Proposal: completely replace the old raw format

#685 Merged at 7a9d6b9
Repository
Branch
new_raw_format
Repository
Branch
unified
Author
  1. David Milligan
Reviewers
Description

http://www.magiclantern.fm/forum/index.php?topic=13152.msg161639#msg161639

Here are the issues I have with the RAW format in it's current form:

  1. The file extension should not be .RAW (this is way too generic, and users think they can open it with something like photoshop), it should be something unique

  2. File header information and metadata should be stored at the beginning of the file, not at the end

  3. There should be more metadata than is currently provided (at the very least, camera model, ML version, etc.)

  4. There should be header information in every file if it is split into chunks

Turns out it's not that hard to modify raw_rec to output valid MLV files without add any extra overhead

Comments (6)

  1. Alex

    You mean, a simplified version of MLV? Sounds great to me.

    I would not change the spec though, since there are already a few converters around. Writing VIDF blocks is not hard - one just needs to shift the capture address a bit in every block and copy the metadata into it.

    (I actually wanted to do it myself for a long time, maybe named mlv_lite or so)

  2. Georg Hofstetter

    well if its possible to maintain the high write speed and still generate a proper MLV file, why not. you can make use of the frameSpace or insert NULL chunks if you have to align for the next EDMAC write address.

    1. David Milligan author

      Yeah, so we just allocate an extra 4K when creating the buffers, and put the vidf header in there, and frame space is just 4096 - sizeof(vidf). Only need to update the timestamp and frame number each frame.

      We could theoretically put the vidf at the end of the first 4K instead of beginning and save a tiny bit of write speed, but it hardly seems like saving ~4K would be worth it.

      This current method is tested and working 60D. Files could be opened just fine with current versions of MLVFS and mlv_dump

      1. Georg Hofstetter

        question is, will misaligning CFDMA start address influence transfer speed? whats your write rate with mlv_lite ?

        1. David Milligan author

          Well, there's this comment in the code.

          /* should be multiple of 512, so there's no write speed penalty (see http://chdk.setepontos.com/index.php?topic=9970 ; confirmed by benchmarks) */
              /* should be multiple of 4096 for proper EDMAC alignment */
          

          If that's true then we should be able to put the VIDF header at the start of the last 512 bytes of the 4KB padding without any penalty.