5D3 - attempt to fix vertical stripes (so it doesn't need any more correction in postprocessing)

#732 Merged at 6b1115f
  1. Alex

While cleaning up some stuff in the raw backend, I revisited the raw type register (aka lv_af_raw), and from this discussion it looks like on 5D3 you can pick a frame with exactly one defect from those two:

  • bad pixels
  • vertical stripes

So, we can either pick an image with bad pixels, but without vertical stripes, or the other way. Currently, the 5D3 uses a raw stream without bad pixels, but with vertical stripes.

Since the bad pixels are easier and faster to correct in postprocessing, what about choosing the image with bad pixels, but without stripes?

How to test:

  • make sure you get files with vertical stripes on the nightly builds (any of them). You probably won't see the artifact on the image, because most converters will correct this defect. Just look in the conversion log for the correction coefficients (numbers around 0.9 ... 1.1).
  • record the same test scene with the ML core built from this PR, and check the conversion log (there should be no more mentions about vertical stripes)
  • make sure the bad pixels are, indeed, corrected by your favorite raw converter

Why bother with this?

  • vertical stripe correction is slow (so you will notice a small speedup in the conversion process if you don't need it)
  • vertical stripe correction is not exact (coefficients do not stay the same, and they are estimated from the first frame in the clip; if the stripes are not visible there, the conversion will not work very well)
  • correcting bad pixels is very fast and much simpler

Why didn't we use this method from the beginning?

  • see the source code comments below (back then, Renato said this type of raw image looks best, so I simply used that without thinking too much about it)
  • back then, the bad pixel correction (when it was implemented with DNG opcodes) wasn't compatible with most raw processing apps.

Comments (3)

  1. DeafEyeJedi

    Ah, missed this PR. Thanks for the reminder @a1ex from recent thread. I am intrigued by this fix and would help out by testing this. I have files that show such stripes in bright areas such as clouds, etc. This whole time I assumed that DarkFraming Avg Process would help this but apparently no according to you guys. Let me know what I can do to get this ball rolling.

    1. Alex author

      Well, I don't have the patience to pixel-peep this, so you can run the tests from the PR description.

      I've included these changes in the latest crop build, so you can simply test that.

  2. Alex author

    This PR was discussed here: http://www.magiclantern.fm/forum/index.php?topic=17795.0

    Conclusions: the raw types that have bad pixels do not completely fix the vertical stripes (they do reduce them to some extent, but you still need to fix them in post). Furthermore, not all current raw converters can handle the bad pixels properly.

    Therefore, I think it's not worth the hassle, so I'm reverting it to the old raw type (no change from unified), but I'll keep the changes to bad pixel and vertical stripe fix in raw2dng/mlv_dump. I'm also keeping the developer menu that lets one experiment with other raw types.

    While I was at it, I've noticed the white level on 5D3 can go from 15000 to 16382 16300, giving about 0.15 stops of extra highlight details in LiveView. Other cameras might benefit for this too, but you need to check (with raw_diag, RawDigger, octave... your choice).