Benchmarks and self-tests refactored as modules

#610 Merged at 04d5360
Repository
Branch
bench-and-tests-as-modules
Repository
Branch
unified
Author
  1. Alex
Reviewers
Description

Two new modules:

bench.mo: contains benchmarks:

  • the old ones: for card, memory, peaking, menu backend
  • a new one, which reveals cache size (thanks g3gg0)

selftest.mo: contains the old burn-in tests

  • stubs API test (important for new ports)
  • stability tests (which try a bunch of predefined scenarios)
  • some infinite backend tests, e.g. redraw tests

Moving this stuff to modules reduced core binary size by around 40K (600D). With lua_fix, MemSiz on 600D is 0x7802c. With patchmgr on top of that, 0x7b82c.

So, this patch is all that's needed to re-enable the 600D builds.

TODO:

  • RPC tests should appear in the menu on 7D (done, but not tested)
  • Long benchmarks should disable powersaving (idea from nikfreak)
  • cleanup APIs (calling low-level core stuff is still messy)

Comments (13)

  1. Audionut

    Card benchmarks fine.

    Memory/Cache benchmarks: Needs some more feedback. Screen goes black and there is no sign of any activity. Maybe an initial screen that tells the user to have patience, which serves two purposes, user sees something happened, user learn to have patience.

    Here you will find plots of the memory tests and a stub test (6D.116 (one test failed)) https://www.dropbox.com/sh/bgh7gnxy6zl1ihn/AADNnlSdKFIwjl3455cIvWywa?dl=0

    Misc Benchmarks/focus peaking: Help "You must also have an image on the card, and for accurate results it needs to have something in focus."

    Self tests/quick test: This test appears to hang after a short time in the QR. If I press half shutter, the test appears to restart and then hang again in the QR screen.

    Self test/most tests: Needs help2 = "enable movie mode".

    1. Alex author

      Solved user feedback for memory benchmark, and some minor menu help tweaks.

      Self tests should work in photo mode as well, but at some point it should ask you to switch to movie mode. Maybe your camera was in photo mode with display off?

      Not sure what to do about hanging in QR. At which test phase it does this?

      1. Audionut

        Just tried quick test again with latest updates. Finished fine.

        Yes I was in photo mode with display off. Only when I enter ML menu that the display told me I needed to enter movie mode, then test runs fine.

        At end of test, shutter was set to bulb. Both wheels would adjust aperture. Tried changing in Canon menu to refresh the setting but did not work. Camera restart fixed it and it powered on with the shutter speed that was set before running the quick test.

  2. Georg Hofstetter

    wonderful :) just a question:

    the model specific ifdef CONFIG_50D / CONFIG_5DC will not be handled correctly, or will they?

      1. Georg Hofstetter

        yeah but they look like it's simple enough to use the module model detection functions.

  3. Licaon Kter

    Is the make-the-screen orangey on powersaving dimming while running a benchmark a normal thing?

    Usually dimming does not introduce a colour change.

    1. Alex author

      No; there is a Kelvin WB test, but it's not combined with the LCD brightness one, and it's not on in the benchmarks.

      Have a video?

      1. Licaon Kter

        Video: https://mega.nz/#!xdwTzCrC!-OG_CxCBcuDghq6JRwoseQfQuAou_fZ7VeZx8fxn6S0

        Some notes of what can be seen:

        • I get a flashing orange screen on ML menu access, most of the time ( I don't remember if that was there before, maybe it was)
        • it's not dimming as I said first, but 'globaldraw off' that does the orangey overlay on powersave
        • once globaldraw off hence the orangey overlay appears it will not dim anymore
        • it won't turn screen off either way (neither if dimming timeouts first)
        • the first mem test fails, no log on card but you can read the message on screen
          1. Alex author

            For dimming, it looks like it first turns off the screen completely, then brings it back at a lower brightness, right? What options you have in Canon menu about LCD brightness?

            The yellowing appears to be an overlay (the effect is not there in the corners, in that tiny space near the info bars). Probably related to fast zebras.

            Do you get the orange overlay if you don't run the memory tests?

            If you select only "turn off global draw after 5 seconds" in the powersave menu, and you have fast zebras enabled, do you get the issue?

            If you turn off fast zebras, but leave global draw on, do you still get the issue?

            edit: I think I've reproduced on 5D3; looks different (diagonal stripes) and crashes with ERR70 after one second.

  4. nikfreak

    Looks good to me -despite being again catched by powersave (2 mins). Couldn't find a bug on 70D. Tried a few times since this PR started and attached screenshot and the stubtest from today. Optional: Maybe unify the environment by turning off some unneeded settings (Global draw etc) before the tests / benchmarks start and reload the config once finished.

    cache0.jpg

    STUBTEST-70D

    1. nikfreak

      Don't know if it's worth but what about usability and maybe saving some more kb's on autoexec.bin. Cleaning up "Debug" menu a little bit more and make one additional module containing:

      • Dump RAM / ROM / image buffers
      • Show tasks / CPU usage / GUi events
      • Show EDMAC
      • Don't click me
    2. Alex author

      Powersave working!

      I ended up changing run_in_separate_task so that everything executed using it runs with powersave disabled. So, the change is also applied on things like dot-tune, focus stack, raw_diag tests, Lua scripts when triggered from menu...

      I don't see any real downside for this change, as most things launched in this way from menu are either short-lived or... the infinite loops from these modules. Background tasks that run continuously (while the camera is powered on) are not started in this way.