#720 Open
Repository
Branch
lua_fix
Repository
Branch
unified

Bitbucket cannot automatically merge this request.

The commits that make up this pull request have been removed.

Bitbucket cannot automatically merge this request due to conflicts.

Review the conflicts on the Overview tab. You can then either decline the request or merge it manually on your local system using the following commands:

hg update dcc15836c1f3
# Note: This will create a new head!
hg merge ab6ed83893c1
hg commit -m 'Merged in lua_fix (pull request #720)'
Author
  1. Alex
Reviewers
Description

Summary:

  • Backend:

    • simple (unloadable) scripts in addition to previous concept of "complex" scripts (dmilligan)
    • simple scripts will 'unload' when finished running, freeing memory
    • umm_malloc (custom memory allocator for Lua - much faster)
    • small memory backend tweaks and fixes to behave a little better with Lua
    • print last error in menu, tag errors with [SCRIPT.LUA]
    • scripts can be edited in EDITOR.LUA from context menu
    • autorun can be turned on/off by user for each script
    • each script can have "command-line" arguments (currently used with editor.lua)
    • strict.lua only loaded once (faster)
    • doc updates
    • bunch of other minor fixes
  • global:

    • removed global shoot() (use camera.shoot instead)
  • camera:
    • camera.shoot(should_af) now calls take_a_pic (syntax changed, removed the wait parameter)
    • camera.bulb(duration_seconds) fixed (before, docs said seconds but the argument was ms)
    • camera.burst(num_pics) - take N pics in burst mode
    • backend: fix waiting after taking a picture
  • battery:
    • battery.time_remaining (estimated)
  • display:
    • display.print: allow transparent text on transparent background (discussed here; changes done in ML core)
    • display.print: rough tab support (4 spaces) for editor.lua (fixed in ML core)
    • display.screenshot: fix color palette outside ML menu (fixed in ML core)
  • dryos:
    • dryos.rename(src, dst)
  • interval:
    • fix data type of interval.running (bool)
  • tasks:
    • task.yield(): workaround to prevent unsafe usage (one interruption is fine, going back and forth between tasks is not)
  • key:
    • key.wait() uses the task.yield() workaround
    • key.press(KEY.HALFSHUTTER) now works (this might affect the behavior of many other core functions that call SW1/SW2, to be tested)
    • fast scrollwheel events on DIGIC 5 models fixed
    • fix handling unknown events
    • use emulated Q button for cameras without it (so modules and scripts can simply test for the Q event)
  • lens:
  • lv:
    • fix lv.start() in various GUI modes and edge cases (fixed in ML core)
  • menu:
    • removed run_in_separate_task
    • menu.set: backend fixes + API tests (changing menu values by string works much better)
    • menu.get: option to return string; returns nil if menu not found
    • menu.set: returns nil if menu not found
    • menu.select(parent, entry): select ML menu item by name
    • removed menu customization fields (these are internal to menu backend)
    • menu.hidden now handles programmatic menu hiding (todo: same for core code)
  • movie:
    • raise error if ML menu is open (as it works by pressing REC/LV button, which is used in menu)
    • backend checks for duplicate menu entries
  • scripts:
    • api_test, calc, editor, hello, pong, sokoban are now "simple" (unloadable) scripts
    • pong: minor fix (ML menu was not always fully erased before starting the game)
    • recdelay: minor fixes (reasonable defaults; close ML menu)
    • editor: minor fixes (stop on closing ML menu; allow command-line arguments)
    • api_test.lua: new tests (keys, picture taking, movie recording, autofocus, multitasking)
    • unload.lua: test for unloading simple scripts (even if they create tasks and register event handlers, they can still be unloaded if certain conditions are met)

Some changes unrelated to Lua:

  • fine-tuned memory backend behavior (found a case where it was running out of memory)
  • selftest module: stub tests for autofocus functionality
  • max 64 modules (from 32)
  • modules made of multiple source files can now include "module.h" without much trouble
  • bumped API version

This PR also includes dynamic-my-menu, because of its tight integration with Lua menu API changes.


Regarding memory:

Looks like what Lua is doing to the memory backend is pure craziness...

Loading all scripts, including those in the extra directory, and rawbench.lua, allocates about 7000 tiny blocks. Memcheck overhead was 55% before these changes, 27% after.

Fragmentation becomes a big issue - I managed to have about 1MB free in AllocateMemory, but the largest block was smaller than 2K. This can easily cause Canon tasks to fail with ERR70.

I think we should implement a way to load the scripts on demand, rather than autostarting all of them.

I'm also thinking to use an allocator optimized for lots of tiny blocks, such as umm_malloc. Would be nice to start with a contiguous block for this allocator (say 256K) and grow it as needed (maybe by adding one more such block when needed). This way, I hope to minimize fragmentation in the main memory pools, and I also hope to speedup the thousands of realloc calls Lua is making.

Anyone has experience with umm_malloc or other similar library?

Other reports:

  • Issues #2808: Lua led blink not working as expected resolved

Comments (5)

  1. Alex author

    Wanted to run a quick half-shutter test, so I merged lua_fix and ran "make install" over the unified version.

    Got a bunch of games, calculator, text editor and so on, all running at startup, then camera locked up. Will try to debug later.

    edit: after formatting the card and trying the same install sequence (unified, then lua_fix), it seems to work fine. Probably user error, or maybe uploading via wifi isn't that reliable.

  2. Garry George

    All

    One thought for this 'fix' is whether it is possible to add a specific location to menu.open(). At the moment it 'only' open to the last ML location.

    From my experience this would help script to script connectivity, i.e. Jumping from one script to another.