check if @pretty_decorator makes the dependency on multiprocess obsolete

Issue #13 closed
Thomas Gilgenast created an issue

currently lib5c depends on multiprocess to allow pickling of functions parallelized via @parallelize_regions

since c095acb @parallelize_regions is itself decorated with @pretty_decorator. it is unknown whether or not the new mechanics of decorators decorated with @pretty_decorator allow them to be pickled without dill, but we should investigate this

Comments (2)

  1. Thomas Gilgenast reporter

    we dug into this and made the following observations:

    • when decorator is not available, @parallelize_regions breaks on some rare edge cases related to default kwargs
    • even when decorator is available, functions decorated with @pretty_decorator decorators still cannot be pickled
    • it is possible to drop the multiprocess dependency if you retain dependence on dill and use that package to dodge the pickling problem

    the recommendations are:

    • make decorator a required dependency
    • remove multiprocess from the dependencies, replacing it with dill
  2. Thomas Gilgenast reporter

    refactored dependencies related to parallelization

    decorator is now a required dependency

    dill replaces multiprocess as a dependency

    modified @parallelize_regions to use dill directly instead of multiprocess

    closes #13

    → <<cset b2835c86f2ee>>

  3. Log in to comment