check if @pretty_decorator makes the dependency on multiprocess obsolete
Issue #13
closed
currently lib5c
depends on multiprocess
to allow pickling of functions parallelized via @parallelize_regions
since c095acb @parallelize_regions
is itself decorated with @pretty_decorator
. it is unknown whether or not the new mechanics of decorators decorated with @pretty_decorator
allow them to be pickled without dill
, but we should investigate this
Comments (2)
-
reporter -
reporter - changed status to closed
refactored dependencies related to parallelization
decorator is now a required dependency
dill replaces multiprocess as a dependency
modified @parallelize_regions to use dill directly instead of multiprocess
closes
#13→ <<cset b2835c86f2ee>>
- Log in to comment
we dug into this and made the following observations:
decorator
is not available,@parallelize_regions
breaks on some rare edge cases related to default kwargsdecorator
is available, functions decorated with@pretty_decorator
decorators still cannot be pickledmultiprocess
dependency if you retain dependence ondill
and use that package to dodge the pickling problemthe recommendations are:
decorator
a required dependencymultiprocess
from the dependencies, replacing it withdill