pickling no longer customizable

Issue #82 resolved
Stephan Rave created an issue

In mpi4py 2.0 it is possible to setup custom pickling methods by monkey patching mpi4py.MPI.pickle. This is in particular useful when one wants to use alternative serialization packages such as dill which allow pickling of lambdas, etc.

With the master branch of mpi4py this is no longer possible since mpi4py.MPI.PyMPI_PICKLE cannot be modified from within Python and the Pickle class does not allow setting its ob_dumps and ob_loads attributes.

A simple workaround would be to make the ob_dumps and ob_loads attributes cdef public. An alternative would be to provide a set_pickle method which would allow the user to provide a custom Pickle instance.

Comments (4)

  1. Lisandro Dalcin

    Use MPI.pickle.__init__(my_dumps, my_loads), and eventually MPI.pickle.__init__() to reset to defaults.

  2. Stephan Rave reporter

    Ah, of course! Somehow it didn't occur to me that one can simply call __init__ a second time. Thanks!

  3. Lisandro Dalcin

    The whole point of all that nasty code is that I want MPI.pickle.{dumps|loads}() to behave as Python 3 MPI.pickle.{dumps|loads}() under both Python 2 and Python 3 runtimes. Under Python 2, cPickle.dumps(obj) users the slow and heavy protocol 0.

  4. Log in to comment