missing method ibcast: non-blocking broadcast for Python objects

[Tharindu Adikari]

Following methods accept buffers/NumPy arrays:

Send, Recv, Bcast, Isend, Irecv, Ibcast

All of above except the last one has corresponding methods that accept serializable Python objects:

send, recv, bcast, isend, irecv

Why isn't there an ibcast method that accepts Python objects?

Is there a fundamental reason behind this?

If not please include such a function.

‌

Reference: https://bitbucket.org/mpi4py/mpi4py/src/master/src/mpi4py/MPI/Comm.pyx

‌

[Lisandro Dalcin]

ibcast() (or any of the other nonblocking COLLECTIVE operations) are not be so straightforward to implement.

[Tharindu Adikari]

I'm not sure if I understand correctly. Wouldn't the following be a naive implementation for ibcast?

Sender:
req = comm.Ibcast(pickle.dumps(object), root=0)

Others:

buf = bytearray(2**10)
req = comm.Ibcast(buf, root=0)

Upon testing other get data using

data = pickle.loads(buf)

Wouldn’t a wrapper around these methods will give ibcast functionality?

Probably I’m missing some key insight here.

[Lisandro Dalcin]

The problem is at the receiving side. What if the sender sends a pickled stream larger than 1 KiB? Unfortunately, you naive implementation may work well in your particular implementation, but it is not general enough.

I already have that exact same issue in irecv() and a couple ways to workaound/ease it, but these workarounds would not look so natural to use in the context of an ibcast() implementation.

‌

[Tharindu Adikari]

Thanks, fair enough! I’m going to move the issue to won’t fix.

[Tharindu Adikari]

Functionality is not so straightforward to implement.