Fix: 100% cpu usage on idle ranks

[Former user]

When only one node is computing, the IDLE process occupies 100% of the CPU. Tested: IntelMPI/2017.0.098 ParaStationMPI/5.1.5-1

mpi4py/2.0.0-Python-2.7.12

I dont know if it is a bug or a feature, but the problem has been solved else where in 2012: See: https://goo.gl/NofOO9 see: https://groups.google.com/forum/#!topic/mpi4py/nArVuMXyyZI

Simple fix for me: def barrier(comm, tag=0, sleep=0.01): """ MPI barrier fonction that solve the problem that Idle process occupies 100% CPU. See: https://goo.gl/NofOO9 see: https://groups.google.com/forum/#!topic/mpi4py/nArVuMXyyZI """ size = comm.Get_size() if size == 1: return rank = comm.Get_rank() mask = 1 while mask < size: dst = (rank + mask) % size src = (rank - mask + size) % size req = comm.isend(None, dst, tag) while not comm.Iprobe(src, tag): time.sleep(sleep) comm.recv(None, src, tag) req.Wait() mask <<= 1

[e-mail]

[Lisandro Dalcin]

mpi4py just uses MPI_Barrier(), the 100% CPU is not mpi4py's choice/bug/feature, but the backend MPI. Please check the Intel MPI docs https://software.intel.com/en-us/node/528821, there is a way to alter the default behavior (look for I_MPI_SPIN_COUNT and I_MPI_SHM_SPIN_COUNT)

The solution you mention is actually a workaround proposed by myself in our mailing list. However, it is just a workaround, and I don't think it is a good idea to change mpi4py's default barrier to anything other than the plain MPI_Barrier(). There are many reason, the first one would the the principle of least surprise, the second one is that is very reasonable to expect that mpi4py users will want to use the builtin barrier implementation from the MPI implementation they are using (again, there are bunch of reasons for this), a third one is that the problem is specific to some MPI implementations (and they usually have tuning parameters to alter behavior).

In general lines, mpi4py tries hard to stay as close as possible to bare-metal MPI. If you still disagree with my point of view, you are most welcome to challenge me in our mailing list. Maybe I'm being too rigid and, other users may very well be on your side, and I would love to know their opinions and participate in further discussions about this topic.