Tests fail with mpich2 on FreeBSD: ERROR: testIDup (test_comm.TestCommSelf)

[yurivict]

running test
[[e-mail]] Python 2.7 (/usr/local/bin/python2.7)
[[e-mail]] MPI 2.2 (MPICH2 1.5.0)
[[e-mail]] mpi4py 3.0.0 (build/lib.freebsd-11.2-STABLE-amd64-2.7/mpi4py)
..............................................................................................................................................................................................................................................................................................................ssssssssssssssssssssssss.........................................................................E...............E...............E...............E.....ssssssssssss.......................................................F....ssss............s............ssssssssssssssssssssssssssssssssssssssssssss..................................................................................ss...s.s..........ssssssssssssss..................................................................................................................................................................................................s...s....................s........s.......s........sssssssssssssssssssssssssssssssssssssssssssssssssssssss..........................................................................................................................
======================================================================
ERROR: testIDup (test_comm.TestCommSelf)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_comm.py", line 96, in testIDup
    comm, request = self.COMM.Idup()
  File "mpi4py/MPI/Comm.pyx", line 160, in mpi4py.MPI.Comm.Idup
Exception: Internal MPI error!, error stack:
PMPIX_Comm_idup(159).............:  MPIX_Comm_idup(comm=MPI_COMM_SELF, newcomm=0x80b102790, request=0x807b01ba0)
PMPIX_Comm_idup(139).............: 
MPIR_Comm_idup_impl(62)..........: 
MPIR_Get_contextid_nonblock(1234): 
gcn_sch(1175)....................: this functionality is not supported when the thread level is greater than MPI_THREAD_SINGLE

======================================================================
ERROR: testIDup (test_comm.TestCommSelfDup)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_comm.py", line 96, in testIDup
    comm, request = self.COMM.Idup()
  File "mpi4py/MPI/Comm.pyx", line 160, in mpi4py.MPI.Comm.Idup
Exception: Internal MPI error!, error stack:
PMPIX_Comm_idup(159).............:  MPIX_Comm_idup(comm=comm=0x84000003, newcomm=0x80b102868, request=0x80bf38720)
PMPIX_Comm_idup(139).............: 
MPIR_Comm_idup_impl(62)..........: 
MPIR_Get_contextid_nonblock(1234): 
gcn_sch(1175)....................: this functionality is not supported when the thread level is greater than MPI_THREAD_SINGLE

======================================================================
ERROR: testIDup (test_comm.TestCommWorld)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_comm.py", line 96, in testIDup
    comm, request = self.COMM.Idup()
  File "mpi4py/MPI/Comm.pyx", line 160, in mpi4py.MPI.Comm.Idup
Exception: Internal MPI error!, error stack:
PMPIX_Comm_idup(159).............:  MPIX_Comm_idup(comm=MPI_COMM_WORLD, newcomm=0x80b1028e0, request=0x80bf386e0)
PMPIX_Comm_idup(139).............: 
MPIR_Comm_idup_impl(62)..........: 
MPIR_Get_contextid_nonblock(1234): 
gcn_sch(1175)....................: this functionality is not supported when the thread level is greater than MPI_THREAD_SINGLE

======================================================================
ERROR: testIDup (test_comm.TestCommWorldDup)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_comm.py", line 96, in testIDup
    comm, request = self.COMM.Idup()
  File "mpi4py/MPI/Comm.pyx", line 160, in mpi4py.MPI.Comm.Idup
Exception: Internal MPI error!, error stack:
PMPIX_Comm_idup(159).............:  MPIX_Comm_idup(comm=comm=0x84000005, newcomm=0x80b1029e8, request=0x80bf38760)
PMPIX_Comm_idup(139).............: 
MPIR_Comm_idup_impl(62)..........: 
MPIR_Get_contextid_nonblock(1234): 
gcn_sch(1175)....................: this functionality is not supported when the thread level is greater than MPI_THREAD_SINGLE

======================================================================
FAIL: testDL1 (test_dl.TestDL)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_dl.py", line 22, in testDL1
    self.assertTrue(dl.dlerror() is None)
AssertionError: False is not true

----------------------------------------------------------------------
Ran 1134 tests in 7.680s

FAILED (failures=1, errors=4, skipped=163)
error: test
*** Error code 1

[Lisandro Dalcin]

FreeBSD is still shipping MPICH2 in 2018? Great!

Can you try this patch and confirm whether it works ? (It will just skip the test)

diff --git a/test/test_comm.py b/test/test_comm.py
index 68519a8..a94e83e 100644
--- a/test/test_comm.py
+++ b/test/test_comm.py
@@ -91,6 +91,7 @@ class BaseTestComm(object):
         info.Free()

     @unittest.skipMPI('mpich(<=3.1.0)', MPI.Query_thread() > MPI.THREAD_SINGLE)
+    @unittest.skipMPI('MPICH2')
     def testIDup(self):
         try:
             comm, request = self.COMM.Idup()

[yurivict]

FreeBSD is still shipping MPICH2 in 2018? Great!

No, mpich2 is still available. It's my fault I didn't realize that 3.x is the current version and used 2.x.

But it also fails with mpich-3.2.1:

======================================================================
FAIL: testDL1 (test_dl.TestDL)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_dl.py", line 22, in testDL1
    self.assertTrue(dl.dlerror() is None)
AssertionError: False is not true

----------------------------------------------------------------------
Ran 1134 tests in 7.869s

FAILED (failures=1, skipped=83)

[Lisandro Dalcin]

That error is totally MPI-unrelated. These test cases are for the mpi4py.dl module, which is just a thin wrapper for dlopen() and related functions. This module is non essential.

To provide a proper patch, I would need your help, I don't have access to FreeBSD. Could you please add print(dl.dlerror()) before the assertion to figure out what's going on? Or maybe just run this code:

from mpi4py import dl
libm = 'libm.so'
handle = dl.dlopen(libm, dl.RTLD_LOCAL|dl.RTLD_LAZY)
assert handle != 0
print(dl.dlerror())

I'm reading this https://www.freebsd.org/cgi/man.cgi?query=dlopen&sektion=3&manpath=freebsd-release-ports

If dlopen() fails, it returns a null pointer, and sets an error condition
which may be interrogated with dlerror().
...
The dlerror() function returns a null-terminated character  string
describing  the last error that occurred during a call to dlopen(),
dladdr(), dlinfo(), dlsym(), dlfunc(), or dlclose().  If no such error
has occurred, dlerror() returns a null pointer.  At each call to
dlerror(),  the error indication is reset

So, if dlopen() succeeds, then handle cannot be NULL (the wrapper translates the pointer to an integer, so the Python return cannot be zero), and then dlerror() should return NULL (and the Python return should be mapped to None).

[Lisandro Dalcin]

@yurivict Any news?

[yurivict]

Tests succeed now with mpich-3. Thanks!

[Lisandro Dalcin]

@yurivict What happened to the dlopen() test? How did you manage to fix it?

[yurivict]

The dlopen error went away with mpich-3 update.

[Lisandro Dalcin]

OK, something weird is going on. Maybe the old MPICH was linked with an older version of libm.