Test failures with openmpi 2.0.0rc2

[Orion Poplawski]

We're testing out openmpi 2.0.0rc2 and getting:

testMemoryModel (test_win.TestWinCreateWorld) ... ok

======================================================================
ERROR: testGetSetInfo (test_file.TestFileSelf)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_file.py", line 86, in testGetSetInfo
    info.Free()
  File "MPI/Info.pyx", line 41, in mpi4py.MPI.Info.Free (src/mpi4py.MPI.c:80825)
Exception: MPI_ERR_INFO: invalid info object

======================================================================
FAIL: testReduceScatterBlock (test_cco_nb_buf.TestCCOBufSelf)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_cco_nb_buf.py", line 225, in testReduceScatterBlock
    self.assertAlmostEqual(value, v_sum)
  File "test/test_cco_nb_buf.py", line 103, in assertAlmostEqual
    raise self.failureException('%r != %r' % (first, second))
AssertionError: -1 != 0

======================================================================
FAIL: testReduceScatterBlock (test_cco_nb_buf.TestCCOBufSelfDup)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_cco_nb_buf.py", line 225, in testReduceScatterBlock
    self.assertAlmostEqual(value, v_sum)
  File "test/test_cco_nb_buf.py", line 103, in assertAlmostEqual
    raise self.failureException('%r != %r' % (first, second))
AssertionError: -1 != 0

======================================================================
FAIL: testReduceScatterBlock (test_cco_nb_buf.TestCCOBufWorld)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_cco_nb_buf.py", line 225, in testReduceScatterBlock
    self.assertAlmostEqual(value, v_sum)
  File "test/test_cco_nb_buf.py", line 103, in assertAlmostEqual
    raise self.failureException('%r != %r' % (first, second))
AssertionError: -1 != 0

======================================================================
FAIL: testReduceScatterBlock (test_cco_nb_buf.TestCCOBufWorldDup)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_cco_nb_buf.py", line 225, in testReduceScatterBlock
    self.assertAlmostEqual(value, v_sum)
  File "test/test_cco_nb_buf.py", line 103, in assertAlmostEqual
    raise self.failureException('%r != %r' % (first, second))
AssertionError: -1 != 0

======================================================================
FAIL: testGetByteOffset (test_file.TestFileSelf)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_file.py", line 140, in testGetByteOffset
    self.assertEqual(disp, offset)
AssertionError: 0 != 1

======================================================================
FAIL: testGetSetView (test_file.TestFileSelf)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_file.py", line 102, in testGetSetView
    self.assertEqual(etype,   et)
AssertionError: <mpi4py.MPI.Datatype object at 0x7f05163f9330> != <mpi4py.MPI.Datatype object at 0x7f04f868f150>

======================================================================
FAIL: testPreallocate (test_file.TestFileSelf)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_file.py", line 46, in testPreallocate
    self.assertEqual(size, 100)
AssertionError: 102 != 100

----------------------------------------------------------------------
Ran 1042 tests in 19.480s

FAILED (failures=7, errors=1)
-------------------------------------------------------
Primary job  terminated normally, but 1 process returned
a non-zero exit code.. Per user-direction, the job has been aborted.
-------------------------------------------------------
--------------------------------------------------------------------------
mpiexec detected that one or more processes exited with non-zero status, thus causing
the job to be terminated. The first process to do so was:

  Process name: [[42372,1],0]
  Exit code:    1
--------------------------------------------------------------------------

[Orion Poplawski]

Full build logs: https://copr-be.cloud.fedoraproject.org/results/@scitech/openmpi2/fedora-rawhide-x86_64/00182813-mpi4py/build.log.gz

[Lisandro Dalcin]

OK, give me some time to perform my own testing. New Open MPI releases usually break mpi4py's testsuite, and 99% of the time is not mpi4py's fault. Eventually, you should just patch mpi4py to disable some tests.

[Lisandro Dalcin]

I can reproduce the issue with a source build of Open MPI 2.0.0rc2. I think most of these failures are regressions in Open MPI, however there are a couple I need to look more closely.

[Orion Poplawski]

Thanks for taking a look at these. If there are any regressions in openmpi, they should be reported here: https://github.com/open-mpi/ompi/issues

[Lisandro Dalcin]

All issues reported upstream:

• https://github.com/open-mpi/ompi/issues/1630

• https://github.com/open-mpi/ompi/issues/1631

• https://github.com/open-mpi/ompi/issues/1632

• https://github.com/open-mpi/ompi/issues/1633

• https://github.com/open-mpi/ompi/issues/1636

[Lisandro Dalcin]

All issues in Open MPI tracker were closed.

[Orion Poplawski]

Still two test failures with current 2.0.0 snapshot:

FAIL: testGetByteOffset (test_file.TestFileSelf)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_file.py", line 140, in testGetByteOffset
    self.assertEqual(disp, offset)
AssertionError: 0 != 1
======================================================================
FAIL: testPreallocate (test_file.TestFileSelf)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_file.py", line 49, in testPreallocate
    self.assertEqual(size, 100)
AssertionError: 10 != 100

Looks like the get_byte_offset is not yet in 2.X - https://github.com/open-mpi/ompi-release/pull/1212

testPreallocate is a different assertion now - looks like the size is going down with the smaller preallocate. I don't know what is correct there.

[Lisandro Dalcin]

From the MPI 3.1 standard: If size is larger than the current file size, the file size increases to size. If size is less than or equal to the current file size, the file size is unchanged.

[Orion Poplawski]

Looks like you commented in https://github.com/open-mpi/ompi/issues/1633 I commented in https://github.com/open-mpi/ompi-release/pull/1146 as well.