Copyright (c) 2004-2007 The Trustees of Indiana University and Indiana
University Research and Technology
Corporation. All rights reserved.
Copyright (c) 2004-2006 The University of Tennessee and The University
of Tennessee Research Foundation. All rights
Copyright (c) 2004-2006 High Performance Computing Center Stuttgart,
University of Stuttgart. All rights reserved.
Copyright (c) 2004-2006 The Regents of the University of California.
All rights reserved.
Copyright (c) 2006-2007 Cisco Systems, Inc. All rights reserved.
Copyright (c) 2006 Voltaire, Inc. All rights reserved.
Copyright (c) 2006 Sun Microsystems, Inc. All rights reserved.
Use is subject to license terms.
Copyright (c) 2006-2007 Los Alamos National Security, LLC. All rights
Additional copyrights may follow
This file contains the main features as well as overviews of specific
bug fixes (and other actions) for each version of Open MPI since
- Fixed race condition in the shared memory fifo's, which led to
- Corrected the size of the shared memory file - subtracted out the
space the header was occupying.
- Add support for MPI_2COMPLEX and MPI_2DOUBLE_COMPLEX.
- Always ensure to create $(includedir)/openmpi, even if the C++
bindings are disabled so that the wrapper compilers don't point to
a directory that doesn't exist. Thanks to Martin Audet for
identifying the problem.
- Fixes for endian handling in MPI process startup.
- Openib BTL initialization fixes for cases where MPI processes in the
same job has different numbers of active ports on the same physical
- Print more descriptive information when displaying backtraces on
OS's that support this functionality, such as the hostname and PID
of the process in question.
- Fixes to properly handle MPI exceptions in C++ on communicators,
windows, and files.
- Much more reliable runtime support, particularly with regards to MPI
job startup scalability, BProc support, and cleanup in failure
scenarios (e.g., MPI_ABORT, MPI processes abnormally terminating,
- Significant performance improvements for MPI collectives,
particularly on high-speed networks.
- Various fixes in the MX BTL component.
- Fix C++ typecast problems with MPI_ERRCODES_IGNORE. Thanks to
Satish Balay for bringing this to our attention.
- Allow run-time specification of the maximum amount of registered
memory for OpenFabrics and GM.
- Users who utilize the wrapper compilers (e.g., mpicc and mpif77)
will not notice, but the underlying library names for ORTE and OPAL
have changed to libopen-rte and libopen-pal, respectively (listed
here because there are undoubtedly some users who are not using the
- Many bug fixes to MPI-2 one-sided support.
- Added support for TotalView message queue debugging.
- Fixes for MPI_STATUS_SET_ELEMENTS.
- Print better error messages when mpirun's "-nolocal" is used when
there is only one node available.
- Added man pages for several Open MPI executables and the MPI API
- A number of fixes for Alpha platforms.
- A variety of Fortran API fixes.
- Build the Fortran MPI API as a separate library to allow these
functions to be profiled properly.
- Add new --enable-mpirun-prefix-by-default configure option to always
imply the --prefix option to mpirun, preventing many rsh/ssh-based
users from needing to modify their shell startup files.
- Add a number of missing constants in the C++ bindings.
- Added tight integration with Sun N1 Grid Engine (N1GE) 6 and the
open source Grid Engine.
- Allow building the F90 MPI bindings as shared libraries for most
compilers / platforms. Explicitly disallow building the F90
bindings as shared libraries on OS X because of complicated
situations with Fortran common blocks and lack of support for
unresolved common symbols in shared libraries.
- Added stacktrace support for Solaris and Mac OS X.
- Update event library to libevent-1.1b.
- Fixed standards conformance issues with MPI_ERR_TRUNCATED and
setting MPI_ERROR during MPI_TEST/MPI_WAIT.
- Addition of "cm" PML to better support library-level matching
interconnects, with support for Myrinet/MX, and QLogic PSM-based
- Addition of "udapl" BTL for transport across uDAPL interconnects.
- Really check that the $CXX given to configure is a C++ compiler
(not a C compiler that "sorta works" as a C++ compiler).
- Properly check for local host only addresses properly, looking
for 127.0.0.0/8, rather than just 127.0.0.1.
- Fix syntax error in a corner case of the event library. Thanks to
Bert Wesarg for pointing this out.
- Add new MCA parameter (mpi_preconnect_oob) for pre-connecting the
"out of band" channels between all MPI processes. Most helpful for
MPI applications over InfiniBand where process A sends an initial
message to process B, but process B does not enter the MPI library
for a long time.
- Fix for a race condition in shared memory locking semantics.
- Add major, minor, and release version number of Open MPI to mpi.h.
Thanks to Martin Audet for the suggestion.
- Fix the "restrict" compiler check in configure.
- Fix a problem with argument checking in MPI_TYPE_CREATE_SUBARRAY.
- Fix a problem with compiling the XGrid components with non-gcc
- Fixed 64-bit alignment issues with TCP interface detection on
intel-based OS X machines.
- Adjusted TCP interface selection to automatically ignore Linux
channel-bonded slave interfaces.
- Fixed the type of the first parameter to the MPI F90 binding for
MPI_INITIALIZED. Thanks to Tim Campbell for pointing out the
- Fix a bunch of places in the Fortran MPI bindings where (MPI_Fint*)
was mistakenly being used instead of (MPI_Aint*).
- Fixes for fortran MPI_STARTALL, which could sometimes return
incorrect request values. Thanks to Tim Campbell for pointing out
- Include both pre- and post-MPI-2 errata bindings for
- Fix math error on Intel OS X platforms that would greatly increase
shared memory latency.
- Fix type casting issue with MPI_ERRCODES_IGNORE that would cause
errors when using a C++ compiler. Thanks to Barry Smith for
bringing this to our attention.
- Fix possible segmentation fault during shutdown when using the
- Remove the "hierarch" coll component; it was not intended to be
included in stable releases yet.
- Fix a race condition with stdout/stderr not appearing properly from
all processes upon termination of an MPI job.
- Fix internal accounting errors with the self BTL.
- Fix typos in the code path for when sizeof(int) != sizeof(INTEGER)
in the MPI F77 bindings functions. Thanks to Pierre-Matthieu
Anglade for bringing this problem to our attention.
- Fix for a memory leak in the derived datatype function
ompi_ddt_duplicate(). Thanks to Andreas Schafer for reporting,
diagnosing, and patching the leak.
- Used better performing basic algorithm for MPI_ALLGATHERV.
- Added a workaround for a bug in the Intel 9.1 C++ compiler (all
versions up to and including 20060925) in the MPI C++ bindings that
caused run-time failures. Thanks to Scott Weitzenkamp for reporting
- Fix MPI_SIZEOF implementation in the F90 bindings for COMPLEX
- Fixes for persistent requests involving MPI_PROC_NULL. Thanks to
Lisandro Dalcin for reporting the problem.
- Fixes to MPI_TEST* and MPI_WAIT* for proper MPI exception reporting.
Thanks to Lisandro Dalcin for finding the issue.
- Various fixes for MPI generalized request handling; addition of
missing MPI::Grequest functionality to the C++ bindings.
- Add "mpi_preconnect_all" MCA parameter to force wireup of all MPI
connections during MPI_INIT (vs. making connections lazily whenever
the first MPI communication occurs between a pair of peers).
- Fix a problem for when $FC and/or $F77 were specified as multiple
tokens. Thanks to Orion Poplawski for identifying the problem and
to Ralf Wildenhues for suggesting the fix.
- Fix several MPI_*ERRHANDLER* functions and MPI_GROUP_TRANSLATE_RANKS
with respect to what arguments they allowed and the behavior that
they effected. Thanks to Lisandro Dalcin for reporting the
- Really fix Fortran status handling in MPI_WAITSOME and MPI_TESTSOME.
- Various datatype fixes, reported by several users as causing
failures in the BLACS testing suite. Thanks to Harald Forbert, Ake
Sandgren and, Michael Kluskens for reporting the problem.
- Correctness and performance fixes for heterogeneous environments.
- Fixed a error in command line parsing on some platforms (causing
mpirun to crash without doing anything).
- Fix for initialization hangs on 64 bit Mac OS X PowerPC systems.
- Fixed some memory allocation problems in mpirun that could cause
random problems if "-np" was not specified on the command line.
- Add Kerberos authentication support for XGrid.
- Added LoadLeveler support for jobs larger than 128 tasks.
- Fix for large-sized Fortran LOGICAL datatypes.
- Fix various error checking in MPI_INFO_GET_NTHKEY and
MPI_GROUP_TRANSLATE_RANKS, and some collective operations
(particularly with regards to MPI_IN_PLACE). Thanks to Lisandro
Dalcin for reporting the problems.
- Fix receiving messages to buffers allocated by MPI_ALLOC_MEM.
- Fix a number of race conditions with the MPI-2 Onesided
- Fix the "tuned" collective componenete where some cases where
MPI_BCAST could hang.
- Update TCP support to support non-uniform TCP environments.
- Allow the "poe" RAS component to be built on AIX or Linux.
- Only install mpif.h if the rest of the Fortran bindings are
- Fixes for BProc node selection.
- Add some missing Fortran MPI-2 IO constants.
- Fix for Fortran string handling in various MPI API functions.
- Fix for Fortran status handling in MPI_WAITSOME and MPI_TESTSOME.
- Various fixes for the XL compilers.
- Automatically disable using mallot() on AIX.
- Memory fixes for 64 bit platforms with registering MCA parameters in
the self and MX BTL components.
- Fixes for BProc to support oversubscription and changes to the
mapping algorithm so that mapping processes "by slot" works as
- Fixes for various abort cases to not hang and clean up nicely.
- If using the Intel 9.0 v20051201 compiler on an IA64 platform, the
ptmalloc2 memory manager component will automatically disable
itself. Other versions of the Intel compiler on this platform seem
to work fine (e.g., 9.1).
- Added "host" MPI_Info key to MPI_COMM_SPAWN and
- Add missing C++ methods: MPI::Datatype::Create_indexed_block,
- Fix OSX linker issue with Fortran bindings.
- Fixed MPI_COMM_SPAWN to start spawning new processes in slots that
(according to Open MPI) are not already in use.
- Added capability to "mpirun a.out" (without specifying -np) that
will run on all currently-allocated resources (e.g., within a batch
job such as SLURM, Torque, etc.).
- Fix a bug with one particular case of MPI_BCAST. Thanks to Doug
Gregor for identifying the problem.
- Ensure that the shared memory mapped file is only created when there
is more than one process on a node.
- Fixed problems with BProc stdin forwarding.
- Fixed problem with MPI_TYPE_INDEXED datatypes. Thanks to Yven
Fournier for identifying this problem.
- Fix some thread safety issues in MPI attributes and the openib BTL.
- Fix the BProc allocator to not potentially use the same resources
across multiple ORTE universes.
- Fix gm resource leak.
- More latency reduction throughout the code base.
- Make the TM PLS (PBS Pro, Torque, Open PBS) more scalable, and fix
some latent bugs that crept in v1.1. Thanks to the Thunderbird crew
at Sandia National Laboratories and Martin Schaffoner for access to
testing facilities to make this happen.
- Added new command line options to mpirun:
--nolocal: Do not run any MPI processes on the same node as mpirun
(compatibility with the OSC mpiexec launcher)
--nooversubscribe: Abort if the number of processes requested would
--quiet / -q: do not show spurious status messages
--version / -V: show the version of Open MPI
- Fix bus error in XGrid process starter. Thanks to Frank from the
Open MPI user's list for identifying the problem.
- Fix data size mismatches that caused memory errors on PPC64
platforms during the startup of the openib BTL.
- Allow propagation of SIGUSR1 and SIGUSR2 signals from mpirun to
back-end MPI processes.
- Add missing MPI::Is_finalized() function.
- Various MPI datatype fixes, optimizations.
- Fixed various problems on the SPARC architecture (e.g., not
correctly aligning addresses within structs).
- Improvements in various run-time error messages to be more clear
about what they mean and where the errors are occurring.
- Various fixes to mpirun's handling of --prefix.
- Updates and fixes for Cray/Red Storm support.
- Major improvements to the Fortran 90 MPI bindings:
- General improvements in compile/linking time and portability
between different F90 compilers.
- Addition of "trivial", "small" (the default), and "medium"
Fortran 90 MPI module sizes (v1.0.x's F90 module was
equivalent to "medium"). See the README file for more
- Fix various MPI F90 interface functions and constant types to
match. Thanks to Michael Kluskens for pointing out the problems
- Allow short messagees to use RDMA (vs. send/receive semantics) to a
limited number peers in both the mvapi and openib BTL components.
This reduces communication latency over IB channels.
- Numerous performance improvements throughout the entire code base.
- Many minor threading fixes.
- Add a define OMPI_SKIP_CXX to allow the user to skip the mpicxx.h from
being included in mpi.h. It allows the user to compile C code with a CXX
compiler without including the CXX bindings.
- PERUSE support has been added. In order to activate it add
--enable-peruse to the configure options. All events described in
the PERUSE 2.0 draft are supported, plus one Open MPI
extension. PERUSE_COMM_REQ_XFER_CONTINUE allow to see how the data
is segmented internally, using multiple interfaces or the pipeline
engine. However, this version only support one event of each type
simultaneously attached to a communicator.
- Add support for running jobs in heterogeneous environments.
Currently supports environments with different endianness and
different representations of C++ bool and Fortran LOGICAL.
Mismatched sizes for other datatypes is not supported.
- Open MPI now includes an implementation of the MPI-2 One-Sided
- Open MPI is now configurable in cross-compilation environments.
Several Fortran 77 and Fortran 90 tests need to be pre-seeded with
results from a config.cache-like file.
- Add --debug option to mpirun to generically invoke a parallel debugger.
1.0.3 (unreleased; all fixes included in 1.1)
- Fix a problem noted by Chris Hennes where MPI_INFO_SET incorrectly
disallowed long values.
- Fix a problem in the launch system that could cause inconsistent
launch behavior, particularly when launching large jobs.
- Require that the openib BTL find <sysfs/libsysfs.h>. Thanks to Josh
Aune for the suggestion.
- Include updates to support the upcoming Autoconf 2.60 and Libtool
2.0. Thanks to Ralf Wildenhues for all the work!
- Fix bug with infinite loop in the "round robin" process mapper.
Thanks to Paul Donohue for reporting the problem.
- Enusre that memory hooks are removed properly during MPI_FINALIZE.
Thanks to Neil Ludban for reporting the problem.
- Various fixes to the included support for ROMIO.
- Fix to ensure that MPI_LONG_LONG and MPI_LONG_LONG_INT are actually
synonyms, as defined by the MPI standard. Thanks to Martin Audet
for reporting this.
- Fix Fortran 90 configure tests to properly utilize LDFLAGS and LIBS.
Thanks to Terry Reeves for reporting the problem.
- Fix shared memory progression in asynchronous progress scenarios.
Thanks to Mykael Bouquey for reporting the problem.
- Fixed back-end operations for predefined MPI_PROD for some
datatypes. Thanks to Bert Wesarg for reporting this.
- Adapted configure to be able to handle Torque 2.1.0p0's (and above)
new library name. Thanks to Brock Palen for pointing this out and
providing access to a Torque 2.1.0p0 cluster to test with.
- Fixed situation where mpirun could set a shell pipeline's stdout
to non-blocking, causing the shell pipeline to prematurely fail.
Thanks to Darrell Kresge for figuring out what was happening.
- Fixed problems with leave_pinned that could cause Badness with the
- Fixed problems with MPI_FILE_OPEN and non-blocking MPI-2 IO access.
- Fixed various InfiniBand port matching issues during startup.
Thanks to Scott Weitzenkamp for identifying these problems.
- Fixed various configure, build and run-time issues with ROMIO.
Thanks to Dries Kimpe for bringing them to our attention.
- Fixed error in MPI_COMM_SPLIT when dealing with intercommunicators.
Thanks to Bert Wesarg for identifying the problem.
- Fixed backwards handling of "high" parameter in MPI_INTERCOMM_MERGE.
Thanks to Michael Kluskens for pointing this out to us.
- Fixed improper handling of string arguments in Fortran bindings
for MPI-IO functionality
- Fixed segmentation fault with 64 bit applications on Solaris when
using the shared memory transports.
- Fixed MPI_COMM_SELF attributes to free properly at the beginning of
MPI_FINALIZE. Thanks to Martin Audet for bringing this to our
- Fixed alignment tests for cross-compiling to not cause errors with
recent versions of GCC.
- Fixed assembly race condition on AMD64 platforms.
- Fixed residual .TRUE. issue with copying MPI attributes set from
- Remove unnecessary logic from Solaris pty I/O forwarding. Thanks to
Francoise Roch for bringing this to our attention.
- Fixed error when count = 0 was given for multiple completion MPI
functions (MPI_TESTSOME, MPI_TESTANY, MPI_TESTALL, MPI_WAITSOME,
- Better handling in MPI_ABORT for when peer processes have already
died, especially under some resource managers.
- Random updates to README file, to include notes about the Portland
- Random, small threading fixes to prevent deadlock.
- Fixed a problem with handling long mpirun app files. Thanks to Ravi
Manumachu for identifying the problem.
- Fix handling of strings in several of the Fortran 77 bindings.
- Fix LinuxPPC assembly issues. Thanks to Julian Seward for reporting
- Enable pty support for standard I/O forwarding on platforms that
have ptys but do not have openpty(). Thanks to Pierre Valiron for
bringing this to our attention.
- Disable inline assembly for PGI compilers to avoid compiler errors.
Thanks to Troy Telford for bringing this to our attention.
- Added MPI_UNSIGNED_CHAR and MPI_SIGNED_CHAR to the allowed reduction
- Fix a segv in variable-length message displays on Opterons running
Solaris. Thanks to Pierre Valiron for reporting the issue.
- Added MPI_BOOL to the intrinsic reduction operations MPI_LAND,
MPI_LOR, MPI_LXOR. Thanks to Andy Selle for pointing this out to us.
- Fixed TCP BTL network matching logic during MPI_INIT; in some cases
on multi-NIC nodes, a NIC could get paired with a NIC on another
network (typically resulting in deadlock). Thanks to Ken Mighell
for pointing this out to us.
- Change the behavior of orterun (mpirun, mpirexec) to search for
argv and the cwd on the target node (i.e., the node where the
executable will be running in all systems except BProc, where the
searches are run on the node where orterun is invoked).
- Fix race condition in shared memory transport that could cause
crashes on machines with weak memory consistency models (including
- Fix warnings about setting read-only MCA parameters on bproc systems.
- Change the exit status set by mpirun when an application process is
killed by a signal. The exit status is now set to signo + 128, which
conforms with the behavior of (almost) all shells.
- Correct a datatype problem with the convertor when partially
unpacking data. Now we can position the convertor to any position
not only on the predefined types boundaries. Thanks to Yvan Fournier
for reporting this to us.
- Fix a number of standard I/O forwarding issues, including the
ability to background mpirun and a loss of data issue when
redirecting mpirun's standard input from a file.
- Fixed bug in ompi_info where rcache and bml MCA parameters would not
- Fixed umask issues in the session directory. Thanks to Glenn Morris
for reporting this to us.
- Fixed tcsh-based LD_LIBRARY_PATH issues with --prefix. Thanks to
Glen Morris for identifying the problem and suggesting the fix.
- Removed extraneous \n's when setting PATH and LD_LIBRARY_PATH in the
rsh startup. Thanks to Glen Morris for finding these typos.
- Fixed missing constants in MPI C++ bindings.
- Fixed some errors caused by threading issues.
- Fixed openib BTL flow control logic to not overrun the number of
send wqes available.
- Update to match newest OpenIB user-level library API. Thanks to
Roland Dreier for submitting this patch.
- Report errors properly when failing to register memory in the openib
- Reduce memory footprint of openib BTL.
- Fix parsing problem with mpirun's "-tv" switch. Thanks to Chris
Gottbrath for supplying the fix.
- Fix Darwin net/if.h configure warning.
- The GNU assembler unbelievably defaults to making stacks executable.
So when using gas, add flags to explicitly tell it to not make
stacks executable (lame but necessary).
- Add missing MPI::Request::Get_status() methods. Thanks to Bill
Saphir for pointing this out to us.
- Improved error messages on memory registration errors (e.g., when
using high-speed networks).
- Open IB support now checks firmware for how many outstanding RDMA
requests are supported. Thanks to Mellanox for pointing this out to
- Enable printing of stack traces in MPI processes upon SIGBUS,
SIGSEGV, and SIGFPE if the platform supports it.
- Fixed F90 compilation support for the Lahey compiler.
- Fixed issues with ROMIO shared library support.
- Fixed internal accounting problems with rsh support.
- Update to GNU Libtool 1.5.22.
- Fix error in configure script when setting CCAS to ias (the Intel
- Added missing MPI::Intercomm collectives.
- Fixed MPI_IN_PLACE handling for Fortran collectives.
- Fixed some more C++ const_cast<> issues. Thanks for Martin Audet
(again) for bringing this to our attention.
- Updated ROMIO with the version from MPICH 1.2.7p1, marked as version
- Fixes for some cases where the use of MPI_BOTTOM could cause
- Properly handle the case where an mVAPI does not have shared receive
queue support (such as the one shipped by SilverStorm / Infinicon
for OS X).
- Fixed assembly on Solaris AMD platforms. Thanks to Pierre Valiron
for bringing this to our attention.
- Fixed long messages in the send-to-self case.
- Ensure that when the "leave_pinned" option is used, the memory hooks
are also enabled. Thanks to Gleb Natapov for pointing this out.
- Fixed compile errors for IRIX.
- Allow hostfiles to have integer host names (for BProc clusters).
- Fixed a problem with message matching of out-of-order fragments in
multiple network device scenarios.
- Converted all the C++ MPI bindings to use proper const_cast<>'s
instead of old C-style casts to get rid of const-ness. Thanks to
Martin Audet for raising the issue with us.
- Converted MPI_Offset to be a typedef instead of a #define because it
causes problems for some C++ parsers. Thanks to Martin Audet for
bringing this to our attention.
- Improved latency of TCP BTL.
- Fixed index value in MPI_TESTANY to be MPI_UNDEFINED if some
requests were not MPI_REQUEST_NULL, but no requests finished.
- Fixed several Fortran MPI API implementations that incorrectly used
integers instead of logicals or address-sized integers.
- Fix so that Open MPI correctly handles the Fortran value for .TRUE.,
regardless of what the Fortran compiler's value for .TRUE. is.
- Improved scalability of MX startup.
- Fix datatype offset handling in the coll basic component's
- Fix EOF handling on stdin.
- Fix missing MPI_F_STATUS_IGNORE and MPI_F_STATUSES_IGNORE
instanatiations. Thanks to Anthony Chan for pointing this out.
- Add a missing value for MPI_WIN_NULL in mpif.h.
- Bring over some fixes for the sm btl that somehow didn't make it
over from the trunk before v1.0. Thanks to Beth Tibbitts and Bill
Chung for helping identify this issue.
- Bring over some fixes for the iof that somehow didn't make it over
from the trunk before v1.0.
- Fix for --with-wrapper-ldflags handling. Thanks to Dries Kimpe for
pointing this out to us.
Initial public release.