UPC++: a PGAS library for C++
UPC++ is a parallel programming library for developing C++ applications with the Partitioned Global Address Space (PGAS) model.
UPC++ has three main objectives:
Provide an object-oriented PGAS programming model in the context of the popular C++ language
Expose useful asynchronous parallel programming idioms unavailable in traditional SPMD models, such as remote function invocation and continuation-based operation completion, to support complex scientific applications
Offer an easy on-ramp to PGAS programming through interoperability with other existing parallel programming systems (e.g., MPI, OpenMP, CUDA)
The rest of this document provides basic information for command-line use of the UPC++ software implementation.
Other topics are covered in the following documents:
- Installing the UPC++ software, see: INSTALL.md
- Tutorial on programming with UPC++, see: UPC++ Programmer's Guide
- Formal details on UPC++ semantics, see: UPC++ Specification
- Running basic UPC++ correctness tests, see: docs/testing.md
- Software change history of UPC++, see: ChangeLog.md
- Debugging UPC++ programs, see: docs/debugging.md
- Using UPC++ and MPI in the same program, see: docs/mpi-hybrid.md.
- Using UPC++ and UPC in the same program, see: docs/upc-hybrid.md.
- Using UPC++ with oversubscribed cores, see: docs/oversubscription.md
- Implementation-defined behavior, see: docs/implementation-defined.md
- Copyright notice and licensing agreement, see: LICENSE.txt
To report problems or request features: issue tracker.
Compiling Against UPC++ on the Command Line
With UPC++ installed, the easiest way to build a UPC++ application from the
command line is to use the
upcxx compiler wrapper, installed in
<upcxx-install-path>/bin/upcxx. The arguments to this wrapper work
just like the C++ compiler used to install UPC++ (analogous to the
mpicxx compiler wrapper often provided for MPI/C++ programs).
For example, to build an application consisting of
export PATH="<upcxx-install-path>/bin/:$PATH" upcxx -O -c my-app1.cpp my-app2.cpp upcxx -O -o my-app my-app1.o my-app2.o -lm
Be sure that all commands used to build one executable consistently pass either a -O option to select the optimized/production version of UPC++ (for performance runs), or a -g option to select the debugging version of UPC++ (for tracking down bugs in your application).
To select a non-default network backend or thread-safe version of the library,
you'll need to pass the -network= or -threadmode= options, or set the
UPCXX_THREADMODE variables prior to invoking compilation.
See the 'UPC++ Backends' section below.
Compiling Against UPC++ in Makefiles
The simplest way to build UPC++ programs from a Makefile is to use the
upcxx compiler wrapper documented in the section above to replace your
normal C++ compiler command.
If your Makefile structure prevents this and/or requires extraction of the
underlying compiler flags to build against UPC++, your build process can
query this information by invoking the
<upcxx-install-path>/bin/upcxx-meta <what> script, where
which form of flags are desired. Valid values are:
CXX: The C++ compiler used to install UPC++, which must also be used for building application code.
CPPFLAGS: Preprocessor flags which will put the upcxx headers in the compiler's search path and define macros required by those headers.
CXXFLAGS: Compiler flags which set debug/optimization settings, and set the minimum C++ language level required by the UPC++ headers.
LDFLAGS: Linker flags usually belonging at the front of the link command line (before the list of object files).
LIBS: Linker flags belonging at the end of the link command line. These will make libupcxx and its dependencies available to the linker.
For example, to build an application consisting of
my-app2.cpp using extracted arguments:
meta="<upcxx-install-path>/bin/upcxx-meta" $($meta CXX) $($meta CPPFLAGS) $($meta CXXFLAGS) -c my-app1.cpp $($meta CXX) $($meta CPPFLAGS) $($meta CXXFLAGS) -c my-app2.cpp $($meta CXX) $($meta LDFLAGS) my-app1.o my-app2.o $($meta LIBS)
For an example of a Makefile which builds UPC++ applications, look at
example/prog-guide/Makefile. This directory also
has code for running all the examples given in the programmer's guide. To use
Makefile, first set the
UPCXX_INSTALL shell variable to the
Using UPC++ with CMake
A UPCXX CMake module is provided in the installation directory. To use it
in a CMake project, append the UPC++ installation directory to the
cmake ... -DCMAKE_PREFIX_PATH=/path/to/upcxx/install/prefix ...),
find_package(UPCXX) in the
CMakeLists.txt file of the project.
If it is able to find a compatible UPC++ installation, the CMake module
will define a
UPCXX:upcxx target (as well as a
variable for legacy projects) that can be added as dependency to
UPC++ provides multiple "backends" offering the user flexibility to choose the
means by which the parallel communication facilities are implemented. Those
backends are characterized by three dimensions: conduit, thread-mode, and
code-mode. The conduit and thread-mode parameters map directly to the GASNet
concepts of the same name (for more explanation, see below). Code-mode selects
between highly optimized code and highly debuggable code. The
script will assume sensible defaults for these parameters based on the
installation configuration. The following environment variables can be set to
influence which backend
UPCXX_NETWORK=[aries|ibv|smp|udp|mpi]: The GASNet network backend to use for communication (the default and available values are system-dependent):
ariesis the high-performance Cray XC network.
ibvis the high-performance InfiniBand network.
smpis the high-performance choice for single-node multi-core runs.
udpis a portable low-performance alternative for testing and debugging.
mpiis a portable low-performance alternative for testing and debugging.
UPCXX_THREADMODE=[seq|par]: The value
seqlimits the application to only calling "communicating" upcxx routines from the thread that invoked
upcxx::init, and only while that thread is holding the master persona. The benefit is that
seqcan be synchronization-free in much of its internals. A thread-mode value of
parselects the thread-safe library that allows any thread in each process to issue communication as allowed by the specification, enabling greater injection concurrency in a multi-threaded application, at the expensive of greater internal synchronization (higher overhead per operation).
The default value is always
O3is for highly compiler-optimized code.
debugproduces unoptimized code, includes extra error checking assertions, and is annotated with the symbol tables needed by debuggers. The default value is always
Running UPC++ Programs
To run a parallel UPC++ application, use the
upcxx-run launcher provided in
<upcxx-install-path>/bin/upcxx-run -n <ranks> <exe> <args...>
This will run the executable and arguments
<exe> <args...> in a parallel
<ranks> number of UPC++ processes.
Upon startup, each UPC++ process creates a fixed-size shared memory heap that will never grow. By
default, this heap is 128 MB per process. This can be adjusted by passing a
upcxx-run, which takes a suffix of KB, MB or GB; e.g. to reserve 1GB per process, call:
<upcxx-install-path>/bin/upcxx-run -shared-heap 1G -n <ranks> <exe> <args...>
There are several additional options that can be passed to
upcxx-run -h for a complete list of options.
The canonical version of this document is located here: https://upcxx.lbl.gov/wiki/README.md
For more information, please visit the UPC++ home page