Wiki
Clone wikiupcxx / docs / alt-compilers
Guide to C++ standard libraries for non-GNU compilers on Linux
This document describes how to use a non-GNU C++ compiler (such as from Clang/LLVM, Intel or PGI/Nvidia) with UPC++ on a Linux system where the default C++ standard library is not recent enough to meet the requirements of UPC++.
Due to its use of C++11 features, UPC++ requires a fairly modern C++ compiler
and associated standard library. The minimum requirements are documented in
README.md. When using a non-GNU compiler, the default is most
often to use the C++ standard library of /usr/bin/g++
. Unfortunately, not
all current Linux distributions have a sufficiently recent g++
to meet the
requirements of UPC++.
This document assumes the reader has installed a version of g++
which does
meet the requirements, and desires to use its libstdc++
with a non-GNU C++
compiler. Be advised that not all C++ compilers are compatible with all
versions of libstdc++
. The most common issues relate to an inability of an
older compiler to parse a newer header. It is beyond the scope of this
document to provide detailed guidance in this respect. However, a good rule
of thumb is to avoid use of a g++
newer than the non-GNU compiler.
The remainder of this document consists of sections each covering one compiler family: Clang/LLVM, Intel or PGI/Nvidia. For each compiler family there is a description of how to query the C++ standard library to be used, and two approaches to override it:
-
A "temporary" override is described in which one must set
$CXX
when installing UPC++ and ensure a compatible invocation of the C++ compiler when compiling any sources to be linked into the same application. In general, this is less convenient than the permanent approach, but "safer" in that it will not have any effect on other uses of the compiler. -
A "permanent" override is described in which no specialized options are required in
$CXX
or when compiling sources to be linked into UPC++ applications (though in some cases a compiler-specific environment variable setting is required at install and compile time). In general, this is simpler to use than the temporary approach, but with the added risks that come from globally modifying the behavior of the compiler.
The "override" instructions can only ensure that a C++ compiler will use the correct header search path and linker search path. That leaves the matter of the runtime linker (loader) search path to be dealt with. The document local-gcc.md describes means to address runtime linking, and the majority of the approaches described there are relevant to non-GNU C++ compilers as well. The sections below will clarify which.
Clang/LLVM
Query
One can determine the C++ standard library to be used by clang++
with a
simple query. For example, the following shows a case in which a GCC 7.2.0
installation is used:
$ clang++ --print-file-name=libstdc++.so
/usr/local/gcc/7.2.0/lib/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../lib64/libstdc++.so
Alternatively, the following demonstrates looking for the information in the verbose compiler output:
$ clang++ -v -E -x c++ /dev/null 2>&1 | grep '^Selected GCC' Selected GCC installation: /usr/local/gcc/7.2.0/lib/gcc/x86_64-pc-linux-gnu/7.2.0
Temporary Override
To use a non-default C++ standard library, clang++
provides the command line
option --gcc-toolchain
. Its argument is an installation directory as would
be passed as the --prefix
when configuring GCC. For example, if
/usr/local/gcc/6.4.0/bin/g++
is a suitable g++
version, then one might
configure UPC++ with CXX="clang++ --gcc-toolchain=/usr/local/gcc/6.4.0"
.
Permanent Override
Detailed instructions on configuring and installing Clang/LLVM are beyond the
scope of this document. Instead, the reader is advised to see LLVM.org's
official Clang Getting Started page.
Text about C++ standard library headers will mention -DGCC_INSTALL_PREFIX
,
which is the cmake
analog to the --gcc-toolchain
option. Use of this
option when running cmake
to configure Clang will establish the default used
in the absence of an explicit --gcc-toolchain
option.
Runtime Linker Search Path
All approaches to control of the runtime linker search path given in
local-gcc.md are also applicable to clang++
. This includes
static linking of compiler-dependent libraries, where clang++
is command line
compatible with g++
.
Intel
Query
One can determine the C++ standard library to be used by icpc
with a simple
query. For example, the following shows a case in which a GCC 8.3.0
installation is used:
$ icpc --print-file-name=libstdc++.so
/usr/local/gcc/8.3.0/lib/gcc/x86_64-pc-linux-gnu/8.3.0/../../../../lib64/libstdc++.so
Alternatively, one can look for GCC directories in the header search path
printed as part of the stderr generated with -v
. For example, the output
of following may be helpful:
$ icpc -v -E -x c++ /dev/null 2>&1 | grep '^ .*/include' /usr/local/intel/2019/compilers_and_libraries_2019.4.227/linux/pstl/include /usr/local/intel/2019/compilers_and_libraries_2019.4.227/linux/compiler/include/icc /usr/local/intel/2019/compilers_and_libraries_2019.4.227/linux/compiler/include /usr/local/gcc/8.3.0/include/c++/8.3.0 /usr/local/gcc/8.3.0/include/c++/8.3.0/x86_64-pc-linux-gnu /usr/local/gcc/8.3.0/include/c++/8.3.0/backward /usr/local/include /usr/local/gcc/8.3.0/lib/gcc/x86_64-pc-linux-gnu/8.3.0/include /usr/local/gcc/8.3.0/lib/gcc/x86_64-pc-linux-gnu/8.3.0/include-fixed /usr/local/gcc/8.3.0/include/ /usr/include
Temporary Override
To use a non-default C++ standard library, icpc
provides the -gxx-name
command line option. The argument to this option is the full path to a
suitable g++
. It has been observed, however, that some installations
of the Intel compilers use mismatched paths for the C++ headers and libraries
unless -gcc-name
is passed. Therefore, we advise use of both options.
For instance, one might configure UPC++ using
CXX="icpc -gxx-name=/usr/local/gcc/6.4.0/bin/g++ -gcc-name=/usr/local/gcc/6.4.0/bin/gcc"
.
Alternatively, one can consult the vendor-provided documentation for
information on "Using Response Files". A response file, prefixed by @
, can
serve as a convenient macro replacement for multiple command line options.
For example, one might set CXX="icpc @$HOME/etc/icpc_upcxx.txt"
and place
the following in $HOME/etc/icpc_upcxx.txt
:
-gxx-name=/usr/local/gcc/6.4.0/bin/g++ -gcc-name=/usr/local/gcc/6.4.0/bin/gcc -Wl,-rpath=/usr/local/gcc/6.4.0/lib64
Permanent Override
For detailed instructions on configuration and installation of the Intel
compiler suite, the reader is advised to consult the vendor-provided
documentation, where one should look for information on "Using
Configuration Files". As described there, one can create an icpc.cfg
file containing options to be processes as if they appear on the command
line before any explicit options. The content is precisely the same
as for a response file, shown immediately above.
If one has administrative privilege, a global icpc.cfg
can be placed in
the same directory as the icpc
executable. However, any user may create a
private icpc.cfg
file, and set the environment variable ICPCCFG
to its
location.
Runtime Linker Search Path
All approaches to control of the runtime linker search path given in
local-gcc.md are also applicable to icpc
. This includes
static linking of compiler-dependent libraries, where icpc
is command line
compatible with g++
.
Additionally, as demonstrated in the examples, both response files and
icpc.cfg
provide a convenient means to pass linker options such as
-Wl,-rpath=...
implicitly.
PGI and Nvidia
Query
One can determine the C++ library to be used by pgc++
or nvc++
by examining its
standard include path looking for GCC installation directories. The following
examples extract relevant information from verbose compiler output, and
indicates that an installation of GCC 9.1.0 is being used.
For PGI versions through 20.4:
$ pgc++ -v -E /dev/null 2>&1 | perl -ne 'if(m/stdinc ([^ ]*)/){print($1=~tr/:/\n/r);}' /usr/local/pgi/linux86-64-llvm/19.7/include-gcc70 /usr/local/pgi/linux86-64-llvm/19.7/include /usr/local/gcc/9.1.0/lib/gcc/x86_64-pc-linux-gnu/9.1.0/include /usr/local/include /usr/local/gcc/9.1.0/include /usr/local/gcc/9.1.0/lib/gcc/x86_64-pc-linux-gnu/9.1.0/include-fixed
For PGI and NVHPC versions 20.9 and newer:
$ nvc++ -v -E /dev/null 2>&1 >/dev/null | perl -0x20 -ne 'if(m,^-I/,) {print("$_\n");}' -I/usr/local/nvidia/hpc_sdk/Linux_x86_64/21.5/compilers/include-gcc70 -I/usr/local/nvidia/hpc_sdk/Linux_x86_64/21.5/compilers/include -I/usr/local/gcc/9.1.0/include/c++/9.1.0 -I/usr/local/gcc/9.1.0/include/c++/9.1.0/x86_64-pc-linux-gnu -I/usr/local/gcc/9.1.0/include/c++/9.1.0/backward -I/usr/local/gcc/9.1.0/lib/gcc/x86_64-pc-linux-gnu/9.1.0/include -I/usr/local/include [truncated]
If the command above yields an empty result, one can look for the
colon-separated list following -stdinc
(for PGI <= 20.4) or the
-I
options (PGI or NVHPC >= 20.9) in the output of the compiler
commands above (the portions before | perl ...
).
Temporary Override
NVHPC compilers 21.11 and newer support a --gcc-toolchain
option to specify a
non-default C++ standard library. Its argument is an installation directory as
would be passed as the --prefix
when configuring GCC. For example, if
/usr/local/gcc/9.1.0/bin/g++
is a suitable g++
version, then one might
configure UPC++ with CXX="nvc++ --gcc-toolchain=/usr/local/gcc/9.1.0"
.
Compilers prior to 21.11 did not provide a simple means to override the GCC
installation from which the C++ standard library and headers are used.
Instead this information is contained in a configuration file, which one
must generate as described in the following "Permanent Override" section.
Given a suitable configuration file, one can provide it to an invocation of
pgc++
using the -rc
command line option. So, one might install UPC++
using CXX="pgc++ -rc=[full path to config file]"
.
Permanent Override
For detailed instructions on configuration and installation of the PGI or
NVHPC compiler suite, the reader is advised to consult the vendor-provided
documentation, where one should look for information on the makelocalrc
utility. The basic usage instructions for makelocalrc
are available by
running it with no arguments. The following assumes one has read and
understands those instructions.
When running makelocalrc
, each of the following environment variables must
be unset (not just empty): CPATH
, C_INCLUDE_PATH
, CPLUS_INCLUDE_PATH
.
If they are set, then their values become embedded in the header search
path variables in the generated localrc file. If necessary, one can run
$ env -u CPATH -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH makelocalrc [args]
By default, makelocalrc
generates a configuration file suitable for the
gcc
, g++
and g77
(or gfortran
) in /usr/bin
. To override these, one
uses the correspondingly-named command line options. It is strongly advised
to override all three compilers with versions from a single installation if
possible. For example, the following would be suitable for the case in which
the desired compilers are found in ones $PATH
:
$ makelocalrc [output options] -gcc $(which gcc) -g++ $(which g++) -g77 $(which gfortran)
Note that while the option -g++
has been used in the example above, the
correct option may be either -g++
or -gpp
, depending on the compiler version.
A system administrator can install a global configuration file. However,
any user may create a private configuration file, and either set the
environment variable PGI_LOCALRC
(<= 20.4) or NVLOCALRC
(>= 20.9) to its
location or use the -rc
command line option to pass it at each compiler
invocation.
Runtime Linker Search Path
Nearly all approaches to control of the runtime linker search path given in
local-gcc.md are also applicable to pgc++
. The exception is
static linking of compiler-dependent libraries, where pgc++
is not command
line compatible with g++
and has no equivalent.
We are not aware of any official documentation on the format of a localrc file. However, much useful information be found online in the NVIDIA Developer Forums, and especially in the "Legacy PGI Compilers" section. Based on information found there, we can recommend (but cannot warranty) addition of a line such as the one below to the end of any localrc file. This instructs the compiler to add the given directory to the executable's RPATH.
set LOCALCOMPLIB=/usr/local/gcc/6.4.0/lib64;
Alternatively, other sources recommend the following:
append USRRPATH=-rpath /usr/local/gcc/6.4.0/lib64;
Updated