Installation

PHIST has been developed for unix-like environments and tested with recent versions of GCC (>=4.9.2) and the Intel compilers (>=14.0), and with MPI libraries from Intel, OpenMPI and MPICH. The build system uses CMake tool in a reasonably recent version (it will complain if the used cmake is too old).

Some sample build scripts are available in phist/buildScripts, they may provide a good starting point, although they might be somewhat outdated and paths may have to be adjusted.

For users of the SPACK package manager, a package.py file for phist can be found here.

Basic steps

make a build directory:

> mkdir build

you need cmake, lapack and MPI, for instance at DLR you can use

> module add PrgEnv/gcc-7.2.0-openmpi lapack

As a start, use the builtin kernel lib without third-party libraries (TPLs):

> cd build/ > cmake -DPHIST_KERNEL_LIB=builtin ..

Available targets for building just a certain component (libraries, drivers, or tests) are

> make libs > make drivers > make tests

Required for installation via

> make install

is only the target libs.

A large number of configuration options can be set to fine-tune the behavior of PHIST. We recommend using the ccmake tool for viewing/editing the configuration after the initial invocation of cmake, e.g.

> ccmake ..

A list of some useful settings is found at the end of this page]]

Testing the installation

To run some unit tests type

> make test

or

> make check

(to get verbose error messages)

The above commands require that CMake can invoke an MPI program, so on a cluster you may need an interactive job. If the tests fail to launch, you may want to use the ccmake command to edit your configuration, press t to toggle advanced options and set the MPIEXEC item (and related ones).

To 'smoke-test' the installation (after 'make install') use

> make test_install

which will try to compile a simple external application with the phist headers and libraries installed.

Selecting the BLAS and LAPACK libraries to be used

By default, phist will try to locate its required dependencies BLAS, LAPACK and LAPACKe on your system. If you are using the Intel(R) compilers, it will use the flag '-mkl=sequential', otherwise it uses a standard CMake mechanism. This mechanism looks for a variety of vendor libraries. To guide the process towards the library you want to use you can set the variable BLA_VENDOR (via the cmake argument -DBLA_VENDOR=[...] or as an environment variable) to any supported string, e.g. "Generic", "GOTO", "Intel10_64lp_seq" (for a list of supported values see the FindBLAS.cmake module of your cmake installation). Note: phist uses these libraries only for small-scale computations and it is *not advisable* to use a parallel BLAS/LAPACK installation. This is why we use the sequential MKL version with the Intel compiler.

As an alternative to this procedure, you can provide the full paths to the libraries to be used via the variable TPL_LAPACKE_LIBRARIES. The Intel(R) MKL includes the lapacke library, so no additional library has to be linked in this case. The header file lapacke.h (or mkl_lapacke.h) is required by PHIST, if it fails to find it you may need to supply it via the variable TPL_LAPACKE_INCLUDE_DIRS.

Installing in a custom location and using phist in another project

In order to install PHIST in a non-standard location (say $PHIST_HOME) (the default is /usr/local/), add the cmake flag -DCMAKE_INSTALL_PREFIX=${PHIST_HOME. In order to use phist from another project, set the environment variables (assuming bash)

> export PATH+=${PHIST_HOME}/bin > export LIBRARY_PATH+=${PHIST_HOME}/lib > export LD_LIBRARY_PATH+=${PHIST_HOME}/lib

For CMake projects (recommended approach):

> export CMAKE_PREFIX_PATH+=${PHIST_HOME}/

Or you can use PkgConfig to find the phist libs and heaqders:

> export PKG_CONFIG_PATH+=${PHIST_HOME}/lib/pkgconfig

Examples of how to set up an external project using CMake or PkgConfig can be found in phist/exampleProjects/.

Dependencies and optional packages

all kernel libraries
- required:
  - LAPACKe, LAPACK and BLAS (see above).
- optional:
  - MPI is recommended but not strictly needed for building PHIST (see option PHIST_USE_MPI=ON/OFF). For instance, the Eigen kernel library is not MPI-parallel ad so it is not needed with phist either.
  - Belos and Anasazi (The Trilinos packages implementing block iterative solvers for linear and eigenvalue problems, respectively)
  - essex/physics: https://bitbucket.org/essex/physics (if not available, some
builtin
- required:
  - MPI
- optional:
  - ParMETIS: http://glaros.dtc.umn.edu/gkhome/metis/parmetis/overview (configured with IDXTYPEWIDTH=64)
  - ColPack: http://cscapes.cs.purdue.edu/coloringpage/software.htm example physics problems like the spin chain and graphene have been copied into the phist repo and will be used, but they might be outdated).
ghost
- required:
  - MPI is required if GHOST was built with MPI.
  - essex/ghost: https://bitbucket.org/essex/ghost and dependencies
- optional:
  - additional libraries depend on the ghost installation (e.g. Zoltan for repartitioning)
epetra/tpetra
- required:
  - Trilinos: http://trilinos.sandia.gov/
- optional: depends on the Trilinos installation, e.g. Ifpack(2), ML, Zoltan/Isorropia

Typically CMake will automatically find the TPL if it is in your PATH, CMAKE_PREFIX_PATH, PKG_CONFIG_PATH or the like. By default phist searches for all supported TPLs, to fine-tune this behavior you can use the variables

PHIST_USE_SOLVER_TPLS=ON/OFF: enable/disable Belos, Anasazi and the like
PHIST_USE_PRECON_TPLS=ON/OFF: enable/disable preconditioning packages for which we have an interface (e.g. Ifpack)
PHIST_USE_GRAPH_TPLS=ON/OFF: enable/disable graph partitioning and coloring tools such as ParMETIS, Zoltan, ColPack
PHIST_USE_TRILINOS_TPLS=ON/OFF: If Trilinos is in your search path but you want to disable it (in particular when building with other kernel libraries such as builtin or ghost), this variable allows you to disable all Trilinos TPLs at once.

For Belos and Anasazi, there are an advanced options to disable using the PHIST orthogonalization routine:

PHIST_BELOS_THEIR_ORTHO_MANAGER=ON
PHIST_ANASAZI_THEIR_ORTHO_MANAGER=ON

The background here is that the Trilinnos implementations may be advantageous if the multi-vectors (mvecs) are stored in column-major order, as in Epetra and Tpetra.

Installation notes for specific machines

The information below is gathered experience of the past few years and is not necessarily up-to-date. It may, however, prove useful if you need to run phist and/or ghost on one of the machines or a similar setup.

Installation instructions for the RRZE systems

----------------------------------------------

Proceed as follows to build PHIST with GHOST (substitute $PREFIX with the path where ghost is installed). This is the easiest way to make PHIST find GHOST. It is also possible to pass the `GHOST_DIR` variable to cmake, insterad.

> module load intel64 > module load cmake > cd build/ > export CC="mpicc -mt_mpi" \ CXX="mpicxx -mt_mpi" \ FC="mpif90 -mt_mpi" > cmake -DPHIST_KERNEL_LIB=ghost \ -DMPIEXEC=mpirun_rrze \ -DCMAKE_INSTALL_PREFIX=${PREFIX} \ .. > make

If you want to use likwid, also set `LIKWID_HOME` and pass `-DLIKWID_PERFMON` to cmake.

Notes for non-Intel machines (e.g. SPARC)

Basically phist should run on any machine as long as the kernel library does. For the builtin kernels, we do assume by default that the CPU supports SSE instructions. If this is not the case, you should pass the following option to cmake (or set the corresponding entry using ccmake):

-DPHIST_HAVE_SSE=OFF

In the future, we will automatically detect which instructions are supported by the CPU.

Notes for SuperMUC

module load intel mkl hwloc git cmake parmetis nano

export CMAKE_PREFIX_PATH=${MKLROOT}:${CMAKE_PREFIX_PATH} (don't know why this is required here but not on Emmy)

I experienced problems on SuperMUC, especially with the high precision kernels in the builtin library and some of the tests with ghost!

Wiki

phist / User_Guide / 000 Installation