Building CHEF using CMake

The list of operating systems explicitly supported by this build mechanism is:

0. Ubuntu 20.04 LTS (using the system Python 3)
1. Ubuntu 18.04 LTS (using the system Python or Python 3)
2. Ubuntu 16.04 LTS (using the system Python or Python 3)
3. macOS 10.13.x -- 10.14.x (High Sierra and Mojave) (using Python 2 or Python 3)
4. Fedora 30 (Using Python 3)

The general organization of the build is that a set of libraries is expected to be on the user's system, typically as installed by the system's package manager. In the case of macOS, we assume the use of Homebrew (see below). On Linux systems, Python packages are assumed to have been installed using the system's package manager. On macOS, Python packages are assumed to have been installed using the package's setup.py script, as is done by pip.

Installing the required packages for Ubuntu 20.04 LTS, 18.04 LTS and Ubuntu 16.04 LTS

We assume that you will be building and installing both CHEF and Synergia, so these instructions provide unified instructions.

For Ubuntu 16.04 LTS (but not later), you need to install a more modern CMake than that which is available from the system's package manager. A binary installation can be installed directly from the cmake.org web site. At the time of this writing (10 Nov 2020) the current version is cmake v3.18.4; newer versions are expected to work. The following instructions will download and install this version:

  cd $HOME
  wget https://github.com/Kitware/CMake/releases/download/v3.18.4/cmake-3.18.4-Linux-x86_64.tar.gz
  tar --strip-components=1 -x -f cmake-3.18.4-Linux-x86_64.tar.gz
  rm cmake-3.18.4-Linux-x86_64.tar.gz

The cmake executable (and others) will be put into ${HOME}/bin, which must appear on {PATH}; add this directory if necessary.

The following list of commands will install the libraries, headers, and Python modules required for building and using CHEF, using Python 2.

apt-get install -y cmake
apt-get install -y fftw3-mpi-dev
apt-get install -y fftw3-dev
apt-get install -y g++
apt-get install -y git
apt-get install -y gsl-dev
apt-get install -y libboost-dev
apt-get install -y libboost-filesystem-dev
apt-get install -y libboost-iostreams-dev
apt-get install -y libboost-python-dev
apt-get install -y libboost-regex-dev
apt-get install -y libboost-serialization-dev
apt-get install -y libboost-system-dev
apt-get install -y libboost-test-dev
apt-get install -y libeigen3-dev
apt-get install -y libgsl-dev
apt-get install -y libhdf5-dev
apt-get install -y libopenmpi-dev
apt-get install -y openmpi-bin

For Python 2 (deprecated), install the following:

apt-get install -y python-dev
apt-get install -y python-mpi4py
apt-get install -y python-nose
apt-get install -y python-numpy
apt-get install -y python-pyparsing
apt-get install -y python-tables

To build using Python 3, install these packages:

apt-get install -y python3-dev
apt-get install -y python3-pip
apt-get install -y python3-mpi4py
apt-get install -y python3-nose
apt-get install -y python3-pyparsing
apt-get install -y python3-tables

Installing the required packages for macOS "Mojave" and "Catalina"

Building CHEF requires installation of Apple's development environment, Xcode, or at least the Xcode command-line tools. Xcode is available, free of charge, from the Apple's App Store. We recommend keeping your Xcode installation up-to-date. Installing Xcode requires an Apple ID, which is also available free of charge. After installing Xcode, the Xcode command-line tools can be installed by running:

xcode-select --install

If you already have the command-line tools installed, this will result in a harmless error message:

xcode-select: error: command line tools are already installed, use "Software Update" to install updates

The following list of commands will install the libraries, headers, and Python modules required for building and using CHEF. We rely upon Homebrew's python, which is newer than that shipped by Apple. Homebrew is available at brew.sh.

We strongly recommend keeping your Homebrew installation up-to-date by regularly using both brew update and brew upgrade.

brew update
brew upgrade
brew install boost
brew install boost-python3
brew install cmake
brew install eigen
brew install fftw
brew install gsl
brew install hdf5
brew install libomp
brew install mpi4py
brew install numpy
brew install pkgconfig
brew install python   # This installs Python 3.x

To be able to run some of tests for Synergia, you will also need some Python modules that are not part of the standard Python library, nor are available through Homebrew. We recommend making use of a virtual environment for this installation. You can create the virtual environment in any directory that is convenient; you will need to retain this directory so that you can activate the virtual environment whenever you want to use any Synergia python script (such as the Synergia tests) that use these modules.

In whatever directory you want to create the virtual environment, use the following commands:

# Make sure to use --system-site-packages, so python can find
# Homebrew-installed packages (numpy, mpi4py,...)
python3 -m venv --system-site-packages work-venv
source work-venv/bin/activate
python -m pip install -U pip
python -m pip install nose
python -m pip install pyparsing
deactivate

Whenever you want to have access to these modules, you must source the activate script:

source work-venv/bin/activate

Installing the required packages for Fedora 30

The following command will install the libraries, headers, and Python modules required for building and using CHEF, using Python 3.

dnf -y install \
    boost-devel \
    boost-python3 \
    boost-python3-devel \
    bzip2 \
    cmake \
    eigen3-devel \
    fftw-openmpi-devel \
    gcc \
    gcc-c++ \
    gsl-devel \
    hdf5-devel \
    hdf5-openmpi-devel \
    make \
    mpi4py-openmpi \
    numpy \
    openmpi \
    openmpi-devel \
    python3-devel \
    python3-ipython \
    python3-tables \
    python3-mpi4py-openmpi \
    python3-nlopt \
    python3-pydot \
    zlib-devel

Building and installing CHEF

These instructions assume that you have already installed the required packages for your operating system, as described above.

0. Create a top-level working directory

   This is the directory into which you will clone the source (for both CHEF and Synergia) and build the software. Note this is distinct from the directory into which you will install the resulting build.

   Define the environment variable SYNERGIA_TOP_DIR to point to this directory.

1. Obtain the source code

   The CHEF code is in a git repository, and can be obtained by

   git clone https://bitbucket.org/fnalacceleratormodeling/chef.git
   cd chef
   git checkout master
   

   Note that this branch is the only one currently supporting this CMake build.

2. Create the build directory

   We use an out-of-source build to prevent accidental pollution of the source code repository, and to simplify having multiple builds (e.g. debug and release) of the same source code.

   mkdir build
   cd build
   

3. Generate makefiles

   Special note for Fedora 30: The Fedora 30 installation of MPI is only active in your environment after being "loaded" by the modules system. The command to load the require module is:

   module load mpi
   

   You must do this in every new shell in which you wish to use CHEF or Synergia.

   End note.

   You must decide where to want to install the CHEF software you build. Define the environment variable CHEF_INSTALL_DIR appropriately; make sure the named directory exists.

   In the example below, we define CHEF_INSTALL_DIR so that CHEF is installed under the user's home directory. If you are building on Fedora 30, please note the extra cmake argument that is required to help find the FFTW3 MPI libraries.

   export CHEF_INSTALL_DIR=$HOME/install-chef
   mkdir -p "${CHEF_INSTALL_DIR}"
   # For the Python 3 build, add -DUSE_PYTHON_3=1 to the cmake command line, before the ..
   # For Fedora 30, add -DFFTW3_LIBRARY_DIRS=/usr/lib64/openmpi/lib to the cmake command line
   cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX="${CHEF_INSTALL_DIR}" ..
   

4. Build the code

   The makefiles generated by CMake support parallel builds. We find setting the level of parallelism (the argument of -j, 4 in the example below) to be equal to the number of cores on your computer generally works well.

    make -j 4
   

5. Test the code

   The CTest system supported by CMake can support parallel tests. It is expected that all provided tests will pass; if any test does not pass on your system, please file an issue.

    ctest -j 4
   

6. Install the code

   Only install the code if all tests have passed. The libraries, headers, and python modules will be installed in CHEF_INSTALL_DIR, as defined above.

   make install
   

When you are done, you may also want to build Synergia2. See Synergia build instructions.