Midway is a large (10,000 cpu) cluster at the University of Chicago operated by the Research Computing Center. Each Midway compute node has 16 Intel SandyBridge cores and 32GB of memory, and connected to a high-performance GPFS storage system via infiniband. For more details on Midway and how to use it, see the RCC documentation.
Preparing your Environment
The RCC uses a modules system to manage libraries and applications installed on Midway. When using CosmoSIS, you will not be using most of the installed software but rather the UPS packages provided by CosmoSIS. In particular we recommend keeping your environment as clean as possible, to avoid possible conflicts between CosmoSIS and installed package (python in particular).
You will need to load the git module in order to install and update your local CosmoSIS directory. One of Midway's MPI environments is also necessary for cosmosis-bootstrap to install mpi4py and make mpirun available.
module load git module load intelmpi/5.0
These commands will need to be run every time you log into Midway, or be placed in your login scripts (.bash_profile or .bashrc).
The CosmoSIS team maintains a central copy of the CosmoSIS UPS packages and Planck data, which can be identified using the
-u flag. The instructions for downloading CosmoSIS are given below.
curl -L --remote-name https://bitbucket.org/dale_mellor/cosmosis-bootstrap/downloads/cosmosis-bootstrap chmod u+x cosmosis-bootstrap ./cosmosis-bootstrap -u /project/databases/cosmosis <new desired target directory> cd <the new desired target directory that just got created> source config/setup-cosmosis make
Be sure to replace <cosmosis target directory> with the location you would like to install CosmoSIS to.
Once you have installed CosmoSIS, every time you log in you just need to source the CosmoSIS environment by
source <cosmosis target directory>/config/setup-cosmosis
Running Jobs On Midway
Midway is a shared resource and it is forbidden to run computation or memory intensive programs on the login nodes. CosmoSIS runs that use the Test Sampler such as Demo 1 and Demo 2 are fine, as is plotting and basic post-processing analysis.
You must run larger tasks on a Midway compute node, either by submitting a slurm batch script or by entering an interactive batch session. See RCC's documentation for more information on interacting with the Slurm batch scheduling system.