Flux is a high-performance cluster available to students at the University of Michigan. For useful information on accessing Flux: Flux in 10 easy steps

Compiling the code on Flux

Follow the guidelines below in order to get access to Flux and be able to compile the code.

• Establish a user account (refer to the flux user guide to fill out the form)
• If you are not on campus or connecting from an insecure wireless network (e.g., MGuest), you must first connect via VPN. Click here for details on getting started with the UMich VPN.
• Once you have an account name and are connected to the UMich network, log on to Flux: ssh -Y <username>@flux-login.arc-ts.umich.edu
• Load appropriate modules
  • Check to see if you have the modules loaded:
    • module list displays all loaded modules
    • module avail displays all available modules
  • Load your desired compilers, for example:

module load gcc/5.4.0
module load openmpi/1.10.2/gcc/5.4.0
module load fftw/3.3.4/gcc/5.4.0

For convenience, you should add the module load commands in your .bashrc file so you do not have to reload them every time you log in.

• Copy over the source code to your home directory scp -r jcode <username>@flux-login.arc-ts.umich.edu:/.
• Make sure the compilers in the Makefile.in point to the MPI compilers that you loaded. In general, simply using mpif90, epic, etc. should work.

Running on Flux

Jobs should be submitted from the scratch directory. To submit a job, execute the command qsub run where run is a PBS script that tells the machine important information, e.g., how many cores to use, how long to run for, the location of the executable... Below is an example run script

#!/bin/bash 

#PBS -N channel
#PBS -M <username>@umich.edu
#PBS -m abe
#PBS -A jcaps_flux
#PBS -q flux
#PBS -l qos=flux
#PBS -l nodes=30:ppn=16,pmem=2gb
#PBS -l walltime=1:00:00
#PBS -j oe
#PBS -V

####  End PBS preamble

if [ -s "$PBS_NODEFILE" ] ; then
    echo "Running on"
    cat $PBS_NODEFILE
fi

if [ -d "$PBS_O_WORKDIR" ] ; then
    cd $PBS_O_WORKDIR
    echo "Running from $PBS_O_WORKDIR"
fi

#  Put your job commands after this line
mpirun -np 480 ~/jcode/bin/jcode input > sim.out

You can check the status of your job using showq -u <username>

To kill a job use qdel <jobid>