HTTPS SSH

param-sweep.sh

Creates separate directories for an array of separate jobs with different parameters and submits those jobs.

More in-depth explanation in one of my blog posts.

To use:

1. Create input file template:

Contents of bcp.input with variable NRLENGTH (can use multiple variables):

1000        # Number of iterations
60          # Polymer length
1           # Nanorod radius
NRLENGTH    # Nanorod length

2. Create submission script template:

Contents of sub.sh with variable TRIALNAME:

#!/bin/sh
#PBS -N TRIALNAME
#PBS -l nodes=1:ppn=12
#PBS -l walltime=01:00:00,mem=2gb

cd $PBS_O_WORKDIR

# Run code that happens to look for bcp.input in the current directory
mpirun $HOME/code/awesome_code.exe

3. Create trials file:

Contents of trials.txt:

name        i:NRLENGTH  s:TRIALNAME
length1     4.0         length1-trial
length2     5.0         length2-trial
length3     6.0         length3-trial

The name column means "make directories with the names below".

The i:NRLENGTH means "look for NRLENGTH in the input file and replace each one with the values below".

The s:TRIALNAME column means "look for TRIALNAME in the submit file and replace each one with the values below".

4. Run command:

$ param-sweep.sh -i bcp.input -s sub.sh -t trials.txt

Or, since bcp.input, sub.sh, and trials.txt are what I have hardcoded into the script, I can just use

$ param-sweep.sh

This creates directories length1/, length2/, and length3/, copies bcp.input and sub.sh into each directory, and makes the variable replacements in the copied files as specified in the trials.txt file, then submits all the jobs using qsub or squeue, depending on what is available on your system.

To create the directories but not submit, use the -n flag (think "no submit"):

$ param-sweep.sh -i bcp.input -s sub.sh -t trials.txt -n