AMBER (scorep)

[jg piccinali]

Scorep/142

Daint

Setup

src

• module swap PrgEnv-cray PrgEnv-gnu
• module swap gcc gcc/4.8.2
• module load craype-accel-nvidia35
• module load cray-netcdf/4.3.3.1

Compile

• export AMBERHOME=/apps/daint/5.2.UP02/sandbox/jgp/amber/GNU482/amber14
• export CUDA_HOME=$CRAY_CUDATOOLKIT_DIR
• ./configure -mpi -cuda -noX11 --with-netcdf $NETCDF_DIR --no-updates -crayxt5 gnu
• MPI=/opt/cray/mpt/default/gni/mpich2-gnu/48
• make NVCC="nvcc -use_fast_math -O3 -arch=sm_35 -I$MPI/include"

Run

• benchmark
• export AMBERHOME="/apps/santis/sandbox/jgp/amber/GNU482/amber14"
• export PATH="${PATH}:${AMBERHOME}/bin"
• export LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:${AMBERHOME}/lib"
• cd Amber14_Benchmark_Suite/PME/JAC_production_NVE_4fs/

aprun -n4 -N1 -d1 -j1  ./pmemd.cuda.MPI -O -i mdin.CPU \
-o o_04c -inf mdinfo.04c \
-x mdcrd.04c -r restrt.04c

Compile (scorep)

• module load scorep/1.4.2

export PREP="scorep --mpp=mpi --cuda"
make \
NVCC="$PREP nvcc -use_fast_math -O3 -arch=sm_35 -I/opt/cray/mpt/default/gni/mpich2-gnu/48/include" \
CC="$PREP cc" \
CXX="$PREP CC" \
CPLUSPLUS="$PREP CC" \
FC="$PREP ftn" \
PMEMD_F90="$PREP ftn -DMPI -DBINTRAJ -DEMIL -DPUBFFT" \
PMEMD_CC="$PREP cc" \
PMEMD_LD="$PREP ftn \$(AMBERBUILDFLAGS)"

Profiling (scorep)

• export SCOREP_ENABLE_PROFILING=true
• export SCOREP_ENABLE_TRACING=false
• export SCOREP_CUDA_ENABLE=no

Filtering (scorep)

• scorep-score scorep-n04N1P/profile.cubex

Estimated aggregate size of event trace:                   212MB
Estimated requirements for largest trace buffer (max_buf): 61MB
Estimated memory requirements (SCOREP_TOTAL_MEMORY):       63MB

Tracing (scorep)

• export SCOREP_ENABLE_PROFILING=false
• export SCOREP_ENABLE_TRACING=true
• export SCOREP_CUDA_ENABLE=yes
• /apps/ela/vampir/8.5.0/bin/vampir scorep-n04N1T/traces.otf2