JACOBI - MPI/OMP - ENERGY

[jg piccinali]

DAINT

Get the src

• ssh -Y daint01
• git clone --single-branch -b jacobi https://github.com/eth-cscs/parallel-debuggers.git
• cd parallel-debuggers/jacobi.git/src/

Compile

CCE

module load PrgEnv-cray/5.1.29
module rm totalview-support/1.1.4
module rm totalview/8.11.0
module swap atp/1.7.1 atp/1.7.4
module swap cce/8.2.4 cce/8.3.1
module swap cray-libsci/12.1.3 cray-libsci/13.0.0
module swap cray-mpich/6.2.2 cray-mpich/7.0.1
module swap craype/2.05 craype/2.1.3
module swap ddt/4.2.1.as ddt/4.3rc2
module swap pmi/5.0.2-1.0000.9906.117.2.ari pmi/5.0.4-1.0000.10161.132.4.ari
module load perftools/621jg

• make clean
• make mpiomp PREP=

cc -D_CSCS_ITMAX=150 -O3  \
-DOMP_MEMLOCALITY -homp -DUSE_MPI -c jacobi_openmp.c \
-o jacobi_mpi+openmp.o

cc -homp jacobi_mpi+openmp.o \
-o ./jacobi_mpi+openmp.CRAY.DAINT.

Instrument the code

pat_build -f -u -g mpi,omp jacobi_mpi+openmp.CRAY.DAINT.

Choose the energy hardware counters

export PAT_RT_SUMMARY=0
export PAT_RT_PERFCTR=default_samp

• More infos about the counters (SandyBridge):
  • $CRAYPAT_ROOT/share/counters/Counters.papi_intel_fam6mod45
  • $CRAYPAT_ROOT/share/counters/CounterGroups.intel_fam6mod4
  • man pmpc
  • aprun -n1 papi_avail

Run

aprun -n 2 -N 1 -d 8 -cc cpu ./jacobi_mpi+openmp.CRAY.DAINT.+pat   4096 4096

CrayPat/X:  Version 6.2.1 Revision 13075  08/27/14 12:39:06
pat[WARNING][0]: HW performance counter multiplexing enabled
Jacobi relaxation Calculation: 4096 x 4096 mesh with 2 processes and 8 threads.
    0, 0.250000
  100, 0.002397
 total: 0.817544 s
Experiment data directory written: ...

Reports

pat_report jacobi_mpi+openmp.CRAY.DAINT.+pat+4104-1727t/
app2 *.ap2

[jg piccinali]

SCOREP/1.3

export SCOREP_ENABLE_PROFILING=false
export SCOREP_ENABLE_TRACING=true
export SCOREP_CUDA_ENABLE=yes
export SCOREP_TOTAL_MEMORY=100M
export SCOREP_METRIC_PLUGINS=pm_plugin
export SCOREP_METRIC_PM_PLUGIN=all