EPCC MicroBenchmark - MPI (MAQAO:LPROFMPI)

[jg piccinali]

Setup

• http://www.vi-hps.org/training/tws/tw18.html
• http://www2.epcc.ed.ac.uk/~markb/mpiopenmpbench/intro.html
• module swap PrgEnv-cray PrgEnv-gnu

Currently Loaded Modulefiles:
  1) modules/3.2.10.3
  2) nodestat/2.2-1.0502.53712.3.109.ari
  3) sdb/1.0-1.0502.55976.5.27.ari
  4) alps/5.2.1-2.0502.9041.11.6.ari
  5) lustre-cray_ari_s/2.5_3.0.101_0.31.1_1.0502.8394.10.1-1.0502.17198.8.51
  6) udreg/2.3.2-1.0502.9275.1.12.ari
  7) ugni/5.0-1.0502.9685.4.24.ari
  8) gni-headers/3.0-1.0502.9684.5.2.ari
  9) dmapp/7.0.1-1.0502.9501.5.219.ari
 10) xpmem/0.1-2.0502.55507.3.2.ari
 11) hss-llm/7.2.0
 12) Base-opts/1.0.2-1.0502.53325.1.2.ari
 13) craype-network-aries
 14) craype-haswell
 15) craype/2.4.0
 16) slurm
 17) cray-mpich/7.2.2
 18) ddt/5.0
 19) gcc/4.9.2
 20) cray-libsci/13.0.4
 21) pmi/5.0.7-1.0000.10678.155.25.ari
 22) atp/1.8.2
 23) PrgEnv-gnu/5.2.40

Compile

• make clean
• make FC="ftn -fopenmp" LFLAGS=-dynamic

Prepare

cat in0

1 # min data size
4 # max data size
1.00 # target time
multiplePingpong
-1 0

Run/Profile

• export G=/apps/ela/sandbox/jgp/graphviz/git/gnu_434
• export PATH=$G/bin:$PATH
• export M=/apps/daint/5.2.UP02/maqao/2.1.1
• export PATH=$Mbin:$PATH
• run without "/usr/bin/time -p"
• sbatch.sh brisi 5 ./mixedModeBenchmark 24 24 1 in.jg \ "LD_PRELOAD=$M/lib/MPI/GNU/libopmpi_gnu_cscs.so"

 --------------------------------------------
 # Multiple Pingpong
 --------------------------------------------
 WARNING: Implementation does not support benchmark
 Inter node benchmark between process         191 and process           0
  Data Size     Msg Size (bytes)     No. Reps     Time (sec)     Time/Rep (s)     Test
 -----------   ------------------   ----------   ------------   --------------   ------
d         1                    4       256000       1.184780       0.000004628     Pass
d         2                    8       256000       1.190468       0.000004650     Pass
d         4                   16       256000       1.207656       0.000004717     Pass
================== MPI PROFILE (Rank 0) ====================
MPI_Barrier      hits: 22    time: 0.000327      size : 0
MPI_Bcast    hits: 34    time: 0.000269      size : 144
MPI_Comm_rank    hits: 3     time: 0.000000      size : 0
MPI_Comm_size    hits: 2     time: 0.000000      size : 0
MPI_Comm_split       hits: 2     time: 1.836133      size : 0
MPI_Finalize     hits: 1     time: 0.000013      size : 0
MPI_Init_thread      hits: 1     time: 0.090983      size : 0
MPI_Recv     hits: 1023006       time: 2.279792      size : 8188056
MPI_Send     hits: 1023007       time: 2.062786      size : 8188183
MPI_Type_size    hits: 1     time: 0.000002      size : 0
MPI_Wtime    hits: 22    time: 0.000008      size : 0

• Ignore: Error: remove_overlap: Graphviz not built with triangulation library
• ===> simulation creates MPI_profile.js

Analyze

• scp -rp ela.cscs.ch:/apps/daint/5.2.UP02/maqao/VIHPS/MAQAOH/LProfMPI/MPI_GUI .
• scp ela.cscs.ch:MPI_profile.js .
• open MPI_GUI/MPI.html
• load MPI_profile.js