summer school - openacc/cce (nvvp/nvprof)

Issue #39 new
jg piccinali repo owner created an issue

DAINT

Get the src

CCE

Compile

  • module load craype-accel-nvidia35
Currently Loaded Modulefiles:
  1) modules/3.2.10.3
  2) nodestat/2.2-1.0502.53712.3.109.ari
  3) sdb/1.0-1.0502.55976.5.27.ari
  4) alps/5.2.1-2.0502.9041.11.6.ari
  5) lustre-cray_ari_s/2.5_3.0.101_0.31.1_1.0502.8394.10.1-1.0502.17198.8.51
  6) udreg/2.3.2-1.0502.9275.1.12.ari
  7) ugni/5.0-1.0502.9685.4.24.ari
  8) gni-headers/3.0-1.0502.9684.5.2.ari
  9) dmapp/7.0.1-1.0502.9501.5.219.ari
 10) xpmem/0.1-2.0502.55507.3.2.ari
 11) hss-llm/7.2.0
 12) Base-opts/1.0.2-1.0502.53325.1.2.ari
 13) craype-network-aries
 14) craype/2.4.0
 15) cce/8.3.12
 16) totalview-support/1.1.4
 17) totalview/8.11.0
 18) cray-libsci/13.0.4
 19) pmi/5.0.7-1.0000.10678.155.25.ari
 20) rca/1.0.0-2.0502.53711.3.127.ari
 21) atp/1.8.2
 22) PrgEnv-cray/5.2.40
 23) craype-sandybridge
 24) slurm
 25) cray-mpich/7.2.2
 26) ddt/5.0
 27) cray-libsci_acc/3.1.1
 28) cudatoolkit/6.5.14-1.0502.9613.6.1
 29) craype-accel-nvidia35
  • make clean
  • make main
ftn -rmd -hacc -O3 -e Z   -c stats.f90 -o stats.o
ftn -rmd -hacc -O3 -e Z   -c data.f90 -o data.o
ftn -rmd -hacc -O3 -e Z   -c operators.f90 -o operators.o
ftn -rmd -hacc -O3 -e Z   -c linalg.f90 -o linalg.o
ftn -rmd -hacc -O3 -e Z   -c io.f90 -o io.o
ftn -rmd -hacc -O3 -e Z  \
stats.o   data.o   operators.o     linalg.o     io.o main.f90  \
-o main

Profile

  • export PMI_NO_FORK=1
  • sbatch.sh santis 5 main 1 1 1 "1024 1024 100 0.0025" "" "-b nvprof -o nvprof.%h.%p"
    • equivalent to: aprun -n1 -b nvprof -o nvprof.%h.%p main 1024 1024 100 0.0025
==27237== Generated result file: nvprof.nid00012.27237

Analyze

  • nvvp nvprof.* nvvp_cce.png

Comments (2)

  1. jg piccinali reporter

    ANALYZE (online)

    • recompile WITHOUT mpi
    • salloc -p ccm
    • module load ccm
    • export PBS_JOBID=$SLURM_JOBID
    • export PMI_NO_FORK=1
    • ccmlogin -V
      • nvvp ./main # 1024 1024 100 0.0025
      • ❗ Overhead
      • ==> nvvp ./main # 256 256 20 0.0001 Screen Shot 2015-06-15 at 23.14.40.png Screen Shot 2015-06-15 at 23.15.01.png Screen Shot 2015-06-15 at 23.50.45.png
  2. Log in to comment