summer school - openacc/pgi (nvvp/nvprof)

[jg piccinali]

DAINT

Get the src

• ssh -Y daint01
• git clone https://github.com/bcumming/summer-school.git
• cd summer-school.git/openacc/fortran/

PGI

Compile

• module swap PrgEnv-cray PrgEnv-pgi
• module load craype-accel-nvidia35

Currently Loaded Modulefiles:
  1) modules/3.2.10.3
  2) nodestat/2.2-1.0502.53712.3.109.ari
  3) sdb/1.0-1.0502.55976.5.27.ari
  4) alps/5.2.1-2.0502.9041.11.6.ari
  5) lustre-cray_ari_s/2.5_3.0.101_0.31.1_1.0502.8394.10.1-1.0502.17198.8.51
  6) udreg/2.3.2-1.0502.9275.1.12.ari
  7) ugni/5.0-1.0502.9685.4.24.ari
  8) gni-headers/3.0-1.0502.9684.5.2.ari
  9) dmapp/7.0.1-1.0502.9501.5.219.ari
 10) xpmem/0.1-2.0502.55507.3.2.ari
 11) hss-llm/7.2.0
 12) Base-opts/1.0.2-1.0502.53325.1.2.ari
 13) craype-network-aries
 14) craype-sandybridge
 15) craype/2.4.0
 16) slurm
 17) cray-mpich/7.2.2
 18) ddt/5.0
 19) pgi/15.3.0
 20) totalview-support/1.1.4
 21) totalview/8.11.0
 22) pmi/5.0.7-1.0000.10678.155.25.ari
 23) atp/1.8.2
 24) PrgEnv-pgi/5.2.40
 25) cudatoolkit/6.5.14-1.0502.9613.6.1
 26) craype-accel-nvidia35
 27) rca/1.0.0-2.0502.53711.3.127.ari

• make clean
• make main

ftn -acc=verystrict -ta=nvidia,nofma,cc35,cuda6.5  -Mpreprocess  \
-O3   -c stats.f90 -o stats.o
ftn -acc=verystrict -ta=nvidia,nofma,cc35,cuda6.5  -Mpreprocess  \
-O3   -c data.f90 -o data.o
ftn -acc=verystrict -ta=nvidia,nofma,cc35,cuda6.5  -Mpreprocess  \
-O3   -c operators.f90 -o operators.o
ftn -acc=verystrict -ta=nvidia,nofma,cc35,cuda6.5  -Mpreprocess  \
-O3   -c linalg.f90 -o linalg.o
ftn -acc=verystrict -ta=nvidia,nofma,cc35,cuda6.5  -Mpreprocess  \
-O3   -c io.f90 -o io.o
ftn -acc=verystrict -ta=nvidia,nofma,cc35,cuda6.5  -Mpreprocess  \
-O3  stats.o   data.o   operators.o     linalg.o     io.o main.f90  -o main

Profile

• export PMI_NO_FORK=1
• sbatch.sh santis 5 main+pat 1 1 1 "1024 1024 100 0.0025" "" "-b nvprof -o nvprof.%h.%p"

==27093== Generated result file: nvprof.nid00012.27093

overhead

Analyze

• nvvp nvprof.nid00012.27093