SIRIUS (scorep)

[jg piccinali]

Scorep/1.4.2

Daint

Setup

• git clone https://github.com/toxa81/sirius.git sirius.git
• gic develop
• module swap PrgEnv-cray PrgEnv-gnu
• module swap gcc/4.8.2
• module load fftw
• module load cray-hdf5
• module load craype-accel-nvidia35

Compile

• /project/csstaff/antonk/sirius_vampir/sirius/platform.json
  • depends on libcx, spglib, gsl, json, elpa
• python configure.py
• make

Run

[jg piccinali]

ptl/625

• /apps/daint/5.2.UP02/sandbox/jgp/sirius.git/apps/dft_loop/TiCoO2

Table 1:  Profile by Function
  Samp% |   Samp | Imb. |  Imb. |Group
        |        | Samp | Samp% | Function
        |        |      |       |  PE=HIDE
        |        |      |       |   Thread=HIDE

 100.0% | 3698.0 |   -- |    -- |Total
|-----------------------------------------------------------------------------
|  74.4% | 2749.5 |   -- |    -- |ETC
||----------------------------------------------------------------------------
||  28.7% | 1061.8 | 32.2 |  3.9% |sirius::DFT_ground_state::scf_loop
||   4.9% |  180.0 |  8.0 |  5.7% |__elpa1_MOD_solve_evp_complex
||   4.4% |  163.2 | 11.8 |  9.0% |sirius::Density::initial_density
||   4.2% |  156.0 | 21.0 | 15.8% |_ZN6sirius4Atom25generate_radial
_integralsE17processing_unit_tRK12Communicator._omp_fn.5
||   4.1% |  151.2 | 12.8 | 10.4% |gomp_team_barrier_wait_end
||   3.8% |  142.0 | 11.0 |  9.6% |cudbgGetAPIVersion
||   3.7% |  137.5 | 76.5 | 47.7% |__elpa1_MOD_cholesky_complex
||   3.2% |  119.8 |  7.2 |  7.6% |gotoblas_zgemv_c_sandybridge
||   2.2% |   81.0 | 12.0 | 17.2% |__elpa1_MOD_mult_ah_b_complex
||   1.7% |   64.0 |  1.0 |  2.1% |gotoblas_zgemv_n_sandybridge
||   1.5% |   57.2 |  6.8 | 14.1% |__amd_bas64_log
||   1.4% |   51.5 |  5.5 | 12.9% |cuMemGetAttribute_v2
||   1.0% |   36.5 |  6.5 | 20.2% |__GI_memset
||============================================================================
|  12.0% |  444.8 |   -- |    -- |BLAS
||----------------------------------------------------------------------------
||   4.4% |  161.5 | 63.5 | 37.6% |gotoblas_zgemm_kernel_n_sandybridge
||   2.6% |   94.8 |  5.2 |  7.0% |gotoblas_zgemm_kernel_l_sandybridge
||   2.2% |   82.0 | 12.0 | 17.0% |gotoblas_zgemm_kernel_r_sandybridge
||   1.4% |   53.5 | 27.5 | 45.3% |gotoblas_zgemm_otcopy_sandybridge
||============================================================================
|   6.5% |  239.2 |   -- |    -- |USER
||----------------------------------------------------------------------------
||   1.7% |   62.5 |  3.5 |  7.1% |gsl_sf_bessel_J_CF1
||============================================================================
|   3.4% |  125.8 |   -- |    -- |PTHREAD
||----------------------------------------------------------------------------
||   3.4% |  124.0 |  3.0 |  3.1% |pthread_join
||============================================================================

[jg piccinali]

scorep doesn't currently have support for C++11 std::thread as there is no
standard way to instrument std::thread.

If std::thread builds upon pthreads, then instrumenting with
--thread=pthread *might* produce some results, but only if you link a
static pthreads library (never tried). I won't encourage you to go this way.