Compilation on FX10

Issue #278 resolved
Jonas Thies created an issue

Please collect all info on Problems, Progress and How-Tos here.
Also include links to Fujitsu issues etc.

Comments (14)

  1. Takahiro Yano

    Triplet of K(/FX10)

    sparc64-unknown-linux(-gnu)

    You can confirm this by executing gcc -v on compute nodes.

    How to build HWLOC

    Building HWLOC using Fujitsu's compiler was failed even native build on compute nodes.
    So I used GCC and it was successful.

    ./configure --prefix=<where to locate HWLOC> --build=sparc64-unknown-linux-gnu
    make
    make install
    

    OpenMP flags for Fujitsu's compiler

    NOTE: CMake executables for SPARC64 is NOT present in the system.
    Thus, you need to build them beforehand.

    After building HWLOC, I tried configuring GHOST.

    CC=fcc CXX=FCC cmake <path to ghost source> \
        -DHWLOC_INCLUDE_DIR=<path to installed HWLOC>/include \
        -DGHOST_USE_CUDA=OFF
    

    Then I got an error about OpenMP flags.

    CMake Error at /***/share/cmake-3.5/Modules/FindPackageHandleStandardArgs.cmake:148 (message):
      Could NOT find OpenMP (missing: OpenMP_C_FLAGS OpenMP_CXX_FLAGS)
    

    I added them as the following.

    CC=fcc CXX=FCC cmake <path to ghost source> \
        -DHWLOC_INCLUDE_DIR=<path to installed HWLOC>/include \
        -DGHOST_USE_CUDA=OFF \
        -DOpenMP_C_FLAGS=-Kopenmp \
        -DOpenMP_CXX_FLAGS=-Kopenmp
    

    Lack of CBLAS in Fujitsu's BLAS/LAPACK library

    After that, I got another error about the location of cblas.h or like that.

    -- Try to find MKL in default paths and via MKLROOT
    -- Try to find GSL in default paths and via find_package
    -- FindGSL: gsl-config not found.
    -- Try to find default Cblas
    CMake Error at CMakeLists.txt:561 (message):
      CBLAS_INCLUDE_DIR not set!
    

    Fujitsu's library does NOT contain the CBLAS interface.
    So Prof. Kengo NAKAJIMA asked Fujitsu to install it (as far as I know).

    Following is WIP.

    To solve this issue instantly, I built the CBLAS library from source
    with some modifications on generating scripts.
    Generating the library itself was succeeded, however, testing level-2 and level-3 interfaces was failed.

  2. Takahiro Yano

    Quick fix for CBLAS (just for finishing configure)

    You locate cblas.h and cblas_f77.h somewhere.

    Fix of CMakeList.txt about using SpMP is also needed.
    (Tested with 275c395)

    UPD: fixed in 911d256

    MPI compilers

    CMake could not detect Fujitsu's MPI compilers.
    Add options to cmake as the following.

    CC=fcc CXX=FCC cmake <path to ghost source> \
        -DHWLOC_INCLUDE_DIR=<path to installed HWLOC>/include \
        -DGHOST_USE_CUDA=OFF \
        -DOpenMP_C_FLAGS=-Kopenmp \
        -DOpenMP_CXX_FLAGS=-Kopenmp \
        -DCBLAS_INCLUDE_DIR=<path to cblas.h and cblas_f77.h> \
        -DGHOST_USE_SPMP=OFF \
        -DMPI_C_COMPILER=mpifcc \
        -DMPI_CXX_COMPILER=mpiFCC
    

    Check unsupported features in Fujitsu's C++ compiler

    Now, time for building GHOST.
    I ran make and got two types of errors on building src/sell_kacz.cpp.

    Initialization of structures

    Two structures are initialized by the C's field initializer form.
    See e.g. SO.
    Fujitsu's compiler does not support this in C++ although GCC and Intel support it.

    Lack of std::to_string()

    std::to_string() is introduced in C++11.
    Fujitsu may not have implemented it.

    And more

    C++ features introduced in C++11 (or later) is partially supported in Fujitsu's C++ compiler.
    We need to collect what features the compiler must support in C++11 to build GHOST.

  3. Takahiro Yano

    Compiler flags

    Fujitsu C++ compiler can't accept -std=c++0x flag.
    Following is tested with -std=c++11 flag.

    More unsupported features

    Can't recognize __inline__ in C compiler

    Quick fix: fix ${HWLOC_INCLUDE_DIR}/hwloc/autogen/config.h

    30a31,32
    > #elif defined(__FUJITSU)
    > #  define __hwloc_inline
    

    C++11 extension of some functions in <complex> is NOT implemented

    • std::norm()
    • std::conj()

    NOT exist intrinsics ("immintrin.h")

    • bench.c
    • dot_avx__*.c
    • sell_spmv_avx__*_*.c
    • sell_spmv_avx2__*_*.c
    • sell_spmv_mic__*_*.c
    • sell_spmv_sse__*_*.c
    • sell_spmv_varblock_avx__*.c
    • tsmm_avx__*_*_2.c
    • tsmm_sse__*_*.c
    • tsmttsm_avx__*_*.c
    • tsmttsm_avx2__*_*.c
    • tsmttsm_sse__*_*.c
    • sell_kacz_bmc_shift__*_*.cpp
  4. Moritz Kreutzer

    GHOST is now compiling, but there are two problems occuring at link time:

    1. Missing CBLAS. In principle, one could use the standard BLAS interface and "emulate" CBLAS but this would require some trickery with the storage of matrices (which can be specified only in CBLAS). Is there any chance to get CBLAS on this machine?

    2. Missing qsort_r() function. This is only available from GLIBC 2.8 (2.7 is present on the machine). Of course this could easily be avoided by not sorting the SELL-C-S rows in parallel in case of a too old GLIBC. So this is not a big issue...

  5. Moritz Kreutzer

    Unfortunately not...

    a10tio297 ~/proj/ghost/build $ ./test/minimal
    jwe0019i-u The program was terminated abnormally with signal number SIGSEGV.
               signal identifier = SEGV_MAPERR, address not mapped to object
     error occurs at _ZNSiC2EPSt15basic_streambufIcSt11char_traitsIcEE loc ffffffff00842e74 offset 000000000000005c
     _ZNSiC2EPSt15basic_streambufIcSt11char_traitsIcEE     at loc ffffffff00842e18 called from loc ffffffff00855db0 in _ZNSiC1EPSt15basic_streambufIcSt11char_traitsIcEE
     _ZNSiC1EPSt15basic_streambufIcSt11char_traitsIcEE     at loc ffffffff00855d7c called from loc ffffffff04712590 in _ZNSt8ios_base13_S_initializeEv
     _ZNSt8ios_base13_S_initializeEv     at loc ffffffff04712540 called from loc ffffffff04712280 in _ZNSt8ios_base4InitC1Ev
     _ZNSt8ios_base4InitC1Ev             at loc ffffffff04712240 called from loc ffffffff0473c398 in __sti___15_locale_impl_cpp_e73c48f6
     __sti___15_locale_impl_cpp_e73c48f6     at loc ffffffff0473c300 called from loc ffffffff047b54ac in
                  at loc ffffffff0473c300 called from loc ffffffff046d850c in
                  at loc ffffffff0473c300 called from loc fffff80101013464 in
                  at loc ffffffff0473c300 called from loc fffff80101013620 in
                  at loc ffffffff0473c300 called from loc fffff80101003da8 in
                  at loc ffffffff0473c300 called from o.s.
    error summary (Fortran)
    error number  error level  error count
      jwe0019i         u           1
    total error count = 1
    

    And a debugger does not really help:

    a10tio297 ~/proj/ghost/build $ fdb ./test/minimal
    FDB [Fujitsu Debugger for C/C++ and Fortran] Version 1.2(glibc2.7)MT/OMP
    Please wait to analyze the DEBUG information.
    fdb* r
    The program: ./test/minimal starting.
    Please wait to analyze the DEBUG information.
    Please wait to analyze the DEBUG information.
    Please wait to analyze the DEBUG information.
    Please wait to analyze the DEBUG information.
    Please wait to analyze the DEBUG information.
    Please wait to analyze the DEBUG information.
    Please wait to analyze the DEBUG information.
    Please wait to analyze the DEBUG information.
    [ Process 15036 ]
    [thread 1 on process 15036] Signaled SIGSEGV(11): std::basic_istream<char, std::char_traits<char>>::basic_istream(std::basic_streambuf<char, std::char_traits<char>> *) [subobject].
    0xffffffff00842e74 (std::basic_istream<char, std::char_traits<char>>::basic_istream + 0x5c)     ldx               [%g4+%g0],%g4
    fdb* t
    #0 0xffffffff00842e74 (_ZNSiC2EPSt15basic_streambufIcSt11char_traitsIcEE + 0x5c) (0xffffffff14818e30,0x0,0x0,0x9010,0x0,0x0)
    #1 0xffffffff00855db0 (_ZNSiC1EPSt15basic_streambufIcSt11char_traitsIcEE + 0x34) (0xffffffff14818e30,0x0,0x0,0x3520,0x0,0x0)
    #2 0xffffffff04712590 (_ZNSt8ios_base13_S_initializeEv + 0x50) (0xffffffff148139f0,0x1,0x0,0x0,0x0,0x0)
    #3 0xffffffff04712280 (_ZNSt8ios_base4InitC1Ev + 0x40) (0xffffffff14814624,0xffffffff148145d0,0xffffffff14811480,0x0,0x0,0x0)
    #4 0xffffffff0473c398 (__sti___15_locale_impl_cpp_e73c48f6 + 0x98) (0xffffffff04700f80,0xffffffff148155e0,0xffffffff148155c0,0xffffffff14818b08,0x0,0xffffffff14818b10)
    #5 0xffffffff047b54ac (__do_global_ctors_aux + 0x2c) (0xffffffff03b7f9f8,0x40,0xffffffff03a776e0,0x7feffffe2e0,0xffffffff03a7c6b0,0x0)
    #6 0xffffffff046d850c (_init + 0x34) (0x1,0x7fefffff6c8,0x7fefffff6d8,0x0,0x8,0x8)
    #7 0xfffff80101013464 (call_init + 0xc8) (0xfffff80100064378,0x1,0x7fefffff6c8,0x7fefffff6d8,0x6fffffff,0xffffffff90000000)
    #8 0xfffff80101013620 (_dl_init_internal + 0x144) (0xfffff80100050000,0x1,0x7fefffff6c8,0x7fefffff6d8,0x0,0x0)
    #9 0xfffff80101003da8 (_dl_start_user + 0xbc) (0xfffff80100050000,0x1,0x7fefffff6c8,0x7fefffff6d8,0x0,0x0)
    
  6. Log in to comment