Error with compilation of HWLOC

[anonymous]

I updated the ET nightly version. The configuration works fine on the local Ubuntu gcc PC, but it fails to compile at CFCA CRAY XC30 cluster with Intel environment. I do not see error message.

Using configuration: test
ModuleCmd_Switch.c(172):ERROR:152: Module 'PrgEnv-cray' is currently not loaded
ModuleCmd_Switch.c(172):ERROR:152: Module 'PrgEnv-cray' is currently not loaded
ModuleCmd_Switch.c(172):ERROR:152: Module 'PrgEnv-cray' is currently not loaded
ModuleCmd_Switch.c(172):ERROR:152: Module 'PrgEnv-cray' is currently not loaded
ModuleCmd_Switch.c(172):ERROR:152: Module 'PrgEnv-cray' is currently not loaded
Reconfiguring test
Writing configuration to: /work/barkovmm/ET/Cactus/configs/test/OptionList
ModuleCmd_Switch.c(172):ERROR:152: Module 'PrgEnv-cray' is currently not loaded
Cactus - version: 4.2.3
Reconfiguring test.
Using configuration options from configure line
  Setting fds to '4,5 -j --'
End of options from configure line
Adding configuration options from '/work/barkovmm/ET/Cactus/configs/test/OptionList'...
  Setting VERSION to 'cfca-2015-04-03'
  Setting CPP to 'cpp'
  Setting FPP to 'cpp'
  Setting CC to 'cc'
  Setting CXX to 'CC'
  Setting F77 to 'ftn'
  Setting F90 to 'ftn'
  Setting CPPFLAGS to '-g'
  Setting FPPFLAGS to '-g -traditional'
  Setting CFLAGS to '-g -debug all -traceback -xHost -align -std=gnu99 -ansi-alias'
  Setting CXXFLAGS to '-g -debug all -traceback -xHost -align -std=gnu++0x -ansi-alias -D__builtin_signbit=__signbit'
  Setting F77FLAGS to '-g -debug all -traceback -xHost -align -pad -safe_cray_ptr'
  Setting F90FLAGS to '-g -debug all -traceback -xHost -align -pad -safe_cray_ptr'
  Setting C_LINE_DIRECTIVES to 'yes'
  Setting F_LINE_DIRECTIVES to 'yes'
  Setting VECTORISE to 'yes'
  Setting VECTORISE_ALIGNED_ARRAYS to 'no'
  Setting VECTORISE_INLINE to 'no'
  Setting DEBUG to 'no'
  Setting CPP_DEBUG_FLAGS to '-DCARPET_DEBUG'
  Setting FPP_DEBUG_FLAGS to '-DCARPET_DEBUG'
  Setting C_DEBUG_FLAGS to ''
  Setting CXX_DEBUG_FLAGS to ''
  Setting F77_DEBUG_FLAGS to '-check bounds -check format'
  Setting F90_DEBUG_FLAGS to '-check bounds -check format'
  Setting OPTIMISE to 'yes'
  Setting CPP_OPTIMISE_FLAGS to ''
  Setting FPP_OPTIMISE_FLAGS to ''
  Setting C_OPTIMISE_FLAGS to '-O2 -no-prec-div'
  Setting CXX_OPTIMISE_FLAGS to '-Ofast'
  Setting F77_OPTIMISE_FLAGS to '-Ofast'
  Setting F90_OPTIMISE_FLAGS to '-Ofast'
  Setting C_NO_OPTIMISE_FLAGS to '-O0'
  Setting CXX_NO_OPTIMISE_FLAGS to '-O0'
  Setting F77_NO_OPTIMISE_FLAGS to '-O0'
  Setting F90_NO_OPTIMISE_FLAGS to '-O0'
  Setting PROFILE to 'no'
  Setting CPP_PROFILE_FLAGS to ''
  Setting FPP_PROFILE_FLAGS to ''
  Setting C_PROFILE_FLAGS to '-pg'
  Setting CXX_PROFILE_FLAGS to '-pg'
  Setting F77_PROFILE_FLAGS to '-pg'
  Setting F90_PROFILE_FLAGS to '-pg'
  Setting OPENMP to 'yes'
  Setting CPP_OPENMP_FLAGS to '-openmp'
  Setting FPP_OPENMP_FLAGS to '-fopenmp'
  Setting C_OPENMP_FLAGS to '-openmp'
  Setting CXX_OPENMP_FLAGS to '-openmp'
  Setting F77_OPENMP_FLAGS to '-openmp'
  Setting F90_OPENMP_FLAGS to '-openmp'
  Setting WARN to 'yes'
  Setting CPP_WARN_FLAGS to ''
  Setting FPP_WARN_FLAGS to '-Wall'
  Setting C_WARN_FLAGS to ''
  Setting CXX_WARN_FLAGS to ''
  Setting F90_WARN_FLAGS to ''
  Setting F77_WARN_FLAGS to ''
  Setting BLAS_DIR to 'NO_BUILD'
  Setting BOOST_DIR to 'BUILD'
  Setting FFTW_DIR to '/opt/fftw/3.3.4.0/haswell/'
  Setting GSL_DIR to '/work/ap/GSL/gsl-1.15/intel/'
  Setting GSL_INC_DIRS to '/work/ap/GSL/gsl-1.15/intel/include'
  Setting GSL_LIB_DIRS to '/work/ap/GSL/gsl-1.15/intel/lib'
  Setting HDF5_DIR to '/opt/cray/hdf5/1.8.13'
  Setting HWLOC_DIR to 'BUILD'
  Setting LAPACK_DIR to 'NO_BUILD'
  Setting MPI_DIR to 'NONE'
  Setting OPENBLAS_DIR to 'NO_BUILD'
  Setting OPENBLAS_LIBS to ''
  Setting PAPI_DIR to '/opt/cray/papi/5.3.2'
  Setting PETSC_DIR to '/opt/cray/petsc/3.5.1.0/real/INTEL/140/haswell/'
  Setting PTHREADS to 'yes'
  Setting ZLIB_DIR to 'BUILD'
End of options from '/work/barkovmm/ET/Cactus/configs/test/OptionList'.



checking for stdint.h... (cached) yes
checking sys/mman.h usability... yes
checking sys/mman.h presence... yes
checking for sys/mman.h... yes
checking for KAFFINITY... no
checking for PROCESSOR_CACHE_TYPE... no
checking for CACHE_DESCRIPTOR... no
checking for LOGICAL_PROCESSOR_RELATIONSHIP... no
checking for RelationProcessorPackage... no
checking for SYSTEM_LOGICAL_PROCESSOR_INFORMATION... no
checking for GROUP_AFFINITY... no
checking for PROCESSOR_RELATIONSHIP... no
checking for NUMA_NODE_RELATIONSHIP... no
checking for CACHE_RELATIONSHIP... no
checking for PROCESSOR_GROUP_INFO... no
checking for GROUP_RELATIONSHIP... no
checking for SYSTEM_LOGICAL_PROCESSOR_INFORMATION_EX... no
checking for PSAPI_WORKING_SET_EX_BLOCK... no
checking for PSAPI_WORKING_SET_EX_INFORMATION... no
checking for main in -lgdi32... no
checking windows.h usability... no
checking windows.h presence... no
checking for windows.h... no
checking sys/lgrp_user.h usability... no
checking sys/lgrp_user.h presence... no
checking for sys/lgrp_user.h... no
checking kstat.h usability... no
checking kstat.h presence... no
checking for kstat.h... no
checking whether fabsf is declared... yes
checking for fabsf in -lm... icc: command line warning #10121: overriding '-xCORE-AVX2' with '-xHost'
yes
checking picl.h usability... no
checking picl.h presence... no
checking for picl.h... no
checking whether _SC_NPROCESSORS_ONLN is declared... yes
checking whether _SC_NPROCESSORS_CONF is declared... yes
checking whether _SC_NPROC_ONLN is declared... no
checking whether _SC_NPROC_CONF is declared... no
checking whether _SC_PAGESIZE is declared... yes
checking whether _SC_PAGE_SIZE is declared... yes
checking whether _SC_LARGE_PAGESIZE is declared... no
checking mach/mach_host.h usability... no
checking mach/mach_host.h presence... no
checking for mach/mach_host.h... no
checking mach/mach_init.h usability... no
checking mach/mach_init.h presence... no
checking for mach/mach_init.h... no
checking sys/param.h usability... yes
checking sys/param.h presence... yes
checking for sys/param.h... yes
checking for sys/sysctl.h... yes
checking whether CTL_HW is declared... no
checking whether HW_NCPU is declared... no
checking whether strtoull is declared... icc: command line warning #10121: overriding '-xCORE-AVX2' with '-xHost'
yes
checking for sysctl... icc: command line warning #10121: overriding '-xCORE-AVX2' with '-xHost'
yes
checking for sysctlbyname... no
checking for pthread_t... yes
checking whether function sched_setaffinity is declared... yes
checking for old prototype of sched_setaffinity... no
checking for working CPU_SET... yes
checking for working CPU_SET_S... yes
checking for working _syscall3... no
checking for KERRIGHED... no
checking for lib... no
checking for bash... /bin/sh
checking for ffs... yes
checking whether function ffs is declared... yes
checking for ffsl... yes
checking whether function ffsl is declared... yes
checking for fls... no
checking for flsl... no
checking for clz... no
checking for clzl... no
checking for openat... yes
checking malloc.h usability... yes
checking malloc.h presence... yes
checking for malloc.h... yes
checking for getpagesize... yes
checking for memalign... yes
checking for posix_memalign... yes
checking sys/utsname.h usability... yes
checking sys/utsname.h presence... yes
checking for sys/utsname.h... yes
checking for uname... yes
checking pthread_np.h usability... no
checking pthread_np.h presence... no
checking for pthread_np.h... no
checking whether pthread_setaffinity_np is declared... yes
checking whether pthread_getaffinity_np is declared... yes
checking for sched_setaffinity... yes
checking for sys/cpuset.h... no
checking for cpuset_setaffinity... no
checking for library containing pthread_getthrds_np... no
checking for cpuset_setid... no
checking numaif.h usability... yes
checking numaif.h presence... yes
checking for numaif.h... yes
checking for numa_available in -lnuma... no
checking nvml.h usability... no
checking nvml.h presence... no
checking for nvml.h... no
checking X11/Xlib.h usability... yes
checking X11/Xlib.h presence... yes
checking for X11/Xlib.h... yes
checking for XOpenDisplay in -lX11... no
checking NVCtrl/NVCtrl.h usability... no
checking NVCtrl/NVCtrl.h presence... no
checking for NVCtrl/NVCtrl.h... no
checking for cpuid... yes
checking for pthread_mutex_lock... yes
checking if plugin support is enabled... no
checking components to build statically...  noos xml synthetic custom xml_nolibxml linux x86
checking components to build as plugins... 
checking for X... no

###
### Configuring hwloc documentation
###
checking if this is a developer build... no (doxygen generation is optional)
checking for doxygen... /usr/bin/doxygen
checking for pdflatex... no
checking for makeindex... no
checking for fig2dev... no
checking for gs... /usr/bin/gs
checking for epstopdf... no
checking if can build doxygen docs... no
checking for a sed that does not truncate output... /usr/bin/sed
checking if the sed -i option requires an argument... no
checking for w3m... /usr/bin/w3m
checking for lynx... no
checking if can build top-level README... yes
checking if will build doxygen docs... no
checking if will install doxygen docs... yes
checking whether to enable "picky" compiler mode... no (default)

###
### Configuring hwloc command line utilities
###
checking for wchar_t... yes
checking for putwc... yes
checking locale.h usability... yes
checking locale.h presence... yes
checking for locale.h... yes
checking for setlocale... (cached) yes
checking for uselocale... yes
checking xlocale.h usability... yes
checking xlocale.h presence... yes
checking for xlocale.h... yes
checking for setlocale... (cached) yes
checking for uselocale... (cached) yes
checking langinfo.h usability... yes
checking langinfo.h presence... yes
checking for langinfo.h... yes
checking for nl_langinfo... yes
checking termcap support using ncurses and ... no
checking termcap support using ncurses and -ltermcap... no
checking termcap support using ncurses and -lncursesw... yes
checking whether diff accepts -u... -u
checking whether diff accepts -w... -w

###
### Configuring hwloc tests
###
checking for pthread_self in -lpthread... yes
checking whether numa_bitmask_alloc is declared... yes
checking infiniband/verbs.h usability... yes
checking infiniband/verbs.h presence... yes
checking for infiniband/verbs.h... yes
checking for ibv_open_device in -libverbs... no
checking myriexpress.h usability... no
checking myriexpress.h presence... no
checking for myriexpress.h... no
checking for xmllint... xmllint
checking for bunzip2... bunzip2
checking if CXX works... yes
checking whether diff accepts -u... -u

###
### Performing final hwloc configuration
###
checking how to print strings... printf
checking for a sed that does not truncate output... (cached) /usr/bin/sed
checking for fgrep... /usr/bin/grep -F
checking for ld used by cc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 3458764513820540925
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands "+="... yes
checking how to convert x86_64-unknown-linux-gnu file names to x86_64-unknown-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-unknown-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for archiver @FILE support... @
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from cc object... ok
checking for sysroot... no
checking for mt... mt
checking if mt is a manifest tool... no
checking for dlfcn.h... yes
checking for objdir... .libs
checking if cc supports -fno-rtti -fno-exceptions... yes
checking for cc option to produce PIC... -fPIC -DPIC
checking if cc PIC flag -fPIC -DPIC works... yes
checking if cc static flag -static works... yes
checking if cc supports -c -o file.o... yes
checking if cc supports -c -o file.o... (cached) yes
checking whether the cc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking whether -lc should be explicitly linked in... no
checking dynamic linker characteristics... icc: warning #10315: specifying -lm before files may supersede the Intel(R) math library and affect performance
icc: warning #10315: specifying -lm before files may supersede the Intel(R) math library and affect performance
icc: warning #10315: specifying -lm before files may supersede the Intel(R) math library and affect performance
icc: warning #10315: specifying -lm before files may supersede the Intel(R) math library and affect performance
icc: warning #10315: specifying -lm before files may supersede the Intel(R) math library and affect performance
icc: warning #10315: specifying -lm before files may supersede the Intel(R) math library and affect performance
icc: warning #10315: specifying -lm before files may supersede the Intel(R) math library and affect performance
icc: warning #10315: specifying -lm before files may supersede the Intel(R) math library and affect performance
icc: warning #10315: specifying -lm before files may supersede the Intel(R) math library and affect performance
icc: warning #10315: specifying -lm before files may supersede the Intel(R) math library and affect performance
icc: warning #10315: specifying -lm before files may supersede the Intel(R) math library and affect performance
icc: warning #10315: specifying -lm before files may supersede the Intel(R) math library and affect performance
icc: warning #10315: specifying -lm before files may supersede the Intel(R) math library and affect performance
icc: warning #10315: specifying -lm before files may supersede the Intel(R) math library and affect performance
GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking for shl_load... no
checking for shl_load in -ldld... no
checking for dlopen... yes
checking whether a program can dlopen itself... no
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... no
checking how to run the C++ preprocessor... CC -E
checking for ld used by CC... /usr/bin/ld -m elf_x86_64
checking if the linker (/usr/bin/ld -m elf_x86_64) is GNU ld... yes
icpc: warning #10315: specifying -lm before files may supersede the Intel(R) math library and affect performance
icpc: warning #10315: specifying -lm before files may supersede the Intel(R) math library and affect performance
icpc: warning #10315: specifying -lm before files may supersede the Intel(R) math library and affect performance
icpc: warning #10315: specifying -lm before files may supersede the Intel(R) math library and affect performance
icpc: warning #10315: specifying -lm before files may supersede the Intel(R) math library and affect performance
icpc: warning #10315: specifying -lm before files may supersede the Intel(R) math library and affect performance
icpc: warning #10315: specifying -lm before files may supersede the Intel(R) math library and affect performance
icpc: warning #10315: specifying -lm before files may supersede the Intel(R) math library and affect performance
icpc: warning #10315: specifying -lm before files may supersede the Intel(R) math library and affect performance
icpc: warning #10315: specifying -lm before files may supersede the Intel(R) math library and affect performance
icpc: warning #10315: specifying -lm before files may supersede the Intel(R) math library and affect performance
icpc: warning #10315: specifying -lm before files may supersede the Intel(R) math library and affect performance
icpc: warning #10315: specifying -lm before files may supersede the Intel(R) math library and affect performance
icpc: warning #10315: specifying -lm before files may supersede the Intel(R) math library and affect performance
icpc: warning #10315: specifying -lm before files may supersede the Intel(R) math library and affect performance
icpc: warning #10315: specifying -lm before files may supersede the Intel(R) math library and affect performance
icpc: warning #10315: specifying -lm before files may supersede the Intel(R) math library and affect performance
checking whether the CC linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking for CC option to produce PIC... -fPIC -DPIC
checking if CC PIC flag -fPIC -DPIC works... yes
checking if CC static flag -static works... yes
checking if CC supports -c -o file.o... yes
checking if CC supports -c -o file.o... (cached) yes
checking whether the CC linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking dynamic linker characteristics... (cached) GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
config.status: creating include/Makefile
config.status: creating src/Makefile
config.status: creating doc/Makefile
config.status: creating doc/doxygen-config.cfg
config.status: creating utils/Makefile
config.status: creating hwloc.pc
config.status: creating tests/Makefile
config.status: creating tests/linux/Makefile
config.status: creating tests/linux/gather/Makefile
config.status: creating tests/xml/Makefile
config.status: creating tests/ports/Makefile
config.status: creating tests/rename/Makefile
config.status: creating tests/linux/hwloc-gather-topology
config.status: creating tests/linux/gather/test-gather-topology.sh
config.status: creating tests/linux/test-topology.sh
config.status: creating tests/xml/test-topology.sh
config.status: creating tests/wrapper.sh
config.status: creating utils/hwloc-assembler-remote
config.status: creating utils/test-hwloc-annotate.sh
config.status: creating utils/test-hwloc-assembler.sh
config.status: creating utils/test-hwloc-calc.sh
config.status: creating utils/test-hwloc-distances.sh
config.status: creating utils/test-hwloc-distrib.sh
config.status: creating utils/test-hwloc-info.sh
config.status: creating utils/test-hwloc-ls.sh
config.status: creating utils/test-fake-plugin.sh
config.status: creating include/private/autogen/config.h
config.status: creating include/hwloc/autogen/config.h
config.status: linking src/topology-solaris.c to tests/ports/topology-solaris.c
config.status: linking src/topology-solaris-chiptype.c to tests/ports/topology-solaris-chiptype.c
config.status: linking src/topology-aix.c to tests/ports/topology-aix.c
config.status: linking src/topology-osf.c to tests/ports/topology-osf.c
config.status: linking src/topology-windows.c to tests/ports/topology-windows.c
config.status: linking src/topology-darwin.c to tests/ports/topology-darwin.c
config.status: linking src/topology-freebsd.c to tests/ports/topology-freebsd.c
config.status: linking src/topology-netbsd.c to tests/ports/topology-netbsd.c
config.status: linking src/topology-hpux.c to tests/ports/topology-hpux.c
config.status: linking src/topology-bgq.c to tests/ports/topology-bgq.c
config.status: linking src/topology-opencl.c to tests/ports/topology-opencl.c
config.status: linking src/topology-cuda.c to tests/ports/topology-cuda.c
config.status: linking src/topology-nvml.c to tests/ports/topology-nvml.c
config.status: linking src/topology-gl.c to tests/ports/topology-gl.c
config.status: executing depfiles commands
config.status: executing chmoding-scripts commands
config.status: executing libtool commands

-----------------------------------------------------------------------------
Hwloc optional build support status (more details can be found above):

Probe / display I/O devices: no
Graphical output (Cairo):    no
XML input / output:          basic
libnuma memory support:      no
Plugin support:              no
-----------------------------------------------------------------------------

hwloc: Building...
Making all in src
  CC       topology.lo
  CC       traversal.lo
  CC       distances.lo
  CC       components.lo
  CC       bind.lo
  CC       bitmap.lo
  CC       misc.lo
  CC       base64.lo
icc: command line warning #10121: overriding '-xCORE-AVX2' with '-xHost'
icc: command line warning #10121: overriding '-xCORE-AVX2' with '-xHost'
icc: command line warning #10121: overriding '-xCORE-AVX2' with '-xHost'
icc: command line warning #10121: overriding '-xCORE-AVX2' with '-xHost'
icc: command line warning #10121: overriding '-xCORE-AVX2' with '-xHost'
icc: command line warning #10121: overriding '-xCORE-AVX2' with '-xHost'
icc: command line warning #10121: overriding '-xCORE-AVX2' with '-xHost'
icc: command line warning #10121: overriding '-xCORE-AVX2' with '-xHost'
  CC       topology-noos.lo
  CC       topology-synthetic.lo
  CC       topology-custom.lo
  CC       topology-xml.lo
icc: command line warning #10121: overriding '-xCORE-AVX2' with '-xHost'
  CC       topology-xml-nolibxml.lo
icc: command line warning #10121: overriding '-xCORE-AVX2' with '-xHost'
icc: command line warning #10121: overriding '-xCORE-AVX2' with '-xHost'
icc: command line warning #10121: overriding '-xCORE-AVX2' with '-xHost'
  CC       topology-linux.lo
icc: command line warning #10121: overriding '-xCORE-AVX2' with '-xHost'
  CC       topology-x86.lo
icc: command line warning #10121: overriding '-xCORE-AVX2' with '-xHost'
icc: command line warning #10121: overriding '-xCORE-AVX2' with '-xHost'
  CCLD     libhwloc.la
Making all in include
Making all in utils
  CC       lstopo_no_graphics-lstopo.o
  CC       lstopo_no_graphics-lstopo-color.o
  CC       lstopo_no_graphics-lstopo-text.o
  CC       lstopo_no_graphics-lstopo-draw.o
  CC       lstopo_no_graphics-lstopo-fig.o
  CC       lstopo_no_graphics-lstopo-xml.o
  CC       hwloc-annotate.o
  CC       hwloc-assembler.o
icc: command line warning #10121: overriding '-xCORE-AVX2' with '-xHost'
icc: command line warning #10121: overriding '-xCORE-AVX2' with '-xHost'
icc: command line warning #10121: overriding '-xCORE-AVX2' with '-xHost'
icc: command line warning #10121: overriding '-xCORE-AVX2' with '-xHost'
icc: command line warning #10121: overriding '-xCORE-AVX2' with '-xHost'
icc: command line warning #10121: overriding '-xCORE-AVX2' with '-xHost'
icc: command line warning #10121: overriding '-xCORE-AVX2' with '-xHost'
icc: command line warning #10121: overriding '-xCORE-AVX2' with '-xHost'
  CC       hwloc-calc.o
  CC       hwloc-bind.o
  CC       hwloc-distances.o
  CC       hwloc-distrib.o
icc: command line warning #10121: overriding '-xCORE-AVX2' with '-xHost'
icc: command line warning #10121: overriding '-xCORE-AVX2' with '-xHost'
  CC       hwloc-info.o
icc: command line warning #10121: overriding '-xCORE-AVX2' with '-xHost'
icc: command line warning #10121: overriding '-xCORE-AVX2' with '-xHost'
  CC       hwloc-ps.o
icc: command line warning #10121: overriding '-xCORE-AVX2' with '-xHost'
Creating hwloc.7 man page...
Creating lstopo.1 man page...
Creating hwloc-annotate.1 man page...
Creating hwloc-bind.1 man page...
Creating hwloc-calc.1 man page...
Creating hwloc-distances.1 man page...
Creating hwloc-distrib.1 man page...
Creating hwloc-info.1 man page...
Creating hwloc-ps.1 man page...
Creating hwloc-assembler.1 man page...
Creating hwloc-assembler-remote.1 man page...
icc: command line warning #10121: overriding '-xCORE-AVX2' with '-xHost'
Creating hwloc-gather-topology.1 man page...
  CCLD     hwloc-annotate
  CCLD     hwloc-assembler
  CCLD     hwloc-distances
icc: command line warning #10121: overriding '-xCORE-AVX2' with '-xHost'
  CCLD     hwloc-distrib
  CCLD     hwloc-ps
icc: command line warning #10121: overriding '-xCORE-AVX2' with '-xHost'
icc: command line warning #10121: overriding '-xCORE-AVX2' with '-xHost'
  CCLD     hwloc-calc
icc: command line warning #10121: overriding '-xCORE-AVX2' with '-xHost'
icc: command line warning #10121: overriding '-xCORE-AVX2' with '-xHost'
  CCLD     hwloc-bind
ld: attempted static link of dynamic object `/work/barkovmm/ET/Cactus/configs/cid_rg/scratch/build/hwloc/hwloc-1.7.2/src/.libs/libhwloc.so'
ld: attempted static link of dynamic object `/work/barkovmm/ET/Cactus/configs/cid_rg/scratch/build/hwloc/hwloc-1.7.2/src/.libs/libhwloc.so'
make[5]: *** [hwloc-assembler] Error 1
make[5]: *** Waiting for unfinished jobs....
make[5]: *** [hwloc-annotate] Error 1
icc: command line warning #10121: overriding '-xCORE-AVX2' with '-xHost'
ld: attempted static link of dynamic object `/work/barkovmm/ET/Cactus/configs/cid_rg/scratch/build/hwloc/hwloc-1.7.2/src/.libs/libhwloc.so'
make[5]: *** [hwloc-distances] Error 1
icc: command line warning #10121: overriding '-xCORE-AVX2' with '-xHost'
ld: attempted static link of dynamic object `/work/barkovmm/ET/Cactus/configs/cid_rg/scratch/build/hwloc/hwloc-1.7.2/src/.libs/libhwloc.so'
make[5]: *** [hwloc-distrib] Error 1
ld: attempted static link of dynamic object `/work/barkovmm/ET/Cactus/configs/cid_rg/scratch/build/hwloc/hwloc-1.7.2/src/.libs/libhwloc.so'
make[5]: *** [hwloc-ps] Error 1
ld: attempted static link of dynamic object `/work/barkovmm/ET/Cactus/configs/cid_rg/scratch/build/hwloc/hwloc-1.7.2/src/.libs/libhwloc.so'
make[5]: *** [hwloc-calc] Error 1
ld: attempted static link of dynamic object `/work/barkovmm/ET/Cactus/configs/cid_rg/scratch/build/hwloc/hwloc-1.7.2/src/.libs/libhwloc.so'
make[5]: *** [hwloc-bind] Error 1
make[4]: *** [all-recursive] Error 1
make[2]: *** [make.checked] Error 2
make[1]: *** [/work/barkovmm/ET/Cactus/configs/cid_rg/lib/libthorn_hwloc.a] Error 2
make: *** [cid_rg] Error 2

Keyword: HWLOC

[Roland Haas]

On Crays you cannot link a dynamic library. However due to issues with hwloc we force it to produce only a dynamic library. This will fail on any Cray where one tries to compile hwloc. I think.

[Roland Haas]

Indeed Sherwood Richers can no longer compile on bluewaters.

[Roland Haas]

Cactus (ET trunk) now fails to build (once again) on a freshly installed Ubuntu 15.04 with libnuma-dev installed (even when using HWLOC_DIR=BUILD and the very newest ubuntu.cfg) due to hwloc's configure script finding libnuma but Cactus not linking against it.

The only real solution to this seems to me to move the call to "configure" from build.sh into detect.sh so that one can inspect the results of configure afterwards.

Just blindly adding numa to HWLOC_LIBS will not work since the required library libnuma.a (for static linking as we now require this) is only present (in Ubunut at least) if libnuma-dev is installed.

As far as I am concerned we need to be able to build on typical user Desktop machines (and laptops) out of the box. Typical, in my opinion, being the current version of OSX, Ubuntu, RedHat and possibly Debian and Fedora. Obviously we also need to be able to build on the public clusters so we cannot revert to dynamic linking again since it breaks all Crays.

The error I get is:

Creating cactus_sim in /home/rhaas/Cactus/exe from EinsteinAnalysis/ADMAnalysis EinsteinBase/ADMBase EinsteinBase/ADMCoupling EinsteinBase/ADMMacros AEIThorns/ADMMass AEIThorns/AEILocalInterp EinsteinAnalysis/AHFinder EinsteinAnalysis/AHFinderDirect ExternalLibraries/BLAS CactusBase/Boundary EinsteinAnalysis/CalcK Carpet/Carpet Carpet/CarpetEvolutionMask Carpet/CarpetIOASCII Carpet/CarpetIOBasic Carpet/CarpetIOHDF5 Carpet/CarpetIOScalar Carpet/CarpetIntegrateTest Carpet/CarpetInterp Carpet/CarpetInterp2 Carpet/CarpetLib Carpet/CarpetMask Carpet/CarpetProlongateTest Carpet/CarpetReduce Carpet/CarpetRegrid Carpet/CarpetRegrid2 Carpet/CarpetRegridTest Carpet/CarpetSlab Carpet/CarpetTracker CactusBase/CartGrid3D CactusNumerical/Cartoon2D EinsteinBase/Constants CactusBase/CoordBase EinsteinBase/CoordGauge Carpet/CycleClock CactusExamples/DemoInterp CactusNumerical/Dissipation EinsteinInitialData/DistortedBHIVP EinsteinAnalysis/EHFinder EinsteinBase/EOS_Base EinsteinEOS/EOS_Hybrid EinsteinEOS/EOS_IdealFluid EinsteinEOS/EOS_Omni EinsteinEOS/EOS_Polytrope EinsteinExact/EinsteinExact_Test CactusElliptic/EllBase CactusElliptic/EllSOR EinsteinInitialData/Exact EinsteinAnalysis/Extract ExternalLibraries/FFTW3 CactusExamples/FleshInfo CactusUtils/Formaline CactusBase/Fortran EinsteinEvolve/GRHydro EinsteinEvolve/GRHydro_InitData ExternalLibraries/GSL EinsteinExact/GaugeWave KrancNumericalTools/GenericFD ExternalLibraries/HDF5 CactusConnect/HTTPD CactusConnect/HTTPDExtra CactusExamples/HelloWorld Carpet/HighOrderWaveTest EinsteinBase/HydroBase EinsteinAnalysis/Hydro_Analysis EinsteinInitialData/Hydro_InitExcision EinsteinInitialData/IDAnalyticBH EinsteinInitialData/IDAxiBrillBH EinsteinInitialData/IDAxiOddBrillBH EinsteinInitialData/IDBrillData EinsteinInitialData/IDConstraintViolate EinsteinInitialData/IDFileADM EinsteinInitialData/IDLinearWaves CactusWave/IDScalarWave CactusWave/IDScalarWaveC CactusWave/IDScalarWaveCXX CactusWave/IDScalarWaveElliptic CactusExamples/IDWaveMoL CactusBase/IOASCII CactusBase/IOBasic CactusPUGHIO/IOHDF5 CactusPUGHIO/IOHDF5Util CactusIO/IOJpeg CactusBase/IOUtil CactusBase/InitBase CactusNumerical/InterpToArray EinsteinExact/KerrSchild ExternalLibraries/LAPACK ExternalLibraries/LORENE CactusNumerical/LocalInterp CactusNumerical/LocalInterp2 CactusNumerical/LocalReduce Carpet/LoopControl McLachlan/ML_ADMConstraints McLachlan/ML_ADMQuantities McLachlan/ML_BSSN McLachlan/ML_BSSN_Helper McLachlan/ML_BSSN_Test McLachlan/ML_CCZ4 McLachlan/ML_CCZ4_Helper McLachlan/ML_CCZ4_Test McLachlan/ML_WaveToy McLachlan/ML_WaveToy_Test ExternalLibraries/MPI CactusUtils/MemSpeed EinsteinInitialData/Meudon_Bin_BH EinsteinInitialData/Meudon_Bin_NS EinsteinInitialData/Meudon_Mag_NS EinsteinExact/Minkowski CactusNumerical/MoL EinsteinExact/ModifiedSchwarzschildBL EinsteinAnalysis/Multipole CactusUtils/NaNCatcher CactusUtils/NaNChecker EinsteinEvolve/NewRad CactusUtils/Nice EinsteinInitialData/NoExcision CactusUtils/NoMPI CactusNumerical/Noise CactusNumerical/Norms PITTNullCode/NullConstr PITTNullCode/NullDecomp PITTNullCode/NullEvolve PITTNullCode/NullExact PITTNullCode/NullGrid PITTNullCode/NullInterp PITTNullCode/NullNews PITTNullCode/NullPsiInt PITTNullCode/NullSHRExtract PITTNullCode/NullVars ExternalLibraries/OpenSSL EinsteinAnalysis/Outflow ExternalLibraries/PAPI CactusPUGH/PUGH CactusPUGH/PUGHInterp CactusPUGH/PUGHReduce CactusPUGH/PUGHSlab CactusNumerical/Periodic LSUThorns/PeriodicCarpet CactusExamples/Poisson AEIThorns/PunctureTracker LSUThorns/QuasiLocalMeasures Carpet/ReductionTest Carpet/ReductionTest2 Carpet/ReductionTest3 CactusNumerical/ReflectionSymmetry Carpet/RegridSyncTest EinsteinInitialData/RotatingDBHIVP CactusNumerical/RotatingSymmetry180 CactusNumerical/RotatingSymmetry90 CactusExamples/SampleBoundary CactusExamples/SampleIO EinsteinUtils/SetMask_SphericalSurface EinsteinExact/ShiftedGaugeWave CactusNumerical/Slab CactusNumerical/SlabTest CactusConnect/Socket CactusNumerical/SpaceMask PITTNullCode/SphericalHarmonicDecomp PITTNullCode/SphericalHarmonicRecon CactusNumerical/SphericalSurface EinsteinBase/StaticConformal LSUThorns/SummationByParts CactusBase/SymBase AEIThorns/SystemStatistics CactusUtils/SystemTopology CactusElliptic/TATelliptic EinsteinUtils/TGRtensor EinsteinInitialData/TOVSolver CactusUtils/TerminationTrigger CactusTest/TestArrays Carpet/TestCarpetGridInfo CactusTest/TestComplex CactusTest/TestCoordinates CactusTest/TestFortranCrayPointers CactusTest/TestFortranDependencies1 CactusTest/TestFortranDependencies2 CactusTest/TestFpointerNULL CactusTest/TestFreeF90 CactusTest/TestGlobalReduce CactusTest/TestInclude1 CactusTest/TestInclude2 CactusNumerical/TestLocalInterp2 CactusNumerical/TestLocalReduce CactusTest/TestLoop Carpet/TestLoopControl CactusTest/TestMath CactusTest/TestMoL CactusTest/TestPar CactusTest/TestReduce CactusTest/TestSchedule CactusTest/TestStrings CactusTest/TestTable CactusTest/TestTimers CactusTest/TestTypes CactusBase/Time CactusExamples/TimerInfo CactusUtils/TimerReport Carpet/Timers EinsteinBase/TmunuBase AEIThorns/Trigger EinsteinInitialData/TwoPunctures EinsteinExact/Vaidya2 LSUThorns/Vectors CactusWave/WaveBinarySource CactusExamples/WaveMoL CactusExamples/WaveToy1DF77 CactusExamples/WaveToy2DF77 CactusWave/WaveToyC CactusWave/WaveToyCXX CactusWave/WaveToyExtra CactusWave/WaveToyF77 CactusWave/WaveToyF90 CactusWave/WaveToyFreeF90 EinsteinAnalysis/WeylScal4 ExternalLibraries/hwloc ExternalLibraries/libjpeg ExternalLibraries/zlib
Formaline: Creating git local repository...
Formaline: Pushing source tree to master git repository...
Formaline: Pushing to local repository /home/rhaas/Cactus/../CactusSourceJar.git...
Formaline: Optimising git repository (slow only the first time)...
Formaline: Done.
Formaline: (Formaline has finished updating the git repositories.
Formaline:  If the build process seems to hang, some other command
Formaline:  is still running, e.g. the final link stage.)
/home/rhaas/Cactus/configs/sim/scratch/external/hwloc/lib/libhwloc.a(topology-linux.o): In function `hwloc_linux_set_thisthread_membind':
/home/rhaas/Cactus/configs/sim/scratch/build/hwloc/hwloc-1.10.1/src/topology-linux.c:1302: undefined reference to `migrate_pages'
/home/rhaas/Cactus/configs/sim/scratch/build/hwloc/hwloc-1.10.1/src/topology-linux.c:1314: undefined reference to `set_mempolicy'
/home/rhaas/Cactus/configs/sim/scratch/build/hwloc/hwloc-1.10.1/src/topology-linux.c:1291: undefined reference to `set_mempolicy'
/home/rhaas/Cactus/configs/sim/scratch/external/hwloc/lib/libhwloc.a(topology-linux.o): In function `hwloc_linux_set_area_membind':
/home/rhaas/Cactus/configs/sim/scratch/build/hwloc/hwloc-1.10.1/src/topology-linux.c:1243: undefined reference to `mbind'
/home/rhaas/Cactus/configs/sim/scratch/build/hwloc/hwloc-1.10.1/src/topology-linux.c:1224: undefined reference to `mbind'
/home/rhaas/Cactus/configs/sim/scratch/external/hwloc/lib/libhwloc.a(topology-linux.o): In function `hwloc_linux_find_kernel_max_numnodes':
/home/rhaas/Cactus/configs/sim/scratch/build/hwloc/hwloc-1.10.1/src/topology-linux.c:1347: undefined reference to `get_mempolicy'
/home/rhaas/Cactus/configs/sim/scratch/external/hwloc/lib/libhwloc.a(topology-linux.o): In function `hwloc_linux_get_area_membind':
/home/rhaas/Cactus/configs/sim/scratch/build/hwloc/hwloc-1.10.1/src/topology-linux.c:1445: undefined reference to `get_mempolicy'
/home/rhaas/Cactus/configs/sim/scratch/external/hwloc/lib/libhwloc.a(topology-linux.o): In function `hwloc_linux_get_thisthread_membind':
/home/rhaas/Cactus/configs/sim/scratch/build/hwloc/hwloc-1.10.1/src/topology-linux.c:1392: undefined reference to `get_mempolicy'
collect2: error: ld returned 1 exit status
/home/rhaas/Cactus/lib/make/make.configuration:149: recipe for target '/home/rhaas/Cactus/exe/cactus_sim' failed
make[1]: *** [/home/rhaas/Cactus/exe/cactus_sim] Error 1
Makefile:254: recipe for target 'sim' failed
make: *** [sim] Error 2

[Erik Schnetter]

For the release, I suggest to require libnuma-dev, and to add hwloc to HWLOC_EXTRA_LIBS.

[Bruno Mundim]

I was able to build with the current ubuntu.cfg in simfactory for Ubuntu 14.04.2 LTS without any serious problem. You do have to install libnuma-dev though. There is quite a bit of annoying warning messages of the kind:

...
Warning: Nonexistent include directory "/home/bruno/tmp/ET_git_dev/Cactus/arrangements/EinsteinEvolve/GRHydro/src/include"
...

I don't know yet exactly where they are coming from. I will increase verbosity and build on one process only to find out.

[Erik Schnetter]

These warning messages probably come from the Fortran compiler. There is an option to disable the warnings, but the option is broken in GCC.

[Roland Haas]

Replying to [comment:6 bmundim]:

I was able to build with the current ubuntu.cfg in simfactory for Ubuntu 14.04.2 LTS without any serious problem. You do have to install libnuma-dev though.

How strange on my freshly installed Ubuntu 15.04 testing VM using the current (as of right now) trunk, I can only compile if I do not install libnuma-dev. The packages (beyond the default set that the installer installs are): build-essential mpich2 libmpich2?-dev gfortran git subversion curl gnuplot gnuplot-x11

[Bruno Mundim]

Ok, I had problems compiling with mpich2 in the past and I switched to openmpi. My configuration with openmpi was

MPI_DIR      = /usr
MPI_INC_DIRS = /usr/lib/openmpi/include /usr/lib/openmpi/include/openmpi
MPI_LIB_DIRS = /usr/lib /usr/lib/openmpi/lib
MPI_LIBS     = mpi mpi_cxx dl hwloc

but it also works as it is now:

MPI_DIR      = /usr

as long as libopenmpi-dev is installed, but that's only my guess on what is different between your configuration and mine (besides the ubuntu release).

[Roland Haas]

That probably explains it, yes. I used ubuntu.cfg with mpich2 as insructed in there (I do this to test the simplified tuttorial so do not want to do anything different than what a first time user who follows the instructions would do). OpenMPI links to libnuma and libhwloc:

rhaas@T1650-403463:~$ ldd /usr/lib/libmpi.so | grep numa libnuma.so.1 => /usr/lib/x86_64-linux-gnu/libnuma.so.1 (0x00007f0b60c35000)

so that you will get the missing symbols through them. Possibly you also end up linking agaisnt the system hwloc and not against the one you just build (probably depends on where in the linker line you encounter -lmpi).

[Erik Schnetter]

Why would build hwloc if there is a system hwloc?

Yes, all the *-dev packages need to be installed.

The linker has MPI before hwloc, since MPI depends on hwloc.

[Roland Haas]

It builds it because HWLOC_DIR is set to BUILD. There is a comment in there stating that the system hwloc is broken:

# Ubuntu's libhwloc-dev is broken; see
# <https://trac.einsteintoolkit.org/ticket/1717>

[Bruno Mundim]

so that you will get the missing symbols through them. Possibly you also end up linking agaisnt the system hwloc and not against the one you just build (probably depends on where in the linker line you encounter -lmpi).

yes, at the moment it links to the system hwloc, but that's what I wanted and what it is in the ubuntu.cfg file at the moment. I haven't tried building hwloc but I guess chances are we are going to pick the system one if -lmpi comes first.

[Bruno Mundim]

Replying to [comment:12 rhaas]:

It builds it because HWLOC_DIR is set to BUILD. There is a comment in there stating that the system hwloc is broken:

`

Ubuntu's libhwloc-dev is broken; see

https://trac.einsteintoolkit.org/ticket/1717

`

Please update your ubuntu.cfg file. There is no need to build hwloc. I added the recommended packages to be installed as requirements. The only one missing is openmpi which I forgot to mention. First we should decide between openmpi and mpich2. Openmpi works fine for me.

[Erik Schnetter]

In this case, MPI should also be set to "build".

We need to decide whether it is okay to ask for new packages to be installed on Ubuntu. Currently we do ask for packages, and if so, asking for hwloc-dev and numa-dev etc. should be fine. With these, hwloc is not broken any more; we just add -lnuma to the list of hwloc libraries.

Alternatively, we say that no new package should be required. We then call the system "hwloc" broken, since hwloc-dev may not be present. We then need to make sure we don't use it, neither directly nor indirectly. Every Ubuntu package that may use hwloc then cannot be used. Thus we need to build MPI ourselves, and possibly also HDF5 if we cannot ensure that the system HDF5 is a serial one. Overall, I don't think this is a good idea for Ubuntu in general.

Maybe we should have a "standalone.cfg" option list that does not expect anything from the system, but builds everything, unless we are really sure that a system library that we find is good. Given that libraries depend on each other, that's difficult to detect.

[Roland Haas]

I would rather not have things compile themselves on the workstation config files unless absolutely required. Those are likely the first ones that new users use and they should be quick to build. MPI does not build quickly (or at least it does not feel as if it does).

Ubuntu.cfg used to be specific to mpich2 only since it had to hard-code the mpi library names. Since some days ago I changed it to use Stave's detect.pl which queries mpicc (same as debian.cfg and fedora.cfg already do). So now one can use the same ubuntu.cfg file for both MPI variants. I think we need hwloc-dev libnuma-dev and libltdl-dev to enable static linking. libltdl-dev is missing from the current (Fri May 8 22:26:40 CEST 2015). libboost1.50-dev should not be added I think since boost is very big and not compiled in by the default ET thornlist. The option "-fno-range-check" was added, this option only affects compilation and allows constructs like "1d0 / 0d0" to produce infinity. Was that intended? The comment is incorrect assuming that libltdl-dev is actually required (and #1717 indicates it is).

Note that this only fixes part of the problem and we still will have failures if a user does not install those libraries since building hwloc may detect libnuma.a (if libnuma-dev is installed) but eg detect.sh does not report this required library (and cannot do so since the library may not be installed and hwloc would compile fine without it).

[Bruno Mundim]

Replying to [comment:16 rhaas]:

I would rather not have things compile themselves on the workstation config files unless absolutely required. Those are likely the first ones that new users use and they should be quick to build. MPI does not build quickly (or at least it does not feel as if it does).

I totally agree with you on this!

Ubuntu.cfg used to be specific to mpich2 only since it had to hard-code the mpi library names. Since some days ago I changed it to use Stave's detect.pl which queries mpicc (same as debian.cfg and fedora.cfg already do). So now one can use the same ubuntu.cfg file for both MPI variants.

The question is: does it work with both MPI variants? It didn't for me with most recent ubuntu releases and mpich2. I haven't tested it recently though. Maybe the MPI package should be added to the header comment too.

I think we need hwloc-dev libnuma-dev and libltdl-dev to enable static linking. libltdl-dev is missing from the current (Fri May 8 22:26:40 CEST 2015).

Ok, it is installed in my system but I missed to list it there. Sorry.

libboost1.50-dev should not be added I think since boost is very big and not compiled in by the default ET thornlist.

Yes, but it is a popular library and I know of at least 4 institutions in the ET community which use codes that link to Boost. Maybe we could rephrase the header comment as follows:

"Some libraries are very popular in the ET community but are not distributed by the toolkit. You might want to install their packages in your system before building ET. One such a popular library is Boost, libboost1.50-dev."

The option "-fno-range-check" was added, this option only affects compilation and allows constructs like "1d0 / 0d0" to produce infinity. Was that intended?

This option slipped from my private configuration file. One of our codes had a strange compilation abort which is fixed by adding this option. I will test it again without it to refresh my memory on this issue.

The comment is incorrect assuming that libltdl-dev is actually required (and #1717 indicates it is).

Yes, please update the comment with this library. I forgot to include it.

Note that this only fixes part of the problem and we still will have failures if a user does not install those libraries since building hwloc may detect libnuma.a (if libnuma-dev is installed) but eg detect.sh does not report this required library (and cannot do so since the library may not be installed and hwloc would compile fine without it).

We could for the time being state that those libraries are mandatory if you want to avoid building the external libraries.

[Erik Schnetter]

Is there a reason to switch to OpenMPI? If MPICH works fine, then let's stick with it for the release. Everything is open again one or two weeks from now.

Regarding Boost: I don't think we need to add such a comment. If people want Boost, then they'll figure out that they either have to install it or to build it themselves (maybe via Cactus). Also, Boost 1.50 is quite outdated; Boost 1.58 is current, and with two releases per year this makes it four years old. I expect many packages to simply not work with Boost 1.50.

We are very shortly before a release. Now is not the time to clean up things. Now is the time to test things, and to fix important things that can't be worked around.

[Frank Löffler]

Replying to [comment:17 bmundim]:

libboost1.50-dev should not be added I think since boost is very big and not compiled in by the default ET thornlist.

Yes, but it is a popular library and I know of at least 4 institutions in the ET community which use codes that link to Boost. Maybe we could rephrase the header comment as follows:

"Some libraries are very popular in the ET community but are not distributed by the toolkit. You might want to install their packages in your system before building ET. One such a popular library is Boost, libboost1.50-dev."

I don't quite get that argument. While boost is interesting and useful, it is not necessary for the Einstein Toolkit. If we would recommend useful packages, boost would only be one of a quite long list. And if you use boost already for something else, then most likely you have it installed already anyway.

[Frank Löffler]

Replying to [comment:18 eschnett]:

Is there a reason to switch to OpenMPI? If MPICH works fine, then let's stick with it for the release. Everything is open again one or two weeks from now.

Regarding Boost: I don't think we need to add such a comment. If people want Boost, then they'll figure out that they either have to install it or to build it themselves (maybe via Cactus). Also, Boost 1.50 is quite outdated; Boost 1.58 is current, and with two releases per year this makes it four years old. I expect many packages to simply not work with Boost 1.50.

We are very shortly before a release. Now is not the time to clean up things. Now is the time to test things, and to fix important things that can't be worked around.

For the first time now I really would like to have a 'like' button in trac. :)

[Bruno Mundim]

Replying to [comment:19 knarf]:

Replying to [comment:17 bmundim]:

libboost1.50-dev should not be added I think since boost is very big and not compiled in by the default ET thornlist.

Yes, but it is a popular library and I know of at least 4 institutions in the ET community which use codes that link to Boost. Maybe we could rephrase the header comment as follows:

"Some libraries are very popular in the ET community but are not distributed by the toolkit. You might want to install their packages in your system before building ET. One such a popular library is Boost, libboost1.50-dev."

I don't quite get that argument. While boost is interesting and useful, it is not necessary for the Einstein Toolkit. If we would recommend useful packages, boost would only be one of a quite long list. And if you use boost already for something else, then most likely you have it installed already anyway.

I wanted ubuntu.cfg to work out of the box for the students here, but that's ok. Please remove those comments. I will keep patching ubuntu.cfg locally.

Cheers!

[Frank Löffler]

Replying to [comment:21 bmundim]:

I wanted ubuntu.cfg to work out of the box for the students here, but that's ok. Please remove those comments. I will keep patching ubuntu.cfg locally.

ubuntu.cfg should work, shouldn't it? The list of recommended packages isn't used anywhere. It's really only a comment. Is the problem that a standard Ubuntu boost installation isn't found by the thorn, if not configured in the option list (I don't know if this is the case, you tell me)?

[Erik Schnetter]

Bruno -- why don't you add your own configuration file? We're always happy about contributions, in particular if they are actually used and thus genuinely useful and tested. ubuntu.cfg may not be the right place since it's supposed to be generic and for newcomers, and for a restricted set of thorns.

[Roland Haas]

I think the comments in the cfg files should ideally list two sets (a) the minimal packages required to build and (b) additional ones that can be used to speed up compilation of the thorn activated by default in the einsteintoolkit.th by simfactory on that machine. The hope is to have set (a) be more robust than set (a)+(b) so that the option list can also be used if users cannot eg install hdf5-serial-dev because petsc needs to be installed and wants hdf5-parallel.

Since we do not include boost in the default thornlist or have simfactory activate it by default I think we should not give the intention that it is required in the thorn lists.

[Bruno Mundim]

Replying to [comment:22 knarf]:

Replying to [comment:21 bmundim]:

I wanted ubuntu.cfg to work out of the box for the students here, but that's ok. Please remove those comments. I will keep patching ubuntu.cfg locally.

ubuntu.cfg should work, shouldn't it? The list of recommended packages isn't used anywhere. It's really only a comment. Is the problem that a standard Ubuntu boost installation isn't found by the thorn, if not configured in the option list (I don't know if this is the case, you tell me)?

It will build Boost and HDF5 libraries with the current ubuntu.cfg because it can't find them in the system directories. Those comments are to help new users on which packages to have installed in the system so that ET compilation goes faster. I suggested also to add openmpi to the list of packages installed in the comment because I had problems with mpich2 a few months ago. Both might work though. I guess Roland is trying a test with mpich2, right?

[Bruno Mundim]

Replying to [comment:23 eschnett]:

Bruno -- why don't you add your own configuration file? We're always happy about contributions, in particular if they are actually used and thus genuinely useful and tested. ubuntu.cfg may not be the right place since it's supposed to be generic and for newcomers, and for a restricted set of thorns.

At the moment (after the release) I think the best to do is to fix the issue of not finding system include and library directories.

[Roland Haas]

I used mpich in my tests so far and could compilewith the version of ubuntu.cfg that was in the repo at the time of comment:8 letting Cactus build hwloc (but could not have libnuma-dev installed). I can also compile with the version present before Monday using mpich as long as libnuma-dev and libtldl-dev are present.

The same option list should work for both mpi stacks since Steve's detect.pl queries mpicc for the correct libraries.

[Frank Löffler]

hwloc needs to be treated differently, but it now at least builds. Thus, closing this ticket.