Multiply defined symbol detail::device_allocator_core<cuda_device>::min_alignment w/ std=c++2a, gcc

[Paul Hargrove]

In a configuration where we build tests (not libupcxx.a) with -std=c++2a I see multiply a defined symbol link error on the memberof.cpp test.

To reproduce on Dirac with gcc/9.2.0 module loaded:

{phargrov@pcp-d-11 B}$ ../configure --with-cxxflags=-Wno-invalid-offsetof
{phargrov@pcp-d-11 B}$ make run SRC=memberof.cpp EXTRAFLAGS=-std=c++2a
[...]
/usr/local/pkg/gcc/9.2.0/bin/g++ -Wno-invalid-offsetof -std=c++11 -g3   -Wno-unused -Wunused-result -Wno-unused-parameter -Wno-address    -DUPCXX_ASSERT_ENABLED=1 -DUPCXX_BACKEND=1 -DUPCXX_MPSC_QUEUE_ATOMIC=1 -DUPCXX_BACKEND_GASNET_SEQ=1 -I/home/pcp1/phargrov/upcxx/B/bld/upcxx.assert1.optlev0.dbgsym1.gasnet_seq.smp/include -I/home/pcp1/phargrov/upcxx/B/bld/upcxx.assert1.optlev0.dbgsym1.gasnet_seq.smp/gen_include   -D_GNU_SOURCE=1 -DGASNET_SEQ     -I/home/pcp1/phargrov/upcxx/B/bld/GASNet-stable -I/home/pcp1/phargrov/upcxx/B/bld/GASNet-stable/smp-conduit -I/home/pcp1/phargrov/upcxx/B/bld/GASNet-stable/other   -I/home/pcp1/phargrov/upcxx/B/bld/GASNet-stable/extended-ref/vis -I/home/pcp1/phargrov/upcxx/B/bld/GASNet-stable/extended-ref/coll -I/home/pcp1/phargrov/upcxx/B/bld/GASNet-stable/extended-ref/ratomic -I/home/pcp1/phargrov/upcxx/B/bld/GASNet-stable/extended-ref -I/home/pcp1/phargrov/upcxx/B/bld/gasnet.debug    -g3  -Wno-unused -Wunused-result -Wno-unused-parameter -Wno-address      /home/pcp1/phargrov/upcxx/test/memberof.cpp -L/home/pcp1/phargrov/upcxx/B/bld/upcxx.assert1.optlev0.dbgsym1.gasnet_seq.smp/lib -lupcxx -L/home/pcp1/phargrov/upcxx/B/bld/gasnet.debug/smp-conduit  -lgasnet-smp-seq    -lrt -L/usr/local/pkg/gcc/9.2.0/lib/gcc/x86_64-pc-linux-gnu/9.2.0 -lgcc -lm   -o /home/pcp1/phargrov/upcxx/B/bld/upcxx.assert1.optlev0.dbgsym1.gasnet_seq.smp/exe/2c720dbda1981cfd3426e73442b7c97c6aaa9ac2 -std=c++2a
/home/pcp1/phargrov/upcxx/B/bld/upcxx.assert1.optlev0.dbgsym1.gasnet_seq.smp/lib/libupcxx.a(cuda.o):(.rodata+0x28): multiple definition of `upcxx::detail::device_allocator_core<upcxx::cuda_device>::min_alignment'
/tmp/cc9zUooU.o:(.rodata._ZN5upcxx6detail21device_allocator_coreINS_11cuda_deviceEE13min_alignmentE[_ZN5upcxx6detail21device_allocator_coreINS_11cuda_deviceEE13min_alignmentE]+0x0): first defined here
collect2: error: ld returned 1 exit status
make[4]: *** [/home/pcp1/phargrov/upcxx/B/bld/upcxx.assert1.optlev0.dbgsym1.gasnet_seq.smp/exe/2c720dbda1981cfd3426e73442b7c97c6aaa9ac2] Error 1

The --with-cxxflags=-Wno-invalid-offsetof is NOT related to the problem, but squelches warnings (expected) on this test.

Adding -std=c++2a to those configured CXXFLAGS, however, eliminates the linker error.
So the problem seems related to mixing the two language levels.

An equivalent tests with Intel and Clang compilers do not reproduce the problem.

I believe EX-dirac-ibv-gcc will show this in automated testing on Monday.

[Paul Hargrove]

@Dan Bonachea I have assigned to you initially to verify this is not something "dodgy" in the test itself.

[Paul Hargrove]

More testing shows this is debug-only.

[Dan Bonachea]

New test test/memberof.cpp just happens to tickle this pre-existing problem.

Minimal reproducer:

#include <upcxx/upcxx.hpp>

int main(){
  upcxx::init();

  volatile bool cuda_enabled = false;
  upcxx::cuda_device *gpu_device;
  upcxx::device_allocator<upcxx::cuda_device> *gpu_alloc;

  if (cuda_enabled) {
//    gpu_device = new upcxx::cuda_device( 0 ); // Open device 0
    gpu_alloc = new upcxx::device_allocator<upcxx::cuda_device>(*gpu_device, 16*1024);

    upcxx::global_ptr<int,upcxx::memory_kind::cuda_device> g = gpu_alloc->allocate<int>(4);
  }

  upcxx::finalize();
  return 0;
}

[Dan Bonachea]

The problem appears to be the following line from cuda.cpp:8:

  constexpr std::size_t detail::device_allocator_core<upcxx::cuda_device>::min_alignment; // required before C++17

If this line is removed and library is built with -std=c++11, the link failure with application -std=c++2a goes away.
However doing so breaks the link (undefined symbol) for applications built with -std=c++11/14.

This smelled suspiciously like issue #181, so I went ahead and adapted the same idea into a solution :
see pull request #182, which in my testing resolves the problem for library=c++11 and app=c++{11,14,17,2a}

[Dan Bonachea]

Fix issue #333: Multiply defined symbol detail::device_allocator_core<cuda_device>::min_alignment w/ std=c++2a

→ <<cset ad6d87386ce2>>

[Dan Bonachea]

Merge pull request #182 into develop

• issue333: Update ChangeLog Fix issue #333: Multiply defined symbol detail::device_allocator_core<cuda_device>::min_alignment w/ std=c++2a

→ <<cset 9af760bbef97>>