hwloc in Cactus fails to link with cuda

Create issue
Issue #1732 closed
Frank Löffler created an issue

I get link errors like this:

/home/knarf/psamr/configs/sim/scratch/external/hwloc/lib/libhwloc.a(topology-cuda.o): In function `hwloc_cuda_backend_notify_new_object':
/home/knarf/psamr/configs/sim/scratch/build/hwloc/hwloc-1.7.2/src/topology-cuda.c:122: undefined reference to `cudaGetDeviceProperties'
/home/knarf/psamr/configs/sim/scratch/external/hwloc/lib/libhwloc.a(topology-cuda.o): In function `hwloc_cuda_query_devices':
/home/knarf/psamr/configs/sim/scratch/build/hwloc/hwloc-1.7.2/src/topology-cuda.c:38: undefined reference to `cudaGetDeviceCount'
/home/knarf/psamr/configs/sim/scratch/external/hwloc/lib/libhwloc.a(topology-cuda.o): In function `hwloc_cudart_get_device_pci_ids':
/home/knarf/psamr/configs/sim/scratch/build/hwloc/hwloc-1.7.2/include/hwloc/cudart.h:49: undefined reference to `cudaGetDeviceProperties'
collect2: error: ld returned 1 exit status

hwloc is built (system version present, but deemed too old). The configuration is using the standard debian option list (on spine). Everything is dev-version, clean checkout.

It seems to me that hwloc picks up a cuda dependency (present on system), but the linker isn't told about it (no cuda present in Cactus config).


Comments (6)

  1. Erik Schnetter
    • removed comment

    Ah yes, the joys of building late... We need to tell Cactus what libraries we want before hwloc is installed, i.e. before its pkgconfig file is written. The standard solutions are either (1) explicitly disable CUDA integration in hwloc, or (2) add these libraries to LDFLAGS and LIBS for this particular machine. Note that setting LIBS overwrites, so you'll need to manually add the automatically detected libraries as well.

  2. Frank Löffler reporter
    • removed comment

    For this particular case I think that a switch for using Cuda or not would be the easiest thing to do. If using Cuda we would need to point hwloc to the right installation anyway, i.e. the one Cactus plans to use. What do you (and others) think?

    The current situation means that the ET does not build if hwloc is built by the ET, and Cuda is installed, but not used by Cactus (which is the default).

  3. Frank Löffler reporter
    • changed status to resolved
    • removed comment

    I disabled cuda and opencl discovery in hwloc for now. If it is needed, we would need to check for support in Cactus first, and point hwloc to the right location as well.

  4. Steven R. Brandt
    • removed comment

    A related issue is that hwloc+Cactus doesn't compile on my platform without adding -lnuma to the final link.

  5. Log in to comment