UPC++ Spack Package

[Jonathan Madsen]

Could y’all update the UPCXX spack package to include a +pic option? It is quite annoying to have to configure spack globally to include cflags: -fPIC, etc. or do spack install upcxx cflags=”-fPIC” cxxflags=”-fPIC” or worse, forgetting to do so and finding out much later that linking failed, especially when shared libraries are so common.

Also, it would also be nice if there was:

• +shared option to build shared libraries

  • Which I think is supported, if not NBD

• Ability to install only specific optimization levels

  • E.g. build_type=all vs. build_type=debug vs. build_type=opt

    • Personally, I find “ReleaseWithDebugInfo” builds to be very convenient (i.e. -O2 -g instead of just -O0 -g and -O3)

• Ability to install only specific backend configs

  • e.g. only O3 + gasnet_par + {udp,smp} (which I think y’all support but its been a while since I manually did this installation)

• A ~python (i.e. non-python) variant (which is optional correct?)

In general, most of these requests are w.r.t. trying to simplify and reduce the build-time of a complex installation that depends on UPC++

[Paul Hargrove]

Re: Shared libraries and PIC: There are non-trivial difficulties with invocation of RPCs in/from shared libraries in systems with ASLR, therefore we do not (yet) officially support or endorse use of relocation for UPC++ libraries or calling code. Therefore we are not adding -fPIC or shared libraries options to the Spack package.

Re: ~python: There was more dependence on python in the past, but currently UPC++ only requires python for the upcxx-run script. However, the configure script will not run to completion without finding a python interpreter. If you’re on a system like cori with PMI-based spawners like srun it’s possible to launch UPC++ programs without upcxx-run, provided you are willing to set all the necessary environment variables manually, however this is not something we officially encourage. All that being said, Spack itself is a Python program, so by definition Python is present on the system, and UPC++ supports a very wide range of Python versions -- so this dependency should be trivial to satisfy with the right Spack options. IOW we don’t see any strong motivation for providing the requested knob.

Re: eliminating threadmode (seq/par) and codemode (opt/debug) variants of GASNet/UPC++ libraries - we agree this could be useful to reduce build times and disk space, but our make infrastructure does not currently support this type of subsetting. So this is not a Spack request - we’d need some non-trivial changes to the package Makefiles first before this could be deployed. We could enter enhancement request issues for these two ideas, if you think they are of sufficient importance, but cannot commit to performing any work on this in the near term.

We DO support limiting the set of conduits built in the configure command, and the Spack package currently exposes the +mpi variant that does this, and aries is automatically enable/disabled based on the detection of the Cray XC target. Spack knobs could be added to explicitly enable/disable other conduits like ibv - is that what you’re trying to disable?

On the topic of build times, our Makefiles DO support parallel make, so instructing Spack to invoke make with -j (if it’s not doing so by default?) could save you some build time.

-- Dan and Paul

[Dan Bonachea]

Mass roll-over of open issues to next release milestone

[Dan Bonachea]

For the record, the Spack incantation to avoid rebuilding python is to specify the system python install in ~/.spack/packages.yaml.

For example, here is an excerpt that works on cori, in the format expected by the current develop version of Spack:

packages:
  python:
    buildable: false
    externals:
    - spec: python@2.7.15.7
      modules:
      - cray-python/2.7.15.7
    - spec: python@3.7.3.2
      modules:
      - cray-python/3.7.3.2

and here's an example from our local cluster running Scientific Linux:

packages:
  python:
    buildable: false

    externals:
    - spec: python@2.7.5
      prefix: /usr
    - spec: python@3.4.9
      prefix: /usr
    - spec: python@2.7.0
      modules:
      - python/2.7.0

[Dan Bonachea]

Mass roll-over of open issues to next release milestone

[Dan Bonachea]

Closing this due to perceived lack of interest on both sides.

Some of these ideas might eventually be deployed as a side-effect once the build system evolves further, but until then this is not something we are pursuing.

[Jonathan Madsen]

Should I assume the:

this is not something we are pursuing.

is related to everything else aside from the PIC issues? Bc not supporting PIC is a huge problem when you are trying distribute a library with UPC++ support.

[Paul Hargrove]

@Jonathan Madsen I think you've misunderstood.
Your quotation omits the key phrase "until then". If you reread the entire sentence you should hopefully see we are stating that we are not considering these ideas until there is a corresponding capability in our build system which we could expose though spack variants. If you want PIC support, then you do want that text to apply to it (otherwise it is dismissed entirely).

Support for UPC++ in/with shared libraries is a bigger issue than simply passing PIC flags to the compiler, as we alluded to in the initial response. We have issue 321 open related to that general topic, and once that is resolved it is reasonable to expect a subsequent spack package update would add a variant to enable a share-lib-friendly build of libupcxx. I am sorry that I don't have a good estimate for a time frame for that work to be completed.

As another example: issue 418 is a request for CMake-level exposure of a RelWithDebInfo build_type. Engineering an equivalent configure/build/install option entirely within the spack package today would be a duplication of effort with respect to that issue. Once our build/install system is capable of supporting that for the purpose of exposure through CMake, it becomes reasonable to consider a corresponding spack variant to also expose it.

If you have remaining un-answered concerns, you may email me privately to discuss them. We have communicated on other topics by email in the past, and so I assume you have my LBNL email address already.