PG: Expand description of upcxx-run to describe use of underlying spawners

Issue #315 resolved
Dan Bonachea created an issue

The upcxx-run -show and upcxx-run -v options provide the means to access the information about how to invoke a UPC++ program directly using the underlying spawner (srun, aprun, etc). The idea is this provides an "escape hatch" whereby users can launch directly using the underlying system spawner (which often provides options for finer-grained or system-specific job control), with all the appropriate envvars to control UPC++ operation.

At the 2020-02-12 meeting, we identified that we should describe this capability/strategy to users in the upcxx-run section of the programming guide.

Comments (5)

  1. Paul Hargrove

    Notes:

    Except for SMP and UDP, -show and -v are going to just show a GASNet-level spawner.
    In general, if upcxx-run lacks the user's desired controls (such as NUMA or GPU bindings), then the GASNet-level one lacks them too.

    Unfortunately, I think the instructions might need to include running upcxx-run -show followed by a run of the GASNet-level spawner with -t just to allow the user to determine what GASNet was configured to use (perhaps by an admin who is not the current user).

    Of course, the main point we are is documenting is that aries (always) and ibv (by default) spawn using exactly the same mechanism(s) as MPI on the same system. That means mpirun, srun, aprun, jsrun all "just work" once one manages the few environment variables set transparently by upcxx-run.

  2. Dan Bonachea reporter

    Except for SMP and UDP, -show and -v are going to just show a GASNet-level spawner.

    upcxx-run -vv is usually sufficient to find the underlying spawn command when one exists (ssh-spawner being the counter-example, where they are just "stuck" and should use mpi-spawner if they want more control). It's just lost in a sea of output so they need a pointer to the right line

  3. Log in to comment