nobs run only works with smp-conduit

[Steven Hofmeyr]

On linux, setting the conduit to udp results in an error for nobs:

export GASNET_CONDUIT=udp
nobs run hello_upcc.cpp
...
*** FATAL ERROR: implicit-master without argv not supported - use amudprun
WARNING: Ignoring call to gasneti_print_backtrace_ifenabled before gasneti_backtrace_init
Aborted (core dumped)

However, I can run successfully with udp if I do the following:

.nobs/art/d0a50efd9b1d4e33a3a1cf4e30861f44654ebd9a/other/amudp/amudprun -np 4 -spawn L $(nobs exe test/hello_upcxx.cpp)

Is this expected behavior? If so, I'd like to document it and maybe we should have a shortcut to amudprun, so the user doesn't have to dig through the .nobs directory to get it.

[Steven Hofmeyr]

It appears that amudprun does not get installed when running just nobs. However, when running ./install, it does get installed.

[Dan Bonachea]

nobs run implicitly assumes smp-conduit.

It is broken for all other conduits, all of which require proper use of a job spawner.

So we can either teach nobs the proper run commands for each conduit, or stop using nobs run for non-smp jobs.

At the very least nobs run should probably print an informative error if conduit is set to something other than smp, rather than running something we know will fail in potentially confusing ways.

CC: @PHHargrove

[Dan Bonachea]

I've added a unified spawn script in 41f5a97

nobs run should now be taught to do the equivalent of:

export GASNET_PREFIX=/path/to/.nobs/hidden/gasnet/buildtree
$NOBS_ROOT/utils/upcxx-run $RANKS $exe

and then we can close this issue.

The run-tests script for udp will probably additionally want to set GASNET_SPAWNFN=L, to avoid the complications of ssh-spawning on remote nodes for a smoke-screen test.

[Steven Hofmeyr]

I have modified run-tests to use either udp or smp conduits. These are the only conduits supported right now, and the udp conduit uses fork/exec to spawn. We should decide what other conduits to support before the release.

[Dan Bonachea]

If John makes the nobs run modification I suggest above to plug its hidden build tree into upcxx-run, then we can rollback Steve's run-tests change and continue using nobs run, which will be faster by skipping an install-to-hidden-directory step and also automatically support additional conduits (aries and ibv being the others supported in this release).

CC: @PHHargrove

[Steven Hofmeyr]

That would be better, but by default nobs run doesn't build amudprun, even if GASNET_CONDUIT=udp is set. The run-tests script simply calls nobs run to do all of the building, just explicitly specifying the conduit. And also note that there will always be an install-to-hidden-directory step, since gasnet has to be installed somewhere.

For now, we have something that works in a limited way.

[Dan Bonachea]

For now, we have something that works in a limited way.

I agree something is better than nothing, and John's time is limited.

And also note that there will always be an install-to-hidden-directory step, since gasnet has to be installed somewhere.

upcxx-run will also work with $GASNET_PREFIX set to the root of the GASNet build directory, so no install required if nobs provides its hidden builddir when invoking upcxx-run (or gives run-tests a way to retrieve that pathname).

However you are correct that nobs does not currently build the amudprun spawner before install. It's possible to augment udp-conduit 'make seq' to do that, but too late to make that change for this release. For now either nobs or run-tests could make -C $gasnetbuilddir/other/amudp amudprun when conduit==udp.

[Steven Hofmeyr]

This makes sense. Tomorrow I'll change run-tests to use upcxx-run and add a make of amudprun to the script, using the existing nobs gasnet source tree.

[Former user]

I'm adding upcxx-run to nobs run today.

[Former user]

Fixed issue 69: nobs run leverages universal spawner. run-tests updated accordingly.

→ <<cset 405cfdc3bd24>>