- changed component to RPC
envvar-tunable RPC eager/rendezvous threshold
In today's SPMV meeting, we identified a need to allow the eager/rendzevous threshold used internally by the UPC++ RPC data transfer (gasnet::am_size_rdzv_cutover
) to be tunable by the end-user, up to the GASNet AM MaxMedium.
We should really add an envvar to turn this voodoo knob (probably not for the September release).
Comments (9)
-
reporter -
I have tried this yesterday on Cori Haswell and improvements are... non existent. Also, the variant in which a LPC is issued from within the RPC is slower than doing the flops directly inside the RPC.
I will forward the plots by email.
-
Mathias, can you try this in combination with maxing out the max-medium allowed by gasnet? To do this, put
GASNET_CONFIGURE_ARGS=--with-areis-max-medium=64896
in your environment while building upcxx. -
OK thanks, I will give it a try today. I will also try to remove the progress call before the diagonal spmv call.
Best,
Mathias Jacquelin Research Scientist Lawrence Berkeley National Laboratory mjacquelin@lbl.gov 1-510-495-2605
-
reporter Spelling correction:
GASNET_CONFIGURE_ARGS=--with-aries-max-medium=64896
this will allow you to change the
gasnet::am_size_rdzv_cutover
up to anywhere close to that size, although note that maxxing the size near 64k may not be optimal. There is presumably a local maxima for some value of the threshold corresponding to a cross-over where rendezvous begins to be profitable (but probably something higher than the hard-coded 512 byte default). My guess would be in the few KB range, but it depends on the exact overhead of the additional messages. -
reporter -
assigned issue to
-
assigned issue to
-
reporter - changed milestone to 2021.3.0 release
- changed version to Development Branch
I've been working on tuning enhancements to the RPC implementation that will include resolving this issue.
This is not yet a PR, but some preliminary results were shown at the 2020-11-19 Pagoda meeting
CC: @Steven Hofmeyr @Rob Egan
-
reporter Proposed resolution now in PR 301
-
reporter - changed status to resolved
issue
#164: Add UPCXX_RPC_EAGER_THRESHOLD knobThis new knob controls the eager/rendezvous threshold for the RPC protocol.
Resolves issue
#164→ <<cset b13c27b9c471>>
- Log in to comment