Proposal - add internal_cx completion hooks to rpc and rpc_ff

[Rob Egan]

An alternate proposal to Issue #448, now that I understand more of the underlying mechanisms.

Reminder that the motivation is to give the user program more information to handle its own need for back pressure on injection of work that may consume (an opaque amount of) resources at runtime, which need to be managed at several levels.

Dan clarified that “There is no user-visible event corresponding to the rendezvous buffer, because that is a runtime-managed temporary resource holding an opaque byte stream.”

So I’d like a discussion on a proposal to add this completion event of this internal buffer consumption as a user visible completion, possibly even a new struct internal_cx, to at least the rpc and rpc_ff functions. This would allow the developer to understand/count/quantify the runtime utilization of the internal resources consumed by user space calls while keeping the internal upcxx code opaque and subject to trivial completion, or other changes and optimizations.

So can we discuss adding internal_cx::as_buffered or internal_cx::as_future or internal_cx::as_promise to the completions available to request in rpc and rpc_ff?

[Dan Bonachea]

This proposal was discussed in the 2021-02-24 meeting which included the UPC++ developers and @Rob Egan from ExaBiome.

The general consensus was that a proposal along these lines exposes too many internal details of the current implementation that exist below the semantic level of the library API and are subject to change. Implementing the proposed event would be expected to add overheads to the RPC critical path (to track additional state info that is not explicitly tracked in the current implementation), and would at best represent only initiator-side resources and provide no information about the target-side resources (the other half of the resource exhaustion puzzle). Information about target-side resources could not be obtained for rpc_ff without additional communication that would essentially mimic the RPC acknowledgement path for operation_cx of round-trip RPC which is already directly available via upcxx::rpc(operation_cx)).

We are currently leaning towards the approach described in spec issue 176, namely provide variants of rpc and rpc_ff that opt-in to throwing an exception on the initiator when initiator-side resources are scarce, allowing applications to exert backpressure and throttle their injection to avoid the (currently fatal) initiator-side exhaustion. As noted in that issue this approach still doesn't address target-side resources, but that makes it "bug compatible" with this proposal and @Rob Egan notably agreed these proposals effectively solve the same problem in different ways. However the spec issue 176 proposal exposes fewer internal details, uses a coarser-grained opt-in mechanism that is easier to separate from the existing calls, and is expected to incur negligible overhead for the non-exceptional critical path, and thus is generally preferred.

[Dan Bonachea]

Proposal declined in favor of spec issue 176