Completions / promise API to upcxx::view

[Rob Egan]

In the effort to manage memory consumption and the back-pressure needs during heavy network communications, it occurred to me that having a completion handle on a view would allow the developer to count outstanding views for a given set of rpcs and thereby make runtime decisions about whether to block on progress() or go ahead with initiating a new rpc and view.

So if the upcxx::view API can be augmented to include something like a promise<> or operation_cx::as_promise which is fulfilled when all locally allocated resources by upcxx internal operations are eventually released.

It would also be nice, though not necessary, to include an interface that returns the private and/or share segment bytes which were consumed or possibly even are presently still consumed by the view (or some kind of weak reference view-handle when the original view is out of scope)

I get that the view’s internals that control of the lifetime and resources consumed are purposely opaque, but that prevents sufficient information communicated ast the user-level to make decisions about the resources that are consumed.

[Dan Bonachea]

@Rob Egan I think there may be some confusion about the semantics of upcxx:view<>. A upcxx:view<T,I> object is always a non-owning container: it's effectively just a pair of iterators of type I, which in most cases just means a pair of raw, "dumb" C++ pointers. view objects do not manage the lifetime of any storage, and do not have any insight/control over storage lifetime. This is not only the semantics as defined by the spec, but also the implementation - view objects never control storage lifetime.

I think the confusion likely arises from the lifetime extension exception for deserialized RPC argument storage, which previously only applied to view-typed arguments, but as of 2020.10.0 now applies to all deserialized RPC arguments of any reference type. Here is the current spec wording for rpc_ff, describing the lifetime of deserialized RPC arguments provided to callback function object:

The result from invoking the function object is discarded. However, the return value affects the lifetime of the deserialized objects that the UPC++ runtime constructs and passes to the function object. If the return value is a non-future value or a ready future, the deserialized objects are destructed immediately after the invocation returns. On the other hand, if the return value is a non-ready future, destruction of the deserialized objects is deferred until after the future becomes ready, allowing the function object to safely initiate further asynchronous computation that operates on those objects.

rpc specifies equivalent argument lifetime semantics, with the only difference being the return value is additionally serialized (after readying, if applicable) and sent back to the initiator.

The effect of these semantics is that at an RPC target, runtime-owned storage backing reference-typed RPC arguments (including view) is guaranteed to remain live until the RPC callback returns or readies a returned future (whichever happens last). In other words, although the runtime owns the network buffer storage, the lifetime is determined entirely by the action of the RPC callback. If you'd like a future for when that lifetime ends, simply copy the future returned from the RPC callback. For an RPC callback that doesn't return a future, it can of course declare a promise and fulfill it before returning to produce the desired future tracking of buffer lifetime.

On the initiator side, the lifetime "contract" for any by-reference input arguments passed to RPC is reversed; the implementation callee does not acquire ownership of any storage referenced by input arguments, whether by view or C++ reference. Rather, the caller (generally user code) is responsible for ensuring that any arguments passed by reference (including views) reference storage that remains live and valid until after the source completion event for the injection (which for RPC defaults to happening synchronously before the injection call returns, although explicit completion can be used to request deferred source completion).

Regarding the request for "interface that returns the private and/or share segment bytes which were consumed or possibly even are presently still consumed by the view": This already exists for view<T, T*>, which is the view specialization received by RPC callbacks corresponding to a view over TrivialllySerializable element type. Namely, view<T, T*>::begin() returns a T* which is exactly that pointer, a non-owning pointer to the trivially-serialized elements residing in the runtime-managed incoming network buffer. Analogous functionality is however not provided for the view<T, deserializing_iterator<T>> specialization (which is the view type received by an RPC for a view argument over NON-TrivialllySerializable elements), because the elements do not exist in a persistent deserialized form at the target, and the format of the serialized representation residing in the network buffer is intentionally unspecified.

Hope this helps clarify and answer your questions..

[Rob Egan]

Thanks Dan. This does help and yes I am mixing up the relationships between the view and rpc’s lifetime guarantees of the view.

So, if I understand correctly, assuming the rendezvous is being used by a rpc_ff, I should be able to use a promise-completion in the rpc_ff to count the outstanding calls which are still consuming shared segment for the transmission (ie the rget on the target side has not yet completed and released the source side buffer)

And on that note, I’m assuming that, with the rendezvous protocol, a rpc_ff(target, …) is expecting an acknowledgment message from the target when that target has completed its rget, and that a rpc(target_rank, …) would get both that acknowledgement message and a second one after the lambda has completed executing on the target, so it is still more network-efficient to use a rpc_ff even considering the rendezvous protocol’s full round-trip.

‌

‌

[Dan Bonachea]

Rob said:

assuming the rendezvous is being used by a rpc_ff, I should be able to use a promise-completion in the rpc_ff to count the outstanding calls which are still consuming shared segment for the transmission (ie the rget on the target side has not yet completed and released the source side buffer)

Unfortunately not. The only completion event for rpc_ff is source completion, and that event fires immediately upon completion of serialization from user buffers into the rendezvous buffer (which currently occurs synchronously before the injection call returns, so the completion will generally trigger during next user progress for the injecting persona). This is because source completion indicates the user-provided input arguments to rpc_ff are fully "consumed", which is ensured once serialization is complete. There is no user-visible event corresponding to the rendezvous buffer, because that is a runtime-managed temporary resource holding an opaque byte stream. The best way to get insight into the resource utilization of that buffer is via the shared segment query functions we just added.

Rob said:

rpc(target_rank, …) would get both that (rendezvous) acknowledgement message and a second one after the lambda has completed executing on the target, so it is still more network-efficient to use a rpc_ff even considering the rendezvous protocol’s full round-trip.

Correct - in the remote rendezvous protocol, the rendezvous acknowledgement AM is sent by the target to free the rendezvous buffer as soon as the RMA get is completed. We do not attempt to coalesce that message with rpc return value acknowledgement, which in general will not be ready to send until (possibly much) later.

[Dan Bonachea]

Resolving invalid since the proposal was based on a misunderstood premise.

Feel free to open new issues for additional concerns.