Share gex_MK_t objects between endpoints

Issue #530 resolved
Dan Bonachea created an issue

Currently the UPC++ runtime creates an internal gex_MK_t object for each high-level gpu_device object creation using native GEX kinds. GASNet allows us to share such objects between endpoints with segments bound to the same physical GPU device. UPC++ does not currently exercise that flexibility, but could do so with some extra bookkeeping work at device open time.

Paul informs me there may (eventually?) be some savings in GDR copy overhead from deploying this enhancement that would be relevant for applications that open the same physical device more than once per process; ie limiting the process-wide number of gex_MK_t objects to one per open physical device.

Comments (3)

  1. Dan Bonachea reporter

    issue #530: Share gex_MK_t objects between endpoints

    Ensure the runtime at each process creates at most one GASNet-level memory kind gex_MK_t object for each open physical GPU device, regardless of how many device segments this process has created on the same physical device.

    This helps reduce overhead in GEX-accelerated device RMA for processes that concurrently open the same physical device more than once.

    Resolves issue #530.

    → <<cset 30e90dfcf39e>>

  2. Log in to comment