Add in-place and vector reductions
Very frequently, when one is trying to reduce / allreduce a large chunk of data, it is advantageous to do it "in place". I do not think our API allows to do that.
One thing that could be done, is to use the move syntax to "move" the input value and place the modified content in the future, that way we avoid making an extra copy.
Any take on this ?
Comments (11)
-
-
reporter Yes, reducing a vector is good example. I think it applies for both versions of the API, let's assume I do:
std::vector<double> buffer; .... broadcast( buffer, 0 );
Can I do this in place ? This is a use case that we will HAVE to handle as well I think.
Best,
Mathias Jacquelin Research Scientist Lawrence Berkeley National Laboratory mjacquelin@lbl.gov 1-510-495-2605
-
Yes, we have a vector broadcast interface:
template <typename T> future<> broadcast(T *buffer, std::size_t count, intrank_t sender, team &team = world());
-
- changed component to Teams and Collectives
-
- changed milestone to 2018.03.31 release
- changed title to Add in-place and vector reductions
At the 9-12 meeting we agreed to defer this issue.
-
- changed milestone to 2017.12.31 release
-
- changed milestone to 2018.09.30 release
This issue was discussed in the 1/10/18 meeting and we agreed to defer it to the September release.
-
The title and comments touch on 2 issues. This first is a vector API for reduction, and is the topic of this comment.
As Amir noted in a 20178-08-30 comment, we do have a vector broadcast.
However, there is not currently a vector interface for reduce as Mathias has requested.I believe possible to use a vector as the type and a lambda as the operator to achieve the same interface. However, if vector reductions with the basic types and standard operators are ever to be off-loaded, then an explicit vector form (like Amir proposes in the 1st comment on this issue) will be required.
I would like the issue of the inclusion/exclusion of this API decided soon, because it impact my priorities for GASNet-EX collectives.
-
The title and comments touch on 2 issues. This second is "in-place" support for reduction, and is the topic of this comment.
I think for memory scaling reasons, it is desirable to support the equivalent of
MPI_IN_PLACE
for all collective operations where is it well defined. At the moment, that appears only to impact the vectorallreduce
with the prototype Amir provides in the first comment on this issue (except for the removal ofconst
fromsrc
). However, the "in-place" behavior is potentially relevant for other collectives we may add, such as gather-to-all and exchange (aka all-to-all). -
- marked as critical
This issue was triaged at the 2018-06-13 Pagoda meeting and assigned a new milestone/priority.
We noted that GEX should have support for vector (ie multi-field) reductions in time for the upcoming release, and we should expose at least that feature in UPC++ collectives.
-
- changed status to resolved
Resolved in 2d9680f (pull request #16)
- Log in to comment
Can you clarify what use case you're asking for? Something like vector reductions, or just the ability to pass the input by pointer?
I imagine supporting both with an MPI-like interface:
However, unless this is high priority for one of our apps, I don't think we can promise this in the near term.