team_id's are not "universal" as documented

The spec defines team_id as "A universal name representing a team" which is EqualityComparable (and TriviallyCopyable). This in turn implies that team_id comparison should always be sufficient to determine whether any two teams anywhere in the job are in fact the same team.

Unfortunately (as discovered by CS267 students @Xinyi Qian and @Raehyun Kim ) the 2020.3.0 implementation does not appear to provide the documented universality.

This attached example demonstrates that team_id is not guaranteed to be a globally unique name for a particular team in the current implementation. Example output for 4 processes with 2 local_team()s: (on dirac/gcc10/udp/debug)

0: id_t1={3541522223009875147,6372108535818588829}      id_t2={426333498038458042,542583166138072248}   id_t3={4985218028745241708,3356046829632903843}    id_t4={16255499421184154785,6415631985977346961}        id_world={1229782938247303441,1229782938247303441}      id_local={2459565876494606882,2459565876494606882}
3: id_t1={3541522223009875147,6372108535818588829}      id_t2={426333498038458042,542583166138072248}   id_t3={4985218028745241708,3356046829632903843}    id_t4={16255499421184154785,6415631985977346961}        id_world={1229782938247303441,1229782938247303441}      id_local={2459565876494606882,2459565876494606882}
2: id_t1={3541522223009875147,6372108535818588829}      id_t2={426333498038458042,542583166138072248}   id_t3={4985218028745241708,3356046829632903843}    id_t4={16255499421184154785,6415631985977346961}        id_world={1229782938247303441,1229782938247303441}      id_local={2459565876494606882,2459565876494606882}
1: id_t1={3541522223009875147,6372108535818588829}      id_t2={426333498038458042,542583166138072248}   id_t3={4985218028745241708,3356046829632903843}    id_t4={16255499421184154785,6415631985977346961}        id_world={1229782938247303441,1229782938247303441}      id_local={2459565876494606882,2459565876494606882}
ERROR: t1.id() values are not unique across ranks!
ERROR: t2.id() values are not unique across ranks!
ERROR: t1.id() values are not unique across ranks!
ERROR: t2.id() values are not unique across ranks!
ERROR: local_team().id() for ranks 0 and 2 should be different!
ERROR: t4.id() for ranks 0 and 2 should be different!
ERROR: t1.id() values are not unique across ranks!
ERROR: t2.id() values are not unique across ranks!
ERROR: local_team().id() for ranks 0 and 3 should be different!
ERROR: t4.id() for ranks 0 and 3 should be different!
done.

In the current implementation, team_id values only uniquely name a team with respect to a given process that is ALSO a member of the team being identified. As one example, local_team().id() is always equal across all processes, even when there are multiple disjoint local_team's. If team_id was in fact a "universal name", one would expect the team_id's of disjoint local_team()s to compare as unequal.

The weaker property currently provided is sufficient to implement team_id::here() and team_id::when_here() (which both have a precondition requiring the calling process to be a member of the referenced team), but is too weak to guarantee correct EqualityComparison via team_id::operator==(team_id) when the calling process is not a member of one of teams being identified. Unfortunately, the latter is a useful idiom when team_id's are stored in the global address space, and one users expect to work as documented (as evidenced by the experience of CS267 students).

We should investigate repairing this defect to provide the documented property.

Comments (5)