- changed component to Storage Management
Provide contrib code that implements the C++ Allocator concept for the shared heap
Implementing the Allocator concept would allow STL and other C++ code to allocate memory in the registered segment.
Comments (12)
-
-
reporter So this should be as simple as the following:
template<typename T> struct allocator { using value_type = T; allocator() {} template<typename U> allocator(const allocator<U> &other) {} [[nodiscard]] T* allocate(std::size_t n) const { if (T* result = upcxx::allocate<T>(n).local()) { return result; } throw std::bad_alloc(); } void deallocate(T* ptr, std::size_t n) const { upcxx::deallocate(ptr); } }; template<typename T, typename U> bool operator==(const allocator<T>& lhs, const allocator<U>& rhs) { return true; } template<typename T, typename U> bool operator!=(const allocator<T>& lhs, const allocator<U>& rhs) { return false; }
-
this would be done in namespace upcxx? so all our allocators evaluate as equal and type conversion is trivial?
-
reporter Yes. This is compliant with the Allocator concept and essentially the same as
std::allocator
. -
This has worked out really nice for me in my preliminary testing
struct particle_t { float x, y, z, vx, vy, vz, charge; uint64_t id; }; typedef std::vector<particle_t,upcxx::allocator<particle_t>> uvector;
A few gotchas for me were having any
uvector
objects that get destructed afterfinalize()
, but that is precisely what we expect to happen. -
Brian encountered a problem when using a statically-scoped vector with this custom shared allocator, because destruction happens after
upcxx::finalize()
and causes an exit-time crash inupcxx::deallocate
If we ever deploy the custom shared allocator as part of upcxx it should probably keep a library "epoch" counter in each object and skip deallocation of objects for expired epochs. The global epoch counter would advance each time an "outermost"
upcxx::finalize()
tears down the library and shared segment. -
Dan, just so I understand clearly, we are not spilling any memory because the shared segment has been teared down by the
upcxx::finalize()
call. Is this correct ?Best,
Mathias Jacquelin Research Scientist Lawrence Berkeley National Laboratory mjacquelin@lbl.gov 1-510-495-2605
-
reporter I think we fundamentally cannot support statically-scoped containers that use this allocator. In the case of non-TriviallyDestructible element type, the container will invoke the destructor of each element, resulting in UB if the memory has been released. While this doesn't effect element types such as
double
s, I'm not sure we should be making it easier to do something that is a conceptual error even if we can make it work in practice. -
It occurs to me that there may also me a use for constructing C++ objects in GPU memory. So, the interaction with Memory Kinds should be considered.
-
- changed milestone to 2018.09.30 release
This issue was triaged at the 2018-06-13 Pagoda meeting and assigned a new milestone/priority.
It was noted that satisfying this use case doesn't necessarily require any spec changes at all, as the required glue could be provided as an external "add-on" utility header that doesn't touch any upcxx internals.
-
- changed milestone to Deferred indefinitely
- marked as minor
- changed title to Provide contrib code that implements the C++ Allocator concept for the shared heap
The consensus seems to be that this is not a spec issue at all, nor does it require access to UPC++ internals. This is a task that someone should provide an appropriate header in a contrib dir to encapsulate Amir's code above for ease of re-use.
Removing from milestone, but leaving it open until someone does this.
-
- changed status to resolved
This has been added to upcxx-extras, a forthcoming optional add-on package for UPC++.
- Log in to comment