Regression: `completion_cx::as_future()` typically leaks

[Benjamin Driscoll]

Most futures returned by the runtime incur a small memory leak every time. Notable offenders are comm operations that complete via as_future() (the default) and dist_object::when_here(). Not affected are the various user-accessible future constructors like make_future, when_all, future::then, and promise::get_future.

Original report by Benjamin Driscoll:

Valgrind seems to report a memory leak of 56 bytes for each call to upcxx::broadcast that returns a upcxx::future. I can trigger this behavior with code as simple as the following:

#include <upcxx/upcxx.hpp>

int main() {
  upcxx::init();

  int i = 0;
  upcxx::broadcast(&i, 1, 0).wait();

  upcxx::finalize();
  return 0;
}

Perhaps this is the expected behavior and I fail to understand something?

[john bachan]

Yikes. I have confirmed this to be a real leak with a quick fix. I will follow up via a PR

[john bachan]

Fix pending in PR: https://bitbucket.org/berkeleylab/upcxx/pull-requests/215/completion-as_future-leaks-every-time

[Dan Bonachea]

Based on behavioral testing I don't see the leak in the 2019.9.0 or 2019.3.2 releases, so I think this defect might have been first released in 2020.3.0 (making it a recent regression).

Hoping John can confirm that with code inspection.

[Dan Bonachea]

The 2020.3.0 installs deployed yesterday on cori now include the hotfix patch for this issue from pull request #215, and I've confirmed they fix the leak I reproduced there in the nightly build.

[john bachan]

Bugfix issue 369. <completion>::as_future() was typically leaking a small 56 byte heap struct per completion notification due to a refcount bungling in detail::persona_tls::fulfill_during_user_of_active().

→ <<cset c676794c6afb>>

[Dan Bonachea]

Bugfix issue 369. <completion>::as_future() was typically leaking a small 56 byte heap struct per completion notification due to a refcount bungling in detail::persona_tls::fulfill_during_user_of_active().

(cherry picked from commit c676794c6afb4818334c76d3c3868bca0326e4b9)

→ <<cset cc789647b6f6>>

[Dan Bonachea]

Merging release upcxx-2020.3.2

• upcxx-2020.3: ChangeLog: update release date 2020.3.2 package version bumps Make future and serialization tests "full" upc++ ChangeLog cleanups Update ChangeLog
• Added test/regression/issue380.cpp - Fixed issue 380 where rput with source+operation cx failed to compile. Fix issue 368. Needed to add a default constructor for bcast_payload_header in runtime.hpp since it has a union containing a non-trivially constructible member (std::atomic). Add a Serialization example with an abstract base class ChangeLog.md bugfix entry Bugfix issue 369. <completion>::as_future() was typically leaking a small 56 byte heap struct per completion notification due to a refcount bungling in detail::persona_tls::fulfill_during_user_of_active(). configure: search PATH for python3 and python2 configure: implement check_tool_path() Update ChangeLog Add issue371 test Fix issue 371: team_id's are not "universal" as documented Update ChangeLog Add issue343 test fix issue 343: ensure that default-construced team_id provide a unique invalid team_id Update ChangeLog Improve bad_shared_alloc to output shared heap status Add a test to demonstrate shared heap allocation failures Tweak shared heap allocation exceptions test/memberof.cpp: silence a harmless warning with -std=c++2a docs: revise Intel/libstdc++ recommendation bench/misc_perf: avoid C++20 deprecation warnings INSTALL.md: Clarify recommendations for using parallel make ChangeLog.md: list fix #356 Fully qualified std::foo to ::std::foo in UPCXX_SERIALIZED_FOO macros. For members injected into user classes by UPCXX_SERIALIZED_* macros: This fixes issue 356 where non-public constructors were inaccessible to UCPXX_SERIALIZED_XXX macros. The fix entails smuggling a public static function into the class that then calls the constructor. Add issue356 regression test Update ChangeLog ChangeLog: document CROSS to UPCXX_CROSS rename configure: automatically cross-compile on Cray XC Start a ChangeLog for hotfix release

→ <<cset e35f012d9670>>