Test code, based on issue478.cpp:
#include "../util.hpp"
using namespace upcxx;
int main(int argc, char **argv) {
upcxx::init();
print_test_header();
future<> ready_empty = make_future();
future<int> ready_nonempty = make_future(3);
int N = 1;
if (argc > 1) N = std::atoi(argv[1]);
for (int i=0; i<N; i++) {
promise<> prom1;
future<> nonready_empty = prom1.get_future();
promise<int> prom2;
future<int> nonready_nonempty = prom2.get_future();
promise<int,int> prom3;
future<int,int> nonready_nonempty2 = prom3.get_future();
#if !SKIP_WHENALL
// nonready + nonready
future<> f1 = when_all(nonready_empty, nonready_empty);
future<int> f2 = when_all(nonready_empty, nonready_nonempty);
future<int> f3 = when_all(nonready_nonempty, nonready_empty);
future<int,int> f4 = when_all(nonready_nonempty, nonready_nonempty);
future<int,int,int> f5 = when_all(nonready_nonempty, nonready_nonempty2);
// two ready, one nonready
future<int> f6 = when_all(ready_empty, ready_empty, nonready_nonempty);
future<int> f7 = when_all(nonready_empty, ready_empty, ready_nonempty);
future<int> f8 = when_all(ready_empty, ready_nonempty, nonready_empty);
future<int> f9 = when_all(ready_nonempty, nonready_empty, ready_empty);
#endif
}
print_test_success();
upcxx::finalize();
}
The program above, run with develop, leaks about 1KiB of memory every loop iteration, as reported by valgrind 3.17.0. The leak only occurs for runs executing when_all
, and appears to correspond to the underlying promise
objects. Even without valgrind, one can see the problem by running the program above with a command-line argument like ./a.out 1000000
and the leak is obvious via ps/top.
when_all
was recently optimized in pull request 345 (which also added the test that initially detected this leak). However based on manual testing it appears this leak was introduced somewhere between 2020.3.2 and 2020.3.8, most likely during pull request 241.
My guess is we're missing a dropref()
somewhere.
Further investigation reveals that 1) this only affects promises whose dependency count never reaches zero, and 2) this is not limited to
when_all()
.More specifically, the following does not leak memory:
I’ve verified that putting in
fulfill_anonymous()
/fulfill_result()
calls also fixes the leak in the original example.The following that uses
then()
rather thanwhen_all()
leaks memory:Again, adding
p1.fulfill_anonymous(1)
fixes the leak.My guess is that the following change to
core.hpp
in commit 27b7692 is what results in the problem:I suspect that this extra reference in a non-ready
future_header_dependent
keeps the cell on which it depends alive, even when there are no longer any user-level references to it. However, I have not yet confirmed this, nor do I have a fix yet.I’ll keep digging.