thread friendly wait option i.e. NOT busy-wait

[Rob Egan]

In MHM2, we have some code that spawns off thread-safe packages of work into a thread pool and then use a promise and a LPC to relay the results back to the calling persona as a future (almost always master for now). This allows the master persona to be attentive to the network, call progress() often and do other tasks like package work for offloading to the GPU.

At the end, we call future::wait() to assure completion of the offloaded work, but these are busy spin waits and could compete with the code executing in the thread pool.

I’m now replacing this code with the following idiom instead of simply using upcxx::future::wait()

while(!fut.ready()) {
  std::this_thread::yield();
  progress();
}
fut.wait();

… but it would be great if future::wait() had the option to yield to other threads with a boolean argument like:

void future::wait(bool spin_wait = true);

Thoughts?

[Dan Bonachea]

Hi Rob - Thanks for the thoughtful suggestion!

You can of course implement a wrapper around future::wait() to do whatever you want, even without changes to UPC++, eg:

template<typename T...>
inline auto my_wait(upcxx::future<T...> fut) {
  while(!fut.ready()) {
    std::this_thread::yield();
    upcxx::progress();
  }
  return fut.wait();
}

and then call my_wait(f) instead of f.wait().

Unfortunately I don't think the proposed code will accomplish your stated goal of reducing busy-wait contention for execution resources on systems you care about. Specifically, my understanding is that recent versions of the Linux CFS scheduler which appear in most modern distros implement the sched_yield() system call as effectively a no-op for the default user-level scheduling policy. IOW instead of relinquishing the scheduling time-slice to some other thread as implied/intended by POSIX, the CFS scheduler always immediately returns control to the calling thread unless its time-slice has expired.

Steve confirmed this unfortunate scheduler behavior affects execution on cori in a series of laborious experiments a few years ago, and here's a good blog post on this topic. This frustrating scheduling policy makes sched_yield() and wrappers around it effectively useless on Linux, leaving no mechanism for unpriviledged processes on Linux to voluntarily relinquish their time-slice without blocking on a kernel object. UPC++'s UPCXX_OVERSUBSCRIBED feature makes sched_yield() calls inside progress() when cores are oversubscribed, but as we document this doesn't actually do much when running as a standard process on Linux.

All that being said, have a look at spec issue 153, which proposes a different mechanism to allow idle threads (without the master persona) to truly sleep and fully release compute resources by blocking on a kernel object. I'm not sure if that proposal would meet your use case, but it's something we could discuss further if it sounds promising.