g++ fails to inline some functions in future/rput/rget when optimizing

[Nenad Vukicevic]

I use Linux FC23 which has gcc 5.3. The default LDFLAGS for upcxx come out as :

-O3 --param max-inline-insns-single=35000 --param inline-unit-growth=10000 --param large-function-growth=200000 -Winline -Wno-unused -Wno-unused-parameter -Wno-address

With -O3 I get lots of warnings about the inlining. E.g. (few out of many):

/eng/upc/dev/nenad/upcxx/bld/upcxx/include/upcxx/future/core.hpp: In function ‘int main()’:
/eng/upc/dev/nenad/upcxx/bld/upcxx/include/upcxx/future/core.hpp:526:17: warning: inlining failed in call to ‘static void upcxx::detail::future_header_ops_general::decref_header(upcxx::detail::future_header*) [with T = {}]’: call is unlikely and code size would grow [-Winline]
     inline void future_header_ops_general::decref_header(future_header *hdr) {
                 ^
In file included from /eng/upc/dev/nenad/upcxx/bld/upcxx/include/upcxx/future/make_future.hpp:5:0,
                 from /eng/upc/dev/nenad/upcxx/bld/upcxx/include/upcxx/future/apply.hpp:5,
                 from /eng/upc/dev/nenad/upcxx/bld/upcxx/include/upcxx/future.hpp:7,
                 from /eng/upc/dev/nenad/upcxx/bld/upcxx/include/upcxx/backend.hpp:8,
                 from /eng/upc/dev/nenad/upcxx/bld/upcxx/include/upcxx/allocate.hpp:8,
                 from /eng/upc/dev/nenad/upcxx/bld/upcxx/include/upcxx/upcxx.hpp:4,
                 from loc_put.cpp:1:
/eng/upc/dev/nenad/upcxx/bld/upcxx/include/upcxx/future/impl_shref.hpp:132:11: warning: called from here [-Winline]
           HeaderOps::template decref_header<T...>(this->hdr_);
           ^
In file included from /eng/upc/dev/nenad/upcxx/bld/upcxx/include/upcxx/future.hpp:4:0,
                 from /eng/upc/dev/nenad/upcxx/bld/upcxx/include/upcxx/backend.hpp:8,
                 from /eng/upc/dev/nenad/upcxx/bld/upcxx/include/upcxx/allocate.hpp:8,
                 from /eng/upc/dev/nenad/upcxx/bld/upcxx/include/upcxx/upcxx.hpp:4,
                 from loc_put.cpp:1:
/eng/upc/dev/nenad/upcxx/bld/upcxx/include/upcxx/future/core.hpp:526:17: warning: inlining failed in call to ‘static void upcxx::detail::future_header_ops_general::decref_header(upcxx::detail::future_header*) [with T = {}]’: call is unlikely and code size would grow [-Winline]
     inline void future_header_ops_general::decref_header(future_header *hdr) {

[Dan Bonachea]

Dan and @PHHargrove are looking into a good solution for this for a future release.

For now, the recommended workaround is to add -Wno-inline at the end of the C++ command-line in your Makefile to silence the warnings if you encounter this issue.

[Dan Bonachea]

I'm working on a GASNet-level fix to remove -Winline from the compile line.

However, this will only hide the warnings, not change the fact the g++ inlining heuristic is choosing to truncate inlining of some functions in upcxx::future that have been explicitly marked with the inline keyword.

It appears this is frequently (usually?) due to the call to decref_header in the ~future_impl_shref() destructor, which is commented as an "unnecessary check", and g++ apparently decides is "unlikely".

John should probably look at the latter issue, to determine if some of the inlining keywords should be adjusted for this case.

[john bachan]

@bonachea do you have an environment I can use to produce these warnings?

[Dan Bonachea]

@jdbachan : Just reproduced the problem on dirac (pcp-d-5) with develop 7d984b7

$ env CXX='g++ -Winline' nobs exe test/dist_object.cpp

[john bachan]

Removing inline on member functions which were defined in their encompassing class definition fixed most of these. Unfortunately, free functions defined in headers need the inline keyword to function correctly, but the functions in test/util.hpp were beefy enough to throw warnings. There is a workaround, but its a little ugly.

// in header.hpp

// this causes -Winline noise
inline void foo() { /* lots of beef */ }

// this does not, and can be called syntactically
// the same
template<typename=void>
void foo() { /* the beef */ }

I could rewrite all the inline non-template header functions this way to guard against future compiler warnings. But I'm not going to. So for now, just the ones which have been observed to throw warnings.

[john bachan]

Fixed issue 43. Removed unnecessary "inline's" which were causing pedantic -Winline compiler warnings.

→ <<cset 9da3bbedec3a>>