bench/put_flood can't be built by end users

[Dan Bonachea]

Following the documentation for our microbenchmarks in bench/README.md:

Build & Run: From UPC++ installation

Can be built like any other upcxx application, ...
Build. Default allocator = std.
<upcxx-install>/bin/upcxx my-bench.cpp -o my-bench

These instructions don't work for the critical bench/put_flood.cpp microbenchmark:

{cori-knl ~/UPC/upcxx/bench} upcxx --version
UPC++ version 20180900 
Copyright (c) 2018, The Regents of the University of California,
through Lawrence Berkeley National Laboratory.
http://upcxx.lbl.gov

icpc (ICC) 18.0.1 20171018
Copyright (C) 1985-2017 Intel Corporation.  All rights reserved.

{cori-knl ~/UPC/upcxx/bench} upcxx put_flood.cpp -o put_flood
upcxx: error: please specify exactly one of -O or -g, otherwise set UPCXX_CODEMODE={O3,debug}

{cori-knl ~/UPC/upcxx/bench} upcxx -O put_flood.cpp -o put_flood
put_flood.cpp(52): catastrophic error: cannot open source file "upcxx/backend/gasnet/runtime_internal.hpp"
  #include <upcxx/backend/gasnet/runtime_internal.hpp>
                                                      ^

compilation aborted for put_flood.cpp (code 4)

The first problem (missing -O) is a simple doc fix.

The second problem is more serious, because the header required by the test is never installed. I'm not sure what the right fix is here, but we really want this to be buildable by end users, ideally using UPC++ modules installed by admins.

We use bench/put_flood in publication to benchmark our system, so the build process really needs to be documented such that it's reproducible by end users.

[john bachan]

Confirmed. I'm not sure how long building outside nobs has been broken, but the Sep18 release made it even worse by changing runtime internals and not updating this benchmark's code to follow. Fix is forthcoming, but unfortunately this benchmark is totally DoA as of this release.

[john bachan]

Fixes issue 183. "bench/put_flood.cpp" was failing to compile due to:

1. Didn't have access to <upcxx/backend/gasnet/runtime_internal.hpp> when building outside of nobs.

2. The Sep18 runtime refactoring broke the code of this benchmark without updating it.

This commit adds a new user accessible (hence installed) header: <upcxx/upcxx_internal.hpp> By including this the user can get at more of our internals, notably gasnet.

→ <<cset 54cec2cc20f3>>

[Dan Bonachea]

bench/nebr_exchange was similarly broken.

Fixed in 0246c1e

[Paul Hargrove]

Resume installing internal headers (see issue #183)

→ <<cset d457209fbdd7>>

[Paul Hargrove]

Merge pull request #134 into develop

• gex-incr: make: implement "gasnet" pass-through target make: sub-divide upcxx-level install steps Update 'wrong GASNet' error for new infrastructure test/README.md: mark as out-of-date/unmaintained upcxx-meta: rename GASNET_INSTALL to GASNET_PREFIX Remove util/install-single.sh and support tests: make hello_upcxx.cpp a suitable smoke test make test_install: wider coverage make: rename UPCXX_TESTS_{VERBOSE,CODEMODE} make: document some internal-use make variables make: check target now honors NETWORKS make check/tests: uniform output treatment fix echovar errors on second make with a broken GEX configure configure: shell checks configure: preserve input and output to config.log make: preserve GASNet-EX configure output configure: record configure command in Makefile configure: quoting of GASNet config args Reduce difference between compile and dep-gen make exe/run: fix dependency tracking with PGI Centralize defn of dependency-generation flags make exe/run: allow .c files make exe/run: include MANUAL_* in hash docs: minor copy-editing make tests: implement UNOFFICIAL_CONDUITS make check/tests: cleanup UPCXX_TESTS defaulting make check/tests: implement UPCXX_TESTS_VERBOSE configure: implement configure --help=recursive configure: fetch/verify of GASNet-EX as first step make check/tests: parallel compiles make check/tests: reduce prerequisite builds make check: suppress redundant library builds Order debug targets ahead of opt Error output on tool's failure to build runtime Implement and document make exe-clean-all Add upcxx-meta to $(top_blddir)/bin Update build-devel.md for the new infrastructure ChangeLog: document the infrastructure changes Remove docs/testing.md and xfef in README.md Fix "TBD" cross-reference in make install output INSTALL.md updates for new infrastructure Simplify user messages regarding make Update .gitignore Remove out-of-date WIP comment REdo "quiet" option to in-build-tree upcxx and upcxx-run Create factored logic for configure warnings Implement tests targets Move GASNet-related macros to gasnet.mak Move lists of tests to it own tests.mak Factor logic for GASNet variable extraction Add -q/--quiet option to in-build-tree upcxx and upcxx-run Honor CONFIG_{CC,CXX} in --enable-single mode. Repair install-hook for --enable-single=debug Improved configure arg parsing Update utils/release.sh w/ new GASNet URL location Add success/next-steps output to all,install,check targets Implement "--enable-single={opt,dbg}" support New targets: {all,install,clean}-{opt,debug} (for CI and maintainers) xSDK-compliant --with(out)-cuda Reintroduce install, w/o --single and w/ a legacy warning Replace some non-POSIX grep options Add explanatory comments requested in PR review Preserve user's -std=gnu++NN same as -std=c++NN More "pilot error" paranoia Protect against a potential "pilot error" Strip "LIB" from LIBUPCXX_{DEFINES,STDCXX} Resolve issue #286 via LIBUPCXX_CXXFLAGS Move libupcxx_sources to a distinct .mak for ease of maintenance Centralize compiler-specific logic to a single .mak file Fix dependency tracking w/ PGI Add CFLAGS to bottom-level upcxx-meta Fix a couple of typos Add --enable-upcxx-cuda and friends Scary warning for --single mode Minimize differences between GASNET= source vs build Remove extra whitespace to favor machine parsing over pretty-for-people Remove top-level run-tests Add utils/install-single.sh and remove install Update copyright date Add --with-default-network=... Fix error seen w/ some shells Cosmetic tweak to configure final output Add $(top_builddir)/bin/{upcxx,upcxx-run} upcxx.sh: new "BUILDDIR" mode restore (improved) --with-gmake=... logic upcxx-run target only needs GASNet, not libupcxx whitespace change to satisfy vim syntax highlighing Remove GMAKE and --with-gmake. Only warn if min version not found. Expose more parallelism, including concurrent GASNet configures Resolve issue with make -j2 all install running concurrent configures Install basic environment modulefile Provide support instructions when 'all', 'check' or 'install' targets fail More helpful configure usage info Add "bench" to places in which to search for SRC Give up on SRC='multiple.cpp files.cpp' in exe and run targets Improve exe and run targets Add useful info at end of configure help output Honor $(DESTDIR) at install time Ensure correct operation of 'make -d' and 'make --debug=...' Ensure use of full path to "bld". Remove --with-gasnet-configure-args option from configure Remove "--" handing in args parsing Move UPCXX_MPSC_QUEUE handling to undocumented section in args parsing Improve GASNet tarball download Ensure explict GASNET is honored over offline tarball or online URL Remove all handling of UPCXX_PYTHON Move Makefile.rules into $(top_srcdir)/bld/ Echo compile step in exe target Remove --with-mpsc-queue from configure's usage message Update INSTALL.md with respect to GMake requirement Add UPCXX_MPSC_QUEUE logic to configure script Correct defaulting behavior of $UPCXX_PYTHON Less aggressive removal of GASNet config.cache Force rebuild upon reconfigure or edits to Makefiles Enforce (sort of) GNU Make 3.80+ Fix a (so far) harmless typo sed based canonicalization of $prefix Implement full dependency tracking for exe and run targets Apply proper suffix to executbales Correct/complete EXTRAFLAGS for exe and run targets Ensure developer builds of libupcxx.a refresh runtime.o watermark on every required rebuild Ensure runtime.o in developer builds embeds the git-describe output Add a mechanism for per-target compiler flags Initial upcxx make target Improve maintainability of build-tree upcxx-run script GREATLY simplify maintenance of upcxx-meta content Implement 'upcxx-run' and 'upcxx-meta' targets Remove debug output not meant for commit cherry-pick upcxx-meta change from commit 48e9362 Keep just a single list of tests Remove unused generality Remove unnecessary in-build-tree upcxx-run Implement a rudimentary "make check" Add missing .PHONY for exe and run respond to PR comments Implement nobs-less run-tests script Implement make run SRC=... [ARGS=...] [RANKS=...] remove spawner and launch targets (keep upcxx-run) Suppress harmless warnings Implement "make (launcher|spawner)" Reduce gratuitous output from recursive makes Factor common "gasnet.$(GASNET_CODEMODE)" Initial implementation of 'make exe SRC=[path]' Simplify the most common usage of UPCXXDIR Work-around GNU Make 3.79->3.80 changed behavior Avoid a harmless warning from older GNU Make automate -std=c++14 for Intel Factor the idiom for using 'echovar' from GASNet Add an important TODO VPATH is computable Suppress excessive output Factor some repeated iteration lists Expose more parallism in libupcxx builds Restore parallel make (fixes depfile generation) Disable parallel builds (for now) due to corrupted dependency files. More factorization of install-common.sh Provide an initial configure script Resume installing internal headers (see issue #183) Comment in Makefile to avoid breaking things later Fix expr: syntax error on macOS Implement GASNET=BLDDIR install --single ... Permit embedding of GASNet-stable.tar.gz Add GASNet-EX version checks Revise version interlock to be parsable Update release.sh to get GEX version from install Default to GASNet-stable URL Better error checking for fetch/unpack of $GASNET Add install --single support Provide new installer which operates w/o nobs Factor reusable installer logic Use dependency info to guide header installation Add dependency tracking for libupcxx cleanup to libupcxx build Add generation of upcxx-meta Complete libupcxx build logic Add framework for building libupcxx.a Add rule to build libgasnet by its full path Initial Makefile for nobs-like conduit builds

→ <<cset c314f74e6b44>>