Wiki

UPCXX nightly test results

Verify that GitLab/pushbuild CI is passing on all supported platforms
1. Manually run the following non-standard GitLab experiments:
  1. Single-rank CI_RANKS=1
  2. Any high-risk non-default configure options not covered in GitLab, eg
    1. Previous GASNet release: --with-gasnet=https://bitbucket.org/berkeleylab/gasnet/get/master.tar.gz
    2. --disable-ccs-rpc
    3. --enable-discontig-ranks
  3. dev-check all the "exhaustive" compiler groupings, to maximize compiler version coverage
  4. dev-check all the c++ standard groups for 14, 17, 20, 23
2. Any test failures should have corresponding triaged issues, with workarounds whenever possible
Valgrind check for memory leaks in GPU tests on dirac CX5: (not covered by GitLab)
- configure --enable-valgrind --enable-cuda --enable-hip --with-hip-home=/opt/rocm --with-hip-platform=nvidia
- env UPCXX_VERBOSE=1 UPCXX_OVERSUBSCRIBED=1 OMP_NUM_THREADS=1 TEST_ARGS_PERSONA_EXAMPLE=100 make -j16 dev-check-debug NETWORKS=smp RUN_WRAPPER='timeout -k 15m 10m $(VALGRIND_WRAPPER)' NO_TESTS=neg-
- Any leaks should be triaged and fixed or annotated as known issues. Currently includes:
  1. test-issue478-*: Issue #536
Update UPCXX_VERSION and UPCXX_SPEC_VERSION definitions in src/version.hpp
1. should be of the form: YYYYMMPP
2. Y and M represent numeric year and month of the release
3. PP = patch number, starting at 0 for a "normal" (ie not hotfix) release
4. PP should be zero for UPCXX_SPEC_VERSION
Update version info in utils/upcxx-run:load_args()
Update feature macros to match specified value in spec AND impl:
1. UPCXX_KIND_CUDA
2. UPCXX_KIND_HIP
3. UPCXX_KIND_ZE
Update GASNet-EX spec version floor in src/backend/gasnet/runtime_internal.hpp and corresponding error message
Update ChangeLog
1. Review list of commits looking for any user-facing bug-fixes, improvements or breaking changes not represented
2. Sort issue list by issue number
3. Ensure that document links for older release sections are updated
4. Insert excerpt of recent GASNet-EX improvements impacting UPC++
Update configure default URL to point at EX release tarball. This should look like: https://gasnet.lbl.gov/EX/GASNet-YYYY.M.P.tar.gz
Note the https and host
Commit final guide PDF to docs/guide.pdf:
1. iworld is our canonical pandoc system for guide publication
2. validate tech report number and version number on the cover and PDF metadata are correct
3. guide and spec publication date should NOT be identical, to avoid nondeterministic rendering in publication lists with unstable date sorts, report guide publication date one day earlier if needed.
4. Post HTML version to a private location and run a W3C link check to validate internal linkage (ignore warnings about "data URIs")
5. upload to berkeleylab/upcxx downloads area with name like upcxx-guide-2017.9.0.pdf
6. TAG the guide repo
7. Advance patch number in upcxx-prog-guide/Makefile
Commit final spec PDF to docs/spec.pdf
1. validate tech report version number on the cover is correct
2. validate UPCXX_SPEC_VERSION and the matching version on the cover and PDF metadata
3. upload to berkeleylab/upcxx downloads area with name like upcxx-spec-2017.9.0.pdf
4. TAG the spec repo
5. Advance draft number and append ".dev" suffix on spec master branch in upcxx-spec.tex
Generate source release-candidate tarball (from git working dir)
1. Run the follow command in the git working directory:
  env REPO=origin BRANCH=develop utils/release.sh
2. Confirm the final output matches the expected version numbers.
3. Upload this release candidate to a private download area (e.g. private fork with group access)
Email local group to review the release-candidate
Ask an ADMIN to grant you WRITE access to the 'master' branch
Merge to master (updating version below)
git remote update
git checkout master
git pull --ff-only
git merge --no-ff --no-log -m 'Merge develop for 2017.9.0 release' origin/develop
(push to a private fork on BitBucket or inspect locally with git log)
--no-log ensures we don't get spurious issue tracker updates from issues mentioned in the auto-generated log
git push origin master
Ask an ADMIN to revoke WRITE access to the 'master' branch
TAG master (updating version below)
git tag -a -m 'UPC++ 2017.9.0 release' upcxx-2017.9.0 master
git push origin upcxx-2017.9.0
Generate final tarball - must come AFTER the previous step has pushed to remote
1. Run the follow command in the git working directory
  env REPO=origin BRANCH=master utils/release.sh
  Confirm the final output matches the expected version numbers.
2. Validate the git hash matches the tip of master holding the release tag.
3. Collect the md5sum for the generated tarball (at end of the output).
4. Upload the final tarball to (public) berkeleylab/upcxx downloads area.
Download the tarball just uploaded into a different directory.
1. Run md5sum on the download and verify it matches the checksum from the previous step.
2. Extract the packaged git hash:
  gzip -cd upcxx-2017.9.0.tar.gz | git get-tar-commit-id
3. Validate the git hash matches the tip of master holding the release tag.
4. Unpack the tarball and inspect the contents of README.md and src/version.hpp for correct versioning.
Advance patch version number on develop branch in src/version.hpp
Change configure URL back to https://gasnet-bugs.lbl.gov/nightly/unlisted/GASNet-stable.tar.gz on develop
Tag the develop branch to match the new content of src/version.hpp
git tag -a -m 'tagging YYYY.M.P for development' upcxx-YYYY.M.P develop
Push the previous several steps (substituting the correct version).
git push origin develop upcxx-YYYY.M.P
Tag the upcxx-extras repo to record the release point: git tag -a -m 'tagging YYYY.M.P for development' upcxx-YYYY.M.P develop
git push origin upcxx-YYYY.M.P
1. Consider also merging upcxx-extras develop branch to master, and tagging as upcxx-extras-YYYY.MM.DD
Review and revise the release announcement
Navigate to the wiki and git clone it to your local system
Add any new publications (eg updated spec or guide) to
wiki/Publications.md
wiki/pagoda.bib
wiki/pubs/*.pdf
~~~Update the GASNet-EX version table in the wiki~~~
Update all 3 issue trackers: add this release to Versions, next one to Milestones and as default Milestone
Update NEWS and Download sections of wiki/Home.md with info on the new release download including links, moving old links to a new line in the Previous Releases section
Run the script wiki/internal/import-release.sh to import the documentation from the release, overwriting the corresponding files in the working copy.
1. git add any new documents and remove any obsolete ones
2. Audit the changes before committing to look for information loss.
Push your wiki changes and audit everything on the live page, especially checking for broken links.
1. Upload new guide HTML to upcxx.lbl.gov:docs/html
  1. update the guide.html symlink
  2. add new file to CVS
  3. cvs up this directory and upcxx directory on the backup webservers
2. Ask Dan to run deadlinkchecker scan on the updated site
3. Run an W3C link check on the guide to validate internal linkage (ignore warnings about "data URIs")
4. Run an W3C link check on the FAQ to validate anchor linkage to the guide
5. Run an W3C link check on INSTALL to validate anchor linkage
Add md5sum of each download (code, guide, spec) to announcement
Send the announcement To: upcxx@googlegroups.com, upcxx-announce@lbl.gov Bcc: pagoda@lbl.gov Reply-to: pagoda@lbl.gov
Ask Dan or Paul to update the SourceForge download mirror
Bug Paul to install on NERSC, Summit, Dirac, Kotten, etc. ...
Email Todd Gamblin and Sameer Shende to request the release cutoff dates for the next Spack + E4S releases
Update this list

Post-release management goop

Generate short URL for the Google Groups announcement
1. Visit the upcxx-announce googlegroup page
2. Click on the announcement to view it
3. Click the to the right of the message date (says "More message actions" when you roll over)
4. Select " Link"
5. Copy to your clipboard (Ctrl-C or command-C, as appropriate)
6. Generate a go.lbl.gov/upcxx-YYYY_M_P short URL from this long hairy URL
Attach PDF (not .pptx) of Milestone Highlight Slide as an attachment to Milestone Reports page in Confluence
Update/add section to Milestone Reports page with links to
1. JIRA entry in the heading
2. Slide attachment (uploaded a few steps earlier)
3. Google Groups announcement go.lbl.gov link
Perform a second upload of the following to JIRA Epic as attachment(s):
1. Milestone Highlight Slide PDF
2. Any supporting evidence described in the Execution Plan that is not included inline in the drafted closure text
Finish the JIRA Epic by hitting Workflow > Done
1. Paste in the drafted Milestone closure text, which should include the Google Groups announcement bit.ly link
Add links to the Confluence Quarterly Report page below "Accomplishments / Milestones Delivered During This Period". Past quarters show examples.
1. Using Create Link and Search will let you find the slide PDF uploaded earlier
Update NEWS on top-level Pagoda Confluence page