Add CarpetX thorns to Einstein Toolkit manifest

[Erik Schnetter]

https://bitbucket.org/einsteintoolkit/manifest/pull-requests/18

[Roland Haas]

This would currently “do harm” and prevent the thorn list from compiling for (at least) two reasons:

‌

• CarpetX and AMReX require C++17 which is not enabled by any (?) of the option lists that ship with Simfactory in the ET
• the thornlist lacks that ExternalLibraries required to compile CarpetX

‌

[Samuel Cupp]

As I said on the PR, this also has the issue that PoissonX inherits from PDESolvers, which is disabled. They probably should both be disabled or both enabled.

[Erik Schnetter]

I corrected the missing ExternalLibraries and disabled the PoissonX thorn.

I am looking for guidance regarding C++17. I can see several ways forward:

• Update currently supported machines, possibly removing some old machines from the list
• Disabling CarpetX by default everywhere
• Explicitly disabling CarpetX on those machines that don’t support C++17 (possibly many machines) and updating others

Sam, do you have a list of machines that you consider “high priority” for the release?

-erik

‌

[Roland Haas]

Since we only really support gcc or (on some systems) clang I ran a quick test to see just how involved this might be:

.../mdb/optionlists$ git grep -l CC.*gcc *
db-sing-cpu.cfg
db-sing-nv.cfg
debian-cuda.cfg
expanse-gnu.cfg
generic.cfg
graham-gpu.cfg
graham.cfg
orca-gcc.cfg
raspbian.cfg
summit.cfg
thornyflat.cfg
wheeler.cfg

which shows that almost none of the big US production clusters (except expanse and summit) use gcc and none clang.

db-sing-nv and db-sing-cpu are are both special for deepbayou and singularity and CarpetX so they are ok.

So we would be disabling everywhere and enable only on db1, expanse and summit. We cannot easily enable on generic since that would push the minimum required gcc on workstations where we do not know what is present (one hopes that everybody runs a gcc new enough, though gcc-11 might be too new for some setups).

I could not find any cluster officially using clang (an only theta uses Cray’s cc wrapper but uses the intel compiler).

[Samuel Cupp]

Sorry for not responding to this promptly. I do not have a complete list of machines that people are currently using. However, I know for sure people are using Deep Bayou, the INL machines, and Falcon. It looks like Falcon has gcc 12, which I believe has all of the '17 standard. Can you confirm what version of gcc is the minimum for CarpetX currently?

I don’t know what machines the RIT group is using, but it sounded like several of their machines probably don’t have access to C++17 (or anything beyond like 2011…).

I agree with Roland that it would likely be untenable to require C++17 by default to compile the toolkit.

[Samuel Cupp]

I logged into the INL machines. The most recent versions listed by module are

• sawtooth: gcc/10.4.0
• lemhi: gcc/11.2.0

‌

[Samuel Cupp]

Based on the call, the consensus seems to be to include CarpetX but have it disabled by default. We can, of course, change this to enabled in a future release.

[Roland Haas]

We have a list of tested compilers on https://github.com/eschnett/CarpetX/wiki/CompilerCompatibility so we could for now add the thornlist to master to see what (not if, we know something will) breaks and then get started fixing things. gcc-9 is the earliest gcc compiler that was found to work which is newer than the current minimum of gcc-6 (for C++11), so we may have to scout out which OS / distros / package manager provide older gcc by default (or disable-thorns the CarpetX thorns in generic.ini which is opaque to users usually).

[Roland Haas]

Applied as git hash 5313eeb "einsteintoolkit.th: add CarpetX thorns" of manifest