include SphericalHarmonicReconASCII from incoming in PITTNULLCode

[Roland Haas]

Christian Reisswig provided a copy of SphericalHarmonicReconASCII and alternative thorn that reads in CCE boundary data (similar SphericalHarmonicRecon) but supporting a wider variety of input file formats (HDF5 among them, irrespective of the name).

Used by SpEC and Llama.

There are no docs or test cases as of now. Test data would be welcome (SpEC, Llama or Cactus provided, I don't think it makes a difference).

Keyword:

[Erik Schnetter]

Where are these thorns?

[Roland Haas]

PITTNull is part of the ET already:

https://svn.einsteintoolkit.org/cactus/PITTNullCode

SphericalHarmonicReconASCII is incoming/PITTNullCode:

https://svn.einsteintoolkit.org/incoming/PITTNullCode/SphericalHarmonicReconASCII/

[Erik Schnetter]

This thorn lacks documentation, examples, and test cases.

Instead of scheduling SphericalHarmonicReconASCII_Startup both at initial and post_recover_variables, I recommend scheduling only at basegrid.

[Roland Haas]

The thorn now has test suites (data and parfile by Nick Taylor). They were generated (by me) using 2 MPI processes. However I found that they fails when run with 1 or 4 MPI processes. The tests use PUGH and output only reduction results (min,max,norm_info,norm2).

The thorn was also renamed to https://svn.einsteintoolkit.org/incoming/PITTNullCode/SphericalHarmonicReconGen/

[Frank Löffler]

Is it known what the cause for the failure on different numbers of processors is (tolerance too low for reduction-based tests)?

[Roland Haas]

I have to admit I have no idea. I just noticed it tonight. Since the thorn won't make it into the release (lack of documentation) this is not terribly high on my list of things to do before the release :-). The test uses the default tolerances so RELTOL=ABSTOL=1e-12. Output seems to differ at about this level. However the maximum should be independent of the number of processes I think.

diff SpEC-h5-test/NewsB[0]_maximum.asc /NewsB[0]_maximum.asc

3c3
< 963.0000000000000     0.0034305367678
---
> 963.0000000000000     0.0034305368375

[Roland Haas]

Replying to [comment:3 eschnett]:

This thorn lacks documentation, examples, and test cases.

Instead of scheduling SphericalHarmonicReconASCII_Startup both at initial and post_recover_variables, I recommend scheduling only at basegrid. Scheduling only at basedgrid will currently fail since the startup routine copies cctk_iteration into its own data structures. Since BaseGrid runs before CCTK_RECOVER_VARIABLES it would see the wrong iteration number. A solution might be to not make copies of cctkGH members in thorn-owned data structures.

[Erik Schnetter]

The iteration number (and other cGH entries) should be set before recover_variables. They should be available in basegrid. Can you open a ticket for this?

[Roland Haas]

I have to admit that I did not check what actually happens and only looked at the schedule output (CarpetIOHDF5 certainly fiddles with the iteration number though). I will investigate and open the required number of new tickets. So the idea is to set the cGH entries as early as recover_parameters?

[Erik Schnetter]

Either there, or some time shortly thereafter. However, all grid functions (and, by extension, all cGH entries) must be available in basegrid. The cGH should be set up at the same time as the grid structure.

[Roland Haas]

I just have (for the first time) actually worked with the code. I believe that it will be best to spend an afternoon and clean up the code.

TODO items:

• make sure all externally visible functions with C linkage start with "SphericalHarmonicReconGeneric_"
• consolidate the (global) variables in the SHR namespace into a single source file, vars.cc
• remove commented out sections of code
• use CCTK_INFO for informational output instead of cout
• use CCTK_Error to abort rather than assert
• clean up double vs CCTK_REAL issues
• terminate cleanly when running out of data in mode files

[Ian Hinder]

Bela has an enhancement to this thorn and would like to commit it. The current status is that this thorn lives at https://svn.einsteintoolkit.org/incoming/PITTNullCode/SphericalHarmonicReconGen/ but apparently has not been included in the toolkit due to a desire to clean up the code. For convenience, I think this should be imported into the main PittNullCode repository. I have created a git svn clone of the SVN repository, and imported this (as a subtree merge) into a new branch "sphericalharmonicrecongen" of the BitBucket PittNullCode repository. I have not applied the username mappings or attempted any other complicated cleanup of the branch. It would be good if someone could tidy up the code on this branch, get it reviewed, and hopefully into the next release, as it has been sitting in incoming for a very long time.

[Frank Löffler]

Needed: a bit of cleanup, but at least some documentation.

[Roland Haas]

Christian Reisswig took care of all remaining issues in svn revision 213 and 214. Unless vetoed in today's call I will apply this by the end of the week of Jul 13 2015.

[Roland Haas]

Included in the ET thorn list in git hash 87de2d10f3296ce1a54ff7a70fc82716b5240404 of the manifest.