SphericalHarmonicRecon test is failing

Create issue
Issue #873 closed
Ian Hinder created an issue

See http://damiana2.aei.mpg.de/~ianhin/testsuites/einsteintoolkit/ (from 05.07). Diffs are:

   NewsB[0]_2D.asc: substantial differences
      significant differences on 2208 (out of 4515) lines
   NewsB[0]_norm2.xg: substantial differences
      significant differences on 1 (out of 2) lines
      maximum absolute difference in column 1 is 0.1
      maximum absolute difference in column 2 is 0.0337938571711373
      maximum relative difference in column 1 is 1
      maximum relative difference in column 2 is 1

The test passed on 05-May-2012. There have not been any changes to this thorn recently (https://trac.einsteintoolkit.org/log/PITTNullCode/SphericalHarmonicRecon/trunk), so this must have been caused by a change to a different component.

Keyword: testsuites

Comments (8)

  1. Ian Hinder reporter
    • removed comment

    Does anyone have an idea for how to fix this? It is the last failing test on Datura. The standard output (http://damiana2.aei.mpg.de/~ianhin/testsuites/einsteintoolkit/einsteintoolkit/2012-05-16-010001/SphericalHarmonicRecon_regression_test.1.log) shows large numbers of the following warning:

    WARNING[L1,P0] (AEILocalInterp): CCTK_InterpLocalUniform(): input datatype 111 not supported! (0-origin) input #in=0

    There was a comment that this might be due to a configuration problem on the machine, but the configuration has not changed. The regression was introduced some time between the 5th and the 7th of May.

    Mon May 7 20:32:21 2012 +0100

    • flesh 474dc19...cef5ed9 (1): > Protect declarations with external "C"
    • simfactory aa415d3...9aca06f (2): > Build documentation by default > Synchronise doc directory Mon May 7 17:32:18 2012 +0100
    • flesh ac43e07...474dc19 (1): > Enable "restrict" as keyword in C++
    • manifest 7b3195d...14405cd (3): > add comment about opencl thorns > Add OpenCL thorn, but keep it commented out > Add CCZ4 formulation Mon May 7 16:32:09 2012 +0100
    • simfactory ba7196d...aa415d3 (4): > Invalidate aliaspattern to avoid conflicts > Set scratchbasedir > Set scratchbasedir > Introduce scratchbasedir instead of scratchdir Mon May 7 07:33:17 2012 +0100
    • arrangements/EinsteinBase/HydroBase bbfe4eb...6c6a71a (1): > Documentation: fix conversion factor for SI units. Add Gaussian one. Mon May 7 03:32:00 2012 +0100
    • simfactory 09478d5...ba7196d (2): > Update > Examine more lines for status output Sun May 6 23:32:00 2012 +0100
    • arrangements/AEIThorns/AEILocalInterp aec6b0b...72130d2 (1): > Parallelize AEILocalInterp with OpenMP Sun May 6 03:31:59 2012 +0100
    • simfactory f63e8d9...09478d5 (1): > Add missing scratchdir Sat May 5 23:32:01 2012 +0100
    • simfactory ebc22d9...f63e8d9 (1): > Correct, update, and simplify configuration Sat May 5 22:32:04 2012 +0100
    • simfactory a30ad51...ebc22d9 (1): > Set FFTW3_DIR instead of FFTW_DIR Sat May 5 16:32:01 2012 +0100
    • simfactory 8f46d8e...a30ad51 (3): > Add Kraken to "known good machines" > Don't delete scratch directory > Clarify description of "scratchdir" Sat May 5 15:32:19 2012 +0100
    • arrangements/EinsteinInitialData/NoExcision 29c738b...6a6b78c (1): > Declare all loop variables private
    • arrangements/EinsteinInitialData/TwoPunctures 7e3d877...59b4352 (1): > Declare all loop variables private
    • arrangements/LSUThorns/SummationByParts 7a5e754...cc09b8e (1): > Declare all loop variables as OpenMP private Sat May 5 13:32:30 2012 +0100
    • repos/carpet 74be6e5...d7663b5 (1): > CarpetLib: Disable OpenMP collapse statements
    • simfactory c2b986b...8f46d8e (4): > Update > Update web page > Update list of "known good" machines on which to test > Update Forge configuration Sat May 5 04:32:22 2012 +0100
    • arrangements/EinsteinEvolve/GRHydro c40fdb9...ddee80c (1): > remove tests using BSSN_MoL for which ML_BSSN alternatives exist
    • arrangements/EinsteinInitialData/Exact de5c5df...0b51a27 (1): > use ML_BSSN instead of BSSN_MoL for testsuites Sat May 5 03:32:11 2012 +0100
    • arrangements/CactusNumerical/Dissipation b152a29...2af5af1 (1): > use ML_BSSN instead of BSSN_MoL in testsuites Sat May 5 02:32:26 2012 +0100
    • arrangements/CactusNumerical/Cartoon2D 31c0fc2...3370381 (1): > use ML_BSSN instead of BSSN_MoL in testsuites
    • arrangements/CactusNumerical/RotatingSymmetry90 59fb60e...d25162a (1): > use ML_BSSN instead BSSN_MoL in testsuites Sat May 5 01:33:29 2012 +0100
    • arrangements/CactusNumerical/RotatingSymmetry180 8f3cdd4...23b7bcf (1): > use ML_BSSN instead of BSSN_MoL in testsuites
    • repos/carpet 0a20793...74be6e5 (2): > CarpetEvolutionMask: use ML_BSSN instead of BSSN_MoL in testsuites > CarpetEvolutionMask: use dd.buffer_widths instead of buffer_width parameter

    In addition to fixing whatever this problem is, I think that the calling thorn should check the result and abort if the interpolation cannot be performed. Currently the error logs are very large.

  2. Ian Hinder reporter
    • removed comment

    I think I have narrowed it down to this commit:

    http://git.barrywardell.net/arrangements/AEIThorns/AEILocalInterp.git/commit/72130d23ede5d69c96102c7822f55975d5b9ef44

    author schnetter <schnetter@0f49ee68-0e4f-0410-9b9c-b2c123ded7ef> Mon, 7 May 2012 00:14:23 +0200 (22:14 +0000) committer schnetter <schnetter@0f49ee68-0e4f-0410-9b9c-b2c123ded7ef> Mon, 7 May 2012 00:14:23 +0200 (22:14 +0000) commit 72130d23ede5d69c96102c7822f55975d5b9ef44 tree 774093ff98e9dea96f46ef96b00fb6b83742c8bf tree parent aec6b0bd9a2f82096fde2b43c159653ee255a07f commit | diff Parallelize AEILocalInterp with OpenMP

    This leads to a slight change in behaviour. Currently, AEILocalInterp traverses the list of points sequentially, and aborts when the first error is encountered. After parallelisation, there is no fixed order in which the points are traversed, and if several errors are encountered, any one of the errors may be returned, not necessarily the first. I am not aware of any thorn that would or should rely on such an ordering.

    This patch also adds "restrict" and "const" statements that may improve performance as it gives the compiler more information about dependencies between pointers.

    git-svn-id: https://svn.aei.mpg.de/numrel/AEIThorns/AEILocalInterp/trunk@55 0f49ee68-0e4f-0410-9b9c-b2c123ded7ef

    Erik, can you see something in this commit that might explain why these errors are generated?

  3. Erik Schnetter
    • changed status to open
    • removed comment

    This change is an optimisation. If it causes problems, then I recommend to revert it until after the release.

  4. Log in to comment