CarpetInterp/waveinterp_2p tests constant in space initial data

Create issue
Issue #1599 closed
Roland Haas created an issue

the test waveinterp_2p sets up initial data using IDWaveMoL (fom CactusExamples) using kx=ky=kz = 0. and slopet = 0.1. Ie the initial data is constant in space and has frequency 0.1. It then evolves this using WaveMoL and interpolates in space and time with CarpetInterp.

Since the function is constant in space (exactly so, see eg the output in InterpToArray::array1d_vars[0]) the interpolation in space just tests roundoff errors. The second time derivative (array1d_vars[2]) should be zero and all we measure in the test is truncation errors on the level of 1e-9. This makes the test very sensitive to -Ofast, -O0, gcc vs. intel etc. and not a very good test.

I would rather either use a Gausian wave for initial data so that there is some actual data in the result and not just numerical noise, or leave out the 2nd time derivative from the test data.

The test fails due to significant errors in a virtual machine using ubuntu.cfg (which usies -ffast-math and -O2).

Keyword: CarpetInterp

Comments (12)

  1. Erik Schnetter
    • removed comment

    I believe I disabled checking the relative error in this test. The absolute error should be fine, and is independent of the amplitude of a wave that would be added.

    Testing the second time derivative is important, we should not omit it.

  2. Roland Haas reporter
    • removed comment

    As far as I can tell relative error checks are not disabled in test.ccl. It only specifies the number of processes to use for each test. I can add a

    RELTOL 1e42
    

    to the test to avoid relative differences.

    I agree that testing the 2nd time derivative is important.

    Would a non-zero 2nd time derivative and not-constant in space data test more of the code? Testing an interpolator with constant (or very low order data) seems not like the best idea. The prolongation tests eg use operators that are as high order as possible while still allowing exact recovery of the result (ie 3rd order polynomial test function for the 3rd order prolongation operators).

  3. Roland Haas reporter
    • changed status to open
    • removed comment

    Turns out that setting reltol to 1e42 as proposed is not the correct solution. The comparison inside of RunTestUtils.pl is

    1745 my $vreltol = $filereltol * $valmax[$count];
    1746 my $vtol = $fileabstol > $vreltol ? $fileabstol : $vreltol;
    1747 if ($absdiff[$count] >= $vtol) {
    1748   $rundata->{"$thorn $test $file NFAILSTRONG"}++;
    1749   last;
    1750 }
    

    ie setting reltol to 1e42 sets $vreltol to some very large number and ignores abstol. In order to only use abstol, one has to set reltol to 0. The tests actually had larger absolute error than allowed (eg on bluewaters, though I do seem to get different results than Erik does):

       arrays1d[2].x.asc: substantial differences
          significant differences on 170 (out of 204) lines
          maximum absolute difference in column 13 is 5.94358425587416e-08
          maximum relative difference in column 13 is 63.81875
       arrays1d[2].y.asc: substantial differences
          significant differences on 17 (out of 51) lines
          maximum absolute difference in column 13 is 5.94358425587416e-08
          maximum relative difference in column 13 is 4.17201225740553
    

    So even with an ABSTOL=1e-8 (and RELTOL=0. so that it does not matter) this will fail. Please note that all bluewaters tests pass if I compile with -O3 and -mno-fma -mno-fma4 ie disable non-value safe optimization and fused multiply add instructions (which I'd also claim to be not value safe).

  4. Ian Hinder
    • removed comment

    Many of the bluewaters failures, e.g. from Exact, look to be fixable by a very slight increase of the tolerance to something larger than 1e-12.

  5. Erik Schnetter
    • removed comment

    Apparently Cactus compares the error to the larger of the absolute and the relative tolerance. If so, then decreasing (sic!) the relative tolerance should not be necessary, and we only need to increase the absolute tolerance.

  6. Roland Haas reporter
    • removed comment

    The tolerances were increase in rev 297, 130, 143 of Exact, IDAxiBrillBH and TwoPunctures.

  7. Log in to comment