- changed status to open
- removed comment
CarpetProlongateTest/test_o9 is failing on several machines
CarpetProlongateTest/test_o9 is failing on Gordon, Mike and Redshift. On Gordon, the differences are
carpetprolongatetest::difference.d.asc: substantial differences
significant differences on 1 (out of 206) lines
maximum absolute difference in column 13 is 1.07288360595703e-06
maximum absolute difference in column 14 is 6.98491930961609e-10
maximum relative difference in column 13 is 1.12625729345722e-12
maximum relative difference in column 14 is 3
(insignificant differences on 19 lines)
carpetprolongatetest::difference.x.asc: differences below tolerance on 26 lines
carpetprolongatetest::difference.y.asc: differences below tolerance on 26 lines
carpetprolongatetest::errornorm..asc: differences below tolerance on 1 lines
carpetprolongatetest::scalar.d.asc: differences below tolerance on 13 lines
carpetprolongatetest::scalar.x.asc: differences below tolerance on 6 lines
carpetprolongatetest::scalar.y.asc: differences below tolerance on 4 lines
and on the other machines the results are similar. test.ccl contains
TEST test_o7
{
ABSTOL 2.0e-11
}
TEST test_o9
{
ABSTOL 5.0e-10
}
TEST test_o11
{
ABSTOL 3.0e-8
}
Higher order prolongation probably leads to more amplification of roundoff differences, which is why the absolute tolerances listed here increase with prolongation order.
On Redshift, which uses -Ofast with gcc, the maximum absolute differences in columns 13 and 14 are just marginally above the tolerance of 5e-10:
maximum absolute difference in column 13 is 9.31322574615479e-10
maximum absolute difference in column 14 is 6.98491930961609e-10
maximum relative difference in column 13 is 9.77653900570505e-16
maximum relative difference in column 14 is 3
However on Gordon and Mike, the column 13 absolute difference is 1e-6, which presumably means the data is large, so the relative tolerance will come into play. The default relative tolerance is 1e-12, and the difference in column 13 is marginally above this.
Gordon:
maximum absolute difference in column 13 is 1.07288360595703e-06
maximum absolute difference in column 14 is 6.98491930961609e-10
maximum relative difference in column 13 is 1.12625729345722e-12
maximum relative difference in column 14 is 3
Mike:
maximum absolute difference in column 13 is 1.07288360595703e-06
maximum absolute difference in column 14 is 6.98491930961609e-10
maximum relative difference in column 13 is 1.12625729345722e-12
maximum relative difference in column 14 is 3
Should we increase both the relative and absolute tolerances for this test to 1e-11? I believe that would make the test pass on all three machines.
Keyword:
Comments (9)
-
-
- changed status to open
- removed comment
-
reporter - changed status to resolved
- removed comment
Committed in f73efa0164d8296c361e486fabdd95c88469766a of Carpet.
-
reporter - changed status to open
- removed comment
It turns out that 1e-11 is larger than 1e-10, so the test now fails. The absolute tolerance should have been increased to 1e-9 instead. Still OK to change?
-
reporter - removed comment
That should have read: "It turns out that 1e-11 is smaller than 1e-10". It also turns out that I frequently make the same mistake over and over again...
-
- changed status to open
- removed comment
-
- changed status to open
- removed comment
Still ok.
-
reporter - removed comment
Committed in baf2333c5b909d6daa93056b1c63b35040796b66 of Carpet. Test passes on Datura. This ticket should be closed if it also passes on gordon, redshift and mike.
-
- changed status to resolved
- removed comment
Fixed.
- Log in to comment
This sounds reasonable.