1. Running the automatic test suite

To run the automatic tests, navigate to the tests directory, and run the run_test_suite.sh script:

>$ cd tests
>$ ./run_test_suite.sh

The tests will begin and the output should look like:

############################################
#   Running RAMSES automatic test suite    #
############################################
Will perform the following tests:
 [ 1] hydro/implosion
 [ 2] hydro/sod-tube
 [ 3] mhd/imhd-tube
 [ 4] mhd/orszag-tang
 [ 5] rt/stromgren2d
 [ 6] sink/smbh-bondi
--------------------------------------------
Test 1/6: hydro/implosion
Cleanup
Compiling source

and so on. Once the tests have completed, a report is generated in a .pdf file named test_results.pdf, alongside a log file test_suite.log.

Options

Run the suite in parallel (on 4 cpus):
```
./run_test_suite.sh -p 4
```
Do not delete results data:
```
./run_test_suite.sh -d
```
Run in verbose mode:
```
./run_test_suite.sh -v
```
Select individual tests (for tests 3 to 5, and 10):
```
./run_test_suite.sh -t 3-5,10
```
Run all tests in mhd directory:
```
./run_test_suite.sh -t mhd
```

2. Creating a new test

The following steps describe how to add a new test to the test suite. In this example, the test will be named sedov-3d.

The first step is to create a new directory sedov-3d in one of the hydro, mhd, rt, or sinks directories. No need to modify the run_test_suite.sh script, the new test will automatically be picked up and added to the list. We will choose to place it inside the hydro directory. Please use hyphens (-) in your test names instead of underscores (_) as latex does not like underscores.

>$ cd hydro
>$ mkdir sedov-3d

Note: use one directory per test. If you want to run a 2D and a 3D sedov test, create separate sedov-2d and sedov-3d directories.

In that directory, you will need:

A config.txt file: usually just contains the Makefile flags, e.g. FLAGS: NDIM=3 PATCH= SOLVER=hydro
A namelist: sedov-3d.nml (the name needs to be the same as the test directory)
A file for plotting and checking the solution against a reference: plot-sedov-3d.py. It is advised to copy a file from the other directories to see how to write this. Note that this file needs to contain at least one call to visu_ramses.check_solution(data["data"], 'sedov-3d').
A reference solution: sedov-3d-ref.txt. To create it, run your test and once the final output (number 2 in this case) has been created, do the following:
```
import visu_ramses
data = visu_ramses.load_snapshot(2)
visu_ramses.check_solution(data["data"], 'sedov-3d', overwrite=True)
```
A Readme.md containing a short description of the test

Optional files:

condinit.f90: you can have your own initial setup if it's not entirely definable in a namelist. REMEMBER to set the correct PATCH in the config.txt file! (e.g. PATCH=../tests/hydro/sedov-3d)
before-test.sh: if this file is present in the test directory, it will be run before the test begins (useful for e.g. creating symbolic links to libraries...)
after-test.sh: if this file is present in the test directory, it will be run after the test begins (useful for e.g. cleaning up symbolic links to libraries...)

Tuning tolerances for solution verification

By default, relative differences between the sums of all the variables inside all leaf cells in the domain and the reference solution cannot exceed 3.0e-13. Sometimes, some variables are more volatile than others when running simulations on different numbers of CPUs, and this limit is too low, leading to false failed tests. The check_solution method in the visu/visu_ramses.py module can be tuned to work for your test using the following options:

tolerance: a dictionary listing the allowed relative difference between the sum over all leaf cells and reference value. The default is {"all":3.0e-13}. To make the check on density less restrictive, use for instance tolerance={"density":1.0e-10}.
threshold: relative value below which a vector component is set to zero. Default is 2.0e-14.
norm_min: minimum value for the norm of a vector, to protect against null vectors. Default is 1.0e-30.
min_variance: if the data differs by less than this value from the average value, it is set to the average. Default is 1.0e-14.

3. Creating a new group of tests

If your test does not fall under the categories already present in the tests directory (hydro, mhd, rt, sinks), you can create a new directory and put your tests in there. You will then have to edit the run_test_suite.sh file to ensure your new tests will be picked up.

Say you want to create 3 new tests, sedov-1d, sedov-2d, and sedov-3d inside a new sedov directory, you have to find the line describing the list of directories to be scanned at the top of the run_test_suite.sh file:

# List of directories to scan
testlist="hydro,mhd,rt,sink";

and add your new directory separated from the previous one by a comma, i.e.

# List of directories to scan
testlist="hydro,mhd,rt,sink,sedov";

Wiki

ramses / Chapter 6. Testing

1. Running the automatic test suite

Options

2. Creating a new test

Tuning tolerances for solution verification

3. Creating a new group of tests