By default, the entire test suite is run. If you want to run only specific tests, you can additionally use the --select-tests option. You can give this option a test name (ending in .par), an arrangement name or a thorn specification in the form <arrangement>/<thorn>.
Since it is necessary to have the test data and Cactus flesh scripts available when the job starts, the required data is copied into the simulation restart directory on job submission (or interactive running). This ensures that the test data and scripts are available when the home directory is not mounted on the compute nodes, and that the test data is not modified between job submission and job running.
The test selection mechanism works by choosing which tests to copy (ensuring that all test.ccl files are copied, as this is necessary to get all the output files compared), so there is always a summary.log file produced in the "standard" format.
The run scripts have been modified to pass the required options, but as I'm typing this I realise this might not be necessary since the required information should be stored in the simulation properties upon job submission
When you use the --testsuite option, it is not necessary to specify a parameter file. The positional arguments syntax (parfile, cores, walltime) is not supported for running the test suite.
This has been tested on my laptop, on Datura and on Kraken, and works in those places. I have also tested that running normal jobs works, though there might be corner cases where something has broken that was not seen in my tests.
I've added a new patch, 0001-Add-testsuite-support-v2, which supersedes the previous two. This one does not add the unnecessary options to the run scripts.