Simfactory's test mechanism should use softlinks instead of rsync

Create issue
Issue #1721 open
Erik Schnetter created an issue

Simfactory's test mechanism copies all test cases from the source tree into the simulation directory. This takes a long time and a lot of space. This should rarely be necessary, since these test usually run soon after being submitted, so that there is little chance of the source tree becoming outdated.


Comments (2)

  1. Ian Hinder
    • removed comment

    The main reason I added support for running the tests using simfactory was so that I could run them on Kraken, where the compute nodes didn't have access to the home filesystem, so the copying was essential. I am also a little nervous about the assumption that the source tree stays around. The tests may sit in the queue for a while, or you may change branch in the source directory. I agree it would be good to reduce the time taken to copy the data, but I seem to remember that a design goal of simfactory was to be correct, and avoid people making mistakes, at the potential expense of efficiency.

    Maybe we could have a "cache" of the required files on the scratch filesystem, and use rsync's link-dest option, to make hard links to that cache. It wouldn't help the first time, but subsequent jobs would benefit.

  2. Log in to comment