Simfactory's test mechanism should use softlinks instead of rsync

Create issue
Issue #1721 open
Erik Schnetter created an issue

Simfactory's test mechanism copies all test cases from the source tree into the simulation directory. This takes a long time and a lot of space. This should rarely be necessary, since these test usually run soon after being submitted, so that there is little chance of the source tree becoming outdated.

Keyword:

Comments (2)

  1. Ian Hinder
    • removed comment

    The main reason I added support for running the tests using simfactory was so that I could run them on Kraken, where the compute nodes didn't have access to the home filesystem, so the copying was essential. I am also a little nervous about the assumption that the source tree stays around. The tests may sit in the queue for a while, or you may change branch in the source directory. I agree it would be good to reduce the time taken to copy the data, but I seem to remember that a design goal of simfactory was to be correct, and avoid people making mistakes, at the potential expense of efficiency.

    Maybe we could have a "cache" of the required files on the scratch filesystem, and use rsync's link-dest option, to make hard links to that cache. It wouldn't help the first time, but subsequent jobs would benefit.

  2. Log in to comment