Rate-limit cleanups

[Erik Schnetter]

Simfactory performs a cleanup each time certain operations are executed. This can slow down simfactory significantly, in particular when many operations are performed successively. There should be a rate limiter for cleanups.

Keyword:

[anonymous]

Cleanups should now be rate limited. It will only attempt to cleanup a simulation every 120 seconds. A new file called timestamp-[some number here] is written on a per simulation basis in <simulationdir>, and checked for against time.time().

[Erik Schnetter]

There could instead be a single file that is acts on all simulations, since automatic cleanup will always act on all simulations. A per-simulation check is not necessary. This file would e.g. have the name cleanup-timestamp, and the age of the file determines whether cleanup is skipped. (The file name itself does not need to encode the age.)

In addition, if the file's timestamp is too far in the future, there would be a warning and the time stamp is deleted/ignored.

Explicit cleanups would ignore this time stamp. If a command acts on a simulation (e.g. "submit"), it would also ignore this time stamp, and still clean up this simulation if necessary.

[anonymous]

Cleanups are now rate limited. (every 30 seconds)