- changed status to resolved
- removed comment
Rate-limit cleanups
Simfactory performs a cleanup each time certain operations are executed. This can slow down simfactory significantly, in particular when many operations are performed successively. There should be a rate limiter for cleanups.
Keyword:
Comments (4)
-
-
reporter - marked as enhancement
- changed status to open
- marked as
- removed comment
There could instead be a single file that is acts on all simulations, since automatic cleanup will always act on all simulations. A per-simulation check is not necessary. This file would e.g. have the name cleanup-timestamp, and the age of the file determines whether cleanup is skipped. (The file name itself does not need to encode the age.)
In addition, if the file's timestamp is too far in the future, there would be a warning and the time stamp is deleted/ignored.
Explicit cleanups would ignore this time stamp. If a command acts on a simulation (e.g. "submit"), it would also ignore this time stamp, and still clean up this simulation if necessary.
-
- changed status to resolved
- removed comment
Cleanups are now rate limited. (every 30 seconds)
-
- changed status to closed
- edited description
- Log in to comment
Cleanups should now be rate limited. It will only attempt to cleanup a simulation every 120 seconds. A new file called timestamp-[some number here] is written on a per simulation basis in <simulationdir>, and checked for against time.time().