simfactory offers two ways to start a simulation that are both demonstrate in the new user’s tutorial: “run” and “submit”. The latter being the common one used on clusters that use a queuing system. It relies on the queuing system to log stdout and stderr of the RunScript to appropriate files. “submit” is asynchronous in that it returns immediately and log output is only accessible via the
show-output command. For machines lacking a queuing system (generic.ini only I think) we mimic this behaviour using background execution and redirection of output and error streams in
generic.sub. The other method, which is mostly useful on private workstations without queuing system, is “run” which effectively has simfactory directly execute the runscript eg
generic.run. Since no output redirection or backgrounding happens log output goes to screen and execution in synchronous.
However log output also only goes to screen and is not saved in any log files which can be confusing (for new users who would expect a behaviour more closely mimicking
submit) or inconvenient (for more experienced users who are aware of this but might have to scroll back to earlier log output or want to have separate stdout and stderr logs).
It would be nice if
sim run would duplicate log output to log files as well as screen. This is not fully trivial for a number of reasons:
- the same runscript ist used with
runcommands and the former requires that the runscript does not redirect stdout and stderr so that they can be merged with stdout and stderr messages produced in the submitscript and possibly submit commands
- simfactory does not actually know the name of the log file that
show-outputdisplays, instead it relies on the
stdout-followoptions in the ini file
stdout and stderr should be captured to different files and both be replicated to their original file descriptors as well. This can be done (in bash) using some redirection tricks and tee:, which may however affect buffering of output to screen (since for Cactus output is no longer to a terminal but to a file thus affecting how output is buffered by default plus tee may do its own buffering). It relies on first making a copy of the original stdout descriptor to descriptor 3 then using tee to duplicate output with some switching of stderr and stdout for use in pipes (eg
>&3outputs to the original stdtout rather than the stream piped to the next tee command).
exec 3>&1 ; ( @EXECUTABLE@ -L 3 @PARFILE@ | tee sim.out >&3 ) 2>&1 | tee sim.err >&2
A way to achieve this may be to introduce a
run option analogous to the
submit option in the ini files.