have simfactory's "run" user command log output to files as well as screen

simfactory offers two ways to start a simulation that are both demonstrate in the new user’s tutorial: “run” and “submit”. The latter being the common one used on clusters that use a queuing system. It relies on the queuing system to log stdout and stderr of the RunScript to appropriate files. “submit” is asynchronous in that it returns immediately and log output is only accessible via the show-output command. For machines lacking a queuing system (generic.ini only I think) we mimic this behaviour using background execution and redirection of output and error streams in generic.sub. The other method, which is mostly useful on private workstations without queuing system, is “run” which effectively has simfactory directly execute the runscript eg generic.run. Since no output redirection or backgrounding happens log output goes to screen and execution in synchronous.

However log output also only goes to screen and is not saved in any log files which can be confusing (for new users who would expect a behaviour more closely mimicking submit) or inconvenient (for more experienced users who are aware of this but might have to scroll back to earlier log output or want to have separate stdout and stderr logs).

It would be nice if sim run would duplicate log output to log files as well as screen. This is not fully trivial for a number of reasons:

the same runscript ist used with submit and run commands and the former requires that the runscript does not redirect stdout and stderr so that they can be merged with stdout and stderr messages produced in the submitscript and possibly submit commands
simfactory does not actually know the name of the log file that show-output displays, instead it relies on the stdout, stderr and stdout-follow options in the ini file
stdout and stderr should be captured to different files and both be replicated to their original file descriptors as well. This can be done (in bash) using some redirection tricks and tee:, which may however affect buffering of output to screen (since for Cactus output is no longer to a terminal but to a file thus affecting how output is buffered by default plus tee may do its own buffering). It relies on first making a copy of the original stdout descriptor to descriptor 3 then using tee to duplicate output with some switching of stderr and stdout for use in pipes (eg >&3 outputs to the original stdtout rather than the stream piped to the next tee command).

exec 3>&1 ; ( @EXECUTABLE@ -L 3 @PARFILE@ | tee sim.out >&3 ) 2>&1 | tee sim.err >&2

A way to achieve this may be to introduce a run option analogous to the submit option in the ini files.

Comments (6)