lazylogger is too lazy when mlogger is writing dotfiles

[Ben Smith]

In SuperCDMS we have mlogger configured to write files of the form ".RUN%05d_DUMP%04d.mid" (note the leading dot). When mlogger closes the file, it renames it to remove the dot (see log_close() in mlogger.cxx).

The lazylogger has logic to make sure it doesn't process a file that is currently being written to. However, it does this by assuming that the most recent file for a run is still being written to if the run is still in progress (see nothing_todo logic in lazy_main() of lazylogger.cxx). In our case, the dotfile does not appear in the list of files that lazylogger considers, so the logic is broken; we DO want lazylogger to handle the most recent file as well.

The problem is that we need to rename and copy some files to process them very quickly. Our subruns are 60s long, so this "off-by-one" causes a 60s delay in the processing (as it waits until there are 2 visible files instead of 1). One solution would be for us to ignore lazylogger completely and write our own solution, but that would be a shame as lazylogger is 99% of the way there.

I'm not sure what the best solution is. It doesn't seem like lazylogger can easily interrogate the logger channels to see whether the channel writing our file is initially writing a hidden dotfile. Perhaps an extra boolean flag could be added to the lazylogger settings to indicate that dotfiles are initially being written? The check in L1903 of lazylogger (else if (behind_files > 1)) could be changed to compare against 0/1 based on the flag. Are there any better solutions?