avoid opening and closing HDF5 files multiple times during output
Issue #2364
open
Reduce file system open/close calls from once per variable per reflevel per iteration of output to once per variable per iteration of output or less (if one_file_per_group or one_file_per_proc is used).
It does however keep more files open which can be an issue if the OS limit on open files is too small. Usually this limit is per process though and as long as one process does not open many files, the whole simulation can still open very many.
Comments (3)
-
reporter -
reporter Might be suitable as a hackathon review task.
-
reporter I did some work during the hackathon. Will have to rework it a bit for a cleaner implementation, maybe add a parameter as well.
- Log in to comment
Pull request is here: https://bitbucket.org/eschnett/carpet/pull-requests/34/cache-file-handles-in-hdf5-output-to-avoid/diff