cache file handles in HDF5 output to avoid opening and closing the same file multiple times
Roland Haas
Branch: rhaas/cacheFH
rhaas/cacheFH
Branch: master
master
Open
#34 · Created  · Last updated
Description
CarpetIOHDF5: keep outND_var files open until all data is written
this reduces file system open/close calls from once per variable per reflevel per iteration of output to once per variable per iteration of output or less (if one_file_per_group or one_file_per_proc is used).
It does however keep more files open which can be an issue if the OS limit on open files is too small. Usually this limit is per process though and as long as one process does not open many files, the whole simulation can still open very many.
CarpetIOHDF5: be more carefull closing all open hdf5 files
CarpetIOHDF5: keep outND_var files open until all data is written
this reduces file system open/close calls from once per variable per reflevel per iteration of output to once per variable per iteration of output or less (if one_file_per_group or one_file_per_proc is used).
It does however keep more files open which can be an issue if the OS limit on open files is too small. Usually this limit is per process though and as long as one process does not open many files, the whole simulation can still open very many.
CarpetIOHDF5: be more carefull closing all open hdf5 files
Â