HDF5File.write() should check function names in parallel
Updated: The following gives no error, but produces corrupt files due to different random function name order on different parallel processes. This can easily happen when using dictionaries to store functions to be output
import dolfin
mesh = dolfin.UnitSquareMesh(4, 4)
V = dolfin.FunctionSpace(mesh, 'CG', 1)
funcs = []
for i in range(10):
name = 'u_%d' % i
funcs.append((name, dolfin.Function(V)))
import random
random.shuffle(funcs)
with dolfin.HDF5File(mesh.mpi_comm(), 'test.h5', 'w') as h5:
h5.write(mesh, '/mesh')
for name, u in funcs:
h5.write(u, '/%s' % name)
Comments (5)
-
-
reporter That makes sense. Although, my "randomization" was just dictionary iteration, which should be deterministic on all processes since dictionary insertion should have been the same on all processes. But, I might have made a mistake somewhere
-
reporter - marked as minor
- edited description
- changed title to HDF5File.write() should check function names in parallel
Updated description to make the real reason apparent. Thanks @michalhabera !
-
Unlike in
pyton2.7
inpython3.3
dictionary iteration is by default not deterministic. So I think the linefor name, u in funcs
just iterates in different order on different processes. See https://stackoverflow.com/questions/14956313/why-is-dictionary-ordering-non-deterministic -
reporter Then I learned something today as well :-) I was sure it was somehow related to different function spaces being output after one another and some obscure bug related to internal state in HDF5File. Then, it was just my fault as normal ;-)
- Log in to comment
Hi Tormod, I tried to run this with
XDMFFile::write_checkpoint
method and it failed onXDMFFile::check_function_name(std::string function_name)
. This check method was added to make sure that function name in parallel is the same on all processes, otherwise HDF paths are completly messed up. Maybe all we can do is add the similar check for HDFFile interface, any other thoughts?