xdmf files with DG0 data unsupported when running in parallel and using ghost cells

Issue #427 resolved

Ettie Unwin created an issue 2014-12-10

It is not possible to save DG0 data to a .xdmf file when using ghost cells in parallel. The following code run on two processes should recreate this:

from dolfin import *
parameters["ghost_mode"] = "shared_facet"
mesh = UnitSquareMesh(1, 1)
DG0 = FunctionSpace(mesh, 'DG', 0)
u = Function(DG)
file = File("file.xdmf")
file << u

Comments (19)

Chris Richardson

Which version of dolfin are you using? This code works for me with the dev version... or is it just that the XDMF which is produced is faulty?

$ h5ls -r file.h5 
/                        Group
/Mesh                    Group
/Mesh/0                  Group
/Mesh/0/cell_indices     Dataset {2048}
/Mesh/0/coordinates      Dataset {1089, 2}
/Mesh/0/topology         Dataset {2048, 3}
/VisualisationVector     Group
/VisualisationVector/0   Dataset {2048, 1}

2014-12-10T12:42:12+00:00

Ettie Unwin reporter

I have the development version. When I try and run the file using mpirun I get this error:

*** -------------------------------------------------------------------------
*** DOLFIN encountered an error. If you are not able to resolve this issue
*** using the information listed below, you can ask for help at
***
***     fenics@fenicsproject.org
***
*** Remember to include the error message listed below and, if possible,
*** include a *minimal* running example to reproduce the error.
***
*** -------------------------------------------------------------------------
*** Error:   Unable to complete call to function write_dataset().
*** Reason:  Assertion status != HDF5_FAIL failed.
*** Where:   This error was encountered inside /Users/xxxxxxxxx/Documents/fenics/dolfin/dolfin/io/HDF5Interface.h (line 303).
*** Process: unknown
*** 
*** DOLFIN version: 1.4.0+
*** Git changeset:  a855849f99f19ded571d51269bd3cf1305b77d45
*** -------------------------------------------------------------------------

HDF5-DIAG: Error detected in HDF5 (1.8.13) MPI-process 1:
  #000: H5F.c line 2070 in H5Fclose(): decrementing file ID failed
    major: Object atom
    minor: Unable to close file
  #001: H5I.c line 1464 in H5I_dec_app_ref(): can't decrement ID ref count
    major: Object atom
    minor: Unable to decrement reference count
  #002: H5F.c line 1839 in H5F_close(): can't close file, there are objects still open
    major: File accessibilty
    minor: Unable to close file
libc++abi.dylib: terminating with uncaught exception of type std::runtime_error: 

*** -------------------------------------------------------------------------
*** DOLFIN encountered an error. If you are not able to resolve this issue
*** using the information listed below, you can ask for help at
***
***     fenics@fenicsproject.org
***
*** Remember to include the error message listed below and, if possible,
*** include a *minimal* running example to reproduce the error.
***
*** -------------------------------------------------------------------------
*** Error:   Unable to complete call to function close_file().
*** Reason:  Assertion status != HDF5_FAIL failed.
*** Where:   This error was encountered inside /Users/xxxxxx/Documents/fenics/dolfin/dolfin/io/HDF5Interface.cpp (line 104).
*** Process: unknown
*** 
*** DOLFIN version: 1.4.0+
*** Git changeset:  a855849f99f19ded571d51269bd3cf1305b77d45
*** -------------------------------------------------------------------------


===================================================================================
=   BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
=   PID 4048 RUNNING AT xxxxxxxxxx
=   EXIT CODE: 6
=   CLEANING UP REMAINING PROCESSES
=   YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
===================================================================================
YOUR APPLICATION TERMINATED WITH THE EXIT STRING: Abort trap: 6 (signal 6)
This typically refers to a problem with your application.
Please see the FAQ page for debugging suggestions

2014-12-10T13:23:47+00:00

Chris Richardson
@ettieunwin - OK, that is weird. Is it only with ghost_mode that this happens?
- 2014-12-10T15:36:25+00:00
Ettie Unwin reporter
Yes it is.
- 2014-12-10T16:14:13+00:00
Prof Garth Wells
- changed status to invalid
Marking as invalid for now because I can't reproduce. Try re-installing HDF5 (with parallel support).
- 2014-12-23T16:47:00+00:00
Ettie Unwin reporter
I've done that and I still get the same error.
- 2015-01-03T12:25:12+00:00
Prof Garth Wells
- changed milestone to 1.6
- 2015-01-05T13:34:40+00:00
Prof Garth Wells
- edited description
Could be a memory error. Targeting 1.6 since shared facets/nodes, etc are still an 'undocumented' feature.
- 2015-01-05T13:37:53+00:00
Prof Garth Wells
- changed status to open
Possible memory issue.
- 2015-01-05T13:38:06+00:00
Prof Garth Wells
- assigned issue to
  
  Chris Richardson
- 2015-01-05T13:38:46+00:00
Prof Garth Wells
- edited description
- 2015-01-05T13:41:53+00:00
Chris Richardson
- removed milestone
- 2015-05-27T14:49:47+00:00
Prof Garth Wells
Is this still an issue?
- 2016-01-26T08:46:48+00:00

Ettie Unwin reporter

I still get this error on my mac:

Fatal error in MPI_Init_thread: Other MPI error, error stack:
MPIR_Init_thread(498)..............: 
MPID_Init(187).....................: channel initialization failed
MPIDI_CH3_Init(89).................: 
MPID_nem_init(320).................: 
MPID_nem_tcp_init(171).............: 
MPID_nem_tcp_get_business_card(418): 
MPID_nem_tcp_init(377).............: gethostbyname failed, 140.144.248.10.in-addr.arpa.private.cam.ac.uk (errno 1)
Fatal error in MPI_Init_thread: Other MPI error, error stack:
MPIR_Init_thread(498)..............: 
MPID_Init(187).....................: channel initialization failed
MPIDI_CH3_Init(89).................: 
MPID_nem_init(320).................: 
MPID_nem_tcp_init(171).............: 
MPID_nem_tcp_get_business_card(418): 
MPID_nem_tcp_init(377).............: gethostbyname failed, 140.144.248.10.in-addr.arpa.private.cam.ac.uk (errno 1)

===================================================================================
=   BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
=   PID 4426 RUNNING AT 140.144.248.10.in-addr.arpa.private.cam.ac.uk
=   EXIT CODE: 1
=   CLEANING UP REMAINING PROCESSES
=   YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
===================================================================================

but it works fine on my linux machine

2016-01-26T09:52:33+00:00

Johannes Ring
This works fine for me on a Mac when using DOLFIN installed by fenics-install.sh (both 1.6 and dev tested), but I get errors when using the FEniCS 1.6 binary package for Mac. @ettieunwin - How did you install DOLFIN and the dependencies?
- 2016-01-26T14:10:00+00:00
Ettie Unwin reporter
I have been installing the dev version of DOLFIN and its dependencies which I have pulled from bitbucket.
- 2016-01-26T14:57:34+00:00
Johannes Ring
What about HDF5? Have you installed that from source as well?
- 2016-01-26T17:01:00+00:00
Ettie Unwin reporter
No I have that installed via homebrew
- 2016-01-27T08:58:47+00:00
Chris Richardson
- changed status to resolved
I am closing this, as it is not reproducible. I think there may be other side issues, such as trying to write to a HDF5 file which is open by another process, which clearly can cause problems.
- 2016-03-31T08:33:04+00:00
Log in to comment

Assignee: Chris Richardson

Type: bug

Priority: minor

Status: resolved

Component: –

Milestone: –

Version: –

Votes: 0

Watchers: 3