- edited description
Reg. HDF5 output of QuasiLocalMeasures variables
Greetings
I am currently facing an issue when attempting to output QuasiLocalMeasures variables in HDF5 (in a BBH simulation similar to the gallery one) files instead of ASCII. I only face this issue when I request any QuasiLocalMeasures variables to be output in IOHDF5.
I am getting the same error message as in 005361 (http://lists.einsteintoolkit.org/pipermail/users/2017-March/005361.html). .
I am attaching the parameter file, the backtrace and the out and err files from the run. Requesting your help.
(ETK messages archive num. 007565)
Thanking you in advance
Vaishak P
Comments (25)
-
reporter -
reporter - attached backtrace.0.txt
-
reporter - attached run.err
-
reporter - attached run.out
-
reporter - edited description
-
- changed status to open
-
-
assigned issue to
- marked as major
-
assigned issue to
-
- attached qlm-ks.par
Reproducer parfile.
-
Summarizing my end of the conversation on mailing list (http://lists.einsteintoolkit.org/pipermail/users/2020-August/007565.html):
To make it fail I need more than 1 refinement level active. It fails on both 1 and 4 MPI ranks (that I have tested on).
Adding a bit of debug output:
std::cerr << "active1: " << active0 << " outputslab: " << outputslab << std::endl;
I get:
active1: bboxset<CCTK_INT4,3>(set<bbox>:{([12,12,12]:[136,136,136]:[4,4,4]/[3,3,3]:[34,34,34]/[32,32,32]/32768)},stride:[4,4,4],offset:[0,0,0]) outputslab: ([0,0,0]:[38,75,0]:[1,1,1]/[0,0,0]:[38,75,0]/[39,76,1]/2964) cactus_sim: /data/rhaas/postdoc/gr/cactus/ET_trunk/arrangements/Carpet/CarpetLib/src/bboxset2.hh:262: CarpetLib::bboxset2::bboxset<T, D> CarpetLib::bboxset2::bboxset<T, D>::binary_operator(const F&, const CarpetLib::bboxset2::bboxset<T, D>&) const [with F = CarpetLib::bboxset2::bboxset<T, D>::operator&<int, 3>::<lambda(const bboxset1&, const bboxset1&)>; T = int; int D = 3]: Assertion `all(stride == other.stride)' failed.
which seems to indicate the issue is really that one should not try to use "active" for anything other than grid functions by adding a:
if (groupdata.grouptype == CCTK_GF)
before the block of code that I suggested be commented out ("active" only makes sense for grid functions, since it refers to [basically] the non-buffer points due to mesh refinement).
-
-
Please review.
-
@Vaishak Prasad Turns out that you can also use:
iohdf5::out_vars = "QuasiLocalMeasures::qlm_newman_penrose"
which handles output of 2D arrays fine (it just cannot create a 2D slice through a 3D grid function).
-
reporter Hi,
iohdf5::out_vars = "QuasiLocalMeasures::qlm_newman_penrose"
Is not working. So are these:
QuasiLocalMeasures::qlm_shapes
QuasiLocalMeasures::qlm_coordinates
QuasiLocalMeasures::qlm_tetrad_l
QuasiLocalMeasures::qlm_tetrad_n
QuasiLocalMeasures::qlm_tetrad_m
QuasiLocalMeasures::qlm_killing_vector
QuasiLocalMeasures::qlm_killed_twometric
QuasiLocalMeasures::qlm_invariant_coordinates
QuasiLocalMeasures::qlm_3determinant
QuasiLocalMeasures::qlm_weyl_scalars
QuasiLocalMeasures::qlm_ricci_scalars
QuasiLocalMeasures::qlm_twometric
Out of these (qlm_tetrad_m, qlm_weyl_scalars and qlm_newman_penrose) are complex variable types.
The run exists with signal 6 with the error:
terminate called after throwing an instance of 'std::out_of_range'
what(): vector::_M_range_check: __n (which is 0) >= this->size() (which is 0)
Thanks
-
- attached qlm-ks-outvars.par
I cannot reproduce that failure.
The attached file qlm-ks-outvars.par uses
IOhdf5::out_every = 1 iohdf5::out_vars = " QuasiLocalMeasures::qlm_newman_penrose QuasiLocalMeasures::qlm_shapes QuasiLocalMeasures::qlm_coordinates QuasiLocalMeasures::qlm_tetrad_l QuasiLocalMeasures::qlm_tetrad_n QuasiLocalMeasures::qlm_tetrad_m QuasiLocalMeasures::qlm_killing_vector QuasiLocalMeasures::qlm_killed_twometric QuasiLocalMeasures::qlm_invariant_coordinates QuasiLocalMeasures::qlm_3determinant QuasiLocalMeasures::qlm_weyl_scalars QuasiLocalMeasures::qlm_ricci_scalars QuasiLocalMeasures::qlm_twometric "
and runs fine for me. Namely:
mpirun -n 2 exe/cactus_sim qlm-ks-outvars.par
produces a file
qlm-ks-outvars/qlm_npmu[0].h5
which contains a dataset:QUASILOCALMEASURES::qlm_npmu[0]\ it=4\ tl=0 Dataset {76/76, 39/39} Attribute: carpet_mglevel scalar Type: native int Data: 0 Attribute: cctk_bbox {4} Type: native int Data: 1, 1, 1, 1 Attribute: cctk_nghostzones {2} Type: native int Data: 0, 0 Attribute: group_timelevel scalar Type: native int Data: 0 Attribute: iorigin {2} Type: native int Data: 0, 0 Attribute: level scalar Type: native int Data: 0 Attribute: name scalar Type: 32-byte null-terminated ASCII string Data: "QUASILOCALMEASURES::qlm_npmu[0]" Attribute: time scalar Type: native double Data: 0.05 Attribute: timestep scalar Type: native int Data: 4 Location: 1:58684 Links: 1 Chunks: {76, 39} 47424 bytes Storage: 47424 logical bytes, 19682 allocated bytes, 240.95% utilization Filter-0: shuffle-2 OPT {16} Filter-1: deflate-1 OPT {6} Type: struct { "real" +0 native double "imag" +8 native double } 16 bytes
Can you try and modify the file qlm-ks-outvars.par to fail in the way you observe, so that there is a quick reproducer?
-
Also, if you could include out and err file as well as backtrace for the failure, that would be helpful.
-
reporter I am able to run the par file 'qlm-ks-outvars.par' as is. I will try to reprouce the error and report back to you.
-
reporter I am attaching the backtrace, err, out, par files for the gallery run. I am still working on qlm-ks-outvars.
-
reporter - attached GWtrial2-0000.e50700
-
reporter - attached GWtrial2-0000.o50700
-
reporter - attached backtrace_GWtrial2_11.txt
-
reporter - attached GWtrial2.rpar
-
reporter I could not get qlm-ks-outvars to fail in the way the bbh case does. I am able to output QLM variables of any dimensionality.
I believe the bbh case is failing due to some other reason.
-
Unless objected I will apply https://bitbucket.org/eschnett/carpet/pull-requests/38/carpetiohdf5-only-store-active-region-for after 2020-09-24.
-
-
- changed status to resolved
- Log in to comment