Reg. HDF5 output of QuasiLocalMeasures variables

Create issue
Issue #2451 resolved
Vaishak Prasad created an issue

Greetings

I am currently facing an issue when attempting to output QuasiLocalMeasures variables in HDF5 (in a BBH simulation similar to the gallery one) files instead of ASCII. I only face this issue when I request any QuasiLocalMeasures variables to be output in IOHDF5.

I am getting the same error message as in 005361 (http://lists.einsteintoolkit.org/pipermail/users/2017-March/005361.html). .

I am attaching the parameter file, the backtrace and the out and err files from the run. Requesting your help.

(ETK messages archive num. 007565)

Thanking you in advance

Vaishak P

Comments (25)

  1. Roland Haas

    Summarizing my end of the conversation on mailing list (http://lists.einsteintoolkit.org/pipermail/users/2020-August/007565.html):

    To make it fail I need more than 1 refinement level active. It fails on both 1 and 4 MPI ranks (that I have tested on).

    Adding a bit of debug output:

    std::cerr << "active1: " << active0 << " outputslab: " << outputslab << std::endl;
    

    I get:

    active1: bboxset<CCTK_INT4,3>(set<bbox>:{([12,12,12]:[136,136,136]:[4,4,4]/[3,3,3]:[34,34,34]/[32,32,32]/32768)},stride:[4,4,4],offset:[0,0,0]) outputslab: ([0,0,0]:[38,75,0]:[1,1,1]/[0,0,0]:[38,75,0]/[39,76,1]/2964)
    cactus_sim: /data/rhaas/postdoc/gr/cactus/ET_trunk/arrangements/Carpet/CarpetLib/src/bboxset2.hh:262: CarpetLib::bboxset2::bboxset<T, D> CarpetLib::bboxset2::bboxset<T, D>::binary_operator(const F&, const CarpetLib::bboxset2::bboxset<T, D>&) const [with F = CarpetLib::bboxset2::bboxset<T, D>::operator&<int, 3>::<lambda(const bboxset1&, const bboxset1&)>; T = int; int D = 3]: Assertion `all(stride == other.stride)' failed.
    

    which seems to indicate the issue is really that one should not try to use "active" for anything other than grid functions by adding a:

    if (groupdata.grouptype == CCTK_GF)
    

    before the block of code that I suggested be commented out ("active" only makes sense for grid functions, since it refers to [basically] the non-buffer points due to mesh refinement).

  2. Roland Haas

    @Vaishak Prasad Turns out that you can also use:

    iohdf5::out_vars = "QuasiLocalMeasures::qlm_newman_penrose"
    

    which handles output of 2D arrays fine (it just cannot create a 2D slice through a 3D grid function).

  3. Vaishak Prasad reporter

    Hi,

    iohdf5::out_vars = "QuasiLocalMeasures::qlm_newman_penrose"

    Is not working. So are these:

    QuasiLocalMeasures::qlm_shapes
    QuasiLocalMeasures::qlm_coordinates
    QuasiLocalMeasures::qlm_tetrad_l
    QuasiLocalMeasures::qlm_tetrad_n
    QuasiLocalMeasures::qlm_tetrad_m
    QuasiLocalMeasures::qlm_killing_vector
    QuasiLocalMeasures::qlm_killed_twometric
    QuasiLocalMeasures::qlm_invariant_coordinates
    QuasiLocalMeasures::qlm_3determinant
    QuasiLocalMeasures::qlm_weyl_scalars
    QuasiLocalMeasures::qlm_ricci_scalars
    QuasiLocalMeasures::qlm_twometric

    Out of these (qlm_tetrad_m, qlm_weyl_scalars and qlm_newman_penrose) are complex variable types.

    The run exists with signal 6 with the error:

    terminate called after throwing an instance of 'std::out_of_range'
    what(): vector::_M_range_check: __n (which is 0) >= this->size() (which is 0)

    Thanks

  4. Roland Haas

    I cannot reproduce that failure.

    The attached file qlm-ks-outvars.par uses

    IOhdf5::out_every = 1
    iohdf5::out_vars = "
    QuasiLocalMeasures::qlm_newman_penrose
    QuasiLocalMeasures::qlm_shapes
    QuasiLocalMeasures::qlm_coordinates
    QuasiLocalMeasures::qlm_tetrad_l
    QuasiLocalMeasures::qlm_tetrad_n
    QuasiLocalMeasures::qlm_tetrad_m
    QuasiLocalMeasures::qlm_killing_vector
    QuasiLocalMeasures::qlm_killed_twometric
    QuasiLocalMeasures::qlm_invariant_coordinates
    QuasiLocalMeasures::qlm_3determinant
    QuasiLocalMeasures::qlm_weyl_scalars
    QuasiLocalMeasures::qlm_ricci_scalars
    QuasiLocalMeasures::qlm_twometric
    "
    

    and runs fine for me. Namely:

    mpirun -n 2 exe/cactus_sim qlm-ks-outvars.par
    

    produces a file qlm-ks-outvars/qlm_npmu[0].h5 which contains a dataset:

    QUASILOCALMEASURES::qlm_npmu[0]\ it=4\ tl=0 Dataset {76/76, 39/39}
        Attribute: carpet_mglevel scalar
            Type:      native int
            Data:  0
        Attribute: cctk_bbox {4}
            Type:      native int
            Data:  1, 1, 1, 1
        Attribute: cctk_nghostzones {2}
            Type:      native int
            Data:  0, 0
        Attribute: group_timelevel scalar
            Type:      native int
            Data:  0
        Attribute: iorigin {2}
            Type:      native int
            Data:  0, 0
        Attribute: level scalar
            Type:      native int
            Data:  0
        Attribute: name scalar
            Type:      32-byte null-terminated ASCII string
            Data:  "QUASILOCALMEASURES::qlm_npmu[0]"
        Attribute: time scalar
            Type:      native double
            Data:  0.05
        Attribute: timestep scalar
            Type:      native int
            Data:  4
        Location:  1:58684
        Links:     1
        Chunks:    {76, 39} 47424 bytes
        Storage:   47424 logical bytes, 19682 allocated bytes, 240.95% utilization
        Filter-0:  shuffle-2 OPT {16}
        Filter-1:  deflate-1 OPT {6}
        Type:      struct {
                       "real"             +0    native double
                       "imag"             +8    native double
                   } 16 bytes
    

    Can you try and modify the file qlm-ks-outvars.par to fail in the way you observe, so that there is a quick reproducer?

  5. Roland Haas

    Also, if you could include out and err file as well as backtrace for the failure, that would be helpful.

  6. Vaishak Prasad reporter

    I am able to run the par file 'qlm-ks-outvars.par' as is. I will try to reprouce the error and report back to you.

  7. Vaishak Prasad reporter

    I am attaching the backtrace, err, out, par files for the gallery run. I am still working on qlm-ks-outvars.

  8. Vaishak Prasad reporter

    I could not get qlm-ks-outvars to fail in the way the bbh case does. I am able to output QLM variables of any dimensionality.

    I believe the bbh case is failing due to some other reason.

  9. Log in to comment