CarpetIOHDF5 commit 69c73fb1 " CarpetIOHDF5: Add "active" attribute to grid function dataset describing which points are active" breaks 'IO::out_mode = "onefile"'

Create issue
Issue #2117 resolved
Roland Haas created an issue

Setting

IO::out_mode            = "onefile"

in a parfile and running with more than 1 MPI ranks causes and out-of-range array access in Output.cc in line 913:

    ostringstream buf;
    buf << (vdd.at(Carpet::map)
                ->local_boxes.at(mglevel)
                .at(refinementlevel)
---->           .at(local_component)
                .active);

due to local_component being -1. This is the backtrace:

#6  0x00007ffff3a03355 in std::__throw_out_of_range_fmt(char const*, ...) ()
   from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#7  0x00005555578f17ee in std::vector<dh::local_dboxes, std::allocator<dh::local_dboxes> >::_M_range_check (__n=<optimized out>, this=<optimized out>) at /usr/include/c++/7/bits/stl_vector.h:825
#8  std::vector<dh::local_dboxes, std::allocator<dh::local_dboxes> >::at (__n=<optimized out>,
    this=<optimized out>) at /usr/include/c++/7/bits/stl_vector.h:846
#9  CarpetIOHDF5::AddAttributes (cctkGH=cctkGH@entry=0x55555f829ab0,
    fullname=fullname@entry=0x5555634efb10 "TMUNUBASE::eTxx", vdim=3, refinementlevel=<optimized out>,
    refinementlevel@entry=0, request=request@entry=0x5555635801b0, bbox=...,
    dataset=360287970189640065, is_index=false)
    at /data/rhaas/postdoc/gr/cactus/ET_vanilla/configs/sim/build/CarpetIOHDF5/Output.cc:913
#10 0x00005555578f2a45 in CarpetIOHDF5::WriteVarChunkedSequential (cctkGH=cctkGH@entry=0x55555f829ab0,
    outfile=outfile@entry=72057594037928056, io_bytes=@0x7fffffffb960: 62212, request=0x5555635801b0,
    called_from_checkpoint=called_from_checkpoint@entry=true, indexfile=indexfile@entry=-1)
    at /data/rhaas/postdoc/gr/cactus/ET_vanilla/configs/sim/build/CarpetIOHDF5/Output.cc:513
#11 0x00005555578d9bdf in CarpetIOHDF5::Checkpoint (cctkGH=0x55555f829ab0, called_from=0)
    at /data/rhaas/postdoc/gr/cactus/ET_vanilla/configs/sim/build/CarpetIOHDF5/CarpetIOHDF5.cc:1218

To reproduce:

wget https://einsteintoolkit.org/gallery/ns/tov.par
OMP_NUM_THREADS=1 mpirun -n 2 exe/cactus_sim tov.par

Keyword:

Comments (10)

  1. Roland Haas reporter
    • changed status to open

    This causes a out-of-range error when checkpointing grid scalar and grid arrays b/c I was not careful enough checking under which circumstances the active region string was added before.

  2. Log in to comment