- changed status to open
- removed comment
add tags to disallow splitting in directions
the attached patch adds a new tag "no_split_directions" which can be used to prevent a grid array to be split in the given directions when processor decompositions is done. I currently explicitly forbid this tag to be used in grid functions since all grid functions share the same processor decomposition so using this tag on a per-function basis is impossible.
Keyword:
Comments (5)
-
reporter -
- changed status to open
- removed comment
The statement
bvect const dims = not split_components || no_split_dims;
seems to be the pivot point of the whole implementation. I assume you tested it... too lazy to figure out whether this is correct.
Please apply.
-
reporter - changed status to resolved
- removed comment
Hmm. We have tested that the tag works with grid arrays (in the thorn for which this functionality was introduced, "Replay" in Zelmani if you are curious). I had not tested whether I had broken split_components. Let me see.
Hmm, alright, I am tempted to claim that Carpet::split_components = "no" is not usable right now. Setting it triggers an assert(nprocs == 1) in SplitRegionsMaps_Automatic_Recursively unless CCTK_NProcs() == 1 in which case splitting components makes no sense anyway (since all(dims) == true). dims is set to all true just before we enter SplitRegionsMaps_Automatic_Recursively (namely in line 1715):
for (int r=0, p=0; r<nregs; p+=mynprocs.AT(r), ++r) { if (recompose_verbose) cout << "SRMA superreg[" << r << "] " << superregs.AT(r) << endl; // bvect const dims = false; bvect const dims = not split_components || no_split_dims; SplitRegionsMaps_Automatic_Recursively (dims, p, mynprocs.AT(r), superregs.AT(r), newregs); } // for r
I am no sure how to construct a parameter file to make this pass. Anyhow, I did not change its behaviour it seems :-).
I'll revive CarpetTest and make it use the tag for one of its arrays.
Committed as git hash 911ab59919a935b1d25146830673b2d200c74fed "Carpet: add new group tag no_split_directions".
-
reporter - removed comment
Hmpf. There was a bug left of course. Just such that for the choice of parameters in my original application it was producing the correct result. Fixed and added a test case to CarpetTest. Turns out that
for(int i = 0 ; i < nvals . ++i) { no_split_dims[i] = true; }
is identical to
for(int i = 0 ; i < nvals . ++i) { no_split_dims[no_split_directions[i]] = true; }
if
no_split_directions = {0,1
}. -
reporter - changed status to closed
- edited description
- Log in to comment