Carpet: npoints[d] <= max issue

[Lunan Sun]

Hi,

I have set my dx=0.015 with xmax/xmin = 15/-15, which results 2000 grid points across.

Then I get the following message:

void Carpet::calculate_grid_points(int, const vect<vect<int, 3>, 2> &, const vect<double, 3> &, const vect<double, 3> &, const vect<double, 3> &, vect<int, 3> &): Assertion `npoints[d] <= max' failed.

It seems that my number of grid points has exceed the INT_MAX limit. What’s the best way to solve this if I prefer the preserve the grid size?

‌

Thanks

[Lunan Sun]

Some additional information (not sure if it’s useful)

Cluster: Stampede2

Nodes:128

MPI processes:512

Also, I have been decreasing the grid boundary for tests. And I find that the code can run with the boundary set at 12.3 but failed at 12.6, with the setup above.

[Erik Schnetter]

Lunan

Cactus (and Carpet) use int to access arrays. This means that the maximum number of grids points is limited to be about 2e9.

Parts of Carpet have been extended to use 64-bit integers. I think that many parts of Carpet will work correctly as long as the process-local grid size is less than 2e9. Unfortunately, there is no good way to flag integer overflow errors in C or C++, and it is thus difficult to debug this.

You might get lucky if you simply disable this check. There are other checks for process-local grid sizes that would trigger if things go wrong. If you don’t get lucky, then there will be weird errors. I assume that most such errors will affect “unimportant” things such as calculating the average over a full grid function, or estimating the current progress rate in terms of number of grid points per second that are evolved.

If you are familiar with C/C++ and integer overflow, then this is worth an attempt. You can also build a version of your code with compiler options that check for signed integer overflow to see whether this triggers an error. I’ll be happy to assist if needed.

If you are not familiar with this, then I would not advise trying this, because the C/C++ standards are not exactly user friendly. In this case, using a smaller domain is the only way to go. You might try using mesh refinement, which generally leads to smaller numbers of grid points.

We’re currently working on a new driver “CarpetX”. We should make sure there are no respective limitations there.

-erik

[Lunan Sun]

Thanks Erik,

I will try to disable the check. If it doesn't work I can live with a smaller grid or mesh refinement.

-Lunan

[Roland Haas]

@Lunan Sun is this still an issue?

[Lunan Sun]

Hi @Roland Haas Nope you can close it. Thanks!