Improve the response matrix dump/read functionality
Both fields_implicit and fields_local over the ability to save and load the response matrices to/from file. Whilst this works it is not as useful as it could be.
The current problems are:
- It only currently makes sense to have one of dump/read response true in a particular run typically.
- In non-linear runs response matrix files will be overwritten each time we change the timestep
- There’s no information about what timestep was used to generate a particular file.
- Others
Proposed enhancements:
- Add the current timestep to the response matrix file name. This should allow us to identify the timestep used to produce a file.
- Restructure the code to allow us to test if the expected response matrix file exists before we try to read from it. This should allow us to enable read response without fear the run will abort because the file is absent, instead we can just calculate the response matrix directly. We could use similar functionality to avoid dump_response from overwriting files?
- Consider storing some meta-data in the response matrix file to better describe compatibility etc.
With proposals 1 and 2 we should be able to have dump_response = .true.
and read_response = .true.
in a nonlinear simulation and expect to be able to reuse previously calculated response matrix data. For example consider a nonlinear simulation that is oscillating between timestep sizes A and B. Currently each time we change the time step we must recalculate the full response matrices, with the proposed enhancements we would only calculate the response matrices for timestep size A once and then same for size B. This could be a decent saving for large simulations and should help us avoid much of the computation penalty of our implicit scheme in nonlinear simulations.
Challenges:
- This is a backwards incompatible change (change to filename).
- Testing
Comments (7)
-
-
reporter File size could be an issue but generally I’d have thought a set of response matrices for a single timestep shouldn’t be any more than
naky*available_memory_per_core
(and in general it can’t be any bigger thantotal_memory_consumed_by_gs2
) so sayavailable_memory_per_core = 4GB
andnaky = 128
we’re talking0.5TB
– this is a lot and we’d need to multiply this by say 4 different timesteps. It’s a lot but not unrealistic for systems where you’re going to be running these large cases. There’s also a program that just generates the dump files for different timesteps – we could use this to precalculateN
timesteps and then only useread_response = .true.
(withdump_response = .false.
) in the main run. This would ensure no additional disk space is required, i.e. we won’t write any extra ones. -
reporter Work started in branch feature/improve_response_matrix_dumping_issue_96 . Currently it implements the first proposed enhancement and fixes a couple of bugs. I hope to soon work on the second enhancement.
-
reporter The branch now contains code to check if the expected response matrix exists and if not quietly carry on with calculating the response matrix instead (this is reported to the error unit). I think this should be enough to get enhancement 2 but I’ve yet to develop a test for this.
-
reporter It might also make sense to skip dumping the response matrix if a file with the expected name already exists. This might have slight performance advantages but should also help reduce peak memory requirements for big jobs that wish to dump the response matrix.
-
reporter PR 293 implements pretty much all of this now. If that is approved and merged I think that will close this issue.
-
reporter - changed status to resolved
Resolved by PR #293 which adds the key features noted here along with some bug fixes.
- Log in to comment
This seems like a very good idea! Is the size of response matrices also a challenge? Is storing multiple copies on disk likely to be a problem?