In fields implicit read_response reads each response file on proc0 and then communicates row by row to the processor which owns this in a blocking way. There’s some suggestion that this may be very slow in some situations. Each processor currently allocates storage to hold the full response matrix anyway so we could instead just allow all processors to read directly from file.
Ideally we may just read the relevant rows of the matrix rather than the entire matrix. Independent of all procs reading the file one could imagine proc0 reading the matrix row by row to avoid having to read the entire matrix into memory at once. This would help avoid potential memory limitations. Ideally we may also write matrices row by row. This would help ensure that fields implicit isn’t subject to the same memory limitations which impact fields local. Just reading the relevant processor local data from the file may also help fields local but likely more challenging to implement.
PR #1204 added the
read_response_communicateinput flag. If set to false then we allow multiple processors to read the required data rather than having a single process read and then communicate this data.This can significantly accelerate the restoration of response data from file.