DefaultDataAccess class crashes in case that two files of same vertical coordinate type have different number of levels.

Issue #282 resolved
Joern Ungermann created an issue

The consistency check of the data access class tries to compare the given vertical levels of all files and skips a file if they are not consistent. Sadly, the comparison did not take into account a differing number of levels. The comparison for different values thus crashes in case that a differing number of levels is encountered, This is obviously not intended as the desired behaviour here is also an error message and a skipping of the file.

Comments (4)

  1. Reimar Bauer

    crashing is bad, bad also we should have some method to sort off wrong files. May be some descriptive meta file based on yaml or json to setup a minimum requirements. And everything what did not match becomes ignored.

  2. Joern Ungermann reporter

    Indeed, the crash is the error, the desired behaviour is skipping of the file PLUS error in the log. As I see your comment, this is not clear. I'll update the issue description.

    With respect to the descriptive meta file:

    One gives the access class a "tag" that is used for identifying files. Files not matching this tag are not used. This is a very simple mechanism, with which one may have multiple associated filetypes in one directory. For example, it allowed the pressure-level and theta-level files of ClAMS to be in two different data groups that allow for differing init_times for the groups.

  3. Reimar Bauer

    Properly treat model files with differing vertical levels.

    The data access class checks for consistency between discovered model data files. In case of a different number of vertical levels in two files, MSS Crashes as the comparison by np.allclose does not likle to arrays of different shape. A second checks before now checks for different shapes.

    Fix issue #282

    → <<cset f0dedc60bcdd>>

  4. Log in to comment