Server is too slow when loading in many netCDF files

[Joern Ungermann]

The current implemtation scans for files at each getCapability call. This enables the discovery of new or chaned data files. This works reasonably well also for a large number of files in case that all files are in the file cache. After some time, the files get purged frm the cache and a new call requires the loading of all files from disk.

If many files need to be loaded from disk, the discovery process takes longer than the client waits for. This causes an error message in the client. A second try to retrieve the capabilities is typically succesful.

I propose to reimplement a simple in-memory caching based on filename and modification time. I.e. access only netCDF files that have changed since the last access.

With this change only the initial get capabilities call after a server restart would take long, which can be performed manually by the administrator. Later calls will need to access only the smaller set of changed/updated files.

While not perfect, this would be an improvement over the current state.

[Reimar Bauer]

Draft for a simple cached data access class

Requires a long time for initial read-in, but reuses info about previously read in files. Even with files being in file-cache this is an order of magnitude faster.

Fix issue #423

→ <<cset ae46f241305b>>