HTTPS SSH

Changelog

  • 2018-04-29, RS: Fixed few small things.
  • 2015-01-16, RS: Created the script from scratch
  • 2015-01-17, RS: Got operational. Also on wetterturnier server.
  • 2015-08-05, RS: Removed os.chdir stuff.

GENSvis (GENSvisualizer)

Visualizing GFS ENSEMBLE (GENS) meteograms. The script is currently running on the ertel2 and on the prognose2.met.fu-berlin.de wetterturnier server with slightly different configs.

Requirements

  • python (currently python 2.7+)
  • numpy
  • matplotlib
  • pydap (python OpenDAP client). Warning: there is an issue with Pydap 3.2.2 reading the OpenDAP data files (results in an error invalid start byte, see https://github.com/pydap/pydap/pull/152). Works well with latest version from github (version Feb 6, 2018).

Installation

Please note that this small tool is not yet a proper python package. I've written that before my python-packaging-time. To get the scripts run simply clone the repository, set up the .config files (see below) and start the script (tested for python 2.7+).

Data source

Using the NCEP nomads OpenDAP servers to load the GFS ensemble data. Based on a given station list the script is downloading segments (areas), does the interpolation and automatically creates the necessary figures.

Output

Creates figures located in the 'imagedir' (see config section [image]). At the moment two files per station are produced based on the same data but with different probabilistic plot styles. The plotting routine was written by Felis Schueller, the rest by Reto Stauffer 2015.

Configuration files

There is a config.conf.template file in the repository as well as some hostname specific files. These are used by different servers. The procedure:

  • If there is a <hostname>_XXXX_config.conf file the script is using this script file.
  • If no hostname specific file is available, the fallback procedure is to search for config.conf. If not existing, stop.

The XXXX can be either GENS (for GENSvis.py) or GFS (for GFSvis.py). Please modify 'all' config files which are in the repository so that I can easily pull/push when updating the tool on one of the servers where the script is in use!!!

Usage of the script

WARNING: here only the GENSvis.py script is described. There is also a second script called GFSvis.py. While the first is processing the GENS (Gfs ENSemble) the latter one is processing the deterministic GFS forecasts. The two scripts are both based on the same methods and the biggest part of the two classes GENSdap and GFSdap is based on GENERALdap. Usage is identical, output procedure identical, data and output format different.

The main usage is simple: - python GENSvis.py

What happens? The script goes to the nomads server and searches for the latest GENS forecast run. If found the script is checking the local 'checkfile' (see config file, [settings] section). If the file does not exist - download/visualize the data. If the 'checkfile' exists the content will be read. The content is of form '%Y-%m-%d %H:%M' containing the last visualized forecast run. If the current (latest) forecast is equal or older the script will exit. If not, the new run will be visualized. At the end the 'checkfile' will be updated. This allows to start the script every 10-15 minutes without revisualizing the same data twice (operational mode).

There are some input options to control the script. One used operationally is the 'stationlist' input -s/--station. WARNING: it is NOT the 'statlist' file as defined in the config. The input -s/--statoin is only the 'HASH' prefix of the station list. If given, the script is expecting a file called HASH_statlist.csv located in the 'statlistdir' as defined in the config file ([settings] section). If not existing, stop. The script then behaves like not given any -s/--station input with the exception that the 'checkfile' will be station HASH dependent. If the 'checkfile' is defined as GENS.lastrun in the config file a run with -s/--statlist will create a HASH_GENS.lastrun file. This allows to run several different regions around the globe in an operational mode. Why? Think about having one station in South America and one in Europe. If you put both in one statoin list the script has to load half of the globe to visualize these two locations. When starting twice (two separate station lists) we only have to download 8 grid points - the four nearest neighbors for these two stations. As an example an EU (Europa) and US (U.S.) station list: - python GENSvis.py -s EU - python GENSvis.py -s US

Furthermore there is the option to recompute a meteogram given a -d/--day and a -r/--runhour. NOTE both have to be given. If not, the script will stop. If these are given the 'checkfile' has no influence - the job will recompute the stations even if the run was allready done somewehen in the past. Of course, these inputs can also be combined with the -s/--statlist input. If no files are available on the nomads server for the given date and runhour (should be 0/6/12/18) the script will .. probably crash. I guess I have no check there. - python GENSvis.py -d 20150117 -r 12 - python GENSvis.py --day 20150117 -runhour 12 - python GENSvis.py -d 20150117 -r 12 -s EU