Wiki
Clone wikiGrass Data Explorer / Home
Welcome to Grass Data Explorer documentation
Copyright:
2015-2016 Thünen Institutes of Climate-Smart Agriculture
Author:
Sören Gebbert
soerengebbert [at] googlemail [dot] com
https://www.ti.bund.de/en/ak/staff/wissenschaftliches-personal/dipl-ing-soeren-gebbert/
Requirements
The Plugin works only with the latest GRASS GIS 7.1 svn version.
To use this Plugin, QGIS must be started from within an active GRASS GIS 7.1 session.
The following Python libraries must be installed:
- matplotlib with Qt4 support (python-matplotlib-qt4 and maybe python-matplotlib-qt5)
- numpy
- PyQt4-devel
On Ubuntu 14.04 make sure that the following packages are installed:
- python-qt4-dev
- python-qt4
- pyqt4-dev-tools
- python-sphinx
- python-matplotlib
Installation
Clone the repository:
git clone https://bitbucket.org/huhabla/grass-data-explorer.git
Switch into the Plugin directory and run make to deploy the Plugin in the QGIS plugin directory:
cd grass-data-explorer make deploy
Then start QGIS from within a GRASS GIS session. Enable this Plugin in QGIS Extension manager.
A new GRASS GIS icon should appear in the toolbar.
Introduction
The Grass Data Explorer QGIS plugin was designed to explore raster, vector and in particular time series data of a GRASS GIS location in a fast way. It is not designed to replace the native GRASS GUI nor the existing GRASS GIS plugin for QGIS. Hence vector digitizing or direct vector and raster manipulation is not supported. However, the integrated GRASS module browser allows fast access to all GRASS processing modules and their documentation.
It is a tool for GRASS GIS poweruser to investigate large spatio-temporal datasets in GRASS GIS locations.
Important Notes
QGIS must be started from within an active GRASS session.
This plugin can only show data from a single GRASS location. It provides access to all mapsets in this location that are in the search path of the active mapset. Switching mapsets within the location is only supported by the provided GUI component of the plugin. Changing the mapset by hand (g.mapset) will put the plugin out of work.
The Grass Data Explorer does not track any changes in the GRASS database. Deletion, renaming, modification etc. of GRASS layers or the change of current region settings are not automatically recognized by this plugin. Hence, many dialogs provide reload, refresh or list buttons to keep the plugin up-to-date with the GRASS database.
GRASS data handling in QGIS
The Grass Data Explorer uses GRASS GIS python libraries to transfer data from GRASS into QGIS. It is fully implemented in Python and makes use of the QGIS Python API, the GRASS GIS temporal framework Python API, PyGRASS and the GRASS scripting library. It demonstrates the capabilities of the different GRASS GIS Python API's.
GRASS Raster layer
GRASS Raster layers are rendered in a dedicated GRASS process using their native resolution, independently from the current GRASS region settings. The extent of the QGIS canvas and its pixel resolution is used to read the visible part of a GRASS raster layer. The resulting RGB image is then transferred to the QGIS rendering system as a QImage. This approach allows a fast rendering of huge raster map layer with sizes beyond 2 billion pixel. However, the QGIS raster setting and styling approach can not be used, since the raster layer is rendered in GRASS. The native GRASS raster color table is used for rendering and symbol creation. Therefore the Grass Data Explorer provides a dedicated settings dialog for GRASS raster layer that can be used to sample the raster layer and to set the GRASS color table and the opacity.
GRASS raster layers are handled via the QgsPluginLayer mechanism.
GRASS Vector layer
GRASS GIS vector layers are converted into the QGIS memory layer format. This allows the use of the QGIS vector layer rendering and analysis tools. Hence, the power of the QGIS vector layer styles can be used to render GRASS GIS vector layer. Be aware that huge vector layers will consume huge amount of main memory. The memory layers can be modified. However, the modified memory layers will not affect the original GRASS vector layers. Memory layers can not be directly written back to GRASS.Raster time series layer (STRDS)
Raster time series layers (Space Time Raster Datasets [STRDS] in GRASS GIS) are collections of time stamped GRASS raster layers. The time stamped raster layers may have time instances or time intervals attached to them. The Grass Data Explorer provides a unique approach to visualize time series data. When a STRDS is loaded into QGIS the temporally first raster layer of the collection is visualized. The STRDS settings dialog allows the selection of an arbitrary raster layer of the collection to be visualized instantly. The arrow keys can be used to switch between raster layers.
GRASS raster time series layers are handled via the QgsPluginLayer mechanism and uses the same rendering mechanism as the GRASS raster layer approach.
Vector time series layer (STVDS)
Vector time series layers (Space Time Vector Datasets [STVDS] in GRASS GIS) are collections of time stamped GRASS vector layers. Vector time series layers are handled like Raster time series layers. A single time stamped layer of the collection can be selected in the dedicated settings dialog for immediate rendering.
Selected vector layers are loaded as memory layer in QGIS. These alyer are cached to avoid rereading of already loaded layers. The number fo layers in stored in the memroy cache can be adjusted, to avoid heavy memory usage.
GRASS vector time series layers are handled via the QgsPluginLayer mechanism and uses the same rendering mechanism as the GRASS vector layer approach.
Graphical user interface
The main window
The main window provides access to GRASS GIS general information, mapsets, raster, vector and time series layers as well as the module browser and time time series rendering. The main windows provides a Reload all button to update all information that are readed from the GRASS GIS location. These are:
- General information about the location, projection and mapsets
- All accessible raster, vector and time series layers
- Available modules specified in GRASS GIS GUI related XML files
- Region settings
The button Reload all will reload the information of all dialogs in the main window. This may take some time in case of tens of thousands of layers.
General Info
The General Info dialog shows information about the current location and its mapsets. It show the current region settings, the location specific coordinate reference system, GRASS specific environmental variables and GRASS GIS build information. A Refresh button can be used to read this information again, just in case something changed in the GRASS database.GRASS Module Browser
The GRASS module browser is designed to search, inspect and start modules to process GRASS data. Modules are organized by a two fold keyword hierarchy. Modules can be searched using the search line input. The search is performed instantly, checking the name of the modules, the keywords and its short description. Hence, the search result will be shown each time a letter was typed into the search line.
Search phrases are separated by spaces. If two phrases are entered, then the modules must contain booth phrases in the module name, keyword list and description.
A single click on a module name will show its manual in the documentation browser at the right side. A double click will start the module graphical user interface.
GRASS Region Settings
This dialog allows the management of GRASS regions. The current region can be loaded and manipulated. The spatial extent of the QGIS map canvas or the currently active layer in the QGIS layer tree can be loaded and manipulated. The region settings can then be set as the current computational region in GRASS.Raster Layer
The Raster Layer dialog lists all accessible GRASS raster layer from different mapsets using a tree layout. The raster layers are sorted by mapset and name. The list of raster layers can be searched using the Pattern option. The list maps button will reread all accessible raster layer applying the provided search pattern.
Metadata information and a preview picture are shown on the right side for the selected raster layer. This information will be updated if a layer is selected by mouse of by the arrow keys.
The load button will load the currently selected layer into QGIS.
Raster Time Series
The Raster Time Series dialog lists mapset specific all accessible space-time raster datasets and their associated raster layer using a tree layout. The datasets and their associated raster layers are sorted by mapset and name. This dialog is designed to load space-time raster datasets as well as single raster layer into QGIS. A WHERE statement can be used to search for specific space-time raster datasets in the temporal database of GRASS. The list time series button will list accessible datasets and their associated raster layer applying the where statement.
Information about the selected space-time raster dataset or raster layer is available at the right side. This includes a preview picture, which is in case of a selected space-time raster dataset its temporally first raster layer.
The load button will load the currently selected space-time raster dataset or raster layer into QGIS.
Raster Sampling
Selecting this dialog will start the GRASS raster layer map tool. This will allow to query all raster and STRDS layer that were loaded into QGIS using the mouse. The result will be shown in the dialogs information text window for each layer. The current GRASS region is used for sampling.Vector Layer
The Vector Layer dialog lists all accessible GRASS vector layer from different mapsets using a tree layout. The vector layers are sorted by mapset and name. The list of vector layers can be searched using the Pattern option. The list maps button will reread all accessible vector layer applying the provided search pattern.
By default, a GRASS vector layers is converted into a QGIS memory layer using its dominant feature type. GRASS vector layer can contain different feature types at the same time (point, line, boundary area). The feature dominance is defined by this order:
- area
- boundary
- line
- point
This can be adjusted in the load settings. In addition the vector layer can be loaded with and without attribute data.
The current canvas extent can be used to limit the number of vector features from the source GRASS vector layer. Only features that intersect with the current extent or located within it are loaded as memory layer.
Metadata information is shown on the right side for the selected vector layer. This information will be updated if a layer is selected by mouse of by the arrow keys.
The load button will load the currently selected layer into QGIS.
Vector Time Series
The Vector Time Series dialog lists mapset specific all accessible space-time vector datasets and their associated vector layer using a tree layout. The datasets and their associated vector layers are sorted by mapset and name. This dialog is designed to load space-time vector datasets as well as single vector layer into QGIS. A WHERE statement can be used to search for specific space-time vector datasets in the temporal database of GRASS. The list time series button will list accessible datasets and their associated vector layer applying the where statement.
The same settings for feature type, attributes and extent are available as for the vector layers.
Information about the selected space-time vector dataset or vector layer is available at the right side.
The load button will load the currently selected space-time vector dataset or vector layer into QGIS.
Synchronized Time Series
GRASS time series of type raster (STRDS) and vector (STVDS) that have been loaded into QGIS can be synchronized to visualize them synchronously. STRDS and STVDS that should be synchronized must be selected in the dialogs time series layer list widget. Clicking the Synchronize button will compute the greatest common temporal granuarity between all selected space-time datasets. The result is shown in the map list widget. Selecting a row in the list widget will show all layer listed in this row in the time series layers.
A WHERE statement can be provided to select a subset of all space-time datasets for synchronization computation.
Be aware that combining of time series with yearly granualrity and granularities smaller than a day for synchronization will result in huge lists of layer and will require a significant amount computational time.
Render Synced Time Series
Synchronized time series can be exported as list of images to create animations. As first requirement the composition of the resulting images must be provided as QGIS composer template file. This file specifies the layout of the time series-, raster- and vector- layers, legends, annotations, ... as well as the size and the pixel resolution of the resulting images.
Compositions can be created using the composer dialog of QGIS. A composition must be saved as composer template file (not as QGIS project file).
Set the path to this file in the Composer template file dialog.
The second requirement is the path to the resulting image files, for example "/tmp/image". The resulting PNG images will be named /tmp/image_000001.png up to /tmp/image_99999.png, depending on the number of synchronized raster/vector layers.
The progress of the rendering will be continuously updated. Path, temporal information and participating layers of the currently rendered image are displayed as well as an image preview.
Use the WHERE statement in the synchronisation dialog, to select a small subset of synchronous layers to test the rendering results.
Important
Be aware that the rendering will run in the main thread of QGIS for technical reasons. Hence, you can't use QGIS while the rendering takes place. You have to kill QGIS to abort the rendering process.Note
Use textual placeholders in your composition that will be replaced by date and time information of the rendered time series layers. Supported placeholder for the temporal granule are (specify with squared brackets):
- [START_DATE] -> the start date, format YY-mm-dd HH:MM:SS
- [END_DATE] -> the end date, format YY-mm-dd HH:MM:SS
- [START_YEAR] as 4 digit integer
- [START_MONTH] as two digit integer
- [START_DAY] as two digit integer
- [START_HOUR] as two digit integer
- [START_MINUTE] as two digit integer
- [START_SECOND] as two digit integer
- [END_YEAR] as 4 digit integer
- [END_MONTH] as two digit integer
- [END_DAY] as two digit integer
- [END_HOUR] as two digit integer
- [END_MINUTE] as two digit integer
- [END_SECOND] as two digit integer
Raster layer settings
GRASS raster layer that have been loaded into QGIS have a dedicated settings dialog, that is significantly different from the QGIS common raster layer settings dialog.
Raster Info
Information about the loaded raster layer including categorical data if available.Opacity and Color
This dialog can be used to set the GRASS color table and the opacity of the layer redered in QGIS.Raster Statistics
This dialog can be used to compute a simple histogram of the raster layer. The input of the graph is provided by the GRASS module r.stats. The number of steps can be specified, default is 200. Click the Generate button to compute the history graph.
Note
Be aware this computation is blocking the main thread and cn not be aborted. However, the computation is usually fast even for large raster layers. The computation is performed using the native resolution of the raster layer, rather then using the current GRASS region.Raster Sampling
Selecting this dialog will start the raster layer sampling map tool. Clicking in the QGIS map canvas will sample this raster layer at the canvas click position and shows the result in the sampling dialog. The sampling is performed using the native resolution of the raster layer.
Raster time series settings
Each space-time raster dataset has its own settings that allow the selection of the currently visible raster layer, to set the color table, to statistically analyze the time series and to sample it.
STRDS Info
Metadata information about the loaded space-time raster dataset can be investigated in this widget. The information is similar to the output of the t.info module.List of Layers
All or a specific subset of raster layers that are contained in the space-time raster dataset can be listed here. Clicking on a layer will show it in the QGIS map canvas. Use the arrow up and down keys to switch between several raster layer.
A WHERE statement can be used to select a subset of layers, examples:
-- Select all layers that have a start_time that is later 1979 start_time > '1980-01-01' -- Select all layers that have a maximum over 25 max > 25 -- Select all layers that have a maximum over 25 in the time frame 1980 - 1990 start_time >= '1980-01-01' AND end_time <= '1990-01-01' AND max > 25Opacity and Color
The opacity for the space-time raster dataset as well as the the color table can be set in this dialog.Raster Statistics
Simple raster statistics can be computed for the currently selected raster layer. The functionality is similar to the raster layer dialog.Raster Sampling
Query the currently selected raster layer with the mouse in the map canvas. The functionality is similar to the raster layer dialog.Time Series Statistics
Simple time series statistics can be computed for the space-time raster dataset. Use the 'WHERE' statement to select a subset of raster layers for computation. Deselect the use raster region to use the current GRASS region settings for each raster layer to compute the time series statistics. Click the Generate button to compute the time series statistics. Be aware that this computation may take a while, depending on the number of selected raster layers and the size of the raster layers. The computation process is running in the background and can be aborted by clicking the Abort button that was formerly the Generate button.
A simple regression line and a fitting function with power of 5 are computed and visualized to get an impression of the time series trend.
The result of the computation is visualized in several diagrams. The raw data is available in the table tab for further investigation.
Time Series Sampling
Clicking at this dialog will enable the time series map tool. The space-time raster dataset can then be queried in the map canvas using the mouse. The resulting list of values is displayed as graph. The raw data is available in the table tab widget.
Vector time series settings
Each space-time vector dataset has its own settings that allow the selection of the currently visible vector layer, to set the rendering style, to sample the currently visualized vector layer and to manage the vector layer cache.
STVDS Info
Metadata information about the loaded space-time vector dataset can be investigated in this widget. The information is similar to the output of the t.info module.List of Layers
All or a specific subset of vector layers that are contained in the space-time vector dataset can be listed here. Clicking on a layer will show it in the QGIS map canvas. Use the arrow up and down keys to switch between several vector layer.
Clicking the right mouse button will show the context menu to load a single vector layer into QGIS. This can be used to create QGIS style files that can be used to set the rendering style of the time series.
A WHERE statement can be used to select a subset of layers, examples:
-- Select all layers that have a start_time that is later 1979 start_time > '1980-01-01' -- Select all layers that have more than 25 areas areas > 25 -- Select all layers that have more than 25 areas in the time frame 1980 - 1990 start_time >= '1980-01-01' AND end_time <= '1990-01-01' AND areas > 25Styles
Styles that have been generated for a single vector layer can be used to define the rendering style of the whole time series. In this dialog the rendering and label style can be set by providing the paths to QGIS style files.Vector Sampling
Selecting this dialog will start the vector time series map tool to query the currently selected vector layer with GRASS functionality. The result is displayed in this dialog as textual output.
The table index (GRASS vector layer) can be selected to query different tables that may be associated with the currently selected vector layer. The sampling distance can be adjusted to find the nearest neighbour feature.
Vector Layer Cache
The number of memory layers that should be kept in memory can be adjusted here. It will show all memory layers that have been already loaded into memory. The cache can be cleared and its size be adjusted.
Updated