[Spectral View] wavelength information from GeetseEarthEngineRasterDataProvider not supported anymore

[Andreas Rabe]

We used to supported spectral profile plotting for GeetseEarthEngineRasterDataProvider WMS layer.

Wavelength information was provided via GeetseEarthEngineRasterDataProvider.wavelength:

‌

Currently, this information is not used anymore by the Spectral View.

We can still plot against Band Numbers…

‌

…but plotting against wavelength is not supported anymore:

‌

‌

[Andreas Rabe]

@Benjamin Jakimow I guess you changed something in the metadata handling and how wavelength information is requested.

Anything I should do different or will you just reconnect the Provider.wavelength logic?

[Andreas Rabe]

Hi @Benjamin Jakimow , here is a code snippet to reproduce the problem:

import numpy as np

from enmapbox.qgispluginsupport.qps.qgsrasterlayerproperties import QgsRasterLayerSpectralProperties
from enmapboxprocessing.driver import Driver
from qgis._core import QgsRasterLayer

# create layer that reports his wavelength view Provider.wavelength
Driver('test.tif').createFromArray(np.zeros((3, 1, 1)))
layer = QgsRasterLayer('test.tif')
provider = layer.dataProvider()
provider.wavelength = lambda bandNo: 42 + bandNo

# get the wavelength
props = QgsRasterLayerSpectralProperties.fromRasterLayer(layer)
print(props)
print(props.__dict__)

‌

The result only contains “mBandCount“ info:

<enmapbox.qgispluginsupport.qps.qgsrasterlayerproperties.QgsRasterLayerSpectralProperties object at 0x000001D445029E50>
{'mBandCount': 3}

‌

[Andreas Rabe]

Oh no, it’s more complicated than I thought.

It looks like you are caching the wavelength information when the layer is added.

In case of GEE TSE WMS layers, the wavelength information is set AFTER you already cached the information (i.e. no wavelength available)

Shall we have a spectralPropertiesChanged signal, that I can emit?

[Andreas Rabe]

Or better use the QgsRasterLayer.dataSourceChanged signal.

[Benjamin Jakimow]

I don’t understand what you mean with “caching”?
The spectral layer properties are read each time the user makes a mouse click. This is implemented in the SpectralProfileBridge.loadProfiles():

for uri, BLOCKS in SOURCE_BLOCKS.items():
    layer: QgsRasterLayer = URI2LAYER[uri]
    spectralProperties = QgsRasterLayerSpectralProperties.fromRasterLayer(layer)

In you code-snippet you create a layer with a gdal data provider. In that case it is expected the provider delivers metadata using the GDAL metadata model (the future C++ implementation of the QgsRasterLayerSpectralProperties will do that the same way):

In case of other providers (provider.name() != 'gdal') the current python prototype looks for your GEE “wavelength” implementation like:

        # wavelength() can be available for custom providers like EE
        if hasattr(provider, 'wavelength'):
            wl = np.array([provider.wavelength(bandNo) for bandNo in range(1, provider.bandCount() + 1)])
            wlu = 'Nanometers'
            self.setBandValues(None, 'wl', wl)
            self.setBandValues(None, 'wlu', wlu)

‌

[Andreas Rabe]

I don’t understand what you mean with “caching”?

Ah ok, I thought you are caching the spectral properties once and not asking the the Provider.wavelength() every time you need wavelength information.

Sorry,

In you code-snippet you create a layer with a gdal data provider. In that case the provider should deliver the metadata as known from GDAL, using the GDAL metadata model.

Ah ok, I will create a test with a WMS layer.

[Andreas Rabe]

Ok, here is a new test snippet:

‌

import numpy as np

from enmapbox import EnMAPBox
from enmapbox.testing import start_app
from enmapboxprocessing.driver import Driver
from qgis._core import QgsRasterLayer

qgsApp = start_app()
enmapBox = EnMAPBox(load_other_apps=False)

# create a layer that reports his wavelength via Provider.wavelength
Driver('test.tif').createFromArray(np.array(list(range(3*2*2))).reshape(((3, 2, 2))))
layer = QgsRasterLayer('test.tif', 'test')
provider = layer.dataProvider()
provider.wavelength = lambda bandNo: 42 + bandNo
provider.name = lambda: 'dummy'  # GDAL provider is not allowed to report his wavelength, so we overwrite the name

enmapBox.onDataDropped([layer])
qgsApp.exec_()

‌

It works as expected, when plotting against Band Numbers:

‌

It’s not working, when plotting against the wavelength:

‌

[Andreas Rabe]

Also outputs a warning:

‌

D:\source\QGISPlugIns\enmap-box\enmapbox\qgispluginsupport\qps\pyqtgraph\pyqtgraph\graphicsItems\ScatterPlotItem.py:974: RuntimeWarning: All-NaN slice encountered
  self.bounds[ax] = (np.nanmin(d) - self._maxSpotWidth*0.7072, np.nanmax(d) + self._maxSpotWidth*0.7072)
D:\source\QGISPlugIns\enmap-box\enmapbox\qgispluginsupport\qps\pyqtgraph\pyqtgraph\graphicsItems\ScatterPlotItem.py:974: RuntimeWarning: All-NaN slice encountered
  self.bounds[ax] = (np.nanmin(d) - self._maxSpotWidth*0.7072, np.nanmax(d) + self._maxSpotWidth*0.7072)
D:\source\QGISPlugIns\enmap-box\enmapbox\qgispluginsupport\qps\pyqtgraph\pyqtgraph\graphicsItems\ScatterPlotItem.py:974: RuntimeWarning: All-NaN slice encountered
  self.bounds[ax] = (np.nanmin(d) - self._maxSpotWidth*0.7072, np.nanmax(d) + self._maxSpotWidth*0.7072)
D:\source\QGISPlugIns\enmap-box\enmapbox\qgispluginsupport\qps\pyqtgraph\pyqtgraph\graphicsItems\ScatterPlotItem.py:974: RuntimeWarning: All-NaN slice encountered
  self.bounds[ax] = (np.nanmin(d) - self._maxSpotWidth*0.7072, np.nanmax(d) + self._maxSpotWidth*0.7072)

‌

[Andreas Rabe]

I also printed every time the wavelength are queried:

‌

That looks good. Prints as expected:

‌

call provider.wavelength
provider.wavelength: [43 44 45]

‌

[Andreas Rabe]

Ok, I think it’s save to say, that this has nothing to do with GEE TSE or WMS layer in general.

[Benjamin Jakimow]

You example does not work right because the monkey patched provider cannot be cloned.

[Andreas Rabe]

1. Do you get an error? I don’t.
2. Can we just skip the test for GDAL providers? That would make it much easier.

‌

[Benjamin Jakimow]

addresses #1270, fixes numpy data type JSON serialization

Signed-off-by: Benjamin Jakimow [e-mail] [e-mail]

→ <<cset 7f051ef48931>>

[Benjamin Jakimow]

@Andreas Janz Long story short. The wavelength information was lost during the JSON serialization (numpy data types are serialized to None). This should now be fixed.
Furthermore I fixed your test example using a proper data provider (snippets/issues/issue1270.py) which can be cloned.
Please check if this works with your GEE provider as well.

[Andreas Rabe]

Many thanks, I will test it

[Andreas Rabe]

Is working now , thanks @Benjamin Jakimow .