Spectral Library: wavelengths units are not read from hdr

[Agustin Lobo]

hdr indicates um, but at selecting um in the Spectral Viewer the X labels are wrong:

‌

[Andreas Janz]

I can’t confirm that on my testdata.

Please make sure that there isn't an .aux.xml sidecar file shadowing your ENVI header.

If so, delete it. Now GDAL will use the ENVI header again.

If the problem remains, please share your raster.

[Agustin Lobo]

Please make sure that there isn't an .aux.xml sidecar file shadowing your ENVI header.

If so, delete it. Now GDAL will use the ENVI header again.

Right, removing the xml solves the problem. But this is very inconvenient, as those xml get produced in the shadows.

‌

[Andreas Janz]

Unfortunately, this is QGIS/GDAL behaviour, which we can’t change.

The user has to be aware of it.

Shall we close this issue?

[Andreas Janz]

Note that the user will be able to change the wavelength inside the GUI with the Band Properties widget (see #913).

[Agustin Lobo]

No way that EnMapBox would enforce the hdr after the gdal reading?

An alternative, let the user set the data units in the spectral window itself.

As it is now, it will be an endless source of confusion. This kind of problems ensure user frustration.

[Andreas Janz]

No way that EnMapBox would enforce the hdr after the gdal reading?

I think that opening an ENVI raster in EnMAP-Box should behave the same as opening it in standard QGIS. On-the-fly manipulations will introduce problems.

An alternative, let the user set the data units in the spectral window itself.

With data units you mean Nanometers, Micrometers, etc?

‌

As it is now, it will be an endless source of confusion. This kind of problems ensure user frustration.

Yes, that is true. Users are already confused. The main idea behind having the Band Properties widget, is to have one place where the user can manage all the spectral (wavelength, fwhm, bad band), temporal (start time, end time) and other (band name, scale, offset) band properties.

This dialog could have additional convenience functions like: “Get reflectance scale factor from ENVI Header” or “Get spectral properties from ENVI Header” etc.

[Agustin Lobo]

An alternative, let the user set the data units in the spectral window itself.

With data units you mean Nanometers, Micrometers, etc?

Yes, spectral units, reflectance factor etc

‌

This dialog could have additional convenience functions like: “Get reflectance scale factor from ENVI Header” or “Get spectral properties from ENVI Header” etc.

Yes, this would be a very good way to enforce the hdr information. Best if this option could also be present

in the Spectral Library menu. Or even as a general user option (“Enforce hdr spectral information for ENVI format files”)

‌

[Andreas Janz]

Yes, spectral units, reflectance factor etc

Why would you need to select the wavelength units on your own? Currently the GUI always show Nanometers. Technically, each band has it’s own wavelength units, but we abstract from that in the GUI.

Shall we have another column, where the user can select the wavelength unit for each band? That would be most general.

Yes, this would be a very good way to enforce the hdr information.

We could do this. Technically we would read the information from the ENVI header and edit the table. After applying the changes, we would write into the GDAL *.aux.xml file. Now GDAL and ENVI information is in sync.

Shall we just have one button “Use ENVI Header information”? This would set the reflectance scale factor, the wavelength and the fwhm.

‌

Best if this option could also be present in the Spectral Library menu.

Not sure about that. I think the user should prepare its raster before using it in the EnMAP-Box.

‌

Or even as a general user option (“Enforce hdr spectral information for ENVI format files”)

Also not sure about that. If the EnMAP-Box would for example use the reflectance scale factor automatically, but native QGIS apps and algorithms won’t, wouldn’t that be confusing?

[Agustin Lobo]

Yes, spectral units, reflectance factor etc

Why would you need to select the wavelength units on your own? Currently the GUI always show Nanometers. Technically, each band has it’s own wavelength units, but we abstract from that in the GUI.

Currently, the user selects the units to be used in the plot. The problem is that the xml makes the data units not being read right, thus the units in the plot are wrong. See my original snapshot: units should have been read as um, but they have not, thus values in the x axis are wrong.

Shall we have another column, where the user can select the wavelength unit for each band? That would be most general.

For each band? This would be very tedious, you normally have the same units for all bands.

Shall we just have one button “Use ENVI Header information”? This would set the reflectance scale factor, the wavelength and the fwhm.

Yes!

Best if this option could also be present in the Spectral Library menu.

Not sure about that. I think the user should prepare its raster before using it in the EnMAP-Box.

Try to make user’s life easy. Sometimes you may want to invest time on these settings at the beginning of the

session as you expect to work for a long time, but in other cases you may just want a fast check of few spectra in the image. Having this option in the Spectral Library would be handy.

Or even as a general user option (“Enforce hdr spectral information for ENVI format files”)

Also not sure about that. If the EnMAP-Box would for example use the reflectance scale factor automatically, but native QGIS apps and algorithms won’t, wouldn’t that be confusing?

Note this would be an option that the user would have to set (may be with an appropriate caution message), so he/she will be aware of the consequences. ( And you said that after modifying the xml these settings would be applied to all qgis).

Not sure about others, but in my experience I rarely use anything else from QGIS if I’m using EnMapBox. If I need it, I even open another QGIS session to make sure I do not mesh things up. EnMap Box is not really a plugin, it is a another QGIS.

‌

‌

‌

[Andreas Janz]

The problem is that the xml makes the data units not being read right, thus the units in the plot are wrong. See my original snapshot: units should have been read as um, but they have not, thus values in the x axis are wrong.

Oh, now it sounds like a bug again. Just to get it right:

1. you have an ENVI raster with “um” units in the *hdr file.
2. Now you open the raster in QGIS and close it again. Now you have an *.aux.xml file that shadows the *.hdr. This is concidered normal QGIS/GDAL behaviour that we can’t change. But the EnMAP-Box Profile-Plot should still work fine. Isn’t that the case? If not, please send your raster for testing.
3. What you aren’t allowed to do, is to edit the *.hdr file AFTER the *.aux.xml file is already created. Did you do that? If so, you would need to delete the *.aux.xml file again.

For each band? This would be very tedious, you normally have the same units for all bands.

Imagin a raster with “nm” and a raster with “um”. Now stack both together. The result will have mixed wavelength units. At algorithm level, we would just copy the bandwise properties and we wouldn’t want to translate.

The model we use is a bit more complex then the ENVI model. In our model we could for example have both, spectral bands (e.g. Landsat B1 - B7) and quality bands (e.g. QA_PIXEL) in a single raster. In that sense it is more comparable to a GoogleEarthEngine Image. We will use that approach in the upcoming GEE Time Series Explorer, that allows you to visualize all GEE Data Catalog archives in QGIS.

‌

‌

[Andreas Janz]

Shall we just have one button “Use ENVI Header information”? This would set the reflectance scale factor, the wavelength and the fwhm.

see #947

‌

[Andreas Janz]

Try to make user’s life easy. Sometimes you may want to invest time on these settings at the beginning of the

session as you expect to work for a long time, but in other cases you may just want a fast check of few spectra in the image. Having this option in the Spectral Library would be handy.

That is true, but I think we rather keep this simple to avoid more complicated implementations.

Note this would be an option that the user would have to set (may be with an appropriate caution message), so he/she will be aware of the consequences. ( And you said that after modifying the xml these settings would be applied to all qgis).

Not sure about others, but in my experience I rarely use anything else from QGIS if I’m using EnMapBox. If I need it, I even open another QGIS session to make sure I do not mesh things up. EnMap Box is not really a plugin, it is a another QGIS.

I understand why a better support for ENVI files would be desirable, but again, I think we stick with the pure GDAL/QGIS behaviour and avoid more complicated implementations.

‌

@Benjamin Jakimow have you any further thoughts on that?

[Andreas Janz]

Note that #950 will introduce functionality to get wavelength infos from the HDR file and set it to the GDAL correctly.

[Andreas Janz]

Done, see #947 for details.