Error in Transform Raster when applying a previously calculated *.pkl (Fit PCA)

[Agustin Lobo]

I get this error in Transform Raster when applying a previously calculated *.pkl (Fit PCA)

QGIS version: 3.22.1-Białowieża

QGIS code revision: 663dcf8fb9

Qt version: 5.15.2

Python version: 3.9.5

GDAL version: 3.2.2

GEOS version: 3.9.0-CAPI-1.16.2

PROJ version: Rel. 7.2.1, January 1st, 2021

PDAL version: 2.2.0 (git-version: Release)

Algorithm started at: 2021-12-17T10:04:06

Algorithm 'Transform Raster' starting…

Input parameters:

{ 'mask' : None, 'outRaster' : '/media/alobo/LaCieNTFS2T/Ignacio2019/RioTinto/PRISMA/PRS_L2D_STD_20210625111917_20210625111921_0001_PCA.bsq', 'raster' : '/media/alobo/LaCieNTFS2T/Ignacio2019/RioTinto/PRISMA/PRS_L2D_STD_20210625111917_20210625111921_0001_SR.tif', 'transformer' : '/media/alobo/LaCieNTFS2T/Ignacio2019/RioTinto/PRISMA/PRS_L2D_STD_20210625111917_20210625111921_0001.pkl' }

start EstimatorTransform, RasterSize(x=1190, y=1178)

Traceback (most recent call last):

File "/home/alobo/.local/share/QGIS/QGIS3/profiles/default/python/plugins/enmapboxplugin/enmapboxgeoalgorithms/provider.py", line 156, in processAlgorithm

result = self.processAlgorithm_()

File "/home/alobo/.local/share/QGIS/QGIS3/profiles/default/python/plugins/enmapboxplugin/enmapboxgeoalgorithms/algorithms.py", line 697, in processAlgorithm_

estimator.transform(filename=filename, raster=raster, mask=mask, progressBar=self._progressBar)

File "/home/alobo/.local/share/QGIS/QGIS3/profiles/default/python/plugins/enmapboxplugin/site-packages/hubflow/core.py", line 5305, in _transform

applier.apply(operatorType=_EstimatorTransform, estimator=self, raster=raster, mask=mask, mask2=mask2,

File "/home/alobo/.local/share/QGIS/QGIS3/profiles/default/python/plugins/enmapboxplugin/site-packages/hubflow/core.py", line 53, in apply

results = hubdc.applier.Applier.apply(self, operatorType=operatorType, description=description,

File "/home/alobo/.local/share/QGIS/QGIS3/profiles/default/python/plugins/enmapboxplugin/site-packages/hubdc/applier.py", line 979, in apply

results = self._runProcessSubgrids()

File "/home/alobo/.local/share/QGIS/QGIS3/profiles/default/python/plugins/enmapboxplugin/site-packages/hubdc/applier.py", line 1043, in _runProcessSubgrids

blockResults.append(_Worker.processSubgrid(**kwargs))

File "/home/alobo/.local/share/QGIS/QGIS3/profiles/default/python/plugins/enmapboxplugin/site-packages/hubdc/applier.py", line 1125, in processSubgrid

return cls.operator._apply(workingGrid=workingGrid, iblock=i, nblock=n, yblock=iy, xblock=ix, nyblock=ny,

File "/home/alobo/.local/share/QGIS/QGIS3/profiles/default/python/plugins/enmapboxplugin/site-packages/hubdc/applier.py", line 1272, in _apply

blockResult = self.ufunc(*self._ufuncArgs, **self._ufuncKwargs)

File "/home/alobo/.local/share/QGIS/QGIS3/profiles/default/python/plugins/enmapboxplugin/site-packages/hubflow/core.py", line 5426, in ufunc

y = sklTransform(X=X)

File "/home/alobo/.local/lib/python3.9/site-packages/sklearn/pipeline.py", line 560, in _transform

Xt = transform.transform(Xt)

File "/home/alobo/.local/lib/python3.9/site-packages/sklearn/preprocessing/_data.py", line 883, in transform

X = self._validate_data(X, reset=False,

File "/home/alobo/.local/lib/python3.9/site-packages/sklearn/base.py", line 421, in _validate_data

X = check_array(X, **check_params)

File "/home/alobo/.local/lib/python3.9/site-packages/sklearn/utils/validation.py", line 63, in inner_f

return f(*args, **kwargs)

File "/home/alobo/.local/lib/python3.9/site-packages/sklearn/utils/validation.py", line 726, in check_array

raise ValueError("Found array with %d sample(s) (shape=%s) while a"

ValueError: Found array with 0 sample(s) (shape=(0, 234)) while a minimum of 1 is required by StandardScaler.

Traceback (most recent call last):
File "/home/alobo/.local/share/QGIS/QGIS3/profiles/default/python/plugins/enmapboxplugin/enmapboxgeoalgorithms/provider.py", line 156, in processAlgorithm
result = self.processAlgorithm_()
File "/home/alobo/.local/share/QGIS/QGIS3/profiles/default/python/plugins/enmapboxplugin/enmapboxgeoalgorithms/algorithms.py", line 697, in processAlgorithm_
estimator.transform(filename=filename, raster=raster, mask=mask, progressBar=self._progressBar)
File "/home/alobo/.local/share/QGIS/QGIS3/profiles/default/python/plugins/enmapboxplugin/site-packages/hubflow/core.py", line 5305, in _transform
applier.apply(operatorType=_EstimatorTransform, estimator=self, raster=raster, mask=mask, mask2=mask2,
File "/home/alobo/.local/share/QGIS/QGIS3/profiles/default/python/plugins/enmapboxplugin/site-packages/hubflow/core.py", line 53, in apply
results = hubdc.applier.Applier.apply(self, operatorType=operatorType, description=description,
File "/home/alobo/.local/share/QGIS/QGIS3/profiles/default/python/plugins/enmapboxplugin/site-packages/hubdc/applier.py", line 979, in apply
results = self._runProcessSubgrids()
File "/home/alobo/.local/share/QGIS/QGIS3/profiles/default/python/plugins/enmapboxplugin/site-packages/hubdc/applier.py", line 1043, in _runProcessSubgrids
blockResults.append(_Worker.processSubgrid(**kwargs))
File "/home/alobo/.local/share/QGIS/QGIS3/profiles/default/python/plugins/enmapboxplugin/site-packages/hubdc/applier.py", line 1125, in processSubgrid
return cls.operator._apply(workingGrid=workingGrid, iblock=i, nblock=n, yblock=iy, xblock=ix, nyblock=ny,
File "/home/alobo/.local/share/QGIS/QGIS3/profiles/default/python/plugins/enmapboxplugin/site-packages/hubdc/applier.py", line 1272, in _apply
blockResult = self.ufunc(*self._ufuncArgs, **self._ufuncKwargs)
File "/home/alobo/.local/share/QGIS/QGIS3/profiles/default/python/plugins/enmapboxplugin/site-packages/hubflow/core.py", line 5426, in ufunc
y = sklTransform(X=X)
File "/home/alobo/.local/lib/python3.9/site-packages/sklearn/pipeline.py", line 560, in _transform
Xt = transform.transform(Xt)
File "/home/alobo/.local/lib/python3.9/site-packages/sklearn/preprocessing/_data.py", line 883, in transform
X = self._validate_data(X, reset=False,
File "/home/alobo/.local/lib/python3.9/site-packages/sklearn/base.py", line 421, in _validate_data
X = check_array(X, **check_params)
File "/home/alobo/.local/lib/python3.9/site-packages/sklearn/utils/validation.py", line 63, in inner_f
return f(*args, **kwargs)
File "/home/alobo/.local/lib/python3.9/site-packages/sklearn/utils/validation.py", line 726, in check_array
raise ValueError("Found array with %d sample(s) (shape=%s) while a"
ValueError: Found array with 0 sample(s) (shape=(0, 234)) while a minimum of 1 is required by StandardScaler.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/alobo/.local/share/QGIS/QGIS3/profiles/default/python/plugins/enmapboxplugin/enmapboxgeoalgorithms/provider.py", line 170, in processAlgorithm
raise Exception('unexpected error')
Exception: unexpected error

Execution failed after 136.05 seconds (2 minutes 16 seconds)

Loading resulting layers

Algorithm 'Transform Raster' finished

‌

‌

‌

[Andreas Janz]

This is the relevant part:
ValueError: Found array with 0 sample(s) (shape=(0, 234)) while a minimum of 1 is required by StandardScaler.

I’ll try to reproduce this with one of my own PRISMA scenes. Looks like a whole data block is masked, which is not correctly handled.

[Royden Yates]

my report also refers - it's a dupe of this one

[Andreas Janz]

Confirmed, this is a bug. I’ll fix this after the holidays, or better overhaul the whole transformer algorithms, to also improve data IO speed.

[Agustin Lobo]

Can you provide an estimated date for this issue to be fixed?

[Andreas Janz]

Hi Agus, next week is quite full, I’ll see what I can do and give it a higher priority.

[Andreas Janz]

resolves #862, #882 and #1003

→ <<cset 164e12383cdb>>

[Agustin Lobo]

Great! could you provide a zip with the new dev version?

[Andreas Janz]

Here it is:
https://bitbucket.org/hu-geomatics/enmap-box/downloads/enmapboxplugin.3.10.0.20220222T163058.TEST.zip