- changed status to open
off-by-1 normalization error in ImageUtility::NormalizeUShort
Issue #1237
resolved
Description
below is current code for function NormalizeUShort on lines 85-92 of \NINA.Image\ImageAnalysis\ImageUtility.cs:
/// <summary>
/// Converts a value from range [0;65535] to [0;1]
/// </summary>
/// <param name="val"></param>
/// <returns></returns>
private static double NormalizeUShort(double val, int bitDepth) {
return val / (double)(1 << bitDepth);
}
UShort has range 0-65535 but this code is using normalization with 65536 as max value for normalization instead of 65535
I suggested changing denominator
(double)(1 << bitdepth)
to
(double)((1<<bitdepth)-1)
Steps to Reproduce
NormalizeUShort(65535, 16)
returns 65535/65536 = 0.9999847412109375
Expected behaviour
NormalizeUShort(65535, 16)
should return 65535/65535 = 1.0
Actual behaviour
NormalizeUShort(65535, 16)
returns 65535/65536 = 0.9999847412109375
Comments (2)
-
repo owner -
repo owner - changed status to resolved
Fix Issue
#1237- off-by-1 normalization error in ImageUtility::NormalizeUShort→ <<cset 41f02aa90c6f>>
- Log in to comment