Ascom multiplane Camera.ImageArray support

Issue #472 closed
robert hasson created an issue

Hi,

first time user of NiNa and long time user of ASCOM. in fact I have developped my own ASCOM camera driver for Lumix Cameras. (available on Github: https://github.com/totoantibes/LumixCameraAscomDriver/) It has worked well for me over the last year or so. I have used it in combination of APT so far and wanted to try Nina.

I seem to have an issue with the image transfer from the driver to Nina with the use of the multidimension camera.ImageArray

NiNa seems to expect a 2D array however i send out a 3D array (multiplane) since i generate a color image which i guess is not so common with ASCOM cameras.

This has worked very well for me on APT, I also pass the ASCOM CONFORM test with no issue or warning.

the error I get at exec time from NiNa is:

Unable to cast object of type ‘SystemInt32[,,]’to type ‘SystemInt32[,]’.

now maybe this is due to my own implementation of the ASCOM driver multiplane imagearray but since i’ve had good success with other apps i am thinking NiNa is not handling it correctly.

(side note: i had to mod the driver to actual send a fake CCDtemperature of 25C instead of the standard exception built in the ascom driver template) in order for NiNa to confirm the connection to the camera driver. this seems quite odd as not all cameras can sent temperature back. )

steps:

  1. select the camera driver from the list.
  2. configure the driver
  3. connect and confirm all data is passed correctly to nina (size x y, temp, read out modes etc.)
  4. try and capture in the imaging tab.
  5. above mentioned error is displayed.

Thanks

here is the excerpt of my imageArray() property.

 Public ReadOnly Property ImageArray() As Object Implements ICameraV2.ImageArray        Get            If (Not cameraImageReady) Then                TL.LogMessage("ImageArray Get", "Throwing InvalidOperationException because of a call to ImageArray before the first image has been taken!")                Throw New ASCOM.InvalidOperationException("Call to ImageArray before the first image has been taken!")            End If            Dim Tiffimagefile As IO.FileStream            Tiffimagefile = New FileStream(TiffFileName, IO.FileMode.Open)            ReDim cameraImageArray(cameraNumX - 1, cameraNumY - 1, 2) ' there are 3 channels: RVB. 
            Dim decoder As New TiffBitmapDecoder(Tiffimagefile, BitmapCreateOptions.PreservePixelFormat, BitmapCacheOption.Default)            Dim stride As Int32            Dim index As Int32            Dim bitmapSource As BitmapSource = decoder.Frames(0)            Dim bytesPerPixel As UShort            bytesPerPixel = bitmapSource.Format.BitsPerPixel / 8 '3 for JPG and 6 for RAW            stride = bitmapSource.PixelWidth * bytesPerPixel
            If CurrentROM = 1 Then                Dim pixels(bitmapSource.PixelHeight * stride) As UShort                bitmapSource.CopyPixels(pixels, stride, 0)                For y = 0 To (cameraNumY - 1)                    For x = 0 To (cameraNumX - 1)                        index = x * 3 + (y * stride / 2) 'because of the 16 bit instead of the 8 bit per channel this /2 is needed.                        cameraImageArray(x, cameraNumY - y - 1, 0) = pixels(index)                        cameraImageArray(x, cameraNumY - y - 1, 1) = pixels(index + 1)                        cameraImageArray(x, cameraNumY - y - 1, 2) = pixels(index + 2)
                    Next x                Next y            Else                Dim pixels(bitmapSource.PixelHeight * stride) As Byte                bitmapSource.CopyPixels(pixels, stride, 0)                For y = 0 To (cameraNumY - 1)                    For x = 0 To (cameraNumX - 1)                        index = x * 3 + (y * stride)                        cameraImageArray(x, cameraNumY - y - 1, 0) = pixels(index + 2) 'R and B are reversed                        cameraImageArray(x, cameraNumY - y - 1, 1) = pixels(index + 1)                        cameraImageArray(x, cameraNumY - y - 1, 2) = pixels(index)
                    Next x                Next y
            End If            Tiffimagefile.Dispose() 'cleaning up aftermyself and removing the Tiff file once it is used            My.Computer.FileSystem.DeleteFile(TiffFileName)
            TL.LogMessage("ImageArray Get", "getting the Array")
            Return cameraImageArray        End Get    End Property

Robert

Comments (9)

  1. Stefan B repo owner

    Hi Robert,

    thank you for your detailed explanation.
    NINA does not implement the 3D plane, as we either expect a monochromatic image, or a mono image with a bayered filter encoding.

    True color sensors are rare and the current flow of NINA does not support this data type, so it would be a non trivial task to add support for this.
    Is the lumix camera you write an ascom driver for a true color camera? Otherwise I don’t recommend sending debayered images over the pipe, as post processing should be done on the raw bayered image instead.

    As for your second point about the CCDTemperature it seems that I have missed to catch a potential PropertyNotImplementedException to not try to access this property further. I will add this check to the code base.

    Best regards,
    Stefan

  2. robert hasson reporter

    Thanks Stefan,

    Lumix/Panasonic is a regular mirror-less camera. (works the same as any NIKON/CANON etc.). the issue is that the only file I can get with my driver is either a RAW file (20 MB and very slow to download over Wifi) or full size JPG or Thumb JPG. (which I expose in the Readoutmodes of the driver).

    I cannot get the sensor matrix. Hence I qualify my sensor as a colour sensor as per here: https://ascom-standards.org/Help/Developer/html/P_ASCOM_DriverAccess_Camera_SensorType.htm

    The main reason I developed the Driver is to integrate with sequencing soft like APT or NiNa so that i can do dithering and importantly platesoving and not for full imaging Postprocessing. for that i relie on the RAW file left in the SD card! In fact when I use the driver it is often in the Thumb mode (1440x1080) so that the transfer is fast and I dont lose time. Also that is enough for me to get the platesolving done and a general sense of the quality of the file. Lastly if i were to get the RAW file (again typically 20sec to download) i would also waste disk space on my imaging PC.

    Honestly i dont think i can generate a sensor matrix (RGGB) type file from the JPG that I get without a lot of reprocessing

    All i am saying is that APT is able to do so maybe NiNa could accept the challenge😉 . I would really love to use the sequencing and mosaic features built into NiNa.

    Robert

  3. robert hasson reporter

    Stefan,

    here is a snippet of how the ASCOM team (Rick B.) manages the input from the ASCOM cameras:

        Array tempArray = (Array)Camera.ImageArray;
    
        int cols = tempArray.GetLength( 0 );
        int rows = tempArray.GetLength( 1 );
        int planes = 1;
    
        if ( tempArray.Rank == 3 )
        {
            planes = tempArray.GetLength( 2 );
    
            // Here we have a multi-plane (RGB) image;
    
            if ( planes != 3 )
            {
                throw new Exception( "The downloaded image has an unsupported number of colors!" );
            }
        }
        else if ( tempArray.Rank > 3 )
        {
            throw new Exception( "The camera provided an unsupported image type (not mono or 3 color)!"
        }
    

    can you see if a similar logic would also work within NINA?

    robert

  4. Stefan B repo owner

    Hi Robert,

    the problem is not in reading the data from the ASCOM driver. It’s the post processing routine in NINA that is expecting an unsigned int flat grayscale array and not a colored image. This change would need quite some adaptions.
    Furthermore working on compressed JPG data is not intended for many workflows as you’ll lose a lot of precision for star analysis, platesolving etc.
    NINA pulls data from a camera, processes it for displaying etc and saves the raw data onto the disk. A workflow to not save it/leave it on a dslr is just not intended. For canon/nikon for example we pull the RAW files from the camera and save them to the disk instead of leaving them at the camera side.

    Regards,
    Stefan

  5. robert hasson reporter

    Hi Stefan,

    i have now a first prototype that sends an RGGB array that seems to be working in my dev environment.

    I will test further and publish an update to my driver. It seems to work with NINA and APT transparently. could you advise if i should keep the color version of just stick to the RGGB? any benefit in general in keeping both?

    Robert

  6. Stefan B repo owner

    Nina will always expect the bayered array. There are no plans currently to support a multi plane color image.

  7. Ingvar Stepanyan

    There are no plans currently to support a multi plane color image.

    If so, would it be possible to improve error message when detecting SensorType == Color? Right now it just emits an obscure

    Unable to cast object of type ‘SystemInt32[,,]’to type ‘SystemInt32[,]’.

    Which doesn’t tell the user what’s wrong. Perhaps it could instead say that RGB images are not supported and suggest user to switch to a Bayer mode in camera settings (which a lot of such cameras support).

    That said, I’d love if RGB support worked at least for Three Point Polar alignment. JPEG is indeed not precise enough for actual stacking so it’s reasonable not to invest time to support it there, but it tends to be good enough for plate solving and is a lot faster to transmit. So when I want just to set up my mount, I could use JPEG mode for the alignment, and then leave the DSLR to do its thing for actual shooting.

  8. Log in to comment