Meridian Flips Failing to recover
Issue #1257
resolved
[X ] Is the issue reproducible?
[ ] Are all prerequisites that are mentioned inside the manual met?
Which Version are you running? 2.3.0.9001
Description
Possibly since the latest update Meridian flips are failing, looking at logs seeing a lot of QHYSDK errors.
Steps to Reproduce
- Start Imaging Sequence
- Wait for Meridian Flip
- Meridian Flip fails on ReCenter
Expected behaviour
Image Sequence Continues
Actual behaviour
Meridian Flips does not complete
Log Files Attached
Comments (28)
-
reporter
-
repo owner
- changed status to closed
Hi, thanks for the report, but this indicates a hardware level failure. When the QHY sdk reports failures to read certain attributes this is an indication of a lost connection. As meridian flips move things around quite a bit, please double check for loose cable connections etc.
-
reporter
Hi, thank you for replying. The thing that is confusing me is that it is consistently happening only ever on Meridian flips and closing out NINA and reconnecting seems to consistently work fine without any issues.
-
What you’re reporting, Sean, is very consistent with a hardware issue (cable most likely; perhaps something intemittantly failing due to the motion of the mount during the MF).
-
reporter
Hi, I understand it's just I would expect to see errors while Slewing, doing manual meridian flips, etc? Which I have done and I haven’t had any recent connectivity issues. I will do some more testing of this but it has been exclusively and consistently only during automated meridian flip.
On a side note, NINA becomes unresponsive once the connection issue occurs and so I guess regardless of the problem it would be beneficial if it was some how possible to not have NINA crash.
-
From NINAs point of view there’s no difference whatever between a manual flip and an automated flip. What’s more, a flip is just a slew, and you said that slews don’t have issues. My suspicion is that it’s the much larger movement in a meridian flip that instigates the problem.
Get Outlook for iOShttps://aka.ms/o0ukef
-
reporter
Sure, I plan on running some manual meridian flips tonight to see if I can instigate any connection issues.
-
reporter
I ran through some manual/forced meridian flips last night and as I kinda of expected did NOT experience any camera connectivity issues until MeridianFlipTrigger in Sequence.
- 2023-09-30T21:42:14.4290 – Started FilterFocusOffset calculator on M34
- 2023-09-30T23:01:02 ~~~ Roughly crossed 45” pass meridian and scope automatically switched sides.
- 2023-09-30T23:52:47 – FilterOffset completed
- 2023-09-30T23:53:07 – Started 5sec capture on loop and slewed back to West side of pier.
- 2023-10-01T01:50:55 – Meridian Flip Trigger
- 2023-10-01T01:51:59 – QhySDK Errors
As previously mentioned I have only ever seen these connectivity errors during a MeridianFlipTrigger, NINA becomes unresponsive. I close it, re-open NINA and restart sequence. Any USB or Camera/FilterWheel connectivity I would expect to also require a power reset/windows restart or something else. But I have only been able to identify NINA as being an issue. Though I am open to any additional suggestions.
2023-09-30T21:42:14.4290|INFO|ImageSolver.cs|Solve|54|Platesolve successful: Coordinates: RA: 21:31:46; Dec: 48° 26' 32"; Epoch: J2000
2023-09-30T21:42:14.4445|INFO|CenteringSolver.cs|Center|85|Centering Solver - Scope Position: RA: 21:32:41; Dec: 48° 32' 45"; Epoch: JNOW; Offset: RA: 00:00:00; Dec: 00° 00' 00"; Distance: 00° 00' 00"; Bearing: 00° 00' 00"; Centering Coordinates: RA: 21:32:41; Dec: 48° 32' 45"; Epoch: JNOW; Solved: RA: 21:32:38; Dec: 48° 33' 00"; Epoch: JNOW; Separation RA: 00:00:03; Dec: -00° 00' 14"; Distance: 00° 00' 30"; Bearing: 62° 24' 32"; Threshold: 1
2023-09-30T21:45:28.2870|INFO|SequenceItem.cs|Run|206|Starting Category: , Item: FilterOffsetCalculator, Loops: 3
...
...
2023-09-30T23:01:02.1759|INFO|CameraVM.cs|Capture|732|Starting Exposure - Exposure Time: 6s; Filter: Blue; Gain: 56; Offset 40; Binning: 2x2;
2023-09-30T23:01:02.9671|INFO|HocusFocusStarDetection.cs|Detect|410|Average HFR: 5.08525027806781, HFR MAD: 0.328391191796999, Detected Stars 75, Region: 0
2023-09-30T23:01:09.6442|INFO|AutoFocusEngine.cs|StartBlindFocusPoints|915|Waiting on remaining AutoFocus analysis tasks
2023-09-30T23:01:10.6754|INFO|FocuserVM.cs|MoveFocuserInternal|205|Moving Focuser to position 22300
2023-09-30T23:01:14.5281|INFO|AutoFocusEngine.cs|ValidateCalculatedFocusPosition|1168|Validating HFR at final focus position 22300
2023-09-30T23:01:14.7863|INFO|CameraVM.cs|Capture|732|Starting Exposure - Exposure Time: 6s; Filter: Blue; Gain: 56; Offset 40; Binning: 2x2;
2023-09-30T23:01:22.5041|INFO|CameraVM.cs|Capture|732|Starting Exposure - Exposure Time: 6s; Filter: Blue; Gain: 56; Offset 40; Binning: 2x2;
2023-09-30T23:01:30.2285|INFO|CameraVM.cs|Capture|732|Starting Exposure - Exposure Time: 6s; Filter: Blue; Gain: 56; Offset 40; Binning: 2x2;
2023-09-30T23:01:38.6568|WARNING|AutoFocusEngine.cs|ValidateCalculatedFocusPosition|1180|Failed assessing HFR at the final focus point
2023-09-30T23:01:38.6598|WARNING|AutoFocusEngine.cs|PerformPostAutoFocusActions|1033|AutoFocus did not complete successfully, so restoring the focuser position to 22332
..
..
2023-09-30T23:52:47.8428|INFO|SequenceItem.cs|Run|224|Finishing Category: , Item: FilterOffsetCalculator, Loops: 3
...
...
2023-09-30T23:53:07.8720|INFO|CameraVM.cs|Capture|732|Starting Exposure - Exposure Time: 5s; Filter: Luminance; Gain: 56; Offset 40; Binning: 2x2;
2023-09-30T23:53:23.0952|INFO|CameraVM.cs|Capture|732|Starting Exposure - Exposure Time: 5s; Filter: Luminance; Gain: 56; Offset 40; Binning: 2x2;
2023-09-30T23:53:31.0143|INFO|HocusFocusStarDetection.cs|Detect|410|Average HFR: 3.36558133069905, HFR MAD: 0.114230583731335, Detected Stars 75, Region: 0
2023-09-30T23:53:31.4748|INFO|CameraVM.cs|Capture|732|Starting Exposure - Exposure Time: 5s; Filter: Luminance; Gain: 56; Offset 40; Binning: 2x2;
2023-09-30T23:53:38.1925|INFO|CameraVM.cs|Capture|732|Starting Exposure - Exposure Time: 5s; Filter: Luminance; Gain: 56; Offset 40; Binning: 2x2;
2023-09-30T23:53:39.3167|INFO|HocusFocusStarDetection.cs|Detect|410|Average HFR: 3.25535231543969, HFR MAD: 0.102796539363814, Detected Stars 75, Region: 0
2023-09-30T23:53:44.9325|INFO|CameraVM.cs|Capture|732|Starting Exposure - Exposure Time: 5s; Filter: Luminance; Gain: 56; Offset 40; Binning: 2x2;
2023-09-30T23:53:45.7892|INFO|HocusFocusStarDetection.cs|Detect|410|Average HFR: 2.83857592469337, HFR MAD: 0.200219238919003, Detected Stars 75, Region: 0
2023-09-30T23:53:51.6555|INFO|CameraVM.cs|Capture|732|Starting Exposure - Exposure Time: 5s; Filter: Luminance; Gain: 56; Offset 40; Binning: 2x2;
2023-09-30T23:53:52.3963|INFO|HocusFocusStarDetection.cs|Detect|410|Average HFR: 3.55065707351229, HFR MAD: 0.55823567857044, Detected Stars 13, Region: 0
2023-09-30T23:53:58.3749|INFO|CameraVM.cs|Capture|732|Starting Exposure - Exposure Time: 5s; Filter: Luminance; Gain: 56; Offset 40; Binning: 2x2;
2023-09-30T23:54:05.1081|INFO|CameraVM.cs|Capture|732|Starting Exposure - Exposure Time: 5s; Filter: Luminance; Gain: 56; Offset 40; Binning: 2x2;
2023-09-30T23:54:11.8372|INFO|CameraVM.cs|Capture|732|Starting Exposure - Exposure Time: 5s; Filter: Luminance; Gain: 56; Offset 40; Binning: 2x2;
2023-09-30T23:54:18.5824|INFO|CameraVM.cs|Capture|732|Starting Exposure - Exposure Time: 5s; Filter: Luminance; Gain: 56; Offset 40; Binning: 2x2;
2023-09-30T23:54:25.3515|INFO|CameraVM.cs|Capture|732|Starting Exposure - Exposure Time: 5s; Filter: Luminance; Gain: 56; Offset 40; Binning: 2x2;
2023-09-30T23:54:32.0970|INFO|CameraVM.cs|Capture|732|Starting Exposure - Exposure Time: 5s; Filter: Luminance; Gain: 56; Offset 40; Binning: 2x2;
2023-09-30T23:54:39.0632|INFO|CameraVM.cs|Capture|732|Starting Exposure - Exposure Time: 5s; Filter: Luminance; Gain: 56; Offset 40; Binning: 2x2;
2023-09-30T23:54:45.8012|INFO|CameraVM.cs|Capture|732|Starting Exposure - Exposure Time: 5s; Filter: Luminance; Gain: 56; Offset 40; Binning: 2x2;
2023-09-30T23:54:46.5454|INFO|HocusFocusStarDetection.cs|Detect|410|Average HFR: 2.68265893257821, HFR MAD: 0.343847140804151, Detected Stars 49, Region: 0
2023-09-30T23:54:52.5213|INFO|CameraVM.cs|Capture|732|Starting Exposure - Exposure Time: 5s; Filter: Luminance; Gain: 56; Offset 40; Binning: 2x2;
2023-09-30T23:54:53.3812|INFO|HocusFocusStarDetection.cs|Detect|410|Average HFR: 2.88580772721799, HFR MAD: 0.188470523929594, Detected Stars 75, Region: 0
2023-09-30T23:55:02.1891|INFO|CameraVM.cs|Capture|732|Starting Exposure - Exposure Time: 5s; Filter: ; Gain: 56; Offset 40; Binning: 2x2;
2023-09-30T23:55:09.2927|INFO|ImageSolver.cs|Solve|41|Platesolving with parameters: FocalLength: 2810 PixelSize: 7.52 SearchRadius: 30 BlindFailoverEnabled: True Regions: 5000 DownSampleFactor: 0 MaxObjects: 500 Reference Coordinates RA: 00:42:45 Dec: 41° 16' 09" Epoch: J2000
-
reporter
- attached 20231001-015516-2.3.0.9001.12180.log
- attached 20230930-193623-2.3.0.9001.22900.log
<div class="preview-container wiki-content"><!-- loaded via ajax --></div> <div class="mask"></div> </div></div> </form>
-
reporter
- attached 20231001-015516-2.3.0.9001.12180.log
- attached 20230930-193623-2.3.0.9001.22900.log
<div class="preview-container wiki-content"><!-- loaded via ajax --></div> <div class="mask"></div> </div></div> </form>
-
repo owner
Why should the connectivity issues occur on normal operation? The problem is most likely due to cable stretch during a meridian flip. It is the largest slew you encounter during a night and that you see these issues ONLY during meridian flips strongly indicates that this is the case, as a meridian flip is nothing special for the camera operation. It basically is just a slew to the same coordinates from the application perspective.
And these issues can be intermittent issues, where a full replug is not required, but the QHY SDK might have trouble and take down the application with it.
-
reporter
Not sure I am following, I did two meridian flips with Camera taking images throughout the whole flip (ie. Scope flipping from Zenith to Zenith) and there wasn’t any issues all images downloaded with-out issue. Yet during every
MeridianFlipVM.csthere is aQhyccdSdk.csERROR. If it was a cable issue that was occurring during everyMeridianFlipVM.csthan it should be reproducible while doing a Manual flip. But it isn’t.
-
repo owner
And inside the MeridianFlipVM there is zero interaction with the camera during the flip itself and there are many users with QHY cameras doing successful flips.
There is no special code for any camera interaction with regards to a meridian flip. -
Sean, There would have to be a plausible explanation for NINA to be involved. As Stefan has said repeatedly, there’s nothing magical about a Meridian flip - it’s just a slew like any other. And Meridian flips don’t involve the camera at all. So if we were to look to find a problem with NINA, there just wouldn’t be any place to start. You also say that this might relate to a recent version of NINA, implying that this used to work; is that right? I ask because there aren't any recent changes to the meridian flip code.
Get Outlook for iOShttps://aka.ms/o0ukef
-
reporter
I am looking at this repo
https://bitbucket.org/Isbeorn/nina/src/master/I dont see any tags or branches for2.3?
Looking at the logs, the it looks like the error is occurring in
AscomTelescope.cs|MeridianFlip. -
reporter
never mind, I see the release comments in the master branch
-
reporter
This looks like the most recent commit hash to that function
commit 36b1c2da0154ef2e6d72830f7012d7882aad076bI will try to v2.3.1 and see if that fixed any of the issues I am seeing.
-
repo owner
In which log do you see an error inside MeridianFlip function - all i can see is that the flip function sets the pier side and then the QHY hardware is going poof. Setting the pier side is however nothing that would affect a camera - except that this causes a meridian flip slew.
-
reporter
Right, thats the last log message before the error. I don't know enough about NINA’s code to have any more insight besides that.
But this error is consistently occurring during the MeridianFlip function in NINA, if it was a cable issue I would expect to see similar connectivity errors during any slew across the Meridian and potentially other larg slews. But these errors have not occurred, even when I have tried to reproduce. Without being able to reproduce it any other way than letting NINA perform the MeridianFlip, points at this point lacking additional data to a potential issue.
I can’t speak to other users, but I had waited a few weeks before posting this issue.
-
repo owner
If anything this would be a very low level issue that only QHY would know about and how that could be related to anything. There is nothing in N.I.N.A. itself that could have an effect like that.
-
reporter
NINA is doing more than just slewing the mount. Without looking into the code,
- It stops or waits for the camera to complete the image.
- Stops PHD/Guiding.
- Slews the mount.
- Performs ReCenter/Slewing.
- Runs AutoFocus
- Starts PHD/Guiding
- Resumes the sequence.
It appears the issue is occurring somewhere between 3 and 4,5,6.
I will try to spend more time looking into this later, and see if I can get a debugging session started with NINA.
-
repo owner
Yes Nina is doing more in the workflow of a flip. No at the time of the log timestamp it is not doing more, as these steps are sequential.
As you can see in the AscomTelescope.cs file the “
Setting pier side to pierEast" log will be followed by the “Slewing to coordinates ….” - but in your logs in between these two logs the QHY SDK is already throwing a bunch of errors. -
repo owner
2023-10-01T01:50:58.7851|INFO|AscomTelescope.cs|MeridianFlip|501|Setting pier side to pierEast 2023-10-01T01:51:59.8084|ERROR|QhyccdSdk.cs|GetControlValue|274|QHYCCD: Failed to Get value for control CONTROL_CURTEMP ... Qhy errors ... 2023-10-01T01:52:26.3133|INFO|AscomTelescope.cs|MeridianFlip|505|Slewing to coordinates2023-09-30T01:26:50.3585|INFO|AscomTelescope.cs|MeridianFlip|501|Setting pier side to pierEast 2023-09-30T01:27:49.1019|ERROR|QhyccdSdk.cs|GetControlValue|274|QHYCCD: Failed to Get value for control CONTROL_CURTEMP ... Qhy errors ... 2023-09-30T01:28:17.8080|INFO|AscomTelescope.cs|MeridianFlip|505|Slewing to coordinates RA: 00:44:03; Dec: 41° 00' 29"; Epoch: JNOW. Attempt 1 / 202023-10-01T01:50:58.7851|INFO|AscomTelescope.cs|MeridianFlip|501|Setting pier side to pierEast 2023-10-01T01:51:59.8084|ERROR|QhyccdSdk.cs|GetControlValue|274|QHYCCD: Failed to Get value for control CONTROL_CURTEMP ... Qhy errors ... 2023-10-01T01:52:26.3133|INFO|AscomTelescope.cs|MeridianFlip|505|Slewing to coordinates RA: 00:44:03; Dec: 41° 00' 29"; Epoch: JNOW. Attempt 1 / 20From this pattern it looks like your mount driver does an actual flip when setting the pier side (which is fine - just not all drivers do it, that’s why a slew follows afterwards) - but the QHY failures that start during that pier side flip are not instantaneous but rather random.
Not sure how you reproduced the meridian flips with your manual steps, but maybe the mount moves differently when setting the pier side compared to a standard slew. Also it was not clear if NINA was connected to the mount and camera during that time. -
Maybe he should try turning off “Use side of pier” and see what happens?
Get Outlook for iOShttps://aka.ms/o0ukef
-
repo owner
The side of pier is always set. The toggle for usage of side of pier is only about relying on it for evaluating if a flip should happen
-
This just looks like classic USB connection problems to me. I suggest running USBLogView to log any device disconnect/connect events that might otherwise go unnoticed.
-
reporter
I downloaded USBLogView, and was not able to find anything. Additionally it appears after upgrading to
2.3.1appears to have resolved the issue. I ran 2 sequences across the Meridian and didn’t experience any issues. -
reporter
- changed status to resolved
Upgrading to 2.3.1 appears to have resolved the issue.
- Log in to comment
