Assign Map Scan information during Parsing

Merged
#67 · Created  · Last updated

Merged pull request

Merged in parse_map_scan (pull request #67)

6f9be71·Author: ·Closed by: ·2022-05-12

Description

It is now clear that we will never be able to rely on DKIST008/9 (DSPS repeat numbers) to assign files to different map scans. In the absence of ViSP-specific keywords to do this we need to infer the map scan organization from other header keys. This has already be done in the RewriteInputFramesToCorrectHeaders paradigm, but that was always a temporary solution until the summit “fixed” the headers. Now that we know the headers will never be fixed it’s time to move the logic of RewriteInputFramesToCorrectHeaders where it belongs: into the parse task. In addition to any organizational benefits, this should drastically reduce run time because the Rewrite.. step had to open and update lots and lots of files.

This PR is huge, but it does fall into a few different categories:

New Functionality in Parse

The “map_repeats” module has grown to include 3 new stems:

  • MapScanStepFlower - Tags files with their corresponding map scan number

  • NumMapScansBud - Produces a constant that is the total number of map scans.

  • PickyMapBud - A PickyBud that makes a few checks that the full collection of (map_scan, raster_step, modstate) data forms a consistent set.

All of these simply replicate the logic that had been in RewriteInputFramesToCorrectHeaders.

In addition to these “top-level” Stems there are two new classes to grease the wheels a bit:

  • SingleScanStep - This is just a fancy tuple. It distills a single VispL0FitsAccess object into its (raster_step, modstate, date_obs) values. It is sortable by date_obs. It's just a fancy tuple.

  • MapScanStemBase - All of the Map related stems listed above have the same setter, which ingests VispL0FitsAccess objects with SingleScanStep and places them in a nested dictionary (which is exactly like RewriteInputFramesToCorrectHeaders). With this base class we don’t have to copy that setter three times.

Removal Of RewriteInputFramesToCorrectHeaders

With Map stuff in the Parse task we no longer need to correct “bad” headers. Thus that task has been removed, as has the workflow that used it. This has Recipe Processing testing implications.

If, in the future, the ViSP team does add map keywords to L0 data it will be as simple as replacing the Flower and Bud in Parse.

Exorcism of DSPS Language

This builds on a branch that @Fraser Watson had started. In every place we had previously using “DSPS” terminology we actually meant “map_scan”. All mentions of DSPS have been removed. This forms the bulk of the “small” diffs in this PR.

Update to Science/WriteL1

In the past WriteL1 used DSPS keys to correctly populate the WCS. Those keys are wrong forever and there are no L0 ViSP keys to take their place. Fortunately there are L1 ViSP keys that capture this information. WriteL1 has been updated to use those keys and there’s a new function in Science called _update_calibrated_header that populates the keys. I’m not totally in love with the idea of writing the L1 keys to the calibrated frames in Science, but that’s the one place where we have the frame and know it’s map info.

Update to AssembleMovie

The default AssembleMovie task in *-common uses DSPS repeats to define the separate Images in the movie. This is no longer true for ViSP and so AssembleVispMovie has been updated to use Map Scans.

This makes me think that the defaults in *-common might need revisiting.

 

0 attachments

0 comments

Loading commits...