e6e1918·Author: Arthur Eigenbrot·Closed by: Arthur Eigenbrot·2022-04-06
Description
This isn’t a strictly necessary change, but it cleans up the Science task a bit with two (related) goals in mind:
Now that we only split beams right when we need them we can have beam 1 and 2 from the same beam at exactly the same time. So let’s take advantage of that and process the two beams when we have them and not worry about having to match them up later.
A consequence of point 1 is that we no longer need INTERMEDIATE_ARRAYS, which saves some I/O time. It’s not the longest pole, but it will probably still be hours on long runs.
Summary
process_frames
A lot of individual processing steps (basic corrections, average beams, telescope correction) have been moved from run to process_frames. process_frames loops over all the stuff and at its core completely processes a single (dsps, raster step) frame. This means the calibrated frame can be immediately be written as a VispTag.calibrated() frame, which removes the need to have INTERMEDIATE_ARRAYS.
Black kind of hides this, but process_frames is really just a big ol' stack of loops with 5 lines at the center. In pseudo code:
def process_frames:
for dsps_repeat:
for raster_step:
for beam:
-------------
if polarimetric:
process_polarimetric_modstates
else:
correct_single_frame
store_beams
--------------
combine_beams
if polarimetric:
telescope_polarization_correction
write_calibrated_array
Â
process_polarimetric_modstates runs correct_single_frame for all modstates and then demodulates the array stack.
correct_single_frame used to be called correct_frames, but it turns out it only even worked on a single frame so it was renamed.
run
To save some reads and clean up the loops in process_frames we no longer load the calibration objects when we expect to need them. Instead we just load every single one once at the start of run. The function that does this is collect_calibration_objects; it just contains the loading code that had been spread out before. The set of calibration objects has a constrained and small size (at most modstate * beam * exp_time) so this should be OK. Once the objects are loaded they are passed to correct_single_frame (via process_frames) where they are used.
This isn’t a strictly necessary change, but it cleans up the Science task a bit with two (related) goals in mind:
Now that we only split beams right when we need them we can have beam 1 and 2 from the same beam at exactly the same time. So let’s take advantage of that and process the two beams when we have them and not worry about having to match them up later.
A consequence of point 1 is that we no longer need INTERMEDIATE_ARRAYS, which saves some I/O time. It’s not the longest pole, but it will probably still be hours on long runs.
Summary
process_frames
A lot of individual processing steps (basic corrections, average beams, telescope correction) have been moved from
runtoprocess_frames.process_framesloops over all the stuff and at its core completely processes a single (dsps, raster step) frame. This means the calibrated frame can be immediately be written as aVispTag.calibrated()frame, which removes the need to have INTERMEDIATE_ARRAYS.Black kind of hides this, but
process_framesis really just a big ol' stack of loops with 5 lines at the center. In pseudo code:def process_frames: for dsps_repeat: for raster_step: for beam: ------------- if polarimetric: process_polarimetric_modstates else: correct_single_frame store_beams -------------- combine_beams if polarimetric: telescope_polarization_correction write_calibrated_arrayÂ
process_polarimetric_modstatesrunscorrect_single_framefor all modstates and then demodulates the array stack.correct_single_frameused to be calledcorrect_frames, but it turns out it only even worked on a single frame so it was renamed.run
To save some reads and clean up the loops in
process_frameswe no longer load the calibration objects when we expect to need them. Instead we just load every single one once at the start ofrun. The function that does this iscollect_calibration_objects; it just contains the loading code that had been spread out before. The set of calibration objects has a constrained and small size (at most modstate * beam * exp_time) so this should be OK. Once the objects are loaded they are passed tocorrect_single_frame(viaprocess_frames) where they are used.So
runis much simplified:def run: collect_calibrations process_frames record_quality_stuffÂ