Add Stems for single-readout exp time and num readouts per FPA
Arthur Eigenbrot
Branch: more_time_parsers
more_time_parsers
Branch: master
master
Merged
#145 · Created · Last updated
Merged pull request
Merged in more_time_parsers (pull request #145)
042e112·Author: Arthur Eigenbrot·Closed by: Arthur Eigenbrot·2023-07-28
Description
We recently learned that the way we have been thinking about “exposure time” in the context of dark correction is slightly wrong. See SDPO-60 for more information, but the upshot is that we need to keep track of an extra header key: the per-readout exposure time (CAM__005/TEXPOSUR) and the number of readouts per FPA (CAM__014/NSUMEXP). This PR adds buds and stems to do that.
In an attempt to minimize code repetition and future-proof this framework the existing exposure time related Buds and Flowers were refactored to use base classes which made adding the new Buds and Flowers very easy.
EDIT: The Parsing for number of readouts per FPA (NSUMEXP) has been removed after some testing with ViSP. The issue is subtle and some design decisions were/are made here in this PR that affect how we deal with this stuff downstream (i.e., the instruments). The issue is that we never need to know NSUMEXP in isolation; we always want to correlate it with TEXPOSUR as well. For example, consider a situation where any task was taken with both multiple TEXPOSUR values and multiple NSUMEXP values. Being able to query a constant like constants.TASK_num_raw_frames_per_fpa does us no good because we don’t know how the set of those values correlates with the set of TEXPOSUR values.
In other words, what we care about can be thought of as a tuple of (TEXPOSUR, NSUMEXP). One of the was to achieve this is to make a bud/flower that exactly that; a tuple of these two coupled values. That solution ended up being a little awkward, however, and resulted in an explosion of loops within the science pipelines.
Instead we have buds/flowers only for TEXPOSUR and XPOSURE. These are the important things needed to find the correct files. Once we have them we can get the NSUMEXP for each file by looking at its header (via a FitsAccess class). This means we don’t have to worry about correlating TEXPOSUR and NSUMEXP values in the constants db because we just read NSUMEXP from the source whenever we need it.
As an example, using this method would look like this:
for readout_exp in self.constants.dark_readout_exp_times:
dark_objs = self.read(.....)
arrays = (o.data / o.num_raw_frames_per_fpa for o in dark_objs)
...
instead of this:
for read_exp in self.constants.dark_readout_exp_times:
for num_raw in self.constants.dark_num_raw_frames_per_fpa_list:
tags = [....]
if self.count(tags):
continue
arrays = self.read(tags, ...)
normed_arrays = (data / num_raw for data in arrays)
...
We recently learned that the way we have been thinking about “exposure time” in the context of dark correction is slightly wrong. See SDPO-60 for more information, but the upshot is that we need to keep track of an extra header key: the per-readout exposure time (CAM__005/TEXPOSUR) and the number of readouts per FPA (CAM__014/NSUMEXP). This PR adds buds and stems to do that.
In an attempt to minimize code repetition and future-proof this framework the existing exposure time related Buds and Flowers were refactored to use base classes which made adding the new Buds and Flowers very easy.
EDIT: The Parsing for number of readouts per FPA (NSUMEXP) has been removed after some testing with ViSP. The issue is subtle and some design decisions were/are made here in this PR that affect how we deal with this stuff downstream (i.e., the instruments). The issue is that we never need to know NSUMEXP in isolation; we always want to correlate it with TEXPOSUR as well. For example, consider a situation where any task was taken with both multiple TEXPOSUR values and multiple NSUMEXP values. Being able to query a constant like
constants.TASK_num_raw_frames_per_fpadoes us no good because we don’t know how the set of those values correlates with the set of TEXPOSUR values.In other words, what we care about can be thought of as a tuple of (TEXPOSUR, NSUMEXP). One of the was to achieve this is to make a bud/flower that exactly that; a tuple of these two coupled values. That solution ended up being a little awkward, however, and resulted in an explosion of loops within the science pipelines.
Instead we have buds/flowers only for TEXPOSUR and XPOSURE. These are the important things needed to find the correct files. Once we have them we can get the NSUMEXP for each file by looking at its header (via a
FitsAccessclass). This means we don’t have to worry about correlating TEXPOSUR and NSUMEXP values in the constants db because we just read NSUMEXP from the source whenever we need it.As an example, using this method would look like this:
for readout_exp in self.constants.dark_readout_exp_times: dark_objs = self.read(.....) arrays = (o.data / o.num_raw_frames_per_fpa for o in dark_objs) ...instead of this:
for read_exp in self.constants.dark_readout_exp_times: for num_raw in self.constants.dark_num_raw_frames_per_fpa_list: tags = [....] if self.count(tags): continue arrays = self.read(tags, ...) normed_arrays = (data / num_raw for data in arrays) ...Comments welcome.