START and FINISH behavior is bad
Issue #58
resolved
A couple concerns about START and FINISH handling by Tempo2:
- Why do par files get written with "1" at the end of those lines? That has bitten me MANY times when I read in new data that gets ignored because it ignores all TOAs outside of the START/FINISH range.
- Why aren't START/FINISH equal to the first and last TOA? For some reason they seem to be getting written with a 1-day padding around them. Why 1 day? This is not always appropriate, depending on the spacing between TOAs.
Comments (7)
-
-
reporter
Excellent, thanks!
-
I’ve pushed to a new branch (startfinish) with some changes. I think that the START/FINISH issues mostly came from plk. I’ve done a couple of things:
- tempo2 keeps START/FINISH to the first/last SAT by default. Internally it will add a 1 microsecond extra bound to the START/FINISH to deal with rounding issues. This can probably be smaller if needed.
- I fixed one instance of using BAT rather than SAT with START/FINISH in “bootstrap.C”
- I’ve changed plk so that it doesn’t change START/FINISH except when you have zoomed in the time axis and have pressed fit. Otherwise it will leave them alone. I also made it so you can change from “date” to MJD to year without having to re-set the zoom window.
I’d be grateful if you can see if this solves some of your issues. I am wary of introducing new bugs in PLK (it’s a messy bit of code), so I want to use it myself a bit before pushing it to the wider world.
-
reporter
Cool! I’ll take a look. A couple of questions first:
- How are START/FINISH (meant to be) defined? It sounds like they are SATs but they don’t have a site or frequency associated with them so that is a poorly defined concept. But nevertheless it would be the MJD values in the .tim files that way.
- When START/FINISH have “1” after them, TOAs outside that range are not included in the fit. Is this comparison done in SATs or BATs? And is that only a feature of the plk plugin and not Tempo2 in general?
-
reporter
OK, I checked out that branch and played a bit. Answering my own questions, it seems like START/FINISH are defined as SATs, and compared that way, despite the conceptual difficulties with that.
Both core Tempo2 and plk fits only include TOAs within START/FINISH if the “1” is on those lines. This is fine.
Many thanks for now making plk not write out the “1”s by default! And, thanks for taking away the huge padding around START/FINISH. Both of those are very helpful fixes for my workflow!
And, it makes sense that if you have zoomed in using “s” and “f” keys in plk, then it does write START and FINISH with the “1”s.
So, that branch looks like an improvement to me!
-
Ok - Glad you think it makes sense. I also am not 100% sure if it makes any sense to define START and FINISH without frequencies, but in general I think this is mostly useful when working on a fixed and known dataset to e.g. fit only to half of the data and then compare to the results of fitting on the other half. Using START/FINISH from one dataset and then applying to another (e.g. fitting on radio and applying to gamma) is not really a use case that anyone has thought about I think. If this is something you would find useful we can look at it, but otherwise I suggest we leave it as is.
-
- changed status to resolved
I'm marking this resolved.
- Log in to comment
I agree. I’ll add it to my list of quality of life fixes and try to look into it soon.
I really hate the way that it seems to randomly add ‘1' to the end of the START/FINISH parameters. I think I looked into this once before and it wasn’t completely obvious how to fix it, but I didn’t put in a lot of effort. I think the padding is something to do with barycentric vs site arrival times and some laziness in which is used for what. I don’t really see why it can’t be fixable.