Only tag guaranteed to be returned that will distinguish them is SeriesInstanceUID
Add series_instance_uid to GeneralStudyModuleAttr model along side the series time and content time already added for ref #114
Add recording of SeriesInstanceUID in rdsr.py
No point doing any logic with SeriesInstanceUID in rdsr.py because it is already covered by the times and the UID doesn't tell you which one is later
In the query function, look for StudyInstanceUID matches, then if found look for SeriesInstanceUID
If there is only one RDSR, we can stop there
If there are multiple RDSR series, we'll always have additional ones
Therefore attempt to get the series time also? Will require SeriesTime to be added to the series C-Find and to the DicomQRRspSeries model
If we have the times, we can work out if the one being considered is newer or older and act accordingly
If the study in the database does not have SeriesInstanceUID recorded, rely on times?
If the study in the database does not have times recorded, we import again
If we import again, the logic will cause the existing study to be deleted and the new one to replace it, which will then have the times and SeriesInstanceUID values, preventing a repeat next time!
I'll do another comment later to consider how this will affect DX and MG images before implementing it.
Current behaviour for MG and DX images is as follows (extractor modules):
Check StudyInstanceUID against database
If existing, check SOPInstanceUID against the database in the field projectionxrayradiationdose__irradeventxraydata__irradiation_event_uid (not limited to the same StudyInstanceUID!)
When importing new images, SOPInstanceUID is recorded as though it were IrradiationEventUID
Therefore for C-Find filtering we need to something similar.
If StudyInstanceUID in database is true:
For each series_rsp if modality is not SR (or is MG or DX or CR):
For each images_rsp check if SOPInstanceUID is in database
Next I need to see what we do for legacy Philips CT...
For legacy Philips CT (ct_philips.py), we do not consider the possibility of there being more than one 'Dose Info' object for the same study, so we don't need to here either.
For legacy Toshiba CT (rdsr_toshiba_fct_from_dose_images.py), we are creating the RDSR so there won't be multiple ones. If we come across the same StudyInstanceUID again, I don't think we want to fetch it again to see if it has changed.
So, to bring it together:
Do all the normal filtering, then if remove_duplicates:
if StudyInstanceUID in database:
for series in series_rsp:
if series.modality == "SR":
if SeriesInstanceUID in database: then delete else
if SeriesTime earlier than series_time in database then delete
if series.modality in ['MG', 'DX', 'CR']:
for image in images_rsp:
if SOPInstanceUID in database: (in IrradiationEventUID field), delete
I'm now thinking it would be best to remove all but the latest date RDSR of a study at the same time? Leave them all in if not remove_duplicates.
Slight modification - will need to _query_images in order to cater for mammo and DX. Currently this only happens if in _get_toshiba_dose_imagesor _check_sr_type_in_study.
Added SeriesInstanceUID to GeneralStudyModuleAttr model. Added SeriesTime to DicomQRRspSeries model.
Added SeriesInstanceUID to rdsr extractor.
Moved remove_duplicates to much later in process to ensure series level data is available. Started to implement ref #610
Added checking of SOP instance UID to DX _create_event. Also changed logic for multiple studies with the same UID - previous code would have fallen over. Need to add UID stuff to new DX study function.
[skip ci] because tested locally. Refs #610
Explicitly named mam log for consistency. Added record_sop for new studies. Added tests for importing new event and duplicate vent a second time. Lots of duplication still... for another time! Refs #610, #628
Tested in production. check_uid.check_uid was preventing UIDs to be collected, and isn't needed in extract_common. Removed. Added debug logging to duplicates QR code. Added query stage updates for web interface. Refs #610, #628