Add QR flag to get Toshiba CT images

Issue #546 resolved
Ed McDonagh
created an issue

Refs #480.

Comments (43)

  1. Ed McDonagh reporter

    @David Platten if you do get an opportunity to test this, you will need to do a quick manage.py makemigrations remapp and manage.py migrate remapp as I've added a field to the series level query model.

    In theory, if you set the -toshiba flag, then if there are no RDSR or ESR objects in a CT study response, and no Dose info series, then it will fetch the following:

    • A series that is secondary capture: get everything
    • Otherwise, get just the first image
  2. David Platten

    When using the -toshiba flag the code fails when it encounters a Toshiba study and then doesn't progress with the rest of the query. It seems that the code removes all the images from the series, rather than all but one.

    Here's the last part of my log file:

    [05/Oct/2017 12:50:43] INFO [remapp.netdicom.qrscu:205] Query_id 961a1b92-1a90-4140-89d6-37d6b61582e5: Query for Toshiba images. Have 2 in this series.
    [05/Oct/2017 12:50:50] INFO [remapp.netdicom.qrscu:205] Query_id 961a1b92-1a90-4140-89d6-37d6b61582e5: Query for Toshiba images. Have 87 in this series.
    [05/Oct/2017 12:50:50] INFO [remapp.netdicom.qrscu:211] Query_id 961a1b92-1a90-4140-89d6-37d6b61582e5: In non secondary capture series, SOPClassUID 1.2.840.10008.5.1.4.1.1.2. Will delete all but first image.
    [05/Oct/2017 12:50:50] INFO [remapp.netdicom.qrscu:214] Query_id 961a1b92-1a90-4140-89d6-37d6b61582e5: Deleted other images, now 0 remaining (should be 1)
    [05/Oct/2017 12:50:56] INFO [remapp.netdicom.qrscu:205] Query_id 961a1b92-1a90-4140-89d6-37d6b61582e5: Query for Toshiba images. Have 59 in this series.
    [05/Oct/2017 12:50:56] INFO [remapp.netdicom.qrscu:211] Query_id 961a1b92-1a90-4140-89d6-37d6b61582e5: In non secondary capture series, SOPClassUID 1.2.840.10008.5.1.4.1.1.2. Will delete all but first image.
    [05/Oct/2017 12:50:56] INFO [remapp.netdicom.qrscu:214] Query_id 961a1b92-1a90-4140-89d6-37d6b61582e5: Deleted other images, now 0 remaining (should be 1)
    [05/Oct/2017 12:52:29] INFO [remapp.netdicom.qrscu:205] Query_id 961a1b92-1a90-4140-89d6-37d6b61582e5: Query for Toshiba images. Have 1261 in this series.
    [05/Oct/2017 12:52:29] INFO [remapp.netdicom.qrscu:211] Query_id 961a1b92-1a90-4140-89d6-37d6b61582e5: In non secondary capture series, SOPClassUID 1.2.840.10008.5.1.4.1.1.2. Will delete all but first image.
    [05/Oct/2017 12:52:29] INFO [remapp.netdicom.qrscu:214] Query_id 961a1b92-1a90-4140-89d6-37d6b61582e5: Deleted other images, now 0 remaining (should be 1)
    [05/Oct/2017 12:52:30] INFO [remapp.netdicom.qrscu:205] Query_id 961a1b92-1a90-4140-89d6-37d6b61582e5: Query for Toshiba images. Have 2 in this series.
    [05/Oct/2017 12:52:30] INFO [remapp.netdicom.qrscu:211] Query_id 961a1b92-1a90-4140-89d6-37d6b61582e5: In non secondary capture series, SOPClassUID 1.2.840.10008.5.1.4.1.1.2. Will delete all but first image.
    [05/Oct/2017 12:52:30] INFO [remapp.netdicom.qrscu:214] Query_id 961a1b92-1a90-4140-89d6-37d6b61582e5: Deleted other images, now 0 remaining (should be 1)
    

    I think the offending line in the code is:

    images.exclude(instance_number__exact='0').delete()
    

    Changing the above to:

    images.exclude(instance_number__exact=images[0].instance_number).delete()
    

    Does result in 1 image being left per series, and the query continues.

    I'm waiting to see what happens next - I have a query running now.

  3. Ed McDonagh reporter

    If we are using images[0], then we could just as well use any identifier such as sop instance UID, but I guess instance number is just as good. I wasn't sure if there is always an instance number '0' or not, and it looks like maybe there isn't?

  4. David Platten

    There seems to be an instance number for what's been looked at by the query so far. As you say, it's probably better to use a field we know will definitely be present, and that is also unique to that image within the series.

  5. Ed McDonagh reporter

    I meant whether there would be an instance number of zero rather than whether the field would exist, but we may as well change it to SOPInstanceUID once you've tested the current code anyway.

  6. David Platten

    The secondary capture series images aren't ever retrieved with the current code. It seems that the series-level retrieve just doesn't work. If I modify to the following then the secondary capture images are retrieved (using an image-level retrieve). Note that this code includes the series.save() that is missing from the current code, and also uses sop_instance_uid to exclude all but one image in non-secondary capture series. I'll commit this later. I don't know why a series-level retrieve fails to move any images.

    def _get_toshiba_dose_images(study_series, assoc, query_id):
        """
        Get images for Toshiba studies with no RDSR
        :param study_series: database set
        :return: None. Non-useful entries will be removed from database
        """
    
        for series in study_series:
            _query_images(assoc, series, query_id)
            images = series.dicomqrrspimage_set.all()
            logger.debug(u"Query_id {0}: Query for Toshiba images. Have {1} in this series.".format(
                query_id, images.count()))
            if images.count() == 0:
                logger.debug("Query_id {0}: No images in series! Deleting series.".format(query_id))
                series.delete()
            if images[0].sop_class_uid != '1.2.840.10008.5.1.4.1.1.7':
                logger.debug("Query_id {0}: In non secondary capture series, SOPClassUID {1}. "
                             "Will delete all but first image.".format(query_id, images[0].sop_class_uid))
                images.exclude(sop_instance_uid__exact=images[0].sop_instance_uid).delete()
                logger.debug("Query_id {0}: Deleted other images, now {1} remaining (should be 1)".format(
                    query_id, images.count()))
                series.image_level_move = True
                series.save()
            if images[0].sop_class_uid == '1.2.840.10008.5.1.4.1.1.7':
                logger.debug("Query_id {0}: In secondary capture series, SOPClassUID {1}. "
                             "Will not delete any images.".format(query_id, images[0].sop_class_uid))
                series.image_level_move = True
                series.save()
    
  7. Ed McDonagh reporter

    I don't understand either. If we leave it as you have proposed above, then it isn't the end of the world as we'll be doing image level moves on series with about two images in.

    However, I'd appreciate it if you would try the following to see if it shines any new light:

    1. Rather than if == SC SOP, just use an else
    2. Leave the debug message, but remove the last two lines that make it an image level move
    3. In movescu, add some logger.debug messages into the else that does the series level move request printing the d so we can see if it just doesn't get there, or if the move request is malformed somehow.

    Thanks

  8. David Platten

    OK - I'll give that a go. I'm now testing this locally on my computer with two instances of Conquest running: one as "PACS" and the other as the OpenREM SCP. I'm running the code in PyCharm so that I can debug it properly.

  9. David Platten

    Here's the qr log with d being written out using this code:

                if series.image_level_move:
                    logger.info(u"Image-level move - d is: {0}".format(d))
                    d.QueryRetrieveLevel = "IMAGE"
                    for image in series.dicomqrrspimage_set.all():
                        d.SOPInstanceUID = image.sop_instance_uid
                        _move_req(my_ae, assoc, d, study_no, series_no)
                else:
                    logger.info(u"Series-level move - d is: {0}".format(d))
                    _move_req(my_ae, assoc, d, study_no, series_no)
    
    [10/Oct/2017 13:04:50] INFO [remapp.netdicom.qrscu:921] qrscu script called
    [10/Oct/2017 13:04:50] INFO [remapp.netdicom.qrscu:941] Modalities are ['CT']
    [10/Oct/2017 13:04:50] INFO [remapp.netdicom.qrscu:947] Date from: 2017-10-04
    [10/Oct/2017 13:04:50] INFO [remapp.netdicom.qrscu:950] Date until: 2017-10-04
    [10/Oct/2017 13:04:51] INFO [remapp.netdicom.qrscu:447] Query_id 53f5d8a8-af5b-4dfa-bf04-1d9b369cc151: Request association with {'Port': 5678, 'AET': 'PACS', 'Address': u'127.0.0.1'} (127.0.0.1 5678 from <AE(OPENREM, stopped daemon 6608)>)
    [10/Oct/2017 13:04:51] INFO [remapp.netdicom.qrscu:621] DICOM Echo ... 
    [10/Oct/2017 13:04:51] INFO [remapp.netdicom.qrscu:634] DICOM FindSCU ... 
    [10/Oct/2017 13:04:51] INFO [remapp.netdicom.qrscu:492] Currently querying for CT studies...
    [10/Oct/2017 13:04:51] INFO [remapp.netdicom.qrscu:686] Checking to see if any of the 2 studies are already in the OpenREM database
    [10/Oct/2017 13:04:51] INFO [remapp.netdicom.qrscu:691] After removing studies we already have in the db, 1 studies are left
    [10/Oct/2017 13:04:51] INFO [remapp.netdicom.qrscu:703] Pruning study responses based on inc/exc options: 
    [10/Oct/2017 13:04:51] INFO [remapp.netdicom.qrscu:706] Now have 1 studies
    [10/Oct/2017 13:04:52] INFO [remapp.netdicom.qrscu:85] Deleting series we can't use
    [10/Oct/2017 13:05:23] INFO [remapp.netdicom.qrscu:746] Now have 1 studies
    [10/Oct/2017 13:05:39] INFO [remapp.netdicom.qrscu:808] Query_id 53f5d8a8-af5b-4dfa-bf04-1d9b369cc151: Move association requested
    [10/Oct/2017 13:05:40] INFO [remapp.netdicom.qrscu:812] Query_id 53f5d8a8-af5b-4dfa-bf04-1d9b369cc151: Preparing to start move request
    [10/Oct/2017 13:05:40] INFO [remapp.netdicom.qrscu:817] Query_id 53f5d8a8-af5b-4dfa-bf04-1d9b369cc151: Requesting move of 1 studies
    [10/Oct/2017 13:05:40] INFO [remapp.netdicom.qrscu:841] Requesting move: modality CT, study 1 (of 1) series 1 (of 5). Series contains 2 objects
    [10/Oct/2017 13:05:40] INFO [remapp.netdicom.qrscu:857] Image-level move - d is: (0008, 0052) Query/Retrieve Level                CS: 'SERIES'
    (0020, 000d) Study Instance UID                  UI: 1.2.840.113619.6.95.31.0.3.4.1.1791.13.5062077
    (0020, 000e) Series Instance UID                 UI: 1.2.392.200036.9116.2.5.1.48.1221172435.1507088174.447101
    [10/Oct/2017 13:05:41] INFO [remapp.netdicom.qrscu:769] Move of study 1, series 1 status is Pending (i.e. one object processed)
    [10/Oct/2017 13:05:41] INFO [remapp.netdicom.qrscu:841] Requesting move: modality CT, study 1 (of 1) series 2 (of 5). Series contains 124 objects
    [10/Oct/2017 13:05:41] INFO [remapp.netdicom.qrscu:857] Image-level move - d is: (0008, 0018) SOP Instance UID                    UI: 1.2.392.200036.9116.2.5.1.48.1221172435.1507088174.454749
    (0008, 0052) Query/Retrieve Level                CS: 'SERIES'
    (0020, 000d) Study Instance UID                  UI: 1.2.840.113619.6.95.31.0.3.4.1.1791.13.5062077
    (0020, 000e) Series Instance UID                 UI: 1.2.392.200036.9116.2.5.1.48.1221172435.1507088291.440666
    [10/Oct/2017 13:05:41] INFO [remapp.netdicom.qrscu:769] Move of study 1, series 2 status is Pending (i.e. one object processed)
    [10/Oct/2017 13:05:41] INFO [remapp.netdicom.qrscu:841] Requesting move: modality CT, study 1 (of 1) series 3 (of 5). Series contains 77 objects
    [10/Oct/2017 13:05:41] INFO [remapp.netdicom.qrscu:857] Image-level move - d is: (0008, 0018) SOP Instance UID                    UI: 1.2.392.200036.9116.2.5.1.48.1221172435.1507088291.449731
    (0008, 0052) Query/Retrieve Level                CS: 'SERIES'
    (0020, 000d) Study Instance UID                  UI: 1.2.840.113619.6.95.31.0.3.4.1.1791.13.5062077
    (0020, 000e) Series Instance UID                 UI: 1.2.392.200036.9116.2.5.1.48.1221172435.1507088430.396322
    [10/Oct/2017 13:05:41] INFO [remapp.netdicom.qrscu:769] Move of study 1, series 3 status is Pending (i.e. one object processed)
    [10/Oct/2017 13:05:41] INFO [remapp.netdicom.qrscu:841] Requesting move: modality CT, study 1 (of 1) series 4 (of 5). Series contains 100 objects
    [10/Oct/2017 13:05:41] INFO [remapp.netdicom.qrscu:857] Image-level move - d is: (0008, 0018) SOP Instance UID                    UI: 1.2.392.200036.9116.2.5.1.48.1221172435.1507088430.527850
    (0008, 0052) Query/Retrieve Level                CS: 'SERIES'
    (0020, 000d) Study Instance UID                  UI: 1.2.840.113619.6.95.31.0.3.4.1.1791.13.5062077
    (0020, 000e) Series Instance UID                 UI: 1.2.392.200036.9116.2.5.1.48.1221172435.1507088440.779778
    [10/Oct/2017 13:05:41] INFO [remapp.netdicom.qrscu:769] Move of study 1, series 4 status is Pending (i.e. one object processed)
    [10/Oct/2017 13:05:41] INFO [remapp.netdicom.qrscu:841] Requesting move: modality CT, study 1 (of 1) series 5 (of 5). Series contains 2 objects
    [10/Oct/2017 13:05:41] INFO [remapp.netdicom.qrscu:863] Series-level move - d is: (0008, 0018) SOP Instance UID                    UI: 1.2.392.200036.9116.2.5.1.48.1221172435.1507088440.921531
    (0008, 0052) Query/Retrieve Level                CS: 'SERIES'
    (0020, 000d) Study Instance UID                  UI: 1.2.840.113619.6.95.31.0.3.4.1.1791.13.5062077
    (0020, 000e) Series Instance UID                 UI: 1.2.392.200036.9116.2.5.1.48.1221172435.1507088343.920457
    [10/Oct/2017 13:05:41] INFO [remapp.netdicom.qrscu:868] Move complete
    [10/Oct/2017 13:05:42] INFO [remapp.netdicom.qrscu:872] Query_id 53f5d8a8-af5b-4dfa-bf04-1d9b369cc151: Move association released
    
  10. David Platten

    This is the corresponding extract from the PACS Conquest:

    [PACS] UPACS THREAD 62: STARTED AT: Tue Oct 10 13:04:51 2017
    [PACS]  Calling Application Title : "OPENREM"
    [PACS]  Called Application Title : "PACS"
    [PACS]  Application Context : "1.2.840.10008.3.1.1.1", PDU length: 16000
    [PACS]  Presentation Context 0 "1.2.840.10008.5.1.4.1.2.2.1" 1
    [PACS]  Presentation Context 1 "1.2.840.10008.5.1.4.1.2.2.2" 1
    [PACS]  Presentation Context 2 "1.2.840.10008.1.1" 1
    [PACS] C-Echo
    [PACS] (StudyRootQuery) search level: STUDY 
    [PACS] C-Find (StudyRoot) located 2 records
    [PACS] (StudyRootQuery) search level: SERIES
    [PACS] C-Find (StudyRoot) located 5 records
    [PACS] (StudyRootQuery) search level: IMAGE 
    [PACS] C-Find (StudyRoot) located 2 records
    [PACS] (StudyRootQuery) search level: IMAGE 
    [PACS] C-Find (StudyRoot) located 124 records
    [PACS] (StudyRootQuery) search level: IMAGE 
    [PACS] C-Find (StudyRoot) located 77 records
    [PACS] (StudyRootQuery) search level: IMAGE 
    [PACS] C-Find (StudyRoot) located 100 records
    [PACS] (StudyRootQuery) search level: IMAGE 
    [PACS] C-Find (StudyRoot) located 2 records
    [PACS] 
    [PACS] UPACS THREAD 63: STARTED AT: Tue Oct 10 13:05:39 2017
    [PACS]  Calling Application Title : "OPENREM"
    [PACS]  Called Application Title : "PACS"
    [PACS]  Application Context : "1.2.840.10008.3.1.1.1", PDU length: 16000
    [PACS]  Presentation Context 0 "1.2.840.10008.5.1.4.1.2.2.1" 1
    [PACS]  Presentation Context 1 "1.2.840.10008.5.1.4.1.2.2.2" 1
    [PACS]  Presentation Context 2 "1.2.840.10008.1.1" 1
    [PACS]  C-Move Destination: "OPENREM "
    [PACS] Number of Images to send: 1
    [PACS] Sending file : C:\DJP\apps\conquest_testing\conquest_pacs\data\5928212310759232\1.2.392.200036.9116.2.5.1.48.1221172435.1507088174.447101_0001_000001_1507622082019d.dcm
    [PACS] C-Move (StudyRoot)
    [PACS]  C-Move Destination: "OPENREM "
    [PACS] Number of Images to send: 1
    [PACS] Sending file : C:\DJP\apps\conquest_testing\conquest_pacs\data\5928212310759232\1.2.392.200036.9116.2.5.1.48.1221172435.1507088291.440666_0002_000001_1507622082019f.dcm
    [PACS] C-Move (StudyRoot)
    [PACS]  C-Move Destination: "OPENREM "
    [PACS] Number of Images to send: 1
    [PACS] Sending file : C:\DJP\apps\conquest_testing\conquest_pacs\data\5928212310759232\1.2.392.200036.9116.2.5.1.48.1221172435.1507088430.396322_0004_000001_1507622085021b.dcm
    [PACS] C-Move (StudyRoot)
    [PACS]  C-Move Destination: "OPENREM "
    [PACS] Number of Images to send: 1
    [PACS] Sending file : C:\DJP\apps\conquest_testing\conquest_pacs\data\5928212310759232\1.2.392.200036.9116.2.5.1.48.1221172435.1507088440.779778_0005_000001_15076220860268.dcm
    [PACS] C-Move (StudyRoot)
    [PACS]  C-Move Destination: "OPENREM "
    [PACS] C-Move (StudyRoot)
    [PACS] UPACS THREAD 63: ENDED AT: Tue Oct 10 13:05:41 2017
    [PACS] UPACS THREAD 63: TOTAL RUNNING TIME: 2 SECONDS
    [PACS] UPACS THREAD 62: ENDED AT: Tue Oct 10 13:05:52 2017
    [PACS] UPACS THREAD 62: TOTAL RUNNING TIME: 61 SECONDS
    

    and this is the extract from the OpenREM SCP Conquest file:

    [PACS] UPACS THREAD 113: STARTED AT: Tue Oct 10 13:04:50 2017
    [PACS]  Calling Application Title : "OPENREMECHO"
    [PACS]  Called Application Title : "OPENREM"
    [PACS]  Application Context : "1.2.840.10008.3.1.1.1", PDU length: 16000
    [PACS]  Presentation Context 0 "1.2.840.10008.1.1" 1
    [PACS] C-Echo
    [PACS] UPACS THREAD 113: ENDED AT: Tue Oct 10 13:04:51 2017
    [PACS] UPACS THREAD 113: TOTAL RUNNING TIME: 1 SECONDS
    [PACS] 
    [PACS] UPACS THREAD 114: STARTED AT: Tue Oct 10 13:05:41 2017
    [PACS]  Calling Application Title : "PACS            "
    [PACS]  Called Application Title : "OPENREM         "
    [PACS]  Application Context : "1.2.840.10008.3.1.1.1", PDU length: 16384
    [PACS]  Presentation Context 0 "1.2.840.10008.5.1.4.1.1.2" 1
    [PACS] Written file: C:\DJP\apps\conquest_testing\conquest_openrem_scp\data\5928212310759232\1.2.392.200036.9116.2.5.1.48.1221172435.1507088174.447101_0001_000001_15076371410020.dcm
    [PACS] UPACS THREAD 114: ENDED AT: Tue Oct 10 13:05:41 2017
    [PACS] UPACS THREAD 114: TOTAL RUNNING TIME: 0 SECONDS
    [PACS] 
    [PACS] UPACS THREAD 115: STARTED AT: Tue Oct 10 13:05:41 2017
    [PACS]  Calling Application Title : "PACS            "
    [PACS]  Called Application Title : "OPENREM         "
    [PACS]  Application Context : "1.2.840.10008.3.1.1.1", PDU length: 16384
    [PACS]  Presentation Context 0 "1.2.840.10008.5.1.4.1.1.2" 1
    [PACS] Written file: C:\DJP\apps\conquest_testing\conquest_openrem_scp\data\5928212310759232\1.2.392.200036.9116.2.5.1.48.1221172435.1507088291.440666_0002_000001_15076371410021.dcm
    [PACS] UPACS THREAD 115: ENDED AT: Tue Oct 10 13:05:41 2017
    [PACS] UPACS THREAD 115: TOTAL RUNNING TIME: 0 SECONDS
    [PACS] 
    [PACS] UPACS THREAD 116: STARTED AT: Tue Oct 10 13:05:41 2017
    [PACS]  Calling Application Title : "PACS            "
    [PACS]  Called Application Title : "OPENREM         "
    [PACS]  Application Context : "1.2.840.10008.3.1.1.1", PDU length: 16384
    [PACS]  Presentation Context 0 "1.2.840.10008.5.1.4.1.1.2" 1
    [PACS] Written file: C:\DJP\apps\conquest_testing\conquest_openrem_scp\data\5928212310759232\1.2.392.200036.9116.2.5.1.48.1221172435.1507088430.396322_0004_000001_15076371410022.dcm
    [PACS] UPACS THREAD 116: ENDED AT: Tue Oct 10 13:05:41 2017
    [PACS] UPACS THREAD 116: TOTAL RUNNING TIME: 0 SECONDS
    [PACS] 
    [PACS] UPACS THREAD 117: STARTED AT: Tue Oct 10 13:05:41 2017
    [PACS]  Calling Application Title : "PACS            "
    [PACS]  Called Application Title : "OPENREM         "
    [PACS]  Application Context : "1.2.840.10008.3.1.1.1", PDU length: 16384
    [PACS]  Presentation Context 0 "1.2.840.10008.5.1.4.1.1.2" 1
    [PACS] Written file: C:\DJP\apps\conquest_testing\conquest_openrem_scp\data\5928212310759232\1.2.392.200036.9116.2.5.1.48.1221172435.1507088440.779778_0005_000001_15076371410023.dcm
    [PACS] UPACS THREAD 117: ENDED AT: Tue Oct 10 13:05:41 2017
    [PACS] UPACS THREAD 117: TOTAL RUNNING TIME: 0 SECONDS
    
  11. Ed McDonagh reporter

    As you've noticed @David Platten, queries are really slow when using the Toshiba flag because we're querying every image in every series.

    I assume it might be a bit quicker without the debug logging, but I'm thinking maybe we could interrupt the generator after the first few responses?

    Unless there is a way of querying for specific images. I'll have a think/play now I have an example study to play with.

  12. David Platten

    It does take ages: a couple of hours for one day of studies...

    I was thinking that we could pass _query_images the -toshiba flag. We could then check the type of the first image in the series: if it's a secondary capture then query all images; if it's anything else then query the first image of the series and then exit the function.

  13. David Platten

    @Ed McDonagh, the long time required for the query causes a problem: celery becomes unresponsive before the move part, so by the time the initial query has finished the move part just sites in the queue. Restarting celery causes the whole thing to start again, rather than just the move part...

    It would be great if the time required for the initial part of the query could be reduced.

  14. David Platten

    This is much quicker, but doesn't work on my test system: it only retrieves a single image from one study series, and no images or dose summaries for the others.

    I've commented out the two shared_task lines in my test system, and also removed the .delay so that I don't need celery to be running.

    Did it successfully move the required images when you tested it?

    See logs:

    Conquest PACS:

    [PACS] UPACS THREAD 23: STARTED AT: Wed Oct 11 11:31:47 2017
    [PACS]  Calling Application Title : "OPENREM"
    [PACS]  Called Application Title : "PACS"
    [PACS]  Application Context : "1.2.840.10008.3.1.1.1", PDU length: 16000
    [PACS]  Presentation Context 0 "1.2.840.10008.1.1" 1
    [PACS] C-Echo
    [PACS] UPACS THREAD 23: ENDED AT: Wed Oct 11 11:31:47 2017
    [PACS] UPACS THREAD 23: TOTAL RUNNING TIME: 0 SECONDS
    [PACS] 
    [PACS] UPACS THREAD 24: STARTED AT: Wed Oct 11 11:31:48 2017
    [PACS]  Calling Application Title : "OPENREM"
    [PACS]  Called Application Title : "PACS"
    [PACS]  Application Context : "1.2.840.10008.3.1.1.1", PDU length: 16000
    [PACS]  Presentation Context 0 "1.2.840.10008.5.1.4.1.2.2.1" 1
    [PACS]  Presentation Context 1 "1.2.840.10008.5.1.4.1.2.2.2" 1
    [PACS]  Presentation Context 2 "1.2.840.10008.1.1" 1
    [PACS] C-Echo
    [PACS] (StudyRootQuery) search level: STUDY 
    [PACS] C-Find (StudyRoot) located 2 records
    [PACS] (StudyRootQuery) search level: SERIES
    [PACS] C-Find (StudyRoot) located 5 records
    [PACS] (StudyRootQuery) search level: SERIES
    [PACS] C-Find (StudyRoot) located 5 records
    [PACS] (StudyRootQuery) search level: IMAGE 
    [PACS] C-Find (StudyRoot) located 160 records
    [PACS] (StudyRootQuery) search level: IMAGE 
    [PACS] C-Find (StudyRoot) located 122 records
    [PACS] (StudyRootQuery) search level: IMAGE 
    [PACS] C-Find (StudyRoot) located 130 records
    [PACS] (StudyRootQuery) search level: IMAGE 
    [PACS] C-Find (StudyRoot) located 2 records
    [PACS] (StudyRootQuery) search level: IMAGE 
    [PACS] C-Find (StudyRoot) located 2 records
    [PACS] (StudyRootQuery) search level: IMAGE 
    [PACS] C-Find (StudyRoot) located 2 records
    [PACS] (StudyRootQuery) search level: IMAGE 
    [PACS] C-Find (StudyRoot) located 124 records
    [PACS] (StudyRootQuery) search level: IMAGE 
    [PACS] C-Find (StudyRoot) located 77 records
    [PACS] (StudyRootQuery) search level: IMAGE 
    [PACS] C-Find (StudyRoot) located 100 records
    [PACS] (StudyRootQuery) search level: IMAGE 
    [PACS] C-Find (StudyRoot) located 2 records
    [PACS] 
    [PACS] UPACS THREAD 25: STARTED AT: Wed Oct 11 11:32:09 2017
    [PACS]  Calling Application Title : "OPENREM"
    [PACS]  Called Application Title : "PACS"
    [PACS]  Application Context : "1.2.840.10008.3.1.1.1", PDU length: 16000
    [PACS]  Presentation Context 0 "1.2.840.10008.5.1.4.1.2.2.1" 1
    [PACS]  Presentation Context 1 "1.2.840.10008.5.1.4.1.2.2.2" 1
    [PACS]  Presentation Context 2 "1.2.840.10008.1.1" 1
    [PACS]  C-Move Destination: "OPENREM "
    [PACS] Number of Images to send: 1
    [PACS] Sending file : C:\DJP\apps\conquest_testing\conquest_pacs\data\1352938289346305\1.2.392.200036.9116.2.5.1.16.1613466634.1507092675.132000_0003_000007_15076220670000.dcm
    [PACS] C-Move (StudyRoot)
    [PACS]  C-Move Destination: "OPENREM "
    [PACS] C-Move (StudyRoot)
    [PACS]  C-Move Destination: "OPENREM "
    [PACS] C-Move (StudyRoot)
    [PACS]  C-Move Destination: "OPENREM "
    [PACS] C-Move (StudyRoot)
    [PACS]  C-Move Destination: "OPENREM "
    [PACS] C-Move (StudyRoot)
    [PACS]  C-Move Destination: "OPENREM "
    [PACS] C-Move (StudyRoot)
    [PACS]  C-Move Destination: "OPENREM "
    [PACS] C-Move (StudyRoot)
    [PACS]  C-Move Destination: "OPENREM "
    [PACS] C-Move (StudyRoot)
    [PACS]  C-Move Destination: "OPENREM "
    [PACS] C-Move (StudyRoot)
    [PACS]  C-Move Destination: "OPENREM "
    [PACS] C-Move (StudyRoot)
    [PACS] UPACS THREAD 25: ENDED AT: Wed Oct 11 11:32:11 2017
    [PACS] UPACS THREAD 25: TOTAL RUNNING TIME: 2 SECONDS
    [PACS] UPACS THREAD 24: ENDED AT: Wed Oct 11 11:32:23 2017
    [PACS] UPACS THREAD 24: TOTAL RUNNING TIME: 35 SECONDS
    

    QR log:

    [11/Oct/2017 11:31:47] INFO [remapp.netdicom.qrscu:924] qrscu script called
    [11/Oct/2017 11:31:47] INFO [remapp.netdicom.qrscu:944] Modalities are ['CT']
    [11/Oct/2017 11:31:47] INFO [remapp.netdicom.qrscu:950] Date from: 2017-10-04
    [11/Oct/2017 11:31:47] INFO [remapp.netdicom.qrscu:953] Date until: 2017-10-04
    [11/Oct/2017 11:31:48] INFO [remapp.netdicom.qrscu:450] Query_id e915cfd0-df68-4061-b9f2-cd43079df103: Request association with {'Port': 5678, 'AET': 'PACS', 'Address': u'127.0.0.1'} (127.0.0.1 5678 from <AE(OPENREM, stopped daemon 6540)>)
    [11/Oct/2017 11:31:48] INFO [remapp.netdicom.qrscu:624] DICOM Echo ... 
    [11/Oct/2017 11:31:48] INFO [remapp.netdicom.qrscu:637] DICOM FindSCU ... 
    [11/Oct/2017 11:31:48] INFO [remapp.netdicom.qrscu:495] Currently querying for CT studies...
    [11/Oct/2017 11:31:48] INFO [remapp.netdicom.qrscu:689] Checking to see if any of the 2 studies are already in the OpenREM database
    [11/Oct/2017 11:31:48] INFO [remapp.netdicom.qrscu:694] After removing studies we already have in the db, 2 studies are left
    [11/Oct/2017 11:31:48] INFO [remapp.netdicom.qrscu:706] Pruning study responses based on inc/exc options: 
    [11/Oct/2017 11:31:48] INFO [remapp.netdicom.qrscu:709] Now have 2 studies
    [11/Oct/2017 11:31:50] INFO [remapp.netdicom.qrscu:85] Deleting series we can't use
    [11/Oct/2017 11:31:51] INFO [remapp.netdicom.qrscu:749] Now have 2 studies
    [11/Oct/2017 11:32:09] INFO [remapp.netdicom.qrscu:811] Query_id e915cfd0-df68-4061-b9f2-cd43079df103: Move association requested
    [11/Oct/2017 11:32:09] INFO [remapp.netdicom.qrscu:815] Query_id e915cfd0-df68-4061-b9f2-cd43079df103: Preparing to start move request
    [11/Oct/2017 11:32:09] INFO [remapp.netdicom.qrscu:820] Query_id e915cfd0-df68-4061-b9f2-cd43079df103: Requesting move of 2 studies
    [11/Oct/2017 11:32:09] INFO [remapp.netdicom.qrscu:844] Requesting move: modality CT, study 1 (of 2) series 1 (of 5). Series contains 160 objects
    [11/Oct/2017 11:32:10] INFO [remapp.netdicom.qrscu:772] Move of study 1, series 1 status is Pending (i.e. one object processed)
    [11/Oct/2017 11:32:10] INFO [remapp.netdicom.qrscu:844] Requesting move: modality CT, study 1 (of 2) series 2 (of 5). Series contains 122 objects
    [11/Oct/2017 11:32:10] INFO [remapp.netdicom.qrscu:844] Requesting move: modality CT, study 1 (of 2) series 3 (of 5). Series contains 130 objects
    [11/Oct/2017 11:32:10] INFO [remapp.netdicom.qrscu:844] Requesting move: modality CT, study 1 (of 2) series 4 (of 5). Series contains 2 objects
    [11/Oct/2017 11:32:10] INFO [remapp.netdicom.qrscu:844] Requesting move: modality CT, study 1 (of 2) series 5 (of 5). Series contains 2 objects
    [11/Oct/2017 11:32:10] INFO [remapp.netdicom.qrscu:844] Requesting move: modality CT, study 2 (of 2) series 1 (of 5). Series contains 2 objects
    [11/Oct/2017 11:32:10] INFO [remapp.netdicom.qrscu:844] Requesting move: modality CT, study 2 (of 2) series 2 (of 5). Series contains 124 objects
    [11/Oct/2017 11:32:10] INFO [remapp.netdicom.qrscu:844] Requesting move: modality CT, study 2 (of 2) series 3 (of 5). Series contains 77 objects
    [11/Oct/2017 11:32:11] INFO [remapp.netdicom.qrscu:844] Requesting move: modality CT, study 2 (of 2) series 4 (of 5). Series contains 100 objects
    [11/Oct/2017 11:32:11] INFO [remapp.netdicom.qrscu:844] Requesting move: modality CT, study 2 (of 2) series 5 (of 5). Series contains 2 objects
    [11/Oct/2017 11:32:11] INFO [remapp.netdicom.qrscu:871] Move complete
    [11/Oct/2017 11:32:11] INFO [remapp.netdicom.qrscu:875] Query_id e915cfd0-df68-4061-b9f2-cd43079df103: Move association released
    
  15. Ed McDonagh reporter

    I have the same situation - after the first move request, none of the subsequent ones get into the generator for loop.

    I'm looking, but I'm in meetings all afternoon now.

  16. David Platten

    I know what the problem is. Below is an extract from the Conquest PACS in debug mode.

    It successfully finds and sends the first image of the first series (series A) in the study.

    What it should then do is send the first image of the second series (series B) of the study. But it doesn't. It tries to find the UID of the second image in series A within series B. Unsurprisingly it finds nothing.

    When trying to send the first image of the third series, it actually tries to find UID of the third image in series A within series C.

    Basically, it always uses SOP instance UIDs from the first series, but the correct UIDs for the series and study. This works for the first image, but nothing else.

    The code in _query_images uses a generator function to go through each image in a function. We're now breaking out of this early. It seems that it remembers where it was when it is called the next time, rather than starting at the first image of the current series. My guess is that this is the problem: before we break out of the loop we need to iterate over the remaining images in the series.

    Changing

        st3 = assoc.StudyRootFindSOPClass.SCU(d3, 1)
    

    to

        st3 = assoc.StudyRootFindSOPClass.SCU(d3, 1)
        st3 = list(st3)
    

    fixes the problem. I think.

        st3 = assoc.StudyRootFindSOPClass.SCU(d3, 1)
        st3 = list(st3)
    
        query_id = uuid.uuid4()
    
        imRspNo = 0
    
        for images in st3:
            if not images[1]:
                continue
            images[1].decode()
            imRspNo += 1
            logger.debug(u"Query_id {0}: Image Response {1}: {2}".format(query_id, imRspNo, images[1]))
            imagesrsp = DicomQRRspImage.objects.create(dicom_qr_rsp_series=seriesrsp)
            imagesrsp.query_id = query_id
            # Mandatory tags
            imagesrsp.sop_instance_uid = images[1].SOPInstanceUID
            imagesrsp.sop_class_uid = images[1].SOPClassUID
            imagesrsp.instance_number = images[1].InstanceNumber
            if not imagesrsp.instance_number:  # just in case!!
                imagesrsp.instance_number = None  # integer so can't be ''
            imagesrsp.save()
            if initial_image_only:
                break
    
    11/10/2017 14:59:41 [PACS] UPACS THREAD 61: STARTED AT: Wed Oct 11 14:59:40 2017
    11/10/2017 14:59:41 [PACS] A-ASSOCIATE-RQ Packet Dump
    11/10/2017 14:59:41 [PACS]  Calling Application Title : "OPENREM"
    11/10/2017 14:59:41 [PACS]  Called Application Title : "PACS"
    11/10/2017 14:59:41 [PACS]  Application Context : "1.2.840.10008.3.1.1.1", PDU length: 16000
    11/10/2017 14:59:41 [PACS]  Number of Proposed Presentation Contexts: 3
    11/10/2017 14:59:41 [PACS]  Presentation Context 0 "1.2.840.10008.5.1.4.1.2.2.1" 1
    11/10/2017 14:59:41 [PACS]  Presentation Context 1 "1.2.840.10008.5.1.4.1.2.2.2" 1
    11/10/2017 14:59:41 [PACS]  Presentation Context 2 "1.2.840.10008.1.1" 1
    11/10/2017 14:59:41 [PACS] Server Command   := 0021
    11/10/2017 14:59:41 [PACS] Message ID       := 0001
    11/10/2017 14:59:41 [PACS]  C-Move Destination: "OPENREM "
    11/10/2017 14:59:41 [PACS] (QualifyOn) (mapped) IP:127.0.0.1, PORT:5688
    11/10/2017 14:59:41 [PACS] MyStudyRootRetrieveGeneric :: SearchOn
    11/10/2017 14:59:41 [PACS] Query On Image
    11/10/2017 14:59:41 [PACS] Issue Query on Columns: DICOMImages.SOPClassUI, DICOMImages.SOPInstanc, DICOMSeries.SeriesInst, DICOMStudies.StudyInsta,DICOMImages.ObjectFile,DICOMImages.DeviceName
    11/10/2017 14:59:41 [PACS] Values: DICOMImages.SOPInstanc = '1.2.392.200036.9116.2.5.1.16.1613466634.1507092677.165873' and DICOMSeries.SeriesInst = '1.2.392.200036.9116.2.5.1.16.1613466634.1507092675.132000' and DICOMStudies.StudyInsta = '1.2.392.200036.9116.2.5.1.16.1613466634.1507092477.524642' and DICOMSeries.StudyInsta = DICOMStudies.StudyInsta and DICOMImages.SeriesInst = DICOMSeries.SeriesInst
    11/10/2017 14:59:41 [PACS] Tables: DICOMImages, DICOMSeries, DICOMStudies
    11/10/2017 14:59:41 [PACS] Records = 1
    11/10/2017 14:59:41 [PACS] Number of Images to send: 1
    11/10/2017 14:59:41 [PACS] MyStudyRootRetrieveGeneric :: RetrieveOn
    11/10/2017 14:59:41 [PACS] Locating file:MAG0 1352938289346305\1.2.392.200036.9116.2.5.1.16.1613466634.1507092675.132000_0003_000007_15076220670000.dcm
    11/10/2017 14:59:41 [PACS] ReadAheadThread: readahead > 0000
    11/10/2017 14:59:41 [PACS] RetrieveOn: givenout < 0000
    11/10/2017 14:59:41 [PACS] Sending file : C:\DJP\apps\conquest_testing\conquest_pacs\data\1352938289346305\1.2.392.200036.9116.2.5.1.16.1613466634.1507092675.132000_0003_000007_15076220670000.dcm
    11/10/2017 14:59:41 [PACS] Image Loaded from Read Ahead Thread, returning TRUE
    11/10/2017 14:59:41 [PACS] C-Move (StudyRoot)
    11/10/2017 14:59:41 [PACS] Server Command   := 0021
    11/10/2017 14:59:41 [PACS] Message ID       := 0001
    11/10/2017 14:59:41 [PACS]  C-Move Destination: "OPENREM "
    11/10/2017 14:59:41 [PACS] (QualifyOn) (mapped) IP:127.0.0.1, PORT:5688
    11/10/2017 14:59:41 [PACS] MyStudyRootRetrieveGeneric :: SearchOn
    11/10/2017 14:59:41 [PACS] Query On Image
    11/10/2017 14:59:41 [PACS] Issue Query on Columns: DICOMImages.SOPClassUI, DICOMImages.SOPInstanc, DICOMSeries.SeriesInst, DICOMStudies.StudyInsta,DICOMImages.ObjectFile,DICOMImages.DeviceName
    11/10/2017 14:59:41 [PACS] Values: DICOMImages.SOPInstanc = '1.2.392.200036.9116.2.5.1.16.1613466634.1507092677.510427' and DICOMSeries.SeriesInst = '1.2.392.200036.9116.2.5.1.16.1613466634.1507092859.165572' and DICOMStudies.StudyInsta = '1.2.392.200036.9116.2.5.1.16.1613466634.1507092477.524642' and DICOMSeries.StudyInsta = DICOMStudies.StudyInsta and DICOMImages.SeriesInst = DICOMSeries.SeriesInst
    11/10/2017 14:59:41 [PACS] Tables: DICOMImages, DICOMSeries, DICOMStudies
    11/10/2017 14:59:41 [PACS] Records = 0
    11/10/2017 14:59:41 [PACS] Retrieve: move search failed
    
  17. Ed McDonagh reporter

    Once you've created a generator, you can go back to it again and again and it will remember where you were and offer the next item, which is what is happening here. But I don't quite understand why calling it again doesn't start again. I'm guessing that because we're using the same association something is persisting between calls. Ggggrrrr.

  18. Ed McDonagh reporter

    I don't think so, but I will look at the pynetdicom coffee again.

    I've implemented an index that gets passed to the function to modify the message ID (currently always 1) to see if that does the trick. Initial run-through just as my train pulled into my station suggested it didn't, but I haven't checked I got the code right yet. I might get an opportunity later tonight, I might not.

  19. David Platten

    It works - many, many thanks @Ed McDonagh. I've just queried for a day of CT examinations here (140 studies). This would normally take around 6 - 7 hours to complete. With the new code the whole query and retrieve was done in 40 minutes. This is going to make things much easier for me.

  20. Log in to comment