1. OpenREM Avatar OpenREM
  2. OpenREM
  3. OpenREM
  4. Issues

Add QR flag to get Toshiba CT images

Issue #546 resolved
Ed McDonagh created an issue

Refs #480.

Comments (43)

  7. Ed McDonagh reporter

    @dplatten if you do get an opportunity to test this, you will need to do a quick manage.py makemigrations remapp and manage.py migrate remapp as I've added a field to the series level query model.

    In theory, if you set the -toshiba flag, then if there are no RDSR or ESR objects in a CT study response, and no Dose info series, then it will fetch the following:

    • A series that is secondary capture: get everything
    • Otherwise, get just the first image
  9. David Platten

    When using the -toshiba flag the code fails when it encounters a Toshiba study and then doesn't progress with the rest of the query. It seems that the code removes all the images from the series, rather than all but one.

    Here's the last part of my log file:

    [05/Oct/2017 12:50:43] INFO [remapp.netdicom.qrscu:205] Query_id 961a1b92-1a90-4140-89d6-37d6b61582e5: Query for Toshiba images. Have 2 in this series.
[05/Oct/2017 12:50:50] INFO [remapp.netdicom.qrscu:205] Query_id 961a1b92-1a90-4140-89d6-37d6b61582e5: Query for Toshiba images. Have 87 in this series.
[05/Oct/2017 12:50:50] INFO [remapp.netdicom.qrscu:211] Query_id 961a1b92-1a90-4140-89d6-37d6b61582e5: In non secondary capture series, SOPClassUID 1.2.840.10008.5.1.4.1.1.2. Will delete all but first image.
[05/Oct/2017 12:50:50] INFO [remapp.netdicom.qrscu:214] Query_id 961a1b92-1a90-4140-89d6-37d6b61582e5: Deleted other images, now 0 remaining (should be 1)
[05/Oct/2017 12:50:56] INFO [remapp.netdicom.qrscu:205] Query_id 961a1b92-1a90-4140-89d6-37d6b61582e5: Query for Toshiba images. Have 59 in this series.
[05/Oct/2017 12:50:56] INFO [remapp.netdicom.qrscu:211] Query_id 961a1b92-1a90-4140-89d6-37d6b61582e5: In non secondary capture series, SOPClassUID 1.2.840.10008.5.1.4.1.1.2. Will delete all but first image.
[05/Oct/2017 12:50:56] INFO [remapp.netdicom.qrscu:214] Query_id 961a1b92-1a90-4140-89d6-37d6b61582e5: Deleted other images, now 0 remaining (should be 1)
[05/Oct/2017 12:52:29] INFO [remapp.netdicom.qrscu:205] Query_id 961a1b92-1a90-4140-89d6-37d6b61582e5: Query for Toshiba images. Have 1261 in this series.
[05/Oct/2017 12:52:29] INFO [remapp.netdicom.qrscu:211] Query_id 961a1b92-1a90-4140-89d6-37d6b61582e5: In non secondary capture series, SOPClassUID 1.2.840.10008.5.1.4.1.1.2. Will delete all but first image.
[05/Oct/2017 12:52:29] INFO [remapp.netdicom.qrscu:214] Query_id 961a1b92-1a90-4140-89d6-37d6b61582e5: Deleted other images, now 0 remaining (should be 1)
[05/Oct/2017 12:52:30] INFO [remapp.netdicom.qrscu:205] Query_id 961a1b92-1a90-4140-89d6-37d6b61582e5: Query for Toshiba images. Have 2 in this series.
[05/Oct/2017 12:52:30] INFO [remapp.netdicom.qrscu:211] Query_id 961a1b92-1a90-4140-89d6-37d6b61582e5: In non secondary capture series, SOPClassUID 1.2.840.10008.5.1.4.1.1.2. Will delete all but first image.
[05/Oct/2017 12:52:30] INFO [remapp.netdicom.qrscu:214] Query_id 961a1b92-1a90-4140-89d6-37d6b61582e5: Deleted other images, now 0 remaining (should be 1)

    I think the offending line in the code is:

    images.exclude(instance_number__exact='0').delete()

    Changing the above to:

    images.exclude(instance_number__exact=images[0].instance_number).delete()

    Does result in 1 image being left per series, and the query continues.

    I'm waiting to see what happens next - I have a query running now.

  10. Ed McDonagh reporter

    I think you are right. That line was worrying me, and I didn't have anything to test it with.

  11. Ed McDonagh reporter

    If we are using images[0], then we could just as well use any identifier such as sop instance UID, but I guess instance number is just as good. I wasn't sure if there is always an instance number '0' or not, and it looks like maybe there isn't?

  12. David Platten

    There seems to be an instance number for what's been looked at by the query so far. As you say, it's probably better to use a field we know will definitely be present, and that is also unique to that image within the series.

  13. Ed McDonagh reporter

    I meant whether there would be an instance number of zero rather than whether the field would exist, but we may as well change it to SOPInstanceUID once you've tested the current code anyway.

  15. David Platten

    The secondary capture series images aren't ever retrieved with the current code. It seems that the series-level retrieve just doesn't work. If I modify to the following then the secondary capture images are retrieved (using an image-level retrieve). Note that this code includes the series.save() that is missing from the current code, and also uses sop_instance_uid to exclude all but one image in non-secondary capture series. I'll commit this later. I don't know why a series-level retrieve fails to move any images.

    def _get_toshiba_dose_images(study_series, assoc, query_id):
    """
    Get images for Toshiba studies with no RDSR
    :param study_series: database set
    :return: None. Non-useful entries will be removed from database
    """

    for series in study_series:
        _query_images(assoc, series, query_id)
        images = series.dicomqrrspimage_set.all()
        logger.debug(u"Query_id {0}: Query for Toshiba images. Have {1} in this series.".format(
            query_id, images.count()))
        if images.count() == 0:
            logger.debug("Query_id {0}: No images in series! Deleting series.".format(query_id))
            series.delete()
        if images[0].sop_class_uid != '1.2.840.10008.5.1.4.1.1.7':
            logger.debug("Query_id {0}: In non secondary capture series, SOPClassUID {1}. "
                         "Will delete all but first image.".format(query_id, images[0].sop_class_uid))
            images.exclude(sop_instance_uid__exact=images[0].sop_instance_uid).delete()
            logger.debug("Query_id {0}: Deleted other images, now {1} remaining (should be 1)".format(
                query_id, images.count()))
            series.image_level_move = True
            series.save()
        if images[0].sop_class_uid == '1.2.840.10008.5.1.4.1.1.7':
            logger.debug("Query_id {0}: In secondary capture series, SOPClassUID {1}. "
                         "Will not delete any images.".format(query_id, images[0].sop_class_uid))
            series.image_level_move = True
            series.save()
  16. Ed McDonagh reporter

    I don't understand either. If we leave it as you have proposed above, then it isn't the end of the world as we'll be doing image level moves on series with about two images in.

    However, I'd appreciate it if you would try the following to see if it shines any new light:

    1. Rather than if == SC SOP, just use an else
    2. Leave the debug message, but remove the last two lines that make it an image level move
    3. In movescu, add some logger.debug messages into the else that does the series level move request printing the d so we can see if it just doesn't get there, or if the move request is malformed somehow.

    Thanks

  17. David Platten

    OK - I'll give that a go. I'm now testing this locally on my computer with two instances of Conquest running: one as "PACS" and the other as the OpenREM SCP. I'm running the code in PyCharm so that I can debug it properly.

  18. David Platten

    Here's the qr log with d being written out using this code:

                if series.image_level_move:
                logger.info(u"Image-level move - d is: {0}".format(d))
                d.QueryRetrieveLevel = "IMAGE"
                for image in series.dicomqrrspimage_set.all():
                    d.SOPInstanceUID = image.sop_instance_uid
                    _move_req(my_ae, assoc, d, study_no, series_no)
            else:
                logger.info(u"Series-level move - d is: {0}".format(d))
                _move_req(my_ae, assoc, d, study_no, series_no)

    [10/Oct/2017 13:04:50] INFO [remapp.netdicom.qrscu:921] qrscu script called
[10/Oct/2017 13:04:50] INFO [remapp.netdicom.qrscu:941] Modalities are ['CT']
[10/Oct/2017 13:04:50] INFO [remapp.netdicom.qrscu:947] Date from: 2017-10-04
[10/Oct/2017 13:04:50] INFO [remapp.netdicom.qrscu:950] Date until: 2017-10-04
[10/Oct/2017 13:04:51] INFO [remapp.netdicom.qrscu:447] Query_id 53f5d8a8-af5b-4dfa-bf04-1d9b369cc151: Request association with {'Port': 5678, 'AET': 'PACS', 'Address': u'127.0.0.1'} (127.0.0.1 5678 from <AE(OPENREM, stopped daemon 6608)>)
[10/Oct/2017 13:04:51] INFO [remapp.netdicom.qrscu:621] DICOM Echo ... 
[10/Oct/2017 13:04:51] INFO [remapp.netdicom.qrscu:634] DICOM FindSCU ... 
[10/Oct/2017 13:04:51] INFO [remapp.netdicom.qrscu:492] Currently querying for CT studies...
[10/Oct/2017 13:04:51] INFO [remapp.netdicom.qrscu:686] Checking to see if any of the 2 studies are already in the OpenREM database
[10/Oct/2017 13:04:51] INFO [remapp.netdicom.qrscu:691] After removing studies we already have in the db, 1 studies are left
[10/Oct/2017 13:04:51] INFO [remapp.netdicom.qrscu:703] Pruning study responses based on inc/exc options: 
[10/Oct/2017 13:04:51] INFO [remapp.netdicom.qrscu:706] Now have 1 studies
[10/Oct/2017 13:04:52] INFO [remapp.netdicom.qrscu:85] Deleting series we can't use
[10/Oct/2017 13:05:23] INFO [remapp.netdicom.qrscu:746] Now have 1 studies
[10/Oct/2017 13:05:39] INFO [remapp.netdicom.qrscu:808] Query_id 53f5d8a8-af5b-4dfa-bf04-1d9b369cc151: Move association requested
[10/Oct/2017 13:05:40] INFO [remapp.netdicom.qrscu:812] Query_id 53f5d8a8-af5b-4dfa-bf04-1d9b369cc151: Preparing to start move request
[10/Oct/2017 13:05:40] INFO [remapp.netdicom.qrscu:817] Query_id 53f5d8a8-af5b-4dfa-bf04-1d9b369cc151: Requesting move of 1 studies
[10/Oct/2017 13:05:40] INFO [remapp.netdicom.qrscu:841] Requesting move: modality CT, study 1 (of 1) series 1 (of 5). Series contains 2 objects
[10/Oct/2017 13:05:40] INFO [remapp.netdicom.qrscu:857] Image-level move - d is: (0008, 0052) Query/Retrieve Level                CS: 'SERIES'
(0020, 000d) Study Instance UID                  UI: 1.2.840.113619.6.95.31.0.3.4.1.1791.13.5062077
(0020, 000e) Series Instance UID                 UI: 1.2.392.200036.9116.2.5.1.48.1221172435.1507088174.447101
[10/Oct/2017 13:05:41] INFO [remapp.netdicom.qrscu:769] Move of study 1, series 1 status is Pending (i.e. one object processed)
[10/Oct/2017 13:05:41] INFO [remapp.netdicom.qrscu:841] Requesting move: modality CT, study 1 (of 1) series 2 (of 5). Series contains 124 objects
[10/Oct/2017 13:05:41] INFO [remapp.netdicom.qrscu:857] Image-level move - d is: (0008, 0018) SOP Instance UID                    UI: 1.2.392.200036.9116.2.5.1.48.1221172435.1507088174.454749
(0008, 0052) Query/Retrieve Level                CS: 'SERIES'
(0020, 000d) Study Instance UID                  UI: 1.2.840.113619.6.95.31.0.3.4.1.1791.13.5062077
(0020, 000e) Series Instance UID                 UI: 1.2.392.200036.9116.2.5.1.48.1221172435.1507088291.440666
[10/Oct/2017 13:05:41] INFO [remapp.netdicom.qrscu:769] Move of study 1, series 2 status is Pending (i.e. one object processed)
[10/Oct/2017 13:05:41] INFO [remapp.netdicom.qrscu:841] Requesting move: modality CT, study 1 (of 1) series 3 (of 5). Series contains 77 objects
[10/Oct/2017 13:05:41] INFO [remapp.netdicom.qrscu:857] Image-level move - d is: (0008, 0018) SOP Instance UID                    UI: 1.2.392.200036.9116.2.5.1.48.1221172435.1507088291.449731
(0008, 0052) Query/Retrieve Level                CS: 'SERIES'
(0020, 000d) Study Instance UID                  UI: 1.2.840.113619.6.95.31.0.3.4.1.1791.13.5062077
(0020, 000e) Series Instance UID                 UI: 1.2.392.200036.9116.2.5.1.48.1221172435.1507088430.396322
[10/Oct/2017 13:05:41] INFO [remapp.netdicom.qrscu:769] Move of study 1, series 3 status is Pending (i.e. one object processed)
[10/Oct/2017 13:05:41] INFO [remapp.netdicom.qrscu:841] Requesting move: modality CT, study 1 (of 1) series 4 (of 5). Series contains 100 objects
[10/Oct/2017 13:05:41] INFO [remapp.netdicom.qrscu:857] Image-level move - d is: (0008, 0018) SOP Instance UID                    UI: 1.2.392.200036.9116.2.5.1.48.1221172435.1507088430.527850
(0008, 0052) Query/Retrieve Level                CS: 'SERIES'
(0020, 000d) Study Instance UID                  UI: 1.2.840.113619.6.95.31.0.3.4.1.1791.13.5062077
(0020, 000e) Series Instance UID                 UI: 1.2.392.200036.9116.2.5.1.48.1221172435.1507088440.779778
[10/Oct/2017 13:05:41] INFO [remapp.netdicom.qrscu:769] Move of study 1, series 4 status is Pending (i.e. one object processed)
[10/Oct/2017 13:05:41] INFO [remapp.netdicom.qrscu:841] Requesting move: modality CT, study 1 (of 1) series 5 (of 5). Series contains 2 objects
[10/Oct/2017 13:05:41] INFO [remapp.netdicom.qrscu:863] Series-level move - d is: (0008, 0018) SOP Instance UID                    UI: 1.2.392.200036.9116.2.5.1.48.1221172435.1507088440.921531
(0008, 0052) Query/Retrieve Level                CS: 'SERIES'
(0020, 000d) Study Instance UID                  UI: 1.2.840.113619.6.95.31.0.3.4.1.1791.13.5062077
(0020, 000e) Series Instance UID                 UI: 1.2.392.200036.9116.2.5.1.48.1221172435.1507088343.920457
[10/Oct/2017 13:05:41] INFO [remapp.netdicom.qrscu:868] Move complete
[10/Oct/2017 13:05:42] INFO [remapp.netdicom.qrscu:872] Query_id 53f5d8a8-af5b-4dfa-bf04-1d9b369cc151: Move association released
  19. David Platten

    This is the corresponding extract from the PACS Conquest:

    [PACS] UPACS THREAD 62: STARTED AT: Tue Oct 10 13:04:51 2017
[PACS]  Calling Application Title : "OPENREM"
[PACS]  Called Application Title : "PACS"
[PACS]  Application Context : "1.2.840.10008.3.1.1.1", PDU length: 16000
[PACS]  Presentation Context 0 "1.2.840.10008.5.1.4.1.2.2.1" 1
[PACS]  Presentation Context 1 "1.2.840.10008.5.1.4.1.2.2.2" 1
[PACS]  Presentation Context 2 "1.2.840.10008.1.1" 1
[PACS] C-Echo
[PACS] (StudyRootQuery) search level: STUDY 
[PACS] C-Find (StudyRoot) located 2 records
[PACS] (StudyRootQuery) search level: SERIES
[PACS] C-Find (StudyRoot) located 5 records
[PACS] (StudyRootQuery) search level: IMAGE 
[PACS] C-Find (StudyRoot) located 2 records
[PACS] (StudyRootQuery) search level: IMAGE 
[PACS] C-Find (StudyRoot) located 124 records
[PACS] (StudyRootQuery) search level: IMAGE 
[PACS] C-Find (StudyRoot) located 77 records
[PACS] (StudyRootQuery) search level: IMAGE 
[PACS] C-Find (StudyRoot) located 100 records
[PACS] (StudyRootQuery) search level: IMAGE 
[PACS] C-Find (StudyRoot) located 2 records
[PACS] 
[PACS] UPACS THREAD 63: STARTED AT: Tue Oct 10 13:05:39 2017
[PACS]  Calling Application Title : "OPENREM"
[PACS]  Called Application Title : "PACS"
[PACS]  Application Context : "1.2.840.10008.3.1.1.1", PDU length: 16000
[PACS]  Presentation Context 0 "1.2.840.10008.5.1.4.1.2.2.1" 1
[PACS]  Presentation Context 1 "1.2.840.10008.5.1.4.1.2.2.2" 1
[PACS]  Presentation Context 2 "1.2.840.10008.1.1" 1
[PACS]  C-Move Destination: "OPENREM "
[PACS] Number of Images to send: 1
[PACS] Sending file : C:\DJP\apps\conquest_testing\conquest_pacs\data\5928212310759232\1.2.392.200036.9116.2.5.1.48.1221172435.1507088174.447101_0001_000001_1507622082019d.dcm
[PACS] C-Move (StudyRoot)
[PACS]  C-Move Destination: "OPENREM "
[PACS] Number of Images to send: 1
[PACS] Sending file : C:\DJP\apps\conquest_testing\conquest_pacs\data\5928212310759232\1.2.392.200036.9116.2.5.1.48.1221172435.1507088291.440666_0002_000001_1507622082019f.dcm
[PACS] C-Move (StudyRoot)
[PACS]  C-Move Destination: "OPENREM "
[PACS] Number of Images to send: 1
[PACS] Sending file : C:\DJP\apps\conquest_testing\conquest_pacs\data\5928212310759232\1.2.392.200036.9116.2.5.1.48.1221172435.1507088430.396322_0004_000001_1507622085021b.dcm
[PACS] C-Move (StudyRoot)
[PACS]  C-Move Destination: "OPENREM "
[PACS] Number of Images to send: 1
[PACS] Sending file : C:\DJP\apps\conquest_testing\conquest_pacs\data\5928212310759232\1.2.392.200036.9116.2.5.1.48.1221172435.1507088440.779778_0005_000001_15076220860268.dcm
[PACS] C-Move (StudyRoot)
[PACS]  C-Move Destination: "OPENREM "
[PACS] C-Move (StudyRoot)
[PACS] UPACS THREAD 63: ENDED AT: Tue Oct 10 13:05:41 2017
[PACS] UPACS THREAD 63: TOTAL RUNNING TIME: 2 SECONDS
[PACS] UPACS THREAD 62: ENDED AT: Tue Oct 10 13:05:52 2017
[PACS] UPACS THREAD 62: TOTAL RUNNING TIME: 61 SECONDS

    and this is the extract from the OpenREM SCP Conquest file:

    [PACS] UPACS THREAD 113: STARTED AT: Tue Oct 10 13:04:50 2017
[PACS]  Calling Application Title : "OPENREMECHO"
[PACS]  Called Application Title : "OPENREM"
[PACS]  Application Context : "1.2.840.10008.3.1.1.1", PDU length: 16000
[PACS]  Presentation Context 0 "1.2.840.10008.1.1" 1
[PACS] C-Echo
[PACS] UPACS THREAD 113: ENDED AT: Tue Oct 10 13:04:51 2017
[PACS] UPACS THREAD 113: TOTAL RUNNING TIME: 1 SECONDS
[PACS] 
[PACS] UPACS THREAD 114: STARTED AT: Tue Oct 10 13:05:41 2017
[PACS]  Calling Application Title : "PACS            "
[PACS]  Called Application Title : "OPENREM         "
[PACS]  Application Context : "1.2.840.10008.3.1.1.1", PDU length: 16384
[PACS]  Presentation Context 0 "1.2.840.10008.5.1.4.1.1.2" 1
[PACS] Written file: C:\DJP\apps\conquest_testing\conquest_openrem_scp\data\5928212310759232\1.2.392.200036.9116.2.5.1.48.1221172435.1507088174.447101_0001_000001_15076371410020.dcm
[PACS] UPACS THREAD 114: ENDED AT: Tue Oct 10 13:05:41 2017
[PACS] UPACS THREAD 114: TOTAL RUNNING TIME: 0 SECONDS
[PACS] 
[PACS] UPACS THREAD 115: STARTED AT: Tue Oct 10 13:05:41 2017
[PACS]  Calling Application Title : "PACS            "
[PACS]  Called Application Title : "OPENREM         "
[PACS]  Application Context : "1.2.840.10008.3.1.1.1", PDU length: 16384
[PACS]  Presentation Context 0 "1.2.840.10008.5.1.4.1.1.2" 1
[PACS] Written file: C:\DJP\apps\conquest_testing\conquest_openrem_scp\data\5928212310759232\1.2.392.200036.9116.2.5.1.48.1221172435.1507088291.440666_0002_000001_15076371410021.dcm
[PACS] UPACS THREAD 115: ENDED AT: Tue Oct 10 13:05:41 2017
[PACS] UPACS THREAD 115: TOTAL RUNNING TIME: 0 SECONDS
[PACS] 
[PACS] UPACS THREAD 116: STARTED AT: Tue Oct 10 13:05:41 2017
[PACS]  Calling Application Title : "PACS            "
[PACS]  Called Application Title : "OPENREM         "
[PACS]  Application Context : "1.2.840.10008.3.1.1.1", PDU length: 16384
[PACS]  Presentation Context 0 "1.2.840.10008.5.1.4.1.1.2" 1
[PACS] Written file: C:\DJP\apps\conquest_testing\conquest_openrem_scp\data\5928212310759232\1.2.392.200036.9116.2.5.1.48.1221172435.1507088430.396322_0004_000001_15076371410022.dcm
[PACS] UPACS THREAD 116: ENDED AT: Tue Oct 10 13:05:41 2017
[PACS] UPACS THREAD 116: TOTAL RUNNING TIME: 0 SECONDS
[PACS] 
[PACS] UPACS THREAD 117: STARTED AT: Tue Oct 10 13:05:41 2017
[PACS]  Calling Application Title : "PACS            "
[PACS]  Called Application Title : "OPENREM         "
[PACS]  Application Context : "1.2.840.10008.3.1.1.1", PDU length: 16384
[PACS]  Presentation Context 0 "1.2.840.10008.5.1.4.1.1.2" 1
[PACS] Written file: C:\DJP\apps\conquest_testing\conquest_openrem_scp\data\5928212310759232\1.2.392.200036.9116.2.5.1.48.1221172435.1507088440.779778_0005_000001_15076371410023.dcm
[PACS] UPACS THREAD 117: ENDED AT: Tue Oct 10 13:05:41 2017
[PACS] UPACS THREAD 117: TOTAL RUNNING TIME: 0 SECONDS
  22. Ed McDonagh reporter

    Try this @dplatten. d.SOPInstanceUID was not being removed when going from image level to series level retrieve, so was looking for object that didn't exist. Refs #546

    → <<cset f4689939952e>>

  23. Ed McDonagh reporter

    As you've noticed @dplatten, queries are really slow when using the Toshiba flag because we're querying every image in every series.

    I assume it might be a bit quicker without the debug logging, but I'm thinking maybe we could interrupt the generator after the first few responses?

    Unless there is a way of querying for specific images. I'll have a think/play now I have an example study to play with.

  24. David Platten

    It does take ages: a couple of hours for one day of studies...

    I was thinking that we could pass _query_images the -toshiba flag. We could then check the type of the first image in the series: if it's a secondary capture then query all images; if it's anything else then query the first image of the series and then exit the function.

  25. David Platten

    @edmcdonagh, the long time required for the query causes a problem: celery becomes unresponsive before the move part, so by the time the initial query has finished the move part just sites in the queue. Restarting celery causes the whole thing to start again, rather than just the move part...

    It would be great if the time required for the initial part of the query could be reduced.

  26. Ed McDonagh reporter

    I've done it, just need to get an internet connection to push it to bitbucket!

    I think it works - my test was too quick to see it happening!

  27. Ed McDonagh reporter

    Added break such that when looking for image SOP Class UID for a series only the first image is queried. Massive speedup for this circumstance! Refs #546

    → <<cset 7df0d29e45e4>>

  28. David Platten

    This is much quicker, but doesn't work on my test system: it only retrieves a single image from one study series, and no images or dose summaries for the others.

    I've commented out the two shared_task lines in my test system, and also removed the .delay so that I don't need celery to be running.

    Did it successfully move the required images when you tested it?

    See logs:

    Conquest PACS:

    [PACS] UPACS THREAD 23: STARTED AT: Wed Oct 11 11:31:47 2017
[PACS]  Calling Application Title : "OPENREM"
[PACS]  Called Application Title : "PACS"
[PACS]  Application Context : "1.2.840.10008.3.1.1.1", PDU length: 16000
[PACS]  Presentation Context 0 "1.2.840.10008.1.1" 1
[PACS] C-Echo
[PACS] UPACS THREAD 23: ENDED AT: Wed Oct 11 11:31:47 2017
[PACS] UPACS THREAD 23: TOTAL RUNNING TIME: 0 SECONDS
[PACS] 
[PACS] UPACS THREAD 24: STARTED AT: Wed Oct 11 11:31:48 2017
[PACS]  Calling Application Title : "OPENREM"
[PACS]  Called Application Title : "PACS"
[PACS]  Application Context : "1.2.840.10008.3.1.1.1", PDU length: 16000
[PACS]  Presentation Context 0 "1.2.840.10008.5.1.4.1.2.2.1" 1
[PACS]  Presentation Context 1 "1.2.840.10008.5.1.4.1.2.2.2" 1
[PACS]  Presentation Context 2 "1.2.840.10008.1.1" 1
[PACS] C-Echo
[PACS] (StudyRootQuery) search level: STUDY 
[PACS] C-Find (StudyRoot) located 2 records
[PACS] (StudyRootQuery) search level: SERIES
[PACS] C-Find (StudyRoot) located 5 records
[PACS] (StudyRootQuery) search level: SERIES
[PACS] C-Find (StudyRoot) located 5 records
[PACS] (StudyRootQuery) search level: IMAGE 
[PACS] C-Find (StudyRoot) located 160 records
[PACS] (StudyRootQuery) search level: IMAGE 
[PACS] C-Find (StudyRoot) located 122 records
[PACS] (StudyRootQuery) search level: IMAGE 
[PACS] C-Find (StudyRoot) located 130 records
[PACS] (StudyRootQuery) search level: IMAGE 
[PACS] C-Find (StudyRoot) located 2 records
[PACS] (StudyRootQuery) search level: IMAGE 
[PACS] C-Find (StudyRoot) located 2 records
[PACS] (StudyRootQuery) search level: IMAGE 
[PACS] C-Find (StudyRoot) located 2 records
[PACS] (StudyRootQuery) search level: IMAGE 
[PACS] C-Find (StudyRoot) located 124 records
[PACS] (StudyRootQuery) search level: IMAGE 
[PACS] C-Find (StudyRoot) located 77 records
[PACS] (StudyRootQuery) search level: IMAGE 
[PACS] C-Find (StudyRoot) located 100 records
[PACS] (StudyRootQuery) search level: IMAGE 
[PACS] C-Find (StudyRoot) located 2 records
[PACS] 
[PACS] UPACS THREAD 25: STARTED AT: Wed Oct 11 11:32:09 2017
[PACS]  Calling Application Title : "OPENREM"
[PACS]  Called Application Title : "PACS"
[PACS]  Application Context : "1.2.840.10008.3.1.1.1", PDU length: 16000
[PACS]  Presentation Context 0 "1.2.840.10008.5.1.4.1.2.2.1" 1
[PACS]  Presentation Context 1 "1.2.840.10008.5.1.4.1.2.2.2" 1
[PACS]  Presentation Context 2 "1.2.840.10008.1.1" 1
[PACS]  C-Move Destination: "OPENREM "
[PACS] Number of Images to send: 1
[PACS] Sending file : C:\DJP\apps\conquest_testing\conquest_pacs\data\1352938289346305\1.2.392.200036.9116.2.5.1.16.1613466634.1507092675.132000_0003_000007_15076220670000.dcm
[PACS] C-Move (StudyRoot)
[PACS]  C-Move Destination: "OPENREM "
[PACS] C-Move (StudyRoot)
[PACS]  C-Move Destination: "OPENREM "
[PACS] C-Move (StudyRoot)
[PACS]  C-Move Destination: "OPENREM "
[PACS] C-Move (StudyRoot)
[PACS]  C-Move Destination: "OPENREM "
[PACS] C-Move (StudyRoot)
[PACS]  C-Move Destination: "OPENREM "
[PACS] C-Move (StudyRoot)
[PACS]  C-Move Destination: "OPENREM "
[PACS] C-Move (StudyRoot)
[PACS]  C-Move Destination: "OPENREM "
[PACS] C-Move (StudyRoot)
[PACS]  C-Move Destination: "OPENREM "
[PACS] C-Move (StudyRoot)
[PACS]  C-Move Destination: "OPENREM "
[PACS] C-Move (StudyRoot)
[PACS] UPACS THREAD 25: ENDED AT: Wed Oct 11 11:32:11 2017
[PACS] UPACS THREAD 25: TOTAL RUNNING TIME: 2 SECONDS
[PACS] UPACS THREAD 24: ENDED AT: Wed Oct 11 11:32:23 2017
[PACS] UPACS THREAD 24: TOTAL RUNNING TIME: 35 SECONDS

    QR log:

    [11/Oct/2017 11:31:47] INFO [remapp.netdicom.qrscu:924] qrscu script called
[11/Oct/2017 11:31:47] INFO [remapp.netdicom.qrscu:944] Modalities are ['CT']
[11/Oct/2017 11:31:47] INFO [remapp.netdicom.qrscu:950] Date from: 2017-10-04
[11/Oct/2017 11:31:47] INFO [remapp.netdicom.qrscu:953] Date until: 2017-10-04
[11/Oct/2017 11:31:48] INFO [remapp.netdicom.qrscu:450] Query_id e915cfd0-df68-4061-b9f2-cd43079df103: Request association with {'Port': 5678, 'AET': 'PACS', 'Address': u'127.0.0.1'} (127.0.0.1 5678 from <AE(OPENREM, stopped daemon 6540)>)
[11/Oct/2017 11:31:48] INFO [remapp.netdicom.qrscu:624] DICOM Echo ... 
[11/Oct/2017 11:31:48] INFO [remapp.netdicom.qrscu:637] DICOM FindSCU ... 
[11/Oct/2017 11:31:48] INFO [remapp.netdicom.qrscu:495] Currently querying for CT studies...
[11/Oct/2017 11:31:48] INFO [remapp.netdicom.qrscu:689] Checking to see if any of the 2 studies are already in the OpenREM database
[11/Oct/2017 11:31:48] INFO [remapp.netdicom.qrscu:694] After removing studies we already have in the db, 2 studies are left
[11/Oct/2017 11:31:48] INFO [remapp.netdicom.qrscu:706] Pruning study responses based on inc/exc options: 
[11/Oct/2017 11:31:48] INFO [remapp.netdicom.qrscu:709] Now have 2 studies
[11/Oct/2017 11:31:50] INFO [remapp.netdicom.qrscu:85] Deleting series we can't use
[11/Oct/2017 11:31:51] INFO [remapp.netdicom.qrscu:749] Now have 2 studies
[11/Oct/2017 11:32:09] INFO [remapp.netdicom.qrscu:811] Query_id e915cfd0-df68-4061-b9f2-cd43079df103: Move association requested
[11/Oct/2017 11:32:09] INFO [remapp.netdicom.qrscu:815] Query_id e915cfd0-df68-4061-b9f2-cd43079df103: Preparing to start move request
[11/Oct/2017 11:32:09] INFO [remapp.netdicom.qrscu:820] Query_id e915cfd0-df68-4061-b9f2-cd43079df103: Requesting move of 2 studies
[11/Oct/2017 11:32:09] INFO [remapp.netdicom.qrscu:844] Requesting move: modality CT, study 1 (of 2) series 1 (of 5). Series contains 160 objects
[11/Oct/2017 11:32:10] INFO [remapp.netdicom.qrscu:772] Move of study 1, series 1 status is Pending (i.e. one object processed)
[11/Oct/2017 11:32:10] INFO [remapp.netdicom.qrscu:844] Requesting move: modality CT, study 1 (of 2) series 2 (of 5). Series contains 122 objects
[11/Oct/2017 11:32:10] INFO [remapp.netdicom.qrscu:844] Requesting move: modality CT, study 1 (of 2) series 3 (of 5). Series contains 130 objects
[11/Oct/2017 11:32:10] INFO [remapp.netdicom.qrscu:844] Requesting move: modality CT, study 1 (of 2) series 4 (of 5). Series contains 2 objects
[11/Oct/2017 11:32:10] INFO [remapp.netdicom.qrscu:844] Requesting move: modality CT, study 1 (of 2) series 5 (of 5). Series contains 2 objects
[11/Oct/2017 11:32:10] INFO [remapp.netdicom.qrscu:844] Requesting move: modality CT, study 2 (of 2) series 1 (of 5). Series contains 2 objects
[11/Oct/2017 11:32:10] INFO [remapp.netdicom.qrscu:844] Requesting move: modality CT, study 2 (of 2) series 2 (of 5). Series contains 124 objects
[11/Oct/2017 11:32:10] INFO [remapp.netdicom.qrscu:844] Requesting move: modality CT, study 2 (of 2) series 3 (of 5). Series contains 77 objects
[11/Oct/2017 11:32:11] INFO [remapp.netdicom.qrscu:844] Requesting move: modality CT, study 2 (of 2) series 4 (of 5). Series contains 100 objects
[11/Oct/2017 11:32:11] INFO [remapp.netdicom.qrscu:844] Requesting move: modality CT, study 2 (of 2) series 5 (of 5). Series contains 2 objects
[11/Oct/2017 11:32:11] INFO [remapp.netdicom.qrscu:871] Move complete
[11/Oct/2017 11:32:11] INFO [remapp.netdicom.qrscu:875] Query_id e915cfd0-df68-4061-b9f2-cd43079df103: Move association released
  29. Ed McDonagh reporter

    I have the same situation - after the first move request, none of the subsequent ones get into the generator for loop.

    I'm looking, but I'm in meetings all afternoon now.

  30. David Platten

    Ta. There's no rush: I'm using the old code on my live system. If we can get this working it'll be great.

  31. Ed McDonagh reporter

    I can only think that there is something new wrong with d that is preventing the move SCU being able to provide any response.

  32. David Platten

    I know what the problem is. Below is an extract from the Conquest PACS in debug mode.

    It successfully finds and sends the first image of the first series (series A) in the study.

    What it should then do is send the first image of the second series (series B) of the study. But it doesn't. It tries to find the UID of the second image in series A within series B. Unsurprisingly it finds nothing.

    When trying to send the first image of the third series, it actually tries to find UID of the third image in series A within series C.

    Basically, it always uses SOP instance UIDs from the first series, but the correct UIDs for the series and study. This works for the first image, but nothing else.

    The code in _query_images uses a generator function to go through each image in a function. We're now breaking out of this early. It seems that it remembers where it was when it is called the next time, rather than starting at the first image of the current series. My guess is that this is the problem: before we break out of the loop we need to iterate over the remaining images in the series.

    Changing 

        st3 = assoc.StudyRootFindSOPClass.SCU(d3, 1)

    to

        st3 = assoc.StudyRootFindSOPClass.SCU(d3, 1)
    st3 = list(st3)

    fixes the problem. I think.

        st3 = assoc.StudyRootFindSOPClass.SCU(d3, 1)
    st3 = list(st3)

    query_id = uuid.uuid4()

    imRspNo = 0

    for images in st3:
        if not images[1]:
            continue
        images[1].decode()
        imRspNo += 1
        logger.debug(u"Query_id {0}: Image Response {1}: {2}".format(query_id, imRspNo, images[1]))
        imagesrsp = DicomQRRspImage.objects.create(dicom_qr_rsp_series=seriesrsp)
        imagesrsp.query_id = query_id
        # Mandatory tags
        imagesrsp.sop_instance_uid = images[1].SOPInstanceUID
        imagesrsp.sop_class_uid = images[1].SOPClassUID
        imagesrsp.instance_number = images[1].InstanceNumber
        if not imagesrsp.instance_number:  # just in case!!
            imagesrsp.instance_number = None  # integer so can't be ''
        imagesrsp.save()
        if initial_image_only:
            break

    11/10/2017 14:59:41 [PACS] UPACS THREAD 61: STARTED AT: Wed Oct 11 14:59:40 2017
11/10/2017 14:59:41 [PACS] A-ASSOCIATE-RQ Packet Dump
11/10/2017 14:59:41 [PACS]  Calling Application Title : "OPENREM"
11/10/2017 14:59:41 [PACS]  Called Application Title : "PACS"
11/10/2017 14:59:41 [PACS]  Application Context : "1.2.840.10008.3.1.1.1", PDU length: 16000
11/10/2017 14:59:41 [PACS]  Number of Proposed Presentation Contexts: 3
11/10/2017 14:59:41 [PACS]  Presentation Context 0 "1.2.840.10008.5.1.4.1.2.2.1" 1
11/10/2017 14:59:41 [PACS]  Presentation Context 1 "1.2.840.10008.5.1.4.1.2.2.2" 1
11/10/2017 14:59:41 [PACS]  Presentation Context 2 "1.2.840.10008.1.1" 1
11/10/2017 14:59:41 [PACS] Server Command   := 0021
11/10/2017 14:59:41 [PACS] Message ID       := 0001
11/10/2017 14:59:41 [PACS]  C-Move Destination: "OPENREM "
11/10/2017 14:59:41 [PACS] (QualifyOn) (mapped) IP:127.0.0.1, PORT:5688
11/10/2017 14:59:41 [PACS] MyStudyRootRetrieveGeneric :: SearchOn
11/10/2017 14:59:41 [PACS] Query On Image
11/10/2017 14:59:41 [PACS] Issue Query on Columns: DICOMImages.SOPClassUI, DICOMImages.SOPInstanc, DICOMSeries.SeriesInst, DICOMStudies.StudyInsta,DICOMImages.ObjectFile,DICOMImages.DeviceName
11/10/2017 14:59:41 [PACS] Values: DICOMImages.SOPInstanc = '1.2.392.200036.9116.2.5.1.16.1613466634.1507092677.165873' and DICOMSeries.SeriesInst = '1.2.392.200036.9116.2.5.1.16.1613466634.1507092675.132000' and DICOMStudies.StudyInsta = '1.2.392.200036.9116.2.5.1.16.1613466634.1507092477.524642' and DICOMSeries.StudyInsta = DICOMStudies.StudyInsta and DICOMImages.SeriesInst = DICOMSeries.SeriesInst
11/10/2017 14:59:41 [PACS] Tables: DICOMImages, DICOMSeries, DICOMStudies
11/10/2017 14:59:41 [PACS] Records = 1
11/10/2017 14:59:41 [PACS] Number of Images to send: 1
11/10/2017 14:59:41 [PACS] MyStudyRootRetrieveGeneric :: RetrieveOn
11/10/2017 14:59:41 [PACS] Locating file:MAG0 1352938289346305\1.2.392.200036.9116.2.5.1.16.1613466634.1507092675.132000_0003_000007_15076220670000.dcm
11/10/2017 14:59:41 [PACS] ReadAheadThread: readahead > 0000
11/10/2017 14:59:41 [PACS] RetrieveOn: givenout < 0000
11/10/2017 14:59:41 [PACS] Sending file : C:\DJP\apps\conquest_testing\conquest_pacs\data\1352938289346305\1.2.392.200036.9116.2.5.1.16.1613466634.1507092675.132000_0003_000007_15076220670000.dcm
11/10/2017 14:59:41 [PACS] Image Loaded from Read Ahead Thread, returning TRUE
11/10/2017 14:59:41 [PACS] C-Move (StudyRoot)
11/10/2017 14:59:41 [PACS] Server Command   := 0021
11/10/2017 14:59:41 [PACS] Message ID       := 0001
11/10/2017 14:59:41 [PACS]  C-Move Destination: "OPENREM "
11/10/2017 14:59:41 [PACS] (QualifyOn) (mapped) IP:127.0.0.1, PORT:5688
11/10/2017 14:59:41 [PACS] MyStudyRootRetrieveGeneric :: SearchOn
11/10/2017 14:59:41 [PACS] Query On Image
11/10/2017 14:59:41 [PACS] Issue Query on Columns: DICOMImages.SOPClassUI, DICOMImages.SOPInstanc, DICOMSeries.SeriesInst, DICOMStudies.StudyInsta,DICOMImages.ObjectFile,DICOMImages.DeviceName
11/10/2017 14:59:41 [PACS] Values: DICOMImages.SOPInstanc = '1.2.392.200036.9116.2.5.1.16.1613466634.1507092677.510427' and DICOMSeries.SeriesInst = '1.2.392.200036.9116.2.5.1.16.1613466634.1507092859.165572' and DICOMStudies.StudyInsta = '1.2.392.200036.9116.2.5.1.16.1613466634.1507092477.524642' and DICOMSeries.StudyInsta = DICOMStudies.StudyInsta and DICOMImages.SeriesInst = DICOMSeries.SeriesInst
11/10/2017 14:59:41 [PACS] Tables: DICOMImages, DICOMSeries, DICOMStudies
11/10/2017 14:59:41 [PACS] Records = 0
11/10/2017 14:59:41 [PACS] Retrieve: move search failed
  33. David Platten

    The downside to putting a list around st3 is that you lose most of the speed benefit...

    Maybe there's another way?

  35. Ed McDonagh reporter

    Once you've created a generator, you can go back to it again and again and it will remember where you were and offer the next item, which is what is happening here. But I don't quite understand why calling it again doesn't start again. I'm guessing that because we're using the same association something is persisting between calls. Ggggrrrr.

  36. David Platten

    Is there an alternative to

    st3 = assoc.StudyRootFindSOPClass.SCU(d3, 1)

    when we know we only want one image from a particular series?

  37. Ed McDonagh reporter

    I don't think so, but I will look at the pynetdicom coffee again.

    I've implemented an index that gets passed to the function to modify the message ID (currently always 1) to see if that does the trick. Initial run-through just as my train pulled into my station suggested it didn't, but I haven't checked I got the code right yet. I might get an opportunity later tonight, I might not.

  38. Ed McDonagh reporter

    Removed break from generator for loop, added specifying instance number instead. Left in msg_id index increment though it is probably of no significance. Refs #546

    → <<cset d4b366d5300c>>

  39. Ed McDonagh reporter

    Try this @dplatten. I think it works now. And I've just seen that my attempt at 'code' above was corrected to 'coffee'. Probably needed more of the stuff!

  40. David Platten

    It works - many, many thanks @edmcdonagh. I've just queried for a day of CT examinations here (140 studies). This would normally take around 6 - 7 hours to complete. With the new code the whole query and retrieve was done in 40 minutes. This is going to make things much easier for me.

  41. David Platten

    Added continue when no images are returned to prevent execution of the rest of the code in the for loop. Without this, the next line causes an IndexError because images[0] does not exist. References issue #546

    → <<cset b4e640c9a33a>>

  44. Log in to comment
Assignee
Type
enhancement
Priority
major
Status
resolved
Component
DICOM Networking
Milestone
0.8.0
Votes
0
Watchers
1
Jira: the preferred issue tracker for Bitbucket. Join the team!