Celery extractor tasks imported with two namespaces

Issue #567 resolved
Ed McDonagh
created an issue

They are listed with both openrem.remapp.extractors.x and remapp.extractors.x

See https://groups.google.com/d/topic/openrem/U6ngf_Cp_-Q/discussion

Comments (28)

  1. Ed McDonagh reporter

    Source of issue identified by @Ben Earner in the google group thread (thanks Ben!):

    I think I might have found the issue. In https://bitbucket.org/openrem/openrem/src/5bfe8f63aa74ec8686cc66de204b586a9277463f/openrem/remapp/extractors/init.py?at=develop&fileviewer=file-view-default

    The extractor stuff is getting imported, which doubles them up in the task list like you said. If I comment these lines out then the netdicom imports suddenly appear on the celery tasks list. Not sure I understand why that has that effect but seems to be the cause...

    Ben

  2. Ben Earner

    So commenting out the imports removes the issue but then causes the extractors to stop working. What I had to do was to stop the celery service, comment out the code in __init__.py, restart the celery service (at this point is must discover which tasks are available), then uncomment __init__.py to enable the extractors to function.

    (As a side note I left

    from rdsr_toshiba_ct_from_dose_images import rdsr_toshiba_ct_from_dose_images
    

    commented out as this was causing errors as it requires JAVA_EXE etc to be in local_settings.py which I didn't have installed. Maybe it should handle the case where rdsr from toshiba is not required/avaliable? Or will the new install docs go about getting these applications installed?)

  3. David Platten

    Hi Ben,

    You should be able to set JAVA_EXE to a dummy string in local_settings.py, as well as some other variables. Defaults are included in local_settings.py in the develop branch:

    # Locations of various tools for DICOM RDSR creation from CT images
    DCMTK_PATH = 'C:\\Users\\David\\Apps\\dcmtk-3.6.0-win32-i386\\bin'
    DCMCONV = os.path.join(DCMTK_PATH, 'dcmconv.exe')
    DCMMKDIR = os.path.join(DCMTK_PATH, 'dcmmkdir.exe')
    JAVA_EXE = 'C:\\Users\\David\\Apps\\doseUtility\\windows\\jre\\bin\\java.exe'
    JAVA_OPTIONS = '-Xms256m -Xmx512m -Xss1m -cp'
    PIXELMED_JAR = 'C:\\Users\\David\\Apps\\doseUtility\\pixelmed.jar'
    PIXELMED_JAR_OPTIONS = '-Djava.awt.headless=true com.pixelmed.doseocr.OCR -'
    
  4. Ed McDonagh reporter

    @Ben Earner, @Tim de Wit - Can you try using the issue567celeryimportnamespace branch to see if everything starts to work?

    It differs from the develop branch in the following ways:

    • All the shared tasks that are imported in extractors/__init__.py are now explicitly named without the openrem. prefix
    • The tasks.py imports are slightly re-ordered - I don't expect this to make any difference
    • The ptsizecsv2db.py function has had a little reformatting - I don't expect this to make any difference
  5. Ed McDonagh reporter

    Using celery multi start on my development laptop with this branch, then celery inspect registered I get the following:

        * openremproject.celery.debug_task
        * remapp.exports.ct_export.ct_csv
        * remapp.exports.ct_export.ctxlsx
        * remapp.exports.dx_export.dxxlsx
        * remapp.exports.dx_export.exportDX2excel
        * remapp.exports.mg_csv_nhsbsp.mg_csv_nhsbsp
        * remapp.exports.mg_export.exportMG2excel
        * remapp.exports.rf_export.exportFL2excel
        * remapp.exports.rf_export.rfopenskin
        * remapp.exports.rf_export.rfxlsx
        * remapp.extractors.ct_philips.ct_philips
        * remapp.extractors.dx.dx
        * remapp.extractors.mam.mam
        * remapp.extractors.ptsizecsv2db.websizeimport
        * remapp.extractors.rdsr.rdsr
        * remapp.extractors.rdsr_toshiba_ct_from_dose_images.rdsr_toshiba_ct_from_dose_images
        * remapp.netdicom.keepalive.keep_alive
        * remapp.netdicom.qrscu.movescu
        * remapp.netdicom.qrscu.qrscu
        * remapp.tools.make_skin_map
    
  6. Tim de Wit

    Just did and after some minor adjustments it's indeed working again!

    For some reason on my (ubuntu) system the scripts starting with the line
    # This Python file uses the following encoding: utf-8
    could not be executed. I had to put the interpreter-line on top.
    Currently the interpreter is:
    #!/usr/local/bin/python
    which didn't exist on my system. I had to change it to /usr/bin/python.

  7. Ed McDonagh reporter

    Log file was being created because the configuration hadn't been added to my local_settings.py. Added line in release docs. Refs #579.
    Added dummy locations for the Toshiba import tools so it doesn't fail if it isn't in local_settings.py and isn't being used.
    Removed imports in extractors/int.py as they are no longer used due to rewriting the scripts (refs #567)

    → <<cset f60a7da98829>>

  8. Log in to comment