1. OpenREM Avatar OpenREM
  2. OpenREM
  3. OpenREM
  4. Issues

Celery extractor tasks imported with two namespaces

Issue #567 resolved
Ed McDonagh created an issue

They are listed with both openrem.remapp.extractors.x and remapp.extractors.x

See https://groups.google.com/d/topic/openrem/U6ngf_Cp_-Q/discussion

Comments (28)

  1. Ed McDonagh reporter

    Source of issue identified by @bearner in the google group thread (thanks Ben!):

    I think I might have found the issue. In https://bitbucket.org/openrem/openrem/src/5bfe8f63aa74ec8686cc66de204b586a9277463f/openrem/remapp/extractors/init.py?at=develop&fileviewer=file-view-default

    The extractor stuff is getting imported, which doubles them up in the task list like you said. If I comment these lines out then the netdicom imports suddenly appear on the celery tasks list. Not sure I understand why that has that effect but seems to be the cause...

    Ben

  2. Ben Earner

    So commenting out the imports removes the issue but then causes the extractors to stop working. What I had to do was to stop the celery service, comment out the code in __init__.py, restart the celery service (at this point is must discover which tasks are available), then uncomment __init__.py to enable the extractors to function.

    (As a side note I left 

    from rdsr_toshiba_ct_from_dose_images import rdsr_toshiba_ct_from_dose_images

    commented out as this was causing errors as it requires JAVA_EXE etc to be in local_settings.py which I didn't have installed. Maybe it should handle the case where rdsr from toshiba is not required/avaliable? Or will the new install docs go about getting these applications installed?)

  3. David Platten

    Hi Ben,

    You should be able to set JAVA_EXE to a dummy string in local_settings.py, as well as some other variables. Defaults are included in local_settings.py in the develop branch:

    # Locations of various tools for DICOM RDSR creation from CT images
DCMTK_PATH = 'C:\\Users\\David\\Apps\\dcmtk-3.6.0-win32-i386\\bin'
DCMCONV = os.path.join(DCMTK_PATH, 'dcmconv.exe')
DCMMKDIR = os.path.join(DCMTK_PATH, 'dcmmkdir.exe')
JAVA_EXE = 'C:\\Users\\David\\Apps\\doseUtility\\windows\\jre\\bin\\java.exe'
JAVA_OPTIONS = '-Xms256m -Xmx512m -Xss1m -cp'
PIXELMED_JAR = 'C:\\Users\\David\\Apps\\doseUtility\\pixelmed.jar'
PIXELMED_JAR_OPTIONS = '-Djava.awt.headless=true com.pixelmed.doseocr.OCR -'
  4. Ben Earner

    Ah ok sorry, I didn't add them in as I wasn't using them, like you say it works fine as long as the dummy settings are there. Thanks!

  6. Ben Earner

    Are you running a daemonised service out of interest? (Not that it should make much difference I guess).

  10. Ed McDonagh reporter

    @bearner, @tcdewit - Can you try using the issue567celeryimportnamespace branch to see if everything starts to work?

    It differs from the develop branch in the following ways:

    • All the shared tasks that are imported in extractors/__init__.py are now explicitly named without the openrem. prefix
    • The tasks.py imports are slightly re-ordered - I don't expect this to make any difference
    • The ptsizecsv2db.py function has had a little reformatting - I don't expect this to make any difference
  11. Ed McDonagh reporter

    @tcdewit - now that you have fixed the log file permission issue, does this branch give you a sensible list of (working) tasks?

    (See comments against 0854ad7a2608)

  13. Ed McDonagh reporter

    @bearner if you are able to try this, I'll merge it in if you agree it works. I'll create a PR.

  14. Ben Earner

    I'm still getting the doubling (see my comments on the commit).

    @tcdewit when you run:

    celery inspect registered

    Do you see doubling of tasks? Celery does work for me though.

  15. Ed McDonagh reporter

    Using celery multi start on my development laptop with this branch, then celery inspect registered I get the following:

        * openremproject.celery.debug_task
    * remapp.exports.ct_export.ct_csv
    * remapp.exports.ct_export.ctxlsx
    * remapp.exports.dx_export.dxxlsx
    * remapp.exports.dx_export.exportDX2excel
    * remapp.exports.mg_csv_nhsbsp.mg_csv_nhsbsp
    * remapp.exports.mg_export.exportMG2excel
    * remapp.exports.rf_export.exportFL2excel
    * remapp.exports.rf_export.rfopenskin
    * remapp.exports.rf_export.rfxlsx
    * remapp.extractors.ct_philips.ct_philips
    * remapp.extractors.dx.dx
    * remapp.extractors.mam.mam
    * remapp.extractors.ptsizecsv2db.websizeimport
    * remapp.extractors.rdsr.rdsr
    * remapp.extractors.rdsr_toshiba_ct_from_dose_images.rdsr_toshiba_ct_from_dose_images
    * remapp.netdicom.keepalive.keep_alive
    * remapp.netdicom.qrscu.movescu
    * remapp.netdicom.qrscu.qrscu
    * remapp.tools.make_skin_map
  21. Tim de Wit

    Just did and after some minor adjustments it's indeed working again!

    For some reason on my (ubuntu) system the scripts starting with the line
    # This Python file uses the following encoding: utf-8
    could not be executed. I had to put the interpreter-line on top.
    Currently the interpreter is:
    #!/usr/local/bin/python
    which didn't exist on my system. I had to change it to /usr/bin/python.

  23. Ed McDonagh reporter

    And the interpreter line is rewritten when the package is installed by pip to match the current python

  27. Ed McDonagh reporter

    Log file was being created because the configuration hadn't been added to my local_settings.py. Added line in release docs. Refs #579. Added dummy locations for the Toshiba import tools so it doesn't fail if it isn't in local_settings.py and isn't being used. Removed imports in extractors/int.py as they are no longer used due to rewriting the scripts (refs #567)

    → <<cset f60a7da98829>>

  29. Log in to comment
Assignee
–
Type
bug
Priority
major
Status
resolved
Component
Tasks
Milestone
0.8.0
Votes
0
Watchers
1
Jira: the preferred issue tracker for Bitbucket. Join the team!