Stores queue consumes too much CPU and memory

[Ed McDonagh]

Need to work out why, and if using a celery task/queue is really the right thing to do...

[David Platten]

Hmmm. Similar to this? http://stackoverflow.com/questions/19925264/django-1-6-and-celery-3-0-memory-leaks

[Ed McDonagh]

I don't know. I don't think it is that. It might just be that running a long task like that is inappropriate for Celery. Maybe I should just run it without Celery, with a loop to check if we've asked it not to run anymore, and Celery Beat to start and restart it if it stops.

[Ed McDonagh]

StoreSCP from web interface no longer starts a celery task. Starts and stops successfully from the web interface. TODO: command line start and rethink logging. Refs #328

→ <<cset 42f4742b33c0>>

[Ed McDonagh]

Added basic logging configuration, with override of key values in the example local_settings.py. Only applies to web_store function so far. Refs #328

→ <<cset ecb74cf3b743>>

[Ed McDonagh]

Changed all of storescp to use the new log. Seems to work. Refs #328

→ <<cset af1368c941af>>

[Ed McDonagh]

Changed all of qrscu to use the new log. Seems to work, and continues to log to the celery log too, but for some reason that is at DEBUG level. Refs #328

→ <<cset 0c0e30119246>>

[Ed McDonagh]

Now logs qrscu to separate file. Still logs to celery logs too, but currently only WARNING message that the worker is ready. Refs #328

→ <<cset 2bddaff1aa78>>

[Ed McDonagh]

StoreSCP now has its own log. Refs #328

→ <<cset 05f786d188a2>>

[Ed McDonagh]

Added basic instructions for the new log settings in the install docs. Refs #328

→ <<cset 0bb39f6bc243>>

[Ed McDonagh]

Added upgrade instructions for log config. Refs #328

→ <<cset 2b0edfc31e46>>

[Ed McDonagh]

Merging issue ref #328 back into develop with a view to building a beta for more testing.

→ <<cset ec85f3f0b304>>

[Ed McDonagh]

Removed store task. Refs #328

→ <<cset 18d887f24e69>>

[Ed McDonagh]

Hopefully simplified docs relating to starting Celery. Refs #328 as it removes the store queue.

→ <<cset bd8aa8ed76c7>>

[Ed McDonagh]

Store SCP no longer uses Celery, so this task should be closed. Ref #337 takes on the new problems that have arisen as a consequence!

[Ed McDonagh]

Adding refs #338, #328 to changes

→ <<cset b32a5a2a4f0a>>