Celery 4 drops support for Windows

[Ed McDonagh]

See release notes

Prompted by this message to the Google group

[Ed McDonagh]

Initial action is to lock the pip install to version 3.1

[Ed McDonagh]

Looks like 3.1.25 is the one to target: http://stackoverflow.com/a/40795761

[Ed McDonagh]

We know that celery==4.0.0 with amqp==2.1.1 works.

We know that celery==4.0.2 doesn't work with amqp==2.1.4 on Windows. We know that celery==4.1.0 doesn't work, but that is probably with amqp<2.1.1, maybe current amqp==2.2.2.

I don't think we know anything else?

We need to document this properly for release 0.8.0, so we need to do some experiments. Any volunteers?

We already have ticket to update the documentation for Celery for Windows in issue #541

[Ed McDonagh]

Moving to future to deal with possible lack of support for Windows in Celery 4 - there are indications support might return in some form.

Changing docs for Windows in the interim is covered by issue #541

[Ed McDonagh]

Referenced in ca72f39

[Ed McDonagh]

@dplatten - are you able to shed any light on the celery and amqp version compatibility?

We also need to change the ssettings.py from amqp which seems to be long since deprecated to CELERY_RESULT_BACKEND = 'rpc://' - I don't know if there are any consequences - presumably we just change the setting and restart Celery?

Also, the CELERYD_PREFETCH_MULTIPLIER = 1 setting and I notice you are using concurrency of 1 - is this a problem? Does it not mean that everything is always waiting in a queue, particularly if you have one long running task?

[David Platten]

On my Windows Server 2012 install I am running celery 4.0.0, seemingly without problem.

I am using this with the following celery settings in settings.py (concurrency is 4 in the settings and in the celery command). Note that CELERY_RESULT_BACKEND is set to rpc, rather than rpc:// - I don't know if this matters, but it appears to work as it is.

# Celery settings
BROKER_URL = 'amqp://guest:guest@localhost//'
CELERY_RESULT_BACKEND = 'rpc'

CELERY_ACCEPT_CONTENT = ['json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'

CELERY_DEFAULT_QUEUE = 'default'

CELERYD_CONCURRENCY = 4
CELERY_ACKS_LATE = True
CELERYD_PREFETCH_MULTIPLIER = 1

My command to run celery is:

celery worker -n default -P solo -Ofair -A openremproject -c 4 -Q default --pidfile=e:\media_root\celery\default.pid --logfile=e:\media_root\celery\default.log

[David Platten]

In terms of updating the settings from ampq to rpc, I just restarted celery - there was nothing else that needed to be done.

[Ed McDonagh]

Thanks. I'll update the settings file and docs accordingly.

[Ed McDonagh]

Changed celery results backend to rpc. Refs #582. Added -Ofair to linux celery command. Added note about versions of Celery on Windows. Refs #455, #541. [skip ci]

→ <<cset facda714c3b1>>

[David Platten]

I think you may need a "-P solo" in the Windows celery command in the docs. This is the line that I use on my live system:

celery worker -n default -P solo -Ofair -A openremproject -c 4 -Q default --pidfile=e:\media_root\celery\default.pid --logfile=e:\media_root\celery\default.log

[David Platten]

Perhaps forget that comment - I think it's already there in the docs.

[Ed McDonagh]

Resolved by removing dependence on Celery for v1.0 (Thanks Jannis!)

[Ed McDonagh]

Resolved in issue ref #788