Celery 4 drops support for Windows

Issue #455 new
Ed McDonagh created an issue

Comments (12)

  1. Ed McDonagh reporter

    We know that celery==4.0.0 with amqp==2.1.1 works.

    We know that celery==4.0.2 doesn't work with amqp==2.1.4 on Windows. We know that celery==4.1.0 doesn't work, but that is probably with amqp<2.1.1, maybe current amqp==2.2.2.

    I don't think we know anything else?

    We need to document this properly for release 0.8.0, so we need to do some experiments. Any volunteers?

    We already have ticket to update the documentation for Celery for Windows in issue #541

  2. Ed McDonagh reporter

    Moving to future to deal with possible lack of support for Windows in Celery 4 - there are indications support might return in some form.

    Changing docs for Windows in the interim is covered by issue #541

  3. Ed McDonagh reporter

    @dplatten - are you able to shed any light on the celery and amqp version compatibility?

    We also need to change the ssettings.py from amqp which seems to be long since deprecated to CELERY_RESULT_BACKEND = 'rpc://' - I don't know if there are any consequences - presumably we just change the setting and restart Celery?

    Also, the CELERYD_PREFETCH_MULTIPLIER = 1 setting and I notice you are using concurrency of 1 - is this a problem? Does it not mean that everything is always waiting in a queue, particularly if you have one long running task?

  4. David Platten

    On my Windows Server 2012 install I am running celery 4.0.0, seemingly without problem.

    I am using this with the following celery settings in settings.py (concurrency is 4 in the settings and in the celery command). Note that CELERY_RESULT_BACKEND is set to rpc, rather than rpc:// - I don't know if this matters, but it appears to work as it is.

    # Celery settings
    BROKER_URL = 'amqp://guest:guest@localhost//'
    CELERY_RESULT_BACKEND = 'rpc'
    
    CELERY_ACCEPT_CONTENT = ['json']
    CELERY_TASK_SERIALIZER = 'json'
    CELERY_RESULT_SERIALIZER = 'json'
    
    CELERY_DEFAULT_QUEUE = 'default'
    
    CELERYD_CONCURRENCY = 4
    CELERY_ACKS_LATE = True
    CELERYD_PREFETCH_MULTIPLIER = 1
    

    My command to run celery is:

    celery worker -n default -P solo -Ofair -A openremproject -c 4 -Q default --pidfile=e:\media_root\celery\default.pid --logfile=e:\media_root\celery\default.log
    
  5. David Platten

    In terms of updating the settings from ampq to rpc, I just restarted celery - there was nothing else that needed to be done.

  6. David Platten

    I think you may need a "-P solo" in the Windows celery command in the docs. This is the line that I use on my live system:

    celery worker -n default -P solo -Ofair -A openremproject -c 4 -Q default --pidfile=e:\media_root\celery\default.pid --logfile=e:\media_root\celery\default.log
    
  7. Log in to comment