Create instructions on how to remove all queues from RabbitMQ

Issue #637 resolved
David Platten created an issue

I sometimes need to stop celery and remove all queues before restarting celery again. This can be useful when a query isn't working properly, for example.

At the moment I do this by going to the web interface of RabbitMQ and manually deleting each queue. This is time-consuming.

It can be scripted instead.

You need to download the python rabbitmqadmin script for your install (see https://www.rabbitmq.com/management-cli.html).

This can then be used to obtain a list of the queues (https://www.cloudamqp.com/blog/2016-06-21-how-to-delete-queues-in-rabbitmq.html), the output of which can be used to delete them.

I've not implemented this on Windows yet, but plan to.

Comments (37)

  1. David Platten reporter

    A Windows batch file as a work-in-progress to stop celery, delete all RabbitMQ jobs and then restart celery:

    ::ECHO OFF
    
    ::-----------------------------------------------------------------------------
    :: Set up some variables
    :: Reminder: there must not be any whitespace on either side of the equals when
    :: assigning variables
    SET openrem_folder=D:\Server_Apps\python27\Lib\site-packages\openrem
    SET celery_pid=E:\media_root\celery\default.pid
    SET python_exe=python.exe
    SET rabbitmqadmin=D:\Server_Apps\rabbit_mq\rabbitmq_server-3.6.9\rabbitmqadmin.py
    SET q_names=E:\rabbit_mq\q_names.txt
    ::-----------------------------------------------------------------------------
    
    
    ::-----------------------------------------------------------------------------
    :: First stop celery
    :: Change to the drive on which OpenREM is installed and navigate to the
    :: OpenREM folder
    D:
    CD %openrem_folder%
    
    :: Attempt to shutdown celery gracefully
    celery -A openremproject control shutdown --timeout=10
    
    :: Pause this file for 10 s to ensure that the above has time to work (you may
    :: need to check that the 'timeout' command is available on your Windows
    :: system. Some systems may have 'sleep' instead, in which case replace the
    :: line below with:
    :: SLEEP 10
    TIMEOUT /T 10
    
    :: Kill any remaining celery tasks (ungraceful) and delete the pid file in case
    :: the above graceful shutdown did not work
    TASKKILL /IM /F celery.exe
    DEL /F %celery_pid%
    ::-----------------------------------------------------------------------------
    
    
    ::-----------------------------------------------------------------------------
    :: Now remove all entries in rabbitmq
    :: First obtain a list of all the RabbitMQ queue names and save this to a file
    :: called q_names.txt. The contents of the rabbitmqadmin.py script can be obtained
    :: by following the instructions here: https://www.rabbitmq.com/management-cli.html
    %python_exe% %rabbitmqadmin% -f tsv -q list queues name > %q_names%
    
    :: Next, read in each line of q_names.txt and delete that queue
    FOR /F "tokens=*" %%A IN (%q_names%) DO (
         %rabbitmqadmin% delete queue name=%%A
    )
    ECHO. >> %q_names%
    ECHO "Queues deleted" >> %q_names%
    ::-----------------------------------------------------------------------------
    
    
    ::-----------------------------------------------------------------------------
    :: Now restart a new instance of celery
    celery worker -n default -P solo -Ofair -A openremproject -c 4 -Q default --pidfile=e:\media_root\celery\default.pid --logfile=e:\media_root\celery\default.log
    ::-----------------------------------------------------------------------------
    
  2. David Platten reporter

    On my Windows system it turns out that there's no need to stop celery before deleting the queues because celery automatically re-creates the essential queues on deletion. So, the following works for me (confirmed as working):

    ECHO OFF
    
    ::-----------------------------------------------------------------------------
    :: Set up some variables
    :: Reminder: there must not be any whitespace on either side of the equals when
    :: assigning variables in a batch file.
    ::
    :: The python executable (you may need to include the full path).
    SET python_exe=python.exe
    :: The rabbitmqadmin.py script. You need to create this file by following the
    :: instructions here: https://www.rabbitmq.com/management-cli.html
    SET rabbitmqadmin=D:\Server_Apps\rabbit_mq\rabbitmq_server-3.6.9\rabbitmqadmin.py
    :: The path and file name of a temporary file to store the queue names
    SET q_names=E:\rabbit_mq\q_names.txt
    ::-----------------------------------------------------------------------------
    
    ::-----------------------------------------------------------------------------
    :: Now remove all entries in rabbitmq
    :: First obtain a list of all the RabbitMQ queue names and save this to a file.
    %python_exe% %rabbitmqadmin% -f tsv -q list queues name > %q_names%
    :: Next, read in the name of each queue from the file and delete that queue
    FOR /F "tokens=*" %%A IN (%q_names%) DO %python_exe% %rabbitmqadmin% delete queue name=%%A
    ::-----------------------------------------------------------------------------
    

    @edmcdonagh, I'm sure there's a simple way of scripting the equivalent to this in Linux...

  3. Ed McDonagh

    Scrap that I can see I just needed to run sudo rabbitmq-plugins enable rabbitmq_management then browse to http://localhost:15672/cli/rabbitmqadmin to get the python file.

  4. Ed McDonagh

    Having a quick look at the API, it wouldn't be hard to have a page in the web interface that reports on queue lengths, aliveness and enables queue clearing. http://localhost:15672/api/

  5. David Platten reporter

    You're right - it looks like using the api would enable this to be done via a web interface. As well as the coding it would involve adding instructions on how to install the RabbitMQ management plugin, but that isn't too involved.

  6. Log in to comment