tsuyukimakoto / qamasu / wiki / Home — bitbucket.org

Qamasu . Job Queue Application written in Python

Qamasu is JobQueue system that respects TheSchwartz.

Requirements

((Python2.5 and simplejson) or Python2.6) and Django1.0

Usage

Set Qamasu up!

Qamasu is a Django application.

You need add qamasu to your or new django project's INSTALLED_APPS.

And manage.py syncdb.

Write your worker.

Define GRAB_FOR in seconds that is max time worker grabbed for a work.

Define def work_safely(manager, job): that is a work you need.

See sample worker in workers directory for detail.

Registration

You need add worker to abilities.

1
2
3

>>> from qamasu import Qamasu
>>> qamasu = Qamasu(['workers.random_wait',])
>>> qamasu.register_func('workers.random_wait')

Queue!

>>> from qamasu import Qamasu
>>> from uuid import uuid4 as uuid
>>> from random import uniform
>>> qamasu = Qamasu(['workers.random_wait',])
>>> for x in xrange(1,500):
      arg = dict(random_number=uniform(1,5))
      qamasu.enqueue('workers.random_wait', arg, uuid().hex)

Add hundreds Queues.

then.

1	>>> qamasu.enqueue('workers.random_wait', dict(random_number=uniform(1,5)), uuid().hex, priority=1)

A highest-priority queue is added.

Work! Work! Work!

1
2
3

>>> from qamasu import Qamasu
>>> qamasu = Qamasu(['workers.random_wait',])
>>> qamasu.work()

FIFO.

1
2
3

>>> from qamasu import Qamasu
>>> qamasu = Qamasu(['workers.random_wait',])
>>> qamasu.work_prioritizing()

Respect to priority.

Caution!

For MySQL backend

You must set worker's transaction isolation level to read commited before working qamasu when you use InnoDB.

>>> from django.db import connection
>>> from qamasu import Qamasu
>>> connection.cursor().execute('set session transaction isolation level read committed')
>>> qamasu = Qamasu(['workers.random_wait',])
>>> qamasu.work()

Or you have to set transaction isolation level read committed. It's global settings and dangerous.

1 2	[mysqld] transaction-isolation=Read-Committed

This revision is from 2009-10-11 16:34