Kolibri is a reusable django app for designing and executing asynchronous processes and workflows. A workflow is a collections of steps in a defined order, processing data in each step. A step can break the flow if an exception is raised and/or a specified step can be executed to handle a specific exception. Kolibri uses celery to handle processing in the background. All processors and workflows can only be started by staff members, but more fine grained access control might be implemented in future versions.
The project got started because I needed to control how I added content to a photo project I'm developing in django. The project involved lots of heavy processes like thumbnail generation and metadata processing. Adding content consists of steps that needs to be done in a specific order, and I need to control what action to take if one step throws an exception. I was using celery, but adding a new step or process was tedious and I wanted more dynamic way of defining and managing processors.
The current implementation is not stable and a proof of concept. Comments very welcome, especially on how to monitor status of celery processes and provide feedback to the user.
- asynchronous processes, which can process items/querysets or execute processes not related to specific models or instances (sending email, scanning filesystems etc)
- connect several processors into workflows, with exception handling, clean-up steps and an optional fluent interface
- template tags to handle execution of processors/workflows for an item or queryset in your templates
- admin action integration for your models
- dashboard listing running processors
- a concept of pending processors and a history of what has been processed so you don't execute unnecessary processesors or workflows
- user exclusive processors so two users can execute the same processor at the same time without touching the same data
- logging and history, with direct link to processed instances
- ajax integration using jquery
- better examples, more detailed tutorial and actual documentation in the source
- full-blown dashboard with feedback and progress from running processes and some way of killing processes
- nicely formatted logs and history for processed items
- a way of telling users that something is going on with the item they're looking at (progressbar, growl notification etc.)
pip install django-kolibri
hg clone https://bitbucket.org/weholt/django-kolibri python setup.py install
- set STATIC_ROOT and STATIC_URL in settings.py
- add 'kolibri' to your installed apps
- add url(r'^kolibri/', include('kolibri.urls')), to your urls.py
It would be smart to read through usage.txt first for a more detailed tutorial or experiment with the working example project provided in the source, available at bitbucket.
- Celery / django-celery
The simplest processor you can define looks something like:
from kolibri.core import * from models import * dirty_words = ('foo', 'fudge', 'bar',) class RemoveProfanity(Processor): model = Article def process(self, user, article, **kwargs): for dirty_word in dirty_words: article.text = article.text.replace(dirty_word,'*'*len(dirty_word)) article.save() manager.register.processor(RemoveProfanity())
It's a very simple processor which replaces all dirty words, defined in dirty_words, with * from instances of a model called Article.
To create a workflow, connecting a series of processors:
from kolibri.core import manager from kolibri.core.workflow import Workflow workflow = Workflow('Publish article', model=Article) workflow.first(RemoveProfanity()).on_exception(ValueError, DirtyWordRemover()).\ then(PublishArticle()).then(ArchiveArticle()) manager.register.workflow(workflow)
Here we create a workflow called "Publish article" for the Article-model. First we remove all profanity using the RemoveProfanity, if RemoveProfanity raises an ValueError we run the DirtyWordRemover-processor, then we publish the article using a processor called PublishArticle and finally we archive it.
See the usage.txt document in the source for more examples and in-depth explanation of features.
- 0.2.0 - support for user input. See bottom of usage description for more info.
- 0.1.1 - Added support for only running a processor once for an instance.
- 0.1.0 - Initial release. Pre-alpha state.