Clone wiki

Migratory / Overview

Overview

So the idea is a django orm database migration system that:

  • Is simple.
  • Doesn't make you use sql. This is an orm, we shouldn't have to use sql.
  • Can be automatic. Predicts the migration script for you so you don't have to think about what has changed.
  • Works well in a version control system, or even distributed ones. Because damnit.
  • During the migration process, allows you to use the state of your previous models as if they were still there. This is key, and is not done anywhere else, as far as I know.

System

So you have a migrations folder in your app. It has zero to many migration files, named things like 2008-12-08-added-a-blog-model.py, and 2008-12-09-removed-the-crappy-blog-model.py as well as a __manifest__.py file that lists, as a python list, all the migrations in order.

Now, when you syncdb and there are outstanding migrations, they are run in the order specified.

python manage.py migrate

To create a migration you do: python manage.py migrate app-name migration-name This checks the state of your models, cross-references it with the previous state of your models. It then creates the migration script for you, and tells you what that script will do, in english (no localization yet). You can then simply syncdb to run those changes, or you can edit the script if the prediction mechanism got anything wrong.

Migrations

The migration files define a simple function up(). It takes one argument database, that allows you to interact with the database and retrieve old models. These models are like old versions since your last syncdb, and you can use them just like a normal model, except that they are 'naked', that is they don't have any extra functions or properties that you defined. But you can still interact with them like you do with regular django models.

Let's say we go from this models.py:

from django.db import models

class Entry(models.Model):
    name = models.CharField(max_length=255)
    text = models.TextField(blank=True)

To this:

from django.db import models

class Entry(models.Model):
    slug = models.SlugField()
    text = models.TextField(blank=True)

We do python manage migrate app remove-names-add-slugs which creates two files for us. First, it creates a snapshot of your current models (2008-12-23-remove-names-add-slugs.snap), and second, it creates the migration script:

"""
Django Migration 2008-12-23-remove-names-add-slugs.py

"""

def up(database):
    database.remove_field('Entry', 'name')
    database.add_field('Entry', 'slug')

But we realize that we will lose all of our names, yet what we really want is to change them to slugs. We want to take our existing names, strip them down to 50 characters, replace all their spaces with dashes, making them slugs, and then insert them into our new slug field. Okay, no problem, edit the file:

"""
Django Migration 2008-12-23-remove-names-add-slugs.py

"""

def up(database):
    entries = database['Entry'].objects.all()
    names = dict((entry.id, entry.name) for entry in entries)

    database.remove_field('Entry', 'name')
    database.add_field('Entry', 'slug')

    for entry in database['Entry'].objects.all():
          entry.slug = names[entry.id][:50].strip().replace(' ', '-).lower()

So even though our current models.py didn't have a field called 'name', we can still use the old representation, which is current to the database itself, using the 'database' object.

Version Control

__manifest__.py isn't just a lazy way to specify the order of migrations, it also serves as an elegant solution to version control conflicts with migrations.

Let's say your manifest looks like this:

[
    "2008-12-23-added-a-whatzit.py",
    "2008-12-24-made-a-whosit.py",
    "2008-12-24-killed-all-whosits.py",
]

And one person adds a new migration in their branch:

[
    "2008-12-23-added-a-whatzit.py",
    "2008-12-24-made-a-whosit.py",
    "2008-12-24-killed-all-whosits.py",
    "2008-12-24-the-new-migration-from-bob.py",
]

And you add a new migration in your branch:

[
    "2008-12-23-added-a-whatzit.py",
    "2008-12-24-made-a-whosit.py",
    "2008-12-24-killed-all-whosits.py",
    "2008-12-24-added-a-wherezit-generator.py",
]

When you merge, you will have a conflict on that line, which you can then resolve during your vcs merge, exactly when you want think about it.

[
    "2008-12-23-added-a-whatzit.py",
    "2008-12-24-made-a-whosit.py",
    "2008-12-24-killed-all-whosits.py",
<<<<<<<
    "2008-12-24-added-a-wherezit-generator.py",
=======
    "2008-12-24-the-new-migration-from-bob.py",
>>>>>>>
]

Updated