Writing database migrations

This document explains how to structure and write database migrations fordifferent scenarios you might encounter. For introductory material onmigrations, see the topic guide.

Data migrations and multiple databases

When using multiple databases, you may need to figure out whether or not torun a migration against a particular database. For example, you may want toonly run a migration on a particular database.

In order to do that you can check the database connection’s alias inside aRunPython operation by looking at the schema_editor.connection.aliasattribute:

from django.db import migrations
 
def forwards(apps, schema_editor):
    if schema_editor.connection.alias != 'default':
        return
    # Your migration code goes here
 
class Migration(migrations.Migration):
 
    dependencies = [
        # Dependencies to other migrations
    ]
 
    operations = [
        migrations.RunPython(forwards),
    ]

You can also provide hints that will be passed to the allow_migrate()method of database routers as **hints:

myapp/dbrouters.py

class MyRouter:
 
    def allow_migrate(self, db, app_label, model_name=None, **hints):
        if 'target_db' in hints:
            return db == hints['target_db']
        return True

Then, to leverage this in your migrations, do the following:

from django.db import migrations
 
def forwards(apps, schema_editor):
    # Your migration code goes here
    ...
 
class Migration(migrations.Migration):
 
    dependencies = [
        # Dependencies to other migrations
    ]
 
    operations = [
        migrations.RunPython(forwards, hints={'target_db': 'default'}),
    ]

If your RunPython or RunSQL operation only affects one model, it’s goodpractice to pass model_name as a hint to make it as transparent as possibleto the router. This is especially important for reusable and third-party apps.

Migrations that add unique fields

Applying a “plain” migration that adds a unique non-nullable field to a tablewith existing rows will raise an error because the value used to populateexisting rows is generated only once, thus breaking the unique constraint.

Therefore, the following steps should be taken. In this example, we’ll add anon-nullable UUIDField with a default value. Modifythe respective field according to your needs.

• Add the field on your model with default=uuid.uuid4 and unique=Truearguments (choose an appropriate default for the type of the field you’readding).

• Run the makemigrations command. This should generate a migrationwith an AddField operation.

• Generate two empty migration files for the same app by runningmakemigrations myapp —empty twice. We’ve renamed the migration files togive them meaningful names in the examples below.

• Copy the AddField operation from the auto-generated migration (the firstof the three new files) to the last migration, change AddField toAlterField, and add imports of uuid and models. For example:

0006_remove_uuid_null.py

# Generated by Django A.B on YYYY-MM-DD HH:MM
from django.db import migrations, models
import uuid
 
class Migration(migrations.Migration):
 
    dependencies = [
        ('myapp', '0005_populate_uuid_values'),
    ]
 
    operations = [
        migrations.AlterField(
            model_name='mymodel',
            name='uuid',
            field=models.UUIDField(default=uuid.uuid4, unique=True),
        ),
    ]

• Edit the first migration file. The generated migration class should looksimilar to this:

0004_add_uuid_field.py

class Migration(migrations.Migration):
 
    dependencies = [
        ('myapp', '0003_auto_20150129_1705'),
    ]
 
    operations = [
        migrations.AddField(
            model_name='mymodel',
            name='uuid',
            field=models.UUIDField(default=uuid.uuid4, unique=True),
        ),
    ]

Change unique=True to null=True – this will create the intermediarynull field and defer creating the unique constraint until we’ve populatedunique values on all the rows.

• In the first empty migration file, add aRunPython orRunSQL operation to generate aunique value (UUID in the example) for each existing row. Also add an importof uuid. For example:

0005_populate_uuid_values.py

# Generated by Django A.B on YYYY-MM-DD HH:MM
from django.db import migrations
import uuid
 
def gen_uuid(apps, schema_editor):
    MyModel = apps.get_model('myapp', 'MyModel')
    for row in MyModel.objects.all():
        row.uuid = uuid.uuid4()
        row.save(update_fields=['uuid'])
 
class Migration(migrations.Migration):
 
    dependencies = [
        ('myapp', '0004_add_uuid_field'),
    ]
 
    operations = [
        # omit reverse_code=... if you don't want the migration to be reversible.
        migrations.RunPython(gen_uuid, reverse_code=migrations.RunPython.noop),
    ]

• Now you can apply the migrations as usual with the migrate command.

Note there is a race condition if you allow objects to be created while thismigration is running. Objects created after the AddField and beforeRunPython will have their original uuid’s overwritten.

Non-atomic migrations

On databases that support DDL transactions (SQLite and PostgreSQL), migrationswill run inside a transaction by default. For use cases such as performing datamigrations on large tables, you may want to prevent a migration from running ina transaction by setting the atomic attribute to False:

from django.db import migrations
 
class Migration(migrations.Migration):
    atomic = False

Within such a migration, all operations are run without a transaction. It’spossible to execute parts of the migration inside a transaction usingatomic() or by passing atomic=True toRunPython.

Here’s an example of a non-atomic data migration that updates a large table insmaller batches:

import uuid
 
from django.db import migrations, transaction
 
def gen_uuid(apps, schema_editor):
    MyModel = apps.get_model('myapp', 'MyModel')
    while MyModel.objects.filter(uuid__isnull=True).exists():
        with transaction.atomic():
            for row in MyModel.objects.filter(uuid__isnull=True)[:1000]:
                row.uuid = uuid.uuid4()
                row.save()
 
class Migration(migrations.Migration):
    atomic = False
 
    operations = [
        migrations.RunPython(gen_uuid),
    ]

The atomic attribute doesn’t have an effect on databases that don’t supportDDL transactions (e.g. MySQL, Oracle). (MySQL’s atomic DDL statement support refers to individualstatements rather than multiple statements wrapped in a transaction that can berolled back.)

Controlling the order of migrations

Django determines the order in which migrations should be applied not by thefilename of each migration, but by building a graph using two properties on theMigration class: dependencies and run_before.

If you’ve used the makemigrations command you’ve probablyalready seen dependencies in action because auto-createdmigrations have this defined as part of their creation process.

The dependencies property is declared like this:

from django.db import migrations
 
class Migration(migrations.Migration):
 
    dependencies = [
        ('myapp', '0123_the_previous_migration'),
    ]

Usually this will be enough, but from time to time you may need toensure that your migration runs before other migrations. This isuseful, for example, to make third-party apps’ migrations run _after_your AUTH_USER_MODEL replacement.

To achieve this, place all migrations that should depend on yours inthe run_before attribute on your Migration class:

class Migration(migrations.Migration):
    ...
 
    run_before = [
        ('third_party_app', '0001_do_awesome'),
    ]

Prefer using dependencies over run_before when possible. You shouldonly use run_before if it is undesirable or impractical to specifydependencies in the migration which you want to run after the one you arewriting.

Migrating data between third-party apps

You can use a data migration to move data from one third-party application toanother.

If you plan to remove the old app later, you’ll need to set the dependenciesproperty based on whether or not the old app is installed. Otherwise, you’llhave missing dependencies once you uninstall the old app. Similarly, you’llneed to catch LookupError in the apps.get_model() call thatretrieves models from the old app. This approach allows you to deploy yourproject anywhere without first installing and then uninstalling the old app.

Here’s a sample migration:

myapp/migrations/0124_move_old_app_to_new_app.py

from django.apps import apps as global_apps
from django.db import migrations
 
def forwards(apps, schema_editor):
    try:
        OldModel = apps.get_model('old_app', 'OldModel')
    except LookupError:
        # The old app isn't installed.
        return
 
    NewModel = apps.get_model('new_app', 'NewModel')
    NewModel.objects.bulk_create(
        NewModel(new_attribute=old_object.old_attribute)
        for old_object in OldModel.objects.all()
    )
 
class Migration(migrations.Migration):
    operations = [
        migrations.RunPython(forwards, migrations.RunPython.noop),
    ]
    dependencies = [
        ('myapp', '0123_the_previous_migration'),
        ('new_app', '0001_initial'),
    ]
 
    if global_apps.is_installed('old_app'):
        dependencies.append(('old_app', '0001_initial'))

Also consider what you want to happen when the migration is unapplied. Youcould either do nothing (as in the example above) or remove some or all of thedata from the new application. Adjust the second argument of theRunPython operation accordingly.

Changing an unmanaged model to managed

If you want to change an unmanaged model (managed=False) to managed, you must removemanaged=False and generate a migration before making other schema-relatedchanges to the model, since schema changes that appear in the migration thatcontains the operation to change Meta.managed may not be applied.