Django 1.3 release notes

March 23, 2011

Welcome to Django 1.3!

Nearly a year in the making, Django 1.3 includes quite a few new features and plenty of bug fixes and improvements to existing features. These release notes cover the new features in 1.3, as well as some backwards-incompatible changes you’ll want to be aware of when upgrading from Django 1.2 or older versions.

Overview

Django 1.3’s focus has mostly been on resolving smaller, long-standing feature requests, but that hasn’t prevented a few fairly significant new features from landing, including:

• A framework for writing class-based views.
• Built-in support for using Python’s logging facilities.
• Contrib support for easy handling of static files.
• Django’s testing framework now supports (and ships with a copy of) the unittest2 library.

Wherever possible, new features are introduced in a backwards-compatible manner per our API stability policy policy. As a result of this policy, Django 1.3 begins the deprecation process for some features.

Python compatibility

The release of Django 1.2 was notable for having the first shift in Django’s Python compatibility policy; prior to Django 1.2, Django supported any 2.x version of Python from 2.3 up. As of Django 1.2, the minimum requirement was raised to Python 2.4.

Django 1.3 continues to support Python 2.4, but will be the final Django release series to do so; beginning with Django 1.4, the minimum supported Python version will be 2.5. A document outlining our full timeline for deprecating Python 2.x and moving to Python 3.x will be published shortly after the release of Django 1.3.

What’s new in Django 1.3

Class-based views

Django 1.3 adds a framework that allows you to use a class as a view. This means you can compose a view out of a collection of methods that can be subclassed and overridden to provide common views of data without having to write too much code.

Analogs of all the old function-based generic views have been provided, along with a completely generic view base class that can be used as the basis for reusable applications that can be easily extended.

See the documentation on class-based generic views for more details. There is also a document to help you convert your function-based generic views to class-based views.

Logging

Django 1.3 adds framework-level support for Python’s logging module. This means you can now easily configure and control logging as part of your Django project. A number of logging handlers and logging calls have been added to Django’s own code as well – most notably, the error emails sent on a HTTP 500 server error are now handled as a logging activity. See the documentation on Django’s logging interface for more details.

Extended static files handling

Django 1.3 ships with a new contrib app – django.contrib.staticfiles – to help developers handle the static media files (images, CSS, JavaScript, etc.) that are needed to render a complete web page.

In previous versions of Django, it was common to place static assets in MEDIA_ROOT along with user-uploaded files, and serve them both at MEDIA_URL. Part of the purpose of introducing the staticfiles app is to make it easier to keep static files separate from user-uploaded files. Static assets should now go in static/ subdirectories of your apps or in other static assets directories listed in STATICFILES_DIRS, and will be served at STATIC_URL.

See the reference documentation of the app for more details or learn how to manage static files.

unittest2 support

Python 2.7 introduced some major changes to the unittest library, adding some extremely useful features. To ensure that every Django project can benefit from these new features, Django ships with a copy of unittest2, a copy of the Python 2.7 unittest library, backported for Python 2.4 compatibility.

To access this library, Django provides the django.utils.unittest module alias. If you are using Python 2.7, or you have installed unittest2 locally, Django will map the alias to the installed version of the unittest library. Otherwise, Django will use its own bundled version of unittest2.

To take advantage of this alias, simply use:

from django.utils import unittest

wherever you would have historically used:

import unittest

If you want to continue to use the base unittest library, you can – you just won’t get any of the nice new unittest2 features.

Transaction context managers

Users of Python 2.5 and above may now use transaction management functions as context managers. For example:

with transaction.autocommit():
    # ...

Configurable delete-cascade

ForeignKey and OneToOneField now accept an on_delete argument to customize behavior when the referenced object is deleted. Previously, deletes were always cascaded; available alternatives now include set null, set default, set to any value, protect, or do nothing.

For more information, see the on_delete documentation.

Contextual markers and comments for translatable strings

For translation strings with ambiguous meaning, you can now use the pgettext function to specify the context of the string.

And if you just want to add some information for translators, you can also add special translator comments in the source.

For more information, see Contextual markers and Comments for translators.

Improvements to built-in template tags

A number of improvements have been made to Django’s built-in template tags:

• The include tag now accepts a with option, allowing you to specify context variables to the included template
• The include tag now accepts an only option, allowing you to exclude the current context from the included context
• The with tag now allows you to define multiple context variables in a single with block.
• The load tag now accepts a from argument, allowing you to load a single tag or filter from a library.

TemplateResponse

It can sometimes be beneficial to allow decorators or middleware to modify a response after it has been constructed by the view. For example, you may want to change the template that is used, or put additional data into the context.

However, you can’t (easily) modify the content of a basic HttpResponse after it has been constructed. To overcome this limitation, Django 1.3 adds a new TemplateResponse class. Unlike basic HttpResponse objects, TemplateResponse objects retain the details of the template and context that was provided by the view to compute the response. The final output of the response is not computed until it is needed, later in the response process.

For more details, see the documentation on the TemplateResponse class.

Caching changes

Django 1.3 sees the introduction of several improvements to the Django’s caching infrastructure.

Firstly, Django now supports multiple named caches. In the same way that Django 1.2 introduced support for multiple database connections, Django 1.3 allows you to use the new CACHES setting to define multiple named cache connections.

Secondly, versioning, site-wide prefixing and transformation have been added to the cache API.

Thirdly, cache key creation has been updated to take the request query string into account on GET requests.

Finally, support for pylibmc has been added to the memcached cache backend.

For more details, see the documentation on caching in Django.

Permissions for inactive users

If you provide a custom auth backend with supports_inactive_user set to True, an inactive User instance will check the backend for permissions. This is useful for further centralizing the permission handling. See the authentication docs for more details.

GeoDjango

The GeoDjango test suite is now included when running the Django test suite with runtests.py when using spatial database backends.

MEDIA_URL and STATIC_URL must end in a slash

Previously, the MEDIA_URL setting only required a trailing slash if it contained a suffix beyond the domain name.

A trailing slash is now required for MEDIA_URL and the new STATIC_URL setting as long as it is not blank. This ensures there is a consistent way to combine paths in templates.

Project settings which provide either of both settings without a trailing slash will now raise a PendingDeprecationWarning.

In Django 1.4 this same condition will raise DeprecationWarning, and in Django 1.5 will raise an ImproperlyConfigured exception.

Everything else

Django 1.1 and 1.2 added lots of big ticket items to Django, like multiple-database support, model validation, and a session-based messages framework. However, this focus on big features came at the cost of lots of smaller features.

To compensate for this, the focus of the Django 1.3 development process has been on adding lots of smaller, long standing feature requests. These include:

• Improved tools for accessing and manipulating the current Site object in the sites framework.
• A RequestFactory for mocking requests in tests.
• A new test assertion – assertNumQueries() – making it easier to test the database activity associated with a view.
• Support for lookups spanning relations in admin’s list_filter.
• Support for HttpOnly cookies.
• mail_admins() and mail_managers() now support easily attaching HTML content to messages.
• EmailMessage now supports CC’s.
• Error emails now include more of the detail and formatting of the debug server error page.
• simple_tag() now accepts a takes_context argument, making it easier to write simple template tags that require access to template context.
• A new render() shortcut – an alternative to django.shortcuts.render_to_response() providing a RequestContext by default.
• Support for combining F expressions with timedelta values when retrieving or updating database values.

Backwards-incompatible changes in 1.3

CSRF validation now applies to AJAX requests

Prior to Django 1.2.5, Django’s CSRF-prevention system exempted AJAX requests from CSRF verification; due to security issues reported to us, however, all requests are now subjected to CSRF verification. Consult the Django CSRF documentation for details on how to handle CSRF verification in AJAX requests.

Restricted filters in admin interface

Prior to Django 1.2.5, the Django administrative interface allowed filtering on any model field or relation – not just those specified in list_filter – via query string manipulation. Due to security issues reported to us, however, query string lookup arguments in the admin must be for fields or relations specified in list_filter or date_hierarchy.

Deleting a model doesn’t delete associated files

In earlier Django versions, when a model instance containing a FileField was deleted, FileField took it upon itself to also delete the file from the backend storage. This opened the door to several data-loss scenarios, including rolled-back transactions and fields on different models referencing the same file. In Django 1.3, when a model is deleted the FileField’s delete() method won’t be called. If you need cleanup of orphaned files, you’ll need to handle it yourself (for instance, with a custom management command that can be run manually or scheduled to run periodically via e.g. cron).

PasswordInput default rendering behavior

The PasswordInput form widget, intended for use with form fields which represent passwords, accepts a boolean keyword argument render_value indicating whether to send its data back to the browser when displaying a submitted form with errors. Prior to Django 1.3, this argument defaulted to True, meaning that the submitted password would be sent back to the browser as part of the form. Developers who wished to add a bit of additional security by excluding that value from the redisplayed form could instantiate a PasswordInput passing render_value=False .

Due to the sensitive nature of passwords, however, Django 1.3 takes this step automatically; the default value of render_value is now False, and developers who want the password value returned to the browser on a submission with errors (the previous behavior) must now explicitly indicate this. For example:

class LoginForm(forms.Form):
    username = forms.CharField(max_length=100)
    password = forms.CharField(widget=forms.PasswordInput(render_value=True))

Clearable default widget for FileField

Django 1.3 now includes a ClearableFileInput form widget in addition to FileInput. ClearableFileInput renders with a checkbox to clear the field’s value (if the field has a value and is not required); FileInput provided no means for clearing an existing file from a FileField.

ClearableFileInput is now the default widget for a FileField, so existing forms including FileField without assigning a custom widget will need to account for the possible extra checkbox in the rendered form output.

To return to the previous rendering (without the ability to clear the FileField), use the FileInput widget in place of ClearableFileInput. For instance, in a ModelForm for a hypothetical Document model with a FileField named document:

from django import forms
from myapp.models import Document
class DocumentForm(forms.ModelForm):
    class Meta:
        model = Document
        widgets = {'document': forms.FileInput}

New index on database session table

Prior to Django 1.3, the database table used by the database backend for the sessions app had no index on the expire_date column. As a result, date-based queries on the session table – such as the query that is needed to purge old sessions – would be very slow if there were lots of sessions.

If you have an existing project that is using the database session backend, you don’t have to do anything to accommodate this change. However, you may get a significant performance boost if you manually add the new index to the session table. The SQL that will add the index can be found by running the sqlindexes admin command:

python manage.py sqlindexes sessions

No more naughty words

Django has historically provided (and enforced) a list of profanities. The comments app has enforced this list of profanities, preventing people from submitting comments that contained one of those profanities.

Unfortunately, the technique used to implement this profanities list was woefully naive, and prone to the Scunthorpe problem. Improving the built-in filter to fix this problem would require significant effort, and since natural language processing isn’t the normal domain of a web framework, we have “fixed” the problem by making the list of prohibited words an empty list.

If you want to restore the old behavior, simply put a PROFANITIES_LIST setting in your settings file that includes the words that you want to prohibit (see the commit that implemented this change if you want to see the list of words that was historically prohibited). However, if avoiding profanities is important to you, you would be well advised to seek out a better, less naive approach to the problem.

Localflavor changes

Django 1.3 introduces the following backwards-incompatible changes to local flavors:

• Canada (ca) – The province “Newfoundland and Labrador” has had its province code updated to “NL”, rather than the older “NF”. In addition, the Yukon Territory has had its province code corrected to “YT”, instead of “YK”.
• Indonesia (id) – The province “Nanggroe Aceh Darussalam (NAD)” has been removed from the province list in favor of the new official designation “Aceh (ACE)”.
• United States of America (us) – The list of “states” used by USStateField has expanded to include Armed Forces postal codes. This is backwards-incompatible if you were relying on USStateField not including them.

FormSet updates

In Django 1.3 FormSet creation behavior is modified slightly. Historically the class didn’t make a distinction between not being passed data and being passed empty dictionary. This was inconsistent with behavior in other parts of the framework. Starting with 1.3 if you pass in empty dictionary the FormSet will raise a ValidationError.

For example with a FormSet:

>>> class ArticleForm(Form):
...     title = CharField()
...     pub_date = DateField()
>>> ArticleFormSet = formset_factory(ArticleForm)

the following code will raise a ValidationError:

>>> ArticleFormSet({})
Traceback (most recent call last):
...
ValidationError: [u'ManagementForm data is missing or has been tampered with']

if you need to instantiate an empty FormSet, don’t pass in the data or use None:

>>> formset = ArticleFormSet()
>>> formset = ArticleFormSet(data=None)

Callables in templates

Previously, a callable in a template would only be called automatically as part of the variable resolution process if it was retrieved via attribute lookup. This was an inconsistency that could result in confusing and unhelpful behavior:

>>> Template("{{ user.get_full_name }}").render(Context({'user': user}))
u'Joe Bloggs'
>>> Template("{{ full_name }}").render(Context({'full_name': user.get_full_name}))
u'<bound method User.get_full_name of <...

This has been resolved in Django 1.3 - the result in both cases will be u'Joe Bloggs'. Although the previous behavior was not useful for a template language designed for web designers, and was never deliberately supported, it is possible that some templates may be broken by this change.

Use of custom SQL to load initial data in tests

Django provides a custom SQL hooks as a way to inject hand-crafted SQL into the database synchronization process. One of the possible uses for this custom SQL is to insert data into your database. If your custom SQL contains INSERT statements, those insertions will be performed every time your database is synchronized. This includes the synchronization of any test databases that are created when you run a test suite.

However, in the process of testing the Django 1.3, it was discovered that this feature has never completely worked as advertised. When using database backends that don’t support transactions, or when using a TransactionTestCase, data that has been inserted using custom SQL will not be visible during the testing process.

Unfortunately, there was no way to rectify this problem without introducing a backwards incompatibility. Rather than leave SQL-inserted initial data in an uncertain state, Django now enforces the policy that data inserted by custom SQL will not be visible during testing.

This change only affects the testing process. You can still use custom SQL to load data into your production database as part of the syncdb process. If you require data to exist during test conditions, you should either insert it using test fixtures, or using the setUp() method of your test case.

Changed priority of translation loading

Work has been done to simplify, rationalize and properly document the algorithm used by Django at runtime to build translations from the different translations found on disk, namely:

For translatable literals found in Python code and templates ('django' gettext domain):

• Priorities of translations included with applications listed in the INSTALLED_APPS setting were changed. To provide a behavior consistent with other parts of Django that also use such setting (templates, etc.) now, when building the translation that will be made available, the apps listed first have higher precedence than the ones listed later.
• Now it is possible to override the translations shipped with applications by using the LOCALE_PATHS setting whose translations have now higher precedence than the translations of INSTALLED_APPS applications. The relative priority among the values listed in this setting has also been modified so the paths listed first have higher precedence than the ones listed later.
• The locale subdirectory of the directory containing the settings, that usually coincides with and is known as the project directory is being deprecated in this release as a source of translations. (the precedence of these translations is intermediate between applications and LOCALE_PATHS translations). See the corresponding deprecated features section of this document.

For translatable literals found in JavaScript code ('djangojs' gettext domain):

• Similarly to the 'django' domain translations: Overriding of translations shipped with applications by using the LOCALE_PATHS setting is now possible for this domain too. These translations have higher precedence than the translations of Python packages passed to the javascript_catalog() view. Paths listed first have higher precedence than the ones listed later.
• Translations under the locale subdirectory of the project directory have never been taken in account for JavaScript translations and remain in the same situation considering the deprecation of such location.

Transaction management

When using managed transactions – that is, anything but the default autocommit mode – it is important when a transaction is marked as “dirty”. Dirty transactions are committed by the commit_on_success decorator or the django.middleware.transaction.TransactionMiddleware, and commit_manually forces them to be closed explicitly; clean transactions “get a pass”, which means they are usually rolled back at the end of a request when the connection is closed.

Until Django 1.3, transactions were only marked dirty when Django was aware of a modifying operation performed in them; that is, either some model was saved, some bulk update or delete was performed, or the user explicitly called transaction.set_dirty(). In Django 1.3, a transaction is marked dirty when any database operation is performed.

As a result of this change, you no longer need to set a transaction dirty explicitly when you execute raw SQL or use a data-modifying SELECT. However, you do need to explicitly close any read-only transactions that are being managed using commit_manually(). For example:

@transaction.commit_manually
def my_view(request, name):
    obj = get_object_or_404(MyObject, name__iexact=name)
    return render_to_response('template', {'object':obj})

Prior to Django 1.3, this would work without error. However, under Django 1.3, this will raise a TransactionManagementError because the read operation that retrieves the MyObject instance leaves the transaction in a dirty state.

No password reset for inactive users

Prior to Django 1.3, inactive users were able to request a password reset email and reset their password. In Django 1.3 inactive users will receive the same message as a nonexistent account.

Password reset view now accepts from_email

The django.contrib.auth.views.password_reset() view now accepts a from_email parameter, which is passed to the password_reset_form’s save() method as a keyword argument. If you are using this view with a custom password reset form, then you will need to ensure your form’s save() method accepts this keyword argument.

Features deprecated in 1.3

Django 1.3 deprecates some features from earlier releases. These features are still supported, but will be gradually phased out over the next few release cycles.

Code taking advantage of any of the features below will raise a PendingDeprecationWarning in Django 1.3. This warning will be silent by default, but may be turned on using Python’s warnings module, or by running Python with a -Wd or -Wall flag.

In Django 1.4, these warnings will become a DeprecationWarning, which is not silent. In Django 1.5 support for these features will be removed entirely.

See also

For more details, see the documentation Django’s release process and our deprecation timeline.

mod_python support

The mod_python library has not had a release since 2007 or a commit since 2008. The Apache Foundation board voted to remove mod_python from the set of active projects in its version control repositories, and its lead developer has shifted all of his efforts toward the lighter, slimmer, more stable, and more flexible mod_wsgi backend.

If you are currently using the mod_python request handler, you should redeploy your Django projects using another request handler. mod_wsgi is the request handler recommended by the Django project, but FastCGI is also supported. Support for mod_python deployment will be removed in Django 1.5.

Function-based generic views

As a result of the introduction of class-based generic views, the function-based generic views provided by Django have been deprecated. The following modules and the views they contain have been deprecated:

• django.views.generic.create_update
• django.views.generic.date_based
• django.views.generic.list_detail
• django.views.generic.simple

Test client response template attribute

Django’s test client returns Response objects annotated with extra testing information. In Django versions prior to 1.3, this included a template attribute containing information about templates rendered in generating the response: either None, a single Template object, or a list of Template objects. This inconsistency in return values (sometimes a list, sometimes not) made the attribute difficult to work with.

In Django 1.3 the template attribute is deprecated in favor of a new templates attribute, which is always a list, even if it has only a single element or no elements.

DjangoTestRunner

As a result of the introduction of support for unittest2, the features of django.test.simple.DjangoTestRunner (including fail-fast and Ctrl-C test termination) have been made redundant. In view of this redundancy, DjangoTestRunner has been turned into an empty placeholder class, and will be removed entirely in Django 1.5.

Changes to url and ssi

Most template tags will allow you to pass in either constants or variables as arguments – for example:

{% extends "base.html" %}

allows you to specify a base template as a constant, but if you have a context variable templ that contains the value base.html:

{% extends templ %}

is also legal.

However, due to an accident of history, the url and ssi are different. These tags use the second, quoteless syntax, but interpret the argument as a constant. This means it isn’t possible to use a context variable as the target of a url and ssi tag.

Django 1.3 marks the start of the process to correct this historical accident. Django 1.3 adds a new template library – future – that provides alternate implementations of the url and ssi template tags. This future library implement behavior that makes the handling of the first argument consistent with the handling of all other variables. So, an existing template that contains:

{% url sample %}

should be replaced with:

{% load url from future %}
{% url 'sample' %}

The tags implementing the old behavior have been deprecated, and in Django 1.5, the old behavior will be replaced with the new behavior. To ensure compatibility with future versions of Django, existing templates should be modified to use the new future libraries and syntax.

Changes to the login methods of the admin

In previous version the admin app defined login methods in multiple locations and ignored the almost identical implementation in the already used auth app. A side effect of this duplication was the missing adoption of the changes made in r12634 to support a broader set of characters for usernames.

This release refactors the admin’s login mechanism to use a subclass of the AuthenticationForm instead of a manual form validation. The previously undocumented method 'django.contrib.admin.sites.AdminSite.display_login_form' has been removed in favor of a new login_form attribute.

reset and sqlreset management commands

Those commands have been deprecated. The flush and sqlflush commands can be used to delete everything. You can also use ALTER TABLE or DROP TABLE statements manually.

GeoDjango

• The function-based TEST_RUNNER previously used to execute the GeoDjango test suite, django.contrib.gis.tests.run_gis_tests, was deprecated for the class-based runner, django.contrib.gis.tests.GeoDjangoTestSuiteRunner.
• Previously, calling transform() would silently do nothing when GDAL wasn’t available. Now, a GEOSException is properly raised to indicate possible faulty application code. A warning is now raised if transform() is called when the SRID of the geometry is less than 0 or None.

CZBirthNumberField.clean

Previously this field’s clean() method accepted a second, gender, argument which allowed stronger validation checks to be made, however since this argument could never actually be passed from the Django form machinery it is now pending deprecation.

CompatCookie

Previously, django.http exposed an undocumented CompatCookie class, which was a bugfix wrapper around the standard library SimpleCookie. As the fixes are moving upstream, this is now deprecated - you should use from django.http import SimpleCookie instead.

Loading of project-level translations

This release of Django starts the deprecation process for inclusion of translations located under the so-called project path in the translation building process performed at runtime. The LOCALE_PATHS setting can be used for the same task by adding the filesystem path to a locale directory containing project-level translations to the value of that setting.

Rationale for this decision:

• The project path has always been a loosely defined concept (actually, the directory used for locating project-level translations is the directory containing the settings module) and there has been a shift in other parts of the framework to stop using it as a reference for location of assets at runtime.

• Detection of the locale subdirectory tends to fail when the deployment scenario is more complex than the basic one. e.g. it fails when the settings module is a directory (ticket #10765).

• There are potential strange development- and deployment-time problems like the fact that the project_dir/locale/ subdir can generate spurious error messages when the project directory is added to the Python path (manage.py runserver does this) and then it clashes with the equally named standard library module, this is a typical warning message:

  /usr/lib/python2.6/gettext.py:49: ImportWarning: Not importing directory '/path/to/project/locale': missing __init__.py.
  import locale, copy, os, re, struct, sys

• This location wasn’t included in the translation building process for JavaScript literals. This deprecation removes such inconsistency.

PermWrapper moved to django.contrib.auth.context_processors

In Django 1.2, we began the process of changing the location of the auth context processor from django.core.context_processors to django.contrib.auth.context_processors. However, the PermWrapper support class was mistakenly omitted from that migration. In Django 1.3, the PermWrapper class has also been moved to django.contrib.auth.context_processors, along with the PermLookupDict support class. The new classes are functionally identical to their old versions; only the module location has changed.

Removal of XMLField

When Django was first released, Django included an XMLField that performed automatic XML validation for any field input. However, this validation function hasn’t been performed since the introduction of newforms, prior to the 1.0 release. As a result, XMLField as currently implemented is functionally indistinguishable from a simple TextField.

For this reason, Django 1.3 has fast-tracked the deprecation of XMLField – instead of a two-release deprecation, XMLField will be removed entirely in Django 1.4.

It’s easy to update your code to accommodate this change – just replace all uses of XMLField with TextField, and remove the schema_path keyword argument (if it is specified).