自定义查找

Django offers a wide variety of built-in lookups forfiltering (for example, exact and icontains). This documentationexplains how to write custom lookups and how to alter the working of existinglookups. For the API references of lookups, see the Lookup API reference.

一个简单的查找示例

Let's start with a simple custom lookup. We will write a custom lookup newhich works opposite to exact. Author.objects.filter(name__ne='Jack')will translate to the SQL:

"author"."name" <> 'Jack'

SQL 会自动适配不同的后端, 所以我们不需要对使用不同的数据库担心.

There are two steps to making this work. Firstly we need to implement thelookup, then we need to tell Django about it. The implementation is quitestraightforward:

from django.db.models import Lookup
 
class NotEqual(Lookup):
    lookup_name = 'ne'
 
    def as_sql(self, compiler, connection):
        lhs, lhs_params = self.process_lhs(compiler, connection)
        rhs, rhs_params = self.process_rhs(compiler, connection)
        params = lhs_params + rhs_params
        return '%s <> %s' % (lhs, rhs), params

To register the NotEqual lookup we will just need to callregister_lookup on the field class we want the lookup to be available. Inthis case, the lookup makes sense on all Field subclasses, so we registerit with Field directly:

from django.db.models.fields import Field
Field.register_lookup(NotEqual)

Lookup registration can also be done using a decorator pattern:

from django.db.models.fields import Field
 
@Field.register_lookup
class NotEqualLookup(Lookup):
    # ...

We can now use foo__ne for any field foo. You will need to ensure thatthis registration happens before you try to create any querysets using it. Youcould place the implementation in a models.py file, or register the lookupin the ready() method of an AppConfig.

Taking a closer look at the implementation, the first required attribute islookupname. This allows the ORM to understand how to interpret name_neand use NotEqual to generate the SQL. By convention, these names are alwayslowercase strings containing only letters, but the only hard requirement isthat it must not contain the string .

We then need to define the as_sql method. This takes a SQLCompilerobject, called compiler, and the active database connection.SQLCompiler objects are not documented, but the only thing we need to knowabout them is that they have a compile() method which returns a tuplecontaining an SQL string, and the parameters to be interpolated into thatstring. In most cases, you don't need to use it directly and can pass it on toprocess_lhs() and process_rhs().

A Lookup works against two values, lhs and rhs, standing forleft-hand side and right-hand side. The left-hand side is usually a fieldreference, but it can be anything implementing the query expression API. The right-hand is the value given by the user. In theexample Author.objects.filter(name__ne='Jack'), the left-hand side is areference to the name field of the Author model, and 'Jack' is theright-hand side.

We call process_lhs and process_rhs to convert them into the values weneed for SQL using the compiler object described before. These methodsreturn tuples containing some SQL and the parameters to be interpolated intothat SQL, just as we need to return from our as_sql method. In the aboveexample, process_lhs returns ('"author"."name"', []) andprocess_rhs returns ('"%s"', ['Jack']). In this example there were noparameters for the left hand side, but this would depend on the object we have,so we still need to include them in the parameters we return.

Finally we combine the parts into an SQL expression with <>, and supply allthe parameters for the query. We then return a tuple containing the generatedSQL string and the parameters.

A simple transformer example

The custom lookup above is great, but in some cases you may want to be able tochain lookups together. For example, let's suppose we are building anapplication where we want to make use of the abs() operator.We have an Experiment model which records a start value, end value, and thechange (start - end). We would like to find all experiments where the changewas equal to a certain amount (Experiment.objects.filter(changeabs=27)),or where it did not exceed a certain amount(Experiment.objects.filter(changeabs__lt=27)).

Note

This example is somewhat contrived, but it nicely demonstrates the range offunctionality which is possible in a database backend independent manner,and without duplicating functionality already in Django.

We will start by writing an AbsoluteValue transformer. This will use the SQLfunction ABS() to transform the value before comparison:

from django.db.models import Transform
 
class AbsoluteValue(Transform):
    lookup_name = 'abs'
    function = 'ABS'

下一步, 让我们为其注册 IntrgerField:

from django.db.models import IntegerField
IntegerField.register_lookup(AbsoluteValue)

We can now run the queries we had before.Experiment.objects.filter(change__abs=27) will generate the following SQL:

SELECT ... WHERE ABS("experiments"."change") = 27

By using Transform instead of Lookup it means we are able to chainfurther lookups afterwards. SoExperiment.objects.filter(changeabslt=27) will generate the followingSQL:

SELECT ... WHERE ABS("experiments"."change") < 27

Note that in case there is no other lookup specified, Django interpretschangeabs=27 as changeabs__exact=27.

When looking for which lookups are allowable after the Transform has beenapplied, Django uses the output_field attribute. We didn't need to specifythis here as it didn't change, but supposing we were applying AbsoluteValueto some field which represents a more complex type (for example a pointrelative to an origin, or a complex number) then we may have wanted to specifythat the transform returns a FloatField type for further lookups. This canbe done by adding an output_field attribute to the transform:

from django.db.models import FloatField, Transform
 
class AbsoluteValue(Transform):
    lookup_name = 'abs'
    function = 'ABS'
 
    @property
    def output_field(self):
        return FloatField()

This ensures that further lookups like abs__lte behave as they would fora FloatField.

编写一个高效的 abs__lt 查找

When using the above written abs lookup, the SQL produced will not useindexes efficiently in some cases. In particular, when we usechangeabslt=27, this is equivalent to changegt=-27 ANDchangelt=27. (For the lte case we could use the SQL BETWEEN).

因此, 我们希望 Experiment.objects.filter(changeabslt=27) 能生成以下 SQL:

SELECT .. WHERE "experiments"."change" < 27 AND "experiments"."change" > -27

实现方式是:

from django.db.models import Lookup
 
class AbsoluteValueLessThan(Lookup):
    lookup_name = 'lt'
 
    def as_sql(self, compiler, connection):
        lhs, lhs_params = compiler.compile(self.lhs.lhs)
        rhs, rhs_params = self.process_rhs(compiler, connection)
        params = lhs_params + rhs_params + lhs_params + rhs_params
        return '%s < %s AND %s > -%s' % (lhs, rhs, lhs, rhs), params
 
AbsoluteValue.register_lookup(AbsoluteValueLessThan)

There are a couple of notable things going on. First, AbsoluteValueLessThanisn't calling process_lhs(). Instead it skips the transformation of thelhs done by AbsoluteValue and uses the original lhs. That is, wewant to get "experiments"."change" not ABS("experiments"."change").Referring directly to self.lhs.lhs is safe as AbsoluteValueLessThancan be accessed only from the AbsoluteValue lookup, that is the lhsis always an instance of AbsoluteValue.

Notice also that as both sides are used multiple times in the query the paramsneed to contain lhs_params and rhs_params multiple times.

The final query does the inversion (27 to -27) directly in thedatabase. The reason for doing this is that if the self.rhs is something elsethan a plain integer value (for example an F() reference) we can't do thetransformations in Python.

Note

In fact, most lookups with __abs could be implemented as range querieslike this, and on most database backends it is likely to be more sensible todo so as you can make use of the indexes. However with PostgreSQL you maywant to add an index on abs(change) which would allow these queries tobe very efficient.

A bilateral transformer example

The AbsoluteValue example we discussed previously is a transformation whichapplies to the left-hand side of the lookup. There may be some cases where youwant the transformation to be applied to both the left-hand side and theright-hand side. For instance, if you want to filter a queryset based on theequality of the left and right-hand side insensitively to some SQL function.

Let's examine the simple example of case-insensitive transformation here. Thistransformation isn't very useful in practice as Django already comes with a bunchof built-in case-insensitive lookups, but it will be a nice demonstration ofbilateral transformations in a database-agnostic way.

We define an UpperCase transformer which uses the SQL function UPPER() totransform the values before comparison. We definebilateral = True to indicate thatthis transformation should apply to both lhs and rhs:

from django.db.models import Transform
 
class UpperCase(Transform):
    lookup_name = 'upper'
    function = 'UPPER'
    bilateral = True

下一步, 让我们注册它:

from django.db.models import CharField, TextField
CharField.register_lookup(UpperCase)
TextField.register_lookup(UpperCase)

现在, 这个 ``Author.objects.filter(name__upper="doe")``查询集会生成一个像这样的不区分大小写的查询:

SELECT ... WHERE UPPER("author"."name") = UPPER('doe')

为现有的查找编写代替实现

Sometimes different database vendors require different SQL for the sameoperation. For this example we will rewrite a custom implementation forMySQL for the NotEqual operator. Instead of <> we will be using !=operator. (Note that in reality almost all databases support both, includingall the official databases supported by Django).

We can change the behavior on a specific backend by creating a subclass ofNotEqual with an as_mysql method:

class MySQLNotEqual(NotEqual):
    def as_mysql(self, compiler, connection):
        lhs, lhs_params = self.process_lhs(compiler, connection)
        rhs, rhs_params = self.process_rhs(compiler, connection)
        params = lhs_params + rhs_params
        return '%s != %s' % (lhs, rhs), params
 
Field.register_lookup(MySQLNotEqual)

We can then register it with Field. It takes the place of the originalNotEqual class as it has the same lookup_name.

When compiling a query, Django first looks for as_%s % connection.vendormethods, and then falls back to as_sql. The vendor names for the in-builtbackends are sqlite, postgresql, oracle and mysql.

How Django determines the lookups and transforms which are used

In some cases you may wish to dynamically change which Transform orLookup is returned based on the name passed in, rather than fixing it. Asan example, you could have a field which stores coordinates or an arbitrarydimension, and wish to allow a syntax like .filter(coords__x7=4) to returnthe objects where the 7th coordinate has value 4. In order to do this, youwould override get_lookup with something like:

class CoordinatesField(Field):
    def get_lookup(self, lookup_name):
        if lookup_name.startswith('x'):
            try:
                dimension = int(lookup_name[1:])
            except ValueError:
                pass
            else:
                return get_coordinate_lookup(dimension)
        return super().get_lookup(lookup_name)

You would then define get_coordinate_lookup appropriately to return aLookup subclass which handles the relevant value of dimension.

There is a similarly named method called get_transform(). get_lookup()should always return a Lookup subclass, and get_transform() aTransform subclass. It is important to remember that Transformobjects can be further filtered on, and Lookup objects cannot.

When filtering, if there is only one lookup name remaining to be resolved, wewill look for a Lookup. If there are multiple names, it will look for aTransform. In the situation where there is only one name and a Lookupis not found, we look for a Transform and then the exact lookup on thatTransform. All call sequences always end with a Lookup. To clarify:

• .filter(myfield__mylookup) will call myfield.get_lookup('mylookup').
• .filter(myfieldmytransformmylookup) will callmyfield.get_transform('mytransform'), and thenmytransform.get_lookup('mylookup').
• .filter(myfield__mytransform) will first callmyfield.get_lookup('mytransform'), which will fail, so it will fall backto calling myfield.get_transform('mytransform') and thenmytransform.get_lookup('exact').