Operator Reference

This section details usage of the operators that are available to construct SQL expressions.

These methods are presented in terms of the Operators and ColumnOperators base classes. The methods are then available on descendants of these classes, including:

  • Column objects

  • ColumnElement objects more generally, which are the root of all Core SQL Expression language column-level expressions

  • InstrumentedAttribute objects, which are ORM level mapped attributes.

The operators are first introduced in the tutorial sections, including:

Comparison Operators

Basic comparisons which apply to many datatypes, including numerics, strings, dates, and many others:

IN Comparisons

The SQL IN operator is a subject all its own in SQLAlchemy. As the IN operator is usually used against a list of fixed values, SQLAlchemy’s feature of bound parameter coercion makes use of a special form of SQL compilation that renders an interim SQL string for compilation that’s formed into the final list of bound parameters in a second step. In other words, “it just works”.

IN against a list of values

IN is available most typically by passing a list of values to the ColumnOperators.in_() method:

  1. >>> print(column('x').in_([1, 2, 3]))
  2. x IN ([POSTCOMPILE_x_1])

The special bound form POSTCOMPILE is rendered into individual parameters at execution time, illustrated below:

  1. >>> stmt = select(User.id).where(User.id.in_([1, 2, 3]))
  2. >>> result = conn.execute(stmt)
  3. SELECT user_account.id
  4. FROM user_account
  5. WHERE user_account.id IN (?, ?, ?)
  6. [...] (1, 2, 3)

Empty IN Expressions

SQLAlchemy produces a mathematically valid result for an empty IN expression by rendering a backend-specific subquery that returns no rows. Again in other words, “it just works”:

  1. >>> stmt = select(User.id).where(User.id.in_([]))
  2. >>> result = conn.execute(stmt)
  3. SELECT user_account.id
  4. FROM user_account
  5. WHERE user_account.id IN (SELECT 1 FROM (SELECT 1) WHERE 1!=1)
  6. [...] ()

The “empty set” subquery above generalizes correctly and is also rendered in terms of the IN operator which remains in place.

NOT IN

“NOT IN” is available via the ColumnOperators.not_in() operator:

  1. >>> print(column('x').not_in([1, 2, 3]))
  2. x NOT IN ([POSTCOMPILE_x_1])

This is typically more easily available by negating with the ~ operator:

  1. >>> print(~column('x').in_([1, 2, 3]))
  2. x NOT IN ([POSTCOMPILE_x_1])

Tuple IN Expressions

Comparison of tuples to tuples is common with IN, as among other use cases accommodates for the case when matching rows to a set of potential composite primary key values. The tuple_() construct provides the basic building block for tuple comparisons. The Tuple.in_() operator then receives a list of tuples:

  1. >>> from sqlalchemy import tuple_
  2. >>> tup = tuple_(column('x', Integer), column('y', Integer))
  3. >>> expr = tup.in_([(1, 2), (3, 4)])
  4. >>> print(expr)
  5. (x, y) IN ([POSTCOMPILE_param_1])

To illustrate the parameters rendered:

  1. >>> tup = tuple_(User.id, Address.id)
  2. >>> stmt = select(User.name).join(Address).where(tup.in_([(1, 1), (2, 2)]))
  3. >>> conn.execute(stmt).all()
  4. SELECT user_account.name
  5. FROM user_account JOIN address ON user_account.id = address.user_id
  6. WHERE (user_account.id, address.id) IN (VALUES (?, ?), (?, ?))
  7. [...] (1, 1, 2, 2)
  8. [('spongebob',), ('sandy',)]

Subquery IN

Finally, the ColumnOperators.in_() and ColumnOperators.not_in() operators work with subqueries. The form provides that a Select construct is passed in directly, without any explicit conversion to a named subquery:

  1. >>> print(column('x').in_(select(user_table.c.id)))
  2. x IN (SELECT user_account.id
  3. FROM user_account)

Tuples work as expected:

  1. >>> print(
  2. ...     tuple_(column('x'), column('y')).in_(
  3. ...         select(user_table.c.id, address_table.c.id).join(address_table)
  4. ...     )
  5. ... )
  6. (x, y) IN (SELECT user_account.id, address.id
  7. FROM user_account JOIN address ON user_account.id = address.user_id)

Identity Comparisons

These operators involve testing for special SQL values such as NULL, boolean constants such as true or false which some databases support:

  • ColumnOperators.is_():

    This operator will provide exactly the SQL for “x IS y”, most often seen as “<expr> IS NULL”. The NULL constant is most easily acquired using regular Python None:

    1. >>> print(column('x').is_(None))
    2. x IS NULL

    SQL NULL is also explicitly available, if needed, using the null() construct:

    1. >>> from sqlalchemy import null
    2. >>> print(column('x').is_(null()))
    3. x IS NULL

    The ColumnOperators.is_() operator is automatically invoked when using the ColumnOperators.__eq__() overloaded operator, i.e. ==, in conjunction with the None or null() value. In this way, there’s typically not a need to use ColumnOperators.is_() explicitly, paricularly when used with a dynamic value:

    1. >>> a = None
    2. >>> print(column('x') == a)
    3. x IS NULL

    Note that the Python is operator is not overloaded. Even though Python provides hooks to overload operators such as == and !=, it does not provide any way to redefine is.

  • ColumnOperators.is_not():

    Similar to ColumnOperators.is_(), produces “IS NOT”:

    1. >>> print(column('x').is_not(None))
    2. x IS NOT NULL

    Is similarly equivalent to != None:

    1. >>> print(column('x') != None)
    2. x IS NOT NULL

  • ColumnOperators.is_distinct_from():

    Produces SQL IS DISTINCT FROM:

    1. >>> print(column('x').is_distinct_from('some value'))
    2. x IS DISTINCT FROM :x_1

  • ColumnOperators.isnot_distinct_from():

    Produces SQL IS NOT DISTINCT FROM:

    1. >>> print(column('x').isnot_distinct_from('some value'))
    2. x IS NOT DISTINCT FROM :x_1

String Comparisons

String Containment

String containment operators are basically built as a combination of LIKE and the string concatenation operator, which is || on most backends or sometimes a function like concat():

String matching

Matching operators are always backend-specific and may provide different behaviors and results on different databases:

  • ColumnOperators.match():

    This is a dialect-specific operator that makes use of the MATCH feature of the underlying database, if available:

    1. >>> print(column('x').match('word'))
    2. x MATCH :x_1

  • ColumnOperators.regexp_match():

    This operator is dialect specific. We can illustrate it in terms of for example the PostgreSQL dialect:

    1. >>> from sqlalchemy.dialects import postgresql
    2. >>> print(column('x').regexp_match('word').compile(dialect=postgresql.dialect()))
    3. x ~ %(x_1)s

    Or MySQL:

    1. >>> from sqlalchemy.dialects import mysql
    2. >>> print(column('x').regexp_match('word').compile(dialect=mysql.dialect()))
    3. x REGEXP %s

String Alteration

  • ColumnOperators.concat():

    String concatenation:

    1. >>> print(column('x').concat("some string"))
    2. x || :x_1

    This operator is available via ColumnOperators.__add__(), that is, the Python + operator, when working with a column expression that derives from String:

    1. >>> print(column('x', String) + "some string")
    2. x || :x_1

    The operator will produce the appropriate database-specific construct, such as on MySQL it’s historically been the concat() SQL function:

    1. >>> print((column('x', String) + "some string").compile(dialect=mysql.dialect()))
    2. concat(x, %s)

  • ColumnOperators.regexp_replace():

    Complementary to ColumnOperators.regexp() this produces REGEXP REPLACE equivalent for the backends which support it:

    1. >>> print(column('x').regexp_replace('foo', 'bar').compile(dialect=postgresql.dialect()))
    2. REGEXP_REPLACE(x, %(x_1)s, %(x_2)s)

  • ColumnOperators.collate():

    Produces the COLLATE SQL operator which provides for specific collations at expression time:

    1. >>> print((column('x').collate('latin1_german2_ci') == 'Müller').compile(dialect=mysql.dialect()))
    2. (x COLLATE latin1_german2_ci) = %s

    To use COLLATE against a literal value, use the literal() construct:

    1. >>> from sqlalchemy import literal
    2. >>> print((literal('Müller').collate('latin1_german2_ci') == column('x')).compile(dialect=mysql.dialect()))
    3. (%s COLLATE latin1_german2_ci) = x

Arithmetic Operators

Using Conjunctions and Negations

The most common conjunction, “AND”, is automatically applied if we make repeated use of the Select.where() method, as well as similar methods such as Update.where() and Delete.where():

  1. >>> print(
  2. ...        select(address_table.c.email_address).
  3. ...        where(user_table.c.name == 'squidward').
  4. ...        where(address_table.c.user_id == user_table.c.id)
  5. ...    )
  6. SELECT address.email_address
  7. FROM address, user_account
  8. WHERE user_account.name = :name_1 AND address.user_id = user_account.id

Select.where(), Update.where() and Delete.where() also accept multiple expressions with the same effect:

  1. >>> print(
  2. ...        select(address_table.c.email_address).
  3. ...        where(
  4. ...            user_table.c.name == 'squidward',
  5. ...            address_table.c.user_id == user_table.c.id
  6. ...        )
  7. ...    )
  8. SELECT address.email_address
  9. FROM address, user_account
  10. WHERE user_account.name = :name_1 AND address.user_id = user_account.id

The “AND” conjunction, as well as its partner “OR”, are both available directly using the and_() and or_() functions:

  1. >>> from sqlalchemy import and_, or_
  2. >>> print(
  3. ...     select(address_table.c.email_address).
  4. ...     where(
  5. ...         and_(
  6. ...             or_(user_table.c.name == 'squidward', user_table.c.name == 'sandy'),
  7. ...             address_table.c.user_id == user_table.c.id
  8. ...         )
  9. ...     )
  10. ... )
  11. SELECT address.email_address
  12. FROM address, user_account
  13. WHERE (user_account.name = :name_1 OR user_account.name = :name_2)
  14. AND address.user_id = user_account.id

A negation is available using the not_() function. This will typically invert the operator in a boolean expression:

  1. >>> from sqlalchemy import not_
  2. >>> print(not_(column('x') == 5))
  3. x != :x_1

It also may apply a keyword such as NOT when appropriate:

  1. >>> from sqlalchemy import Boolean
  2. >>> print(not_(column('x', Boolean)))
  3. NOT x

Conjunction Operators

The above conjunction functions and_(), or_(), not_() are also available as overloaded Python operators:

Note

The Python &, | and ~ operators take high precedence in the language; as a result, parenthesis must usually be applied for operands that themselves contain expressions, as indicated in the examples below.

  • Operators.__and__() (Python “&” operator):

    The Python binary & operator is overloaded to behave the same as and_() (note parenthesis around the two operands):

    1. >>> print((column('x') == 5) & (column('y') == 10))
    2. x = :x_1 AND y = :y_1

  • Operators.__or__() (Python “|” operator):

    The Python binary | operator is overloaded to behave the same as or_() (note parenthesis around the two operands):

    1. >>> print((column('x') == 5) | (column('y') == 10))
    2. x = :x_1 OR y = :y_1

  • Operators.__invert__() (Python “~” operator):

    The Python binary ~ operator is overloaded to behave the same as not_(), either inverting the existing operator, or applying the NOT keyword to the expression as a whole:

    1. >>> print(~(column('x') == 5))
    2. x != :x_1
    4. >>> from sqlalchemy import Boolean
    5. >>> print(~column('x', Boolean))
    6. NOT x

Operator Customization

TODO