Function overloading

Sometimes the arguments and types in a function depend on each otherin ways that can’t be captured with a Union. For example, supposewe want to write a function that can accept x-y coordinates. If we passin just a single x-y coordinate, we return a ClickEvent object. However,if we pass in two x-y coordinates, we return a DragEvent object.

Our first attempt at writing this function might look like this:

  1. from typing import Union, Optional
  2.  
  3. def mouse_event(x1: int,
  4.                 y1: int,
  5.                 x2: Optional[int] = None,
  6.                 y2: Optional[int] = None) -> Union[ClickEvent, DragEvent]:
  7.     if x2 is None and y2 is None:
  8.         return ClickEvent(x1, y1)
  9.     elif x2 is not None and y2 is not None:
  10.         return DragEvent(x1, y1, x2, y2)
  11.     else:
  12.         raise TypeError("Bad arguments")

While this function signature works, it’s too loose: it implies mouse_eventcould return either object regardless of the number of argumentswe pass in. It also does not prohibit a caller from passing in the wrongnumber of ints: mypy would treat calls like mouse_event(1, 2, 20) as beingvalid, for example.

We can do better by using overloadingwhich lets us give the same function multiple type annotations (signatures)to more accurately describe the function’s behavior:

  1. from typing import Union, overload
  2.  
  3. # Overload *variants* for 'mouse_event'.
  4. # These variants give extra information to the type checker.
  5. # They are ignored at runtime.
  6.  
  7. @overload
  8. def mouse_event(x1: int, y1: int) -> ClickEvent: ...
  9. @overload
  10. def mouse_event(x1: int, y1: int, x2: int, y2: int) -> DragEvent: ...
  11.  
  12. # The actual *implementation* of 'mouse_event'.
  13. # The implementation contains the actual runtime logic.
  14. #
  15. # It may or may not have type hints. If it does, mypy
  16. # will check the body of the implementation against the
  17. # type hints.
  18. #
  19. # Mypy will also check and make sure the signature is
  20. # consistent with the provided variants.
  21.  
  22. def mouse_event(x1: int,
  23.                 y1: int,
  24.                 x2: Optional[int] = None,
  25.                 y2: Optional[int] = None) -> Union[ClickEvent, DragEvent]:
  26.     if x2 is None and y2 is None:
  27.         return ClickEvent(x1, y1)
  28.     elif x2 is not None and y2 is not None:
  29.         return DragEvent(x1, y1, x2, y2)
  30.     else:
  31.         raise TypeError("Bad arguments")

This allows mypy to understand calls to mouse_event much more precisely.For example, mypy will understand that mouse_event(5, 25) willalways have a return type of ClickEvent and will report errors forcalls like mouse_event(5, 25, 2).

As another example, suppose we want to write a custom container class thatimplements the getitem method ([] bracket indexing). If thismethod receives an integer we return a single item. If it receives aslice, we return a Sequence of items.

We can precisely encode this relationship between the argument and thereturn type by using overloads like so:

  1. from typing import Sequence, TypeVar, Union, overload
  2.  
  3. T = TypeVar('T')
  4.  
  5. class MyList(Sequence[T]):
  6.     @overload
  7.     def __getitem__(self, index: int) -> T: ...
  8.  
  9.     @overload
  10.     def __getitem__(self, index: slice) -> Sequence[T]: ...
  11.  
  12.     def __getitem__(self, index: Union[int, slice]) -> Union[T, Sequence[T]]:
  13.         if isinstance(index, int):
  14.             # Return a T here
  15.         elif isinstance(index, slice):
  16.             # Return a sequence of Ts here
  17.         else:
  18.             raise TypeError(...)

Note

If you just need to constrain a type variable to certain types orsubtypes, you can use a value restriction.

Runtime behavior

An overloaded function must consist of two or more overload variants_followed by an _implementation. The variants and the implementationsmust be adjacent in the code: think of them as one indivisible unit.

The variant bodies must all be empty; only the implementation is allowedto contain code. This is because at runtime, the variants are completelyignored: they’re overridden by the final implementation function.

This means that an overloaded function is still an ordinary Pythonfunction! There is no automatic dispatch handling and you must manuallyhandle the different types in the implementation (e.g. by usingif statements and isinstance checks).

If you are adding an overload within a stub file, the implementationfunction should be omitted: stubs do not contain runtime logic.

Note

While we can leave the variant body empty using the pass keyword,the more common convention is to instead use the ellipsis (…) literal.

Type checking calls to overloads

When you call an overloaded function, mypy will infer the correct returntype by picking the best matching variant, after taking into considerationboth the argument types and arity. However, a call is never typechecked against the implementation. This is why mypy will report callslike mouse_event(5, 25, 3) as being invalid even though it matches theimplementation signature.

If there are multiple equally good matching variants, mypy will selectthe variant that was defined first. For example, consider the followingprogram:

  1. from typing import List, overload
  2.  
  3. @overload
  4. def summarize(data: List[int]) -> float: ...
  5.  
  6. @overload
  7. def summarize(data: List[str]) -> str: ...
  8.  
  9. def summarize(data):
  10.     if not data:
  11.         return 0.0
  12.     elif isinstance(data[0], int):
  13.         # Do int specific code
  14.     else:
  15.         # Do str-specific code
  16.  
  17. # What is the type of 'output'? float or str?
  18. output = summarize([])

The summarize([]) call matches both variants: an empty list couldbe either a List[int] or a List[str]. In this case, mypywill break the tie by picking the first matching variant: outputwill have an inferred type of float. The implementor is responsiblefor making sure summarize breaks ties in the same way at runtime.

However, there are two exceptions to the “pick the first match” rule.First, if multiple variants match due to an argument being of typeAny, mypy will make the inferred type also be Any:

  1. dynamic_var: Any = some_dynamic_function()
  2.  
  3. # output2 is of type 'Any'
  4. output2 = summarize(dynamic_var)

Second, if multiple variants match due to one or more of the argumentsbeing a union, mypy will make the inferred type be the union of thematching variant returns:

  1. some_list: Union[List[int], List[str]]
  2.  
  3. # output3 is of type 'Union[float, str]'
  4. output3 = summarize(some_list)

Note

Due to the “pick the first match” rule, changing the order of youroverload variants can change how mypy type checks your program.

To minimize potential issues, we recommend that you:

  • Make sure your overload variants are listed in the same order asthe runtime checks (e.g. isinstance checks) in your implementation.
  • Order your variants and runtime checks from most to least specific.(See the following section for an example).

Type checking the variants

Mypy will perform several checks on your overload variant definitionsto ensure they behave as expected. First, mypy will check and make surethat no overload variant is shadowing a subsequent one. For example,consider the following function which adds together two Expressionobjects, and contains a special-case to handle receiving two Literaltypes:

  1. from typing import overload, Union
  2.  
  3. class Expression:
  4.     # ...snip...
  5.  
  6. class Literal(Expression):
  7.     # ...snip...
  8.  
  9. # Warning -- the first overload variant shadows the second!
  10.  
  11. @overload
  12. def add(left: Expression, right: Expression) -> Expression: ...
  13.  
  14. @overload
  15. def add(left: Literal, right: Literal) -> Literal: ...
  16.  
  17. def add(left: Expression, right: Expression) -> Expression:
  18.     # ...snip...

While this code snippet is technically type-safe, it does contain ananti-pattern: the second variant will never be selected! If we try callingadd(Literal(3), Literal(4)), mypy will always pick the first variantand evaluate the function call to be of type Expression, not Literal.This is because Literal is a subtype of Expression, which meansthe “pick the first match” rule will always halt after considering thefirst overload.

Because having an overload variant that can never be matched is almostcertainly a mistake, mypy will report an error. To fix the error, we caneither 1) delete the second overload or 2) swap the order of the overloads:

  1. # Everything is ok now -- the variants are correctly ordered
  2. # from most to least specific.
  3.  
  4. @overload
  5. def add(left: Literal, right: Literal) -> Literal: ...
  6.  
  7. @overload
  8. def add(left: Expression, right: Expression) -> Expression: ...
  9.  
  10. def add(left: Expression, right: Expression) -> Expression:
  11.     # ...snip...

Mypy will also type check the different variants and flag any overloadsthat have inherently unsafely overlapping variants. For example, considerthe following unsafe overload definition:

  1. from typing import overload, Union
  2.  
  3. @overload
  4. def unsafe_func(x: int) -> int: ...
  5.  
  6. @overload
  7. def unsafe_func(x: object) -> str: ...
  8.  
  9. def unsafe_func(x: object) -> Union[int, str]:
  10.     if isinstance(x, int):
  11.         return 42
  12.     else:
  13.         return "some string"

On the surface, this function definition appears to be fine. However, it willresult in a discrepancy between the inferred type and the actual runtime typewhen we try using it like so:

  1. some_obj: object = 42
  2. unsafe_func(some_obj) + " danger danger"  # Type checks, yet crashes at runtime!

Since some_obj is of type object, mypy will decide that unsafe_funcmust return something of type str and concludes the above will type check.But in reality, unsafe_func will return an int, causing the code to crashat runtime!

To prevent these kinds of issues, mypy will detect and prohibit inherently unsafelyoverlapping overloads on a best-effort basis. Two variants are considered unsafelyoverlapping when both of the following are true:

  • All of the arguments of the first variant are compatible with the second.
  • The return type of the first variant is not compatible with (e.g. is not asubtype of) the second. So in this example, the int argument in the first variant is a subtype ofthe object argument in the second, yet the int return type is not a subtype ofstr. Both conditions are true, so mypy will correctly flag unsafe_func asbeing unsafe.

However, mypy will not detect all unsafe uses of overloads. For example,suppose we modify the above snippet so it calls summarize instead ofunsafe_func:

  1. some_list: List[str] = []
  2. summarize(some_list) + "danger danger"  # Type safe, yet crashes at runtime!

We run into a similar issue here. This program type checks if we look just at theannotations on the overloads. But since summarize(…) is designed to be biasedtowards returning a float when it receives an empty list, this program will actuallycrash during runtime.

The reason mypy does not flag definitions like summarize as being potentiallyunsafe is because if it did, it would be extremely difficult to write a safeoverload. For example, suppose we define an overload with two variants that accepttypes A and B respectively. Even if those two types were completely unrelated,the user could still potentially trigger a runtime error similar to the ones above bypassing in a value of some third type C that inherits from both A and B.

Thankfully, these types of situations are relatively rare. What this does mean,however, is that you should exercise caution when designing or using an overloadedfunction that can potentially receive values that are an instance of two seeminglyunrelated types.

Type checking the implementation

The body of an implementation is type-checked against thetype hints provided on the implementation. For example, in theMyList example up above, the code in the body is checked withargument list index: Union[int, slice] and a return type ofUnion[T, Sequence[T]]. If there are no annotations on theimplementation, then the body is not type checked. If you want toforce mypy to check the body anyways, use the —check-untyped-defsflag (more details here).

The variants must also also be compatible with the implementationtype hints. In the MyList example, mypy will check that theparameter type int and the return type T are compatible withUnion[int, slice] and Union[T, Sequence] for thefirst variant. For the second variant it verifies the parametertype slice and the return type Sequence[T] are compatiblewith Union[int, slice] and Union[T, Sequence].

Note

The overload semantics documented above are new as of mypy 0.620.

Previously, mypy used to perform type erasure on all overload variants. Forexample, the summarize example from the previous section used to beillegal because List[str] and List[int] both erased to just List[Any].This restriction was removed in mypy 0.620.

Mypy also previously used to select the best matching variant using a differentalgorithm. If this algorithm failed to find a match, it would default to returningAny. The new algorithm uses the “pick the first match” rule and will fall backto returning Any only if the input arguments also contain Any.