Extension types (aka. cdef classes)

To support object-oriented programming, Cython supports writing normalPython classes exactly as in Python:

  1. class MathFunction(object):
  2. def __init__(self, name, operator):
  3. self.name = name
  4. self.operator = operator
  5.  
  6. def __call__(self, *operands):
  7. return self.operator(*operands)

Based on what Python calls a “built-in type”, however, Cython supportsa second kind of class: extension types, sometimes referred to as“cdef classes” due to the keywords used for their declaration. Theyare somewhat restricted compared to Python classes, but are generallymore memory efficient and faster than generic Python classes. Themain difference is that they use a C struct to store their fields and methodsinstead of a Python dict. This allows them to store arbitrary C typesin their fields without requiring a Python wrapper for them, and toaccess fields and methods directly at the C level without passingthrough a Python dictionary lookup.

Normal Python classes can inherit from cdef classes, but not the otherway around. Cython requires to know the complete inheritancehierarchy in order to lay out their C structs, and restricts it tosingle inheritance. Normal Python classes, on the other hand, caninherit from any number of Python classes and extension types, both inCython code and pure Python code.

So far our integration example has not been very useful as it onlyintegrates a single hard-coded function. In order to remedy this,with hardly sacrificing speed, we will use a cdef class to represent afunction on floating point numbers:

  1. cdef class Function:
  2. cpdef double evaluate(self, double x) except *:
  3. return 0

The directive cpdef makes two versions of the method available; onefast for use from Cython and one slower for use from Python. Then:

  1. from libc.math cimport sin
  2.  
  3. cdef class Function:
  4. cpdef double evaluate(self, double x) except *:
  5. return 0
  6.  
  7. cdef class SinOfSquareFunction(Function):
  8. cpdef double evaluate(self, double x) except *:
  9. return sin(x ** 2)

This does slightly more than providing a python wrapper for a cdefmethod: unlike a cdef method, a cpdef method is fully overridable bymethods and instance attributes in Python subclasses. It adds alittle calling overhead compared to a cdef method.

To make the class definitions visible to other modules, and thus allow forefficient C-level usage and inheritance outside of the module thatimplements them, we define them in a sin_of_square.pxd file:

  1. cdef class Function:
  2. cpdef double evaluate(self, double x) except *
  3.  
  4. cdef class SinOfSquareFunction(Function):
  5. cpdef double evaluate(self, double x) except *

Using this, we can now change our integration example:

  1. from sin_of_square cimport Function, SinOfSquareFunction
  2.  
  3. def integrate(Function f, double a, double b, int N):
  4. cdef int i
  5. cdef double s, dx
  6. if f is None:
  7. raise ValueError("f cannot be None")
  8. s = 0
  9. dx = (b - a) / N
  10. for i in range(N):
  11. s += f.evaluate(a + i * dx)
  12. return s * dx
  13.  
  14. print(integrate(SinOfSquareFunction(), 0, 1, 10000))

This is almost as fast as the previous code, however it is much more flexibleas the function to integrate can be changed. We can even pass in a newfunction defined in Python-space:

  1. >>> import integrate
  2. >>> class MyPolynomial(integrate.Function):
  3. ... def evaluate(self, x):
  4. ... return 2*x*x + 3*x - 10
  5. ...
  6. >>> integrate(MyPolynomial(), 0, 1, 10000)
  7. -7.8335833300000077

This is about 20 times slower, but still about 10 times faster thanthe original Python-only integration code. This shows how large thespeed-ups can easily be when whole loops are moved from Python codeinto a Cython module.

Some notes on our new implementation of evaluate:



  • The fast method dispatch here only works because evaluate was
    declared in Function. Had evaluate been introduced in
    SinOfSquareFunction, the code would still work, but Cython
    would have used the slower Python method dispatch mechanism
    instead.

  • In the same way, had the argument f not been typed, but only
    been passed as a Python object, the slower Python dispatch would
    be used.

  • Since the argument is typed, we need to check whether it is
    None. In Python, this would have resulted in an AttributeError
    when the evaluate method was looked up, but Cython would instead
    try to access the (incompatible) internal structure of None as if
    it were a Function, leading to a crash or data corruption.


There is a compiler directive nonecheck which turns on checksfor this, at the cost of decreased speed. Here’s how compiler directivesare used to dynamically switch on or off nonecheck:

  1. # cython: nonecheck=True
  2. # ^^^ Turns on nonecheck globally
  3.  
  4. import cython
  5.  
  6. cdef class MyClass:
  7. pass
  8.  
  9. # Turn off nonecheck locally for the function
  10. @cython.nonecheck(False)
  11. def func():
  12. cdef MyClass obj = None
  13. try:
  14. # Turn nonecheck on again for a block
  15. with cython.nonecheck(True):
  16. print(obj.myfunc()) # Raises exception
  17. except AttributeError:
  18. pass
  19. print(obj.myfunc()) # Hope for a crash!

Attributes in cdef classes behave differently from attributes in regular classes:



  • All attributes must be pre-declared at compile-time

  • Attributes are by default only accessible from Cython (typed access)

  • Properties can be declared to expose dynamic attributes to Python-space


  1. from sin_of_square cimport Function
  2.  
  3. cdef class WaveFunction(Function):
  4.  
  5. # Not available in Python-space:
  6. cdef double offset
  7.  
  8. # Available in Python-space:
  9. cdef public double freq
  10.  
  11. # Available in Python-space, but only for reading:
  12. cdef readonly double scale
  13.  
  14. # Available in Python-space:
  15. @property
  16. def period(self):
  17. return 1.0 / self.freq
  18.  
  19. @period.setter
  20. def period(self, value):
  21. self.freq = 1.0 / value