Views and inplace operations

Theano allows the definition of Ops which return a view on oneof their inputs or operate inplace on one or severalinputs. This allows more efficient operations on numpy’s ndarraydata type than would be possible otherwise.However, in order to work correctly, these Ops need toimplement an additional interface.

Theano recognizes views and inplace operations specially. It ensuresthat they are used in a consistent manner and it ensures thatoperations will be carried in a compatible order.

An unfortunate fact is that it is impossible to return a view on aninput with the double type or to operate inplace on it (Pythonfloats are immutable). Therefore, we can’t make examples of theseconcepts out of what we’ve just built. Nonetheless, we will presentthe concepts:

Views

A “view” on an object x is an object y which shares memorywith x in some way. In other words, changing x might alsochange y and vice versa. For example, imagine a vector structurewhich contains two fields: an integer length and a pointer to a memorybuffer. Suppose we have:

  1. x = vector {length: 256,
  2.             address: 0xDEADBEEF}
  3.  
  4. y = vector {length: 224,
  5.             address: 0xDEADBEEF + 0x10}
  6.  
  7. z = vector {length: 256,
  8.             address: 0xCAFEBABE}

So x uses the memory range 0xDEADBEEF - 0xDEADBFEF, y therange 0xDEADBEFF - 0xDEADBFDF and z the range 0xCAFEBABE - 0xCAFEBBBE. Since the ranges for x and y overlap, y isconsidered to be a view of x and vice versa.

Suppose you had an Op which took x as input and returnedy. You would need to tell Theano that y is a view of x. For thispurpose, you would set the view_map field as follows:

  1. myop.view_map = {0: [0]}

What this means is that the first output (position 0) is a view of thefirst input (position 0). Even though the interface allows a list ofinputs that are viewed by a given output, this feature is currentlyunsupported. Here are more examples:

  1. myop.view_map = {0: [0]} # first output is a view of first input
  2. myop.view_map = {0: [1]} # first output is a view of second input
  3. myop.view_map = {1: [0]} # second output is a view of first input
  4.  
  5. myop.view_map = {0: [0], # first output is a view of first input
  6.                  1: [1]} # *AND* second output is a view of second input
  7.  
  8. myop.view_map = {0: [0], # first output is a view of first input
  9.                  1: [0]} # *AND* second output is *ALSO* a view of first input
  10.  
  11. myop.view_map = {0: [0, 1]} # THIS IS NOT SUPPORTED YET! Only put a single input number in the list!

Inplace operations

An inplace operation is one that modifies one or more of itsinputs. For example, the expression x += y where x and yare numpy.ndarray instances would normally represent an inplaceoperation on x.

Note

Inplace operations in Theano still work in a functional setting:they need to return the modified input. Symbolically, Theanorequires one Variable standing for the input before being modifiedand another Variable representing the input after beingmodified. Therefore, code using inplace operations would look likethis:

  1. from theano.tensor import dscalars, log
  2. from theano.tensor.inplace import add_inplace
  3.  
  4. x, y = dscalars('x', 'y')
  5. r1 = log(x)
  6.  
  7. # r2 is x AFTER the add_inplace - x still represents the value before adding y
  8. r2 = add_inplace(x, y)
  9.  
  10. # r3 is log(x) using the x from BEFORE the add_inplace
  11. # r3 is the SAME as r1, even if we wrote this line after the add_inplace line
  12. # Theano is actually going to compute r3 BEFORE r2
  13. r3 = log(x)
  14.  
  15. # this is log(x) using the x from AFTER the add_inplace (so it's like log(x + y))
  16. r4 = log(r2)

Needless to say, this goes for user-defined inplace operations aswell: the modified input must figure in the list of outputs yougive to Apply in the definition of make_node.

Also, for technical reasons but also because they are slightlyconfusing to use as evidenced by the previous code, Theano does notallow the end user to use inplace operations by default. However,it does allow optimizations to substitute them in in a laterphase. Therefore, typically, if you define an inplace operation,you will define a pure equivalent and an optimization whichsubsitutes one for the other. Theano will automatically verify ifit is possible to do so and will refuse the substitution if itintroduces inconsistencies.

Take the previous definitions of x, y and z and suppose an Op whichadds one to every byte of its input. If we give x as an input tothat Op, it can either allocate a new buffer of the same size as x(that could be z) and set that new buffer’s bytes to the variable ofthe addition. That would be a normal, pure Op. Alternatively,it could add one to each byte in the buffer x, thereforechanging it. That would be an inplace Op.

Theano needs to be notified of this fact. The syntax is similar tothat of view_map:

  1. myop.destroy_map = {0: [0]}

What this means is that the first output (position 0) operates inplace on thefirst input (position 0).

  1. myop.destroy_map = {0: [0]} # first output operates inplace on first input
  2. myop.destroy_map = {0: [1]} # first output operates inplace on second input
  3. myop.destroy_map = {1: [0]} # second output operates inplace on first input
  4.  
  5. myop.destroy_map = {0: [0], # first output operates inplace on first input
  6.                     1: [1]} # *AND* second output operates inplace on second input
  7.  
  8. myop.destroy_map = {0: [0], # first output operates inplace on first input
  9.                     1: [0]} # *AND* second output *ALSO* operates inplace on first input
  10.  
  11. myop.destroy_map = {0: [0, 1]} # first output operates inplace on both the first and second input
  12. # unlike for views, the previous line is legal and supported

Destructive Operations

While some operations will operate inplace on their inputs, some mightsimply destroy or corrupt them. For example, an Op could do temporarycalculations right in its inputs. If that is the case, Theano alsoneeds to be notified. The way to notify Theano is to assume that someoutput operated inplace on whatever inputs are changed or corrupted bythe Op (even if the output does not technically reuse any of theinput(s)’s memory). From there, go to the previous section.

Warning

Failure to correctly mark down views and inplace operations usingviewmap and destroy_map can lead to nasty bugs. In theabsence of this information, Theano might assume that it is safe toexecute an inplace operation on some inputs _before doing othercalculations on the previous values of the inputs. For example,in the code: y = log(x); x2 = add_inplace(x, z) it isimperative to do the logarithm before the addition (because afterthe addition, the original x that we wanted to take the logarithmof is gone). If Theano does not know that add_inplace changesthe value of x it might invert the order and that willcertainly lead to erroneous computations.

You can often identify an incorrect viewmap or destroy_mapby using debugmode. _Be sure to use DebugMode when developinga new Op that uses view_map and/or destroy_map.

Inplace optimization and DebugMode

It is recommended that during the graph construction, all Ops are not inplace.Then an optimization replaces them with inplace ones. Currently DebugMode checksall optimizations that were tried even if they got rejected. One reason an inplaceoptimization can get rejected is when there is another Op that is already being appliedinplace on the same input. Another reason to reject an inplace optimization isif it would introduce a cycle into the graph.

The problem with DebugMode is that it will trigger a useless error whenchecking a rejected inplace optimization, since it will lead to wrong results.In order to be able to use DebugMode in more situations, your inplaceoptimization can pre-check whether it will get rejected by using thetheano.gof.destroyhandler.fast_inplace_check() function, that will tellwhich Ops can be performed inplace. You may then skip the optimization if it isincompatible with this check. Note however that this check does not cover allcases where an optimization may be rejected (it will not detect cycles).