Conditions

IfElse vs Switch

  • Both ops build a condition over symbolic variables.
  • IfElse takes a boolean condition and two variables as inputs.
  • Switch takes a tensor as condition and two variables as inputs.switch is an elementwise operation and is thus more general than ifelse.
  • Whereas switch evaluates both output variables, ifelse is lazy and onlyevaluates one variable with respect to the condition.

Example

  1. from theano import tensor as T
  2. from theano.ifelse import ifelse
  3. import theano, time, numpy
  4.  
  5. a,b = T.scalars('a', 'b')
  6. x,y = T.matrices('x', 'y')
  7.  
  8. z_switch = T.switch(T.lt(a, b), T.mean(x), T.mean(y))
  9. z_lazy = ifelse(T.lt(a, b), T.mean(x), T.mean(y))
  10.  
  11. f_switch = theano.function([a, b, x, y], z_switch,
  12.                            mode=theano.Mode(linker='vm'))
  13. f_lazyifelse = theano.function([a, b, x, y], z_lazy,
  14.                                mode=theano.Mode(linker='vm'))
  15.  
  16. val1 = 0.
  17. val2 = 1.
  18. big_mat1 = numpy.ones((10000, 1000))
  19. big_mat2 = numpy.ones((10000, 1000))
  20.  
  21. n_times = 10
  22.  
  23. tic = time.clock()
  24. for i in range(n_times):
  25.     f_switch(val1, val2, big_mat1, big_mat2)
  26. print('time spent evaluating both values %f sec' % (time.clock() - tic))
  27.  
  28. tic = time.clock()
  29. for i in range(n_times):
  30.     f_lazyifelse(val1, val2, big_mat1, big_mat2)
  31. print('time spent evaluating one value %f sec' % (time.clock() - tic))

In this example, the IfElse op spends less time (about half as much) than Switchsince it computes only one variable out of the two.

  1. $ python ifelse_switch.py
  2. time spent evaluating both values 0.6700 sec
  3. time spent evaluating one value 0.3500 sec

Unless linker='vm' or linker='cvm' are used, ifelse will compute bothvariables and take the same computation time as switch. Although the linkeris not currently set by default to cvm, it will be in the near future.

There is no automatic optimization replacing a switch with abroadcasted scalar to an ifelse, as this is not always faster. Seethis ticket.

Note

If you use test values, then all branches ofthe IfElse will be computed. This is normal, as using test_valuemeans everything will be computed when we build it, due to Python’sgreedy evaluation and the semantic of test value. As we build bothbranches, they will be executed for test values. This doesn’t causeany changes during the execution of the compiled Theano function.