Glossary

  • Apply
  • Instances of Apply represent the application of an Opto some input Variable (or variables) to produce some outputVariable (or variables). They are like the application of a [symbolic]mathematical function to some [symbolic] inputs.
  • Broadcasting
  • Broadcasting is a mechanism which allows tensors withdifferent numbers of dimensions to be used in element-by-element(elementwise) computations. It works by(virtually) replicating the smaller tensor alongthe dimensions that it is lacking.

For more detail, see Broadcasting, and also SciPy documentation about numpy’s broadcasting OnLamp article about numpy’s broadcasting

  • Constant
  • A variable with an immutable value.For example, when you type
  1. >>> x = tensor.ivector()
  2. >>> y = x + 3

Then a constant is created to represent the 3 in the graph.

See also: gof.Constant

  • Elementwise
  • An elementwise operation f on two tensor variables M and Nis one such that:

f(M, N)[i, j] == f(M[i, j], N[i, j])

In other words, each element of an input matrix is combinedwith the corresponding element of the other(s). There are nodependencies between elements whose [i, j] coordinates donot correspond, so an elementwise operation is like a scalaroperation generalized along several dimensions. Elementwiseoperations are defined for tensors of different numbers of dimensions bybroadcasting the smaller ones.

  • Expression
  • See Apply
  • Expression Graph
  • A directed, acyclic set of connected Variable andApply nodes that express symbolic functional relationshipbetween variables. You use Theano by defining expression graphs, andthen compiling them with theano.function.

See also Variable, Op, Apply, andType, or read more about Graph Structures.

  • Destructive
  • An Op is destructive (of particular input[s]) if itscomputation requires that one or more inputs be overwritten orotherwise invalidated. For example, inplace Ops aredestructive. Destructive Ops can sometimes be faster thannon-destructive alternatives. Theano encourages users not to putdestructive Ops into graphs that are given to theano.function,but instead to trust the optimizations to insert destructive opsjudiciously.

Destructive Ops are indicated via a destroy_map Op attribute. (Seegof.Op.

  • Graph
  • see expression graph
  • Inplace
  • Inplace computations are computations that destroy their inputs as aside-effect. For example, if you iterate over a matrix and doubleevery element, this is an inplace operation because when you are done,the original input has been overwritten. Ops representing inplacecomputations are destructive, and by default these can only beinserted by optimizations, not user code.
  • Linker
  • Part of a function Mode – an object responsible for ‘running’the compiled function. Among other things, the linker determines whether computations are carried out with C or Python code.
  • Mode
  • An object providing an optimizer and a linker that ispassed to theano.function. It parametrizes how an expressiongraph is converted to a callable object.
  • Op
  • The .op of an Apply, together with its symbolic inputsfully determines what kind of computation will be carried out for thatApply at run-time. Mathematical functions such as addition(T.add) and indexing x[i] are Ops in Theano. Much of thelibrary documentation is devoted to describing the various Ops thatare provided with Theano, but you can add more.

See also Variable, Type, and Apply,or read more about Graph Structures.

  • Optimizer
  • An instance of Optimizer, which has the capacity to providean optimization (or optimizations).
  • Optimization
  • A graph transformation applied by an optimizer duringthe compilation of a graph by theano.function.
  • Pure
  • An Op is pure if it has no destructive side-effects.
  • Storage
  • The memory that is used to store the value of a Variable. In mostcases storage is internal to a compiled function, but in some cases(such as constant and shared variable the storage is not internal.
  • Shared Variable
  • A Variable whose value may be shared between multiple functions. See shared and theano.function.
  • theano.function
  • The interface for Theano’s compilation from symbolic expression graphsto callable objects. See function.function().
  • Type
  • The .type of aVariable indicates what kinds of values might be computed for it in acompiled graph.An instance that inherits from Type, and is used as the.type attribute of a Variable.

See also Variable, Op, and Apply,or read more about Graph Structures.

  • Variable
  • The the main data structure you work with when using Theano.For example,
  1. >>> x = theano.tensor.ivector()
  2. >>> y = -x**2

x and y are both Variables, i.e. instances of the Variable class.

See also Type, Op, and Apply,or read more about Graph Structures.

  • View
  • Some Tensor Ops (such as Subtensor and Transpose) can be computed inconstant time by simply re-indexing their inputs. The outputs from[the Apply instances from] such Ops are called Views because theirstorage might be aliased to the storage of other variables (the inputsof the Apply). It is important for Theano to know which Variables areviews of which other ones in order to introduce DestructiveOps correctly.

View Ops are indicated via a view_map Op attribute. (Seegof.Op.