Tutorial

Let us start an interactive session (e.g. with python or ipython) and import Theano.

>>> from theano import *

Several of the symbols you will need to use are in the tensor subpackageof Theano. Let us import that subpackage under a handy name likeT (the tutorials will frequently use this convention).

>>> import theano.tensor as T

If that succeeded you are ready for the tutorial, otherwise check yourinstallation (see Installing Theano).

Throughout the tutorial, bear in mind that there is a Glossary as wellas index and modules links in the upper-right corner of each page to helpyou out.

Prerequisites

• Python tutorial
• NumPy refresher
  • Matrix conventions for machine learning
  • Broadcasting

Basics

• Baby Steps - Algebra
  • Adding two Scalars
  • Adding two Matrices
  • Exercise
• More Examples
  • Logistic Function
  • Computing More than one Thing at the Same Time
  • Setting a Default Value for an Argument
  • Using Shared Variables
  • Copying functions
  • Using Random Numbers
    • Brief Example
    • Seeding Streams
    • Sharing Streams Between Functions
    • Copying Random State Between Theano Graphs
    • Other Random Distributions
    • Other Implementations
  • A Real Example: Logistic Regression
• Derivatives in Theano
  • Computing Gradients
  • Computing the Jacobian
  • Computing the Hessian
  • Jacobian times a Vector
    • R-operator
    • L-operator
  • Hessian times a Vector
  • Final Pointers
• Conditions
  • IfElse vs Switch
• Loop
  • Scan
  • Exercise
• How Shape Information is Handled by Theano
  • Shape Inference Problem
  • Specifing Exact Shape
  • Future Plans
• Broadcasting

Advanced

• Sparse
  • Compressed Sparse Format
    • Which format should I use?
  • Handling Sparse in Theano
    • To and Fro
    • Properties and Construction
    • Structured Operation
    • Gradient
• Using the GPU
  • GpuArray Backend
    • Testing Theano with GPU
    • Returning a Handle to Device-Allocated Data
    • What Can be Accelerated on the GPU
    • Tips for Improving Performance on GPU
    • GPU Async Capabilities
    • Changing the Value of Shared Variables
    • Exercise
  • CUDA backend
  • Software for Directly Programming a GPU
  • Learning to Program with PyCUDA
    • Exercise
    • Exercise
  • Note
• Using multiple GPUs
  • Defining the context map
  • A simple graph on two GPUs
  • Explicit transfers of data
• Convolution arithmetic tutorial
  • About this tutorial
  • Refresher: discrete convolutions
  • Theano terminology
  • Convolution arithmetic
    • No zero padding, unit strides
    • Zero padding, unit strides
    • Special cases
      • Half (same) padding
      • Full padding
    • No zero padding, non-unit strides
    • Zero padding, non-unit strides
  • Transposed convolution arithmetic
    • Convolution as a matrix operation
    • Transposed convolution
    • No zero padding, unit strides, transposed
    • Zero padding, unit strides, transposed
    • Special cases
      • Half (same) padding, transposed
      • Full padding, transposed
    • No zero padding, non-unit strides, transposed
    • Zero padding, non-unit strides, transposed
  • Miscellaneous convolutions
    • Dilated convolutions
  • Quick reference

Advanced configuration and debugging

• Configuration Settings and Compiling Modes
  • Configuration
  • Exercise
  • Mode
  • Linkers
  • Using DebugMode
• Printing/Drawing Theano graphs
  • Pretty Printing
  • Debug Print
  • Picture Printing of Graphs
  • Interactive Graph Visualization
• Debugging Theano: FAQ and Troubleshooting
  • Isolating the Problem/Testing Theano Compiler
  • Interpreting Error Messages
  • Using Test Values
  • “How do I Print an Intermediate Value in a Function?”
  • “How do I Print a Graph?” (before or after compilation)
  • “The Function I Compiled is Too Slow, what’s up?”
  • “Why does my GPU function seem to be slow?”
  • “How do I Step through a Compiled Function?”
  • How to Use pdb
  • Dumping a Function to help debug
  • Breakpoint during Theano function execution
• Dealing with NaNs
  • Check Superparameters and Weight Initialization
  • Run in NanGuardMode, DebugMode, or MonitorMode
  • Numerical Stability
  • Algorithm Related
  • Cuda Specific Option
  • NaN Introduced by AllocEmpty
• Profiling Theano function

Further readings

• Graph Structures
  • Traversing the graph
  • Graph Structures
    • Apply
    • Op
    • Type
    • Variable
      • Constant
  • Graph Structures Extension
  • Automatic Differentiation
  • Optimizations
• Loading and Saving
  • The Basics of Pickling
  • Short-Term Serialization
  • Robust Serialization
  • Long-Term Serialization
• PyCUDA/CUDAMat/Gnumpy compatibility
  • PyCUDA
    • Transfer
    • Compiling with PyCUDA
    • Theano Op using a PyCUDA function
  • CUDAMat
  • Gnumpy
• Understanding Memory Aliasing for Speed and Correctness
  • The Memory Model: Two Spaces
  • Borrowing when Creating Shared Variables
  • Borrowing when Accessing Value of Shared Variables
    • Retrieving
    • Assigning
  • Borrowing when Constructing Function Objects
• Python Memory Management
  • Basic Objects
  • Internal Memory Management
  • Pickle
• Multi cores support in Theano
  • Convolution and Pooling
  • BLAS operation
  • Parallel element wise ops with OpenMP
• Frequently Asked Questions
  • How to update a subset of weights?