Release Notes

Theano 0.9.0 (20th of March, 2017)

This is a final release of Theano, version 0.9.0, with a lot ofnew features, interface changes, improvements and bug fixes.

We recommend that everybody update to this version.

  • Highlights (since 0.8.0):
    • Better Python 3.5 support
    • Better numpy 1.12 support
    • Conda packages for Mac, Linux and Windows
    • Support newer Mac and Windows versions
    • More Windows integration:
      • Theano scripts (theano-cache and theano-nose) now works on Windows
      • Better support for Windows end-lines into C codes
      • Support for space in paths on Windows
    • Scan improvements:
      • More scan optimizations, with faster compilation and gradient computation
      • Support for checkpoint in scan (trade off between speed and memory usage, useful for long sequences)
      • Fixed broadcast checking in scan
    • Graphs improvements:
      • More numerical stability by default for some graphs
      • Better handling of corner cases for theano functions and graph optimizations
      • More graph optimizations with faster compilation and execution
      • smaller and more readable graph
    • New GPU back-end:
      • Removed warp-synchronous programming to get good results with newer CUDA drivers
      • More pooling support on GPU when cuDNN isn’t available
      • Full support of ignore_border option for pooling
      • Inplace storage for shared variables
      • float16 storage
      • Using PCI bus ID of graphic cards for a better mapping between theano device number and nvidia-smi number
      • Fixed offset error in GpuIncSubtensor
    • Less C code compilation
    • Added support for bool dtype
    • Updated and more complete documentation
    • Bug fixes related to merge optimizer and shape inference
    • Lot of other bug fixes, crashes fixes and warning improvements

A total of 123 people contributed to this release since 0.8.0, see list below.

  • Interface changes:
    • Merged CumsumOp/CumprodOp into CumOp
    • In MRG module:
      • Replaced method multinomial_wo_replacement() with new method choice()
      • Random generator now tries to infer the broadcast pattern of its output
    • New pooling interface
    • Pooling parameters can change at run time
    • Moved softsign out of sandbox to theano.tensor.nnet.softsign
    • Using floatX dtype when converting empty list/tuple
    • Roll make the shift be modulo the size of the axis we roll on
    • round() default to the same as NumPy: half_to_even
  • Convolution updates:
    • Support of full and half modes for 2D and 3D convolutions including in conv3d2d
    • Allowed pooling of empty batch
    • Implement conv2d_transpose convenience function
    • Multi-cores convolution and pooling on CPU
    • New abstract 3d convolution interface similar to the 2d convolution interface
    • Dilated convolution
  • GPU:
    • cuDNN: support versoin 5.1 and wrap batch normalization (2d and 3d) and RNN functions
    • Multiple-GPU, synchrone update (via platoon, use NCCL)
    • Gemv(matrix-vector product) speed up for special shape
    • cublas gemv workaround when we reduce on an axis with a dimensions size of 0
    • Warn user that some cuDNN algorithms may produce unexpected results in certain environmentsfor convolution backward filter operations
    • GPUMultinomialFromUniform op now supports multiple dtypes
    • Support for MaxAndArgMax for some axis combination
    • Support for solve (using cusolver), erfinv and erfcinv
    • Implemented GpuAdvancedSubtensor
  • New features:
    • OpFromGraph now allows gradient overriding for every input
    • Added Abstract Ops for batch normalization that use cuDNN when available and pure Theano CPU/GPU alternatives otherwise
    • Added gradient of solve, tensorinv (CPU), tensorsolve (CPU), searchsorted (CPU), DownsampleFactorMaxGradGrad (CPU)
    • Added Multinomial Without Replacement
    • Allowed partial evaluation of compiled function
    • More Rop support
    • Indexing support ellipsis: a[…, 3]`, a[1,…,3]
    • Added theano.tensor.{tensor5,dtensor5, …}
    • compiledir_format support device
    • Added New Theano flag conv.assert_shape to check user-provided shapes at runtime (for debugging)
    • Added new Theano flag cmodule.age_thresh_use
    • Added new Theano flag cuda.enabled
    • Added new Theano flag nvcc.cudafe to enable faster compilation and import with old CUDA back-end
    • Added new Theano flag print_global_stats to print some global statistics (time spent) at the end
    • Added new Theano flag profiling.ignore_first_call, useful to profile the new gpu back-end
    • remove ProfileMode (use Theano flag profile=True instead)
  • Others:
    • Split op now has C code for CPU and GPU
    • theano-cache list now includes compilation times
    • Speed up argmax only on GPU (without also needing the max)
    • More stack trace in error messages
    • Speed up cholesky grad
    • log(sum(exp(…))) now get stability optimized
  • Other more detailed changes:
    • Added Jenkins (gpu tests run on pull requests in addition to daily buildbot)
    • Removed old benchmark directory and other old files not used anymore
    • Use of 64-bit indexing in sparse ops to allow matrix with more then 231-1 elements
    • Allowed more then one output to be an destructive inplace
    • More support of negative axis
    • Added the keepdims parameter to the norm function
    • Make scan gradient more deterministic
  • Commiters since 0.8.0:
    • Frederic Bastien
    • Arnaud Bergeron
    • Pascal Lamblin
    • Steven Bocco
    • Ramana Subramanyam
    • Simon Lefrancois
    • Gijs van Tulder
    • Benjamin Scellier
    • khaotik
    • Chiheb Trabelsi
    • Chinnadhurai Sankar
    • Cesar Laurent
    • Reyhane Askari
    • Mohammad Pezeshki
    • Alexander Matyasko
    • Alexandre de Brebisson
    • Mathieu Germain
    • Nan Rosemary Ke
    • Pierre Luc Carrier
    • Olivier Mastropietro
    • Thomas George
    • Saizheng Zhang
    • Iulian Vlad Serban
    • Francesco Visin
    • Caglar
    • Faruk Ahmed
    • Harm de Vries
    • Samira Shabanian
    • Vincent Dumoulin
    • Nicolas Ballas
    • Jakub Sygnowski
    • Jan Schlüter
    • Samira Ebrahimi Kahou
    • Mikhail Korobov
    • Fei Wang
    • Kv Manohar
    • Jesse Livezey
    • Kelvin Xu
    • Matt Graham
    • Ruslana Makovetsky
    • Sina Honari
    • Bryn Keller
    • Ciyong Chen
    • Vitaliy Kurlin
    • Zhouhan LIN
    • Gokula Krishnan
    • Kumar Krishna Agrawal
    • Ozan Çağlayan
    • Vincent Michalski
    • affanv14
    • Amjad Almahairi
    • Ray Donnelly
    • Tim Cooijmans
    • happygds
    • mockingjamie
    • Christos Tsirigotis
    • Florian Bordes
    • Ilya Kulikov
    • RadhikaG
    • Taesup (TS) Kim
    • Ying Zhang
    • Anton Chechetka
    • Karthik Karanth
    • Kirill Bobyrev
    • Rebecca N. Palmer
    • Yang Zhang
    • Yaroslav Ganin
    • Jonas Degrave
    • Liwei Cai
    • Lucas Beyer
    • Michael Harradon
    • Morgan Stuart
    • Tim Gasper
    • Xavier Bouthillier
    • p
    • texot
    • Andrés Gottlieb
    • Ben Poole
    • Bhavishya Pohani
    • Carl Thomé
    • David Bau
    • Dimitar Dimitrov
    • Evelyn Mitchell
    • Fei Zhan
    • Fuchai
    • Fábio Perez
    • Gennadiy Tupitsin
    • Gilles Louppe
    • Greg Ciccarelli
    • He
    • Huan Zhang
    • Kaixhin
    • Kevin Keraudren
    • Maltimore
    • Marc-Alexandre Cote
    • Marco
    • Marius F. Killinger
    • Martin Drawitsch
    • Maxim Kochurov
    • Micah Bojrab
    • Neil
    • Nizar Assaf
    • Rithesh Kumar
    • Rizky Luthfianto
    • Robin Millette
    • Roman Ring
    • Sander Dieleman
    • Sebastin Santy
    • Shawn Tan
    • Wazeer Zulfikar
    • Wojciech Głogowski
    • Yann N. Dauphin
    • gw0 [http://gw.tnode.com/]
    • hexahedria
    • hsintone
    • jakirkham
    • joncrall
    • root
    • superantichrist
    • tillahoffmann
    • valtron
    • wazeerzulfikar
    • you-n-g