Theano 0.9.0

Release Notes

Theano 0.9.0 (20th of March, 2017)

This is a final release of Theano, version 0.9.0, with a lot of new features, interface changes, improvements and bug fixes.

We recommend that everybody update to this version.

Highlights (since 0.8.0):

• Better Python 3.5 support

• Better numpy 1.12 support

• Conda packages for Mac, Linux and Windows

• Support newer Mac and Windows versions

• More Windows integration:

  • Theano scripts (theano-cache and theano-nose) now works on Windows

  • Better support for Windows end-lines into C codes

  • Support for space in paths on Windows

• Scan improvements:

  • More scan optimizations, with faster compilation and gradient computation

  • Support for checkpoint in scan (trade off between speed and memory usage, useful for long sequences)

  • Fixed broadcast checking in scan

• Graphs improvements:

  • More numerical stability by default for some graphs

  • Better handling of corner cases for theano functions and graph optimizations

  • More graph optimizations with faster compilation and execution

  • smaller and more readable graph

• New GPU back-end:

  • Removed warp-synchronous programming to get good results with newer CUDA drivers

  • More pooling support on GPU when cuDNN isn’t available

  • Full support of ignore_border option for pooling

  • Inplace storage for shared variables

  • float16 storage

  • Using PCI bus ID of graphic cards for a better mapping between theano device number and nvidia-smi number

  • Fixed offset error in GpuIncSubtensor

• Less C code compilation

• Added support for bool dtype

• Updated and more complete documentation

• Bug fixes related to merge optimizer and shape inference

• Lot of other bug fixes, crashes fixes and warning improvements

A total of 123 people contributed to this release since 0.8.0, see list below.

Interface changes:

• Merged CumsumOp/CumprodOp into CumOp

• In MRG module:

  • Replaced method multinomial_wo_replacement() with new method choice()

  • Random generator now tries to infer the broadcast pattern of its output

• New pooling interface

• Pooling parameters can change at run time

• Moved softsign out of sandbox to theano.tensor.nnet.softsign

• Using floatX dtype when converting empty list/tuple

• Roll make the shift be modulo the size of the axis we roll on

• round() default to the same as NumPy: half_to_even

Convolution updates:

• Support of full and half modes for 2D and 3D convolutions including in conv3d2d

• Allowed pooling of empty batch

• Implement conv2d_transpose convenience function

• Multi-cores convolution and pooling on CPU

• New abstract 3d convolution interface similar to the 2d convolution interface

• Dilated convolution

GPU:

• cuDNN: support versoin 5.1 and wrap batch normalization (2d and 3d) and RNN functions

• Multiple-GPU, synchrone update (via platoon, use NCCL)

• Gemv(matrix-vector product) speed up for special shape

• cublas gemv workaround when we reduce on an axis with a dimensions size of 0

• Warn user that some cuDNN algorithms may produce unexpected results in certain environments for convolution backward filter operations

• GPUMultinomialFromUniform op now supports multiple dtypes

• Support for MaxAndArgMax for some axis combination

• Support for solve (using cusolver), erfinv and erfcinv

• Implemented GpuAdvancedSubtensor

New features:

• OpFromGraph now allows gradient overriding for every input

• Added Abstract Ops for batch normalization that use cuDNN when available and pure Theano CPU/GPU alternatives otherwise

• Added gradient of solve, tensorinv (CPU), tensorsolve (CPU), searchsorted (CPU), DownsampleFactorMaxGradGrad (CPU)

• Added Multinomial Without Replacement

• Allowed partial evaluation of compiled function

• More Rop support

• Indexing support ellipsis: a[..., 3]`, a[1,...,3]

• Added theano.tensor.{tensor5,dtensor5, ...}

• compiledir_format support device

• Added New Theano flag conv.assert_shape to check user-provided shapes at runtime (for debugging)

• Added new Theano flag cmodule.age_thresh_use

• Added new Theano flag cuda.enabled

• Added new Theano flag nvcc.cudafe to enable faster compilation and import with old CUDA back-end

• Added new Theano flag print_global_stats to print some global statistics (time spent) at the end

• Added new Theano flag profiling.ignore_first_call, useful to profile the new gpu back-end

• remove ProfileMode (use Theano flag profile=True instead)

Others:

• Split op now has C code for CPU and GPU

• theano-cache list now includes compilation times

• Speed up argmax only on GPU (without also needing the max)

• More stack trace in error messages

• Speed up cholesky grad

• log(sum(exp(...))) now get stability optimized

Other more detailed changes:

• Added Jenkins (gpu tests run on pull requests in addition to daily buildbot)

• Removed old benchmark directory and other old files not used anymore

• Use of 64-bit indexing in sparse ops to allow matrix with more then 2³¹-1 elements

• Allowed more then one output to be an destructive inplace

• More support of negative axis

• Added the keepdims parameter to the norm function

• Make scan gradient more deterministic

Commiters since 0.8.0:

• Frederic Bastien

• Arnaud Bergeron

• Pascal Lamblin

• Steven Bocco

• Ramana Subramanyam

• Simon Lefrancois

• Gijs van Tulder

• Benjamin Scellier

• khaotik

• Chiheb Trabelsi

• Chinnadhurai Sankar

• Cesar Laurent

• Reyhane Askari

• Mohammad Pezeshki

• Alexander Matyasko

• Alexandre de Brebisson

• Mathieu Germain

• Nan Rosemary Ke

• Pierre Luc Carrier

• Olivier Mastropietro

• Thomas George

• Saizheng Zhang

• Iulian Vlad Serban

• Francesco Visin

• Caglar

• Faruk Ahmed

• Harm de Vries

• Samira Shabanian

• Vincent Dumoulin

• Nicolas Ballas

• Jakub Sygnowski

• Jan Schlüter

• Samira Ebrahimi Kahou

• Mikhail Korobov

• Fei Wang

• Kv Manohar

• Jesse Livezey

• Kelvin Xu

• Matt Graham

• Ruslana Makovetsky

• Sina Honari

• Bryn Keller

• Ciyong Chen

• Vitaliy Kurlin

• Zhouhan LIN

• Gokula Krishnan

• Kumar Krishna Agrawal

• Ozan Çağlayan

• Vincent Michalski

• affanv14

• Amjad Almahairi

• Ray Donnelly

• Tim Cooijmans

• happygds

• mockingjamie

• Christos Tsirigotis

• Florian Bordes

• Ilya Kulikov

• RadhikaG

• Taesup (TS) Kim

• Ying Zhang

• Anton Chechetka

• Karthik Karanth

• Kirill Bobyrev

• Rebecca N. Palmer

• Yang Zhang

• Yaroslav Ganin

• Jonas Degrave

• Liwei Cai

• Lucas Beyer

• Michael Harradon

• Morgan Stuart

• Tim Gasper

• Xavier Bouthillier

• p

• texot

• Andrés Gottlieb

• Ben Poole

• Bhavishya Pohani

• Carl Thomé

• David Bau

• Dimitar Dimitrov

• Evelyn Mitchell

• Fei Zhan

• Fuchai

• Fábio Perez

• Gennadiy Tupitsin

• Gilles Louppe

• Greg Ciccarelli

• He

• Huan Zhang

• Kaixhin

• Kevin Keraudren

• Maltimore

• Marc-Alexandre Cote

• Marco

• Marius F. Killinger

• Martin Drawitsch

• Maxim Kochurov

• Micah Bojrab

• Neil

• Nizar Assaf

• Rithesh Kumar

• Rizky Luthfianto

• Robin Millette

• Roman Ring

• Sander Dieleman

• Sebastin Santy

• Shawn Tan

• Wazeer Zulfikar

• Wojciech Głogowski

• Yann N. Dauphin

• gw0 [http://gw.tnode.com/]

• hexahedria

• hsintone

• jakirkham

• joncrall

• root

• superantichrist

• tillahoffmann

• valtron

• wazeerzulfikar

• you-n-g