Deep learning toolkits

There are wide variety of software packages (here is list of some software) in different languages which can be used to implement deep learning models. This post gives general introduction of Theano, Torch, and TensorFlow, and gives the comparison in the end. For installation steps and sample programs visit my blog post here.

Theano

Theano is a python library that lets you define, optimize and evaluate mathematical expressions especially ones with multi-dimensional arrays. It combines the aspects of computer algebra system with aspects of an optimizing compiler. It can generate customized C code for mathematical operations. It is useful in those cases where the expressions are evaluated  repeatedly and evaluation speed is critical.

Theano is written in LISA and is used in development of efficient machine learning algorithms. It uses GPU computation, constant folding, similar subgraph merging, arithmetic simplification, inplace operations, memory aliasing, lazy evaluation, etc to optimize expression evaluation. It support all numPy and SciPy functionality.

Torch

Torch is a scientific computing framework with wide support for machine learning algorithms that puts GPU first. It is easy and efficient to use. This framework was implemented by google before the development of TensorFlow. LuaJIT scripting language is used to generate machine learning model, which makes it fast and easy to understand. It support neural network and energy based models and has implemented numeric optimization routines and embeddable with ports to iOS, Android and FPGA backends. Torch comes with already implemented community packages which has been implemented in computer vision, NLP, signal processing, etc.

TensorFlow

TensorFLow is an open source software library for numerical computations using data flow graphs. Nodes in graph represent mathematical operations, while graph edges represent multidimensional data arrays also called as tensors.

TensorFlow allows easy deployment of models in mobile desktop or server and usage of multiple CPU or GPU for computations. It was developed by google brain team. TensorFlow is highly flexible, it can be used in any system where you can express computation as data flow graph. It is highly portable, production ready, it has automatic differentiation capabilities, means it can calculate derivative automatically. TensorFlow gives language option for programmers to implement models in C++ or python and it can be implemented in distributed servers and maximizes performance by utilizing CPU and GPU processing capability if possible.

Comparison of TensorFlow, Torch and Theano

Few major deep learning Toolkits are evaluated (here) based on different parameters. In terms of flexibility and modelling Torch is the most flexible toolkit. In term of interface and model deployment TensorFlow is the best option as it is easy and deployment ready. In terms of performance, Torch and TensorFLow are similar. Theano performance is poor because of its startup time as it compiles CUDA to executable binary. TensorFlow and Torch do not work on Windows.

For installation steps and sample programs, please visit my wordpress blog post here.