Torch (machine learning)

Short description: Deep learning software

Torch

Original author(s)| Ronan Collobert, Samy Bengio, Johnny Mariéthoz[1]
Initial release| October 2002; 23 years ago (2002-10)[1]
Final release|

7.0 / February 27, 2017; 9 years ago (2017-02-27)[2]

Written in| Lua, C, C++
Operating system| Linux, Android, Mac OS X, iOS
Type| Library for machine learning and deep learning
License| BSD License
Website| torch.ch

Torch is an open-source machine learning library, a scientific computing framework, and a scripting language based on Lua.[3] It provides LuaJIT interfaces to deep learning algorithms implemented in C. It was created by the Idiap Research Institute at EPFL. Torch development moved in 2017 to PyTorch, a port of the library to Python.[4][5][better source needed]

## Contents

* 1 torch
* 2 nn
* 3 Other packages
* 4 Applications
* 5 See also
* 6 References
* 7 External links

## torch

The core package of Torch is `torch`. It provides a flexible N-dimensional array or Tensor, which supports basic routines for indexing, slicing, transposing, type-casting, resizing, sharing storage and cloning. This object is used by most other packages and thus forms the core object of the library. The Tensor also supports mathematical operations like `max`, `min`, `sum`, statistical distributions like uniform, normal and multinomial, and BLAS operations like dot product, matrix–vector multiplication, matrix–matrix multiplication and matrix product.

The following exemplifies using torch via its REPL interpreter:

> a = torch.randn(3,4)

> =a
-0.2381 -0.3401 -1.7844 -0.2615
0.1411  1.6249  0.1708  0.8299
-1.0434  2.2291  1.0525  0.8465
[torch.DoubleTensor of dimension 3x4]

> a[1][2]
-0.34010116549482

> a:narrow(1,1,2)
-0.2381 -0.3401 -1.7844 -0.2615
0.1411  1.6249  0.1708  0.8299
[torch.DoubleTensor of dimension 2x4]

> a:index(1, torch.LongTensor{1,2})
-0.2381 -0.3401 -1.7844 -0.2615
0.1411  1.6249  0.1708  0.8299
[torch.DoubleTensor of dimension 2x4]

> a:min()
-1.7844365427828

The `torch` package also simplifies object-oriented programming and serialization by providing various convenience functions which are used throughout its packages. The `torch.class(classname, parentclass)` function can be used to create object factories (classes). When the constructor is called, torch initializes and sets a Lua table with the user-defined metatable, which makes the table an object.

Objects created with the torch factory can also be serialized, as long as they do not contain references to objects that cannot be serialized, such as Lua coroutines, and Lua userdata. However, userdata can be serialized if it is wrapped by a table (or metatable) that provides `read()` and `write()` methods.

## nn

The `nn` package is used for building neural networks. It is divided into modular objects that share a common `Module` interface. Modules have a `forward()` and `backward()` method that allow them to feedforward and backpropagate, respectively. Modules can be joined using module composites, like `Sequential`, `Parallel` and `Concat` to create complex task-tailored graphs. Simpler modules like `Linear`, `Tanh` and `Max` make up the basic component modules. This modular interface provides first-order automatic gradient differentiation. What follows is an example use-case for building a multilayer perceptron using Modules:

> mlp = nn.Sequential()
> mlp:add( nn.Linear(10, 25) ) -- 10 input, 25 hidden units
> mlp:add( nn.Tanh() ) -- some hyperbolic tangent transfer function
> mlp:add( nn.Linear(25, 1) ) -- 1 output
> =mlp:forward(torch.randn(10))
-0.1815
[torch.Tensor of dimension 1]

Loss functions are implemented as sub-classes of `Criterion`, which has a similar interface to `Module`. It also has `forward()` and `backward()` methods for computing the loss and backpropagating gradients, respectively. Criteria are helpful to train neural network on classical tasks. Common criteria are the Mean Squared Error criterion implemented in `MSECriterion` and the cross-entropy criterion implemented in `ClassNLLCriterion`. What follows is an example of a Lua function that can be iteratively called to train an `mlp` Module on input Tensor `x`, target Tensor `y` with a scalar `learningRate`:

function gradUpdate(mlp, x, y, learningRate)
local criterion = nn.ClassNLLCriterion()
pred = mlp:forward(x)
local err = criterion:forward(pred, y);
mlp:zeroGradParameters();
local t = criterion:backward(pred, y);
mlp:backward(x, t);
mlp:updateParameters(learningRate);
end

It also has `StochasticGradient` class for training a neural network using Stochastic gradient descent, although the `optim` package provides much more options in this respect, like momentum and weight decay regularization.

## Other packages

Many packages other than the above official packages are used with Torch. These are listed in the torch cheatsheet.[6] These extra packages provide a wide range of utilities such as parallelism, asynchronous input/output, image processing, and so on. They can be installed with LuaRocks, the Lua package manager which is also included with the Torch distribution.

## Applications

Torch is used by the Facebook AI Research Group,[7] IBM,[8] Yandex[9] and the Idiap Research Institute.[10] Torch has been extended for use on Android[11][better source needed] and iOS.[12][better source needed] It has been used to build hardware implementations for data flows like those found in neural networks.[13]

Facebook has released a set of extension modules as open source software.[14]

## See also

* Comparison of deep learning software
* PyTorch

## References

1. ↑ 1.0 1.1 "Torch: a modular machine learning software library". 30 October 2002. CiteSeerX 10.1.1.8.9850.
2. ↑ Collobert, Ronan. "Torch7". http://torch.ch/.
3. ↑ "Torch7: A Matlab-like Environment for Machine Learning". Neural Information Processing Systems. 2011. http://ronan.collobert.com/pub/matos/2011_torch7_nipsw.pdf.
4. ↑ Torch GitHub repository ReadMe
5. ↑ PyTorch GitHub repository
6. ↑ "Cheatsheet · torch/torch7 Wiki". https://github.com/torch/torch7/wiki/Cheatsheet.
7. ↑ KDnuggets Interview with Yann LeCun, Deep Learning Expert, Director of Facebook AI Lab
8. ↑ Hacker News
9. ↑ Yann Lecun's Facebook Page
10. ↑ IDIAP Research Institute : Torch
11. ↑ Torch-android GitHub repository
12. ↑ Torch-ios GitHub repository
13. ↑ NeuFlow: A Runtime Reconfigurable Dataflow Processor for Vision
14. ↑ "Facebook Open-Sources a Trove of AI Tools". 16 January 2015. https://www.wired.com/2015/01/facebook-open-sources-trove-ai-tools/.

## External links

* v
* t
* e

Deep learning software

Open-source|

* Apache SINGA
* Caffe
* Deeplearning4j
* Dlib
* Keras
* Microsoft Cognitive Toolkit
* MXNet
* OpenNN
* PyTorch
* TensorFlow
* Theano
* Torch
* ONNX

Proprietary|

* Maple
* Neural Designer
* Wolfram Mathematica
* Apple Core ML

* Category
* Comparison

* v
* t
* e

Lua programming language

Interpreters and compilers|

* Lua (reference implementation from PUC-Rio)
* LuaJIT
* Parrot

Package manager|

* LuaRocks

Ports and distributions|

* Plua
* Lua Player
* Corona

IDEs|

* Decoda
* SciTE
* Comparison of Lua IDEs

Applications and frameworks|

* Prosody
* Torch
* Tarantool
* IUP

Lua in Wikipedia

0.00

(0 votes)

Original source: https://en.wikipedia.org/wiki/Torch (machine learning). Read more

|

Retrieved from "https://handwiki.org/wiki/index.php?title=Software:Torch_(machine_learning)&oldid=737726"
*[v]: View this template
*[t]: Discuss this template
*[e]: Edit this template