Torch Internals. Ronan Collobert torch.ch

Size: px

Start display at page:

Download "Torch Internals. Ronan Collobert torch.ch"

Marvin Brendan Hines
5 years ago
Views:

1 Torch Internals Ronan Collobert torch.ch

2 Introduction

3 Machine Learning Library Around since versions, implemented in C, C++, Objective C Implements main machine learning algorithms: SVMs, Neural Networks, HMMs, GMMs Goal: research on large-scale machine learning

4 It is for researchers It has to be modular! organized as packages It must be easy to extend! It must be simple to understand! a high-level language is essential It must be fast (large-scale machine learning) JIT-compile or efficient C interface for core routines

5 Overview

6 Overview optim torch (lua) nn nngraph TH (C) svm sndfile image

7 Overview core optim torch (lua) nn nngraph TH (C) svm sndfile image

8 Overview packages core optim torch (lua) nn nngraph TH (C) svm sndfile image

9 Overview packages core optim torch (lua) nn nngraph TH (C) svm sndfile image

10 Overview packages core optim torch (lua) nn nngraph TH (C) svm sndfile image

11 Overview packages core optim torch (lua) nn nngraph TH (C) svm sndfile image

12 Overview packages glue core optim cwrap torch (lua) nn nngraph luat (C) TH (C) svm sndfile image

13 Which Programming Language? LISP? (Lush A subset could be compiled Could inline C code JIT-compiled Languages? (e.g. LuaJIT) Only a subset is actually compiled Python? err Lua? Looks like pseudo-code (yet very powerful) Simple C API

14 Which Programming Language? C++? (Torch3, PLearn, EBLearn ) Objective C? (Torch4) Avoid too many abstractions Avoid complicated syntax We chose C for Torch7

15 The Core TH and torch

16 The core library (TH+torch) ML algorithms manipulate all kind of data Represent data as a n-dimensional Tensor 1D: a bunch of features 2D: a gray image, some audio features 3D: RGB images 4D: Videos The core lib provides many tensor operations Available from C (TH lib) and Lua (torch package)

17 The Core Library (TH+torch) Avoid memory copies A Tensor is a view of a Storage (memory chunk) Storages might have different views (Tensors) Storages can be in-memory, or on-disk (mmap!)

18 The Core Library (TH+torch) Avoid memory copies A Tensor is a view of a Storage (memory chunk) Storages might have different views (Tensors) Storages can be in-memory, or on-disk (mmap!) Easy to get different views from another tensor x: x:narrow(dim, idx, size) x:select(dim, idx) x[idx] x:unfold(dim, kw, dw) x:transpose(dim1, dim2)

19 The Core Library (TH+torch) Avoid memory copies A Tensor is a view of a Storage (memory chunk) Storages might have different views (Tensors) Storages can be in-memory, or on-disk (mmap!) Easy to get different views from another tensor x: x:narrow(dim, idx, size) x:select(dim, idx) x[idx] new tensors share same storage x:unfold(dim, kw, dw) x:transpose(dim1, dim2)

20 The Core Library (TH+torch) Avoid memory copies All C functions are of the form THTensor_foobar(THTensor *dst, THTensor *src1, THTensor *src2) The Lua interface supports the destination as an optional argument: torch.foobar([dst], src1, src2) or dst:foobar(src1, src2)

21 The Glue luat and cwrap

22 C/Lua Glue Lua is a stack-based language In Lua: In C: a = f("how", t.x, 14) How to extend it? (luat or FFI) Lua provides the userdata type (a GCed C pointer) Need to do proper type checking both in C and Lua How to limit the pain? (cwrap or FFI)

23 C/Lua Glue (luat)

24 C/Lua Glue (luat) check arguments on the stack

25 C/Lua Glue (luat) check arguments on the stack push the result

26 C/Lua Glue (luat) check arguments on the stack push the result call TH

27 C/Lua Glue (cwrap)

28 C/Lua Glue (cwrap) for all tensor types

29 C/Lua Glue (cwrap) for all tensor types torch.cmul(dst, src) or dst:cmul(src)

30 C/Lua Glue (cwrap)

31 C/Lua Glue (cwrap) another default value

32 C/Lua Glue (cwrap) two different C functions another default value

34 C/Lua Glue (FFI) Cut-and-paste the C header Caveats: Need to do proper argument checking (sub-classes ) Need to overload methods/functions properly Less robust across different systems (#define ) Need also to handle different types (templates)

35 Tensor Types

36 Dynamic Typing (lua) Write generic code with torch.tensor() Specify alias with (e.g.) torch.setdefaulttensortype( torch.floattensor )

37 Dynamic Typing (lua) Write generic code with torch.tensor() Specify alias with (e.g.) torch.setdefaulttensortype( torch.floattensor ) return a torch.tensor()

38 CUDA cutorch package defines CudaTensor Most CPU Tensor methods are available Relies on few custom-made iterators and thrust x:cuda() or x:float() for GPU <-> CPU copies cunn package provides a GPU backend to nn Most popular nn layers have a GPU backend

39 Deep Learning nn

40 nn package

41 nn package

42 nn package Two main classes: Module and Criterion Three main methods: updateoutput(input) updategradinput(input, gradoutput) accgradparameters(input, gradoutput)

43 nn package

44 Tutorial Torch7 nn package itorch Installation instructions Get the tutorial

Deep Learning with Torch

Deep Learning with Torch The good, the bad, the ugly since 2002 Jimmy Ba jimmy@psi.utoronto.ca What is Torch? Year 2012 Google Answer: Torch7 provides a Matlab-like environment for state-of-the-art machine