2015 The MathWorks, Inc. 1

Transcription

1 2015 The MathWorks, Inc. 1

2 개발에서구현까지 MATLAB 환경에서의딥러닝 김종남 Application Engineer 2015 The MathWorks, Inc. 2

3 3

4 Why MATLAB for Deep Learning? MATLAB is Productive MATLAB is Fast MATLAB Integrates with Open Source 4

5 What is Deep Learning? 6

6 Deep Learning Model learns to perform classification tasks directly from data. x1000 x1000 x1000 Deep Learning Model Image Classifier x1000 x1000 7

7 Data Types for Deep Learning Signal Text Image 8

8 Deep Learning is Versatile Detection of cars and road in autonomous driving systems Rain Detection and Removal 1 Iris Recognition 99.4% accuracy 2 1. Deep Joint Rain Detection and Removal from a Single Image" Wenhan Yang, Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features for iris recognition Signal Processing in Medicine and Biology Symposium (SPMB), 2016 IEEE Shervin Minaee ; Amirali Abdolrashidiy ; Yao Wang; An experimental study of deep convolutional features for iris recognition 9

9 How is deep learning performing so well? 10

10 Deep Learning Uses a Neural Network Architecture Input Layer Hidden Layers (n) Output Layer 11

11 Thinking about Layers Layers are like blocks Stack on top of each other Replace one block with a different one Each hidden layer processes the information from the previous layer 12

12 Thinking about Layers Layers are like blocks Stack them on top of each other Replace one block with a different one Each hidden layer processes the information from the previous layer Layers can be ordered in different ways 13

13 Deep Learning in 6 Lines of MATLAB Code 14

14 Why MATLAB for Deep Learning? MATLAB is Productive MATLAB is Fast MATLAB integrates with Open Source 15

15 I love to label and preprocess my data ~ Said no engineer, ever. 16

16 Caterpillar Case Study World s leading manufacturer of construction and mining equipment. Similarity between these projects? Autonomous haul trucks Pedestrian detection Equipment classification Terrain mapping 17

17 Computer Must Learn from Lots of Data ALL data must first be labeled to create these autonomous systems. We were spending way too much time ground-truthing [the data] --Larry Mianzo, Caterpillar 18

18 How Did Caterpillar Do with Our Tools? Semi-automated labeling process Used MATLAB for entire development workflow. Because everything is in MATLAB, development time is short 19

19 How Does MATLAB Come into Play? 20

20 21

21 22

22 23

23 MATLAB is Productive Image Labeler App semi-automates labeling workflow Bootstrapping Improve automatic labeling by updating algorithm as you label more images correctly. Easy to load metadata even when labeling manually 24

24 MATLAB is Fast Performance Training Deployment 25

25 What is Training? Feed labeled data into neural network to create working model x1000 x1000 x1000 Convolution Neural Network Image Classifier Model x1000 x

26 Amplitude Frequency Speech Recognition Example Audio signal Spectrogram Image Classification algorithm Time Time 27

27 Another Network for Signals - LSTM LSTM = Long Short Term Memory (Networks) Signal, text, time-series data Use previous data to predict new information I live in France. I speak. c 0 C 1 C t 28

28 1. Create Datastore Datastore creates reference for data Do not have to load in all objects into memory 29

29 Frequency Amplitude 2. Compute Speech Spectrograms Time 30

30 3. Split datastores Training Validation Test 70% 15% 15% Trains the model Computer learns from this data Checks accuracy of model during training Tests model accuracy Not used until validation accuracy is good 31

31 4. Define Architecture and Parameters Neural Network Architecture Model Parameters 32

32 5. Train Network 33

33 Deep Learning on CPU, GPU, Multi-GPU and Clusters H OW TO TA RG E T? Single CPU Single CPU Single GPU Single CPU, Multiple GPUs On-prem server with GPUs Cloud GPUs (AWS) 34

34 Training is an Iterative Process Parameters adjusted according to performance 36

35 MATLAB is Fast for Deployment Target a GPU for optimal performance NVIDIA GPUs use CUDA code We only have MATLAB code. Can we translate this? 37

36 GPU Coder Automatically generates CUDA Code from MATLAB Code can be used on NVIDIA GPUs CUDA extends C/C++ code with constructs for parallel computing 38

37 GPU Coder Performance 39

38 Why MATLAB? MATLAB is Productive MATLAB is Fast MATLAB Integrates with Open Source 41

39 MATLAB Integrates with Open Source Frameworks Access to many pretrained models through add-ons Users wanted to import latest models Import models directly from Tensorflow or Caffe Allows for improved collaboration 43

40 Keras-Tensorflow Importer 44

41 MATLAB Integrates with Open Source Frameworks MATLAB supports entire deep learning workflow Use when it is convenient for your workflow Access to latest models Improved collaboration with other users 45

42 Why MATLAB for Deep Learning? MATLAB is Productive MATLAB is Fast (Performance) MATLAB Integrates with Open Source (Frameworks) 46