Deep Learning with R. Francesca Lazzeri Data Scientist II - Microsoft, AI Research

Transcription

1 with R Francesca Lazzeri Data Scientist II - Microsoft, AI Research

2 Agenda with R What is Demo Better understanding of R DL tools

3 Fundamental concepts in Forward Propagation Algorithm Activation Functions Gradient Descent Backpropagation

4 Fundamental concepts in Forward Propagation Algorithm Activation Functions Gradient Descent Backpropagation

5 Example as seen by linear regression Age Bank Balance Retirement Status Number of Transactions

6 Interactions o Neural networks account for interactions really well o learning uses especially powerful neural networks for: * Text * Images * Videos * Audio * Source code

7 learning models capture interactions Age Bank Balance Retirement Status Number of Transactions

8 Interactions in neural networks Input Layer Age Hidden Layer Bank Balance Retirement Status Output Layer Number of Transactions # Accounts

9 Forward Propagation Algorithm # Children Input Hidden 5 2 Output # Transactions 1 1 # Accounts 3

10 Fundamental concepts in Forward Propagation Algorithm Activation Functions Gradient Descent Backpropagation

11 Activation Functions # Children Input Hidden tanh (2+3) 2 Output # Transactions 1 tanh (-2+3) # Accounts 3

12 ReLU Activation Function Input Hidden Hidden 0-3 Output

13 Representation o networks internally build representations of patterns in the data o Partially replace the need for feature engineering o Subsequent layers build increasingly sophisticated representations of raw data o Modeler doesn t need to specify the interactions o When you train the model, the neural network gets weights that find the relevant patterns to make better predictions

14 Fundamental concepts in Forward Propagation Algorithm Activation Functions Gradient Descent Backpropagation

15 The Need for Optimization o Predictions with multiple points * Making accurate predictions gets harder with more points * At any set of weights, there are many values of the error * Correspond to the many points we make predictions for o Loss function * Aggregate errors in predictions from many data points into single number * Measure of model s predictive performance

16 The Need for Optimization o Squared error loss function Prediction Actual Error Squared Error o Total Squared Error: 150 o Mean Squared Error: 50 o Lower loss function value means a better model o Goal: find the weights that give the lowest value for the loss function o Gradient descent!

17 Gradient Descent Loss(w) w

18 Gradient Descent o Slope calculation example Actual Target Value = 10 o To calculate the slope for a weight, need to multiply: * Slope of the loss function w.r.t value at the node we feed into * The value of the node that feeds into our weight * Slope of activation function w.r.t value we feed into

19 Fundamental concepts in Forward Propagation Algorithm Activation Functions Gradient Descent Backpropagation

20 Backpropagation Input 3 Hidden 26 Hidden 0 Output

21 Backpropagation o Allows gradient descent to update all weights in neural network (by getting gradients for all weights) o Go back one layer at a time o Important to understand the process, but you will generally use a library that implements this

22 with R MXNetR Feed-forward neural network Convolutional neural network (CNN) darch Restricted Boltzmann machine belief network deepnet Feed-forward neural network Restricted Boltzmann machine belief network Stacked autoencoders H2O Feed-forward neural network autoencoders deepr Simplify some functions from H2O net packages

23 with R MNIST Iris Forest Cover Type Model/Dataset Accuracy (%) Runtime (sec) Accuracy (%) Runtime (sec) Accuracy (%) Runtime (sec) MXNetR (CPU) MXNetR (GPU) darch darch 500/ deepnet DBN deepnet DNN H2O Random Forest

24 with R R interface to Keras

25 Demo with R on Azure with Keras and CNTK DSVM CNTK R & Keras

26 Demo Keras Workflow Steps to Build your Model o Specify architecture o Compile the model o Fit the model o Predict

27 Demo Preparing the Data

28 Demo Defining the Model

29 Demo Defining the Model

30 Demo Defining the Model

31 Demo Training and Evaluation

32 References

33 Thank You! Francesca Lazzeri Data Scientist II - Microsoft, AI Research