Accelerating Convolutional Neural Nets. Yunming Zhang

Transcription

1 Accelerating Convolutional Neural Nets Yunming Zhang

2 Focus Convolutional Neural Nets is the state of the art in classifying the images The models take days to train Difficult for the programmers to tune the parameters to the model

3 The structure of a CNN Step 1: Serialize the images and store them in a data storage Step 2: Run through the convolutional neural nets batch by batch Feed forward: forward propagate the batch of images through the convolutional neural nets and calculate the accumulated loss Backward propagation: update the parameters starting from the accumulated batch loss calculated in feed forward Step 3: Validate the trained model on the validation set Performs only the feed forward process

4 Structure of a CNN The architecture of a neural network is organized into layers The network is a Directed Acyclic Graph Node: Layer Edge: Dependence between the layers Can have multiple branches going in parallel with multiple loss layers in the end

5 Architecture of a CNN

6 Categories of Layer Types Different Categories of Layer Types in CAFE Data Convolution Pooling Neuron Common Utility Regularization Loss A total of 40+ different layer types

7 Data Layer Purpose: read the images into the database Supports multiple formats: leveldb (protobuf) HDF5 in-memory image Currently it is very slow and single threaded by default It can take an entire day to serialize the images

8 Neuron Layers Different Types of Neurons sigmoid RELU tanh absolute value power binomial normal log likelihood Brings non linearity to the neural nets

9 RELU(Rectified Linear Units) Purpose: The most commonly used neuron layers Function: f(x) = max(0, wx+b) Used to build the artificial neurons, which can be used to build any compute function Other options include sigmoid, tan(h) neurons

10 Convolutional Layer Purpose: performs convolution on the images Parameters num_output: number of output filters (could be around 96) kernel_size: the dimension of the sliding window in the original image stride: number of pixels between adjacent windows weight_filter: bias_filter:

11 Convolutional Layer The most compute intensive stage in the convolutional neural nets takes up to 90% of the total execution time in an unoptimized convolutional neural nets much more expensive than usual neural nets (calculating the cost tutorial/lenet.html )

12 Pooling Partitions the input image into a set of nonoverlapping rectangles and, for each sub-region, outputs the maximum value Eliminates non-maximal values and reduces computation for upper layers Provides a form of translation invariance

13 Regularization A weight decay factor specified in the overall hyper parameter Drop Out Layer deactivate half of the neurons randomly Forward propagate and back propagate Repeat the steps

14 Common Utility Common utility layers Inner Product Splitting Flattening Concatenation Slicing Elementwise Argmax Softmax Mean variance normalization

15 Loss Layer Calculates the loss between the predicted output and the actual output in the output layer Different types of loss layers softmax contrastive loss layer square loss hinge loss inforgain loss multinomial logisttic loss accuracy sigmoid cross entropy

16 Organization of the layers There needs to be a neuron layer after a convolution layer (RELU is the most common neuron layer after convolution) except when multiple convolution are chained together Convolution + relu + pooling (common sequence) There needs to be a loss layer in the end

17 Context CAFE: the most popular library, integrated CuDnN CuDNN: a GPU based implementation by nvidia that claims to be the fastest implementation Cuda-convnet2: a GPU based implementation supported by Google

18 Current Benchmarks measured in a batch of 128 images roughly 2-3 ms per image But each unit of computation is still a batch. There are parallelism within the batch

19 Next steps Understand the problem better How are the layers implemented and optimized? What is the current performance bottleneck? It used to be convolution layer, but we don t know if it is still the bottleneck Understand the tools better Can we use existing tools such as Halide, StreaMIT or Simmit to improve the performance? How? How much performance improvement can we get? Language Design CAFE is a well modularized library. It abstracted out common utility layers. Each layer seems to have its distinct functionality.

20 Source of Parallelism Theoretical Parallelism Each image in the batch can be processed in parallel Possible pipeline parallelism between different stages in the neural network Multiple branches in the neural networks can potentially be processed in parallel I am not sure whether the parallelisms are being exploited in the current implementations

21 More Questions Can we do a GPU/CPU hybrid? Current implementation seems to focus on GPU exclusively Is it possible that some layers are faster in CPU and others on GPU? Is it worthwhile to implement a distributed version? The distribution might increase the latency Are the applications limited by the current performance? Would it help if ML people can use larger images (current images are 128x128)? Would it help if ML people can chain together more compute intensive layers? Would it help ML people if they can build deeper networks that run fast enough? Other questions for Aditya?