Dynamic Routing Between Capsules

Size: px
Start display at page:

Download "Dynamic Routing Between Capsules"

Transcription

1 Report Explainable Machine Learning Dynamic Routing Between Capsules Author: Michael Dorkenwald Supervisor: Dr. Ullrich Köthe 28. Juni 2018

2 Inhaltsverzeichnis 1 Introduction 2 2 Motivation 2 3 CapusleNet Capsules Architecture Routing Algorithm Example How the Routing Algorithm Works Margin Loss Reconstruction Results MNIST Data Set Instantation Parameters Robustness MultiMNIST CIFAR Discussion 9 1

3 1 Introduction Artificial neural networks may be the hottest topic in Machine Learning. In the last few years, there were a lot of new developments that have enhanced neural networks and made them more accessible. However, they were mostly incremental, like adding more layers or improving types of layers like Batch Normalization, but did not introduce a new type of architecture or topic. That is the reason why this paper is really interesting because on the one hand it is published by Geoffrey Hinton and his team and on the other, it introduces a completely new architecture based on capsules. Hinton is one of the founders of deep learning and an inventor of various models and algorithms that are widely used today. He has had the idea of capsules for quite a long time and has finally managed to publish a functioning network that achieves state-of-the-art performance on MNIST. 2 Motivation Convolutional Neural Network (ConvNet or CNN) is a specific type of deep neural networks in which a model learns to perform classification tasks directly from images or videos. They are useful in finding patterns in images to recognize objects, faces, and scenes. ConvNets have been successful in identifying faces, objects and traffic signs, which is an important component for powering vision in robots and self-driving cars. During training the different layers of the Convolutional Neural Network learn some different types of features. The convolutional layers that are closer to the input learn low-level features like edges or color gradients. The convolutional layers which are close to the fully connected layers (output) learn high-level features. These high-level features are combinations of low-level features. The dense layers combine these high-level features and produce a classification task. This is shown in the figure below. Abbildung 1: Differnt Types of Features of a CNN 2 For example, if you want to classify a ship or a horse the innermost layer understands the small curves and edges. The 2nd layer might understand the straight lines or the smaller shapes, like the mast of a ship or the curvature of the entire tail. Higher up layers start understanding more complex shapes like the entire tail or the ship hull. Final layers try to see a more holistic picture like the entire ship or the entire horse. In Convolutional Neural Network we use Max-Pooling to reduce the spatial size of the features. Because of this it computes in a reasonable time and avoids overfitting due to the fact that it decreases the number of neurons in the network. On the other hand, it extracts the dominant features of a specific 2

4 field of view. Thereby we lose the spatial information where this value comes from and because of this, we have a small invariance in the change of the viewpoint. This is illustrated in the following figure. Abbildung 2: Differnt Types of Features of a CNN 3 Max-Pooling is a crutch that makes CNN works really well. When we take a look on the performance of Convolutional Neural Networks we see that the error for nearly all datasets decreases over the last year and achieve superhuman performance in several benchmarks. But the loss of spatial relation is a thorn in the flesh for Geoffrey Hinton. He says that: The pooling operation used in convolutional neural networks is a big mistake and the fact that it works so well is a disaster. 4 Hinton is looking for equivariance, this means changes in viewpoint leads to corresponding changes in neural activities. Therefore we need a different architecture which does not use Max-Pooling layers. 3 CapusleNet 3.1 Capsules A capsule is a small group of neurons that learns to detect a particular object (e.g., a rectangle) within a given region of the image. There are a lot of different ways to implement the basic idea of capsules. They decided that the output vector of a capsule should represent the probability that a specific pattern or object is present in the input image. These specific patterns are the instantiation parameters of a specific entity and is learned from the network. In the results, the individual dimensions of the output vector are shown. They used a squashing function to ensure that the output represents a probability and to have a non-linearity in their network. The difference of this non-linearity compared to one of a neural network like a ReLu or sigmoid is that it is applied to the whole capsule instead of each neuron separately. The squashing is defined as: v j = s j s j 2 s j s j 2 with v j the output vector of capsule j and s j the input. 3.2 Architecture Like regular neural networks, the Capsule Network consists of multiple layers but is shallow which means it consists only of two convolutional layers and one fully connected layer. A simple CapsNet architecture is shown in the figure below. The first layer is a standard convolutional layer with 256 channels and a 9x9 window and is followed from a ReLU activation. The job of the first layer is to detect basic features in the 2D input image. Therefore it transforms the pixel values from the input image to local feature activities which are the input for the first primary capsules. The second layer is a convolutional capsule layer with 32 primary capsules whose job is to combine the basic features which are detected in the first layer. Each capsule applies eight convolutional kernels to the input volume and creates a 6x6x8 output tensor. Since there are 32 such capsules, the output 3

5 volume has the shape of 6x6x8x32. The third and last layer consists of the 10 digit capsules, one for each digit (MNIST dataset). Each digit capsule takes as input the 6x6x32 8D vectors and output for each digit a 16D capsule. Between these two layers, we have a routing algorithm that is where all the magic of this paper appears. Abbildung 3: Architecture of the Capsule Network Routing Algorithm Between the two capsule layers, the routing algorithm is applied. The algorithm follows a specific procedure which can be seen in figure 7. This procedure can also be applied to Capsule Networks with more capsule layers. For all capsule layers except the first one, they apply a weight Matrix W ij to the output of the previous capsule layer, to encode important relationships between lower level features (e.g. mouth, nose) and higher level feature (e.g. face). û j i = W ij u i û j i is the prediction vector which is an estimation what the output of the layer would be and u i is the output of the previous layer. With the help of the prediction vector we can compute the output of the capsule which is a weighted sum over all these vectors s j = i c ij û i j v j = squash(s j ) where the c ij are the coupling coefficients. The ultimate output is v j. At the first glance, this calculation looks similar to the one where neurons weight its inputs before adding them up in fully connected layers. In the neuron case, these weights are updated during backpropagation, but in the case of capsules, they are determined by the dynamic routing process. This can be interpreted as follows: the dynamic routing algorithm determines where each capsule s output goes. The coefficients of the coupling variable are normalized which means that between capsule i and all the capsules in the layer above they sum to 1. Furthermore, they are determined by a routing softmax whose initial logits b ij are the log prior probabilities that capsule i should be coupled to capsule j. c ij = exp(b ij) k exp(b ik) The b ij can be learned at the same time as all other weights. They depend on the region and type of the two capsules but not on the current image. The initial coupling coefficients are iteratively updated depending on the agreement which is defined as: a ij = v j û j i 4

6 Therefore, the agreement score takes into account both likeliness and the feature properties, instead of just likeliness in neurons. Also, b ij remains low if the activation u i of capsule i is low since û j i length is proportional to u i, for example, b ij should remain low between the mouth capsule and the face capsule if the mouth capsule is not activated. The agreement can just be added to the log priors because at the beginning of each iteration b ij will be normalized with the routing softmax. On the MNIST dataset, they showed that 1 to 3 iterations are sufficient. The dynamic routing is not a complete replacement of the backpropagation, because the transformation matrix W ij is still trained with it. We compute c ij to quantify the connection between a capsule and its parent capsules and this value is important but short lived. They re-initialize it to 0 for every data point before the dynamic routing calculation. To calculate a capsule output, training or testing, you always have to redo the dynamic routing calculation. Abbildung 4: Routing Algorithm Example How the Routing Algorithm Works In this example, we focus on a rectangle and a triangle capsule and with them, we can build a boat or a house. Furthermore, we reduce the number of the instantiation parameters to one which represents the rotation. In the figure below you can see in the lowest window the input image which consists of a boat (left) and a house (right). By passing the input image into the primary capsule layer the corresponding capsules gets activated. The length of the vector corresponds to the probability that a rectangle or a triangle is present in the image and the rotation of the vector corresponds to the rotation of the rectangle or triangle. Abbildung 5: 2 layer capsule network 6 For now, we focus only on the boat as an input image and look how the CapsNet proceeds with it. The rectangle and the triangle could be part of a house or a boat. Now we have to take the pose of 5

7 the rectangle into account and the output house or boat would have to be slightly rotated, left side of the figure below. For the triangle the same, because of the rotation of it, the output house would stand almost upside down. As a result, the triangle and the rectangle strongly agree with the object of a boat but disagree with the output of a house. Abbildung 6: predict the presence and pose of objects based on the presence and pose of object parts 6 Since it is very likely that the output is a boat, it would make sense to send the output of the primary capsule more to the boat capsule and less to the house capsule. Because of this, the boat capsule receives a more useful input signal. This is realized by updating the coupling coefficients. Because of the strong agreement, the weights of the boat will be big and the one of the house very small. In two to three iterations the corresponding house capsule (top window fig 5) will shrink to a tiny vector and the boat capsule will be huge which corresponds to a large probability that a boat is present in the input image. 3.5 Margin Loss The length of instantiation vector corresponds to the probability that a specific entity is present in the input image. We would like the top-level capsule for digit class k to have a long instantiation vector if and only if that digit is present in the image. To allow images with multiple labels (more than one digit), they used a separate margin loss, L k for each digit capsule, k: where T k = 1 if a digit of class k is present and m + = 0.9, m = 0.1 and λ = 0.5. The loss function is very similar to the SVM loss function. During training, for each image, one loss value will be calculated for each of the 10 vectors according to the formula above and then the 10 values will be added together to calculate the final loss. This forces the model that the top-level capsule has a probability greater than 0.9 and the rest smaller than Reconstruction They reconstruct the image from the correct digit capsule (16D vector), the other digit capsules are masked out. Therefore, they use a decoder, like a typical one from an autoencoder. The decoder is used as a regularizer, it takes the output of the correct DigitCap as input and learns to recreate a 28 by 28 pixels image, with the loss function being Euclidean distance between the reconstructed image and the input image. As a result, the decoder forces the capsules to learn features that are useful for reconstructing the original image. The reconstruction loss is scaled down by a factor of so that it does not dominate the margin loss. 6

8 Abbildung 7: Decoder Network 1 4 Results They evaluated their capsule network on different data sets and achieved for the MNIST and MultiMNIST data set state of the art performance. The test error for these experiments are in the figure below. Abbildung 8: CapsNet classification test accuracy MNIST Data Set First, they evaluated their model on MNIST a dataset consisting of images with on digit. They only shifted the images by up to 2 pixels in each direction with zero padding as data augmentation. The dataset consists of 60k training examples and 10k test examples. They compared their model only to those which used the same type of data augmentation. The figure above (figure 10) shows the importance of the routing and reconstruction regularizer. Without the reconstruction part, the capsule net with 3 iterations would not be better than the version with one iteration. As a baseline model, they used a CNN (convolutional neural network) with 3 convolutional layers with 256,256 and 128 channels and 5x5 kernels with stride 1. The last convolutional layers are followed by two fully connected layers of size 328, 192. The last fully connected layer is connected with dropout to a 10 class softmax layer with cross-entropy loss. The baseline is also trained on 2-pixel shifted MNIST with Adam optimizer. The capsule network achieved a test error of (0.25%) only on 3 layers similar results are only achieved by deeper networks. 7

9 4.1.1 Instantation Parameters They investigated what each individual dimension of a capsule represents. They feed in a perturbed version of this activity vector and looked how this influenced the reconstructed image. This is shown in the figure below. One dimension of the digit capsule always represents the width of the digit. Other dimensions could represent the localization, scale, and thickness. These are easier to interpret as the layers from a standard convolutional neural network. Abbildung 9: Dimension perturbations Robustness They looked at the robustness of their model and compared it to a traditional convolutional neural network. Therefore, they applied small affine transformations on the MNIST images and investigated the impact of the results. They achieved 79 % accuracy, the traditional CNN with the same number of parameters as the capsule network achieved only 66 %. The models are trained on the normal MNIST dataset and only evaluated on this one. 4.2 MultiMNIST First of all, they had to create this dataset. Therefore, they took every image from the MNIST dataset and overlay it with all digits from a different class. Each digit is shifted up to 4 pixels in each direction resulting in a image. They generate for each digit in the MNIST dataset 1K MultiMNIST examples. As a result, the training set consists of 60 M images and the testing set of 10M. Examples of images can be seen in the figure below in the first row. For this experiment, they used a capsule network Abbildung 10: MultiMNIST 1 8

10 with 3 routing iterations. The two reconstructed digits are overlayed in green and red in the second row. L (l1, l2 ) are the labels for the input image and R (r1,r2) are the labels used for the reconstruction. The two rightmost columns show two images with wrong classification (R) reconstructed from the input label and the predicted label. The other columns have correct classifications and show that the model accounts for all the pixels while being able to assign one pixel to two digits in extremely difficult scenarios. They treated the two most active digit capsules as the classification produced by the capsules network. For the reconstruction, they picked one digit at a time and used the activity vector of the chosen digit capsule to reconstruct the image of the chosen digit. Their 3 layer network achieved a better performance than the baseline model. The baseline models consist of 2 convolutional layers (followed by max-pooling layers) and 2 fully connected layers for classification. The number of parameters is 25 M of the baseline model, for the capsule net only 11M. 4.3 CIFAR10 They evaluated CapsNet on the CIFAR10 dataset and achieved a test error of 10.6 % with an ensemble of 7 models each trained with 3 routing iterations. The architecture is similar to the one for the MNIST dataset, they only increased the number of primary capsule because the input image has 3 channels. They stated that the standard CNN had the same error rate when they were first applied to this dataset. 5 Discussion The authors of the paper have presented a routing algorithm which provides a way how capsules interact with each other. This kind of network could perhaps replace one-day convolutional neural networks. CNN s have become dominant in computer vision tasks for instance object detection, but there are signs that these may be replaced by other networks. One sign is, for instance, the difficulty in generalization to novel viewpoints. If the viewpoint changes in CapsNet the neural activities vary correspondingly rather than eliminating the viewpoint variation from the neural activity like in CNN. On MNIST CapsNet has achieved state-of-the-art performance, but it has not yet been tested on large data sets such as Imagenet. They have also shown that their model produces better results on MultiMNIST as a regular convolutional neural network. The reconstructions illustrated that CapsNet is able to segment the image into the two original digits. This is very promising for later use in object detection. Furthermore, CapsNet also has shown better robustness to affine transformation than a regular convolutional neural network with the same number of parameters. Overall, the activation vectors are easier to interpret, this could be seen in fig 9 and the dimensions represent for instance the thickness, scale or rotation. A problem with this routing algorithm is the time it needs for training, because of the inner loop. However the research on capsule networks are on an early stage and there are good reasons for believing that it is a better approach as the current networks, but it will take a lot of effort to out-perform a highly developed network. 9

11 Literatur [1] Paper Dynamic Routing Between Capsules: [2] CNN Features Image: bengioy/talks/dl-tutorial-nips2015.pdf [3] Max Pooling Image: [4] Quote Geoffrey Hinton: geoffrey hinton/clyj4jv/ [5] AlexNet image: segmentation.html [6] Routing Example: 10

Dynamic Routing Between Capsules. Yiting Ethan Li, Haakon Hukkelaas, and Kaushik Ram Ramasamy

Dynamic Routing Between Capsules. Yiting Ethan Li, Haakon Hukkelaas, and Kaushik Ram Ramasamy Dynamic Routing Between Capsules Yiting Ethan Li, Haakon Hukkelaas, and Kaushik Ram Ramasamy Problems & Results Object classification in images without losing information about important parts of the picture.

More information

Capsule Networks. Eric Mintun

Capsule Networks. Eric Mintun Capsule Networks Eric Mintun Motivation An improvement* to regular Convolutional Neural Networks. Two goals: Replace max-pooling operation with something more intuitive. Keep more info about an activated

More information

Research on Pruning Convolutional Neural Network, Autoencoder and Capsule Network

Research on Pruning Convolutional Neural Network, Autoencoder and Capsule Network Research on Pruning Convolutional Neural Network, Autoencoder and Capsule Network Tianyu Wang Australia National University, Colledge of Engineering and Computer Science u@anu.edu.au Abstract. Some tasks,

More information

Machine Learning 13. week

Machine Learning 13. week Machine Learning 13. week Deep Learning Convolutional Neural Network Recurrent Neural Network 1 Why Deep Learning is so Popular? 1. Increase in the amount of data Thanks to the Internet, huge amount of

More information

Does the Brain do Inverse Graphics?

Does the Brain do Inverse Graphics? Does the Brain do Inverse Graphics? Geoffrey Hinton, Alex Krizhevsky, Navdeep Jaitly, Tijmen Tieleman & Yichuan Tang Department of Computer Science University of Toronto How to learn many layers of features

More information

Does the Brain do Inverse Graphics?

Does the Brain do Inverse Graphics? Does the Brain do Inverse Graphics? Geoffrey Hinton, Alex Krizhevsky, Navdeep Jaitly, Tijmen Tieleman & Yichuan Tang Department of Computer Science University of Toronto The representation used by the

More information

Why equivariance is better than premature invariance

Why equivariance is better than premature invariance 1 Why equivariance is better than premature invariance Geoffrey Hinton Canadian Institute for Advanced Research & Department of Computer Science University of Toronto with contributions from Sida Wang

More information

Deep Learning with Tensorflow AlexNet

Deep Learning with Tensorflow   AlexNet Machine Learning and Computer Vision Group Deep Learning with Tensorflow http://cvml.ist.ac.at/courses/dlwt_w17/ AlexNet Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton, "Imagenet classification

More information

COMP 551 Applied Machine Learning Lecture 16: Deep Learning

COMP 551 Applied Machine Learning Lecture 16: Deep Learning COMP 551 Applied Machine Learning Lecture 16: Deep Learning Instructor: Ryan Lowe (ryan.lowe@cs.mcgill.ca) Slides mostly by: Class web page: www.cs.mcgill.ca/~hvanho2/comp551 Unless otherwise noted, all

More information

Deep Learning for Computer Vision II

Deep Learning for Computer Vision II IIIT Hyderabad Deep Learning for Computer Vision II C. V. Jawahar Paradigm Shift Feature Extraction (SIFT, HoG, ) Part Models / Encoding Classifier Sparrow Feature Learning Classifier Sparrow L 1 L 2 L

More information

Exploring Capsules. Binghui Peng Runzhou Tao Shunyu Yao IIIS, Tsinghua University {pbh15, trz15,

Exploring Capsules. Binghui Peng Runzhou Tao Shunyu Yao IIIS, Tsinghua University {pbh15, trz15, Exploring Capsules Binghui Peng Runzhou Tao Shunyu Yao IIIS, Tsinghua University {pbh15, trz15, yao-sy15}@mails.tsinghua.edu.cn 1 Introduction Nowadays, convolutional neural networks (CNNs) have received

More information

Using Capsule Networks. for Image and Speech Recognition Problems. Yan Xiong

Using Capsule Networks. for Image and Speech Recognition Problems. Yan Xiong Using Capsule Networks for Image and Speech Recognition Problems by Yan Xiong A Thesis Presented in Partial Fulfillment of the Requirements for the Degree Master of Science Approved November 2018 by the

More information

COMP9444 Neural Networks and Deep Learning 7. Image Processing. COMP9444 c Alan Blair, 2017

COMP9444 Neural Networks and Deep Learning 7. Image Processing. COMP9444 c Alan Blair, 2017 COMP9444 Neural Networks and Deep Learning 7. Image Processing COMP9444 17s2 Image Processing 1 Outline Image Datasets and Tasks Convolution in Detail AlexNet Weight Initialization Batch Normalization

More information

Know your data - many types of networks

Know your data - many types of networks Architectures Know your data - many types of networks Fixed length representation Variable length representation Online video sequences, or samples of different sizes Images Specific architectures for

More information

Machine Learning. Deep Learning. Eric Xing (and Pengtao Xie) , Fall Lecture 8, October 6, Eric CMU,

Machine Learning. Deep Learning. Eric Xing (and Pengtao Xie) , Fall Lecture 8, October 6, Eric CMU, Machine Learning 10-701, Fall 2015 Deep Learning Eric Xing (and Pengtao Xie) Lecture 8, October 6, 2015 Eric Xing @ CMU, 2015 1 A perennial challenge in computer vision: feature engineering SIFT Spin image

More information

Neural Network and Deep Learning. Donglin Zeng, Department of Biostatistics, University of North Carolina

Neural Network and Deep Learning. Donglin Zeng, Department of Biostatistics, University of North Carolina Neural Network and Deep Learning Early history of deep learning Deep learning dates back to 1940s: known as cybernetics in the 1940s-60s, connectionism in the 1980s-90s, and under the current name starting

More information

Improving the way neural networks learn Srikumar Ramalingam School of Computing University of Utah

Improving the way neural networks learn Srikumar Ramalingam School of Computing University of Utah Improving the way neural networks learn Srikumar Ramalingam School of Computing University of Utah Reference Most of the slides are taken from the third chapter of the online book by Michael Nielson: neuralnetworksanddeeplearning.com

More information

Convolutional Neural Networks

Convolutional Neural Networks Lecturer: Barnabas Poczos Introduction to Machine Learning (Lecture Notes) Convolutional Neural Networks Disclaimer: These notes have not been subjected to the usual scrutiny reserved for formal publications.

More information

Perceptron: This is convolution!

Perceptron: This is convolution! Perceptron: This is convolution! v v v Shared weights v Filter = local perceptron. Also called kernel. By pooling responses at different locations, we gain robustness to the exact spatial location of image

More information

arxiv: v2 [cs.cv] 7 Nov 2017

arxiv: v2 [cs.cv] 7 Nov 2017 Dynamic Routing Between Capsules Sara Sabour Nicholas Frosst arxiv:1710.09829v2 [cs.cv] 7 Nov 2017 Geoffrey E. Hinton Google Brain Toronto {sasabour, frosst, geoffhinton}@google.com Abstract A capsule

More information

Deep Learning. Visualizing and Understanding Convolutional Networks. Christopher Funk. Pennsylvania State University.

Deep Learning. Visualizing and Understanding Convolutional Networks. Christopher Funk. Pennsylvania State University. Visualizing and Understanding Convolutional Networks Christopher Pennsylvania State University February 23, 2015 Some Slide Information taken from Pierre Sermanet (Google) presentation on and Computer

More information

Study of Residual Networks for Image Recognition

Study of Residual Networks for Image Recognition Study of Residual Networks for Image Recognition Mohammad Sadegh Ebrahimi Stanford University sadegh@stanford.edu Hossein Karkeh Abadi Stanford University hosseink@stanford.edu Abstract Deep neural networks

More information

INTRODUCTION TO DEEP LEARNING

INTRODUCTION TO DEEP LEARNING INTRODUCTION TO DEEP LEARNING CONTENTS Introduction to deep learning Contents 1. Examples 2. Machine learning 3. Neural networks 4. Deep learning 5. Convolutional neural networks 6. Conclusion 7. Additional

More information

Rotation Invariance Neural Network

Rotation Invariance Neural Network Rotation Invariance Neural Network Shiyuan Li Abstract Rotation invariance and translate invariance have great values in image recognition. In this paper, we bring a new architecture in convolutional neural

More information

Deep Learning Workshop. Nov. 20, 2015 Andrew Fishberg, Rowan Zellers

Deep Learning Workshop. Nov. 20, 2015 Andrew Fishberg, Rowan Zellers Deep Learning Workshop Nov. 20, 2015 Andrew Fishberg, Rowan Zellers Why deep learning? The ImageNet Challenge Goal: image classification with 1000 categories Top 5 error rate of 15%. Krizhevsky, Alex,

More information

Convolutional Neural Networks. Computer Vision Jia-Bin Huang, Virginia Tech

Convolutional Neural Networks. Computer Vision Jia-Bin Huang, Virginia Tech Convolutional Neural Networks Computer Vision Jia-Bin Huang, Virginia Tech Today s class Overview Convolutional Neural Network (CNN) Training CNN Understanding and Visualizing CNN Image Categorization:

More information

Deep Learning. Vladimir Golkov Technical University of Munich Computer Vision Group

Deep Learning. Vladimir Golkov Technical University of Munich Computer Vision Group Deep Learning Vladimir Golkov Technical University of Munich Computer Vision Group 1D Input, 1D Output target input 2 2D Input, 1D Output: Data Distribution Complexity Imagine many dimensions (data occupies

More information

CMU Lecture 18: Deep learning and Vision: Convolutional neural networks. Teacher: Gianni A. Di Caro

CMU Lecture 18: Deep learning and Vision: Convolutional neural networks. Teacher: Gianni A. Di Caro CMU 15-781 Lecture 18: Deep learning and Vision: Convolutional neural networks Teacher: Gianni A. Di Caro DEEP, SHALLOW, CONNECTED, SPARSE? Fully connected multi-layer feed-forward perceptrons: More powerful

More information

Neural Networks for unsupervised learning From Principal Components Analysis to Autoencoders to semantic hashing

Neural Networks for unsupervised learning From Principal Components Analysis to Autoencoders to semantic hashing Neural Networks for unsupervised learning From Principal Components Analysis to Autoencoders to semantic hashing feature 3 PC 3 Beate Sick Many slides are taken form Hinton s great lecture on NN: https://www.coursera.org/course/neuralnets

More information

Report: Privacy-Preserving Classification on Deep Neural Network

Report: Privacy-Preserving Classification on Deep Neural Network Report: Privacy-Preserving Classification on Deep Neural Network Janno Veeorg Supervised by Helger Lipmaa and Raul Vicente Zafra May 25, 2017 1 Introduction In this report we consider following task: how

More information

Convolutional Neural Networks

Convolutional Neural Networks NPFL114, Lecture 4 Convolutional Neural Networks Milan Straka March 25, 2019 Charles University in Prague Faculty of Mathematics and Physics Institute of Formal and Applied Linguistics unless otherwise

More information

CPSC 340: Machine Learning and Data Mining. Deep Learning Fall 2016

CPSC 340: Machine Learning and Data Mining. Deep Learning Fall 2016 CPSC 340: Machine Learning and Data Mining Deep Learning Fall 2016 Assignment 5: Due Friday. Assignment 6: Due next Friday. Final: Admin December 12 (8:30am HEBB 100) Covers Assignments 1-6. Final from

More information

Deep Learning. Volker Tresp Summer 2014

Deep Learning. Volker Tresp Summer 2014 Deep Learning Volker Tresp Summer 2014 1 Neural Network Winter and Revival While Machine Learning was flourishing, there was a Neural Network winter (late 1990 s until late 2000 s) Around 2010 there

More information

Deep neural networks II

Deep neural networks II Deep neural networks II May 31 st, 2018 Yong Jae Lee UC Davis Many slides from Rob Fergus, Svetlana Lazebnik, Jia-Bin Huang, Derek Hoiem, Adriana Kovashka, Why (convolutional) neural networks? State of

More information

An Exploration of Computer Vision Techniques for Bird Species Classification

An Exploration of Computer Vision Techniques for Bird Species Classification An Exploration of Computer Vision Techniques for Bird Species Classification Anne L. Alter, Karen M. Wang December 15, 2017 Abstract Bird classification, a fine-grained categorization task, is a complex

More information

Multi-Glance Attention Models For Image Classification

Multi-Glance Attention Models For Image Classification Multi-Glance Attention Models For Image Classification Chinmay Duvedi Stanford University Stanford, CA cduvedi@stanford.edu Pararth Shah Stanford University Stanford, CA pararth@stanford.edu Abstract We

More information

Deep Model Compression

Deep Model Compression Deep Model Compression Xin Wang Oct.31.2016 Some of the contents are borrowed from Hinton s and Song s slides. Two papers Distilling the Knowledge in a Neural Network by Geoffrey Hinton et al What s the

More information

Deep Learning Explained Module 4: Convolution Neural Networks (CNN or Conv Nets)

Deep Learning Explained Module 4: Convolution Neural Networks (CNN or Conv Nets) Deep Learning Explained Module 4: Convolution Neural Networks (CNN or Conv Nets) Sayan D. Pathak, Ph.D., Principal ML Scientist, Microsoft Roland Fernandez, Senior Researcher, Microsoft Module Outline

More information

Using Machine Learning for Classification of Cancer Cells

Using Machine Learning for Classification of Cancer Cells Using Machine Learning for Classification of Cancer Cells Camille Biscarrat University of California, Berkeley I Introduction Cell screening is a commonly used technique in the development of new drugs.

More information

Inception and Residual Networks. Hantao Zhang. Deep Learning with Python.

Inception and Residual Networks. Hantao Zhang. Deep Learning with Python. Inception and Residual Networks Hantao Zhang Deep Learning with Python https://en.wikipedia.org/wiki/residual_neural_network Deep Neural Network Progress from Large Scale Visual Recognition Challenge (ILSVRC)

More information

Weighted Convolutional Neural Network. Ensemble.

Weighted Convolutional Neural Network. Ensemble. Weighted Convolutional Neural Network Ensemble Xavier Frazão and Luís A. Alexandre Dept. of Informatics, Univ. Beira Interior and Instituto de Telecomunicações Covilhã, Portugal xavierfrazao@gmail.com

More information

Machine Learning. MGS Lecture 3: Deep Learning

Machine Learning. MGS Lecture 3: Deep Learning Dr Michel F. Valstar http://cs.nott.ac.uk/~mfv/ Machine Learning MGS Lecture 3: Deep Learning Dr Michel F. Valstar http://cs.nott.ac.uk/~mfv/ WHAT IS DEEP LEARNING? Shallow network: Only one hidden layer

More information

Deep Convolutional Neural Networks. Nov. 20th, 2015 Bruce Draper

Deep Convolutional Neural Networks. Nov. 20th, 2015 Bruce Draper Deep Convolutional Neural Networks Nov. 20th, 2015 Bruce Draper Background: Fully-connected single layer neural networks Feed-forward classification Trained through back-propagation Example Computer Vision

More information

Object Recognition in 3D data using Capsules

Object Recognition in 3D data using Capsules Syracuse University SURFACE Theses - ALL May 2018 Object Recognition in 3D data using Capsules Ayesha Ahmad Syracuse University Follow this and additional works at: https://surface.syr.edu/thesis Part

More information

Deep Learning. Deep Learning provided breakthrough results in speech recognition and image classification. Why?

Deep Learning. Deep Learning provided breakthrough results in speech recognition and image classification. Why? Data Mining Deep Learning Deep Learning provided breakthrough results in speech recognition and image classification. Why? Because Speech recognition and image classification are two basic examples of

More information

Deep Face Recognition. Nathan Sun

Deep Face Recognition. Nathan Sun Deep Face Recognition Nathan Sun Why Facial Recognition? Picture ID or video tracking Higher Security for Facial Recognition Software Immensely useful to police in tracking suspects Your face will be an

More information

Intro to Deep Learning. Slides Credit: Andrej Karapathy, Derek Hoiem, Marc Aurelio, Yann LeCunn

Intro to Deep Learning. Slides Credit: Andrej Karapathy, Derek Hoiem, Marc Aurelio, Yann LeCunn Intro to Deep Learning Slides Credit: Andrej Karapathy, Derek Hoiem, Marc Aurelio, Yann LeCunn Why this class? Deep Features Have been able to harness the big data in the most efficient and effective

More information

Deep Learning for Computer Vision

Deep Learning for Computer Vision Deep Learning for Computer Vision Lecture 7: Universal Approximation Theorem, More Hidden Units, Multi-Class Classifiers, Softmax, and Regularization Peter Belhumeur Computer Science Columbia University

More information

A Deep Learning Approach to Vehicle Speed Estimation

A Deep Learning Approach to Vehicle Speed Estimation A Deep Learning Approach to Vehicle Speed Estimation Benjamin Penchas bpenchas@stanford.edu Tobin Bell tbell@stanford.edu Marco Monteiro marcorm@stanford.edu ABSTRACT Given car dashboard video footage,

More information

A Quick Guide on Training a neural network using Keras.

A Quick Guide on Training a neural network using Keras. A Quick Guide on Training a neural network using Keras. TensorFlow and Keras Keras Open source High level, less flexible Easy to learn Perfect for quick implementations Starts by François Chollet from

More information

INF 5860 Machine learning for image classification. Lecture 11: Visualization Anne Solberg April 4, 2018

INF 5860 Machine learning for image classification. Lecture 11: Visualization Anne Solberg April 4, 2018 INF 5860 Machine learning for image classification Lecture 11: Visualization Anne Solberg April 4, 2018 Reading material The lecture is based on papers: Deep Dream: https://research.googleblog.com/2015/06/inceptionism-goingdeeper-into-neural.html

More information

Dynamic Routing Using Inter Capsule Routing Protocol Between Capsules

Dynamic Routing Using Inter Capsule Routing Protocol Between Capsules 2018 UKSim-AMSS 20th International Conference on Modelling & Simulation Dynamic Routing Using Inter Capsule Routing Protocol Between Capsules Sanjib Kumar Sahu GGS Indraprastha University Delhi, India,

More information

Su et al. Shape Descriptors - III

Su et al. Shape Descriptors - III Su et al. Shape Descriptors - III Siddhartha Chaudhuri http://www.cse.iitb.ac.in/~cs749 Funkhouser; Feng, Liu, Gong Recap Global A shape descriptor is a set of numbers that describes a shape in a way that

More information

Neural Networks for Machine Learning. Lecture 15a From Principal Components Analysis to Autoencoders

Neural Networks for Machine Learning. Lecture 15a From Principal Components Analysis to Autoencoders Neural Networks for Machine Learning Lecture 15a From Principal Components Analysis to Autoencoders Geoffrey Hinton Nitish Srivastava, Kevin Swersky Tijmen Tieleman Abdel-rahman Mohamed Principal Components

More information

EP-CapsNet: Extending Capsule Network with Inception Module for Electrophoresis Binary Classification

EP-CapsNet: Extending Capsule Network with Inception Module for Electrophoresis Binary Classification EP-CapsNet: Extending Capsule Network with Inception Module for Electrophoresis Binary Classification Elizabeth Tobing, Ashraf Murtaza, Keejun Han, Mun Y. Yi Graduate School of Knowledge Service Engineering

More information

Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization. Presented by: Karen Lucknavalai and Alexandr Kuznetsov

Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization. Presented by: Karen Lucknavalai and Alexandr Kuznetsov Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization Presented by: Karen Lucknavalai and Alexandr Kuznetsov Example Style Content Result Motivation Transforming content of an image

More information

3D model classification using convolutional neural network

3D model classification using convolutional neural network 3D model classification using convolutional neural network JunYoung Gwak Stanford jgwak@cs.stanford.edu Abstract Our goal is to classify 3D models directly using convolutional neural network. Most of existing

More information

Fuzzy Set Theory in Computer Vision: Example 3, Part II

Fuzzy Set Theory in Computer Vision: Example 3, Part II Fuzzy Set Theory in Computer Vision: Example 3, Part II Derek T. Anderson and James M. Keller FUZZ-IEEE, July 2017 Overview Resource; CS231n: Convolutional Neural Networks for Visual Recognition https://github.com/tuanavu/stanford-

More information

CENG 783. Special topics in. Deep Learning. AlchemyAPI. Week 11. Sinan Kalkan

CENG 783. Special topics in. Deep Learning. AlchemyAPI. Week 11. Sinan Kalkan CENG 783 Special topics in Deep Learning AlchemyAPI Week 11 Sinan Kalkan TRAINING A CNN Fig: http://www.robots.ox.ac.uk/~vgg/practicals/cnn/ Feed-forward pass Note that this is written in terms of the

More information

Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs Supplementary Material

Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs Supplementary Material Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs Supplementary Material Peak memory usage, GB 10 1 0.1 0.01 OGN Quadratic Dense Cubic Iteration time, s 10

More information

CS 6501: Deep Learning for Computer Graphics. Training Neural Networks II. Connelly Barnes

CS 6501: Deep Learning for Computer Graphics. Training Neural Networks II. Connelly Barnes CS 6501: Deep Learning for Computer Graphics Training Neural Networks II Connelly Barnes Overview Preprocessing Initialization Vanishing/exploding gradients problem Batch normalization Dropout Additional

More information

Classifying Depositional Environments in Satellite Images

Classifying Depositional Environments in Satellite Images Classifying Depositional Environments in Satellite Images Alex Miltenberger and Rayan Kanfar Department of Geophysics School of Earth, Energy, and Environmental Sciences Stanford University 1 Introduction

More information

Classification of objects from Video Data (Group 30)

Classification of objects from Video Data (Group 30) Classification of objects from Video Data (Group 30) Sheallika Singh 12665 Vibhuti Mahajan 12792 Aahitagni Mukherjee 12001 M Arvind 12385 1 Motivation Video surveillance has been employed for a long time

More information

Overall Description. Goal: to improve spatial invariance to the input data. Translation, Rotation, Scale, Clutter, Elastic

Overall Description. Goal: to improve spatial invariance to the input data. Translation, Rotation, Scale, Clutter, Elastic Philippe Giguère Overall Description Goal: to improve spatial invariance to the input data Translation, Rotation, Scale, Clutter, Elastic How: add a learnable module which explicitly manipulate spatially

More information

Ryerson University CP8208. Soft Computing and Machine Intelligence. Naive Road-Detection using CNNS. Authors: Sarah Asiri - Domenic Curro

Ryerson University CP8208. Soft Computing and Machine Intelligence. Naive Road-Detection using CNNS. Authors: Sarah Asiri - Domenic Curro Ryerson University CP8208 Soft Computing and Machine Intelligence Naive Road-Detection using CNNS Authors: Sarah Asiri - Domenic Curro April 24 2016 Contents 1 Abstract 2 2 Introduction 2 3 Motivation

More information

CSE 559A: Computer Vision

CSE 559A: Computer Vision CSE 559A: Computer Vision Fall 2018: T-R: 11:30-1pm @ Lopata 101 Instructor: Ayan Chakrabarti (ayan@wustl.edu). Course Staff: Zhihao Xia, Charlie Wu, Han Liu http://www.cse.wustl.edu/~ayan/courses/cse559a/

More information

Restricted Boltzmann Machines. Shallow vs. deep networks. Stacked RBMs. Boltzmann Machine learning: Unsupervised version

Restricted Boltzmann Machines. Shallow vs. deep networks. Stacked RBMs. Boltzmann Machine learning: Unsupervised version Shallow vs. deep networks Restricted Boltzmann Machines Shallow: one hidden layer Features can be learned more-or-less independently Arbitrary function approximator (with enough hidden units) Deep: two

More information

Quantifying Translation-Invariance in Convolutional Neural Networks

Quantifying Translation-Invariance in Convolutional Neural Networks Quantifying Translation-Invariance in Convolutional Neural Networks Eric Kauderer-Abrams Stanford University 450 Serra Mall, Stanford, CA 94305 ekabrams@stanford.edu Abstract A fundamental problem in object

More information

Tutorial on Machine Learning Tools

Tutorial on Machine Learning Tools Tutorial on Machine Learning Tools Yanbing Xue Milos Hauskrecht Why do we need these tools? Widely deployed classical models No need to code from scratch Easy-to-use GUI Outline Matlab Apps Weka 3 UI TensorFlow

More information

Computer Vision Lecture 16

Computer Vision Lecture 16 Computer Vision Lecture 16 Deep Learning for Object Categorization 14.01.2016 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Announcements Seminar registration period

More information

Structured Prediction using Convolutional Neural Networks

Structured Prediction using Convolutional Neural Networks Overview Structured Prediction using Convolutional Neural Networks Bohyung Han bhhan@postech.ac.kr Computer Vision Lab. Convolutional Neural Networks (CNNs) Structured predictions for low level computer

More information

arxiv: v1 [cs.cv] 20 Dec 2016

arxiv: v1 [cs.cv] 20 Dec 2016 End-to-End Pedestrian Collision Warning System based on a Convolutional Neural Network with Semantic Segmentation arxiv:1612.06558v1 [cs.cv] 20 Dec 2016 Heechul Jung heechul@dgist.ac.kr Min-Kook Choi mkchoi@dgist.ac.kr

More information

CS 523: Multimedia Systems

CS 523: Multimedia Systems CS 523: Multimedia Systems Angus Forbes creativecoding.evl.uic.edu/courses/cs523 Today - Convolutional Neural Networks - Work on Project 1 http://playground.tensorflow.org/ Convolutional Neural Networks

More information

Inception Network Overview. David White CS793

Inception Network Overview. David White CS793 Inception Network Overview David White CS793 So, Leonardo DiCaprio dreams about dreaming... https://m.media-amazon.com/images/m/mv5bmjaxmzy3njcxnf5bml5banbnxkftztcwnti5otm0mw@@._v1_sy1000_cr0,0,675,1 000_AL_.jpg

More information

Deep Residual Learning

Deep Residual Learning Deep Residual Learning MSRA @ ILSVRC & COCO 2015 competitions Kaiming He with Xiangyu Zhang, Shaoqing Ren, Jifeng Dai, & Jian Sun Microsoft Research Asia (MSRA) MSRA @ ILSVRC & COCO 2015 Competitions 1st

More information

Pouya Kousha Fall 2018 CSE 5194 Prof. DK Panda

Pouya Kousha Fall 2018 CSE 5194 Prof. DK Panda Pouya Kousha Fall 2018 CSE 5194 Prof. DK Panda 1 Observe novel applicability of DL techniques in Big Data Analytics. Applications of DL techniques for common Big Data Analytics problems. Semantic indexing

More information

Channel Locality Block: A Variant of Squeeze-and-Excitation

Channel Locality Block: A Variant of Squeeze-and-Excitation Channel Locality Block: A Variant of Squeeze-and-Excitation 1 st Huayu Li Northern Arizona University Flagstaff, United State Northern Arizona University hl459@nau.edu arxiv:1901.01493v1 [cs.lg] 6 Jan

More information

Ship Classification Using an Image Dataset

Ship Classification Using an Image Dataset Ship Classification Using an Image Dataset Okan Atalar (okan@stanford.edu), Burak Bartan (bbartan@stanford.edu) Abstract In this project, we developed three different sets of classification algorithms

More information

Ensemble methods in machine learning. Example. Neural networks. Neural networks

Ensemble methods in machine learning. Example. Neural networks. Neural networks Ensemble methods in machine learning Bootstrap aggregating (bagging) train an ensemble of models based on randomly resampled versions of the training set, then take a majority vote Example What if you

More information

Plankton Classification Using ConvNets

Plankton Classification Using ConvNets Plankton Classification Using ConvNets Abhinav Rastogi Stanford University Stanford, CA arastogi@stanford.edu Haichuan Yu Stanford University Stanford, CA haichuan@stanford.edu Abstract We present the

More information

ImageNet Classification with Deep Convolutional Neural Networks

ImageNet Classification with Deep Convolutional Neural Networks ImageNet Classification with Deep Convolutional Neural Networks Alex Krizhevsky Ilya Sutskever Geoffrey Hinton University of Toronto Canada Paper with same name to appear in NIPS 2012 Main idea Architecture

More information

PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space

PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space Sikai Zhong February 14, 2018 COMPUTER SCIENCE Table of contents 1. PointNet 2. PointNet++ 3. Experiments 1 PointNet Property

More information

Convolutional Layer Pooling Layer Fully Connected Layer Regularization

Convolutional Layer Pooling Layer Fully Connected Layer Regularization Semi-Parallel Deep Neural Networks (SPDNN), Convergence and Generalization Shabab Bazrafkan, Peter Corcoran Center for Cognitive, Connected & Computational Imaging, College of Engineering & Informatics,

More information

Deep Learning Benchmarks Mumtaz Vauhkonen, Quaizar Vohra, Saurabh Madaan Collaboration with Adam Coates, Stanford Unviersity

Deep Learning Benchmarks Mumtaz Vauhkonen, Quaizar Vohra, Saurabh Madaan Collaboration with Adam Coates, Stanford Unviersity Deep Learning Benchmarks Mumtaz Vauhkonen, Quaizar Vohra, Saurabh Madaan Collaboration with Adam Coates, Stanford Unviersity Abstract: This project aims at creating a benchmark for Deep Learning (DL) algorithms

More information

Multi-Task Self-Supervised Visual Learning

Multi-Task Self-Supervised Visual Learning Multi-Task Self-Supervised Visual Learning Sikai Zhong March 4, 2018 COMPUTER SCIENCE Table of contents 1. Introduction 2. Self-supervised Tasks 3. Architectures 4. Experiments 1 Introduction Self-supervised

More information

6. Convolutional Neural Networks

6. Convolutional Neural Networks 6. Convolutional Neural Networks CS 519 Deep Learning, Winter 2017 Fuxin Li With materials from Zsolt Kira Quiz coming up Next Thursday (2/2) 20 minutes Topics: Optimization Basic neural networks No Convolutional

More information

ECE 5470 Classification, Machine Learning, and Neural Network Review

ECE 5470 Classification, Machine Learning, and Neural Network Review ECE 5470 Classification, Machine Learning, and Neural Network Review Due December 1. Solution set Instructions: These questions are to be answered on this document which should be submitted to blackboard

More information

Smart Content Recognition from Images Using a Mixture of Convolutional Neural Networks *

Smart Content Recognition from Images Using a Mixture of Convolutional Neural Networks * Smart Content Recognition from Images Using a Mixture of Convolutional Neural Networks * Tee Connie *, Mundher Al-Shabi *, and Michael Goh Faculty of Information Science and Technology, Multimedia University,

More information

CS230: Lecture 3 Various Deep Learning Topics

CS230: Lecture 3 Various Deep Learning Topics CS230: Lecture 3 Various Deep Learning Topics Kian Katanforoosh, Andrew Ng Today s outline We will learn how to: - Analyse a problem from a deep learning approach - Choose an architecture - Choose a loss

More information

CS 1674: Intro to Computer Vision. Neural Networks. Prof. Adriana Kovashka University of Pittsburgh November 16, 2016

CS 1674: Intro to Computer Vision. Neural Networks. Prof. Adriana Kovashka University of Pittsburgh November 16, 2016 CS 1674: Intro to Computer Vision Neural Networks Prof. Adriana Kovashka University of Pittsburgh November 16, 2016 Announcements Please watch the videos I sent you, if you haven t yet (that s your reading)

More information

MoonRiver: Deep Neural Network in C++

MoonRiver: Deep Neural Network in C++ MoonRiver: Deep Neural Network in C++ Chung-Yi Weng Computer Science & Engineering University of Washington chungyi@cs.washington.edu Abstract Artificial intelligence resurges with its dramatic improvement

More information

Object Detection Lecture Introduction to deep learning (CNN) Idar Dyrdal

Object Detection Lecture Introduction to deep learning (CNN) Idar Dyrdal Object Detection Lecture 10.3 - Introduction to deep learning (CNN) Idar Dyrdal Deep Learning Labels Computational models composed of multiple processing layers (non-linear transformations) Used to learn

More information

Lecture 2 Notes. Outline. Neural Networks. The Big Idea. Architecture. Instructors: Parth Shah, Riju Pahwa

Lecture 2 Notes. Outline. Neural Networks. The Big Idea. Architecture. Instructors: Parth Shah, Riju Pahwa Instructors: Parth Shah, Riju Pahwa Lecture 2 Notes Outline 1. Neural Networks The Big Idea Architecture SGD and Backpropagation 2. Convolutional Neural Networks Intuition Architecture 3. Recurrent Neural

More information

Deep Learning for Embedded Security Evaluation

Deep Learning for Embedded Security Evaluation Deep Learning for Embedded Security Evaluation Emmanuel Prouff 1 1 Laboratoire de Sécurité des Composants, ANSSI, France April 2018, CISCO April 2018, CISCO E. Prouff 1/22 Contents 1. Context and Motivation

More information

Fig 2.1 Typical prediction results by linear regression with HOG

Fig 2.1 Typical prediction results by linear regression with HOG Facial Keypoints Detection Hao Zhang Jia Chen Nitin Agarwal (hzhang10, 61526092) (jiac5, 35724702) (agarwal,84246130) 1. Introduction The objective of the project is to detect the location of keypoints

More information

Vulnerability of machine learning models to adversarial examples

Vulnerability of machine learning models to adversarial examples Vulnerability of machine learning models to adversarial examples Petra Vidnerová Institute of Computer Science The Czech Academy of Sciences Hora Informaticae 1 Outline Introduction Works on adversarial

More information

Alternatives to Direct Supervision

Alternatives to Direct Supervision CreativeAI: Deep Learning for Graphics Alternatives to Direct Supervision Niloy Mitra Iasonas Kokkinos Paul Guerrero Nils Thuerey Tobias Ritschel UCL UCL UCL TUM UCL Timetable Theory and Basics State of

More information

Convolutional Neural Network for Facial Expression Recognition

Convolutional Neural Network for Facial Expression Recognition Convolutional Neural Network for Facial Expression Recognition Liyuan Zheng Department of Electrical Engineering University of Washington liyuanz8@uw.edu Shifeng Zhu Department of Electrical Engineering

More information

Convolutional Neural Networks for Facial Expression Recognition

Convolutional Neural Networks for Facial Expression Recognition Convolutional Neural Networks for Facial Expression Recognition Shima Alizadeh Stanford University shima86@stanford.edu Azar Fazel Stanford University azarf@stanford.edu Abstract In this project, we have

More information

Lecture 37: ConvNets (Cont d) and Training

Lecture 37: ConvNets (Cont d) and Training Lecture 37: ConvNets (Cont d) and Training CS 4670/5670 Sean Bell [http://bbabenko.tumblr.com/post/83319141207/convolutional-learnings-things-i-learned-by] (Unrelated) Dog vs Food [Karen Zack, @teenybiscuit]

More information