Transfer Learning. Style Transfer in Deep Learning
|
|
- Chastity Andrews
- 6 years ago
- Views:
Transcription
1 Transfer Learning & Style Transfer in Deep Learning 4-DEC-2016 Gal Barzilai, Ram Machlev Deep Learning Seminar School of Electrical Engineering Tel Aviv University
2 Part 1: Transfer Learning in Deep Learning 6-OCT-2013 (976 cited) Yangqing Jia, author of Caffe and DeCAF.
3 One of The Main Problems in Deep Learning Approaches: with limited training data, fully-supervised deep architectures generally overfit many visual recognition challenges have tasks with few training examples 3
4 Transfer Learning Concept learning the features on large-scale data in a supervised setting, then transferring them to different tasks with different labels. Input A (for example: cars) Task A Transfer Input B (for example: trucks) Layer n AnB: Frozen Weights New Learned Weights Back-propagation Task B 4
5 Accuracy experiments in : J Yosinski Input A- 500 Classes from ImageNet Layer 3 Task A Transfer Input B- 500 Classes from ImageNet A3B: Frozen Weights Back-propagation Task B 5
6 DeCAF Approach- Deep convolutional representations are learned on a set of related problems but applied to new tasks which have too few training examples to learn a full deep representation. The model can be considered as : deep architecture for transfer learning based on a supervised pre-training phase. Or simply as- convolutional network weights learned on a set of pre-defined object recognition tasks. 6
7 Adopted Network Deep CNN architecture proposed by Krizhevsky [Krizhevsky NIPS 2012]. 5 convolutional layers (with pooling and ReLU) 3 fully-connected layers won ImageNet Large Scale Visual recognition Challenge (ILSVRC) 2012 (10,000,000 labeled images depicting 10,000+ object categories) as training top-1 validation error rate of 40.7% follow architecture and training protocol with two differences input 256 x 256 images rather than 224 x 224 images no data augmentation trick 7
8 DeCAF1 DeCAF2 DeCAF3 DeCAF4 DeCAF5 DeCAF6 DeCAF Activations of The nth Hidden Layer of The Deep Convolutional Neural Network As a Feature DeCAFn. 8
9 Feature Generalization and Visualization Features that were compared : ImageNet Training Database (ILVRC-2012) GIST Features GIST features LLC features Feature extraction Krizhevsky s net DeCAFn LLC Features Known Known features features extraction extraction approach approach (Locality-constrained Linear Coding. J Wang, 2010) t-sne map t-sne map (a low dimensional representation of the scene, not require any form of segmentation, Oliva A and Torralba 2001) visualize features in the following way: run t- SNE algorithm - a 2-dimensional embedding of the highdimensional feature space. plot features as points colored depending on their semantic 9 category.
10 t-sne feature visualizations on the ILSVRC-2012 validation set. (after trained on ILSVRC-2012 training set, prevent overfitting) LLC FEATURES GIST FEATURES GIST or LLC fail to capture the semantic difference between images 11
11 t-sne feature visualizations on the ILSVRC-2012 validation set. (after trained on ILSVRC-2012 training set, prevent overfitting) DeCAF1 DeCAF6 DeCAF 1 FEATURES first layers learn low-level features DeCAF 6 FEATURES latter layers learn semantic or high level features. 12
12 DeCAF 6 features trained on ILSVRC-2012 generalized to SUN-397 SUN-397: Large-scale scene recognition from abbey to zoo. (899 categories and 130,519 images) Different semantic categories 13
13 Experiments all the network s hidden layer weights are frozen to those learned on the ILSVRC-2012 dataset. New dataset Krizhevsky s net After train on ILVRC linear Classifier Frozen weights Activation feature Of new dataset Train New Task Results on multiple datasets to evaluate the strength of DeCAF for Object recognition Domain adaptation subcategory recognition scene recognition each task differ somewhat from that for which the architecture was trained. 14
14 Experiment: Object Recognition Caltech Pictures of objects belonging to 101 categories. About 40 to 800 images per category. Most categories have about 50 images. Evaluating linear classifier performance on DeCAF6 and DeCAF7. using dropout Compared also with the two-layers convolutional network of Jarret et al (2009) 15
15 Experiment: Domain Adaptation Office dataset (Saenko et al., 2010), which has 3 domains (31 categories in each domain): Amazon: images taken from amazon.com Webcam & Dslr: images taken in office environment using a webcam or digital SLR camera Domain shift: Source -> target Domain shift: Source -> target Trained Liner classifiers Adaptive methods ecent deep domain adaptation
16 Experiments Domain Adaptation DeCAF robust to resolution changes DeCAF provides better category clustering than SURF DeCAF clusters same category instances across domains SURF FEATURES DeCAF 6 FEATURES 17
17 Experiment: Subcategory Recognition Fine grained recognition involves recognizing subclasses of the same object class such as different bird species, dog breeds, flower types, etc. Caltech-UCSD birds dataset ( ~6000 photos of 200 bird species) - First,adopt ImageNet-like pipeline, DeCAF6 and a multi-class logistic regression ( as previous experiments) - Second, adopt deformable part descriptors (DPD) method [Zhang et al., 2013] 18
18 Experiment: Subcategory Recognition (only) (applied DeCAF in the same pre-trained DPM model and part predictions and used the same pooling weights). 19
19 Experiment: Scene Recognition Goal: classify the scene of the entire image SUN-397 large-scale scene recognition database (899 categories and 130,519 images) Outperforms Xiao ed al. (2010), the current state-of-the-art method 20
20 DeCAF demonstrate: Discussion Achieve high classification accuracy on tasks with sparse labeled data using simple linear classifiers. outperforming current state-of-the-art approaches based on sophisticated multi-kernel learning techniques with traditional hand-engineered features. the features tend to cluster images into interesting semantic categories on which the network was never explicitly trained. can substantially improve the performance of a wide variety of existing methods across a spectrum of visual recognition tasks 21
21 An Open-Source Convolutional Model Caffe ( at first it was called decaf) is a deep learning Python framework made with expression, speed, and modularity in mind. It is developed by the Berkeley Vision and Learning Center (BVLC) and by community contributors. Yangqing Jia created the project during his PhD at UC Berkeley. The framework allows one to easily train networks consisting of various layer types and to execute pre-trained networks efficiently without being restricted to a GPU. able to process about 40 images per second with an 8-core commodity machine when the CNN model is executed in a minibatch mode. In addition, they have released the network parameters used in their experiments to allow for out-of-the-box feature extraction without the need to re-train the large network 22
22 Image Style Transfer Using Convolutional Neural Networks 4-DEC-2016 Gal Barzilai, Ram Machlev Deep Learning Seminar School of Electrical Engineering Tel Aviv University
23 Part 2: Style Transfer
24 Texture Transfer - Review Transferring the style from one image onto another can be consider a problem of texture transfer. Our goal synthesize a texture from a source image while constraining the texture synthesis in order to preserve the semantic content of a target image. 25
25 Several Examples 26
26 Texture Transfer Former Approaches Large range of powerful non-parametric algorithms can synthesize photorealistic natural textures by resampling the pixels of a given source texture. For example Texture synthesis by non-parametric sampling These algorithms suffer from the limitation that they use only low-level image features of the target image to inform the texture transfer. There is a need for an algorithm that uses the high level image features for style transfer, and this article addresses this issue. 27
27 Texture Transfer Deep Learning Approach The article proposes a novel algorithm A neural Algorithm of Artistic Style (also suggested by the authors in an article with the same name). Many available implementations on Github (for example: ) The algorithm uses a CNN network that was trained for object recognition and localization (the chosen network was a VGG network). 28
28 Image Representations in CNN The number of different filters increase along the processing hierarchy. The size of filtered images is reduced by down-sampling mechanism, in our network average-pooling, leading to a decrease in the total number of units per layer of the network.
29 CNN Network VGG The VGG network was designed by the Visual Geometry Group in Oxford. Article: Very Deep Convolutional Networks for Large-Scale Image Recognition Developed a 16 layer and 19 layer models. Network parameters: normalized version, 16 convolutional, 5 pooling layers of the 19 VGG network. The normalization is of the weights such that the mean activation of each convolutional filter over images and positions is equal to one. No Fully connected layers. Average pooling instead of max pooling (gave better results, no theoretical explanation was provided. 30
30 Style Transfer Algorithm
31 Content Representation - Notations A layer with N l distinct filters has N l feature maps of size M l, Where M l is the height times the width of the feature maps. The responses in layer l can be stored in a matrix F l R N l M l, Where F ij l is the activation of the i th filter at position j in layer l. p is the original content image, x is the image generated (initialized from white noise). P l and F l are their respective feature representations in layer l. 32
32 Style Transfer Algorithm
33 Content Representation - Calculations Squared-error loss between two feature representations The derivative of the loss with respect to the activation in layer l The gradient with respect to the image x can be computed using standard error back-propagation. 34
34 Style Transfer Algorithm L total = L content Content Representation, ignoring style effect
35 Style Representation 1 To obtain a representation of the style of an input image, the authors used a feature space designed to capture texture information (the authors published it Texture Synthesis Using Convolutional Neural Networks. The feature space can be built on top of the filter responses in any layer of the network. It consists of the correlations between the different filter responses. The feature correlations are given by the gram matrix G l R N l N l l where G ij is the inner product between the vectorised feature maps i and j in layer l. 36
36 Style Representation 2 We include the feature correlations of multiple layers, and gain a stationary, multi-scale representation of the input image which captures its texture. We can visualize the information captured by the style feature spaces built on different layers of the network by constructing an image that matches the style representation of the style image. This is done by using gradient decent from a white noise image to minimize the mean-square distance between the entries of the Gram matrices from the style image and the gram matrices of the image to be generated. 37
37 Style Representation 3 a is the original style image, x is the image generated (initialized from white noise). A l and G l are their respective style representation in layer l. The contribution of each layer l to the total loss is: The total style loss is 38
38 Style Transfer Algorithm
39 Style Representation 4 w l are weighting factors of the contribution of each layer to the total loss. The derivative of E l with resect to the activations in layer l can be computed as: The gradients of E l with respect to the pixel values of x can be computed using standard error back-propagation. 40
40 Style Transfer Algorithm L total = L style Style Representation, ignoring content effect
41 Style Transfer Now we want to find a compromise between the style of the style picture and the content of the content image. We will now jointly minimize the distance of the feature representations of a white noise from the content representation of the photograph in one layer (a high one) and the style representation of the painting defined on a number of layer of the CNN. The loss function to be minimized is α and β are weighting factors. 42
42 Style Transfer Algorithm
43 Style Transfer Implementation Consideration The optimization strategy is L-BFGS which the authors found best for image synthesis. This is a limited memory BFGS, BFGS is an iterative method for solve unconstrained nonlinear optimization problems. The style image was resized to the size of the content image in order to extract image information on comparable size. 44
44 Style Transfer Algorithm
45 The Main Result The representation of content and style in CNN are well separable. Therefore we can manipulate both representations independently to produce new, perceptually meaningful images. 46
46 Trade-Off Between Content And Style Matching The higher the ratio of α the content of the β picture resembles to the content of the content image and less to the style of the style image 47
47 The Effect of Matching The Content Representation in Different Layers of The Network On the lower layer of the network (conv2_2) the texture of the painting is blended over the photograph. On the higher layer of the network (conv4_2) the new picture looks as if the content of the original picture was preserved in the style of the painting. Therefore, usually the more appealing image are creating from matching the style from the higher layers. Both images were produced with a ratio α β =
48 Initialization of Gradient Decent The initial guess changes the output image! Initializing from a predefined image leads to one image (neglecting the stochasticity of the gradient-decent)! Image A initialized from content image. Image B initialized from style image. The last four images were initialized from white noise. Initializing from white noise gives us an infinite number of potential output images. Small Bias toward initial guess (A or B). 49
49 Photorealistic Style Transfer Style New York. Content London. The photo-realism doesn t fully preserved. 50
50 Discussion Slide 1 In this article it was demonstrated how to use feature representations from CNN to transfer image style between arbitrary images. Limitations resolution of the synthesized images the speed of generating an image depends linearly in the number of pixels (both for the optimization problem and the number of units in the CNN). In this article a 512x512 pixels images were generated with an Nvidia K40 GPU and it could take an hour. The meaning of this limitation is that the algorithm can t be used for online and interactive applications. 51
51 Discussion Slide 2 Synthesized images are sometimes subject to some lowlevel noise. This is less problematic for artistic style transfer, and more relevant when both content and style images are photographs, because the photorealism of the image is affected. The authors say the noise resembles the filters of units in the network, and suggest to develop a de-noising technique to post process the image after the optimization. The separation of image content from style is not a well defined problem. This is because it s hard to define what is style in an image. 52
52 Questions 53
DeCAF: a Deep Convolutional Activation Feature for Generic Visual Recognition
DeCAF: a Deep Convolutional Activation Feature for Generic Visual Recognition ECS 289G 10/06/2016 Authors: Jeff Donahue, Yangqing Jia, Oriol Vinyals, Judy Hoffman, Ning Zhang, Eric Tzeng and Trevor Darrell
More informationA Neural Algorithm of Artistic Style. Leon A. Gatys, Alexander S. Ecker, Matthias Bethge
A Neural Algorithm of Artistic Style Leon A. Gatys, Alexander S. Ecker, Matthias Bethge Presented by Shishir Mathur (1 Sept 2016) What is this paper This is the research paper behind Prisma It creates
More informationImageNet Classification with Deep Convolutional Neural Networks
ImageNet Classification with Deep Convolutional Neural Networks Alex Krizhevsky Ilya Sutskever Geoffrey Hinton University of Toronto Canada Paper with same name to appear in NIPS 2012 Main idea Architecture
More informationDeep Learning with Tensorflow AlexNet
Machine Learning and Computer Vision Group Deep Learning with Tensorflow http://cvml.ist.ac.at/courses/dlwt_w17/ AlexNet Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton, "Imagenet classification
More informationMachine Learning. Deep Learning. Eric Xing (and Pengtao Xie) , Fall Lecture 8, October 6, Eric CMU,
Machine Learning 10-701, Fall 2015 Deep Learning Eric Xing (and Pengtao Xie) Lecture 8, October 6, 2015 Eric Xing @ CMU, 2015 1 A perennial challenge in computer vision: feature engineering SIFT Spin image
More informationDeep Learning for Computer Vision with MATLAB By Jon Cherrie
Deep Learning for Computer Vision with MATLAB By Jon Cherrie 2015 The MathWorks, Inc. 1 Deep learning is getting a lot of attention "Dahl and his colleagues won $22,000 with a deeplearning system. 'We
More informationA FRAMEWORK OF EXTRACTING MULTI-SCALE FEATURES USING MULTIPLE CONVOLUTIONAL NEURAL NETWORKS. Kuan-Chuan Peng and Tsuhan Chen
A FRAMEWORK OF EXTRACTING MULTI-SCALE FEATURES USING MULTIPLE CONVOLUTIONAL NEURAL NETWORKS Kuan-Chuan Peng and Tsuhan Chen School of Electrical and Computer Engineering, Cornell University, Ithaca, NY
More informationCSE 559A: Computer Vision
CSE 559A: Computer Vision Fall 2018: T-R: 11:30-1pm @ Lopata 101 Instructor: Ayan Chakrabarti (ayan@wustl.edu). Course Staff: Zhihao Xia, Charlie Wu, Han Liu http://www.cse.wustl.edu/~ayan/courses/cse559a/
More informationCOMP9444 Neural Networks and Deep Learning 7. Image Processing. COMP9444 c Alan Blair, 2017
COMP9444 Neural Networks and Deep Learning 7. Image Processing COMP9444 17s2 Image Processing 1 Outline Image Datasets and Tasks Convolution in Detail AlexNet Weight Initialization Batch Normalization
More informationTowards Large-Scale Semantic Representations for Actionable Exploitation. Prof. Trevor Darrell UC Berkeley
Towards Large-Scale Semantic Representations for Actionable Exploitation Prof. Trevor Darrell UC Berkeley traditional surveillance sensor emerging crowd sensor Desired capabilities: spatio-temporal reconstruction
More informationInception and Residual Networks. Hantao Zhang. Deep Learning with Python.
Inception and Residual Networks Hantao Zhang Deep Learning with Python https://en.wikipedia.org/wiki/residual_neural_network Deep Neural Network Progress from Large Scale Visual Recognition Challenge (ILSVRC)
More informationPart Localization by Exploiting Deep Convolutional Networks
Part Localization by Exploiting Deep Convolutional Networks Marcel Simon, Erik Rodner, and Joachim Denzler Computer Vision Group, Friedrich Schiller University of Jena, Germany www.inf-cv.uni-jena.de Abstract.
More informationConvolutional Neural Networks. Computer Vision Jia-Bin Huang, Virginia Tech
Convolutional Neural Networks Computer Vision Jia-Bin Huang, Virginia Tech Today s class Overview Convolutional Neural Network (CNN) Training CNN Understanding and Visualizing CNN Image Categorization:
More informationUsing Machine Learning for Classification of Cancer Cells
Using Machine Learning for Classification of Cancer Cells Camille Biscarrat University of California, Berkeley I Introduction Cell screening is a commonly used technique in the development of new drugs.
More informationAll You Want To Know About CNNs. Yukun Zhu
All You Want To Know About CNNs Yukun Zhu Deep Learning Deep Learning Image from http://imgur.com/ Deep Learning Image from http://imgur.com/ Deep Learning Image from http://imgur.com/ Deep Learning Image
More informationImage Transformation via Neural Network Inversion
Image Transformation via Neural Network Inversion Asha Anoosheh Rishi Kapadia Jared Rulison Abstract While prior experiments have shown it is possible to approximately reconstruct inputs to a neural net
More informationDeep Neural Networks:
Deep Neural Networks: Part II Convolutional Neural Network (CNN) Yuan-Kai Wang, 2016 Web site of this course: http://pattern-recognition.weebly.com source: CNN for ImageClassification, by S. Lazebnik,
More informationCOMP 551 Applied Machine Learning Lecture 16: Deep Learning
COMP 551 Applied Machine Learning Lecture 16: Deep Learning Instructor: Ryan Lowe (ryan.lowe@cs.mcgill.ca) Slides mostly by: Class web page: www.cs.mcgill.ca/~hvanho2/comp551 Unless otherwise noted, all
More informationDeep Learning for Computer Vision II
IIIT Hyderabad Deep Learning for Computer Vision II C. V. Jawahar Paradigm Shift Feature Extraction (SIFT, HoG, ) Part Models / Encoding Classifier Sparrow Feature Learning Classifier Sparrow L 1 L 2 L
More informationDeep Learning. Visualizing and Understanding Convolutional Networks. Christopher Funk. Pennsylvania State University.
Visualizing and Understanding Convolutional Networks Christopher Pennsylvania State University February 23, 2015 Some Slide Information taken from Pierre Sermanet (Google) presentation on and Computer
More informationA Neural Algorithm of Artistic Style. Leon A. Gatys, Alexander S. Ecker, Mattthias Bethge Presented by Weidi Xie (1st Oct 2015 )
A Neural Algorithm of Artistic Style Leon A. Gatys, Alexander S. Ecker, Mattthias Bethge Presented by Weidi Xie (1st Oct 2015 ) What does the paper do? 2 Create artistic images of high perceptual quality.
More informationarxiv: v1 [cs.cv] 22 Feb 2017
Synthesising Dynamic Textures using Convolutional Neural Networks arxiv:1702.07006v1 [cs.cv] 22 Feb 2017 Christina M. Funke, 1, 2, 3, Leon A. Gatys, 1, 2, 4, Alexander S. Ecker 1, 2, 5 1, 2, 3, 6 and Matthias
More informationScalable and Modularized RTL Compilation of Convolutional Neural Networks onto FPGA
Scalable and Modularized RTL Compilation of Convolutional Neural Networks onto FPGA Yufei Ma, Naveen Suda, Yu Cao, Jae-sun Seo, Sarma Vrudhula School of Electrical, Computer and Energy Engineering School
More informationConvolutional Neural Networks
Lecturer: Barnabas Poczos Introduction to Machine Learning (Lecture Notes) Convolutional Neural Networks Disclaimer: These notes have not been subjected to the usual scrutiny reserved for formal publications.
More informationRich feature hierarchies for accurate object detection and semantic segmentation
Rich feature hierarchies for accurate object detection and semantic segmentation Ross Girshick, Jeff Donahue, Trevor Darrell, Jitendra Malik Presented by Pandian Raju and Jialin Wu Last class SGD for Document
More informationConvolutional Neural Networks + Neural Style Transfer. Justin Johnson 2/1/2017
Convolutional Neural Networks + Neural Style Transfer Justin Johnson 2/1/2017 Outline Convolutional Neural Networks Convolution Pooling Feature Visualization Neural Style Transfer Feature Inversion Texture
More informationKaggle Data Science Bowl 2017 Technical Report
Kaggle Data Science Bowl 2017 Technical Report qfpxfd Team May 11, 2017 1 Team Members Table 1: Team members Name E-Mail University Jia Ding dingjia@pku.edu.cn Peking University, Beijing, China Aoxue Li
More informationSpatial Localization and Detection. Lecture 8-1
Lecture 8: Spatial Localization and Detection Lecture 8-1 Administrative - Project Proposals were due on Saturday Homework 2 due Friday 2/5 Homework 1 grades out this week Midterm will be in-class on Wednesday
More informationCNN Basics. Chongruo Wu
CNN Basics Chongruo Wu Overview 1. 2. 3. Forward: compute the output of each layer Back propagation: compute gradient Updating: update the parameters with computed gradient Agenda 1. Forward Conv, Fully
More informationCS 2750: Machine Learning. Neural Networks. Prof. Adriana Kovashka University of Pittsburgh April 13, 2016
CS 2750: Machine Learning Neural Networks Prof. Adriana Kovashka University of Pittsburgh April 13, 2016 Plan for today Neural network definition and examples Training neural networks (backprop) Convolutional
More informationDeep Tracking: Biologically Inspired Tracking with Deep Convolutional Networks
Deep Tracking: Biologically Inspired Tracking with Deep Convolutional Networks Si Chen The George Washington University sichen@gwmail.gwu.edu Meera Hahn Emory University mhahn7@emory.edu Mentor: Afshin
More informationReturn of the Devil in the Details: Delving Deep into Convolutional Nets
Return of the Devil in the Details: Delving Deep into Convolutional Nets Ken Chatfield - Karen Simonyan - Andrea Vedaldi - Andrew Zisserman University of Oxford The Devil is still in the Details 2011 2014
More informationPerceptron: This is convolution!
Perceptron: This is convolution! v v v Shared weights v Filter = local perceptron. Also called kernel. By pooling responses at different locations, we gain robustness to the exact spatial location of image
More informationMachine Learning 13. week
Machine Learning 13. week Deep Learning Convolutional Neural Network Recurrent Neural Network 1 Why Deep Learning is so Popular? 1. Increase in the amount of data Thanks to the Internet, huge amount of
More informationDeep Learning in Visual Recognition. Thanks Da Zhang for the slides
Deep Learning in Visual Recognition Thanks Da Zhang for the slides Deep Learning is Everywhere 2 Roadmap Introduction Convolutional Neural Network Application Image Classification Object Detection Object
More informationComputer Vision Lecture 16
Computer Vision Lecture 16 Deep Learning for Object Categorization 14.01.2016 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Announcements Seminar registration period
More informationContent-Based Image Recovery
Content-Based Image Recovery Hong-Yu Zhou and Jianxin Wu National Key Laboratory for Novel Software Technology Nanjing University, China zhouhy@lamda.nju.edu.cn wujx2001@nju.edu.cn Abstract. We propose
More informationClassification of objects from Video Data (Group 30)
Classification of objects from Video Data (Group 30) Sheallika Singh 12665 Vibhuti Mahajan 12792 Aahitagni Mukherjee 12001 M Arvind 12385 1 Motivation Video surveillance has been employed for a long time
More informationPhoto-realistic Renderings for Machines Seong-heum Kim
Photo-realistic Renderings for Machines 20105034 Seong-heum Kim CS580 Student Presentations 2016.04.28 Photo-realistic Renderings for Machines Scene radiances Model descriptions (Light, Shape, Material,
More information3D model classification using convolutional neural network
3D model classification using convolutional neural network JunYoung Gwak Stanford jgwak@cs.stanford.edu Abstract Our goal is to classify 3D models directly using convolutional neural network. Most of existing
More informationCS 229 Final Report: Artistic Style Transfer for Face Portraits
CS 229 Final Report: Artistic Style Transfer for Face Portraits Daniel Hsu, Marcus Pan, Chen Zhu {dwhsu, mpanj, chen0908}@stanford.edu Dec 16, 2016 1 Introduction The goal of our project is to learn the
More informationRestricted Boltzmann Machines. Shallow vs. deep networks. Stacked RBMs. Boltzmann Machine learning: Unsupervised version
Shallow vs. deep networks Restricted Boltzmann Machines Shallow: one hidden layer Features can be learned more-or-less independently Arbitrary function approximator (with enough hidden units) Deep: two
More informationA Deep Learning Framework for Authorship Classification of Paintings
A Deep Learning Framework for Authorship Classification of Paintings Kai-Lung Hua ( 花凱龍 ) Dept. of Computer Science and Information Engineering National Taiwan University of Science and Technology Taipei,
More informationYOLO9000: Better, Faster, Stronger
YOLO9000: Better, Faster, Stronger Date: January 24, 2018 Prepared by Haris Khan (University of Toronto) Haris Khan CSC2548: Machine Learning in Computer Vision 1 Overview 1. Motivation for one-shot object
More informationObject detection using Region Proposals (RCNN) Ernest Cheung COMP Presentation
Object detection using Region Proposals (RCNN) Ernest Cheung COMP790-125 Presentation 1 2 Problem to solve Object detection Input: Image Output: Bounding box of the object 3 Object detection using CNN
More informationNeural Networks. Single-layer neural network. CSE 446: Machine Learning Emily Fox University of Washington March 10, /10/2017
3/0/207 Neural Networks Emily Fox University of Washington March 0, 207 Slides adapted from Ali Farhadi (via Carlos Guestrin and Luke Zettlemoyer) Single-layer neural network 3/0/207 Perceptron as a neural
More informationAn Exploration of Computer Vision Techniques for Bird Species Classification
An Exploration of Computer Vision Techniques for Bird Species Classification Anne L. Alter, Karen M. Wang December 15, 2017 Abstract Bird classification, a fine-grained categorization task, is a complex
More informationCS 1674: Intro to Computer Vision. Neural Networks. Prof. Adriana Kovashka University of Pittsburgh November 16, 2016
CS 1674: Intro to Computer Vision Neural Networks Prof. Adriana Kovashka University of Pittsburgh November 16, 2016 Announcements Please watch the videos I sent you, if you haven t yet (that s your reading)
More informationDeep Learning for Visual Manipulation and Synthesis
Deep Learning for Visual Manipulation and Synthesis Jun-Yan Zhu 朱俊彦 UC Berkeley 2017/01/11 @ VALSE What is visual manipulation? Image Editing Program input photo User Input result Desired output: stay
More informationCaffe tutorial. Seong Joon Oh
Caffe tutorial Seong Joon Oh What is Caffe? Convolution Architecture For Feature Extraction (CAFFE) Open framework, models, and examples for deep learning 600+ citations, 100+ contributors, 7,000+ stars,
More informationCaffe2C: A Framework for Easy Implementation of CNN-based Mobile Applications
Caffe2C: A Framework for Easy Implementation of CNN-based Mobile Applications Ryosuke Tanno and Keiji Yanai Department of Informatics, The University of Electro-Communications, Tokyo 1. INTRODUCTION Deep
More informationKnow your data - many types of networks
Architectures Know your data - many types of networks Fixed length representation Variable length representation Online video sequences, or samples of different sizes Images Specific architectures for
More informationTutorial on Keras CAP ADVANCED COMPUTER VISION SPRING 2018 KISHAN S ATHREY
Tutorial on Keras CAP 6412 - ADVANCED COMPUTER VISION SPRING 2018 KISHAN S ATHREY Deep learning packages TensorFlow Google PyTorch Facebook AI research Keras Francois Chollet (now at Google) Chainer Company
More informationMACHINE LEARNING CLASSIFIERS ADVANTAGES AND CHALLENGES OF SELECTED METHODS
MACHINE LEARNING CLASSIFIERS ADVANTAGES AND CHALLENGES OF SELECTED METHODS FRANK ORBEN, TECHNICAL SUPPORT / DEVELOPER IMAGE PROCESSING, STEMMER IMAGING OUTLINE Introduction Task: Classification Theory
More informationSemantic Segmentation
Semantic Segmentation UCLA:https://goo.gl/images/I0VTi2 OUTLINE Semantic Segmentation Why? Paper to talk about: Fully Convolutional Networks for Semantic Segmentation. J. Long, E. Shelhamer, and T. Darrell,
More informationINTRODUCTION TO DEEP LEARNING
INTRODUCTION TO DEEP LEARNING CONTENTS Introduction to deep learning Contents 1. Examples 2. Machine learning 3. Neural networks 4. Deep learning 5. Convolutional neural networks 6. Conclusion 7. Additional
More informationConvolutional Neural Networks
NPFL114, Lecture 4 Convolutional Neural Networks Milan Straka March 25, 2019 Charles University in Prague Faculty of Mathematics and Physics Institute of Formal and Applied Linguistics unless otherwise
More informationRotation Invariance Neural Network
Rotation Invariance Neural Network Shiyuan Li Abstract Rotation invariance and translate invariance have great values in image recognition. In this paper, we bring a new architecture in convolutional neural
More informationA performance comparison of Deep Learning frameworks on KNL
A performance comparison of Deep Learning frameworks on KNL R. Zanella, G. Fiameni, M. Rorro Middleware, Data Management - SCAI - CINECA IXPUG Bologna, March 5, 2018 Table of Contents 1. Problem description
More informationRecurrent Neural Networks and Transfer Learning for Action Recognition
Recurrent Neural Networks and Transfer Learning for Action Recognition Andrew Giel Stanford University agiel@stanford.edu Ryan Diaz Stanford University ryandiaz@stanford.edu Abstract We have taken on the
More informationReal-time Object Detection CS 229 Course Project
Real-time Object Detection CS 229 Course Project Zibo Gong 1, Tianchang He 1, and Ziyi Yang 1 1 Department of Electrical Engineering, Stanford University December 17, 2016 Abstract Objection detection
More informationDeep Convolutional Neural Networks. Nov. 20th, 2015 Bruce Draper
Deep Convolutional Neural Networks Nov. 20th, 2015 Bruce Draper Background: Fully-connected single layer neural networks Feed-forward classification Trained through back-propagation Example Computer Vision
More informationSafety verification for deep neural networks
Safety verification for deep neural networks Marta Kwiatkowska Department of Computer Science, University of Oxford UC Berkeley, 8 th November 2016 Setting the scene Deep neural networks have achieved
More informationProceedings of the International MultiConference of Engineers and Computer Scientists 2018 Vol I IMECS 2018, March 14-16, 2018, Hong Kong
, March 14-16, 2018, Hong Kong , March 14-16, 2018, Hong Kong , March 14-16, 2018, Hong Kong , March 14-16, 2018, Hong Kong TABLE I CLASSIFICATION ACCURACY OF DIFFERENT PRE-TRAINED MODELS ON THE TEST DATA
More informationAutomatic detection of books based on Faster R-CNN
Automatic detection of books based on Faster R-CNN Beibei Zhu, Xiaoyu Wu, Lei Yang, Yinghua Shen School of Information Engineering, Communication University of China Beijing, China e-mail: zhubeibei@cuc.edu.cn,
More informationFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun Presented by Tushar Bansal Objective 1. Get bounding box for all objects
More informationRegionlet Object Detector with Hand-crafted and CNN Feature
Regionlet Object Detector with Hand-crafted and CNN Feature Xiaoyu Wang Research Xiaoyu Wang Research Ming Yang Horizon Robotics Shenghuo Zhu Alibaba Group Yuanqing Lin Baidu Overview of this section Regionlet
More informationNeural style transfer
1/32 Neural style transfer Victor Kitov v.v.kitov@yandex.ru 2/32 Neural style transfer Input: content image, style image. Style transfer - application of artistic style from style image to content image.
More informationCS230: Lecture 3 Various Deep Learning Topics
CS230: Lecture 3 Various Deep Learning Topics Kian Katanforoosh, Andrew Ng Today s outline We will learn how to: - Analyse a problem from a deep learning approach - Choose an architecture - Choose a loss
More informationREGION AVERAGE POOLING FOR CONTEXT-AWARE OBJECT DETECTION
REGION AVERAGE POOLING FOR CONTEXT-AWARE OBJECT DETECTION Kingsley Kuan 1, Gaurav Manek 1, Jie Lin 1, Yuan Fang 1, Vijay Chandrasekhar 1,2 Institute for Infocomm Research, A*STAR, Singapore 1 Nanyang Technological
More informationarxiv: v1 [cs.cv] 6 Jul 2016
arxiv:607.079v [cs.cv] 6 Jul 206 Deep CORAL: Correlation Alignment for Deep Domain Adaptation Baochen Sun and Kate Saenko University of Massachusetts Lowell, Boston University Abstract. Deep neural networks
More informationObject Detection Based on Deep Learning
Object Detection Based on Deep Learning Yurii Pashchenko AI Ukraine 2016, Kharkiv, 2016 Image classification (mostly what you ve seen) http://tutorial.caffe.berkeleyvision.org/caffe-cvpr15-detection.pdf
More informationUnsupervised Domain Adaptation by Backpropagation. Chih-Hui Ho, Xingyu Gu, Yuan Qi
Unsupervised Domain Adaptation by Backpropagation Chih-Hui Ho, Xingyu Gu, Yuan Qi Problems Deep network: requires massive labeled training data. Labeled data: Available sometimes: Image recognition Speech
More informationUnsupervised learning in Vision
Chapter 7 Unsupervised learning in Vision The fields of Computer Vision and Machine Learning complement each other in a very natural way: the aim of the former is to extract useful information from visual
More informationCENG 783. Special topics in. Deep Learning. AlchemyAPI. Week 11. Sinan Kalkan
CENG 783 Special topics in Deep Learning AlchemyAPI Week 11 Sinan Kalkan TRAINING A CNN Fig: http://www.robots.ox.ac.uk/~vgg/practicals/cnn/ Feed-forward pass Note that this is written in terms of the
More informationDeep Face Recognition. Nathan Sun
Deep Face Recognition Nathan Sun Why Facial Recognition? Picture ID or video tracking Higher Security for Facial Recognition Software Immensely useful to police in tracking suspects Your face will be an
More informationApparel Classifier and Recommender using Deep Learning
Apparel Classifier and Recommender using Deep Learning Live Demo at: http://saurabhg.me/projects/tag-that-apparel Saurabh Gupta sag043@ucsd.edu Siddhartha Agarwal siagarwa@ucsd.edu Apoorve Dave a1dave@ucsd.edu
More informationRich feature hierarchies for accurate object detection and semant
Rich feature hierarchies for accurate object detection and semantic segmentation Speaker: Yucong Shen 4/5/2018 Develop of Object Detection 1 DPM (Deformable parts models) 2 R-CNN 3 Fast R-CNN 4 Faster
More informationPARTIAL STYLE TRANSFER USING WEAKLY SUPERVISED SEMANTIC SEGMENTATION. Shin Matsuo Wataru Shimoda Keiji Yanai
PARTIAL STYLE TRANSFER USING WEAKLY SUPERVISED SEMANTIC SEGMENTATION Shin Matsuo Wataru Shimoda Keiji Yanai Department of Informatics, The University of Electro-Communications, Tokyo 1-5-1 Chofugaoka,
More information11. Neural Network Regularization
11. Neural Network Regularization CS 519 Deep Learning, Winter 2016 Fuxin Li With materials from Andrej Karpathy, Zsolt Kira Preventing overfitting Approach 1: Get more data! Always best if possible! If
More informationTransferrable Representations for Visual Recognition
Transferrable Representations for Visual Recognition Jeffrey Donahue Electrical Engineering and Computer Sciences University of California at Berkeley Technical Report No. UCB/EECS-2017-106 http://www2.eecs.berkeley.edu/pubs/techrpts/2017/eecs-2017-106.html
More informationReal-Time Depth Estimation from 2D Images
Real-Time Depth Estimation from 2D Images Jack Zhu Ralph Ma jackzhu@stanford.edu ralphma@stanford.edu. Abstract ages. We explore the differences in training on an untrained network, and on a network pre-trained
More informationDeep Learning Basic Lecture - Complex Systems & Artificial Intelligence 2017/18 (VO) Asan Agibetov, PhD.
Deep Learning 861.061 Basic Lecture - Complex Systems & Artificial Intelligence 2017/18 (VO) Asan Agibetov, PhD asan.agibetov@meduniwien.ac.at Medical University of Vienna Center for Medical Statistics,
More informationDeep Learning. Vladimir Golkov Technical University of Munich Computer Vision Group
Deep Learning Vladimir Golkov Technical University of Munich Computer Vision Group 1D Input, 1D Output target input 2 2D Input, 1D Output: Data Distribution Complexity Imagine many dimensions (data occupies
More informationCNNS FROM THE BASICS TO RECENT ADVANCES. Dmytro Mishkin Center for Machine Perception Czech Technical University in Prague
CNNS FROM THE BASICS TO RECENT ADVANCES Dmytro Mishkin Center for Machine Perception Czech Technical University in Prague ducha.aiki@gmail.com OUTLINE Short review of the CNN design Architecture progress
More informationStudy of Residual Networks for Image Recognition
Study of Residual Networks for Image Recognition Mohammad Sadegh Ebrahimi Stanford University sadegh@stanford.edu Hossein Karkeh Abadi Stanford University hosseink@stanford.edu Abstract Deep neural networks
More informationGAN Related Works. CVPR 2018 & Selective Works in ICML and NIPS. Zhifei Zhang
GAN Related Works CVPR 2018 & Selective Works in ICML and NIPS Zhifei Zhang Generative Adversarial Networks (GANs) 9/12/2018 2 Generative Adversarial Networks (GANs) Feedforward Backpropagation Real? z
More informationDL Tutorial. Xudong Cao
DL Tutorial Xudong Cao Historical Line 1960s Perceptron 1980s MLP BP algorithm 2006 RBM unsupervised learning 2012 AlexNet ImageNet Comp. 2014 GoogleNet VGGNet ImageNet Comp. Rule based AI algorithm Game
More informationDepth Estimation from a Single Image Using a Deep Neural Network Milestone Report
Figure 1: The architecture of the convolutional network. Input: a single view image; Output: a depth map. 3 Related Work In [4] they used depth maps of indoor scenes produced by a Microsoft Kinect to successfully
More informationOverall Description. Goal: to improve spatial invariance to the input data. Translation, Rotation, Scale, Clutter, Elastic
Philippe Giguère Overall Description Goal: to improve spatial invariance to the input data Translation, Rotation, Scale, Clutter, Elastic How: add a learnable module which explicitly manipulate spatially
More informationClassifying Depositional Environments in Satellite Images
Classifying Depositional Environments in Satellite Images Alex Miltenberger and Rayan Kanfar Department of Geophysics School of Earth, Energy, and Environmental Sciences Stanford University 1 Introduction
More informationStructured Prediction using Convolutional Neural Networks
Overview Structured Prediction using Convolutional Neural Networks Bohyung Han bhhan@postech.ac.kr Computer Vision Lab. Convolutional Neural Networks (CNNs) Structured predictions for low level computer
More informationFully Convolutional Networks for Semantic Segmentation
Fully Convolutional Networks for Semantic Segmentation Jonathan Long* Evan Shelhamer* Trevor Darrell UC Berkeley Chaim Ginzburg for Deep Learning seminar 1 Semantic Segmentation Define a pixel-wise labeling
More informationSupplementary material for the paper Are Sparse Representations Really Relevant for Image Classification?
Supplementary material for the paper Are Sparse Representations Really Relevant for Image Classification? Roberto Rigamonti, Matthew A. Brown, Vincent Lepetit CVLab, EPFL Lausanne, Switzerland firstname.lastname@epfl.ch
More informationINF 5860 Machine learning for image classification. Lecture 11: Visualization Anne Solberg April 4, 2018
INF 5860 Machine learning for image classification Lecture 11: Visualization Anne Solberg April 4, 2018 Reading material The lecture is based on papers: Deep Dream: https://research.googleblog.com/2015/06/inceptionism-goingdeeper-into-neural.html
More informationIn Defense of Fully Connected Layers in Visual Representation Transfer
In Defense of Fully Connected Layers in Visual Representation Transfer Chen-Lin Zhang, Jian-Hao Luo, Xiu-Shen Wei, Jianxin Wu National Key Laboratory for Novel Software Technology, Nanjing University,
More informationSEMANTIC COMPUTING. Lecture 8: Introduction to Deep Learning. TU Dresden, 7 December Dagmar Gromann International Center For Computational Logic
SEMANTIC COMPUTING Lecture 8: Introduction to Deep Learning Dagmar Gromann International Center For Computational Logic TU Dresden, 7 December 2018 Overview Introduction Deep Learning General Neural Networks
More informationCMU Lecture 18: Deep learning and Vision: Convolutional neural networks. Teacher: Gianni A. Di Caro
CMU 15-781 Lecture 18: Deep learning and Vision: Convolutional neural networks Teacher: Gianni A. Di Caro DEEP, SHALLOW, CONNECTED, SPARSE? Fully connected multi-layer feed-forward perceptrons: More powerful
More informationASCII Art Synthesis with Convolutional Networks
ASCII Art Synthesis with Convolutional Networks Osamu Akiyama Faculty of Medicine, Osaka University oakiyama1986@gmail.com 1 Introduction ASCII art is a type of graphic art that presents a picture with
More informationLearning Visual Semantics: Models, Massive Computation, and Innovative Applications
Learning Visual Semantics: Models, Massive Computation, and Innovative Applications Part II: Visual Features and Representations Liangliang Cao, IBM Watson Research Center Evolvement of Visual Features
More information