Deep Learning For Video Classification. Presented by Natalie Carlebach & Gil Sharon
|
|
- Sibyl Wood
- 6 years ago
- Views:
Transcription
1 Deep Learning For Video Classification Presented by Natalie Carlebach & Gil Sharon
2 Overview Of Presentation Motivation Challenges of video classification Common datasets 4 different methods presented in 3 papers: 1. 3D convolutions 2. Spatial + Optical flow fusion 3. Temporal pooling 4. LSTM Recap of a different elegant method Conclusions
3 Motivation 500 Hours of video uploaded to YouTube every minute Analyzing these videos is needed for search, recommendation, ranking etc. Action recognition, abnormal event detection, activity understanding
4 Challenges In Video Classification current ConvNets are not able to take full advantage of temporal information Several orders of magnitude more data compared with photos Variations in motion and viewpoint Datasets are noisy or small - videos are difficult to collect, annotate and store Complex Context compared to photos
5 Motivation of Temporal Information For example, what is happening in this video? A CNN would probably classify this as crying or shouting Temporal information is needed
6 Challenges In Video Classification current ConvNets are not able to take full advantage of temporal information Several orders of magnitude more data compared with photos Variations in motion and viewpoint Datasets are noisy or small - videos are difficult to collect, annotate and store Complex Context compared to photos
7 First paper 3D Convolution (C3D) October 2015
8 Motivation To combine temporal information with spatial information. 2D covnets are not enough. Proposal: spatiotemporal feature learning using deep 3D ConvNets.
9 2D Conv vs 3D Conv 2D conv on an image Input: image Output: image 2D conv on a video Input: volume (multiple frames as multiple channels) Output: image 3D conv on a video Input: volume Output: volume preserves temporal information of the input
10 3D Conv Kernels fixed the spatial kernels to 3x3 vary only the temporal depth of the 3D convolution kernels.
11 Network Settings Input: 16 frames non overlapping clips (were split from each video - resized to ) Output: class labels (belong to 101 different actions) All convolution kernels are d 3 3 (d temporal depth) Max pooling layer 2-5 : kernel are Max pooling layer 1: kernel is Input: 3x16x 112x 112
12 Training Technical Details Data set: UCF101 We train the networks from scratch using mini-batches of 30 clips. Learning Rate: initial learning rate of The learning rate is divided by 10 after every 4 epochs. Stopping criteria: after 16 epochs.
13 Datasets Action Recognition UCF101-13,000 videos of 101 human action categories Short clips, steady camera, less natural
14 Training Technical Details Data set: UCF101 We train the networks from scratch using mini-batches of 30 clips. Learning Rate: initial learning rate of The learning rate is divided by 10 after every 4 epochs. Stopping criteria: after 16 epochs.
15 Varying Network Architectures a) homogeneous temporal depth: d= 1,3,5,7 b) varying temporal depth: increasing: d = decreasing: d =
16 Varying Network Architectures homogeneous temporal depth: varying temporal depth: homogeneous temporal depth of 3 was chosen
17 Learning Spatiotemporal Features Dataset : Sport1M (long videos) Training: randomly extract five 2-second long clips from every training video C3D trained from scratch/ C3D pre- trained Testing: For video predictions, we average clip predictions of 10 clips
18 Datasets - Sports Video Classification Sport1M 1 million you tube videos, 487 sport categories Few minutes videos, in the wild, camera less steady, noisier labels
19 Learning Spatiotemporal Features Dataset : Sport1M (long videos) Training: randomly extract five 2-second long clips from every training video C3D trained from scratch/ C3D pre- trained Testing: For video predictions, we average clip predictions of 10 clips
20 Sport1M results The method is not state of the art. We note that the method of [29] uses long clips, thus its clip-level accuracy is not directly comparable to that of C3D and DeepVideo
21 C3D Video Descriptor A model is trained on Sport1M and kept constant 4096 dim video descriptor - averaging FC6 of this model, on 16 frames clips with stride 8, followed by L2 normalization. Multiclass Linear SVM is used on descriptor The descriptor is compared to other descriptors on several datasets
22 Visualization of C3D Descriptor We observe that C3D starts by focusing on appearance in the first few frames and tracks the salient motion in the subsequent frames
23 Results of Action Recognition on UCF101 Using C3D descriptor, results were state of the art only when combined with hand crafted video descriptor idt. Best among methods with RGB input only CNN RGB based only all possible feature combinations
24 C3D descriptor characteristics Compact- Results of UCF101 when reducing dimensions using PCA, better than other descriptors More generic - visualized by t-sne, compared with features extracted with 2D convolutions Fast 313 fps on GPU, 100 times faster than idt
25 Deconvolution Examples A feature map from Conv2 is learning moving edges and blobs
26 Deconvolution Examples A feature map from Conv3 is learning moving body parts
27 Deconvolution Examples A feature map from Conv5 is learning more complex movements like biking
28 Disadvantages of C3D C3D is limited temporal support of 16 consecutive frames. No optical flow is added to the CNN - Other works showed that adding it is improving the results. Other methods had better performance on each data set.
29 Second Paper April 2015, presented at CVPR2015
30 Motivation combining information over longer videos than previous methods Proposals Convolutional temporal feature pooling architecture LSTM cells connected to CNN convolutional output
31 Optical Flow d t (u, v): the displacement vector at the point (u, v) in frame t, which moves the point to the corresponding point in the following frame t + 1. (c) A close up of dense optical flow in the outlined area. The horizontal and vertical components of the vector field, d t x and d t y, can be seen as image channels (d), (e)
32 Optical Flow Stacking To represent the motion across a sequence of frames, we stack the flow channels d t x, d t y of L consecutive frames to form a total of 2L input channels
33 Motivation of Combination RGB - brush and hair, brush and teeth Optical Flow - hand moves periodically at some spatial location The combination discriminates the action
34 First Method Convolutional Temporal Feature Pooling Proposed approaches Temporal feature pooling performed on last convolutional layer of GoogLeNet.
35 First Method Implementation Details Datasets: Sport1m Cropping clips of 120 frames at 1 fps Feed forward of each frame through GoogLeNet Several methods for feature pooling between frames in the clip For prediction averaging clips with different starting points
36 Results: Convolutional Temporal Feature Pooling Data set: Sport1M Late pooling is the worst, doesn t preserve spatial information
37 Second Method - LSTM architecture Pooling is order invariant, LSTM usage is more natural for sequences
38 Video classification LSTM architecture 5 layers of LSTM, 512 cells each. Input is last convolutional layer of GoogLeNet from each frame Prediction is made after each frame In training, weight of loss is linearly growing from 0 to 1 through the video
39 Optical flow fusion implementation Processing videos in 1 Fps loses local motion information. Optical flow stream is added Each stream is fed forward in the same 2 methods as mentioned before Fusion is made only at the softmax layer
40 Results Sport1M results of different methods Optical flow fusion does not improve due to shaky videos State of the art performance on Sport1M
41 Results UCF101 results based on raw frames only UCF101 state of the art results (not anymore) when fusing optical flow
42 Disadvantages Of The 2 Methods Feature pooling is less generic for arbitrary length of a video LSTM is tested only on 30 frames, less global context Optical flow calculation is slow, and less elegant end-to-end training Fusion with optical flow is only at the softmax, not ideal information sharing
43 Third Paper Two Stream Fusion April 2016, presented at CVPR2016
44 Motivation Spatial + Optical flow fusion: registering appearance recognition (spatial cue) with optical flow recognition (temporal cue), at the pixel level Temporal fusion: how these cues evolve over time
45 Two Networks to Fuse Spatial fusion: Per frame Temporal fusion: Between frames
46 Challenges in the Fusion Process How to spatially fuse? Which channel in one network corresponds to a channel of the other network? Where to fuse the networks spatially? How to perform temporal fusion between frames?
47 Declarations Fusion Function: Feature Maps: Output Map: For simplicity, assume that W Width H height D number of channels of the respective feature maps
48 Possible Spatial Fusion Methods Sum Fusion: Max Fusion: Concatenation Fusion: Conv Fusion: Bilinear Fusion:
49 Where to Spatially Fuse? Two Examples (based on VGG): fusion after the 4th conv-layer. Only a single network tower is used from the point of fusion two layers (after conv5 and fc8) both network towers are kept Influence on the number of parameters
50 Spatial Fusion Methods - Comparison DataSet: UCF101 Model: 8 layers VGG-M
51 Spatial Fusion Methods - Comparison DataSet: UCF101 Model: 8 layers VGG-M How to spatially fuse? Conv Fusion Answers to our questions:
52 Where to Spatially Fuse? Dataset: UCF101 Model: 8 layers VGG-M
53 Where to Spatially Fuse? Dataset: UCF101 Model: 8 layers VGG-M Answers to our questions: Where to spatially fuse? At ReLU5 or at ReLU5+FC8 (but nearly doubles the parameters involved)
54 Temporal Fusion At Pool5 Spatiotemporal: Ignores time, spatial pooling only. Averaging the network predictions over time Stacking feature maps across frames, pools from local spatiotemporal neighborhood. No pooling across channels Additionally performs a convolution with a fusion kernel that spans the feature channels from both streams, space and time before 3D pooling. This is replacing the single frame Conv fusion. Kernel size:
55 Combining it all together Model: Vgg-16 Fusion Relu5 + after softmax (2 towers are kept). Spatiotemporal fusion+ 3D pooling in fused tower, 3D pooling in temporal tower L = 10 :Number of optical flow images around each frame T = 5 : Number of frames per video clip (for testing and training) τ Frame distance between sampled frames. Selected randomly [3,10] Prediction is averaged over both towers
56 Results
57 Disadvantages of this method Tested only on 5 frames per video. Not enough for context of longer videos Not tested on bigger and more general dataset of Sport1M Relies heavily on optical flow, may not work on many real life not stabilized videos As before, using optical flow is slow and not allowing end-to-end training
58 Summery & Conclusions 4 different approaches of video classification were shown Each method performed better on different tasks or datasets Optical flow improves the performance. Adding hand crafted features improved the results, hinting that there is still a room for improvement on CNN approaches
59 Disadvantages Of LSTM Methods [slide source: cs231n course, Fei-Fei Li & Andrej Karpathy & Justin Johnson lecture 14, slide 36]
60 Another Elegant Method
61 Brief Overview [slide source: cs231n course, Fei-Fei Li & Andrej Karpathy & Justin Johnson lecture 14, slide 31]
62 Brief Overview [slide source: cs231n course, Fei-Fei Li & Andrej Karpathy & Justin Johnson lecture 14, slide 33]
63 Brief Overview [slide source: cs231n course, Fei-Fei Li & Andrej Karpathy & Justin Johnson lecture 14, slide 37]
64 Questions?
Two-Stream Convolutional Networks for Action Recognition in Videos
Two-Stream Convolutional Networks for Action Recognition in Videos Karen Simonyan Andrew Zisserman Cemil Zalluhoğlu Introduction Aim Extend deep Convolution Networks to action recognition in video. Motivation
More informationQuo Vadis, Action Recognition? A New Model and the Kinetics Dataset. By Joa õ Carreira and Andrew Zisserman Presenter: Zhisheng Huang 03/02/2018
Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset By Joa õ Carreira and Andrew Zisserman Presenter: Zhisheng Huang 03/02/2018 Outline: Introduction Action classification architectures
More informationCS231N Section. Video Understanding 6/1/2018
CS231N Section Video Understanding 6/1/2018 Outline Background / Motivation / History Video Datasets Models Pre-deep learning CNN + RNN 3D convolution Two-stream What we ve seen in class so far... Image
More informationLarge-scale Video Classification with Convolutional Neural Networks
Large-scale Video Classification with Convolutional Neural Networks Andrej Karpathy, George Toderici, Sanketh Shetty, Thomas Leung, Rahul Sukthankar, Li Fei-Fei Note: Slide content mostly from : Bay Area
More informationPerson Action Recognition/Detection
Person Action Recognition/Detection Fabrício Ceschin Visão Computacional Prof. David Menotti Departamento de Informática - Universidade Federal do Paraná 1 In object recognition: is there a chair in the
More informationMachine Learning 13. week
Machine Learning 13. week Deep Learning Convolutional Neural Network Recurrent Neural Network 1 Why Deep Learning is so Popular? 1. Increase in the amount of data Thanks to the Internet, huge amount of
More informationLong-term Temporal Convolutions for Action Recognition INRIA
Longterm Temporal Convolutions for Action Recognition Gul Varol Ivan Laptev INRIA Cordelia Schmid 2 Motivation Current CNN methods for action recognition learn representations for short intervals (116
More informationKnow your data - many types of networks
Architectures Know your data - many types of networks Fixed length representation Variable length representation Online video sequences, or samples of different sizes Images Specific architectures for
More informationMachine Learning. Deep Learning. Eric Xing (and Pengtao Xie) , Fall Lecture 8, October 6, Eric CMU,
Machine Learning 10-701, Fall 2015 Deep Learning Eric Xing (and Pengtao Xie) Lecture 8, October 6, 2015 Eric Xing @ CMU, 2015 1 A perennial challenge in computer vision: feature engineering SIFT Spin image
More informationRecurrent Neural Networks and Transfer Learning for Action Recognition
Recurrent Neural Networks and Transfer Learning for Action Recognition Andrew Giel Stanford University agiel@stanford.edu Ryan Diaz Stanford University ryandiaz@stanford.edu Abstract We have taken on the
More informationCNN Basics. Chongruo Wu
CNN Basics Chongruo Wu Overview 1. 2. 3. Forward: compute the output of each layer Back propagation: compute gradient Updating: update the parameters with computed gradient Agenda 1. Forward Conv, Fully
More informationFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun Presented by Tushar Bansal Objective 1. Get bounding box for all objects
More informationLecture 37: ConvNets (Cont d) and Training
Lecture 37: ConvNets (Cont d) and Training CS 4670/5670 Sean Bell [http://bbabenko.tumblr.com/post/83319141207/convolutional-learnings-things-i-learned-by] (Unrelated) Dog vs Food [Karen Zack, @teenybiscuit]
More informationDeep neural networks II
Deep neural networks II May 31 st, 2018 Yong Jae Lee UC Davis Many slides from Rob Fergus, Svetlana Lazebnik, Jia-Bin Huang, Derek Hoiem, Adriana Kovashka, Why (convolutional) neural networks? State of
More informationConvolutional Neural Networks. Computer Vision Jia-Bin Huang, Virginia Tech
Convolutional Neural Networks Computer Vision Jia-Bin Huang, Virginia Tech Today s class Overview Convolutional Neural Network (CNN) Training CNN Understanding and Visualizing CNN Image Categorization:
More informationDeep Learning. Visualizing and Understanding Convolutional Networks. Christopher Funk. Pennsylvania State University.
Visualizing and Understanding Convolutional Networks Christopher Pennsylvania State University February 23, 2015 Some Slide Information taken from Pierre Sermanet (Google) presentation on and Computer
More informationIntroduction to Neural Networks
Introduction to Neural Networks Jakob Verbeek 2017-2018 Biological motivation Neuron is basic computational unit of the brain about 10^11 neurons in human brain Simplified neuron model as linear threshold
More informationClassification of objects from Video Data (Group 30)
Classification of objects from Video Data (Group 30) Sheallika Singh 12665 Vibhuti Mahajan 12792 Aahitagni Mukherjee 12001 M Arvind 12385 1 Motivation Video surveillance has been employed for a long time
More informationDeep Learning for Computer Vision II
IIIT Hyderabad Deep Learning for Computer Vision II C. V. Jawahar Paradigm Shift Feature Extraction (SIFT, HoG, ) Part Models / Encoding Classifier Sparrow Feature Learning Classifier Sparrow L 1 L 2 L
More informationFuzzy Set Theory in Computer Vision: Example 3
Fuzzy Set Theory in Computer Vision: Example 3 Derek T. Anderson and James M. Keller FUZZ-IEEE, July 2017 Overview Purpose of these slides are to make you aware of a few of the different CNN architectures
More informationCOMP9444 Neural Networks and Deep Learning 7. Image Processing. COMP9444 c Alan Blair, 2017
COMP9444 Neural Networks and Deep Learning 7. Image Processing COMP9444 17s2 Image Processing 1 Outline Image Datasets and Tasks Convolution in Detail AlexNet Weight Initialization Batch Normalization
More informationCS 2750: Machine Learning. Neural Networks. Prof. Adriana Kovashka University of Pittsburgh April 13, 2016
CS 2750: Machine Learning Neural Networks Prof. Adriana Kovashka University of Pittsburgh April 13, 2016 Plan for today Neural network definition and examples Training neural networks (backprop) Convolutional
More informationarxiv: v1 [cs.cv] 19 Jun 2018
Multimodal feature fusion for CNN-based gait recognition: an empirical comparison F.M. Castro a,, M.J. Marín-Jiménez b, N. Guil a, N. Pérez de la Blanca c a Department of Computer Architecture, University
More informationEasyChair Preprint. Real-Time Action Recognition based on Enhanced Motion Vector Temporal Segment Network
EasyChair Preprint 730 Real-Time Action Recognition based on Enhanced Motion Vector Temporal Segment Network Xue Bai, Enqing Chen and Haron Chweya Tinega EasyChair preprints are intended for rapid dissemination
More informationP-CNN: Pose-based CNN Features for Action Recognition. Iman Rezazadeh
P-CNN: Pose-based CNN Features for Action Recognition Iman Rezazadeh Introduction automatic understanding of dynamic scenes strong variations of people and scenes in motion and appearance Fine-grained
More informationDeep Convolutional Neural Networks. Nov. 20th, 2015 Bruce Draper
Deep Convolutional Neural Networks Nov. 20th, 2015 Bruce Draper Background: Fully-connected single layer neural networks Feed-forward classification Trained through back-propagation Example Computer Vision
More informationINTRODUCTION TO DEEP LEARNING
INTRODUCTION TO DEEP LEARNING CONTENTS Introduction to deep learning Contents 1. Examples 2. Machine learning 3. Neural networks 4. Deep learning 5. Convolutional neural networks 6. Conclusion 7. Additional
More informationBody Joint guided 3D Deep Convolutional Descriptors for Action Recognition
1 Body Joint guided 3D Deep Convolutional Descriptors for Action Recognition arxiv:1704.07160v2 [cs.cv] 25 Apr 2017 Congqi Cao, Yifan Zhang, Member, IEEE, Chunjie Zhang, Member, IEEE, and Hanqing Lu, Senior
More informationActivity Recognition in Temporally Untrimmed Videos
Activity Recognition in Temporally Untrimmed Videos Bryan Anenberg Stanford University anenberg@stanford.edu Norman Yu Stanford University normanyu@stanford.edu Abstract We investigate strategies to apply
More informationNVIDIA FOR DEEP LEARNING. Bill Veenhuis
NVIDIA FOR DEEP LEARNING Bill Veenhuis bveenhuis@nvidia.com Nvidia is the world s leading ai platform ONE ARCHITECTURE CUDA 2 GPU: Perfect Companion for Accelerating Apps & A.I. CPU GPU 3 Intro to AI AGENDA
More informationDeep Learning in Visual Recognition. Thanks Da Zhang for the slides
Deep Learning in Visual Recognition Thanks Da Zhang for the slides Deep Learning is Everywhere 2 Roadmap Introduction Convolutional Neural Network Application Image Classification Object Detection Object
More informationLearning Spatio-Temporal Features with 3D Residual Networks for Action Recognition
Learning Spatio-Temporal Features with 3D Residual Networks for Action Recognition Kensho Hara, Hirokatsu Kataoka, Yutaka Satoh National Institute of Advanced Industrial Science and Technology (AIST) Tsukuba,
More informationUsing Machine Learning for Classification of Cancer Cells
Using Machine Learning for Classification of Cancer Cells Camille Biscarrat University of California, Berkeley I Introduction Cell screening is a commonly used technique in the development of new drugs.
More informationComputer Vision Lecture 16
Computer Vision Lecture 16 Deep Learning for Object Categorization 14.01.2016 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Announcements Seminar registration period
More informationStructured Prediction using Convolutional Neural Networks
Overview Structured Prediction using Convolutional Neural Networks Bohyung Han bhhan@postech.ac.kr Computer Vision Lab. Convolutional Neural Networks (CNNs) Structured predictions for low level computer
More informationTHE goal of action detection is to detect every occurrence
JOURNAL OF L A T E X CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 1 An End-to-end 3D Convolutional Neural Network for Action Detection and Segmentation in Videos Rui Hou, Student Member, IEEE, Chen Chen, Member,
More informationA Deep Learning Approach to Vehicle Speed Estimation
A Deep Learning Approach to Vehicle Speed Estimation Benjamin Penchas bpenchas@stanford.edu Tobin Bell tbell@stanford.edu Marco Monteiro marcorm@stanford.edu ABSTRACT Given car dashboard video footage,
More informationMulti-View 3D Object Detection Network for Autonomous Driving
Multi-View 3D Object Detection Network for Autonomous Driving Xiaozhi Chen, Huimin Ma, Ji Wan, Bo Li, Tian Xia CVPR 2017 (Spotlight) Presented By: Jason Ku Overview Motivation Dataset Network Architecture
More informationEvaluation of Triple-Stream Convolutional Networks for Action Recognition
Evaluation of Triple-Stream Convolutional Networks for Action Recognition Dichao Liu, Yu Wang and Jien Kato Graduate School of Informatics Nagoya University Nagoya, Japan Email: {liu, ywang, jien} (at)
More informationFace Recognition A Deep Learning Approach
Face Recognition A Deep Learning Approach Lihi Shiloh Tal Perl Deep Learning Seminar 2 Outline What about Cat recognition? Classical face recognition Modern face recognition DeepFace FaceNet Comparison
More informationMultilayer and Multimodal Fusion of Deep Neural Networks for Video Classification
Multilayer and Multimodal Fusion of Deep Neural Networks for Video Classification Xiaodong Yang, Pavlo Molchanov, Jan Kautz INTELLIGENT VIDEO ANALYTICS Surveillance event detection Human-computer interaction
More informationDeep Learning and Its Applications
Convolutional Neural Network and Its Application in Image Recognition Oct 28, 2016 Outline 1 A Motivating Example 2 The Convolutional Neural Network (CNN) Model 3 Training the CNN Model 4 Issues and Recent
More informationVideo Gesture Recognition with RGB-D-S Data Based on 3D Convolutional Networks
Video Gesture Recognition with RGB-D-S Data Based on 3D Convolutional Networks August 16, 2016 1 Team details Team name FLiXT Team leader name Yunan Li Team leader address, phone number and email address:
More informationDeep Learning with Tensorflow AlexNet
Machine Learning and Computer Vision Group Deep Learning with Tensorflow http://cvml.ist.ac.at/courses/dlwt_w17/ AlexNet Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton, "Imagenet classification
More informationDeep Learning for Computer Vision with MATLAB By Jon Cherrie
Deep Learning for Computer Vision with MATLAB By Jon Cherrie 2015 The MathWorks, Inc. 1 Deep learning is getting a lot of attention "Dahl and his colleagues won $22,000 with a deeplearning system. 'We
More informationArbitrary Style Transfer in Real-Time with Adaptive Instance Normalization. Presented by: Karen Lucknavalai and Alexandr Kuznetsov
Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization Presented by: Karen Lucknavalai and Alexandr Kuznetsov Example Style Content Result Motivation Transforming content of an image
More informationSSD: Single Shot MultiBox Detector. Author: Wei Liu et al. Presenter: Siyu Jiang
SSD: Single Shot MultiBox Detector Author: Wei Liu et al. Presenter: Siyu Jiang Outline 1. Motivations 2. Contributions 3. Methodology 4. Experiments 5. Conclusions 6. Extensions Motivation Motivation
More informationarxiv: v2 [cs.cv] 2 Apr 2018
Depth of 3D CNNs Depth of 2D CNNs Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet? arxiv:1711.09577v2 [cs.cv] 2 Apr 2018 Kensho Hara, Hirokatsu Kataoka, Yutaka Satoh National Institute
More informationSemantic Segmentation
Semantic Segmentation UCLA:https://goo.gl/images/I0VTi2 OUTLINE Semantic Segmentation Why? Paper to talk about: Fully Convolutional Networks for Semantic Segmentation. J. Long, E. Shelhamer, and T. Darrell,
More informationDisguised Face Identification (DFI) with Facial KeyPoints using Spatial Fusion Convolutional Network. Nathan Sun CIS601
Disguised Face Identification (DFI) with Facial KeyPoints using Spatial Fusion Convolutional Network Nathan Sun CIS601 Introduction Face ID is complicated by alterations to an individual s appearance Beard,
More informationMartian lava field, NASA, Wikipedia
Martian lava field, NASA, Wikipedia Old Man of the Mountain, Franconia, New Hampshire Pareidolia http://smrt.ccel.ca/203/2/6/pareidolia/ Reddit for more : ) https://www.reddit.com/r/pareidolia/top/ Pareidolia
More informationarxiv: v1 [cs.cv] 22 Nov 2017
D Nets: New Architecture and Transfer Learning for Video Classification Ali Diba,4,, Mohsen Fayyaz,, Vivek Sharma, Amir Hossein Karami 4, Mohammad Mahdi Arzani 4, Rahman Yousefzadeh 4, Luc Van Gool,4 ESAT-PSI,
More informationMIXED PRECISION TRAINING: THEORY AND PRACTICE Paulius Micikevicius
MIXED PRECISION TRAINING: THEORY AND PRACTICE Paulius Micikevicius What is Mixed Precision Training? Reduced precision tensor math with FP32 accumulation, FP16 storage Successfully used to train a variety
More informationFuzzy Set Theory in Computer Vision: Example 3, Part II
Fuzzy Set Theory in Computer Vision: Example 3, Part II Derek T. Anderson and James M. Keller FUZZ-IEEE, July 2017 Overview Resource; CS231n: Convolutional Neural Networks for Visual Recognition https://github.com/tuanavu/stanford-
More informationConvolutional Neural Networks + Neural Style Transfer. Justin Johnson 2/1/2017
Convolutional Neural Networks + Neural Style Transfer Justin Johnson 2/1/2017 Outline Convolutional Neural Networks Convolution Pooling Feature Visualization Neural Style Transfer Feature Inversion Texture
More informationStoryline Reconstruction for Unordered Images
Introduction: Storyline Reconstruction for Unordered Images Final Paper Sameedha Bairagi, Arpit Khandelwal, Venkatesh Raizaday Storyline reconstruction is a relatively new topic and has not been researched
More informationAn Exploration of Computer Vision Techniques for Bird Species Classification
An Exploration of Computer Vision Techniques for Bird Species Classification Anne L. Alter, Karen M. Wang December 15, 2017 Abstract Bird classification, a fine-grained categorization task, is a complex
More informationSupplementary Material: Unsupervised Domain Adaptation for Face Recognition in Unlabeled Videos
Supplementary Material: Unsupervised Domain Adaptation for Face Recognition in Unlabeled Videos Kihyuk Sohn 1 Sifei Liu 2 Guangyu Zhong 3 Xiang Yu 1 Ming-Hsuan Yang 2 Manmohan Chandraker 1,4 1 NEC Labs
More informationConvolution Neural Networks for Chinese Handwriting Recognition
Convolution Neural Networks for Chinese Handwriting Recognition Xu Chen Stanford University 450 Serra Mall, Stanford, CA 94305 xchen91@stanford.edu Abstract Convolutional neural networks have been proven
More informationWeighted Convolutional Neural Network. Ensemble.
Weighted Convolutional Neural Network Ensemble Xavier Frazão and Luís A. Alexandre Dept. of Informatics, Univ. Beira Interior and Instituto de Telecomunicações Covilhã, Portugal xavierfrazao@gmail.com
More informationConvolutional-Recursive Deep Learning for 3D Object Classification
Convolutional-Recursive Deep Learning for 3D Object Classification Richard Socher, Brody Huval, Bharath Bhat, Christopher D. Manning, Andrew Y. Ng NIPS 2012 Iro Armeni, Manik Dhar Motivation Hand-designed
More informationObject Recognition II
Object Recognition II Linda Shapiro EE/CSE 576 with CNN slides from Ross Girshick 1 Outline Object detection the task, evaluation, datasets Convolutional Neural Networks (CNNs) overview and history Region-based
More informationIntro to Deep Learning. Slides Credit: Andrej Karapathy, Derek Hoiem, Marc Aurelio, Yann LeCunn
Intro to Deep Learning Slides Credit: Andrej Karapathy, Derek Hoiem, Marc Aurelio, Yann LeCunn Why this class? Deep Features Have been able to harness the big data in the most efficient and effective
More informationarxiv: v2 [cs.cv] 31 May 2018
Graph Edge Convolutional Neural Networks for Skeleton Based Action Recognition Xikun Zhang, Chang Xu, Xinmei Tian, and Dacheng Tao June 1, 2018 arxiv:1805.06184v2 [cs.cv] 31 May 2018 Abstract This paper
More informationInception and Residual Networks. Hantao Zhang. Deep Learning with Python.
Inception and Residual Networks Hantao Zhang Deep Learning with Python https://en.wikipedia.org/wiki/residual_neural_network Deep Neural Network Progress from Large Scale Visual Recognition Challenge (ILSVRC)
More informationarxiv: v1 [cs.cv] 14 Jul 2017
Temporal Modeling Approaches for Large-scale Youtube-8M Video Understanding Fu Li, Chuang Gan, Xiao Liu, Yunlong Bian, Xiang Long, Yandong Li, Zhichao Li, Jie Zhou, Shilei Wen Baidu IDL & Tsinghua University
More informationCENG 783. Special topics in. Deep Learning. AlchemyAPI. Week 11. Sinan Kalkan
CENG 783 Special topics in Deep Learning AlchemyAPI Week 11 Sinan Kalkan TRAINING A CNN Fig: http://www.robots.ox.ac.uk/~vgg/practicals/cnn/ Feed-forward pass Note that this is written in terms of the
More informationMulti-Glance Attention Models For Image Classification
Multi-Glance Attention Models For Image Classification Chinmay Duvedi Stanford University Stanford, CA cduvedi@stanford.edu Pararth Shah Stanford University Stanford, CA pararth@stanford.edu Abstract We
More informationObject Detection Based on Deep Learning
Object Detection Based on Deep Learning Yurii Pashchenko AI Ukraine 2016, Kharkiv, 2016 Image classification (mostly what you ve seen) http://tutorial.caffe.berkeleyvision.org/caffe-cvpr15-detection.pdf
More informationConvolu'onal Neural Networks
Convolu'onal Neural Networks Dr. Kira Radinsky CTO SalesPredict Visi8ng Professor/Scien8st Technion Slides were adapted from Fei-Fei Li & Andrej Karpathy & Jus8n Johnson A bit of history: Hubel & Wiesel,
More informationA Torch Library for Action Recognition and Detection Using CNNs and LSTMs
A Torch Library for Action Recognition and Detection Using CNNs and LSTMs Gary Thung and Helen Jiang Stanford University {gthung, helennn}@stanford.edu Abstract It is very common now to see deep neural
More informationConvolutional Neural Networks: Applications and a short timeline. 7th Deep Learning Meetup Kornel Kis Vienna,
Convolutional Neural Networks: Applications and a short timeline 7th Deep Learning Meetup Kornel Kis Vienna, 1.12.2016. Introduction Currently a master student Master thesis at BME SmartLab Started deep
More informationVGR-Net: A View Invariant Gait Recognition Network
VGR-Net: A View Invariant Gait Recognition Network Human gait has many advantages over the conventional biometric traits (like fingerprint, ear, iris etc.) such as its non-invasive nature and comprehensible
More informationTemporal Activity Detection in Untrimmed Videos with Recurrent Neural Networks
Temporal Activity Detection in Untrimmed Videos with Recurrent Neural Networks Alberto Montes al.montes.gomez@gmail.com Santiago Pascual TALP Research Center santiago.pascual@tsc.upc.edu Amaia Salvador
More informationSpatial Localization and Detection. Lecture 8-1
Lecture 8: Spatial Localization and Detection Lecture 8-1 Administrative - Project Proposals were due on Saturday Homework 2 due Friday 2/5 Homework 1 grades out this week Midterm will be in-class on Wednesday
More informationLSTM and its variants for visual recognition. Xiaodan Liang Sun Yat-sen University
LSTM and its variants for visual recognition Xiaodan Liang xdliang328@gmail.com Sun Yat-sen University Outline Context Modelling with CNN LSTM and its Variants LSTM Architecture Variants Application in
More informationClass 9 Action Recognition
Class 9 Action Recognition Liangliang Cao, April 4, 2013 EECS 6890 Topics in Information Processing Spring 2013, Columbia University http://rogerioferis.com/visualrecognitionandsearch Visual Recognition
More informationConvolutional Neural Networks
NPFL114, Lecture 4 Convolutional Neural Networks Milan Straka March 25, 2019 Charles University in Prague Faculty of Mathematics and Physics Institute of Formal and Applied Linguistics unless otherwise
More informationAdaDepth: Unsupervised Content Congruent Adaptation for Depth Estimation
AdaDepth: Unsupervised Content Congruent Adaptation for Depth Estimation Introduction Supplementary material In the supplementary material, we present additional qualitative results of the proposed AdaDepth
More informationMask R-CNN. presented by Jiageng Zhang, Jingyao Zhan, Yunhan Ma
Mask R-CNN presented by Jiageng Zhang, Jingyao Zhan, Yunhan Ma Mask R-CNN Background Related Work Architecture Experiment Mask R-CNN Background Related Work Architecture Experiment Background From left
More informationAdvanced Video Analysis & Imaging
Advanced Video Analysis & Imaging (5LSH0), Module 09B Machine Learning with Convolutional Neural Networks (CNNs) - Workout Farhad G. Zanjani, Clint Sebastian, Egor Bondarev, Peter H.N. de With ( p.h.n.de.with@tue.nl
More informationDeep Tracking: Biologically Inspired Tracking with Deep Convolutional Networks
Deep Tracking: Biologically Inspired Tracking with Deep Convolutional Networks Si Chen The George Washington University sichen@gwmail.gwu.edu Meera Hahn Emory University mhahn7@emory.edu Mentor: Afshin
More informationImage and Video Understanding
Image and Video Understanding 2VO 70.095 WS Christoph Feichtenhofer, Axel Pinz Slide credits: Many thanks to all the great computer vision researchers on which this presentation relies on. Most material
More informationKeras: Handwritten Digit Recognition using MNIST Dataset
Keras: Handwritten Digit Recognition using MNIST Dataset IIT PATNA January 31, 2018 1 / 30 OUTLINE 1 Keras: Introduction 2 Installing Keras 3 Keras: Building, Testing, Improving A Simple Network 2 / 30
More informationFully Convolutional Networks for Semantic Segmentation
Fully Convolutional Networks for Semantic Segmentation Jonathan Long* Evan Shelhamer* Trevor Darrell UC Berkeley Chaim Ginzburg for Deep Learning seminar 1 Semantic Segmentation Define a pixel-wise labeling
More informationActiveStereoNet: End-to-End Self-Supervised Learning for Active Stereo Systems (Supplementary Materials)
ActiveStereoNet: End-to-End Self-Supervised Learning for Active Stereo Systems (Supplementary Materials) Yinda Zhang 1,2, Sameh Khamis 1, Christoph Rhemann 1, Julien Valentin 1, Adarsh Kowdle 1, Vladimir
More informationPerceptron: This is convolution!
Perceptron: This is convolution! v v v Shared weights v Filter = local perceptron. Also called kernel. By pooling responses at different locations, we gain robustness to the exact spatial location of image
More informationComputer Vision Lecture 16
Announcements Computer Vision Lecture 16 Deep Learning Applications 11.01.2017 Seminar registration period starts on Friday We will offer a lab course in the summer semester Deep Robot Learning Topic:
More informationComputer Vision Lecture 16
Computer Vision Lecture 16 Deep Learning Applications 11.01.2017 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Announcements Seminar registration period starts
More informationCS 1674: Intro to Computer Vision. Neural Networks. Prof. Adriana Kovashka University of Pittsburgh November 16, 2016
CS 1674: Intro to Computer Vision Neural Networks Prof. Adriana Kovashka University of Pittsburgh November 16, 2016 Announcements Please watch the videos I sent you, if you haven t yet (that s your reading)
More informationCS 1674: Intro to Computer Vision. Object Recognition. Prof. Adriana Kovashka University of Pittsburgh April 3, 5, 2018
CS 1674: Intro to Computer Vision Object Recognition Prof. Adriana Kovashka University of Pittsburgh April 3, 5, 2018 Different Flavors of Object Recognition Semantic Segmentation Classification + Localization
More informationCSE 559A: Computer Vision
CSE 559A: Computer Vision Fall 2018: T-R: 11:30-1pm @ Lopata 101 Instructor: Ayan Chakrabarti (ayan@wustl.edu). Course Staff: Zhihao Xia, Charlie Wu, Han Liu http://www.cse.wustl.edu/~ayan/courses/cse559a/
More informationRecognition of Animal Skin Texture Attributes in the Wild. Amey Dharwadker (aap2174) Kai Zhang (kz2213)
Recognition of Animal Skin Texture Attributes in the Wild Amey Dharwadker (aap2174) Kai Zhang (kz2213) Motivation Patterns and textures are have an important role in object description and understanding
More informationInception Network Overview. David White CS793
Inception Network Overview David White CS793 So, Leonardo DiCaprio dreams about dreaming... https://m.media-amazon.com/images/m/mv5bmjaxmzy3njcxnf5bml5banbnxkftztcwnti5otm0mw@@._v1_sy1000_cr0,0,675,1 000_AL_.jpg
More informationRGBD Occlusion Detection via Deep Convolutional Neural Networks
1 RGBD Occlusion Detection via Deep Convolutional Neural Networks Soumik Sarkar 1,2, Vivek Venugopalan 1, Kishore Reddy 1, Michael Giering 1, Julian Ryde 3, Navdeep Jaitly 4,5 1 United Technologies Research
More informationCAP 6412 Advanced Computer Vision
CAP 6412 Advanced Computer Vision http://www.cs.ucf.edu/~bgong/cap6412.html Boqing Gong April 21st, 2016 Today Administrivia Free parameters in an approach, model, or algorithm? Egocentric videos by Aisha
More informationCamera-based Vehicle Velocity Estimation using Spatiotemporal Depth and Motion Features
Camera-based Vehicle Velocity Estimation using Spatiotemporal Depth and Motion Features Moritz Kampelmuehler* kampelmuehler@student.tugraz.at Michael Mueller* michael.g.mueller@student.tugraz.at Christoph
More informationHello Edge: Keyword Spotting on Microcontrollers
Hello Edge: Keyword Spotting on Microcontrollers Yundong Zhang, Naveen Suda, Liangzhen Lai and Vikas Chandra ARM Research, Stanford University arxiv.org, 2017 Presented by Mohammad Mofrad University of
More informationFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks Shaoqing Ren Kaiming He Ross Girshick Jian Sun Present by: Yixin Yang Mingdong Wang 1 Object Detection 2 1 Applications Basic
More informationDeep Face Recognition. Nathan Sun
Deep Face Recognition Nathan Sun Why Facial Recognition? Picture ID or video tracking Higher Security for Facial Recognition Software Immensely useful to police in tracking suspects Your face will be an
More information