Large-Scale Live Active Learning: Training Object Detectors with Crawled Data and Crowds
|
|
- Letitia Parks
- 5 years ago
- Views:
Transcription
1 Large-Scale Live Active Learning: Training Object Detectors with Crawled Data and Crowds Sudheendra Vijayanarasimhan Kristen Grauman Department of Computer Science University of Texas at Austin Austin, Texas
2 Object Detection Challenge: Best results require large amount of cleanly labeled training examples. 2
3 Ways to Reduce Effort Active learning minimize effort by focusing label requests on the most informative examples [Kapoor et al. ICCV 2007, Qi et al. CVPR 2008, Vijayanarasimhan et al. CVPR 2009, Joshi et al. CVPR 2009, Siddique et al. CVPR 2010] 3
4 Ways to Reduce Effort Active learning Labeled data Unlabeled data minimize effort by focusing label requests on the most informative examples [Kapoor et al. ICCV 2007, Qi et al. CVPR 2008, Vijayanarasimhan et al. CVPR 2009, Joshi et al. CVPR 2009, Siddique et al. CVPR 2010] 3
5 Ways to Reduce Effort Active learning Labeled data Current category model Unlabeled data minimize effort by focusing label requests on the most informative examples [Kapoor et al. ICCV 2007, Qi et al. CVPR 2008, Vijayanarasimhan et al. CVPR 2009, Joshi et al. CVPR 2009, Siddique et al. CVPR 2010] 3
6 Ways to Reduce Effort Active learning Labeled data Current category model Unlabeled data Active Selection Function minimize effort by focusing label requests on the most informative examples [Kapoor et al. ICCV 2007, Qi et al. CVPR 2008, Vijayanarasimhan et al. CVPR 2009, Joshi et al. CVPR 2009, Siddique et al. CVPR 2010] 3
7 Ways to Reduce Effort Active learning Labeled data Current category model Unlabeled data Active Selection Function Actively selected example(s) minimize effort by focusing label requests on the most informative examples [Kapoor et al. ICCV 2007, Qi et al. CVPR 2008, Vijayanarasimhan et al. CVPR 2009, Joshi et al. CVPR 2009, Siddique et al. CVPR 2010] 3
8 Ways to Reduce Effort Active learning Labeled data Annotator(s) Current category model Unlabeled data Annotation request Active Selection Function Actively selected example(s) minimize effort by focusing label requests on the most informative examples [Kapoor et al. ICCV 2007, Qi et al. CVPR 2008, Vijayanarasimhan et al. CVPR 2009, Joshi et al. CVPR 2009, Siddique et al. CVPR 2010] 3
9 Ways to Reduce Effort Active learning Labeled data Annotator(s) Current category model Unlabeled data Annotation request Active Selection Function Actively selected example(s) minimize effort by focusing label requests on the most informative examples [Kapoor et al. ICCV 2007, Qi et al. CVPR 2008, Vijayanarasimhan et al. CVPR 2009, Joshi et al. CVPR 2009, Siddique et al. CVPR 2010] 3
10 Ways to Reduce Effort Active learning Crowd-sourced annotations Labeled data Annotator(s) Current category model Unlabeled data Annotation request Active Selection Function minimize effort by focusing label requests on the most informative examples Actively selected example(s) [Kapoor et al. ICCV 2007, Qi et al. CVPR 2008, Vijayanarasimhan et al. CVPR 2009, Joshi et al. CVPR 2009, Siddique et al. CVPR 2010] package annotation tasks to obtain from online human workers [von Ahn et al. CHI 2004, Russell et al. IJCV 2007, Sorokin et al. 2008, Welinder et al. ACVHL 2010, Deng et al, CVPR 2009] 3
11 Problem Thus far techniques are only tested in artificially controlled settings: 4
12 Problem Thus far techniques are only tested in artificially controlled settings: use sandbox datasets - dataset s source and scope is fixed 4
13 Problem Thus far techniques are only tested in artificially controlled settings: use sandbox datasets - dataset s source and scope is fixed computational cost of active selection and retraining the model generally ignored - linear/quadratic time 4
14 Problem Thus far techniques are only tested in artificially controlled settings: use sandbox datasets - dataset s source and scope is fixed computational cost of active selection and retraining the model generally ignored - linear/quadratic time crowd-sourced collection requires iterative fine-tuning 4
15 Goal Goal Take active learning and crowd-sourced annotation collection out of the sandbox. 5
16 Goal Goal Take active learning and crowd-sourced annotation collection out of the sandbox. break free from dataset-based learning 5
17 Goal Goal Take active learning and crowd-sourced annotation collection out of the sandbox. break free from dataset-based learning collect information on the fly (no manual intervention) 5
18 Goal Goal Take active learning and crowd-sourced annotation collection out of the sandbox. break free from dataset-based learning collect information on the fly (no manual intervention) large-scale data 5
19 Goal Goal Take active learning and crowd-sourced annotation collection out of the sandbox. break free from dataset-based learning collect information on the fly (no manual intervention) large-scale data Our Approach: Live Learning Live active learning system that autonomously builds models for object detection 5
20 Goal Goal Take active learning and crowd-sourced annotation collection out of the sandbox. break free from dataset-based learning collect information on the fly (no manual intervention) large-scale data Our Approach: Live Learning Live active learning system that autonomously builds models for object detection bicycle Category model 5
21 Our Approach: Live Learning bicycle 6
22 Our Approach: Live Learning bicycle Unlabeled images 6
23 Our Approach: Live Learning bicycle Generate object windows Unlabeled images Unlabeled windows 6
24 Our Approach: Live Learning bicycle Select images to annotate Unlabeled images Generate object windows Unlabeled windows Actively selected examples 6
25 Our Approach: Live Learning Labeled data Online annotation collection bicycle Select images to annotate Unlabeled images Generate object windows Unlabeled windows Actively selected examples 6
26 Our Approach: Live Learning Labeled data Online annotation collection Object representation and classifier bicycle Category model Select images to annotate Unlabeled images Generate object windows Unlabeled windows Actively selected examples 6
27 Main Contributions Linear classification part-based linear detector based on non-linear feature coding 7
28 Main Contributions Linear classification part-based linear detector based on non-linear feature coding Large-scale active selection sub-linear time hashing scheme for efficiently selecting uncertain examples [ Jain, Vijayanarasimhan & Grauman, NIPS 2010] 7
29 Main Contributions Linear classification part-based linear detector based on non-linear feature coding Large-scale active selection sub-linear time hashing scheme for efficiently selecting uncertain examples [ Jain, Vijayanarasimhan & Grauman, NIPS 2010] Live learning results for active detection of unprecedented scale and autonomy for the first time 7
30 Outline Labeled data Online annotation collection? Object representation and classifier bicycle Category model Select images to annotate Unlabeled images Generate object windows Unlabeled windows Actively selected examples 8
31 Outline Labeled data bicycle? Object representation and classifier Online annotation collection Linear classification fast/incremental training using linear SVM Category model Select images to annotate Unlabeled images Generate object windows Unlabeled windows Actively selected examples 8
32 Outline Labeled data bicycle Generate object windows? Object representation and classifier Category model Online annotation collection Select images to annotate Actively selected examples Linear classification fast/incremental training using linear SVM efficient active selection using our hyperplane hash functions Unlabeled images Unlabeled windows 8
33 Object Representation and Classifier Part based object representation 9
34 Object Representation and Classifier Part based object representation Root 9
35 Object Representation and Classifier Part based object representation Root Parts 9
36 Object Representation and Classifier Part based object representation Root Parts Context 9
37 Object Representation and Classifier Part based object representation Sparse Max Pooling Root [φ(r) Parts Context 9
38 Object Representation and Classifier Part based object representation Sparse Max Pooling Root Parts [φ(r), φ(p 1 ) φ(p P ) Context 9
39 Object Representation and Classifier Part based object representation Sparse Max Pooling Root Parts Context [φ(r), φ(p 1 ) φ(p P ), φ(c 1 ) φ(c C )] 9
40 Object Representation and Classifier Part based object representation Sparse Max Pooling Root Parts Context [φ(r), φ(p 1 ) φ(p P ), φ(c 1 ) φ(c C )] Sparse Max Pooling [Yang et al. 10] similar to bag of words: 9
41 Object Representation and Classifier Part based object representation Sparse Max Pooling Root Parts Context [φ(r), φ(p 1 ) φ(p P ), φ(c 1 ) φ(c C )] Sparse Max Pooling [Yang et al. 10] similar to bag of words: sparse coding fuller representation of original features 9
42 Object Representation and Classifier Part based object representation Sparse Max Pooling Root Parts Context [φ(r), φ(p 1 ) φ(p P ), φ(c 1 ) φ(c C )] Sparse Max Pooling [Yang et al. 10] similar to bag of words: sparse coding fuller representation of original features max pooling better discriminability in clutter [Boureau 10]. 9
43 Object Representation and Classifier Part based object representation Score windows as linear sum: Sparse Max Pooling Root Parts Context [φ(r), φ(p 1 ) φ(p P ), φ(c 1 ) φ(c C )] f (O) = w T ϕ(o) P = w r ϕ(r) + i=1 C + w ci ϕ(c i ), i=1 w - linear SVM weights w pi ϕ(p i ) Sparse Max Pooling [Yang et al. 10] similar to bag of words: sparse coding fuller representation of original features max pooling better discriminability in clutter [Boureau 10]. 9
44 Object Representation and Classifier Relationship to existing detection models Spatial pyramid (SP) [Lazebnik et al. 06, Vedaldi et al. 09] Hard VQ +avg pooling local features, discard locs per window + + Latent SVM (LSVM) [Felzenswalb et al. 09] + root parts deformations dense gradients at fixed locs within window + 10
45 Object Representation and Classifier Relationship to existing detection models Spatial pyramid (SP) [Lazebnik et al. 06, Vedaldi et al. 09] Hard VQ +avg pooling local features, discard locs per window + + Sparse code +max pooling Ours root Latent SVM (LSVM) [Felzenswalb et al. 09] + root parts deformations dense gradients at fixed locs within window + parts + local features, discard locs per window (r) (p 1 ) (p P ) 10
46 Object Representation and Classifier Relationship to existing detection models Spatial pyramid (SP) [Lazebnik et al. 06, Vedaldi et al. 09] Hard VQ +avg pooling local features, discard locs per window + + Sparse code +max pooling Ours root Latent SVM (LSVM) [Felzenswalb et al. 09] + root parts deformations dense gradients at fixed locs within window + parts + local features, discard locs per window (r) (p 1 ) (p P ) faster training (linear SVM) 10
47 Object Representation and Classifier Relationship to existing detection models Spatial pyramid (SP) [Lazebnik et al. 06, Vedaldi et al. 09] Hard VQ +avg pooling local features, discard locs per window + + Sparse code +max pooling Ours root Latent SVM (LSVM) [Felzenswalb et al. 09] + root parts deformations dense gradients at fixed locs within window + parts + local features, discard locs per window (r) (p 1 ) (p P ) faster training (linear SVM) results comparable to non-linear detectors 10
48 Selecting Images to Annotate Labeled data Online annotation collection Part-based linear detector bicycle Jumping window prediction Category model Select images to annotate? Actively selected examples Unlabeled images Unlabeled windows 11
49 Selecting Images to Annotate Labeled data Online annotation collection bicycle Jumping window prediction Part-based linear detector Category model Select images to annotate? Actively selected examples Efficient active selection select most useful windows from millions Unlabeled images Unlabeled windows 11
50 Active Selection of Object Windows SVM SVM margin margin criterion criterion for activefor selection active selection? w Select point Select nearest point to nearest to hyperplane hyperplane decision decision boundary for labeling. boundary for labeling. x = argmin w T x i x i U [Tong & Koller, 2000; Schohn & Cohen, 2000; [Tong & Koller, Campbell 2000; etschohn al. 2000] & Cohn, 2000; Campbell et al. 2000] Problem: With massive unlabeled pool, cannot afford exhaustive linear scan to make selection. 12
51 Active Selection of Object Windows SVM SVM margin margin criterion criterion for activefor selection active selection? w Select point Select nearest point to nearest to hyperplane hyperplane decision decision boundary for labeling. boundary for labeling. x = argmin w T x i x i U [Tong & Koller, 2000; Schohn & Cohen, 2000; [Tong & Koller, Campbell 2000; etschohn al. 2000] & Cohn, 2000; Campbell et al. 2000] Problem: With massive unlabeled pool, cannot afford exhaustive linear scan to make selection. Problem: With massive unlabeled pool, cannot afford exhaustive linear scan to make selection. 12
52 Active Selection of Object Windows Sub-linear time selection through hyperplane hashing hash function h(.) - high probability of collision when ϕ(o) close to w [Jain, Vijayanarasimhan and Grauman, NIPS 2010] 13
53 Active Selection of Object Windows Sub-linear time selection through hyperplane hashing hash function h(.) - high probability of collision when ϕ(o) close to w [Jain, Vijayanarasimhan and Grauman, NIPS 2010] Unlabeled windows 13
54 Active Selection of Object Windows Sub-linear time selection through hyperplane hashing [Jain, Vijayanarasimhan and Grauman, NIPS 2010] hash function h(.) - high probability of collision when ϕ(o) close to w preprocess - hash unlabeled windows into table h( ϕ O )) ( i 1100 Hash table Unlabeled windows 13
55 Active Selection of Object Windows Sub-linear time selection through hyperplane hashing [Jain, Vijayanarasimhan and Grauman, NIPS 2010] hash function h(.) - high probability of collision when ϕ(o) close to w preprocess - hash unlabeled windows into table h( ϕ O )) ( i Hash table Unlabeled windows 13
56 Active Selection of Object Windows Sub-linear time selection through hyperplane hashing [Jain, Vijayanarasimhan and Grauman, NIPS 2010] hash function h(.) - high probability of collision when ϕ(o) close to w preprocess - hash unlabeled windows into table active learning loop - hash classifier w and retrieve examples Category model Unlabeled windows h( ϕ O )) ( i Hash table 13
57 Active Selection of Object Windows Sub-linear time selection through hyperplane hashing [Jain, Vijayanarasimhan and Grauman, NIPS 2010] hash function h(.) - high probability of collision when ϕ(o) close to w preprocess - hash unlabeled windows into table active learning loop - hash classifier w and retrieve examples Category model Unlabeled windows h( w) h( ϕ O )) ( i Hash table 13
58 Active Selection of Object Windows Sub-linear time selection through hyperplane hashing [Jain, Vijayanarasimhan and Grauman, NIPS 2010] hash function h(.) - high probability of collision when ϕ(o) close to w preprocess - hash unlabeled windows into table active learning loop - hash classifier w and retrieve examples Category model Unlabeled windows h( w) h( ϕ O )) ( i Hash table Sub-linear Time Retrieval 13
59 Active Selection of Object Windows Sub-linear time selection through hyperplane hashing [Jain, Vijayanarasimhan and Grauman, NIPS 2010] hash function h(.) - high probability of collision when ϕ(o) close to w preprocess - hash unlabeled windows into table active learning loop - hash classifier w and retrieve examples evaluate 10 3 windows vs for exhaustive Category model Unlabeled windows h( w) h( ϕ O )) ( i Hash table Sub-linear Time Retrieval 13
60 Online Annotation Collection Labeled data? Online annotation collection bicycle Part-based linear detector Category model h( w) on the fly reliable annotations without pruning 1111 Jumping window prediction h( ϕ O )) ( i Hash table Actively selected examples Unlabeled images Unlabeled windows 14
61 Online Annotation Collection Mechanical Turk Interface 15
62 Online Annotation Collection Mechanical Turk Interface post same image to multiple (5-10) annotators 15
63 Online Annotation Collection Mechanical Turk Interface post same image to multiple (5-10) annotators cluster all bounding boxes to obtain consensus 15
64 Online Annotation Collection Mechanical Turk Interface post same image to multiple (5-10) annotators cluster all bounding boxes to obtain consensus 15
65 Summary: Live Learning Labeled data Consensus (Mean shift) Part-based linear detector bicycle h( w) 1100 Category model Jumping window prediction ( O )) h ϕ ( i Hash table Actively selected examples Unlabeled images Unlabeled windows 16
66 Results PASCAL 2007 challenge 20 different objects under changes in viewpoint, scale, and background clutter training and test examples given an image detect all objects 17
67 Results PASCAL 2007 challenge 20 different objects under changes in viewpoint, scale, and background clutter training and test examples given an image detect all objects Live learning on Flickr 6 of the most challenging PASCAL objects New Flickr testset 17
68 Results PASCAL 2007 challenge 20 different objects under changes in viewpoint, scale, and background clutter training and test examples given an image detect all objects Live learning on Flickr 6 of the most challenging PASCAL objects New Flickr testset Features 30,000 SIFT features densely extracted 60,000 words with hierarchical kmeans sparse coding using LLC [Yang et al. 10] 17
69 Results PASCAL 2007 challenge 20 different objects under changes in viewpoint, scale, and background clutter training and test examples given an image detect all objects Live learning on Flickr 6 of the most challenging PASCAL objects New Flickr testset Features 30,000 SIFT features densely extracted 60,000 words with hierarchical kmeans sparse coding using LLC [Yang et al. 10] Implementation 12 parts from LSVM detector 100 images per active iteration 17
70 Sandbox Results (PASCAL 2007) Comparison to state-of-art aero. cat dog sheep sofa train bicyc. bird boat bottl bus Mean BoF SP Ours part-based, single feature representation, linear model
71 Sandbox Results (PASCAL 2007) Comparison to state-of-art aero. cat dog sheep sofa train bicyc. bird boat bottl bus Mean BoF SP Ours LSVM+HOG SP+MKL [Felzenszwalb et al. 09] 2 [Vedaldi et al. 09] part-based, single feature representation, linear model competitive with state-of-art (better for 6 classes) 18
72 Live Learning Results Live learning tested on PASCAL testset 19
73 Live Learning Results Live learning tested on PASCAL testset bird boat dog potted plant sheep chair Ours Previous best Significant improvements in state-of-art on challenging categories 19
74 Live Learning Results Live learning tested on PASCAL testset bird boat dog potted plant sheep chair Ours Previous best Significant improvements in state-of-art on challenging categories Computation Time Active selection Training Detection per image Ours + active 10 mins 5 mins 150 secs LSVM [Felzenszwalb et al. 2009] 3 hours 4 hours 2 secs SP+MKL [Vedaldi et al. 2009] 93 hours > 2 days 67 secs Our approach s efficiency makes live learning feasible. 19
75 Live Learning Results Live learning tested on Flickr testset General test set of web images 20
76 Live Learning Results Live learning tested on Flickr testset General test set of web images keyword+image: randomly select a keyword filtered image to get bounding box keyword+window: randomly select a window within a keyword filtered image and obtain binary label 20
77 Live Learning Results Live learning tested on Flickr testset General test set of web images Average Precision Live active (ours) Keyword+image Keyword+window boat dog bird pottedplant sheep Annotations added, out of 3 million examples chair keyword+image: randomly select a keyword filtered image to get bounding box keyword+window: randomly select a window within a keyword filtered image and obtain binary label 20
78 Live Learning Results Live learning tested on Flickr testset General test set of web images Average Precision Live active (ours) Keyword+image Keyword+window boat dog bird pottedplant sheep Annotations added, out of 3 million examples chair keyword+image: randomly select a keyword filtered image to get bounding box keyword+window: randomly select a window within a keyword filtered image and obtain binary label dramatic improvements for most categories 20
79 Live Learning Results Live learning tested on Flickr testset General test set of web images Average Precision Live active (ours) Keyword+image Keyword+window boat dog bird pottedplant sheep Annotations added, out of 3 million examples chair keyword+image: randomly select a keyword filtered image to get bounding box keyword+window: randomly select a window within a keyword filtered image and obtain binary label dramatic improvements for most categories outperforms status quo approach of learning 20
80 Conclusions autonomous online learning - break-free from sandbox learning no intervention in example/annotation selection or pruning obtains results better than state-of-art on challenging PASCAL dataset largest scale active learning results to our knowledge 21
Large-Scale Live Active Learning: Training Object Detectors with Crawled Data and Crowds
To appear, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2011. Large-Scale Live Active Learning: Training Object Detectors with Crawled Data and Crowds Sudheendra
More informationInstances on a Budget
Retrieving Similar or Informative Instances on a Budget Kristen Grauman Dept. of Computer Science University of Texas at Austin Work with Sudheendra Vijayanarasimham Work with Sudheendra Vijayanarasimham,
More informationCategory-level localization
Category-level localization Cordelia Schmid Recognition Classification Object present/absent in an image Often presence of a significant amount of background clutter Localization / Detection Localize object
More informationBag-of-features. Cordelia Schmid
Bag-of-features for category classification Cordelia Schmid Visual search Particular objects and scenes, large databases Category recognition Image classification: assigning a class label to the image
More informationBeyond Sliding Windows: Object Localization by Efficient Subwindow Search
Beyond Sliding Windows: Object Localization by Efficient Subwindow Search Christoph H. Lampert, Matthew B. Blaschko, & Thomas Hofmann Max Planck Institute for Biological Cybernetics Tübingen, Germany Google,
More informationSemantic Pooling for Image Categorization using Multiple Kernel Learning
Semantic Pooling for Image Categorization using Multiple Kernel Learning Thibaut Durand (1,2), Nicolas Thome (1), Matthieu Cord (1), David Picard (2) (1) Sorbonne Universités, UPMC Univ Paris 06, UMR 7606,
More informationLearning Representations for Visual Object Class Recognition
Learning Representations for Visual Object Class Recognition Marcin Marszałek Cordelia Schmid Hedi Harzallah Joost van de Weijer LEAR, INRIA Grenoble, Rhône-Alpes, France October 15th, 2007 Bag-of-Features
More informationDetection III: Analyzing and Debugging Detection Methods
CS 1699: Intro to Computer Vision Detection III: Analyzing and Debugging Detection Methods Prof. Adriana Kovashka University of Pittsburgh November 17, 2015 Today Review: Deformable part models How can
More informationDeformable Part Models
CS 1674: Intro to Computer Vision Deformable Part Models Prof. Adriana Kovashka University of Pittsburgh November 9, 2016 Today: Object category detection Window-based approaches: Last time: Viola-Jones
More informationObject Detection Based on Deep Learning
Object Detection Based on Deep Learning Yurii Pashchenko AI Ukraine 2016, Kharkiv, 2016 Image classification (mostly what you ve seen) http://tutorial.caffe.berkeleyvision.org/caffe-cvpr15-detection.pdf
More informationDevelopment in Object Detection. Junyuan Lin May 4th
Development in Object Detection Junyuan Lin May 4th Line of Research [1] N. Dalal and B. Triggs. Histograms of oriented gradients for human detection, CVPR 2005. HOG Feature template [2] P. Felzenszwalb,
More informationAttributes. Computer Vision. James Hays. Many slides from Derek Hoiem
Many slides from Derek Hoiem Attributes Computer Vision James Hays Recap: Human Computation Active Learning: Let the classifier tell you where more annotation is needed. Human-in-the-loop recognition:
More informationPart based models for recognition. Kristen Grauman
Part based models for recognition Kristen Grauman UT Austin Limitations of window-based models Not all objects are box-shaped Assuming specific 2d view of object Local components themselves do not necessarily
More information24 hours of Photo Sharing. installation by Erik Kessels
24 hours of Photo Sharing installation by Erik Kessels And sometimes Internet photos have useful labels Im2gps. Hays and Efros. CVPR 2008 But what if we want more? Image Categorization Training Images
More informationMulti-Level Active Prediction of Useful Image Annotations
Multi-Level Active Prediction of Useful Image Annotations Sudheendra Vijayanarasimhan and Kristen Grauman Department of Computer Sciences University of Texas at Austin Austin, Texas 78712 (svnaras,grauman)@cs.utexas.edu
More informationDeep condolence to Professor Mark Everingham
Deep condolence to Professor Mark Everingham Towards VOC2012 Object Classification Challenge Generalized Hierarchical Matching for Sub-category Aware Object Classification National University of Singapore
More informationTA Section: Problem Set 4
TA Section: Problem Set 4 Outline Discriminative vs. Generative Classifiers Image representation and recognition models Bag of Words Model Part-based Model Constellation Model Pictorial Structures Model
More informationIs 2D Information Enough For Viewpoint Estimation? Amir Ghodrati, Marco Pedersoli, Tinne Tuytelaars BMVC 2014
Is 2D Information Enough For Viewpoint Estimation? Amir Ghodrati, Marco Pedersoli, Tinne Tuytelaars BMVC 2014 Problem Definition Viewpoint estimation: Given an image, predicting viewpoint for object of
More informationCategory-level Localization
Category-level Localization Andrew Zisserman Visual Geometry Group University of Oxford http://www.robots.ox.ac.uk/~vgg Includes slides from: Ondra Chum, Alyosha Efros, Mark Everingham, Pedro Felzenszwalb,
More informationStructured Models in. Dan Huttenlocher. June 2010
Structured Models in Computer Vision i Dan Huttenlocher June 2010 Structured Models Problems where output variables are mutually dependent or constrained E.g., spatial or temporal relations Such dependencies
More informationPreviously. Part-based and local feature models for generic object recognition. Bag-of-words model 4/20/2011
Previously Part-based and local feature models for generic object recognition Wed, April 20 UT-Austin Discriminative classifiers Boosting Nearest neighbors Support vector machines Useful for object recognition
More informationHierarchical Learning for Object Detection. Long Zhu, Yuanhao Chen, William Freeman, Alan Yuille, Antonio Torralba MIT and UCLA, 2010
Hierarchical Learning for Object Detection Long Zhu, Yuanhao Chen, William Freeman, Alan Yuille, Antonio Torralba MIT and UCLA, 2010 Background I: Our prior work Our work for the Pascal Challenge is based
More informationLearning Object Representations for Visual Object Class Recognition
Learning Object Representations for Visual Object Class Recognition Marcin Marszalek, Cordelia Schmid, Hedi Harzallah, Joost Van de Weijer To cite this version: Marcin Marszalek, Cordelia Schmid, Hedi
More informationUnified, real-time object detection
Unified, real-time object detection Final Project Report, Group 02, 8 Nov 2016 Akshat Agarwal (13068), Siddharth Tanwar (13699) CS698N: Recent Advances in Computer Vision, Jul Nov 2016 Instructor: Gaurav
More informationObject Detection with Partial Occlusion Based on a Deformable Parts-Based Model
Object Detection with Partial Occlusion Based on a Deformable Parts-Based Model Johnson Hsieh (johnsonhsieh@gmail.com), Alexander Chia (alexchia@stanford.edu) Abstract -- Object occlusion presents a major
More informationCPSC340. State-of-the-art Neural Networks. Nando de Freitas November, 2012 University of British Columbia
CPSC340 State-of-the-art Neural Networks Nando de Freitas November, 2012 University of British Columbia Outline of the lecture This lecture provides an overview of two state-of-the-art neural networks:
More informationModeling 3D viewpoint for part-based object recognition of rigid objects
Modeling 3D viewpoint for part-based object recognition of rigid objects Joshua Schwartz Department of Computer Science Cornell University jdvs@cs.cornell.edu Abstract Part-based object models based on
More informationThe Caltech-UCSD Birds Dataset
The Caltech-UCSD Birds-200-2011 Dataset Catherine Wah 1, Steve Branson 1, Peter Welinder 2, Pietro Perona 2, Serge Belongie 1 1 University of California, San Diego 2 California Institute of Technology
More informationFind that! Visual Object Detection Primer
Find that! Visual Object Detection Primer SkTech/MIT Innovation Workshop August 16, 2012 Dr. Tomasz Malisiewicz tomasz@csail.mit.edu Find that! Your Goals...imagine one such system that drives information
More informationSparse coding for image classification
Sparse coding for image classification Columbia University Electrical Engineering: Kun Rong(kr2496@columbia.edu) Yongzhou Xiang(yx2211@columbia.edu) Yin Cui(yc2776@columbia.edu) Outline Background Introduction
More informationBy Suren Manvelyan,
By Suren Manvelyan, http://www.surenmanvelyan.com/gallery/7116 By Suren Manvelyan, http://www.surenmanvelyan.com/gallery/7116 By Suren Manvelyan, http://www.surenmanvelyan.com/gallery/7116 By Suren Manvelyan,
More informationConstrained Convolutional Neural Networks for Weakly Supervised Segmentation. Deepak Pathak, Philipp Krähenbühl and Trevor Darrell
Constrained Convolutional Neural Networks for Weakly Supervised Segmentation Deepak Pathak, Philipp Krähenbühl and Trevor Darrell 1 Multi-class Image Segmentation Assign a class label to each pixel in
More informationHierarchical Image-Region Labeling via Structured Learning
Hierarchical Image-Region Labeling via Structured Learning Julian McAuley, Teo de Campos, Gabriela Csurka, Florent Perronin XRCE September 14, 2009 McAuley et al (XRCE) Hierarchical Image-Region Labeling
More informationSyllabus. 1. Visual classification Intro 2. SVM 3. Datasets and evaluation 4. Shallow / Deep architectures
Syllabus 1. Visual classification Intro 2. SVM 3. Datasets and evaluation 4. Shallow / Deep architectures Image classification How to define a category? Bicycle Paintings with women Portraits Concepts,
More informationCrowdsourcing Annotations for Visual Object Detection
Crowdsourcing Annotations for Visual Object Detection Hao Su, Jia Deng, Li Fei-Fei Computer Science Department, Stanford University Abstract A large number of images with ground truth object bounding boxes
More informationFast, Accurate Detection of 100,000 Object Classes on a Single Machine
Fast, Accurate Detection of 100,000 Object Classes on a Single Machine Thomas Dean etal. Google, Mountain View, CA CVPR 2013 best paper award Presented by: Zhenhua Wang 2013.12.10 Outline Background This
More informationImageCLEF 2011
SZTAKI @ ImageCLEF 2011 Bálint Daróczy joint work with András Benczúr, Róbert Pethes Data Mining and Web Search Group Computer and Automation Research Institute Hungarian Academy of Sciences Training/test
More informationObject Classification Problem
HIERARCHICAL OBJECT CATEGORIZATION" Gregory Griffin and Pietro Perona. Learning and Using Taxonomies For Fast Visual Categorization. CVPR 2008 Marcin Marszalek and Cordelia Schmid. Constructing Category
More informationPart-Based Models for Object Class Recognition Part 3
High Level Computer Vision! Part-Based Models for Object Class Recognition Part 3 Bernt Schiele - schiele@mpi-inf.mpg.de Mario Fritz - mfritz@mpi-inf.mpg.de! http://www.d2.mpi-inf.mpg.de/cv ! State-of-the-Art
More informationSegmentation as Selective Search for Object Recognition in ILSVRC2011
Segmentation as Selective Search for Object Recognition in ILSVRC2011 Koen van de Sande Jasper Uijlings Arnold Smeulders Theo Gevers Nicu Sebe Cees Snoek University of Amsterdam, University of Trento ILSVRC2011
More informationSupplementary Material: Pixelwise Instance Segmentation with a Dynamically Instantiated Network
Supplementary Material: Pixelwise Instance Segmentation with a Dynamically Instantiated Network Anurag Arnab and Philip H.S. Torr University of Oxford {anurag.arnab, philip.torr}@eng.ox.ac.uk 1. Introduction
More informationAn Object Detection Algorithm based on Deformable Part Models with Bing Features Chunwei Li1, a and Youjun Bu1, b
5th International Conference on Advanced Materials and Computer Science (ICAMCS 2016) An Object Detection Algorithm based on Deformable Part Models with Bing Features Chunwei Li1, a and Youjun Bu1, b 1
More informationLearning Collections of Part Models for Object Recognition
Learning Collections of Part Models for Object Recognition Ian Endres, Kevin J. Shih, Johnston Jiaa, Derek Hoiem University of Illinois at Urbana-Champaign {iendres2,kjshih2,jiaa1,dhoiem}@illinois.edu
More informationRich feature hierarchies for accurate object detection and semantic segmentation
Rich feature hierarchies for accurate object detection and semantic segmentation BY; ROSS GIRSHICK, JEFF DONAHUE, TREVOR DARRELL AND JITENDRA MALIK PRESENTER; MUHAMMAD OSAMA Object detection vs. classification
More informationPASCAL VOC Classification: Local Features vs. Deep Features. Shuicheng YAN, NUS
PASCAL VOC Classification: Local Features vs. Deep Features Shuicheng YAN, NUS PASCAL VOC Why valuable? Multi-label, Real Scenarios! Visual Object Recognition Object Classification Object Detection Object
More informationFine-tuning Pre-trained Large Scaled ImageNet model on smaller dataset for Detection task
Fine-tuning Pre-trained Large Scaled ImageNet model on smaller dataset for Detection task Kyunghee Kim Stanford University 353 Serra Mall Stanford, CA 94305 kyunghee.kim@stanford.edu Abstract We use a
More informationDiscriminative classifiers for image recognition
Discriminative classifiers for image recognition May 26 th, 2015 Yong Jae Lee UC Davis Outline Last time: window-based generic object detection basic pipeline face detection with boosting as case study
More informationAnalysis: TextonBoost and Semantic Texton Forests. Daniel Munoz Februrary 9, 2009
Analysis: TextonBoost and Semantic Texton Forests Daniel Munoz 16-721 Februrary 9, 2009 Papers [shotton-eccv-06] J. Shotton, J. Winn, C. Rother, A. Criminisi, TextonBoost: Joint Appearance, Shape and Context
More informationPart-based and local feature models for generic object recognition
Part-based and local feature models for generic object recognition May 28 th, 2015 Yong Jae Lee UC Davis Announcements PS2 grades up on SmartSite PS2 stats: Mean: 80.15 Standard Dev: 22.77 Vote on piazza
More informationc 2011 by Pedro Moises Crisostomo Romero. All rights reserved.
c 2011 by Pedro Moises Crisostomo Romero. All rights reserved. HAND DETECTION ON IMAGES BASED ON DEFORMABLE PART MODELS AND ADDITIONAL FEATURES BY PEDRO MOISES CRISOSTOMO ROMERO THESIS Submitted in partial
More informationA Discriminatively Trained, Multiscale, Deformable Part Model
A Discriminatively Trained, Multiscale, Deformable Part Model Pedro Felzenszwalb University of Chicago pff@cs.uchicago.edu David McAllester Toyota Technological Institute at Chicago mcallester@tti-c.org
More informationApprenticeship Learning: Transfer of Knowledge via Dataset Augmentation
Apprenticeship Learning: Transfer of Knowledge via Dataset Augmentation Miroslav Kobetski and Josephine Sullivan Computer Vision and Active Perception, KTH, 114 28 Stockholm kobetski@kth.se, sullivan@nada.kth.se
More informationObject Detection with Discriminatively Trained Part Based Models
Object Detection with Discriminatively Trained Part Based Models Pedro F. Felzenszwelb, Ross B. Girshick, David McAllester and Deva Ramanan Presented by Fabricio Santolin da Silva Kaustav Basu Some slides
More informationColorado School of Mines. Computer Vision. Professor William Hoff Dept of Electrical Engineering &Computer Science.
Professor William Hoff Dept of Electrical Engineering &Computer Science http://inside.mines.edu/~whoff/ 1 People Detection Some material for these slides comes from www.cs.cornell.edu/courses/cs4670/2012fa/lectures/lec32_object_recognition.ppt
More informationon learned visual embedding patrick pérez Allegro Workshop Inria Rhônes-Alpes 22 July 2015
on learned visual embedding patrick pérez Allegro Workshop Inria Rhônes-Alpes 22 July 2015 Vector visual representation Fixed-size image representation High-dim (100 100,000) Generic, unsupervised: BoW,
More informationExperiments of Image Retrieval Using Weak Attributes
Columbia University Computer Science Department Technical Report # CUCS 005-12 (2012) Experiments of Image Retrieval Using Weak Attributes Felix X. Yu, Rongrong Ji, Ming-Hen Tsai, Guangnan Ye, Shih-Fu
More informationLocality-constrained Linear Coding for Image Classification
Locality-constrained Linear Coding for Image Classification Jinjun Wang, Jianchao Yang,KaiYu, Fengjun Lv, Thomas Huang, and Yihong Gong Akiira Media System, Palo Alto, California Beckman Institute, University
More informationObject Recognition II
Object Recognition II Linda Shapiro EE/CSE 576 with CNN slides from Ross Girshick 1 Outline Object detection the task, evaluation, datasets Convolutional Neural Networks (CNNs) overview and history Region-based
More informationComputer Vision: Summary and Discussion
12/05/2011 Computer Vision: Summary and Discussion Computer Vision CS 143, Brown James Hays Many slides from Derek Hoiem Announcements Today is last day of regular class Second quiz on Wednesday (Dec 7
More informationYiqi Yan. May 10, 2017
Yiqi Yan May 10, 2017 P a r t I F u n d a m e n t a l B a c k g r o u n d s Convolution Single Filter Multiple Filters 3 Convolution: case study, 2 filters 4 Convolution: receptive field receptive field
More informationDeep learning for object detection. Slides from Svetlana Lazebnik and many others
Deep learning for object detection Slides from Svetlana Lazebnik and many others Recent developments in object detection 80% PASCAL VOC mean0average0precision0(map) 70% 60% 50% 40% 30% 20% 10% Before deep
More informationVisual Object Recognition
Perceptual and Sensory Augmented Computing Visual Object Recognition Tutorial Visual Object Recognition Bastian Leibe Computer Vision Laboratory ETH Zurich Chicago, 14.07.2008 & Kristen Grauman Department
More informationDeformable Part Models with Individual Part Scaling
DUBOUT, FLEURET: DEFORMABLE PART MODELS WITH INDIVIDUAL PART SCALING 1 Deformable Part Models with Individual Part Scaling Charles Dubout charles.dubout@idiap.ch François Fleuret francois.fleuret@idiap.ch
More informationMixture Component Identification and Learning for Visual Recognition
Mixture Component Identification and Learning for Visual Recognition Omid Aghazadeh, Hossein Azizpour, Josephine Sullivan, and Stefan Carlsson Computer Vision and Active Perception laboratory (CVAP), KTH,
More informationSelf-Paced Learning for Semisupervised Image Classification
Self-Paced Learning for Semisupervised Image Classification Kevin Miller Stanford University Palo Alto, CA kjmiller@stanford.edu Abstract In this project, we apply three variants of self-paced learning
More informationCompressed local descriptors for fast image and video search in large databases
Compressed local descriptors for fast image and video search in large databases Matthijs Douze2 joint work with Hervé Jégou1, Cordelia Schmid2 and Patrick Pérez3 1: INRIA Rennes, TEXMEX team, France 2:
More informationBeyond Bags of Features
: for Recognizing Natural Scene Categories Matching and Modeling Seminar Instructed by Prof. Haim J. Wolfson School of Computer Science Tel Aviv University December 9 th, 2015
More informationString distance for automatic image classification
String distance for automatic image classification Nguyen Hong Thinh*, Le Vu Ha*, Barat Cecile** and Ducottet Christophe** *University of Engineering and Technology, Vietnam National University of HaNoi,
More informationSpatial Localization and Detection. Lecture 8-1
Lecture 8: Spatial Localization and Detection Lecture 8-1 Administrative - Project Proposals were due on Saturday Homework 2 due Friday 2/5 Homework 1 grades out this week Midterm will be in-class on Wednesday
More informationIterative Category Discovery via Multiple Kernel Metric Learning
Int J Comput Vis (2014) 108:115 132 DOI 10.1007/s11263-013-0679-z Iterative Category Discovery via Multiple Kernel Metric Learning Carolina Galleguillos Brian McFee Gert R. G. Lanckriet Received: 26 February
More informationObject Category Detection. Slides mostly from Derek Hoiem
Object Category Detection Slides mostly from Derek Hoiem Today s class: Object Category Detection Overview of object category detection Statistical template matching with sliding window Part-based Models
More informationLecture 5: Object Detection
Object Detection CSED703R: Deep Learning for Visual Recognition (2017F) Lecture 5: Object Detection Bohyung Han Computer Vision Lab. bhhan@postech.ac.kr 2 Traditional Object Detection Algorithms Region-based
More informationThree things everyone should know to improve object retrieval. Relja Arandjelović and Andrew Zisserman (CVPR 2012)
Three things everyone should know to improve object retrieval Relja Arandjelović and Andrew Zisserman (CVPR 2012) University of Oxford 2 nd April 2012 Large scale object retrieval Find all instances of
More informationLearning Visual Semantics: Models, Massive Computation, and Innovative Applications
Learning Visual Semantics: Models, Massive Computation, and Innovative Applications Part II: Visual Features and Representations Liangliang Cao, IBM Watson Research Center Evolvement of Visual Features
More informationRecap Image Classification with Bags of Local Features
Recap Image Classification with Bags of Local Features Bag of Feature models were the state of the art for image classification for a decade BoF may still be the state of the art for instance retrieval
More informationKernels for Visual Words Histograms
Kernels for Visual Words Histograms Radu Tudor Ionescu and Marius Popescu Faculty of Mathematics and Computer Science University of Bucharest, No. 14 Academiei Street, Bucharest, Romania {raducu.ionescu,popescunmarius}@gmail.com
More informationLocal Features and Kernels for Classifcation of Texture and Object Categories: A Comprehensive Study
Local Features and Kernels for Classifcation of Texture and Object Categories: A Comprehensive Study J. Zhang 1 M. Marszałek 1 S. Lazebnik 2 C. Schmid 1 1 INRIA Rhône-Alpes, LEAR - GRAVIR Montbonnot, France
More informationUsing the Deformable Part Model with Autoencoded Feature Descriptors for Object Detection
Using the Deformable Part Model with Autoencoded Feature Descriptors for Object Detection Hyunghoon Cho and David Wu December 10, 2010 1 Introduction Given its performance in recent years' PASCAL Visual
More informationObject Recognition with Deformable Models
Object Recognition with Deformable Models Pedro F. Felzenszwalb Department of Computer Science University of Chicago Joint work with: Dan Huttenlocher, Joshua Schwartz, David McAllester, Deva Ramanan.
More informationLearning to Localize Objects with Structured Output Regression
Learning to Localize Objects with Structured Output Regression Matthew Blaschko and Christopher Lampert ECCV 2008 Best Student Paper Award Presentation by Jaeyong Sung and Yiting Xie 1 Object Localization
More informationRecognition Tools: Support Vector Machines
CS 2770: Computer Vision Recognition Tools: Support Vector Machines Prof. Adriana Kovashka University of Pittsburgh January 12, 2017 Announcement TA office hours: Tuesday 4pm-6pm Wednesday 10am-12pm Matlab
More informationEECS 442 Computer vision Databases for object recognition and beyond Segments of this lectures are courtesy of Prof F. Li, R. Fergus and A.
EECS 442 Computer vision Databases for object recognition and beyond Segments of this lectures are courtesy of Prof F. Li, R. Fergus and A. Zisserman Why are we talking about datasets? This is computer
More informationMetric Learning for Large Scale Image Classification:
Metric Learning for Large Scale Image Classification: Generalizing to New Classes at Near-Zero Cost Thomas Mensink 1,2 Jakob Verbeek 2 Florent Perronnin 1 Gabriela Csurka 1 1 TVPA - Xerox Research Centre
More informationApplying Supervised Learning
Applying Supervised Learning When to Consider Supervised Learning A supervised learning algorithm takes a known set of input data (the training set) and known responses to the data (output), and trains
More informationA Discriminatively Trained, Multiscale, Deformable Part Model
A Discriminatively Trained, Multiscale, Deformable Part Model by Pedro Felzenszwalb, David McAllester, and Deva Ramanan CS381V Visual Recognition - Paper Presentation Slide credit: Duan Tran Slide credit:
More informationDeformable Part Models
Deformable Part Models References: Felzenszwalb, Girshick, McAllester and Ramanan, Object Detec@on with Discrimina@vely Trained Part Based Models, PAMI 2010 Code available at hkp://www.cs.berkeley.edu/~rbg/latent/
More informationAttentionNet for Accurate Localization and Detection of Objects. (To appear in ICCV 2015)
AttentionNet for Accurate Localization and Detection of Objects. (To appear in ICCV 2015) Donggeun Yoo, Sunggyun Park, Joon-Young Lee, Anthony Paek, In So Kweon. State-of-the-art frameworks for object
More informationAn Incremental Structured Part Model for Object Recognition
An Incremental Structured Part Model for Object Recognition Xiao Bai a, Peng Ren b,, Huigang Zhang a, Jun Zhou c a School of Computer Science and Engineering, Beihang University, Beijing 100191, China.
More informationAction recognition in videos
Action recognition in videos Cordelia Schmid INRIA Grenoble Joint work with V. Ferrari, A. Gaidon, Z. Harchaoui, A. Klaeser, A. Prest, H. Wang Action recognition - goal Short actions, i.e. drinking, sit
More informationObject recognition (part 2)
Object recognition (part 2) CSE P 576 Larry Zitnick (larryz@microsoft.com) 1 2 3 Support Vector Machines Modified from the slides by Dr. Andrew W. Moore http://www.cs.cmu.edu/~awm/tutorials Linear Classifiers
More informationFeature Sets and Dimensionality Reduction for Visual Object Detection
HUSSAIN, TRIGGS: FEATURE SETS AND DIMENSIONALITY REDUCTION... Feature Sets and Dimensionality Reduction for Visual Object Detection Sibt ul Hussain Sibt.ul.Hussain@gmail.com Bill Triggs Bill.Triggs@imag.fr
More informationRich feature hierarchies for accurate object detection and semant
Rich feature hierarchies for accurate object detection and semantic segmentation Speaker: Yucong Shen 4/5/2018 Develop of Object Detection 1 DPM (Deformable parts models) 2 R-CNN 3 Fast R-CNN 4 Faster
More informationRecognition of Animal Skin Texture Attributes in the Wild. Amey Dharwadker (aap2174) Kai Zhang (kz2213)
Recognition of Animal Skin Texture Attributes in the Wild Amey Dharwadker (aap2174) Kai Zhang (kz2213) Motivation Patterns and textures are have an important role in object description and understanding
More informationA Comparison of l 1 Norm and l 2 Norm Multiple Kernel SVMs in Image and Video Classification
A Comparison of l 1 Norm and l 2 Norm Multiple Kernel SVMs in Image and Video Classification Fei Yan Krystian Mikolajczyk Josef Kittler Muhammad Tahir Centre for Vision, Speech and Signal Processing University
More informationStructured Prediction using Convolutional Neural Networks
Overview Structured Prediction using Convolutional Neural Networks Bohyung Han bhhan@postech.ac.kr Computer Vision Lab. Convolutional Neural Networks (CNNs) Structured predictions for low level computer
More informationLeveraging flickr images for object detection
Leveraging flickr images for object detection Elisavet Chatzilari Spiros Nikolopoulos Yiannis Kompatsiaris Outline Introduction to object detection Our proposal Experiments Current research 2 Introduction
More informationCS 1674: Intro to Computer Vision. Attributes. Prof. Adriana Kovashka University of Pittsburgh November 2, 2016
CS 1674: Intro to Computer Vision Attributes Prof. Adriana Kovashka University of Pittsburgh November 2, 2016 Plan for today What are attributes and why are they useful? (paper 1) Attributes for zero-shot
More informationGeneral Principal. Huge Dataset. Images. Info from Most Similar Images. Input Image. Associated Info
Devin Montes General Principal Huge Dataset Input Image Images Associated Info image matching Info from Most Similar Images Hopefully, If you have enough images, the dataset will contain very similar images
More informationSelective Search for Object Recognition
Selective Search for Object Recognition Uijlings et al. Schuyler Smith Overview Introduction Object Recognition Selective Search Similarity Metrics Results Object Recognition Kitten Goal: Problem: Where
More informationModern Object Detection. Most slides from Ali Farhadi
Modern Object Detection Most slides from Ali Farhadi Comparison of Classifiers assuming x in {0 1} Learning Objective Training Inference Naïve Bayes maximize j i logp + logp ( x y ; θ ) ( y ; θ ) i ij
More information