Introduction to Object Recognition & Bag of Words (BoW) Models
|
|
- Lester West
- 5 years ago
- Views:
Transcription
1 : Introduction to Object Recognition & Bag of Words (BoW) Models Professor Fei Fei Li Stanford Vision Lab 1
2 What we will learn today? Introduction to object recognition Representation Learning Recognition Bag of Words models (Problem Set 4 (Q2)) Basic representation Different learning and recognition algorithms 2
3 What are the different visual recognition tasks? 3
4 Classification: Does this image contain a building? [yes/no] Yes! 4
5 Classification: Is this an beach? 5
6 Image Search Organizing photo collections 6
7 Detection: Does this image contain a car? [where?] car 7
8 Detection: Which object does this image contain? [where?] Building clock person car 8
9 Detection: Accurate localization (segmentation) clock 9
10 Detection: Estimating object semantic & geometric attributes Object: Building, 45º pose, 8 10 meters away It has bricks Object: Person, back; 1 2 meters away Object: Police car, side view, 4 5 m away 10
11 Applications of computer vision Computational photography Assistive technologies Surveillance Security Assistive driving 11
12 Categorization vs Single instance recognition Does this image contain the Chicago Macy building s? 12
13 Categorization vs Single instance recognition Where is the crunchy nut? 13
14 Applications of computer vision Recognizing landmarks in mobile platforms + GPS 14
15 Activity or Event recognition What are these people doing? 15
16 Visual Recognition Design algorithms that are capable to Classify images or videos Detect and localize objects Estimate semantic and geometrical attributes Classify human activities and events Why is this challenging? 16
17 How many object categories are there? 17
18 Challenges: viewpoint variation Michelangelo
19 Challenges: illumination image credit: J. Koenderink 19
20 Challenges: scale 20
21 Challenges: deformation 21
22 Challenges: occlusion Magritte,
23 Challenges: background clutter Kilmeny Niland
24 Challenges: intra class variation 24
25 Some early works on object categorization Turk and Pentland, 1991 Belhumeur, Hespanha, & Kriegman, 1997 Schneiderman & Kanade 2004 Viola and Jones, 2000 Amit and Geman, 1999 LeCun et al Belongie and Malik, 2002 Schneiderman & Kanade, 2004 Argawal and Roth, 2002 Poggio et al. 1993
26 Basic issues Representation How to represent an object category; which classification scheme? Learning How to learn the classifier, given training data Recognition How the classifier is to be used on novel data 26
27 Representation Building blocks: Sampling strategies Interest operators Multiple interest operators Dense, uniformly Randomly Image credits: L. Fei Fei, E. Nowak, J. Sivic 27
28 Representation Appearance only or location and appearance 28
29 Representation Invariances View point Illumination Occlusion Scale Deformation Clutter etc. 29
30 Representation To handle intra class variability, it is convenient to describe an object categories using probabilistic models Object models: Generative vs Discriminative vs hybrid 30
31 Object categorization: the statistical viewpoint p( zebra image) vs. p( no zebra image) Bayesrule: p( zebra image) p( no zebra image) 31
32 Object categorization: the statistical viewpoint p( zebra image) vs. p( no zebra image) Bayes rule: p( zebra image) p( no zebra image) p( image zebra) p( image no zebra) p( zebra) p( no zebra) posterior ratio likelihood ratio prior ratio
33 Object categorization: the statistical viewpoint Discriminative methods model posterior Generative methods model likelihood and prior Bayes rule: p( zebra image) p( no zebra image) p( image zebra) p( image no zebra) p( zebra) p( no zebra) posterior ratio likelihood ratio prior ratio
34 Discriminative models Modeling the posterior ratio: p( zebra image) p( no zebra image) Decision boundary Zebra Non zebra 34
35 Discriminative models Nearest neighbor Neural networks 10 6 examples Shakhnarovich, Viola, Darrell 2003 Berg, Berg, Malik LeCun, Bottou, Bengio, Haffner 1998 Rowley, Baluja, Kanade 1998 Support Vector Machines Latent SVM Structural SVM Boosting Guyon, Vapnik, Heisele, Serre, Poggio Felzenszwalb 00 Ramanan 03 Viola, Jones 2001, Torralba et al. 2004, Opelt et al. 2006, Source: Vittorio Ferrari, Kristen Grauman, Antonio Torralba 35
36 Generative models Modeling the likelihood ratio: p( image zebra) p( image no zebra) 5 4 p(x C 2 ) class densities p(x C 1 ) x 36
37 Generative models p( image zebra) High Low p( image no zebra) Low High 5 4 p(x C 2 ) class densities 3 2 p(x C 1 )
38 Generative models Naïve Bayes classifier Csurka Bray, Dance & Fan, 2004 Hierarchical Bayesian topic models (e.g. plsa and LDA) Object categorization: Sivic et al. 2005, Sudderth et al Natural scene categorization: Fei Fei et al D Part based models Constellation models: Weber et al 2000; Fergus et al 200 Star models: ISM (Leibe et al 05) 3D part based models: multi aspects: Sun, et al,
39 Basic issues Representation How to represent an object category; which classification scheme? Learning How to learn the classifier, given training data Recognition How the classifier is to be used on novel data 39
40 Learning Learning parameters: What are you maximizing? Likelihood (Gen.) or performances on train/validation set (Disc.) 40
41 Learning Learning parameters: What are you maximizing? Likelihood (Gen.) or performances on train/validation set (Disc.) Level of supervision Manual segmentation; bounding box; image labels; noisy labels Batch/incremental Priors 41
42 Learning Learning parameters: What are you maximizing? Likelihood (Gen.) or performances on train/validation set (Disc.) Level of supervision Manual segmentation; bounding box; image labels; noisy labels Batch/incremental Priors Training images: Issue of overfitting Negative images for discriminative methods 42
43 Basic issues Representation How to represent an object category; which classification scheme? Learning How to learn the classifier, given training data Recognition How the classifier is to be used on novel data 43
44 Recognition Recognition task: classification, detection, etc.. 44
45 Recognition Recognition task Search strategy: Sliding Windows Simple Computational complexity (x,y, S,, N of classes) BSW by Lampert et al 08 Also, Alexe, et al 10 Viola, Jones 2001, 45
46 Recognition Recognition task Search strategy: Sliding Windows Simple Computational complexity (x,y, S,, N of classes) BSW by Lampert et al 08 Also, Alexe, et al 10 Localization Objectsare not boxes Viola, Jones 2001, 46
47 Recognition Recognition task Search strategy: Sliding Windows Simple Computational complexity (x,y, S,, N of classes) BSW by Lampert et al 08 Also, Alexe, et al 10 Localization Objectsare not boxes Proneto false positive Non max suppression: Canny 86. Desai et al, 2009 Viola, Jones 2001, 47
48 Recognition Recognition task Search strategy Attributes Savarese, 2007 Sun et al 2009 Liebelt et al., 08, 10 Farhadi et al 09 Category: car Azimuth = 225º Zenith = 30º It has metal it is glossy has wheels Farhadi et al 09 Lampert et al 09 Wang & Forsyth 09 48
49 Recognition Recognition task Search strategy Attributes Context Semantic: Torralba et al 03 Rabinovich et al 07 Gupta & Davis 08 Heitz & Koller 08 L J Li et al 08 Yao & Fei Fei 10 Geometric Hoiem, et al 06 Gould et al 09 Bao, Sun, Savarese 10 49
50 Basic issues Representation How to represent an object category; which classification scheme? Learning How to learn the classifier, given training data Recognition How the classifier is to be used on novel data 50
51 Part 1: Bag of words models This segment is based on the tutorial Recognizing and Learning Object Categories: Year 2007, by Prof L. Fei Fei, A. Torralba, and R. Fergus 51
52 Related works Early bag of words models: mostly texture recognition Cula & Dana, 2001; Leung & Malik 2001; Mori, Belongie & Malik, 2001; Schmid 2001; Varma & Zisserman, 2002, 2003; Lazebnik, Schmid & Ponce, 2003; Hierarchical Bayesian models for documents (plsa, LDA, etc.) Hoffman 1999; Blei, Ng & Jordan, 2004; Teh, Jordan, Beal & Blei, 2004 Object categorization Csurka, Bray, Dance & Fan, 2004; Sivic, Russell, Efros, Freeman & Zisserman, 2005; Sudderth, Torralba, Freeman & Willsky, 2005; Natural scene categorization Vogel & Schiele, 2004; Fei Fei & Perona, 2005; Bosch, Zisserman & Munoz,
53 Object Bag of words 53
54 Analogy to documents Of all the sensory impressions proceeding to the brain, the visual experiences are the dominant ones. Our perception of the world around us is based essentially on the messages that reach the brain from our eyes. For a long time it was thought that the retinal image was transmitted sensory, point brain, by point to visual centers in the brain; the cerebral cortex was a visual, perception, movie screen, so to speak, upon which the image in retinal, the eye was cerebral projected. Through cortex, the discoveries of eye, Hubel cell, and Wiesel optical we now know that behind the origin of the visual perception in the nerve, brain there image is a considerably more complicated Hubel, course of Wiesel events. By following the visual impulses along their path to the various cell layers of the optical cortex, Hubel and Wiesel have been able to demonstrate that the message about the image falling on the retina undergoes a stepwise analysis in a system of nerve cells stored in columns. In this system each cell has its specific function and is responsible for a specific detail in the pattern of the retinal image. China is forecasting a trade surplus of $90bn ( 51bn) to $100bn this year, a threefold increase on 2004's $32bn. The Commerce Ministry said the surplus would be created by a predicted 30% jump in exports to $750bn, compared with a 18% rise in imports to $660bn. The figures China, are likely trade, to further annoy the surplus, US, which has commerce, long argued that China's exports are unfairly helped by a deliberately exports, undervalued imports, yuan. Beijing US, agrees the yuan, surplus bank, is too high, domestic, but says the yuan is only one factor. Bank of China governor Zhou foreign, Xiaochuan increase, said the country also needed to do trade, more to value boost domestic demand so more goods stayed within the country. China increased the value of the yuan against the dollar by 2.1% in July and permitted it to trade within a narrow band, but the US wants the yuan to be allowed to trade freely. However, Beijing has made it clear that it will take its time and tread carefully before allowing the yuan to rise further in value. 54
55 definition of BoW Independent features face bike violin 55
56 definition of BoW Independent features histogram representation codewords dictionary 56
57 Representation recognition feature detection & representation codewords dictionary image representation learning category models (and/or) classifiers 57 category decision
58 1.Feature detection and representation 58
59 1.Feature detection and representation Regular grid Vogel & Schiele, 2003 Fei Fei & Perona,
60 1.Feature detection and representation Regular grid Vogel & Schiele, 2003 Fei Fei & Perona, 2005 Interest point detector Csurka, et al Fei Fei & Perona, 2005 Sivic, et al
61 1.Feature detection and representation Regular grid Vogel & Schiele, 2003 Fei Fei & Perona, 2005 Interest point detector Csurka, Bray, Dance & Fan, 2004 Fei Fei & Perona, 2005 Sivic, Russell, Efros, Freeman & Zisserman, 2005 Other methods Random sampling (Vidal Naquet & Ullman, 2002) Segmentation based patches (Barnard, Duygulu, Forsyth, de Freitas, Blei, Jordan, 2003) 61
62 1.Feature detection and representation Compute SIFT descriptor [Lowe 99] Normalize patch Detect patches [Mikojaczyk and Schmid 02] [Mata, Chum, Urban & Pajdla, 02] [Sivic & Zisserman, 03] Slide credit: Josef Sivic 62
63 1.Feature detection and representation 63
64 2. Codewords dictionary formation 64
65 2. Codewords dictionary formation Cluster center = code word Clustering/ vector quantization 65
66 2. Codewords dictionary formation Fei-Fei et al
67 Image patch examples of codewords Sivic et al
68 Visual vocabularies: Issues How to choose vocabulary size? Too small: visual words not representative of all patches Too large: quantization artifacts, overfitting Computational efficiency Vocabulary trees (Nister & Stewenius, 2006) 68
69 3. Bag of word representation Nearest neighbors assignment K D tree search strategy Codewords dictionary 69
70 3. Bag of word representation frequency. codewords Codewords dictionary 70
71 Representation 1. feature detection & representation 2. codewords dictionary image representation 3. 71
72 Learning and Recognition codewords dictionary category models (and/or) classifiers 72 category decision
73 Learning and Recognition 1. Discriminative method: -NN - SVM 2.Generative method: - graphical models category models (and/or) classifiers 73
74 Discriminative classifiers category models Model space Class 1 Class N 74
75 Discriminative classifiers Query image Model space Winning class: pink 75
76 Nearest Neighbors classifier Query image Model space Winning class: pink Assign label of nearest training data point to each test data point 76
77 K- Nearest Neighbors classifier Query image Model space Winning class: pink For a new point, find the k closest points from training data Labels of the k points vote to classify Works well provided there is lots of data and the distance function is good 77
78 K- Nearest Neighbors classifier from Duda et al. Voronoi partitioning of feature space for 2 category 2 D and 3 D data For k dimensions: k D tree = space partitioning data structure for organizing points in a k dimensional space Enable efficient search Nice tutorial: /pbasic.pdf 78
79 Functions for comparing histograms L1 distance χ 2 distance Quadratic distance (cross bin) N i i h i h h h D ) ( ) ( ), ( Jan Puzicha, Yossi Rubner, Carlo Tomasi, Joachim M. Buhmann: Empirical Evaluation of Dissimilarity Measures for Color and Texture. ICCV 1999 N i i h i h i h i h h h D ) ( ) ( ) ( ) ( ), ( j i ij j h i h A h h D, )) ( ) ( ( ), ( 79
80 Learning and Recognition 1. Discriminative method: -NN - SVM 2.Generative method: - graphical models 80
81 Discriminative classifiers (linear classifier) category models Model space Class 1 Class N 81
82 Support vector machines Find hyperplane that maximizes the margin between the positive and negative examples Support vectors: x i w b 1 x w b Distance between point i and hyperplane: w Margin = 2 / w Solution: w y x i i i i Support vectors Credit slide: S. Lazebnik Margin w Classification function (decision boundary): x b y x x i i i i b 82
83 Support vector machines Classification w x b y x x i i i i b Test point if x w b 0 class 1 if x w b 0 class 2 Margin C. Burges, A Tutorial on Support Vector Machines for Pattern Recognition, Data Mining and Knowledge Discovery,
84 Nonlinear SVMs Datasets that are linearly separable work out great: 0 x But what if the dataset is just too hard? 0 x We can map it to a higher dimensional space: x 2 0 x Slide credit: Andrew Moore 84
85 Nonlinear SVMs General idea: the original input space can always be mapped to some higher dimensional feature space where the training set is separable: Φ: x φ(x) lifting transformation Slide credit: Andrew Moore 85
86 Nonlinear SVMs Nonlinear decision boundary in the original feature space: i y i K ( x i, x ) i b The kernel K = product of the lifting transformation φ(x): K(x i,x j j) = φ(x i ) φ(x j ) NOTE: It is not required to compute φ(x) explicitly: The kernel must satisfy the Mercer inequality C. Burges, A Tutorial on Support Vector Machines for Pattern Recognition, Data Mining and Knowledge Discovery,
87 Kernels for bags of features Histogram intersection kernel: N I ( h1, h 2 ) min( h1 ( i), h 2 ( i)) i 1 Generalized Gaussian kernel: 1 K ( h, h 2 ) exp D ( h1, h A ) D can be Euclidean distance, χ 2 distance etc J. Zhang, M. Marszalek, S. Lazebnik, and C. Schmid, Local Features and Kernels for Classifcation of Texture and Object Categories: A Comprehensive Study, IJCV
88 Pyramid match kernel Fast approximation of Earth Mover s Distance Weighted sum of histogram intersections at mutliple resolutions (linear in the number of features instead of cubic) K. Grauman and T. Darrell. The Pyramid Match Kernel: Discriminative Classification with Sets of Image Features, ICCV
89 Spatial Pyramid Matching Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories. S. Lazebnik, C. Schmid, and J. Ponce. CVPR
90 What about multi class SVMs? No definitive multi class SVM formulation In practice, we have to obtain a multi class SVM by combining multiple two class SVMs One vs. others Traning: learn an SVM for each class vs. the others Testing: apply each SVM to test example and assign to it the class of the SVM that returns the highest decision value One vs. one Training: learn an SVM for each pair of classes Testing: each learned SVM votes for a class to assign to the test example Credit slide: S. Lazebnik 90
91 SVMs: Pros and cons Pros Many publicly available SVM packages: machines.org/software Kernel based framework is very powerful, flexible SVMs work very well in practice, even with very small training sample sizes Cons No direct multi class SVM, must combine two class SVMs Computation, memory During training time, must compute matrix of kernel values for every pair of examples Learning can take a very long time for large scale problems 91
92 Object recognition results ETH 80 database of 8 object classes (Eichhorn and Chapelle 2004) Features: Harris detector PCA SIFT descriptor, d=10 Kernel Complexity Recognition rate Match [Wallraven et al.] 84% Bhattacharyya affinity [Kondor & Jebara] 85% Pyramid match 84% Slide credit: Kristen Grauman
93 Discriminative models Nearest neighbor Neural networks 10 6 examples Shakhnarovich, Viola, Darrell 2003 Berg, Berg, Malik LeCun, Bottou, Bengio, Haffner 1998 Rowley, Baluja, Kanade 1998 Support Vector Machines Latent SVM Structural SVM Boosting Guyon, Vapnik, Heisele, Serre, Poggio Felzenszwalb 00 Ramanan 03 Viola, Jones 2001, Torralba et al. 2004, Opelt et al. 2006, Source: Vittorio Ferrari, Kristen Grauman, Antonio Torralba 93
94 Learning and Recognition 1. Discriminative method: NN SVM 2.Generative method: graphical models Model the probability distribution that produces a given bag of features 94
95 Generative models 1. Naïve Bayes classifier Csurka Bray, Dance & Fan, Hierarchical Bayesian text models (plsa and LDA) Background: Hoffman 2001, Blei, Ng & Jordan, 2004 Object categorization: Sivic et al. 2005, Sudderth et al Natural scene categorization: Fei Fei et al
96 Some notations w: a collection of all N codewords in the image w = [w1,w2,,wn] c: category of the image 96
97 the Naïve Bayes model c w N Graphical model Posterior = probability that image I is of category c p( c w) p( c) p( w c) Prior prob. of the object classes Image likelihood given the class
98 the Naïve Bayes model c w N Graphical model c arg max c p( c w) p( c) p( w c) p( c) N n 1 p( w n c) Object class decision Likelihood of ith visual word given the class Estimated by empirical frequencies of code words in images from a given class
99 Csurka et al
100 Csurka et al
101 Other generative BoW models Hierarchical Bayesian topic models (e.g. plsa and LDA) Object categorization: Sivic et al. 2005, Sudderth et al Natural scene categorization: Fei Fei et al
102 Generative vs discriminative Discriminative methods Computationally efficient & fast Generative models Convenient for weakly or un supervised, incremental training Prior information Flexibility in modeling parameters 102
103 Weakness of BoW the models No rigorous geometric information of the object components It s intuitive to most of us that objects are made of parts no such information Not extensively tested yet for View point invariance Scale invariance Segmentation and localization unclear 103
104 What have learned today? Introduction to object recognition Representation Learning Recognition Bag of Words models (Problem Set 4 (Q2)) Basic representation Different learning and recognition algorithms 104
EECS 442 Computer vision. Object Recognition
EECS 442 Computer vision Object Recognition Intro Recognition of 3D objects Recognition of object categories: Bag of world models Part based models 3D object categorization Computer Vision: Algorithms
More informationCEE598 - Visual Sensing for Civil Infrastructure Eng. & Mgmt.
CEE598 - Visual Sensing for Civil Infrastructure Eng. & Mgmt. Session 19 Object Recognition II Mani Golparvar-Fard Department of Civil and Environmental Engineering 3129D, Newmark Civil Engineering Lab
More informationLecture 12 Visual recognition
Lecture 12 Visual recognition Bag of words models for object recognition and classification Discriminative methods Generative methods Silvio Savarese Lecture 11 17Feb14 Challenges Variability due to: View
More informationRecognition with Bag-ofWords. (Borrowing heavily from Tutorial Slides by Li Fei-fei)
Recognition with Bag-ofWords (Borrowing heavily from Tutorial Slides by Li Fei-fei) Recognition So far, we ve worked on recognizing edges Now, we ll work on recognizing objects We will use a bag-of-words
More informationLecture 15: Object recognition: Bag of Words models & Part based generative models
Lecture 15: Object recognition: Bag of Words models & Part based generative models Professor Fei Fei Li Stanford Vision Lab 1 Basic issues Representation How to represent an object category; which classification
More informationBy Suren Manvelyan,
By Suren Manvelyan, http://www.surenmanvelyan.com/gallery/7116 By Suren Manvelyan, http://www.surenmanvelyan.com/gallery/7116 By Suren Manvelyan, http://www.surenmanvelyan.com/gallery/7116 By Suren Manvelyan,
More informationPattern recognition (3)
Pattern recognition (3) 1 Things we have discussed until now Statistical pattern recognition Building simple classifiers Supervised classification Minimum distance classifier Bayesian classifier Building
More informationColorado School of Mines. Computer Vision. Professor William Hoff Dept of Electrical Engineering &Computer Science.
Professor William Hoff Dept of Electrical Engineering &Computer Science http://inside.mines.edu/~whoff/ 1 Object Recognition in Large Databases Some material for these slides comes from www.cs.utexas.edu/~grauman/courses/spring2011/slides/lecture18_index.pptx
More informationObject Classification for Video Surveillance
Object Classification for Video Surveillance Rogerio Feris IBM TJ Watson Research Center rsferis@us.ibm.com http://rogerioferis.com 1 Outline Part I: Object Classification in Far-field Video Part II: Large
More informationDistances and Kernels. Motivation
Distances and Kernels Amirshahed Mehrtash Motivation How similar? 1 Problem Definition Designing a fast system to measure the similarity il it of two images. Used to categorize images based on appearance.
More informationIndexing local features and instance recognition May 16 th, 2017
Indexing local features and instance recognition May 16 th, 2017 Yong Jae Lee UC Davis Announcements PS2 due next Monday 11:59 am 2 Recap: Features and filters Transforming and describing images; textures,
More informationAdvanced Techniques for Mobile Robotics Bag-of-Words Models & Appearance-Based Mapping
Advanced Techniques for Mobile Robotics Bag-of-Words Models & Appearance-Based Mapping Wolfram Burgard, Cyrill Stachniss, Kai Arras, Maren Bennewitz Motivation: Analogy to Documents O f a l l t h e s e
More informationDiscriminative classifiers for image recognition
Discriminative classifiers for image recognition May 26 th, 2015 Yong Jae Lee UC Davis Outline Last time: window-based generic object detection basic pipeline face detection with boosting as case study
More informationIndexing local features and instance recognition May 14 th, 2015
Indexing local features and instance recognition May 14 th, 2015 Yong Jae Lee UC Davis Announcements PS2 due Saturday 11:59 am 2 We can approximate the Laplacian with a difference of Gaussians; more efficient
More informationCS6670: Computer Vision
CS6670: Computer Vision Noah Snavely Lecture 16: Bag-of-words models Object Bag of words Announcements Project 3: Eigenfaces due Wednesday, November 11 at 11:59pm solo project Final project presentations:
More informationIndexing local features and instance recognition May 15 th, 2018
Indexing local features and instance recognition May 15 th, 2018 Yong Jae Lee UC Davis Announcements PS2 due next Monday 11:59 am 2 Recap: Features and filters Transforming and describing images; textures,
More informationCS 4495 Computer Vision Classification 3: Bag of Words. Aaron Bobick School of Interactive Computing
CS 4495 Computer Vision Classification 3: Bag of Words Aaron Bobick School of Interactive Computing Administrivia PS 6 is out. Due Tues Nov 25th, 11:55pm. One more assignment after that Mea culpa This
More informationLecture 14: Indexing with local features. Thursday, Nov 1 Prof. Kristen Grauman. Outline
Lecture 14: Indexing with local features Thursday, Nov 1 Prof. Kristen Grauman Outline Last time: local invariant features, scale invariant detection Applications, including stereo Indexing with invariant
More informationVisual Object Recognition
Perceptual and Sensory Augmented Computing Visual Object Recognition Tutorial Visual Object Recognition Bastian Leibe Computer Vision Laboratory ETH Zurich Chicago, 14.07.2008 & Kristen Grauman Department
More informationSupervised learning. y = f(x) function
Supervised learning y = f(x) output prediction function Image feature Training: given a training set of labeled examples {(x 1,y 1 ),, (x N,y N )}, estimate the prediction function f by minimizing the
More informationToday. Main questions 10/30/2008. Bag of words models. Last time: Local invariant features. Harris corner detector: rotation invariant detection
Today Indexing with local features, Bag of words models Matching local features Indexing features Bag of words model Thursday, Oct 30 Kristen Grauman UT-Austin Main questions Where will the interest points
More informationCS4495/6495 Introduction to Computer Vision. 8C-L1 Classification: Discriminative models
CS4495/6495 Introduction to Computer Vision 8C-L1 Classification: Discriminative models Remember: Supervised classification Given a collection of labeled examples, come up with a function that will predict
More informationBag of Words Models. CS4670 / 5670: Computer Vision Noah Snavely. Bag-of-words models 11/26/2013
CS4670 / 5670: Computer Vision Noah Snavely Bag-of-words models Object Bag of words Bag of Words Models Adapted from slides by Rob Fergus and Svetlana Lazebnik 1 Object Bag of words Origin 1: Texture Recognition
More informationRecognizing Object Instances. Prof. Xin Yang HUST
Recognizing Object Instances Prof. Xin Yang HUST Applications Image Search 5 Years Old Techniques Applications For Toys Applications Traffic Sign Recognition Today: instance recognition Visual words quantization,
More informationEECS 442 Computer vision. Object Recognition
EECS 442 Computer vision Object Recognition Intro Recognition of 3D objects Recognition of object categories: Bag of world models Part based models 3D object categorization Challenges: intraclass variation
More informationRecognizing object instances
Recognizing object instances UT-Austin Instance recognition Motivation visual search Visual words quantization, index, bags of words Spatial verification affine; RANSAC, Hough Other text retrieval tools
More informationPart-based and local feature models for generic object recognition
Part-based and local feature models for generic object recognition May 28 th, 2015 Yong Jae Lee UC Davis Announcements PS2 grades up on SmartSite PS2 stats: Mean: 80.15 Standard Dev: 22.77 Vote on piazza
More informationWindow based detectors
Window based detectors CS 554 Computer Vision Pinar Duygulu Bilkent University (Source: James Hays, Brown) Today Window-based generic object detection basic pipeline boosting classifiers face detection
More informationPatch Descriptors. CSE 455 Linda Shapiro
Patch Descriptors CSE 455 Linda Shapiro How can we find corresponding points? How can we find correspondences? How do we describe an image patch? How do we describe an image patch? Patches with similar
More informationPart based models for recognition. Kristen Grauman
Part based models for recognition Kristen Grauman UT Austin Limitations of window-based models Not all objects are box-shaped Assuming specific 2d view of object Local components themselves do not necessarily
More informationIntroduction to object recognition. Slides adapted from Fei-Fei Li, Rob Fergus, Antonio Torralba, and others
Introduction to object recognition Slides adapted from Fei-Fei Li, Rob Fergus, Antonio Torralba, and others Overview Basic recognition tasks A statistical learning approach Traditional or shallow recognition
More informationBag-of-features. Cordelia Schmid
Bag-of-features for category classification Cordelia Schmid Visual search Particular objects and scenes, large databases Category recognition Image classification: assigning a class label to the image
More informationBeyond bags of features: Adding spatial information. Many slides adapted from Fei-Fei Li, Rob Fergus, and Antonio Torralba
Beyond bags of features: Adding spatial information Many slides adapted from Fei-Fei Li, Rob Fergus, and Antonio Torralba Adding spatial information Forming vocabularies from pairs of nearby features doublets
More informationPreviously. Part-based and local feature models for generic object recognition. Bag-of-words model 4/20/2011
Previously Part-based and local feature models for generic object recognition Wed, April 20 UT-Austin Discriminative classifiers Boosting Nearest neighbors Support vector machines Useful for object recognition
More informationMachine Learning Crash Course
Machine Learning Crash Course Photo: CMU Machine Learning Department protests G20 Computer Vision James Hays Slides: Isabelle Guyon, Erik Sudderth, Mark Johnson, Derek Hoiem The machine learning framework
More informationObject detection as supervised classification
Object detection as supervised classification Tues Nov 10 Kristen Grauman UT Austin Today Supervised classification Window-based generic object detection basic pipeline boosting classifiers face detection
More informationSupervised learning. y = f(x) function
Supervised learning y = f(x) output prediction function Image feature Training: given a training set of labeled examples {(x 1,y 1 ),, (x N,y N )}, estimate the prediction function f by minimizing the
More informationObject Recognition. Computer Vision. Slides from Lana Lazebnik, Fei-Fei Li, Rob Fergus, Antonio Torralba, and Jean Ponce
Object Recognition Computer Vision Slides from Lana Lazebnik, Fei-Fei Li, Rob Fergus, Antonio Torralba, and Jean Ponce How many visual object categories are there? Biederman 1987 ANIMALS PLANTS OBJECTS
More informationPatch Descriptors. EE/CSE 576 Linda Shapiro
Patch Descriptors EE/CSE 576 Linda Shapiro 1 How can we find corresponding points? How can we find correspondences? How do we describe an image patch? How do we describe an image patch? Patches with similar
More informationBeyond Bags of Features
: for Recognizing Natural Scene Categories Matching and Modeling Seminar Instructed by Prof. Haim J. Wolfson School of Computer Science Tel Aviv University December 9 th, 2015
More informationFeature Matching + Indexing and Retrieval
CS 1699: Intro to Computer Vision Feature Matching + Indexing and Retrieval Prof. Adriana Kovashka University of Pittsburgh October 1, 2015 Today Review (fitting) Hough transform RANSAC Matching points
More informationCV as making bank. Intel buys Mobileye! $15 billion. Mobileye:
CV as making bank Intel buys Mobileye! $15 billion Mobileye: Spin-off from Hebrew University, Israel 450 engineers 15 million cars installed 313 car models June 2016 - Tesla left Mobileye Fatal crash car
More informationInstance recognition
Instance recognition Thurs Oct 29 Last time Depth from stereo: main idea is to triangulate from corresponding image points. Epipolar geometry defined by two cameras We ve assumed known extrinsic parameters
More informationImage classification Computer Vision Spring 2018, Lecture 18
Image classification http://www.cs.cmu.edu/~16385/ 16-385 Computer Vision Spring 2018, Lecture 18 Course announcements Homework 5 has been posted and is due on April 6 th. - Dropbox link because course
More informationLocal Features and Bag of Words Models
10/14/11 Local Features and Bag of Words Models Computer Vision CS 143, Brown James Hays Slides from Svetlana Lazebnik, Derek Hoiem, Antonio Torralba, David Lowe, Fei Fei Li and others Computer Engineering
More informationRecognizing object instances. Some pset 3 results! 4/5/2011. Monday, April 4 Prof. Kristen Grauman UT-Austin. Christopher Tosh.
Recognizing object instances Some pset 3 results! Monday, April 4 Prof. UT-Austin Brian Bates Christopher Tosh Brian Nguyen Che-Chun Su 1 Ryu Yu James Edwards Kevin Harkness Lucy Liang Lu Xia Nona Sirakova
More informationSupervised learning. f(x) = y. Image feature
Coffer Illusion Coffer Illusion Supervised learning f(x) = y Prediction function Image feature Output (label) Training: Given a training set of labeled examples: {(x 1,y 1 ),, (x N,y N )} Estimate the
More informationVisual Navigation for Flying Robots. Structure From Motion
Computer Vision Group Prof. Daniel Cremers Visual Navigation for Flying Robots Structure From Motion Dr. Jürgen Sturm VISNAV Oral Team Exam Date and Time Student Name Student Name Student Name Mon, July
More informationCEE598 - Visual Sensing for Civil Infrastructure Eng. & Mgmt.
CEE598 - Visual Sensing for Civil Infrastructure Eng. & Mgmt. Session 18 Object Recognition I Mani Golparvar-Fard Department of Civil and Environmental Engineering 3129D, Newmark Civil Engineering Lab
More informationObject recognition (part 1)
Recognition Object recognition (part 1) CSE P 576 Larry Zitnick (larryz@microsoft.com) The Margaret Thatcher Illusion, by Peter Thompson Readings Szeliski Chapter 14 Recognition What do we mean by object
More informationLecture 16: Object recognition: Part-based generative models
Lecture 16: Object recognition: Part-based generative models Professor Stanford Vision Lab 1 What we will learn today? Introduction Constellation model Weakly supervised training One-shot learning (Problem
More informationImage Features and Categorization. Computer Vision Jia-Bin Huang, Virginia Tech
Image Features and Categorization Computer Vision Jia-Bin Huang, Virginia Tech Administrative stuffs Final project Proposal due 11:59 PM on Thursday, Oct 27 Submit via CANVAS Send a copy to Jia-Bin and
More informationOverview of section. Lecture 10 Discriminative models. Discriminative methods. Discriminative vs. generative. Discriminative methods.
Overview of section Lecture 0 Discriminative models Object detection with classifiers Boosting Gentle boosting Object model Object detection Nearest-Neighbor methods Multiclass object detection Context
More informationPreviously. Window-based models for generic object detection 4/11/2011
Previously for generic object detection Monday, April 11 UT-Austin Instance recognition Local features: detection and description Local feature matching, scalable indexing Spatial verification Intro to
More informationCourse Administration
Course Administration Project 2 results are online Project 3 is out today The first quiz is a week from today (don t panic!) Covers all material up to the quiz Emphasizes lecture material NOT project topics
More informationEE290T : 3D Reconstruction and Recognition
EE290T : 3D Reconstruction and Recognition Acknowledgement Courtesy of Prof. Silvio Savarese. Introduction There was a table set out under a tree in front of the house, and the March Hare and the Hatter
More informationArtistic ideation based on computer vision methods
Journal of Theoretical and Applied Computer Science Vol. 6, No. 2, 2012, pp. 72 78 ISSN 2299-2634 http://www.jtacs.org Artistic ideation based on computer vision methods Ferran Reverter, Pilar Rosado,
More informationBeyond Bags of features Spatial information & Shape models
Beyond Bags of features Spatial information & Shape models Jana Kosecka Many slides adapted from S. Lazebnik, FeiFei Li, Rob Fergus, and Antonio Torralba Detection, recognition (so far )! Bags of features
More informationImage Features and Categorization. Computer Vision Jia-Bin Huang, Virginia Tech
Image Features and Categorization Computer Vision Jia-Bin Huang, Virginia Tech Administrative stuffs Final project Got your proposals! Thanks! Will reply with feedbacks this week. HW 4 Due 11:59pm on Wed,
More informationImage Analysis. Window-based face detection: The Viola-Jones algorithm. iphoto decides that this is a face. It can be trained to recognize pets!
Image Analysis 2 Face detection and recognition Window-based face detection: The Viola-Jones algorithm Christophoros Nikou cnikou@cs.uoi.gr Images taken from: D. Forsyth and J. Ponce. Computer Vision:
More informationThe bits the whirl-wind left out..
The bits the whirl-wind left out.. Reading: Szeliski Background: 1 Edge Features Reading: Szeliski - 1.1 + 4.2.1 (Background: Ch 2 + 3.1 3.3) Background: 2 Edges as Gradients Edge detection = differential
More informationGeneric Object-Face detection
Generic Object-Face detection Jana Kosecka Many slides adapted from P. Viola, K. Grauman, S. Lazebnik and many others Today Window-based generic object detection basic pipeline boosting classifiers face
More informationDeformable Part Models
CS 1674: Intro to Computer Vision Deformable Part Models Prof. Adriana Kovashka University of Pittsburgh November 9, 2016 Today: Object category detection Window-based approaches: Last time: Viola-Jones
More informationLoose Shape Model for Discriminative Learning of Object Categories
Loose Shape Model for Discriminative Learning of Object Categories Margarita Osadchy and Elran Morash Computer Science Department University of Haifa Mount Carmel, Haifa 31905, Israel rita@cs.haifa.ac.il
More informationEfficient Kernels for Identifying Unbounded-Order Spatial Features
Efficient Kernels for Identifying Unbounded-Order Spatial Features Yimeng Zhang Carnegie Mellon University yimengz@andrew.cmu.edu Tsuhan Chen Cornell University tsuhan@ece.cornell.edu Abstract Higher order
More informationMining Discriminative Adjectives and Prepositions for Natural Scene Recognition
Mining Discriminative Adjectives and Prepositions for Natural Scene Recognition Bangpeng Yao 1, Juan Carlos Niebles 2,3, Li Fei-Fei 1 1 Department of Computer Science, Princeton University, NJ 08540, USA
More informationAnnouncements. Recognition. Recognition. Recognition. Recognition. Homework 3 is due May 18, 11:59 PM Reading: Computer Vision I CSE 152 Lecture 14
Announcements Computer Vision I CSE 152 Lecture 14 Homework 3 is due May 18, 11:59 PM Reading: Chapter 15: Learning to Classify Chapter 16: Classifying Images Chapter 17: Detecting Objects in Images Given
More informationSelection of Scale-Invariant Parts for Object Class Recognition
Selection of Scale-Invariant Parts for Object Class Recognition Gy. Dorkó and C. Schmid INRIA Rhône-Alpes, GRAVIR-CNRS 655, av. de l Europe, 3833 Montbonnot, France fdorko,schmidg@inrialpes.fr Abstract
More informationObject Classification Problem
HIERARCHICAL OBJECT CATEGORIZATION" Gregory Griffin and Pietro Perona. Learning and Using Taxonomies For Fast Visual Categorization. CVPR 2008 Marcin Marszalek and Cordelia Schmid. Constructing Category
More informationImproving Recognition through Object Sub-categorization
Improving Recognition through Object Sub-categorization Al Mansur and Yoshinori Kuno Graduate School of Science and Engineering, Saitama University, 255 Shimo-Okubo, Sakura-ku, Saitama-shi, Saitama 338-8570,
More informationGenerative and discriminative classification techniques
Generative and discriminative classification techniques Machine Learning and Category Representation 013-014 Jakob Verbeek, December 13+0, 013 Course website: http://lear.inrialpes.fr/~verbeek/mlcr.13.14
More informationRecognition Tools: Support Vector Machines
CS 2770: Computer Vision Recognition Tools: Support Vector Machines Prof. Adriana Kovashka University of Pittsburgh January 12, 2017 Announcement TA office hours: Tuesday 4pm-6pm Wednesday 10am-12pm Matlab
More informationRecap Image Classification with Bags of Local Features
Recap Image Classification with Bags of Local Features Bag of Feature models were the state of the art for image classification for a decade BoF may still be the state of the art for instance retrieval
More informationObject Category Detection. Slides mostly from Derek Hoiem
Object Category Detection Slides mostly from Derek Hoiem Today s class: Object Category Detection Overview of object category detection Statistical template matching with sliding window Part-based Models
More informationLecture 12 Recognition
Institute of Informatics Institute of Neuroinformatics Lecture 12 Recognition Davide Scaramuzza 1 Lab exercise today replaced by Deep Learning Tutorial Room ETH HG E 1.1 from 13:15 to 15:00 Optional lab
More informationUnsupervised Identification of Multiple Objects of Interest from Multiple Images: discover
Unsupervised Identification of Multiple Objects of Interest from Multiple Images: discover Devi Parikh and Tsuhan Chen Carnegie Mellon University {dparikh,tsuhan}@cmu.edu Abstract. Given a collection of
More informationObject Category Detection: Sliding Windows
04/10/12 Object Category Detection: Sliding Windows Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem Today s class: Object Category Detection Overview of object category detection Statistical
More informationTemplates and Background Subtraction. Prof. D. Stricker Doz. G. Bleser
Templates and Background Subtraction Prof. D. Stricker Doz. G. Bleser 1 Surveillance Video: Example of multiple people tracking http://www.youtube.com/watch?v=inqv34bchem&feature=player_embedded As for
More informationHigh Level Computer Vision
High Level Computer Vision Part-Based Models for Object Class Recognition Part 2 Bernt Schiele - schiele@mpi-inf.mpg.de Mario Fritz - mfritz@mpi-inf.mpg.de http://www.d2.mpi-inf.mpg.de/cv Please Note No
More informationLocal Features based Object Categories and Object Instances Recognition
Local Features based Object Categories and Object Instances Recognition Eric Nowak Ph.D. thesis defense 17th of March, 2008 1 Thesis in Computer Vision Computer vision is the science and technology of
More informationBasic Problem Addressed. The Approach I: Training. Main Idea. The Approach II: Testing. Why a set of vocabularies?
Visual Categorization With Bags of Keypoints. ECCV,. G. Csurka, C. Bray, C. Dance, and L. Fan. Shilpa Gulati //7 Basic Problem Addressed Find a method for Generic Visual Categorization Visual Categorization:
More informationComparing Local Feature Descriptors in plsa-based Image Models
Comparing Local Feature Descriptors in plsa-based Image Models Eva Hörster 1,ThomasGreif 1, Rainer Lienhart 1, and Malcolm Slaney 2 1 Multimedia Computing Lab, University of Augsburg, Germany {hoerster,lienhart}@informatik.uni-augsburg.de
More informationTA Section: Problem Set 4
TA Section: Problem Set 4 Outline Discriminative vs. Generative Classifiers Image representation and recognition models Bag of Words Model Part-based Model Constellation Model Pictorial Structures Model
More informationBias-Variance Trade-off + Other Models and Problems
CS 1699: Intro to Computer Vision Bias-Variance Trade-off + Other Models and Problems Prof. Adriana Kovashka University of Pittsburgh November 3, 2015 Outline Support Vector Machines (review + other uses)
More informationPart-Based Models for Object Class Recognition Part 2
High Level Computer Vision Part-Based Models for Object Class Recognition Part 2 Bernt Schiele - schiele@mpi-inf.mpg.de Mario Fritz - mfritz@mpi-inf.mpg.de https://www.mpi-inf.mpg.de/hlcv Class of Object
More informationPart-Based Models for Object Class Recognition Part 2
High Level Computer Vision Part-Based Models for Object Class Recognition Part 2 Bernt Schiele - schiele@mpi-inf.mpg.de Mario Fritz - mfritz@mpi-inf.mpg.de https://www.mpi-inf.mpg.de/hlcv Class of Object
More informationDetermining Patch Saliency Using Low-Level Context
Determining Patch Saliency Using Low-Level Context Devi Parikh 1, C. Lawrence Zitnick 2, and Tsuhan Chen 1 1 Carnegie Mellon University, Pittsburgh, PA, USA 2 Microsoft Research, Redmond, WA, USA Abstract.
More informationCEE598 - Visual Sensing for Civil Infrastructure Eng. & Mgmt.
CEE598 - Visual Sensing for Civil Infrastructure Eng. & Mgmt. Session 20 Object Recognition 3 Mani Golparvar-Fard Department of Civil and Environmental Engineering 3129D, Newmark Civil Engineering Lab
More informationFuzzy based Multiple Dictionary Bag of Words for Image Classification
Available online at www.sciencedirect.com Procedia Engineering 38 (2012 ) 2196 2206 International Conference on Modeling Optimisation and Computing Fuzzy based Multiple Dictionary Bag of Words for Image
More informationVisual words. Map high-dimensional descriptors to tokens/words by quantizing the feature space.
Visual words Map high-dimensional descriptors to tokens/words by quantizing the feature space. Quantize via clustering; cluster centers are the visual words Word #2 Descriptor feature space Assign word
More informationMethods for Representing and Recognizing 3D objects
Methods for Representing and Recognizing 3D objects part 1 Silvio Savarese University of Michigan at Ann Arbor Object: Building, 45º pose, 8-10 meters away Object: Person, back; 1-2 meters away Object:
More informationLecture 12 Recognition. Davide Scaramuzza
Lecture 12 Recognition Davide Scaramuzza Oral exam dates UZH January 19-20 ETH 30.01 to 9.02 2017 (schedule handled by ETH) Exam location Davide Scaramuzza s office: Andreasstrasse 15, 2.10, 8050 Zurich
More informationToday. Introduction to recognition Alignment based approaches 11/4/2008
Today Introduction to recognition Alignment based approaches Tuesday, Nov 4 Brief recap of visual words Introduction to recognition problem Recognition by alignment, pose clustering Kristen Grauman UT
More informationMultiple-Choice Questionnaire Group C
Family name: Vision and Machine-Learning Given name: 1/28/2011 Multiple-Choice naire Group C No documents authorized. There can be several right answers to a question. Marking-scheme: 2 points if all right
More informationCEE598 - Visual Sensing for Civil Infrastructure Eng. & Mgmt.
CEE598 - Visual Sensing for Civil Infrastructure Eng. & Mgmt. Session 20 Object Recognition 3 Mani Golparvar-Fard Department of Civil and Environmental Engineering Department of Computer Science 3129D,
More informationLecture 15 Visual recognition
Lecture 15 Visual recognition 3D object detection Introduction Single instance 3D object detectors Generic 3D object detectors Silvio Savarese Lecture 15-26-Feb-14 3D object detection Object: Building
More informationLecture 18: Human Motion Recognition
Lecture 18: Human Motion Recognition Professor Fei Fei Li Stanford Vision Lab 1 What we will learn today? Introduction Motion classification using template matching Motion classification i using spatio
More informationCS231M Mobile Computer Vision Spring 2014
CS231M Mobile Computer Vision Spring 2014 Instructors: - Prof Silvio Savarese (Stanford, CVGLab) - Dr Kari Pulli (Nvidia Research) CAs: - Saumitro Dasgupta saumitro@stanford.edu - Francois Chaubard fchaubar@stanford.edu
More informationBayes Risk. Classifiers for Recognition Reading: Chapter 22 (skip 22.3) Discriminative vs Generative Models. Loss functions in classifiers
Classifiers for Recognition Reading: Chapter 22 (skip 22.3) Examine each window of an image Classify object class within each window based on a training set images Example: A Classification Problem Categorize
More informationWhat do we mean by recognition?
Announcements Recognition Project 3 due today Project 4 out today (help session + photos end-of-class) The Margaret Thatcher Illusion, by Peter Thompson Readings Szeliski, Chapter 14 1 Recognition What
More information