Detecting Object Instances Without Discriminative Features
|
|
|
- Helen Delilah Banks
- 5 years ago
- Views:
Transcription
1 Detecting Object Instances Without Discriminative Features Edward Hsiao June 19, 2013 Thesis Committee: Martial Hebert, Chair Alexei Efros Takeo Kanade Andrew Zisserman, University of Oxford 1
2 Object Instance Detection Find this object under arbitrary viewpoint, lighting, clutter and occlusions 2
3 3
4 4
5 Robotic Manipulation 5
6 Scene Understanding 6
7 Scene Understanding Microwave Coffee maker Paper towel Faucet Refrigerator Stove Dishwasher 7
8 Visual Search 8
![Recognition Using [SIFT, Lowe 2004]](/docs-images/87/96183703/images/9-0.jpg "Discriminative Features model test")
9 Recognition Using [SIFT, Lowe 2004] Discriminative Features model test image 9
![[SIFT, Lowe 2004] Extract](/docs-images/87/96183703/images/10-0.jpg "Keypoints model test image")
10 [SIFT, Lowe 2004] Extract Keypoints model test image 10
11 [SIFT, Lowe 2004] Generate 1-To-1 Correspondences model test image 11
12 Enforce Geometric Constraints [SIFT, Lowe 2004] model test image 12
13 [SIFT, Lowe 2004] Recognized Object model test image 13
14 Failure of Feature Matching model test image 0 correct correspondences 14
15 Overview Lack of Discriminative Features Ambiguous Keypoint Features Feature-poor objects Occlusions 15
16 Overview Lack of Discriminative Features Ambiguous Keypoint Features Feature-poor objects Occlusions 16
17 Ambiguous Keypoint Features 17
18 Repeated Patterns 18
19 Failure of Discriminative Matching mdesc 1 mdesc 2... Image keypoint descriptor Model descriptors Geometric model 19
20 Failure of Discriminative Matching? or One-to-one matching mdesc 1 mdesc 2... Image keypoint descriptor Model descriptors Geometric model 20
21 Failure of Discriminative Matching? or One-to-one matching mdesc 1 mdesc 2... Image keypoint descriptor Model descriptors Geometric model Most approaches discard ambiguous features 21
22 Quantized Matching qdesc 1 qdesc 2... Image keypoint descriptor Quantized model descriptors Geometric model 22
23 Quantized Matching qdesc 1 Quantized matching qdesc 2... Image keypoint descriptor Quantized model descriptors Geometric model Preserve ambiguity of match until geometric verification 23
24 Detection Performance CMU Grocery Dataset 0.9 Average Precision (higher is better) images, 10 household objects 0 one-to-one matching [Collet et al. 2009] quantized matching 24
25 Failure of Feature Matching model test image 0 correct correspondences 25
26 Keypoint Comparison Success Failure 26
27 Uninformative Keypoints 27
28 Uninformative Keypoints 28
29 Uninformative Keypoints 29
30 Informative Keypoints 980 keypoints 10 keypoints Keypoints contained entirely within the object 30
31 Informative Keypoints 980 keypoints 10 keypoints Keypoints due to specularities 31
32 Feature-richness More keypoints Less keypoints 32
33 Feature-richness More keypoints Less keypoints 33
34 Feature-richness More keypoints Less keypoints 34
35 Feature-richness More keypoints Less keypoints 35
36 Feature Matching Experiment 36
37 Feature Matching Experiment 37
38 Feature Matching Experiment 38
39 Feature Matching Experiment At least 5 good correspondences between all pairs of images 39
40 Works Fails More keypoints Less keypoints 40
41 Works Fails More keypoints Less keypoints 41
42 Works Fails More keypoints Less keypoints 42
43 Feature-rich Feature-poor More keypoints Less keypoints 43
44 Feature-rich Feature-poor More keypoints Less keypoints 44
45 Overview Lack of Discriminative Features Ambiguous Keypoint Features Feature-poor objects Occlusions 45
46 46
47 Feature-poor Objects Shape Matching Template shape Input window Matched shape 47
![Representing Feature-poor Objects Sparse Edge Points [Berg 2005], [Leordeanu 2007], [Duchenne 2009], [Hinterstoisser 2011] Lines & Contour](/docs-images/87/96183703/images/48-0.jpg "Fragments [Ferrari 2006 & 2008], [Opelt 2006], [Srinivasan 2010] Histogram of Oriented Gradients (HOG) [Dalal and Triggs 2005], [Lai 2011]")
48 Representing Feature-poor Objects Sparse Edge Points [Berg 2005], [Leordeanu 2007], [Duchenne 2009], [Hinterstoisser 2011] Lines & Contour Fragments [Ferrari 2006 & 2008], [Opelt 2006], [Srinivasan 2010] Histogram of Oriented Gradients (HOG) [Dalal and Triggs 2005], [Lai 2011] 48
49 Sparse Edge Points Local information: gradient orientation and color 49
50 Sparse Edge Points Matched Not matched 50
51 Sparse Edge Points Matched Not matched 51
52 Sparse Edge Points Matched Not matched Edge connectivity is lost 52
53 Lines & Contour Fragments 53
54 Lines & Contour Fragments Dependent on edge extraction Splines sensitive to occlusions Line fitting is brittle Difficult to parameterize 54
55 Lines & Contour Fragments Dependent on edge extraction Splines sensitive to occlusions Line fitting is brittle Difficult to parameterize 55
56 Histogram of Oriented Gradients Coarse statistics of gradient orientation and magnitude 56
57 Histogram of Oriented Gradients patch HOG patch HOG Corrupted by background clutter Ambiguous shape 57
58 Histogram of Oriented Gradients patch HOG patch HOG Corrupted by background clutter Ambiguous shape 58
59 Gradient Networks Our Approach 1. Match shape explicitly 2. Enforce connectivity without extracting edges 59
60 Gradient Networks Overview Shape template Input window 60
61 Gradient Networks Overview Shape template Input window 61
62 Gradient Networks Local Shape Potential How well does each pixel match locally? 62
63 Gradient Networks Predicted Shape Match Find long connected components which follow shape 63
64 Local Shape Potential Distance to template Local orientation Color Edge potential 64
65 Local Shape Potential Distance to template Local orientation Color Edge potential 65
66 Local Shape Potential Distance to template Local orientation Color Edge potential 66
67 Local Orientation Potential local orientation potential model test 67
68 Local Shape Potential Distance to template Local orientation Color Edge potential 68
69 Local Shape Potential Distance to template Local orientation Color Edge potential 69
70 Local Shape Potential 70
71 Gradient Networks p p Each pixel is a node in the network 71
72 Gradient Networks p Q 0 p p q p Q 1 Connect each node to neighbors in tangent direction 72
73 Gradient Networks p p Find paths in the network that match the shape well 73
74 [Bhat et al. 2010] Message Passing Local shape potential p shape similarity local shape potential message from left message from right 74
75 Message Passing Local shape potential p Initially, it is just the local shape potential 75
76 Message Passing Local shape potential p 76
77 Message Passing Local shape potential p 77
78 Message Passing Local shape potential p 78
79 Predicted Shape Match Message passing Local shape potential Predicted match 79
80 CMU Kitchen Occlusion Dataset 1600 images of 8 feature-poor objects Single and multiple viewpoints Cluttered scenes and occlusions Objects Example images 80
81 Shape Matching Results Template Input window Local shape potential Predicted match 81
82 Shape Matching Results Template Input window Local shape potential Predicted match 82
83 Shape Matching Results Template Input window Local shape potential Predicted match 83
84 Object Detection Sliding Window 84
85 Object Detection Sliding Window 85
86 Detection Performance better 86
87 False positives with shape only Object False positive window GN point-wise confidences 88
88 Interior Appearance Object False positive window GN point-wise confidences 89
89 BaRT Boundary and Region Templates 90
90 BaRT Boundary and Region Templates 91
91 Boundary Explicit shape: rline2d and GN 92
92 BaRT Boundary and Region Templates 93
93 Region Consider appearance within the object interior HOG and color 94
94 BaRT Boundary and Region Templates 95
95 BaRT Combines explicit boundary and region information 96
96 HOG Uniform Regions Uniform regions not represented well 97
97 HOG Normalization Each cell normalized with respect to magnitude of neighbors 98
98 HOG Normalization Amplifies noise if magnitude close to 0 99
99 Uniform Regions 100
100 Learning? HOG + SVM Multiple images weight = 0 HOG + exemplar SVM Single image weight = random 101
101 Learning? HOG + SVM Multiple images weight = 0 HOG + exemplar SVM Single image weight = random 102
102 Learning? HOG + SVM Multiple images weight = 0 HOG + exemplar SVM Single image weight = random 103
103 Modify HOG Normalization HOG Modified HOG Set cell to zero if normalization below threshold 104
104 Matching Uniform Regions HOG Ours Test image: HOG Ours 105
105 Matching Uniform Regions HOG Ours Test image: HOG Ours 106
106 Matching Uniform Regions HOG Ours Test image: HOG Ours More accurate confidences in uniform regions 107
107 Example Detections detection zoomed in boundary (GN) region (HOG+color) 108
108 Example Detections detection zoomed in boundary (GN) region (HOG+color) 109
109 Example Detections detection zoomed in boundary (GN) region (HOG+color) 110
110 Detection Performance 112
111 Detection Performance 113
112 Detection Performance Under Different Occlusion Levels 114
113 Detection Performance Under Different Occlusion Levels 115
114 Overview Lack of Discriminative Features Ambiguous Keypoint Features Feature-poor objects Occlusions 116
115 Occlusions 117
116 Occlusions 118
117 Occlusions happen in 3D 119
118 Occlusions happen in 3D 120
119 Occlusions happen in 3D 121
120 Occlusions happen in 3D 122
121 Occlusion Reasoning Matched Not matched Which of these hypotheses is most likely? 123
122 Occlusion Reasoning Matched Not matched Which of these hypotheses is most likely? 124
123 Occlusion Reasoning Matched Not matched Which of these hypotheses is most likely? 125
124 Occlusion Reasoning Matched Not matched Which of these hypotheses is most likely? 126
125 Occlusion Reasoning Local Coherency Fransens 06, Wang 09 Object Detection Depth Ordering Wu 05, Wang 11 Learn Occlusion Structure Gao 11, Kwak
126 Structure of Occlusions Occlusion Conditional Likelihood Probability a point is visible given the visibility labeling of all other points Binary variable that equals 1 if is visible Matched Not matched Occlusion under a given camera view point c 128
127 Occlusion Reasoning Per Environment H obj L obj W obj Estimate of object dimensions Distribution of object dimensions for a given environment 129
128 Occlusion Model 130
129 Occlusion Model Object Occluder 131
130 Occlusion Model Wˆobj Object Ĥ obj ĥ Occluder ŵ 132
131 Occlusion Model Wˆobj Object Ĥ obj ĥ Occluder ŵ 133
132 Occlusion Conditional Likelihood X i X j X j A Vj,O c A Vi,V j,o c Integral Geometry 134
133 Occlusion Conditional Likelihood X i X j X j A Vj,O c A Vi,V j,o c Area covering all positions where X j is visible and object occluded 135
134 Occlusion Conditional Likelihood X i X j X j A Vj,O c A Vi,V j,o c Area covering all positions where X j is visible and object occluded 136
135 Occlusion Conditional Likelihood X i X j X j A Vj,O c A Vi,V j,o c Area covering all positions where X j is visible and object occluded 137
136 Occlusion Conditional Likelihood X i X j X j A Vj,O c A Vi,V j,o c Area covering all positions where X j and X j are visible and object occluded 138
137 Occlusion Conditional Likelihood X j 139
138 Occlusion Conditional Likelihood Under Different Viewpoints 140
139 Occlusion Conditional Likelihood Under Different Viewpoints 141
140 Occlusion Conditional Likelihood Penalty (OCLP) X i Matched Not matched f OCLP : High penalty if unlikely to be occluded by a valid object on same support surface 142
141 Occlusion Conditional Likelihood Penalty (OCLP) X i Matched Not matched f OCLP : Low penalty if likely to be occluded by a valid object on same support surface 143
142 Occlusion Conditional Likelihood Penalty (OCLP) X i Matched Not matched f OCLP : Low penalty if likely to be occluded by a valid object on same support surface 144
143 Example Detections 145
144 Detection Performance 146
145 Detection Performance Under Different Occlusion Levels 147
146 Limitation Binary Matching Pattern Occlusion Conditional Likelihood 148
147 Limitation Binary Matching Pattern Occlusion Conditional Likelihood Misclassifications can have impact on distribution 149
148 Occlusion Efficient Subwindow Search (OESS) Probabilistic Matching Pattern Probabilistic Matching Pattern 150
149 OESS for True Positive Occlusion can be explained well 151
150 OESS for True Positive 95% explained 152
151 OESS for False Positive 153
152 OESS for False Positive Only 50% explained 154
153 OESS Scoring Matching Pattern p = p = 0 score = (1) + (1) + (-1) + (-1) = 0 155
154 OESS Scoring Matching Pattern p = 1 rewarded +1 Occluding block -1 p = 0 score = (1) + (1) + (1) + (-1) = 2 156
155 OESS Scoring Matching Pattern penalized p = p = 0 Occluding block score = (-1) + (1) + (1) + (-1) = 0 157
156 OESS Reformulate as Efficient Subwindow Search (ESS) 158
157 OESS Find best occluder object 159
158 OESS Remove all explained points 160
159 OESS Iterate 161
160 OESS Iterate 162
161 OESS Iterate 163
162 OESS Final prediction 164
163 Results detection window boundary region oboxes predicted groundtruth 165
164 Results detection window boundary region oboxes predicted groundtruth 166
165 Results detection window boundary region oboxes predicted groundtruth 167
166 Results detection window boundary region oboxes predicted groundtruth 168
167 Occlusion Prediction Performance predicted vs. groundtruth Average Intersection over Union (IoU) 169
168 Occlusion Prediction Performance vs. predicted groundtruth 170
169 Detection Performance 171
170 172
171 Summary Lack of Discriminative Features Gradient Networks Occlusion Conditional Likelihood Boundary and Region Templates Occlusion Efficient Subwindow Search Ambiguous Keypoint Features Feature-poor objects Occlusions 173
172 Main Contributions Ambiguous Keypoint Features Making specific features less discriminative 174
173 Main Contributions Representing Feature-poor Objects Gradient Networks Explicit shape matching without extracting edges Boundary and Region Templates Capture explicit boundary and region information 175
174 Main Contributions Representing Feature-poor Objects Gradient Networks Explicit shape matching without extracting edges Boundary and Region Templates Capture explicit boundary and region information 176
175 Main Contributions Representing Feature-poor Objects Gradient Networks Explicit shape matching without extracting edges Boundary and Region Templates Capture explicit boundary and region information 177
176 Main Contributions Occlusion Reasoning Occlusion Conditional Likelihood Representing occlusion structure under arbitrary viewpoint Occlusion Efficient Subwindow Search Directly search for occluding blocks to explain matching pattern 178
177 Main Contributions Occlusion Reasoning Occlusion Conditional Likelihood Representing occlusion structure under arbitrary viewpoint Occlusion Efficient Subwindow Search Directly search for occluding blocks to explain matching pattern 179
178 Main Contributions Occlusion Reasoning Occlusion Conditional Likelihood Representing occlusion structure under arbitrary viewpoint Occlusion Efficient Subwindow Search Directly search for occluding blocks to explain matching pattern 180
179 Acknowledgements Martial Hebert Alexei Efros Takeo Kanade Andrew Zisserman 181
180 182
181 183
182 Background 184
183 Augmented Reality 3D model Target environment 185
184 Augmented Reality 3D model Target environment 186
185 Instance vs. Category Recognition Instance Arbitrary viewpoint and lighting Single image per view Category Intra-class variations Many images per view 187
186 Ambiguous Viewpoint 188
187 Failure of SIFT Matching 189
188 Invariant Approaches 190
189 Future Directions Fine-grained verification Scalability 3D 191
190 Fine-grained Verification 192
191 Scalability 193
192 3D 194
193 Datasets 195
194 CMU Grocery Dataset 620 images of household objects 10 objects 25 single instance, 25 double instance 12 with ground truth pose Clutter, viewpoint, lighting, occlusion
195 CMU Kitchen Occlusion Dataset 1600 images of 8 household objects Single and multiple viewpoints Cluttered scenes and occlusions Objects Example images Hsiao and Hebert, CVPR
196 Gradient Networks 198
197 Local Shape Potential Region of influence Appearance Edge 199
198 Local Appearance Gradient Orientation Color 200
199 Potentials Unary Pairwise 201
200 Message Passing Shape Similarity 202
201 Probability Calibration Weibull fit to tail of negative distribution CDF of NOT Object Density of NOT Object Probability of Object NOT Object Object scores 203 Scheirer et al. CVPR 2012
202 Soft Shape Model 204
203 Additional Results 205
204 Color Potential 206
205 LINE2D Similarity Quantized gradient orientation of model point, p i θ i Quantized gradient orientation of the best matching image point in a local neighborhood p i Model point score LINE N 2 D = cos( θi ) i= 1 o cos( 0 ) = 1.00 o cos( 45 ) = 0.71 LINE2D (Hinterstoisser et al., PAMI 2011) 207
206 Robust LINE2D Similarity Quantized gradient orientation of model point, p i θ i Quantized gradient orientation of the best matching image point in a local neighborhood p i Model point N score rline 2 D = δ ( θi = 0) i= 1 rline2d (Hsiao and Hebert, CVPR 2012) 208
207 Message Passing Iterations 209
208 Probability Calibration 210
209 F-Measure of Shape Matching 211
210 Single View 212
211 Multiple View 213
212 Detection 1.0 FPPI 214
213 Detection 1.0 FPPI 215
214 False Positives 216
215 BaRT 217
216 Grid Optimization Un-optimized : 57 cells Optimized : 60 cells 218
217 HOG Normalization Amplifies noise in uniform region! 219
218 HOG Normalization Sensitive to shading effects! 220
219 HOG Normalization Pedestrians 221
220 Average Precision 222
221 Single View 223
222 Multiple View 224
223 False positives Match both boundary and region 225
224 BaRT False Positives Insufficient edge evidence Unlikely occlusion configuration Region information is only informative after there is a plausible hypothesis based on the boundary 226
225 Occlusion Reasoning 227
226 Occlusion Model 228
227 Occlusion Scoring Object detector Occlusion model Score of window Sliding window Occlusion hypothesis (binary) 229
228 Occlusion Conditional Likelihood 230
229 Occlusion Conditional Likelihood Approximation Approximate Analytic 231
230 Distribution of Physical Dimensions Household Objects 232
231 Occlusion Statistics 233
232 Validity of Occlusion Model 234
233 Occlusion Penalty Occlusion Prior Penalty (OPP) Occlusion Conditional Likelihood Penalty (OCLP) 235
234 Average Precision 236
235 Performance vs. Occlusion 237
236 Learning from Data 238
237 Parameter Sensitivity 239
238 OESS 240
239 Occlusion Upper Bound 241
240 OESS Algorithm 242
241 OESS vs. Brute Force 243
242 Occlusion Prediction 244
243 Object Detection Performance 245
244 Ambiguous Features 246
245 Problem Not enough correct matches Difficult to obtain matches Result of our system
246 Discriminative hierarchical matching (DHM) discriminative match Model features (Level 0) Image features discriminative match Quantized features (Level 1) aggregate Candidate correspondences discriminative match Quantized features (Level 2)
247 DHM example All features
248 DHM result Ratio test 3 correct matches (soymilk can) DHM 11 correct matches (soymilk can)
249 Simulated Affine (SA) Morel & Yu 2009
250 Baseline systems Gordon & Lowe SIFT + RANSAC Levenberg-Marquardt non-linear optimization Enhanced PnP (EPnP) Gordon & Lowe EPnP non-iterative pose estimation algorithm Collet et al. Gordon & Lowe Mean-shift spatial clustering of image features
251 Averaged precision-recall
252 Average Precision
253 Object detection results
254 Failure cases Pose ambiguity Repeated patterns Extreme lighting, occlusion, viewpoint etc
Coherent Occlusion Reasoning for Instance Recognition
oherent Occlusion Reasoning for Instance Recognition Edward Hsiao and Martial Hebert The Robotics Institute arnegie Mellon University Pittsburgh, PA 15213 Abstract Occlusions are common in real world scenes
Deformable Part Models
CS 1674: Intro to Computer Vision Deformable Part Models Prof. Adriana Kovashka University of Pittsburgh November 9, 2016 Today: Object category detection Window-based approaches: Last time: Viola-Jones
Category-level localization
Category-level localization Cordelia Schmid Recognition Classification Object present/absent in an image Often presence of a significant amount of background clutter Localization / Detection Localize object
Object Category Detection. Slides mostly from Derek Hoiem
Object Category Detection Slides mostly from Derek Hoiem Today s class: Object Category Detection Overview of object category detection Statistical template matching with sliding window Part-based Models
Detection III: Analyzing and Debugging Detection Methods
CS 1699: Intro to Computer Vision Detection III: Analyzing and Debugging Detection Methods Prof. Adriana Kovashka University of Pittsburgh November 17, 2015 Today Review: Deformable part models How can
Category vs. instance recognition
Category vs. instance recognition Category: Find all the people Find all the buildings Often within a single image Often sliding window Instance: Is this face James? Find this specific famous building
Three-Dimensional Object Detection and Layout Prediction using Clouds of Oriented Gradients
ThreeDimensional Object Detection and Layout Prediction using Clouds of Oriented Gradients Authors: Zhile Ren, Erik B. Sudderth Presented by: Shannon Kao, Max Wang October 19, 2016 Introduction Given an
Feature descriptors. Alain Pagani Prof. Didier Stricker. Computer Vision: Object and People Tracking
Feature descriptors Alain Pagani Prof. Didier Stricker Computer Vision: Object and People Tracking 1 Overview Previous lectures: Feature extraction Today: Gradiant/edge Points (Kanade-Tomasi + Harris)
Shape-based instance detection under arbitrary viewpoint
Shape-based instance detection under arbitrary viewpoint Edward Hsiao and Martial Hebert Abstract Shape-based instance detection under arbitrary viewpoint is a very challenging problem. Current approaches
Object Detection by 3D Aspectlets and Occlusion Reasoning
Object Detection by 3D Aspectlets and Occlusion Reasoning Yu Xiang University of Michigan Silvio Savarese Stanford University In the 4th International IEEE Workshop on 3D Representation and Recognition
2D Image Processing Feature Descriptors
2D Image Processing Feature Descriptors Prof. Didier Stricker Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche Intelligenz http://av.dfki.de 1 Overview
Recognition of Animal Skin Texture Attributes in the Wild. Amey Dharwadker (aap2174) Kai Zhang (kz2213)
Recognition of Animal Skin Texture Attributes in the Wild Amey Dharwadker (aap2174) Kai Zhang (kz2213) Motivation Patterns and textures are have an important role in object description and understanding
Occlusion Reasoning for Object Detection under Arbitrary Viewpoint
Occlusion Reasoning for Object Detection under Arbitrary Viewpoint Edward Hsiao and Martial Hebert Robotics Institute, Carnegie Mellon University, USA {ehsiao,hebert}@cs.cmu.edu Abstract We present a unified
OCCLUSIONS are common in real world scenes and
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. X, NO. X, MONTH 23 Occlusion Reasoning for Object Detection under Arbitrary Viewpoint Edward Hsiao, Student Member, IEEE, and Martial
Object Recognition with Invariant Features
Object Recognition with Invariant Features Definition: Identify objects or scenes and determine their pose and model parameters Applications Industrial automation and inspection Mobile robots, toys, user
Local features and image matching. Prof. Xin Yang HUST
Local features and image matching Prof. Xin Yang HUST Last time RANSAC for robust geometric transformation estimation Translation, Affine, Homography Image warping Given a 2D transformation T and a source
Object Category Detection: Sliding Windows
04/10/12 Object Category Detection: Sliding Windows Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem Today s class: Object Category Detection Overview of object category detection Statistical
Making specific features less discriminative to improve point-based 3D object recognition
Making specific features less discriminative to improve point-based 3D object recognition Edward Hsiao, Alvaro Collet and Martial Hebert Robotics Institute, Carnegie Mellon University, USA {ehsiao,acollet,hebert}@cscmuedu
Image Features: Local Descriptors. Sanja Fidler CSC420: Intro to Image Understanding 1/ 58
Image Features: Local Descriptors Sanja Fidler CSC420: Intro to Image Understanding 1/ 58 [Source: K. Grauman] Sanja Fidler CSC420: Intro to Image Understanding 2/ 58 Local Features Detection: Identify
Stereo. Outline. Multiple views 3/29/2017. Thurs Mar 30 Kristen Grauman UT Austin. Multi-view geometry, matching, invariant features, stereo vision
Stereo Thurs Mar 30 Kristen Grauman UT Austin Outline Last time: Human stereopsis Epipolar geometry and the epipolar constraint Case example with parallel optical axes General case with calibrated cameras
Object Detection with Partial Occlusion Based on a Deformable Parts-Based Model
Object Detection with Partial Occlusion Based on a Deformable Parts-Based Model Johnson Hsieh (johnsonhsieh@gmail.com), Alexander Chia (alexchia@stanford.edu) Abstract -- Object occlusion presents a major
A novel template matching method for human detection
University of Wollongong Research Online Faculty of Informatics - Papers (Archive) Faculty of Engineering and Information Sciences 2009 A novel template matching method for human detection Duc Thanh Nguyen
Evaluation and comparison of interest points/regions
Introduction Evaluation and comparison of interest points/regions Quantitative evaluation of interest point/region detectors points / regions at the same relative location and area Repeatability rate :
HISTOGRAMS OF ORIENTATIO N GRADIENTS
HISTOGRAMS OF ORIENTATIO N GRADIENTS Histograms of Orientation Gradients Objective: object recognition Basic idea Local shape information often well described by the distribution of intensity gradients
Visuelle Perzeption für Mensch- Maschine Schnittstellen
Visuelle Perzeption für Mensch- Maschine Schnittstellen Vorlesung, WS 2009 Prof. Dr. Rainer Stiefelhagen Dr. Edgar Seemann Institut für Anthropomatik Universität Karlsruhe (TH) http://cvhci.ira.uka.de
Beyond Bags of features Spatial information & Shape models
Beyond Bags of features Spatial information & Shape models Jana Kosecka Many slides adapted from S. Lazebnik, FeiFei Li, Rob Fergus, and Antonio Torralba Detection, recognition (so far )! Bags of features
Discriminative classifiers for image recognition
Discriminative classifiers for image recognition May 26 th, 2015 Yong Jae Lee UC Davis Outline Last time: window-based generic object detection basic pipeline face detection with boosting as case study
Visuelle Perzeption für Mensch- Maschine Schnittstellen
Visuelle Perzeption für Mensch- Maschine Schnittstellen Vorlesung, WS 2009 Prof. Dr. Rainer Stiefelhagen Dr. Edgar Seemann Institut für Anthropomatik Universität Karlsruhe (TH) http://cvhci.ira.uka.de
Combining PGMs and Discriminative Models for Upper Body Pose Detection
Combining PGMs and Discriminative Models for Upper Body Pose Detection Gedas Bertasius May 30, 2014 1 Introduction In this project, I utilized probabilistic graphical models together with discriminative
CS 231A Computer Vision (Winter 2018) Problem Set 3
CS 231A Computer Vision (Winter 2018) Problem Set 3 Due: Feb 28, 2018 (11:59pm) 1 Space Carving (25 points) Dense 3D reconstruction is a difficult problem, as tackling it from the Structure from Motion
OCCLUSION BOUNDARIES ESTIMATION FROM A HIGH-RESOLUTION SAR IMAGE
OCCLUSION BOUNDARIES ESTIMATION FROM A HIGH-RESOLUTION SAR IMAGE Wenju He, Marc Jäger, and Olaf Hellwich Berlin University of Technology FR3-1, Franklinstr. 28, 10587 Berlin, Germany {wenjuhe, jaeger,
Human detection solution for a retail store environment
FACULDADE DE ENGENHARIA DA UNIVERSIDADE DO PORTO Human detection solution for a retail store environment Vítor Araújo PREPARATION OF THE MSC DISSERTATION Mestrado Integrado em Engenharia Eletrotécnica
Object Detection Design challenges
Object Detection Design challenges How to efficiently search for likely objects Even simple models require searching hundreds of thousands of positions and scales Feature design and scoring How should
Automatic Learning and Extraction of Multi-Local Features
Automatic Learning and Extraction of Multi-Local Features Oscar Danielsson, Stefan Carlsson and Josephine Sullivan School of Computer Science and Communications Royal Inst. of Technology, Stockholm, Sweden
CS 231A Computer Vision (Winter 2014) Problem Set 3
CS 231A Computer Vision (Winter 2014) Problem Set 3 Due: Feb. 18 th, 2015 (11:59pm) 1 Single Object Recognition Via SIFT (45 points) In his 2004 SIFT paper, David Lowe demonstrates impressive object recognition
CS 223B Computer Vision Problem Set 3
CS 223B Computer Vision Problem Set 3 Due: Feb. 22 nd, 2011 1 Probabilistic Recursion for Tracking In this problem you will derive a method for tracking a point of interest through a sequence of images.
Learning and Inferring Depth from Monocular Images. Jiyan Pan April 1, 2009
Learning and Inferring Depth from Monocular Images Jiyan Pan April 1, 2009 Traditional ways of inferring depth Binocular disparity Structure from motion Defocus Given a single monocular image, how to infer
CS4670: Computer Vision
CS4670: Computer Vision Noah Snavely Lecture 6: Feature matching and alignment Szeliski: Chapter 6.1 Reading Last time: Corners and blobs Scale-space blob detector: Example Feature descriptors We know
CS 231A Computer Vision (Fall 2012) Problem Set 3
CS 231A Computer Vision (Fall 2012) Problem Set 3 Due: Nov. 13 th, 2012 (2:15pm) 1 Probabilistic Recursion for Tracking (20 points) In this problem you will derive a method for tracking a point of interest
Find that! Visual Object Detection Primer
Find that! Visual Object Detection Primer SkTech/MIT Innovation Workshop August 16, 2012 Dr. Tomasz Malisiewicz tomasz@csail.mit.edu Find that! Your Goals...imagine one such system that drives information
Bus Detection and recognition for visually impaired people
Bus Detection and recognition for visually impaired people Hangrong Pan, Chucai Yi, and Yingli Tian The City College of New York The Graduate Center The City University of New York MAP4VIP Outline Motivation
Lecture 12 Recognition
Institute of Informatics Institute of Neuroinformatics Lecture 12 Recognition Davide Scaramuzza 1 Lab exercise today replaced by Deep Learning Tutorial Room ETH HG E 1.1 from 13:15 to 15:00 Optional lab
Colorado School of Mines. Computer Vision. Professor William Hoff Dept of Electrical Engineering &Computer Science.
Professor William Hoff Dept of Electrical Engineering &Computer Science http://inside.mines.edu/~whoff/ 1 People Detection Some material for these slides comes from www.cs.cornell.edu/courses/cs4670/2012fa/lectures/lec32_object_recognition.ppt
CS 558: Computer Vision 13 th Set of Notes
CS 558: Computer Vision 13 th Set of Notes Instructor: Philippos Mordohai Webpage: www.cs.stevens.edu/~mordohai E-mail: Philippos.Mordohai@stevens.edu Office: Lieb 215 Overview Context and Spatial Layout
CRF Based Point Cloud Segmentation Jonathan Nation
CRF Based Point Cloud Segmentation Jonathan Nation jsnation@stanford.edu 1. INTRODUCTION The goal of the project is to use the recently proposed fully connected conditional random field (CRF) model to
Selective Search for Object Recognition
Selective Search for Object Recognition Uijlings et al. Schuyler Smith Overview Introduction Object Recognition Selective Search Similarity Metrics Results Object Recognition Kitten Goal: Problem: Where
Large-scale visual recognition The bag-of-words representation
Large-scale visual recognition The bag-of-words representation Florent Perronnin, XRCE Hervé Jégou, INRIA CVPR tutorial June 16, 2012 Outline Bag-of-words Large or small vocabularies? Extensions for instance-level
Separating Objects and Clutter in Indoor Scenes
Separating Objects and Clutter in Indoor Scenes Salman H. Khan School of Computer Science & Software Engineering, The University of Western Australia Co-authors: Xuming He, Mohammed Bennamoun, Ferdous
Templates and Background Subtraction. Prof. D. Stricker Doz. G. Bleser
Templates and Background Subtraction Prof. D. Stricker Doz. G. Bleser 1 Surveillance Video: Example of multiple people tracking http://www.youtube.com/watch?v=inqv34bchem&feature=player_embedded As for
Lecture 12 Recognition. Davide Scaramuzza
Lecture 12 Recognition Davide Scaramuzza Oral exam dates UZH January 19-20 ETH 30.01 to 9.02 2017 (schedule handled by ETH) Exam location Davide Scaramuzza s office: Andreasstrasse 15, 2.10, 8050 Zurich
Three things everyone should know to improve object retrieval. Relja Arandjelović and Andrew Zisserman (CVPR 2012)
Three things everyone should know to improve object retrieval Relja Arandjelović and Andrew Zisserman (CVPR 2012) University of Oxford 2 nd April 2012 Large scale object retrieval Find all instances of
Previously. Window-based models for generic object detection 4/11/2011
Previously for generic object detection Monday, April 11 UT-Austin Instance recognition Local features: detection and description Local feature matching, scalable indexing Spatial verification Intro to
Pose estimation using a variety of techniques
Pose estimation using a variety of techniques Keegan Go Stanford University keegango@stanford.edu Abstract Vision is an integral part robotic systems a component that is needed for robots to interact robustly
Learning and Matching Line Aspects for Articulated Objects
Learning and Matching Line Aspects for Articulated Objects Xiaofeng Ren Toyota Technological Institute at Chicago 1427 E. 6th Street, Chicago, IL 6637 xren@tti-c.org Abstract Traditional aspect graphs
Augmenting Reality, Naturally:
Augmenting Reality, Naturally: Scene Modelling, Recognition and Tracking with Invariant Image Features by Iryna Gordon in collaboration with David G. Lowe Laboratory for Computational Intelligence Department
Introduction. Introduction. Related Research. SIFT method. SIFT method. Distinctive Image Features from Scale-Invariant. Scale.
Distinctive Image Features from Scale-Invariant Keypoints David G. Lowe presented by, Sudheendra Invariance Intensity Scale Rotation Affine View point Introduction Introduction SIFT (Scale Invariant Feature
Previously. Part-based and local feature models for generic object recognition. Bag-of-words model 4/20/2011
Previously Part-based and local feature models for generic object recognition Wed, April 20 UT-Austin Discriminative classifiers Boosting Nearest neighbors Support vector machines Useful for object recognition
Seminar Heidelberg University
Seminar Heidelberg University Mobile Human Detection Systems Pedestrian Detection by Stereo Vision on Mobile Robots Philip Mayer Matrikelnummer: 3300646 Motivation Fig.1: Pedestrians Within Bounding Box
Classifying Images with Visual/Textual Cues. By Steven Kappes and Yan Cao
Classifying Images with Visual/Textual Cues By Steven Kappes and Yan Cao Motivation Image search Building large sets of classified images Robotics Background Object recognition is unsolved Deformable shaped
Mobile Human Detection Systems based on Sliding Windows Approach-A Review
Mobile Human Detection Systems based on Sliding Windows Approach-A Review Seminar: Mobile Human detection systems Njieutcheu Tassi cedrique Rovile Department of Computer Engineering University of Heidelberg
LEARNING TO GENERATE CHAIRS WITH CONVOLUTIONAL NEURAL NETWORKS
LEARNING TO GENERATE CHAIRS WITH CONVOLUTIONAL NEURAL NETWORKS Alexey Dosovitskiy, Jost Tobias Springenberg and Thomas Brox University of Freiburg Presented by: Shreyansh Daftry Visual Learning and Recognition
Recap Image Classification with Bags of Local Features
Recap Image Classification with Bags of Local Features Bag of Feature models were the state of the art for image classification for a decade BoF may still be the state of the art for instance retrieval
Large-Scale Traffic Sign Recognition based on Local Features and Color Segmentation
Large-Scale Traffic Sign Recognition based on Local Features and Color Segmentation M. Blauth, E. Kraft, F. Hirschenberger, M. Böhm Fraunhofer Institute for Industrial Mathematics, Fraunhofer-Platz 1,
Action recognition in videos
Action recognition in videos Cordelia Schmid INRIA Grenoble Joint work with V. Ferrari, A. Gaidon, Z. Harchaoui, A. Klaeser, A. Prest, H. Wang Action recognition - goal Short actions, i.e. drinking, sit
Segmentation and Tracking of Partial Planar Templates
Segmentation and Tracking of Partial Planar Templates Abdelsalam Masoud William Hoff Colorado School of Mines Colorado School of Mines Golden, CO 800 Golden, CO 800 amasoud@mines.edu whoff@mines.edu Abstract
Tri-modal Human Body Segmentation
Tri-modal Human Body Segmentation Master of Science Thesis Cristina Palmero Cantariño Advisor: Sergio Escalera Guerrero February 6, 2014 Outline 1 Introduction 2 Tri-modal dataset 3 Proposed baseline 4
Window based detectors
Window based detectors CS 554 Computer Vision Pinar Duygulu Bilkent University (Source: James Hays, Brown) Today Window-based generic object detection basic pipeline boosting classifiers face detection
Development in Object Detection. Junyuan Lin May 4th
Development in Object Detection Junyuan Lin May 4th Line of Research [1] N. Dalal and B. Triggs. Histograms of oriented gradients for human detection, CVPR 2005. HOG Feature template [2] P. Felzenszwalb,
C. Premsai 1, Prof. A. Kavya 2 School of Computer Science, School of Computer Science Engineering, Engineering VIT Chennai, VIT Chennai
Traffic Sign Detection Via Graph-Based Ranking and Segmentation Algorithm C. Premsai 1, Prof. A. Kavya 2 School of Computer Science, School of Computer Science Engineering, Engineering VIT Chennai, VIT
CS4495/6495 Introduction to Computer Vision. 3B-L3 Stereo correspondence
CS4495/6495 Introduction to Computer Vision 3B-L3 Stereo correspondence For now assume parallel image planes Assume parallel (co-planar) image planes Assume same focal lengths Assume epipolar lines are
Outline 7/2/201011/6/
Outline Pattern recognition in computer vision Background on the development of SIFT SIFT algorithm and some of its variations Computational considerations (SURF) Potential improvement Summary 01 2 Pattern
Shape recognition with edge-based features
Shape recognition with edge-based features K. Mikolajczyk A. Zisserman C. Schmid Dept. of Engineering Science Dept. of Engineering Science INRIA Rhône-Alpes Oxford, OX1 3PJ Oxford, OX1 3PJ 38330 Montbonnot
SIFT: SCALE INVARIANT FEATURE TRANSFORM SURF: SPEEDED UP ROBUST FEATURES BASHAR ALSADIK EOS DEPT. TOPMAP M13 3D GEOINFORMATION FROM IMAGES 2014
SIFT: SCALE INVARIANT FEATURE TRANSFORM SURF: SPEEDED UP ROBUST FEATURES BASHAR ALSADIK EOS DEPT. TOPMAP M13 3D GEOINFORMATION FROM IMAGES 2014 SIFT SIFT: Scale Invariant Feature Transform; transform image
5*4%),- CS4495/6495 Introduction to Computer Vision. 8C-L1 Classification: Discriminative models
CS4495/6495 Introduction to Computer Vision 8C-L1 Classification: Discriminative models Remember: Supervised classification Given a collection of labeled examples, come up with a function that will predict
Classification and Detection in Images. D.A. Forsyth
Classification and Detection in Images D.A. Forsyth Classifying Images Motivating problems detecting explicit images classifying materials classifying scenes Strategy build appropriate image features train
SURF. Lecture6: SURF and HOG. Integral Image. Feature Evaluation with Integral Image
SURF CSED441:Introduction to Computer Vision (2015S) Lecture6: SURF and HOG Bohyung Han CSE, POSTECH bhhan@postech.ac.kr Speed Up Robust Features (SURF) Simplified version of SIFT Faster computation but
HOG-based Pedestriant Detector Training
HOG-based Pedestriant Detector Training evs embedded Vision Systems Srl c/o Computer Science Park, Strada Le Grazie, 15 Verona- Italy http: // www. embeddedvisionsystems. it Abstract This paper describes
Local features: detection and description. Local invariant features
Local features: detection and description Local invariant features Detection of interest points Harris corner detection Scale invariant blob detection: LoG Description of local patches SIFT : Histograms
Semi-Supervised Hierarchical Models for 3D Human Pose Reconstruction
Semi-Supervised Hierarchical Models for 3D Human Pose Reconstruction Atul Kanaujia, CBIM, Rutgers Cristian Sminchisescu, TTI-C Dimitris Metaxas,CBIM, Rutgers 3D Human Pose Inference Difficulties Towards
Multiple-Person Tracking by Detection
http://excel.fit.vutbr.cz Multiple-Person Tracking by Detection Jakub Vojvoda* Abstract Detection and tracking of multiple person is challenging problem mainly due to complexity of scene and large intra-class
Beyond Bags of Features
: for Recognizing Natural Scene Categories Matching and Modeling Seminar Instructed by Prof. Haim J. Wolfson School of Computer Science Tel Aviv University December 9 th, 2015
3D Object Recognition and Scene Understanding from RGB-D Videos. Yu Xiang Postdoctoral Researcher University of Washington
3D Object Recognition and Scene Understanding from RGB-D Videos Yu Xiang Postdoctoral Researcher University of Washington 1 2 Act in the 3D World Sensing & Understanding Acting Intelligent System 3D World
Midterm Wed. Local features: detection and description. Today. Last time. Local features: main components. Goal: interest operator repeatability
Midterm Wed. Local features: detection and description Monday March 7 Prof. UT Austin Covers material up until 3/1 Solutions to practice eam handed out today Bring a 8.5 11 sheet of notes if you want Review
Visual Object Recognition
Perceptual and Sensory Augmented Computing Visual Object Recognition Tutorial Visual Object Recognition Bastian Leibe Computer Vision Laboratory ETH Zurich Chicago, 14.07.2008 & Kristen Grauman Department
An Implementation on Histogram of Oriented Gradients for Human Detection
An Implementation on Histogram of Oriented Gradients for Human Detection Cansın Yıldız Dept. of Computer Engineering Bilkent University Ankara,Turkey cansin@cs.bilkent.edu.tr Abstract I implemented a Histogram
Bias-Variance Trade-off (cont d) + Image Representations
CS 275: Machine Learning Bias-Variance Trade-off (cont d) + Image Representations Prof. Adriana Kovashka University of Pittsburgh January 2, 26 Announcement Homework now due Feb. Generalization Training
Instance-level recognition part 2
Visual Recognition and Machine Learning Summer School Paris 2011 Instance-level recognition part 2 Josef Sivic http://www.di.ens.fr/~josef INRIA, WILLOW, ENS/INRIA/CNRS UMR 8548 Laboratoire d Informatique,
Structured light 3D reconstruction
Structured light 3D reconstruction Reconstruction pipeline and industrial applications rodola@dsi.unive.it 11/05/2010 3D Reconstruction 3D reconstruction is the process of capturing the shape and appearance
Object Detection with Discriminatively Trained Part Based Models
Object Detection with Discriminatively Trained Part Based Models Pedro F. Felzenszwelb, Ross B. Girshick, David McAllester and Deva Ramanan Presented by Fabricio Santolin da Silva Kaustav Basu Some slides
Part based models for recognition. Kristen Grauman
Part based models for recognition Kristen Grauman UT Austin Limitations of window-based models Not all objects are box-shaped Assuming specific 2d view of object Local components themselves do not necessarily
Chaplin, Modern Times, 1936
Chaplin, Modern Times, 1936 [A Bucket of Water and a Glass Matte: Special Effects in Modern Times; bonus feature on The Criterion Collection set] Multi-view geometry problems Structure: Given projections
3D Photography: Stereo
3D Photography: Stereo Marc Pollefeys, Torsten Sattler Spring 2016 http://www.cvg.ethz.ch/teaching/3dvision/ 3D Modeling with Depth Sensors Today s class Obtaining depth maps / range images unstructured
Bundling Features for Large Scale Partial-Duplicate Web Image Search
Bundling Features for Large Scale Partial-Duplicate Web Image Search Zhong Wu, Qifa Ke, Michael Isard, and Jian Sun Microsoft Research Abstract In state-of-the-art image retrieval systems, an image is
Min-Hashing and Geometric min-hashing
Min-Hashing and Geometric min-hashing Ondřej Chum, Michal Perdoch, and Jiří Matas Center for Machine Perception Czech Technical University Prague Outline 1. Looking for representation of images that: is
Human detection using histogram of oriented gradients. Srikumar Ramalingam School of Computing University of Utah
Human detection using histogram of oriented gradients Srikumar Ramalingam School of Computing University of Utah Reference Navneet Dalal and Bill Triggs, Histograms of Oriented Gradients for Human Detection,
Augmented Reality VU. Computer Vision 3D Registration (2) Prof. Vincent Lepetit
Augmented Reality VU Computer Vision 3D Registration (2) Prof. Vincent Lepetit Feature Point-Based 3D Tracking Feature Points for 3D Tracking Much less ambiguous than edges; Point-to-point reprojection
Video Google faces. Josef Sivic, Mark Everingham, Andrew Zisserman. Visual Geometry Group University of Oxford
Video Google faces Josef Sivic, Mark Everingham, Andrew Zisserman Visual Geometry Group University of Oxford The objective Retrieve all shots in a video, e.g. a feature length film, containing a particular
Feature Detection. Raul Queiroz Feitosa. 3/30/2017 Feature Detection 1
Feature Detection Raul Queiroz Feitosa 3/30/2017 Feature Detection 1 Objetive This chapter discusses the correspondence problem and presents approaches to solve it. 3/30/2017 Feature Detection 2 Outline
Local features: detection and description May 12 th, 2015
Local features: detection and description May 12 th, 2015 Yong Jae Lee UC Davis Announcements PS1 grades up on SmartSite PS1 stats: Mean: 83.26 Standard Dev: 28.51 PS2 deadline extended to Saturday, 11:59