Augmented Reality VU. Computer Vision 3D Registration (2) Prof. Vincent Lepetit
|
|
- Edwina Holmes
- 5 years ago
- Views:
Transcription
1 Augmented Reality VU Computer Vision 3D Registration (2) Prof. Vincent Lepetit
2 Feature Point-Based 3D Tracking
3 Feature Points for 3D Tracking Much less ambiguous than edges; Point-to-point reprojection error, not point-to-line. More robust to occlusions than template matching. BUT not for all objects.
4 Feature Point Extraction How to extract the same physical points?
5 Good Features to Track x J. Shi and C. Tomasi. Good features to track. CVPR'94. Defines a "cornerness" measure. Idea: Look for the points x easy to match under a 2D translation
6 Local Feature Detection: The Math x y W (u,v) Consider shifting the window W by (u,v) how do the pixels in W change? compare each pixel before and after by summing up the squared differences
7 Local Feature Detection: The Math x y (u,v) W summing up the squared differences of pixel intensities:
8 Small Motion Assumption Taylor Series expansion of I(x + u, y + v): If the motion (u,v) is small, then the first order approximation is good Plugging this into the formula from the previous slide
9 Local Feature Detection: The Math
10 Local Feature Detection: The Math E(u, v) X (x,y)2w X (x,y)2w X (x,y)2w [uv] apple [I x I y ] apple Ix apple u v apple u [uv] [I I x I y ] y v apple apple I 2 [uv] x I x I y u I x I y Iy 2 v 1 X apple apple I 2 x I x I y A u I x I y v (x,y)2w 2 I 2 y
11 Local Feature Detection: The Math 0 E(u, v) [uv]@ X (x,y)2w apple I 2 x I x I y I x I y I 2 y 1 A apple u v we are looking for (x, y) images locations such that E(u, v) is large for all directions [u, v] How is it related to H?
12 Local Feature Detection: The Math H = X (x,y)2w apple I 2 x I x I y I x I y is a 2 2 symmetric matrix. It can be decomposed into: H =[x + x ] apple where λ + and λ - are the eigenvalues of H; I 2 y x + and x - are the eigenvectors of H. 2 4 x> + x > 3 5
13 Local Feature Detection: The Math apple H =[x + x ] where λ + and λ - are the eigenvalues of H; x + and x - are the eigenvectors of H. E(u, v) [uv] H apple u v x + = direction for (u, v) of largest increase in E λ + = amount of increase in direction x + x - = direction for (u, v) of smallest increase in E λ - = amount of increase in direction x x> + x > x + x - 3 5
14 Harris Cornerness Computation g x g y (g y ) 2 (g x ) 2 Gauss(.) g x g y Gauss(.) (g x ) 2 Gauss(.) (g y ) 2
15 min(λ 1,λ 2 )
16 Feature Point Tracking Two general approaches: KLT Kanade-Lucas-Tomasi tracker: Detection in the first frame then tracking; Detection in every frame and matching.
17 KLT Kanade-Lucas-Tomasi Tracker [Shi & Tomasi CVPR94] Detection (Good Features to Track CVPR 94): Tracking: Make use of the Lucas-Kanade algorithm j ( I t (f(m j ;p i + Δ i )) T(m j )) 2 à Correlation measure: Sum of Square Differences à f: translation model or affine model. Monitoring the templates: Stop tracking when low similarity (Correlation measure > Threshold)
18 Disadvantages of the KLT Tracker Looses all the features after a while: Potential solution: regularly redetect feature points, but can be confused by still be tracked features. There is a better solution.
19 Detection + Matching in Every Frame Detecting in every frame; Matching consecutive frames.
20 Feature Point Matching Possible correlation measures: +di +dj ( ) 2 C = I 1 (x + di,y + dj) I 2 ( x # + di, y # + dj) i= di j= dj 1 Left image Right image 5 Point 1 Point 2 Point 3 Point 4 Point 5 C = +di +dj i= di j= dj ( I 1 (x + di, y + dj) I 1 ) I 2 ( x # + di, y # + dj) I 2 σ 1 σ 2 ( ) x di x' di dj dj y y' I 1 I 2
21 Cross-correlation measure: Feature Point Matching 3 2 C = +di i= di +dj j= dj ( I 1 (x + di,y + dj) I 1 ) I 2 ( x # + di, y # + dj) I 2 σ 1 σ 2 - invariant to affine changes of the lighting; - between -1 (completly different patches) and +1 (equal patches); In practice: accept patches when C > 0.8 ( )
22 Feature Point Matching 2 1. For each point, search for the correspondent that maximizes the correlation. Search limited to a Region of interest centered on the point.
23 Feature Point Matching 2 1. For each point, search for the correspondent that maximizes the correlation. Search limited to a Region of interest. Retain the best correspondent according to the correlation.
24 Feature Point Matching 1. For each point, search for the correspondent that maximizes the correlation. Search limited to a Region of interest. 2. Reverse the role of the images.
25 Feature Point Matching 1. For each point, search for the correspondent that maximizes the correlation. Search limited to a Region of interest. 2. Reverse the role of the images.
26 Feature Point Matching Keep the points that choose each other.
27 Using Interest Points for 3D Tracking: Tracking planes Simon et al., "Pose Estimation for Planar Structures", CGA02. For a plane: H w,t =H t x H t-1 x..h 1 x H w,0 H w,0 H 1 H 2 H t Estimation of the homographies H t from matches.
28 Interest Point-Based Tracking Advantage: Robust to occlusions
29 Reference frame-based tracking [Vacchetti et al PAMI04] Reference frames are images of the object, captured and registered offline.
30 Reference Frame-Based Tracking Method Frame at time t
31 Reference Frame-Based Tracking Method How to match points with points in a (registered) reference frame of the object? Reference frame Frame at time t
32 Wide baseline matching During the tracking we roughly know where the camera is. Frame at time t.
33 Wide baseline matching During the tracking we roughly know where the camera is. We re-render the reference frame from the viewpoint estimated at time t-1. Reference frame «Rerendered reference frame» rendered from the viewpoint estimated at time t-1. Frame at time t.
34 Reference frame Wide baseline matching During the tracking we roughly know where the camera is. We re-render the reference frame from the viewpoint estimated at time t-1. The «re-rendered reference frame» is an intermediate image that can easily be matched with the current frame. «Rerendered reference frame» rendered from the viewpoint estimated at time t-1. Frame at time t.
35 Reference Frame-Based Tracking Method Works but not accurate à the virtual objects jitter:
36 In the reference frames-based tracking method, the successive frames were tracked independently. reference frames time t-1 time t
37 Stable Tracking Method Idea: track interest points on the object over the successive frames and use them to improve the accuracy of the camera registration. reference frames time t-1 time t
38 Stable Tracking Method The tracked points are the projections of 3D points lying on the object surface: à we should also optimize on these points 3D positions. reference frame time t-1 time t
39 The tracked points are the projections of 3D points lying on the object surface: à we should also optimize on these points 3D positions. We also optimize not only on the current camera position but also on the previous ones. The problem becomes: min camera positions upto time t 3D positions of the tracked points Stable Tracking Method reprojection errors of the tracked points + reprojection errors of points matched with reference frames with the constraint that the tracked points lie on the object surface. reference frame time t-1 time t
40 Stable Tracking Method 1. We consider only the current and the previous frames to keep reasonable computation times. 2. The optimization of the tracked points 3D positions under the constraint they lie on the object surface can easily be performed using a transfer function Ψ. (Ying Shan et al. Model-Based Bundle Adjustment with Application to Face Modeling ICCV01): à the 3D positions are not explicitly computed. Object n i Ψ( n i m i ) Error to minimize camera at time t-1 camera at time t
41 Full Method
42 Results
43 Augmented Reality
44 Face Tracking Face assumed to be rigid; Generic 3D model of the face; 1 reference frame built manually on a frontal view of the face; Automatic reinitialisation using a 2D detection.
45 Vision-Based 3D Tracking
46 Recursive Tracking t = 0 t = 1 t = 2...
47 3D Object Detection Keypoint detection (Harris, extrema of Laplacian, affine regions,...); Keypoint recognition (descriptor matching or classification); Robust pose estimation (RANSAC+P3P,...). Registered image(s) of the object to detect Input image
48 Keypoint-Based Object Detection
49 Step 1: Detection invariant to scale and rotation, or perspective transformation
50 Step 2: Patch rectification
51 Step 3: Build description vector
52 Step 4: Match description vectors
53 Feature Detector in SIFT: Invariant to Rotation and Scale
54 Scale-Space Theory Original image Successive convolutions with a Gaussian filter or Gaussian derivative filter while increasing σ [Lindeberg 9*]
55 Laplacian of Gaussian (G xx + G yy ) for feature point detection Laplacian operator
56 Fast Approximation of the Laplacian of Gaussian Convolution with Laplacian of Gaussian is not separable, and therefore slow. However, the Laplacian of Gaussian can be approximated by the difference of two Gaussians: G(σ) G(σ') G(σ) - G(σ')
57 Efficient Scale-Space Detection
58 Resample Blur Subtract Accurate Keypoint localization Keypoint locations: Extrema of Difference-of-Gaussian in scale space: Sub-pixel and sub-scale interpolation: The Taylor expansion around point is: Offset of extremum (use finite differences for derivatives):
59 Results
60 Affine Region Detectors: Invariant to Affine Transformations
61 Harris-Affine & Hessian-Affine Region Detector Harris-Affine: Uses the Auto-correlation matrix as in the classic Harris detector: " 2 % Gauss(.) I x Gauss(.) (I x I y ) M = $ ' $ 2 Gauss(.) (I x I y ) Gauss(.) I ' # y & Local maxima of the smallest eigenvalue indicate the presence of a corner. Hessian-Affine: Considers the Hessian matrix: " H = I xx I xy % $ ' # I xy I yy & Local maxima of determinant or of the smallest eigenvalue indicate the presence of a blob structure.
62 Scale Selection Both the Harris-Affine and the Hessian-Affine use the Laplacian to select the "characteristic" scale: σ 2 Lap(x,σ)
63 Affine Transformation Estimation Warp by Affine Transformation M 1/2, where M is the auto-correlation matrix.
64 Harris-Affine & Hessian-Affine Region Detector Algorithm: 1. Detect initial region with Harris or Hessian detector and select the scale; 2. Estimate the shape with the second moment matrix (=auto-correlation matrix); 3. Normalize the affine region to the circular one; 4. Go to step 2 if the eigenvalues of the second moment matrix for new point are not equal.
65 Maximally Stable Extremal region detector Binary thresholding with thresholds from 0 to 255; Regions that remain unchanged over a large ranges of thresholds are kept.
66 Affine Normalization Warp by M 1 1/2 Warp by M 2 1/2 We still have to correct for the orientation!
67 Select Canonical Orientation Create histogram of local gradient directions computed over the image patch; Each gradient contributes for its norm, weighted by its distance to patch center; Assign canonical orientation at peak of smoothed histogram. 0 2π
68 SIFT Description Vector Made of local histograms of gradients: In practice: 8 orientations x 4 x 4 histograms = 128 dimensions vector.
69 Handling Lighting Changes Gains do not affect gradients; Normalization to unit length removes contrast; Saturation affects magnitudes much more than orientation: magnitudes are thresholded.
70 Standard Approach Step 4: Match description vectors
71 Matching: Approximate Nearest Neighbour Best-Bin-First: Approximate nearest-neighbour search in k-d tree q q
72
SIFT: SCALE INVARIANT FEATURE TRANSFORM SURF: SPEEDED UP ROBUST FEATURES BASHAR ALSADIK EOS DEPT. TOPMAP M13 3D GEOINFORMATION FROM IMAGES 2014
SIFT: SCALE INVARIANT FEATURE TRANSFORM SURF: SPEEDED UP ROBUST FEATURES BASHAR ALSADIK EOS DEPT. TOPMAP M13 3D GEOINFORMATION FROM IMAGES 2014 SIFT SIFT: Scale Invariant Feature Transform; transform image
More informationScale Invariant Feature Transform
Why do we care about matching features? Scale Invariant Feature Transform Camera calibration Stereo Tracking/SFM Image moiaicing Object/activity Recognition Objection representation and recognition Automatic
More informationOutline 7/2/201011/6/
Outline Pattern recognition in computer vision Background on the development of SIFT SIFT algorithm and some of its variations Computational considerations (SURF) Potential improvement Summary 01 2 Pattern
More informationFeature Detection. Raul Queiroz Feitosa. 3/30/2017 Feature Detection 1
Feature Detection Raul Queiroz Feitosa 3/30/2017 Feature Detection 1 Objetive This chapter discusses the correspondence problem and presents approaches to solve it. 3/30/2017 Feature Detection 2 Outline
More informationScale Invariant Feature Transform
Scale Invariant Feature Transform Why do we care about matching features? Camera calibration Stereo Tracking/SFM Image moiaicing Object/activity Recognition Objection representation and recognition Image
More informationVisual Tracking (1) Tracking of Feature Points and Planar Rigid Objects
Intelligent Control Systems Visual Tracking (1) Tracking of Feature Points and Planar Rigid Objects Shingo Kagami Graduate School of Information Sciences, Tohoku University swk(at)ic.is.tohoku.ac.jp http://www.ic.is.tohoku.ac.jp/ja/swk/
More informationFeatures Points. Andrea Torsello DAIS Università Ca Foscari via Torino 155, Mestre (VE)
Features Points Andrea Torsello DAIS Università Ca Foscari via Torino 155, 30172 Mestre (VE) Finding Corners Edge detectors perform poorly at corners. Corners provide repeatable points for matching, so
More informationThe SIFT (Scale Invariant Feature
The SIFT (Scale Invariant Feature Transform) Detector and Descriptor developed by David Lowe University of British Columbia Initial paper ICCV 1999 Newer journal paper IJCV 2004 Review: Matt Brown s Canonical
More informationMotion Estimation and Optical Flow Tracking
Image Matching Image Retrieval Object Recognition Motion Estimation and Optical Flow Tracking Example: Mosiacing (Panorama) M. Brown and D. G. Lowe. Recognising Panoramas. ICCV 2003 Example 3D Reconstruction
More informationLocal invariant features
Local invariant features Tuesday, Oct 28 Kristen Grauman UT-Austin Today Some more Pset 2 results Pset 2 returned, pick up solutions Pset 3 is posted, due 11/11 Local invariant features Detection of interest
More informationFeature Based Registration - Image Alignment
Feature Based Registration - Image Alignment Image Registration Image registration is the process of estimating an optimal transformation between two or more images. Many slides from Alexei Efros http://graphics.cs.cmu.edu/courses/15-463/2007_fall/463.html
More informationComputer Vision for HCI. Topics of This Lecture
Computer Vision for HCI Interest Points Topics of This Lecture Local Invariant Features Motivation Requirements, Invariances Keypoint Localization Features from Accelerated Segment Test (FAST) Harris Shi-Tomasi
More information3D from Photographs: Automatic Matching of Images. Dr Francesco Banterle
3D from Photographs: Automatic Matching of Images Dr Francesco Banterle francesco.banterle@isti.cnr.it 3D from Photographs Automatic Matching of Images Camera Calibration Photographs Surface Reconstruction
More informationSchool of Computing University of Utah
School of Computing University of Utah Presentation Outline 1 2 3 4 Main paper to be discussed David G. Lowe, Distinctive Image Features from Scale-Invariant Keypoints, IJCV, 2004. How to find useful keypoints?
More informationBSB663 Image Processing Pinar Duygulu. Slides are adapted from Selim Aksoy
BSB663 Image Processing Pinar Duygulu Slides are adapted from Selim Aksoy Image matching Image matching is a fundamental aspect of many problems in computer vision. Object or scene recognition Solving
More informationAK Computer Vision Feature Point Detectors and Descriptors
AK Computer Vision Feature Point Detectors and Descriptors 1 Feature Point Detectors and Descriptors: Motivation 2 Step 1: Detect local features should be invariant to scale and rotation, or perspective
More informationCS 4495 Computer Vision A. Bobick. CS 4495 Computer Vision. Features 2 SIFT descriptor. Aaron Bobick School of Interactive Computing
CS 4495 Computer Vision Features 2 SIFT descriptor Aaron Bobick School of Interactive Computing Administrivia PS 3: Out due Oct 6 th. Features recap: Goal is to find corresponding locations in two images.
More informationObject Recognition with Invariant Features
Object Recognition with Invariant Features Definition: Identify objects or scenes and determine their pose and model parameters Applications Industrial automation and inspection Mobile robots, toys, user
More informationLocal Feature Detectors
Local Feature Detectors Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr Slides adapted from Cordelia Schmid and David Lowe, CVPR 2003 Tutorial, Matthew Brown,
More informationObtaining Feature Correspondences
Obtaining Feature Correspondences Neill Campbell May 9, 2008 A state-of-the-art system for finding objects in images has recently been developed by David Lowe. The algorithm is termed the Scale-Invariant
More informationBuilding a Panorama. Matching features. Matching with Features. How do we build a panorama? Computational Photography, 6.882
Matching features Building a Panorama Computational Photography, 6.88 Prof. Bill Freeman April 11, 006 Image and shape descriptors: Harris corner detectors and SIFT features. Suggested readings: Mikolajczyk
More informationLocal features and image matching. Prof. Xin Yang HUST
Local features and image matching Prof. Xin Yang HUST Last time RANSAC for robust geometric transformation estimation Translation, Affine, Homography Image warping Given a 2D transformation T and a source
More informationVisual Tracking (1) Pixel-intensity-based methods
Intelligent Control Systems Visual Tracking (1) Pixel-intensity-based methods Shingo Kagami Graduate School of Information Sciences, Tohoku University swk(at)ic.is.tohoku.ac.jp http://www.ic.is.tohoku.ac.jp/ja/swk/
More informationScott Smith Advanced Image Processing March 15, Speeded-Up Robust Features SURF
Scott Smith Advanced Image Processing March 15, 2011 Speeded-Up Robust Features SURF Overview Why SURF? How SURF works Feature detection Scale Space Rotational invariance Feature vectors SURF vs Sift Assumptions
More informationCorner Detection. GV12/3072 Image Processing.
Corner Detection 1 Last Week 2 Outline Corners and point features Moravec operator Image structure tensor Harris corner detector Sub-pixel accuracy SUSAN FAST Example descriptor: SIFT 3 Point Features
More informationCEE598 - Visual Sensing for Civil Infrastructure Eng. & Mgmt.
CEE598 - Visual Sensing for Civil Infrastructure Eng. & Mgmt. Section 10 - Detectors part II Descriptors Mani Golparvar-Fard Department of Civil and Environmental Engineering 3129D, Newmark Civil Engineering
More informationIntroduction. Introduction. Related Research. SIFT method. SIFT method. Distinctive Image Features from Scale-Invariant. Scale.
Distinctive Image Features from Scale-Invariant Keypoints David G. Lowe presented by, Sudheendra Invariance Intensity Scale Rotation Affine View point Introduction Introduction SIFT (Scale Invariant Feature
More informationFeature Detection and Matching
and Matching CS4243 Computer Vision and Pattern Recognition Leow Wee Kheng Department of Computer Science School of Computing National University of Singapore Leow Wee Kheng (CS4243) Camera Models 1 /
More informationAutonomous Navigation for Flying Robots
Computer Vision Group Prof. Daniel Cremers Autonomous Navigation for Flying Robots Lecture 7.1: 2D Motion Estimation in Images Jürgen Sturm Technische Universität München 3D to 2D Perspective Projections
More informationLocal Image Features
Local Image Features Computer Vision CS 143, Brown Read Szeliski 4.1 James Hays Acknowledgment: Many slides from Derek Hoiem and Grauman&Leibe 2008 AAAI Tutorial This section: correspondence and alignment
More informationProf. Feng Liu. Spring /26/2017
Prof. Feng Liu Spring 2017 http://www.cs.pdx.edu/~fliu/courses/cs510/ 04/26/2017 Last Time Re-lighting HDR 2 Today Panorama Overview Feature detection Mid-term project presentation Not real mid-term 6
More informationHarder case. Image matching. Even harder case. Harder still? by Diva Sian. by swashford
Image matching Harder case by Diva Sian by Diva Sian by scgbt by swashford Even harder case Harder still? How the Afghan Girl was Identified by Her Iris Patterns Read the story NASA Mars Rover images Answer
More informationImage processing and features
Image processing and features Gabriele Bleser gabriele.bleser@dfki.de Thanks to Harald Wuest, Folker Wientapper and Marc Pollefeys Introduction Previous lectures: geometry Pose estimation Epipolar geometry
More informationMotion illusion, rotating snakes
Motion illusion, rotating snakes Local features: main components 1) Detection: Find a set of distinctive key points. 2) Description: Extract feature descriptor around each interest point as vector. x 1
More informationSUMMARY: DISTINCTIVE IMAGE FEATURES FROM SCALE- INVARIANT KEYPOINTS
SUMMARY: DISTINCTIVE IMAGE FEATURES FROM SCALE- INVARIANT KEYPOINTS Cognitive Robotics Original: David G. Lowe, 004 Summary: Coen van Leeuwen, s1460919 Abstract: This article presents a method to extract
More informationImage matching. Announcements. Harder case. Even harder case. Project 1 Out today Help session at the end of class. by Diva Sian.
Announcements Project 1 Out today Help session at the end of class Image matching by Diva Sian by swashford Harder case Even harder case How the Afghan Girl was Identified by Her Iris Patterns Read the
More informationLocal Features: Detection, Description & Matching
Local Features: Detection, Description & Matching Lecture 08 Computer Vision Material Citations Dr George Stockman Professor Emeritus, Michigan State University Dr David Lowe Professor, University of British
More informationHarder case. Image matching. Even harder case. Harder still? by Diva Sian. by swashford
Image matching Harder case by Diva Sian by Diva Sian by scgbt by swashford Even harder case Harder still? How the Afghan Girl was Identified by Her Iris Patterns Read the story NASA Mars Rover images Answer
More informationVisual Tracking (1) Feature Point Tracking and Block Matching
Intelligent Control Systems Visual Tracking (1) Feature Point Tracking and Block Matching Shingo Kagami Graduate School of Information Sciences, Tohoku University swk(at)ic.is.tohoku.ac.jp http://www.ic.is.tohoku.ac.jp/ja/swk/
More informationImage Features. Work on project 1. All is Vanity, by C. Allan Gilbert,
Image Features Work on project 1 All is Vanity, by C. Allan Gilbert, 1873-1929 Feature extrac*on: Corners and blobs c Mo*va*on: Automa*c panoramas Credit: Ma9 Brown Why extract features? Mo*va*on: panorama
More informationEdge and corner detection
Edge and corner detection Prof. Stricker Doz. G. Bleser Computer Vision: Object and People Tracking Goals Where is the information in an image? How is an object characterized? How can I find measurements
More informationFeature descriptors. Alain Pagani Prof. Didier Stricker. Computer Vision: Object and People Tracking
Feature descriptors Alain Pagani Prof. Didier Stricker Computer Vision: Object and People Tracking 1 Overview Previous lectures: Feature extraction Today: Gradiant/edge Points (Kanade-Tomasi + Harris)
More informationSCALE INVARIANT FEATURE TRANSFORM (SIFT)
1 SCALE INVARIANT FEATURE TRANSFORM (SIFT) OUTLINE SIFT Background SIFT Extraction Application in Content Based Image Search Conclusion 2 SIFT BACKGROUND Scale-invariant feature transform SIFT: to detect
More informationFeature Tracking and Optical Flow
Feature Tracking and Optical Flow Prof. D. Stricker Doz. G. Bleser Many slides adapted from James Hays, Derek Hoeim, Lana Lazebnik, Silvio Saverse, who 1 in turn adapted slides from Steve Seitz, Rick Szeliski,
More informationComparison of Feature Detection and Matching Approaches: SIFT and SURF
GRD Journals- Global Research and Development Journal for Engineering Volume 2 Issue 4 March 2017 ISSN: 2455-5703 Comparison of Detection and Matching Approaches: SIFT and SURF Darshana Mistry PhD student
More informationScale Invariant Feature Transform by David Lowe
Scale Invariant Feature Transform by David Lowe Presented by: Jerry Chen Achal Dave Vaishaal Shankar Some slides from Jason Clemons Motivation Image Matching Correspondence Problem Desirable Feature Characteristics
More informationEECS150 - Digital Design Lecture 14 FIFO 2 and SIFT. Recap and Outline
EECS150 - Digital Design Lecture 14 FIFO 2 and SIFT Oct. 15, 2013 Prof. Ronald Fearing Electrical Engineering and Computer Sciences University of California, Berkeley (slides courtesy of Prof. John Wawrzynek)
More informationMulti-modal Registration of Visual Data. Massimiliano Corsini Visual Computing Lab, ISTI - CNR - Italy
Multi-modal Registration of Visual Data Massimiliano Corsini Visual Computing Lab, ISTI - CNR - Italy Overview Introduction and Background Features Detection and Description (2D case) Features Detection
More informationCS 556: Computer Vision. Lecture 3
CS 556: Computer Vision Lecture 3 Prof. Sinisa Todorovic sinisa@eecs.oregonstate.edu Interest Points Harris corners Hessian points SIFT Difference-of-Gaussians SURF 2 Properties of Interest Points Locality
More informationCS5670: Computer Vision
CS5670: Computer Vision Noah Snavely Lecture 4: Harris corner detection Szeliski: 4.1 Reading Announcements Project 1 (Hybrid Images) code due next Wednesday, Feb 14, by 11:59pm Artifacts due Friday, Feb
More informationSIFT - scale-invariant feature transform Konrad Schindler
SIFT - scale-invariant feature transform Konrad Schindler Institute of Geodesy and Photogrammetry Invariant interest points Goal match points between images with very different scale, orientation, projective
More informationLocal features: detection and description May 12 th, 2015
Local features: detection and description May 12 th, 2015 Yong Jae Lee UC Davis Announcements PS1 grades up on SmartSite PS1 stats: Mean: 83.26 Standard Dev: 28.51 PS2 deadline extended to Saturday, 11:59
More informationKeypoint detection. (image registration, panorama stitching, motion estimation + tracking, recognition )
Keypoint detection n n Many applications benefit from features localized in (x,y) (image registration, panorama stitching, motion estimation + tracking, recognition ) Edges well localized only in one direction
More informationCAP 5415 Computer Vision Fall 2012
CAP 5415 Computer Vision Fall 01 Dr. Mubarak Shah Univ. of Central Florida Office 47-F HEC Lecture-5 SIFT: David Lowe, UBC SIFT - Key Point Extraction Stands for scale invariant feature transform Patented
More informationKey properties of local features
Key properties of local features Locality, robust against occlusions Must be highly distinctive, a good feature should allow for correct object identification with low probability of mismatch Easy to etract
More informationCOMPUTER VISION > OPTICAL FLOW UTRECHT UNIVERSITY RONALD POPPE
COMPUTER VISION 2017-2018 > OPTICAL FLOW UTRECHT UNIVERSITY RONALD POPPE OUTLINE Optical flow Lucas-Kanade Horn-Schunck Applications of optical flow Optical flow tracking Histograms of oriented flow Assignment
More informationRequirements for region detection
Region detectors Requirements for region detection For region detection invariance transformations that should be considered are illumination changes, translation, rotation, scale and full affine transform
More informationProblems with template matching
Problems with template matching The template represents the object as we expect to find it in the image The object can indeed be scaled or rotated This technique requires a separate template for each scale
More informationComputer Vision I. Announcement. Corners. Edges. Numerical Derivatives f(x) Edge and Corner Detection. CSE252A Lecture 11
Announcement Edge and Corner Detection Slides are posted HW due Friday CSE5A Lecture 11 Edges Corners Edge is Where Change Occurs: 1-D Change is measured by derivative in 1D Numerical Derivatives f(x)
More informationCS 558: Computer Vision 4 th Set of Notes
1 CS 558: Computer Vision 4 th Set of Notes Instructor: Philippos Mordohai Webpage: www.cs.stevens.edu/~mordohai E-mail: Philippos.Mordohai@stevens.edu Office: Lieb 215 Overview Keypoint matching Hessian
More information2D Image Processing Feature Descriptors
2D Image Processing Feature Descriptors Prof. Didier Stricker Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche Intelligenz http://av.dfki.de 1 Overview
More informationEE368 Project Report CD Cover Recognition Using Modified SIFT Algorithm
EE368 Project Report CD Cover Recognition Using Modified SIFT Algorithm Group 1: Mina A. Makar Stanford University mamakar@stanford.edu Abstract In this report, we investigate the application of the Scale-Invariant
More informationLocal features: detection and description. Local invariant features
Local features: detection and description Local invariant features Detection of interest points Harris corner detection Scale invariant blob detection: LoG Description of local patches SIFT : Histograms
More information3D Vision. Viktor Larsson. Spring 2019
3D Vision Viktor Larsson Spring 2019 Schedule Feb 18 Feb 25 Mar 4 Mar 11 Mar 18 Mar 25 Apr 1 Apr 8 Apr 15 Apr 22 Apr 29 May 6 May 13 May 20 May 27 Introduction Geometry, Camera Model, Calibration Features,
More informationApplication questions. Theoretical questions
The oral exam will last 30 minutes and will consist of one application question followed by two theoretical questions. Please find below a non exhaustive list of possible application questions. The list
More informationStitching and Blending
Stitching and Blending Kari Pulli VP Computational Imaging Light First project Build your own (basic) programs panorama HDR (really, exposure fusion) The key components register images so their features
More informationEppur si muove ( And yet it moves )
Eppur si muove ( And yet it moves ) - Galileo Galilei University of Texas at Arlington Tracking of Image Features CSE 4392-5369 Vision-based Robot Sensing, Localization and Control Dr. Gian Luca Mariottini,
More informationLocal Image Features
Local Image Features Computer Vision Read Szeliski 4.1 James Hays Acknowledgment: Many slides from Derek Hoiem and Grauman&Leibe 2008 AAAI Tutorial Flashed Face Distortion 2nd Place in the 8th Annual Best
More informationAnno accademico 2006/2007. Davide Migliore
Robotica Anno accademico 6/7 Davide Migliore migliore@elet.polimi.it Today What is a feature? Some useful information The world of features: Detectors Edges detection Corners/Points detection Descriptors?!?!?
More informationAutomatic Image Alignment (feature-based)
Automatic Image Alignment (feature-based) Mike Nese with a lot of slides stolen from Steve Seitz and Rick Szeliski 15-463: Computational Photography Alexei Efros, CMU, Fall 2006 Today s lecture Feature
More informationFeature descriptors and matching
Feature descriptors and matching Detections at multiple scales Invariance of MOPS Intensity Scale Rotation Color and Lighting Out-of-plane rotation Out-of-plane rotation Better representation than color:
More informationMidterm Wed. Local features: detection and description. Today. Last time. Local features: main components. Goal: interest operator repeatability
Midterm Wed. Local features: detection and description Monday March 7 Prof. UT Austin Covers material up until 3/1 Solutions to practice eam handed out today Bring a 8.5 11 sheet of notes if you want Review
More informationImplementing the Scale Invariant Feature Transform(SIFT) Method
Implementing the Scale Invariant Feature Transform(SIFT) Method YU MENG and Dr. Bernard Tiddeman(supervisor) Department of Computer Science University of St. Andrews yumeng@dcs.st-and.ac.uk Abstract The
More informationMatching. Compare region of image to region of image. Today, simplest kind of matching. Intensities similar.
Matching Compare region of image to region of image. We talked about this for stereo. Important for motion. Epipolar constraint unknown. But motion small. Recognition Find object in image. Recognize object.
More informationCS231A Section 6: Problem Set 3
CS231A Section 6: Problem Set 3 Kevin Wong Review 6 -! 1 11/09/2012 Announcements PS3 Due 2:15pm Tuesday, Nov 13 Extra Office Hours: Friday 6 8pm Huang Common Area, Basement Level. Review 6 -! 2 Topics
More informationECE Digital Image Processing and Introduction to Computer Vision
ECE592-064 Digital Image Processing and Introduction to Computer Vision Depart. of ECE, NC State University Instructor: Tianfu (Matt) Wu Spring 2017 Recap, SIFT Motion Tracking Change Detection Feature
More informationCS 556: Computer Vision. Lecture 3
CS 556: Computer Vision Lecture 3 Prof. Sinisa Todorovic sinisa@eecs.oregonstate.edu 1 Outline Matlab Image features -- Interest points Point descriptors Homework 1 2 Basic MATLAB Commands 3 Basic MATLAB
More informationAdvanced Video Content Analysis and Video Compression (5LSH0), Module 4
Advanced Video Content Analysis and Video Compression (5LSH0), Module 4 Visual feature extraction Part I: Color and texture analysis Sveta Zinger Video Coding and Architectures Research group, TU/e ( s.zinger@tue.nl
More informationCapturing, Modeling, Rendering 3D Structures
Computer Vision Approach Capturing, Modeling, Rendering 3D Structures Calculate pixel correspondences and extract geometry Not robust Difficult to acquire illumination effects, e.g. specular highlights
More informationRobotics Programming Laboratory
Chair of Software Engineering Robotics Programming Laboratory Bertrand Meyer Jiwon Shin Lecture 8: Robot Perception Perception http://pascallin.ecs.soton.ac.uk/challenges/voc/databases.html#caltech car
More informationFeature Tracking and Optical Flow
Feature Tracking and Optical Flow Prof. D. Stricker Doz. G. Bleser Many slides adapted from James Hays, Derek Hoeim, Lana Lazebnik, Silvio Saverse, who in turn adapted slides from Steve Seitz, Rick Szeliski,
More informationLine, edge, blob and corner detection
Line, edge, blob and corner detection Dmitri Melnikov MTAT.03.260 Pattern Recognition and Image Analysis April 5, 2011 1 / 33 Outline 1 Introduction 2 Line detection 3 Edge detection 4 Blob detection 5
More informationLeow Wee Kheng CS4243 Computer Vision and Pattern Recognition. Motion Tracking. CS4243 Motion Tracking 1
Leow Wee Kheng CS4243 Computer Vision and Pattern Recognition Motion Tracking CS4243 Motion Tracking 1 Changes are everywhere! CS4243 Motion Tracking 2 Illumination change CS4243 Motion Tracking 3 Shape
More informationComputer Vision II Lecture 4
Computer Vision II Lecture 4 Color based Tracking 29.04.2014 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Course Outline Single-Object Tracking Background modeling
More informationSIFT: Scale Invariant Feature Transform
1 / 25 SIFT: Scale Invariant Feature Transform Ahmed Othman Systems Design Department University of Waterloo, Canada October, 23, 2012 2 / 25 1 SIFT Introduction Scale-space extrema detection Keypoint
More informationWikipedia - Mysid
Wikipedia - Mysid Erik Brynjolfsson, MIT Filtering Edges Corners Feature points Also called interest points, key points, etc. Often described as local features. Szeliski 4.1 Slides from Rick Szeliski,
More informationLecture 10 Detectors and descriptors
Lecture 10 Detectors and descriptors Properties of detectors Edge detectors Harris DoG Properties of detectors SIFT Shape context Silvio Savarese Lecture 10-26-Feb-14 From the 3D to 2D & vice versa P =
More informationCS664 Lecture #21: SIFT, object recognition, dynamic programming
CS664 Lecture #21: SIFT, object recognition, dynamic programming Some material taken from: Sebastian Thrun, Stanford http://cs223b.stanford.edu/ Yuri Boykov, Western Ontario David Lowe, UBC http://www.cs.ubc.ca/~lowe/keypoints/
More informationEE795: Computer Vision and Intelligent Systems
EE795: Computer Vision and Intelligent Systems Spring 2012 TTh 17:30-18:45 FDH 204 Lecture 09 130219 http://www.ee.unlv.edu/~b1morris/ecg795/ 2 Outline Review Feature Descriptors Feature Matching Feature
More informationIMAGE MATCHING USING SCALE INVARIANT FEATURE TRANSFORM (SIFT) Naotoshi Seo and David A. Schug
IMAGE MATCHING USING SCALE INVARIANT FEATURE TRANSFORM (SIFT) Naotoshi Seo and David A. Schug University of Maryland ENEE631 Digital Image and Video Processing Final Project Professor: Min Wu ABSTRACT
More informationA Comparison of SIFT and SURF
A Comparison of SIFT and SURF P M Panchal 1, S R Panchal 2, S K Shah 3 PG Student, Department of Electronics & Communication Engineering, SVIT, Vasad-388306, India 1 Research Scholar, Department of Electronics
More informationVisual Tracking. Image Processing Laboratory Dipartimento di Matematica e Informatica Università degli studi di Catania.
Image Processing Laboratory Dipartimento di Matematica e Informatica Università degli studi di Catania 1 What is visual tracking? estimation of the target location over time 2 applications Six main areas:
More informationKanade Lucas Tomasi Tracking (KLT tracker)
Kanade Lucas Tomasi Tracking (KLT tracker) Tomáš Svoboda, svoboda@cmp.felk.cvut.cz Czech Technical University in Prague, Center for Machine Perception http://cmp.felk.cvut.cz Last update: November 26,
More informationComputer Vision I. Announcements. Fourier Tansform. Efficient Implementation. Edge and Corner Detection. CSE252A Lecture 13.
Announcements Edge and Corner Detection HW3 assigned CSE252A Lecture 13 Efficient Implementation Both, the Box filter and the Gaussian filter are separable: First convolve each row of input image I with
More informationImage features. Image Features
Image features Image features, such as edges and interest points, provide rich information on the image content. They correspond to local regions in the image and are fundamental in many applications in
More informationCS201: Computer Vision Introduction to Tracking
CS201: Computer Vision Introduction to Tracking John Magee 18 November 2014 Slides courtesy of: Diane H. Theriault Question of the Day How can we represent and use motion in images? 1 What is Motion? Change
More informationCoarse-to-fine image registration
Today we will look at a few important topics in scale space in computer vision, in particular, coarseto-fine approaches, and the SIFT feature descriptor. I will present only the main ideas here to give
More informationLocal Features Tutorial: Nov. 8, 04
Local Features Tutorial: Nov. 8, 04 Local Features Tutorial References: Matlab SIFT tutorial (from course webpage) Lowe, David G. Distinctive Image Features from Scale Invariant Features, International
More informationImplementation and Comparison of Feature Detection Methods in Image Mosaicing
IOSR Journal of Electronics and Communication Engineering (IOSR-JECE) e-issn: 2278-2834,p-ISSN: 2278-8735 PP 07-11 www.iosrjournals.org Implementation and Comparison of Feature Detection Methods in Image
More information3D Photography. Marc Pollefeys, Torsten Sattler. Spring 2015
3D Photography Marc Pollefeys, Torsten Sattler Spring 2015 Schedule (tentative) Feb 16 Feb 23 Mar 2 Mar 9 Mar 16 Mar 23 Mar 30 Apr 6 Apr 13 Apr 20 Apr 27 May 4 May 11 May 18 May 25 Introduction Geometry,
More information