Ulas Bagci

Size: px
Start display at page:

Download "Ulas Bagci"

Transcription

1 CAP5415- Computer Vision Lecture 5 and 6- Finding Features, Affine Invariance, SIFT Ulas Bagci bagci@ucf.edu 1

2 Outline Concept of Scale Pyramids Scale- space approaches briefly Scale invariant region selecqon SIFT: an image region descriptor David Lowe, IJCV 2004 paper [Please read it!] 2

3 Reminder: MoQvaQon Image Matching Fundamental aspect of many problems Object RecogniQon 3D Structures Stereo Correspondence MoQon Tracking. 3

4 Reminder: MoQvaQon Image Matching Fundamental aspect of many problems Object RecogniQon 3D Structures Stereo Correspondence MoQon Tracking. What are the desired features to conduct these tasks? 4

5 Review of the Corner DetecQon Corners are invariant to translaqon rotaqon scaling 5

6 SCALE The extrema in a signal and its first a few derivaqves provide a useful general purpose descripqon for many kinds of signals Edges Corners 6

7 SCALE The extrema in a signal and its first a few derivaqves provide a useful general purpose descripqon for many kinds of signals Edges Corners In physics, objects live on a range of scales. 7

8 SCALE The extrema in a signal and its first a few derivaqves provide a useful general purpose descripqon for many kinds of signals Edges Corners In physics, objects live on a range of scales. Neighborhood operaqons can only extract local features (at a scale of at most a few pixels), however, images contain informaqon at larger scales 8

9 Scale- Space Any given image can contain objects that exist at scales different from other objects in the same image 9

10 Scale- Space Any given image can contain objects that exist at scales different from other objects in the same image Or, that even exist at mulqple scales simultaneously 10

11 Scale- Space Any given image can contain objects that exist at scales different from other objects in the same image Or, that even exist at mulqple scales simultaneously 11

12 MulQ- scale RepresentaQon Image Small Scale Features Large Scale Features Smaller Filter Masks Larger Filter Masks 12

13 MulQ- scale RepresentaQon Image Small Scale Features Large Scale Features Smaller Filter Masks Larger Filter Masks ComputaQonal Cost increases! Doubling the scale leads to four- fold increase in the number of operaqons in 2D. 13

14 MulQ- Scale RepresentaQon: Pyramid Pyramid is one way to represent images in mulq- scale Pyramid is built by using mulqple copies of image. Each level in the pyramid is 1/4 of the size of previous level. The lowest level is of the highest resoluqon. The highest level is of the lowest resoluqon. 14

15 MulQ- Scale RepresentaQon: Pyramid 15

16 Pyramid can capture global and local features 16

17 Gaussian Pyramids 17 Source: Forsyth

18 Laplacian Pyramids 18 Source: Forsyth

19 Laplacian Pyramids Laplacian of Gaussian (LoG): Gaussian first to smooth images then perform Laplacian operaqon 5 2 (G I) =I 5 2 G 19

20 Laplacian Pyramids 20

21 Laplacian Pyramids G x (x, y) = x 2 4 e (x 2 +y 2 )/(2 2 ) 21

22 Laplacian Pyramids G x (x, y) = x 2 4 e (x 2 +y 2 )/(2 2 ) Repeat same derivaqve for y component, and then take second derivaqves. 5 2 G (x, y) = (x 2 + y ) 2 e (x2 +y 2 )/(2 2 ) 22

23 Laplacian Pyramid ConstrucQon 23

24 Laplacian Pyramid ConstrucQon 24

25 DecimaQon and InterpolaQon Lowpass filter (i.e., Gaussian) 25

26 DecimaQon and InterpolaQon Lowpass filter (i.e., Gaussian) y(n) =x(n) h(n) = X k h(k)x(n k) 26

27 DecimaQon and InterpolaQon Lowpass filter (i.e., Gaussian) y(n) =x(n) h(n) = X k h(k)x(n k) z(n) =y(2n) 27

28 DecimaQon and InterpolaQon Lowpass filter (i.e., Gaussian) y(n) =x(n) h(n) = X k h(k)x(n k) z(n) =y(2n) z(n) = X k h(k)x(2n k) 28

29 Difference of Gaussian (DoG) It is a common approximaqon of LoG (bejer run Qme). 29

30 Difference of Gaussian (DoG) It is a common approximaqon of LoG (bejer run Qme). D, (x, y) =L(x, y, ) L(x, y, ) 30

31 Difference of Gaussian (DoG) It is a common approximaqon of LoG (bejer run Qme). D, (x, y) =L(x, y, ) L(x, y, ) It is the difference between a blurred copy of image I and an even more blurred copy of I. 31

32 Difference of Gaussian (DoG) It is a common approximaqon of LoG (bejer run Qme). D, (x, y) =L(x, y, ) L(x, y, ) It is the difference between a blurred copy of image I and an even more blurred copy of I. 5G (x, y) G (x, y) G (x, y) ( 1) 2 32

33 Scale SelecQon- Automated Find scale that gives local maxima of some function f in both position and scale. f Image 1 f Image 2 K. Grauman, s 1 region size s 2 33 region size

34 Good FuncQon? A good function for scale detection: has one stable sharp peak f bad f bad f Good! region size region size region size For usual images: a good funcqon would be a one which responds to contrast (sharp local intensity change) 34

35 Patch Size Corresponding to Scale f ( Ii σ 1 im i1 im ( x, σ )) = f ( I ( x, )) How to find corresponding patch sizes? 35

36 Lecture 5&6 Finding Features, Affine Invariance, SIFT Patch Size Corresponding to Scale f ( I i1 im ( x, σ )) K. Grauman, B. Leibe f ( I i1 im ( x, σ )) 36

37 Lecture 5&6 Finding Features, Affine Invariance, SIFT Patch Size Corresponding to Scale f ( I i1 im ( x, σ )) K. Grauman, B. Leibe f ( I i1 im ( x, σ )) 37

38 Lecture 5&6 Finding Features, Affine Invariance, SIFT Patch Size Corresponding to Scale f ( I i1 im ( x, σ )) K. Grauman, B. Leibe f ( I i1 im ( x, σ )) 38

39 Patch Size Corresponding to Scale f ( I (, )) i im 1 x i i σ m f ( I 1 ( x, σ )) K. Grauman, B. Leibe 39

40 Lecture 5&6 Finding Features, Affine Invariance, SIFT Patch Size Corresponding to Scale f ( I i1 im ( x, σ )) K. Grauman, B. Leibe f ( I i1 im ( x, σ )) 40

41 Lecture 5&6 Finding Features, Affine Invariance, SIFT Patch Size Corresponding to Scale FuncQon responses for increasing scale (scale signature) f ( I i1 im ( x, σ )) K. Grauman, B. Leibe f ( I i1 im ( x, σ )) 41

42 NormalizaQon 42

43 Scale Invariant Feature Transform (SIFT) Lowe., D. 2004, IJCV cited > 25K 43

44 Scale Invariant Feature Transform (SIFT) Shows existence, importance, and value of invariant types of detector Demonstrates the richness that feature descriptors can bring to feature matching Instead of using LoG (Laplacian of Gaussian) like in Harris and Hessian- based operators, SIFT uses DoG (Difference of Gaussian). 44

45 Scale Invariant Feature Transform (SIFT) Image content is transformed into local feature coordinates that are invariant to translation, rotation, scale, and other imaging parameters 45

46 Scale Invariant Feature Transform (SIFT) Image content is transformed into local feature coordinates that are invariant to translation, rotation, scale, and other imaging parameters 46

47 Overall Procedure at a High Level Scale- Space Extrema DetecQon Keypoint LocalizaQon Search over mulqple scales and image locaqons Fit a model to determine locaqon and scale. Select Keypoints based on a measure of stability. OrientaQon Assignment Compute best orientaqon(s) for each keypoint region. Keypoint DescripQon Use local image gradients at selected scale and rotaqon to describe each keypoint region. 47

48 Basic idea: SIFT Descriptor Take n x n (i.e., n=16) square window (around a feature/interest point) Divide them into m x m (i.e, m=4) cells Compute gradient orientation for each cell Create histogram over edge orientations weighted by magnitude 0 2 angle histogram 48

49 SIFT Descriptor 16 histograms x 8 orientaqons = 128 features 49

50 SIFT Descriptor Full version Divide the 16x16 window into a 4x4 grid of cells (2x2 case shown below) Compute an orientation histogram for each cell 16 cells * 8 orientations = 128 dimensional descriptor Threshold normalize the descriptor: such that:

51 ProperQes of SIFT Extraordinarily robust matching technique Can handle changes in viewpoint Can handle significant changes in illuminaqon Efficient (real Qme) Lots of code available hjp://people.csail.mit.edu/albert/ladypack/wiki/index.php/known_implementaqons_of_sift 51

52 Revisit SIFT Steps (1) Scale-space extrema detection Extract scale and rotation invariant interest points (i.e., keypoints). (2) Keypoint localization Determine location and scale for each interest point. Eliminate weak keypoints (3) Orientation assignment Assign one or more orientations to each keypoint. (4) Keypoint descriptor Use local image gradients at the selected scale. 52

53 (1) Scale- Space Extrema DetecQon We want to find points that give us informaqon about the objects in the image. The informaqon about the objects is in the object s edges. We will represent the image in a way that gives us these edges as this representaqons extrema points. 53

54 (1) Scale- Space Extrema DetecQon Harris-Laplace scale Find local maxima of: Harris detector in space LoG in scale y Harris x LoG SIFT scale Find local maxima of: Hessian in space DoG in scale y DoG Hessian x 54

55 (1) Scale- Space Extrema DetecQon 55

56 (1) Scale- Space Extrema DetecQon ( 2 σ ) D k D( kσ ) Extract local extrema (i.e., minima or maxima) in DoG pyramid. - Compare each point to its 8 neighbors at the same level, 9 neighbors in the level above, and 9 neighbors in the level below (i.e., 26 total). D( σ ) 56

57 (1) Scale- Space Extrema DetecQon Laplacian of Gaussian kernel Scale normalized (x by scale 2 ) Proposed by Lindeberg Scale-space detection Find local maxima across scale/space A good blob detector 57

58 (2) Keypoint LocalizaQon There are still a lot of points, some of them are not good enough. The locations of keypoints may be not accurate. Eliminating edge points. 58

59 (2) Keypoint LocalizaQon Reject (1) points with low contrast (flat) (2) poorly localized along an edge (edge)

60 (2) Keypoint LocalizaQon Reject (1) points with low contrast (flat) (2) poorly localized along an edge (edge) if Tr(H) H < (r + 1)2 r Reject Where r = 1 2 and pracqcally SIFT uses r=10.

61 Inaccurate Keypoint LocalizaQon Poor contrast True Extrema Detected Extrema Sampling x 61

62 Inaccurate Keypoint LocalizaQon The SoluQon: Taylor expansion:! D x D! x T! D! 2 x ( ) T = D + x + x x Minimize to find accurate extrema: 2 D 1 D x ˆ =! 2! x x If offset from sampling point is larger than Keypoint should be in a different sampling point.! T! Brown & Lowe

63 Local extremas 63

64 Local extremas Remove low contrast features 64

65 Local extremas Remove low edges 65

66 SIFT Descriptor 66

67 (3) OrientaQon Assignment Create histogram of gradient directions, within a region around the keypoint, at selected scale: L(x, y,σ ) = G(x, y,σ )* I(x, y) m(x, y) = (L(x +1, y) L(x 1, y)) 2 + (L(x, y +1) L(x, y 1)) 2 θ(x, y) = a tan2((l(x, y +1) L(x, y 1)) / (L(x +1, y) L(x 1, y))) 36 bins (i.e., 10 o per bin) 0 2 pi Histogram entries are weighted by (i) gradient magnitude and (ii) a Gaussian funcqon with σ equal to 1.5 Qmes the scale of the keypoint. 67

68 (3) OrientaQon Assignment 68

69 (3) OrientaQon Assignment 69

70 (3) OrientaQon Assignment 70

71 (3) OrientaQon Assignment 71

72 (3) OrientaQon Assignment 72

73 (4) Keypoint Descriptor Partial Voting: distribute histogram entries into adjacent bins (i.e., additional robustness to shifts) Each entry is added to all bins, multiplied by a weight of 1-d, where d is the distance from the bin it belongs. 73

74 (4) Keypoint Descriptor Descriptor depends on two main parameters: (1) number of orientaqons n x n array of orientaqon histograms 74

75 EvaluaQng Results How can we measure the performance of a feature matcher? 1 the matcher correctly found a match # true positives matched # true positives 0.7 true positive rate features that really do have a match false positive rate 1 # false positives matched # true negatives the matcher said yes when the right answer was no 75

76 EvaluaQng Results How can we measure the performance of a feature matcher? 1 ROC curve ( Receiver Operator Characteristic ) the matcher correctly found a match # true positives matched # true positives 0.7 true positive rate features that really do have a match false positive rate 1 # false positives matched # true negatives the matcher said yes when the right answer was no 76

77 Change of IlluminaQon Change of brightness => doesn t effect gradients (difference of pixels value). Change of contrast => doesn t effect gradients (up to normalization). Saturation (non- linear change of illumination) => affects magnitudes much more than orientation. => Threshold gradient magnitudes to 0.2 and renormalize. 77

78 IlluminaQon invariance? 1. Before you start, you can normalize the images. 2. Difference based metrics can be used (Haar, SIFT, ) 78

79 79

80 Extract features 80

81 Extract features Compute putamve matches 81

82 Extract features Compute putamve matches Loop: Hypothesize transformaqon T (small group of putaqve matches that are related by T) 82

83 Extract features Compute putamve matches Loop: Hypothesize transformaqon T (small group of putaqve matches that are related by T) Verify transformaqon (search for other matches consistent with T) 83

84 Extract features Compute putamve matches Loop: Hypothesize transformaqon T (small group of putaqve matches that are related by T) Verify transformaqon (search for other matches consistent with T) 84

85 For training images: Object RecogniQon ExtracQng keypoints by SIFT. CreaQng descriptors database. For query images: ExtracQng keypoints by SIFT. For each descriptor - finding nearest neighbor in DB. Finding cluster of at- least 3 keypoints. Performing detailed geometric fit check for each cluster. 85

86 RecogniQon/Matching 86

87 Image RegistraQon 87

Local Features: Detection, Description & Matching

Local Features: Detection, Description & Matching Local Features: Detection, Description & Matching Lecture 08 Computer Vision Material Citations Dr George Stockman Professor Emeritus, Michigan State University Dr David Lowe Professor, University of British

More information

CAP 5415 Computer Vision Fall 2012

CAP 5415 Computer Vision Fall 2012 CAP 5415 Computer Vision Fall 01 Dr. Mubarak Shah Univ. of Central Florida Office 47-F HEC Lecture-5 SIFT: David Lowe, UBC SIFT - Key Point Extraction Stands for scale invariant feature transform Patented

More information

SIFT: SCALE INVARIANT FEATURE TRANSFORM SURF: SPEEDED UP ROBUST FEATURES BASHAR ALSADIK EOS DEPT. TOPMAP M13 3D GEOINFORMATION FROM IMAGES 2014

SIFT: SCALE INVARIANT FEATURE TRANSFORM SURF: SPEEDED UP ROBUST FEATURES BASHAR ALSADIK EOS DEPT. TOPMAP M13 3D GEOINFORMATION FROM IMAGES 2014 SIFT: SCALE INVARIANT FEATURE TRANSFORM SURF: SPEEDED UP ROBUST FEATURES BASHAR ALSADIK EOS DEPT. TOPMAP M13 3D GEOINFORMATION FROM IMAGES 2014 SIFT SIFT: Scale Invariant Feature Transform; transform image

More information

Scale Invariant Feature Transform

Scale Invariant Feature Transform Why do we care about matching features? Scale Invariant Feature Transform Camera calibration Stereo Tracking/SFM Image moiaicing Object/activity Recognition Objection representation and recognition Automatic

More information

Scale Invariant Feature Transform

Scale Invariant Feature Transform Scale Invariant Feature Transform Why do we care about matching features? Camera calibration Stereo Tracking/SFM Image moiaicing Object/activity Recognition Objection representation and recognition Image

More information

School of Computing University of Utah

School of Computing University of Utah School of Computing University of Utah Presentation Outline 1 2 3 4 Main paper to be discussed David G. Lowe, Distinctive Image Features from Scale-Invariant Keypoints, IJCV, 2004. How to find useful keypoints?

More information

SUMMARY: DISTINCTIVE IMAGE FEATURES FROM SCALE- INVARIANT KEYPOINTS

SUMMARY: DISTINCTIVE IMAGE FEATURES FROM SCALE- INVARIANT KEYPOINTS SUMMARY: DISTINCTIVE IMAGE FEATURES FROM SCALE- INVARIANT KEYPOINTS Cognitive Robotics Original: David G. Lowe, 004 Summary: Coen van Leeuwen, s1460919 Abstract: This article presents a method to extract

More information

Introduction. Introduction. Related Research. SIFT method. SIFT method. Distinctive Image Features from Scale-Invariant. Scale.

Introduction. Introduction. Related Research. SIFT method. SIFT method. Distinctive Image Features from Scale-Invariant. Scale. Distinctive Image Features from Scale-Invariant Keypoints David G. Lowe presented by, Sudheendra Invariance Intensity Scale Rotation Affine View point Introduction Introduction SIFT (Scale Invariant Feature

More information

CS 4495 Computer Vision A. Bobick. CS 4495 Computer Vision. Features 2 SIFT descriptor. Aaron Bobick School of Interactive Computing

CS 4495 Computer Vision A. Bobick. CS 4495 Computer Vision. Features 2 SIFT descriptor. Aaron Bobick School of Interactive Computing CS 4495 Computer Vision Features 2 SIFT descriptor Aaron Bobick School of Interactive Computing Administrivia PS 3: Out due Oct 6 th. Features recap: Goal is to find corresponding locations in two images.

More information

Feature descriptors. Alain Pagani Prof. Didier Stricker. Computer Vision: Object and People Tracking

Feature descriptors. Alain Pagani Prof. Didier Stricker. Computer Vision: Object and People Tracking Feature descriptors Alain Pagani Prof. Didier Stricker Computer Vision: Object and People Tracking 1 Overview Previous lectures: Feature extraction Today: Gradiant/edge Points (Kanade-Tomasi + Harris)

More information

SCALE INVARIANT FEATURE TRANSFORM (SIFT)

SCALE INVARIANT FEATURE TRANSFORM (SIFT) 1 SCALE INVARIANT FEATURE TRANSFORM (SIFT) OUTLINE SIFT Background SIFT Extraction Application in Content Based Image Search Conclusion 2 SIFT BACKGROUND Scale-invariant feature transform SIFT: to detect

More information

Computer Vision for HCI. Topics of This Lecture

Computer Vision for HCI. Topics of This Lecture Computer Vision for HCI Interest Points Topics of This Lecture Local Invariant Features Motivation Requirements, Invariances Keypoint Localization Features from Accelerated Segment Test (FAST) Harris Shi-Tomasi

More information

The SIFT (Scale Invariant Feature

The SIFT (Scale Invariant Feature The SIFT (Scale Invariant Feature Transform) Detector and Descriptor developed by David Lowe University of British Columbia Initial paper ICCV 1999 Newer journal paper IJCV 2004 Review: Matt Brown s Canonical

More information

Local invariant features

Local invariant features Local invariant features Tuesday, Oct 28 Kristen Grauman UT-Austin Today Some more Pset 2 results Pset 2 returned, pick up solutions Pset 3 is posted, due 11/11 Local invariant features Detection of interest

More information

SIFT - scale-invariant feature transform Konrad Schindler

SIFT - scale-invariant feature transform Konrad Schindler SIFT - scale-invariant feature transform Konrad Schindler Institute of Geodesy and Photogrammetry Invariant interest points Goal match points between images with very different scale, orientation, projective

More information

2D Image Processing Feature Descriptors

2D Image Processing Feature Descriptors 2D Image Processing Feature Descriptors Prof. Didier Stricker Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche Intelligenz http://av.dfki.de 1 Overview

More information

SIFT: Scale Invariant Feature Transform

SIFT: Scale Invariant Feature Transform 1 / 25 SIFT: Scale Invariant Feature Transform Ahmed Othman Systems Design Department University of Waterloo, Canada October, 23, 2012 2 / 25 1 SIFT Introduction Scale-space extrema detection Keypoint

More information

Local Features Tutorial: Nov. 8, 04

Local Features Tutorial: Nov. 8, 04 Local Features Tutorial: Nov. 8, 04 Local Features Tutorial References: Matlab SIFT tutorial (from course webpage) Lowe, David G. Distinctive Image Features from Scale Invariant Features, International

More information

Scale Invariant Feature Transform by David Lowe

Scale Invariant Feature Transform by David Lowe Scale Invariant Feature Transform by David Lowe Presented by: Jerry Chen Achal Dave Vaishaal Shankar Some slides from Jason Clemons Motivation Image Matching Correspondence Problem Desirable Feature Characteristics

More information

Local features: detection and description. Local invariant features

Local features: detection and description. Local invariant features Local features: detection and description Local invariant features Detection of interest points Harris corner detection Scale invariant blob detection: LoG Description of local patches SIFT : Histograms

More information

Outline 7/2/201011/6/

Outline 7/2/201011/6/ Outline Pattern recognition in computer vision Background on the development of SIFT SIFT algorithm and some of its variations Computational considerations (SURF) Potential improvement Summary 01 2 Pattern

More information

Feature Detection. Raul Queiroz Feitosa. 3/30/2017 Feature Detection 1

Feature Detection. Raul Queiroz Feitosa. 3/30/2017 Feature Detection 1 Feature Detection Raul Queiroz Feitosa 3/30/2017 Feature Detection 1 Objetive This chapter discusses the correspondence problem and presents approaches to solve it. 3/30/2017 Feature Detection 2 Outline

More information

EECS150 - Digital Design Lecture 14 FIFO 2 and SIFT. Recap and Outline

EECS150 - Digital Design Lecture 14 FIFO 2 and SIFT. Recap and Outline EECS150 - Digital Design Lecture 14 FIFO 2 and SIFT Oct. 15, 2013 Prof. Ronald Fearing Electrical Engineering and Computer Sciences University of California, Berkeley (slides courtesy of Prof. John Wawrzynek)

More information

CS4670: Computer Vision

CS4670: Computer Vision CS4670: Computer Vision Noah Snavely Lecture 6: Feature matching and alignment Szeliski: Chapter 6.1 Reading Last time: Corners and blobs Scale-space blob detector: Example Feature descriptors We know

More information

Implementing the Scale Invariant Feature Transform(SIFT) Method

Implementing the Scale Invariant Feature Transform(SIFT) Method Implementing the Scale Invariant Feature Transform(SIFT) Method YU MENG and Dr. Bernard Tiddeman(supervisor) Department of Computer Science University of St. Andrews yumeng@dcs.st-and.ac.uk Abstract The

More information

Feature Matching and Robust Fitting

Feature Matching and Robust Fitting Feature Matching and Robust Fitting Computer Vision CS 143, Brown Read Szeliski 4.1 James Hays Acknowledgment: Many slides from Derek Hoiem and Grauman&Leibe 2008 AAAI Tutorial Project 2 questions? This

More information

Obtaining Feature Correspondences

Obtaining Feature Correspondences Obtaining Feature Correspondences Neill Campbell May 9, 2008 A state-of-the-art system for finding objects in images has recently been developed by David Lowe. The algorithm is termed the Scale-Invariant

More information

Harder case. Image matching. Even harder case. Harder still? by Diva Sian. by swashford

Harder case. Image matching. Even harder case. Harder still? by Diva Sian. by swashford Image matching Harder case by Diva Sian by Diva Sian by scgbt by swashford Even harder case Harder still? How the Afghan Girl was Identified by Her Iris Patterns Read the story NASA Mars Rover images Answer

More information

CEE598 - Visual Sensing for Civil Infrastructure Eng. & Mgmt.

CEE598 - Visual Sensing for Civil Infrastructure Eng. & Mgmt. CEE598 - Visual Sensing for Civil Infrastructure Eng. & Mgmt. Section 10 - Detectors part II Descriptors Mani Golparvar-Fard Department of Civil and Environmental Engineering 3129D, Newmark Civil Engineering

More information

Key properties of local features

Key properties of local features Key properties of local features Locality, robust against occlusions Must be highly distinctive, a good feature should allow for correct object identification with low probability of mismatch Easy to etract

More information

Local Feature Detectors

Local Feature Detectors Local Feature Detectors Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr Slides adapted from Cordelia Schmid and David Lowe, CVPR 2003 Tutorial, Matthew Brown,

More information

Features Points. Andrea Torsello DAIS Università Ca Foscari via Torino 155, Mestre (VE)

Features Points. Andrea Torsello DAIS Università Ca Foscari via Torino 155, Mestre (VE) Features Points Andrea Torsello DAIS Università Ca Foscari via Torino 155, 30172 Mestre (VE) Finding Corners Edge detectors perform poorly at corners. Corners provide repeatable points for matching, so

More information

Motion illusion, rotating snakes

Motion illusion, rotating snakes Motion illusion, rotating snakes Local features: main components 1) Detection: Find a set of distinctive key points. 2) Description: Extract feature descriptor around each interest point as vector. x 1

More information

Harder case. Image matching. Even harder case. Harder still? by Diva Sian. by swashford

Harder case. Image matching. Even harder case. Harder still? by Diva Sian. by swashford Image matching Harder case by Diva Sian by Diva Sian by scgbt by swashford Even harder case Harder still? How the Afghan Girl was Identified by Her Iris Patterns Read the story NASA Mars Rover images Answer

More information

EE368 Project Report CD Cover Recognition Using Modified SIFT Algorithm

EE368 Project Report CD Cover Recognition Using Modified SIFT Algorithm EE368 Project Report CD Cover Recognition Using Modified SIFT Algorithm Group 1: Mina A. Makar Stanford University mamakar@stanford.edu Abstract In this report, we investigate the application of the Scale-Invariant

More information

Motion Estimation and Optical Flow Tracking

Motion Estimation and Optical Flow Tracking Image Matching Image Retrieval Object Recognition Motion Estimation and Optical Flow Tracking Example: Mosiacing (Panorama) M. Brown and D. G. Lowe. Recognising Panoramas. ICCV 2003 Example 3D Reconstruction

More information

Local features: detection and description May 12 th, 2015

Local features: detection and description May 12 th, 2015 Local features: detection and description May 12 th, 2015 Yong Jae Lee UC Davis Announcements PS1 grades up on SmartSite PS1 stats: Mean: 83.26 Standard Dev: 28.51 PS2 deadline extended to Saturday, 11:59

More information

CS 558: Computer Vision 4 th Set of Notes

CS 558: Computer Vision 4 th Set of Notes 1 CS 558: Computer Vision 4 th Set of Notes Instructor: Philippos Mordohai Webpage: www.cs.stevens.edu/~mordohai E-mail: Philippos.Mordohai@stevens.edu Office: Lieb 215 Overview Keypoint matching Hessian

More information

Local Patch Descriptors

Local Patch Descriptors Local Patch Descriptors Slides courtesy of Steve Seitz and Larry Zitnick CSE 803 1 How do we describe an image patch? How do we describe an image patch? Patches with similar content should have similar

More information

BSB663 Image Processing Pinar Duygulu. Slides are adapted from Selim Aksoy

BSB663 Image Processing Pinar Duygulu. Slides are adapted from Selim Aksoy BSB663 Image Processing Pinar Duygulu Slides are adapted from Selim Aksoy Image matching Image matching is a fundamental aspect of many problems in computer vision. Object or scene recognition Solving

More information

Midterm Wed. Local features: detection and description. Today. Last time. Local features: main components. Goal: interest operator repeatability

Midterm Wed. Local features: detection and description. Today. Last time. Local features: main components. Goal: interest operator repeatability Midterm Wed. Local features: detection and description Monday March 7 Prof. UT Austin Covers material up until 3/1 Solutions to practice eam handed out today Bring a 8.5 11 sheet of notes if you want Review

More information

Comparison of Feature Detection and Matching Approaches: SIFT and SURF

Comparison of Feature Detection and Matching Approaches: SIFT and SURF GRD Journals- Global Research and Development Journal for Engineering Volume 2 Issue 4 March 2017 ISSN: 2455-5703 Comparison of Detection and Matching Approaches: SIFT and SURF Darshana Mistry PhD student

More information

Local Image Features

Local Image Features Local Image Features Computer Vision CS 143, Brown Read Szeliski 4.1 James Hays Acknowledgment: Many slides from Derek Hoiem and Grauman&Leibe 2008 AAAI Tutorial This section: correspondence and alignment

More information

Feature Based Registration - Image Alignment

Feature Based Registration - Image Alignment Feature Based Registration - Image Alignment Image Registration Image registration is the process of estimating an optimal transformation between two or more images. Many slides from Alexei Efros http://graphics.cs.cmu.edu/courses/15-463/2007_fall/463.html

More information

Feature descriptors and matching

Feature descriptors and matching Feature descriptors and matching Detections at multiple scales Invariance of MOPS Intensity Scale Rotation Color and Lighting Out-of-plane rotation Out-of-plane rotation Better representation than color:

More information

Building a Panorama. Matching features. Matching with Features. How do we build a panorama? Computational Photography, 6.882

Building a Panorama. Matching features. Matching with Features. How do we build a panorama? Computational Photography, 6.882 Matching features Building a Panorama Computational Photography, 6.88 Prof. Bill Freeman April 11, 006 Image and shape descriptors: Harris corner detectors and SIFT features. Suggested readings: Mikolajczyk

More information

Lecture 10 Detectors and descriptors

Lecture 10 Detectors and descriptors Lecture 10 Detectors and descriptors Properties of detectors Edge detectors Harris DoG Properties of detectors SIFT Shape context Silvio Savarese Lecture 10-26-Feb-14 From the 3D to 2D & vice versa P =

More information

Augmented Reality VU. Computer Vision 3D Registration (2) Prof. Vincent Lepetit

Augmented Reality VU. Computer Vision 3D Registration (2) Prof. Vincent Lepetit Augmented Reality VU Computer Vision 3D Registration (2) Prof. Vincent Lepetit Feature Point-Based 3D Tracking Feature Points for 3D Tracking Much less ambiguous than edges; Point-to-point reprojection

More information

Computer Vision. Recap: Smoothing with a Gaussian. Recap: Effect of σ on derivatives. Computer Science Tripos Part II. Dr Christopher Town

Computer Vision. Recap: Smoothing with a Gaussian. Recap: Effect of σ on derivatives. Computer Science Tripos Part II. Dr Christopher Town Recap: Smoothing with a Gaussian Computer Vision Computer Science Tripos Part II Dr Christopher Town Recall: parameter σ is the scale / width / spread of the Gaussian kernel, and controls the amount of

More information

3D Reconstruction From Multiple Views Based on Scale-Invariant Feature Transform. Wenqi Zhu

3D Reconstruction From Multiple Views Based on Scale-Invariant Feature Transform. Wenqi Zhu 3D Reconstruction From Multiple Views Based on Scale-Invariant Feature Transform Wenqi Zhu wenqizhu@buffalo.edu Problem Statement! 3D reconstruction 3D reconstruction is a problem of recovering depth information

More information

Image matching. Announcements. Harder case. Even harder case. Project 1 Out today Help session at the end of class. by Diva Sian.

Image matching. Announcements. Harder case. Even harder case. Project 1 Out today Help session at the end of class. by Diva Sian. Announcements Project 1 Out today Help session at the end of class Image matching by Diva Sian by swashford Harder case Even harder case How the Afghan Girl was Identified by Her Iris Patterns Read the

More information

Local features and image matching. Prof. Xin Yang HUST

Local features and image matching. Prof. Xin Yang HUST Local features and image matching Prof. Xin Yang HUST Last time RANSAC for robust geometric transformation estimation Translation, Affine, Homography Image warping Given a 2D transformation T and a source

More information

AK Computer Vision Feature Point Detectors and Descriptors

AK Computer Vision Feature Point Detectors and Descriptors AK Computer Vision Feature Point Detectors and Descriptors 1 Feature Point Detectors and Descriptors: Motivation 2 Step 1: Detect local features should be invariant to scale and rotation, or perspective

More information

Distinctive Image Features from Scale-Invariant Keypoints

Distinctive Image Features from Scale-Invariant Keypoints International Journal of Computer Vision 60(2), 91 110, 2004 c 2004 Kluwer Academic Publishers. Manufactured in The Netherlands. Distinctive Image Features from Scale-Invariant Keypoints DAVID G. LOWE

More information

Implementation and Comparison of Feature Detection Methods in Image Mosaicing

Implementation and Comparison of Feature Detection Methods in Image Mosaicing IOSR Journal of Electronics and Communication Engineering (IOSR-JECE) e-issn: 2278-2834,p-ISSN: 2278-8735 PP 07-11 www.iosrjournals.org Implementation and Comparison of Feature Detection Methods in Image

More information

Pictures at an Exhibition

Pictures at an Exhibition Pictures at an Exhibition Han-I Su Department of Electrical Engineering Stanford University, CA, 94305 Abstract We employ an image identification algorithm for interactive museum guide with pictures taken

More information

A Novel Algorithm for Color Image matching using Wavelet-SIFT

A Novel Algorithm for Color Image matching using Wavelet-SIFT International Journal of Scientific and Research Publications, Volume 5, Issue 1, January 2015 1 A Novel Algorithm for Color Image matching using Wavelet-SIFT Mupuri Prasanth Babu *, P. Ravi Shankar **

More information

Performance Evaluation of Scale-Interpolated Hessian-Laplace and Haar Descriptors for Feature Matching

Performance Evaluation of Scale-Interpolated Hessian-Laplace and Haar Descriptors for Feature Matching Performance Evaluation of Scale-Interpolated Hessian-Laplace and Haar Descriptors for Feature Matching Akshay Bhatia, Robert Laganière School of Information Technology and Engineering University of Ottawa

More information

Image Features: Detection, Description, and Matching and their Applications

Image Features: Detection, Description, and Matching and their Applications Image Features: Detection, Description, and Matching and their Applications Image Representation: Global Versus Local Features Features/ keypoints/ interset points are interesting locations in the image.

More information

Image Features. Work on project 1. All is Vanity, by C. Allan Gilbert,

Image Features. Work on project 1. All is Vanity, by C. Allan Gilbert, Image Features Work on project 1 All is Vanity, by C. Allan Gilbert, 1873-1929 Feature extrac*on: Corners and blobs c Mo*va*on: Automa*c panoramas Credit: Ma9 Brown Why extract features? Mo*va*on: panorama

More information

Faster Image Feature Extraction Hardware

Faster Image Feature Extraction Hardware IOSR Journal of Computer Engineering (IOSR-JCE) e-issn: 2278-0661,p-ISSN: 2278-8727 PP 33-38 www.iosrjournals.org Jibu J.V, Sherin Das, Mini Kumari G Assistant Professor,College of Engineering, Chengannur.Alappuzha,

More information

Advanced Video Content Analysis and Video Compression (5LSH0), Module 4

Advanced Video Content Analysis and Video Compression (5LSH0), Module 4 Advanced Video Content Analysis and Video Compression (5LSH0), Module 4 Visual feature extraction Part I: Color and texture analysis Sveta Zinger Video Coding and Architectures Research group, TU/e ( s.zinger@tue.nl

More information

CS 556: Computer Vision. Lecture 3

CS 556: Computer Vision. Lecture 3 CS 556: Computer Vision Lecture 3 Prof. Sinisa Todorovic sinisa@eecs.oregonstate.edu 1 Outline Matlab Image features -- Interest points Point descriptors Homework 1 2 Basic MATLAB Commands 3 Basic MATLAB

More information

CS 556: Computer Vision. Lecture 3

CS 556: Computer Vision. Lecture 3 CS 556: Computer Vision Lecture 3 Prof. Sinisa Todorovic sinisa@eecs.oregonstate.edu Interest Points Harris corners Hessian points SIFT Difference-of-Gaussians SURF 2 Properties of Interest Points Locality

More information

Image Matching. AKA: Image registration, the correspondence problem, Tracking,

Image Matching. AKA: Image registration, the correspondence problem, Tracking, Image Matching AKA: Image registration, the correspondence problem, Tracking, What Corresponds to What? Daisy? Daisy From: www.amphian.com Relevant for Analysis of Image Pairs (or more) Also Relevant for

More information

CS231A Section 6: Problem Set 3

CS231A Section 6: Problem Set 3 CS231A Section 6: Problem Set 3 Kevin Wong Review 6 -! 1 11/09/2012 Announcements PS3 Due 2:15pm Tuesday, Nov 13 Extra Office Hours: Friday 6 8pm Huang Common Area, Basement Level. Review 6 -! 2 Topics

More information

Eppur si muove ( And yet it moves )

Eppur si muove ( And yet it moves ) Eppur si muove ( And yet it moves ) - Galileo Galilei University of Texas at Arlington Tracking of Image Features CSE 4392-5369 Vision-based Robot Sensing, Localization and Control Dr. Gian Luca Mariottini,

More information

Distinctive Image Features from Scale-Invariant Keypoints

Distinctive Image Features from Scale-Invariant Keypoints Distinctive Image Features from Scale-Invariant Keypoints David G. Lowe Computer Science Department University of British Columbia Vancouver, B.C., Canada Draft: Submitted for publication. This version:

More information

Corner Detection. GV12/3072 Image Processing.

Corner Detection. GV12/3072 Image Processing. Corner Detection 1 Last Week 2 Outline Corners and point features Moravec operator Image structure tensor Harris corner detector Sub-pixel accuracy SUSAN FAST Example descriptor: SIFT 3 Point Features

More information

Local Image Features

Local Image Features Local Image Features Computer Vision Read Szeliski 4.1 James Hays Acknowledgment: Many slides from Derek Hoiem and Grauman&Leibe 2008 AAAI Tutorial Flashed Face Distortion 2nd Place in the 8th Annual Best

More information

SURF. Lecture6: SURF and HOG. Integral Image. Feature Evaluation with Integral Image

SURF. Lecture6: SURF and HOG. Integral Image. Feature Evaluation with Integral Image SURF CSED441:Introduction to Computer Vision (2015S) Lecture6: SURF and HOG Bohyung Han CSE, POSTECH bhhan@postech.ac.kr Speed Up Robust Features (SURF) Simplified version of SIFT Faster computation but

More information

Patch-based Object Recognition. Basic Idea

Patch-based Object Recognition. Basic Idea Patch-based Object Recognition 1! Basic Idea Determine interest points in image Determine local image properties around interest points Use local image properties for object classification Example: Interest

More information

TA Section 7 Problem Set 3. SIFT (Lowe 2004) Shape Context (Belongie et al. 2002) Voxel Coloring (Seitz and Dyer 1999)

TA Section 7 Problem Set 3. SIFT (Lowe 2004) Shape Context (Belongie et al. 2002) Voxel Coloring (Seitz and Dyer 1999) TA Section 7 Problem Set 3 SIFT (Lowe 2004) Shape Context (Belongie et al. 2002) Voxel Coloring (Seitz and Dyer 1999) Sam Corbett-Davies TA Section 7 02-13-2014 Distinctive Image Features from Scale-Invariant

More information

CS664 Lecture #21: SIFT, object recognition, dynamic programming

CS664 Lecture #21: SIFT, object recognition, dynamic programming CS664 Lecture #21: SIFT, object recognition, dynamic programming Some material taken from: Sebastian Thrun, Stanford http://cs223b.stanford.edu/ Yuri Boykov, Western Ontario David Lowe, UBC http://www.cs.ubc.ca/~lowe/keypoints/

More information

Scale Invariant Feature Transform (SIFT) CS 763 Ajit Rajwade

Scale Invariant Feature Transform (SIFT) CS 763 Ajit Rajwade Scale Invariant Feature Transform (SIFT) CS 763 Ajit Rajwade What is SIFT? It is a technique for detecting salient stable feature points in an image. For ever such point it also provides a set of features

More information

Object Recognition with Invariant Features

Object Recognition with Invariant Features Object Recognition with Invariant Features Definition: Identify objects or scenes and determine their pose and model parameters Applications Industrial automation and inspection Mobile robots, toys, user

More information

Distinctive Image Features from Scale-Invariant Keypoints

Distinctive Image Features from Scale-Invariant Keypoints Distinctive Image Features from Scale-Invariant Keypoints David G. Lowe Computer Science Department University of British Columbia Vancouver, B.C., Canada lowe@cs.ubc.ca January 5, 2004 Abstract This paper

More information

EE795: Computer Vision and Intelligent Systems

EE795: Computer Vision and Intelligent Systems EE795: Computer Vision and Intelligent Systems Spring 2012 TTh 17:30-18:45 FDH 204 Lecture 09 130219 http://www.ee.unlv.edu/~b1morris/ecg795/ 2 Outline Review Feature Descriptors Feature Matching Feature

More information

Coarse-to-fine image registration

Coarse-to-fine image registration Today we will look at a few important topics in scale space in computer vision, in particular, coarseto-fine approaches, and the SIFT feature descriptor. I will present only the main ideas here to give

More information

Edge and corner detection

Edge and corner detection Edge and corner detection Prof. Stricker Doz. G. Bleser Computer Vision: Object and People Tracking Goals Where is the information in an image? How is an object characterized? How can I find measurements

More information

Line, edge, blob and corner detection

Line, edge, blob and corner detection Line, edge, blob and corner detection Dmitri Melnikov MTAT.03.260 Pattern Recognition and Image Analysis April 5, 2011 1 / 33 Outline 1 Introduction 2 Line detection 3 Edge detection 4 Blob detection 5

More information

Feature Detection and Matching

Feature Detection and Matching and Matching CS4243 Computer Vision and Pattern Recognition Leow Wee Kheng Department of Computer Science School of Computing National University of Singapore Leow Wee Kheng (CS4243) Camera Models 1 /

More information

CS334: Digital Imaging and Multimedia Edges and Contours. Ahmed Elgammal Dept. of Computer Science Rutgers University

CS334: Digital Imaging and Multimedia Edges and Contours. Ahmed Elgammal Dept. of Computer Science Rutgers University CS334: Digital Imaging and Multimedia Edges and Contours Ahmed Elgammal Dept. of Computer Science Rutgers University Outlines What makes an edge? Gradient-based edge detection Edge Operators From Edges

More information

Local Image preprocessing (cont d)

Local Image preprocessing (cont d) Local Image preprocessing (cont d) 1 Outline - Edge detectors - Corner detectors - Reading: textbook 5.3.1-5.3.5 and 5.3.10 2 What are edges? Edges correspond to relevant features in the image. An edge

More information

A Comparison and Matching Point Extraction of SIFT and ISIFT

A Comparison and Matching Point Extraction of SIFT and ISIFT A Comparison and Matching Point Extraction of SIFT and ISIFT A. Swapna A. Geetha Devi M.Tech Scholar, PVPSIT, Vijayawada Associate Professor, PVPSIT, Vijayawada bswapna.naveen@gmail.com geetha.agd@gmail.com

More information

A Comparison of SIFT, PCA-SIFT and SURF

A Comparison of SIFT, PCA-SIFT and SURF A Comparison of SIFT, PCA-SIFT and SURF Luo Juan Computer Graphics Lab, Chonbuk National University, Jeonju 561-756, South Korea qiuhehappy@hotmail.com Oubong Gwun Computer Graphics Lab, Chonbuk National

More information

3D Object Recognition using Multiclass SVM-KNN

3D Object Recognition using Multiclass SVM-KNN 3D Object Recognition using Multiclass SVM-KNN R. Muralidharan, C. Chandradekar April 29, 2014 Presented by: Tasadduk Chowdhury Problem We address the problem of recognizing 3D objects based on various

More information

Verslag Project beeldverwerking A study of the 2D SIFT algorithm

Verslag Project beeldverwerking A study of the 2D SIFT algorithm Faculteit Ingenieurswetenschappen 27 januari 2008 Verslag Project beeldverwerking 2007-2008 A study of the 2D SIFT algorithm Dimitri Van Cauwelaert Prof. dr. ir. W. Philips dr. ir. A. Pizurica 2 Content

More information

Feature Detectors and Descriptors: Corners, Lines, etc.

Feature Detectors and Descriptors: Corners, Lines, etc. Feature Detectors and Descriptors: Corners, Lines, etc. Edges vs. Corners Edges = maxima in intensity gradient Edges vs. Corners Corners = lots of variation in direction of gradient in a small neighborhood

More information

Digital Image Processing (CS/ECE 545) Lecture 5: Edge Detection (Part 2) & Corner Detection

Digital Image Processing (CS/ECE 545) Lecture 5: Edge Detection (Part 2) & Corner Detection Digital Image Processing (CS/ECE 545) Lecture 5: Edge Detection (Part 2) & Corner Detection Prof Emmanuel Agu Computer Science Dept. Worcester Polytechnic Institute (WPI) Recall: Edge Detection Image processing

More information

Edge Detection (with a sidelight introduction to linear, associative operators). Images

Edge Detection (with a sidelight introduction to linear, associative operators). Images Images (we will, eventually, come back to imaging geometry. But, now that we know how images come from the world, we will examine operations on images). Edge Detection (with a sidelight introduction to

More information

Image features. Image Features

Image features. Image Features Image features Image features, such as edges and interest points, provide rich information on the image content. They correspond to local regions in the image and are fundamental in many applications in

More information

Designing Applications that See Lecture 7: Object Recognition

Designing Applications that See Lecture 7: Object Recognition stanford hci group / cs377s Designing Applications that See Lecture 7: Object Recognition Dan Maynes-Aminzade 29 January 2008 Designing Applications that See http://cs377s.stanford.edu Reminders Pick up

More information

Discovering Visual Hierarchy through Unsupervised Learning Haider Razvi

Discovering Visual Hierarchy through Unsupervised Learning Haider Razvi Discovering Visual Hierarchy through Unsupervised Learning Haider Razvi hrazvi@stanford.edu 1 Introduction: We present a method for discovering visual hierarchy in a set of images. Automatically grouping

More information

3D Vision. Viktor Larsson. Spring 2019

3D Vision. Viktor Larsson. Spring 2019 3D Vision Viktor Larsson Spring 2019 Schedule Feb 18 Feb 25 Mar 4 Mar 11 Mar 18 Mar 25 Apr 1 Apr 8 Apr 15 Apr 22 Apr 29 May 6 May 13 May 20 May 27 Introduction Geometry, Camera Model, Calibration Features,

More information

Other Linear Filters CS 211A

Other Linear Filters CS 211A Other Linear Filters CS 211A Slides from Cornelia Fermüller and Marc Pollefeys Edge detection Convert a 2D image into a set of curves Extracts salient features of the scene More compact than pixels Origin

More information

Image Features: Local Descriptors. Sanja Fidler CSC420: Intro to Image Understanding 1/ 58

Image Features: Local Descriptors. Sanja Fidler CSC420: Intro to Image Understanding 1/ 58 Image Features: Local Descriptors Sanja Fidler CSC420: Intro to Image Understanding 1/ 58 [Source: K. Grauman] Sanja Fidler CSC420: Intro to Image Understanding 2/ 58 Local Features Detection: Identify

More information

Templates, Image Pyramids, and Filter Banks

Templates, Image Pyramids, and Filter Banks Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown Slides: Hoiem and others Reminder Project due Friday Fourier Bases Teases away fast vs. slow changes in the image. This change

More information

CS534: Introduction to Computer Vision Edges and Contours. Ahmed Elgammal Dept. of Computer Science Rutgers University

CS534: Introduction to Computer Vision Edges and Contours. Ahmed Elgammal Dept. of Computer Science Rutgers University CS534: Introduction to Computer Vision Edges and Contours Ahmed Elgammal Dept. of Computer Science Rutgers University Outlines What makes an edge? Gradient-based edge detection Edge Operators Laplacian

More information

Bias-Variance Trade-off (cont d) + Image Representations

Bias-Variance Trade-off (cont d) + Image Representations CS 275: Machine Learning Bias-Variance Trade-off (cont d) + Image Representations Prof. Adriana Kovashka University of Pittsburgh January 2, 26 Announcement Homework now due Feb. Generalization Training

More information