SIFT: Scale Invariant Feature Transform

Transcription

1 1 / 25 SIFT: Scale Invariant Feature Transform Ahmed Othman Systems Design Department University of Waterloo, Canada October, 23, 2012

2 2 / 25 1 SIFT Introduction Scale-space extrema detection Keypoint localization Orientation assignment Keypoint descriptor 2 Speeded Up Robust Feature -SURF 3 Conclusion

3 Introduction Objectives Extract different features from object image. These features are: Invariant to image scale and rotation. Used to recognize this object in different images. 3 / 25

4 Introduction SIFT Output Testing Data 4 / 25

5 Introduction SIFT Output Matching Process. 5 / 25

6 Introduction SIFT Output 6 / 25

7 Introduction SIFT Steps SIFT is consist of four stages. 1 Scale-space extrema detection: Search over all scales of the image. Detect candidate key points. 2 Keypoint localization. Measure the stability of the candidate point. 3 Orientation assignment. Image gradient directions are used to assign orientations to the keypoint locations 4 Keypoint descriptor. Each point is represented with a descriptor vector. 7 / 25

8 8 / 25 Scale-space extrema detection Scale-space extrema detection Detecting key points that are invariant to different scales of the image. Use Gaussian filter to detect the candidate points. Define the scale space of an image L(x, y, ) as: L(x, y, )=G(x, y, ) I(x, y), (1) G(x, y, )= exp (x 2 +y 2 )/2 2 (2) Difference of Gaussian function D(x, y, ) are used to detect stable points by convolving to the image. D(x, y, )=(G(x, y, k ) G(x, y, )) I(x, y) (3) D(x, y, )=L(x, y, k ) L(x, y, ). (4) k is a constant multiplicative factor.

9 9 / 25 Scale-space extrema detection Difference of Gaussians.

10 Scale-space extrema detection Scale-space extrema detection Every point is compared to: 1 Its eight neighbors in 3 3 map. 2 Its nine neighbors in the scale above and below. The point is selected if : 1 Larger than the other 26 points (local maxima). 2 Less than the other 26 points (local minima). 10 / 25

11 Keypoint localization Keypoint localization The candidate points are revisited and outliers are discarded. Two types of points are Discarded: 1 Points with low contrast. 2 points that poorly localized along an edge. Use the Taylor expansion of the scale space D(x, y, ). The sample point is in the origin. 11 / 25

12 Keypoint localization Keypoint localization - Low contrast points removal The Taylor series of the scale space is defined as: D(x) x x 2 x (5) The location of the extremum ˆx is determined as: ˆx (6) The Taylor value at the extremum is: D(ˆx) =D + ˆx (7) Points that its D(x) < 0.03 are discarded. 12 / 25

13 Keypoint localization Keypoint localization - Poor edge point removal DOG have strong response along the edge. The location along the edge may be poorly determined. The poor point in the DOG have: 1 A large principal curvature across the edge. 2 A small principal curvature in the perpendicular direction. A thresholded principle curvature (PC) is used to remove poor edge points. 13 / 25

14 14 / 25 Keypoint localization Keypoint localization -Poor edge point removal PC computed from 2 2 Hessian matrix: H = apple Dxx D xy D xy D yy (8) The trace Tr(H) and the determinant Det(H)are computed. Tr (H) =D xx + D yy = +, (9) Det(H) =D xx D yy (D xy ) 2 =. (10) the largest magnitude eignvalue is the lowest one.

15 Keypoint localization Keypoint localization -Poor edge point removal let r be: r =. (11) The ratio of the principle curvature is checked by checking : Tr (H) 2 (r + 1)2 <. (12) Det(H) 2 r Points with r greater than 10 are discarded. 15 / 25

16 Keypoint localization Keypoint localization 16 / 25

17 Orientation assignment Orientation assignment Based on image properties, an orientation is assigned to each keypoint. This makes the descriptor invariant to image rotation. The gradient magnitude m(x, y) and the orientation (x, y) of every smoothed image L(x, y) are used. p m(x, y) = (L(x + 1, y) L(x 1, y)) 2 +(L(x, y + 1) L(x, y 1)) 2 )) (13) (x, y) =tan 1(((L(x, y + 1) L(x, y 1)))/(L(x + 1, y) L(x 1, y))) (14) 17 / 25

18 18 / 25 Orientation assignment Orientation assignment The process is as follow: 1 A 36 (for 360 degree) bins histogram is built from the orientation of the points within a neighbiours of the key points. 2 Sample added to the histogram weighted by: Gradient magnitude, A Gaussian filter with equal 1.5 times that of the scale of the keypoint. 3 The highest peak and the 80% of the highest peak are detected.

19 Orientation assignment Orientation assignment Orientation Assignement. 19 / 25

20 Keypoint descriptor Keypoint descriptor Use the orientation histogram of the previous step. Look at around the key point. Divide region into four 4 4 blocks 8 bins each. For each block compute a histogram of gradient magnitude and orientation. The descriptor vector of size 128 is formed for every point. SIFT Descriptors. 20 / 25

21 21 / 25 SURF- Speeded Up Robust Feature Key points are detected using Hessian matrix. A descriptor vector is built describing the neighbourhood of every keypoint Matching process is performed based on the descriptor vector.

22 22 / 25 SURF- Descriptor vector Orientation assignment is done using Haar wavelet response in x and y directions. A 20s rectangle is generated around each keypoint. The rectangle is splitted into 4 4 subregions.

23 23 / 25 SURF- Descriptor vector For each subregion: the Haar wavelet response in horizontal and vertical direction d x, and d y. d x, and d y are summed over each subregion. The sum of the absolute value of the responses d x and d y are calculated. A four dimension vector [d x d y d x d y ] is extracted. Each sub region have 4 vectors come up to 64. Matching process is performed based on the descriptor vector.

24 24 / 25 SIFT - SURF Keypoints are detected. Different orientations are assigned around the keypoints. A set of discriptors are assigned to the key points. These descriptors are used for object recognition.

25 25 / 25 References Distinctive image features from scale-invariant keypoints. by David Lowe, International journal of computer vision volume 60, pages (91 110), Vedio lecture by Dr. Mubarak Shah, Professor of Computer Science at university of Central Florida, USA. Google images for SIFT. SURF: Speeded Up Robust Features. Herbert Bay, Tinne Tuytelaars, and Luc Van Gool Computer Vision 2006.