Keypoint detection. (image registration, panorama stitching, motion estimation + tracking, recognition )

Transcription

1 Keypoint detection n n Many applications benefit from features localized in (x,y) (image registration, panorama stitching, motion estimation + tracking, recognition ) Edges well localized only in one direction è detect corners?? n Desirable properties of keypoint detector l Accurate localization l Invariance against shift, rotation, scale, brightness change l Robustness against noise, high repeatability Digital Image Processing: Bernd Girod, 2013 Stanford University -- Keypoint Detection 1

2 Keypoint detection n Laplacian detector n Determinant of Hessian detector n Harris detector n FAST detector Digital Image Processing: Bernd Girod, 2013 Stanford University -- Keypoint Detection 2

3 Laplacian keypoint detector LoG convolution Thresholding Input f [x,y] Detect local min/max keypoints Digital Image Processing: Bernd Girod, 2013 Stanford University -- Keypoint Detection 3

4 Input images Digital Image Processing: Bernd Girod, 2013 Stanford University -- Keypoint Detection 4

5 LoG response Digital Image Processing: Bernd Girod, 2013 Stanford University -- Keypoint Detection 5

6 Thresholded LoG response Digital Image Processing: Bernd Girod, 2013 Stanford University -- Keypoint Detection 6

7 Local extrema of thresholded LoG response Digital Image Processing: Bernd Girod, 2013 Stanford University -- Keypoint Detection 7

8 Superimposed LoG keypoints 500 strongest keypoints Digital Image Processing: Bernd Girod, 2013 Stanford University -- Keypoint Detection 8

9 Determinant of Hessian keypoint detector D xx D yy D xy H[ x, y] = = f xx [x, y] f xy [x, y] f xy [x, y] f yy [x, y] D xx [ x, y] f [x, y] D xy [x, y] f [x, y] D xy [x, y] f [x, y] D yy [x, y] f [x, y] det H[ x, y] = f xx [x, y] f yy [x, y] ( f xy [x, y] ) 2 Digital Image Processing: Bernd Girod, 2013 Stanford University -- Keypoint Detection 9

10 Input images Digital Image Processing: Bernd Girod, 2013 Stanford University -- Keypoint Detection 10

11 DoH response Digital Image Processing: Bernd Girod, 2013 Stanford University -- Keypoint Detection 11

12 Thresholded DoH response Digital Image Processing: Bernd Girod, 2013 Stanford University -- Keypoint Detection 12

13 Local maxima of DoH response Digital Image Processing: Bernd Girod, 2013 Stanford University -- Keypoint Detection 13

14 Superimposed DoH keypoints 500 strongest keypoints Digital Image Processing: Bernd Girod, 2013 Stanford University -- Keypoint Detection 14

15 n n What patterns can be localized most accurately? Local displacement sensitivity (assuming continuous f(x,y)) S Δx,Δy Linear approximation for small f x + Δx, y + Δy ( ) = f ( x, y) f ( x + Δx, y + Δy) ( x,y) window ( ) f ( x, y) + f x x, y Δx, Δy ( )Δx + f y ( x, y)δy f x (x,y) horizontal image gradient f y (x,y) vertical image gradient 2 ( ) f x ( x, y) f y ( x, y) S Δx,Δy ( x,y) window ( ) = Δx Δy ( ) ( x,y) window ( ) M Δx = Δx Δy Δy f x Δx Δy 2 2 f x ( x, y) f x ( x, y) f y ( x, y) ( x, y) f y ( x, y) 2 f y ( x, y) Δx Δy n Iso-sensitivity curves are ellipses Digital Image Processing: Bernd Girod, 2013 Stanford University -- Keypoint Detection 15

16 Harris detector λ 2 Edge λ 2 >> λ 1 Corner λ 1 and λ 2 are large Based on eigenvalues λ 1, λ 2 of structure matrix (aka normal matrix aka second-moment matrix ) M = 2 f x x, y f x x, y x,y window x,y window f x x, y f y x, y 2 f y x,y window x,y window f y x, y x, y f x [x,y] horizontal image gradient f y [x,y] vertical image gradient Flat region Edge λ 1 >> λ 2 λ 1 Digital Image Processing: Bernd Girod, 2013 Stanford University -- Keypoint Detection 16

17 Harris cornerness C = det(m) k ( trace( M) ) 2 = λ 1 λ 2 k ( λ 1 + λ 2 ) λ 2 λ 2 k = k = λ λ Digital Image Processing: Bernd Girod, 2013 Stanford University -- Keypoint Detection 17

18 Input images Digital Image Processing: Bernd Girod, 2013 Stanford University -- Keypoint Detection 18

19 Harris cornerness Digital Image Processing: Bernd Girod, 2013 Stanford University -- Keypoint Detection 19

20 Thresholded cornerness Digital Image Processing: Bernd Girod, 2013 Stanford University -- Keypoint Detection 20

21 Local maxima of cornerness Digital Image Processing: Bernd Girod, 2013 Stanford University -- Keypoint Detection 21

22 Superimposed Harris keypoints 500 strongest keypoints Digital Image Processing: Bernd Girod, 2013 Stanford University -- Keypoint Detection 22

23 Robustness of Harris detector n Invariant to brightness offset: f [x,y] à f [x,y] + c n Invariant to shift and rotation è n Not invariant to scaling 1 è Repeatability edge corner Scale factor Digital Image Processing: Bernd Girod, 2013 Stanford University -- Keypoint Detection 23

24 Features from Accelerated Segment Test (FAST) P n n n Compare nucleus p to circle of sixteen pixels Nucleus is feature point, iff at least n=9 contiguous circle pixels are either all brighter, or all darker, by θ Optimize pixel comparisons to reject non-corners early Digital Image Processing: Bernd Girod, 2013 Stanford University -- Keypoint Detection 24

25 Input images Digital Image Processing: Bernd Girod, 2013 Stanford University -- Keypoint Detection 25

26 FAST corners superimposed Digital Image Processing: Bernd Girod, 2013 Stanford University -- Keypoint Detection 26

27 FAST corner detection on smartphone Digital Image Processing: Bernd Girod, 2013 Stanford University -- Keypoint Detection 27

28 FAST keypoint tracking on smartphone Digital Image Processing: Bernd Girod, 2013 Stanford University -- Keypoint Detection 28