CS 556: Computer Vision. Lecture 3

Transcription

1 CS 556: Computer Vision Lecture 3 Prof. Sinisa Todorovic sinisa@eecs.oregonstate.edu

2 Interest Points Harris corners Hessian points SIFT Difference-of-Gaussians SURF 2

3 Properties of Interest Points Locality -- robust to occlusion, noise Saliency -- rich visual cue Stable under affine transforms Distinctiveness -- differ across distinct objects Efficiency -- easy to compute 3

4 Example of Detecting Harris Corners 4

5 Harris Corner Detector scanning window homogeneous region no change in all directions edge no change along the edge corner change in all directions 5 Source: Frolova, Simakov, Weizmann Institute

6 Harris Corner Detector E(x, y) =w(x, y) [I(x + u, y + v) I(x, y)] 2 2D convolution 6 Source: Frolova, Simakov, Weizmann Institute

7 Harris Corner Detector E(x, y) = X m,n w(m, n)[i(x + u + m, y + v + n) I(x + m, y + n)] 2 change weighting window shifted image original image 7 Source: Frolova, Simakov, Weizmann Institute

8 Harris Detector Example I(x, y) input image E(x, y) 2D map of changes 8

9 Harris Corner Detector Taylor series expansion For small shifts u 0 v 0 I(x + u, y + v) v I(x + u, y + v) I(x, y)+[i x I y ] apple u v 9 image derivatives along x and y axes

10 Harris Corner Detector E(x, y) =w(x, y) I(x, y)+[i x I y ] apple u 2 v I(x, y) apple u 2 = w(x, y) [I x I y ] v apple u v T apple u v = w(x, y) [I x I y ] [I x I y ] 0

11 Harris Corner Detector E(x, y) =w(x, y) [u v] apple Ix I y [I x I y ] apple u v =[u v] apple I 2 apple x(x, y) I w(x, y) x (x, y)i y (x, y) u I x (x, y)i y (x, y) Iy(x, 2 y) v {z } M(x,y)

12 Weighted Image Gradient w(x, y; ) I x (x, y) =w x (x, y; ) I(x, y) w(x, y; ) I y (x, y) =w y (x, y; ) I(x, y) Image is discrete Gradient is approximate We always find the gradient of the kernel! 2

13 Harris Corner Detector E(x, y) =[u v]m(x, y) apple u v apple (wx I) 2 (w x I)(w y I) M(x, y) = (w x I)(w y I) (w y I) 2 (x,y) 3

14 Eigenvalues and Eigenvectors of M 2 E(x, y) =[u v]m(x, y) apple u v Eigenvectors of M -- directions of the largest change of E(x,y) Eigenvalues of M -- the amount of change along eigenvectors 4

15 Detection of Harris Corners using the Eigenvalues of M 2 5

16 Harris Detector f( )= ( ) 2 ( ) ( )+ 2 ( ) = det(m( )) trace(m( )) objective function 6

17 Harris Detector Example input image 7

18 Harris Detector Example f values color coded red = high values blue = low values 8

19 Harris Detector Example f values thresholded 9

20 Harris Detector Example Harris features = Spatial maxima of f values 20

21 Example of Detecting Harris Corners 2

22 Hessian detector (Beaudet, 978) Hessian determinant I xx I yy Hessian ( I) = & I $ % I xx xy I I xy yy #! " I xy det( Hessian( I )) = I xx I yy I 2 xy 22 Source: Tuytelaars

23 Hessian Detector E(x, y) =[u v]h(x, y) apple u v H(x, y; )=w(x, y; ) apple Ixx (x, y) I xy (x, y) I xy (x, y) I yy (x, y) H(x, y; )= apple wxx (x, y; ) I(x, y) w xy (x, y; ) I(x, y) w xy (x, y; ) I(x, y) w yy (x, y; ) I(x, y) 23

24 Hessian Detector f( )= ( ) 2 ( ) ( )+ 2 ( ) = det(h( )) trace(h( )) objective function 24

25 Properties of Harris Corners Invariance to variations of imaging parameters: Illumination? Camera distance, i.e., scale? Camera viewpoint, i.e., affine transformation? 25

26 Harris/Hessian Detector is NOT Scale Invariant edge edge corner edge 26 Source: L. Fei-Fei

27 Automatic scale selection Function responses for increasing scale Scale trace (signature) f ( I ( x, σ )) i i m 27 Source: Tuytelaars

28 Automatic scale selection Function responses for increasing scale Scale trace (signature) f ( I ( x, σ )) i i m 28 Source: Tuytelaars

29 Automatic scale selection Function responses for increasing scale Scale trace (signature) f ( I ( x, σ )) i i m 29 Source: Tuytelaars

30 Automatic scale selection Function responses for increasing scale Scale trace (signature) f ( I ( x, σ )) i i m 30 Source: Tuytelaars

31 Automatic scale selection Function responses for increasing scale Scale trace (signature) f ( I ( x, σ )) i i m 3 Source: Tuytelaars

32 Automatic scale selection Function responses for increasing scale Scale trace (signature) f ( I ( x, σ )) i i m 32 Source: Tuytelaars

33 Automatic scale selection Function responses for increasing scale Scale trace (signature) f ( I ( x, σ )) i i m 33 f ( I ( x, σ )) " i i m Source: Tuytelaars

34 Automatic scale selection Function responses for increasing scale Scale trace (signature) f ( I ( x, σ )) i i m 34 f ( I ( x, σ )) " i i m Source: Tuytelaars

35 Automatic scale selection Function responses for increasing scale Scale trace (signature) f ( I ( x, σ )) i i m 35 f ( I ( x, σ )) " i i m Source: Tuytelaars

36 Automatic scale selection Function responses for increasing scale Scale trace (signature) f ( I ( x, σ )) i i m 36 f ( I ( x, σ )) " i i m Source: Tuytelaars

37 Automatic scale selection Function responses for increasing scale Scale trace (signature) f ( I ( x, σ )) i i m 37 f ( I ( x, σ )) " i i m Source: Tuytelaars

38 Automatic scale selection Function responses for increasing scale Scale trace (signature) f ( I ( x, σ )) i i m 38 f ( I ( x, σ )) " i i m Source: Tuytelaars

39 Automatic scale selection Function responses for increasing scale Scale trace (signature) f ( I ( x, σ )) i i m 39 f ( I ( x", σ ")) i i m Source: Tuytelaars

40 Popular Kernels: Difference of Gaussians w( ) I Gaussian G(x, y; )= p 2 exp x 2 + y Difference of Gaussians DoG( )=G(x, y; k ) G(x, y; ) 40

41 Selected Scales = Extrema of DoG/Laplacian Convolve the image with Gaussians whose sigma increases Then, subsample, and repeat the convolutions Finally, find extrema in the 3D DoG or Laplacian space 4 Source: D. Lowe

42 scale y SIFT Detector DoG x DoG Find a local maximum of E(x, y; )=DOG(x, y; ) I(x, y) simultaneously in: 2D space of the image Scale 42

43 SURF Detector E(x, y) =[u v]h(x, y) Hessian matrix apple u v H(x, y; )= apple wxx (x, y; ) I(x, y) w xy (x, y; ) I(x, y) w xy (x, y; ) I(x, y) w yy (x, y; ) I(x, y) implemented using Haar wavelets 43