CS 556: Computer Vision. Lecture 3

Transcription

1 CS 556: Computer Vision Lecture 3 Prof. Sinisa Todorovic sinisa@eecs.oregonstate.edu 1

2 Outline Matlab Image features -- Interest points Point descriptors Homework 1 2

3 Basic MATLAB Commands 3

4 Basic MATLAB Commands I= im2double(imread(ʻimage_nameʼ)) 3

5 Basic MATLAB Commands I= im2double(imread(ʻimage_nameʼ)) imwrite(uint8(i),ʼimage_nameʼ) 3

6 Basic MATLAB Commands I= im2double(imread(ʻimage_nameʼ)) imwrite(uint8(i),ʼimage_nameʼ) imshow(i) 3

7 Basic MATLAB Commands I= im2double(imread(ʻimage_nameʼ)) imwrite(uint8(i),ʼimage_nameʼ) imshow(i) print -djpeg Fig1.jpg 3

8 Basic MATLAB Commands I= im2double(imread(ʻimage_nameʼ)) imwrite(uint8(i),ʼimage_nameʼ) imshow(i) print -djpeg Fig1.jpg [x,y] = size(i) 3

9 Basic MATLAB Commands I= im2double(imread(ʻimage_nameʼ)) imwrite(uint8(i),ʼimage_nameʼ) imshow(i) print -djpeg Fig1.jpg [x,y] = size(i) rgb2gray(i) 3

10 Basic MATLAB Commands I= im2double(imread(ʻimage_nameʼ)) imwrite(uint8(i),ʼimage_nameʼ) imshow(i) print -djpeg Fig1.jpg [x,y] = size(i) rgb2gray(i) list = find(img == 0) 3

11 Basic MATLAB Commands I= im2double(imread(ʻimage_nameʼ)) imwrite(uint8(i),ʼimage_nameʼ) imshow(i) print -djpeg Fig1.jpg [x,y] = size(i) rgb2gray(i) list = find(img == 0) w=fspecial(ʻgaussʼ, 3, 0.01) 3

12 Basic MATLAB Commands I= im2double(imread(ʻimage_nameʼ)) imwrite(uint8(i),ʼimage_nameʼ) imshow(i) print -djpeg Fig1.jpg [x,y] = size(i) rgb2gray(i) list = find(img == 0) w=fspecial(ʻgaussʼ, 3, 0.01) imfilter(i,w) 3

13 Image Features -- Interest Points Harris corners Hessian points SIFT, Difference-of-Gaussians Harris-Laplacian Harris-Difference-of-Gaussians Point descriptors -- DAISY 4

14 Properties of Interest Points Locality -- robustness to occlusion, noise Saliency -- rich visual cue Stable under affine transforms Distinctiveness -- differ across distinct objects Efficiency -- easy to compute 5

15 Harris Corner Detector homogeneous region no change in all directions edge no change along the edge corner change in all directions Source: Frolova, Simakov, Weizmann Institute 6

16 Harris Corner Detector E(x, y) = m,n w(m, n)[i(x + u + m, y + v + n) I(x, y)] 2 change weighting window shifted image original image Source: Frolova, Simakov, Weizmann Institute 7

17 Harris Corner Detector E(x, y) =w(x, y) [I(x + u, y + v) I(x, y)] 2 2D convolution Source: Frolova, Simakov, Weizmann Institute 8

18 2D Convolution or 2D Correlation 9

19 Harris Detector Example I(x, y) input image E(x, y) 2D map of changes 10

20 Harris Corner Detector Taylor series expansion For small shifts u 0 v 0 I(x + u, y + v) I(x, y)+ I x u + I y v I(x + u, y + v) I(x, y)+[i x I y ] u v image derivatives along x and y axes 11

21 Harris Corner Detector 2 E(x, y) =w(x, y) I(x, y)+[i x I y ] u v I(x, y) u 2 = w(x, y) [I x I y ] v u v T u v = w(x, y) [I x I y ] [I x I y ] 12

22 Harris Corner Detector Ix u v E(x, y) =w(x, y) [u v] I y [I x I y ] =[u v] I 2 x(x, y) I w(x, y) x (x, y)i y (x, y) I x (x, y)i y (x, y) Iy(x, 2 y) M(x,y) u v 13

23 Harris Corner Detector E(x, y) =[u v]m(x, y) u v M(x, y; σ) =w(x, y; σ) I 2 x(x, y) I x (x, y)i y (x, y) I x (x, y)i y (x, y) I 2 y(x, y) M(x, y; σ) = w(x, y; σ) I 2 x(x, y) w(x, y; σ) I x (x, y)i y (x, y) w(x, y; σ) I x (x, y)i y (x, y) w(x, y; σ) I 2 y(x, y) 14

24 Image Gradient I x (x, y) =I(x +1,y) I(x, y) = D x (x,y) I(x, y) 15

25 Image Gradient I y (x, y) =I(x, y + 1) I(x, y) = D y (x,y) I(x, y) 16

26 Weighted Image Gradient w(x, y; σ) I x (x, y) =w(x, y; σ) D x (x, y) I(x, y) =[w(x, y; σ) D x (x, y)] I(x, y) convolution is associative 17

27 Weighted Image Gradient w(x, y; σ) I x (x, y) =w(x, y; σ) D x (x, y) I(x, y) =[w(x, y; σ) D x (x, y)] I(x, y) =[D x (x, y) w(x, y; σ)] I(x, y) convolution is commutative 18

28 Weighted Image Gradient w(x, y; σ) I x (x, y) =w(x, y; σ) D x (x, y) I(x, y) =[w(x, y; σ) D x (x, y)] I(x, y) =[D x (x, y) w(x, y; σ)] I(x, y) = w x (x, y; σ) I(x, y) derivative of the filter 19

29 Weighted Image Gradient w(x, y; σ) I x (x, y) =w x (x, y; σ) I(x, y) w(x, y; σ) I y (x, y) =w y (x, y; σ) I(x, y) Image is discrete Gradient is approximate We always find the gradient of the kernel! 20

30 Harris Corner Detector E(x, y) =[u v]m(x, y) u v M(x, y; σ) =w(x, y; σ) I 2 x(x, y) I x (x, y)i y (x, y) I x (x, y)i y (x, y) I 2 y(x, y) M(x, y; σ) = w(x, y; σ) I 2 x(x, y) w(x, y; σ) I x (x, y)i y (x, y) w(x, y; σ) I x (x, y)i y (x, y) w(x, y; σ) I 2 y(x, y) 21

31 Eigenvalues and Eigenvectors of M λ1 λ2 E(x, y) =[u v]m(x, y) u v Eigenvectors of M -- directions of the largest change of E(x,y) Eigenvalues of M -- the amount of change along eigenvectors 22

32 Detection of Harris Corners using the Eigenvalues of M λ 1 λ 2 23

33 Harris Detector f(σ) = λ 1(σ)λ 2 (σ) λ 1 (σ)+λ 2 (σ) = det(m(σ)) trace(m(σ)) objective function 24

34 Harris Detector Example input image 25

35 Harris Detector Example f values color coded red = high values blue = low values 26

36 Harris Detector Example f values thresholded 27

37 Harris Detector Example Harris features = Spatial maxima of f values 28

38 Example of Detecting Harris Corners 29

39 Hessian detector (Beaudet, 1978) Hessian determinant I xx I yy Hessian ( I) = & I $ % I xx xy I I xy yy #! " I xy det( Hessian ( I )) = I xx I yy ' I 2 xy Source: Tuytelaars 30

40 Hessian Detector u v E(x, y) =[u v]m(x, y) Ixx (x, y) I xy (x, y) M(x, y; σ) =w(x, y; σ) I xy (x, y) I yy (x, y) wxx (x, y; σ) I(x, y) w xy (x, y; σ) I(x, y) M(x, y; σ) = w xy (x, y; σ) I(x, y) w yy (x, y; σ) I(x, y) 31

41 Hessian Detector f(σ) = λ 1(σ)λ 2 (σ) λ 1 (σ)+λ 2 (σ) = det(m(σ)) trace(m(σ)) objective function 32

42 Properties of Harris Corners Invariance to variations of imaging parameters: Illumination? Camera distance, i.e., scale? Camera viewpoint, i.e., affine transformation? 33

43 Harris/Hessian Detector is NOT Scale Invariant edge edge corner edge Source: L. Fei-Fei 34

44 Automatic scale selection Function responses for increasing scale Scale trace (signature) f ( I 1 ( x,! )) i!i m Source: Tuytelaars 35

45 Automatic scale selection Function responses for increasing scale Scale trace (signature) f ( I 1 ( x,! )) i!i m Source: Tuytelaars 36

46 Automatic scale selection Function responses for increasing scale Scale trace (signature) f ( I 1 ( x,! )) i!i m Source: Tuytelaars 37

47 Automatic scale selection Function responses for increasing scale Scale trace (signature) f ( I 1 ( x,! )) i!i m Source: Tuytelaars 38

48 Automatic scale selection Function responses for increasing scale Scale trace (signature) f ( I 1 ( x,! )) i!i m Source: Tuytelaars 39

49 Automatic scale selection Function responses for increasing scale Scale trace (signature) f ( I 1 ( x,! )) i!i m Source: Tuytelaars 40

50 Automatic scale selection Function responses for increasing scale Scale trace (signature) f ( I 1 ( x,! )) i!i m f ( I 1 ( x,! )) " i i! Source: Tuytelaars m 41

51 Automatic scale selection Function responses for increasing scale Scale trace (signature) f ( I 1 ( x,! )) i!i m f ( I 1 ( x,! )) " i i! Source: Tuytelaars m 42

52 Automatic scale selection Function responses for increasing scale Scale trace (signature) f ( I 1 ( x,! )) i!i m f ( I 1 ( x,! )) " i i! Source: Tuytelaars m 43

53 Automatic scale selection Function responses for increasing scale Scale trace (signature) f ( I 1 ( x,! )) i!i m f ( I 1 ( x,! )) " i i! Source: Tuytelaars m 44

54 Automatic scale selection Function responses for increasing scale Scale trace (signature) f ( I 1 ( x,! )) i!i m f ( I 1 ( x,! )) " i i! Source: Tuytelaars m 45

55 Automatic scale selection Function responses for increasing scale Scale trace (signature) f ( I 1 ( x,! )) i!i m f ( I 1 ( x,! )) " i i! Source: Tuytelaars m 46

56 Automatic scale selection Function responses for increasing scale Scale trace (signature) f ( I 1 ( x,! )) i!i m f ( I 1 ( x",! ")) i! i m Source: Tuytelaars 47

57 Issues with the Automatic Scale Selection f f bad bad region size region size f(σ) =Kernel(σ) I Often difficult to identify the global maximum 48

58 Popular Kernels for Scale Selection w(σ) I Gaussian G(x, y; σ) = 1 2πσ exp x2 + y 2 2σ 2 Laplacian of Gaussians L(σ) =σ 2 (G xx (x, y; σ)+g yy (x, y; σ)) 49

59 Hessian/Harris-Laplacian Detector scale y! Harris " x! Laplacian " Find a local maximum of Hessian/Harris function E(x, y; σ) =σ 2 (G xx (x, y; σ)+g yy (x, y; σ)) I(x, y) simultaneously in : 2D space of the image Scale 50

60 Mikolajczyk: Harris Laplace 1. Initialization: Multiscale Harris corner detection 2. Scale selection based on Laplacian Harris points Harris-Laplace points Source: Tuytelaars 51

61 Popular Kernels for Scale Selection w(σ) I Gaussian G(x, y; σ) = 1 2πσ exp x2 + y 2 2σ 2 Difference of Gaussians DoG(σ) =G(x, y; kσ) G(x, y; σ) 52

62 Source: Tuytelaars Lowe s DoG Difference of Gaussians as approximation of the Laplacian of Gaussian - = 53

63 Selected Scales = Extrema of DoG/Laplacian Convolve the image with Gaussians whose sigma increases Then, subsample, and repeat the convolutions Finally, find extrema in the 3D DoG or Laplacian space Source: D. Lowe 54

64 scale y SIFT Detector! DoG " x! DoG " Find a local maximum of E(x, y; σ) =DOG(x, y; σ) I(x, y) simultaneously in: 2D space of the image Scale 55

65 Source: Tuytelaars Mikolajczyk: Harris Laplace 1. Initialization: Multiscale Harris corner detection! 4! 3! 2! Computing Harris function Detecting local maxima 56

66 Next Class Other interest points Point descriptors 57