Edge Detection. Computer Vision Shiv Ram Dubey, IIIT Sri City

Transcription

1 Edge Detection Computer Vision Shiv Ram Dubey, IIIT Sri City

2 Previous two classes: Image Filtering Spatial domain Smoothing, sharpening, measuring texture * = FFT FFT Inverse FFT = Frequency domain Denoising, sampling

3 Today s class Detecting edges Finding straight lines

4 Why finding edges is important? Cues for 3D shape Group pixels into objects or parts Shape analysis Recover geometry and viewpoint Vanishing line Vanishing point Vertical vanishing point (at infinity) Vanishing point

5 Origin of Edges surface normal discontinuity depth discontinuity surface color discontinuity illumination discontinuity Edges are caused by a variety of factors Source: Steve Seitz

6 Closeup of edges

7 Closeup of edges

8 Closeup of edges

9 Closeup of edges

10 Closeup of edges

11 Closeup of edges

12 Characterizing edges An edge is a place of rapid change in the image intensity function image

13 Characterizing edges An edge is a place of rapid change in the image intensity function image intensity function (along horizontal scanline)

14 Characterizing edges An edge is a place of rapid change in the image intensity function image intensity function (along horizontal scanline) first derivative

15 Characterizing edges An edge is a place of rapid change in the image intensity function image intensity function (along horizontal scanline) first derivative edges correspond to extrema of derivative

16 Intensity profile

17 Intensity profile Intensity

18 Intensity profile Intensity Gradient

19 With a little Gaussian noise Gradient

20 Effects of noise Consider a single row or column of the image Plotting intensity as a function of position gives a signal Source: S. Seitz

21 Effects of noise Consider a single row or column of the image Plotting intensity as a function of position gives a signal Where is the edge? Source: S. Seitz

22 Effects of noise Consider a single row or column of the image Plotting intensity as a function of position gives a signal Where is the edge? The larger the noise the stronger the response

23 Effects of noise Consider a single row or column of the image Plotting intensity as a function of position gives a signal What can we do about it?

24 Solution: smooth first f Source: S. Seitz

25 Solution: smooth first f g Source: S. Seitz

26 Solution: smooth first f g f * g Source: S. Seitz

27 Solution: smooth first f g f * g d dx ( f g) To find edges, look for peaks in ( f g) d dx Source: S. Seitz

28 Derivative theorem of convolution Differentiation is convolution, and convolution is associative: d ( f g) f This saves us one operation: dx d dx g f d dx g f d dx g Source: S. Seitz

29 Gaussian Kernel Source: C. Rasmussen

30 Gaussian filters = 1 pixel = 5 pixels = 10 pixels = 30 pixels

31 Derivative of Gaussian filter * [1 0-1] = Gaussian derivative of Gaussian (x)

32 Derivative of Gaussian filter x-direction y-direction

33 Designing an edge detector Criteria for a good edge detector: Good detection: find all real edges, ignoring noise or other artifacts Source: L. Fei-Fei

34 Designing an edge detector Criteria for a good edge detector: Good detection: find all real edges, ignoring noise or other artifacts Good localization detect edges as close as possible to the true edges return one point only for each true edge point Source: L. Fei-Fei

35 Canny edge detector The most widely used edge detector J. Canny, A Computational Approach To Edge Detection, IEEE Trans. Pattern Analysis and Machine Intelligence, 8: , 1986.

36 Example input image ( Lena )

37 Derivative of Gaussian filter x-direction y-direction

38 Compute Gradients (DoG) Input Image X-Derivative of Gaussian

39 Compute Gradients (DoG) Input Image X-Derivative of Gaussian Y-Derivative of Gaussian

40 Compute Gradients (DoG) Input Image X-Derivative of Gaussian Y-Derivative of Gaussian Gradient Magnitude

41 Get Orientation at Each Pixel Threshold at minimum level Get orientation theta = atan2(gy, gx)

42 Non-maximum suppression for each orientation At q, we have a maximum if the value is larger than those at both p and at r. Interpolate to get these values. Source: D. Forsyth

43 Examples from Sidebar: Interpolation options nearest Copy value from nearest known Very fast but creates blocky edges bilinear Weighted average from four nearest known pixels Fast and reasonable results bicubic (default) Non-linear smoothing over larger area Slower, visually appealing, may create negative pixel values

44 Before Non-max Suppression

45 After non-max suppression

46 Hysteresis thresholding Threshold at low/high levels to get weak/strong edge pixels Do connected components, starting from strong edge pixels

47 Hysteresis thresholding Check that maximum value of gradient value is sufficiently large drop-outs? use hysteresis use a high threshold to start edge curves and a low threshold to continue them. Source: S. Seitz

48 Final Canny Edges

49 Canny edge detector 1. Filter image with x, y derivatives of Gaussian 2. Find magnitude and orientation of gradient 3. Non-maximum suppression: Thin multi-pixel wide ridges down to single pixel width 4. Thresholding and linking (hysteresis): Define two thresholds: low and high Use the high threshold to start edge curves and the low threshold to continue them MATLAB: edge(image, canny ) Source: D. Lowe, L. Fei-Fei

50 Canny edge detector 1. Filter image with x, y derivatives of Gaussian 2. Find magnitude and orientation of gradient 3. Non-maximum suppression: Thin multi-pixel wide ridges down to single pixel width 4. Thresholding and linking (hysteresis): Define two thresholds: low and high Use the high threshold to start edge curves and the low threshold to continue them MATLAB: edge(image, canny ) Source: D. Lowe, L. Fei-Fei

51 Canny edge detector 1. Filter image with x, y derivatives of Gaussian 2. Find magnitude and orientation of gradient 3. Non-maximum suppression: Thin multi-pixel wide ridges down to single pixel width 4. Thresholding and linking (hysteresis): Define two thresholds: low and high Use the high threshold to start edge curves and the low threshold to continue them MATLAB: edge(image, canny ) Source: D. Lowe, L. Fei-Fei

52 Canny edge detector 1. Filter image with x, y derivatives of Gaussian 2. Find magnitude and orientation of gradient 3. Non-maximum suppression: Thin multi-pixel wide ridges down to single pixel width 4. Thresholding and linking (hysteresis): Define two thresholds: low and high Use the high threshold to start edge curves and the low threshold to continue them MATLAB: edge(image, canny ) Source: D. Lowe, L. Fei-Fei

53 Canny edge detector 1. Filter image with x, y derivatives of Gaussian 2. Find magnitude and orientation of gradient 3. Non-maximum suppression: Thin multi-pixel wide ridges down to single pixel width 4. Thresholding and linking (hysteresis): Define two thresholds: low and high Use the high threshold to start edge curves and the low threshold to continue them MATLAB: edge(image, canny ) Source: D. Lowe, L. Fei-Fei

54 Effect of (Gaussian kernel spread/size) original Canny with Canny with The choice of depends on desired behavior large detects large scale edges small detects fine features Source: S. Seitz

55 Learning to detect boundaries image human segmentation gradient magnitude Berkeley segmentation database:

56 Pb Boundary Detector Martin, Fowlkes, Malik 2004: Learning to Detection Natural Boundaries Figure from Fowlkes

57 Pb Boundary Detector I-Intensity, B-Brightness gradient, C-Color gradient, T-Texture gradient Figure from Fowlkes

58 Brightness Color Texture Combined Human

59 Results Pb (0.88) Human (0.95)

60 Results Pb Global Pb (0.88) Pb Human Human (0.96)

61 Results Pb (0.63) Human (0.95)

62 Results Pb (0.35) Human (0.90) For more: /vision/bsds/bench/html/ color.html

63 Global Pb Boundary Detector Figure from Fowlkes

64 Edge Detection with Structured Random Forests (Dollar and Zitnick ICCV 2013) Goal: quickly predict whether each pixel is an edge Insights Predictions can be learned from training data Predictions for nearby pixels should not be independent Solution Train structured random forests to split data into patches with similar boundaries based on features Predict boundaries at patch level, rather than pixel level, and aggregate (average votes) Boundaries in patch

65 Edge Detection with Structured Random Forests - Algorithm 1. Extract overlapping 32x32 patches at three scales 2. Features are pixel values and pairwise differences in feature maps (color, gradient magnitude, oriented gradient) 3. Predict T boundary maps in the central 16x16 region using T trained decision trees 4. Average predictions for each pixel across all patches

66 Edge Detection with Structured Random Forests Ground truth Results

67 DeepContour Shen et al., Deep Contour, CVPR 2015

68 DeepContour Shen et al., Deep Contour, CVPR 2015

69 Holistically-Nested Edge Detection: Using Convolution Xie and Tu, Holistically-Nested Edge Detection, ICCV 2015

70 Holistically-Nested Edge Detection: Using Convolution Xie and Tu, Holistically-Nested Edge Detection, ICCV 2015

71 State of edge detection Local edge detection is mostly solved Intensity gradient, color, texture Often used in combination with object detectors or region classifiers Deep learning approach is more common nowadays

72 Finding straight lines

73 Finding line segments using connected components 1. Compute canny edges Compute: gx, gy (DoG in x,y directions) Compute: theta = atan(gy / gx)

74 Finding line segments using connected components 1. Compute canny edges Compute: gx, gy (DoG in x,y directions) Compute: theta = atan(gy / gx) 2. Assign each edge to one of 8 directions

75 Finding line segments using connected components 1. Compute canny edges Compute: gx, gy (DoG in x,y directions) Compute: theta = atan(gy / gx) 2. Assign each edge to one of 8 directions 3. For each direction d, get edgelets: find connected components for edge pixels with directions in {d-1, d, d+1}

76 Finding line segments using connected components 1. Compute canny edges Compute: gx, gy (DoG in x,y directions) Compute: theta = atan(gy / gx) 2. Assign each edge to one of 8 directions 3. For each direction d, get edgelets: find connected components for edge pixels with directions in {d-1, d, d+1} 4. Compute straightness and theta of edgelets using eig of x,y 2 nd moment matrix of their points

77 Finding line segments using connected components 1. Compute canny edges Compute: gx, gy (DoG in x,y directions) Compute: theta = atan(gy / gx) 2. Assign each edge to one of 8 directions 3. For each direction d, get edgelets: find connected components for edge pixels with directions in {d-1, d, d+1} 4. Compute straightness and theta of edgelets using eig of x,y 2 nd moment matrix of their points M 2 x x x x y y 2 x x y y y y [ v, λ] eig( Μ) Larger eigenvector atan 2( v(2,2), v(1,2)) conf 2 / 1

78 Finding line segments using connected components 1. Compute canny edges Compute: gx, gy (DoG in x,y directions) Compute: theta = atan(gy / gx) 2. Assign each edge to one of 8 directions 3. For each direction d, get edgelets: find connected components for edge pixels with directions in {d-1, d, d+1} 4. Compute straightness and theta of edgelets using eig of x,y 2 nd moment matrix of their points M 2 x x x x y y 2 x x y y y y [ v, λ] eig( Μ) Larger eigenvector atan 2( v(2,2), v(1,2)) conf 2 / 1 5. Threshold on straightness, store segment

79 Canny lines straight edges

80 Things to remember Canny edge detector = smooth derivative thin threshold link Pb: learns weighting of gradient, color, texture differences More recent learning approaches give at least as good accuracy and are faster Straight line detector = canny + gradient orientations orientation binning linking check for straightness

81 Acknowledgements Thanks to the following researchers for making their teaching/research material online Forsyth Steve Seitz Noah Snavely J.B. Huang Derek Hoiem D. Lowe A. Bobick S. Lazebnik K. Grauman R. Zaleski

82 Next classes: Correspondence and Alignment Detecting interest points Tracking points Object/image alignment and registration Image stitching