What is an edge? Paint. Depth discontinuity. Material change. Texture boundary

Transcription

1 EDGES AND TEXTURES The slides are from several sources through James Hays (Brown); Srinivasa Narasimhan (CMU); Silvio Savarese (U. of Michigan); Bill Freeman and Antonio Torralba (MIT), including their own slides.

2 What is an edge?

3 What is an edge? Paint Depth discontinuity Material change Texture boundary 32

4 Characterizing edges An edge is a place of rapid change in the image intensity function image intensity function (along horizontal scanline) first derivative edges correspond to extrema of derivative

5 Partial derivatives of an image f ( x, y ) f ( x, y ) x y -1 1 or or 1-1

6 Image gradient The gradient of an image: T T T T The gradient points in the direction of most rapid increase in intensity How does this direction relate to the direction of the edge? The gradient direction is given by The edge strength is given by the gradient magnitude Source: Steve Seitz

7 Effects of noise Consider a single row or column of the image Plotting intensity as a function of position gives a signal Where is the edge? Source: S. Seitz

8 Solution: smooth first f g f* g d ( f g) dx To find edges, look for peaks in ( f g) d dx Source: S. Seitz

9 Derivative theorem of convolution Differentiation is convolution, and convolution is associative: d d ( f g) = f g dx dx This saves us one operation: f d dx g f d dx g Source: S. Seitz

10 Derivative of Gaussian filter x-direction y-direction These filters are separable.

11 Derivative of Gaussian filter x-direction y-direction Which one finds horizontal/vertical edges? vertical edges horizontal edges

12 Scale of Gaussian derivative filter 1 pixel 3 pixels 7 pixels Smoothed derivative removes noise, but blurs edge. Also finds edges at different scales Source: D. Forsyth

13 Review: Smoothing vs. derivative filters Smoothing filters Gaussian: remove high-frequency components; l filt low-pass filter. Can the values of a smoothing filter be negative? Gaussian = No. What should the values sum to of the filter? One: constant regions are not affected by the filter. Derivative filters Derivatives of Gaussian. Can the values of a derivative filter be negative? Yes. What should the values sum to of the filter? Zero: no response in constant regions. High absolute value at points of high contrast.

14 Canny Edge Detection Most widely used edge detector in computer vision. First derivative of the Gaussian closely approximates the operator that optimizes the product of signal-tonoise ratio and localization. Analysis based on "step-edges" corrupted by "additive Gaussian noise". Least squares with binomial weights ~~~ edge detector. J. Canny, A Computational Approach To Edge Detection. IEEE Trans. Pattern Analysis and Machine Intelligence, 8: , 1986.

15 Edge Detection Criteria Criteria for optimal edge detection (Canny 86): Good detection accuracy: minimize the probability of false positives (detecting spurious edges caused by noise), false negatives (missing real edges) Good localization: edges must be detected as close as possible to the true edges. Single response constraint: minimize the number of local maxima around the true edge (i.e. detector must return single point for each true edge point)

16 Examples... valid mostly for straight edges... True edge Poor robustness to noise Poor localization Too many responses

17 Steps: Canny Edge Detection 1+2. Gaussian smoothing together with derivative of Gaussian (~discrete) 3. Find magnitude and orientation of gradient 4. Extract edge points: Non-maximum suppression 5. Linking and thresholding: Hysteresis MATLAB: edge(i, canny )

18 First Two Steps Smoothing I y) g(x, ' I ), ( y x e y x g I g I g S y x g g y g x g g I g g y x I g I g y x Derivative Can be done with two one-dimensional filters. 2

19 two dimensional Gaussian h = g h x (x, y) = h(x, y) x = x x 2 +y 2 2πσ 4 e 2σ 2 h y (x, y) = h(x, y) y x 2 +y y 2 2πσ 4 e 2σ 2 = Scale

20 Example: I S x S y S g I g I S S x S y = gradient vector

21 sigma 1 pixel 2 pixels Increased smoothing: Eliminates noise edges. Makes edges smoother and thicker. Removes fine detail.

22 Third Step magnitude and direction of S S x S y magnitude direction (S 2 x tan S 1 2 y S S y x ) image gradient magnitude

23 Non-maximum suppression along the direction of gradient Fourth Step At q, we have a maximum if the value is larger than those at both p and at r. Interpolate to get these values. Source: D. Forsyth

24 Example: Non-Maximum Suppression Original image Gradient magnitude Non-maxima suppressed courtesy of G. Loy Slide credit: Christopher Rasmussen

25 high threshold strong edges only low threshold weak edges too

26 Fifth Step: Hysteresis Thresholding Hysteresis: no LOW maybe HIGH sure. Maintain two thresholds k high and k low Use k high to find strong edges to start edge chain. Use k low to find weak edges along the edge chain. Typical ratio of thresholds is roughly k high / k low = 2-2.5

27 Closing edge gaps Gradient magnitude Check that maximum value of gradient value is sufficiently large and... t 1 t 2... use hysteresis. use a high threshold to start edge curves and a low threshold to continue them. Not an edge Pixel number in linked list along gradient maxima Labeled as edge maybe a line

28 Example gap is gone Original image Strong + connected weak edges Strong edges only Weak edges too courtesy of G. Loy

29 Effect of (Gaussian kernel spread/size) original Canny with Canny with The choice of depends on desired behavior large detects large scale edges small detects fine features Source: S. Seitz

30 Example of Canny edge detection original image (Lena)

31 Compute Gradients (DoG) X-Derivative of Gaussian Y-Derivative of Gaussian Gradient Magnitude and orientation

32 Before non-max suppression...

33 ...after non-max suppression

34 Before the hysteresis thresholding Threshold at low/high levels to get weak/strong edge pixels Do connected components, starting from strong edge pixels

35 Final Canny Edges

36 A hidden advantage for the human observer. She/he first see the original image and only after the edges detected. What happens if she/he cannot see the original image first and therefore can rely on it? This is how all the computer vision algorithms has to work... all the time!

37

38

39 Learning to detect boundaries image human segmentation gradient magnitude Berkeley segmentation database:

40 - All are 3D objects in 2D projections! - The top-down recognition is very ad-hoc now. Edge detection is only the very beginning. - Edges are based on local, low-level pixel information. Much more is needed to detect boundaries or to recognize objects.

41 Representing Texture 3D is "present" and the textures are therefore more complex Source: Forsyth

42 Texture and Material ~2D

43 Texture and Orientation ~2D

44 Texture and Scale ~2D 3D

45 Representation of sets 3D ~2D

46 What is texture? Regular or stochastic patterns caused by orderly markings of a small unit. 3D textures are much more complicated in 2D representation. Will highlight only 2D textures. Even there the representations are far from perfect....an example:

47 Overcomplete representation: filter banks Leung and Malik (2001); 48 anisotropic and isotropic filters Gaussian and first derivative, Laplacian sensitive to texture rotation LM Filter Bank x sigma y 3 sigma 36 elongated filters: 3 scales; 6 orientations; first/second derivative Gaussian 8 Laplacian (difference of Gaussian); 4 low-pass Gaussian Code for filter banks:

48 Can you match the texture to the response? Filters Record simple statistics (e.g., mean, std.) of absolute filter responses. A 1 B 2 C 3 Mean abs responses

49 Filters Record simple statistics (e.g., mean, std.) of absolute filter responses A 1 B B 2 C C 3 A Mean abs responses

50 Another representation of textures Take absolute values of the output at each pixel. Lighter pixels, stronger response. 0 middle lighter + how coarse is the scale is important

51 Low-level edges and perceived contours Background Texture Shadows Kristen Grauman, UT-Austin...the reality is more complicated...