Filtering and Edge Detection. Computer Vision I. CSE252A Lecture 10. Announcement

Transcription

1 Filtering and Edge Detection CSE252A Lecture 10 Announcement HW1graded, will be released later today HW2 assigned, due Wed. Nov. 7 1

2 Image formation: Color Channel k " $ $ # $ I r I g I b % " ' $ ' = ( f d n l) $ &' # $ D r D g D b % " ' $ ' + ( f s (θ)n l) $ &' # $ S r S g S b % ' ' &' Image color lies in span of diffuse color D and specular color S. +, David Kriegman Data-dependent SUV Color Space [R] = [ S U V] t U,V spans a plane orthogonal to S First row of R is the RGB color value of light source color S, normalized to unit length. Other rows are orthogonal to S, David Kriegman 2

3 Convolution: R= K*I * Kernel (K) Image (I) Note: Typically Kernel is relatively small in vision applications. Convolution: R= K*I m=2 I R Kernel size is m+1 by m+1 R( i, j) = m/ 2 m/ 2 h= m/ 2 k = m/ 2 K( h, k) I( i h, j k) 3

4 Properties of convolution Let f,g,h be images and * denote convolution f * g = f (x u, y v)g(u, v)dudv Commutative: f*g=g*f Associative: f*(g*h)=(f*g)*h Linear: for scalars a & b and images f,g,h (af+bg)*h=a(f*h)+b(g*h) Differentiation rule f ( f * g) = * g = x x g f * x Gaussian Noise: sigma=1 Gaussian Noise: sigma=16 4

5 An Isotropic Gaussian The plot shows a Gaussian smoothing kernel proportional to e x 2 +y 2 2σ 2 Where σ is usually measured in pixels Gaussian Smoothing original σ = 2 σ = 2.8 σ = 4 5

6 Gaussian Smoothing by Charles Allen Gillbert by Harmon & Julesz Gaussian Smoothing 6

7 Halftone image (binary) Gaussian Blurred Efficient Implementations Both, the BOX filter and the Gaussian filter are separable: First convolve each row with a 1-D filter Then convolve each column with a 1-D filter. = [ 1 1 1] 1 = 1 1 For Gaussian kernels g 1 (x) and g 2 (x), If g 1 & g 2 respectively have variance σ 12 & σ 2 2 Then g 1 *g 2 has variance σ 12 + σ 2 2 K K r c 7

8 Some other useful filtering techniques Median filter Anisotropic diffusion Bilateral filter The Median Filter Method : 1. Take a neighborhood of intensities, and order them. 2. take middle value as output Median filter non-linear filter no new grey levels emerge... 8

9 Median filters: Example for window size of 3 1,1,1,7,1,1,1,1?,1,1,1.1,1,1,? Advantage of this type of filter is that it eliminates spikes (salt & pepper noise). Median filters : example Filter width of 5 9

10 Median filter : images 3 x 3 median filter sharpens edges, destroys edge cusps and protrusions Median filters : Gauss revisited Comparison with Gaussian e.g. upper lip smoother, eye better preserved 10

11 Example of median 10 times 3 X 3 median patchy effect important details lost (e.g. ear-ring) On segmentation 11

12 Edges Intensity discontinuities Corners 12

13 Physical causes of edges 1. Object boundaries 2. Surface normal discontinuities 3. Reflectance (albedo) discontinuities 4. Lighting discontinuities Object Boundaries 13

14 Surface normal discontinuities Boundaries of materials properties 14

15 Boundaries of lighting Profiles of image intensity edges 15

16 Noisy Step Edge Derivative is high everywhere. Must smooth before taking gradient. Edge is Where Change Occurs: 1-D Change is measured by derivative in 1D Ideal Edge Smoothed Edge First Derivative Second Derivative Biggest change, derivative has maximum magnitude Or 2nd derivative is zero. 16

17 Numerical Derivatives f(x) X 0 -h X 0 X 0 +h x Take Taylor series expansion of f(x) about x 0 f(x) = f(x 0 )+f (x 0 )(x-x 0 ) + ½ f (x 0 )(x-x 0 ) 2 + Consider samples taken at increments of h and first two terms of the expansion, we have f(x 0 +h) = f(x 0 )+f (x 0 )h+ ½ f (x 0 )h 2 f(x 0 -h) = f(x 0 )-f (x 0 )h+ ½ f (x 0 )h 2 Subtracting and adding f(x 0 +h) and f(x 0 -h) respectively yields f ( x + h) f ( x h) 0 0 f '( x0) = Convolve with 2h First Derivative: [-1 0 1] f ( x + h) 2 f ( x0) + f ( x h) 0 0 Second Derivative: [1-2 1] f ''( x0) = 2 h Implementing 1-D Edge Detection 1. Filter out noise: convolve with Gaussian 2. Take a derivative: convolve with [-1 0 1] We can combine steps 1 and 2 3. Find the peak: Two issues: Should be a local maximum Should be sufficiently high 17

18 2D Edge Detection: Canny 1. Filter out noise Use a 2D Gaussian Filter. 2. Take a derivative Compute the magnitude of the gradient: J = (J x, J y ) = J x, J is the gradient y J = J x 2 + J y 2 is the magnitude of the gradient tan 1 (J y, J y ) the direction of the gradient J = I *G Next step Can we use the same next step, finding the peak, be the same for 2-D edge detector as we used for 1-D detection?? NO!! 18

19 Smoothing and Differentiation Need two derivatives, in x and y direction. Filter with Gaussian and then compute Gradient, OR Use a derivative of Gaussian filter because differentiation is convolution, and convolution is associative Directional Derivatives σ G x θ σ G y Gσ G cosθ + sinθ x y σ 19

20 Finding derivatives Is this di/dx or di/dy? y x σ = 1 σ = 2 There are three major issues: 1. The gradient magnitude at different scales is different; which scale should we choose? 2. The gradient magnitude is large along a thick trail; how do we identify the significant points? 3. How do we link the relevant points up into curves? 20

21 There is ALWAYS a tradeoff between smoothing and good edge localization! Image with Edge (No Noise) Edge Location Image + Noise Derivatives detect edge and noise Smoothed derivative removes noise, but blurs edge 1 pixel 3 pixels 7 pixels The scale of the smoothing filter affects derivative estimates 21

22 Magnitude of Gradient Tangent Normal: In direction of grad We wish to mark points along the curve where the magnitude is biggest. We can do this by looking for a maximum along a slice normal to the curve (non-maximum suppression). These points should form a curve. There are then two algorithmic issues: which point is the maximum, and where is the next point on the curve? Non-maximum suppression Using normal at q, find two points p and r on adjacent rows (or columns) q is a maximum if is larger than J (r) J ( p) J (q) and Interpolate to get values 22

23 Before Non-max Suppression Gradient magnitude (x4 for visualization) After non-max suppression Gradient magnitude (x4 for visualization) 23

24 Non-maximum suppression Predicting the next edge point The marked point is an edge point. From edge tangent (normal to gradient), predict next point along edge curve (here either r or s) Link together to create edge curve Input image 24

25 Single Threshold T=15 T=5 When threshold is to high, important edges may be missed or be broken When threshold is too low, many extraneous edges, but non missed Hysteresis thresholding: Best of both Hysteresis Thresholding Start tracking an edge chain at pixel location that is local maximum of gradient magnitude where gradient magnitude > τ high. Follow edge in direction orthogonal to gradient. Stop when gradient magnitude < τ low. i.e., use a high threshold to start edge curves and a low threshold to continue them. I τ high τ low Position along edge curve 25

26 Single Threshold T=15 T=5 Hysteresis T high =15 T low = 5 Hysteresis thresholding Canny Edge Detection Algorithm 1. Three parameters σ, τ high, τ low 2. Filter with symmetric Gaussian of width σ 3. Computer gradient, magnitude, direction 4. Non-maximal supression 5. Hysteresis thresholding using τ high, τ low 26

27 fine scale high threshold 27

28 coarse scale, high high threshold coarse scale Low high threshold 28

29 Why is Canny so Dominant Widely used for 30 years. Theory is nice Details are good Magnitude of gradient, Non-max supression Hysteresis thresholding Most subsequent detectors weren t much better until learning-based detectors came along Code was distributed 29