Announcement Edge and Corner Detection Slides are posted HW due Friday CSE5A Lecture 11 Edges Corners Edge is Where Change Occurs: 1-D Change is measured by derivative in 1D Numerical Derivatives f(x) Ideal Edge Smoothed Edge First Derivative Second Derivative Biggest change, derivative has maximum magnitude Or nd derivative is zero. X -h X X +h Take Taylor series expansion of f(x) about x f(x) = f(x )+f (x )(x-x ) + ½ f (x )(x-x ) + Consider samples taken at increments of h and first two terms of the expansion, we have f(x +h) = f(x )+f (x )h+ ½ f (x )h f(x -h) = f(x )-f (x )h+ ½ f (x )h Subtracting and adding f(x +h) and f(x -h) respectively yields f ( x + h) f ( x h) f '( x) = Convolve with h f ( x + h) f ( x) + f ( x h) f ''( x) = h x First Derivative: [-1 1] Second Derivative: [1-1] 1
Implementing 1-D Edge Detection 1. Filter out noise: convolve with Gaussian. Take a derivative: convolve with [-1 1] We can combine 1 and. 3. Find the peak: Two issues: Should be a local maximum. Should be sufficiently high. D Edge Detection: Canny 1. Filter out noise Use a D Gaussian Filter. Compute the derivative (gradient in D) J = G * I J = (J x, J y ) = J x, J is the Gradient y 3. Compute the magnitude of the gradient: J = J x + J y Next step Can next step, be the same for -D edge detector as in 1-D detector? Finding the maximum? NO!! Smoothing and Differentiation Need two derivatives, in x and y direction. Filter with Gaussian and then compute Gradient, OR Use a derivative of Gaussian filter because differentiation is convolution, and convolution is associative The maxima of a function are isolated points, not curves. σ G x Directional Derivatives θ Note: derivative can be positive or negative Finding derivatives Is this di/dx or di/dy? σ G y Gσ G cosθ + sinθ x y σ y x Image Derivative
Magnitude of the Gradient There is ALWAYS a tradeoff between smoothing and good edge localization! There are three major issues: 1. The gradient magnitude at different scales is different; which scale should we choose?. The gradient magnitude is large along a thick trail; how do we identify the significant points? 3. How do we link the relevant points up into curves? σ = 1 σ = Image + Noise Image with Edge (No Noise) Derivatives detect edge and noise Edge Location Smoothed derivative removes noise, but blurs edge Magnitude of Gradient Tangent Normal: In direction of grad σ = 1 pixel σ = 3 pixels σ = 7 pixels The scale of the smoothing filter affects derivative estimates We wish to mark points along the curve where the magnitude is biggest. We can do this by looking for a maximum along a slice normal to the curve (non-maximum suppression). These points should form a curve. There are then two algorithmic issues: which point is the maximum, and where is the next point on the curve? Non-maximum suppression Non-maximum suppression For every pixel in the image (e.g., q) we have an estimate of edge direction and edge normal (shown at q) Using normal at q, find two points p and r on adjacent rows (or columns). We have a maximum if the value is larger than those at both p and at r. Interpolate to get values. 3
Non-maximum suppression Predicting the next edge point Assume the marked point is an edge point. Then we construct the tangent to the edge curve (which is normal to the gradient at that point) and use this to predict the next points (here either r or s). Link together to create Image curve I HysteresisThresholding Start tracking an edge chain at pixel location that is local maximum of gradient magnitude where gradient magnitude > τ high. Follow edge in direction orthogonal to gradient. Stop when gradient magnitude < τ low. i.e., use a high threshold to start edge curves and a low threshold to continue them. τ high τ low Position along edge curve Input image Single Threshold T=15 T=5 Hysteresis T h =15 T l = 5 Hysteresis thresholding fine scale high threshold 4
coarse scale, high high threshold coarse scale Low high threshold Why is Canny so Dominant Widely used for 5 years. 1. Theory is nice (but most other detectors weren t better until learning-based detectors came along).. Details good (magnitude of gradient, non-max supression). 3. Hysteresis an important heuristic. 4. Code was distributed. Learning-based detectors: Not edges, but boundaries Brightness Color Texture Boundary detection Corner Detection Precision is the fraction of detections that are true positives rather than false positives, while recall is the fraction of true positives that are detected rather than missed. From Contours to Regions: An Empirical Evaluation, Arbelaez, M. Maire C. Fowlkes, and J. Malik, CVPR 8 5
Feature extraction: Corners and blobs Why extract features? Motivation: panorama stitching We have two images how do we combine them? Why extract features? Why extract features? Motivation: panorama stitching We have two images how do we combine them? Motivation: panorama stitching We have two images how do we combine them? Step 1: extract features Step : match features Step 1: extract features Step : match features Step 3: align images The Basic Idea We should easily recognize the point by looking through a small window Shifting a window in any direction should give a large change in intensity Edge Detectors Tend to Fail at Corners Source: A. Efros flat region: no change in all directions edge : no change along the edge direction corner : significant change in all directions Intuition: Right at corner, gradient is ill-defined. Near corner, gradient has two different values. 6
Distribution of gradients for different image patches Formula for Finding Corners Shi-Tomasi Detector We look at matrix: Sum over a small region Gradient with respect to x, times gradient with respect to y # I x I x I & y C(x, y) = % ( $ % I x I y I y '( Matrix is symmetric WHY THIS? General Case: What is region like if: 1. λ 1 =?. λ =? Because C is a symmetric positive definite matrix, it can be factored as follows: λ C = R 1 1 R λ 3. λ 1 = and λ =? 4. λ 1 > and λ >? Where R is a x rotation matrix and λ i is non-negative. Shi-Tomasi Corner Detector Filter image with a Gaussian. Compute the gradient everywhere. Move window over image and construct C over the window. # I x I x I & C(x, y) = % y ( $ % I x I y I y '( Use linear algebra to find λ 1 and λ. If they are both large, we have a corner. 1. Let e(x,y) = min(λ 1 (x,y), λ (x,y)). (x,y) is a corner if it s local maximum of e(x,y) and e(x,y) > τ Parameters: Gaussian std. dev, window size, threshold Corner Detection Sample Results Threshold=5, Threshold=1, Threshold=5, 7