Announcements Assignment 2 due Tuesday, May 4. Edge Detection, Lines Midterm: Thursday, May 6. Introduction to Computer Vision CSE 152 Lecture 10 Edges Last Lecture 1. Object boundaries 2. Surface normal discontinuities 3. Reflectance (albedo) discontinuities 4. Lighting discontinuities (shadow boundaries) Numerical Derivatives f(x) I / x I = I / y Gradients: Is this di/dx or di/dy? X 0 -h X 0 X 0 +h x Take Taylor series expansion of f(x) about x 0 f(x) = f(x 0 )+f (x 0 )(x-x 0 ) + ½ f (x 0 )(x-x 0 ) 2 + Consider Samples taken at increments of h and first two terms, we have f(x 0 +h) = f(x 0 )+f (x 0 )h+ ½ f (x 0 )h 2 f(x 0 -h) = f(x 0 )-f (x 0 )h+ ½ f (x 0 )h 2 Subtracting and adding f(x 0 +h) and f(x 0 -h) respectively yields y x 1
Corner and Edge Detection Smoothing and Differentiation Need two derivatives, in x and y direction. Filter with Gaussian and then compute gradient, or Use a derivative of Gaussian filter because differentiation is convolution, and convolution is associative Corners: The Basic Idea We should easily identify corners by looking through a small window. At corners, gradients point in many directions Source: A. Efros Formula for Finding Corners We look at matrix: Sum over a small region Gradient with respect to x, times gradient with respect to y 2 I x I x I y C(x, y) = 2 I x I y I y General Case Since C is symmetric, we have C = R λ 1 1 0 R 0 λ 2 Where R is a 2x2 rotation matrix that can be parameterized by an angle θ. We can visualize C as an ellipse with axis lengths determined by the eigenvalues and orientation determined by R Ellipse equation: direction of the fastest change (λ max ) -1/2 (λ min ) -1/2 direction of the slowest change Matrix is symmetric WHY THIS? For intuition, consider case where R is identity matrix : What is region like if: 1. λ 1 = 0 and λ 2 = 0? 2. λ 1 = 0? 3. λ 2 = 0? 4. λ 1 > 0 and λ 2 > 0? Corner Detection Algorithm Filter image with a Gaussian. Compute the gradient everywhere. Move window over image and construct C over the window. Use linear algebra to find λ 1 and λ 2. If they are both big, we have a corner. 1. Let e(x,y) = min(λ 1 (x,y), λ 2 (x,y)) 2. (x,y) is a corner if it s local maximum of e(x,y) and e(x,y) > τ Parameters: Gaussian std. dev, window size, threshold 2
Corner Detection Sample Results Threshold=25,000 Threshold=10,000 Back to Edges Threshold=5,000 Edge is Where Change Occurs: 1-D Change is measured by derivative in 1D Ideal Edge Smoothed Edge First Derivative 2D Edge Detection: Canny 1. Filter out noise Use a 2D Gaussian Filter. 2. Take a derivative Compute the magnitude of the gradient: Second Derivative Biggest change, derivative has maximum magnitude Or 2nd derivative is zero. There is ALWAYS a tradeoff between smoothing and good edge localization! σ = 1 σ = 2 Image with Edge Gradient Magnitude: Edge Location There are three major issues: 1. The gradient magnitude at different scales is different; which scale should we choose? 2. The gradient magnitude is large along thick trail; how do we identify the significant points? 3. How do we link the relevant points up into curves? Image + Noise Derivatives detect edge and noise Smoothed derivative removes noise, but blurs edge 3
Magnitude of Gradient Non-maximum suppression Normal: In direction of grad Tangent For every pixel in the image (e.g., q) we have an estimate of edge direction and edge normal (shown at q) We wish to mark points along the curve where the magnitude is biggest. We can do this by looking for a maximum along a slice normal to the curve (non-maximum suppression). These points should form a curve. There are then two algorithmic issues: which point is the maximum, and where is the next point on the curve? Non-maximum suppression Non-maximum suppression Predicting the next edge point Using normal at q, find two points p and r on adjacent rows (or columns). We have a maximum if the value at q is larger than those at both p and at r. Interpolate to get values. Assume the marked point is an edge point. Then we construct the tangent to the edge curve (which is normal to the gradient at that point) and use this to predict the next points (here either r or s). HysteresisThresholding Input image Start tracking an edge chain at pixel location that is local maximum of gradient magnitude where gradient magnitude > τ high. Follow edge in direction orthogonal to gradient. Stop when gradient magnitude < τ low. i.e., use a high threshold to start edge curves and a low gradient magnitude τ high threshold to continue them. τ low Position along edge curve 4
Single Threshold T=15 T=5 Hysteresis T h =15 T l = 5 Hysteresis thresholding fine scale coarse scale, High thresholds high thresholds Why is Canny so Dominant coarse scale Lower high thresholds Still widely used after 25 years. 1. Theory is nice. 2. Details good (magnitude of gradient, non-max suppression). 3. Hysteresis tresholding an important heuristic. 4. Code was distributed. 5
What to do with edges? Segment linked edge chains into curve features (e.g., line segments). Group unlinked or unrelated edges into lines (or curves in general). Hough Transform [ Patented 1962 ] Accurately fitting parametric curves (e.g., lines) to grouped edge points. Finding lines in an image Option 1: Search for the line at every possible position/orientation What is the cost of this operation? Option 2: Use a voting scheme: Hough transform Finding lines in an image y b (m 0,b 0 ) x m image space Hough space Connection between image (x,y) and Hough (m,b) spaces A line in the image corresponds to a point in Hough space To go from image space to Hough space: given a set of points (x,y), find all (m,b) such that y = mx + b Finding lines in an image y b (x 0,y 0 ) Hough transform algorithm Because vertical lines can t be represented, Typically use a different parameterization x image space m Hough space d is the perpendicular distance from the line to the origin θ is the angle this perpendicular makes with the x axis Why? Connection between image (x,y) and Hough (m,b) spaces A line in the image corresponds to a point in Hough space To go from image space to Hough space: given a set of points (x,y), find all (m,b) such that y = mx + b What does a point (x 0, y 0 ) in the image space map to? A: the solutions of b = -x 0 m + y 0 this is a line in Hough space 6
Hough transform algorithm Basic Hough transform algorithm 1. Initialize H[d, θ]=0 ; H is called accumulator array 2. for each edge point I[x,y] in the image for θ = 0 to 180 H[d, θ] += 1 3. Find the value(s) of (d, θ) where H[d, θ] is maximum 4. The detected line in the image is given by What s the running time (measured in # votes)? Extensions Extension 1: Use the image gradient 1. same 2. for each edge point I[x,y] in the image compute unique (d, θ) based on image gradient at (x,y) H[d, θ] += 1 3. same 4. same What s the running time measured in votes? Extension 2 give more votes for stronger edges Extension 3 change the sampling of (d, θ) to give more/less resolution Extension 4 The same procedure can be used with circles, squares, or any other shape Hough Transform: 20 colinear points Hough Transform: Noisy line R, θ representation of line Maximum accumulator value is 20 R, θ representation of line Maximum accumulator value is 6 Hough Transform: Random points Mechanics of the Hough transform Difficulties how big should the cells be? (too big, and we cannot distinguish between quite different lines; too small, and noise causes lines to be missed) How many lines? count the peaks in the Hough array R, θ representation of line Maximum accumulator value is 4 Which edgels belongs to which line? tag the votes Complications, problems with noise and cell size 7
Number of votes that the real line of 20 points gets with increasing noise As the noise increases in a picture without a line, the number of points in the max cell goes up 8