CS4670: Computer Vision Noah Snavely Lecture 2: Edge detection From Sandlot Science Announcements Project 1 released, due Friday, September 7 1
Edge detection Convert a 2D image into a set of curves Extracts salient features of the scene More compact than pixels Origin of Edges surface normal discontinuity depth discontinuity surface color discontinuity illumination discontinuity Edges are caused by a variety of factors 2
Images as functions Edges look like steep cliffs Characterizing edges An edge is a place of rapid change in the image intensity function image intensity function (along horizontal scanline) first derivative Source: L. Lazebnik edges correspond to extrema of derivative
Image derivatives How can we differentiate a digital image F[x,y]? Option 1: reconstruct a continuous image, f, then compute the derivative Option 2: take discrete derivative (finite difference) How would you implement this as a linear filter? : 1-1 : -1 1 Source: S. Seitz The gradient of an image: Image gradient The gradient points in the direction of most rapid increase in intensity The edge strength is given by the gradient magnitude: The gradient direction is given by: how does this relate to the direction of the edge? Source: Steve Seitz 4
Image gradient Source: L. Lazebnik Effects of noise Noisy input image Where is the edge? Source: S. Seitz 5
Solution: smooth first f h f * h To find edges, look for peaks in Source: S. Seitz Associative property of convolution Differentiation is convolution, and convolution is associative: This saves us one operation: f Source: S. Seitz 6
2D edge detection filters Gaussian derivative of Gaussian (x) Derivative of Gaussian filter x-direction y-direction 7
The Sobel operator Common approximation of derivative of Gaussian -1 0 1-2 0 2-1 0 1 1 2 1 0 0 0-1 -2-1 The standard defn. of the Sobel operator omits the 1/8 term doesn t make a difference for edge detection the 1/8 term is needed to get the right gradient value Sobel operator: example Source: Wikipedia 8
Example original image (Lena) Finding edges gradient magnitude 9
Finding edges where is the edge? thresholding Non-maximum supression Check if pixel is local maximum along gradient direction requires interpolating pixels p and r 10
Finding edges thresholding Finding edges thinning (non-maximum suppression) 11
Canny edge detector MATLAB: edge(image, canny ) 1. Filter image with derivative of Gaussian 2. Find magnitude and orientation of gradient. Non-maximum suppression 4. Linking and thresholding (hysteresis): Define two thresholds: low and high Use the high threshold to start edge curves and the low threshold to continue them Source: D. Lowe, L. Fei-Fei Canny edge detector Still one of the most widely used edge detectors in computer vision J. Canny, A Computational Approach To Edge Detection, IEEE Trans. Pattern Analysis and Machine Intelligence, 8:679-714, 1986. Depends on several parameters: : width of the Gaussian blur high threshold low threshold 12
Canny edge detector original Canny with Canny with The choice of depends on desired behavior large detects large-scale edges small detects fine edges Source: S. Seitz Scale space (Witkin 8) first derivative peaks larger Gaussian filtered signal Properties of scale space (w/ Gaussian smoothing) edge position may shift with increasing scale ( ) two edges may merge with increasing scale an edge may not split into two with increasing scale 1
Questions? Image Scissors Aging Helen Mirren Today s Readings Intelligent Scissors, Mortensen et. al, SIGGRAPH 1995 14
Extracting objects How could this be done? hard to do manually hard to do automatically ( image segmentation ) pretty easy to do semi-automatically Intelligent Scissors (demo) 15
Intelligent Scissors Approach answers basic question Q: how to find a path from seed to mouse that follows object boundary as closely as possible? A: define a path that stays as close as possible to edges Basic Idea Intelligent Scissors Define edge score for each pixel edge pixels have low cost Find lowest cost path from seed to mouse mouse Questions How to define costs? How to find the path? seed 16
Let s look at this more closely Treat the image as a graph q p c Graph node for every pixel p link between every adjacent pair of pixels, p,q cost c for each link Note: each link has a cost this is a little different than the figure before where each pixel had a cost Defining the costs s q p c r Want to hug image edges: how to define cost of a link? good (low-cost) links follow the intensity edge want intensity to change rapidly to the link c intensity of r intensity of s 17
Defining the costs s q p c r c can be computed using a cross-correlation filter assume it is centered at p Defining the costs s q 1 1 w p c r 1-1 -1-1 c can be computed using a cross-correlation filter assume it is centered at p A couple more modifications Scale the filter response by length of link c. Why? Make c positive Set c = (max- filter response *length) where max = maximum filter response *length over all pixels in the image 18
Dijkstra s shortest path algorithm 4 9 1 0 2 5 link cost Algorithm 1. init node costs to, set p = seed point, cost(p) = 0 2. expand p as follows: for each of p s neighbors q that are not expanded set cost(q) = min( cost(p) + c pq, cost(q) ) Dijkstra s shortest path algorithm 4 9 5 4 9 1 1 0 2 5 Algorithm 2 1. init node costs to, set p = seed point, cost(p) = 0 2. expand p as follows: for each of p s neighbors q that are not expanded set cost(q) = min( cost(p) + c pq, cost(q) ) if q s cost changed, make q point back to p put q on the ACTIVE list (if not already there) 19
Dijkstra s shortest path algorithm 4 9 5 2 2 9 5 4 1 1 0 4 2 5 Algorithm 2 1. init node costs to, set p = seed point, cost(p) = 0 2. expand p as follows: for each of p s neighbors q that are not expanded set cost(q) = min( cost(p) + c pq, cost(q) ) if q s cost changed, make q point back to p put q on the ACTIVE list (if not already there). set r = node with minimum cost on the ACTIVE list 4. repeat Step 2 for p = r Dijkstra s shortest path algorithm 4 6 2 9 5 4 2 1 1 0 4 2 5 5 4 2 Algorithm 1. init node costs to, set p = seed point, cost(p) = 0 2. expand p as follows: for each of p s neighbors q that are not expanded set cost(q) = min( cost(p) + c pq, cost(q) ) if q s cost changed, make q point back to p put q on the ACTIVE list (if not already there). set r = node with minimum cost on the ACTIVE list 4. repeat Step 2 for p = r 20
Dijkstra s shortest path algorithm Properties It computes the minimum cost path from the seed to every node in the graph. This set of minimum paths is represented as a tree Running time, with N pixels: O(N 2 ) time if you use an active list O(N log N) if you use an active priority queue (heap) takes fraction of a second for a typical (640x480) image Once this tree is computed once, we can extract the optimal path from any point to the seed in O(N) time. it runs in real time as the mouse moves What happens when the user specifies a new seed? Example Result Peter Davis 21
Questions? 22