Fitting: Voting and the Hough Transform

Transcription

1 Last time: Grouping Fitting: Voting and the Hough Transform Monda, Fe 14 Prof. UT Austin Bottom-up segmentation via clustering To find mid-level regions, tokens General choices -- features, affinit functions, and clustering algorithms Eample clustering algorithms Mean shift and mode finding: K-means, Mean shift Graph theoretic: Graph cut, normalized cuts Grouping also useful for quantization Teton histograms for teture within local region Recall: Images as graphs Goal: Segmentation Graph Cuts q p w pq w w A B C Full-connected graph node for ever piel link etween ever pair of piels, p,q similarit w pq for each link» similarit is inversel proportional to difference in color and position Break graph into segments Delete links that cross etween segments Easiest to reak links that have low similarit similar piels should e in the same segments dissimilar piels should e in different segments Slide Steve Seitz Points 1 10 Last time: Measuring affinit 40 data points affinit matri A Eample: weighted graphs Points , ep 2 2 } Feature dimension 2 Suppose we have a 4-piel image (i.e., a 2 2 matri) Each piel descried 2 features 1. What do the locks signif? 2. What does the smmetr of the matri signif? 3. How would the matri change with larger value of σ? Feature dimension 1 Dimension of data points : d = 2 Numer of data points : N = 4 CS 376 Lecture 8 Fitting 1

2 Eample: weighted graphs Computing the distance matri: 0.01 D(1,:)= (0) Eample: weighted graphs Computing the distance matri: D(1,:)= (0) (0) for i=1:n for j=1:n D(i,j) = i - j 2 for i=1:n for j=1:n D(i,j) = i - j 2 Eample: weighted graphs Computing the distance matri: Eample: weighted graphs D Distancesaffinities A for i=1:n for j=1:n D(i,j) = i - j 2 N N matri for i=1:n for j=1:n D(i,j) = i - j 2 for i=1:n for j=i+1:n A(i,j) = ep(-1/(2*σ^2)* i - j 2 ); A(j,i) = A(i,j); Scale parameter σ affects affinit Distance matri D= Affinit matri with increasing σ: Visualizing a shuffled affinit matri If we permute the order of the vertices as the are referred to in the affinit matri, we see different patterns: CS 376 Lecture 8 Fitting 2

3 Putting these two aspects together Cuts in a graph: Min cut Points 1 10 Data points A B Affinit matrices σ=.1 σ=.2 σ=1 Points 31 40, ep 2 2 } Link Cut set of links whose removal makes a graph disconnected cost of a cut: cut( A, B) Find minimum cut gives ou a segmentation fast algorithms eist Weakness of Min cut w p, q pa, qb Source: Steve Seitz Cuts in a graph: Normalized cut Eample results: segments from Ncuts A B Fi ias of Min Cut normalizing for size of segments: cut( A, B) cut( A, B) assoc( A, V ) assoc( B, V ) assoc(a,v) = sum of weights of all edges that touch A Ncut value is small when we get two clusters with man edges with high weights, and few edges of low weight etween them. Approimate solution: generalized eigenvalue prolem. J. Shi and J. Malik, Normalized Cuts and Image Segmentation, CVPR, 1997 Steve Seitz Normalized cuts: pros and cons Segments as primitives for recognition Multiple segmentations Pros: Generic framework, fleile to choice of function that computes weights ( affinities ) etween nodes Does not require model of the data distriution Cons: Time compleit can e high Dense, highl connected graphs man affinit computations Solving eigenvalue prolem Preference for alanced partitions B. Russell et al., Using Multiple Segmentations to Discover Ojects and their Etent in Image Collections, CVPR 2006 Slide credit: Lana Lazenik CS 376 Lecture 8 Fitting 3

4 Motion segmentation Now: Fitting Want to associate a model with oserved features Input sequence Image Segmentation Motion Segmentation Input sequence Image Segmentation Motion Segmentation A. Baru, S.C. Zhu. Generalizing Swsen-Wang to sampling aritrar posterior proailities, IEEE Trans. PAMI, August [Fig from Marszalek & Schmid, 2007] For eample, the model could e a line, a circle, or an aritrar shape. Fitting: Main idea Choose a parametric model to represent a set of features Memership criterion is not local Can t tell whether a point elongs to a given model just looking at that point Three main questions: What model represents this set of features est? Which of several model instances gets which feature? How man model instances are there? Computational compleit is important It is infeasile to eamine ever possile set of parameters and ever possile comination of features Eample: Line fitting Wh fit lines? Man ojects characterized presence of straight lines Wait, wh aren t we done just running edge detection? Slide credit: L. Lazenik Difficult of line fitting Etra edge points (clutter), multiple models: which points go with which line, if an? Onl some parts of each line detected, and some parts are missing: how to find a line that ridges missing evidence? Noise in measured edge points, orientations: how to detect true underling parameters? Voting It s not feasile to check all cominations of features fitting a model to each possile suset. Voting is a general technique where we let the features vote for all models that are compatile with it. Ccle through features, cast votes for model parameters. Look for model parameters that receive a lot of votes. Noise & clutter features will cast votes too, ut tpicall their votes should e inconsistent with the majorit of good features. CS 376 Lecture 8 Fitting 4

5 Fitting lines: Hough transform Given points that elong to a line, what is the line? How man lines are there? Which points elong to which lines? Hough Transform is a voting technique that can e used to answer all of these questions. Main idea: 1. Record vote for each possile line on which each edge point lies. 2. Look for lines that get man votes. Finding lines in an image: Hough space image space m Hough (parameter) space Connection etween image (,) and Hough (m,) spaces A line in the image corresponds to a point in Hough space To go from image space to Hough space: given a set of points (,), find all (m,) such that = m + 0 m 0 Slide credit: Steve Seitz Finding lines in an image: Hough space Finding lines in an image: Hough space 0 0 ( 0, 0 ) ( 1, 1 ) = 1 m image space m Hough (parameter) space 0 image space m Hough (parameter) space Connection etween image (,) and Hough (m,) spaces A line in the image corresponds to a point in Hough space To go from image space to Hough space: given a set of points (,), find all (m,) such that = m + What does a point ( 0, 0 ) in the image space map to? What are the line parameters for the line that contains oth ( 0, 0 ) and ( 1, 1 )? It is the intersection of the lines = 0 m + 0 and = 1 m + 1 Answer: the solutions of = - 0 m + 0 this is a line in Hough space Slide credit: Steve Seitz Finding lines in an image: Hough algorithm image space m Hough (parameter) space How can we use this to find the most likel parameters (m,) for the most prominent line in the image space? Let each edge point in image space vote for a set of possile parameters in Hough space Accumulate votes in discrete set of ins; parameters with the most votes indicate line in image space. Image rows Polar representation for lines Issues with usual (m,) parameter space: can take on infinite values, undefined for vertical lines. [0,0] d Image columns : perpicular distance from line to origin : angle the perpicular makes with the -ais cos sin d Point in image space sinusoid segment in Hough space d CS 376 Lecture 8 Fitting 5

6 Hough line demo a/hough.html Hough transform algorithm Using the polar parameterization: cos sin d H: accumulator arra (votes) Basic Hough transform algorithm d 1. Initialize H[d, ]=0 2. for each edge point I[,] in the image for = [ min to ma ] // some quantization d cos sin H[d, ] += 1 3. Find the value(s) of (d, ) where H[d, ] is maimum 4. The detected line in the image is given d cos sin Time compleit (in terms of numer of votes per pt)? Source: Steve Seitz Original image Cann edges Vote space and top peaks Showing longest segments found Impact of noise on Hough Impact of noise on Hough d edge coordinates Votes What difficult does this present for an implementation? edge coordinates Votes Here, everthing appears to e noise, or random edge points, ut we still see peaks in the vote space. CS 376 Lecture 8 Fitting 6

7 Etensions Etension 1: Use the image gradient 1. same 2. for each edge point I[,] in the image = gradient at (,) d cos sin H[d, ] += 1 3. same 4. same (Reduces degrees of freedom) Etension 2 give more votes for stronger edges Etension 3 change the sampling of (d, ) to give more/less resolution Etension 4 The same procedure can e used with circles, squares, or an other shape Etensions Etension 1: Use the image gradient 1. same 2. for each edge point I[,] in the image compute unique (d, ) ased on image gradient at (,) H[d, ] += 1 3. same 4. same (Reduces degrees of freedom) Etension 2 give more votes for stronger edges (use magnitude of gradient) Etension 3 change the sampling of (d, ) to give more/less resolution Etension 4 The same procedure can e used with circles, squares, or an other shape Source: Steve Seitz Circle: center (a,) and radius r ( i a) ( i ) r For a fied radius r, unknown gradient direction Circle: center (a,) and radius r ( i a) ( i ) r For a fied radius r, unknown gradient direction Intersection: most votes for center occur here. Hough space a Hough space Circle: center (a,) and radius r ( i a) ( i ) r For an unknown radius r, unknown gradient direction r Circle: center (a,) and radius r ( i a) ( i ) r For an unknown radius r, unknown gradient direction r? a Hough space a Hough space CS 376 Lecture 8 Fitting 7

8 Circle: center (a,) and radius r ( i a) ( i ) r For an unknown radius r, known gradient direction Hough space For ever edge piel (,) : For each possile radius value r: For each possile gradient direction : // or use estimated gradient at (,) a = r cos() // column = + r sin() // row H[a,,r] += 1 Time compleit per edgel? Check out online demo : Eample: detecting circles with Hough Eample: detecting circles with Hough Original Edges Votes: Penn Comined Original detections Edges Votes: Quarter Note: a different Hough transform (with separate accumulators) was used for each circle radius (quarters vs. penn). Coin finding sample images from: Vivek Kwatra Eample: iris detection Eample: iris detection Gradient+threshold Hough space (fied radius) Ma detections Hemerson Pistori and Eduardo Rocha Costa An Iris Detection Method Using the Hough Transform and Its Evaluation for Facial and Ee Movement, Hideki Kashima, Hitoshi Hongo, Kunihito Kato, Kazuhiko Yamamoto, ACCV CS 376 Lecture 8 Fitting 8

9 Voting: practical tips Minimize irrelevant tokens first Choose a good grid / discretization Too fine? Too coarse Vote for neighors, also (smoothing in accumulator arra) Use direction of edge to reduce parameters 1 To read ack which points voted for winning peaks, keep tags on the votes. Hough transform: pros and cons Pros All points are processed indepentl, so can cope with occlusion, gaps Some roustness to noise: noise points unlikel to contriute consistentl to an single in Can detect multiple instances of a model in a single pass Cons Compleit of search time increases eponentiall with the numer of model parameters Non-target shapes can produce spurious peaks in parameter space Quantization: can e trick to pick a good grid size Generalized Hough Transform What if we want to detect aritrar shapes? Intuition: Generalized Hough Transform Define a model shape its oundar points and a reference point. Offline procedure: Displacement vectors p 1 a p 2 At each oundar point, compute displacement vector: r = a p i. Ref. point Model image Novel image Now suppose those colors encode gradient directions Vote space Model shape Store these vectors in a tale indeed gradient orientation. [Dana H. Ballard, Generalizing the Hough Transform to Detect Aritrar Shapes, 1980] Generalized Hough Transform Detection procedure: For each edge point: Use its gradient orientation to inde into stored tale Use retrieved r vectors to vote for reference point Assuming translation is the onl transformation here, i.e., orientation and scale are fied. p 1 Novel image Generalized Hough for oject detection Instead of indeing displacements gradient orientation, inde matched local patterns. training image visual codeword with displacement vectors B. Leie, A. Leonardis, and B. Schiele, Comined Oject Categorization and Segmentation with an Implicit Shape Model, ECCV Workshop on Statistical Learning in Computer Vision 2004 Source: L. Lazenik CS 376 Lecture 8 Fitting 9

10 Generalized Hough for oject detection Instead of indeing displacements gradient orientation, inde visual codeword Summar Grouping/segmentation useful to make a compact representation and merge similar features associate features ased on defined similarit measure and clustering ojective Fitting prolems require finding an supporting evidence for a model, even within clutter and missing features. associate features with an eplicit model test image B. Leie, A. Leonardis, and B. Schiele, Comined Oject Categorization and Segmentation with an Implicit Shape Model, ECCV Workshop on Statistical Learning in Computer Vision 2004 Source: L. Lazenik Voting approaches, such as the Hough transform, make it possile to find likel model parameters without searching all cominations of features. Hough transform approach for lines, circles,, aritrar shapes defined a set of oundar points, recognition from patches. CS 376 Lecture 8 Fitting 10