Fitting. Instructor: Jason Corso (jjcorso)! web.eecs.umich.edu/~jjcorso/t/598f14!! EECS Fall 2014! Foundations of Computer Vision!

Transcription

1 Fitting EECS Fall 2014! Foundations of Computer Vision!! Instructor: Jason Corso (jjcorso)! web.eecs.umich.edu/~jjcorso/t/598f14!! Readings: FP 10; SZ 4.3, 5.1! Date: 10/8/14!! Materials on these slides have come from many sources in addition to myself; individual slides reference specific sources.!

2 Plan 2 Problem Formulation! Least Squares Methods! RANSAC! Hough Transform! Multi-model Fitting! Expectation-Maximization! Examples of Uses of Fitting!

3 What is Fitting? 3 Goals:! Choose a parametric model to fit a certain quantity from data! Estimate model parameters! - Lines! - Curves! - Homographic transformation! - Fundamental matrix! - Shape model!

4 4 Example: fitting lines! (for computing vanishing points)!

5 Example: Estimating an homographic!!! transformation! 5 H!

6 Example: Estimating F! 6

7 Example: fitting a 2D shape template! 7 A

8 Example: fitting a 3D object model! 8

9 Fitting 9 Critical issues:! Noisy data! Outliers! Missing data!

10 10 Critical issues: noisy data!

11 Critical issues: noisy data! (intra-class variability)! 11 A

12 12 Critical issues: outliers! H!

13 Critical issues: missing data! 13 (occlusions)!

14 Fitting! 14 Goal: Choose a parametric model to! fit a certain quantity from data! Techniques:! Least square methods! RANSAC! Hough transform! EM (Expectation Maximization)!

15 Least squares methods - fitting a line - Data: (x 1, y 1 ),, (x n, y n ) 15 Line equation: y i = m x i + b y=mx+b Find (m, b) to minimize! (x i, y i ) E = n i = 1 ( y i mx i b) 2

16 Least squares methods - fitting a line - 16 E = n i = 1 ( y i mx i b) 2

17 Least squares methods - fitting a line - [ ] 2 2 n 1 n 1 n 1 i 2 i i XB Y b m 1 x 1 x y y b m 1 x y E = = = =!!! = + = XB X Y X db de T T Normal equation = = n i i i b mx y E 1 2 ) ( Y X XB X T T = ( ) Y X X X B T T 1 = (XB) (XB) Y 2(XB) Y Y XB) (Y XB) (Y T T T T + = = Find (m, b) that minimize E! 17

18 Least squares methods - fitting a line - 18 E = n i = 1 (yi mxi b) 2 y=mx+b B ( T ) 1 T m = X X X Y B = b (x i, y i ) Limitations! Fails completely for vertical lines!

19 Least squares methods - fitting a line - 19 Distance between point! (x n, y n ) and line ax+by=d ax+by=d Find (a, b, d) to minimize the sum of squared perpendicular distances! E = n i = 1 ( ax i + by i d 2 ) (x i, y i ) U N = 0 data! model parameters!

20 Least squares methods - fitting a line - 20 A h = 0 Minimize A h subject to h = 1 A = UDV T h = last column of V

21 Least squares methods -- fitting an homography y! H! y! x! x! U N = 0 data! model parameters!

22 Least squares: Robustness to noise 22

23 Least squares: Robustness to noise 23 outlier!!

24 24 Critical issues: outliers! H! CONCLUSION: Least square is not robust w.r.t. outliers!

25 Least squares: Robust estimators!! Instead of minimizing!! u i = error (residual) of i th point w.r.t. model parameters β = (a,b,d) ρ = robust function of u i with scale parameter σ!! We minimize! E = E i = ρ! n i = 1 ( u ) ρ ;σ i ( ax i u i + b y = i ax i d 2 ) + b y i d The robust function ρ! Favors a configuration! with small residuals! Penalizes large residuals" 25 u!

26 Least squares: Robust estimators!! Instead of minimizing!! u i = error (residual) of i th point w.r.t. model parameters β = (a,b,d) ρ = robust function of u i with scale parameter σ!! We minimize! E = E i = n i = 1 ( u ) ρ ;σ i ( ax i u i + b y = i ax i d 2 ) + b y i d 26 The robust function ρ! Favors a configuration! with small residuals! Penalizes large residuals"

27 Least squares: Robust estimators 27 Good scale parameter σ! The effect of the outlier is eliminated!

28 Least squares: Robust estimators 28 Bad scale parameter σ (too small!)! Fits only locally! Sensitive to initial condition!!

29 Least squares: Robust estimators 29 Bad scale parameter σ (too large!)!! Same as standard LSQ!! CONCLUSION: Robust estimator useful if prior info about the distribution of points is known! Robust fitting is a nonlinear optimization problem (iterative solution)! Least squares solution provides good initial condition!

30 Fitting! 30 Goal: Choose a parametric model to! fit a certain quantity from data! Techniques:! Least square methods! RANSAC! Hough transform! EM (Expectation Maximization)!!

31 Basic philosophy (voting scheme) 31 Data elements are used to vote for one (or multiple) models! Robust to outliers and missing data! Assumption 1: Noisy features will not vote consistently for any single model ( few outliers)!! Assumption 2: there are enough features to agree on a good model ( few missing data)!

32 RANSAC (RANdom SAmple Consensus) :! Learning technique to estimate! parameters of a model by random! sampling of observed data! Fischler & Bolles in 81.! 32 δ f { P O} π : I, such that:! (P, β ) < δ min π O f ( P, β) = β Model parameters! ( T ) 1 T P P P

33 RANSAC 33 Sample set = set of points in 2D! Algorithm:!! 1. Select random sample of minimum required size to fit model! 2. Compute a putative model from sample set! 3. Compute the set of inliers to this model from whole data set! Repeat 1-3 until model with the most inliers over all samples is found!

34 RANSAC 34 Sample set = set of points in 2D! Algorithm:!! 1. Select random sample of minimum required size to fit model [?]! 2. Compute a putative model from sample set! 3. Compute the set of inliers to this model from whole data set! Repeat 1-3 until model with the most inliers over all samples is found!

35 RANSAC 35 Sample set = set of points in 2D! Algorithm:!! 1. Select random sample of minimum required size to fit model [?]! 2. Compute a putative model from sample set! 3. Compute the set of inliers to this model from whole data set! Repeat 1-3 until model with the most inliers over all samples is found!

36 RANSAC 36 δ Sample set = set of points in 2D! O = 14! Algorithm:!! 1. Select random sample of minimum required size to fit model [?]! 2. Compute a putative model from sample set! 3. Compute the set of inliers to this model from whole data set! Repeat 1-3 until model with the most inliers over all samples is found!

37 RANSAC 37 δ O = 6! Algorithm:!! 1. Select random sample of minimum required size to fit model [?]! 2. Compute a putative model from sample set! 3. Compute the set of inliers to this model from whole data set! Repeat 1-3 until model with the most inliers over all samples is found!

38 How many samples? 38 Number of samples S" p = probability at least one random sample is valid (free from outliers)! e = outlier ratio (1-p)! P is total probability of success after S trials! Likelihood in one trial that all s samples are inliers is! s = minimum number needed to fit the model! Likelihood that S such trials will all fail is! Hence the required number of minimum trials is! proportion of outliers e s 5% 10% 20% 25% 30% 40% 50%

39 39 Estimating H! by RANSAC! H 8 DOF! Need 4 correspondences! Sample set = set of matches between 2 images! Algorithm:!! 1. Select a random sample of minimum required size [?]! 2. Compute a putative model from these! 3. Compute the set of inliers to this model from whole sample space! Repeat 1-3 until model with the most inliers over all samples is found!

40 40 Estimating F! Outlier matches! by RANSAC! F 7 DOF! Need 7 (8) correspondences! Sample set = set of matches between 2 images! Algorithm:!! 1. Select a random sample of minimum required size [?]! 2. Compute a putative model from these! 3. Compute the set of inliers to this model from whole sample space! Repeat 1-3 until model with the most inliers over all samples is found!

41 RANSAC Conclusions 41 Good:! Simple and easily implementable! Successful in different contexts! Bad:! Many parameters to tune! Trade-off accuracy-vs-time! Cannot be used if ratio inliers/outliers is too small!

42 Fitting! 42 Goal: Choose a parametric model to! fit a certain quantity from data! Techniques:! Least square methods! RANSAC! Hough transform!!

43 43 Hough transform! P.V.C. Hough, Machine Analysis of Bubble Chamber Pictures, Proc. Int. Conf. High Energy Accelerators and Instrumentation, 1959! Given a set of points, find the curve or line that explains the data points best! y! x!

44 44 Hough transform! P.V.C. Hough, Machine Analysis of Bubble Chamber Pictures, Proc. Int. Conf. High Energy Accelerators and Instrumentation, 1959! Given a set of points, find the curve or line that explains the data points best! y! (x 1, y 1 )! m! y 1 = m x 1 + n! x! Hough space! n! y = m x + n!

45 45 Hough transform! y! m! x! y! m! n! 3! 5! 3! 3! 2! 2! x! 3! 7! 11! 10! 4! 3! 2! 3! 1! 4! 5! 2! 2! 1! 0! 1! 3! 3! n!

46 46 Hough transform! P.V.C. Hough, Machine Analysis of Bubble Chamber Pictures, Proc. Int. Conf. High Energy Accelerators and Instrumentation, 1959! Issue : parameter space [m,n] is unbounded! Use a polar representation for the parameter space! y! ρ ρ θ x! x cosθ + ysinθ = ρ Hough space! θ

47 47 Hough transform - experiments! features! votes!

48 48 Hough transform - experiments! Noisy data! features!! How to compute the intersection point?! IDEA: introduce a grid a count intersection points in each cell! Issue: Grid size needs to be adjusted! votes!

49 49 Hough transform - experiments! features votes Issue: spurious peaks due to uniform noise!

50 50 Hough transform - conclusions! Good:! All points are processed independently, so can cope with occlusion/outliers! Some robustness to noise: noise points unlikely to contribute consistently to any single bin! Bad:! Spurious peaks due to uniform noise! Trade-off noise-grid size (hard to find sweet point)!

51 51 Hough transform - experiments! Courtesy of TKK Automation Technology Laboratory!!

52 52 Credit slide: K. Grauman!

53 Generalized Hough transform [more on forthcoming lectures]! D. Ballard, Generalizing the Hough Transform to Detect Arbitrary Shapes, Pattern Recognition 13(2), 1981! Identify a shape model by measuring the location of its parts and shape centroid! 53 Measurements: orientation theta, location of p! Each measurement casts a vote in the Hough space: p + r(θ)!!!! a p θ r(θ)

54 Generalized Hough transform B. Leibe, A. Leonardis, and B. Schiele, Combined Object Categorization and Segmentation with an Implicit Shape Model, ECCV Workshop on Statistical Learning in Computer Vision 2004! 54

55 Plan 55 Problem Formulation! Least Squares Methods! RANSAC! Hough Transform! Multi-model Fitting! Expectation-Maximization! Examples of Uses of Fitting!

56 Fitting multiple models! 56 Incremental fitting! E.M. (probabilistic fitting)! Hough transform!

57 Incremental line fitting 57 Scan data point sequentially (using locality constraints)! Perform following loop:! 1. Select N point and fit line to N points! 2. Compute residual R N! 3. Add a new point, re-fit line and re-compute R N+1! 4. Continue while line fitting residual is small enough,! Ø When residual exceeds a threshold, start fitting new model (line)!

58 58 Hough transform! Courtesy of unknown! Same cons and pros as before!

59 Plan 59 Problem Formulation! Least Squares Methods! RANSAC! Hough Transform! Multi-model Fitting! Expectation-Maximization! Examples of Uses of Fitting!

60 Fitting helps matching!! 60 Feature are matched (for instance, based on correlation)!

61 Fitting helps matching!! 61 Image 1! Image 2! Matches bases on appearance only! Red: good matches! Green: bad matches! Idea:! Fitting an homography H (by RANSAC) mapping features from images 1 to 2! Bad matches will be labeled as outliers (hence rejected)!!

62 Fitting helps matching!! 62

63 Recognising Panoramas! M. Brown and D. G. Lowe. Recognising Panoramas. In Proceedings of the 9th International Conference on Computer Vision -- ICCV2003! 63

64 64 Fitting helps matching!! Images courtesy of Brandon Lloyd!

65 65

66 Next Lecture: Moving on to Motion Module 66 Readings: FP 10.6; SZ 8; TV 8! (TV is Trucco and Verri, which is not a required book.)!

67 67

68 Least squares methods - fitting a line - 68 Ax = b More equations than unknowns! Look for solution which minimizes Ax-b = (Ax-b) T (Ax-b) T Solve! ( Ax b) ( Ax b) = 0 x i LS solution!! x = ( A T A) 1 A T b

69 Least squares methods - fitting a line -! 69 Solving! x = t ) 1 ( A A A t b A + = t 1 (A A) A t = pseudo-inverse of A! A = U t V = SVD decomposition of A! A = V 1 1 U A + = V + U + 1 with! equal to for all nonzero singular! values and zero otherwise!

70 Least squares methods - fitting an homography -! 70 From n>=4 corresponding points:! A h = 0 h h! h 1,1 1,2 3,3 = 0