Image Analysis. Segmentation by Fitting a Model

Transcription

1 Image Aalysis Segmetatio by Fittig a Model Christophoros Nikou cikou@cs.uoi.gr Images take from: D. Forsyth ad J. Poce. Computer Visio: A Moder Approach, Pretice Hall, Computer Visio course by Svetlaa Lazebik, Uiversity of North Carolia at Chapel Hill ( Uiversity of Ioaia - Departmet of Computer Sciece

2 2 Fittig

3 3 Fittig: motivatio We ve leared how to detect edges, corers, blobs. Now what? 9300 Harris Corers Pkwy, Charlotte, NC We would like to form a higher-level, more compact represetatio of the features i the image by groupig multiple features accordig to a simple model.

4 4 Fittig: motivatio (cot.) Choose a parametric model to represet a set of features. simple model: lies simple model: circles complicated model: car

5 5 Fittig: issues Choose a parametric model to represet a set of features. Membership criterio is ot local Ca t tell whether a poit belogs to a give model just by lookig at that poit. Three mai questios: What model represets this set of features best? Which feature to which of several models? How may models are there? Computatioal complexity is importat It is ifeasible to examie every possible set of parameters ad every possible combiatio of features

6 6 Fittig: issues (cot.) Case study: Lie detectio Noise i the measured feature locatios. Extraeous data: clutter (outliers), multiple lies. Missig data: occlusios.

7 7 Fittig: issues (cot.) If we kow which poits belog to the lie, how do we fid the optimal lie parameters? Least squares. What if there are outliers? Robust fittig, RANSAC. What if there are may lies? Votig methods: RANSAC, Hough trasform. What if we re ot eve sure it s a lie? Model selectio.

8 8 Least squares lie fittig Data: (x 1, y 1 ),, (x, y ) Lie equatio: y i = m x i + b Fid (m, b) to miimize E = i = 1 ( y i mx i b) 2 Is there ay problem with this formulatio? We will fid the solutio first ad we will discuss it afterwards.

9 9 Least squares lie fittig (cot.) = = Y X XB X db de T T [ ] ) ( ) ( ) 2( ) ( ) ( XB XB Y XB Y Y XB Y XB Y XB Y b m x x y y b m x y E T T T T i i i + = = = = = = Least squares solutio to XB=Y: = = i i i b mx y E 1 2 ) ( T T X XB X Y = 1 ( ) T T B X X X Y =

10 10 Least squares lie fittig (cot.) 1 ( T T B= X X) X Y 1 2 xi xi xiyi i= 1 i= 1 i= 1 m B = b = 1 xi yi i= 1 i= 1

11 11 Least squares lie fittig (cot.) Problems Not rotatio-ivariat. It measures the error of the vertical coordiate oly. Fails completely for vertical lies.

12 12 Total least squares Distace betwee poit P(x, y ) ad lie (ε): ax+by+c=0: ax + by + c d( P, ε ) = 2 2 a + b E = i = 1 2 ( axip(x + b yi i, d) y i ) (ε): ax+by+c=0 Uit ormal: N=(a, b) We impose the costrait a 2 +b 2 =1 because λax+λby+λc=0 is the same lie.

13 13 Total least squares (cot.) Fid the parameters of the lie (a, b, c) that miimize the sum of squared perpedicular distaces of each poit from the lie: Eabc (,, ) = ( axi + byi + c) i= 1 2 subject to a 2 +b 2 =1. (ε): ax+by+c=0 E = i = 1 ( ax + b y i i d) P(x i, y i ) 2 Uit ormal: N=(a, b)

14 14 Total least squares (cot.) E a b = 2( ax + i by + i c) = 0 c= xi yi = ax by i= 1 i= 1 c i= 1 E( abc,, ) = ( axi + byi + c) i= 1 2 Substitutig c ito E(a,b,c) ad itroducig the costriat: i= 1 ( ) 2 ( 2 2) i i λ Eab (,, λ) = ax ( x) + by ( y) + 1 a + b

15 15 Total least squares (cot.) E 1 1 = 0 a x ( x) b x y x y λa a + = 2 2 i i i i= 1 i= 1 E 1 1 = 0 a x y x y b y ( y) λb b + = 2 2 i i i i= 1 i= 1 which may be writte i matrix-vector form: 2 ( xi x) ( xi x)( yi y) i= 1 i= 1 a a 2 b = λ b ( xi x)( yi y) ( yi y) i= 1 i= 1

16 16 Total least squares (cot.) 2 ( xi x) ( xi x)( yi y) i= 1 i= 1 a a 2 b = λ b ( xi x)( yi y) ( yi y) i= 1 i= 1 Aw= λw The solutio is the eigevector of A correspodig to the smallest eigevalue (miimizatio of E) of the covariace matrix. The secod eigevalue (the largest) correspods to a perpedicular directio.

17 17 Total least squares (cot.) 2 ( xi x) ( xi x)( yi y) i= 1 i= 1 a a 2 b = λ b ( xi x)( yi y) ( yi y) i= 1 i= 1 Covariace matrix N = (a, b) ( x, y) ( x x, y y) i i

18 18 Least squares as a maximum likelihood problem Geerative model: lie poits are corrupted by Gaussia oise i the directio perpedicular to the lie x y = u v + a ε b ax+by+c=0 (u, v) ε (x, y) poit o the lie oise: ormal zero-mea directio Gaussia with std. dev. σ

19 19 Least squares as a maximum likelihood problem (cot.) Geerative model: x y = u v + a ε b ε ax+by+c=0 (u, v) (x, y) Likelihood of poits give lie parameters (a, b, c): 2 ( axi + byi + c) Px ( 1,, x abd,, ) = Px ( i abc,, ) exp 2 i= 1 i= 1 2σ 1 2 Log-likelihood: L( x,, x a, b, c) = ( ax + by + c) 1 2 i i 2σ i = 1

20 20 Least squares for geeral curves We would like to miimize the sum of squared geometric distaces betwee the data poits ad the curve. d((x i, y i ), C) (x i, y i ) curve C

21 21 Least squares for coics Equatio of a 2D geeral coic: C(a, x) = a x = ax 2 + bxy + cy 2 + dx + ey + f = 0, a = [a, b, c, d, e, f], x = [x 2, xy, y 2, x, y, 1] Algebraic distace: C(a, x). Algebraic distace miimizatio by liear least squares: = f e d c b a y x y y x x y x y y x x y x y y x x

22 22 Least squares for coics Least squares system: Da = 0 Need costrait o a to prevet trivial solutio Discrimiat: b 2 4ac Negative: ellipse Zero: parabola Positive: hyperbola Miimizig squared algebraic distace subject to costraits leads also to a geeralized eigevalue problem. A. Fitzgibbo, M. Pilu, ad R. Fisher, Direct least-squares fittig of ellipses, IEEE Trasactios o Patter Aalysis ad Machie Itelligece, 21(5), , May 1999.

23 23 Who came from which lie? Assume we kow how may lies there are - but which lies are they? easy, if we kow who came from which lie Three strategies Icremetal lie fittig K-meas Probabilistic (later!)

24 24 Icremetal lie fittig A really useful idea; it s the easiest way to fit a set of lies to a curve.

25 25 Icremetal lie fittig (cot.)

26 26 Icremetal lie fittig (cot.)

27 27 Icremetal lie fittig (cot.)

28 28 Icremetal lie fittig (cot.)

29 29 Icremetal lie fittig (cot.)

30 30 Allocatig poits to lies by K-meas

31 31 Least squares ad outliers

32 32 Least squares ad outliers (cot.) Problem: squared error heavily pealizes outliers.

33 33 Least squares ad outliers (cot.) The y-coordiate of a poit corrupted. The error seems small

34 34 Least squares ad outliers (cot.) Zoom ito the previous example to highlight the error.

35 35 Robust estimators I the least squares case, we miimize the residual r(x i, θ) with respect to the model parameters θ: 2 mi r x i ; θ i θ ( ) Outliers cotribute too much to the estimatio of the parameters θ. A solutio is to replace the square with aother fuctio less iflueced by outliers. This fuctio should behave like the square for good poits.

36 36 Robust estimators (cot.) ( r( x ) ) mi ρ i, θ ; σ θ i The robust fuctio ρ behaves like a squared distace for small values of its argumet but saturates for larger values of it. Notice the role of parameter σ. We have to select it carefully.

37 37 Robust estimators (cot.) Estimatio of the model parameters is ow obtaied by: i ( ) ( r x, ; ) ρ θ σ i θ = 0 ( ) where ψ(x;σ) is the ifluece fuctio of the robust estimator characterizig the ifluece of the residuals to the solutio. ( r x, ; ) i r( x, ) ρ θ σ θ i = i r θ ( ) r x, ( ; ) ( (, ); ) i d x ψ r xi θ σ θ = 0, with ψ( x; σ) = ρ σ θ dx i 0

38 38 Robust estimators (cot.) Stadard least squares estimator ad its ifluece fuctio 38

39 39 Robust estimators (cot.) Trucated least squares estimator ad its ifluece fuctio 39

40 40 Robust estimators (cot.) Gema-McClure estimator ad its ifluece fuctio 40

41 41 Robust estimators (cot.) Correct selectio of σ: the effect of the outlier is elimiated

42 42 Robust estimators (cot.) Small σ: the error value is almost the same for every poit ad the fit is very poor. All the poits are cosidered as outliers.

43 43 Robust estimators (cot.) Large σ: the error value is like least squares ad the fit is very poor. All the poits are cosidered as good.

44 44 Robust estimatio (cot.) Robust fittig is a oliear optimizatio problem that must be solved iteratively. Least squares solutio ca be used for iitializatio. Adaptive choice of scale base o the media absolute deviatio (MAD) of the residuals at each iteratio (t): { ( ) { ( )}} i θ k θ MAD = media r x ; media r x ; () t () t () t i σ = MAD () t () t It is based o the MAD of ormally distributed poits with stadard deviatio of 1. P. Rousseeuw ad A. Leory. Robust regressio ad outlier detectio. J. Wiley k

45 45 RANSAC Robust fittig ca deal with a few outliers what if we have may of them? Radom sample cosesus (RANSAC) Choose a small subset of poits uiformly at radom. Fit a model to that subset. Fid all remaiig poits that are close to the model ad reject the rest as outliers. Do this may times ad choose the best model. M. A. Fischler ad R. C. Bolles. Radom Sample Cosesus: A Paradigm for Model Fittig with Applicatios to Image Aalysis ad Automated Cartography. Commuicatios of the ACM, Vol 24, pp , 1981.

46 46 RANSAC (cot.) Isight Search for good : poits i the data set. Suppose we wat to fit a lie to a poit set degraded by at 50% by outliers. Choosig radomly 2 poits has a chace of 0.25 to obtai good poits. We ca idetify them by oticig that a may other poits will be close to the lie fitted to the pair. The we fit a lie to all these good poits. M. A. Fischler ad R. C. Bolles. Radom Sample Cosesus: A Paradigm for Model Fittig with Applicatios to Image Aalysis ad Automated Cartography. Commuicatios of the ACM, Vol 24, pp , 1981.

47 47 RANSAC (cot.) Repeat k times: Draw poits uiformly at radom. Fit lie to these poits. Fid iliers to this lie amog the remaiig poits (i.e., poits whose distace from the lie is less tha t). If there are d or more iliers, accept the lie (ad the cosesus set) ad refit to all iliers. Select the largest cosesus set. Parameters k,, t ad d are crucial.

48 48 RANSAC (cot.) Number of samples ad umber of iteratios k. The miimum umber to fit the model (2 poits for lies). If w is the fractio of good poits, the expected umber of draws required to get a good sample is: E[k]=1 x p(oe good sample i oe draw) + 2 x p(oe good sample i two draws) + = w +2(1-w ) w +3(1-w ) 2 w + = w -. We add a few stadard deviatios to this umber: σ k = 1 w w

49 49 RANSAC (cot.) Number of iteratios k (alterative approach). We look for a umber of samples that guaratees a that at least oe of them is free from outliers with a probability of p. Probability of seeig a bad sample of poits i oe draw: (1 w ) ad the probability of seeig oly bad samples i k draws is: 1 p = (1 w ) k Solvig for k: k = log(1 p) log(1 w )

50 50 RANSAC (cot.) The umber of iteratios k required to esure, with a probability p=0.99, that at least oe sample has o outliers for a give size of sample,, ad proportio of outliers 1-w. k = log(1 p) log(1 w ) proportio of outliers 1-w 5% 10% 20% 25% 30% 40% 50%

51 51 RANSAC (cot.) Threshold t to decide if a poit is close to the lie or ot. We geerally specify it as a part of the modelig process. The same as i the maximum likelihood formulatio of lie fittig (parameter σ). Look similar data sets, fit some lies by eye ad determie it empirically.

52 52 RANSAC (cot.) Number of poits d that must be close to the lie (the size of the cosesus set). A rule of thumb is to termiate if the size of the cosesus set is similar to the umber of iliers believed to be i the data set, give the assumed proportio of outliers. For N data poits ad if the percetage of iliers is w: d = wn

53 53 RANSAC (cot.) Pros Simple ad geeral. Applicable to may differet problems. Ofte works well i practice. Cos Lots of parameters to tue. Ca t always get a good iitializatio of the model based o the miimum umber of samples. Sometimes too may iteratios are required. Ca fail for extremely low ilier ratios. We ca ofte do better tha brute-force samplig.

54 54 Votig schemes Let each feature vote for all the models that are compatible with it Hopefully the oise features will ot vote cosistetly for ay sigle model Missig data does t matter as log as there are eough features remaiig to agree o a good model

55 55 Hough trasform A early type of votig scheme. Geeral outlie: Discretize parameter space ito bis For each feature poit i the image, put a vote i every bi i the parameter space that could have geerated this poit Fid bis that have the most votes Image space Hough parameter space P.V.C. Hough, Machie Aalysis of Bubble Chamber Pictures, Proc. It. Cof. High Eergy Accelerators ad Istrumetatio, 1959

56 56 Hough trasform (cot.) A lie i the image correspods to a poit i Hough space Image space Hough parameter space Source: S. Seitz

57 57 Hough trasform (cot.) What does a poit (x 0, y 0 ) i the image space map to i the Hough space? Image space Hough parameter space

58 58 Hough trasform (cot.) What does a poit (x 0, y 0 ) i the image space map to i the Hough space? Aswer: the solutios of b = x 0 m + y 0 This is a lie i Hough space Image space Hough parameter space

59 59 Hough trasform (cot.) Where is the lie that cotais both (x 0, y 0 ) ad (x 1, y 1 )? Image space Hough parameter space (x 1, y 1 ) (x 0, y 0 ) b = x 1 m + y 1

60 60 Hough trasform (cot.) Where is the lie that cotais both (x 0, y 0 ) ad (x 1, y 1 )? It is the itersectio of the lies b = x 0 m + y 0 ad b = x 1 m + y 1 Image space (x 1, y 1 ) Hough parameter space (x 0, y 0 ) b = x 1 m + y 1

61 61 Hough trasform (cot.) Problems with the (m,b) space: Ubouded parameter domai Vertical lies require ifiite m

62 62 Hough trasform (cot.) Problems with the (m,b) space: Ubouded parameter domai Vertical lies require ifiite m Alterative: polar represetatio x cosθ + y siθ = ρ Each poit will add a siusoid i the (θ,ρ) parameter space

63 63 Hough trasform (cot.)

64 64 Hough trasform (cot.) A: itersectio of curves correspodig to poits 1, 3, 5. B: itersectio of curves correspodig to poits 2, 3, 4. Q, R ad S show the reflective adjacecy at the edges of the parameter space. They do ot correspod to poits.

65 65 Hough trasform (cot.) features votes 20 poits draw from a lie ad the Hough trasform. The largest vote is 20.

66 66 Hough trasform (cot.) Square Circle The four sides of the square get the most votes. I the circle, all of the lies get two votes ad the taget lies get oe vote.

67 67 Hough trasform (cot.) Several lies

68 68 Hough trasform (cot.) features votes Radom perturbatio i [0, 0.05] of each poit. Peak gets fuzzy ad hard to locate. Max umber of votes is ow 6.

69 69 Hough trasform (cot.) Number of votes for a lie of 20 poits with icreasig oise:

70 70 Hough trasform (cot.) features votes Radom poits: uiform oise ca lead to spurious peaks i the accumulator array. Max votes = 4.

71 71 Hough trasform (cot.) As the level of uiform oise icreases, the maximum umber of votes icreases too ad makes the task difficult.

72 72 Practical details Try to get rid of irrelevat features Take oly edge poits with sigificat gradiet magitude. Choose a good grid / discretizatio Too coarse: large votes obtaied whe too may differet lies correspod to a sigle bucket. Too fie: miss lies because some poits that are ot exactly colliear cast votes for differet buckets. Icremet also eighborig bis (smoothig i accumulator array).

73 73 Hough trasform: Pros Ca deal with o-locality ad occlusio. Ca detect multiple istaces of a model i a sigle pass. Some robustess to oise: oise poits ulikely to cotribute cosistetly to ay sigle bi.

74 74 Hough trasform: Cos Complexity of search time icreases expoetially with the umber of model parameters. No-target shapes ca produce spurious peaks i parameter space. It s hard to pick a good grid size.