How Easy is Matching 2D Line Models Using Local Search?

Transcription

1 564 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 19, NO. 6, JUNE 1997 How Easy is Matching 2D Line Models Using Local Seach? J. Ross Beveidge and Edwad M. Riseman, Senio Membe, IEEE Abstact Local seach is a well established and highly effective method fo solving complex combinatoial optimization poblems. Hee, local seach is adapted to solve difficult geometic matching poblems. Matching is posed as the poblem of finding the optimal many-to-many coespondence mapping between a line segment model and image line segments. Image data is assumed to be fagmented, noisy, and clutteed. The algoithms pesented have been used fo obot navigation, photo intepetation, and scene undestanding. This pape exploes how local seach pefoms as model complexity inceases, image clutte inceases, and additional model instances ae added to the image data. Expected un-times to find optimal matches with 95 pecent confidence ae detemined fo 48 distinct poblems involving six models. Nonlinea egession is used to estimate un-time gowth as a function of poblem size. Both polynomial and exponential gowth models ae fit to the un-time data. Fo poblems with andom clutte, the polynomial model fits bette and gowth is compaable to that fo tee seach. Fo poblems involving symmetic models and multiple model instances, whee tee seach is exponential, the polynomial gowth model is supeio to the exponential gowth model fo one seach algoithm and compaable fo anothe. Index Tems Object ecognition, optimal model matching, line segment models, un-time pefomance chaacteization, andomstats local seach. 1 INTRODUCTION L OCAL seach [1], [2], [3] is known within the combinatoial optimization liteatue as an effective means of solving difficult combinatoial optimization poblems. Conceptually, local seach is efeshingly simple. A tactable neighbohood is defined within an intactable combinatoial seach space, and seach epeatedly moves to neighboing states until a state is found which is bette than all its neighbos. In andom stats local seach, local seach is initiated fom states dawn andomly fom the space. Multiple tials of local seach incease the pobability of finding a nea optimal solution. Ou contibution has been to adapt these ideas to geometic matching poblems [4], [5], [6] associated with object ecognition. Ou local seach algoithms have been used fo semiautonomous photo-intepetation [7], obot navigation [8], [9], [10], and scene undestanding [11]. A matching system based upon the ideas pesented hee is now included in the KBVision system poduced by AAI in Amhest, Mass. When estimates fo object position and oientation ae available, a 3D vesion finds optimal matches in domains with significant 3D pespective [12]. Ove the couse of ou wok, we have continued to ty to undestand how local seach scales up to lage and moe difficult poblems. 1 To addess this question, hee we pes- 1. Johnson and Papadimitiou [13] ae thanked fo inspiing the title to this pape with thei own title, How Easy is Local Seach? J.R. Beveidge is with the Compute Science Depatment, Coloado State Univesity, Fot Collins Coloado. oss@cs.colostate.edu. E.M. Riseman is with the Compute Science Depatment at the Univesity of Massachusetts, Amhest Massachusetts. Manuscipt eceived 10 May 1996; evised 4 Ap Recommended fo acceptance by P. Flynn. Fo infomation on obtaining epints of this aticle, please send to: tanspami@compute.og, and efeence IEEECS Log Numbe ent esults fom ou lagest empiical study to date. A shot vesion of this study appeas in [14] and the esults significantly extend those pesented in [6]. We use nonlinea egession to compae two altenatives un-time gowth is exponential O(e n ), un-time gowth is polynomial O(n b ), whee n is the poblem size. Ove a test suite of 48 poblems vaying in size fom n = 12 to n = 1,296, the polynomial hypothesis is a significantly bette fit to ou obseved un-times. The exponent b is oughly two, and hence, untimes appea to gow as a linea function of n 2. Poblem size n is typically the poduct of the numbe of model featues m and data featues d, i.e. n = md. Ou matching algoithms conside many-to-many mappings between featues, and thus, thee ae 2 n possible matches. It is pehaps supising that un-time gowth appeas polynomial, while seach space gowth is exponential. These findings suggest that local seach compaes favoably with two of the bette known and well undestood altenative matching algoithms: Gimson s tee seach [15], [16] and Cass [17] pose equivalence analysis. Gimson has shown that fo tee seach, excluding poblems with symmetic models and multiple instances, aveage case complexity is O(m 2 d 2 ). Since n = md, this aveage case complexity is compaable to that which we obseve fo local seach. Howeve, Gimson also shows that if models ae symmetic, o moe than one model instance is pesent, then tee seach becomes exponential: O(d m ) o O(m d ) depending on fomulation. Ou test suite of poblems includes both symmetic models and multiple model instances, and, thus, we ae obseving n 2 aveage case gowth on poblems whee tee seach is exponential. Pose equivalence analysis clevely combines seach in pose and coespondence space. Fo 2D poblems involving /97/$ IEEE

2 BEVERIDGE AND RISEMAN: HOW EASY IS MATCHING 2D LINE MODELS USING LOCAL SEARCH? 565 otation, tanslation, and scale, pose equivalence analysis has an analytic wost-case complexity bound of O(k 4 n 4 ). Hee, k is the numbe of sides on a convex polygon within which coesponding featues must appea. The fou deives fom the fou degees of feedom in a 2D similaity tansfom. While the existence of this bound is significant, the dependence upon n 4 pecludes lage poblems in the wost case, and aveage case pefomance has not been epoted. We begin this pape with an oveview of local seach matching including examples on eal data. Section 3 and Section 4 pesent ou matching objective function and two local seach algoithms. Section 5 develops a andom sampling methodology and descibes how we chaacteize the difficulty of individual matching poblems. Section 6 pesents ou nonlinea egession analysis of un-times on ou test suite of 48 matching poblems. Section 7 summaizes un-times fo the eal data examples pesented ealie. We finish with a discussion of the elationship between andom stats local seach and algoithms that use featue subsets fo pose indexing. (a) 2 LOCAL SEARCH MATCHING OVERVIEW Let us look at an illustation of the key concepts of local seach matching and eview some esults on eal data. 2.1 A Thumbnail Sketch of Local Seach Matching Fig. 1 intoduces many of the essential elements of local seach matching. Fig. 1a shows an object model, impefect data, and the associated best match. Both model and data ae epesented simply as sets of 2D staight line segments. While the segments in this model fom a closed contou, closed contous ae not equied. Models ae always fit to coesponding data when evaluating match quality: Note that the model has been otated, tanslated, and scaled to best match the data. Fitting is an essential pat of ou appoach and is descibed in detail in Section Fig. 1b illustates one un, o tial, of local seach. Each ow indicates a paticula match. Those pais of segments included in the match ae indicated with a black cicle. The initial match, shown in ow one, is selected at andom. This intoduction of andomness is impotant and is explained below. Each successive ow indicates a new match geneated by a single step of the local seach pocess. Seach consides the addition o emoval of a single pai of coesponding segments, and then makes the change which educes the match eo by the geatest amount. This is an example of a local seach neighbohood and neighbohood seach stategy. Neighbohoods and seach stategies ae discussed in Sections 4.1 and 4.2. The match eo, shown fo each successive match, defines what constitutes the best match and guides the local seach though the combinatoic space of possible matches. Match eo takes into account the quality of the fit between the model and data, and how well the coesponding data explains o coves the model. Match eo details ae pesented in Section 3.3. (b) Fig. 1. Illustation of local seach matching. (a) A model, impefect data, and an optimal match. (b) Successive ows show local seach impoving upon an initial andom match. A black cicle indicates coespondence of a pai. Local seach will not always aive at the best match. Typically thee ae many local optima, and the cux of this pape lies in demonstating that local seach emains a viable tool, despite the existence of local optima. Section 5 discusses the use of andomly sampling to mitigate the impotance of local optima. Let us suggest the type of analysis developed in Section 5 by consideing actual pefomance numbes fo the poblem shown in Fig. 1. On this poblem, local seach finds the best match in 25 out 100 ties initiated fom independently chosen andom stating matches. This tells us that the pobability of seeing this best match on any single tial is oughly Consequently, if 12 independent tials ae un, the best match will be seen at least once with pobability bette than Running on a Sun Spac 10 wokstation, ou algoithm takes just unde one second to un 12 tials: Run-times pe tial ange fom 0.04 to 0.12 seconds. 2.2 Some Sample Results on Actual Image Data Figs. 2 and 3 illustate how local seach matching might be used to find buildings in aeial photogaphs. In these examples, models have been hand built. Howeve, while these examples ae hand built, local seach has been used with eal building models on the RADIUS calibated teain boad imagey [18]. It has also been used with aeial photogaphs to egiste othoectified images [7]. The data line segments fo these examples ae poduced using the Buns algoithm [19].

3 566 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 19, NO. 6, JUNE 1997 (a) (b) (c) (d) Fig. 2. Matching a building in an aeial photogaph. (a) Model in white. (b) Test image. (c) Data segments. (d) Optimal match. Fo the match in Fig. 2d, the model is otated by 120 o, illustating that the oiginal oientation of the model does not matte. In this example, thee is little clutte and the building has a distinctive fom. We ll see in Section 7 that this is not a difficult poblem. The match in Fig. 3 is moe difficult. The data is highly fagmented and clutteed, the matching algoithm must find the exact set of 12 data line segments which, when taken togethe, ae the optimal match to the model. 2 Fig. 4 illustates a diffeent kind of matching poblem. Figs. 4a and 4b show two images of a ca coming towad a camea. Fig. 4c shows a labeled set of 27 model line segments extacted fom Fig. 4a. Fig. 4d shows a labeled set of data line segments extacted fom Fig. 4b. In this example, thee is no clutte. Howeve, the model itself is complex and has locally ambiguous stuctue. The matix in Fig. 4e indicates the coespondence mapping found to be the optimal match. A squae indicates a pai which potentially match, and a filled-in squae indicates a pai belonging to the optimal match. Some paiings between model and data segments ae uled out based upon an initial estimate of how much the model has moved fom Fig. 4a to Fig. 4b. The amount of time equied to find this match is discussed in Section 7. The pais of closely spaced lines bodeing the windshield on the top and sides pesent a challenge to a matching algoithm. The oute segments could locally match the inne segments as well as the oute segments, and the matching algoithm must ovecome this local ambiguity. Fitting the object model as a whole effectively disambiguates this stuctue. (a) (b) (c) (d) 3 MATCHING AS COMBINATORIAL OPTIMIZATION Seveal key concepts undelie ou appoach. 1) Matching is the poblem of finding a discete coespondence mapping, possibly many-to-many, between model and data segments, which minimizes a heuistic measue of match quality (Section 3.1). 2) Efficient global fitting techniques align 2D models to 2D data (Section 3.3.1). 3) Global alignment is the basis fo evaluating all matches, and alignment implicitly guides seach though the combinatoial space of matches (Sections 3.3.3, 3.3.4, and 4.1). 4) Tactable neighbohoods make exploation of combinatoial match space feasible (Sections 4.1 and 4.2). 5) Random sampling finds, with abitaily high pobability, globally optimal matches between model and data line segments 3 (Sections 5 and 6). 2. Fo this model, as fo the ectangle shown ealie, thee ae two equally good matches. These two diffe by otating the model by 180 o. 3. Often, but not always, in the domain of matching, visual inspection of esults is enough to make a detemination that an algoithm has, in fact, found the globally optimal match. In some subtle poblems the tue global optima may not be known to us, and, in these cases, we ae efeing to the best ou algoithms have eve found. Fig. 3. Match with high clutte and fagmentation. (a) Image. (b) Line segments. (c) Simple building model. (d) Optimal match.

4 BEVERIDGE AND RISEMAN: HOW EASY IS MATCHING 2D LINE MODELS USING LOCAL SEARCH? 567 (a) (b) (c) (d) Pais Pais Pais Pais Pais Pais A 20 F 35 K 55 P 52 U 0 Z 13 B 22 G 9 L 53, 54 Q 0 V 2 AA 42 C 21 H 11 M 51 R W 31 D 3 I 15 N 48 S 41 X E 30 J 0 49 T 6 Y 29 (e) Fig. 4. Matching an oncoming ca. (a) Image one. (b) Image two. (c) Model extacted fom image one. (d) Data fom Image two with matching segments in black and othes in gay. (e) Optimal coespondence mapping between segments. 3.1 Discete Space of Many-to-Many Coespondences Match eo is defined ove a discete space of possible model-to-image segment mappings. Let M be the set of model segments, D the set of data segments, and S the coss poduct of M and D, the coespondence space C is the powe set of S: C = 2 S. In othe wods, C contains all possible many-to-many mappings between model and data segments. Fig. 5 shows an example of a match fo a stylized tee. Relating the above definitions to this example, M = {A, B, C, D, E, F, G, H, I, J, K, L} D = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12} S = {(A, 0), (A, 1), º, (A, 12), (B, 0), º, (L, 12)} C = 2 S The sets M, D, S, and C contain 12, 13, 156, and elements, espectively. 3.2 Match Eo A match eo E match is defined such that E match : C Æ, E match (c) 0 " c Œ C The goal of matching is to find the optimal match c* whee E match (c*) E match (c) " c Œ C Ou match eo, E match, measues thee things: 1) the degee to which the model fits the coesponding data segments,

5 568 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 19, NO. 6, JUNE 1997 Making the quality of a match to one instance dependent upon the size of the othe instance leads to unpedictable and undesiable behavio. Howeve, in the context of matching lase ange data we have found a data omission tem to be helpful [20]. To evaluate E match, fist a global alignment of model to data must be detemined based upon the coespondence c. Next, the model must be tansfomed to this configuation and omission measued. Finally, the best fitting tansfomation )* is compaed to some acceptable ange and a penalty is added if the tansfomation lies outside these bounds. (a) 2) the degee to which potions of the model ae omitted fom the match, and 3) the degee to which the best-fit tansfomation is acceptable. In its geneal fom, it may be witten as: af af af af Ematch c = F Efit c + Eom c + E) c = EIaf c Eomaf c Eaf c H G 1I K J ) s (1) whee E I (c) is a nomalized esidual afte least-squaes fitting of the model to the data and the weighting coefficient s contols the elative impotance of fit vesus omission. Ou expeience [4], [5], [6], [12] suggests E match (c) induces easonable ankings ove matches fo a given model. It is not intended fo compaing matches to diffeent models. Developing measues to compae complex vesus simple models is itself a subtle poblem [20]. In pat inspied by Wells [21], we have expeimented with a data omission tem penalizing matches that leave data unmatched. On poblems of the type pesented hee, data omission huts athe than helps. Conside what happens when two instances of a model appea in one image. (b) (c) (d) Fig. 5. An example match fom the test suite to be pesented in Section 6. (a) Model line segments. (b) Data line segments. (c) Model shown matched to data. (d) Coespondence matix with matching pais blacked out. The match eo E match = fo this best match. It is the sum of fit eo E fit = and the omission eo E om = Match Evaluation Using Global Alignment Of seveal altenative ways to fit a line segment model to line segment data, pehaps the most obvious but flawed appoach is to minimize point-to-point distance between coesponding segment endpoints o midpoints. Point-topoint fitting has poblems when line segments ae fagmented o oveextended. Both Lowe and Ayache [22],[23] appopiately suggested it is bette to minimize a pependicula point-to-line distance measue. Ayache [23] developed a closed-fom solution fo the otation, tanslation, and scale which minimizes squaed pependicula distance fom endpoints of model line segments to infinitely extended data lines. The weakness of this appoach is that often line segments extacted fom imagey fagment [24]. Extending a fagmented segment amplifies oientation eos and in tuns skews the oveall esulting fit. We have developed closed-fom solutions with the ole of data and model evesed so that the inheently moe stable model segments ae infinitely extended [4], [24]. In anothe impovement to least-squaes fitting of line models, the pependicula distance is integated along the data line segment athe than being allowed to concentate at the ends. Consequently, the optimal least-squaes fit is completely invaiant with espect to beakpoints in data line segments. Moe pecisely, if a single data segment is boken into two adjacent segments at any position along the segment, the esulting fit will not change. As an aside, the case of igid 2D fitting deseves comment. Fo many matching techniques, such as tee seach [16] and Hough tansfoms [25], focing scale to emain constant makes poblems easie to solve. In a somewhat counte-intuitive discovey, we have found that leastsquaes fitting of line models fo the igid case is hade than fo the vaiable scale case. Vaiable scale equies the solution of a quadatic equation, while fixing scale leads to a quatic equation [6] Minimizing Pependicula Distance Models ae fit to data so as to minimize the sum of the integated squaed pependicula distance (ISPD) between h coesponding pais of segments in c. The pependicula distance fom the endpoint of a data line segment to the coesponding infinitely extended model line segment may be paameteized by a similaity tansfomation ). The tansfomation ) otates, tanslates and scales the model elative to the data and may be witten as:

6 BEVERIDGE AND RISEMAN: HOW EASY IS MATCHING 2D LINE MODELS USING LOCAL SEARCH? 569 F ) = = H G cos f tx d I F DM, T DM, s MD i t smd K J sin f,, y (2) The tems F DM and T DM otate and tanslate data segments data elative to the model, while s MD scales the model elative to the data segments. This asymmetic teatment of otation and tanslation vesus scaling simplifies the solution fo the best-fit tansfomation. The subscipts indicate the diection of the tansfomation: DM fo data efeence fame to model and MD fo model to data. Rotation may be witten with the vecto F DM povided we intoduce a matix to epesent 2D points. Thus, the jth endpoint of a data line segment D i is witten as: D ij x = y ij ij -y x The tem s MD scales the distance fom the oigin to the neaest point on the infinitely extended model line. Hence, fo a model segment M i, ij ij i i ik (3) = N $ M (4) whee N $ i is the unit nomal vecto to the line M i, and M ik is any point lying on M i. The pependicula distance v ij fom the endpoint j of a data line segment D i to the coesponding infinitely extended model line segment M i may be witten as a function of ): v ) = N$ D F + T - s (5) bge e j ij i ij DM DM MD i The integated squaed pependicula distance may be defined in tems of v ij : Integated (E I ) and endpoint concentated (E P ) measues ae shown to highlight the diffeence between the two: E I E P 1 L D h i  2 2 vi v 2e 1 i1j (6) i= 1 = F H G I K J + 1 L D h  i= 1 i 2 2 vi vi vi v 3e i1j (7) = F H G I K J + + The tem i is the length of the data segment D i and L D is the cumulative length ove all matched data line segments. Nomalizing by L D yields a weighted aveage of squaed pependicula distance ove all pais of matching model and data line segments. Multiplying out the tems in (7) and collapsing the sums yields the following: F EI = A T B T CT H G 1I T T T LD K J ef F + 2 F + T T 2-2U Fs + 2V Ts + ksj (8) j  h T A = i / D NN $ $ T T i i i Di + D NN $ $ T T i i i Di + Di NN $ $ T 3 i i D i= 1 i h 2 B =  i NiNi D i=  / $ $ T c 2h 1 j= 1 ij h C =  in$ in $ T i= 1 i h 2 T U =  i idijn i=  c / 2h $ 1 j= 1 i h V =  iin$ i= 1 i h 2 k =  c he j i= 1 Fo simplicity the subscipts have been dopped fom the tansfomation tems: s = s MD, F = F DM, and T = T DM. The best-fit alignment of the model to the data ) * = F *, T *, s* (9) d i (10) minimizes (8) subject to the constaint F T F = 1 (11) To solve fo )*, set to zeo the deivative of (8) with espect to s and solve fo s* as a function of F and T. s* = U T + V T F T / k e j (12) Substitute s* fo s in (8) and dop the nomalization tem (L D ) to yield: T T T E = F DF + 2 T EF + T FT (13) I whee T D = A - UU / k T E = B - VU / k T F = C -VV / k Set the patial deivative of (13) with espect to T to zeo and solve fo T * as a function of F: T* =-F -1 EF (14) Finally, substitute T * into (13) to get T T -1 E = F GF, whee G = D = E F E (15) I By Rayleigh s Pinciple ([26 p. 429]) the vecto F * which minimizes 15 is the unit eigenvecto associated with the lesse eigenvalue of the matix G. Fo some configuations, such as cones, the best-fit tansfomation F*, T *, s * is undeconstained. Adding a egulaizing tem, squaed Euclidean distance between midpoints of coesponding line segments, esolves these cases. We use 10-3 as the weight fo this egulaization tem Fit Eo is Nomalized Residual afte Global Fitting The fit eo is the esidual of E I fom (7) weighted by sigma: 1 E c EI F*, T*, s* (16) 2 s fitaf F = HG I K J d i It is possible now to give a cleae intepetation to the

7 570 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 19, NO. 6, JUNE 1997 elative weighting tem s used in (1). Since E I is aveage squaed pependicula distance, the fit eo E fit (c) eaches 1.0 when the aveage pependicula distance eaches s. Pactically speaking, since the omission eo is nomalized to the ange [0, 1], s is the lagest amount of integated pependicula distance allowed in a match. This is because if E fit (c) exceeds 1.0, the match eo will favo emoval of those segments causing the poo fit. As a paamete of the matching system, s is descibed as the maximumdisplacement, and unless othewise stated, is set to 4.0 pixels in all the expeiments which follow. The symbol s is commonly used fo the standad deviation of a noise pocess, and ou use of the tem is meant to be suggestive. While the connection is not igoous, one may think infomally of s as the standad deviation of a noise pocess which skews data line segments elative to thei tue position. Fo an excellent teatment on noise pocess models and staight line extaction, see [27] Omission Eo is a Nonlinea Function of Coveage The omission eo fo a model segment M is defined as a nonlinea function of the pecent p of the model line unaccounted fo by data. Let M 1 and M 2 be the endpoints of M and define a paametic expession fo points lying on M. M2 - M1 Pt af= Tt+ M1, 0 t, T= (17) M - M 2 1 whee is the length of M in pixels and T is a unit tangent vecto pointing fom M 1 to M 2. The t value fo the pependicula pojection of any abitay point D may be witten: t = D - M T (18) c 1 A data segment D i with endpoints D i1 and D i2 tansfomed into the best-fit configuation coves M ove the inteval [t i1, t i2 ] whee ti1 = cdi1 - M1h T t = D - M T c h i2 i2 1 The pecent omitted p is the potion of M (ange [0, ]) not included in the pojection ange [t i1, t i2 ] of any coesponding data segment. While p itself could be used as an eo, thee ae easons to favo a nonlinea function of p. Fist, even unde the best of cicumstances, a small amount of omission is to be expected (e.g., the ends of lines ae often difficult to extact). Thus, small values of p should incu a elatively small penalty. Second, as inceasingly lage potions fail to be found, the penalty should begin to gow substantially. The following nonlinea function of p captues this elationship: E om bg p = R S T gp e e p g -1-1 h if g π0 othewise (19) Since p lies in the ange [0, 1], the omission eo also lies in this ange. The degee of nonlineaity is contolled by g. Howeve, the exact manne in which changing g changes the fom of E om (p) is less than obvious. It is helpful to intoduce an auxiliay paamete a, and then define g in tems of a: F HG I KJ 2 g = 2ln -1 (20) a The paamete a attenuates the omission eo fo small amounts of omission. In the special case a = 1.0, E om is a linea function of p and as a deceases E om dops below the linea case. A value of a = 0.5 dops E om by 50 pecent at the midpoint of the cuve, i.e. whee p = 0.5. In all the expeiments pesented in this pape, a = Omission eo fo a match is a weighted sum of omission fo each model segment: E om F HG m =  af E p L omc mh (21) mœm M I KJ whee is the length of each model line segment and L M is the cumulative length of all model line segments. Weighting by elative length makes the omission eo moe diectly compaable to the nomalized fit eo E fit. 3.4 Discouaging Excess Scale Change We discouage coespondences that imply nea pathological tansfomations, such as shinking the model to a point. In ode to make such matches less desiable the following E ) (c) is defined: E)af c = Rb S T g 1/ s* - if s* < 1/ 0.0 if 1/ s* s* - if s* > (22) The scale change s* indicates how much the model changes size and the paamete defines an allowable ange of size changes. If s* becomes lage than, then the eo stats out at 0 and gows linealy with s*. If s* becomes smalle than the ecipocal of, then eo gows linealy in the ecipocal of s*. The eo appoaches infinity as s* appoaches infinity o 0. Fo all the expeiments pesented hee = 2: Models may ange fom half to twice thei oiginal size without penalty. 4 TWO LOCAL SEARCH MATCHING ALGORITHMS A Hamming-distance-one neighbohood and the associated steepest-descent algoithm is descibed in the following section. A neighbohood stuctue specifically tailoed to geometic matching poblems is defined in Section Steepest-Descent Local Seach To undestand how local seach is caied out, it helps to fist undestand ou bit-sting encoding of the seach space C and its elationship to the local seach neighbohood. This encoding assigns a unique bit to evey pai of model and image line segments in the set S = M D. Theefoe, if S = n, then the seach space C maps to the space of all possible bit stings of length n. A one in the ith position of the sting indicates the ith pai s i Œ S is pat of the match.

8 BEVERIDGE AND RISEMAN: HOW EASY IS MATCHING 2D LINE MODELS USING LOCAL SEARCH? 571 The local seach neighbohood is now defined to contain all stings within Hamming-distance-one of the cuent match. Local seach is itself just a loop in which the n neighbos of some cuent match ae evaluated and if a bette neighbo is found it becomes the cuent match. Moe specifically, a steepest-descent local seach algoithm in the Hamming-distance-one neighbohood toggles each bit in the bit sting encoding of the cuent match and evaluates the match eo E match (c). The algoithm ecods the toggle which yields the geatest dop in E match (c) and uses this to ceate a new bette state. Seach teminates when none of the neighboing matches ae an impovement upon the cuent match. This algoithm has aleady been illustated in Fig. 1b. This neighbohood can add o emove a single pai of model-data segments in one move: It cannot swap one data segment fo anothe. A Hamming-distance-two neighbohood pemits swapping of segments. Howeve, the size of the Hammingdistance-two neighbohood is n 2. Ealy expeiments wee made with this neighbohood [24], but the n 2 gowth in neighbohood size makes this an unattactive altenative fo even medium-sized poblems. Results using the Hamming-distance-one neighbohood show un-times to solve complete poblems appea to gow as a function of n 2. If the neighbohood alone gows as n 2, then the esulting local seach algoithm cannot help but do wose. Fo efficiency, local seach exploits the fact that the n neighbos ae slight petubations of the cuent match. In pinciple, fo evey neighbo tested, the model must be completely fit to the coesponding data and the associated omission ove the entie model computed. Howeve, the change in fit eo can be moe efficiently computed incementally elative to the cuent match [6]. The incemental computation of E fit equies about 20 floating point additions and multiplications and the finding of one squae oot. A useful heuistic is, theefoe, to see if the change in fit eo appeas to peclude impovement. Most neighbos being tested suggest adding a pai of model-data segments. The ule of thumb is if that if the fit eo gows moe than can be possibly made up fo by an associated dop in omission fo that model segment, then do not bothe to compute the complete change in omission eo. Applying this heuistic educes equied computation by nealy an ode of magnitude [28]. It is a heuistic because it neglects subtle inteaction effects in which a small match change might dop omission eo fo many model line segments. Such cases exist but ae ae. 4.2 Subset-Convegent Local Seach Subset-convegent local seach tests whethe subsets of a locally optimal match in the Hamming-distance-one neighbohood ae consistent with the oveall match. Fo a tuly good match, Hamming-distance-one local seach initiated fom subsets of the matching pais should convege back to the same match. Altenatively, if the match is poo, then subsets of the match ae pobably incompatible, and seach initiated fom a subset may well lead to an oveall bette match. Ou expeiments, including those pesented below, have shown this intuition to be coect. Subset-convegent local seach begins by unning the steepest-descent algoithm until a Hamming-distance-one local optima is encounteed. This match is ecoded and seach is initiated fom new matches containing only subsets of the model-data pais pesent in the local optima. Subsets ae defined elative to the model segments. If M' Ã M is a designated subset, then only pais s containing model segments in M' ae etained in the new match. These matches almost always scoe wose than the local optima since emoving data inceases omission eo. Howeve, steepest-descent initiated fom these subset matches often leads to bette matches. If seach fom all the subsets fails to yield a bette match, then subset-convegent local seach teminates. Thee ae many ways to define the subsets and subset selection is pehaps the least studied aspect of ou algoithm. One guiding pinciple has been that the total numbe of subsets emain small and not gow as a function of model size. The heuistic we have chosen begins with a list of all mm-1 a f pais of model segments and passes this list 2 though thee filtes: Remove nealy paallel pais of segments: Remove pais diffeing in oientation by less then five degees. Retain pais with poximal endpoints: Sot the emaining pais in ascending ode accoding to the minimum Euclidean distance between endpoints. Retain the fist m pais in this list. Retain the fou longest disjoint pais: Sot the m pais in descending ode accoding to the sum of the lengths of the two segments. Select the fist fou disjoint pais in this list. If thee ae fewe than eight model segments then do not equie the pais to be disjoint. Povided thee ae at least fou pais of nonpaallel model segments to begin with, this algoithm will always select fou pais of model segments to seve as subsets fo the subset-convegent local seach algoithm. 5 LOCAL OPTIMA AND RANDOM SAMPLING Both of the local seach algoithms descibed above ae deteministic: Stating fom any match c in the space C, local seach will move pedictably to an associated local optima. As one might expect, fo all but tivial poblems, thee ae a temendous numbe of local optima in the space C. Often this fact causes people to pematuely dismiss local seach as a useful technique. Tovey [29] makes seveal vey insightful obsevations about local seach and local optima. The fist is that a deteministic local seach algoithm imposes a foest stuctue upon the seach space. To be moe specific, the space may be viewed as a foest of tees, with the oot of each individual tee a locally optimal match. Fom nodes which ae not locally optimal, thee is path leading down the tee to the oot. These paths epesent the successive matches found by the local seach algoithm. Thee is one tee, the globally optimal tee, whose oot is the global optima. 4 To visualize what this foest looks like, imagine placing each node in the seach space at a height off the gound coesponding to the match eo: Nodes with lage eo ae highe. Only one banch leads down fom each node, 4. Fo simplicity, ties fo best ae ignoed.

9 572 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 19, NO. 6, JUNE 1997 and this epesents the move fom one state to anothe taken by steepest-descent local seach. At the bottom of each tee is a oot node epesenting the local optima found by local seach initiated fom any banch in the tee. The shape of this foest, the numbe of tees and thei elative size, ae the combined poduct of the local seach neighbohood definition, the citeion function and the specific poblem instance. 5 Tovey [29] poves that fo seveal classes of NP-complete poblems the expected numbe tees in the foest gows exponentially. He futhe expesses a belief that all NP-complete poblems have exponentially many local optima, but stops shot of offeing a poof (a poof would amount to a poof that P π NP). Fom a pactical standpoint, Tovey s most impotant obsevation is that while the numbe of local optima may explode, the elative size of the tees containing local optima vesus the size of the tee containing the global optima may emain such that andom stats local seach will continue to pefom well. Let us fomalize some of these notions. Fist, let O be the faction of the seach space C containing the global optima. Next, let us assume local seach is initiated fom a state c i unifomly sampled fom C. Unde these conditions, the pobability P s of successfully aiving at the global optima on any single independent andom tial is: P s = O (23) If P s fo a given poblem is known, then the pobability of failing to see the global optima ove t independent tials is simply Q f = (P f ) t, whee P f = 1 - P s (24) Fom (24) it is possible to compute the numbe of tials t s equied to find the optimal match with pobability Q s = 1 - Q f : t s = Èlog Pf Q f (25) In all the wok pesented in this pape, Q s is set to The Wok Requied to Solve a Specific Poblem Given a test poblem with a known solution, it is usually possible to detemine how many tials ae equied to find that solution with pobability Q s. We un k tials of local seach and ecod the numbe of times the global optima is seen, o. In classifying the esult as tue fo an optimal match and false othewise, we ae equating ou multiple tials with a binomial pocess with unknown pobability of etuning tue P t. A tue value fom the binomial pocess means success at finding an optimal match, and consequently P s = P t. This means that the maximum likelihood estimate fo P s is the atio: o $P s = (26) k It is also possible to pedict the degee of uncetainty in the estimate P $ s and these bounds fo diffeent combinations of tials and P $ s appea in [6]. 5. This inteplay between the neighbohood definition and the evaluation function makes fomal analysis of local seach difficult [29]. A good ule of thumb is that the best match must be seen some easonable numbe of times: pehaps moe than 10 times. While all the poblems pesented in this pape may be studied in this fashion, clealy thee ae limits. At some point it becomes pohibitive to un sufficient tials to eliably estimate $ P s. Typically we un 100 tials fo easy poblems and 1,000 fo had poblems. To detemine an expected un-time s to solve a poblem, take the estimated pobability of success P s and the aveage un-time fo a single tial of local seach, compute the equied numbe of tials t s to solve the poblem with 95 pecent confidence using (25), and multiply time-petial by the numbe of tials: s = t s (27) This measue of poblem difficulty will be used in Section 6.2 to test two altenative hypothesis about how untime gows as a function of poblem size Biased Random Sampling The choice of initial andom stating matches need not be unifom, and it is common [3], [29] to bias andom selection of stating states in ode to impove the likelihood that the state is in the tee leading to the global optima. While intoducing bias destoys the stict intepetation of P s as the faction of the space spanned by the globally optimal tee, P s may still be eliably estimated using andom sampling, and (25), (26), and (27) still hold. Random initial coespondences ae geneated with, on aveage, l data segments matched to each model segment. This is done by defining a binding pobability P Mi fo each model segment M i : P Mi c Kih (28) = min 0.5, l / whee K i is the numbe of model-data pais included in M i. When building the initial match, a pai s Œ S is included in the initial coespondence c i with pobability P Mi. Fo the example in Fig. 1, l = 4. Elsewhee, l = 2 unless othewise noted How-Many-Tials? To illustate these ideas with a concete example, conside the global optima shown in Fig. 5. Fo the subsetconvegent algoithm, this match was found in 761 out of 1,000 andom tials, and the aveage un-time pe tial was = 2.2 seconds. Consequently, P$ = 076. t = 3tials = 66. seconds (29) s s s Fo the steepest-descent algoithm on this same poblem, the global optima was found in 70 out of 1,000 tials and the aveage un-time pe tial was = 0.8 seconds. Thus, P$ = t = 42 tials = 33.6 seconds (30) s s s 5.2 Example Optima The Good, the Bad, and the Ugly Fig. 6 shows a sampling of local optima fo the Tee example pesented in Fig. 5. Figs. 6a and 6b demonstates that

10 BEVERIDGE AND RISEMAN: HOW EASY IS MATCHING 2D LINE MODELS USING LOCAL SEARCH? 573 local seach is finding patial symmeties in the model. These local optima tell us something about the stuctue of ou models: Fo instance, evealing the self-simila stuctue of the tee banching stuctue. Howeve, most local optima ae uninteesting. Figs. 6c and 6d show two such matches. The match eo anks these local optima as wose than those aising out of the symmety in the tee banching stuctue. (a) (a) (b) (b) Fig. 8. Test data. (a) Random clutte. (b) Multiple instances. (c) (d) Fig. 6. Local optima: (a) Model shifted up with E match = (b) Model shifted down with E match = (c) E match = (d) E match = CHARACTERIZING PERFORMANCE A test suite of 48 distinct matching poblems is used in this study. They ae deived fom the six stick figue models shown in Fig. 7. We and othes [30] have used this test suite in the past to benchmak local seach matching algoithms [6], [14] and to compae local seach with genetic algoithms [28]. The test suite as well as ou optimal matches ae available though ou web site: vision Each model is defined by a set of 2D staight line segments. In matching, these models may be otated and tanslated to lie anywhee in the image. In addition, model size is allowed to vay. The models have been selected to be simple enough to pemit study yet vaied enough to test fo possible weaknesses in a matching algoithm. Fo example, the dandelion exhibits a 16-fold nea symmety. Symmeties in models complicate matching fo Fig. 7. Six stick figue models used in tests. many well established techniques [16]. The leaf pesents an example whee model and data line segments appoximate a cuved contou. In this case, a many-to-many mapping between model and image segments is needed to account fo beakpoints falling at diffeent positions along the cuve.

11 574 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 19, NO. 6, JUNE 1997 TABLE 1 EXPECTED RUN-TIMES SORTED BY PROBLEM SIZE N s (seconds) Ratio M C. I. n SD SC SD SC Po Po Re Po Po Re Re Po De Po Po De Po Re Re Re Re T Re De De T De De Da T Le De Le De T Da Da T Le T T Da Le Le Da , Da , T , Le Le , , Da 0 3 1,130 8, , Le 0 4 1, , Da 0 4 1,293 10, , M C I n SD SC s Ratio LEGEND Model Numbe of Clutte Lines Numbe of Model Instances Numbe of Model-Data Pais Steepest-descent Subset-convegent Time to 95 pecent Pob. Optimal Amount Faste Than Othe

12 BEVERIDGE AND RISEMAN: HOW EASY IS MATCHING 2D LINE MODELS USING LOCAL SEARCH? 575 Fig. 9. Estimated un-times s as a function of n. Times ae boken out by algoithm and andom clutte vesus multiple model instances. Both polynomial and exponential egession lines ae shown. A Monte Calo simulato poduces coupted image data. The simulato otates, tanslates, and scales the model so placement and size is unknown. Model segments ae also fagmented and skewed. In 24 of the poblems, zeo, 10, 20, and 30 additional clutte segments ae andomly placed about the image. A sampling of this data is shown in Fig. 8a. In the othe 24 poblems, one, two, thee, and fou instances of the model ae added to the image. A sampling of this data is shown in Fig. 8b. 6.1 Steepest-Descent Vesus Subset-Convegent Local Seach Having un each algoithm 1,000 times on each of the 48 poblems on a Spac 10, we have eliable estimates of and P s. Fom these, we compute t s and s. Values fo s fo each poblem ae given in Table 1. Size n = md is indicated fo each poblem. Due to fagmentation, d may be lage than m, even when no clutte is pesent. Table 1 indicates SC does bette on 30 out of the 48 poblems. Moeove, while SD is neve moe than 3.3 times faste than subset-convegent on any poblem, SC is as much as 38.5 times faste than SD. Oveall, to solve the entie test suite, SC would equie 6.9 hous compaed to 8.9 hous using SD. An inteesting diffeence does emege elative to which models do bette using which algoithms. Fo both the pole and tee, the SC algoithm is bette on eight out of eight poblems. Fo the dandelion, SC is still doing well, pefoming bette on six out of eight poblems. Fo both the ectangle and dee, SC is bette on only thee out of eight poblems. Finally, fo the leaf poblem, SC does bette on only two out of eight poblems. It appeas the diffeence between these two algoithms may depend upon model stuctue, but it is not immediately appaent why these diffeences aise. 6.2 Run-Time Vesus Poblem Size The estimated un-times s ae chated on log plots shown in Fig. 9. The multiple model instance poblems have been boken out fom the andom clutte poblems. By sepaating these two cases, the gowth tend can be examined fo each independently. Within these two poblem classes, poblems deiving fom the six diffeent models have not been distinguished. Also shown in Fig. 9 ae two nonlinea egession cuves. These ae deived using standad nonlinea egession techniques, as descibed in [31]. Two altenative statistical models ae poposed fo how un-time vaies as a function of poblem size n: Polynomial s = an b Exponential s = ae bn (31) The exponential egession comes out as a staight line on these log plots and is easily distinguished fom the polynomial egession cuve. Qualitatively, it appeas the polynomial model is a bette fit to the data. Moeove, the nomalized coefficients of detemination, o R 2 values shown in Table 2 suppot this intepetation. R 2 measues the popotion of the vaiation explained by the egession cuve. Fo both algoithms applied to andom clutte poblems, the R 2 values ae substantially highe fo the polynomial model. Fo the SD algoithm on the multiple instance poblems, R 2 fo the polynomial model is also highe. The one ambiguous case is the SC algoithm applied to multiple instance poblems. Hee, each egession model is

13 576 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 19, NO. 6, JUNE 1997 TABLE 2 SUMMARY OF RUN-TIME REGRESSION STATISTICS Random Clutte Multiple Instances Poly. Exp. Poly. Exp. Alg. R 2 b R 2 R 2 b R 2 SD SC TABLE 3 R2 VALUES FOR LN(PS) VS. PROBLEM ATTRIBUTES m d n x d & n n & x d & x Random Multiple equally bad. The compaatively low R 2 values ae indicative of the high poblem-to-poblem vaiance in un-time elative to poblem size. Fo SC, some poblems ae unning much faste, and some ae not. The side-by-side plots fo SD and SC algoithms have the same vetical scaling, so one can see that the most significant diffeence is the emegence of some much lowe un-times on the SC plot. The exponents b fo the polynomial model ae also shown in Table 2. Fo the andom clutte cases, these empiical estimates ae supisingly close to the n 2 aveage case bounds deived fo tee seach by Gimson [15], [16]. Howeve, note that ou andom clutte data includes the Dandelion model, which because of its nea symmety would cause tee seach geat difficulty. Fo the multipleinstance poblems, the gowth ate is highe, tending up towad 2.5 athe than Relating P s to Poblem Attibutes By unning many tials on each poblem, we have estimated P s and hence t s and s fo each poblem. While this analysis says much about how local seach behaves, it does not addess a key poblem: How many tials should be un on a novel poblem instance. While in geneal we must leave a detailed study of this issue to futue wok, we have looked into a numbe of diffeent models of how t s might depend upon measuable poblem attibutes. The stongest elationship we find is between ln(p s ) and the numbe of data segments d. This elationship holds fo the SD algoithm. It also appeas helpful to define a poblem attibute x which notes the numbe of patial symmeties in the model. Fo the dandelion, x = 16, fo the tee x = 3, and fo all othes x = 1. Table 3 gives coefficients of detemination (R 2 Values) fo diffeent combinations of poblem attibutes and shows that the combination of d and x has the highest R 2 values. Thee diffeent chaacteizations of poblem size ae tested in Table 3: m, the numbe of model segments, d the numbe of data segments, and n = md the numbe of possibly matching pais of segments. Recall that the nomalized R 2 expesses the pecent of vaiation in the dependent vaiable ln(p s ) explained by the independent vaiables. The combination of d and x explains 77 pecent of the vaiation in the andom clutte data and 71 pecent of the vaiation in the multiple instance poblems. A simila analysis fo the SC algoithm was less poductive. Thee is a highe vaiance in P s not explained by any of the attibutes consideed: The highest R 2 value was 0.16 fo the andom clutte poblems and 0.43 fo the multiple instance poblems. Fo the SD algoithm, the egession coefficients give the following elationship between ln(p s ) and vaiables d and x -ln(p s ) = d x ln(p s ) = d x (32) fo the andom clutte and multiple instance poblems espectively. This nonlinea model of how P s vaies with d and x along with (25) lets us compute the numbe of tials ou egession model pedicts t s,m = Èlog Pf 0.05 (33) Fig. 10 plots t s,m vesus the actual numbe of tials equied t s fo the 48 test poblems. Fig. 10. Tials equied actual vesus estimates deived fom multivaiate egession model Being based upon egession, t s,m undeestimates some cases and oveestimates othes. A moe consevative numbe of tials may be geneated by multiplying t s,m by a constant. Fo the 48 poblems, t s,mc = 3t s,m tials is sufficient to

14 BEVERIDGE AND RISEMAN: HOW EASY IS MATCHING 2D LINE MODELS USING LOCAL SEARCH? 577 TABLE 4 RUN-TIMES FOR REAL DATA PROBLEMS AND COMPARISONS Statistics Time Pedicted Matching Poblem n $ Ps t s s f 1 (n) f 2 (n) Building, Fig. 2 (No Placement) 1,376 6/ , ,694 Building, Fig. 3 (Placement) / Building, Fig. 3 (No Placement) 1,788 12/1, ,293 1,123 3,061 Ca, Fig. 4 (Placement) / Ca, Fig. 4 (No Placement) 1,701 25/ ,000 1,021 2,735 Times pedicted by egession models. TABLE 4 LEGEND n Numbe of Model-Data Pais $P s Times global optimum found out of total tials t s s f 1 (n) f 2 (n) Tials equied to find global optimum with 95 pecent confidence Aveage time pe tial (seconds) Seconds equied to solve poblem with 95 pecent confidence Seconds pedicted by polynomial egession with andom clutte Seconds pedicted by polynomial egession with multiple instances oveestimate t s on all but two poblems. To see if using a numbe of tials that was no longe poblem specific changes ou obseved elationship between n and un-time, we epeated the un-time egession analysis pesented in Section 6.2 using t s,mc in place of t s. On the andom clutte poblems R 2 = 0.91 fo the polynomial gowth model and R 2 = 0.85 fo the exponential model. Fo the multiple instance poblems, R 2 = 0.93 fo the polynomial model and R 2 = 0.82 fo the exponential model. Thus, using t s,mc slightly lessens the diffeence between the two models, but the polynomial gowth model is still explaining moe of the un-time vaiation elative to n. The exponents on the polynomial model ae within 0.01 of those found using t s. 7 PERFORMANCE ON THE REAL DATA EXAMPLES Table 4 summaizes how the SC algoithm pefoms on the matching poblems fom Figs. 2, 3, and 4. Fo the examples in Figs. 3 and 4, esults ae shown both with and without an initial placement estimate fo whee the model appeas in the image. Table 4 compaes un-times needed to solve these specific poblems with un-times pedicted by the polynomial egession lines shown in Fig. 9. It is pehaps supising how well these un-times backet the actual un-times fo fou out of the five cases. The one exception is the example fom Fig. 3 when no placement estimate is available. Exactly why this poblem is so had is not cetain. Howeve, a easonable conjectue is that the un-time scaling as a function of n seen in the test suite should not be expected when n gows solely due to lage numbes of image segments. 8 SOME OBSERVATIONS The pefomance of local seach as a geneal method fo finding matches appeas as good o bette than any of the known altenative geneal methods [16], [17], [32]. Howeve, thee ae some impotant caveats. Fist, while local seach pobabilistically finds optimal matches, these othe techniques deteministically find acceptable (not optimal) matches. Compaison at a coase level is infomative but also somewhat poblematic. Second, while local seach does well with clutte, multiple model instances, highly fagmented data, and symmetic models, in its cuent fom on cuent machines it will not solve poblems involving tens of thousands of possibly matching pais of line segments. In pactice, eithe the complexity of an image must not be excessive, o altenatively, constaints must povide focus of attention within the image. Finally, we have not yet mentioned anything about methods which use some fom of pose indexing to geneate match hypotheses and then exploe these eithe in sequence o in paallel. This line of wok is useful and impotant, so let conside biefly how it elates to ou local seach technique. 8.1 A Comment About Indexing The stengths and weaknesses of ou method come in lage pat fom the lack of any indexing phase. The initial matches ae dawn at andom fom the seach space with no attempt to discen o capitalize upon localized stuctue o domain specific constaints. This lack of eliance upon indexing sets ou appoach apat fom much of the pio wok on geometic object ecognition and makes ou algoithm obust acoss a wide ange of poblem types. Howeve, it also places limits on what poblems can be solved. Going back to Robets [33], thee has been a ich tadition of wok that says essentially: To find an object, fist find a small subset of featues that pedict the pesence of the object. This geneal appoach to ecognition can be taced though many woks including [22], [34], wok on geometic hashing schemes [35], and on though a collection of excellent ecent woks [36], [37], [38], [39], [40]. Gimson et al. povide a nice geneal analysis of the poblem [41]. The fundamental difficulty in designing indexing algoithms is efficiently finding eliable sets of domain inde-

15 578 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 19, NO. 6, JUNE 1997 TABLE 5 TRIALS TO DRAW K GOOD PAIRS VS. TRIALS OF SC LOCAL SEARCH Tials 95 % Confidence Poblem Desciption Daw 3 Daw 4 Local Seach Model g n Good Pais Good Pais Optimal Match Rectangle (Fig. 1) ,948 16,899 8 Tee (Fig. 5) ,175 62,118 3 Tee (30 Clutte Lines) ,334 7,435,809 2 Building (Fig. 2) 19 1,375 1,135,402 82,167, Building (Fig. 3) 12 1,788 9,909,727 1,476,549, pendent indexing featues. Hence, indexing is fequently solved using domain specific heuistics. In some application domains, such as 2D pat ecognition, these heuistics ae easily developed. Howeve, they may not genealize acoss domains fo instance, fom polygonal to nonpolygonal models. Random sampling plays such a key ole in ou appoach that it is woth undestanding that andom sampling alone is not a good way of selecting consistent indexing featues. Random sampling to find small subsets of consistent featues has been suggested and put to good use unde some conditions [42], [43], [44]. Howeve, if fo the poblems studied in this pape the set of pais S is patitioned into good and bad pais, S = G B and G B = (34) whee pais in G belong to the optimal match, then the pobability of dawing k good pais at andom fom S is: P g n k whee g G, n S (35) = F H G I K J = = Table 5 lists values of g and n fo a sampling of matching poblems as well as how many independent andom tials it would take to daw subsets of thee o fou good pais with 95 pecent confidence. The numbe of tials is detemined by inseting the pobabilities fom (35) into (25). Just dawing thee-tuples o fou-tuples at andom fom S clealy does not scale well ove the poblems shown. Upwads of millions to billions of tials ae needed. Fo the sake of compaison, the numbe of tials of SC local seach ae shown in the final column of Table Conclusion Ou wok adds a new tool to the elatively small set of geneal matching techniques. Past wok has shown ou algoithm pefoms well in seveal application domains whee ough placement constaints deived fom othe souces ae available. This is tue fo both 2D and 3D [12] ecognition poblems. The empiical tests pesented in this pape suggest local seach does well on a wide ange of 2D line matching poblems even when no initial estimate of model placement is available. Local seach handles many-to-many featue mappings and optimizes global geometic consistency between model and data. by inceasing the numbe andom tials, the pobability of finding the optimal match may be made abitaily high, and though adjusting the numbe of tials the same algoithm scales between easy and had poblems. Finally, the expected aveage un-time equied to solve a poblem appea to gow as n 2 whee n is the numbe of potential paiings of model and image featues. This is compaable to, o bette than, the un-time of any othe known geneal matching technique. ACKNOWLEDGMENTS This wok was sponsoed by the Defense Advanced Reseach Pojects Agency (DARPA) Image Undestanding Pogam unde gant DAAH04-93-G-422, monitoed by the U.S. Amy Reseach Office, as well as by the National Science Foundation unde gants CDA and IRI REFERENCES [1] B.W. Kenighan and S. Lin, An Efficient Heuistic Pocedue fo Patitioning Gaphs, Bell Systems Tech. J., vol. 49, pp , [2] S. Lin and B. Kenighan, An Effective Heuistic Algoithm fo the Taveling Salesman Poblem, Opeations Reseach, vol. 21, pp , [3] C.H. Papadimitiou and K. Steiglitz, Local Seach, Combinatoial Optimization: Algoithms and Complexity. Englewood Cliffs, N.J.: Pentice-Hall, pp , [4] J.R. Beveidge, R. Weiss, and E.M. Riseman, Combinatoial Optimization Applied to Vaiable Scale 2D Model Matching, Poc. IEEE Int l Conf. Patten Recognition 1990, Atlantic City, N.J., pp , June [5] J.R. Beveidge, R. Weiss, and E.M. Riseman, Optimization of 2-Dimensional Model Matching, Selected Papes on Automatic Object Recognition, oiginally appeaed in DARPA Image Undestanding Wokshop, 1989, H. Nas, ed., SPIE Milestone Seies. Bellingham, Wash., SPIE, [6] J.R. Beveidge, Local Seach Algoithms fo Geometic Object Recognition: Optimal Coespondence and Pose, PhD thesis, Univ. of Massachusetts, Amhest, May [7] R.T. Collins and J.R. Beveidge, Matching Pespective Views of Coplana Stuctues Using Pojective Unwaping and Similaity Matching., Poc IEEE CS Conf. Compute Vision and Patten Recognition, New Yok, NY, pp , June [8] C. Fennema, A. Hanson, E. Riseman, J.R. Beveidge, and R. Kuma, Model-Diected Mobile Robot Navigation, IEEE Tans. Systems, Man, and Cybenetics, vol. 20, no. 6, pp. 1,352-1,369, Nov./Dec [9] E.M. Riseman, A.R. Hanson, J.R. Beveidge, R. Kuma, and H. Sawhney, Landmak-Based Navigation and the Acquisition of Envionmental Models, Visual Navigation: Fom Biological Systems to Unmanned Gound Vehicles, Y. Aloimonos, ed., pp Lawence Elbaum Associates, Inc., [10] J.R. Beveidge. C. Gaves, and C.E. Leshe, Local Seach as a Tool fo Hoizon Line Matching, Poc. Image Undestanding Wokshop, Los Altos, Calif., Feb. 1996, pp , ARPA. Mogan Kaufmann.

16 BEVERIDGE AND RISEMAN: HOW EASY IS MATCHING 2D LINE MODELS USING LOCAL SEARCH? 579 [11] B.A. Dape, Leaning Object Recognition Stategies, PhD. thesis, Univ. of Massachusetts, Amhest, May [12] J.R. Beveidge and E.M. Riseman, Optimal Geometic Model Matching Unde Full 3D Pespective, Compute Vision and Image Undestanding, vol. 61, no. 3, pp , Shot vesion in Poc. IEEE Second CAD-Based Vision Wokshop. [13] D.S. Johnson, C.H. Papadimitiou, and M. Yannakakis, How Easy Is Local Seach?, J. Compute and System Sciences, vol. 37, pp , [14] J.R. Beveidge, E.M. Riseman, and C. Gaves, Demonstating Polynomial Run-Time Gowth fo Local Seach Matching Poc. Int l Symp. Compute Vision, Coal Gables, Fla., pp , Nov Los Alamitos, Calif.: CS Pess. [15] W.E.L. Gimson, The Combinatoics of Object Recognition in Clutteed Envionments Using Constained Seach, Atificial Intelligence, vol. 44, no. 1, pp , July [16] W. Eic and L. Gimson, Object Recognition by Compute: The Role of Geometic Constaints. Cambidge, Mass. MIT Pess, [17] T.A. Cass, Polynomial-Time Object Recognition in the Pesence of Clutte, Occlusion, and Uncetainty, Poc. Image Undestanding Wokshop, San Mateo, Calif., pp , Jan. 1992, DARPA. Mogan Kaufmann. [18] R. Collins, A. Hanson, R. Riseman, and Y. Cheng, Model Matching and Extension fo Automated 3D Site Modeling Poc. Image Undestanding Wokshop, Los Altos, Calif., Ap. 1993, pp , ARPA. Mogan Kaufmann. [19] J.B. Buns, A.R. Hanson, and E.M. Riseman, Extacting Staight Lines, IEEE Tans. Patten Analysis and Machine Intelligence, vol. 8, no. 4, pp , July [20] M.R. Stevens and J.R. Beveidge, Pecise Matching of 3D Taget Models to Multisenso Data IEEE Tans. Image Pocessing, vol. 6, no. 1, pp , Jan [21] W.M. Wells III, Map Model Matching, Poc. CVPR 91, pp , [22] D.G. Lowe, Peceptual Oganization and Visual Recognition. Kluwe Academic Publishes, [23] N. Ayache and O.D. Faugeas, Hype: A New Appoach fo the Recognition and Positioning Of 2D Objects, IEEE Tans. Patten Analysis and Machine Intelligence, vol. 8, no. 1, pp , Jan [24] J.R. Beveidge, R. Weiss, and E.M. Riseman, Optimization of 2-Dimensional Model Matching, Poc. Image Undestanding Wokshop, Los Altos, Calif., pp , June 1989, DARPA.: Mogan Kaufmann. [25] L.S. Davis, Hieachical Genealized Hough Tansfoms and Line-Segment Based Genealized Hough Tansfoms, Patten Recognition, vol. 15, no. 4, pp , [26] B. Noble and J.W. Daniel, Applied Linea Algeba, 2d. ed. Englewood Cliffs, N.J.: Pentice-Hall, Inc, [27] V. Ramesh, Pefomance Chaacteization of Image Undestanding Algoithms, PhD thesis, Univ. of Washington, [28] D. Whitley, J.R. Beveidge, C. Gaves, and K. Mathias, Test Diving Thee 1995 Genetic Algoithms: New Test Functions and Geometic Matching J. Heuistics, vol. 1, pp , [29] C.A. Tovey, Hill Climbing with Multiple Local Optima, SIAM J. Alg. Disc. Meth., vol. 6, no. 3, pp , July [30] C. Loade, Local Seach Algoithms fo 2D Geometic Object Recognition MS thesis, Univ. of Westen Austalia, [31] Jay L. Devoe, Pobability and Statistics fo Engineeing and the Sciences, Nonlinea and Multiple Regession, pp Monteey, Calif.: Books/Cole Pub. Co., [32] H.S. Baid, Model-Based Image Matching Using Location. Cambidge, Mass: MIT Pess, [33] L.G. Robets, Machine Peception of Thee Dimensional Solids, Optical and Electo-Optical Infomation Pocessing, J.T. Tippett, ed., chapte 9, pp Cambidge, Mass.: MIT Pess, [34] R.C. Bolles and R.A. Cain, Recognizing and Locating Patially Visible Objects: The Local-Featue-Focus Method Int l J. of Robotics Reseach, vol. 1, no. 3, pp , [35] Y. Lamdan, J.T. Schwatz, and H.J. Wolfson, Affine Invaiant Model-Based Object Recognition, IEEE Tans. Robotics and Automation, vol. 6, no. 5, pp , Oct [36] F. Stein and Géad Medioni, Recognition of 3D Objects Fom 2d Goupings, Poc. Image Undestanding Wokshop, San Mateo, Calif., pp , Jan. 1992, DARPA. Mogan Kaufmann. [37] A.R. Pope and D.G. Lowe, Leaning Object Recognition Models Fom Images, ICCV, 1993, pp [38] A.R. Pope, Model-Based Object Recognition, Technical Repot, Univ. of Bitish Columbia, Jan [39] C.F. Olson, Time and Space Efficient Pose Clusteing, Poc. CVPR94, 1994, pp [40] C.F. Olson, On the Speed And Accuacy Of Object Recognition When Using Impefect Gouping, Poc. SCV95, 1995, pp , [41] W.E.L. Gimson, D.P. Huttenloche, and D.W. Jacobs, A Study of Affine Matching With Bounded Senso Eo Poc. IJCV, vol. 13, no. 1, pp. 7 32, Sept [42] M.A. Fischle and R.C. Bolles, A Paadigm fo Model Fitting with Applications to Image Analysis and Automated Catogaphy, epinted in Readings in Compute Vision, M.A. Fischle, ed. Comm. ACM, vol. 24, no. 6, pp , June [43] Gehad Roth, Extacting Geometic Pimitives, Compute Vision, Gaphics, and Image Pocessing-Image Undestanding, vol. 58, no. 1, pp. 1-22, July [44] R. Kuma and A.R. Hanson, Robust Methods fo Estimating Pose And A Sensitivity Analysis, Poc. CVGIP: Image Undestanding, vol. 11, J. Ross Beveidge eceived his BS degee in applied mechanics and engineeing science fom the Univesity of Califonia at San Diego in 1980, and his MS and PhD degees in compute science fom the Univ. of Massachusetts in 1987 and 1993, espectively. He has been an assistant pofesso in the Compute Science Depatment at Coloado State Univesity since He is on the editoial boad of Patten Recognition. His pesent inteests include object ecognition, senso fusion, image featue extaction, and softwae development envionments fo compute vision. He is a membe of the ARPA Image Undestanding Envionment Technical Advisoy Committee. Edwad M. Riseman eceived his BS degee fom Clakson College of Technology in 1964, and his MS and PhD degees in electical engineeing fom Conell Univesity in 1966 and 1969, espectively. He joined the Compute Science Depatment as assistant pofesso in 1969, has been a full pofesso since 1978, and seved as chaiman of the depatment fom 1981 to Pofesso Riseman has been diecto of the Compute Vision Laboatoy since its inception in Recent eseach pojects in the lab include knowledge-based scene intepetation, motion analysis, mobile obot navigation, site model constuction fo aeial photo intepetation and teain econstuction fo visualization. Pofesso Riseman cuently seves on the editoial boad fo the Intenational Jounal of Compute Vision, is a senio membe of IEEE, and a fellow of the Ameican Association of Atificial Intelligence. He is co-edito of Compute Vision Systems (Academic Pess, 1978), and is a founde of Ameinex Atificial Intelligence, Inc. (AAI), and Dataview Copoation.