Object and image indexing based on region connection calculus and oriented matroid theory

Transcription

1 Discrete Applied Mthemtics 147 (2005) Oject nd imge indexing sed on region connection clculus nd oriented mtroid theory Ernesto Stffetti, Antoni Gru, Frncesc Serrtos c, Alerto Snfeliu d Deprtment of Computer Science, University of North Crolin t Chrlotte, 9201 University City Blvd., Chrlotte NC, USA Deprtment of Automtic Control, Technicl University of Ctloni, Pu Grgllo 5, Brcelon, Spin c Deprtment of Computer Engineering nd Mthemtics, Rovir i Virgili University, Av. Pïsos Ctlns 26, Trrgon, Spin d Institute of Industril Rootics (CSIC-UPC), Llorens i Artigs 4-6, Brcelon, Spin Received 10 Decemer 2003; received in revised form 18 My 2004; ccepted 3 Septemer 2004 Astrct In this pper novel method for indexing views of 3D ojects is presented. The topologicl properties of the regions of views of n oject or of set of ojects re used to define n index sed on region connection clculus nd oriented mtroid theory. Both re formlisms for qulittive sptil representtion nd resoning nd re complementry in the sense tht, wheres region connection clculus chrcterize connectivity of couples of connected regions of views, oriented mtroids encode reltive position of disjoint regions of views nd give locl nd glol topologicl informtion out their sptil distriution. This indexing technique hs een pplied to hypothesis genertion from single view to reduce the numer of cndidtes in 3D oject recognition processes Elsevier B.V. All rights reserved. Keywords: Imge indexing; Oject recognition from single view; Oriented mtroid theory; Region connection clculus E-mil ddresses: ernesto.stffetti@ieee.org (E. Stffetti), ntoni.gru@upc.es (A. Gru), frncesc.serrtos@etse.urv.es (F. Serrtos), snfeliu@iri.upc.es (A. Snfeliu) X/$ - see front mtter 2004 Elsevier B.V. All rights reserved. doi: /j.dm

2 346 E. Stffetti et l. / Discrete Applied Mthemtics 147 (2005) Introduction Given one or more imges contining different views of the sme oject or of the sme set of ojects, fundmentl prolem of computer vision is tht of recognizing the ojects represented in the imges using models of ojects known priori. The components of n oject recognition system re represented in Fig. 1. It consists in generl of dtse of models of views of known ojects, in which they re represented y those ttriutes tht re relevnt to chrcterize ech of them in reltion to the others. The informtion stored in the dtse of models depends on the method employed for recognition nd vice vers. Another component of n oject recognition system is feture extrctor which computes the relevnt fetures of the imges of the ojects to recognize. These fetures re used y the hypothesis genertor to ssign likelihoods to the ojects present in the imge. Hypothesis genertion reduces the serch spce for the recognition process. The hypothesis verifier then uses oject models to refine the likelihood nd finlly the systems selects the ojects with the highest likelihood s the ojects present in the views. However, the reltive importnce ssigned to these two components in different oject recognition systems vries. Oject representtion for recognition cn e sed either on ttriutes of the 3D geometry of the ojects or on ttriutes of their 2D projections. These two pproches re referred to s oject-centered nd view-centered representtions, respectively. View-centered representtions chrcterize 3D ojects y mens of set of 2D views, resulting in significnt reduction in the dimensionlity of the prolem y compring 2D imges rther thn compring 3D ojects. In this cse 3D ojects re represented s imge fetures nd reltionships mong them. Ech 3D oject hs infinitely mny different views tht correspond to the infinitely mny possile points of view. However, the complete set of views of 3D oject is redundnt nd for efficiency resons the set of views used to chrcterize n ojects must e reduced to miniml set. Therefore, the concept of spect or chrcteristic view [21] hs een developed. Indeed, the view spce cn e prtitioned into finite numer of regions clled chrcteristic view domins so tht views from different chrcteristic view domins re topologiclly distinct wheres views elonging to the sme domin re not. The different chrcteristic views of n oject cn e relted using dt structure clled spect grphs [11]. An spect grph enumertes ll the possile ppernces of n oject nd their djcency reltionships. In this context, the chnge in ppernce t the oundry etween different spects is clled visul event. It is esy to see tht, using chrcteristic views, oth the numer of stored view models nd the numer of comprisons tht must e performed t run time for oject recognition is minimized. Using set of view the prolem of oject recognition cn e reduced to the prolem of 2D imge recognition. We suppose to use single view for 3D oject recognition. Views Feture extrction Fetures Hypothesis genertion Cndidte ojects Hypothesis verifiction Oject Model dtse Fig. 1. Components of n oject recognition system.

3 E. Stffetti et l. / Discrete Applied Mthemtics 147 (2005) In oject recognition systems sed on single view the ojects cn e chrcterized y glol, locl or reltionl fetures of their views. Glol fetures re usully chrcteristics of interior points or of the oundry of regions of the views wheres locl fetures mke reference to smll portions of the views. A etter chrcteriztion of view of n oject cn e done using reltionl fetures in which the reltive position of glol or locl fetures is tken into ccount. The pproch to pttern recognition sed on reltionl fetures is lso clled structurl pttern recognition nd grphs re the most used reltionl dt structures. Clssifiction is the stndrd oject recognition method from single views ut in some cses clssifier cnnot e implemented, for instnce ecuse the priori knowledge out feture proilities nd clss proilities is not ville. In such cses direct mtching of the models to the unknown oject cn e pplied nd the est mtching model is selected. If the views re modeled using reltionl fetures, the mtching will e symolic, nd, if grphs re used s reltionl dt structures, the oject recognition prolem is then trnsformed into grph mtching prolem [1,10,13,16,17]. Given view of one or more 3D ojects, the prolem of oject recognition using single view ecomes the prolem of finding suset of the set of regions of the view with reltionl structure similr to tht of set of views stored in the dtse. Given the grph tht represents view of n oject, the symolic mtching technique descried ove requires sequentil comprison etween this grph nd ll the grphs tht chrcterize ll the views of the dtse. It is esy to see tht this strtegy is prohiitive with lrge numer of view models. To llevite this prolem, the concept of spect grph introduced ove cn e used to reduce the numer of grphs models in the dtse y representing ech chrcteristic view using single grph. Nevertheless, in the presence of lrge numer of ojects, it is essentil to reduce the serch spce using hypothesis genertor. Feture indexing cn e used for this purpose. The ide ehind fetures indexing is tht when certin feture is detected in the view of the oject to recognize, it cn e used to reduce the serch spce. Indeed, the serch for the est mtching model cn e restricted to the suset of the set of view models of the dtse tht contin tht feture. However, to e of prcticl interest for oject recognition, hypothesis genertion should e reltively fst lthough imprecise procedure in which reduced numer of possile cndidtes for mtching re generted. In this wy the verifiction cn e crried out using more complex, nd therefore, slower procedure ut over reduced numer of cndidtes [17]. In this pper n hypothesis genertion strtegy sed on indexing technique tht comines region connection clculus nd oriented mtroid theory is presented [19]. More precisely, the type of connectivity etween connected regions of the views is descried y mens of the formlism of region connection clculus [7], wheres the topologicl properties of the disconnected regions of the views re encoded into dt structure clled set of cocircuits [4]. The set of cocircuits, tht re one of the severl comintoril dt structure referred to s oriented mtroids, encode incidence reltions nd reltive position of the regions of n imge nd give locl nd glol topologicl informtion out their sptil distriution. Oriented mtroids provide strightforwrd mthemticl interprettion of the concept of chrcteristic view nd intrinsiclly contins informtion out their djcency reltionship, i.e, out the spect grph of the oject. Resoning with region connection clculus is sed on rules, while oriented mtroids permit lgeric techniques to e used. These two reltionl dt structures re used together to crete n index of the

4 348 E. Stffetti et l. / Discrete Applied Mthemtics 147 (2005) dtse of views. This indexing method hs een pplied to hypothesis genertion for 3D oject recognition from single view tht cn e regrded s qulittive counterprt of the geometric hshing technique [12]. For other pproch to shpe representtion nd indexing sed on comintoril geometry see [6,2,3]. Region connection clculus nd oriented mtroid theory re introduced in Section 2 wheres Section 3 descries the proposed indexing method. In Section 4, some experimentl results re reported nd, finlly, Section 5 contins the conclusions. 2. Qulittive sptil representtion Qulittive resoning is sed on comprtive knowledge rther thn on metric informtion. Mny methods for shpe representtion nd nlysis re sed on extrcting points nd edges tht re used to define projectively invrint descriptors. In this pper, insted of points, regions of the imges re tken into ccount. The motivtion ehind this choice is tht the regions of n imge cn e more relily extrcted thn vertices, edges or contours. Segmented views of 3D ojects re considered, i.e., imges prtitioned into simply connected regions hving perceptully homogeneous chrcteristics. Since our pproch to imge representtion nd indexing is comintoril, it is importnt tht the numer of regions is low. Therefore, we implicitly suppose tht we cn control the mximum numer of regions resulting from imge segmenttion, s in the method descried in [8], in such wy tht too fine segmenttion tht would produce prohiitive complexity of the resulting representtion cn e voided. In the following sections two formlisms for qulittive representtion nd resoning re descried: the first one is sed on region connection clculus nd the second one is derived from oriented mtroid theory Region connection clculus For ech simply connected region of n imge we cn qulittively distinguish the interior, the oundry, nd the exterior of the region, without tking into ccount its concrete shpe or size. A set-theoreticl nlysis of the possile reltions etween ojects sed on the ove prtition is provided y Egenhofer nd Frnzos [9]. The reltion etween ojects tht they exmine is the intersection etween their oundries nd interiors. This setting is sed on the distinction of the vlues empty nd non-empty for the intersection. If the exterior is regrded s prt of the oject itself, there re three prts for ech oject tht must e compred with three prts of nother oject. This gives rise to mny set-theoreticlly distinguishle reltions, ut it is shown in [9] tht some of them cnnot occur. Actully only nine of them hve meningful interprettion in physicl spce nd re referred to s disjoint, meet, equl, inside, covered y, contins, covers, nd overlp (oth with disjoint or intersecting oundries). Consider for instnce region A with hole B. Using this formlism, the reltionship etween A nd B is descried s A contins B or B inside A. Some vrints of this theory were developed y Cohn nd his coworkers in series of ppers (see for exmple [7]). In this work the distinction etween interior nd the oundry of n oject is ndoned, nd eight topologicl reltions derived from the single inry

5 E. Stffetti et l. / Discrete Applied Mthemtics 147 (2005) DC(, ) EC(, ) PO(,) TPP(,) NTPP(, ) EQ(,) () () (c) (d) (e) (f) Fig. 2. Some of the 8 possile reltive positions of two regions nd the corresponding descriptions using the formlism of region connection clculus. The other two cn e otined from (d) nd (e) interchnging with. In the sitution () is disconnected from, in () is externlly connected to, in the sitution (c) is prtilly overlpped to, in (d) is tngentil proper prt of, in (e) is non-tngentil proper prt of nd, finlly, in the sitution (f) nd coincide. OUT(, ) P-INS(,) INS(, ) () () (c) Fig. 3. Some of the possile positions of convex region with respect to the convex hull of non-convex one. Fig. 4. With the formlism of region connection clculus the reltion etween these two disconnected non-convex regions, where is prtilly inside the convex hull of nd vice vers, is denoted y P-INS P-INSi DC(, ). reltion connected to re tken into ccount. Some of them re represented in Fig. 2. Two of these reltions, nmely those of Fig. 2(d) nd (e), re not symmetricl nd, following the nottion used in [7], their inverses re denoted TPPi(, ) nd NTTPi(, ), respectively. Furthermore in [7] the theory is extended to hndle concve ojects y distinguishing the regions inside nd outside of the convex hull of the ojects. A convex oject cn e inside, prtilly inside or outside the convex hull of non-convex one (Fig. 3). If oth regions re non-convex 23 reltions etween them cn e defined. These reltions permit qulittive description of rther complex reltions, such s tht represented in Fig. 4. Moreover, y mens of this formlism clled region connection clculus it is possile, for instnce, to infer the type of connection etween two regions knowing the type of connection they hve with respect to third one. Resoning with region connection clculus is essentilly sed on rules. For other comprehensive descriptions of techniques for resoning out qulittive sptil reltionships see [18,22].

6 350 E. Stffetti et l. / Discrete Applied Mthemtics 147 (2005) Oriented mtroids Oriented mtroid theory [4,5,14] is rod setting in which some comintoril properties of geometricl configurtions relevnt for shpe representtion nd indexing, such s, incidence nd reltive position, cn e descried nd nlyzed. It provides common generliztion of lrge numer of different mthemticl ojects usully treted t the level of usul coordintes. In this section oriented mtroids will e introduced over sets of points clled rrngements of points using two comintoril dt structures clled chirotope nd set of cocircuits, which represent the min tools to trnslte geometric prolems into this formlism. In the strction process from the concrete rrngement of points to the oriented mtroid, metric informtion is lost ut the structurl properties of the rrngement of points re represented t purely comintoril level. Then technique to compute the oriented mtroid representtion for the set of regions tht compose n imge, clled rrngement of regions is presented Oriented mtroids of rrngements of points Given rrngement of points in R d 1 whose coordintes re the columns of the mtrix P =(p 1,p 2,...,p n ) of (R d 1 ) n, the ssocited rrngement of vectors is finite sequence of linerly independent vectors {x 1,x 2,...,x n } in R d represented s columns of the mtrix X = (x 1,x 2,...,x n ) of (R d ) n where ech point p i is represented in homogeneous coordintes s x i = ( p i 1 ). To encode the comintoril properties of the rrngement of points we cn use dt structure clled chirotope [14], which cn e computed y mens of the ssocited rrngement of vectors X. The chirotope of X is the mp χ X :{1, 2,...,n} d {+, 0, } (λ 1, λ 2,...,λ d ) sign ([x λ1,x λ2,...,x λd ]), (1) in which the rckets stnd for determinnt, tht ssigns to ech d-tuple of vectors of the finite rrngement X sign + or depending on whether it forms sis of R d hving positive or negtive orienttion, respectively. Function (1) ssigns the vlue 0 to those d- tuples tht do not constitute sis of R d. The chirotope descries the incidence structure etween the points of X nd the hyperplnes spnned y the sme points nd, t the sme time, encodes the reltive loction of the points of the rrngement with respect to these hyperplnes themselves. This representtion sys, for exmple, which points of X lie on the positive side of given hyperplne, which on the negtive side, nd which on the hyperplne itself. Consider the rrngement of points P represented in Fig. 5 whose ssocited rrngement of vectors X is given in Tle 1. The chirotope χ X of this rrngement of vectors is given y the orienttions listed in Tle 2. The element χ(1, 2, 3) =+indictes tht in the tringle formed y p 1, p 2, nd p 3 these points re counterclockwise ordered (Fig 6). These orienttions cn e rerrnged in n equivlent dt structure clled set of cocircuits of X shown in Tle 3. The set of cocircuits of X is the set of ll prtitions generted y the lines pssing through two points of the rrngement. For exmple, (0, 0, +, +, +, +)

7 E. Stffetti et l. / Discrete Applied Mthemtics 147 (2005) p 1 p 5 p 2 p 6 p 3 p 4 Fig. 5. A plnr point configurtion. Tle 1 Vectors tht corresponds to the points of the plnr rrngement represented in Fig. 5 x 1 = (0, 3, 1) T x 2 = ( 3, 1, 1) T x 3 = ( 2, 2, 1) T x 4 = (2, 2, 1) T x 5 = (2, 2, 1) T x 6 = (0, 0, 1) T Tle 2 Chirotope of the plnr rrngement of points represented in Fig. 5 χ(1, 2, 3) =+ χ(2, 3, 4) =+ χ(3, 4, 5) =+ χ(4, 5, 6) =+ χ(1, 2, 4) =+ χ(2, 3, 5) =+ χ(3, 4, 6) =+ χ(1, 2, 5) =+ χ(2, 3, 6) =+ χ(3, 5, 6) = 0 χ(1, 2, 6) =+ χ(2, 4, 5) =+ χ(1, 3, 4) =+ χ(2, 4, 6) =+ χ(1, 3, 5) =+ χ(2, 5, 6) = χ(1, 3, 6) =+ χ(1, 4, 5) =+ χ(1, 4, 6) = χ(1, 5, 6) = p 1 p 5 p 2 + p 6 p 3 p 4 Fig. 6. In the tringle formed y p 1, p 2, nd p 3 these points re counterclockwise ordered. This is recorded in the chirotope y χ(1, 2, 3) =+.

8 352 E. Stffetti et l. / Discrete Applied Mthemtics 147 (2005) Tle 3 Set of cocircuits of the plnr rrngement of points represented in Fig. 5 Ordered couples of points Cocircuits p 1 p 2 p 3 p 4 p 5 p 6 (p 1,p 2 ) (p 1,p 3 ) (p 1,p 4 ) (p 1,p 5 ) 0 0 (p 1,p 6 ) (p 2,p 3 ) (p 2,p 4 ) (p 2,p 5 ) (p 2,p 6 ) (p 3,p 4 ) (p 3,p 5 ) (p 3,p 6 ) (p 4,p 5 ) (p 4,p 6 ) (p 5,p 6 ) p 1 p 5 p 2 p 6 p 3 p 4 + Fig. 7. Prtition of the plne generted y the oriented ry from the point p 1 to p 4. corresponding to the oriented couple (p 1,p 2 ) mens tht the points p 3, p 4, p 5, nd p 6 lie on the hlf plne determined y the oriented ry from p 1 to p 2. Reversing ll the signs of the set of cocircuits we otin n equivlent description of the plnr rrngement of points (Fig 7). Besides chirotopes nd cocircuits there re severl dt structures cple of encoding the topologicl properties of n rrngement of points. In [14] their definitions cn e found nd it is shown tht ll of them re equivlent nd re referred to s oriented mtroids Oriented mtroid of rrngements of plnr regions Consider segmented view of 3D oject. Computing the oriented mtroid of the rrngement of regions of view is not strightforwrd since the regions tht form the

9 E. Stffetti et l. / Discrete Applied Mthemtics 147 (2005) Fig. 8. Steps of encoding of the comintoril properties of view of n oject into chirotope. imge cnnot e reduced to points, tking for instnce their centroids, without losing essentil topologicl informtion for oject recognition. Therefore, the convex hull [15] of ech region is employed to represent the region itself. Then, couples of the resulting convex hulls re considered nd the oriented mtroid is computed sed on the loction of the other convex regions of the imge with respect to the two lines rising while merging them. Note tht, since we decompose n imge into set of simply connected regions, holes re considered s seprte regions. In this wy we do not need to compute the convex hull of regions with holes tht would eliminte the hole thus generting chnge in the topology of the regions of the imge. Consider, for instnce, the couple of convex hull of regions (S, T ) of Fig. 8(). It is esy to see tht the convex hull of these two plnr convex disconnected polygonl regions is polygon whose set of vertices is included in the union of the set of vertices of S nd T. On the contrry, the set of edges of the convex hull of S nd T is not included in the union of their set of edges. Indeed, two new ridging edges, e 1 nd e 2, pper s illustrted in Fig. 8(). Actully, efficient lgorithms for merging convex hulls re sed on finding these two edges [20]. Consider the two lines l 1 nd l 2 tht support e 1 nd e 2. These two lines divide the imge into three or four zones depending on the loction of their intersection point with respect to the imge. Let R S,T, L S,T of Fig. 8() e, respectively, the rightmost nd leftmost zones with respect to l 1 nd l 2 nd I S,T the zone of the imge comprised etween them. Since, R S,T, L S,T nd I S,T cn e univoclly determined from the ordered couple of regions S nd T, the loction of region U with respect to the regions (S, T ) of the imge is encoded into chirotope using the following rule χ(s, T, U) = { + if U LS,T, 0 if U I S,T, if U R S,T. (2) It hs een implicitly ssumed in (2) tht U is completely contined into either R S,T L S,T or I S,T ut, in generl, it elongs to more tht one of them. In this cse, since the rtio of res is n ffine invrint, introducing n pproximtion, we cn choose the sign sed on which region contins the lrgest portion of the re of U. If this region is split into 2 eqully sized prts etween R S,T nd I S,T, or etween L S,T nd I S,T, the sign tht corresponds to R S,T nd L S,T will e ssigned, respectively. On the contrry, if region is split into three eqully sized prts, mong R S,T, I S,T nd L S,T, the 0 sign will e ssigned. For instnce, if regions U, V nd Z re locted s in Fig. 8(c) we hve tht χ(s, T, U) =+, χ(s, T, V ) = 0 nd χ(s, T, Z) =.

10 354 E. Stffetti et l. / Discrete Applied Mthemtics 147 (2005) Invrince of the representtion Consider 3D rrngement of points nd one of its views. The chirotope of the 3D rrngement of points nd tht of its 2D perspective projection re relted in the following wy: if x 0 represents in homogeneous coordintes the center of the cmer, p 0, we hve tht sign[ x i, x j, x k ]=sign[x i,x j,x k,x 0 ], (3) where x i, x j nd x k re the homogeneous coordintes of the 3D points p i, p j nd p k, nd x i, x j nd x k re those of the corresponding points in the view, p i, p j nd p k. Eq. (3) cn e regrded s projection eqution for chirotopes. It is esy to see tht, wheres the mtrix tht represents in homogeneous coordintes the vertices of projected set of points is coordinte-dependent, the oriented mtroid of the projected set of points is coordinte-free representtion. Furthermore, it is topologicl invrint, tht is, n invrint under homeomorphisms. Roughly speking, this mens tht the oriented mtroid tht represents the rrngement of points of view of n oject does not chnge when the points undergo continuous trnsformtion tht does not chnge ny orienttion, tht is, ny sign of the chirotope. Due to this property, this representtion is roust to discretiztion errors of the imge s well s to smll chnges of the point of view tht does not chnge ny orienttion of the chirotope. This does not occur when the viewing direction is nerly prllel to the plne spnned y the three oserved points. It is esy to see tht in this cse smll displcement of the point of view my flip the plne nd hence the sign of the determinnt formed y these three points nd the point of view. The concept of chirotope of the set of views of 3D oject is relted to the concepts of chrcteristic view nd spect grph of 3D oject. An equivlency reltionship cn e defined in the set of views of 3D oject so tht ll the views from the sme chrcteristic view domin elongs to one equivlency clss, the so-clled chrcteristic view clss. Ech chrcteristic view clss cn e represented y ny of its view clled spect or chrcteristic view. The equivlence reltionship of chrcteristic view clsses is defined s follows. Two views re equivlent if nd only if they hve isomorphic grph representtion nd cn e relted y 3D geometric trnsformtion tht cn e Eucliden or projective trnsformtion. It is esy to see tht the concept of chrcteristic view clss hs nturl mthemticl interprettion y mens of the concept of chirotope. Indeed, s explined in [4, Chpter 1] the oriented mtroid of view of 3D oject contins informtion out the underlying grph representtion of the view nd isomorphic grphs hve the sme oriented mtroid representtion. Furthermore, since projective trnsformtions cn e regrded s specil homeomorphisms, we cn ssert tht the representtion of the projected set of points sed on oriented mtroids is projectively invrint. However, since ffine nd Eucliden trnsformtions re specil projective trnsformtions, the oriented mtroid of the projected set of points of view of n oject does not chnge under rottions, trnsltions, nd ffine trnsformtions of the plnr rrngement of points themselves. Moreover, the oriented mtroid representtion of set of views of 3D oject intrinsiclly contins informtion out their djcency reltionships, i.e., out the spect grph of the oject. These considertions cn e extended to the cse in which oriented mtroids represent n rrngement of plnr regions. Since the rtio of res is not invrint under projective

11 E. Stffetti et l. / Discrete Applied Mthemtics 147 (2005) Fig. 9. Three different views of the sme scene which hve the sme representtion using oriented mtroids. trnsformtions this representtion will e invrint to ffine nd Eucliden trnsformtions of the views. In Fig. 9 three different views of the sme scene hving the sme oriented mtroids re represented. 3. Indexing views of 3D ojects The process of indexing dtse of views of set of ojects strts with some preliminry choices, nmely the fetures used to chrcterize the regions of the segmented views of the set of 3D ojects. Suppose tht hue nd re re used to chrcterize ech region. Another prmeter to choose is the numer of levels in which the hue is quntized nd the numer of regions hving the sme hue tht will e tken into ccount. These choices, of course, depend on the properties of the views of the dtse. Then, the views re segmented ccording to these choices nd the convex hull of ech region is computed. As consequence, the resulting imges re compositions of convex polygonl regions clled relevnt regions which cn e disconnected or prtilly or completely overlpped. In Fig. 10 re represented two views of two ojects in which hue quntiztion with 6 levels W, R, Y, G, B nd N hs een pplied nd only the two regions hving lrgest re with the sme hue vlue re tken into ccount. Let (W, R, Y, G, B, N) e the ordered tuple of hue levels considered. For exmple, lels G 1 nd G 2 in Fig. 10 denote, respectively, the first nd the second regions of the views with the lrgest re hving the sme hue vlue G. The type of connection etween the existing regions is descried using the formlism of region connection clculus. For ech couple of disconnected regions the set of cocircuits is computed. This is done for ech view of the dtse nd this informtion is comined into unique index tle whose entries re reltionl fetures nd whose records contin list of the views in which ech feture is present. The order of the rows is not relevnt: it is consequence of the ordering of the hue levels in the tuple (W, R, Y, G, B, N). The size of

12 356 E. Stffetti et l. / Discrete Applied Mthemtics 147 (2005) Fig. 10. Two views of two ojects whose topologicl properties re indexed in Tle 4. Tle 4 Index of the topologicl properties of the two views v 1,1 nd v 1,2 of the two ojects represented in Fig. 10 Ordered couples Type of Cocircuits Views of regions connection W R Y G 1 G 2 B 1 B 2 N (W, R) DC v 1,1 (W, Y ) DC v 1,2 (W, G 1 ) NTPP v 1,1 (W, G 1 ) DC v 1,2 (W, G 2 ) DC v 1,1 (W, B 1 ) DC v 1,1 (W, B 1 ) NTPP v 1,2 (W, B 2 ) DC v 1,1 (W, B 2 ) NTPPi v 1,2 (W, N) DC v 1,1 (W, N) DC v 1,2 (R, Y ) (R, G 1 ) NTPP v 1,1 (B 2, N) DC v 1,1 (B 2, N) DC v 1,2 the index tle depends on oth the numer of hue levels nd the numer of regions hving the sme hue level tht re tken into ccount. In Tle 4 the index of the topologicl properties of the two views v 1,1 nd v 1,2 of the ojects represented in Fig. 10 is reported. In the first column the reltion etween ordered couples of regions is descried in terms of region connection clculus. The symol for certin couple (S, T ) indictes tht no view contins two regions hving fetures S nd T. This is the cse of the ordered couple of regions (R, Y ). When S nd T re disconnected,

13 E. Stffetti et l. / Discrete Applied Mthemtics 147 (2005) the corresponding cocircuit is present in the index. The symol in correspondence with certin feture indictes tht no region with tht feture is present in the views listed in the record. For exmple, the cocircuit corresponding to the couple (W, R) contins in the column Y ecuse no region with the Y feture is present in v 1,1. If (S, T ) is couple of connected regions, the corresponding row of the index is empty ecuse the cocircuit cnnot e computed Hypothesis genertion for oject recognition Consider dtse of views of set of known 3D ojects nd view v i of one of them tht we wnt to recognize. As explined in the introduction, hypothesis genertion for oject recognition from single view entils retrieving from the dtse the m views most similr to v i to reduce the serch spce in oject recognition from single view. Roughly speking n index indictes which views contin certin reltionl feture. In this pper strtegy for hypothesis genertion sed on indexing of reltionl fetures will e presented. The reltionl fetures used to crete the index re type of connection nd reltive loction of the relevnt regions of the views. In the hypothesis genertion process, the sme type reltionl fetures re clculted for the view v i nd used to ccess the index tle. Then, the m views tht est mtch v i re selected sed on counting the numer of correspondences they hve with v i in terms of type of connection nd set cocircuits which together cn e thought of s feture mtrix in which ech reltionl feture is represented y row nd ssocited to couple of relevnt regions of v i. To find the m views tht est mtch v i we follow n pproch sed on voting [12,6]. For ech reltionl feture of v i, i.e., for ech couple of relevnt regions of v i, sy (U, V ), we ccess the dtse to retrieve the list of views in which the reltionl feture corresponding to the couple (U, V ) is present. At ech itertion ech of the retrieved views receives one vote. The votes otined y ech view t ech itertion re dded together nd the m est mtches of v i will e those views of the dtse tht received the m highest scores. It is esy to see tht this method for hypothesis genertion, which cn e regrded s qulittive version of the geometric hshing technique [12], is lso roust to prtil occlusions of the ojects. Indeed, if region of n imge is occluded, the set of cocircuits cn still e computed nd therefore, the numer of correspondences with the views of the dtse cn still e clculted. In this cse, oviously, its selectivity decreses Computtionl complexity Suppose tht the segmented views re chrcterized y h hue vlues, nd tht the index is uilt tking into ccount the k lrgest regions for ech vlue of hue. Then the mximum numer of relevnt regions of view used for indexing is n = kh. Since the index is sed on comintoril chrcteriztion of the views in which ll the possile ordered couples or relevnt regions re considered, the numer of entries of the index tle, i.e., the numer of different reltionl fetures tht will e considered once the vlue n hs een chosen, is given y the product of two fctors. The first fctor is the numer of different reltionl fetures tht cn exist considering ll the comintions of couples of regions. This corresponds to the numer of comintions of the relevnt

14 358 E. Stffetti et l. / Discrete Applied Mthemtics 147 (2005) n regions tken 2 y 2, tht is n(n 1)/2. Consider couple of relevnt regions of the imge. The second fctor represents the numer of different reltionl fetures tht cn exist in correspondence to the sme couple of relevnt regions. It is given y the numer of comintions of the other (n 2) relevnt regions tken 2 y 2, tht is, (n 2)(n 3)/2. Ech reltionl feture hs (n + 1) elements, nmely 1 symol tht indictes the type of connection nd n symols tht represent the cocircuit. Thus, the memory spce needed to store the index tle is O(n 5 ). This vlue is independent of the numer of views on the dtse. Additionlly ech entry contins list of views whose size depends on the numer of views of the dtse. Feture extrction from ech imge returns n(n 1)/2 reltionl feture ech composed y (n + 1) elements. In the hypothesis genertion process the index tle hs to e ccessed to understnd which views contin certin feture vector. This entils mtching (n + 1)- dimensionl feture vectors tht represent reltionl fetures. Ech reltionl feture of v i corresponding to certin ordered couple of relevnt regions hs to e compred only with the entries of the index tle corresponding to the sme ordered couples of relevnt regions. This mounts to n(n 1)/2 (n 2)(n 3)/2 comprisons. The resulting computtionl cost is O(Cn 4 +D), where C is the cost of compring two (n+1)-tuples of symols nd D is the cost of finding the view of the dtse hving the mximum numer of correspondences with the given view. The vlue Cn 4 is independent of the numer of view of the dtse. It is importnt to er in mind tht in the hypothesis genertion method descried in this pper n is usully smll numer. If we use, for instnce, 16 hue levels nd consider the 2 lrgest regions hving the sme view n = 32. These re the vlues used in the experiments descried in the next section. 4. Experimentl results The method descried in this pper hs een fully implemented nd severl experiments hve een crried out to ssess its effectiveness in hypothesis genertion for oject recognition nd to compre it to other oject recognition techniques sed on reltionl fetures. In prticulr, it hs een compred to two other structurl methods sed on grphs. The first experiment hs een sed on the djcency grph of the regions of the views wheres in the second one structure clled function-descried grph is used to chrcterize ech view. In oth cses distnce etween grphs [16] hs een computed to select the view most similr to given view. Fig. 11 schemticlly shows the lerning nd recognition processes of the three methods (see [17] for more detils). Sixteen views of ech oject with ngulr seprtion of 22.5 hve een used for the experiments. These imges hve een segmented using the method descried in [8]. For illustrtive purposes, Fig. 12 shows one view of ech oject nd the corresponding segmented imge, with the extrcted djcency grph in which the ttriutes of the nodes represent the verge hue of the region. For ech oject, the reference set ws composed y the views tken from the ngles 0, 45, 90, 135, 180, 225, 270 nd 315 nd used to synthesize the structure of the dtse. The test set ws composed y the eight views not used in the reference set, tht is, the views tken t ngles: 22.5, 67.5, 115.5, 157.5, 202.5, 247.5, nd

15 E. Stffetti et l. / Discrete Applied Mthemtics 147 (2005) Ref. Set 1 () Ref. Set 2 Ref. Set n } AG Distnce Test Set Ref. Set 1 FDG 1 () Ref. Set 2 Ref. Set n FDG 2 FDG n } FDG Distnce Test Set Ref. Set 1 (c) Ref. Set 2 Ref. Set n } Index Tle Voting Test Set Fig. 11. The three structurl oject recognition methods from single view considered in the experiments. () Method sed on djcency grphs. () Method sed on function descried grphs. (c) Method presented in this pper. Wheres in () nd () distnce mesure is used, in (c) the selection of the m views of the test set most similr to given view is sed on counting the numer of correspondences sed on the type of connection etween regions nd on the set of cocircuits. Fig. 12. Some of the views used for the experiments of oject recognition, their segmented views nd the djcency grphs of the regions of the segmented views. Tle 5 shows the correctness of the djcency grph (AG), function-descried grphs (FDG) nd the method presented in this pper sed on indexing the topologicl properties

16 360 E. Stffetti et l. / Discrete Applied Mthemtics 147 (2005) Tle 5 Correctness of the three oject recognition methods from single view considered in the experiments Method Correctness AG 59% FDG 78% RCC & OMT 87% of the regions of the views of 3D oject using region connection clculus nd oriented mtroid theory (RCC & OMT) 5. Conclusions In this pper new method for indexing dtse of views of 3D ojects hs een presented. It is sed on the comintion of two qulittive representtions derived from region connection clculus nd oriented mtroid theory. It hs een shown tht this comintion of qulittive representtions chrcterizes the locl nd glol topology of the regions of n imge, is invrint under ffine nd Eucliden trnsformtion of the views, intrinsiclly roust to discretiztion errors of the imge nd insensitive to smll displcements of the point of view. This indexing method hs een pplied to hypothesis genertion for 3D oject recognition from single view. The experimentl results re encourging nd currently we re refining the method introducing similrity mesure etween sets of cocircuits. References [1] S. Berretti, A. Del Bimo, E. Vicrio, Efficient mtching nd indexing of grph models in content-sed retrievl, IEEE Trns. Pttern Anl. Mch. Intell. 23 (10) (2001) [2] S. Berretti, A. Del Bimo, E. Vicrio, Sptil rrngement of colors in retrievl y visul similrity, Pttern Recognition 35 (8) (2002) [3] S. Berretti, A. Del Bimo, E. Vicrio, Weighted wlkthrough etween entities for retrievl y sptil rrngement, IEEE Trns. Multimedi 5 (1) (2003) [4] A. Björner, M. Ls Vergns, B. Sturmfels, N. White, G.M. Ziegler, Oriented Mtroids, Encyclopedi of Mthemtics nd its Applictions, vol. 43, Cmridge University Press, Cmridge, [5] J. Bokowski, B. Sturmfels, Computtionl Synthetic Geometry, Lecture Notes in Mthemtics, vol. 1355, Springer, Berlin, [6] S. Crlsson, Comintoril geometry for shpe representtion nd indexing, in: J. Ponce, A. Zissermn (Eds.), Oject Representtion in ComputerVision II, Lecture Notes in Computer Science, vol. 1144, Springer, Berlin, 1996, pp [7] A. Cohn, B. Bennett, J. Goody, N.M. Gotts, Qulittive sptil representtion nd resoning with the region connection clculus, GeoInformtic 1 (3) (1997) [8] D. Comniciu, P. Meer, Men shift: roust pproch towrd feture spce nlysis, IEEE Trns. Pttern Anl. Mch. Intell. 24 (5) (2002) [9] M.J. Egenhofer, R.D. Frnzos, Point set topologicl reltions, Internt. J. Geogr. Inform. Systems 5 (2) (1991) [10] S. Gunter, H. Bunke, Self-orgnizing mp for clustering in the grph domin, Pttern Recognition Lett. 23 (4) (2002)

17 E. Stffetti et l. / Discrete Applied Mthemtics 147 (2005) [11] J. Koenderink, A. Vn Doorn, The internl representtion of solid shpe with respect to vision, Biol. Cyern. 32 (1979) [12] Y. Lmdn, J.T. Schwrtz, H.J. Wolfson, Affine invrint model-sed oject recognition, IEEE Trns. Root. Automt. 6 (5) (1990) [13] B. Luo, E.R. Hncock, Structurl grph mtching using the EM lgorithm nd singulr vlue decomposition, IEEE Trns. Pttern Anl. Mch. Intell. 23 (10) (2001) [14] J. Richter-Geert, G.M. Ziegler, Oriented mtroids, in: J.E. Goodmn, J. O Rourke (Eds.), Hndook of Discrete nd Computtionl Geometry, CRC Press, Boc Rton, FL, [15] J. O Rourke, Computtionl Geometry in C, Cmridge University Press, Cmridge, [16] A. Snfeliu, K.-S. Fu, A distnce mesure etween ttriuted reltionl grphs for pttern recognition, IEEE Trns. Systems, Mn Cyern. 13 (1983) [17] F. Serrtos, R. Alquézr, A. Snfeliu, Function-descried grphs for modeling ojects represented y ttriuted grphs, Pttern Recognition 36 (3) (2003) [18] A.P. Sistl, C. Yu, Resoning out qulittive sptil reltionships, J. Automted Reson. 25 (4) (2000) [19] E. Stffetti, A. Gru, F. Serrtos, A. Snfeliu, Shpe representtion nd indexing sed on region connection clculus nd oriented mtroid theory, in: I. Nyström, G. Snniti di Bj, S. Svensson (Eds.), Discrete Geometry for Computer Imgery, Lecture Notes in Computer Science, vol. 2886, Springer, Berlin, 2003, pp [20] G.T. Toussint, Solving geometric prolems with the rotting clipers, in: Proceedings of IEEE MELECON 83, Athens, Greece, [21] R. Wng, H. Freemn, Oject recognition sed on chrcteristic view clsses, in: Proceedings of the Interntionl Conference on Pttern Recognition, 1990, pp [22] S. Winter, Uncertin topologicl reltions etween imprecise regions, Internt. J. Geogr. Inform. Sci. 14 (5) (2000)