AN ATTRIBUTE DRIVEN APPROACH FOR IMAGE REGISTRATION USING ROAD NETWORKS

AN ATTRIBUTE DRIVEN APPROACH FOR IMAGE REGISTRATION USING ROAD NETWORKS Caixia Wang, Peggy Agouris, Anthony Stefaniis Center for Earth Observing an Space Research Departent of Earth Systes an Geoinforation Sciences (cwangg; pagouris; astefani)@ gu.eu George Mason University, Fairfax, VA 22030, USA Coission IV, WG IV/2 KEY WORDS: Georeferencing, Matching, Iagery, GIS, Feature, Transforation ABSTRACT: Geospatial analysis is becoing increasingly epenent on the integration of ata fro heterogeneous sources. In this paper, we present an autoate, feature-base approach for geoetric co-registration using networks of roas (or other siilar features). This approach is base on a graph atching schee that oels networks as graphs with ebee invariant attributes. The ain avantages of our approach resie in its ability of using both geoetric an topological attributes to reuce the abiguity in search space for inexact atching as well as its invariance to translation, rotation an scale ifferences (through the use of appropriate attributes). Furtherore, our approach requires no user-efine threshol to justify local atches. Using the inforation erive fro this atching process, the registration of two atasets can be accoplishe. 1. INTRODUCTION Geospatial analysis is becoing increasingly epenent on the integration of ata fro heterogeneous sources. The geoetric co-registration of these atasets still reains a challenging an crucial task, especially given the eergence of novel ata capturing approaches, like the use of unanne aerial vehicles (UAVs) to capture long iage sequences. In this context, registration ay refer either to the registration of iages to iages, to generate for exaple long osaics, or to the registration of iages to aps, to ientify their orientation paraeters. This registration proble becoes increasingly coplex when we consier ifferences in coverage, scale, an resolution as corresponing objects in two atasets ay also iffer to a certain extent. Roa networks usually are coon features in areas of interest. Unlike points or point-like features e.g. anhole covers (Drewniok an Rohr, 1996) or builing corners (Rohr, 2001), roa networks contain inherently substantial seantic inforation in their structure (e.g. topology an geoetry), an thus are consiere robust entities for atching in our approach base on graph atching. A great eal of effort has been evote to graph atching by the coputer vision counity. In the work of Barrow an Popplestone (1971), relational graph atching was first stuie where a relational graph is esigne to represent scene structure for atching. After that, it has been wiely aopte an evelope for atching probles. Two ajor approaches can be ientifie. One involves the construction of structural graph oel where geoetric attributes of coponents are not taken into consier. Matching techniques are evelope solely base on structure pattern, like the graph an sub-graph isoorphis approaches (Shapiro an Haralick, 1985; Pellilo, 1999; Bunke, 1999; Jain et. al., 2002). The ajor rawbacks in these graph-theoretical ethos are their coputational coplexity an inability to hanle inexact atching ue to noise or corruptions in the graph. Later works in Wilson an Hancock (1997) an Luo & Hancock (2001) exeplify soe enhanceents base on pure structural graph oel. The secon approach to the proble appreciates the easureents of network coponents an represents networks as attribute relational graphs. Matching techniques are evelope to copute graph siilarity base on these easureents an network relational structure, such as relaxation labelling algorith (Rosenfel et. al., 1976; Li,1992; Gautaa & Borgharaef, 2005), inforation theory principles (Shi an Malik, 1998), Markov Rano Fiel etho (Li, 1994). In these approaches, invariant easureents of network coponents are essential for the atching as they can reuce abiguities in local siilarity an the corresponing searching space. But ue to ifferent scope of coputer vision applications (e.g. face recognition, content-base iage retrieval) research has aresse geoetric an topological attributes of the network in a rather liite anner, focusing instea ore on perforance etrics (e.g. faster convergence). In this work, we evelop an efficient algorith of inexact graph atching using invariant attributes (geoetry an topology) inclue in geographic networks an is base on the relaxation labelling introuce by Huel an Zucker (1983). The challenges we are facing inclue the coputational coplexity of atching network coponents (i.e. junctions an polygons), as well as errors in feature extraction ue to the presence of noise in scenes, like builing-inuce shaows an occlusions. In this paper, the utilization of point networks an revise relaxation labelling provies the ability to utilize structures an geoetric attributes erive fro the network to iprove the atching algorith an thus achieve relatively efficient coputation. The process is fully autoatic in ters of no input neee fro users. These unique avantages serve both as the otivation for our work an constitute the ain contributions of this paper. The reainer of the paper is organize as follows: Section 2 escribes the foral representation of roa networks in ters of attribute relational graph. The attributes evelope for 193

The International Archives of the Photograetry, Reote Sensing an Spatial Inforation Sciences. Vol. XXXVII. Part B4. Beijing 2008 relaxation atching are escribe in Section 3. In Section 4, our revise relaxation labelling algorith for atching is escribe in etail. Experiental results are presente in Section 5. Finally, Section 6 presents conclusion an outlines our future work. 2. NETWORK PREPARATION Geographic features (roa networks) extracte fro both ata sets are first transfore into graph structure as input to our approach: extracte intersections are oelle as vertices in the graph, while roa segents between intersections are oelle as straight eges in the graph. The etection of intersections is not a topic aresse by this paper, as this is a well-researche topic in photograetry an coputer vision. We assue that roa intersections have been etecte in both atasets being registere. Figure 1 exeplifies the transforation of the roa networks in an iage. one ege, while value 0 represents no ege. By efinition ajacency atrix is invariant with respect to translation, rotation, an even scale variations between the iage an the corresponing geospatial ataset. Typically Eucliean istance is an iportant easureent of the geoetry. It is invariant to translations an rotations, but not to scale changes. In orer to overcoe this proble we use the relative istance between roa intersections as a noe-linking attribute (instea of Eucliean istance). Relative istance is efine as: where Dˆ ij Dij = (1) ( D + D ) * 0.5 ij it i, j, t = three roa intersections = relative istance between i an j Dˆ ij D ij = eucliean istance between i an j D it = eucliean istance between i an t A thir attribute (basic loop attribute) can be erive fro ajacency atrix. It is use to oel higher network topological structures, specifically the foration of close loops in it. In the case of networks the close loops are of triangle, quarangular or higher fors, an accoringly the basic loop is efine as: Figure 1. Graph representation of roa networks on iagery For the sake of clarity, we nae the graph fro iage space as G an the one fro corresponing object space as G. Corresponingly, V 1 is a vertex of G, E 1 an ege of G, V 1 a vertex of G, E 1 an ege of G Definition 2. If vertex V i has two ajacent (connecte) vertices, each of which also has one coon ajacent vertex other than V i, V i has one quarangle associate to it. 1 3 4 3. FORMALIZATION OF INVARIANT ATTRIBUTES 2 5 Invariant attributes are essential for atching as they can reuce abiguities in local siilarity an the corresponing search space. Developing invariant attributes, however, is a non-trivial issue. In one han, as the involve iagery an GIS atasets ay iffer in ters of resolution, scale, coverage, an orientation in general, the conjugate features ay also iffer to a certain extent. On the other han, as roa networks usually involve high volue of ata, it is iportant to evelop attributes that require less coputational efforts. In this section, we introuce attributes erive fro the geoetry an topology of roa networks, which are invariant to translations, rotations an scale changes. We start with connectivity attribute represente by the foral ajacency atrix (note as A), which can be use to oel the topological structure of roa networks. Definition 1. If there is at least one single roa segent connecting roa intersections i an j, i is sai being connecte to j (or vice versa). The entry for ij in the ajacency atrix A is of value 1. Otherwise, it is 0. The ajacency atrix can be erive fro the graph. The entry values of the atrix correspon to the existence of eges between corresponing vertices, i.e. value 1 escribes at least Figure 2. The quarangle in networks This is exeplifie in Figure 2. V 1 has two ajacent vertices V 2 an V 4. Both V 2 an V 4 are ajacent to V 3. Thus, V 1 is associate to one quarangle fore by V 1, V 2, V 3, V 4. Sae as V 4, V 3 an V 2. As entione above, the property can easily be extene to ore coplex, polygonal loops, if so esire. 4. MATCHING TWO ROAD NETWORKS Accoringly, the roa network is efine through the graph ebee two topological (connectivity an basic loop) an one geoetric attribute (relative istance). The reaer can easily unerstan that aitional attributes ay also be use as neee. This type of graph is tere attribute graph here (an, siilarly, attribute network). Using the above notations for these two networks, our ai in atching is to optially correspon (label) noes V i in graph G to those in graph G satisfying certain atching criteria. The roa network atching is forulate as a graph-labeling proble. Base on relaxation labelling, the atching process iteratively re-labels the ata noes with oel noes by changing their 194

The International Archives of the Photograetry, Reote Sensing an Spatial Inforation Sciences. Vol. XXXVII. Part B4. Beijing 2008 corresponing weights. The weights are optiize accoring to their local geoetric an topological siilarity. After each iteration, the global atching (i.e. global copatibility) is easure. The process reaches an optial atching when the global copatibility easureent becoes unchange or varies to a liite threshol. We etails the atching process in following subsections. V1 V 3 V2 V 4 V1 V 2 V 3 V4 V 5 4.1 Local Siilarity Once we have constructe the attribute graphs fro two networks we procee with their optial atching. Given V k fro G as the current label of V i in G, the gooness of such apping (V i V k ) can be easure through a oifie version of the exponential function (Li, 1992). Our ai is to iteratively re-label the noes of the ata graph with the oel graph so as to optiize a global copatibility easure by the structures an geoetries of atche noes. The gooness of the local fit can be easure with H (V i, V k ): a) If not both V i an V k are associate with the basic loop Where (, ) exp( H V V = in Dˆ Dˆ i k i,{ s, α} k,{ t, τ} i, s, α = roa intersections in the ataset to be registere, where s & α are connecte with i k, t, τ = roa intersections in oel ataset, where t & τ are connecte with k = su of relative istances of D ˆ i ( s, α ) intersections i, s an i,α D ˆ = su of relative istances of k ( t, τ ) intersections k, t an k, τ b) If both V i an V k are associate with the basic loop Where H( V i, V ˆ ˆ k ) = exp( in Di D ),{ s, α} k,{ t, τ } + exp( Dˆ Dˆ ) ( s, α ), j ( t, τ ), κ i, s, α, j = roa intersections in the ataset To be registere an for the basic loop of i k, t, τ, κ = roa intersections in oel ataset an for the basic loop of k D ˆ = su of relative istances of ( s, α ), j intersections s, j an α, j D ˆ = su of relative istances of ( t,τ ), κ intersections t, κ an τ, κ Use Figure 3 as an exaple. If we consier labelling V 2 for V 1, V 1 has two connecte vertices V 2 an V 3, which also both connect with vertex V 4. At the sae tie, V 2 also has two connecte vertices V 1 an V 5 both connecting with vertex V 4. In this case, H shoul be easure with the forula (3). If V k has ore than two ajacent vertices as V 1, we choose the two vertices that iniize the power value in function H. ) (2) (3) Figure 3. Vertices with inexact egrees The novel feature of this local consistency easure H is its copoun structure, which istinguishes it fro any alternatives in the literature. Specifically, the geoetry an topology for easuring local apping gooness is fore by noes both irectly connecte (arke with yellow circles in Figure 3) an inirectly connecte (arke with blue circles in Figure 3) with the apping noes. The unerlying avantages with these two easureents is that the constructe H function will not be affecte by the presence of noise (i.e. the aitional link V 3 in Figure 3) an the abiguity will be reuce as low as possible. Siilarly, the presence of noise (i.e. aitional links) in V woul not affect our atching. 4.2 Global Copatibility With function H, the local ifference between V k an V i uner the inial relative istance constraint is appe into a siilarity easure for assigning V k to V i. As the continuous relaxation labeling fraework, weighte values other than logical assertions (1 or 0) are attache to all possible assignents for each vertex in G. The weight (enote by p i (λ)) with which label V λ is assigne to vertex V i belongs to [0,1]. In aition, the su of the weights for all possible assignents to any vertex shoul be equal to 1. Let Θ be all available assignents with V to V. The global copatibility function can be fore as: Λ( Θ, ) = i, j H ( V k, κ ij k, V κ ) p ( V i k ) p ( V Thus, the optial labeling of V with V will be the assignent that axiizes the above function. We use the graient ascent algorith, which iteratively coputes the length an irection of the upate vector to upate the weight p such that the global copatibility function Λ will increase with each upating of p. The iteration terinates when the algorith converges, generally proucing an optial labeling (or atching). Intereste reaers are referre to (Huel an Zucker, 1983) for aitional etails. 5. EXPERIMENTS We teste propose approach in two experients in orer to eonstrate its perforance an robustness. All tests are ipleente in MATLAB environent. 5.1 Test 1 The two roa networks use in this experient are etecte respectively fro a ap, typically having larger coverage, an fro an iage with saller coverage. They are shown on the left in Figure 4, with their corresponing graphs on the right. The two networks reflect typical registration conitions, whereby an iage an a corresponing ap ay iffer j κ ) (4) 195

The International Archives of the Photograetry, Reote Sensing an Spatial Inforation Sciences. Vol. XXXVII. Part B4. Beijing 2008 substantially in ters of translations, rotations, an scale changes. It shoul also be note that a link (between noes a an e) in the ap network oes not exist in the iage network. This introuces inexact atching in the two networks, but only in their structure. a e b Figure 6. Detecte network M 2 fro the ap M 1 f v 1 v 2 c G 1 In aition, as shown in Figure 7, there are four coponents in M 2 arke with colors that have sae topological pattern as M. Thus, topological attributes only woul prouce ultiple results. v 6 v 5 v 3 M G v 4 Figure 4. Detecte networks an their graphs Figure 5 illustrates the convergence of the global copatibility function uner successive iterations until a axiu value is reache. The result using all three attributes is shown by the thinner curve (top) an its global copatibility increases faster an converges earlier than when using two attributes (connectivity an relative istance) only. The run tie for this experient is 1.2218 secons (with two attributes) an 1.4821 secons (with all three attributes). M Figure 7. Topologically siilar coponents The convergence of the global copatibility is shown in Figure 8. Siilar as Figure 5, the result using all three attributes is shown by the re curve (top) an its global copatibility increases faster an converges earlier than when using two attributes only. In this test, the atching starts to converge after 30 iterations, slower than in Test 1 as we have relatively coplex networks for atching. Figure 5. Coparison uner inexact atching The atching result is suarize in Table 1. It can be easily seen that all noes were atche correctly espite ifferences in orientation (rotation, shift, an scales) between the two networks, or even ifferences in their actual structure (the presence of the a-e link). Matching result 5.2 Test 2 a b c e f V 6 V 1 V 2 V 5 V 4 V 3 Table 1. Matching result Figure 8. Global copatibility vs. iteration ties The atching result is graphically escribe in Figure 9. Despite the topological siilarity probles shown in Figure 7, intersections in M are correctly appe to M 2. In this test, we exaine the robustness of our approach in exact atching. M 2 in Figure 6 an M in Figure 4 are two etecte networks use in this experient. It shoul be note that M 2 has 9 intersections an 13 eges, while M only has 6 intersections an 7 eges. These two atasets vary not only in structures like the exaple in Test 1, but also in noes of the graph. Figure 9. Matching result 196

The International Archives of the Photograetry, Reote Sensing an Spatial Inforation Sciences. Vol. XXXVII. Part B4. Beijing 2008 6. CONCLUSION AND FUTURE WORK This paper introuce a novel atching approach to the georegistration proble base on graph atching. It offers the ability to utilize inforation about the topology an geoetry of a network to establish corresponence. The ability to utilize both allows us to reuce the abiguity of local consistency, especially when inexact atching takes place. Furtherore, the approach oes not require user input, other than etecting roa intersections through iage processing. Thus our approach offers a robust an general solution to the iage-to-x registration proble using networks. Future work will further investigate aitional attributes to give rise to invariant escription of patterns in networks. It will also inclue an extension of the propose approach to ore coplex networks. ACKNOWLEDGEMENT This work was supporte by the National Geospatial- Intelligence Agency through NURI grant NMA 401-02-1-2008. REFERENCES Barrow, H. G. an Popplestone, R. J., 1971. Relational escriptions in picture processing. Machine Intelligence, 6, Einburgh. Bunke, H., 1999. Error correcting graph atching: On the influence of the unerlying cost function. IEEE Trans. PAMI, 21(9), pp. 917 922. Drewniok, C. an Rohr, K., 1996. Autoatic exterior orientation of aerial iages in urban environents. In: Proceeings of ISPRS, Vienna, Austria, Vol. XXXI, Part B3, pp. 146-152. Gautaa S., & Borgharaef, A., 2005. Detecting change in roa networks using continuous relaxation labelling. In: ISPRS Hannover Workshop 2005, Hannover. Huel, R.A. an Zucker, S.W., 1983. On the founations of relaxation labeling processes. IEEE Transactions on Pattern Analysis an Machine Intelligence, PAMI-5(3), pp. 267-287. Jain, B. J. an Wysotzki, F., 2002. Fast winner-takes-all networks for the axiu clique proble. In: Proceeings of the 25th Annual Geran Conference on AI: Avances in Artificial Intelligence, pp.163-173. Li, S. Z., 1992. Matching: Invariant to translations, rotations an scale changes. Pattern Recognition, 25(6), pp. 583-594. Li, S. Z., 1994. A arkov rano fiel oel for object atching uner contextual constraints. In Proceeings of the IEEE Coputer Vision an Pattern Recognition Conference (CVPR), Seattle, Washington, pp. 866 pp.869. Luo B. & Hancock, E. R., 2001. Structural graph atching using the EM algorith an singular value ecoposition. IEEE Trans. PAMI, 23(10), pp. 1120-1136. Pelillo, M., 1999. Replicator equations, axial cliques, an graph isoorphis. Neural Coputation, 11(9), pp. 1933-1955. Rohr, K., 2001. Lanark-base iage analysis: using geoetric an intensity oels. Kluwer Acaeic Publisher. Rosenfel, A., Huel, R. an Zucker, S., 1976. Scene labeling by relaxation operations. IEEE Trans. Syst. Man. Cybern., 6, pp. 420-433. Shapiro, L.G. an Haralick, R.M., 1985. A etric for coparing relational escriptions. IEEE Trans. PAMI, 7(1):90 94. Shi, J. an Malik, J., 1998. Self inucing relational istance an its application to iage segentation. Lecture Notes in Coputer Science, 1406, pp. 528. Wilson, R.C. & Hancock, E. R., 1997. Structural atching by iscrete relaxation. IEEE PAMI, 19(6), pp. 634-648. 197

The International Archives of the Photograetry, Reote Sensing an Spatial Inforation Sciences. Vol. XXXVII. Part B4. Beijing 2008 198