A MULTIRESOLUTION AND OPTIMIZATION-BASED IMAGE MATCHING APPROACH: AN APPLICATION TO SURFACE RECONSTRUCTION FROM SPOT5-HRS STEREO IMAGERY M. Pieot-Deseillign N. Papaoditis MATIS laboato Institut Géogaphique National 2, avenue Pasteu 94165 Saint-Mandé Cedex - Fistname.lastname@ign.f Commission I, WG I/5 & WG I/6 EY WORDS: High esolution mapping, satellite image SPOT5, steeo, automation, image matching, optimization, suface econstuction, digital suface models ABSTRACT: This pape addesses the multi-esolution implementation of a Cox&Roy optimal flow image matching algoithm. This minimization algoithm aims at solving the suface econstuction poblem fomalized as a minimization of an enegy. This multi-esolution appoach is necessay fo achieving easonable pocessing times on extended aeas and impoving obustness by estaining matching ambiguities. Some vey good esults ae shown on a 10m gound sample distance (GSD) SPOT5-HRS steeopai. Intemediate esults at diffeent steps of the econstuction pocess show all the Level Of Details povided by the diffeent pyamid scale levels. The tuning of the egulaization paamete is also discussed fo diffeent landscapes. 1.INTRODUCTION Digital Suface Models ae a key poduct fo many applications anging fom othoimage ectification to the geneation of 3D city models but also ae cucial fo applications such as change detection fo emegency mapping. Latest eathquakes have shown again the impotance of obtaining this suface infomation on extended aeas in the shotest time as possible in conditions in which aibone platfoms ae not adapted. The 120 km swath and the high evisit capacity and the steeoscopic imaging capabilities along the tack (educes diachonic atifacts that usually alte the matching qualit thus makes SPOT5-HRS a vey good tool fo these emegency missions. Many obust image matching and suface econstuction techniques have been developed in the last yeas paticulaly adapted fo digital aeial cameas in the case of multi-view imagey [Papaoditis&al 2001]. Unfotunatel in these emegency cases, most of the time only two images ae available. Thus the necessay obustness has to be achieved though dense optimization based matching techniques. 2. SURFACE RECONSTRUCTION AS AN ENERGY MINIMIZATION PROBLEM Regulaization-based algoithms In the set of all image matching-based suface econstuction techniques, we will focus on those who can be fomalized unde the fom of the minimization of an enegy function. The methodological advantages of these appoaches is to explicitly sepaate the what (the enegy function) fom the how ( the minimization algoithm). These appoaches define a global function on the image field which can be witten as: whee E( = A( + α * F( G( ) Z is the unknown altitude function we ae looking fo; A( is the data attachment tem measuing the image consistency/similaity at the image pojections of point ( ; fo example A( = 1 Co( whee Co ( Z ( is the nomalized cosscoelation scoe ; F ( G( Z )) is a positive function which depends on the vaiations of Z, it is the egulaization tem which expesses the a pioi knowledge of the suface egulaity; α is a paamete weighting the elative impotance of data attachment and egulaization. In this appoach, we can distinguish to subclasses. On the one hand, the diffeential appoaches which make the assumption that the elief and F ( G( Z )) ae diffeentiable
(e.g. a quadatic function) and suppose the existence of an initial solution close enough to the final solution. On the othe hand, the combinatoial appoaches, that take fo example fo F ( G( Z )) a tem such as F ( G ( Z )) = Z ( x+ 1, Z ( + Z ( y+ 1) Z (, that we will note Ξ(Z ) ; these appoaches ae looking fo a global minimum with gaph theoy algoithms, they ae implemented by sampling the possible values of Z (voxels in object space) ; We will focus on the combinatoial appoaches because, on the one hand we do not always have these close enough initial solutions and on the othe hand, we can not assume that the elief is diffeentiable in uban aeas. We will note E α the enegy function defined by: E ( = 1 Co( + α * Ξ( α Optimal flow algoithms Up to ecentl no algoithm allowing the calculation of a global optimum of the function E α had been poposed ; in geneal a sub-optimal solution was calculated by analyzing the image line by line. Indeed looking fo the optimum within a line (o a column) can be seen as a classical dynamic pogamming poblem (e.g. [Baillad 97]); the dawback of these methods is to intoduce a vey stong dissymmety in the pocessing of image lines and columns which has as a consequence the intoduction of impotant atifacts in the esult. In 1999, the founde pape of Roy&Cox [Roy&Cox 98] showed that one could constuct a gaph such as: the nodes ae the possible values of Z; the edges ae the pais of neighboing Z (in the plane o in altimet; the planimetic edges ae assigned the egulaization cost and the altimetic edges ae assigned the data attachment cost; One can than demonstate that the sufaces ae the set of gaph cuts between the set of nodes of maximal Z and nodes of minimal Z; moeove the weight of a cut associated to a suface is exactly equal to E α. Finding the suface minimizing E α can be seen as finding a minimal cut in a gaph. Thus the poblem can be solved in polynomial time with classical minimal cut and maximal flow gaph theoy algoithms. 3.MULTI-RESOLUTION IMPLEMENTATION Even though polynomial, the majo dawback of the Roy&Cox is its elative slowness and its memoy geediness. In pactice, it is vey difficult to use it such as fo massive image matching on aibone o spacebone imagey. It would take seveal months to complete the matching of a complete HRS steeopai (2 images of 700 Mo) with a Pentium 4 PC. We have thus implemented a multi-esolution vaiant of the Roy&Cox algoithm. We constuct a pyamid of N 2 1; images of esolution [ N ]. At the coasest esolution, we look fo an optimal suface in the object/teain space matching cube (composed of a similaity scoe [Papaoditis&al 00] fo each voxel (X,Y,) and in the complete Z eseach inteval. In pactice on a HRS steeopai, we choose N=16 o 32, leading to images of few Mo and dispaity intevals of few dozens of pixels. Then at the cuent step, of esolution 2, we use the 2 +1 pevious step, of esolution, as a pedicto enabling a combinatoy eduction. Moe pecisel the multi-esolution pediction is opeated in the following way. Let Z (, be the suface at step + 1 x + 1 1 calculated with a sampling distance Z 2 +. To calculate a suface at step with a sampling distance Z2, we fist define an estimation y * 1( x 0 Z ( y ) = 2 Z +, ). Let 2 2 be the plana dilatation and the altimety xy dilatation; We note o λ the mophological dilatation by a squae of size λ ( λ fo eosion). We define Z + ( x, = Z 0 ( o + and Z xy Z 0 ( y ) = Z ( z. We then limit possible values of Z to the ones bounded by the uppe Z + suface and the lowe suface. xy Z Z 4. RESULTS Results on SPOT5-HRS steeopai ove a mountainous aea The method descibed has been used to geneate DSMs on 10 mete GSD SPOT-HRS steeopais. The images wee ectified in epipola geomety. Fig.1. illustates the pocess evolution on a mountainous and deset aea. Fo this kind of landscape, the paametes used wee: dilatation of 4 in altimety and 8 in planimety ; dispaity sampling distance (in epipola geomet of 1 pixel ; template matching window sizes of 3x3 ; egulaization paamete of 0.2 ; The calculation time with a 3 GhZ PC, was 3 days fo the complete steeopai (two images of 700 Mo of size 12000*60000). The quantitative evaluation is not yet finished but an expeiment on a test site of les dentelles de Montmiail» has shown an RMS of 3.5 metes. All Digital Suface Models shown have been omnidiectionally shaded to incease the visibility of all mophological details.
Extact of a SPOT5-HRS steeopai Resolution 1/4 Resolution 1/16 Resolution 1/2 Resolution 1/8 Resolution 1
Raste DSM, 70 cm GSD, ti-steeo (backwad, nadi, fowad) Resolution 1 with dequantification Figue 1: DSM at diffeent pyamid levels Results on a simulated Pléaides data set ove uban aeas The same method has been used to geneate uban aeas DSMs fom simulated data sets of the CNES Pléiades satellite. The main diffeence with the ual aea test is in the tuning of the egulaisation paamete; indeed in an HR uban context, due to facade depth discontinuities, thee is no need fo injecting a stong a pioi knowledge on the egulaity of the elief. The paamete was wey weak and tuned to 0.04. Fig. 2. shows some esults on along-tack backwad, nadi, and fowad ti-steeo. We have not yet pefomed quantitative evaluations on this data set. Nevetheless, the esults ae sufficiently obust and accuate to be used as an input fo automated 3D city models econstuction algoithms. The ight image of Fig. 3. shows an example of the 3D models geneated with such a famewok [Maillet&al 06] using the 70 cm GSD DMS. The accuacy of the 3D building models is a little less than one mete. 3D city model geneated semi-automatically using a aste DSM Figue 2 : Results on simulated images of the futue Pléiades satellite 5. CONCLUSION We have pesented a multi-esolution optimization-based appoach fo image matching. It povides obust econstuctions in easonable pocessing time and econstucted sufaces ae mophologically peseved due to the fact that the template windows ae chosen small (3x3) thanks to the optimization pocess. This appoach has also been applied vey successfully to multi-view imagey such as encounteed in vey high esolution imagey povided by aeial digital fame cameas. Aknowledgments Raste DSM, 50 cm GSD, ti-steeo (backwad, nadi, fowad) The authos would like to thank Spot-Image and IGN- Espace fo poviding the SPOT5 HRS steeopais, and CNES fo poviding the simulated set of Pléiades images, and also thei colleagues fom MATIS fo poviding the vecto DSM in Fig.3. Refeences Baillad, C., 1997. Analyse d images aéiennes stééoscopiques pou la estitution 3-D des milieux ubains. Détection et caactéisation du
susol, thèse de doctoat, ENST, laboatoie MATIS, IGN-SR-97-005-C-THE-CB. Maillet, G., Flamanc., D., Buissat, H., Cantou, J-P. «Pepaing the use of Pléaides-H images fo mapping puposes: peliminay assessments at IGN-Fance», in Poc. of the Commission I wokshop Topogaphic mapping fom space, Ed. Ugu Muat Leloglu (CD-Rom), Ankaa, Tuke Febua 2006 Papaoditis N., Thom C., Jibini H., 2000. «Suface econstuction in uban aeas fom multiple views of aeial digital fames», in poc. IAPRS, vol. XXIII, Amstedam, 2000. Papaoditis N., Maillet G., Taillandie F., Jibini H., Jung F., Guigues L. and Boldo D., 2001, Multi-image 3D featue and DSM extaction fo change detection and building econstuction. In E.P. Baltsavias, A. Guen and L. Van Gool (eds) AutomaticExtaction of Man-Made Objects fom Aeial and Space Images III (Lisse, The Nethelands: Balkema), pp. 217-230. Ro S., Co I.J., 1998. «A Maximum-Flow Fomulation of the N-camea Steeo Coespondence Poblem», Poc. IEEE Inten. Confeence on Compute Vision, pp. 492-499, Bombay.