HOPC: A NOVEL SIMILARITY METRIC BASED ON GEOMETRIC STRUCTURAL PROPERTIES FOR MULTI-MODAL REMOTE SENSING IMAGE MATCHING

Transcription

1 ISPRS Annls of the Photogrmmetry, Remote Sensing nd Sptil Informtion Sciences, Volume III-1, 216 XXIII ISPRS Congress, July 216, Prgue, Czech Republic : A NOVEL SILARITY METRIC BASED ON GEOMETRIC STRUCTURAL PROPERTIES FOR MULTI-MODAL REMOTE SENSING IMAGE MATCHING Yunxin YE,b, *, Li SHEN,b Stte-province Joint Engineering Lbortory of Sptil Informtion Technology for High-speed Rilwy Sfety, Southwest Jiotong University, , Chin - (yeyunxin, lishen)@home.swjtu.edu.cn b Collbortive innovtion center for ril trnsport sfety, Ministry of Eduction, Southwest Jiotong University, , Chin - (yeyunxin, lishen)@home.swjtu.edu.cn Commission I, WG I/2 KEY WORDS: Multi-modl Remote Sensing Imge, Imge Mtching, Phse Congruency, Similrity metric,, ABSTRACT: Automtic mtching of multi-modl remote sensing imges (e.g., opticl, LiDAR, SAR nd mps) remins chllenging tsk in remote sensing imge nlysis due to significnt non-liner rdiometric differences between these imges. This pper ddresses this problem nd proposes novel similrity metric for multi-modl mtching using geometric structurl properties of imges. We first extend the phse congruency model with illumintion nd contrst invrince, nd then use the extended model to build dense descriptor clled the Histogrm of Orientted Phse Congruency () tht cptures geometric structure or shpe fetures of imges. Finlly, is integrted s the similrity metric to detect tie-points between imges by designing fst templte mtching scheme. This novel metric ims to represent geometric structurl similrities between multi-modl remote sensing dtsets nd is robust ginst significnt non-liner rdiometric chnges. hs been evluted with vriety of multi-modl imges including opticl, LiDAR, SAR nd mp dt. Experimentl results show its superiority to the recent stte-of-the-rt similrity metrics (e.g.,,, etc.), nd demonstrte its improved mtching performnce. 1. INTRODUCTION Imge mtching is prerequisite step for vriety of remote sensing pplictions including imge fusion, chnge detection nd imge mosic. The ccurcy of imge mtching hs significnt impct on these pplictions. Although there hs been rpid development of utomtic imge mtching techniques in the lst decde, in prctice these techniques often require the mnul selection of tie-points(or correspondences) for multi-modl remote sensing imges, especilly for the opticl-to-synthetic Aperture Rdr (SAR) or opticl-to-light Detection nd Rnging equipment (LiDAR) imges. This is becuse there cn be significnt geometric distortions nd rdiometric (intensity) differences between these imges. () (b) Figure 1. () opticl imge. (b) SAR imge Current technologies enble remote sensing imges to be directly georeferenced by pplying the physicl model of sensors nd the nvigtion instrument's onbord stellites, which results in the imges only hving the position offsets of severl or couple of dozen pixels reltive to ny other precisely georeferenced imgery (Gonclves et l. 212). This llows globl geometric distortions such s obvious trnsltion, rottion nd scle differences between imges to be removed by the direct georeferencing techniques. In view of this, the min difficulties remining for multi-modl remote sensing imge mtching re non-liner rdiometric differences. Figure 1 shows pir of opticl nd SAR imges. The two imges hve quite different intensity nd texture ptterns despite cpturing the sme scene, mking tie-point detection much more difficult thn previously. Therefore, the gol of this pper is to find robust mtching method tht is resistnt to non-liner rdiometric differences between multi-modl remote sensing imges. In generl, imge mtching methods cn be clssified s feturebsed nd re-bsed methods (Zitov nd Flusser 23). Feture-bsed methods first extrct the fetures from imges nd then use the similrities between these fetures to detect tiepoints between imges. Common feture-bsed methods include point-bsed methods (Hn et l. 214), line or edge-bsed methods (Sun et l. 215), region-bsed methods (Gonçlves et l. 211), nd locl invrint fetures-bsed methods(sedght nd Ebdi 215). However, when these types of methods re pplied to process multi-modl imges, significnt non-liner rdiometric differences between imges mke it difficult to detect highly-repetble shred fetures, degrding the mtching performnce (Suri nd Reinrtz 21). Are-bsed methods re nother type of processing method, which use similrity metrics to detect tie-points between imges using templte mtching scheme. Compred with feturebsed methods, re-bsed methods hve the following dvntges: (1) re-bsed methods void the requirement for the shred feture detection tht usully hs low-repetbility between multi-modl imges; (2) re-bsed methods cn detect * Corresponding uthor. Yunxin YE, yeyunxin@home.swjtu.edu.cn doi:1.5194/isprsnnls-iii

2 ISPRS Annls of the Photogrmmetry, Remote Sensing nd Sptil Informtion Sciences, Volume III-1, 216 XXIII ISPRS Congress, July 216, Prgue, Czech Republic tie-points within smll serch region becuse most remote sensing imges cn be directly georeferenced so tht there re only few pixels differences between such imges. Additionlly, some current commercil softwre pckges for remote sensing imge processing such s ERDAS nd ENVI, use re-bsed methods for their utomtic imge mtching functionl modules. This indictes tht re-bsed methods my be somewht more suitble for prcticl pplictions. Similrity metrics ply decisive role in re-bsed methods. The common similrity metrics re normlized cross correltion (), mutul informtion () (Cole-Rhodes et l. 23), nd Mtching by Tone Mpping () (Hel-Or et l. 214). However, these similrity metrics cnnot effectively hndle non-liner rdiometric differences between imges becuse the intensity informtion is directly used to detect tie-points. In contrst, geometric structurl properties of imges re more resistnt to non-liner rdiometric differences. Figure 1 shows tht contour shpes nd geometric structures re quite similr between the opticl nd SAR imges, despite their very different intensity nd texture chrcteristics. This observtions motivte us to develop novel similrity metric using geometric structurl fetures of imges to ddress the problem of nonliner rdiometric differences between multi-modl imges in the frmework of re-bsed methods. originl imge contrst vrition imge phse congruency grdient Figure 2. Comprison of phse congruency with grdient It should be noted tht geometric structurl fetures cn be usully represented by grdient informtion, but it is sensitive to rdiometric chnges between imges. In comprison, the phse congruency feture hs been demonstrted to be more resistnt to illumintion nd contrst vrition (Kovesi 1999), s shown in Figure 2. This chrcteristic mkes it insensitive to rdiometric chnges. However, the conventionl phse congruency model cn only obtin its mgnitude tht is insufficient for geometric structurl feture description. Therefore, we expnd the phse congruency model to build its orienttion informtion. nd use the mgnitude nd orienttion of this model to construct novel descriptor tht cptures geometric structure fetures, which is referred to s the Histogrm of Orientted Phse Congruency (). The between descriptors is used s the similrity metric (lso nmed ), nd fst templte mtching scheme is designed to chieve tie-points between imges. The min contributions of this pper re the follows: (1) extend the phse congruency model to build the orienttion representtion of this model; (2) develop novel similrity metric bsed on geometric structure properties for multi-modl remote sensing imge mtching using the mgnitude nd orienttion of phse congruency, nd design fst templte mtching scheme for. The code nd supplementry mterils cn be downloded from this website 1 2. METHODOLOGY Given reference imge nd sensed imge, the im of imge mtching is to identify the tie points between the two imges. In this section, we will present geometric structure feture descriptor nmed nd its use to define the similrity between two imges bsed on the of the descriptors. Our pproch is bsed on the ssumption tht multi-modl imges shre similr geometric structurl properties despite hving very different intensities nd textures. In this section, the phse congruency model is first extended to construct its orienttion representtion, nd then novel similrity metric bsed on geometric structurl properties is developed using the extended phse congruency model. 2.1 Extended Phse Congruency Mny current feture detectors nd descriptors re bsed on grdient informtion, including Sobel, Cnny, SIFT, etc. These opertors re usully sensitive to imge illumintion nd contrst chnges. By comprison, phse informtion is more robust to these chnges. Oppenheim et l. nlyzed the function of phse informtion for imge processing, nd found tht phse informtion is even more importnt thn mplitude informtion (Oppenheim nd Lim 1981). This conclusion is clerly illustrted in Figure 3. We tke the Fourier trnsforms of imge I nd I b, nd use the phse informtion of I nd the mgnitude informtion of Ib to construct new, synthetic Fourier trnsform which is then bck-trnsformed to produce new imge I c. It cn be observed tht I c minly reflects the contour informtion of I, which shows tht the contour nd structurl fetures of imges re minly provided by phse informtion. () (b) (c) Figure 3. The importnce of phse informtion of imges. () the imge I. (b) the imge I b. (c) the synthetic imge constructed using the phse informtion of I nd the mgnitude informtion of I b. Since phse informtion ws demonstrted to be so importnt for imge perception, it is nturl to use phse informtion for feture detection. Phse congruency is feture detector bsed on the locl phse informtion of imges, which postultes tht the fetures such s corners nd edges re presented where the Fourier components re mximlly in phse. Phse congruency cn be clculted using log Gbor wvelets over multiple scles nd orienttions by following formul Wo( x, y) A no( x, y) no( x, y) T o n PC( x, y) (1) A ( x, y) o n no 1 supply.rr?oref=e&n= doi:1.5194/isprsnnls-iii

3 ISPRS Annls of the Photogrmmetry, Remote Sensing nd Sptil Informtion Sciences, Volume III-1, 216 XXIII ISPRS Congress, July 216, Prgue, Czech Republic where PC( x, y ) is the mgnitude of the phse congruency; ( xy, ) indictes the coordintes of the point in n imge; Wo ( x, y ) is the weight fctor for frequency spred; Ano( x, y ) is the mplitude t ( xy, ) t wvelet scle n nd orienttion o ; ( xy, ) is more sensitive phse no devition; T is noise threshold, nd is smll constnt to void division by zero. denotes tht the enclosed quntity is equl to itself when its vlue is positive, nd zero otherwise. () Figure 4. The log Gbor odd-symmetric wvelet. () the 3-D shpe of this wvelet. (b) the 2-D shpe of this wvelet Orienttion of Phse Congruency: The bove mentioned phse congruency model is insufficient to describe imge fetures such s geometric structurl informtion becuse only mgnitude informtion cn be cquired from this model. Therefore, we extend the phse congruency model to chieve its orienttion informtion using log Gbor odd-symmetric wvelets. The orienttion of phse congruency, similr to grdient orienttion, represents the most rpid direction of feture vrition, which is crucil for feture description. () Figure 5. The orienttion of phse congruency. () the imge. (b) its orienttion of phse congruency. Figure 4 shows the log Gbor odd-symmetric wvelet. This wvelet is smooth derivtive filter tht cn compute the imge derivtive in single direction (Moreno et l. 29). Since log Gbor odd-symmetric wvelets with multiple orienttions re used in the computtion of phse congruency, we project the convolution results of the wvelets in the horizontl nd verticl direction to obtin the horizontl derivtive nd the verticl derivtive b respectively. The orienttion of phse congruency is defined by Eq. (2). Figure 5 illustrtes the orienttion of phse congruency, which hs vlues rnging is from to (b) (b) 36. ( o ( )cos( )) b ( ono( )sin( )) (2) rctn( b, ) where is the orienttion of phse congruency nd o ( ) denotes the convolution results of odd-symmetric wvelet. 2.2 Geometric structure similrity metric no In this subsection, we will develop feture descriptor nmed, which cptures geometric structurl properties by the mgnitude nd orienttion of phse congruency, nd we lso build geometric structurl similrity metric on the bsis of this descriptor. is inspired from Histogrms of Oriented Grdient (HOG) (Dll nd Triggs 25) tht cn effectively describe locl object ppernce nd shpe through the distribution of locl grdient mgnitudes nd orienttions. Our descriptor is bsed on evluting dense grid of wellnormlized locl histogrms of phse congruency orienttions over templte window selected in n imge. Figure 6 presents the min processing chin of the descriptor. The steps of this process is s follows. (1) The first step is to select templte window with certin size in n imge, nd then compute the phse congruency mgnitude nd orienttion for ech pixel in this templte window, in order to provide the feture informtion for. (2) The second step is to divide the templte window into some overlpping blocks, where ech block consists of m m some smll sptil regions, clled "cells" contining n n pixels. This process forms the fundmentl frmework of. (3) The third step is to ccumulte locl histogrm of phse congruency orienttions over ll the pixels within cells of ech block. Ech cell is first divided into number of orienttion bins which re used to form the orienttion histogrms, nd then the histogrms re weighted by phse congruency mgnitudes using triliner interpoltion method. The histogrms for the cells in ech block re normlized by the L2 norm to chieve better invrince to illumintion nd shdowing. This process produces the descriptor for ech block. It should be noted tht phse congruency orienttions need to be limited to rnge between nd 18 to construct orienttion histogrms of blocks, in order to hndle the intensity inversion between multi-modl imges. (4) Finlly, we collect the descriptors from ll blocks within dense overlpping grid covering the templte window into combined feture vector which cn be used for the templte mtching. no Templte window Phse congruency mgnitude nd oriention Divide the window into blocks consist of some cells Accumulte orienttion histogrms for cells nd blocks Collect s for ll blocks over templte window cell block overlp of block Feture vectors v={x 1,.x n} Figure 6. The min processing chin of the descriptor doi:1.5194/isprsnnls-iii

4 ISPRS Annls of the Photogrmmetry, Remote Sensing nd Sptil Informtion Sciences, Volume III-1, 216 XXIII ISPRS Congress, July 216, Prgue, Czech Republic As mentioned bove, is feture descriptor tht cptures the internl geometric lyouts of imges. As such, this descriptor cn be used to mtch two imges with different intensity ptterns s long s they both hve similr lyouts or shpes. Figure 7 shows the descriptors computed from the corner nd edge regions of the visible nd infrred imges of the sme scene. The descriptors re quite similr despite the lrge rdiometric differences between the two imges. Figure 7. descriptors of the visible nd infrred imges in the corner nd edge regions Considering the similrity of geometric structurl fetures between multi-modl imges, the between the descriptors is regrded s the similrity metric (lso nmed ) for imge mtching. To illustrte 's dvntge in mtching multi-modl imges, it is compred to, nd by the similrity curve. A pir of imges (visible nd SAR) with high resolution is used in the test. There re very obviously significnt non-liner rdiometric differences between these imges. A templte window (68 68 pixels) is first selected from the visible imge. Then,,, nd re ech clculted for x-direction trnsltions (-1 to 1 pixels) within serch window of the SAR imge. visible infrred visible X(pixels) corner corner edge edge X(pixels) X(pixels) Figure 8 Similrity curves of,, nd. Figure 8 shows the similrity curves of,, nd. is expected to fil to detect the tie-point, nd nd lso hve some loction errors cused by the SAR X(pixels) locl regions descriptors similrity descriptors locl regions significnt rdiometric differences. In contrst, not only correctly detects the tie-point, but lso hs the more distinguishble curve pek. This exmple is preliminry indiction tht is more robust thn the other similrity metrics to non-liner rdiometric differences. More nlysis on the performnce of will be given in Section Fst clcultion scheme for During the templte mtching process, templte window moves pixel by pixel within serch region or n imge. For ech templte window to be mtched, it is obvious tht the descriptor needs to be clculted. Since most of the pixels overlp between djcent templte windows, This requires lot of extr computtion. To ddress this issue, fst mtching scheme is designed for the descriptor. To extrct the descriptor, the templte window is divided into overlpping block regions, nd the descriptors of ll the blocks re collected to form finl dense descriptor. Therefore, block cn be regrded s the fundmentl element for the descriptor. In order to reduce the computtion required for templte mtching, we define block region s being centered by ech pixel in serch region or n imge, nd extrct the descriptor of ech block (herefter referred to s block- descriptor). Ech pixel will then hve multidimensionl vectors to form the 3D descriptors for the whole imge, which is clled the block- imge. The block- descriptor is then collected t intervls of severl pixels (such s hlf block width) to generte the descriptor for the templte window. The fst computing scheme is shown in Figure 9. () (b) (c) Figure 9. The fst computing scheme for the descriptor. () the imge. (b) the block- imge. (c) the block- descriptors t certin intervl. (d) the finl descriptor. This scheme cn eliminte much of repetitive computtion for djcent templte windows. For exmple, ssume tht it spends T time extrcting the descriptor for templte window with size of n n pixels, where ech block contins m m pixels nd the overlp between djcent blocks is hlf block width. If the templte window slides pixel by pixel cross 2 serch region with size of M M pixels, it will spends MT time extrcting the descriptors for ll the templte windows tht re used for mtching. In contrst, the computtionl expense of our scheme rises minly from two tsks: (1)extrction of the block- descriptors for ll pixels in the serch region; (2) collection of the block- descriptor t intervls of hlf block width for ll the templte windows tht re used for mtching. The computtionl expense of the ltter tsk cn lmost be ignored compred to tht of the former tsk becuse it simply ssembles the block- descriptors t certin smpling intervl. The former tsk will spends Tm /2n time extrcting the block- descriptor for pixel becuse templte window contins 2 n/ m blocks. In 2 totl, it spends M Tm /2n time for ll the pixels in the serch region, where the block with m is constnt tht is usully 3 or 4 pixels. Compred with the trditionl scheme which tkes (d) doi:1.5194/isprsnnls-iii

5 ISPRS Annls of the Photogrmmetry, Remote Sensing nd Sptil Informtion Sciences, Volume III-1, 216 XXIII ISPRS Congress, July 216, Prgue, Czech Republic 2 MT time, our scheme hs significnt time dvntge especilly in lrge templte size. Figure 1 shows the run times of the two schemes with regrd to incresed templte sizes, where 2 interest points re mtched, the serch region is 2 2 pixels. It cn be clerly observed tht our scheme require less time thn the trditionl scheme, nd this dvntge becmes more nd more obvious when the templte size increses. Figure 1. The run time of the trditionl mtch scheme nd our scheme using with incresed templte sizes. 3. EXPERIMENT In this section, the mtching performnce of is evluted with different types of multi-modl remote sensing imges, nd is compred with the three stte-of-the-rt similrity metrics, nd. 3.1 Dt sets To evlute the effectiveness of the proposed lgorithm, we select ten sets of multi-modl imge pirs, which re divided into four ctegories: Visible-to-Infrred (Vis-to-Inf), LiDAR-to- Visible (Lid-to-Vis), Visible-to-SAR (Vis-to-SAR), nd Imgeto-Mp (Img-to-Mp). The test imge pirs hve vriety of low-, medium-, nd high-resolution imges with resolutions from.5 to 3m, nd cover different terrins including urbn nd suburbn res. All of these imge pirs hve been systemticlly corrected by their physicl models, nd lso resmpled in the sme ground smple distnce (GSD). Thus there re lmost no obvious trnsltion, rottion nd scle differences between the reference nd sensed imges. However, significnt rdiometric differences re common between these imges becuse they re cquired by different imging modlities nd from vrious spectr. The descriptions of the test dt re listed in Tble 1, nd the chrcteristics of ech set re described below. Visible-to-Infrred: Test 1 nd test 2 re the visible nd infrred imges, which re pir of high resolution imges nd pir of medium resolution imges respectively. The high resolution imge is locted in the urbn re, nd hve rich geometric structurl fetures. In contrst, the medium resolution imge covers suburbn re with reltively poor geometric structurl fetures. LiDAR-to-Visible: Three pirs of LiDAR nd visible dt re selected for the experiments. Test 3 nd test 4 re two pirs of LiDAR intensity nd visible imges covering the urbn re with high buildings, nd hve obvious locl geometric distortions cused by the relief displcement of the building. Moreover, the LiDAR intensity imges hve significnt noise which increses the difficulty of mtching. Test 5 includes pir of LiDAR depth nd visible imges, nd vstly differences cn be observed from the intensity chrcteristics of the two imges, which mke mtching the two imges quite chllenging. Visible-to-SAR: Test 6 to test 8 re composed of the visible nd SAR imges. Test 6 is pir of imges locted in suburbn re with medium resolution, nd hs rich geometricl structurl fetures. Test 7 nd test 8 re high resolution imges covering n urbn re with high buildings, resulting in obvious locl distortions. Additionlly, there is temporl difference of fourteen months between the imges of test 8, so some ground objects hve chnged during this period. These differences in this test mke it very difficult to mtch the two imges. Imge-to-Mp: Test 9 nd test 1 re two pirs of visible nd mp dt, which hs been downloded from Google Mps. As both pirs of dt re locted in n urbn re with high buildings, locl distortions cn be observed between the two dt of ech pir. Moreover, the intensity detils between visible nd mp dt look lmost completely different. As shown in Figure 12, the texturl informtion of the mps is much poorer thn the imges, nd There is lso some lbelled text in the mp. Therefore, it is very chllenging to detect tiepoints between the two dt. Ctegory Imge pir Size nd GSD Dt Test 1 Dedlus visible ,.5m 2/4 Dedlus infrred ,.5m 2/4 Test 2 TM bnd1(visible) 8 8, 3m 25/1 TM bnd4(nir) 8 8, 3m 25/1 Test 3 LiDAR intensity 6 6, 2m 21/1 WorldView2 visible 6 6, 2m 211/1 Test 4 LiDAR intensity , 2m 21/1 WorldView2 visible , 2m 211/1 Test 5 LiDAR depth , 2.5m 212/6 Airborne visible , 2.5m 212/6 Test 6 TM bnd3 6 6, 3m 27/5 TerrSAR-X 6 6, 3m 28/3 Test 7 Google Erth , 3m 27/11 TerrSAR-X , 3m 27/12 Test 8 Google Erth , 3m 29/3 TerrSAR-X , 3m 28/1 Test 9 Google Mps 7 7,.5m Google Mps 7 7,.5m unknown Vis-to- Inf Lid-Vis Vis-to-SAR Img-to- Mp Test 1 Google Mps Google Mps , 1.5m , 1.5m Tble 1 Descriptions of the test dt 3.2 Implementtion detils nd evlution criterion unknown The block-bsed Hrris opertor (Ye nd Shn 214) is first used to detect the 2 evenly-distributed interest points in the reference imge. Then,, nd re pplied to detect tie-points within the serch region with fixed size (- 1 to 1 pixels) of the sensed imge using templte mtching strtegy, followed by fitting the similrity curves with qudrtic polynomil to determine the subpixel position (M et l. 21). The prmeters of re set to 8 orienttion bins, 3 3 cell blocks of 4 4 pixel cells nd hlf block width overlp The correct mtch rte (CMR) is chosen s the evlution criterion. Where CMR CM / C, the correct mtch (CM) is the number of correctly mtched point pirs in the mtching results, nd the correspondence (C) is the totl number of mtch point pirs. The CM number is determined by the following strtegy. For ech imge pir, 4-6 evenly distributed points were selected s the check points. A trnsformtion model T is then computed using the check points. The T used for test 2 nd test 6 is the projective trnsform since these imges re medium resolution imges covering suburbn res. For the other test doi:1.5194/isprsnnls-iii

6 ISPRS Annls of the Photogrmmetry, Remote Sensing nd Sptil Informtion Sciences, Volume III-1, 216 XXIII ISPRS Congress, July 216, Prgue, Czech Republic imge pirs covering urbn res, the cubic polynomil is employed becuse it is usully more suitble thn other globl trnsformtion models such s projective nd second-order polynomil models for pre-fitting non-rigid deformtions between imges (Ye nd Shn 214). The point pir with locliztion error less thn Thre is regrded s the correct mtch. The vlue of Thre is set to 1. pixel for the medium resolution imges tht hve few locl distortions (test 2 nd test 6). For the high resolution imges, the vlue of Thre is set to 1.5 pixels for more flexibility since their rigorous geometric trnsformtion reltionships re usully unknown nd cubic polynomil models cn only pre-fit the geometric distortions. 3.3 Mtching performnce The mtching performnce of is evluted by comprison with, nd in terms of two spects: the CMR vlue nd the computtionl efficiency. In the mtching process, templte windows of different sizes (from 2 2 to pixels) re used to detect tie-points to nlyze the sensitivities of these similrity metrics with respect to chnges in the templte size Correct Mtch Rtio: Figure 11()-(b) show the CMRs between the visible nd infrred imges (test 1 nd test 2). It cn be seen tht performs the best, followed by nd, while chieves the lowest CMRs. This is becuse is only invrint to liner rdiometric differences nd cnnot hndle complex rdiometric chnges between imges. Additionlly, the CMRs of re less ffected by templte sizes compred with, which is very sensitive to templte size chnges. The reson for this is tht is required to compute the joint entropy between imges, which is quite sensitive to the smple sizes (nmely the templte sizes)(hel-or et l, 214). In ddition, the 's CMRs of test 2 decline slightly compred with test 1 in the sme templte sizes[figure 11(b)]. This is becuse the imges of this test contin reltively poor geometric structurl informtion, resulting in tht hrdly extrcts the distinguished structurl fetures from smll templte size. However, chieves the high CMRs in the lrger templte windows (more thn pixels). Figure 11(c)-(e) show the CMRs between the LiDAR dt nd visible imges (test 3-test 5). For test 3 nd test 4, chieves reltively higher CMRs thn the other similrity metrics despite significnt rdiometric differences nd noise existing between the imges. For test 5 where the LiDAR depth nd visible imges present very different intensity ptterns, performs much better thn the other similrity metrics such s nd. As shown in Figure 11(e), the CMR of cn rech lmost 1%, while tht of nd only chieve CMR of 5% in the lrge templte sizes. This is lrgely ttributed to the fct tht the geometric structurl chrcteristics re very similr (Figure 12(e)) despite the lrge rdiometric differences between the imges. Thus, representing the geometric structurl similrity, hs n obvious dvntge over nd. The CMRs between the visible nd SAR imges (test 6-test 8) re illustrted in Figure 11(f)-(h). chieves higher CMRs for l three tests. In ddition, performs much better thn the other similrity metrics such s nd, especilly for test 7 nd test 8 which consist of two pirs of high resolution imges within urbn res. Figure 11(g)-(h) show tht cn respectively chieve the CMRs of 99% nd 91% for test 8 nd test 9 in the lrge templte size. In contrst, the CMRs of re only 64% nd 66%, nd those of re 61% nd 42% in the sme templte sizes for the two tests, respectively. The reson for this is tht the imges of the two tests contin rich geometric structures nd contour informtion such s buildings nd rods. This demonstrtes tht clerly outperforms the other similrity metrics for multi-modl imges tht include bundnt structurl fetures. Test 9 nd test 1 re two pirs of the visible imges nd the mp dt, where the mp dt hve been rsterized. This is chllenging test becuse the two dt hrdly hve ny significnt shred fetures prt from some similr boundries of buildings nd streets. Figure 12(i-j) shows the CMRs for the four similrity metrics. Similr to the previous tests, performs better thn, nd. The CMR of rises s the templte size increses, nd cn respectively chieve the CMRs of 78% nd 75% in the lrge templte size such s pixels, which is n cceptble CMR for multi-modl imge mtching. Figure 12 shows the tie-points detected using with templte size of 1 1 pixels between the multi-modl imges. In the enlrged subimges, it cn be clerly observed tht these tie-points re locted in the correct positions precisely. It cn be observed from the bove experiments tht outperforms lmost ll of the other similrity metrics in ny templte size for ll the tests. chieves the second highest CMRs in most cses. Although the performnce of is very sensitive to templte sizes. In comprison, is more stble to chnges in templte sizes, cn chieve reltively considerble CMR even in smll templte size () (b) (c) (d) (e) (f) (g) (h) (i) (j) Figure 11. The CMRs of,, nd. () test 1. (b) test 2. (c) test 3. (d) test 4. (e) test 5. (f) test 6. (g) test 7. (h) test 8. (i) test 9. (j) test 1. doi:1.5194/isprsnnls-iii

7 ISPRS Annls of the Photogrmmetry, Remote Sensing nd Sptil Informtion Sciences, Volume III-1, 216 XXIII ISPRS Congress, July 216, Prgue, Czech Republic () (b) (c) (d) (e) (f) (g) (j) (h) (i) Figure 12. The correct mtching points of ll the tests by in the templte sizes of 1 1 pixels. () test 1. (b) test 2. (c) test 3. (d) test 4. (e) test 5. (f) test 6. (g) test 7. (h) test 8. (i) test 9. (j) test 1. (j) Computtionl Efficiency: As well s the CMR, the computtionl efficiency is nother importnt indictor for evluting the mtching performnce of similrity metrics. The experimentl pltform used is Inter Core i7-471mq 2.5GHz PC. Figure 13 shows the run times of,, nd with incresed templte sizes. It cn been seen tht spends the lest time mong these similrity metrics. This is becuse is quickly clculted over the whole serch region, voiding to serch correspondences pixel by pixel (Hel- Or et l. 214). In contrst, is the most time-consuming becuse it needs to compute joint histogrm for every mtched templte window pir, which requires certin mount of computtion (Hel-Or et l. 214). In ddition, since needs to extrct the descriptors nd clculte the between such descriptors, it tkes more run time compred with nd. However, the computtionl efficiency of is still better thn tht of within the rnge of templte sizes (less thn pixels) used in our experiment. This is beneficil for prcticl ppliction becuse the lrge templte size increses the computtion of imge mtching, nd the CMRs of nd do not usully increse substntilly when the templte size is more thn certin rnge such s 1 1 pixels (Figure 12). In short, is most time-consuming, followed by, nd Figure 13. Run times of,, nd with incresed templte sizes Bsed on the bove experiments, it cn be concluded tht chieves the higher CMRs thn the other similrity metrics, followed by, nd is lso less time-consuming thn within limited rnge of templte sizes. Although requires more time thn nd, its CMR is much doi:1.5194/isprsnnls-iii

8 ISPRS Annls of the Photogrmmetry, Remote Sensing nd Sptil Informtion Sciences, Volume III-1, 216 XXIII ISPRS Congress, July 216, Prgue, Czech Republic higher thn these two similrity metrics tht re both reltively vulnerble to non-liner rdiometric differences. Therefore, tking the CMR nd computtionl efficiency into considertion, is more distinguished similrity metric for multi-modl imge mtching. 4. CONCLUSION In this pper, novel similrity metric (nmed ) for multi-modl remote sensing imge mtching is proposed to ddress the issue of significnt non-liner rdiometric differences. First, the phse congruency model is extended to build its orienttion representtion. Then, the mgnitude nd orienttion of phse congruency re used to construct, followed by fst templte mtching scheme designed for this metric. ims to cpture the geometric structurl similrity between imges, which cn effectively hndle complex rdiometric vrition. Thus this metric cn robustly find tie-points in different modlities. hs been evluted ginst ten pirs of multi-modl imges, nd compred to the stte-of-the-rt similrity metrics such s,, nd. The experimentl results demonstrte tht outperforms the other similrity metrics, especilly for the imge pirs contining rich geometric structure fetures. Moreover by designing fst mtching scheme, hve lower run time thn tht chieves the second highest correct mtch rte in most experiments. However, is still more timeconsuming compred with nd. The min reson is tht requires high-dimensionl geometric structurl feture descriptor to be clculted. In subsequent works, this issue will be resolved by reducing the dimensions of the descriptor using dimension-reduction technique such s PCA. In ddition, it is worth noting tht the performnce of my decline if the imges include few structure or shpe informtion becuse depends on geometric structurl properties, In this cse, n imge enhncement pproch cn be pplied to enhnce shpe or edge fetures, which my be helpful for imge mtching, A more thorough evlution will be ddressed in future using more multi-modl remote sensing imges. ACKNOWLEDGEMENTS This pper is supported by the Ntionl Bsic Reserch Progrm(973 progrm) of Chin (No. 212CB71991), the Ntionl Nturl Science Foundtion of Chin (No nd No ). The uthors would like to thnk Dr. Kovesi for the public MATLAB code of implementing phse congruency (Kovesi 2). REFERENCES Cole-Rhodes, A. A., Johnson, K. L., LeMoigne, J., et l, 23. Multiresolution registrtion of remote sensing imgery by optimiztion of mutul informtion using stochstic grdient. IEEE Trnsctions on Imge Processing, 12(12), pp Gonclves, H., Gonclves J. A. Corte-Rel L., et l, 212. CHAIR: utomtic imge registrtion bsed on correltion nd Hough trnsform. Interntionl Journl of Remote Sensing, 33(24), pp Hn, Y., Choi, J., Byun, Y., et l, 214. Prmeter optimiztion for the extrction of mtching points between high-resolution multisensor imges in urbn res. Geoscience nd Remote Sensing, IEEE Trnsctions on 52(9), pp Hel-Or, Y., Hel-Or H., nd Dvid, E., 214. Mtching by tone mpping: photometric invrint templte mtching. Pttern Anlysis nd Mchine Intelligence, IEEE Trnsctions on, 36(2), pp Kovesi, P., Imge fetures from phse congruency. Videre: Journl of Computer Vision Reserch, 1(3), pp Kovesi, P., 2. MATLAB nd Octve functions for computer vision nd imge processing, Online: Moreno, P., Bernrdino, A. nd Sntos-Victor J., 29. Improving the SIFT descriptor with smooth derivtive filters. Pttern Recognition Letters 3(1), pp M, J. L., Chn, J. C. W. nd Cnters, F., 21. Fully utomtic subpixel imge registrtion of multingle CHRIS/Prob dt. IEEE Trnsctions on Geoscience nd Remote Sensing, 48(7), pp Oppenheim, A. V. nd Lim J. S., The importnce of phse in signls. Proceedings of the IEEE, 69(5), pp Sedght, A. nd Ebdi, H., 215. Remote Sensing Imge Mtching Bsed on Adptive Binning SIFT Descriptor. IEEE Trnsctions on Geoscience nd Remote Sensing, 53(1), pp Sun, Y., Zho, L., Hung, S., et l, 215. Line mtching bsed on plnr homogrphy for stereo eril imges. ISPRS Journl of Photogrmmetry nd Remote Sensing, 14(215), pp Suri, S. nd Reinrtz, P., 21. Mutul-informtion-bsed registrtion of TerrSAR-X nd ikonos imgery in urbn res. IEEE Trnsctions on Geoscience nd Remote Sensing, 48(2), pp Ye, Y. nd Shn J., 214. A locl descriptor bsed registrtion method for multispectrl remote sensing imges with non-liner intensity differences. ISPRS Journl of Photogrmmetry nd Remote Sensing, 9(214), pp Zitov, B. nd Flusser J., 23. Imge registrtion methods: survey. Imge nd Vision Computing, 21(11), pp Dll,N nd Triggs B., 25. Histogrms of oriented grdients for humn detection. Proc. IEEE Conf. Computer Vision nd Pttern Recognition 25, pp Gonçlves, H., Gonçlves, J. nd Corte-Rel, L., 211. "HAIRIS: A method for utomtic imge registrtion through histogrm-bsed imge segmenttion. Imge Processing, IEEE Trnsctions on, 2(3), pp doi:1.5194/isprsnnls-iii