Title: Robust Registration of Multimodal Remote Sensing Images Based on Structural Similarity.

Transcription

1 7 IEEE. Persoal use of this material is permitted. Permissio from IEEE must be obtaied for all other uses, i ay curret or future media, icludig repritig/republishig this material for advertisig or promotioal purposes, creatig ew collective works, for resale or redistributio to servers or lists, or reuse of ay copyrighted compoet of this work i other works. Title: Robust Registratio of Multimodal Remote Sesig Images Based o Structural Similarity. This paper appears i: IEEE Trasactios o Geosciece ad Remote Sesig Date of Publicatio: 3//7 Author(s): Yuaxi Ye, Jie Sha, Lorezo Bruzzoe, ad Li She Volume:PP, Issue: 99 Page(s): - 8.9/TGRS

2 > Robust Registratio of Multimodal Remote Sesig Images Based o Structural Similarity Yuaxi Ye, Member, IEEE, Jie Sha, Seior Member, IEEE, Lorezo Bruzzoe, Fellow, IEEE, ad Li She Abstract Automatic registratio of multimodal remote sesig data (e.g., optical, LiDAR, SAR) is a challegig task due to the sigificat o-liear radiometric differeces betwee these data. To address this problem, this paper proposes a ovel feature descriptor amed the Histogram of Orietated Phase Cogruecy (HOPC), which is based o the structural properties of images. Furthermore, a similarity metric amed HOPC cc is defied, which uses the ormalized correlatio coefficiet () of the HOPC descriptors for multimodal registratio. I the defiitio of the proposed similarity metric, we first exted the phase cogruecy model to geerate its orietatio represetatio, ad use the exteded model to build HOPC cc. The a fast template matchig scheme for this metric is desiged to detect the cotrol poits betwee images. The proposed HOPC cc aims to capture the structural similarity betwee images, ad has bee tested with a variety of optical, LiDAR, SAR ad map data. The results show that HOPC cc is robust agaist complex o-liear radiometric differeces ad outperforms the state-of-the-art similarities metrics (i.e., ad mutual iformatio) i matchig performace. Moreover, a robust registratio method is also proposed i this paper based o HOPC cc, which is evaluated usig six pairs of multimodal remote sesig images. The experimetal results demostrate the effectiveess of the proposed method for multimodal image registratio. iformatio cotet of multimodal remote sesig images, it is ecessary to itegrate these images for Earth observatio applicatios. Image registratio, which is a fudametal prelimiary task i remote sesig image processig, aligs two or more images captured at differet times, by differet sesors or from differet viewpoits []. The accuracy of image registratio has a sigificat impact o may remote sesig aalysis tasks, such as image fusio, chage detectio, ad image mosaic. Although remarkable progress has bee made i automatic image registratio techiques i the last few decades, their practical implemetatio for multimodal remote sesig image registratio ofte requires maual selectio of the cotrol poits (CPs) [] (e.g., optical-to-sythetic Aperture Radar (SAR) image or optical-to-light Detectio ad Ragig (LiDAR) data registratio) due to the sigificat geometric distortios ad o-liear radiometric (itesity) differeces betwee these images. Idex Terms image registratio, multimodal image aalysis, structural similarity, phase cogruecy. W I. INTRODUCTION ITH the rapid developmet of geospatial iformatio techology, remote sesig systems have etered a era where multimodal, multispectral, ad multiresolutio images ca be acquired ad joitly used. Due to the complemetary This paper is supported by the Natioal key ad Developmet Research Program of Chia (No. 6YFB543 ad No. 6YFB563), the Natioal Natural Sciece Foudatio of Chia (No ad No.44374), the Sciece ad Techology Program of Sichua, Chia (No. 5SZ46), ad the Fudametal Research Fuds for the Cetral Uiversities (No. 686CX83). Y. Ye is with the Faculty of Geoscieces ad Evirometal Egieerig, Southwest Jiaotog Uiversity, Chegdu 63, Chia, ad also with the Departmet of Iformatio Egieerig ad Computer Sciece, Uiversity of Treto, 383 Treto, Italy ( yeyuaxi@home.swjtu.edu.c). L. Bruzzoe is with the Departmet of Iformatio Egieerig ad Computer Sciece, Uiversity of Treto, 383 Treto, Italy ( lorezo. bruzzoe@ig.uit.it). J. Sha is with the School of Civil Egieerig, Purdue Uiversity, West Lafayette, IN 4797 USA ( jsha@purdue.edu). L. She is with the Faculty of Geoscieces ad Evirometal Egieerig, Southwest Jiaotog Uiversity, Chegdu 63, Chia( rssheli@outlook.com). (a) (b) Fig.. Apparet o-liear radiometric differeces betwee (a) optical image ad (b) SAR image. Curret techologies eable the direct georeferecig of remote sesig images usig physical sesor models ad avigatio devices aboard the platforms. These techologies ca produce images that have a offset of oly doze or so pixels [3, 4] ad are capable of removig early all the global geometric distortios from the images, such as obvious rotatio ad scale differeces. The cetral difficulty for multimodal remote sesig image registratio is related to o-liear radiometric differeces. Fig. shows a pair of optical ad SAR images of the same scee with differet itesity ad texture patters, which makes CP detectio much more difficult tha that uder sigle-modal images. Therefore, the goal of this paper is to develop a effective registratio method that is robust to o-liear radiometric differeces betwee multimodal remote sesig images. A typical automatic image registratio process icludes the

3 Illumiatio variatio Origial image > followig four steps: i) feature detectio, ii) feature matchig, iii) trasformatio model estimatio, ad iv) image resamplig. Depedig o the process adopted for registratio, most multimodal remote sesig image registratio methods ca be classified ito two categories: feature-based ad area-based []. Feature-based methods first extract the remarkable features from both cosidered images, ad the match them based o their similarities i order to achieve registratio. Commo image features iclude poit features [5], lie features [6], ad regio features [7]. Recetly, local ivariat features have bee widely applied to image registratio. Mikolajczyk et al. compared the performace of umerous local features for image matchig ad foud that Scale Ivariat Feature Trasform (SIFT) [8] performed best for most of the tests [9]. Due to its ivariace to image scale ad rotatio chages, SIFT has bee widely used for remote sesig image registratio [-]. However, SIFT is ot effective for the registratio of multimodal images, especially for optical ad SAR images, because of its sesitivity to o-liear radiometric differeces [3]. Past researchers proposed some ew local ivariat features based o SIFT, such as Speeded Up Robust Features [4], Orieted FAST ad Rotated BRIEF [5], ad Fast Retia Keypoit [6]. Although these ew local features improve the computatioal efficiecy, they are also vulerable to complex radiometric chages. Fudametally, the aforemetioed feature-based methods maily deped o detectig highly-repeatable commo features betwee images, which ca be difficult i multimodal images due to their o-liear radiometric differeces [7]. Thus, these methods ofte do ot achieve satisfactory performace for multimodal images. Area-based methods (sometimes called template matchig) usually use a template widow of a predefied size to detect the CPs betwee two images. After the template widow i a image is defied, the correspodig widow over the other image is searched usig certai similarity metrics. The ceters of the matchig widows are regarded as the CPs, which are the used to determie the aligmet betwee the two images. Area-based methods have the followig advatages compared with feature-based methods: () they avoid the step of feature detectio, which usually has a low repeatability betwee multimodal images; ad () they ca detect CPs withi a small search regio because most remote sesig images are iitially georefereced up to a offset of several or dozes of pixels. Similarity metrics play a decisive role i area-based methods. Commo similarity metrics iclude the sum of squared differeces (SSD), the ormalized cross correlatio (), ad the mutual iformatio (). SSD is probably the simplest similarity metric because it detects CPs by directly computig the itesity differeces betwee two images. However, SSD is quite sesitive to radiometric chages despite its high computatioal efficiecy. is a very popular similarity metric ad is widely applied to the registratio of remote sesig images because of its ivariace to liear itesity variatios [8, 9]. However, is vulerable to o-liear radiometric differeces []. I cotrast, is more robust to complex radiometric chages ad is extesively used i multimodal image registratio [-3]. Ufortuately, is computatioally expesive because it must compute the joit histogram of each widow to be matched [9] ad is very sesitive to the widow size for template matchig []. These drawbacks limit its broad use i multimodal remote sesig image registratio. I geeral, all three similarity metrics caot effectively hadle sigificat radiometric distortios betwee images because they are maily applied o image itesities. Past researchers improved the performace of registratio by applyig these metrics to image descriptors such as gradiet features [4] ad wavelet-like features [5, 6]. However, these features are difficult to use for reflectig the commo properties of multimodal images. Image Phase cogruecy Gradiet Fig. Compariso of phase cogruecy with gradiet. Recetly, i multimodal medical image processig, structure ad shape features have bee itegrated as similarity metrics for image registratio ad have achieved better performace tha traditioal similarity metrics [7-3]. These methods are based o the assumptio that structure ad shape properties are preserved across differet modalities ad are relatively idepedet of radiometric chages. Ispired by that assumptio, the work preseted i this paper explores the performace of structural properties for multimodal remote sesig image registratio. As show i Fig., the cotour structures ad geometric shapes are quite similar betwee the optical ad SAR images despite their very differet itesity characteristics. To address this issue, a ovel similarity metric is proposed i this paper to exploit the similarity betwee structural features to deal with the o-liear radiometric differeces betwee multimodal images. I geeral, structural features ca be represeted by gradiet iformatio of images, but gradiet iformatio is sesitive to the radiometric chages betwee images. I cotrast, the phase cogruecy feature has bee demostrated to be more robust to illumiatio ad cotrast variatio [3] (see Fig. ). This characteristic makes it isesitive to radiometric chages. However, the covetioal phase cogruecy model ca oly obtai its amplitude, which is isufficiet for structural feature descriptio [3]. This paper therefore exteds the covetioal phase cogruecy model to build its orietatio represetatio. Both amplitude ad orietatio the are used to costruct a ovel feature descriptor that ca capture the structures of images. This descriptor,

4 > 3 amed the Histogram of Orietated Phase Cogruecy (HOPC), ca be efficietly calculated i a dese maer over the whole image. The idea of HOPC is ispired from the histogram of orieted gradiet (HOG), which has bee very successful i target recogitio [33]. The HOPC descriptor reflects the structural properties of images, which are relatively idepedet of the particular itesity distributio patter across two images. The HOPC descriptor ca be extracted for each image separately ad the directly compared across images usig a simple itesity metric such as. Therefore, the of the HOPC descriptors is used as the similarity metric (amed HOPC cc ), ad a fast template matchig scheme is desiged to detect CPs betwee images. I additio, i accordace with the characteristics of remote sesig images, a automatic registratio method is desiged based o HOPC cc. The mai cotributios of this paper are as follows. () Extesio of the phase cogruecy model to build its orietatio represetatio. () Developmet of a ovel similarity metric (amed HOPC cc ) based o both the amplitude ad orietatio of phase cogruecy to address the o-liear radiometric differeces betwee multimodal images as well as a fast template matchig scheme to detect CPs betwee images ad a automatic registratio method for multimodal remote sesig images based o HOPC cc. This paper exteds a prelimiary versio of this work [3] by addig () a detailed pricipled derivatio of HOPC cc ; () a detailed aalysis of the effects of the various parameters o HOPC cc ; (3) a effective multimodal registratio method based o HOPC cc ; ad (4) a more thorough evaluatio process through the use of a larger quatity of multimodal remote sesig data. The code of the proposed method ca be dowloaded i this website. The remaider of this paper is orgaized as follows. Sectio II describes the proposed similarity metric (HOPC cc ) for multimodal registratio. Sectio III proposes a robust registratio method based o HOPC cc. Sectio IV aalyzes the parameter sesitivity of HOPC cc ad compares it with the state-of-the-art similarity metrics by usig various multimodal remote sesig datasets. Sectio V evaluates the proposed registratio method based o HOPC cc. Sectio VI presets the coclusios ad recommedatios for future work. II. HOPC : STRUCTURAL SILARITY METRIC Give a master image I ( x, y ) ad a slave image I ( x, y ), the aim of image registratio is to fid the optimal geometric trasformatio model that maximizes the similarity metric betwee I ( x, y ) ad the trasformed I ( x, y ) amed I ( (, )) T x y, which ca be expressed as: Tˆ( x, y ) arg max ( I ( T ( x, y )), I ( x, y )) () T ( x, y) where T( x, y) is the geometric trasformatio model, ad HOPC%-%mex.rar? (.) is the similarity metric. I this sectio, we preset a ovel structural descriptor amed HOPC ad defie the similarity betwee two images o the basis of HOPC. The proposed descriptor is based o the assumptio that multimodal images share similar structural properties despite havig differet itesity ad texture iformatio. First, the phase cogruecy model is exteded to geerate its orietatio represetatio, which the is used to costruct the structural similarity metric HOPC cc ; ad a fast template matchig scheme for this metric is desiged to detect the CPs betwee images. A. Aalysis of Importace of Phase May feature detectors ad descriptors are based o gradiet iformatio, such as Sobel, Cay [34], ad SIFT. As already metioed, these operators are usually sesitive to image illumiatio ad cotrast chages. By compariso, the phase iformatio of images is more robust to these chages. Let us cosider a image I( x ), ad its Fourier trasform F( ) F( ) e j ( ). ( ) F ad ( ) are the amplitude ad the phase of the Fourier trasform, respectively. Oppeheim et al. [35] aalyzed the phase fuctio for image processig ad foud that the phase of a image is more importat tha the amplitude. This coclusio is clearly illustrated i Fig. 3. Images a ad b are first aalyzed with the Fourier trasform to obtai the phase (a) ad amplitude Fa ( ) of image a, as well as the phase ( b) ad amplitude Fb () of image b, respectively. The, ew image b (a) ad Fb () are used to sythetize a a by applyig iverse Fourier trasform. ( b) ad Fa ( ) also are composed as a ew image b a through the same procedure. It ca be clearly observed that a b ad b a both maily preset the iformatio of the image that provides the phase, which shows that the cotour ad structural features of the images are maily provided by the phase. B. Phase Cogruecy Sice phase has bee demostrated to be importat for image perceptio, it is atural to use it for feature detectio. Phase cogruecy is a feature detector based o the local phase of a image, which postulates that features such as corers ad edges are preset where the Fourier compoets are maximally i phase. Morroe ad Burr [36] have demostrated that this model is coformed to the huma visual perceptio of image features. Phase cogruecy is ivariat to illumiatio ad cotrast chages because of its idepedece of the amplitude of sigals [3]. Give a sigal f( x ), its Fourier series expasio is f ( x) A cos( ( x)), where A is the th amplitude of the Fourier compoet, ad is the local phase of the Fourier compoet at positio x. The phase cogruecy of this sigal is defied as

5 Fourier trasform Iverse Fourier trasform Fourier trasform Iverse Fourier trasform > 4 Phase ( a) Image a Image a b Amplitude Fa ( ) Amplitude Fb ( ) Image b Image b a Phase () b Fig. 3 Illustrative example of importace of image phase. A( x)cos( ( x) ( x)) PC ( x) max ( x) [,) A ( x) where ( x) is the amplitude weighted mea local phase of all the Fourier terms at positio x to maximize this equatio. Sice this model caot accurately locate features i oisy ad blurred images, Kovesi have improved the calculatio model of phase cogruecy by usig log Gabor wavelets over multiple scales ad orietatios [3]. I the frequecy domai, the log Gabor fuctio is defied as () e e eo ( x, y), oo ( x, y) I( x, y) M o, I( x, y) O o (4) where eo ( x, y ) ad oo ( x, y ) are the resposes of e M o ad o M o at scale ad orietatio o. The amplitude A o ad phase o of the trasform at a wavelet scale ad orietatio o are give by.5 A e ( x, y) o ( x, y) o o o ata( e ( x, y), o ( x, y)) o o o.6 (5) (log( / )) g( ) exp (log( / )) (3) where is the cetral frequecy ad is the related width parameter. The correspodig filter of the log Gabor wavelet i the spatial domai ca be achieved by applyig the iverse Fourier trasform. The real ad imagiary compoets of the filter are respectively referred as the log Gabor eve-symmetric M ad odd-symmetric e o M wavelets (see Fig. 4). Give a iput image I ( x, y ), its covolutio results with the two wavelets ca be regarded as a respose vector. o o (a) (b) Fig. 4 Log Gabor wavelets. (a) Eve-symmetric wavelet. (b) Odd-symmetric wavelet. Cosiderig the oise ad blur of images, the improved phase cogruecy model (amed PC ) proposed by Kovesi is defied as

6 > 5 PC ( x, y) o Wo ( x, y) A o ( x, y) o ( x, y) T o A ( x, y) where ( xy, ) idicates the coordiates of the poit i a image, W (, ) o x y is the weightig factor for the give frequecy spread, A (, ) o x y is the amplitude at ( xy, ) for the wavelet scale ad orietatio.., T is a oise threshold, is a small costat to avoid divisio by zero, o (6) deotes that the eclosed quatity is equal to itself whe its value is positive or zero otherwise. ( xy, ) is a more sesitive phase deviatio defied as o A ( x, y) ( x, y)= e ( x, y). ( x, y) o ( x, y). ( x, y) o o o e o o e ( x, y). ( x, y) o ( x, y). ( x, y) o o o e where ( x, y)= e ( x, y) / E( x, y) ad e o ( x, y)= o ( x, y) / E( x, y). The term E( x, y ) is the o o o local eergy fuctio ad is expressed as o E( x, y) eo ( x, y) oo ( x, y) o o. Covolutio with the odd-symmetric wavelets of multiple directios Projectio of x-directio: a arcta( ba, ) Phase cogruecy orietatio Projectio of y-directio: b Fig. 5 Illustrative example of calculatio of the orietatio of phase cogruecy. (7) C. Orietatio of Phase Cogruecy The above covetioal phase cogruecy model oly cosiders feature amplitudes of pixels (like gradiet amplitudes). However, it caot achieve their feature orietatios (like gradiet orietatios) that represet the sigificat directios of feature variatio. This traditioal phase cogruecy model caot effectively describe the feature distributio of the local regios of images. Thus, it is isufficiet to use oly the amplitude of phase cogruecy to costruct robust feature descriptors. Takig the SIFT operator as a example, apart from gradiet amplitudes, gradiet orietatios are also used to build the feature descriptors. Therefore, we exted the phase cogruecy model to build its orietatio represetatio for costructig the feature descriptor. As metioed i the above subsectio, the phase cogruecy feature is computed by the log Gabor odd-symmetric ad eve-symmetric wavelets. The log Gabor odd-symmetric wavelet is a smooth derivative filter (see Fig. 4(b)) which ca compute the image derivative i a sigle directio (like gradiets) [37]. Cosiderig that the log Gabor odd-symmetric wavelets of multiple directios are used i the computatio of phase cogruecy, the covolutio result of each directioal wavelet ca be projected oto the horizotal directio (x-directio) ad vertical directio (y-directio), yieldig the x-directio derivative a ad the y-directio derivative b of the images, respectively (see Fig. 5). The orietatio of phase cogruecy is defied as a ( o ( )cos( )) o b ( oo ( )si( )) (8) arcta( ba, ) where is the orietatio of phase cogruecy ad o ( ) deotes the covolutio results of log Gabor odd-symmetric wavelet at orietatio. Fig. 5 illustrates the process of calculatig the orietatio of phase cogruecy, which has a o domai i the rage [, 36 o ). D. Structural Feature Descriptor The aim of the work i this paper is to fid a descriptor that is as idepedet as possible of the itesity patters of images from differet modality. I this subsectio, a feature descriptor amed HOPC is proposed which uses both the amplitude ad orietatio of phase cogruecy. The HOPC descriptor captures the structural properties of images. It is ispired from HOG, which ca effectively describe local object appearaces ad shapes through the distributio of the gradiet amplitudes ad orietatios of local image regios. HOG has bee successfully applied to object recogitio [38], image classificatio [39] ad image retrieval [4] because it represets the shape ad structural features of images. This descriptor characterizes the structural properties of images usig gradiet iformatio. Phase cogruecy, similarly to o

7 > 6 gradiets, also reflects the sigificace of the features of local image regios. Moreover, this model is more robust to image illumiatio ad cotrast chages. As such, the amplitude ad orietatio of phase cogruecy are utilized to build the HOPC descriptor based o the framework of HOG. As show i Fig. 6, HOPC is calculated based o the evaluatio of a dese grid of well-ormalized local histograms of phase cogruecy orietatios over a template widow selected i a image. The mai steps for extractig the HOPC descriptor are described below. cell block Template widow Compute cogruecy amplitudes ad orietatios Divide the widow ito blocks cosistig of some cells i order to emphasize the cotributios of the pixels ear the ceter of the cell. The, the histograms are weighted by phase cogruecy amplitudes usig a triliear iterpolatio method. The histograms for the cells i each block are ormalized by the L orm to achieve a better robustess to illumiatio chages. This process produces the HOPC descriptor for each block. It should be oted that the phase cogruecy orietatios eed to be limited o to the rage [, 8 o ) to costruct the orietatio histograms, which ca hadle the itesity iversio betwee multimodal images. (4) The fial step collects the HOPC descriptors of all the blocks i the template widow ito a combied feature vector, which ca be used for template matchig. The HOPC descriptor ca capture the structural features of images ad is more robust to illumiatio chages compared with the HOG descriptor. Fig. 7 shows a example of the HOPC descriptors computed from a local regio of two images with sigificat illumiatio variatios. There are more similarities betwee the HOPC descriptors tha the HOG descriptors. Local regio HOPC HOG cell Accumulate the histogram of phase cogruecy orietatios for each cell block Normalize the histograms withi overlappig blocks of cells Fig. 7 Compariso of HOPC with HOG. It is possible to see that the HOPC descriptors are more robust to illumiatio chages tha the HOG descriptors. Feature vectors v={x,.x } Collect HOPCs for all blocks over the template widow Fig. 6 Mai processig chai for calculatig the HOPC descriptor. () The first step selects a template widow with a certai size i a image, ad the computes the phase cogruecy amplitude ad orietatio for each pixel i this template widow, which provides the feature iformatio for HOPC. () The secod step divides the template widow ito overlappig blocks, where each block cosists of m m spatial regios, called "cells," each cotaiig pixels. This process defies the fudametal framework of HOPC. (3) The third step accumulates a local histogram of the phase cogruecy orietatios of all the pixels withi the cells of each block. Each cell is first divided ito a umber of orietatio bis, which are used to form the orietatio histograms. A Gaussia spatial widow is applied to each pixel before accumulatig orietatio votes ito the cells E. Similarity Metric Based o Structural Properties As metioed above, HOPC is a feature descriptor that captures the iteral structures of images. Sice structural properties are relatively idepedet of itesity distributio patters of images, this descriptor ca be used to match two images havig sigificat o-liear radiometric differeces as log as they both have similar shapes. Therefore, the of the HOPC descriptors is take as the similarity metric (amed HOPC cc ) for image registratio, which is defied as where HOPC cc k ( V ( k) V )( V ( k) V ) A A B B ( VA( k) VA) ( VB ( k) VB ) k k (9) V A ad V B are HOPC descriptors of the image regio A ad image regio B respectively. meas of V A ad V B, respectively. V A ad V B deote the

8 Ru time(sec) Ru time(sec) > 7 F. Fast Matchig Scheme Durig the template matchig processig, a template widow moves pixel-by-pixel withi a search regio or a image. For each pair of template widows to be matched, we have to compute its HOPC cc. Sice most of the pixels overlap betwee adjacet template widows, this requires may repetitive computatios. To address this issue, a fast matchig scheme is desiged for HOPC cc. The CP detectio usig HOPC cc icludes two steps: extractig the HOPC descriptors ad computig the betwee such descriptors. The first step speds the most time i the matchig process. To extract the HOPC descriptor, the template widow is divided ito some overlappig blocks, ad the descriptors for each of these blocks are collected to form the fial dese descriptor. Therefore, a block ca be regarded as the fudametal elemet of the HOPC descriptor. I order to reduce the computatioal time of template matchig, we defie a block regio cetered o each pixel i a image (or a search regio), ad extract the HOPC descriptor of each block (hereafter referred to as the block-hopc descriptor). Each pixel will the have a block-hopc descriptor that forms the 3D descriptors for the whole image, which is called the block-hopc image. The the block-hopc descriptors are collected at a iterval of several pixels (such as a half block width ) to geerate the HOPC descriptor for the template widow. Fig. 8 illustrates the fast computig scheme. assembles the block-hopc descriptor at a certai iterval samplig. The former eeds O M N operatios for extractig the block-hopc descriptor for each pixel i the whole search regio. Compared with the traditioal scheme, our scheme has a sigificat computatioal advatage i the large size of template widow or search regio. Fig. 8 shows the ru times for the two schemes versus the size of template widow ad search regio, whe iterest poits are matched. Oe ca observe that our scheme requires much less time tha the traditioal scheme, ad this advatage becomes more ad more obvious by icreasig the template widow ad search regio size Traditoal scheme Our scheme (a) Fig. 8 Compariso of ru time take from the traditioal matchig scheme ad our scheme usig HOPC cc.(a) Ru time versus the template size, where the size of the search regio is pixels. (b) Ru time versus the search regio size, where the template size is pixels Traditoal scheme Our scheme Search regio size(pixels) (b) (a) (b) (c) (d) Fig. 8 Illustratio of the fast computig scheme for the HOPC descriptor. (a) Image. (b) Block-HOPC image. (c) Block-HOPC descriptors at a certai iterval. (d) Fial HOPC descriptor. This scheme ca elimiate much of the repetitive computatio betwee adjacet template widows. Let us ow compare the computatioal efficiecy of our matchig scheme with the traditioal matchig scheme. For a template widow ( N N pixels) that has a movig search regio 3 with a size of M M pixels, the traditioal scheme takes O M N operatios because the template widow slides pixel-by-pixel across the search regio. Differetly from the traditioal scheme, the computatioal time take from our scheme maily icludes the two parts: () extractio of the block-hopc descriptors for all pixels i the whole search regio ( M N pixels); () collectio of the block-hopc descriptors at itervals of a half block width for all of the template widows used to match. The computatioal cost of the latter ca almost be igored compared to that of the former because it simply This makes the adjacet blocks have the overlap of 5% to build the HOPC descriptor. 3 This refers to the movig rage of ceter pixel of template widow. III. MULTIMODAL REGISTRATION METHOD BASED ON HOPC I this sectio, a ovel robust image registratio method is itroduced for multimodal images based o HOPC cc, which cosists of the followig six steps. Fig. shows the flowchart of the proposed method. Master image Georeferecig by avigatio data Iterest poit detectio CP detectio usig Mismatched CP elimiatio Slave image Image registratio via PL trasformatio Fig. Flowchart of the proposed image registratio method. () The master ad slave images are first coarsely rectified usig the direct georeferecig techiques to remove their obvious traslatio ad rotatio differeces. The, the two images are resampled to the same groud sample distace

9 > 8 (GSD) to elimiate possible resolutio differeces. () I order to evely distribute the CPs over the image, the block-based Harris operator [3] is used to detect the iterest poits i the master image. The image is first divided ito o-overlappig blocks, ad the Harris values are computed for each block. The, the Harris values are raked from the largest to the smallest i each block, ad the top k poits are selected as the iterest poits. (3) Oce a set of iterest poits is extracted i the master image, HOPC cc is used to detect the CPs usig a template matchig scheme i a small search widow of the slave image, which is determied through the georeferecig iformatio of the images. To icrease the robustess of the image matchig, a bidirectioal matchig techique [4] is applied, which icludes two steps (forward matchig ad backward matchig). I the forward step, for a iterest poit p i the master image, its match poit p is foud by the maximum of HOPC cc betwee the template widow i the master image ad the search widow i the slave image. I the backward step, the match poit of p is foud i the master image by the same method. Oly whe the two matchig steps achieve cosistet results, the matched poit pair ( p, p ) is cosidered as CPs. (4) Due to existig ucertaity factors, such as occlusio ad shadow, the obtaied CPs are ot error-free. Large CP errors are elimiated usig a global cosistecy check method based o a global trasformatio [5]. The trasformatio model chose is vital for the cosistecy check ad depeds o the types of relative geometric deformatios betwee images. I this paper, the projective trasformatio model is chose for the cosistecy check because it ca effectively hadle commo global trasformatio (traslatio, rotatio, scale, ad shear) [4]. (5) Mismatched CPs are removed by a iterative refiig procedure. A projective trasformatio model is first set up usig the least squares method with all the CPs. The residuals ad the root mea-square error (RMSE) of CPs the are computed, ad the CP with the largest residual is removed. The above process is repeated util the RMSE is less tha a give threshold (e.g., pixel). (6) After the CPs with large errors are removed, it is ecessary to determie a trasformatio model to rectify the slave image. A piecewise liear (PL) trasformatio model [43] is chose to address the local distortios caused by terrai relief. This model first divides the images ito triagular regios usig Delauay s triagulatio method [44], ad a affie trasformatio (see ()) is applied to map each triagular regio i the slave image oto the correspodig regio i the master image [45]. x a a x a y () y b b x b y where ( x, y ) ad ( x, y ) are the coordiates of the CPs i the master ad slave images, respectively. IV. EXPERIMENTAL RESULTS: HOPC MATCHING PERFORMANCE I this sectio, the matchig performace of HOPC cc is evaluated usig differet types of multimodal remote sesig images by cosiderig three metrics: the similarity curve, the correct match ratio (CMR), ad the computatioal efficiecy. The experimets maily have two objectives: () test the iflueces of the various parameters for HOPC cc ad () compare HOPC cc with the state-of-the-art similarity metrics such as ad. I the experimets coducted, is computed by a histogram with 3 bis because it achieves the optimal matchig performace for the datasets used. I additio, sice HOPC cc uses the framework of HOG to build the similarity metric, the HOG descriptor is also itegrated as a similarity metric for the compariso with HOPC cc. Based o our aalysis of the literature, to the authors kowledge, the HOG descriptor has ot bee previously used as a similarity metric for multimodal remote sesig image registratio i a template matchig scheme. The of the HOG descriptors is used as the similarity metric (amed HOG cc ) for image matchig. It is empirically foud that the origial parameter settig [33] of the HOG descriptor could ot be efficietly applied to multimodal remote sesig image matchig, which is likely because these parameters are desiged for target detectio oly. Therefore, HOG cc is set to the same parameters as HOPC cc (see Sectio IV-C) for image matchig i order to make a fair compariso. The test data, implemetatio details, ad experimetal aalysis are as follows. A. Descriptio of Datasets Two categories of multimodal remote sesig image pairs (sythetic images with o-liear radiometric differeces ad real multimodal images) are used to evaluate the effectiveess of HOPC cc. Sythetic Datasets Two differet types of itesity warped models are used to geerate the sythetic images. A high resolutio image (38 38 pixels) located i a urba area is used to perform the sythetic experimet. The master ad slave images are simulated by applyig a spatially-varyig itesity warped model (see ()) ad a piecewise liear mappig fuctio to the image, respectively. Moreover, a Gaussia oise with mea = ad variace =. is imposed o the slave image. K [ xy ; ] k ((8) ) I( x, y) I( x, y).(. e ) () k K where, I ( x, y ) deotes the image rescaled to [,]. The last term i the brackets models the locally-varyig itesity field with a mixture of K radomly cetered Gaussias [46] with K set to 3 to geerate the sythetic image. The spatially-varyig itesity warped model geerates a image havig o-uiform illumiatio ad cotrast chages, while the piecewise liear mappig fuctio itroduces a o-liear radiometric distortio model to warp the image.

10 Output > 9 Such radiometric distortio models have bee applied for the simulatio of multimodal matchig i the literature [9,, 46]. Fig. shows the process used for geeratig the sythetic master ad slave images which preset the sigificat radiometric differeces Origial image Iput Piecewise liear mappig fuctio Fig. Illustratio of the process used to geerate the sythetic images used i the experimets. Real Datasets Spatially-varyig itesity field Warped image Add oise Master image Slave image Te sets of real multimodal image pairs are used to evaluate the effectiveess of HOPC cc. These images are divided ito four categories: Visible-to-Ifrared (Visib-Ifra), LiDAR-to-Visible (LiDAR-Visib), Visible-to-SAR (Visib-SAR), ad Image-to-Map (Img-Map). The tested image pairs are a variety of medium resolutio (3m) ad high-resolutio (.5m to 3m) images that icluded differet terrais ad both urba ad suburba areas. All of the image pairs have bee systematically corrected by usig their physical models, ad each image pair is respectively resampled ito the same GSD. Cosequetly, there are oly a few obvious traslatio, rotatio, ad scale differeces betwee the master ad slave images. However, sigificat radiometric differeces are expected betwee images because they are acquired by differet imagig modalities ad at various spectra. Fig 9 shows the test data, ad Table I presets descriptios of the data. The characteristics of each test set are as follows. Visible-to-Ifrared: Visib-Ifra ad Visib-Ifra are visible ad ifrared data which iclude a pair of highresolutio images ad a pair of medium-resolutio images. The high-resolutio images represet a urba area, while the medium-resolutio images cover a suburba area. LiDAR-to-Visible: Three pairs of LiDAR ad visible data are selected for the experimets. LiDAR-Visib ad LiDAR-Visib are two pairs of iterpolated raster LiDAR itesity ad visible images coverig urba areas with high buildigs. They have obvious local geometric distortios caused by the relief displacemet of buildigs. Moreover, the LiDAR itesity images have sigificat oise, which icrease the difficulty of matchig. LiDAR-Visib 3 icludes a pair of iterpolated raster LiDAR height ad visible images. Large differeces ca be observed from the itesity characteristics of the two images (see Fig. 9), which make matchig the two images quite challegig. Visible-to-SAR: Visib-SAR to Visib-SAR 3 are composed of visible ad SAR images. Visib-SAR cotais a pair of medium-resolutio images located i a suburba area. Visib-SAR ad Visib-SAR 3 are high resolutio images coverig urba areas with high buildigs, thus resultig i obvious local distortios. Additioally, there is a temporal differece of 4 moths betwee the images i Visib-SAR 3, ad some groud objects therefore chaged durig this period. These differeces make it very difficult to match the two images. Image-to-Map: Img-Map ad Img-Map are two pairs of visible images ad map data dowloaded from Google Maps. The map data have bee rasterized. Sice both pairs of data represet urba areas with high buildigs, local distortios are evidet betwee the two images of each pair. I additio, the radiometric properties betwee the visible images ad the map data are almost completely differet. As show i Fig. 9, the texture iformatio of the maps is much poorer tha that of the images, ad there are also some labeled texts i the map. Therefore, it is very challegig to detect the CPs betwee the two data. B. Implemetatio Details ad Evaluatio Criteria First, the block-based Harris operator (see Sectio III) is used to detect the iterest poits i the master image, where the image is divided ito o-overlappig blocks, ad two iterest poits are extracted from each block, for a total of iterest poits. The,, HOG cc, ad HOPC cc are applied to detect the CPs withi a search regio of a fixed size ( pixels) of the slave image usig a template matchig strategy, after which the similarity surface is fitted usig a quadratic polyomial to determie the subpixel positio []. CMR is chose as the evaluatio criterio ad is calculated as CMR=CM/C, where CM is the umber of correctly matched poit pairs i the matchig results, ad C is the total umber of match poit pairs. The matched poit pairs with localizatio errors smaller tha a give threshold value are regarded as the CM. For the sythetic datasets, a small threshold value (.5 pixels) is used to determie the CM because of the kow exact geometric distortios betwee the master ad slave images. For the real datasets, we determie the CM by selectig a umber of evely distributed check poits for each image pair. I geeral, the check poits are determied by maual selectio. However, for some multimodal image pairs, especially for LiDAR-Visib ad Visib-SAR, it is very difficult to locate the CPs precisely by visual ispectio due to their varyig itesity ad texture characteristics. Accordigly, differet strategies are desiged to select the check poits based o the characteristics of the datasets. For Visib-Ifra, the images have relatively more similar radiometric characteristics tha those of other datasets, ad a set of 4-6 evely distributed check poits are maually selected betwee the master ad slave images. For the other datasets, especially for

11 Average CMR(%) Img-Map Visib-SAR LiDAR-Visib Visib-Ifra > TABLE I DESCRIPTION OF DATASETS USED IN THE MATCHING EXPERIMENTS Category No. Image pair Size ad GSD Date Image Characteristic Daedalus visible Daedalus ifrared 5 5,.5m 5 5,.5m 4/ 4/ Urba area Ladsat 5 TM bad (visible) Ladsat 5 TM bad 4(ifrared) 8 8, 3m 8 8, 3m 9/ 3/ Suburba area, temporal differece of 6 moths LiDAR itesity WorldView visible 6 6, m 6 6, m / / Urba area with high buildigs, sigificat local distortios, temporal differece of moths, ad sigificat oise i the LiDAR data LiDAR itesity WorldView visible 6 67, m 6 6, m / / Urba area, temporal differece of moths, ad serious oise i the LiDAR data 3 LiDAR height Airbore visible 54 54,.5m 54 54,.5m 6/ 6/ Urba area, ad large differece i itesity characteristics Ladsat 5 TM bad 3 TerraSAR-X 6 6, 3m 6 6, 3m 5/7 3/8 Suburba area, ad sigificat oise i the SAR data Image from Google Earth TerraSAR-X 58 54, 3m , 3m /7 /7 Urba area, ad sigificat oise i the SAR data 3 Image from Google Earth TerraSAR-X 68 68, 3m 68 68, 3m 3/9 /8 Urba area, local distortios, temporal differece of 4 moths, ad sigificat oise i the SAR data. Image from Google Maps Map from Google Maps 7 7,.5m 7 7,.5m N/A Urba area with high buildigs, obvious local distortios, large differeces i itesity characteristics Image from Google Maps Map from Google Maps 6 64,.5m 6 64,.5m N/A Urba area with high buildigs, obvious local distortios, large differeces i itesity characteristics the LiDAR ad SAR data, HOPC cc has bee used to detect evely distributed CPs betwee images by a large template size ( pixels) because the experimets show that a larger template widow ca achieve higher CMR values (see Sectio IV-E). The, the CPs with large errors are elimiated usig the global cosistecy check method described i Sectio III. Fially 4-6 CPs with the least residuals are selected as the check poits. Oce the check poits are selected, the projective trasformatio model computed usig these poits is employed to calculate the localizatio error of each matched poit pair. The threshold value of the error is set to. pixel to determie the CM for the image pairs of Visib-Ifra ad Visib-SAR because they have few local distortios. For the other high resolutio image pairs, the threshold value is set to.5 pixels for achievig higher flexibility sice their rigorous geometric trasformatio relatioships are usually ukow ad a projective trasformatio model ca oly pre-fit the geometric distortios. C. Parameter Tuig This subsectio systematically aalyzes the effects of various parameters o the performace of HOPC cc. HOPC cc is costructed usig blocks havig a degree of overlap. Each block cosists of m m cells cotaiig pixels, ad each cell is divided ito orietatio bis. Thus,, m,, are the parameters to be tued; ad their iflueces are tested o the te sets of multimodal images described i Table I. I this experimet, HOPC cc is used to detect the CPs betwee the images by a template matchig scheme, where the template size is set to pixels. The average CMR is used to assess the ifluece of the various parameters because multiple sets of data are used i the experimet Bi umber We first test the ifluece of the umber of orietatio bis o HOPC cc, whe HOPC cc is costructed by 3 3 cell blocks of 4 4 pixel cells, ad the overlap betwee blocks is Fig. Average CMR values versus the orietatio bi umber.

12 HOGcc Average CMR(%) > set to a half-block width ( =/ ). Fig. shows the average CMR values versus the umber of orietatio bis. It ca be observed that the average CMR value geerally icreases with the umber of orietatio bis. It reaches the maximum value whe the bi umber is 8. Therefore =8 is regarded as a good-selected for HOPC cc /4 / 3/4 Overlap Fig. 3 Average CMR values versus the degree of overlap. I the procedure for buildig HOPC cc, the blocks are overlapped so that each cell i a block cotributes several compoets to the fial descriptor. Therefore, the degree of overlap affects the performace of HOPC cc. Fig. 4 shows that the average CMR value icreases as the amout of overlap i the rage betwee ad 3/4 block widths is icreased, but the differeces betwee the overlaps of / ad 3/4 block widths is small. Sice a larger overlap is more time-cosumig, a half block width ( =/ ) is chose as the default settig for HOPC cc value, followed by 3 3 cell blocks of 4 4 pixel cells. However, the differece betwee their CMR values is oly.%, ad the choice of 3 3 cell blocks of 4 4 pixel cells has a obvious advatage i computatioal efficiecy compared to 3 3 cell blocks of 3 3 pixel cells. Therefore, 3 3 cell blocks of 4 4 pixel cells are used as the optimum values i these experimets. Based o the above results, the followig parameters are idetified to compute HOPC cc : =8 orietatio bis; 3 3 cell blocks of 4 4 pixel cells; ad =/ block width overlap. These parameters have bee used i the experimets described i the ext subsectio. Cell size (pixels) TABLE II AVERAGE CMR VALUES AND RUN TIMES AT DIFFERENT BLOCK AND CELL SIZES Block size (cells) CMR Time CMR Time CMR Time CMR Time % 3.4s 89.4% 54.8s 9.6% 44.6s 9.% 5.7s % 3.4s 88.6% 3.3s 9.4% 8.9s 9.3% 9.s % 4.s 87.%.s 88.4% 6.7s 87.8%.5s % 4.s 8.4% 3.4s 84.8% 4.s 8.8% 3.4s % 8.5s 77.%.5s 79.6% 9.8s 73.9%.4s D. Aalysis of Similarity Curve The similarity curve ca qualitatively aalyze the matchig performace of similarity metrics [47]. I geeral, the similarity curve is maximal whe the CPs are located at the correct matchig positio. A pair of visible ad SAR images with high resolutio are used i this experimet. A template widow (68 68 pixels) is first selected from the visible image. The,,, HOG cc, ad HOPC cc are calculated withi a search widow ( pixels) of the SAR image X(pixels) X(pixels) (a) (c) (d) Fig. 4 Average CMR values versus differet block ad cell sizes. The block ad cell sizes ( m m cell blocks of pixel cells) affect the performace of HOPC cc. Fig. 4 shows the average CMR values versus the differet block ad cell sizes with a half-block overlap, ad Table II lists the average CMR values ad ru times. It ca be see that the average CMR value drops whe the cell size icreases. Ideed, 3-4 pixel wide cells achieve the best results irrespective of the block size. I additio, 3 3 cell blocks perform best. The valuable spatial iformatio is suppressed if the block becomes too large or too small, which is ufavorable for image matchig. I this aalysis, 3 3 cell blocks of 3 3 pixel cells achieves the highest CMR SAR (b) X(pixels) (e) X(pixels) Fig. 5 Similarity curves of,, HOG cc, ad HOPC cc. (a) Visible image. (b) SAR image. (C) Similarity curve of. (d) Similarity curve of. (e) Similarity curve of HOG cc. (f) Similarity curve of HOPC cc (f)

13 CMR(%) CMR(%) CMR(%) CMR(%) CMR(%) CMR(%) CMR(%) CMR(%) CMR(%) CMR(%) CMR(%) > HOGcc 8 HOGcc HOGcc HOGcc HOGcc (a) (b) (c) (d) (e) 8 HOGcc 8 HOGcc 8 HOGcc HOGcc HOGcc (f) (g) (h) (i) (j) Fig. 8 CMR values versus the template size of,, HOG cc ad HOPC cc for real multimodal images. (a) Visib-Ifra. (b) Visib-Ifra. (c) LiDAR-Visib. (d) LiDAR-Visib. (e) LiDAR-Visib 3. (f) Visib-SAR. (g) Visib-SAR. (h) Visib-SAR 3. (i) Img-Map. (j) Img-Map. Fig. 5 shows the similarity curves of,, HOG cc, ad HOPC cc. Oe ca clearly see that the sigificat radiometric differeces cause both ad to fail to detect the CP. Eve if HOG cc achieves the correct CP at the maximum, its curve peak is ot very sigificat. By compariso, HOPC cc ot oly detects the correct CP, but also exhibits a smoother similarity curve ad more distiguishable peak. This example idicates that HOPC cc is more robust tha the other similarity metrics to the o-liear radiometric differeces. A more detailed aalysis of the performace of HOPC cc is provided i the ext subsectios. E. Aalysis of Correct Matchig Ratio I this subsectio, the performace of,, HOG cc, ad HOPC cc is evaluated by usig the sythetic ad real datasets i terms of CMR. I the matchig processig, template widows of differet sizes (from to pixels) are used to detect the CPs for aalyzig the sesitivity of these similarity metrics with respect to chages i the template size. liear itesity mappig used to geerate the sythetic images [9, ]. Moreover, the CMR values of HOPC cc, HOG cc, ad icrease as the template size icreases, while does ot preset a similar regularity. Compared with HOG cc, HOPC cc exhibits a slight superiority because the simulated radiometric distortio models yield the o-uiform illumiatio ad cotrast chages betwee images (see Fig. ), ad the phase cogruecy feature (used for HOPC cc ) is more robust to these chages compared with gradiet iformatio (used for HOG cc ). Fig. 7 shows the CPs detected usig HOPC cc with a template size of pixels betwee the sythetic images. I the elarged sub-images, oe ca see that the CPs are correctly located i the exact positios. HOGcc Fig. 6 CMR values versus the template size for the sythetic image pair. Results o Sythetic Datasets Fig. 6 shows the CMR values versus the template size for the sythetic image pair with o-liear radiometric differeces. It ca be clearly see that HOPC cc performs best i ay template size, followed by HOG cc ad, whereas achieves the lowest CMR values because it is vulerable to the piecewise Fig. 7 CPs detected by HOPC cc with the template size of pixels (sythetic images). Results o Real Datasets To comprehesively evaluate the proposed similarity metric i a real situatio, experimets also are performed o differet kids of multimodal remote sesig images (Visib-Ifra, LiDAR-Visib, Visib-SAR, ad Img-Map). The performace of the similarity metrics for differet kids of image pairs maily depeds o the radiometric distortios betwee each pair of

14 Ru time(sec) > 3 (a) (b) (c) (d) (e) (f) (g) (h) (i) (j) Fig. 9 CPs idetified by HOPC cc with the template size of pixels (real images). (a) Visib-Ifra. (b) Visib-Ifra. (c) LiDAR-Visib. (d) LiDAR-Visib. (e) LiDAR-Visib 3. (f) Visib-SAR. (g) Visib-SAR. (h) Visib-SAR 3. (i) Img-Map. (j) Img-Map. images. I geeral, the matchig of Visib-SAR ad Img-Map is more difficult tha that of Visib-Ifra due to the presece of more sigificat radiometric differeces ad oises HOGcc Fig. Ru time versus the template size to,, HOG cc ad HOPC cc. Fig. 8 shows the comparative CMR values of the four similarity metrics for the real multimodal images. I almost all the tests, HOPC cc outperforms the other similarity metrics for ay template size, ad HOG cc achieves the secod highest CMR values, followed by. I cotrast, is quite sesitive for multimodal images ad achieves the lowest CMR values compared with the other similarity metrics. Apart from havig higher CMR values, the performace of is less affected by template size compared with. Takig LiDAR-Visib 3 as a example [see Fig.9(e)], the performace of is very sesitive to template size chages, ad its CMR value is less tha 5% whe the template size is small (less tha pixels). I cotrast, HOPC cc achieves a CMR value of 75%. The reaso for this behavior is that computes the joit etropy betwee images, which is quite sesitive to sample sizes (i.e., template sizes) []. I additio, HOPC cc performs much better tha for the high-resolutio multimodal images (LiDAR-Visib 3, Visib-SAR ad 3, ad Img-Map ad ). As show i Fig. 8 (h), the CMR value of HOPC cc reaches 9%, while has a CMR value of oly 54.5% with a large template size ( pixels). Similar results are show i Fig. 8 (g), (i) ad (j). These results are maily due to the fact that high-resolutio images usually have saliet structural features. Thus, HOPC cc represetig the structural similarity has a obvious superiority to. I the experimets, HOPC cc ad HOG cc have achieved the two highest CMR values, which cofirms that the similarity

15 LiDAR-Visib SAR-Visib Img-Map Visib-Ifra > 4 TABLE III DESCRIPTION OF DATASETS USED IN THE MULTIMODAL REGISTRATION EXPERIMENTS Category No. Dataset descriptio Master image Slave image Image characteristic Sesor: SPOT 4 bad GSD: 3m Date: 9/ Size: Sesor: Ladsat 5 TM bad5 GSD: 3m Date: 4/ Size: Images cover a suburba area located i the south part of Wuha, Chia. There is a temporal differece of 9 moths betwee the images Source: Google Maps GSD: m Date: ukow Size: Source: Google Maps GSD: m Date: ukow Size: Images cover a urba area located i Foster City, USA. Their itesity iformatio are largely differet Sesor: TerraSAR-X GSD: 3m Date: 3/8 Size: 38 5 Sesor: TM bad3 GSD: 3m Date: 5/7 Size: 8 5 Images cover a suburba area located i Ruge, Germay. The images have the sigificat radiometric differeces. Sesor: TerraSAR-X GSD: 3m Date: /8 Size: 69 Source: Google Earth GSD: 3m Date: 3/9 Size: 6 3 Images cover a urba area located i Roseheim, Germay. The images have sigificat radiometric differeces ad local distortios. Moreover, they have a temporal differece of 4 moths. Sesor: LiDAR height GSD: m Date: / Size: Sesor: WorldView GSD: m Date: / Size: Images cover a urba area with high buildigs located i Sa Fracisco, USA. The images have sigificat radiometric differeces ad local distortios. Moreover, they have a temporal differece of moths, ad the LiDAR height image is affected by sigificat oise. Sesor: LiDAR itesity GSD: m Date: / Size: Sesor: WorldView GSD: m Date: / Size: 95 3 Images cover a urba area with high buildigs located i Sa Fracisco, USA. The images have sigificat radiometric differeces ad local distortios. Moreover, they have a temporal differece of moths, ad the LiDAR itesity image is affected by sigificat oise. metrics capturig structural properties are more robust to the oliear radiometric differeces tha the other similarity metrics. HOPC cc exhibits better performace tha HOG cc because HOPC cc is based o phase cogruecy, which is more robust to radiometric distortios (illumiatio ad cotrast chages) tha the gradiets used to build HOG cc. All the above results demostrate the effectiveess ad advatage of the proposed structural similarity metric i the matchig performace. The CPs detected by usig HOPC cc o all the real multimodal images are show i Fig. 9. F. Aalysis of Computatioal Efficiecy Computatioal efficiecy is aother importat idicator for evaluatig the matchig performace of similarity metrics. Fig. shows the ru time take from,, HOG cc, ad HOPC cc versus the template size. HOPC cc ad HOG cc are both calculated by the proposed fast matchig scheme (see Sectio II-F). The experimets have bee performed o a Itel Core i7-47mq.5ghz PC. Oe ca see that requires the least amout of ru time amog the similarity metrics due to its lowest computatioal complexity []. Sice HOPC cc ad HOG cc eed to extract the structural descriptors ad calculate the betwee such descriptors, they are both more time-cosumig tha. However, their computatioal efficiecy is better tha that of because calculates the joit histogram for every matched template widow pair, which requires extesive computatio [9]. The results depicted i Fig. illustrate that HOPC cc requires more ru time tha HOG cc maily because HOPC cc is required to extract the phase cogruecy feature, which is more time-cosumig tha calculatig the gradiets used to costruct HOG cc. V. EXPERIMENTAL RESULTS: MULTIMODAL REGISTRATION To validate the effectiveess of the proposed registratio method based o HOPC cc (see Sectio III), maual registratio ad a registratio method based o SIFT are used for compariso. I the proposed method, the block-based Harris operator is set to extract 3 evely distributed iterest poits for image registratio. For maual registratio, 3 CPs are selected evely over the master ad slave images, ad the PL trasformatio model is applied to achieve image registratio. I the SIFT-based registratio, the feature poits are first extracted from both images through the SIFT algorithm, the a oe-to-oe matchig betwee feature poits is performed usig the Euclidea distace ratio betwee the first ad the secod earest eighbor. RANdom SAmple Cosesus [48] is used to remove the outliers to achieve the fial CPs. Fially, the slave image is rectified by the PL trasformatio model. To assess the registratio accuracy, 4-6 check poits are selected evely betwee the master ad registered images by the method described i Sectio IV-B, ad the RMSE of check poits is used for accuracy evaluatio.

16 > 5 Master images Slave images Registratio results Elarged sub-images of registratio results Fig. Registratio results for all the test sets. The lies,,3,4,5, ad 6 correspod to Visib-Ifra, Img-Map, SAR-Visib, SAR-Visib, LiDAR-Visib, ad LiDAR-Visib, respectively. A. Descriptio of Datasets Six sets of multimodal images are used to validate the proposed method. Also for these experimets, the test sets iclude various kids of multimodal images such as Visib-Ifra, LiDAR-Visib, SAR-Visib, ad Img-Map. The master ad slave images of each test are captured by differet sesors ad at differet spectral regios, which results i sigificat o-liear radiometric differeces. The descriptio of datasets is give i Table III. B. Registratio Results Table IV reports the registratio accuracies for the six test sets. The proposed method is successful i registerig all the image pairs ad achieves the highest registratio accuracy. For the SAR-to-Visible ad LiDAR-to-Visible image registratio (SAR-Visib, SAR-Visib, LiDAR-Visib, ad LiDAR-Visib ), the proposed method outperforms maual registratio. Oe reaso for this outcome is that the image pairs of these test sets have sigificat radiometric differeces ad the SAR ad LiDAR data cotai sigificat oise, which results i a large differece betwee the itesity details of the two images ad makes it difficult to locate the CPs precisely by visual ispectio. Aother reaso is that the proposed method detects may more CPs tha maual registratio, which is very beeficial to the PL trasformatio model for fittig complex deformatios betwee images [43]. I additio, the SIFT-based registratio fails i most of the tests except Visib-Ifra

17 LiDAR-Visib SAR-Visib Img-Map Visib-Ifra > 6 because the SIFT algorithm is ot able to extract the highly-repeatable commo features preset i multimodal images due to their sigificat radiometric differeces [49]. O the other had, oe ca see that the proposed method achieves differet registratio accuracies for differet test sets because of the differeces i the image characteristics. I geeral, the test sets havig images with lower resolutios achieve relatively higher registratio accuracy tha those with higher resolutios. For example, Visib-Ifra ad SAR-Visib achieve a sub-pixel registratio accuracy, whereas the other test sets have a RMSE larger tha pixel. This is maily attributed to the fact that the test sets which iclude the images with low resolutios cover flat areas ad there is almost o complex geometric deformatio betwee the images. The higher resolutio images coverig urba areas, such as the image pairs of SAR-Visib, LiDAR-Visib ad, have sigificat local distortios caused by relief displacemet of buildigs. This is a itrisic problem for high-resolutio registratio, which caot be resolved by a image-to-image registratio util a true orthorectificatio is applied [5]. Fig. shows the registratio results of all the test sets. From the elarged sub-images, oe ca see that the registratios are satisfactory ad accurate for all the test sets. The above results demostrate the effectiveess of the proposed method for registerig multimodal remote sesig images. TABLE IV REGISTRATION RESULTS FOR ALL THE CONSIDERED TEST SETS Category No. Method Matched CPs CPs with error elimiatio RMSE (pixels) Proposed Maual SIFT Proposed Maual SIFT 5 Failed Proposed Maual SIFT 8 Failed Proposed VI. CONCLUSION This paper has preseted a ovel similarity metric amed HOPC cc for multimodal remote sesig image registratio. This metric addresses the issues related to the sigificat o-liear radiometric differeces usually preset i images acquired by differet sesors. First, the phase cogruecy model is exteded to build its orietatio represetatio. The, the amplitude ad orietatio of phase cogruecy are used to costruct HOPC cc, ad a fast template matchig scheme is desiged for this metric to detect CPs. HOPC cc aims to capture the structural similarity betwee images ad has bee evaluated agaist various kids of multimodal datasets, icludig Visible-to-Ifrared, LiDAR-to-Visible, Visible-to-SAR, ad Image-to-Map. The experimetal results demostrate clearly that HOPC cc outperforms the two popular similarity metrics icludig ad, especially for image pairs that cotaied rich structural features, such as the high-resolutio visible ad SAR images (Visib-SAR ad Visib-SAR 3 i Table I), ad the LiDAR height ad visible images (LiDAR-Visib 3 i Table I). Moreover, whe HOPC cc is implemeted with the proposed fast matchig scheme, less computatio time is required compared to. A robust registratio method based o HOPC cc for multimodal images is itroduced that uses various techiques icludig the block-based Harris operator, HOPC cc, bidirectioal matchig, ad PL trasformatio. The experimetal results usig six differet pairs of multimodal images cofirms that the ew method ca detect a large umber of evely distributed CPs betwee the images ad its registratio accuracy is better tha the maual ad SIFT-based registratio methods. Sice HOPC cc uses the framework of HOG to build the descriptor, the HOG descriptor is also itegrated as a similarity metric (amed HOG cc ) for image registratio. The experimetal results show that both HOPC cc ad HOG cc perform better tha ad, which demostrates that the framework of HOG is effective for buildig a structural descriptor for multimodal registratio. Whe compared with HOG cc, HOPC cc improves the matchig performace by usig phase cogruecy istead of gradiet iformatio to build the descriptor. I future efforts, we will attempt to itegrate other features (e.g., wavelets ad self-similarity [5,5]) ito the framework of HOG for multimodal remote sesig image registratio. Maual SIFT 8 Failed Proposed Maual SIFT Failed Proposed Maual SIFT 76 Failed Although our experimets show that HOPC cc is robust to o-liear radiometric differeces, some improvemets to HOPC cc should be cosidered. Oe limitatio of HOPC cc is that it is ot ivariat for scale ad rotatio chages, which could be critical i cases where sigificat chages of scale ad rotatio are preset betwee images. I practice, these deformatios betwee images eed to be elimiated usig the direct georeferecig techique based o the avigatio istrumets aboard satellites. A Fourier aalysis method for rotatio-ivariat local descriptor [53] may also address this issue to some degree. Although HOPC cc is applied through a fast matchig scheme, it is still more time-cosumig

18 > 7 compared with sice HOPC cc requires the calculatio of a high-dimesioal structural feature descriptor to be calculated. I future work, this issue could be resolved by reducig the dimesios of the descriptor usig a dimesio-reductio techique, such as pricipal compoet aalysis. I additio, it is worth otig that the performace of HOPC cc may degrade if the images of iterest iclude less structure or shape iformatio because HOPC cc depeds o the structural properties of images. I this case, a image ehacemet approach could be applied to ehace the cotour or edge features, which may be helpful to image registratio. ACKNOWLEDGMENT The authors would like to thak the aoymous reviewers for their helpful commets ad good suggestios. REFERENCES [] B. Zitova ad J. Flusser, Image registratio methods: a survey, Image Vis. Comput., vol., o., pp. 977-, Oct. 3. [] H. Gocalves, L. Corte-Real, ad J. A. Gocalves, Automatic image registratio through image segmetatio ad SIFT, IEEE Tras. Geosci. Remote Ses., vol. 49, o. 7, pp , Mar.. [3] H. Gocalves, J. A. Gocalves, L. Corte-Real, ad A. C. Teodoro, CHAIR: Automatic image registratio based o correlatio ad Hough trasform, It. J. Remote Ses., vol. 33, o. 4, pp , Jul.. [4] P. Butig, F. Labrosse, ad R. Lucas, A multi-resolutio area-based techique for automatic multi-modal image registratio, Image Vis. Comput.., vol. 8, o. 8, pp. 3-9, Aug.. [5] L. Yu, D. R. Zhag, ad E. J. Holde, A fast ad fully automatic registratio approach based o poit features for multi-source remote-sesig images, Comput. Geosci., vol. 34, o. 7, pp , Jul. 8. [6] H. G. Sui, C. Xu, J. Y. Liu, ad F. Hua, Automatic optical-to-sar image registratio by iterative lie extractio ad vorooi itegrated spectral poit matchig, IEEE Tras. Geosci. Remote Ses., vol. 53, o., pp , Nov. 5. [7] H. Goçalves, J. Goçalves, ad L. Corte-Real, HAIRIS: A method for automatic image registratio through histogram-based image segmetatio, IEEE Tras. Image Process., vol., o. 3, pp , Sep.. [8] D. G. Lowe, Distictive image features from scale-ivariat keypoits, It. J. Comput. Vis., vol. 6, o., pp. 9-, Nov. 4. [9] K. Mikolajczyk ad C. Schmid, A performace evaluatio of local descriptors, IEEE Tras. Patter Aal. Mach. Itell., vol. 7, o., pp , Oct. 5. [] J. L. Ma, J. C. W. Cha, ad F. Caters, Fully automatic subpixel image registratio of multiagle CHRIS/Proba data, IEEE Tras. Geosci. Remote Ses., vol. 48, o. 7, pp , Jul.. [] A. Sedaghat, M. Mokhtarzade, ad H. Ebadi, Uiform robust scale-ivariat feature matchig for optical remote sesig images, IEEE Tras. Geosci. Remote Ses., vol. 49, o., pp , Nov.. [] A. Sedaghat, ad H. Ebadi, Remote sesig image matchig based o adaptive biig SIFT descriptor, IEEE Tras. Geosci. Remote Ses., vol. 53, o., pp , Oct. 5. [3] Y. Ye, ad J. Sha, A local descriptor-based registratio method for multispectral remote sesig images with o-liear itesity differeces, ISPRS J. Photogramm. Remote Ses., vol. 9, pp , Apr. 4. [4] H. Bay, A. Ess, T. Tuytelaars, ad L. Va Gool, Speeded-up robust features (SURF), Comput. Visio Image Uderstadig, vol., o. 3, pp , Ju. 8. [5] E. Rublee, V. Rabaud, K. Koolige, ad G. Bradski, ORB: A efficiet alterative to SIFT or SURF, i Proc. IEEE It. Cof. Comput. Vis.,, pp [6] A. Alahi, R. Ortiz, ad P. Vadergheyst, FREAK: Fast retia keypoit, i Proc. IEEE Comput. Vis. Patter Recogit.,, pp [7] A. Kelma, M. Sofka, ad C. V. Stewart, Keypoit descriptors for matchig across multiple image modalities ad o-liear itesity variatios, i Proc. IEEE Comput. Vis. Patter Recogit., 7, pp [8] J. Iglada ad A. Giros, O the possibility of automatic multisesor image registratio, IEEE Tras. Geosci. Remote Ses., vol. 4, o., pp. 4-, Oct. 4. [9] Y. Hel-Or, H. Hel-Or, ad E. David, Fast template matchig i o-liear toe-mapped images, i Proc. IEEE It. Cof. Comput. Vis.,, pp [] Y. Hel-Or, H. Hel-Or, ad E. David, Matchig by toe mappig: Photometric ivariat template matchig, IEEE Tras. Patter Aal. Mach. Itell., vol. 36, o., pp , Feb. 4. [] A. A. Cole-Rhodes, K. L. Johso, J. LeMoige, ad I. Zavori, Multiresolutio registratio of remote sesig imagery by optimizatio of mutual iformatio usig a stochastic gradiet, IEEE Tras. Image Process., vol., o., pp , Dec. 3. [] H. M. Che, M. K. Arora, ad P. K. Varshey, Mutual iformatio-based image registratio for remote sesig data, It. J. Remote Ses., vol. 4, o. 8, pp , Sep. 3. [3] D. Bruer, G. Lemoie, ad L. Bruzzoe, Earthquake damage assessmet of buildigs usig VHR optical ad SAR imagery, IEEE Tras. Geosci. Remote Ses., vol. 48, o. 5, pp. 43-4, May. [4] M. Ravabakhsh, ad C. S. Fraser, A comparative study of DEM registratio approaches, J. Spat. Sci., vol. 58, o., pp , Apr. 3. [5] J. M. Murphy, J. Le Moige, ad D. J. Hardig, Automatic image registratio of multimodal remotely sesed data with global shearlet features, IEEE Tras. Geosci. Remote Ses., vol. 54, o. 3, pp , Mar. 6. [6] I. Zavori ad J. Le Moige, Use of multiresolutio wavelet feature pyramids for automatic registratio of multisesor imagery, IEEE Tras. Image Process., vol. 4, o. 6, pp , Ju. 5. [7] M. P. Heirich, M. Jekiso, M. Bhusha, T. Mati, F. V. Gleeso, J. M. Brady, ad J. A. Schabel, "No-local shape descriptor: a ew similarity metric for deformable multi-modal registratio," i Medical Image Computig ad Computer-Assisted Itervetio,, pp [8] M. P. Heirich, M. Jekiso, M. Bhusha, T. Mati, F. V. Gleeso, S. M. Brady, ad J. A. Schabel, ND: Modality idepedet eighbourhood descriptor for multi-modal deformable registratio, Med. Image Aal., vol. 6, o. 7, pp , Oct.. [9] H. Rivaz, Z. Karimaghaloo, ad D. L. Collis, Self-similarity weighted mutual iformatio: A ew origid image registratio metric, Med. Image Aal., vol. 8, o., pp , Feb. 4. [3] Z. Li, D. Mahapatra, J. A. W. Tielbeek, J. Stoker, L. J. va Vliet, ad F. M. Vos, Image registratio based o autocorrelatio of local structure, IEEE Tras. Med. Imagig, vol. 35, o., pp , Ja. 6. [3] P. Kovesi, Image features from phase cogruecy, Videre: Joural of Computer Visio Research, vol., o. 3, pp. -6, 999. [3] Y. Ye ad L. She, HOPC: A ovel similarity metric based o geometric structural properties for multi-modal remote sesig image matchig, ISPRS A. Photogramm. Remote Ses. Spatial If. Sci., pp. 9-6, Ju. 6. [33] N. Dalal ad B. Triggs, "Histograms of orieted gradiets for huma detectio." i Proc. IEEE Comput. Vis. Patter Recogit., 5, pp [34] J. Cay, A computatioal approach to edge-detectio, IEEE Tras. Patter Aal. Mach. Itell., vol. 8, o. 6, pp , Nov [35] A. V. Oppeheim ad J. S. Lim, The importace of phase i sigals, Proceedigs of the IEEE, vol. 69, o. 5, pp , May 98. [36] M. C. Morroe ad R. A. Owes, Feature detectio from local eergy, Patter Recogitio Letters, vol. 6, o. 5, pp , Dec [37] P. Moreo, A. Berardio, ad J. Satos-Victor, Improvig the SIFT descriptor with smooth derivative filters, Patter Recogitio Letters, vol. 3, o., pp. 8-6, Ja. 9. [38] P. F. Felzeszwalb, R. B. Girshick, D. McAllester, ad D. Ramaa, Object detectio with discrimiatively traied part-based models, IEEE Tras. Patter Aal. Mach. Itell., vol. 3, o. 9, pp ,. [39] H. Harzallah, F. Jurie, ad C. Schmid, Combiig efficiet object localizatio ad image classificatio, i Proc. IEEE It. Cof. Comput. Vis., 9, pp

19 > 8 [4] A. Shrivastava, T. Malisiewicz, A. Gupta, ad A. A. Efros, Data-drive visual similarity for cross-domai image matchig, ACM Tras. Graph., vol. 3, o. 6, Dec.. [4] L. Di Stefao, M. Marchioi, ad S. Mattoccia, A fast area-based stereo matchig algorithm, Image Vis. Comput., vol., o., pp , Oct. 4. [4] A. Wog ad D. A. Clausi, ARRSI: Automatic registratio of remote-sesig images, IEEE Tras. Geosci. Remote Ses., vol. 45, o. 5, pp , May 7. [43] A. Goshtasby, Piecewise liear mappig fuctios for image registratio, Patter Recogit., vol. 9, o. 6, pp , 986. [44] V. J. D. Tsai, Delauay triagulatios i TIN creatio: A overview ad a liear-time algorithm, It. J. Geogr. If. Sci., vol. 7, o. 6, pp. 5-54, 993. [45] Y. Ha, J. Choi, Y. Byu, ad Y. Kim, Parameter optimizatio for the extractio of matchig poits betwee high-resolutio multisesor images i urba areas, IEEE Tras. Geosci. Remote Ses., vol. 5, o. 9, pp , Sep. 4. [46] A. Myroeko ad X. B. Sog, Itesity-based image registratio by miimizig residual complexity, IEEE Trasactios o Medical Imagig, vol. 9, o., pp , Nov.. [47] J. Pluim, J. Maitz, ad M. Viergever, "Image registratio by maximizatio of combied mutual iformatio ad gradiet iformatio." i Medical Image Computig ad Computer-Assisted Itervetio,, pp [48] M. A. Fischler ad R. C. Bolles, Radom sample cosesus: A paradigm for model fittig with applicatios to image aalysis ad automated cartography, Commuicatios of the ACM, vol. 4, o. 6, pp , 98. [49] C. L. Tsai, C. Y. Li, G. Yag, ad K. S. Li, The edge-drive dual-bootstrap iterative closest poit algorithm for registratio of multimodal fluorescei agiogram sequece, IEEE Tras. Med. Imagig, vol. 9, o. 3, pp , Mar.. [5] G. Hog ad Y. Zhag, Wavelet-based image registratio techique for high-resolutio remote sesig images, Comput. Geosci., vol. 34, o., pp. 78-7, Dec. 8. [5] E. Shechtma ad M. Irai, Matchig local self-similarities across images ad videos, i Proc. IEEE Comput. Vis. Patter Recogit., 7, pp. -8. [5] Y. Ye, L. She, M. Hao, J. Cheg, ad Z. Xu, Robust Optical-to-SAR Image Matchig Based o Shape Properties, IEEE Geosci. Remote Ses. Lett., 7 (i press). [53] K. Liu, H. Skibbe, T. Schmidt, T. Blei, K. Palme, T. Brox, ad O. Roeberger, Rotatio-ivariat HOG descriptors usig Fourier aalysis i polar ad spherical coordiates, It. J. Comput. Vis., vol. 6, o. 3, pp , Feb. 4. Yuaxi Ye (M 7) received the B.S. degree i Remote Sesig Sciece ad Techology from Southwest Jiaotog Uiversity, Chegdu, Chia, i 8, ad the Ph.D. degree i Photogrammetry ad Remote Sesig from Wuha Uiversity, Wuha, Chia, i 3. Sice Sep. 3, he has bee a Assistat Professor with the Faculty of Geoscieces ad Evirometal Egieerig, Southwest Jiaotog Uiversity, Chegdu, Chia. He is curretly a postdoctoral fellow i the Remote Sesig Laboratory i the Departmet of Iformatio Egieerig ad Computer Sciece, Uiversity of Treto. He achieved The ISPRS Prizes for Best Papers by Youg Authors of 3th Iteratioal Society for Photogrammetry ad Remote Sesig Cogress (Prague, July 6). His research iterests iclude remote sesig image processig, image registratio, feature extractio, ad chage detectio. Jie Sha (SM 4) received his Ph.D. degree i photogrammetry ad remote sesig from Wuha Uiversity, Chia. He has held faculty positios at uiversities i Chia ad Swede, ad has bee a Research Fellow i Germay. He is with the Lyles School of Civil Egieerig, Purdue Uiversity, West Lafayette, IN, USA. His areas of iterests iclude sesor geometry, patter recogitio from images ad light detectio ad ragig (LiDAR) data, object extractio ad recostructio, urba remote sesig, ad automated mappig. He is a Associate Editor for the IEEE Trasactios o Geosciece ad Remote Sesig. He is a recipiet of multiple academic awards, icludig the Talbert Abrams Grad Award ad the Evirometal Systems Research Istitute Award for Best Scietific Paper i Geographic Iformatio Systems (First Place). Lorezo Bruzzoe (S 95 - M 98 - SM 3 - F ) received the Laurea (M.S.) degree i electroic egieerig (summa cum laude) ad the Ph.D. degree i telecommuicatios from the Uiversity of Geoa, Italy, i 993 ad 998, respectively. He is curretly a Full Professor of telecommuicatios at the Uiversity of Treto, Italy, where he teaches remote sesig, radar, patter recogitio, ad electrical commuicatios. Dr. Bruzzoe is the fouder ad the director of the Remote Sesig Laboratory i the Departmet of Iformatio Egieerig ad Computer Sciece, Uiversity of Treto. His curret research iterests are i the areas of remote sesig, radar ad SAR, sigal processig, ad patter recogitio. He promotes ad supervises research o these topics withi the frameworks of may atioal ad iteratioal projects. Amog the others, he is the Pricipal Ivestigator of the Radar for icy Moo exploratio (RIME) istrumet i the framework of the JUpiter ICy moos Explorer (JUICE) missio of the Europea Space Agecy. He is the author (or coauthor) of 86 scietific publicatios i referred iteratioal jourals (34 i IEEE jourals), more tha 6 papers i coferece proceedigs, ad book chapters. He is editor/co-editor of 6 books/coferece proceedigs ad scietific book. His papers are highly cited, as prove form the total umber of citatios (more tha 68) ad the value of the h-idex (66) (source: Google Scholar). He was ivited as keyote speaker i 3 iteratioal cofereces ad workshops. Sice 9 he is a member of the Admiistrative Committee of the IEEE Geosciece ad Remote Sesig Society. Dr. Bruzzoe raked first place i the Studet Prize Paper Competitio of the 998 IEEE Iteratioal Geosciece ad Remote Sesig Symposium (Seattle, July 998). Sice that time he was recipiet of may iteratioal ad atioal hoors ad awards. Dr. Bruzzoe was a Guest Co-Editor of differet Special Issues of iteratioal jourals. He is the co-fouder of the IEEE Iteratioal Workshop o the Aalysis of Multi-Temporal Remote-Sesig Images (MultiTemp) series

20 > 9 ad is curretly a member of the Permaet Steerig Committee of this series of workshops. Sice 3 he has bee the Chair of the SPIE Coferece o Image ad Sigal Processig for Remote Sesig. Sice 3 he has bee the fouder Editor-i-Chief of the IEEE Geosciece ad Remote Sesig Magazie. Curretly he is a Associate Editor for the IEEE Trasactios o Geosciece ad Remote Sesig ad the Joural of Applied Remote Sesig. Sice he has bee appoited Distiguished Speaker of the IEEE Geosciece ad Remote Sesig Society. Li She received the B.S. degree i Photogrammetry ad Remote Sesig from Wuha Uiversity, Wuha, Chia, i 8, ad the Ph.D. degree from the College of Resources Sciece ad Techology, Beijig Normal Uiversity, Beijig, Chia, i 3. He is curretly a Assistat Professor with the Faculty of Geoscieces ad Evirometal Egieerig, Southwest Jiaotog Uiversity, Chegdu, Chia. His research iterests iclude remote sesig image processig, patter recogitio, ad remote sesig applicatios i atural disaster reductio ad detectio of potetial hazards alog the high-speed railway lie.