Feature-Area Optmzaton: A Novel SAR Image Regstraton Method Fuqang Lu, Fukun B, Lang Chen, Hao Sh and We Lu Abstract Ths letter proposes a synthetc aperture radar (SAR) mage regstraton method named Feature-Area Optmzaton (FAO). Frst, the tradtonal area-based optmzaton model s reconstructed and decomposed nto three key but uncertan factors: ntalzaton, slce set and regularzaton. Next, structural features are extracted by scale nvarant feature transform (SIFT) n dual-resoluton space (SIFT-DRS), a novel SIFT-Lke method dedcated to FAO. Then, the three key factors are determned based on these features. Fnally, solvng the factor-determned optmzaton model can get the regstraton result. A seres of experments demonstrate that the proposed method can regster mult-temporal SAR mages accurately and effcently. Index Terms SAR mage regstraton, Area-based optmzaton model, ntalzaton and regularzaton, scale nvarant feature transform (SIFT), dual-resoluton space S I. INTRODUCTION YNTHETIC aperture radar (SAR) mage regstraton s a fundamental process for varous applcatons, such as change detecton, mage fuson, and mult-temporal analyss. However, t s more dffcult to regster SAR mages than optcal mages as a result of multplcatve nose, whch s referred to as speckle []. In general, feature-based regstraton methods are adopted to mprove the adaptablty of SAR mage regstraton. Based on scale nvarant feature transform (SIFT) [2], Schwnd et al. [3] proposed the SIFT-Octave (SIFT-OCT) whch mproves the probablty of correct matches by skppng the frst octave of the scale space pyramd. You and Fu [4] proposed the blateral flter SIFT (BF-SIFT) to ft SAR mage regstraton by adoptng an ansotropc Gaussan scale space. The above two methods can reduce the number of false matches, but they both need to decrease mage resoluton and thus lose nformaton. Dellnger et al. [5] proposed SAR-SIFT, whch adopted a new gradent defnton, a new detector employng SAR-Harrs scale-space and a new orentaton assgnment for SAR mage regstraton. However, the results of SAR-SIFT stll carry almost 0% false matches [5], whch cause regstraton bases. These methods requre an accurate feature extractor and a consstent feature descrptor to make sure that the same features n dfferent Ths work was supported by the Natonal Natural Scence Foundaton of Chna (Grant No. 6794) Fu-qang Lu, Lang Chen and Hao Sh are wth School of Informaton and Electronc, Bejng Insttute of Technology, Bejng 0008, Chna. (e-mal: fqlu@outlook.com; chenl@bt.edu.cn) Fu-kun BI s wth College of Informaton Engneerng, North Chna Unversty of Technology, Bejng 0044, Chna. pre-regstraton mages can be matched accurately. Then, the geometrc transformaton can be executed based on a number of feature matches. Unlke the feature-based methods, area-based methods [7] do not extract and match features. Therefore, they are not dsturbed by false matches. These methods turn regstraton nto an optmzaton problem and tune the geometrc transformaton based on the correspondng spatal relatonshp between mages. If the optmzaton model can converge to a global extreme effcently, t can be used to accurate SAR mage regstraton. However, compared to feature-based methods, area-based methods are less commonly used n SAR mage regstraton [6]. Under the condtons that the SAR mages are usually large and occuped by serous speckles, these methods are computatonally expensve and lable to get local optma [6]. To overcome the above problems, we propose a novel SAR mage regstraton method named Feature-Area Optmzaton (FAO). In FAO, features are used to help area-based optmzaton model converge to a global optmum and reduce ts complexty. Frst, we decompose the tradtonal area-based optmzaton model nto three key but uncertan factors: ntalzaton, slce set and regularzaton. Next, structural features are extracted by SIFT n dual-resoluton space (SIFT-DRS). SIFT-DRS s a novel SIFT-lke method dedcated to extractng features from pre-regstraton mages fast. Then, the three factors are determned based on these features as follows: the ntalzaton s set equal to the transformaton matrx computed based on feature matches; the orgnal mages are replaced by slces of superposton; and the regularzaton tem s determned based on the determned ntalzaton. Fnally, solvng the factor-determned optmzaton model can get the SAR regstraton result. In tradtonal area-based methods [7], the ntalzaton s set emprcally or close to zero, and the regularzaton cannot be determned f there s no pror nformaton for SAR regstraton. The nnovaton of the proposed method s to use structural features extracted from pre-regstraton mages to determne these factors. Thanks to the three key factors determned based on features, the area-based optmzaton model avods convergng to local optma. And by an optmzng process, the regstraton result of the proposed method can be better than the results of feature-based methods. FAO can be regarded as a composton of feature-based method and area-based method. A seres of experments show that the proposed method s more effectve and more effcent for SAR mage regstraton.
II. OPTIMIZATION MODEL FOR SAR IMAGE REGISTRATION AND FACTOR DECOMPOSITION In ths secton, we descrbe the affne transformaton model and the area-based optmzaton model for SAR mage regstraton. And then we propose the factor decomposton of the optmzaton model. A. Affne Transformaton Model The affne transformaton model [5] s sutable for mage regstraton. Its formula s shown as follows: { x 2 = a x + b y + c y 2 = a 2 x + b 2 y + c 2 () where (x, y ) and (x 2, y 2 ) denote the coordnates of the pre-regstraton mages. () can be shaped nto matrx form as follows: P 2 = H P (2) a b c H = [ a 2 b 2 c 2 ] (3) 0 0 wth P (x, y, ) denotng the generc SAR pxel of mage I and H denotng the transformaton matrx. Though SAR s ntrnscally two-dmensonal, the thrd coordnates of P s essental for obeyng the rule of matrx multplcaton and testng result. Generally, the purpose of SAR mage regstraton s to estmate the coeffcents of H. B. Area-Based Optmzaton Model for Regstraton In tradtonal area-based methods [7], the Eucldean norm s adopted as the crteron to estmate the transformaton matrx. Assumng P denotes the generc SAR pxel, and M = [P, P 2,, P n ] denotes the set of pxels n I, the optmzaton model s shown as follows [8]: mn H m I 2(H M ) I (M ) 2 (4) wth m donatng the number of the pxels nvolved n computaton. The extreme of (4) s the result of SAR mage regstraton. However, as a result of the large scale of SAR mages and speckles, solvng (4) drectly s computatonally expensve and (4) s lable to converge to a local extreme. C. Factor Decomposton of Area-Based Optmzaton Model Generally, f only parts of pre-regstraton mages are nvolved n (4), solvng the optmzaton model wll be computatonally cheaper than solvng the model nvolvng orgnal SAR mages. We name the collecton of these parts as slce set. Involvng only parts of orgnal mages n computaton s to estmate the global optmum by a smlar local optmum. Ths operaton causes a problem named over-fttng. Over-fttng problems can be solved by an approprate regularzaton [0]. In addton, an accurate ntalzaton for the optmzaton model can avod the convergence problems and make solvng the optmzaton model easly. Based on the above states, we reconstruct (4) as (5) and decompose (5) nto three factors: ntalzaton, slce set and regularzaton. mn H m D 2 (H M ) D (M ) 2 + λ reg (5) where reg s the regularzaton tem, λ s weght decay parameter and D denotes the slce of orgnal mage. The superscrpt s the ndex and the subscrpt shows whch mage the slce belongs to. All slces n (5) consttute a slce set set. The weght decay parameter λ s consdered wth regularzaton together. Defne some operatons of D: D D j returns the acreage of the superposton; sum(d, D j ) returns a new slce whch nvolves full place of D, D j ; area(d) returns the acreage of D. We set two lmts for D: one s that there s no superposton between any two slces of one certan mage; another one s that some parts of the two slces wth same ndex should be same. The frst lmt avods repeat computaton and the second one makes sure these slces can be regstered. The above lmts can be wrtten as (6) (7), respectvely. D mage j D mage = 0, j (6) D ndex D j ndex 0, j (7) Proporton denotes the rato between slce set and pre-regstraton mages. We use proporton to descrbe the property of the slce set. Defne proporton as follows: proporton = area(sum(d, D n ))+area(sum(d 2, D 2 n )) area(i )+area(i 2 ) (8) wth D, D n, D 2, D 2 n set. The proporton s the man property of slce set. The three factors nfluence whether (5) can be solved effcently. But the approprate values or forms of the three factors are uncertan. To make (5) soluble, we should fnd an accurate ntalzaton, determne an approprate proporton of the slce set, and choose the form of the regularzaton tem. III. FACTOR DETERMINATION BASED ON FEATURES EXTRACTED BY SIFT-DRS Ths secton proposes usng structural features to determne ntalzaton, slce set, and regularzaton. Frst, dual-resoluton space (SIFT-DRS) s desgned to extract the dstnctve features from SAR mages. Then, we detal how to determne the three factors based on these structural features. A. Feature Extracton by SIFT n Dual-Resoluton Space In ths part, we propose an effectve and effcent method, SIFT n dual-resoluton space (SIFT-DRS), to extract dstnctve features from pre-regstraton mages. These features are devoted to determne the three key factors: ntalzaton, slce set and regularzaton. The dagram of SIFT-DRS s shown n Fg.. There s no relatonshp between dual-resoluton space and the Gaussan Pyramd, whch means that we do not adapt the nternal processes of SIFT.
smlar to the ntalzaton. Based on the above statement, the form of regularzaton s determned by ntalzaton. Fg.. Dagram of SIFT-DRS SIFT-DRS s composed by two man operatons, whch are showed as Low Resoluton and Orgnal Resoluton respectvely n Fg.. And ths s the orgn of dual-resoluton space. The mage after dowmsamplng s loosely regarded as low-resoluton mage. The mage after downsamplng should contan enough area for feature extracton. The mnmum radus of the regon where SIFT creates descrpton s [2], and we need dozens of low-resoluton features. As a result, we lmt that the scale of the downsampled mage s 28 28 at least. Defnng N as the downsamplng rate and L as the mnmum scale of the orgnal mage, N needs to satsfy the followng: L 28 (9) N In Low Resoluton, features are extracted by SIFT frst, and then they are matched wth the low-resoluton features from another mage. These matches demarcate the superposton of the two mages roughly. In Orgnal Resoluton, the area where low-resoluton feature matches demarcate s ncsed nto some squares randomly. Ths step reduces the cost of further feature extracton. The sze of each square s max {N 2, 64 64} to make sure that there are enough area for feature extracton and descrpton. Then features n orgnal resoluton are extracted from these squares by SIFT. The feature set s composed of these features. By the dual resoluton space, we can extract features from the pre-regstraton mages more effcent than by SIFT n orgnal mages. And the feature set s used for determnng the three key factors. B. Factor Determnaton In ths part, we detal how to determne ntalzaton, slce set and regularzaton based on the structural features. Frst, matchng pars are found from the features extracted by SIFT-DRS. Then transformaton matrx s computed by random sample consensus (RANSAC) [2]. As showed n Fg. 2, the RANSAC result, the transformaton matrx, s set as the ntalzaton. If two features from dfferent mage are matched, the neghborhoods of the features wll satsfy (6) (7), and they can be chosen as the slces. We call the step of choosng whch slces are nvolved n the slce set Random Select and Proporton Lmt. If the ntalzaton s close to the global optmum, the regstraton result wll be Fg. 2. Dagram of determnng ntalzaton, slce set and regularzaton based on structural features ) Intalzaton After SIFT-DRS, two feature sets of the pre-regstraton mages are obtaned. The matchng pars of these features are found frst. Then those matchng pars are selected and the transformaton matrx, H trans, s computed by random sample consensus (RANSAC) [2]. Usually, feature-based methods regard the transformaton matrx as the fnal regstraton result [2-6]. Here, we set H trans as the ntalzaton, H 0. 2) Slce Set The matchng pars mark part of the superposton of the two pre-regstraton mages. In practce, the pre-regstraton mages are cut nto amounts of slces whose centers are the key ponts from those matchng pars. The szes of the slces are set as 256 256 emprcally. Each matchng par can determne two related slces, whch s named related slce par, from two mages. Because the slces are chosen from the superposton, related slce pars wll satsfy (7). Then we select some of related slce pars randomly to compose the slce set, only f slces from the same mage satsfy (6). We use proporton, whch s defned as (8), to lmt how many slces we select. 3) Regularzaton We use part of pre-regstraton mages to replace the orgnal mages. In another word, the global optmum s estmated by an approprate local optmum. Ths operaton makes the optmzaton model over-fttng. As a result, a regularzaton tem s added as (5). Prevously, the regularzaton cannot be determned, because there s no pror nformaton. If the ntalzaton s close to the global optmum, the fnal result, regstraton matrx, wll be smlar to the ntalzaton. Thus, we assume that H H 0 s close to zero. Then, the above assumpton can be regarded as pror nformaton, and the regularzaton s set as H H 0 2 n practce. The fnal form of (5) s showed as follows: mn H m D 2 (H M ) D (M ) 2 + λ H H 0 2 (0) where the weght decay parameter λ controls the relatve mportance of the two tems. We set λ as 0.00. It may not be
the optmal value for every SAR regstraton case, but t works well n our experment. Through the above three dscussons, three key factors are determned. Solvng (0) can get the regstraton matrx. All numbers determned n ths secton are vald for FAO. However, there are some uncertan parameters ncludng downsamplng rate N, proporton, and maxmum generaton. Maxmum generaton represents the maxmum teraton tmes to solve (0). These parameters nfluence the effectveness and effcency of FAO. IV. EXPERIMENTS AND ANALYSIS Ths secton descrbes a seres of experments. How to choose the above uncertan parameters s dscussed n experment A. And experment B valdates our method by two regstraton examples and comparsons wth prevously works. All experments are mplemented by Matlab n a computer equpped wth an Intel(R) Core(TM) 5-3470 and 4 GB of RAM. Moreover, the expermental data are acqured by the HJ-C satellte wth 5m ground resoluton. A dataset contanng 0 pars of SAR mages s carefully selected. These SAR mages whose szes are 4096 4096 contan varous scenes, ncludng mountan, sea, urban, rver, forest and desert. A. Experment on Parameter Determnaton The parameters ncludng downsamplng rate N, proporton, and maxmum generaton are determned n experment A. ) Downsamplng Rate As mentoned prevously, SIFT-DRS ams to extract relable features rapdly. SIFT-DRS wll be computatonally cheap and the features wll be smlar to the results of SIFT, f N s chosen approprately. SIFT-DRSs n varous N are mplemented throughout the dataset, and the mean tmes are recorded, whch s shown n Fg. 3(a). The mean tme how long operate SIFT throughout the dataset s 97 s. To measure the accuracy of SIFT-DRS, () s employed to measure the smlarty between features generated by SIFT-DRS and those extracted by SIFT. error = C C 2 () where C s the coordnate of the feature of SIFT and C s the coordnate of the related feature of SIFT-DRS. Fg. 3(b) shows the mean errors n dfferent downsamplng rate. Fg. 3. Tmes and errors of SIFT-DRS n dfferent downsamplng rates As Fg. 3 (a) shows, SIFT-DRS can drastcally decrease the computatonal cost compared to SIFT. Nevertheless, a hgh downsamplng rate does not mean a low cost because the squares n orgnal resoluton space are large. As Fg. 3(b) shows, the error s larger than pxel when the downsamplng rate s over :8. Based on ths experment, we set N as :4 or :8 generally. 2) Proporton Ths experment s to fnd the approprate proporton. Images from the above dataset are nvolved n ths test. SIFT-DRS wth downsamplng rate :4 are completed frst, and we only record the teraton step on solvng (0). Root-mean-square error (RMSE) s adopted to measure the regstraton accuracy; t can be wrtten as follows: RMSE = n = (x x ) 2 + (y y ) 2 n (2) where (x, y ) and (x, y ) are the coordnates of the th matchng key ponts and n s the number of matchng key ponts. Fg. 4. (a) shows the relatonshp between tme and proporton, and (b) shows the relatonshp between RMSE and proporton The maxmum generaton s set as 000. Then we record the computaton tmes and the RMSEs n dfferent proportons. Fg. 4(a) shows the relatonshp between tme and proporton, and Fg. 4(b) shows the relatonshp between RMSE and proporton. As Fg. 4(b) shows, when the proporton s less than 5%, the RMSE s larger than. And a larger proporton leads to a smaller RMSE, but also leads to a more complex computaton, whch s shown n Fg. 4(a). We set the proporton as 5% so that we can get regstraton results wth errors less than a pxel n a relatvely short tme. 3) Maxmum Generaton In ths experment, downsampng rate s set as :4 and the proporton s set as 5%. Repeat the above experment wth dfferent maxmum generatons, and record the tmes and the RMSEs. The result s shown n Table I. TABLE I TIME AND RMSE VARIES WITH GENERATION Generaton 0 50 00 200 300 Tme/s - 5 9 33 5 RMSE.87.55.26.22 0.895 0.893 As Table I shows, the RMSE converges wth the generaton. And when generaton s over than 200, the RMSE vares barely. As a result, the maxmum generaton s chosen as 200. Ths experment also shows that thanks to the accurate ntalzaton, the optmzaton model converges quckly. From experment ) to 3), the relatvely optmal values of downsamplng rate, proporton and maxmum generaton are determned. Downsamplng rate nfluences the accuracy of the ntalzaton and the tme of feature extracton; proporton nfluences the accuracy of regstraton and the tme of model soluton; the maxmum generaton nfluence the tme of model soluton. These parameters can be set dfferently for dfferent regstraton requrement.
Fg. 5. Regstraton results of the proposed method. (b) s detals of red block place n (a), and (e) s detals of red block place n (d). (c) and (f) are results of cross correlaton method B. Regstraton Result and Comparson wth prevous works In ths experment, mult-temporal regstraton cases are carred out to valdate our method. And we propose the result comparson wth some prevous works. Fg. 5(a) covers where the lattude s 35.44643 and the longtude s 39.694373, and Fg. 5(d) covers where the lattude s 42.66588 and the longtude s 4.78835. Here, downsamplng rate s set as :4; proporton s set as 5%; and maxmum generaton s set as 200. Fg. 5(a) and Fg. 5(d) are results of FAO. The superposton n co-regstered mages s marked by heghtenng the gray value facttously. Fg. 5(b) and Fg. 5(e) are the detals of red block place n Fg. 5(a) and Fg. 5(d), respectvely. Fg. 5(c) and Fg. 5(f), whch cover the same places of Fg. 5(b) and Fg. 5(e), are the results of cross correlaton mothed. TABLE II ACCURACIES AND TIMES OF DIFFERENT METHODS FOR FIG. 5(A) Method FAO SIFT-OCT BF-SIFT Normalzed Cross Correlaton RMSE.0.77.37 5.78 Tme/s 68 8 23 75 TABLE III ACCURACIES AND TIMES OF DIFFERENT METHODS FOR FIG. 5(B) Method FAO SIFT-OCT BF-SIFT Normalzed Cross Correlaton RMSE 0.83.3.6 4.69 Tme/s 65 70 205 622 These regstraton cases cannot be realzed by solvng (4) drectly. But FAO can regster these mult-temporal mages effcently and accurately. The same cases are realzed by SIFT-OCT [3], BF-SIFT [4], and normalzed cross correlaton [0]. Table II shows RMSEs and tmes of dfferent method for Fg. 5(a). And Table III shows RMSEs and tmes of dfferent methods for Fg. 5(b). Table II and III show that the performance of FAO s better than the tradtonal methods, both n accuracy and effcency. FAO uses features to avod convergence problem and reduce the computaton tme. Compared to tradtonal feature-based methods, FAO only extract features from downsampled mages and parts of orgnal mages, whch s the reason that FAO needs less tme. And by solvng area-based optmzaton model, the regstraton result s tuned to be better than results of tradtonal feature-based methods. V. CONCLUSION In ths letter, we ntroduce a novel SAR mage regstraton method, FAO. FAO use features extracted by SIFT-DRS to determne ntalzaton, slce set and regularzaton. From features to factors n area-based optmzaton model, FAO can be regarded as a composton of feature-based method and area-based method. Thanks to feature-based method, area-based model avods convergence problem and becomes computatonally cheap. The result of a feature-based method (SIFT-DRS can be regarded a fast part of feature-based method) s tuned to be better by an optmzng process n area-based methods. And even though t s a composton of two type methods, t s more effcently than tradtonal works. There s stll a problem for the proposed method. The optmal values of parameters n FAO are vared wth the dfferent regstraton applcaton. In ths letter, we set constant values and they work well n these regstraton cases. But a method on tunng the parameters automatcally stll deserves more works n the future. REFERENCES [] Sansost, Eugeno, et al. "Geometrcal SAR mage regstraton." Geoscence and Remote Sensng, IEEE Transactons on 44.0 (2006): 286-2870 [2] Lowe, Davd G. "Dstnctve mage features from scale-nvarant keyponts."internatonal journal of computer vson 60.2 (2004): 9-0. [3] Schwnd, P., et al. "Applcablty of the SIFT operator to geometrc SAR mage regstraton." Internatonal Journal of Remote Sensng 3.8 (200): 959-980. [4] Wang, Shanhu, Hongjan You, and Kun Fu. "BFSIFT: A novel method to fnd feature matches for SAR mage regstraton." Geoscence and Remote Sensng Letters, IEEE 9.4 (202): 649-653. [5] Dellnger, Flora, et al. "SAR-SIFT: A SIFT-lke algorthm for applcatons on SAR mages."geoscence and Remote Sensng Symposum (IGARSS), 202 IEEE Internatonal. IEEE, 202. [6] Wang, Bangsong, et al. "A Unform SIFT-Lke Algorthm for SAR Image Regstraton."Geoscence and Remote Sensng Letters, IEEE 2.7 (205): 426-430. [7] Ztova B, Flusser J. "Image regstraton methods: A survey"image Vson Comput 2(2003): 977-000 [8] Chen, Tanze, Lmn Chen, and Y Su. "A SAR mage regstraton method based on pxel mgraton of edge-pont feature."geoscence and Remote Sensng Letters, IEEE.5 (204): 906-90. [9] Hong, Gang, and Yun Zhang. "Combnaton of feature-based and area-based mage regstraton technque for hgh resoluton remote sensng mage."geoscence and Remote Sensng Symposum, 2007. IGARSS 2007. IEEE Internatonal. IEEE, 2007. [0] Yoo, Jae Chern, and T. H. Han. "Fast Normalzed Cross-correlaton." Crcuts Systems & Sgnal Processng82.2(2009):44-56.