Contourlet-Based Image Fusion using Information Measures

Transcription

1 Proceedngs of the 2nd WSEAS Internatonal Symposum on WAVELETS THEORY & APPLICATIONS n Appled Mathematcs, Sgnal Processng & Modern Scence (WAV '08), Istanbul, Turkey, May 2730, 2008 ContourletBased Image Fuson usng Informaton Measures Shrn Mahmoud Islamc Azad unversty, Islamshahr Branch Abstract: The fuson of hghspectral but low spatal resoluton multspectral and lowspectral but hgh spatal resoluton panchromatc satellte mages s a very useful technque n varous remote sensng applcatons. Recently, some studes showed that waveletbased mage fuson methods provde a hgh qualty spectral content of fused mages. However, the spatal resoluton of waveletbased fused mages s less than the Brovey, HIS, and PCAbased fuson methods. Ths s mostly due to the fact that wavelet transform cannot effcently represent the sngularty of lnear/curve edges contourlet transform whch better represents edges. Snce edges play a fundamental role n mage understandng, one good way to enhance spatal resoluton s to enhance the edges. Wwe have computed an automatc threshold for dscardng weak edges of panchromatc satellte mages usng an nformaton measure. Strong edges were then fused to form relable multspectral mages. We have tested the proposed approach on IRS satellte mages and compared our results wth other exstng methods. Then, the pan mage s extended to have the same average and varance as the frst prncpal component. The frst prncpal component s then substtuted by the pan mage. At last, the nverse transform s appled on the resulted mage [3]. Intensty Hue Saturaton Fuson Statstcal methods Arthmetc methods Multresoluton transform Keywords: Image fuson, Contourlet transform, Informaton measure. Brovey transform Bayesan dstrbuton Weghted averagng Intensty modulaton 1 Introducton There are many dfferent methods whch can be used n fuson processes. These methods am at producng one mage whch contans the mportant features of nput mages. In other words, as the panchromatc satellte mage has a hgh spatal resoluton and the multspectral satellte mages have hgh spectral resolutons, the mage fuson algorthm can produce one mage whch contans hgh spectral and spatal resolutons. These methods are grouped n Fgure 1. Intenstyhuesaturaton method s the oldest method used n mage fuson. It performs n RGB doman. The RGB nput mage s then transformed to IHS doman. The pan hstogram and the ntensty component of multspectral mage are then matched and the ntensty component s substtuted by the pan mage. Inverse IHS transform s used to convert the mage to RGB doman [1]. Brovey transform s based on the chromacty transform. In the frst step, the RGB nput mage s normalzed and multpled by the other mage. The resulted mage s then added to the ntensty component of the RGB nput mage [2]. Prncpal component analyssbased mage fuson methods are too smlar to IHS methods, wthout any lmtaton n the number of fused bands. At frst, the multspectral mages are decomposed nto the same number of prncpal components. Prncpal component analyss pass flters Gaussan dstrbuton Hdden Markov model Averagng Nonsubsapmled Fgure 1. Image fuson methods. Segmentbased Waveletbased Curveletbased Contourletbased In pass flters, the hgh frequences components of panchromatc mages are added to the multspectral mages [4]. In Statstcal methods, fuson rules are determned based on some statstcal features. In the Bayesan based approaches, the fuson rule s the maxmum prorty. In other words, the mage value s added to the fused mage wth hgher probablty. In the Gaussanbased approaches, the dstrbuton of each object pxels s consdered to be Gaussan, so the edge pxels wll correspond to two or more dfferent dstrbuton functons. Dfferent objects can be separated usng the sgma flter. The pxels values of each object s produced by averagng and the edge pxels values s taken from panchromatc mage [5]. As n ISBN: ISSN:

2 Proceedngs of the 2nd WSEAS Internatonal Symposum on WAVELETS THEORY & APPLICATIONS n Appled Mathematcs, Sgnal Processng & Modern Scence (WAV '08), Istanbul, Turkey, May 2730, 2008 waveletbased approaches (we revew them next) some edges n the fused mage wll be lost, n [6] the hdden Markov model s used to determne the qualty of pxels presented n the fused mage. Dfferent arthmetc methods have been used n mage fuson algorthms. The smplest one s the averagng method. In that approach, the value of each fused mage pxel s the average of all correspondng pxels of nput mages. In the other averagebased approach, consderng the mportance of mage objects, dfferent weghts are assgned to pxels. Each pxel value s multpled by the assgned weght and used n averagng process. Multresoluton transformbased fuson methods are based on decomposton of fused mages n dfferent levels. Dfferent operatons can be used n decomposton process (such as Laplacan and Gaussan pyramds, wavelet, complex wavelet, curvelet and contourlet transform). In [7,8], Laplacan and Gaussan pyramdbased fuson methods are used to decompose nput mages. The decomposed subbands are then segmented by fuzzy, devaton operators, regon growng, or other segmentaton methods. The resulted segments are compared and weghted and the hgh weghted segment wll contrbute n the fused mage. Waveletbased fuson methods are n general mplemented n 3 man steps [9,10]. In the frst step, the pan and multspectral mages are decomposed nto the same number of levels. In the second phase, the hgh frequences components of pan mages are selected and added to the multspectral mages. At last the resulted mage s reconstructed usng nverse wavelet transform. Complex waveletbased fuson methods are smlar to waveletbased ones. The man dfference s n magnary parts of complex wavelet coeffcents whch offer better drectonal selectvty and shft nvarance propertes [11]. In [12], curveletbased fuson method was proposed. The method s mplemented trough followng steps. 1. R, G and B bands of the pan mage s added to the correspondent bands of multspectral mages. 2. Resulted bands are decomposed usng wavelet transform. 3. resoluton bands are selected and transformed by rdgelet transform. 4. Rdgelet coeffcents are verfed to mprove edges. 5. Fused coeffcents are reconstructed. Spatal nformaton of fused mage s as mportant as the spectral nformaton n many remote sensng applcatons. In partcular, t mproves the effcency of mage fuson process n many applcatons (such as unsupervsed mage classfcaton for change detecton). Contourlet transform [13, 14] s a two dmensonal transform that can capture the ntrnsc geometrcal structures that are key features n vsual nformaton. The dea s to construct a dscrete doman multresoluton and multdrecton expanson usng nonseparable flter banks, n much the same way that wavelets were derved from flter banks. Ths constructon results n a flexble multresoluton, local, and drectonal mage expanson usng contour segments, and thus t s named the contourlet transform (also called pyramdal drectonal flter bank, PDFB). The dscrete contourlet transform has a fast terated flter bank algorthm whose computatonal complexty s O(N) for Npxel mage. Wth parabolc scalng and suffcent drectonal vanshng moments, contourlet can acheve the optmal approxmaton rate for pecewse smooth functons wth dscontnutes along twce contnuously dfferentable curves. In ths paper, a new and effectve contourletbased mage fuson approach s proposed. One of the man advantages of ths approach s that the edge features of panchromatc satellte mages are mproved. The mproved edges are then fused usng multspectral mages. The contourlet transform s adopted because of ts low redundancy and good edge representaton n dfferent drectons. An automatc threshold s computed from decomposed coeffcents and s used for dscardng weak edges. Ths paper s organzed as follows. In Secton 2, two knds of contourlet transform and the prorty of contourlet aganst wavelets and complex wavelet transform are dscussed. In Secton 3, the proposed algorthm s explaned. Expermental results are gven n Secton. Secton 5 concludes the paper. 2 Contourlet Transform 2.1 Defnton Contourlet transform was proposed by Do and Lu n 2003 as a drectonal extenson for wavelets that fxes ths subband mxng problem and mproves ts drectonalty. They ntroduced a double flter bank structure by combnng the Laplacan pyramd wth a drectonal flter bank. Based on a nonredundant checkerboard flter bank, the drectonal extenson for wavelet works wth both crtcallysampled wavelet transform as well as undecmated wavelet transform [15]. Beng crtcally sampled, the contourlet wll not ncrease the redundancy of the overall transform. Although nonseparable n essence was an effcent mplementaton based on 1D operatons, t can be easly generalzed to hgher dmensons. The structure of crtcally sampled contourlet transform s shown n Fg. 2. Ths step can be terated on the lowpass subband for multple decomposton levels. The synthess s gven by synthess part of the checkboard flter banks and the nverse wavelet transform. Nonsubsampled contourlet transform was made smply by turnng the downsampler unts n subsampled contourlet by consderng some ssues about alasng problem [16]. WT LL HL LH Wavelet Decomposton Drectonal Flter Bank Fgure 2: 1level contourlet decomposton. 2.2 Contourlet superorty over wavelet and complex wavelet transform The tradtonal way to construct 2D wavelets s to use tensor products of ther 1D counterparts. The advantage of ths approach s ts smple separable mplementaton. Unfortunately, ths also mposes serous lmts on drectonalty of resultng ISBN: ISSN:

3 Proceedngs of the 2nd WSEAS Internatonal Symposum on WAVELETS THEORY & APPLICATIONS n Appled Mathematcs, Sgnal Processng & Modern Scence (WAV '08), Istanbul, Turkey, May 2730, 2008 frequency parttons. As shown n Fg 3, the 2D wavelet transform produces one lowpass subband (LL), and three hghpass subbands (HL, LH, ), correspondng to the horzontal, vertcal, and dagonal drectons, respectvely. Furthermore, dagonal subbands mx the drectonal nformaton orented at 45 and 135. The man dea of contourlet s to fnd some drectonal extenson to further dvde each hghpass subband nto two branches. In partcular, the desre frequency parttonng n contourlet transform s shown n Fg. 3, whch contans of sx drectonal subbands roughly orented at 15, 45, 75, 105, 135, and 165. Ths s the same frequency decomposton provded by the 2D dualtree complex wavelet transform [16]; whch has ω 2 ( π, π ) ω 2 ( π, π ) HL ( π, π ) LH LL LH HL ω 1 Fgure 3: Dvson of 2D frequency spectrum. Frequency decomposton of wavelet transform.. Frequency decomposton of contourlet transform [16]. been shown to be effcent n several mage processng applcatons. However, the 2D complex wavelet transform s 4 tmes redundant and uses a dfferent flter bank structure compared to wavelets [16]. 3 Proposed Fuson Method Our database ncludes samples from dfferent parts of Bam cty, n Iran. Regstered hgh spatal panchromatc mage and hgh spectral mages were used n the fuson process. The block dagram of proposed method s showed below. Now, consder PI, MI, and DPI() as nput panchromatc mage, nput multspectral mage, and decomposton of panchromatc mage n level, 1<<n, where n maxmum number of decomposton. The algorthm conssts of the followng steps: 1 Interpolaton: as the pan and multspectral mages have dfferent spatal resolutons (5.8m n pan and 23.4 n multspectral mages), n ths step, 1:4 bcubc nterpolaton functon was appled on multspectral mages. Fgure 5, shows the panchromatc and nterpolated verson of multspectral mage. 2 Contourlet Decomposton: n ths step, we decomposed panchromatc mage n n resoluton levels from fne to coarse where each hghpass subband n these levels has 2 n drectons. Fgure 6, shows 3 decomposton levels of pan mage. Contourlet Decomposton ω 1 ( π, π ) Informaton Content Computaton 2 Threshold Computaton Fgure 4: Proposed algorthm. 3 Informaton Content Computaton: n ths step, we computed the nformaton content of panchromatc mage. We dd ths step n two phases. Frst we, splt coeffcents to 256 levels and the then computed ther prorty and nformaton content. In the frst phase, coeffcents at each decomposton subband are grouped n 256 bands; 256 levels for splttng the coeffcents were extracted from grayscale mages. The length of steps n each subband was obtaned by (eq.1) and the contourlet decomposton coeffcents were grouped usng (eq.2). SL = (max( DI) mn( DI)) / 256 (1) coeff DI f (mn( DI ) + ( 1) SL < coeff ) and ( coeff < mn( DI ) + SL ) C = C + 1, coeff S, = 1,2,..., 256 where S and C are the group and the number of members n group S, respectvely. In the second phase, by applyng the C n the prevous phase, the prorty of each band (eq.3) and the nformaton of each band of coeffcents were calculated. Then, the correspondng nformaton matrx to decomposton subbands was obtaned. P( ) = C / NCL, = 1,2,...,256 NCL=Number of coeffcent n subband mage Info ( ) P( ) log( P( ), = 1,2,..., NCL (3) = (4) 4 Computng the Threshold: a thresholdng method was used to dscard weak edges and strengthen strong edges at each subband. It s calculate usng the standard devaton of the nformaton matrx (computed n Step 3) usng (eq.5). TL = mn( Info ( ) + dff ( Info ( )) std ( Info ( )) TH = max( Info ( ) + dff ( Info ( )) std ( Info ( )) where = 1,2,...,256, dff ( x) = max( x) mn( x) (5) (2) Panchromatc Image Dscardng Weak Edges Contourlet Reconstructon ISBN: ISSN:

4 Proceedngs of the 2nd WSEAS Internatonal Symposum on WAVELETS THEORY & APPLICATIONS n Appled Mathematcs, Sgnal Processng & Modern Scence (WAV '08), Istanbul, Turkey, May 2730, Dscardng Weak Edges: the obtaned thresholds are used to elmnate weak coeffcents correspondng to weak edges of the orgnal mage. The threshold was specfed and appled usng (eq.6). = 1, oeff coeff DI, unchanged (6) coeff < TL, coeff = 0 2 < < n, f coeff > TH, coeff = coeff (1 + std( Info( ))) 6 Contourlet Reconstructon: after thresholdng, the changed coeffcents at dfferent levels were used to reconstruct the mage usng the nverse contourlet transform (see Fg. 6). 7 Fuson: after completng Step 4, reconstructed panchromatc mage and multspectral mage were fused usng the add operaton. 4 Expermental Results To evaluate the performance of the proposed method, we compared our results wth the results of two major approaches n fusng satellte mages, namely IHS and wavelet. The proposed algorthm was tested on satellte mages taken wth dfferent resolutons, detals, and contrast. The results shown n Fg. 7 ndcate that our method has been able to overcome the cloud effects n panchromatc and multspectral mages. As Fg. 8 shows, by applyng ths algorthm on these mages, the effects of clouds on panchromatc and multspectral mages taken after a bg earthquake n Bam cty were dscarded and the fused mage s clearer than the orgnal mage. As shown n Fg. 7, our contourletbased fuson method has saved spectral content as good as waveletbased methods and also has provded better spatal content than IHS and waveletbased methods. All revewed methods are compared wth the proposed method and the results are lsted n Table Concluson In ths paper, we proposed a fuson method based on Contourlet transform. The proposed method s compared wth exstng fuson methods lke, IHS, Brovey, PCA, statstcal, pass flters, arthmetc and wavelet, complex wavelet,based methods. Color dstorton, blockng artfact, nose strengthen, texture dstorton, computatonal complexty and run tme measures are used n comparsons. Causng a reasonable complexty and run tme, contourlet based methods could result a hgh spatalhgh spectral fused mage, wthout any color and texture dstorton, blockng artfact and nose strengthen. Acknowledgement Ths work was n part supported by a grant from ITRC. Fgure 5: Sample mages. Panchromatc IRS Bam photo. Interpolated multspectral IRS Bam photo, Fgure 6: Panchromatc mage. Contourlet decomposton of panchromatc mage. Reconstructed panchromatc mage. (c) Fgure 7. Fused mage usng: contourlet, IHS, (c) wavelet. ISBN: ISSN:

5 Proceedngs of the 2nd WSEAS Internatonal Symposum on WAVELETS THEORY & APPLICATIONS n Appled Mathematcs, Sgnal Processng & Modern Scence (WAV '08), Istanbul, Turkey, May 2730, 2008 Fgure 8: Dscardng cloud effects. Orgnal panchromatc mage. Orgnal multspectral mage. (c) Fused mage. Table 1. Performance comparson of fuson methods. Method Intenstyhuesaturaton Brovey transform Prncpal component analyss Statstcal methods pass flters Arthmetc approaches Wavelet transform Complex wavelet approaches Curvelet transform Contourlet transform Color dstorton Blockng artfacts Nose strengthen Texture dstorton Computatonal complexty Run tme 6 References [1]L. Alparone and et. al, Landsat ETM+ and SAR mage fuson based on generalzed ntensty modulaton, IEEE transacton on geoscence and remote sensng, vol. 42, no. 12, Dec [2]A. R. Gllespe and et. al, Color enhancement of hghly correlated magesii. Channes rato and chromatcty transformaton technques, Remote Sensng Envronment., vol. 22, pp , [3]P. S. Chavez and et. al, Comparson of three dfferent methods to merge multresoluton and multspectral data: Landsat TM and SPOT panchromatc, Photogramm Engneerng Remote Sensng., vol. 57, no. 3, pp , [4]S. de Bethune and et. al, Fuson of multspectral and panchromatc mages by local mean and varance matchng flterng technques, Fuson of Earth Data, Jan. 2830, [5] Stennocher, Applcaton of Adaptve flters for multsensoral mage fuson, IEEE geoscnce and remote sensng, vol. 2, pp , [6] Hu and et al, Image fuson for a dgtal camera applcaton based on wavelet doman hdden markov models, Proceedng of the Thrd nternatonal conference on machne learnng and cybernetcs, Shangha, 2629, August [7] A. Toct, Morphologcal pyramd mage decomposton, Pattern recognton letter, pp , [8] P. J. Burt, Gradent pyramd bass for pattern selectve mage fuson, n proceedngs of the socety for nformaton dsplay conference, [9] M. Beauchemn and et. al, A method based on local varance for qualty assessment of multresoluton mage fuson, Photogrammetrc computer vson symposum, [10] J. Zhguo and et. al, A wavelet based algorthm for multfocus mcromage fuson, ISBN: ISSN:

6 Proceedngs of the 2nd WSEAS Internatonal Symposum on WAVELETS THEORY & APPLICATIONS n Appled Mathematcs, Sgnal Processng & Modern Scence (WAV '08), Istanbul, Turkey, May 2730, 2008 IEEE nternatonal conference on mage and graphcs, pp , [11] A. Jalobeanu and et. al, Satellte mage deblurrng usng complex wavelet packets, Internatonal journal of computer vson, vol. 51, no. 3, pp , [12] M. Cho and et. al, The curvelet transform for mage fuson, ISPRS, part B8, pp. 5964, [13] M. N. Do, M. Vetterl, The Contourlet Transform: An Effcent Drectonal Multresoluton Image Representaton, IEEE Transactons On Image Processng, Vol. 14, pp , [14] M. N. Do, M.Vetterl, Contourlets n Proc. Beyond Wavelets, Academc Press, NewYork, pp. 127, [15] Yue. Lu and Mnh. N. Do, A Drectonal Extenson for Multdmensonal Wavelet transforms, IP EDICS: 2WAVP (Wavelets and Multresoluton Processng), Draft, Aprl., [16] R. G. Baranuk, N. Kngsbury, and I. W. Selesnck, The dualtree complex wavelet transform a coherent framework for multscale sgnal and mage processng, IEEE SP Mag., ISBN: ISSN: