GENERALIZED MODEL FOR REMOTELY SENSED DATA PIXEL-LEVEL FUSION

Transcription

1 GENERAIZED MODE FOR REMOTEY SENSED DATA PIXE-EVE FUSION Zhag Jixia, Yag Jighui*, i aitao, Ya Qi Chiese Academy of Surveyig ad Mappig, Beitaipig Road 6, Beijig 0009, P. R. Chia *: Correspodig author. - jhyag@casm.ac.c. Tel: Commissio VII, WG VII/6 KEYWORDS: Remotely Sesed Data, Fusio, Geeralized Model, Implemetatio, PCA ABSTRACT: A geeralized model characterizig most remotely sesed data pixel-level fusio techiques is very importat for theoretical aalysis ad applicatios. This paper focuses o the establishmet of a geeralized model for most data fusio methods, which is helpful to quatitatively aalyze ad quicly implemet differet data fusio techiques. As a example, the PCA fusio method is selected to demostrate the availability of the geeralized model through the geeralized model based implemetatio.. NOMENCATURE : the th bad of the lower resolutio multispectral image; pa : the higher resolutio pachromatic bad; pa : the degraded pachromatic bad; pa A : approximatio coefficiets after level GP Geeralized aplacia Pyramid) or a trous wavelet decompositio; pa D : detail coefficiets after level GP or a trous wavelet decompositio; : the th bad of multispectral image resampled or relatively processed to have same size as the pachromatic bad; : the th bad of the higher resolutio multispectral image after fusio;, : the pixel value of locatio of the bad, : the pixel value of locatio of the bad ; ; : spatial ad textural details of locatio extracted from the pachromatic bad; : the fusio coefficiets modulatig i j, ito 2. INTRODUCTION. So far, may pixel-level fusio methods Carper,990, Shettigara,992, ill, 999, iu,2000, Zhou,998, Rachi,200) for remote sesig image have bee presesed where the multispectral image s spatial details are ehaced by adoptig the higher resolutio pachromatic image correspodig to the lower resolutio multispectral image. Therefore, the mai priciple of remote sesig data fusio focuses o the maximum ehacemet of its spatial details o the coditio of miimizig distortio of multispectral image s spectral characteristics. Whe correlatio betwee the multispectral ad pachromatic images is ot high, it is ofte a mutual cotradictio betwee maiteace of spectral characteristics ad ehacemet of spatial details. Thus the choice of fusio algorithm is determied to emphasize spectral features or spatial details accordig to a specific applicatio. Typical algorithms of remote sesig data fusio ca be divided ito three geeral categories Zhag ad Yag,2006): compoet substitutio fusio techique Chavez,99, Carper,990, Shettigara,992, ill,999), modulatio-based fusio techique Chavez,99, Vrabel, 2000, iu, 2000, Zhag ad Yag,2006) ad multi-scale aalysis based fusio techique Zhou,998, Rachi,200, N u ez,999, Pradha,2006, Aiazz2002). The typical algorithms of compoet substitutio fusio techique are IS trasform fusio algorithm Carper, 990), PCA trasform fusio algorithm Shettigara,992), CM ocal Correlatio Modelig) fusio algorithm ill,999) ad RVS Regressio Variable Substitute) fusio algorithm Shettigara,992); the fusio algorithms of the modulatio-based techique iclude Brovey trasform fusio algorithm Vrabel,2000), SFIM Smoothig Filter Based Itesity Moulatio) fusio algorithm iu,2000) ad high pass filter fusio algorithm Chavez,99); the fusio algorithms based o the multi-scale aalysis maily iclude wavelet decompositio based fusio techique Zhou,998, Rachi,200, N u ez, 999, Pradha,2006) ad aplacia pyramid decompositio based fusio techique Aiazz2002). Whe various fusio algorithms are studied, a issuse whether these algorithms ca be described by a geeralized mathematical model Tu,200, Wag,2005) is igored. The model ca reflect the mai features of the fusio process by a simple mathematical formula. The establishmet of a geeralized model will cotribute to relatively theoretical aalysis ad fusio algorithm desig i the light of a specific applicatio. Also the model is beeficial to qualitative ad quatitative aalysis of fusio techology from differet aspects. The most importat aspect is that the establishmet of a geeralized model will reveal that differet fusio techique 05

2 The Iteratioal Archives of the Photogrammetry, Remote Sesig ad Spatial Iformatio Scieces. Vol. XXXVII. Part B7. Beijig 2008 rely o the differece of the calculatioal mehtod of the mathematical model parameters. Thus calulatig model parameters correspodig to the fusio method is the mai tas whe implemetatig the method. Compared to Wag s wor Wag,2005), the paper maily cocetrates o two aspects. First, the paper presets a geeralized model for remotely sesed data pixel-level fusio, which has a wide rage of applicability. The various commoly used remote sesig data fusio algorithms ca be deduced to the geeralized model. Secod, the implemetatio techique base o the geeralized model oly calculates the model parameters impactig the last fusio results ad discards the processig steps ot affectig the fusio results, savig computatioal time.. TE GENERAIZED MODE Accordig to the imagig mechaism ad the ideal pa-sharpeig results of multispectral image, the preseted geeralized model is formulated by, = +,,, : Spatial ad textural details extracted from the pachromatic bad by a certai calculatio. : The coefficiets modulatig i j, ito. The preseted model expressed by equatio ) ca clearly describe the mathematical relatioships amog the origial multispectral image, the spatial details extracted from the high-resolutio pachromatic image, ad the adopted fusio strategy. I aother word, the spatial ad textural features extracted from the pachromatic bad are imported ito the multispectral image i terms of the fusio coefficiets ad the fusio result is the image whose features are ehaced by the pachromatic image. The fusio operatios are fulfiled pixel by pixel, bad by bad after calculatio of i j ad, but i the course of calculatio of i j ad, ot oly the pixel value of locatio of the lower resolutio multispectral th bad ad the pachromatic bad but also the whole statistical iformatio ad eighbor pixels of loacatio are used. the methods calculatig parameters i j iclude: ) the liear combiatio method: obtaiig the x y after subtractig multispectral bads liear combiatio from the pachromatic bad, such as IS, PCA, RVS, Brovey, Bloc-regressio Zhag ad Yag,2006); 2) filter method ad multi-scale aalysis method: obtaiig the x, y) after subtractig its filtered or multi-level decompositio results from the pachromatic bad, such as SFIM, CM,A trous N u ez,999), GP Aiazz2002), ARSIS method Rachi,200). ), x, y) is determied by followig factors: the pachromatic ad multispectral relative spectral respose, spectral rage of the pachromatic ad multispectral bads, the GIFOVGroud projected Istataeous Field Of View) of pachromatic ad multispectral bads, the ladscape properties ad lad cover classes, radiometric calibratio method of differet sesors, the temporal properties, the correlatio betwee the pachromatic ad multispectral bads, the average value,variace ad other statictical characteristics of the multispectral ad pachromatic bads., x, y) The methods calculatig parameters iclude: ) Costat Value, such as IS,PCA, RVS, A trous, CM; 2) Spectral Distortio Miimum: such as SFIM,Brovey, A trous, Bloc-regressio; ) Cotext-based Decisio CBD),such as GP,ARSIS. The model formulated by equatio ) is more comprehesive ad applicable tha Wag s model. The fusio coefficiets of wag s model are limited to the cases of costat value ad spectral distortio miimum, ad ca ot describe the fusio coefficiets for CM, ARSIS ad GP fusio algorithms. For the method extractig spatial ad textural details, Wag s model iclude the filter ad liear combiatio mehtods while the geeralized mode proposed i this paper supports the additioal methods used i CM ad GP fusio algorithms. I a word, the geeralized model ca characterize most of commoly used remote sesig data fusio algorithms icludig ot oly the IS,PCA,A trous,brovey,pf algorithms but also RVS, GP, CM, ARSIS, wavelet decompositio plus PCA trasform, wavelet decompositio plus IS trasform, ad the authors proposed Bloc-regressio. 4. DEDUCTION FOR COMMONY USED FUSION AGORITMS I this sectio, three categories of fusio algorithms metioed i sectio are deduced to the geeralized model, i.e. the proposed equatio ), through the mathematical trasformatio. Through the deductio, the coclusio ca be draw that differet fusio techique rely o the differece of the calculatio of parameters, ad. 4. Compoet Substitutio Fusio Techique The typical algorithms applyig compoet substitutio fusio techique iclude IS, PCA, CM ad RVS fusio algorithms. To illustrate the deductio for this techique, followig is the trasformatio steps taig PCA fusio algorithm as a example. The lower resolutio multispectral bad is resampled to have the same size as the higher resolutio pachromatic bad pa ) after those bads are co-registrated: = rsp, ad after the resamplig the implemetatiom steps for PCA fusio algorithm are as follows Shettigara,992): ) Calculatig the correlatio matrix of the lower resolutio multispectral bads, is equal to 4; 052

3 The Iteratioal Archives of the Photogrammetry, Remote Sesig ad Spatial Iformatio Scieces. Vol. XXXVII. Part B7. Beijig ) Calculatig the eigevalues ad eigevectors accordig to the correlatio matrix; 6) Replacig the first pricipal compoet by the higher resolutio bad; ) Sortig the eigevalues ad eigevectors; 4) Calculatig the pricipal compoets oe by oe accordig the PCA trasform; pc = Φ 2) pa = pc +, 7) Obtaiig the fusio results after iverse PCA trasform ) 5) Selectig the first pricipal compoet; = Ω ' pc,where Ω = Φ 4) = = pc = pa 2 4 pc pc pc2 = 4 pc 2 4 pc 4 44 pc pc pc = ) + pc pc pc pc 2 24 pc2 + 2 = pc 4 44 pc pc 24 pc 4 pc ) Thus, the whole algorithm maily cosists of calculatig the correlatio matrix, forward PCA trasform ad iverse PCA trasform. Through deductio the fial fusio results are pa = c + c2 2 + c + c ) = Modulatio-based Fusio Techique, where = pa pc 6) The typical algorithms applyig the modulatio-based fusio techique iclude Brovey, SFIM ad PF fusio algorithms. To illustrate the deductio for the modulatio-based fusio techique, followig is the trasformatio steps taig Bloc-regressio fusio algorithm preseted by the authors as a example. The lower resolutio multispectral bad is resampled to have the same size as the higher resolutio pachromatic bad pa after those bads are co-registrated: = rsp ), ad after the resamplig the implemetatiom steps for Bloc-regressio based fusio algorithm are as follows Zhag ad Yag,2006): ) Obtaiig liear regressio coefficiets through multiple liear regressio betwee the blocs from the pachromatic bad ad from the multispectral bads. is equal to 4; 2) Computig the liear combiatio of blocs from multispectral bads i terms of coefficiets. sy = c + c2 2 + c + c4 4 ) Fiishig the fusio operatio for every bloc usig the followig expressio: 2 2 pa 2 sy + 2 = = = + ) sy sy sy sy 2 2 = + = sy sy sy sy 9) 8) 05

4 The Iteratioal Archives of the Photogrammetry, Remote Sesig ad Spatial Iformatio Scieces. Vol. XXXVII. Part B7. Beijig 2008 Thus, the Bloc-regressio based fusio algorithm ca be modeled by the followig simple formula: σ mi,, pa = + σ 0, if ρ if ρ, θ < θ 2), =, + c, 0) = pa c ) Where + c2 2 + c + c4 4, c is the regressio coefficiets of the bloc icludig pixel i,. 4. Multi-scale Aalysis based Fusio Techique Multi-scale aalysis based fusio techique adopts multi-scale decompositio methods such as multi-scale wavelet Zhou,998, Rachi,200, N u ez,999, Pradha,2006), aplacia pyramid Aiazz2002) to decomposize multispectral ad pachromatic images with differet levels, ad the derives spatial details which are imported ito fier scales of the multispectral images i the light of the relatioship betwee the pachromatic ad multispectral images i coarser scales Rachi,200, Aiazz2002, Garzell2005), resultig i ehacemet of spatial details. The typical fusio algorithms based o multi-scale aalysis iclude a trous fusio algorithm which adopts traslatio ivariat ad udecimated wavelet trasform ad aplacia pyramid decompositio based fusio algorithm. Some fusio methods combiig compoet substitutio ad wavelet decompositio are recetly preseted, such as methods combiig wavelet decompositio ad PCA trasform or IS trasform. To illustrate the deductio for the multi-scale aalysis based fusio techique, followig is the trasformatio steps taig ARSIS as a example. The lower resolutio multispectral bad is resampled to have the same size as the higher resolutio pachromatic bad pa after those bads are co-registrated: = rsp ), ad after the resamplig the implemetatiom steps for ARSIS fusio algorithm are as follows Rachi,200): pa ) Obtaiig A, which is approximatio coefficiets after level GP or a trous, or UDWTUdecimated Discrete Wavelet Trasform) decompositio of the pachromatic bad = pa pa A = pa pa. 2) Fiishig the fusio operatio for every pixel usig the followig expressio: D where: ρ, ) is the correlatio coefficiet betwee the s N x N eibors 9x9 for IKONOS,7x7 for SPOT pa A -4) ad s relative pixel widow. θ 2) is the threshold value for the th bad, ragig from 0. ~ 0.6, which depeds o whole correlatio betwee the multispectral bad ad the pachromatic bad. The smaller the correlatio, the bigger the threshold value is. σ j ), ) is stadard deviatio of the s N x N pa σ i, eibors, ad pa A is stadard deviatio of the s N x N eibors. is also determided by Rachi-Wald-Magolii RWM) Rachi, 200). 5. PCA FUSION IMPEMENTATION METOD BASED ON TE GENERAIZED MODE To demostrate availability of the geeralized model, the geeralized model based implemetatio ad experimets are give as well as compariso with the regular implemetatio taig PCA fusio techique as a example. The regular PCA fusio implemetatio maily cosists of three steps. Firstly, the multispectral bads data is forward trasformed ito a ew data space; secodly, the priciple compoet of ew data space is substituted by the higher resolutio bad, i.e., the pachromatic bad; lastly, the data space after replacemet is iversely trasformed ito the origial space. I geeral, the regular implemetatio operatios iclude calculatio of trasformatio matrix, forward trasformatio ad iverse trasformatio. Through the above mathmatical trasformatio i the sectio 4., the PCA fusio techique ca be deduced to the form of equatio ). Thus, the geeralized model based implemetatio for this fusio techique ca be fulfilled by equatio ) pixel by pixel. = + ) = pa pc 4) j ) j ) = + ) pc = ϕ + ϕ2 2 + ϕ + ϕ4 4 5) Cotext-based decisio CBD) model Rachi,200): ϕ The parameters ad are elemets of forward ad iverse trasform matrix, respectively. Compared to the regular implemetatio, the ew implemetatio does ot require the 054

5 The Iteratioal Archives of the Photogrammetry, Remote Sesig ad Spatial Iformatio Scieces. Vol. XXXVII. Part B7. Beijig 2008 forward ad iverse trasformatio ad just cocetrates o the calculatio of ad i j, decreasig computatioal requiremets. a) Origial multispectral,true color composite 44 x 4 b) PCA fusio results,76 x 649 c) a slice of regular implemetatio d) a slice of geeralized model based implemetatio Fig. Fusio results of the regular implemetatio ad the geeralized model based implemetatio The experimetal data cosists of a slice of IKONOS pachromatic bad with 76 x 649 pixels, ad the coorespodig multispectral image with 44 x 4 pixels icludig B, G, R, NIR bads. We write Matlab programs to perform the fusio operatios usig the regular method ad geeralized model based method, respectively, i the same hardware ad software platform. The experimetal results show that the rutime ot icludig resamplig time of the multispectral image, iput ad output time for the ew implemetatio is.098s, while the time for the regular implemetatio is.906s, savig s or 2.5%. At the same time, the pixel value of the two types of fusio results, show i fig. c) for regular implemetatio ad fig. d) for the geeralized model based implemetatio, is the same. Fig. a) ad fig. b) are the origial multispectral image ad PCA fusio results, respectively. 6. CONCUSIONS This paper presets the geeralized model for remotely sesed data pixel-level fusio, which ca clearly describe the relatioships amog the origial multispectral image, the spatial details extracted from the high-resolutio pachromatic image, ad the adopted fusio strategy by meas of mathematical expressio. The geeralized model ca characterize most commoly used remote sesig data fusio algorithms ad these algorithms ca be deduced to the geeralized model through mathematical trasformatio. Therefore, the establishmet of the geeralized model will cotribute to desig of fusio algorithms whose pricipal cotradictios cocetrate o the methods extractig spatial details from high-resolutio pachromatic images ad the strategies addig the spatial details to multispectral images. At the same time, the proposed remotely sesed data fusio algorithms ca bee theoretically aalysized ad traslated ito the framewor of the geeralized model after estabishig the model. It proves that the implemetatio techique based o the geeralized model have the advatages reducig the computatioal requiremets ad havig uiversal applicabilities by meas of the experimetal compariso ad mathematical trasformatio.the implemetatio techique ca be applied to most of remote sesig data fusio algorithms, i particular for the data fusio algorithms demadig huge calculatios, the decreased operatios is large. 055

6 The Iteratioal Archives of the Photogrammetry, Remote Sesig ad Spatial Iformatio Scieces. Vol. XXXVII. Part B7. Beijig 2008 REFERENCES A. Garzell F. Neci2005. Iterbad structure modelig for Pa-sharpeig of very high-resolutio multispectral images. Iformatio Fusio, vol.6, pp.2 ~ 224. B. Aiazz. Alparoe, S. Barot A. Garzell2002. Cotext-drive fusio of high spatial ad spectral resolutio images based o oversampled multiresolutio aalysis. IEEE Trasactios o Geosciece ad Remote Sesig, vol.40, pp.200 ~ 22. J. G. iu, Smoothig Filter-based Itesity Modulatio: a spectral preserve image fusio techique for improvig spatial details. Iteratioal Joural of Remote Sesig, vol.2, pp.46 ~ 472. J. ill, C. Diemer, O. Stöver, T. Udelhove,999. A ocal Correlatio Approach for the Fusio of Remote Sesig Data with Differet Spatial Resolutios i Forestry Applicatios. Iteratioal Archives of Photogrammetry ad Remote Sesig, Valladolid, Spai, vol. 2, Part 7-4- W6. T. Rachi, B. Aiazz. Alparoe, S. Barot ad.wald,200. Image fusio the ARSIS cocept ad some successful implemetatio schemes. ISPRS Joural of Photogrammetry ad Remote Sesig, vol.58, pp.4-8. V. K. Shettigara,992. A geeralized compoet substitutio techique for spatial ehacemet of multispectral images usig a higher resolutio data set. Photogramm. Eg. Remote Ses., vol. 58, pp Z. J. Wag, Z. Djemel, C. Armeais, D. R. ad Q. Q A Comparative Aalysis of Image Fusio Methods. IEEE Trasactios o Geosciece ad Remote Sesig, vol.4, pp.9 ~ 402. ACKNOWEDGEMENTS This wor was supported by the Major State Basic Research Developmet Program of Chia 97 Program) uder Grat No. 2006CB700 ad 86 Program uder Grat No.2007AA2Z5. J. N u ez, X. Otazu, O. Fors, A. Prades, V. c Pal`a, ad R.Arbiol,999. Multiresolutio-Based Image Fusio with Additive Wavelet Decompositio. IEEE Trasactios o Geosciece ad Remote Sesig, vol.7, pp.204 ~ 2. J. Vrabel,2000. Multispectral imagery advaced bad sharpeig study. Photogramm. Eg. Remote Ses., vol.66, pp. 7 ~ 79. J. W. Carper, T. M. illesad, ad R. W. Kiefer,990. The use of itesity hue saturatio trasformatios for mergig SPOT pachromatic ad multispectral image data. Photogramm. Eg. Remote Ses., vol.56, pp J. X. Zhag, J.. Yag, Z. Zhao,2006. Bloc-regressio based Fusio of Optical ad SAR Imagery for Feature Ehacemet. Pleary Presetatio, ISPRS Mid-term Symposium Remote Sesig: From Pixels to Processes. ITC, the Netherlads. J. X. Zhag, J.. Yag, Z. Zhao,. T. ad Y.. Zhag. Bloc-regressio based Fusio of Optical ad SAR Imagery for Feature Ehacemet. Iteratioal Joural of Remote Sesig, to be published. J. Zhou, D.. Civco, ad J. A Silader,998. A Wavelet Trasform Method to Merge adsat TM ad SPOT Pachromatic Data. Iteratioal Joural of Remote Sesig, vol.9, pp.74 ~ 757. P. S. Pradha, R.. Kig, N.. Youa, ad D. W. olcomb,2006. Estimatio of the Number of Decompositio evels for a Wavelet-Based Multiresolutio Multisesor Image Fusio. IEEE Trasactios o Geosciece ad Remote Sesig, vol. 44, pp.674 ~ 686. S. Chavez, C. Sides, ad A. Aderso,99.Compariso of three differet methods to merge multiresolutio ad multispectral data: adsat TM ad SPOT pachromatic. Photogramm. Eg. Remote Ses., vol.57, pp. 295 ~ 0. T. M. Tu, S. C. Su,. C. Shyu ad P. S. uag,200. A ew loo at IS-lie image fusio methods. Iformatio Fusio, vol.2, pp.77 ~