Multiresolution Image Fusion Based on the Wavelet-based Contourlet Transform

Transcription

1 Multiresolutio Image Fusio Based o the Wavelet-based Cotourlet Trasform Lei Tag Istitute of utomated Commad PL Uiversit of Sciece ad Techolog Najig rm Commad College Najig, Chia tttaglei@gmail.com bstract - The wavelet-based cotourlet trasform (WBCT) is a ew directioal trasform. This trasform uses the wavelet trasform ad the directioal filter bak (DFB) to obtai a multiscale ad multidirectio decompositio of image. The wavelet trasform ad the DFB are o-redudat ad perfect recostructio. So the WBCT ca be regarded as a o-redudat versio of the cotourlet trasform. ew image fusio scheme based o the WBCT was preseted. Firstl, the WBCT is used to perform a multiscale ad multidirectio decompositio of each image. The the WBCT coefficiets of fused image are costructed usig multiple operators accordig to differet fusio rules. The experimetal results show that this ew fusio scheme is effective ad the fused images are better tha that of usig the Laplacia pramid trasform, the wavelet trasform ad the cotourlet trasform. Kewords: image fusio wavelet trasform, cotourlet trasform, directioal filter bak, redudac, multiscale ad multidirectio decompositio. 1 Itroductio With the developmet of iformatio techolog ad sesor techolog, ma kids of image sesors are beig widel used. How to deal with the images from differet sesors becomes a ke research field. Image fusio is meat the processig of images about a give regio obtaied from differet sesors b a specific algorithm, so that the resultat image is more reliable, clear ad more itelligible. Image fusio ca take place o pixel-level, feature-level, ad decisio-level. Pixel-level image fusio is the basic for other levels ad multiresolutio image fusio based o multiscale decompositio is a mai research brach i it. I recet ears, ma research achievemets are gotte i multiresolutio image fusio field. The most methods are based o the pramid trasform (such as Laplacia pramid, ratio pramid, gradiet pramid), the wavelet trasform, the cotourlet trasform ad so o. Image fusio based o the pramid trasform decomposes a image from coarse to fie, ad at each level geerates a badpass image. The process is iterated o the coarse Zog-gui Zhao The 8th Research Istitute Chia Electroics Techolog Group Corporatio Najig, Chia tttaglei@gmail.com sigal. This kid of fusio method ca obtai good visual results. But pramid trasform has redudac, the image data after decompositio is icreased ad the fused image is ot the best. The wavelet trasform is widel used i image fusio field ad ma research achievemets are gotte. Wavelets have show their abilit i represetig atural images that cotai smooth areas separated with edges. However, atural images cosist of edges that are smooth curves ad which caot be captured efficietl b the wavelet trasform. For more powerful represetatio of -D image, Mih N. Do ad Marti Vetterli preset a ew sstem of represetatios amed cotourlets, which is a true two dimesioal trasform that ca efficietl represet images cotaiig cotours ad textures [1][][3]. I the cotourlet trasform, the multiscale aalsis ad the multidirectio aalsis are separated. Firstl the Laplacia Pramid trasform is used to do multiscale decompositio secodl the Directioal Filter Bak (DFB) is used to filter the high frequec from each Laplacia Pramid chael. The fused image usig cotourlet-based image fusio method represets better tha usig wavelets [4]. But the cotourlet trasform has a redudac ratio that is less tha 4/3,which comes from the Laplacia pramid trasform. d the multiresolutio structure is ufixed as the directio umber of the DFB is chageable. Based o the cotourlet trasform, Rami Eslami ad Hader Radha preset a ew trasform amed wavelet-based cotourlet trasform (WBCT) [5][6], which is o-redudac ad has a mulitiresolutio structure that is similar to the -D wavelet decompositio. I this paper we preset a ew image fusio method based o the WBCT. I sectio, we itroduce the WBCT. I sectio 3, the fusio scheme ad the fusio rules are give. Experimetal results are preseted to demostrate the performace of the proposed method i sectio 4. Fiall, the coclusio are preseted i sectio 5. The Wavelet-based Cotourlet Trasform Similar to the cotourlet trasform, the WBCT cosists of two filter bad stages. The first stage uses the wavelet trasform [7] to provide multiscale decompositio i

2 cotrast to the Laplacia pramid trasform used i cotourlet trasform. The secod stage of the WBCT uses the DFB [3] to provide agular decompositio. The framework of WBCT is show i Fig.1. We appl DFB with the same umber of directios to LH, HL ad HH bads i a give wavelet level. Startig from the maximum muber of directios o the fiest level of the wavelet trasform, we decrease the muber of directios at ever other dadic scale whe we proceed through the coarser levels. Fig.3 illustrates a schematic plot of the WBCT usig 3 wavelet levels ad directioal levels (8 directios i fiest level). The low frequec part is ot aalzed b the DFB. Fig.1 framework of the WBCT. The image is first decomposed ito subbads b the wavelet trasform ad the each detail image is aalzed b the DFB. I the resolutio j, a image f ( x, L ( R R) ca be decomposed ito image j f ad the detail images 1 3 D j f, D j f, D j f. j f = f x, Φ ( x, Φ ( x), Φ ( ), D D D ( m m 1 f ( x, Ψ m( x, Φ ( x), Ψ ( ), 1 j f = m f ( x, Ψ m( x, Ψ ( x), Φ ( ), j f = m 3 f ( x, Ψ m( x, Ψ ( x), Ψ ( ), 3 j f = m ( Z j f is the lower horizotal ad vertical frequecies of j 1 f, recorded as LL. D 1 f is the horizotal low j frequecies ad vertical high frequecies, recorded as LH. D j f is the horizotal high frequecies ad vertical low 3 frequecies, recorded as HL. D j f is the high frequecies i both horizotal ad vertical directios, recorded as HH. The image j f also ca be decomposed ito four images i the resolutio j +1. s show i Fig., the DFB is desiged to capture the high frequec directioal compoets of images. Therefore, the low frequec compoet is removed before the DFB b the wavelet trasform. m) (1) Fig. Directioal filter bak with 8 frequec bads. Subbads 0-3 correspod to the mostl horizotal directios, while subbads 4-7 correspod to the mostl vertical directios. Fig.3 schematic plot of the WBCT usig 3 dadic wavelet scales ad directios. The top left corer of this plot is the low frequec part. Fig.4 The WBCT coefficiets of the Barbara image. For clear visualizatio the image is ol decomposed ito wavelet levels ad 8-4 directioal subbads. Small coefficiets are show i black while large coefficiets are show i white. Fig.4 show a example of the WBCT coefficiets of the Barbara image. Here for clear visualizatio we ol used wavelet levels ad 8 directioal subbads for fier level, 4 directioal subbads for coarser level. We ca see that most of the large coefficiets are i the HL subbads accord with the characteristic of the Barbara image. Both the wavelet trasform ad the DFB use perfectrecostructio filters, the the WBCT achieves perfect recostructio. The wavelet trasform ad the DFB have o redudac ratio, so the WBCT is a o-redudat trasform cotrast to the cotourlet trasform.

3 3 The Image Fusio Method Based o the WBCT 3.1 Fusio Scheme For each image the backgroud iformatio belogs to the low frequec part, edge ad texture iformatio belogs to the high frequec parts. So we ca use the WBCT to decompose each image ito the low frequec part ad high frequec parts, the differet fusio rules ca be used i each part. The WBCT-based image fusio scheme is show i Fig.5. The image fusio approach is as follows: The two images are geometricall registered to each other. First use the WBCT to decompose the image ad B ito multiscale ad multidirectio as show i Fig.4. The usig differet fusio rules (show i sectio 3.) to fuse the low frequec part ad the high frequec parts. Fiall, the fused image F is recostructed b iverse WBCT. 3. Fusio Rules Some kids of fusio rules are used to fuse the multiscale ad multidirectio images from decompositio. The low frequec part icludes most of the backgroud iformatio the fusio rules is the weighted average operators. Namel, the low frequec coefficiets are weighted averaged. The high frequec parts iclude most of the image detail iformatio (edge ad texture iformatio. The correlatio betwee a pixel ad its eighborig pixels is ofte bigger tha others, so the fusio rules is about the regio i which the pixel is the ceter. The fusio rules computes the regio eerg of ceter pixel ad its eighborig pixels. The expressio of regio eerg is show below: E = ω ( m, f ( m, () ( m, w Here i-k m i+k, j-k j+k, the width of regio is w(w=k+1), ω(m, is the weighted value, f(m, is the pixel gra value. The regio eerg is bigger where the image feature are remarkable, so we compare the regio eerg for each pixel ad choose the pixel s high frequec coefficiet which regio eerg is bigger as the fused image s high frequec coefficiet. I this wa we ca preserve the edge ad texture iformatio of the source image. I this paper we use the regio-based measure fusio operator just based o the above priciple [8]. Whe the differece of a pixel s regio eerg (the regio ca be 3- b-3 widow or 5-b-5 widow) betwee two source image is big, choose the bigger oe s high frequec coefficiet as the fused image s high frequec coefficiet, otherwise weighted average the pixels high frequec coefficiets. It ca be give b: (, E > EB, RB > T F( = B(, E < EB, RB > T ω1 ( + ωb(, RB T (3) Here T is the eerg match degree threshold, ω 1 +ω =1. We defie the eerg match degree R B ( as: ω( m, f ( m, f ( m, ( m, w RB = (4) E + EB Here R B ( value is betwee 0 ad 1. ccordig to this fusio rules, the high frequec coefficiets of the remarkable feature regio ca be saved. Iverse trasform is used to recostruct the fused image so that the edge ad texture iformatio of the source images are preserved ad fused ito the result image. 4 Experimetal Stud ad alsis 4.1 Fusio with Multifocus Images We use two multifocus images to test the performace of this WBCT-based method. The two images are decomposed ito 3 wavelet levels ad directioal subbads, usig 3-b-3 widow. B Fig.5 The image fusio framework based o the WBCT.

4 (a) Source image (b) Source image B So the fused result usig WBCT-based method is the best i the give methods. 4. Fusio with Differet Visual Images We also use two differet visual images to test the performace of this WBCT-based method. The two images are decomposed ito 3 wavelet levels ad directioal subbads, usig 3-b-3 widow. Fig.7(a) is a uderexposure image ad a bright target at the bottom ceter of the image is clear. Fig.7(b) is a overexposure image ad a dark target at the bottom right of the image is clear. Fig.7(c)~(f) are fused image with Laplacia pramid method, wavelet method, cotourlet method ad WBCT-based method. (c) Laplacia pramid method (d) Wavelet method (a) Uderexposure image (b) Overexposure image B (e) Cotourlet method (f) WBCT method Fig.6 Fusio of multifocus images I Fig.6(a)~(b) the left part of source image is clear ad the right part is blurr, the left part of source image B is blurr ad the right part is clear. Fig.6(c)~(f) are fused image with Laplacia pramid method, wavelet method, cotourlet method ad WBCT-based method. These fused images are better tha a oe of the source images ad all the parts are clear. I this paper we use four objective idicators to compare the experimetal results of image fusio usig the four differet methods. These four objective idicators are the sharpess, the etrop, the mea cross etrop (MCE) ad the root cross etrop (RCE)[9]. Objective evaluatio for fusig multifocus images is show i Table 1. Table 1. Objective evaluatio for fusig multifocus images Method Sharpess Etrop MCE RCE Laplacia pramid Wavelet Cotourlet WBCT From the objective idicators give i Table 1 we ca see that the sharpess ad the etrop is the biggest, the MCE ad the RCE is the smallest usig WBCT-based method. (c) Laplacia pramid method (d) Wavelet method (e) Cotourlet method (f) WBCT method Fig.7 Fusio of differet visual images These fused images are better tha a oe of the source images. The bright target ad the dark target i mark zoe are all clear i fused images. Objective evaluatio for fusig differet visual images is show i Table. I Table we ca see that the sharpess ad the etrop is the biggest, the MCE ad the RCE is the smallest usig WBCT-based method. So the fused result usig WBCTbased method is also the best i the give methods.

5 Table. Objective evaluatio for fusig differet visual images Method Sharpess Etrop MCE RCE Laplacia pramid Wavelet Cotourlet WBCT Fusio with Ifrared ad Visual Images We also use oe ifrared image ad oe visual image to test the performace of this ew method. The decompositio ad fusio parameters are same as above experimet. (a) Ifrared image (c) Laplacia pramid method (b) Visual image B (d) Wavelet method (e) Cotourlet method (f) WBCT method Fig.8 Fusio of ifrared ad visual images Fig.8(a) is a ifrared image, the huma target is bright ad clear. I mark zoe there are two small bright tergets. Fig.8(b) is a visual image. Because of the differet reflectivit of light, trees, stoes ad road i backgroud are clearl demarcated. Fig.8(c)~(f) are fused image with Laplacia pramid method, wavelet method, cotourlet method ad WBCT-based method. These fused images are better tha a oe of the source images. The huma target ad the two small tergets are clear. The backgroud has clear visualizatio. Objective evaluatio for fusig ifrared ad visual images is show i Table 3. Table 3. Objective evaluatio for fusig ifrared ad visual images Method Sharpess Etrop MCE RCE Laplacia pramid Wavelet Cotourlet WBCT From the objective idicators give i Table 3 we ca see that the sharpess ad the etrop is the biggest, the MCE ad the RCE is the smallest usig WBCT-based method. I a word, the fused result usig WBCT-based method is the best i the give methods. 5 Coclusio I this paper, we itroduce the WBCT ito image fusio field ad preset a ew image fusio method based o the WBCT. I this method the image is first decomposed ito multiscale ad multidirectio subbads b the WBCT ad the each multiscale ad multidirectio image is fused b the regio-based measure fusio rules. Fiall the fused image is recostructed b iverse trasform. Experimetal results show that this method performs better i preservig the image detail iformatio tha that of the other image fusio methods i multiresolutio image fusio field. t the same time this ew method ma has wide applicatio i ma fields such as medical diagosis, remote sesig, machie visio battlefield surveillace, target trackig ad so o. ckowledgmet This work was partl supported b the 11th Five-Year advaced research topic of PL Geeral rmamets Departmet uder the grat Refereces [1] M. N. Do ad M. Vetterl The Cotourlet Trasform: Efficiet Directioal Multiresolutio Image Represetatio IEEE Trasactios O Image Processig, vol. 14, o. 1, pp , Dec [] M. N. Do ad M. Vetterl Cotourlets, i Beod Wavelets, J.Stoeckler ad G.V.Wellad, Eds. cademic Press, New York, 003. [3] M. N. Do, Directioal Multiresolutio Image Represetatios, Ph.D. Thesis, Departmet of Commuicatio Sstems, Swiss Federal Istitute of Techolog Lausae, November 001. [4] Miao Qi-guag ad Wag Bao-shu, The Cotourlet Trasform for Image Fusio IEEE Iteratioal Workshop o Haptic udio Visual Eviromets ad their pplicatios, 006.

6 [5] Rami Eslami ad Hader Radha, Wavelet-based Cotourlet Codig Usig a SPIHT-like lgorith, IEEE Iteratioal Coferece o Image Processig, Oct, 004. [6] Rami Eslami ad Hader Radha, Wavelet-based Cotourlet Packet Image Codig, Coferece o Iformatio Scieces ad Sstems, The Johs Hopkis Uiversit, March 16-18, 005. [7] S. G. Mallat, Theor for Multiresolutio Sigal Decomposi-tio: the Wavelet Represetatio IEEE Trasactios o Patter alsis ad Machie Itelligece, vol. 11, o. 7, pp , Jul [8] P. J. Burt ad R. J. Koiczsk Ehacemet with applicatio to image fusio Proceedig of the 4th Iteratioal Coferece o Computer Visio pp , [9] C. I. Chag, K. Che J. W. Wag ad M. L. G. lthouse, relative etrop-based approach to image thresholdig, Patter Recogitio vol. 7, o. 9, pp , Lei Tag was bor i Najig, Chia, i He received the B.Eg. degree from the Istitute of Commuicatio Egieerig i 000 ad M.Eg. degree from the PL Uiversit of Sciece ad Techolog i 004. He is curretl studig i Istitute of utomated Commad, PL Uiversit of Sciece ad Techolog as a Ph.D. cadidate. His research iterests iclude data fusio image fusio ad image processig. Zog-gui Zhao was bor i Haicheg, Chia, i He is a Research Professor of the 8th Research Istitute of Chia Electroics Techolog Group Corporatio. He is a fellow of the CIE. His research iterests iclude data fusio ad automated commad sstem.