A REGION-BASED TECHNIQUE FOR FUSION OF HIGH RESOLUTION IMAGES USING MEAN SHIFT SEGMENTATION

Transcription

1 REGION-BSED TECHNIQUE FOR FUSION OF HIGH RESOLUTION IMGES USING MEN SHIFT SEGMENTTION Li Shuang a, Li Zhilin a, b a LIESMRS, Wuhan Univeit, P.R. China - lihuang9@gmail.com b Dept. of Land Suveing and Geo-Infomatic, The Hong Kong Poltechnic Univeit, Hong Kong - lzlli@polu.edu.hk Commiion VII, WG VII/6 KEY WORDS: Image poceing, Shapening, Image undetanding, Fuion, Geogaph BSTRCT: Thi pape decibe a egion-baed technique fo fuion of high-eolution image. In thi technique, mean hift egmentation i adopted to extact the featue fo high eolution image a a ubtitution of othe egmentation method (e.g. Cann opeato) and Stuctue Similait Index Metic (SSIM) i ued to meaue the egion imilait. Expeiment on IKONOS image ae caied out to compae the eult obtained fom thi new technique and thoe with Cann egmentation. It ha been found that the eult obtained thi new technique i much bette than the conventional one in tem of patial hapne and pectal eevation.. INTRODUCTION Image fuion i a poce to combine two o moe diffeent image to fom a new image b uing cetain algoithm (Phol and Gendeen,998). It take place at thee level: pixel, featue and deciion. Image fuion at pixel-level i the lowet poceing level conideing individual pixel o aociated local neighbouhood of pixel fo fuion deciion. In the pat decade, lage numbe of pixel level image fuion method i popoed, e.g. Bove, Intenit-Hue-Satuation, Pinciple Component nali, Wavelet tanfom, etc (Zhen and He,004). Howeve, uch method often intoduce colou ditotion and/o block effect to high eolution image fuion. Thi i becaue a pixel i onl a baic unit of infomation with no emantic ignificance. t the featue level, featue fom the input image will be fit extacted (e.g. uing egmentation pocedue); and then fuion of thee featue will be opeated b ome ule. Compaing with pixel level image fuion, featue-level method i moe meaningful. Becaue it can full exploe the chaacteitic of featue to guide the image fuion poce, uch a egion activit level, egion imilait match meaue and o on. Moe ecentl, a numbe of egion-baed featue-level image fuion technique have been popoed (Zhang,997; Piella,003; Lewi,007). Thee technique fit tanfom the ouce image and B to multi-cale epeentation b wavelet tanfom; egmentation i caied out on the ouce image to get egion epeentation of both image. Then b ovelaing the two egion epeentation, a haed egion epeentation fo thee two image i obtained. nd egion activit level and imilait match meauement ae calculated fom each egion to guide the fuion poce. Duing the whole poce, egmentation i the mot impotant pat becaue it diectl influence the effect of fuion eult. Peviou wok emplo the watehed egmentation o Cann edge detection method and the eult fo fuion of high-eolution image ae not ve good. Theefoe, thi tud aim to develop a new technique fo fuion of high-eolution image. It ha been found (e.g. Mo et al,006) that the mean hift egmentation i moe uitable fo the egmentation of high eolution image and thu will be adopted in thi tud. Moeove, the Stuctue Similait Index Metic (SSIM) popoed b Wang,(00) fo image qualit aement will be ued (intead of egion match meaue which i commonl ued) to guide the fuion poce. Section eview the egion baed fuion. Mean hift egmentation fo featue extaction i intoduced in ection 3. SSIM ued fo fuion deciion making i decibed in ection 4. Section 5 decibe the evaluation of the popoed method and concluion ae made in ection 6.. REGION BSED FUSION: N OVERVIEW ND PROPOSL The concept of egion baed fuion wa fit intoduced b Zhang et al.,(997) and developed b Piella et al.,(003). Piella geneic egion baed image fuion famewok i hown in Figue. 67

2 The Intenational chive of the Photogammet, Remote Sening and Spatial Infomation Science. Vol. XXXVII. Pat B7. Beijing 008 Image Image B Ue of mean hift egmentation to ubtitute Cann egmentation; Wavelet tanfom Segmentation Match Wavelet tanfom ue of the oiginal input image to get the bina image of haed egion and then map of the haed egion image to each level b down-ampling to enue the conitenc of egmentation at each level; and ctivit ctivit ue of Stuctue Similait Index Metic (SSIM) popoed b Wang,(00, 004) to guide the fuion poce intead of egion match meaue becaue SSIM ha moe phical meaning. Deciion Combination Invee wavelet tanfom Fued image Figue. Geneic famewok of egion baed image fuion (Piella,003 ) Fit, the ouce image ae decompoed b wavelet to get the appoximate and detailed ub-image; and then egmentation i caied fo thee ub-image to get the egion of each level. Thee egion ae ued to guide fuion poce. The activit level and match degee meaue of the wavelet coefficient of ouce image ae computed in thee egion; and the maximum value ule and the weighted aveage ule ae epectivel ued to combine the coefficient of detailed ub-image and appoximate ub-image. t lat, the combination coefficient ae inveel tanfomed b wavelet to obtain the final fuion image. The choice of egmentation i vitall impotant becaue it diectl influence the fuion deciion. n appopiate egmentation will give ueful infomation to image fuion, while an inappopiate egmentation will povide mileading infomation to guide the fuion poce. Cuentl, the popula image egmentation method ued in the egion baed image fuion famewok (Zhang,997; Piella,003; Wang,005; Lewi,005) ae c-mean cluteing, watehed algoithm, and Cann edge detection method. But thee egmentation method can be ubtituted b othe. The election of appopiate egmentation method i the fit iue to be conideed. Moeove, in the taditional egion-baed fuion famewok, the effect of egmenting ub-image will be moe eiou than that of egmenting oiginal image becaue ub-image contain lee infomation a the numbe of decompoition level inceae. Thee ma be inaccuac in egmented egion at each level no matte what egmentation method ae ued. When inveel tanfomed b wavelet, the inaccuac will inceae level b level. To educe the inaccuac i the econd iue to be conideed. Fomalization of appopiate ule to guide the fuion poce i the thid iue to be conideed. To develop a moe obut technique fo the fuion of higheolution, in thi tud, the following tateg i adopted: 3. MEN SHIFT SEGMENTTION FOR EXTRCTION OF FETURES FROM HIGH-RESOLUTION IMGES Mean hift anali i a newl developed nonpaametic cluteing technique baed on denit etimation fo the anali of complex featue pace. It ha found man ucceful application uch a image egmentation and tacking (Comaniciu,999; Luo,003). The mean hift pocedue i an adaptive local teepet gadient acent method. The mean hift vecto i computed b the following fomula: m G f, K h c h f () G Whee the ubcipt G and K ae kenel, thei coeponding pofile atif ; i the denit gadient etimato of kenel ' g( x) k f K K ; f G i the pobabilit denit of new kenel G ; h i the bandwidth and c i a contant; x i the cente of kenel(window). It indicate that, at location the mean hift vecto computed with kenel G i popotional to the nomalized denit gadient etimate obtained with kenel K. Theefoe, to get the diection of f h, K, onl the vecto m h, G hould be calculated. The mean hift vecto thu alwa point towad the diection of maximum inceae in the denit. The mean hift pocedue i achieved b a -tep iteation: ) Compute the mean hift vecto m G, ) Tanlate the kenel (window) G (x) b m h, G until convegence. Since the contol paamete ha clea phical meaning, both ga level and colo image ae poceed b the ame algoithm. n image i tpicall epeented a a -D lattice of p-dimenional vecto (pixel). When p, it denote ge image. When p3, it denote colo image. When p>3, it denote multi-pectal image. The pace of lattice i known a the 68

3 The Intenational chive of the Photogammet, Remote Sening and Spatial Infomation Science. Vol. XXXVII. Pat B7. Beijing 008 patial domain, while the ga level and multi-pectal infomation ae epeented in the colo domain. When the location and colo vecto ae concatenated in the joint patial- SSIM (μ μb + C)(σ B + C) ( μ + μ + C )( σ + σ + C ) B B (7) colo domain of dimenion dp+, thu, the multivaiate kenel i defined a: K h, h C h h p X k h X k h () Whee X i the patial pat and X the colo pat of the featue vecto; i the common pofile ued in both of the two domain; and ae the kenel bandwidth; and C i the coeponding nomalization contant. The qualit of egmentation i contolled b the patial domain and colo h domain. k(x) h h h μ, μ whee B ae the mean value of egion in image and B; andσ, σ B ae the vaiance of image and B. σ B i the covaiance of and B. ae two contant. C,C The SSIM index ma be illutated geometicall in a vecto pace of image component. Thee image component can be eithe pixel intenitie o othe extacted featue. It i moe meaningful to compae the imilait of two egion than to compae egion match meaue. nothe paamete -- the egion activit level -- i defined a follow (Piella, 003): a c(i, (8) N c 4. STRUCTURE SIMILRITY INDEX METRIC (SSIM) FOR FUSION DECISION MKING When the egion ae obtained b the ovelaing poce, the egion tuctue imilait index metic ae computed to guide the wavelet coefficient fuion. Wang,(00) fitl popoed a Univeal Image Qualit Index (UIQI) which achieve atifacto eult fo aeing compeed image qualit. ftewad, Wang,(004) impoved on it and named it Stuctue Similait Index Metic (SSIM) which i a global metic to meaue the imilait of two image. In thi pape, we ue the SSIM to calculate the imilait of coeponding egion in two input image. It i defined a follow: α β γ [ l( ] [ c( ] [ ( ) SSIM ( ] (3) The luminance, contat and tuctue compaion meaue wee given a follow: μ xμ + C l( μ + μ + C x x σ xσ + C c( σ + σ + C ( σ x + C3 σ σ + C, C, C3 (3), the SSIM index i given b x C ae contant. When α β γ 3 (4) (5) (6) in fomula Whee N denote the pixel numbe in egion and c denote the coeponding wavelet coefficient at location. Fo each egion, the coefficient ae fued accoding to the SSIM. If SSIM i le than a theholdα, we will pefom election; and othewie we will pefom aveaging. if a ab SSIM α cf cb if a < ab SSIM α + cb, SSIM > α Fo each edge, the fuion ule i a follow: (9), onl if c i at an edge cf cb, onl if cb i at an edge (0) + cb both ae at edge Once the compoite coefficient ae obtained, the fued image can be poduced b invee wavelet pocedue. 5. EVLUTION OF PROPOSED METHODS To evaluate the popoed method, two et of image ae ued. One i the built-up aea and the othe i ual aea. Both ae fom IKONOS- eno. The dimenion of Panchomatic (Pan) and Multi-Spectal (MS) image ae 5*5 and8*8, epectivel. The popoed fuion eult i compaed with the 69

4 The Intenational chive of the Photogammet, Remote Sening and Spatial Infomation Science. Vol. XXXVII. Pat B7. Beijing 008 conventional egion-baed image fuion uing Cann egmentation b both viual inpection and objective anali. Fou objective meaue ae ued in thi evaluation, i.e. Entop, Mutual Infomation (MI), Spatial Fequenc (SF) and Relative Dimenionle Global Eo in Snthei (ERGS) (Ekicioglu, et al,995). Built-up aea data et Figue. Fuion of built-up aea image. (a)oiginal Pan image, (b)oiginal MS image, (c) Cann egmentation of Pan image, (d) Cann egmentation of MS image, (e) mean hift egmentation of Pan image, (f) mean hift egmentation of MS image, (g) Cann egmentation fued eult, (i) Ou popoed fued eult viual inpection of eult hown in Figue eveal that the Cann egmentation of Pan and MS image poduce ove egmented eult and the eultant egion ae too tinn to tell which one i ueful. In contat, the mean hift egmentation poduce ounde eult. The egmentation tend of Pan and MS image ae neal the ame. The total numbe of egion in MS image i lightl moe than that in Pan image with the mean hift egmentation but the elationhip i eveed with Cann egmentation. It i clea that the image fued with mean ift egmentation i cleae than that with Cann egmentation. The patial textue of image fued with Cann egmentation i ditubed due to ove egmentation. (a) (c) (e) (d) (f) (b) image Entop MI SF ERGS B G R Ni B G R Ni Table. Quantitative eult on built-up aea. (Image fom Cann egmentation fuion and image fom popoed fuion) The objective evaluation i made to veif the viual anali and the eult ae given in Table. Fom Table we can ee, that the Entop in image i lage than that in image except fo the blue band. The MI in image i alo a lightl lage than that in image. Of coue, thi doen t mean the qualit of image i bette than of image a the noie will alo inceae the infomation which can be veified b viual anali. But the SF in image i much highe than in image. Thi mean the clait in fome image i highe than latte image which ae conitent with the viual inpection. The malle ERGS implie eo in all the band i malle. Rual aea data et (g) (h) (a) (b) (i) ( 70

5 The Intenational chive of the Photogammet, Remote Sening and Spatial Infomation Science. Vol. XXXVII. Pat B7. Beijing 008 (c) (d) image Entop MI SF ERGS B G R Ni B G R Ni Table. The quantitative eult on ual aea. (Image fom Cann egmentation fuion and image fom popoed fuion) (e) (g) (h) (f) 6. CONCLUSIONS In thi pape, a new technique fo the fuion of high-eolution image ha been decibed. In thi technique, mean hift egmentation i adopted to extact the featue fom image. SSIM i ued to meaue the egion imilait which ha moe phical meaning. The SSIM i then ued to guide the deciion making in the fuion poce. Expeimental evaluation have been conducted fo built-up aea and ual aea. The eult how thi new technique pefom bette than the conventional technique with Cann detection opeato. It ha been found that, fo high eolution image, the Cann detection tend to poduce untable egmentation, i.e. ove egmentation in a ub-egion and unde egment in anothe egion of the ame image. On the othe hand, mean hift egmentation i moe eliable. The hapne and pectal eevation of image fued b ou popoed technique ae bette than thoe b conventional method with Cann egmentation. Thi concluion made hee ae baed on the limited tet. Moe compehenive tet will be conducted in the futue. CKNOWLEDGEMENT Thi eeach wa uppoted b State 973 poject gant 006CB (i) Figue 3. Fuion of ual aea image. (a)oiginal Pan image, (b)oiginal MS image, (c) Cann egmentation of Pan image, (d) Cann egmentation of MS image, (e) mean hift egmentation of Pan image, (f) mean hift egmentation of MS image, (g) Cann egmentation fued eult, (i) Ou popoed fued eult figue 3 how, the tet on ual aea how a imila eult to the built-up aea. The image in Figue 3(c) i ove egmented. In Figue 3(d), ome pat ae ove egmented and ome ae unde egmented. The egmentation eult hown in Figue 3(e) and (f) ae moe eaonable than thoe in Figue 3(d) and (e). The fued eult baed on Cann egmentation i blued a can be een fom Figue 3(g) and (h). The hapne and pectal eevation of the image fued b thi new method ae bette. Moe detailed quantitative evaluation eult i given below: ( REFERENCES Comaniciu, D., Mee, P.,999. Mean hift anali and application. The poceeding of the eventh IEEE intenational confeence on Compute Viion, pp Ekicioglu,.M., Fihe, P.S.,995. Image quantit meaue and thei pefomance. IEEE Tanaction on Communication, 43 (), pp Lewi, J.J., O Callaghan, R.J., Nikolov, S.G., Bull, D.R., Canagaaja N.,007. Pixel- and egion-baed image fuion uing complex wavelet. Infomation Fuion (8), pp Luo, J.B., Guo, C.E.,003. Peceptual gouping of egmented egion in colo image. Patten Recognition, 36(), pp Mo, D.K., Lin,H., Li, J.P., Sun, H., Xiong, Y.J.,006. VHR Image Multi-Reolution Segmentation Baed on Mean Shift. Jounal of Guangxi Nomal Univeit, 4(4), pp

6 The Intenational chive of the Photogammet, Remote Sening and Spatial Infomation Science. Vol. XXXVII. Pat B7. Beijing 008 Piella, G.,003. geneal famewok fo multieolution image fuion: Fom pixel to egion. Infomation Fuion, 4(4), pp Pohl, C., Gendeen, J. L. V.,998. Multieno image fuion in emote ening: concept, method and application. Intenational Jounal of Remote Sening, 9(5), pp Wald, L., Ranchin, T., Mangolini, M.,997. Fuion of atellite image of diffeent patial eolution: eing the qualit of eulting image. Photogammetic Engineeing & Remote Sening, 63(6), pp Wang, R., Gao, L. Q., Yang, S., Chai, Y.H., Liu,Y.C.,005. n Image Fuion ppoach Baed On Segmentation Region. Intenational Jounal of Infomation Technolog, (7), pp Wang, Z., Bovik,.C.,00. univeal image qualit index. IEEE Signal Poceing Lette, 9(3), pp Wang, Z., Bovik,.C., Sheik H.R., Simoncelli, E.P.,004. Image qualit aement: Fom eo viibilit to tuctual imilait. IEEE Tanaction on Image Poceing, 3(4), pp Zhang, Z., Blum, R.S.,997. Region-baed image fuion cheme fo concealed weapon detection. Poceeding of 3t nnual Confeence on Infomation Science and Stem, Mac pp Zhen, J., He, G.J.,005. Shotage of the data fuion appoache to high-eolution atellite image and expected impovement. Remote ening infomation, pil, pp