Rong Wang, L-Qun Gao, Shu Yang, Yu-Hua Cha, and Yan-Chun Lu An Image Fuson Approach Based On Segmentaton Regon An Image Fuson Approach Based on Segmentaton Regon Rong Wang, L-Qun Gao, Shu Yang 3, Yu-Hua Cha 4, and Yan-Chun Lu 5 35# Insttute of Informaton Scence & Engneerng, Northeastern Unversty, 0004 Shenyang, Chna dbdxwangrong@63.com, lqungao064@yahoo.com, 3 shuyang04@63.com, 4 yhcha@63.com, 5 luyanchun@63.com Abstract An Image fuson method based on segmentaton regon s proposed n ths paper. Frst, the source mages are decomposed by wavelet to get the approxmate and detaled sub-mages, and the segmentaton by watershed method for these sub-mages are used to get the regons of each level, these regons are used to gude fuson process. The actvty level and match degree measure of the wavelet coeffcents of source mages are carred out n these regons, and the maxmum value rule and the weghted average rule are respectvely used to combne the coeffcents of detaled sub-mages and approxmate sub-mage. At last, the combnaton coeffcents are nversely transformed by wavelet to get the fnal fuson mage. The expermental results show that the fuson effect s better. Key words: Image fuson; mage segmentaton; wavelet transform. Introducton Wth the fast development of the technque of sensor, mcro-electronc and communcatons, the technque of nformaton fuson has been pad more and more attenton to by people. In the applcaton feld of mage processng, mage fuson has a wde foreground. The am of mage fuson s to ntegrate complementary and redundant nformaton from multple mages to create a composte that contans a better descrpton of the scene than any of the ndvdual source mages. The actual fuson process can tae place at dfferent levels of nformaton representaton[]. A common categorzaton s to dstngush between pxel, feature and symbol level. Image fuson at pxel-level means fuson at the lowest processng level referrng to the mergng of measured physcal parameters. Fuson at feature-level requres frst the extracton (e.g., by segmentaton procedures) of the features can be dentfed by characterstcs such as sze, shape, contrast and texture. The fuson s thus based on those extracted features and enables the detecton of useful features wth hgher confdence. Fuson at symbol level allows the nformaton to be effectvely combned at the hghest level of abstracton. The choce of the approprate level depends on many dfferent factors such as data sources, applcaton and avalable tools. 9
Internatonal Journal of Informaton Technology, Vol., No. 7, 005 Currently, t seems that most mage fuson applcatons employ pxel-based methods[], [3]. The advantage of pxel fuson s that the mages used contan the orgnal nformaton. Furthermore, the algorthms are rather easy to mplement and tme effcent. The exstng fuson methods based on pxel-level are very senstve to msregstraton, thus the accuracy of mage regstraton s demanded n sub-pxel level. In some examples of mage fuson, the fuson method based on the regon feature has more meanng than the one based on pxel method. The robustcty of the method based on regon feature s better than the one based on pxel, whch avods the problem exstng n pxel fuson method, such as the senstvty to nose and msregstraton [4], [5], [6], [7]. The regon-based approach have some advantages that the fuson process becomes more robust and avods some of the well-nown problems n pxellevel fuson such as blurrng effects and hgh senstvty to nose and msregstraton. Image fuson methods based on wavelet transform have been wdely used n recent years. The method proposed n reference 5 uses wavelet method to fuse mages, but t uses a slp wndow regon wth fxed scale to compute the area power to determne whch one wll be consdered n the fused mage, so t maybe brngs the problem of the nconsstent coeffcents selecton to the same object. In ths paper, a wavelet transform mage fuson method based on segmentaton regons s proposed, n whch each regon corresponds to one object n mage, and all the fuson procedures are carred out n ths regons, thus the abuse of nconsstent coeffcents selecton to the same object maybe occurred n fxed wndow regon s avoded. It combnes effectvely the pxel-level and the feature-level method to get a better fuson mage. The expermental results show that the performance of our method s better than the one proposed n reference 5. The Fast Algorthm of -D Wavelet Transform There s a quc algorthm for the -D orthogonal wavelet transform ---- alat algorthm, whch converts the calculaton problem of wavelet transform nto the one of the coeffcent after wavelet transform. Suppose H={h n }, G={g n } are the low-pass and the hgh-pass flters respectvely when the mage s decomposed, then the process of calculatng decomposton coeffcents of the tensor product wavelet s, c ( m, n) = f( m, n) + c ( m, n) = h( m) h( l n) c (, l) D ( m, n) = h( m) g( l n) c (, l) + D ( m, n) = g( m) h( l n) c (, l) + D ( m, n) = g( m) g( l n) c (, l) + 3 l, l, l, l, Where c (m, n) s the wavelet coeffcents at the level, also the orgnal mage data; c + (m, n) s the mage data at the + level and the low frequency component data after the mage data c (m, n) at the level are decomposed by wavelet, and s the general pcture of c (m, n). The contour of c + (m, n) s smlar to that of c (m, n); () 93
Rong Wang, L-Qun Gao, Shu Yang, Yu-Hua Cha, and Yan-Chun Lu An Image Fuson Approach Based On Segmentaton Regon whle D + (m, n), D + (m, n) and D 3 + (m, n) are the hgh-frequency detaled sgnals after c (m, n) s decomposed by wavelet. 3 The Prncple of Image Fuson Based on Segmentaton Regon 3. Fuson ethod Based on Segmentaton Regon The regon-based fuson method s shown as Fg., there are 7 modules n the whole fuson scheme: mage segmentaton, wavelet transformaton, match measure, actvty measure, decson of selectng coeffcents, coeffcents combnaton and nverse wavelet transformaton. Image Image Y Segmentaton R Wavelet Transform Wavelet Transform match my Image Actvty Image Y Actvty Decson map Combnaton coeffcents Inverse Wavelet Transform Fuson Image Fg.. The scheme of fuson method based on segmentaton regon The man tas of mage segmentaton s to obtan a set of segmentaton regons R (R, R,, R ), R represents the segmentaton regons at level. Ths representaton 94
Internatonal Journal of Informaton Technology, Vol. No. 5 wll gude the other blocs of fuson process: actvty measure, match measure and fuson operator are all carred out n them. In our fuson scheme, the segmentaton s a preparatory step toward actual fuson. If the accuracy of segmentaton s hgh, then t s very favorable for mprovng the fuson performance. In ths module, a segmentaton method based on watershed and wavelet s used. The concrete steps are as follows. After creatng the pyramd mage usng a wavelet transform, the lowest-resoluton mage I s segmented through the applcaton of a watershed algorthm and a partton R from I s generated. Generally, watershed algorthms produce meanngful mage segmentaton. However, when an mage s degraded by nose, t becomes over-segmented. So t may requre further mergng of some regons. Our decson on whch regons to merge s determned through homogenety and smlarty crtera based on the wavelet coeffcents. Each of the segmented regons wll have mean, second-order and thrd-order central moment values of the wavelet coeffcents calculated. All the features are computed on the lowest-resoluton sub-mage. For each regon R of the segmented mage R of the mage segmentaton phase, we calculate the mean (), second-order ( µ ) and thrd-order ( µ 3 ). They are defned as, = (, ), R x y x y R num( R ) µ = ( R ( x, y) ) num( R ) µ = ( R ( x, y) ) num( R ) 3 () Where, num(r ) s the number of pxels of segmented regon. To merge the segmented regons usng smlarty crtera (d), we can use the followng equaton: mv = ( R ( ) + R ( µ ) + R ( µ 3 )) =,..., N N d R R mv mv j N j (, j ) = ( j), {,...,, for } (3) Where, mv s the smlarty value of segmented regon and N s the number of segmented regons. R(), R( µ ) and R( µ 3 ) are the mean, second-order and thrd-order moment values of the segmented regon respectvely. If the mv values of the adjacent regons satsfy a specfed value, two adjacent regons wll be merged. The specfed values are found by the experment. Once the merged mage s generated at the mage partton R, t must be projected down n order to reconstruct the dfferent-resoluton mage. To project the segmented mage wth label onto the next level mage, wth the mult-resoluton property to wavelet transformaton, we can frst obtan a coarse merged mage wth label on the low-resoluton mage. Then, the merged mage wth label can be progressvely 95
Rong Wang, L-Qun Gao, Shu Yang, Yu-Hua Cha, and Yan-Chun Lu An Image Fuson Approach Based On Segmentaton Regon refned level by level untl full resoluton s reached. The projecton method used to generate - s carred out n followng steps:. Inverse wavelet transform s appled to, so a - s obtaned. The projected mage - produces new values because each regon of mage has an average pxel value.. Watershed lne mage B - from the mage I - s generated by the watershed algorthm. 3. The boundary and labels of the mage - are refned accordng to the mage B -. To refne the mage -, we (a) determne the central poston at each regon of the mage B -. (b) select a label that s the great many ncluded labels n the regon of mage - ncludng the determned central poston n the above step. (c) assgn the selected label to the regon of mage B-. 4. The result s the refned segment mage at level (-) of the pyramd, dentfed as -. In ths step, the mage - has many more regons than the number of regons of mage. However, the adjacent regons of - are assgned the same label. Therefore, the number of regons of the segmented mages - and s equal. 3. Fuson ethod Actvty Level easure. The actvty level of wavelet coeffcents reflects the local energy n the space. In general, the ponts wth larger actvty level are selected. The actvty of each coeffcent s defned as, AI ( p) = ϖ ( s, t) DI ( m + s, n + t,, l) (4) s S, t T Where, A I ( p) s the actvty level (I =, Y ), p = (m, n,, l ) the wavelet coeffcents of mage, ( m, n ) the specal poston n a gven frequency, the decomposton level, l the frequency band of decomposton, ϖ (s, t ) a weght, S and T are sets of horzontal and vertcal ndexes that descrbe the current wndow (typcally 3 3 or 5 5), D I ( m + s, n + t,, l ) the absolute value of the correspondng coeffcent n a small wndow. The actvty level of each regon R R s defned as, A ( ) I ( R p) = R r N R A ( ) I r ( N p) (5) Where, N r = ( m, n ) R, R R, R s the area of regon R, A () I ( N r p ) the actvty level of each coeffcent n regon R, A () I ( R p ) s the actvty level of regon R. 96
Internatonal Journal of Informaton Technology, Vol. No. 5 atch Degree measure. The match degree s the smlarty measure between the correspondng pxels n source mages, t s usually expressed n terms of a local correlaton measure, alternatvely, the relatve ampltude of the coeffcents or some other crtera can be used. A match measure functon Y ( p ) between the correspondng pxels n source mages s defned as, Y ( p) = s S, t T ϖ (,) s td( m+ sn, + tld,,) ( m+ sn, + tl,,) A ( p) + A ( p) Y Y (6) Where, ϖ (s, t ) s a weght, S and T are sets of horzontal and vertcal ndexes that descrbe the current wndow (typcally 3 3 or 5 5), A and A Y the actvtes calculated by equaton (4), D I ( m + s, n + t,, l ) the absolute value of the correspondng coeffcent n a small wndow (I =, Y ). The defnton of match degree measure of regon s, ( ) Y ( R p) = R r N R ( ) Y r ( N p) (7) Where, () Y ( N r p ) s the match degree between the correspondng pxels n regon R n source mages, Y () ( R p ) the match degree between the correspondng regons n source mages. The Decson of Coeffcents Selecton and Combnaton. The decson of selectng coeffcents s the ey of combng coeffcents, a decson map can be constructed accordng to the actvty measure, match measure and segmentaton regon obtaned by calculatng before, whch mage s coeffcents are selected to combne the coeffcents are determned by ths map. The equaton of coeffcent combnaton s as follows, D ( R p) = ϖ D ( R p) + ϖ D ( R p) (8) F Y Where, D F (R p) s the combnaton coeffcent n regon R, D (R p) and D Y (R p) respectvely the coeffcents of source mages and Y correspondng to R, ϖ and ϖ Y weghts, and ϖ + ϖ Y =. In fact, the calculaton of constructng decson map s equvalent to the one of weghts ϖ and ϖ Y, n ths paper, the calculaton of weghts are consdered as two nstances: maxmum value rule and weghted average rule. axmum value rule s, ϖ ϖ =, ϖ = 0, A ( R p) > A ( R p) Y Y = 0, ϖ =, A ( R p) < A ( R p) Y Y Gven a matchng threshold α, then the weghted average rule s, Y (9) 97
Rong Wang, L-Qun Gao, Shu Yang, Yu-Hua Cha, and Yan-Chun Lu An Image Fuson Approach Based On Segmentaton Regon When Y (p) < α, the calculaton of weghts are as equaton (9), when Y (p) > α, the weghts are as follows, ( Y ),, ( ) ϖ = ϖy = ϖ A R p < AY ( R p ) α ( Y ),, ( ) ϖ = + ϖy = ϖ A R p > AY ( R p ) α (0) Because of ther dfferent physcal meanng, the approxmaton mage and detaled ones are usually treated by dfferent methods. Detaled coeffcents havng large absolute values correspond to sharp ntensty changes and hence to salent features n the mage, such as edges, lnes and regon boundares. The nature of the approxmaton coeffcents, however, s dfferent from the detaled ones. The approxmaton mage s a coarse representaton of the orgnal mage and may have nherted some of the latter s propertes, such as the mean ntensty or texture nformaton. In ths paper, the composte approxmaton coeffcents of the hghest decomposton level adopt the weghted average method and the composte detaled coeffcents adopt the maxmum value method. So the robustcty s mproved. 4 The Performance Estmaton and Expermental Results The qualty of fuson mage s generally estmated by the root mean square error ( RSE ) between the fuson mage and the deal mage, t s defned as, N RSE = ( G( m, n) GF( m, n)) N = = () m n Where, G ( m, n ) s the gray value of pxel pont ( m, n ) n deal mage, G F ( m, n ) the gray value of the correspondng pxel pont ( m, n ) n fuson mage, N the mage magntude. The smaller RSE s, the better the fuson effect s. Some researchers ponted that the number of the optmal levels of wavelet decomposton for mult-focus mage s 3[8]. In our experments, we use wavelet bor. to decompose the source mages to three levels, and a set of segmentaton regons are got usng the method mentoned n secton 3. The combnaton coeffcents of approxmate sub-mage and detaled sub-mages are respectvely selected accordng to the two fuson rules, weghted average rule and maxmum value rule. When the threshold α = 0.85, the fuson effect s better. To verfy the performance of our algorthm, we use a set of source mages (56 56). The comparson wth the method proposed n reference 5 s also done. Fg. (a) and (b) are the two source mages whch have dfferent focuses, the am of fuson s to solve the local blurry problem n mage caused by the dfferent focus pont. Fg. (a) focus on the bg plane, t s clear, however, the small one s blurrng; Fg. (b) 98
Internatonal Journal of Informaton Technology, Vol. No. 5 focus on the small plane, the bg one s blurrng and the small one s clear. Fg. (c) s the segmentaton regons used for gudng fuson ( level), Fg. (d) the deal mage, Fg. (e) the fuson mage got from the method proposed n reference 5, Fg. (f) the fuson mage obtaned by usng our method, the bg and small planes are all clear n ths mage, Fg. (g) the dfference mage between the fuson mage got from method n reference 5 and the deal mage, and Fg. (h) the dfference mage between the fuson mage got from our method and the deal mage. Seen from the fuson mage and the dfference mage, the performance of our method s superor to the one n reference 5. (a) (b) (c) (d) (e) (f) (g) (h) Fg.. Source mages and expermental results To verfy the fuson algorthm proposed n ths paper, we also use equaton () to estmate the performance of fuson mage. The standard devaton of the fuson mage got from reference 5 s 3.403 and the one of our fuson mage s 8.958. Known from the expermental results, the fuson effect of our method s better than the one of reference 5. 5 Conclusons An mage fuson algorthm based on segmentaton regon s proposed n ths paper, t combne effectvely the pxel-level and feature-level fuson method. Because t s guded by the segmented regons whch represent dfferent object, so the performance s better than the one whch s not based on the object regons. Frst, the source mages are decomposed by wavelet to get a set of segmentaton regons of each level, and the actvty measure, match measure and coeffcents combnaton are carred out n these segmentaton regons, n the end, the fnal fuson mage s obtaned by the nverse wavelet transformaton of combnaton coeffcents. Seen from the results, because our fuson method s carred out based on the segmentaton regons and the 99
Rong Wang, L-Qun Gao, Shu Yang, Yu-Hua Cha, and Yan-Chun Lu An Image Fuson Approach Based On Segmentaton Regon match degree between source mages, actvty level and regonal features are consdered at the same tme, the salent nformaton of source mage are better reserved, so the fuson performance s better. References [] C. Pohl, J.L. Genderen.: ult-sensor Image Fuson n Remote Sensng: Concepts, ethods and Applcatons. Internatonal Journal of Remote Sensng, Vol. 9 (5) (998), pp. 83-854. [] C.Y. Wen, J.K. Chen.: ult-resoluton Image Fuson Technque and Its Applcaton to Forensc Scence. Forensc Scence Internatonal. Vol. 40 (004), pp. 7-3. [3] Gonzalo Pajares, Jesus anuel de la Cruz.: A wavelet-based Image Fuson Tutoral. Pattern Recognton. Vol. 37 (004), pp. 855-87. [4] YongAn Zheng.: The Adaptve Fuson Algorthm of Weghtng Regonal Features Based on Wavelet Transformaton. Computer Engneerng and Applcaton. Vol. 6 (004), pp. 94-96. [5] Hao Yan.: ult-scale Image Fuson Based on Wavelet Transformaton. Computer Engneerng and Applcaton. Vol. 9 (003), pp. 3-34. [6] B.J. atuszews, L.K. Shar.: Regon-based Wavelet Fuson of Ultrasonc, Radographc and Shearographyc Non-destructve Testng Images. Proceedngs of the 5th World Conference on Non-destructve Testng. (000), pp. 358-363. [7] Z.Zhang, R. Blum.: A Regon-based Image Fuson Scheme for Concealed Weapon Detecton. Proceedngs of the 3th Annual Conference on Informaton Scences and System. (997), pp. 68-73. [8] Wewe Wang.: ult-focus Image Fuson Algorthm n Wavelet Doman. Systems Engneerng and Electroncs. Vol. 5 (004), pp. 668-67. Rong Wang s a Ph.D. student n Insttute of Informaton Scence & Engneerng, Northeastern Unversty, the research feld s pattern recognton and ntellgent system. L-Qun Gao s a professor n Insttute of Informaton Scence & Engneerng, Northeastern Unversty, the research feld s pattern recognton and ntellgent system. 00