A PARAMETRIC MODELING APPROACH FOR WIRELESS CAPSULE ENDOSCOPY HAZY IMAGE RESTORATION

Transcription

1 A PARAMETRIC MODELING APPROACH FOR WIRELESS CAPSULE ENDOSCOPY HAZY IMAGE RESTORATION Y. Wan 1, C. Cai 2, J. Liu 1, Y. X. Zou 1 * 1 ADSPLAB/ELIP, Shool of ECE, Pekin University, Shenzhen , China 2 Department of Computer Siene, Collee of Information Enineerin, Northwest A&F University, Yanlin , China ABSTRACT Wireless apsule endosopy (WCE) is an innovative solution for astrointestinal disease detetion. The imae quality of WCE is not satisfatory for medial appliations sine some of them are dark or hazy. For the purpose of improvin WCE imae quality, we take a new way to establish a parametri imae eneration model (alled WCE hazy model) between the aptured imae and the ideal imae by onsiderin adverse effets due to inhomoeneous lihtin, unfoused and liht refletion. Some experiments have been arried out to validate this model. Aordinly, the retinex theory and dark-hannel prior have been adopted to estimate the model parameters adaptively. Hene, the WCE hazy imae restoration is ahieved by the inverse proess of the hazy model. Intensive experiments have been onduted with hazy WCE imaes of the testers. Experimental results usin the subjetive and objetive performane measures further verify the effetiveness of proposed method. Index Terms wireless apsule endosopy, imae restoration, parametri eneration model, retinex theory, dark-hannel prior 1. INTRODUCTION In the past year, diestive system aner had been the seond aner-related killer in U.S.A. and aused 61,950 death [1]. Early detetion and treatment of astrointestinal disease are very ruial. Wireless apsule endosopy (WCE) is an innovative solution for astrointestinal disease detetion. It was invented in 2000 and put in use in 2001 [2]. However, the imaes aptured by WCE system are not as ood as those taken by traditional endosopy due to several hardware onstraints [3]: 1) the battery apaity is limited resultin in inhomoeneous lihtin, 2) the amera used in WCE is low-foal-lenth amera whih may produe unfoused imaes; 3) ompliated irumstane of astrointestinal trat and movin imain method also lead to poor imae quality. Fi. 1 shows several WCE imaes with low quality, where Fis. 1 (1) and (2) are with weak lihtin, Fis. 1 (3) and (4) are hazy, whih provide less dianosti information for physiians. As a result, the imae quality improvement is hihly demanded in WCE medial appliations. There are several imae enhanement approahes, suh as historam equalization-related methods [4], Filter based methods [5], transform domain based methods [6], and anisotropi diffusion based approahes [3, 7]. Experimental results show that these methods have some ood effets on enhanin lowluminosity or low-ontrast imaes, but don t have desired effet in (1) (2) (3) (4) Fi. 1. Low quality WCE imaes (Dark and hazy) handlin hazy imaes. Moreover, it is noted that the hazy imaes do not lak luminosity while they have distorted olor and lowontrast properties. Viewed from another perspetive, the hazy effet is similar to the blurred effet whih usually is aused by motion or poor fousin. The blurred imae ould be restored under the onvolution model by usin the inverse filterin in its frequeny domain. Wiener filterin is widely used and quite effetive in solvin deblurrin problem [8, 9], but it needs pre-set parameters hosen by trial and error, whih is impratial without the knowlede of point spread funtion (PSF). Besides, whether the hazy effet of WCE imaes equals to blurred effet is unknown. Some disussions are presented in Setion 2. In this irumstane, a more eneral model is expeted to handle the quality improvement of the hazy imaes. By investiatin the mehanism of the WCE imae eneration proess, we believe that inhomoeneous distributed illumination and lost fousin are main fators ontributin to the low quality of WCE imaes. Motivated by the methods developed for removin the fo from the natural imaes, in this paper, we make an effort to establish the parametri imae eneration model (alled hazy model here) between the aptured imae and the ideal imae by onsiderin adverse effets due to the inhomoeneous lihtin, unfoused and liht refletion. To make the problem tratable, the followin assumptions are made: 1) the aptured WCE imae has a ounterpart ideal imae. 2) the transformation of the ideal imae to the apture imae is nonlinear but an be modeled in a parametri way. 3) the parameters an be estimated. In the rest of this paper, Se. 2 introdues a parametri WCE imae eneration model and the validation of the model by experiments. Se. 3 shows the derivation of the adaptive WCE imae restoration alorithm. Se. 4 presents the experimental results and performane analysis. Finally, Se. 5 ives the onlusion. 2. PROBLEM FORMULATION In natural imae dehazin researh, the most ommonly used imain model for imaes is the additive model iven as [10, 11]: /15/$ IEEE 922 ICASSP 2015

2 I( x, y) T( x, y) R( x, y) [1 T( x, y)] a (1) where (x,y) is the pixel index. I and R are the enerated or observed imae and ideal imae (sene radiane), respetively. T is the transmission matrix and T(x, y) (0,1). Transmission is the property of a substane to permit the passae of liht. Here the element of T is a ratio of sene radiane reahin the amera. a is the lobal atmospheri liht. It is onstant due to its homoeneousness. In this study, we follow the same spirit in (1) to model WCE imaes. Obviously, for eneratin WCE imaes, the ambient illumination was reated by multi liht soures, so it is no loner homoeneous and shows the inhomoeneous property. Thus it is reasonable to establish a eneration model for WCE imaes as follows: I( x, y) T( x, y) R( x, y) [1 T( x, y)] L( x, y) (2) where L is the inhomoeneous ambient illumination presented in imain proedure. From (2), I an be seen as the omposition of weihted sene radiane R and weihted ambient illumination L. It is lear that the weihts are ontrolled by the transmission T. Carefully examinin (2), we have the followin observations: 1) When T(x, y) = 0, (2) ives I(x, y) = L(x, y). This indiates that no information of sene radiane left in I, so it s impossible to reover R from I. 2) When T(x, y) (0,1) but L(x, y) = 0, (2) turns to be the multipliative model denoted as [12] I( x, y) T( x, y) R( x, y) (3) where I is a portion of R and an be rouhly estimated by imae enhanement alorithms suh as historam equalization. 3) When T(x, y) (0,1) while L(x, y) > 0, (2) presents its eneral form and I is influened by R, L and their respondin weiht. 4) When T(x, y) = 1, (2) shows the ideal imain status and we have I(x, y) = R(x, y). Unlukily, this ase rarely ours in real appliations. In order to put model (2) in use, we propose the followin in modellin approahes for transmission T and illumination L. Followin the researh in [13], transmission T for WCE imain an be modeled as N I ( x, y) F ( x, y) R( x, y) (4) temp n n Itemp ( x, y) min( Itemp ) T( x, y) max( I ) min( I ) temp temp N where n is the weiht, 1 n and in our study n = 1 / N, N = 3. Fn(x, y) is a multi-sale Gaussian funtion with the standard deviation σn, where σn is set to different values at eah n. * is the onvolution operator. It an be seen that model (4) simulates the transmission property by applyin the multi-sale Gaussian blurrin effets on R. In addition, (5) ives the normalized transmission model where we assume the transmission T(x, y) is proportional to I temp (x, y). I temp is the raysale version onverted from Itemp from RGB hannels. Now, let s onsider illumination L for WCE imain. Motivated by the researh in [14], it will be a ood option to model L as: L( x, y) F( x, y) R( x, y) (6) where F(x, y) is a sinle Gaussian funtion, whih is able to model inhomoeneous lobal illumination properly. So far, we have established the eneral WCE imae eneration model by (2), (5) and (6). Let s have a demonstration. We selet (5) Oriinal lear WCE imae: R T(x, y)r(x, y) The enerated imae I Radiane throuh solute Ambient illumination L Plus Ambient lihtin (1) (2) (4) Fi. 2. WCE hazy imae eneration Fi. 3. Syntheti hazy effet. First row shows lear imaes and seond rows shows enerated hazy imaes. one ood quality WCE imae as R shown in Fi. 2 (1). If illumination L is zero or lose to zero, but the transmission is modeled in (5), the resultin imae by model (2) is shown in Fi. 2 (2). We an see that the quality of the imae R has been deraded reatly due to the transmission matrix T. Moreover, omparin the imae in Fi. 2 (2) with the imaes in Fis. 1 (1) and (2), it is easy to observe their similarities with dark and low ontrast properties. This may explain the dark WCE imaes are enerated due to low transmission. Fi. 2 (3) is enerated by (6). It is a typial blurred imae where lobal illumination is improved and some informative details are smoothed. Just like bubble loates in top left orner whih an be observed in Fis 2 (1) and (4), it vanishes in Fi. 2 (3) where it is expeted to be shown. Moreover, If transmission and illumination are all non-ideal, whih are modeled in (5) and (6), the WCE imae I enerated by model (2) is shown in Fi. 2 (4). It is easy to identify that imae in Fi. 2 (4) presents some ommon features of the hazy imaes shown in Fis. 1 (3) and (4). They all look low ontrast and lobal briht, and still ontain some informative details. By the omparisons between Fi. 2(3) and Fi. 2(4), and between Fi. 2 (4) and Fis 1 (3) (4), it is easy to identify the differenes between the blurred imae and the hazy imae. The latter ontains more useful information about R than the former. With the disussion shown above, it is not appropriate to use traditional onvolution model desined for blurred imaes to deal with WCE hazy problem. More synthesized hazy WCE imaes by the proposed model in (2) are shown in Fi. 3. Comparin the synthesized hazy imaes in Fi. 2 and Fi. 3 with those in Fi. 1, we may onlude that the proposed eneration model (2) is suitable for modelin the real WCE hazy imaes. As a result, if illumination L and transmission matrix T an be estimated properly, it is straihtforward to reover R from I by solvin the inverse problem of model (2) and the reovered R will have improved quality ompared with I. (3) 923

3 Input WCE hazy imae I ambient lihtness L transmission T Proposed WCE hazy model real radiane R Automati illumination ompensation Fi. 5. The diaram of the WCE hazy imae restoration alorithm (WCE-HIR) The restored imae Fi. 4. The sequene of translation vetors for the Dn in FMR 3. THE PROPOSED WCE HAZY IMAGE RESTORATION METHOD In this setion, we will first present the estimation methods for T and L, respetively. Then the proposed parametri modelin based WCE hazy imae restoration (WCE-PM-HIR) method will be introdued in details The estimation of ambient illumination As assumed in Setion 2, illumination L is normal distributed modeled in (6). Here, we need to estimate L from the aptured WCE imae I. In our study, Frankle-MCann Retinex (FMR) approah is adopted to solve this problem. In FMR theory, it assumes that the illumination in pixel (x, y) is not only affeted by its neihborin pixels, but also by the pixels at far distane, whih makes imae illumination looks more natural. Aordin to [15, 16], the estimation of illumination L usin FMR method an be implemented as follows: the illumination L0-hat is initialized to be the maximum value of observed imae I, then FMR exeutes the iterative proedure: ˆ ˆ ( ˆ ˆ Ln I Ln Dn Ln ) L max{, } (7) 2 2 where Ln-hat is the estimated illumination by iterative approah, whih will be assined to L when iteration is stopped. Dn(.) is a spatially shiftin operator whih will shift the imae by the n th element of a sequene of spirally deayin translation vetors {dn}, where dn an be denoted as vn,n+1, just as shown in Fi. 4. The relation between two adjaent translation vetors satisfy: dn = 2 dn+1. The iteration of FMR will be terminated when dn <=1. Speifi details an be referred to [16] Transmission After we et the estimation of L, we need to estimate the transmission T from I based on model (2). Motivated by the researh in [11], we solve this problem based on the dark-hannel prior (DCP). To make the presentation ompleteness, the dark-hannel prior is briefly introdued first. It is noted that DCP is onluded from analyzin the statistis of outdoor haze-free imaes [11]. It finds that in most non-sky reions, at least one olor hannel has some pixels whose intensity is very low (lose to zero, alled dark hannel), whih an be formulated as: dark R x, y min min R x, y0 (8) x, yω x, y r,,b R where indiates the hannel imae of R, Ω(x, y) is an imae path whih enters at (x, y), and (, ) is a pixel index in path Ω. For R x y, from (2), by normalizin related illumination, it yields:,, I x y R x y, 1, T x y T x y (9) L ( x, y) L ( x, y) As noted in Setion 2, the transmission T is modeled by (4) and (5), so it is reasonable to assume T as a onstant over the path Ω(x, y), whih is denoted as Tp(x,y), and then puttin the minimum operator on both sides of (9) ives: min { min ( I x, y L x, y )} m 1 Tp x, y (10) x, yω x, y r,,b where m min { min ( R x, y L x, y )} Tp x, y. x, yω x, y r,,b If the ondition in (8) satisfied, then m is zero or lose to zero. Then with simple manipulations, from (10), we an derive: Tp x, y 1 min { min ( I x, y L x, y)} (11) x, yω x, y r,,b In pratie, if all the hazy effet is removed, we may lose the feelin of depth and feel unnatural. So a onstant parameter is introdued into equation (11) to preserve a small amount of haze. Then we et: Tp x, y 1 min { min ( I x, y L x, y)} (12) x, yω x, y r,,b Based on experiments, we find =0.75 is the best for WCE visualization. 3.3 The Reovery of Sene Radiane With the estimated values of ambient illumination and transmission matrix desribed in Se. 3.1 and 3.2, the sene radiane R(x, y) an be estimated diretly usin model (2). Moreover, it is pratial to restrit the transmission by a lower bound t0 to avoid it lose to zero in ase R(x, y) is irreoverable. Therefore, we reah the followin: I x, y Lx, y Rx, y Lx, y (13) max T x, y, t p where t0 is set to be 0.05 in our study. In our researh, we noted that the sene radiane omputed by the WCE-PM-HIR method may look darker in some ase, beause of the removal of ambient illumination. To deal with this situation, the illumination ompensation approah is developed as follows. Throuh several omparisons between the lobal illumination of the estimated sene radiane and that of orrespondin oriinal imae, we propose to ompensate sene radiane as follows: R,if ( R I) l R omp( R),else 0 (14) 924

4 Table 1 PSNR and SSIM results (1) Oriinal (2) FMR (3) AAD (4) HR-DCP (5) WCE-PM-HIR Fi. 6. Clear imae and its restored imaes (1) Oriinal (2) FMR (3) AAD (4) HR-DCP (5) WCE-PM-HIR Fi. 7. Dark imae and its restored imaes (1) Oriinal (2) FMR (3) AAD (4) HR-DCP (5) WCE-PM-HIR Fi. 8. Hazy imae and its restored imaes where (.) is a funtion to measure the differene between the estimated sene radiane and the aptured imae I, l is a threshold value, omp(.) is a method or proedure to improve illumination of the imae. In this study, we selet the adaptive anisotropi diffusion method as the ompensation approah sine it is ood at improvin illumination and preservin details. The proposed WCE-PM-HIR alorithm desribed in this setion is summarized in Fi EXPERIMENTS AND RESULTS In this setion, several experiments have been arried out based on WCE imaes supplied from Shenzhen JiFu Tehnoloy Ltd., whih were aptured data from trial patients. The size of WCE imaes is with RGB three hannels. We ompared the results of our proposed method with that of three existin methods inludin the FMR [16], adaptive anisotropi diffusion (AAD) [3, 7] and haze removal based on dark-hannel prior (HR-DCP) [11]. First of all, to obtain objetive performane of our proposed WCE- PM-HIR, 15 lear WCE imaes are piked as known ideal imaes, then the proposed WCE parametri imae eneration model in (2) is used to enerate low quality imaes, whih then will be restored by different alorithms mentioned above. The peak sinal-to-noise ratio (PSNR) and the averaed strutural similarity (SSIM) [17] are used to measure the performane of the alorithms. The results are shown in Table 1. One visual example is shown in Fi. 6. In Table 1, it is lear to see that proposed WCE-PM-HIR outperforms other alorithms from objetive evaluation. It ets the hihest value in PSNR and SSIM in most ases out of 15 or on averae. Speifially, the PSNR value of WCE-PM-HIR is hiher about 2.84dB than that of HR-DCP and hiher about 7.93dB than that of AAD on averae. The SSIM index of WCE-PM-HIR is hiher about 1.50% that that of HR-DCP and hiher about 1.14% than that of AAD on averae. It is onlude that WCE-PM-HIR does reover muh more useful information (olor and loal details) of R from I than other alorithms. In syntheti ase Fi. 6, it is lear to see that FMR performs worst. Althouh the resultin lobal illumination is brihtest, it fails to reover the true olor of the oriinal imae. The illumination of the imae produed by AAD is a little darker, but it No. FMR PSNR(dB) AAD HR- DCP WCE -PM- HIR FMR performs well in reoverin loal details. HR-DCP and our proposed WCE-PM-HIR all performed well to reover olor and some loal details. They look quite similar in visualization, but the WCE-PM-HIR performs better in olor restoration. Moreover, the experimental results for improvin the low quality WCE imaes are shown in Fi. 7 and Fi. 8. From Fi. 7, where the oriinal imae is a low illumination one, it an see that WCE-PM-HIR is inferior to AAD but superior to FMR and HR- DCP. However, from Fi. 8, where the oriinal imae is a hazy imae, we an see WCE-PM-HIR outperforms other alorithms for visualization purpose. Espeially, HR-DCP and WCE-PM-HIR work well in reoverin olor information and improvin the ontrast for hazy imaes. However, from the results in Table 1, we an see that HR-DCP reovers more unwanted information (noise) than WCE-PM-HIR. From all results shown above, it is quite onfident that our proposed WCE-PM-HIR alorithm is quite effetive both in visual pereption and quality improvement for hazy WCE imaes. 5. CONCLUSION AAD This paper works on WCE hazy imae restoration problem. Motivated by the eneral imae eneration model, a hazy WCE imae eneration model is proposed based on ertain assumptions. The retinex and dark-hannel prior theory have been employed to estimate the model parameters. Intensive experimental results verify the effetiveness of the proposed WCE hazy imae restoration method. Results show that the proposed WCE-PM-HIR method is able to improve the ontrast, olor and loal details of the deraded WCE imaes, whih makes the reovered imaes look more visual attrative. ACKNOWLEDGMENT SSIM(%) HR- DCP WCE -PM- HIR av This work was partially supported by the Shenzhen Siene & Tehnoloy Fundamental Researh Proram (No: JCYJ ). 925

5 REFERENCE [1] R. Sieel, D. Naishadham, and A. Jemal, "Caner statistis, 2013," CA: A Caner Journal for Cliniians, vol. 63, pp , [2] G. Iddan, G. Meron, A. Glukhovsky, and P. Swain, "Wireless apsule endosopy," Nature, vol. 405, p. 417, [3] L. Li, Y. Zou, and Y. Li, "Wireless apsule endosopy imaes enhanement based on adaptive anisotropi diffusion," in Sinal and Information Proessin (ChinaSIP), 2013 IEEE China Summit & International Conferene on, pp , [4] M. Abdullah-Al-Wadud, M. H. Kabir, M. A. A. Dewan, and O. Chae, "A dynami historam equalization for imae ontrast enhanement," Consumer Eletronis, IEEE Transations on, vol. 53, pp , [5] A. Polesel, G. Ramponi, and V. J. Mathews, "Imae enhanement via adaptive unsharp maskin," Imae Proessin, IEEE Transations on, vol. 9, pp , [6] J.-L. Stark, F. Murtah, E. J. Candes, and D. L. Donoho, "Gray and olor imae ontrast enhanement by the urvelet transform," Imae Proessin, IEEE Transations on, vol. 12, pp , [7] B. Li and M. Q.-H. Men, "Wireless apsule endosopy imaes enhanement via adaptive ontrast diffusion," Journal of Visual Communiation and Imae Representation, vol. 23, pp , [8] J. Amudha, N. Pradeepa, and R. Sudhakar, "A Survey on Diital Imae Restoration," in International Conferene on Modellin Optimization and Computin. vol. 38, R. Rajesh, K. Ganesh, and S. C. L. Koh, Eds., ed Amsterdam: Elsevier Siene Bv, pp , [9] C. Solomon and T. Brekon, Fundamentals of diital imae proessin: A pratial approah with examples in Matlab: John Wiley & Sons, [10] R. Fattal, "Sinle imae dehazin," Am Transations on Graphis, vol. 27, Au [11] K. He, J. Sun, and X. Tan, "Sinle imae haze removal usin dark hannel prior," Pattern Analysis and Mahine Intelliene, IEEE Transations on, vol. 33, pp , [12] L. Lei, Y. Zhou, and J. Li, "An Investiation of Retinex alorithms for imae enhanement," Journal of Eletronis (China), vol. 24, pp , [13] Z.-U. Rahman, D. J. Jobson, and G. A. Woodell, "Multi-sale retinex for olor imae enhanement," in Imae Proessin, Proeedins., International Conferene on, pp , [14] D. J. Jobson, Z.-U. Rahman, and G. A. Woodell, "Properties and performane of a enter/surround retinex," Imae Proessin, IEEE Transations on, vol. 6, pp , [15] R. Kimmel, M. Elad, D. Shaked, R. Keshet, and I. Sobel, "A variational framework for retinex," International Journal of omputer vision, vol. 52, pp. 7-23, [16] D.-G. Hwan, W.-R. Lee, Y.-J. Oh, and B.-M. Jun, "Frankle- MCann Retinex by Shufflin," in Converene and Hybrid Information Tehnoloy, ed: Spriner, pp , [17] W. Zhou, A. C. Bovik, H. R. Sheikh, and E. P. Simonelli, "Imae quality assessment: from error visibility to strutural similarity," Imae Proessin, IEEE Transations on, vol. 13, pp ,