WIRELESS CAPSULE ENDOSCOPY IMAGE CLASSIFICATION BASED ON VECTOR SPARSE CODING.

Transcription

1 WIRELESS CAPSULE ENDOSCOPY IMAGE CLASSIFICATION BASED ON VECTOR SPARSE CODING Tao Ma 1, Yuexan Zou 1 *, Zhqang Xang 1, Le L 1 and Y L 1 ADSPLAB/ELIP, School of ECE, Pekng Unversty, Shenzhen , Chna Shenzhen JFu Technology Ltd. {*zouyx@pkusz.edu.cn} ABSTRACT 1 Wreless capsule endoscopy (WCE) s a promsng technology for gastrontestnal dsease detecton. Snce there are more than 50,000 frames n one WCE vdeo of a patent, classfyng the whole frame set of the dgestve tract nto subsets correspondng to esophagus, stomach, small ntestne, and colon s necessary, whch can help physcans revew and dagnose rapdly and accurately. The dgestve organ classfcaton n WCE s a challengng task due to the dffcultes n feature representaton of WCE mages. Ths paper presents a new method of WCE organ classfcaton by ncorporatng a proposed localty constrant based vector sparse codng (LCVSC) algorthm wth the support vector machne classfer. Expermental results valdate the effectveness of the proposed method and t s encouragng to see that a good classfcaton performance s acheved. Index Terms wreless capsule endoscopy, dgestve organs, mage classfcaton, vector sparse codng 1. INTRODUCTION Wreless Capsule Endoscopy (WCE) s a novel technology for recordng the vdeos of the dgestve tract of patents, whch was frst appeared n [1] and put n use by Gven Imagng Ltd., Israel n 001. WCE uses a small 11 6-mm capsule, one end of whch contans an optcal dome wth whte lght emttng dodes (LEDs) and a color camera that captures about two mages per second. When a WCE s swallowed by a patent, t wll be propelled by perstalss to move through the gastrontestnal tract, and the captured mages are transmtted to a data recorder by usng the wreless communcaton channel. Compared wth tradtonal endoscopy, the man advantages of WCE are that patents can avod cross nfecton and suffer no pan. WCE also brngs great benefts to elder and weak patents []. However, the WCE vdeo of one patent contans more than 1 Ths work was supported by the Shenzhen Scence & Technology Fundamental Research Program (No: JCYJ ) and the collaboraton research proect funded by Shenzhen JFu Technology Ltd. The authors would thank Shenzhen JFu Technology Ltd. for provdng the WCE mage dataset. (a) (b) (c) (d) Fg. 1. Image samples of the dgestve organs: (a) esophagus; (b) stomach; (c) small ntestne; (d) colon. 50,000 frames and t wll take about two or more hours for an experenced physcan to assess one WCE vdeo, whch s too tme-consumng and lmts the number of examnatons. For promotng the practcal applcatons of the WCE technque, the automatc classfcaton of the dgestve organs s requred, whch wll let the physcans locate the organs easly and help the physcans to reduce the assessment tme. Typcal samples of the dgestve organs are shown n Fg. 1. Generally computer vson based methods can be employed to dscrmnate the dgestve organs ncludng esophagus, stomach, small ntestne, and colon. Vson-based automatc classfcaton of dgestve organs s a typcal pattern recognton problem. Most prevous works follow the general framework ncludng mage feature representaton and classfer desgn. There are some solutons proposed for solvng the problem. Berens et al. [3] proposed a method of automatcally dscrmnatng stomach, ntestne and colon tssue by computng hue saturaton chromatcty hstograms whch are compressed usng a hybrd transform, ncorporatng the DCT and PCA. The K- nearest neghbor (KNN) classfer and the Support Vector Machne (SVM) were adopted and the performance comparsons were gven. Cunha et al. [4] extracted MPEG-7 descrptors as low-level mage features, and then the SVM classfer and Bayesan classfer have been employed. The research showed that usng SVM nstead of Bayesan sgnfcantly mproves the classfcaton results. In [5], a feature vector combnng color, texture, and moton nformaton was created and the mages extracted from the WCE vdeo were classfed nto meanngful parts (esophagus, stomach, small ntestne, and colon) usng the nonlnear SVM bult wthn the framework of a hdden Markov model. In ths paper, amng to develop an effectve WCE organ classfcaton method, we frst use the SIFT (scale nvarant /14/$ IEEE 58 ChnaSIP 014

2 feature transform) descrptors [7] to capture the dscrmnatve nformaton and gve a robust prmary representaton of WCE mages. Then the proposed localty constrant based vector sparse codng (LCVSC) algorthm s used to map all the extracted SIFT vectors nto a sparse feature doman, where the lnear SVM classfer can be adopted to acheve encouragng classfcaton performance. Snce the lnear SVM asks for much lower computatonal complexty n tranng and testng compared wth the nonlnear SVM [6], t s preferred over the nonlnear SVM to provde the tradeoff between computatonal complexty and classfcaton accuracy n practcal applcatons of the WCE technque, where the scalablty of tranng and the speed of testng are very mportant. The system framework of our proposed method s shown n Fg.. In the rest of ths paper, Secton descrbes the feature representaton of WCE mages, ncludng the SIFT extracton and vector sparse codng on SIFT feature vectors. Secton 3 ntroduces the supervsed classfcaton usng the lnear SVM classfer. Secton 4 shows the expermental results and Secton 5 gves the concluson.. FEATURE REPRESENTATION We consder use the grayscale mages n the WCE organ classfcaton. Each WCE mage s frstly converted nto a grayscale mage as the nput of feature representaton. To facltate the descrpton, let I s denote a WCE mage. Y s s the SIFT feature matrx of I s, whch conssts of all the SIFT feature vectors extracted from I s. X s denotes the sparse matrx obtaned by dong the vector sparse codng on each SIFT feature vector n Y s..1. SIFT Extracton SIFT descrptor s emprcally shown to outperform many other local features [8] for mage classfcaton. A SIFT feature vector (SIFT-FV) s created by frst computng the gradent orentaton and magntude at each mage sample pont n a regon around an anchor pont. The regon s parttoned nto r r subregons. A gradent orentaton hstogram for each subregon s then formed by accumulatng samples wthn the subregon, weghted by the correspondng gradent magntudes. All orentaton hstograms from subregons are concatenated to gve a SIFT-FV [9, 10]. Generally, the SIFT-FV wth the best performance s extracted from every pxel patch, whch s dvded nto 4 4 subregons. Then, the obtaned 4 4 array of hstograms wth 8 orentaton bns n each creates one 18-by-1 SIFT-FV (4 4 8). Motvated by the effectve mage representaton ablty of densely sampled SIFT descrptors [7], we use a dense regular grd nstead of commonly adopted nterest ponts to extract SIFT feature vectors, whch turns out to capture more dscrmnatve nformaton of WCE mages. For Input WCE Images Organ Types Feature Representaton Lnear SVM (Decson) Spatal Poolng Lnear SVM (Tranng) Fg.. The system framework of the proposed method for WCE dgestve organ classfcaton. mage I s, each 18 1 SIFT-FV s extracted from a pxel patch, and the SIFT-FVs are densely sampled on the regular grd wth the stepsze of 8 pxels. All the extracted SIFT-FVs consttute the SIFT feature matrx Y s... Vector Sparse Codng on SIFT Descrptors In general, sparse codng conssts of two phases: dctonary learnng and vector sparse codng. Let = [y 1,, y N ] be a matrx formed by a set of SIFT-FVs, where y ( = 1,,, N) s a M 1 (M = 18) SIFT-FV and N s the total number of the SIFT-FVs n. The sparse codng method proposed by Yang et al. n [11] s based on the followng optmzaton: mn X, D N 1 1 s. t. d 1, 1,,..., L y Dx x where D = [d 1,, d L ] s the M L dctonary and d ( = 1,,, L) denotes an M 1 base vector n D. L s generally greater than M to obtan an over-complete dctonary. X = [x 1,, x N ] denotes the sparse matrx formed by the sparse vectors assocated wth, and x ( = 1,,, N) s the L 1 sparse coded vector obtaned by codng y over D. The L 1 - norm (. 1 ) of x s the sparsty regularzaton term and s a free parameter that enforces the sparsty of the soluton. Lee et al. [1] proved that (1) s an optmzaton problem whch s not convex n both X and D smultaneously, but t s convex n X when D s fxed and convex n D when X s fxed. The conventonal procedure for the optmzaton n (1) s to solve t teratvely by alternatngly optmzng over D or X whle fxng the other. When D s fxed, Eq. (1) can be processed by optmzng over each x ndvdually as follows: mn x 1 (1) y Dx x () Ths s essentally a lnear regresson problem wth L 1 -norm regularzaton on x. Eq. () can be solved effcently by usng the feature-sgn search algorthm [1]. When X s fxed, the optmzaton n (1) can be reduced to the followng: mn DX s. t. d 1, 1,,..., L (3) D F Eq. (3) s a least square problem wth L -norm constrants on each base vector d n D. The Lagrange dual algorthm 583

3 proposed n [1] can be employed to effcently solve (3). As mentoned before, sparse codng has a dctonary learnng phase and a vector sparse codng phase. The algorthm to obtan the dctonary D s summarzed n Table I, where the nput SIFT matrx needs to be gven. In our mplementaton, we collected 5,000 WCE mages ncludng varous esophagus, stomach, small ntestne, and colon mages to extract SIFT-FVs from patches. As a result, about 1,000,000 SIFT-FVs are obtaned and used to form the for dctonary learnng. In the vector sparse codng phase, Eq. (1) s solved wth respect to X only when D s avalable and fxed. The sparse coded vector x of each SIFT-FV y wll be obtaned by solvng the L 1 -norm optmzaton n ()..3. Localty Constrant Based Vector Sparse Codng To favor sparsty, common sparse codng algorthm mght select qute dfferent bases from the dctonary for codng smlar SIFT-FVs, whch means that smlar mage patches mght have qute dfferent sparse codes. To acheve good classfcaton performance, the codng scheme should generate smlar sparse codes for smlar SIFT descrptors, whch asks for capturng the correlatons between smlar descrptors by sharng the bases n D. In ths subsecton, we propose the localty constrant based vector sparse codng (LCVSC) algorthm to mprove the tradtonal sparse codng algorthm descrbed n Secton. by ntroducng the localty constrant. When the dctonary D s obtaned as detaled n Secton. and Table I, nstead of codng each y by solvng Eq. () drectly, we frst perform a K-nearest-neghbor (KNN) search n D to buld the local base set D for each nput SIFT-FV y. In our study, the d and y are normalzed. The smlarty between d and y n KNN s measured as: M Sm( y, d ) y d (4) k k k1 where y k and d k represent the k-th element of y and d, respectvely. After KNN searchng we can obtan the local base set D whch conssts of K base vectors that are most smlar to y. Then y can be encoded over D nstead of over D va the followng mnmzaton problem: mn x 1 y D x x (5) where x s the K-by-1 sparse coded vector assocated wth D. To keep the good dscrmnatve ablty of hghdmensonal feature representaton, x s then proected to the L-dmensonal sparse feature doman to generate the L- by-1 sparse coded vector x, and the coeffcents n x correspondng to the (L K) unselected bases n D are set to 0. Moreover, as K s usually smaller compared wth L, solvng (5) s faster than drectly solvng () due to the smaller sze of dctonary. 3. SUPERVISED WCE ORGAN CLASSIFICATION Table I. The dctonary learnng algorthm. Input: The gven SIFT matrx = [y 1,, y N ]. Intalzaton: Randomly generate L base vectors n D, each of whch s normalzed to a unt vector. Repeat 1: Fxng D, Eq. () s solved for each y to form a temporary X. : Fxng X, Eq. (3) s solved to get a temporary D. Untl the maxmum number of teratons s exceeded. Output: The fnal D s the dctonary we want. The WCE organ classfcaton s mplemented by a SVM classfer. Frstly, we need to prepare the tranng and testng samples of SVM. Each sample, whch s a feature vector denoted by z s, represents the fnal feature representaton of mage I s. The spatal pyramd matchng (SPM) proposed n [7] s adopted n the spatal poolng phase to get the fnal feature vector for each mage. The SPM method parttons an mage nto b b segments n dfferent scales b = 0, 1,. For each of the 1 subregons, the sparse coded vectors wthn t are pooled together wth a poolng functon to get the correspondng pooled sparse coded vector (PSCV). For ease of explanaton, let z s (p,q,b) denote the resultng PSCV of the (p,q)-th subregon n the b-th scale of I s. Careful evaluaton shows that the max poolng functon outperforms other alternatve poolng functons, such as the mean of absolute values and the mean square root. Hence, the spatal poolng n each subregon s based on the followng equaton: z ( p, q, b) max{ x, x,..., x }, 1,,..., L (6) s 1 T where zs ( p, q, b ) s the -th element of the PSCV z s (p,q,b), x t denotes the -th element of the t-th sparse coded vector n the (p,q,b)-th subregon, and T s the total number of sparse coded vectors wthn the subregon. L s the dmenson value of each sparse coded vector whch s equal to the number of bases n D. When the max poolng across all the subregons of a WCE mage s fnshed, all these PSCVs from 1 subregons are then drectly concatenated and normalzed to form the fnal feature vector z s of the mage I s. A multclass lnear SVM classfer s employed to classfy the WCE mages nto dfferent dgestve organs. We take the wdely used one-vs-all (OVA, or one-vs-rest) strategy to tran m bnary lnear SVM classfers for buldng the multclass lnear SVM classfer, where m s the number of classes that need to be classfed. 4. EXPERIMENTAL RESULTS 4.1. Expermental Setup As WCE s a medcal fronter technology, there s no publc WCE dataset for performance assessment. The dataset we used s provded by Shenzhen JFu Technology Ltd., whch was captured from 5 tral patents. The sze of each mage s of pxels. Four dgestve organs are consdered to 584

4 be classfed: esophagus, stomach, small ntestne, and colon. We collected 9,000 WCE mages by random selecton from the whole dataset, ncludng 1,000 esophagus mages,,000 stomach mages, 3,000 small ntestne mages, and 3000 colon mages. Each mage has been labeled by the provder wth the correspondng organ class for evaluatng the performance. To compare wth our proposed localty constrant based vector sparse codng (LCVSC) method, we also mplemented two very popular mage classfcaton methods. One s the method that usng tradtonal cluster-based vector quantzaton (K-means) to codng the nput descrptors, whch s proposed n [7] and termed as the CVQC method n ths secton. The other s the common L 1 -norm constrant based vector sparse codng method proposed n [11] as descrbed n Secton., whch s termed as L 1 -norm-vsc here. For performance comparson purpose, three methods follow the same framework of SIFT-Codng-SPM-SVM but wth dfferent codng algorthms. Smulaton parameters are set as follows. Each WCE mage s reszed to pxels usng the bcubc nterpolaton to reduce the computatonal cost. For CVQC [7], the codebook sze s fxed as 51 whch acheves optmal performance. For L 1 -norm-vsc [11] and our proposed LCVSC, the dctonary sze L s selected to be 104 as used n [11], and the free parameter used n Eq. () and Eq. (5) s set as 0.15 emprcally. For our proposed LCVSC, K s set to Expermental Results Followng the common benchmarkng procedure of multclass classfcaton, we repeat the classfcaton by 10 tmes wth dfferent random selecton of the SVM tranng and testng mages. The tranng set was formed by usng 50, 100, 00, and 400 mages per dgestve organ class respectvely. The testng set s formed by the rest. For each tral, per-class accuracy values are recorded and ther average value s computed. Then we report the fnal averaged classfcaton accuracy by the mean of the results from the ndvdual trals. The expermental results are shown n Table II. From the results, we can see that the ncrease of the SVM tranng sze leads to better classfcaton accuracy for all the three methods. For all the cases of tranng sze, the proposed LCVSC method outperforms CVQC by 4 ~ 6 percent, and outperforms L 1 -norm-vsc by more than 1 percent. To evaluate the mpact of the parameter K on the classfcaton performance of the proposed LCVSC, one experment s conducted. Fve dfferent values of K have been tested: 64, 18, 56, 51, and 104, and 400 tranng mages per organ class are used. The results are gven n Table III. It s clear to see that the classfcaton accuracy vares wth dfferent K values, and smaller K and bgger K gve nferor performance. The best selecton of K s 56 under ths expermental setup. Intutvely, f K s too small, Table II. Averaged classfcaton accuracy comparson. SVM tranng mages (per class) CVQC [7] 88.75% 91.93% 9.83% 94.67% L 1 -norm-vsc [11] 94.8% 95.46% 96.5% 97.78% LCVSC (proposed) 95.54% 96.6% 97.49% 98.79% Table III. The effects of the number of nearest-neghbors on the proposed method. K Accuracy 9.54% 97.9% 98.79% 98.13% 97.76% Table IV. The confuson matrx for CVQC/L 1 -norm-vsc/lcvsc. Predcted Small Esophagus Stomach Colon Class Intestne Esophagus 575/583/594 18/11/4 3//0 4/4/ Stomach 3/14/6 1490/1538/ / 3/4 /16/9 Small 11/4/0 3/10/7 545/58/590 1/4/3 Intestne Colon 53/10/1 39/9/6 107/4/15 401/557/567 the local base set D loses the ablty to represent the nput features whch leads to lower dscrmnant power; f K s too large, the localty constrant gets weaker and the LCVSC wll be close to L 1 -norm-vsc. To further demonstrate the classfcaton ablty of the algorthms, a confuson matrx between the four dgestve organs averaged over 10 trals for the three methods s computed and shown n Table IV. In ths experment, 400 tranng mages per class and K=56 are used. It s noted that there are 600 esophagus mages, 1,600 stomach mages,,600 small ntestne mages, and,600 colon mages n the testng set. In Table IV, each element records the number of correct classfcaton or msclassfcaton by usng the CVQC, L 1 -norm-vsc, and LCVSC respectvely. From the confuson matrx, we can see that the proposed LCVSC outperforms CVQC and L 1 -norm-vsc n almost every organ class. It s noted that the hghest number of msclassfcaton occurs between the adacent organs, such as stomach / small ntestne and small ntestne / colon, for all the methods. The proposed LCVSC obtaned the lowest msclassfcaton as well. 5. CONCLUSION A new method has been proposed to acheve good dgestve organ classfcaton accuracy for the large volume of the WCE mages under the SIFT-Codng-SPM-SVM framework, where the localty constrant based vector sparse codng (LCVSC) method has been developed to obtan better dscrmnatve feature representaton capacty. Intensve experments have been carred out to evaluate the classfcaton performance. It s encouragng to see that the proposed WCE dgestve organ classfcaton system s able to acheve the averaged classfcaton accuracy hgher than 95% for almost all testng trals. Future work wll focus on further reducng the computatonal cost. 585

5 REFERENCES [1] G. Iddan, G. Meron, A. Glukhovsky, et al., Wreless capsule endoscopy, Nature, Vol. 405, pp , 000. [] L. L, Y. X. Zou, and Y. L, Wreless capsule endoscopy mages enhancement based on adaptve ansotropc dffuson, n IEEE Chna Summt & Internatonal Conference on Sgnal and Informaton Processng (ChnaSIP), pp , 013. [3] J. Berens, M. Mackewcz, and D. Bell, Stomach, ntestne, and colon tssue dscrmnators for wreless capsule endoscopy mages, n Proc. of SPIE Conference on Medcal Imagng, Vol. 5747, pp , Bellngham, WA, Aprl 005. [4] J. P. S. Cunha, M. Combra, P. Campos, et al., Automated topographc segmentaton and transt tme estmaton n endoscopc capsule exams, IEEE Transactons on Medcal Imagng, Vol. 7, pp. 19-7, 008. [5] M. Mackewcz, J. Berens, and M. Fsher, Wreless capsule endoscopy color vdeo segmentaton, IEEE Transactons on Medcal Imagng, Vol. 7, No. 1, pp , 008. [6] J. Wang, J. Yang, K. Yu et al., Localty-constraned lnear codng for mage classfcaton, n IEEE Conference on Computer Vson and Pattern Recognton (CVPR), pp , 010. [7] S. Lazebnk, C. Schmd, and J. Ponce, Beyond bags of features: Spatal pyramd matchng for recognzng natural scene categores, n IEEE Computer Socety Conference on Computer Vson and Pattern Recognton (CVPR), pp , 006. [8] K. Mkolaczyk and C. Schmd, A performance evaluaton of local descrptors, IEEE Transactons on Pattern Analyss and Machne Intellgence, Vol. 7, No. 10, pp , 005. [9] X. Ma and W. E. L. Grmson, Edge-based rch representaton for vehcle classfcaton, n IEEE Internatonal Conference on Computer Vson (ICCV), pp , 005. [10] T. Ma, Y. X. Zou, and Q. Dng, Urban vehcle classfcaton based on lnear SVM wth effcent vector sparse codng, n IEEE Internatonal Conference on Informaton and Automaton (ICIA), pp , 013. [11] J. Yang, K. Yu, Y. Gong, et al., Lnear spatal pyramd matchng usng sparse codng for mage classfcaton, n IEEE Conference on Computer Vson and Pattern Recognton (CVPR), pp , 009. [1] H. Lee, A. Battle, R. Rana, et al., Effcent sparse codng algorthms, Advances n Neural Informaton Processng Systems (NIPS),