Level set-based histology image segmentation with region-based comparison

Transcription

1 Level set-based hstology mage segmentaton wth regon-based comparson Adel Hafane, Flz Bunyak, and Kannappan Palanappan Department of Computer Scence, Unversty of Mssour-Columba, Columba, MO USA Abstract Automated hstologcal gradng of tssue bopses for clncal cancer care s a challengng problem that requres sophstcated algorthms for mage segmentaton, tssue archtecture characterzaton, global texture feature extracton, and hgh-dmensonal clusterng and classfcaton algorthms. Currently there are no automatc mage-based gradng systems for establshng the pathology of cancer tssues. A prmary step n computatonal hstology s accurate mage segmentaton to detect sgnfcant regons such as nucle, lumen and epthelal cytoplasm whch together make up a gland structure. We descrbe a new approach for tssue segmentaton usng fuzzy spatal clusterng and level set actve contours. The proposed technque shows mprovement n segmentaton accuracy and outperform the classcal clusterng and level set methods, when compared to ground truth segmentaton. I. INTRODUCTION The avalablty of hgh resoluton multspectral multmodal magng of tssue bopses provdes a new opportunty to develop mproved tssue segmentaton algorthms for developng computer-aded dagnostc classfcaton of hstologcal mages n a clncal settng. Automated quanttatve gradng of prostate cancer tssue patches that s begnnng to compare favorably wth vsual analyss by experts for assgnng a Gleason grade to hstogologcal magery was demonstrated usng a combnaton of low level mage texture features and hgh level graph-based tssue archtecture features [1]. A multresoluton approach usng global texture features ncludng frst- and second-order statstcs combned wth a Gabor flter set was able to acheve over 90% overall accuracy n dstngushng between cancerous and bengn tssue, and nearly 77% n dstngushng between two complex grades of cancergleason grade 3 and 4 adenocarcnoma). However, the archtectural features of gland structures ncludng spatal dstrbuton of cell nucle and the arrangement of glands were determned usng manual segmentaton n[1]. Recently, sem-automated mage segmentaton algorthms requrng pror probablty estmates for the lumen structures and pxel-wse classfcaton was developed to facltate the extracton of spatal arrangement nformaton [2]. In ths paper, we develop a fully automatc robust mage segmentaton algorthm for hstopathology magery usng fuzzy spatal clusterng for class ntalzaton, tssue class refnement usng vector-based level sets or geodesc to accurately extract, nucle, lumen and epthelal cytoplasm regons. The proposed technques are compared wth related approaches usng qualtatve and quanttatve evaluaton method. The paper s organzed as follows: Secton II descrbes the Fuzzy C-means wth spatal constrant algorthm, Secton III presents the level sets multphase scheme. Secton IV descrbes the geodesc technque. The evaluaton process s explaned n secton V results are gven n Secton VI, and the conclusons n Secton VII. II. FUZZY C-MEANS WITH SPATIAL CONSTRAINT In ths secton we descrbe the method used to ntalze the level set procedure. The FCM algorthm mnmzes the objectvefuncton JU,V )whchsdefnedbythesumof smlarty measures. The objectve functon s gven by JU,V ) = =1 j=1 u m j x j v 2 1) where X = {x 1,x 2,...,x N }denotethesetofdatapxelfeaturevector). V = {v 1,v 2,...,v C }representstheprototypes, knownastheclusterscenters. U = [u j ]stheparttonmatrx whch satsfes the condton: u j = 1 j 2) m s a fuzzfer whch ndcate the fuzzness of membershp for each pont. FCM algorthm s based on an teratve process by mnmzng the dstance between each pont and the prototypes. The objectve functon Eq. 1 does not ncorporate any spatal nformaton. It s shown that the spatal nformaton brngs more robustness and effcency to the fuzzy c-means algorthm [3], where a second term to nclude the spatal nformaton s ncorporated n the FCM objectve functon. Ths s expressed by the followng equaton: J M U,V ) = + α =1 j=1 =1 u m j x j v 2 j=1 u m j exp k u m k) 3) where sasetofneghbors.theparameter αsaweghtthat controls the nfluence of the second termspatal nformaton). The objectve functon3) has two components. The frst componentsthesameasfcm,thesecondsapenaltyterm.ths component reaches a mnmum when the membershp value

2 of neghbors n a partcular cluster s large. The optmzaton of3)wthrespectto UhavebeensolvedbyusngLagrange multpler technque. J M U,V ) = + =1 u m k)) u m j x j v 2 + α exp j=1 k λ j 1 u j ) 4) j=1 =1 φ1 φ2 thedervatveof4)wthrespectto u j J M u j = mu j x j v 2 + α exp k u m k)) λ j 5) solvngfor u j wehave u j = λ j m x j v 2 + α exp k um k )) ) 1 solvngfor λ j wthrespecttotheconstrant2)weobtan =1 λ j m x j v 2 + α exp k um k )) ) 1 = 1 As λ j doesnotdependnthetermofthesumthsyeld λ 1 j = m x j v 2 + α exp u m k) k =1 ) 1 The obtaned membershp update functon s gven by u j = 1 ) C xj v 2 1 +α exp k um k ) p=1 x j v p 2 +α exp k um pk ) Theneghborngmembershpvaluesu pk )nfluence u j to follow the neghborhood behavor. For nstance f a gven pont has a hgh membershp value to a partcular cluster and ts spatal neghbors have a small membershp values to thscluster,thepenaltytermplaystheroletoforcethepont tobelongtothesameclusterastsneghbors.theweght α controls the mportance of the regularzaton term. The prototype update equaton s the same as standard FCM. v = N j=1 um j x j N j=1 um j 6) 7) 8) 9) 10) The spatal constrant FCMSCFCM) algorthm preforms the same steps as the orgnal fuzzy c-means algorthm but, the membershp functon s computed accordng to the equaton 9. Fg.1. Amult-contourlevelsetntalzatonwthtwolevelsetfunctons φ 1and φ 2 wthnooverlap),forsegmentngacolormage u 0. III. MULTIPHASE VECTOR-BASED ACTIVE CONTOURS In [4], Chan and Vese presented a multphase extenson of ther two-phase level set mage segmentaton algorthm [5]. The multphase approach enable effcent parttonng of the mage nto n classes usng just logn) level sets wthout leavng any gaps or havng overlaps between level sets. Ths ensures that each pxel s properly assgned to a unque class durng the segmentaton process. The Chan and Vese multphase level set mage segmentaton approach nvolves mnmzaton of a reduced or weak Mumford-Shah functonal F n c,φ),otherwsereferredtoasamnmalpartton Mumford-Shah functonal[6]: F n c,φ) = u 0 c ) 2 χ dx + 1 n=2 m λ }{{} 1 n=2 m µ Energy Term χ } {{ } Length Term 11) where, nsthetotalnumberofclassesassocatedwth mlevel set functons, u 0 s the gray-level mage beng segmented, Φsavectoroflevelsetfunctons, csavectorofmean gray-levelvalues.e., c = meanu 0 )ofclassorphase ), χ sthecharacterstcfunctonforeachclass represented by the assocated Heavsde functons Hφ ), and λ, µ ) are constants assocated wth each energy and length term ofthefunctonal F n c,φ).inordertosmplfycomputaton of the length term n the above reduced Mumford-Shah energy functon, we replace the measure of the characterstc functonsbythesumofthelengthofthezero-levelsetsof φ, 1 m µ Hφ ).Insteadofanunweghtedtotal length, ths approxmaton weghts some edges more than others, but s faster to compute and stll leads to satsfactory segmentaton results. Chan and Vese also extended ther two-phase level set mage segmentaton algorthm for scalar valued mages to vectorvalued mages such as color or multspectral mages[7] where the mage s parttoned nto pecewse constant vectors n the sprt of spatally-based vector quantzaton. In ths paper we combne the multphase approach wth the feature vector approach to handle both multple mage classes and vector-

3 valued magery such as segmentng color or multspectral mages. In the case of hstopathology magng derved from H&E staned cancer tssue bopses four mage classes have been shown to produce good feature sets for mage classfcatonbased cancer gradng[1]. For the two level set case.e., m = 2) the mage doman s parttoned nto at most fourclasses.let c = {c 00,c 01,c 10,c 11 }representthesetof average mage feature vectorse three channel color) wthn each class or phase c j, and Φ = φ 1,φ 2 ) represent the two level set functons. The multphase vector-based energy functonal F n c,φ)sdefnedas, F n c,φ) =λ 1 u 0 c Hφ 1 ))1 Hφ 2 ))dx + λ 2 u 0 c Hφ 1 ))Hφ 2 ) dx + λ 3 u 0 c 10 2 Hφ 1 )1 Hφ 2 )) dx + λ 4 u 0 c 11 2 Hφ 1 )Hφ 2 ) dx + µ 1 Hφ 1 ) dx + µ 2 Hφ 2 ) dx 12) The Euler-Lagrange equatons are obtaned by mnmzng Eq. 12andembeddng cand Φnadynamcalsystemas[4] dφ { 1 = δφ 1 ) µ 1 dv φ 1 ) dt φ 1 { λ 1 u 0 c 11 2 λ 3 u 0 c 01 2 )Hφ 2 ) + λ 2 u 0 c 10 2 λ 4 u 0 c 00 2 )1 Hφ 2 )) }}, dφ { 2 = δφ 2 ) µ 2 dv φ 2 ) dt φ 2 { λ 1 u 0 c 11 2 λ 2 u 0 c 10 2 )Hφ 1 ) + λ 3 u 0 c 01 2 λ 4 u 0 c 00 2 )1 Hφ 1 )) }} 13) where, c j s the mean vector of all pxel-based vectors assocated wth each class or phase. c 11 = u 0 Hφ 1 )Hφ 2 ) dx Hφ 1)Hφ 2 ) dx c 10 = u 0 Hφ 1 )1 Hφ 2 )) dx Hφ 1)1 Hφ 2 )) dx c 01 = u 0 1 Hφ 1 ))Hφ 2 ) dx 1 Hφ 1))Hφ 2 ) dx c 00 = u 0 1 Hφ 1 ))1 Hφ 2 )) dx 1 Hφ 1))1 Hφ 2 )) dx and δφ k ) = H φ k )sthedracdeltafuncton.fornumercal stablty of the delta functon, Chan and Vese propose usng a regularzed Heavsde functon wth H 2,ǫ x) = 1 2 [ π δ ǫ x) = 1 π { tan 1 x ǫ ǫ π 2 + ǫ 2 ) }] The motvaton for usng a multphase, rather than a two-phase, level set framework s to accurately detect adjacent regons that meetatajuncton.e.,thetrplejunctonn[4]).however,as the number of regons grows exponentally wth the number oflevelsetfunctons,tsbesttouseasmallsetoflevelset functonstypcally two or three or equvalently, four or eght regons, respectvely). IV. GEODESIC LEVEL-SETS SEGMENTATION In classcal level set-based geodesc actve contours[8], the level set functon φ s evolved usng the speed functon, φ t = gu 0)F c + Kφ)) φ + φ gu 0 ) 14) where F c saconstant, Ksthecurvatureterm, ) φ K = dv = φ xxφ 2 y 2φ x φ y φ xy + φ yy φ 2 x φ φ 2 x + φ 2 y) ) and gu 0 ) s the edge stoppng functon. Edge stoppng can be any decreasng functon of the mage gradent. For hstopathology mage segmentaton, we use Beltram color edge stoppng functon defned as where Es 1 + E = =R,G,B =R,G,B gu 0 ) = exp absdete))) 16) u 0, x ) 2 u0, u 0, u 0, x x y =R,G,B u 0, y 1 + =R,G,B ) 2 u0, y 17) The constant velocty F c pushes the curve nwards or outwards dependng on ts sgn. The regularzaton term K ensures boundary smoothness. The external mage dependent force gu 0 ) s used to stop the curve evoluton at object boundares.theterm g φsusedtoncreasethebasn of attracton for evolvng the curve to the boundares of the objects. The classcal geodesc actve contours s two phase and can segmentanmagentoonlytwoclasses.inordertosegment the three-class hstopathology mages we use two level sets, frstlevelsetsegmentsthelumenregonsfromtherestof the mage, second level set segments nucle regons from the rest of the mage. Both level sets are ntalzed outsde of ther regon of nterestlumen and nucle regons respectvely) andtomovenwards.scfcmmasksectonii)susedfor ntalzaton, two bnary masks one for lumen one for nucle classes are produced from the mult-class SCFCM mask1- for lumen 0-for everythng else and 1-for nucle and zero for everythng else respectvely). Both masks are dlated wth a large enough structurng element to ensure that they fully contan the regons of nterests. The geodesc actve contours are ntalzed from the dlated masks.

4 V. EVALUATION To evaluate the segmentaton accuracy we use supervsed crteron [9]. The crteron s based on regon overlappng where each regon from the segmented mage s overlapped wth each regon of the ground truth mage. The hgh percentageoftheoverlappngndcatehowmuchthetwomagesare close comparng each other. However the under-segmentaton can provde large overlappng areas whch ntroduce mss estmaton n segmentaton qualty. The used crteron take nto account ths aspect and penalze ether the under-segmentaton or over-segmentaton. The evaluaton measure ncorporate the followng ponts: Localzaton: the detected regons should be spatally coherenteg. poston, shape, sze...) wth those present n the reference, Over-segmentaton: ths stuaton s consdered as dsturbngandhastobepenalzednthequaltyndex, Under-segmentaton: ths stuaton s consdered as a segmentaton error and has also to be penalzed. Let R Ref and R Seg j be two classes belongng respectvely to thereference I Ref andtothesegmentatonresult I Seg = 1..NR Ref, j = 1..NR Seg where NR Ref sthenumberof regonsofthereferenceand NR Seg thenumberofregonsof thesegmentatonresult.thematchngndex M I soverall regons s gven by: M I = j,max CardR Ref R Seg ) j CardR Ref CardR Ref R Seg j ) R Seg j ) ρ j 18) where CardX)sthenumberofpxelsof X.Theweghtng factor ρ j expressesthemportanceoftheregon jnthemage and permts to gve to small regons less nfluence n the qualty measure. ρ j = CardRSeg j ) CardI Seg ) 19) The equaton 18) express a morphologcal relaton between two regons/classes. Each class of a segmentaton result s compared wth the correspondng one n the reference by takng nto account the maxmum overlappng surface.fornstance,ftherearetworegonsn I Seg that ntersectwtharegonof I Ref,themeasureconsdersthe maxmum of ntersecton. However, the no perfect matchng spenalzedbythenormalzatonterm CardR Ref Inthecaseoftheperfectmatchngthendex M I sequalto1. R Seg j ). The over- and under-segmentaton errors are descrbed by NR Ref /NR Seg f NR Seg NR Ref η = log1 + NR Seg /NR Ref ) otherwse 20) The fnal evaluaton crteron H s then gven by the followng equaton: H = M I + m η 1 + m 21) where m s a weghtng coeffcent whch controls the mportance of the over-/under-segmentaton errors n the judgment. As shown n [9] m = 0.2 realze a good trade off between between dfferent evaluaton parameters. For the all experment m s set to 0.2. VI. RESULTS AND DISCUSSION The experments have been performed over hstopathology mages 1. We appled clusterng based methods, Level set based actve contours segmentaton methods and combnaton of both. The tested clusterng methods are: K-means, Fuzzy C-means,andSCFCM seesectonii).wehaveusedtwo categores of level sets methods mult-phase descrbed n secton III and geodesc descrbed n secton IV. Fgure 2 shows the segmentaton results of the dfferent technques. Fgure 2a) represents the orgnal mages Gleason Grade 3 tssue composed of nucle, lumen, and epthelal cytoplasm. Next, fgure 2b) the manual segmentaton wth 3 classes: red nucle), greenlumen) and yellowcytoplasm). The result of k-means and FCM algorthms are shown respectvely n fgure 2c) and fgure 2d). The two algorthms provde smlar results. Both methods produce more lumen regons compare to the ground truth. Ths s partly because some small domnantly whte regons wthn the cytoplasm are not fnely classfed n the coarse manual segmentaton and partly because both K-means and Fuzzy C-means do not ncorporate any spatal nformaton and tend to fragment regons. The SCFCM overcome the latter defcency usng spatal correlaton to reduce ths effect as shown n fgure 2e), the segmented regons look more spatally coherent but the segmentaton s stll poor comparng to the ground truth. Both of the level set-based segmentaton approaches geodesc fgure 2f) and vector multphasefgures 2g), 2h)) mprove the segmentaton qualty and provde more accurate regons. But for hstopathology mage segmentaton, vector multphase level s more relable and robust compared to geodesc actve contours. Geodesc actve contours s more senstve to ntalzatonshould start ether completely nsde or outsde of the regons of nterest). It suffers from contour leakng on weak edges e. some nucle edges), and early stoppng on background edgese. cytoplasm texture). Due to these problems some sgnfcant nucle regons dsappear fgure2f)). The multphase vector-based level setmvls) s used wth two dfferent ntalzaton concepts: random crclesfgure 1) and SCFCM segmented regons. As expected MVLS combned wth SCFCMfgure 2h)) produces better results than MVLSfgure 2g)) wth random ntalzaton specally n the lumen regons. MVLS-SCFCM segmentaton results appear more accurate than the other algorthms comparng to the orgnal mage and the ground truth. Statstcs of algorthms comparson s explaned n the next paragraph. Table I compares performance of dfferent segmentaton methodsforeachclassntermsofareapercentageeq.22, 1 HstopathologymageryprovdedbyMchaelFeldmanDept.ofSurgcal Pathology, Unv. of Pennsylvana) and ground truth from Anant Madabhush Rutgers).

5 Method Area% Overlap% Normalzed overlap% Class1 Class2 Class3 Class1 Class2 Class3 Class1 Class2 Class3 Groundtruth K-means FCM SCFCM Geodesc-SCFCM MVLS-random MVLS-SCFCM TABLE I COMPARISON STATISTIC MEASURES a) Grade 3 b) Gound truth c) K-means d) FCM e) SCFCM f) Geodesc g) MVLS Random h) MVLS SCFCM Fg. 2. Automatc segmentaton of Gleason grade 3 hstopathology mage wth nucle shown n red, lumen n green, epthelal cytoplasm n yellow.

6 overlap percentage Eq. 23, and normalzed overlap percentage Eq. 24. These addtonal measures comes to enforce the evaluaton crtera descrbed n Secton V. In table I Class 1, Class 2, and Class 3 represent nucle, lumen and cytoplasm classes respectvely. Area = cardr ) ) Total number of ponts CardR Ref R Seg ) Overlap = ) Total number of ponts Normalzed overlap = CardRRef CardR Ref R Seg ) R Seg ) ) Dstrbuton of the dfferent classes n the reference mage s:nucle29%,lumen11%,andcytoplasm60%.forthenucle class all tested algorthms provde areas smlar to the area n the reference mage. The major dfference resde n lumen and cytoplasm classes due to the addtonal small lumen regons detected. Geodesc-SCFCM produces the best results for the area measure, followed by MVLS-SCFCM. The overlap measures between ground truth class and the correspondng one n the automatc segmentaton are gven n thrd major column of the table I. There are hgh overlap percentages for nucle, but much less for lumen and cytoplasm. Level set methods provdes better spatal precson partcularly for lumen class. They tend to shrnk lumen and nucle regons, ths reduces the overlap area wth the reference mage. Although the low percentage of overlappng n lumen area, level set methods provdes better results n terms of accuracy and localzaton. ThssshownnthenextfourthmajorcolumnoftableI.For the nucle class all the methods presents good performances 64% n average. The lumen class produces low percentage 27% for K-means and FCM, 31% for SCFCM, Geodesc- SCFCM wth 48%, MVLS-Random 40% and MVLS-SCFCM provdes 43%. For cytoplasm class the level set algorthms gve better results an average of 64%. As more relable measures, we compute the crtera H Eq.21and M I Eq18seesectonV).Thesecrteraarenot symmetrc, so the measure s appled n two dfferent ways, automatc segmented mageseg) versus groud truthgt) and vce versa. Table II shows the segmentaton qualty compared to the reference mage over all classes. Geodesc-SCFCM gveshghervaluesfor M I measureproducng78%closerto the reference mage. But when the over-under segmentaton penalty s ntroduced n H crteron, MVLS-SCFCM outperforms all the tested methods. VII. CONCLUSION In ths paper we descrbed a robust algorthm for fully automatc tssue segmentaton of glandular structures n hstopathology magery. An accurate unsupervsed ntalzaton s provded usng the spatal constrant fuzzy c-means developed prevously by our group. The ntal mage clusters whch may not be spatally contguous wth bologcal regons of nterest are refned usng an extended actve contour algorthms to handle complex bologcal structures n color magery. We evaluate the segmentaton accuracy accordng to the manual segmentatonground truth). The proposed method outperforms the classcal methods for all classes of regons, nucle, lumen and cytoplasm. The qualty of segmentaton s mportant snce t wll be used for tssue classfcaton n further future work. REFERENCES [1] S. Doyle, M. Hwang, K. Shah, A. Madabhush, M. Feldman, and J. Tomaszewesk, Automated gradng of prostate cancer usng archtectural and textural mage features, n IEEE Int. Symp. Bomedcal Imagng: From Nano to Macro, Aprl 2007, pp [2] S. Nak, S. Doyle, M. Feldman, J. Tomaszewsk, and A. Madabhush, Gland segmentaton and computerzed Gleason gradng of prostate hstology by ntegratng low-, hgh-level and doman specfc nformaton, n Proc. 2nd MICCAI Workshop Mcroscopc Image Analyss wth Appl. n BologyMIAAB), Pscataway, NJ, USA, [3] A. Hafane, B. Zavdovque, and S. Chaudhur, A modfed FCM wth optmal Peano scans for mage segmentaton, n IEEE ICIP, Genova, Italy, [4] L.VeseandT.Chan, Amultphaselevelsetframeworkformage segmentaton usng the Mumford and Shah model, Int. J. Computer Vson,vol.50,no.3,pp ,2002. [5] T. Chan and L. Vese, Actve contours wthout edges, IEEE Trans. ImageProc.,vol.10,no.2,pp ,Feb [6] D. Mumford and J. Shah, Optmal approxmatons by pecewse smooth functons and assocated varatonal problems, Comm. Pure Appl. Math., vol. 42, pp , [7] T. Chan, B. Sandberg, and L. Vese, Actve contours wthout edges for vector-valued mages, J. of Vsual Communcaton and Image Representaton, vol. 11, pp , [8] V.Caselles, R.Kmmel, and G.Sapro, Geodesc actve contours, Int. J. Computer Vson, vol. 22, no. 1, pp , [9] A. Hafane, S. Chabrer, C. Rosenberger, and H. Laurent, A new supervsed evaluaton crteron for regon based segmentaton methods., n ACIVS. 2007, vol of Lecture Notes n Computer Scence, pp , Sprnger. Algorthm M I % H% M I % H% Seg GT Seg GT GT Seg GT Seg K-means FCM SCFCM Geodesc-SCFCM MVLS-random MVLS-SCFCM TABLE II SEGMENTATIONEVALUATIONUSING M AND HCRITERIA