Weighted Voting in 3D Random Forest Segmentation

Transcription

1 Weighed Voing in 3D Random Fores Segmenaion M. Yaqub,, P. Mahon 3, M. K. Javaid, C. Cooper, J. A. Noble NDORMS, Universiy of Oxford, IBME, Deparmen of Engineering Science, Universiy of Oxford, 3 MRC Epidemiology Resource Cenre, Universiy of Souhampon Absrac. he radiional random foress echnique has shown good classificaion accuracy for D objec segmenaion in naural images. However, he echnique suffers from a few problems when exending i o 3D or 4D images which are of grea ineres in biomedical image analysis. In his paper, we develop an auomaic 3D random foress mehod which is applied o segmen he feal femur in 3D ulrasound. he proposed echnique rains balanced rees from imbalanced daa. A weighed voing mechanism is proposed o generae he probabilisic class label. A cross validaion on 0 3D feal ulrasound volumes shows promising resuls. Experimens show ha our echnique achieves segmenaion and measuremens close o he accuracy of exper delineaions. he mehod runs in a few seconds on a sandard PC and hence is well-suied for clinical applicaions. Inroducion he novely of his work is o exend he convenional Random Foress [] (RF) echnique o provide an efficien mehod for 3D or 4D image segmenaion. In addiion, we provide a robus esing by weighing he class decision of each ree. he convenional RF echnique has already been used o segmen D images [, 3] bu grea ineres in medical image analysis raise he issue of having such echnique o accuraely and efficienly segmen volumeric objecs. 3D feaures are required o represen a 3D objec of ineres herefore we illusrae how o exend several feaures o 3D efficienly. he echnique has been validaed and applied o segmen he feal femur in 3D ulrasound images alhough our echnique is equally applicable o oher problems. Manual measuremens can be inaccurae, edious and ime consuming. Anoher major problem wih manual segmenaion is inra and iner-observer reproducibiliy. he problem becomes harder when measuring volumeric srucures where he errors propagae. herefore, here is an urgen need o auomae his process, enhance reproducibiliy and minimize he source of errors. Recenly, learning-based echniques have been proposed for segmenaion. Random foress [] is a learning-based echnique in which raining using a gold sandard segmenaion is done by building muliple decision rees in which every node excep he leaves is a decision node ha conains a feaure (his is called a variable in saisics erms) and is corresponding hreshold. Every leaf node conains a probabilisic class disribuion (hisogram of class labels for he voxels ha have reached ha node). esing is performed by raversing voxels over he rees saring from he roo of each ree o a leaf node. he voxels are spli a a given node depending on he classificaion of he feaure/hreshold a ha node. he average probabilisic decision of he class disribuion from all rees is considered he final probabilisic class disribuion of he es case (voxel label in his scenario). For more informaion see [-3] and Fig.. RF can achieve comparable accuracy o boosing while being faser [4]. In addiion, randomness in ) choosing a sample raining se for each ree and ) choosing a subse of feaures o ry a each node provides beer generalizaion and helps avoid over-fiing. RF has also shown o have robusness o noise and ambiguiy beween classes in he raining daa which makes he echnique suiable o segmen ulrasound daa []. If an equal voe from each ree is used, he decision can be biased by he srengh of he classifiers on he decision node pah (see green nodes in Fig. ). For example, if he fores has 0 rees and he firs one has a max deph of 5 while he oher 9 rees have a max deph of hen he decision made by he firs ree depends on up o 5 classifiers which may provide a poor accuracy compared o he rees which have up o classifiers on every pah. In addiion, he accuracy of each classifier affecs he decision. his implies ha i would be a beer sraegy if each ree conribues a weighed voe oward he final decision.

2 ... Decision node Decision node pah Leaf node (class disribuion) Classified voxel Fig.. Random foress which conain decision rees. Decision is made as a combinaion of class disribuion (red circles wih green ouline) from every ree ( i ). Mehod. Problem Descripion he ulimae goal of his work is o exend he radiional RF echnique o 3D image segmenaion and provide robus and meaningful 3D feaure ses ha can be compued efficienly in 3D. In addiion, we provide a weighed decision ha depends on he srengh of he feaures used in each ree.. 3D Feaure Ses Each node in he classificaion ree in he RF framework is a classifier. he classifier is in realiy a feaure and is hreshold. In convenional RF, n' feaures are randomly seleced ou of n feaures in he pool (in 3D images, n can be a very huge number, e.g., 0 8 ). his sub-secion describes how o creae his feaure pool. Several challenging properies of ulrasound daa like shadowing, speckle and oher arifacs make he problem hard. herefore, "inelligen" feaures are required o capure all variaions. Several feaure ses are consruced for a given image. We use he phrase "feaure se" o denoe he group of feaures of he same ype bu wih differen window sizes and locaions around a Voxel Of Ineres (VoxOI). Unary3D, binary3d, recangle3d [3], Haar3D [5, 6] feaure ses are used and averaged recangle3d and posiion3d feaure ses are proposed. Fig. illusraes some of hese feaure ses. hese feaures are exraced from image voxels. A unary3d feaure is he inensiy value of a random voxel wihin a random size window around VoxOI. A binary3d feaure is he sum, difference or absolue difference of wo random voxels in a random window around VoxOI [3]. A recangle3d feaure is he sum of all voxels of a random size recangular cuboid saring from a random coordinae around he VoxOI. hese feaure ses have shown good performance in naural image segmenaion [, 3]. We exend he D inegral images [7] o 3D, see equaions () and (), o find he 3D recangular sum efficienly. Noice ha he 3D inegral image can be efficienly compued in one pass. Since recangle3d feaures depend on he size of recangular cuboid which is biased o is dimensions we propose an averaged recangle3d. he averaged recangle3d feaure se is acually a unary3d feaure se of he sub-sampled image in a muli-resoluions image. Haar3D feaures are used o capure edge regions [6]. Finally, posiion3d feaures are used o capure he spaial locaions of he voxels. his feaure se helps discard many regions ha have similar inensiy and edge informaion o he objec of ineres (e.g., in our case allowing o disinguish he femur from oher srucures like ibia). In a 3 D img i : ii ( x, y, z) i( x, y, z) () x y z i j k 3D rec sum ( x, y, z ), ( x, y, z ) ii ( x, y, z ) ii ( x, y, z ) ii ( x, y, z ) ii ( x, y, z ) ii ( x, y, z ) ii ( x, y, z ) ii ( x, y, z ) ii ( x, y, z ) () In each feaure se many feaures exis. For insance, in a recangle3d feaure se 6 feaures can be generaed wih a maximum recangle3d size of (,, ) saring from a random voxel wihin a window of a maximum size (,, ) around VoxOI. Calculaing such feaures in 3D requires considerably more ime han in D. In addiion, many of hese feaures are redundan and many are poor o be used for classificaion. herefore, a weighed decision from each ree should give a more accurae classificaion.

3 (a) binary (b) Recangle (c) Verical Haar Fig.. Examples of he feaure ses. he green color voxels are summed and subraced from he summed red voxels..3 raining Random Foress In he radiional RF, he raining phase proceeds by building randomized decision rees. he number of rees is se before hand. A op-down consrucion for every ree is performed saring from he roo node. Each ree is rained on a random se of he raining poins wih replacemen. For each node in he ree n' feaures from he feaure pool are randomly seleced wihou replacemen. he "bes" feaure ou of n' wih he "bes" hreshold is seleced as a classifier in he ree node. Informaion gain is usually used o decide he performance of a classifier. he raining se is hen divided ino wo sub-ses according o he resuls of he classifier o lef and righ branches. he same process is coninued recursively for each sub-se unil he maximum ree heigh is reached or no more gain is achieved. For more informaion see []. Afer rees consrucion, every leaf node conains a probabilisic class disribuion P(c i l) for each class which is he hisogram of he raining examples of class label c i ha reached leaf node l. p.4 Segmenaion & Measuremens In radiional RF, classifying new voxels proceeds by esing each voxel on he feaures/hresholds for every ree saring from he roo o a leaf node. he probabiliy for a voxel v i o belong o a specific class c j is he percenage of voxels of class c j ha reached he leaf node wih respec o all voxels reached i (v leaf ) during raining (3). he probabiliies from all rees are averaged o generae he final probabilisic decision of a voxel v i belonging o a class c j (4). p( c j v, ree i his ( v c, l ) leaf j ) (3) p( c v, RF ) i p( c v, ree ) i (4) his ( v, l ) leaf Here is he number of he rees and l is a leaf node a ree. One major issue is he equal voe (/) from each ree where some rees may provide a bad classificaion accuracy. One soluion could be o increase he number of rees bu his significanly increases he raining and esing ime in he RF. herefore, we propose a weighed voing in which he voe is weighed depending on he feaures used in each ree saring from he roo unil he leaf node. he decision from each ree is based on he classificaion accuracy of he nodes visied for every voxel v i. o embed his ino he RF framework, a weighed sum of rees probabiliies is proposed and equaion (4) is generalized o (5). ( c v, RF ) i p( c v, ree ) i (5) where Score ( ree ) f F f (6) F Score ( ree ) f i F F i f Here F is he oal number of feaures on he pah from he roo o he leaf when classifying a voxel and Score f (ree ) is he raining score of a feaure f on a pah a ree. Finally, he volume of he segmenaion of class c j is easily found by muliplying he number of segmened voxels by he voxel spacing..5 Pos-processing his sep is applicaion-specific and is mainly applied here o rejec regions wih similar local shape and inensiy disribuion o he objec of ineres. Alhough RF provides good

4 classificaion accuracy, i is a discriminaive model ha capures local similariies. As a resul, any srucure which looks similar in inensiy disribuion and local shape can be regarded as he objec of ineres. A posiion feaure se is added o rejec such regions. Unforunaely, in our specific applicaion some femur like srucures are close o he femur and herefore posiion feaures may no be able o disinguish beween he wo (e.g., he femur is conneced o ibia via he knee ligamens). o accommodae his, he larges 3D conneced componen was auomaically seleced (he femur). 3 Experimenal Resuls Several measuremens of feal srucures from D ulrasound images are imporan o diagnose he growh of he feus and esimae gesaional age and birh weigh [8, 9]. Clinicians usually measure head circumference, biparieal diameer, abdominal circumference and femur lengh. Several research groups have sudied and manually measured he feal femur [8-0]. hey have mainly focused on measuring femur lengh o correlae i wih gesaional age or birh weigh. Several research groups have ried o auomae he process of segmening and measuring such srucures in D ulrasound images [5, ]. o our knowledge we are he firs o invesigae he problem of auomaic femur volume segmenaion in 3D ulrasound images. 3. Daase We esed our echnique on 0 3D ulrasound volumes [9] acquired on 9 weeks feuses ±6 days using a GE Voluson 730 scanner. Volumes dimensions are approximaely wih a ( ) mm 3 voxel spacing. Alhough ou-of-bag error esimae can be used as a classificaion error measure [], cross validaion was performed on he 0 volumes by using 8 images for raining and wo for esing. Cross validaion provides a more general and realisic error measure compared o he ou-of-bag error in his applicaion. 3. Validaion mehodology Experimens on he radiional and weighed RF are repored o suppor he proposed echnique. RF requires several parameers o be se. he parameers were fixed for all experimens ( = 0, max-ree-deph = 0, n' = 00). Recall and precision were calculaed o measure how well he segmenaion of he proposed echnique compared o an exper manual segmenaion according o (7) and (8) respecively. Recall = P P FP (7) Precision = P P FN (8) Where: P is rue Posiive FP is False Posiive FN is False Negaive Recall and precision comparisons of he 0 volumes for he radiional RF and he weighed RF are shown in able. Noice ha he higher he recall he closer he segmenaion is o he ground ruh. Bland-Alman plos for he volume measuremens o compare he manual segmenaion and he radiional and weighed RF echniques show ha he weighed RF has he minimum bias and ighes sandard deviaion bounds (Fig. 3). Visual comparisons beween he manual segmenaion and he boh RF mehods are shown in Fig. 4. he raining and segmenaion imes for he RF echnique are shown in able. hese imes are for one experimen where 8 ulrasound images were used for raining and one for esing, =0, n'=00, Max-ree-deph=0, n~=8*0 6. Fig. 3. Bland-Alman plos for he segmened femur volumes. Lef: manual vs. radiional. RF. Righ: manual vs. weighed RF.

5 able. Recall & precision for he radiional and weighed RF mehods. µ±σ Recall µ±σ Precision rad. RF 64%±8% 88%±% Weighed RF 70%±5% 88%±% able. raining & es ime for radiional and weighed RF mehods. raining ime (Hours) Segmenaion ime (Sec) rad. RF 8 3 Weighed RF 8 Fig. 4. A D slice of he segmenaion using 3D random foress. op is an original longiudinal (lef) and ground ruh (righ). Boom is he segmenaion using radiional RF (lef) and weighed RF (righ). 4 Conclusions & Fuure work In his paper, he RF echnique has been exended from he radiional D RF o 3D. We have shown ha using weighed class decision from each ree in RF ouperforms he convenional mehod. he echnique has shown good accuracy and performance on he problem of feal femur segmenaion in 3D ulrasound daa. Validaion has been performed on a good size daase which showed promising resuls. One major issue o consider is o eliminae irrelevan feaures in he huge feaure pool. his will heoreically provide beer classificaion accuracy. A second issue is how o inegrae global shape informaion in he RF framework since RF mainly capure he local shape informaion of he objec of ineres. Researchers have looked ino his issue by applying a generaive model o he resuls of he discriminaive model (e.g., Boosing and is variaions, RF, ec...) []. Specific o our applicaion, he feaure se could also be exended o accoun for he signal aenuaion for boh he disal and proximal ends of he femur. We also plan o sudy he inra and iner-observer reproducibiliy by doing muliple manual segmenaions from muliple expers. Finally, his approach is general and no resriced o he femur or indeed ulrasound. We plan o look a oher applicaions oo. References [] L. Breiman, Random Foress, Machine Learning, vol. 45(), pp. 5-3, 00. [] F. Schroff e al., Objec Class Segmenaion using Random Foress, in BMVC, 008. [3] J. Shoon e al., Semanic exon Foress for Image Caegorizaion and Segmenaion, in CVPR, 008. [4] R. Caruana, and A. Niculescu-Mizil, An Empirical Comparison of Supervised Learning Algorihms, in ICML, 006. [5] G. Carneiro e al., Deecion and measuremen of feal anaomies from ulrasound images using a consrained probabilisic boosing ree, IEEE MI, vol. 7(9), pp , 008. [6] M. Oren e al., Pedesrian deecion using wavele emplaes, in IEEE CVPR, 997, pp [7] P. Viola, and M. Jones, Robus Real ime Objec Deecion, IJCV, 00. [8] L. S. Chiy, and D. G. Alman, Chars of feal size: limb bones, Briish Journal of Obserics and Gynaecology, vol. 09, pp , 00. [9] P. A. Mahon, Ulrasound assessmen of feal musculo-skeleal developmen, PhD. hesis, Universiy of Souhampon, Souhampon, 007,PhD hesis. [0] C.-H. Chang e al., Prenaal Deecion of Feal Growh Resricion by Feal Femur Volume: Efficacy Assessmen Using hree-dimensional Ulrasound UMB, vol. 33(3), pp , 007. [] S. M. Jardim, and M. A. Figueiredo, Segmenaion of feal ulrasound images, UMB, vol. 3(), pp , 005. [] Z. u e al., Brain anaomical srucure segmenaion by hybrid discriminaive/generaive models, IEEE MI, vol. 7(4), pp , 008.