How can physicians quantify brain degeneration?

Transcription

1 Chapter 11 How can physcans quantfy bran degeneraton? If the human bran were so smple that we could understand t, we would be so smple that we couldn't E. M. Pugh M. Bach Cuadra( ), J.-Ph. Thran( ), F. Marques(*) ( ) Ecole Polytechnque Fédérale de Lausanne, Swtzerland (*) Unverstat Poltècnca de Catalunya, Span Lfe expectaton s ncreasng every year. Along wth ths agng populaton, the rsk of neurologcal dseases (e.g. dementa 1 ) s consderably ncreasng 2 as well. Such dsorders of the human nervous system affect the patent from both a physcal and socal pont of vew, as n the case of Alzhemer s dsease, one of most known bran dsorders (Mazzotta, Toga et al. 2000). To pose ther dagnoss, n addton to the usual clncal exam, physcans ncreasngly rely on the nformaton obtaned by means of 2D or 3D magng systems: the so-called mage modaltes. Nowadays there exsts a wde range of 3D medcal mage modaltes that allow neuroscentsts to see nsde a lvng human bran. 3D magng s of precous help snce t allows, for nstance, the physcan to better localze specfc areas nsde the 1 Dementa s a progressve degeneratve bran syndrome whch affects memory, thnkng, behavor and emoton. 2 In 2006, 24.3 mllon people n the world have dementa, wth 4.6 mllon new cases annually. By 2040 the number wll have rsen to 81.1 mllon. Source:

2 2 M. Bach Cuadra, J.-Ph. Thran, F. Marques body and to understand the relatonshps between them. Among all medcal mage modaltes, Magnetc Resonance Imagng (MRI) has been ncreasngly used snce ther nventon n 1973 by P. Lauterbur and Sr P. Mansfeld 3. As a matter of fact, MR magng has useful propertes: t provdes an anatomcal vew of the tssues and deep structures of the body (see Fg. 11.1) thanks to the magnetc propertes of the water, wthout the use of onc radaton (Haacke, Brown et al. 1999). Fg D MR mage of a human bran: coronal, sagtal, axal and 3D vews. The red (dark grey) cross ndcates the same spatal pont n the three vews. For a long tme MRI has been used qualtatvely, for vsualzaton purpose only, to help physcans n ther dagnoss or treatment plannng. For nstance when physcans look at MR mages to evaluate lesons or tumors, they frst segment the suspcous growth mentally and then use ther tranng and experence to assess ts propertes. However, due to several factors such as the ever-ncreasng amount of clncal data generated n clncal envronments and ther ncreasngly complex nature, t s nowadays necessary to rely on computerzed segmentaton technques (Sur, Wlson et al. 2005). By segmentaton we mean dvdng an mage nto homogeneous regons for a set of relevant propertes such as color, ntensty or texture whch allow a quanttatve analyss of medcal mages (automatc estmaton of quanttatve measures and regons of nterests). In the specfc case of bran dsorders, accurate and robust bran tssue segmentaton from MR bran mages s a key ssue for quanttatve studes (Bach Cuadra, Cammoun et al. 2005). Studyng MR bran mages of such patents s a key step, frst to dfferentate from a normal tssue loss due to normal agng; second, to better understand and characterze the anatomy of such patents; and thrd, to better assess drug treatments. Thus, there s a need n quantfyng MR mage changes to determne where the tssue loss s and ts amount. 3 They were awarded the 2003 Nobel Prze n Medcne or Physology for ther dscoveres concernng MRI.

3 How can physcans quantfy bran degeneraton? 3 In ths Chapter we wll address the problem of quantfyng bran degeneraton n a patent sufferng from focal bran degeneraton, causng a progressve aphasa. To do so, we wll do a longtudnal study, that s, we wll compare two MR mages from the same patent acqured n a year nterval (I1 n 2000 and I2 n 2001). Frst, the mage classfcaton problem must be solved for every 3D mage acquston (see Fg. 11.2). Second, to detect and quantfy the tssue loss we need to compare the I1 segmentaton wth the I2 segmentaton n a pont to pont bass. Therefore, an mage regstraton problem arses: a pont-to-pont correspondence s not ensured between both MR scans snce the patent has not been lyng n the same poston durng the two acquston process. Ths can be solved by a rgd regstraton process whch looks for the spatal transformaton that wll brng one mage to a pont-to-pont correspondence wth the other one. In ths applcaton, only translaton and rotaton are needed to compensate for a dfferent poston of the patent n the two scans. Even n such a smplfed framework (only translaton and rotaton), mage regstraton s a very complex problem (Hajnal, Hll et al. 2001) whose complete dscusson s out of the scope of ths Chapter. For the sake of completeness, the regstraton problem s brefly addressed n Secton Fg Image processng tools presented n ths Chapter

4 4 M. Bach Cuadra, J.-Ph. Thran, F. Marques The Chapter s organzed as follows. In Secton 11.1 we present the theoretcal framework to solve the mage classfcaton problem. In Secton 11.2 a MATLAB proof of concept for the mage classfcaton wll be developed. As prevously commented, the problem of mage regstraton s brefly dscussed n Secton Fnally, n Secton 11.4 a dscusson concludes the Chapter Background 1 Statstcal Pattern Recognton for Image Classfcaton Numerous approaches have been proposed for mage classfcaton. They can be dvded nto two man groups: () supervsed classfcaton, whch explctly needs user nteracton, whle () unsupervsed classfcaton s completely automatc. In ths Chapter we descrbe an unsupervsed statstcal classfcaton approach. In fact, ths can be seen as an estmaton problem: estmate the underlyng class (hdden data) of every voxel (3D extenson of the pxel concept) from the 3D ntensty mage (observed data). Ths concept s llustrated n Fg Fg Classfcaton problem seen as an estmaton problem. MR mage ntenstes (on the left) are the observed data. The labeled mage (on the rght sde) s what we are lookng for: blue (dark grey) voxels are the Cerebrospnal Flud class (CSF), red (medum grey) voxels are Grey Matter (GM), and green (lght grey) voxels are Whte Matter (WM).

5 How can physcans quantfy bran degeneraton? Statstcal framework Formally, let us consder N data ponts (typcally voxels) and the observed data features y, wth S {1,2,..., N } 4. In the MR mage case, Y s the random varable assocated to the ntensty of voxel, wth the set of possble outcomes D and y represents the observed ntensty of voxel. The ensemble of all random varables defnng the MR mage random process s denoted by Y and any smultaneous observaton of the random Y, (that s, any observed MR mage) s denoted by 1 2 N N y y, y,, y D (11.1) N The classfcaton process ams at classfyng the data nto one of the hdden underlyng classes labeled by one of the symbols L = {CSF, GM, WM}. CSF stands for Cerebrospnal Flud (blue or dark grey voxels n Fg. 11.3), GM for Grey Matter (red or medum grey voxels n Fg. 11.3), whereas WM for Whte Matter (green or lght grey voxels n Fg. 11.3). X s the random varable assocated to the class of voxel, wth the set of possble outcomes L, and x represents the class assgned to voxel. The ensemble of all random varables defnng the hdden underlyng random process s denoted by X and any smultaneous confguraton of the random varables X, (that s, any label map) s gven by 1 2 N x x, x,, x L (11.2) In ths Chapter, we are lookng for the most probable label map x* gven the mage observaton y. As already ntroduced n Chapter 4, ths can be expressed mathematcally by the so-called Bayesan, or Maxmum a Posteror (MAP), decson rule as 5 : N 4 S represents the grd n whch voxels are defned and, therefore, s a specfc ste n ths grd 5 As n Chapter 4, here we wll often use a shortcut notaton for probabltes, when ths does not brng confuson. The probablty P(A=a B=b) that a dscrete random varable A s equal to a gven the fact that random varable B s equal to b wll smply be wrtten P(a b). Moreover, we wll use the same notaton when A s a contnuous random varable for referrng to probablty densty pa B=b (a).

6 6 M. Bach Cuadra, J.-Ph. Thran, F. Marques * x arg max P( x y, ) (11.3) xx where x represents a possble label map and the condtonal probablty s evaluated over all possble label maps X, and represents the set of parameters used to estmate the probablty dstrbuton. For nstance, n the Gaussan case, these parameters are the mean vector and the covarance matrx of the process. Bayes rule can be then appled to Eq. (11.3), yeldng: P( y x, ) P( x) P( x y, ) P( y ) (11.4) where P( y x, ) represents the contrbuton of the so-called ntensty mage model (.e., the lkelhood that a specfc model x has produced the ntensty mage observaton y) that models the mage ntensty formaton process for each tssue type; P(x) represents the contrbuton of the socalled spatal dstrbuton model (.e., the a pror probablty of the correspondng label map) that models the spatal coherence of the mages, and P( y ) stands for the a pror probablty of the observed mage. Based on the above, we thus have to address the followng classfcaton problem. Gven an mage ntensty y, fnd the most probable label map x* such that: P( y x, ) P( x) x* arg max xx P( y ) (11.5) The condtonal probablty P( y ) s often assumed to be constant for all hypotheses of label maps and, therefore, t can be gnored n the maxmzaton process, so that Eq. (11.5) smplfes to: x* arg max P( y x, ) P( x ) (11.6) xx whch tself mples two steps: Image ntensty modelng: Image modelng refers to the estmaton of P( y x, ), and ths s typcally carred out usng Gaussan Mxture Models (GMM; see Secton ). As we wll see, assumng data ndependency, classfcaton can be performed relyng only on ths model. However, ths assumpton s too strong and the resultng

7 How can physcans quantfy bran degeneraton? 7 classfers lead to low qualty classfcaton results (see Secton ). Spatal dstrbuton modelng: Spatal pror probabltes of classfed mages P(x) are modeled usng Markov Random Felds (MRF; see Secton ). Actually, the global model to be used comes from the combnaton of the two prevous models. Ths way, n Secton , we wll ntroduce the concept of Hdden Markov Random Felds (HMRF) that allows ncludng dependency among random varables. Fnally, n Secton , Gaussan Hdden Markov Random Felds (GHMRF) wll be presented as a specfc case of the prevous HMRF model. Note that there are n fact many aspects of the theory that we wll present here that are shared wth Chapter 4. The same MAP method s used, thus the mathematcal formalsm s almost the same. As we shall see, however, some dfferences exst: frst, here we wll deal wth 3D data; and second, the pror probablty P(x) wll be modeled by Hdden Markov Random Felds where emsson probabltes wll not be estmated as n Chapter 4, but modeled by an exponental functon thanks to the Hammerd-Clford theorem Gaussan Mxture Models (GMM) We defne n ths Secton the ntal mathematcal model of the ntensty dstrbuton of an MR mage volume. Let us suppose, for now, that all the random varables representng the mage ntensty, Y (wth S), are dentcally and ndependently dstrbuted. Ths s a very strong assumpton on the data, whch wll be later refned usng of the noton of condtonal ndependence n the context of Hdden Markov Random Felds (see Secton ). Nevertheless, under the prevous assumptons, the probablty densty functon of the ntensty voxel at -th locaton can be defned by: where P X l (11.7) P( Y ) P( X l) P( Y X l), l L s the probablty of voxel beng of tssue class l, and P( Y X l) s the probablty densty functon of Y gven the tssue class l. A densty functon of ths form s called fnte mxture (FM) densty. The condtonal denstes P( Y X l) are called component denstes and, n ths case, they encode the ntensty nformaton. The a pror probabl-

8 8 M. Bach Cuadra, J.-Ph. Thran, F. Marques tes P( X l), are the mxng parameters and they encode the spatal nformaton (see Secton ). The smplest component densty consders that the probablty densty functon for the observed ntensty Y, gven the pure tssue class X l, s gven by the Gaussan functon: 2 ( Yl) l l l P( Y X l, ) e f ( Y ), 2 l (11.8) where the model parameters l { l, l} are respectvely the mean and the varance of the Gaussan functon f. Let us note that ths s a good approxmaton snce the nose present n an MRI follows a Rcan dstrbuton (Gudbjartsson and Patz 1995; Haacke, Brown et al. 1999) that, at hgh sgnal-to-nose rato, can be approxmated by a Gaussan dstrbuton. In the partcular case where the random varables X (wth S ) are also assumed to be ndependent of each other 6, whch means that: P( X l) (11.9) l Then, the probablty densty functon for the observed ntenstes Y nvolved n the mage ntensty model term of Eq. (11.6) can be wrtten as: P( Y ) l f ( Y l ). (11.10) ll Note that the mxng parameters l are not known n general and must therefore also be ncluded among the unknown parameters. Thus, the mxture densty parameter estmaton tres to estmate the parameters, such that: l l l 1. (11.11) ll 6 As t wll be shown n Secton , ths mples that no Markov Random Feld model s consdered.

9 How can physcans quantfy bran degeneraton? The Expectaton-Maxmzaton algorthm (EM) A possble approach to solve the parameter estmaton problem n s to fnd the value of usng ether the Maxmum Lkelhood (ML), or Maxmum Log Lkelhood crteron 7 (Blmes 1998): * arg max P( Y X, ) arg max log( P( Y X, )) (11.12) One of the most used methods to solve the maxmzaton problem s the EM algorthm (see Chapter 4). For the partcular case of Gaussan dstrbutons, the EM algorthm leads to the followng equatons: (0) Intalzaton Step. Choose the best ntalzaton for. Expectaton Step. Calculate the a posteror probabltes: ( k1) ( k1) P y ( ) ( ) ˆ ( 1) l P X l k k (, l ) ( 1) ( 1) ˆ k ˆ k l ll Pˆ X l y ˆ ˆ P y X l, P ( X l) (11.13) Maxmzaton Step: ˆ 1 ˆ ˆ (11.14) ( ) ( ) ( ) ( 1) ˆ k k ( ) k (, k l P X l P X l y l ) N S ˆ y Pˆ X l y, ˆ ( k) ( k1) l ( k ) S l ˆ( k) ˆ ( k1) P X l y, l S ( y ˆ ) Pˆ X l y, ˆ ( k ) 2 ( k ) ( k 1) l l ( k ) 2 S l ˆ( k) ( 1), ˆ k P X l y l S ( ˆ ) (11.15) (11.16) 7 Although both crtera are equvalent, t s usually more convenent to work wth the sum of log lkelhoods (see Chapter 4).

10 10 M. Bach Cuadra, J.-Ph. Thran, F. Marques Markov Random Felds (MRF) In Secton , we have made the assumpton that all random varables X (wth S ) were ndependent of each other (see Eq. (11.9)). If we want to ntroduce some relatonshp between neghbor varables n the random feld (whch seems a very natural constran), we have to refne our random feld model We have seen n Chapter 4 that a Markov model (or Markov chan) can be used for estmatng the probablty of a random sequence, based on the assumpton that the value taken by th observaton n the sequence only depends on ts mmedate neghborhood,.e. on the (-1) th observaton. Smlarly, a Markov Random Feld (MRF) can be used to estmate the probablty of a random mage 8, based on the assumpton that the dependency of the value of a gven pxel (or voxel) on the values taken by all other pxels n an mage, PX ( ), can be reduced to the nformaton contaned n a local neghborhood of ths pxel. More formally, let us relate all the pxel (or voxel) locatons (stes) n a grd S wth a neghborhood system N N ( S), where N s the set of stes neghborng, wth N and N j j N. A random feld X s then sad to be an MRF on S wth respect to a neghborhood system N f and only f, for all n S: {} P x x P x x (11.17) S N where x denotes the actual value of X, that s of the random feld X at locaton ; xs denotes the values of the random feld X at all the locatons {} n S except ; and x N the values of the random feld X at all locatons wthn the neghborhood of locaton. Ths property of an MRF s known as Markovanty respectvely. Accordng to the Hammersley-Clfford theorem, an MRF can be equvalently characterzed by a Gbbs dstrbuton, U ( x, ) e P( x ) (11.18) Z 8 The defntons here proposed can be related to 2D or 3D mages. In the case of 2D mages, the sngle elements are pxels whereas n the case of 3D mages, they are voxels.

11 How can physcans quantfy bran degeneraton? 11 where x represents a smultaneous confguraton of the random varables, X ; that s, a gven label map (see Secton ). The expresson n Eq. (11.18) has several free parameters to be determned: the normalzaton factor Z, the spatal parameter, and the energy functon U. Let us brefly dscuss how these parameters can be determned n the partcular framework of mage segmentaton. We refer the nterested reader to (L 2001) for further detals. The energy functon U Gven the prevously commented Hammersley-Clfford theorem, we can nclude a specfc defnton of the spatal dstrbuton model n Eq. (11.6). Ths way, the maxmzaton proposed n Eq. (11.6) can be formulated n terms of the energy functon: log P( y x, ) P( x) log P( y x, ) U( x, ) const (11.19) The defnton of the energy functon s arbtrary and several choces for U(x) can be found n the lterature related to mage segmentaton. A general expresson for the energy functon can be denoted by: U( x, ) V ( x ) Vj ( x, x j ) S 2 (11.20) jn Ths s known as the Potts model wth an external feld, V( x ), that weghts the relatve mportance of the varous classes present n the mage (see, for nstance, (Van Leemput, Maes et al. 2003)). However, the use of an external feld ncludes addtonal parameter estmaton, thus ths model s less used n mage segmentaton. Instead, a smplfed Potts model wth no external energy, V( x) 0, s used. Then, only the local spatal transtons Vj ( x, x j ) are taken nto account and all the classes n the label mage are consdered equally probable. A common way to defne them s, for nstance, as: 1, f x xj Vj ( x, x j ) ( x, x j ) (11.21) 0, otherwse.

12 12 M. Bach Cuadra, J.-Ph. Thran, F. Marques Intutvely, the equaton above encourages one voxel to be classfed as the tssue to whch most of ts neghbors belong. That s, t favors confguratons where homogenous classes are present n the mage. The spatal parameter Gven the energy functon defned n Eq. (11.20) and (11.21), the maxmzaton proposed n Eq. (11.6) can be formulated as a mnmzaton problem: x* arg mn log P( y x, ) U( x ) (11.22) xx As t can be observed, the spatal value of β controls the nfluence of the energy factor due to the spatal pror, U( x ), over the factor due to the ntensty, log P( y x, ). Note that ts nfluence on the fnal segmentaton s mportant. For nstance, = 0 corresponds to a unform dstrbuton over the possble states, that s, the maxmzaton s done only on the condtonal dstrbuton of the observed data P( y x, ). On the contrary, f the spatal nformaton s domnant over the ntensty nformaton, that s f β, MAP tends to classfy all voxels nto a sngle class. The value of β can be estmated by ML estmaton. However, many problems arse due to the complexty of MRF models and alternatve approxmatons have to be done. Commonly, ths s done by smulated annealng, such as Monte-Carlo Smulatons (Geman and Geman 1984), or by maxmum pseudo-lkelhood approaches (Besag J. 1974). The β parameter can be also determned arbtrarly by gradually ncreasng ts value over the algorthm teratons. Here, the value of β wll be fxed emprcally and we wll study ts nfluence on the fnal segmentaton n Secton The normalzaton factor Z The normalzaton factor of Gbbs dstrbuton s theoretcally defned as: Z( U) e xx U ( x, ) (11.23) but ths defnton mples a hgh computatonal cost or t s even ntractable snce the sum among all the possble confguratons of X s usually not known (Geman S. and Geman D. 1984). Note also ts dependence on the

13 How can physcans quantfy bran degeneraton? 13 defnton of the energy functon U. Nevertheless, n optmzaton problems, the soluton does not depend on the normalzaton factor snce t s a constant value shared by all possble solutons. Therefore, commonly, the exact computaton of the normalzaton factor s crcumvented Hdden Markov Random Felds (HMRF) In Secton and , we have examned how GMMs can be used for estmatng the mage ntensty model, whle assumng that Y are ndependent and dentcally dstrbuted. In ths Secton, we wll ntroduce some form of dependency among varables Y through the use of an underlyng, hdden MRF for spatal dstrbuton modelng. The theory of Hdden Markov Random Feld (HMRF) models s derved from Hdden Markov Models (HMMs), as seen n Chapter 4. HMMs are defned as stochastc processes generated by a Markov chan whose state sequence X cannot be observed drectly, but only through a sequence of observatons Y. As a result, HMMs model a random sequence of observatons Y wth two sets of probabltes: the emsson and the transton probabltes, whch respectvely account for the local and sequental randomness of the observatons. Here we consder a specal case snce, nstead of a Markov chan, an MRF s used as the underlyng stochastc process; that s, the sequental correlaton between observatons wll be replaced by the spatal correlaton between voxels. The HMM concept can then be adapted to MRFs, leadng to the noton of Hdden MRF (HMRF). As HMMs, HMRFs are defned wth respect to a par of random varable famles (X,Y) where X accounts for spatal randomness, whle Y accounts for local randomness. In summary, a HMRF model s characterzed by the followng: Hdden Random Feld: X { X, S} s an underlyng MRF assumng values n a fnte state space L wth probablty dstrbuton as defned n the Gbbs equaton (11.18). Observable Random Feld: Y { Y, S} s a random feld wth a fnte state space D. Gven any partcular confguraton, x L, every Y follows a known condtonal probablty dstrbuton P(Y X = x) of the same functonal form f( Y x ), where x are the nvolved parameters. Ths dstrbuton s called emsson probablty functon and, Y s also referred to as the emtted random feld. Condtonal Independence: For any x L, the random varables Y are supposed to be ndependent, whch means that :

14 14 M. Bach Cuadra, J.-Ph. Thran, F. Marques P( y x ) P( y x) (11.24) S So now, as n Secton , t s possble to compute the probablty densty functon for the observed ntenstes Y nvolved n the mage ntensty model term of Eq. (11.6). In the HMRF case, the probablty densty functon of Y dependent on the parameter set and the values n the assocated neghborhood of the hdden random feld, x N, s P( Y X, ) P( Y, X l x, ) N N l ll ll P( X l x ) P( Y ). N l (11.25) where {, x }. Ths s agan a fnte mxture (FM) densty as n Eq. x (11.10) where, now, the a pror probabltes P( X l x ) are the mxng parameters. N Gaussan Hdden Markov Random Feld Model (GHMRF) In the Gaussan case we can assume: 2 ( Yl) l l l P( Y ) e f ( Y ), 2 l (11.26) where, f( Y X ), s a Gaussan dstrbuton defned by l { l, l}. The probablty densty functon of the ntensty voxels can now be wrtten as functon of the parameter set and of the voxel neghborhood X as: N P( Y X, ) P( X l x ) f ( Y ). N N l ll (11.27) where P( X l x ) represents the locally dependent probablty of the N tssue class l. Note that n Secton we smplfed the analogous probablty: P( X l) P( X l). Here, the ndependence assumpton l s not vald any more.

15 How can physcans quantfy bran degeneraton? 15 To solve the parameter estmaton problem, an adapted verson of the EM algorthm presented above, called the HMRF-EM, s used (Zhang, Brady et al. 2001). The update equatons for the l { l, l} parameters are actually the same update equatons as for the Fnte Gaussan Mxture Model (FGMM) (Equatons and 11.16) except that Pˆ ( X l y, ˆ ) ˆ ( k1) ˆ( k1) P y l P X l xn ( 1) ( 1), ˆ ˆ l N ( k) ( k1) l k k ll P y X l P X l x Note that the calculaton of ˆ ( k1) P X l xn (11.28) nvolves a prevous estmaton of the class labels, X ; that s, the classfcaton step. Typcally, an ntal classfcaton s computed usng the GMM approach, thus wthout the HMRF model (see Secton ) MATLAB proof of concept D data vsualzaton As prevously commented, we wll do a longtudnal study, that s, we wll compare two MR mages from the same patent acqured n a year nterval (I1 n 2000 and I2 n 2001, see Fg. 11.2). Studes on patterns of bran atrophy n medcal mages dffer however from the proof-of-concept presented here. Usually, such studes are performed statstcally, n comparson to a probablstc pattern that represents the normal anatomy, that s, a transversal study. Nevertheless, the statstcal classfcaton framework presented here remans vald. The mage data used here are 3D MR mages of the human bran. They are T1-weghted 9 mages of 186 x 186 x 121 voxels of dmenson 1 mm x 1mm x 1.3 mm. Voxel type s unsgned short ntegers coded n 16 bts, that s voxel values can vary from 0 to Dfferent mage contrast can be generated n the MR magng process; the most often used s T1-wegthed but also T2 and Proton Densty (Haacke et al. 1999).

16 16 M. Bach Cuadra, J.-Ph. Thran, F. Marques MATLAB provdes several technques for vsualzng 3D scalar volume data 10. One of them s the contourslce functon: load ImageVolume.mat phandles=contourslce(d,[],[45,64,78,90,100,110,120,127,135,14 2,150,160,175,190,200,210,220,228],[60,65,70,75,80,85],7); vew(-169,-54); colormap(gray) Fg Vsualzaton of 3D MRI data The resultng volume s shown n Fg The vew angle can be changed by usng Tools and Rotate 3D n the fgure menu. However, medcal doctors usually vsualze such data usng the 3 man vews, namely the Axal, Coronal and Sagttal vews. After loadng the volume nto the MATLAB Workspace an Axal matrx that contans the mage ntenstes appears. Its dmensons are: load Image1_vo.mat sze(axal) ans= Let us have a look at the three vews of the Axal matrx: subplot(1,3,1); magesc(squeeze(axal(90,:,:))); subplot(1,3,2); magesc(squeeze(axal(:,65,:))); subplot(1,3,3); magesc(squeeze(axal(:,:,60))'); 10 Search for Technques for Vsualzng Scalar Volume Data n the Help menu of MATLAB.

17 How can physcans quantfy bran degeneraton? 17 Fg Coronal, Axal and Sagttal vews of a 3D MR bran mage at tme 1 Thus the same volume as n Fg s seen here from three dfferent 2D vews: from the front of the head (Coronal), from the top of the head (Axal) and from a sde of the head (Saggttal). Note that only the bran s presented, snce the skull has been removed 11. The color map s as follows: dark gray s Cerebrospnal Flud, or CSF; mddle gray s Gray Matter, or GM; and lght gray s Whte Matter, or WM. Note that few patches of the skull are stll present. Now, we can run the same code for the mage at tme 2 (see Fg. 11.6): load(image2_vo.mat); Fg Coronal, Axal and Sagttal vews of a 3D MR bran mage at tme 2 11 Several technques can be used to remove skull and other non-bran tssues from bran. Here morphologcal operatons (dlatons and closngs) have been appled.

18 18 M. Bach Cuadra, J.-Ph. Thran, F. Marques Note that both mages look very smlar: they show the same bran, both have the same sze and there s a spatal correspondence (they have been already rgdly matched); that s, they have a pont to pont correspondence (see Secton 11.3). However, as the colorbar on the rght sde shows, there s a dfference n the ntensty level range. Thus we propose hereafter to study the mage hstogram Image hstogram The functon nvolved n the computaton of the mage hstogram s: [H,X,NbrBn,BnSze]=hstogram(Axal,loffset,roffset); The outputs are the values of the mage hstogram H, the bns vector X, the total number of bns, NbrBn, and the bn sze s BnSze. loffset and roffset correspond respectvely to the left and rght offsets that defne the start and end of the mage ntenstes consdered n the computaton. Note that mages n Fg and Fg present large black areas (0 grey level) whch do not provde any nformaton. To remove these areas from the statstcal analyss, we set the left offset to 1. In turn, the rght offset s set to 0. Results are shown n Fg. 11.7: loffset=1; roffset=0; load Image1_vo.mat [H,X,NbrBn,BnSze]=hstogram(Axal,loffset,roffset); load Image2_vo.mat [H,X,NbrBn,BnSze]=hstogram(Axal,loffset,roffset); Fg Image hstogram, left s mage 1 (year 2000) and rght s mage 2 (year 2001)

19 How can physcans quantfy bran degeneraton? 19 As expected, ntensty range values are dfferent (as already seen wth the colorbar n Fg and Fg. 11.6). Three peaks can be observed: the frst one s the darkest value and corresponds to CSF, the second one represents GM and the thrd one corresponds to WM. The current number of bns used n the representaton s NbrBn ans = 1179; % For Image 1 ans = 301; % For Image 2 Our hstogram functon calls the hstc MATLAB functon: H=hstc(Axal(:),X); Ths functon counts the number of values n Axal that fall between the elements n the X vector of bns (whch must contan monotoncally non-decreasng values). Then, H s a vector of length length(x) contanng these counts. We can change the range of bns X nteractvely 12 by choosng a dfferent left and rght offset: lms=getpts; dsp(' ') loffset=floor(lms(1)) roffset=floor(nbrbn-lms(2)) loffset = 23 roffset = 696 The number of bns can be modfed by not only varyng the extremes of vector X but also by changng the bn sze. Ths s mplemented by changng the step value n the ASP_bran_segmentaton_hstogram.m functon: f roffset==0 X=loffset:step:n; else X=loffset:step:(n-roffset); end Ths way the hstogram resoluton can be modfed as seen n Fg where step value has been set to 0.1 (left plot) and 40 (rght plot): 12 The getpts functon allows you to select ponts from the current fgure wth the mouse (left-button) and select last pont wth the rght-button.

20 20 M. Bach Cuadra, J.-Ph. Thran, F. Marques Fg Image hstograms usng step=0.1 (left) and step=40 (rght) The selecton of the number and the sze of the bns n the prevous example s only a matter of vsualzaton. However, these parameters can have an nfluence n the computaton tme and precson of some optmzaton problems where mage hstograms are computed very often. We wll show n the next Secton the nfluence of the bn sze on the Gaussan fttng optmzaton problem. In ths work, we wll approxmate the probablty densty functon (PDF) of the ntensty values by the normalzed mage hstogram. The normalzaton of the mage hstogram nh s computed as follows: NbrSamples=sum(sum(H)); nh=h*1/nbrsamples; Gaussan Mxture Model (GMM) As presented n Secton , to solve the parameter estmaton of the Gaussan Mxture Model we use the Expectaton Maxmzaton (EM) algorthm. Such a model assumes that the probablty densty functon of the mage ntensty of every class follows a Gaussan dstrbuton. Before dscussng the estmaton problem, let us have a look at the MATLAB functons nvolved n the generaton of the Gaussan probablty denstes: y = exp(-0.5*(x-mu).*(x-mu)/(sgma*sgma))... /(sqrt(2*p)*sgma); Ths s packed nto the ASP_bran_segmentaton_gauss.m functon and corresponds to Eq. (11.8). In ths equaton, x s the mage ntensty values and mu and sgma must be estmated. In fact, we are lookng for the parameters of a weghted sum of Gaussan dstrbutons that better ft the mage ntensty hstogram (lkelhood estmaton). NbrComp s the

21 How can physcans quantfy bran degeneraton? 21 number of mxtures used for the fttng whch, n our case, s equal to three snce three classes are used: CSF, GM and WM. for =1:NbrComp for j=1:length(x) G(,j)=Theta(,3).*gauss(X(j),Theta(,1),Theta(,2)); end end G s a NbrComp x NbrBn matrx, n whch Theta(:,1) are the mean, Theta(:,2) are the standard devatons and Theta(:,3) are the weghts. Intal values of means, standard devatons and weghts must be set. Remember that the EM algorthm s senstve to ntal parameters and, partcularly, to ntal mean values (see Chapter 4 for an example). We can modfy the ntal parameters by manually selectng the mean values and, ths way, observe the convergence behavor. Let us frst set 3 mean values on the left sde of the mage hstogram: mus=[ ]; s=sze(mus); NbrComp=s(1); Note that mean values can be also selected by clckng on the mage hstogram; the correspondng MATLAB code s provded n the man scrpt, ASP_bran_segmentaton.m. Then, the standard devaton and the ntal weght are usually set as: for =1:NbrComp sg(,1)=5.0; pw(,1)=1/nbrcomp; end That s, standard devaton can be set to a small but non-zero value and weghts are all set equally. As a matter of fact, the EM algorthm s not as senstve to these parameters as to the mean values. Fnally, ntal parameters are saved n Theta: % compose Theta Theta=cat(2,mus,sg,pw); Theta = Now, the EM algorthm can be appled: [S,e,Theta,ftper,G]=segmentem(Axal, loffset, roffset, Theta);

22 22 M. Bach Cuadra, J.-Ph. Thran, F. Marques Ths functon computes the followng output parameters: the new estmated Theta values, the correspondng Gaussan dstrbutons G, the squared error ftper between the mage hstogram and the ft wth the Gaussan mxture, the threshold boundares e between each mxture, and the estmated classfcaton S obtaned by applyng a threshold to the ntensty values wth e. The result s plotted n Fg In t, three dfferent types of functons can be observed. The orgnal hstogram s presented n dot lne, whereas the approxmated hstogram s n sold lne. Ths approxmated hstogram s the sum of the three Gaussan hstograms that have been estmated for each class, whch are presented n dashed lnes. The vertcal lnes represent the two dfferent thresholds that have been fnally obtaned and appled to perform the classfcaton. ftper = Estmated means and varance: ul = sl = Estmated thresholds for Bayesan decson: e = Fg Fttng of the mage hstogram: 3 Gaussans are used. Intal means are 130, 220, and 330 and bn sze s 1

23 How can physcans quantfy bran degeneraton? 23 Let us now vsualze the mage segmentaton S after thresholdng the nput mage usng the e values: fgure; clf; subplot(1,3,1); magesc(squeeze(s(90,:,:))); ptsetpref('imshowborder', 'tght'); ttle('coronal vew'); hold on; subplot(1,3,2); magesc(squeeze(s(:,65,:))); ptsetpref('imshowborder', 'tght'); ttle('axal vew'); hold on; subplot(1,3,3); magesc((squeeze(s(:,:,60)))'); axs fll; ptsetpref('imshowborder', 'tght'); colorbar; ttle('sagttal vew'); Fg Bayesan classfcaton n 3 classes: 0 s background, 1 s CSF, 2 s a thn transton class, and 3 s GM and WM all together. Intal means are 130, 220, and 330 and bn sze s 1. Obvously, the resultng mage classfcaton of Fg s wrong snce the Gaussan fttng has faled. GM and WM have been mxed together nto a sngle class. In order to further analyze the senstvty of the EM algorthm wth respect to the ntalzaton, let us now use a set of ntal mean values closer to the three peaks of the mage hstogram, keepng the ntal values of standard devaton and weghts as n the prevous example: mus = [ ]'; Theta =

24 24 M. Bach Cuadra, J.-Ph. Thran, F. Marques Ths tme, the result of ASP_bran_segmentaton_segmentem.m s ftper = Estmated mean and varance: ul = sl = Estmated thresholds for Bayesan decson: e = Ths result s presented n Fg followng the same layout that has been used n Fg Fg Fttng of the mage hstogram: 3 Gaussans are used. Intal means are 115, 345, and 410 and bn sze s 1. The fttng and classfcaton results are now more satsfactory: classes converged to the underlyng anatomcal tssues. The resultng mage classfcaton n Fg shows a good convergence of the algorthm. Unfortunately t s qute nosy; note the spurous ponts that are msclassfed as gray matter n the nteror of whte matter. Ths s due to the Bayesan decson, whch s taken based on the ntensty gray level nformaton only. Better results can be obtaned by ncludng local spatal nformaton about

25 How can physcans quantfy bran degeneraton? 25 the neghborng voxels. Ths can be done applyng the MRF theory as seen prevously n the theory Secton and presented hereafter. Fg Bayesan classfcaton n 3 classes: 0 s background (n dark blue), 1 s CSF (n lght blue), 2 s GM (n yellow), and 3 s WM (n dark red). Intal means are 115, 345, and 410. As mentoned n the prevous Secton, the result of the Gaussan fttng also depends on the selected number and sze of bns. Logcally, f we change the reference mage hstogram the fnal fttng wll change. Let us llustrate ths usng a bn sze of 20 nstead of 1 and keepng the same ntalzaton as n the prevous case (see Fg ): mus = [ ]'; ftper = Estmated means and varance: ul = sl = Estmated thresholds for Bayesan decson: e =

26 26 M. Bach Cuadra, J.-Ph. Thran, F. Marques Fg Fttng of the mage hstogram usng a dfferent bn sze to compute the mage hstogram. Intal means are 115, 345, and 410 and bn sze s 20. In terms of computatonal tme, the smaller the number of bns, the faster the fttng. However, as llustrated n ths example, there s a compromse snce the use of too few bns leads to an naccurate hstogram estmate. As t can be seen, the selected thresholds are not as accurate as n the prevous case (see Fg ). Moreover, n that partcular case, the use of a too small number of bns presents another effect. Note that the threshold obtaned between class 1 (CSF) and class 2 (GM) s not concdent wth the ntersecton of the Gaussan functons representng ther ndvdual densty functons. Ths s due to the rough quantfcaton nto a too small number of bns: the theoretcal value (whch s the ntersecton pont between both densty functons) s approxmated by the center of the closest bn Hdden Gaussan Mxture Model The strategy underlyng the HMRF-EM algorthm conssts n applyng teratvely the followng two steps: 1. Estmate the mage labelng, x, gven the current model and then use t to form the complete data set x, y (y beng the ntensty mage observaton). The labeled mage s ntally estmated only usng GMM

27 How can physcans quantfy bran degeneraton? 27 (Secton ). Then, solve the followng mnmzaton problem (nstead of maxmzng the posteror probablty): x* arg mn log P( y x, ) U( x ) (11.29) xx The MATLAB functons nvolved are: nl=daloop(nl,axal,ul,sl,nbrcomp,beta); whch computes the new classfed mage (nl) as n Eq. (11.22) wth the current estmaton of mean (ul) and varance (sl). It ncludes two functons for the calculaton of the two energes: calcuyx(mu(l),sgma(l),image); calcux_3class(l,sx,sy,sz,l); where l s the current label, Sx, Sy and Sz are the voxel dmensons (1 mm, 1 mm and 1.3 mm, respectvely) and L s the current classfed mage. 2. Estmate a new by maxmzng the expectaton of the complete data log lkelhood, E[log P( xy, )]. That s, re-estmate the mean and varance vectors as descrbed n Secton and update the equatons for mean and varance vectors n the EM algorthm. The MATLAB functon nvolved s [ul,sl]=calculsl(axal, nl, nh, X, ul, sl,nbrcomp); The two parameters to set n the HMRF-EM algorthm are the number of teratons and the value of (see Eq. (11.22)) that weghts the energy U( x) wth respect to log P( y x, ). The loop s: nt=5; beta=0.6; for =1:nt nl=daloop(nl,axal,ul,sl,nbrcomp,beta); [ul,sl]=calculsl(axal, nl, nh, X, ul, sl,nbrcomp); end The fnal label map after 5 MRF teratons s shown n Fg Note that t s much less nosy than the label map shown n Fg

28 28 M. Bach Cuadra, J.-Ph. Thran, F. Marques Fg HMRF classfcaton n 3 classes after 5 teratons and β set to Influence of the spatal parameter As mentoned n Secton , the β parameter s set here expermentally. In ths Secton we wll analyze ts nfluence on the fnal segmentaton. Let us frst observe the HMRF classfcaton wth β = 0.1, β = 1.2, and β = 10 after 60 teratons (see Fg ). Let us now study the HMRF convergence. For ths purpose we compute the percentage of voxels (related to the total number of voxels of the mage, wthout consderng the background) that changed of assgned label, between two consecutve teratons, durng 20 teratons (see Fg ). Ths wll gve us an dea about the speed of convergence to a local optmal soluton for the MAP decson. The correspondng MATLAB code s ncluded n the man loop of HRMF: nt=60; beta=10; nlold=nl; Change=zeros(nt,1); for =1:nt nl=daloop(nl,axal,ul,sl,nbrcomp,beta); Dff=(nLold-nL); Change()=length(fnd(Dff))./length(fnd(nL))*100; nlold=nl; [ul,sl]=calculsl(axal, nl, nh, X, ul, sl,nbrcomp); end

29 How can physcans quantfy bran degeneraton? 29 Fg Segmentaton after 60 HMRF teratons.top left: Bayesan classfcaton wthout HMRF; Top rght: HMRF classfcaton wth β = 0.1. Bottom left: HMRF classfcaton wth β = 1.2; Bottom rght: same wth β = 10. Fg HMRF convergence study

30 30 M. Bach Cuadra, J.-Ph. Thran, F. Marques As seen n Fg , the larger β s, the slower the convergence to a MAP soluton s. There s thus a compromse between speed (to reach a soluton) and qualty of the fnal segmentaton. In ths work we wll keep the values of β = 0.6 and nt = 5 teratons. Ths choce s stll arbtrary Localzaton and quantfcaton of bran degeneraton Our goal here s to localze and quantfy the GM degeneraton n the prevous example. Globally, studes on patterns of bran atrophy n medcal mages are done manly by two dfferent approaches. The frst conssts of the manual detecton and classfcaton of a regon of nterest (ROI), leadng to a tme consumng and subjectve procedure (Galton, Patterson et al. 2001). The second, and most used approach, s the voxel-based morphometry (VBM), usually havng the followng steps (Ashburner and Frston 2000): Normalzaton of all the subjects nto the same space, Gray matter extracton and smoothng of the normalzed segmented mages, Statstcal analyss of the dfferences between a pror reference data and the subjects. The method proposed here s n fact based on the VBM theory. In our study, the probablstc reference s not avalable so the evolvng degeneraton s only studed from the sequence mages; that s, a so-called longtudnal study s performed. Our nterest s not n the fnal tssue classfcaton (label map), but n the tssue probablty maps; that s, functons (mages n our case) n whch each pont (pxel) has assocated the lkelhood of belongng to a gven tssue. Partcularly, as GM degeneraton s studed, cerebrospnal flud (CSF) and GM posteror probablty maps are retaned from the classfcaton step. Workng wth probablty maps wll allow us to be more precse n the computaton of average concentraton maps. The MATLAB functons nvolved are: ProbMaps(Axal, nl, ul, sl, NbrComp,flenameMRI(1:end-4)); Ths functon outputs the posteror probablty maps nto.raw fles for every tssue class: %Wrte output fles fname=strcat(flename,'_pcsf.raw'); fd = fopen(fname,'wb'); fwrte(fd,a*255,'float');

31 How can physcans quantfy bran degeneraton? 31 fclose(fd) All probablty maps are re-scaled to gray levels n [0, 255] before beng stored n.raw fles. These probablty maps are n fact computed as: ptly(l,:,:,:)=calcptly(image, L, l, nh, X, mu(l), sgma(l), NbrComp); whch mplements the equaton descrbed n Eq. (11.28). Then, a smoothng of the tssue probablty maps by a large Gaussan kernel s appled. Ths way, each voxel of the tssue probablty map represents the tssue concentraton wthn a gven regon defned by the standard devaton of the Gaussan flter beng used. In order to determne the standard devaton, morphometry theory, as descrbed n (Ashburner and Frston 2000), suggests a value smlar to the sze of the changes that are expected. For the partcular case of the mage sequence under study, the value of σ = 11 mm s used. The resultng concentraton maps of CSF and GM at tme 1 are shown n Fg and Fg , respectvely. Ccsf_1=smooth_3DGauss('Image1_vo_Pcsf.raw',s(1),s(2),s(3), 'float',7,11); Cgm_1=smooth_3DGauss('Image1_vo_Pgm.raw',s(1),s(2),s(3), 'float',7,11); Ccsf_2=smooth_3DGauss('Image2_vo_Pcsf.raw',s(1),s(2),s(3), 'float',7,11); Cgm_2=smooth_3DGauss('Image2_vo_Pgm.raw',s(1),s(2),s(3), 'float',7,11); Fg Smoothed probablty maps for the GM class.

32 32 M. Bach Cuadra, J.-Ph. Thran, F. Marques Fg Smoothed probablty maps for the CSF class. Let us defne the GM degeneraton map as D C C (11.30) l l l l where C s the concentraton map of tssue l at tme t, for t l { CSF, GM}, t {1,2}, and D shows the regons wth more probable gray matter loss 12 between mages 1 and 2. The correspondng MATLAB code s: D12=abs(Ccsf_1-Ccsf_2).*abs(Cgm_1-Cgm_2); magesc(squeeze(d12(:,99,:))); magesc(squeeze(d12(84,:,:))); Fg Detected degeneraton areas: Left s an axal vew, rght s the coronal vew. Hghest values represent the hghest degenerated regons

33 How can physcans quantfy bran degeneraton? 33 The obtaned degeneraton maps (Fg ) may contan some regons that actually do not correspond to a real GM degeneraton (for nstance due to regstraton errors n the cortex or trunk area). However, the most mportant regons of degeneraton (ROD) can be solated by applyng a smple threshold to the prevous mages. The result s presented n Fg where t can be observed that degeneraton can now be easly quantfed. volume_ro_1=sum(cgm_1(d12>500)) e+006 volume_ro_2=sum(cgm_2(d12>500)) e+006 Fg Degeneraton areas after thresholdng degeneraton_ro=(volume_ro_1-volume_ro_2)/volume_ro_1* Medcal dagnoss of ths patent s GM degeneraton n the entorhnal regon. Ths corresponds approxmately to the regon shown n Fg : ROD = D12(45:115,70:120,70:121);

34 34 M. Bach Cuadra, J.-Ph. Thran, F. Marques Fg Degeneraton map n the entorhnal cortex Once the canddates to defne the GM degeneraton regons have been detected, degeneraton can be quantfed more precsely wthn these regons: volume_ro_1=sum(cgm_1(rod~=0)) e+004 volume_ro_2=sum(cgm_2(rod~=0)) e+004 degeneraton_ro=(volume_ro_1-volume_ro_2)/volume_ro_1* Ths quantfcaton gves an dea about the GM loss n percentage n these regons of nterest. However, ths longtudnal study s lmted to two tme nstants only. It would be worth comparng the localzaton (D12) and quantfcaton (degeneraton_ro) of the degeneraton at addtonal tme nstants, as well as statstcally comparng t to a group of healthy subjects Gong further There s a large amount of dfferent types of nformaton avalable on medcal mage modaltes (see the Introducton n ths Chapter). All these n-

35 How can physcans quantfy bran degeneraton? 35 formaton types can be effcently combned by medcal mage regstraton (or matchng). Image regstraton, whch has not been handled n ths Chapter, conssts on fndng the transformaton that brngs two medcal mages nto a voxel-to-voxel correspondence. Many varables partcpate n the regstraton paradgm, makng the classfcaton of regstraton technques a dffcult task. A wde overvew of medcal mage regstraton s done n (Mantz J.B.A. and Vergever M.A. 1998). In mage regstraton, two mages are matched one to the other: the target mage, also called reference mage or scene (f), and the mage that wll be transformed, also called floatng mage or deformable model (g). If the mages to regster belong to the same modalty t s sad to be monomodal regstraton and f they belong to dfferent modaltes t s sad to be multmodal regstraton. We can also dstngush between ntra-subject and nter-subject regstraton. In the frst case, both reference and floatng mages belong to the same patent. The goal of ntra-subject regstraton s usually to compensate the varable postonng of the patent n the acquston process as well as other possble geometrcal artfacts n the mages. Ths knd of regstraton s usually needed n surgcal plannng, leson localzaton or pathology evoluton. In the second case, mages are acqured from dfferent patents. The goal of ths knd of regstraton s to compensate the nter-subject anatomcal varablty n order to perform statstcal studes or to proft from reference data 13, for nstance for surgcal plannng. The mage modaltes to regster, as well as the envsaged applcaton, determne the remanng characterstcs of the regstraton process: the nature and doman of possble transformatons τ, the features to be matched, the cost functon to optmze. The mage regstraton problem can be formulated by the followng mnmzaton equaton: * T f T g T argmn cost(, ) (11.31) Note that all transformatons n τ have to follow some physcal constrants n order to model a realstc deformaton between two brans (we can deform a bran nto an apple but t s not very lkely to happen). 13 Reference data s also called Atlas mages that represent the human anatomy.

36 36 M. Bach Cuadra, J.-Ph. Thran, F. Marques Nature and Doman of the transformaton In mage regstraton, transformatons are commonly ether rgd (only translatons and rotatons are allowed), affne (parallel lnes are mapped onto parallel lnes), projectve (lnes are mapped onto lnes) or curved (lnes are mapped onto curves). Then, the transformaton doman can ether be local or global. A transformaton s called global when a change n any one of the transformaton parameters nfluences the transformaton of the mage as a whole. In a local transformaton a change n any transformaton parameter only affects a part of the mage Features and cost functon The feature selecton depends on the mage modalty and the defnton of the cost functon depends on the selected features. The ntensty mage can be drectly used as features for the matchng process (ths approach s called voxel-based regstraton) but other dentfable anatomcal elements can be used such as pont landmarks, lnes or surfaces, whch are prevously extracted n both reference and floatng mages (ths s called model-based regstraton). Once feature selecton s done, the cost functon can be determned, commonly usng: Cost functon = - Smlarty measure. The smlarty measure can be ntensty-based (n voxel-based regstraton) or dstance-based (n model-based regstraton). Some typcal smlarty measures based on voxel ntensty (Bankman I.N. 2000) are absolute or squared ntensty dfference, normalzed cross-correlaton, measures based on the optcal flow concept 14, nformaton theory measures such as mutual nformaton, etc. Some common dstance-based measures are: Procrustean metrc, Eucldean dstance, curvature, etc Optmzaton An optmzaton algorthm (Fg ) s needed to ether maxmze a smlarty measure or mnmze an error measure, over the search space defned by the parameters of the transformaton. Varous optmzaton strateges are avalable, such as Powell s or Smplex optmzaton, Steepest 14 The optcal flow concept has been presented n Chapter 9 wthn the context of moton estmaton.