Non-Rigid Registration and Correspondence Finding in Medical Image Analysis Using Multiple-Layer Flexible Mesh Template Matching

Transcription

1 Non-Rgd Regstraton and Correspondence Fndng n Medcal Image Analyss Usng Multple-Layer Flexble Mesh Template Matchng Janhua Yao, Russell Taylor 2. Dagnostc Radology Department, Clncal Center, Natonal Insttutes of Health, Bethesda, MD 20892, jyao@cc.nh.gov 2. Computer Scence Department, The Johns Hopkns Unversty, Baltmore MD 220 Abstract In ths paper we present a novel technque for non-rgd medcal mage regstraton and correspondence fndng based on a multple-layer flexble mesh template matchng technque. A statstcal anatomcal model s bult n the form of a tetrahedral mesh, whch ncorporates both shape and densty propertes of the anatomcal structure. After the affne transformaton and global deformaton of the model are computed by optmzng an energy functon, a multple-layer flexble mesh template matchng s appled to fnd the vertex correspondence and acheve local deformaton. The multple-layer structure of the template can be used to descrbe dfferent scale of anatomcal features; furthermore, the template matchng s flexble whch makes the correspondence fndng robust. A leave-one-out valdaton has been conducted to demonstrate the effectveness and accuracy of our method. Keyword: Non-rgd regstraton, statstcal model, multple-layer flexble mesh template, correspondence. Background and ntroducton Non-rgd medcal mage regstraton s an essental step n many automated medcal mage analyss and has been wdely nvestgated n recent years [-]. Medcal mage regstraton can be categorzed based on the subjects nvolved n the regstraton. One category s related to regstraton between two mages, such as regstraton between mages from dfferent modaltes, mages from dfferent ndvduals, and mages taken at dfferent tmes [, 2-4]. Ths category of regstraton usually relates the nformaton n one mage to nformaton n another mage by determnng the oneto-one pxel correspondences. Another category of regstraton s between an anatomcal model/atlas

2 and a medcal mage [2-7, 5-20]. In ths category, the anatomcal model or atlas s transformed and deformed to match the anatomc structure n the mage. The ablty to regster an anatomcal model to ndvdual patent mages provdes the bass for solvng several mportant problems n medcal mage nterpretaton. Once the model s regstered to a partcular mage, structures of nterest can be labeled and extracted for further analyss, and knowledge ncorporated n the model can be transferred to the patent. The result of the regstered model generates segmentaton for the structure of nterest, whch can then be used for mage analyss purposes such as measurement and vsualzaton. Model-to-mage regstraton also allows populaton studes to be analyzed n a common frame of reference. In the model-to-mage non-rgd regstraton, the correspondences between features on the model and those on the mage need to be correctly establshed n order to obtan the desred transformaton and deformaton. Some methods are based on anatomcal features (landmarks) such as ponts, curves and surfaces [3, 6, 5, 2-24]. Other methods depend on the ntensty nformaton n the mage [, 2, 25-27]. Hybrd algorthms have also been proposed n whch both geometrc features and mage ntenstes are used [7-9]. In landmark-based methods, geometrc features such as ponts, curves or surfaces are brought nto algnment. The landmark ponts can be ether ntrnsc or extrnsc. Intrnsc ponts are derved from naturally occurrng features, e.g. anatomc landmark ponts. Extrnsc ponts are derved from artfcally appled markers or fducals fxed to the patent and vsble n both mages and models. In surface-based methods, the geometrc features (such as rdge curves) on the surface are used to establsh the correspondence [24]. After the correspondences between landmarks are establshed, the mage and/or model are warped to algn the corresponded landmarks. Intenstybased regstraton stems from the observaton that although mages from dfferent modaltes or dfferent subjects exhbt dfferent data values, there s usually a large amount of shared nformaton between mages of the same structures. The correspondence between the model and the mage s establshed based on the voxel ntensty dstrbuton [, 2, 25-27]. An anatomcal model s a tool to represent human anatomcal structures and normal anatomcal varablty. Medcal mages are usually complex, nosy and possbly ncomplete, whch makes ther nterpretaton very dffcult wthout pror knowledge of the anatomy. Therefore, many researchers turned to pror statstcal models for assstance n medcal mage regstraton. Shen et al. [6] proposed a statstcal surface model usng an affne nvarant geometrc attrbute vector to fnd the vertex correspondences. Chen et al. [] bult an average bran atlas based on statstcal analyss of voxel ntensty values. Cootes et al. [2] proposed an Actve Appearance Model (AAM) that ncorporates both the shape varablty and densty varablty. The AAM models were expermented on 2D MRI bran slces and 2D human face mages. Cootes s models were extended to 3D surface models by Fleute et al. [3]. 2

3 Fgure. Multple resoluton pelvs model Most exstng model-to-mage regstraton methods used ether a surface model or an ntensty grd model [3, 5, 6, 7, 8, 28]. Hence the exteror shape and the nternal densty dstrbuton cannot be matched at the same tme. To address ths problem, our method s based on a statstcal volumetrc model ncorporatng both shape and densty propertes of the anatomcal structure. As mentoned n some statstcal model based regstraton methods [2, 6], the pror model extracted from tranng models s not suffcent to characterze all the varatons n a new mage. To compensate ths, a multple-layer flexble mesh template matchng technque s developed to fnd the feature correspondences and acheve local deformaton. Ths technque takes advantage of our model topologcal structure (tetrahedral mesh) and propertes ncorporated n the model (shape and densty). The remander of ths paper s organzed as follows. Secton 2 brefly ntroduces the statstcal bone densty model and ts constructon from a set of tranng mages. Secton 3 presents our method for non-rgd regstraton between the statstcal model and a CT mage. Secton 4 proposes a multple-layer flexble mesh template matchng method for correspondence fndng and local deformaton of the model. Fnally, Secton 5 and Secton 6 conclude wth valdaton experments and dscussons. 2. Statstcal bone densty model We proposed a unque model representaton to characterze both the boundary surface and nternal densty dstrbuton of the bone structures. The model s represented as a tetrahedral mesh equpped wth embedded Bernsten polynomal densty functons on the barycentrc coordnates of each tetrahedron. Multple level-of-detals of the anatomcal structure are characterzed by a herarchcal representaton. And pror nformaton of both shape propertes and densty propertes s ncorporated n the model. We developed an effcent tetrahedral mesh reconstructon from contours method to construct tetrahedral meshes for bone structures from a CT mage. The method produces tetrahedral meshes wth hgh flexblty and s able to accommodate any anatomcal shape. The meshes are bult from contours consstent wth the cortcal bone boundares. The contours are extracted from the mage 3

4 -3 standard devaton Mean shape (a) +3 standard devaton -3 standard devaton Mean model (b) Fgure 2. Shape and densty varaton of the model +3 standard devaton slce-by-slce usng an actve contour technque [29]. Then the contours are tled nto a tetrahedral mesh by solvng a seres of tlng, correspondng and branchng problems [30]. An analytcal densty functon s assgned for every tetrahedron to mnmze the resdual errors n the densty dstrbuton. Currently, the densty functons are wrtten as n-degree Bernsten polynomals n barycentrc coordnates of a tetrahedron: ( ) = n n C, j, k, lb, j, k, l ( µ + j+ k + l = n D µ ), () C j, k, l where, s the polynomal coeffcent, and µ = (µ x, µ y, µ z, µ w ). n n! j k l B, j, k, l ( µ ) = µ xµ yµ z µ (2) w! j! k! l! s a barycentrc Bernsten bass functon, and (µ x, µ y, µ z, µ w ) are the barycentrc coordnates, wth µ x + µ y + µ z +µ w =. The advantages of such a representaton are: ) t s n an explct form; and 2) t s a contnuous functon n 3D space. Therefore, t s convenent to ntegrate, to dfferentate, and to nterpolate. We also developed a tetrahedral mesh smplfcaton algorthm based on edge collapsng operatons [3] to buld a multple level-of-detal (LOD) model representaton. We desgned a tranng strategy to compute a statstcal model from a collecton of tranng models. A model algnng procedure s frst performed to map all tranng models nto a common mesh topologcal structure. Then the Prncpal Component Analyss (PCA) method s appled to compute the varablty of both shape propertes and densty propertes of the anatomcal structure. Usng the PCA method, the model can be approxmated by a set of statstcal mode parameters {b }: Y = M ( Y, b) = Y + Pb (3) where Y s a model nstance, M ( Y, b) s the nstantaton operaton, Y s the average model representaton, and P s the egenvector matrx ncorporatng the pror nformaton. We have bult a statstcal densty model for hem-pelvs from eght tranng mages. Fgure shows the exteror surface of a multple-resoluton hem-pelvs model. Fgure 2(a) shows the shape 4

5 varaton and Fgure 2(b) shows the densty varaton of the model by projectng the model to a 2D plane usng a ray castng technque. Fgure 2 demonstrates that a new model nstance can be nstantated from the average model by settng the statstcal mode parameters. Detaled descrpton of our model and reconstructon method can be found n [32, 33]. 3. Non-rgd regstraton scheme Non-rgd regstraton between our statstcal bone densty model and CT mages s one of the purposes n our nvestgaton. Our method utlzes both shape propertes and densty propertes of the model, together wth the statstcal nformaton extracted from a populaton of tranng models. The result from the regstraton can be used as a model-based segmentaton of the bone structure. The regstraton process s dvded nto three stages: affne transformaton, global deformaton, and local deformaton. The result of precedng stage s used as ntal values for next stage. In the affne transformaton stage, the translaton, rotaton, and scale of the model are optmzed to determne the locaton and orentaton of the model. In the global deformaton stage, the statstcal mode parameters are optmzed to match the model wth the anatomcal structure n the mages. Due to the lmted number of tranng models n the model tranng stage, the pror nformaton n the statstcal model does not nclude all the varablty nherent n the anatomy. To compensate for ths, a local deformaton step s taken to buld correspondences between vertces on the model and local features n the mage and adaptvely warp the model. The regstraton process s governed by a scrpt fle and s hghly automated. 3. Optmzaton algorthm and energy functon Both the affne transformaton stage and the global deformaton stage are essentally multdmensonal nonlnear optmzaton problems desgned to mnmze an energy functon between the anatomcal model and the CT mage. An energy functon s defned as the objectve functon to evaluate the dfference between the model and the CT mage, wrtten as: E E E ( s) ( d ) ( mdl, mg) w E ( mdl, mg) + w E ( mdl mg) ( s) ( d ) = (4) s d, N ( v) ( mdl) ( mg) ( mdl, mg) = ( g ( v ) g ( v )) = N ( t ( mdl, mg ) = = (5) ) d ( mdl ) ( mg ) 2 ( t, µ ) d ( t, µ ) (6) ( mdl ) d ( t, ) µ µ 5

6 where mdl represents the model; mg represents the mage. The energy functon conssts two parts: E (s) measures the shape dfference; E (d) measures the densty dfference. v s a vertex on the model; ( mdl) g ( mg) ( v ) s the surface normal at v on the model; g ( v ) s the mage ntensty gradent drecton at locaton v. The surface normal of v s computed by fttng a quadratc surface of the neghborng ( mdl) vertces of v. d ( t, µ ) s the densty value at a voxel µ wthn tetrahedron t j n the model; d ( mg ) j ( t, µ ) s the densty value at the correspondng voxel coordnate (t j, µ) n the mage. E (s) s the j ( mdl) sum of the dot product of g ( mg) ( v ) and g ( v ) over all vertces on the model; N(v) s the total number of vertces n the model. E (d) s computed by ntegratng the densty dfference over all tetrahedra usng the local barycentrc coordnates of each tetrahedron; N(t) s the total number of tetrahedra n the model. E (s) and E (d) are assgned weghts w s and w d respectvely, and w s +w d =. w s and w d are determned accordng to the mages and applcatons. For nstance, w s should be large n mages where edges are more promnent (such as mages of bony structures). Whle for mages where densty dstrbutons are more mportant (such as mages of brans and soft tssues), E (d) has a larger weght w d. In our pelvs regstraton problem, w s = 0.7 and w d = 0.3 were used. Some optmzaton algorthms [34], such as gradent descent methods, requre computng dervatves of the energy functon. In our method, the dervatves of the energy functon wth respect to both the affne transformaton parameters and the statstcal mode parameters are very dffcult to compute. Therefore, Powell s method [34] was chosen as our optmzaton algorthm snce t does not requre the dervatves of the objectve functon. To further mprove the effcency and robustness of the algorthm, the process s executed n a multple-resoluton framework. Ths nvolves frst searchng for the match n a coarser mage usng the lower resoluton model, and then refnng the soluton n a seres of hgher resoluton mages and models. The multple-resoluton mage space s mplemented usng a Gaussan mage pyramd descrbed n [35]. The mage on lower resoluton level s formed by smoothng the mage on ts prevous hgher level, followed by sub-samplng to obtan an mage wth half the number of pxels n each dmenson. The multple-resoluton model space s created based on edge collapsng technques, where the number of vertces on models on subsequent levels s approxmately halved. The algorthm s also mplemented n a multple-step-sze manner, n whch t starts wth a large step sze and gradually reduces the step sze as gettng closer to the optmal soluton. The multple-resoluton and multple stepsze scheme leads to a faster optmzaton process, and also makes the process less lkely to fall nto a local mnmum. 6

7 3.2 Affne transformaton An affne transformaton ncludes translaton T(t x, t y, t z ), rotaton R(r x, r y, r z ), and scale S(s x, s y, s z ). The affne transformaton process s a nne-dmensonal nonlnear optmzaton problem, whch can be wrtten as: arg mn R, S, T ( E( R S mdl + T, mg) ) (7) Here E(.) s the energy functon defned n Equaton 4; R, S and T are translaton, rotaton and scale, respectvely. The nne parameters n the affne transformaton are dvded nto three subsets - translaton, rotaton, and scale; durng each pass, only one subset of the parameters s optmzed. The parameters are optmzed back and forth several tmes to get the optmal parameters that mnmze the energy functon. After the optmzed affne transformaton s acheved, the locaton, orentaton, and sze of the model should roughly match wth the anatomcal structure. 3.3 Global deformaton As mentoned n Secton 2, gven a set of statstcal mode parameters b={b }, an nstance of the model can be nstantated. The new nstance Y of the model s equvalent to a warped verson of the average model (See Fgure 2). The operaton of changng the statstcal mode parameters to obtan a new model nstance s referred as the global deformaton of the model. The global deformaton stage can also be treated as an optmzaton procedure n the statstcal mode parameter space to mnmze an energy functon between the model and the mage. For each statstcal mode parameter beng evaluated, an nstance of the model s generated. Ths hypothess s then compared wth the mage usng the energy functon defned n Equaton 4. As n the affne transformaton stage, Powell s method s adopted to optmze the statstcal mode parameter set. The optmzaton problem can be wrtten as: b ( E( M ( mdl, b), mg) ) arg mn (8) Here, b={b } s the statstcal mode parameter of the anatomcal model, E(.) s the energy functon defned n Equaton 4, and M(.) (Equaton 3) s the nstantaton operaton to apply the statstcal mode parameters to the average model as defned. The set of parameters that mnmzes the energy functon s used to nstantate the regstered model. In our experment on the hem-pelvs model, frst fve most sgnfcant statstcal modes are optmzed. Our experment showed that these fve statstcal modes can cover about 93% of varablty n the tranng set. The global deformaton brngs the model very close to the anatomcal structure n the mage, but some dscrepances stll exst due to the lmted varablty ncorporated n 7

8 Input: A model M before local deformaton, and an mage I Output: A model M* after local deformaton Step : Intalzaton Step 2: for every vertex v on model M 2.) a multple-layer flexble mesh template T(v ) from model M s constructed (Secton 4.) 2.2) the attrbute vector A (m) (v ) for T(v ) s computed (Secton 4.2) 2.3) the correspondng mage coordnate c for v s located wthn a searchng range (Secton 4.3). end for Step 3: {v } are adaptvely deformed to correspondng mage coordnate {c }. (Secton 4.4) Step 4: Step 2 and Step 3 are repeated untl the dfference between models n two teratons s below a threshold or the maxmum number of teratons s reached. Fgure 3. Pseudo-code of local deformaton and correspondence procedure the model. The global deformaton stage provdes an ntal value and start pont for further local deformaton and correspondence fndng. 4. Correspondence and local deformaton Human anatomcal structures vary sgnfcantly among ndvduals, but the statstcal model only characterzes the varablty exhbted n the tranng set. Gven a specfc mage whch s not n the tranng set, there always exst varatons that cannot be reproduced from the statstcal model through model nstantaton. Therefore, a local deformaton stage s necessary after the global deformaton stage to address small dscrepances. The purpose of the local deformaton stage s to locally adjust the locaton of each vertex on the model to match local mages features. We proposed a multplelayer flexble mesh template matchng method to frst fnd the correspondences between model vertces and mage features, and then perform the local deformaton of the model. Fgure 3 provdes the pseudo-code of the correspondence fndng and local deformaton procedure. In the procedure, a multple-layer flexble mesh template s frst constructed for each vertex on the model, and the attrbute vector for each template s computed. Then the correspondng mage coordnate for each vertex s located usng flexble mesh template matchng technques. Fnally, the vertces are adaptvely deformed to ther correspondng mage coordnates. The above process s terated several tmes untl converges or the maxmum number of teratons s reached. 8

9 v (2) v 2 (2) v 3 (2) v 8 (2) v () v 4 () v (0) v 3 () v 2 () v 4 (2) v 7 (2) v 6 (2) v 5 (2) Fgure 4. Multple-layer flexble mesh template 4. Multple-layer flexble mesh template Due to the complexty n human anatomcal structures, t s not easy to fnd the correspondence between vertces on the model and mage voxel locatons. Shen and Davatzkos et al. [6] presented an affne nvarant attrbute vector usng the volumes of tetrahedra formed by the neghbors of vertces on the surface mesh to fnd the vertex correspondences. Inspred by ther work, we proposed a robust template matchng method to correlate each vertex on the model to ts correspondng voxel locaton. The tetrahedral mesh topologcal structure of the model s taken advantage of to construct a multple-layer flexble mesh template for each vertex n the model. For each vertex on the model, ts topologcal neghbors on the tetrahedral mesh and tself naturally form a mesh template centered at the vertex. The mesh template s retreved drectly from the model on the fly, so no extra storage s needed to keep the template. Fgure 4 shows a flexble mesh template n a 2D case, whch llustratng a two-layer mesh template wth 3 nodes. The vertces and edges n the fgure are vertces and edges n the tetrahedral mesh. The structure of the template centered at a vertex v (0) conssts of an array of nodes {v (k) }. For a node v (k), k s the topologcal dstance between the node and v (0) n the tetrahedral mesh, k s also referred as the layer number of node v (k), and s the ndex number of the node n layer k. Several attrbutes are assgned to a node v (k) (k) : ) the relatve poston to the center, p = v (k) -v (0) ; 2) a searchng sphere r (k) ; 3) an attrbute value and a weght. The template assocated wth a vertex v (0) has multple layers. v (0) s the center of the template, {v () } are nodes on the frst layer neghbors of v (0), and {v (2) } are on the second layer neghbors of v (0), and so on. The more layers a template has, the larger and more global structure t s able to characterze. Conversely, the fewer layers n a template, the smaller and more local structure t 9

10 descrbes. If too many layers are used, the structure may become too bg to have any correspondence n a local neghborhood. On the other hand, f too few layers are used, many correspondences may be avalable and t s dffcult to sngle out the correct correspondence. The example shown n Fgure 4 has two layers. The frst layer has four nodes {v (), v () 2, v () 3, v () 4 }, and the second layer has eght nodes {v (2),,v (2) 8 }. In most cases, a two-layer mesh template s adequate for fndng local correspondences. The template s also flexble. Ths means the locaton of a node on the template s not fxed, (k) nstead t can be any locaton wthn a searchng sphere r assocated wth the node. The crcles around each node n Fgure 4 represent the searchng spheres. In the flexble template matchng, the nodes on the template move wthn ther searchng spheres to fnd the best match. Due to the local deformaton, the mesh template may not have exact matched features n the mage. The searchng sphere makes the template flexble and able to match wth smlar features wthn a reasonable range. The center of the template has a searchng sphere wth zero radus, whch means the center node of the template s the only node on the template wth fxed locaton. The nodes farther from the template center have larger searchng spheres. Currently n our mplementaton, the radus of the searchng sphere s defned as /0 of the dstance from a node to the center node of the template. Ths s a value based on experence, whch may vary n dfferent applcatons. The mesh template has the followng advantages over an ordnary fxed-sze matrx template: ) t s retreved drectly from the tetrahedral mesh topology; 2) the multple-layer structure can be used to descrbe dfferent scales of anatomcal features; and 3) the flexble searchng sphere makes the template matchng robust to correlate smlar features. 4.2 Template attrbute vector and dynamc mage attrbute vector An attrbute vector s assgned to every mesh template. The attrbute vector descrbes both the densty propertes and shape propertes of the mesh template. Each node on the template s assocated wth an attrbute value ( d; g ) and a weght w, where d s the densty value of the node and g s the gradent drecton at the node locaton. The attrbute value of a node v (k) s drectly computed from the densty model. The densty value d s computed from the Bernsten polynomal densty functon assgned to each tetrahedron (Secton 2). The gradent orentaton s computed from mesh layer surface. The template attrbute vector s the concatenaton of the attrbute values of all nodes on the template, and can be wrtten as ( ) (0) (0) () () () () (0) (0) () () () () A m = ( w d, w d, w d ; w g, w g, w g ) (9)

11 A (c) A (c) 2 A (c) 3 v 0 v v 2 v 3 v4 (a) (b) (c) (d) Fgure 5. Example of flexble template matchng (see text for explanatons) here ( w ( w ( k ), d, g (0) (0) 0 d0, ( k ), g ( k ) (0) 0 ) are the weght and attrbute value of the center node of the template, and ) are the weght and attrbute value of node on layer k,.e. node v (k). A template attrbute vector A (m) s computed for every vertex on the model. In the template matchng and correspondence fndng process (detals n Secton 4.3), the mesh template of a vertex v s moved n the mage feld and the best matched voxel locaton for vertex v s located. For any voxel locaton c n the mage space, an attrbute vector A (c) s defned for the gven mesh template of vertex v. The mage attrbute vector has the same format as the template attrbute vector A (m) of v, but the attrbute values of nodes are obtaned from correspondng voxel locatons. Because a node v (k) on the mesh template s assocated wth a searchng sphere r (k), when retrevng the attrbute value at a voxel locaton, the regon wthn the searchng sphere r (k) (k) of node v s searched for the most alke attrbute value. The attrbute vector of a voxel s called the dynamc mage attrbute vector snce t vares for dfferent mesh templates. The goal of template matchng s to fnd the correspondng voxel locaton for each vertex, where the template attrbute vector A (m) of the vertex and the dynamc mage attrbute vector A (c) are best matched. 4.3 Flexble mesh template matchng and correspondence fndng A template matchng method s employed to fnd the vertex correspondence because of ts smplcty and computatonal effcency. Template matchng s a smple flterng method to detect a partcular shape or object n an mage. An object can be detected f ts appearance s known accurately n terms of a template. In our applcaton, we want to detect the anatomcal structure defned by the mesh structure assocated wth each vertex on the model. The dea of rgd template matchng s straghtforward. The template s moved n the mage feld, and the dfference between the template and the mage s calculated at each voxel locaton. An optmal match s reported at the

12 locaton where the mnmum dfference between the template and the mage s reached. A shortcomng of the rgd template matchng s that t requres the template be very precse, so t s senstve to shape and densty varaton. We proposed a flexble mesh template matchng to overcome ths problem. Each node on the template s assocated wth a searchng sphere r (k). And the search for best matched voxel locaton for each node s conducted wthn ts searchng sphere. Essentally n flexble template matchng, the process s broken down nto two steps: frst the best match for each node on the template s located wthn ts searchng sphere, and then the match of the entre template s computed usng the best match of each node. The goal of our template matchng method s to fnd the correspondng voxel locaton n the mage space for each vertex on the model, so that a vertex can be warped to ts correspondng voxel locaton. Each vertex s assocated wth a mesh template and a template attrbute vector A (m). The approach to fnd the correspondng mage coordnate s to move the mesh template over the mage space to fnd the most smlar dynamc mage attrbute vector. Mnmzng the dfference between the template attrbute vector and dynamc mage attrbute vectors, a best template matchng can be found, and so does the correspondng mage coordnate for the vertex. The followng equaton s the way to compute the dfference between two attrbute vectors A (m) and A (c) : Dff ( A ( m), A ( c) ) = ( len( A m) ) ( m) len( A ) = ( m) ( d d w ( m) d c) + ( len( A m) ) ( m) len( A ) = w ( m) ( c) ( g g ) 2 (0) here A (m) s the attrbute vector of the mesh template, and A (c) s the mage attrbute vector of a voxel locaton. It contans two parts, the densty vector dfference and the gradent vector dfference, and each part has been normalzed to range (0 ). The densty vector dfference evaluates the average percentage dfference of densty values, and the gradent vector dfference measures the average dot product of gradent drectons. The two parts are weghed by ther weght. In order to demonstrate the flexble mesh template matchng, Fgure 5 shows a smple example n 2D space. The mesh template n Fgure 5a s a smplfed 2D two-layer mesh template wth fve nodes. v 0 s the center node of the mesh template; v and v 2 are the frst layer neghbors; v 3 and v 4 are the second layer neghbors. Its template attrbute vector can be wrtten as: ( ) A m = w d, w d, w d, w d, w d ; w g, w g, w g, w g, w g ) () ( By movng ths mesh template over the mage space, dynamc mage attrbute vectors are evaluated at dfferent voxel locatons, denoted as A (c), A (c) 2, A (c) 3,. Fgure 5b-5d demonstrates some dynamc mage attrbute vectors at dfferent voxel locatons. Fgure 5b shows a totally unmatched voxel locaton. Fgure 5c shows a locaton wth partal matchng by applyng the rgd template matchng. After searchng for the best match of each node wthn ts searchng sphere, a perfect match s found 2

13 wthout morphng (a) Gaussan morphng (b) (c) Fgure 6. Gaussan morphng n Fgure 5d. The template dfference between Fgure 5c and Fgure 5d (the dfference s exaggerated for the purpose of vsualzaton) demonstrates the dea of the flexble template matchng. Hence, the voxel locaton of the mage attrbute vector A (c) 3 n Fgure 5d s the correspondng voxel locaton of the center vertex v 0 of the mesh template. At the end, vertex v 0 can be locally and adaptvely warped to ts correspondng voxel locaton. 4.4 Adaptve deformaton and constrants If any deformablty s allowed, one can always deform one object nto any other objects (e.g. morphng a head nto a teapot). Some constrants are necessary for local deformaton, especally n anatomcal structures. In order to make the local deformaton smooth and keep the tetrahedral mesh structure vald, several strateges were adopted, ncludng Gaussan morphng, adaptve deformaton focus, and maxmum deformaton range. Among these technques, adaptve focusng and Gaussan morphng are nspred by Shen s work [6]. To keep the deformaton contnuous and smooth, a Gaussan morphng strategy s adopted. In ths strategy, when one vertex s moved, ts neghbors wll also be morphed accordngly. The followng equaton descrbes the Gaussan morphng operaton, v = v v 0 0' l l 2 2σ v = v e here v 0 s the vertex locaton before deformaton, v 0 s the correspondng locaton obtaned from the template matchng, v s the movement of the deformed vertex, 0 v s the movement of ts l th layer l neghbors, σ s the morphng parameter. Fgure 6 llustrates the effect of Gaussan morphng n 2D. Fgure 6a shows a vertex and ts neghbors. In Fgure 6b, the vertex deforms wthout the morphng of ts neghbors. In Fgure 6c, the vertex deforms wth Gaussan morphng of ts neghbors. The (2) 3

14 Reach low threshold Intermedate Reach hgh threshold Free Neghbor of stable vertces Stable Neghbor of stable vertces Restrcted Reach hgh threshold Fgure 7. Status transton of vertces n local deformaton transton from Fgure 6a to Fgure 6c s much more contnuous and smoother than that from Fgure 6a to Fgure 6b. In addton, an adaptve deformaton focus strategy s appled,.e. the vertex wth hghest matched mage attrbute vector wll deform frst and drag ts neghbors to morph. Then, the focus wll move to the next hghest matched vertex. The adaptve focusng can prevent the deformaton from runnng randomly by restrctng the deformaton to the most promnent matches frst. The thrd strategy to prevent nvald deformaton s to assgn a maxmum deformaton range to each vertex v, whch essentally restrcts the possble deformaton range of a vertex wthn a sphere n one teraton. The radus of the maxmum deformaton range s defned as half of the dstance between a vertex and ts closest neghbor. Ths range s adaptvely updated at the end of each teraton. Durng the local deformaton stage, each vertex s labeled wth a status value. There are four types of status values: Free, Intermedate, Restrcted, and Stable. All vertces are ntalzed as Free vertces. When the template matchng of a vertex (.e., the dfference between ts template attrbute vector and ts best matched mage attrbute vector) s hgher than a low threshold, t becomes an Intermedate vertex. When the template matchng of a vertex reaches a hgh threshold, t becomes a Stable vertex. Stable vertces cannot be deformed any more. The frst layer neghbors of Stable vertces are defned as Restrcted vertces. Free, Intermedate and Restrcted vertces have dfferent deformaton ranges. Intermedate vertces have less freedom n deformaton than Free vertces. And Restrcted vertces have even restrctve deformaton ranges. Fgure 7 llustrates the status transton of vertces. Only Intermedate and Restrcted vertces can be deformaton focus. Free vertces can t be deformaton focus and can only be morphed by ts 4

15 (a) (b) (c) (d) Fgure 8. Vsual results of non-rgd regstraton between the statstcal model and CT mages (a) Intal state; (b) after affne transformaton; (c) after global deformaton; (d) after local deformaton neghbors. The low threshold and the hgh threshold used n our nvestgaton are 0.5 and 0.9, where 0 ndcates totally unmatched and ndcates perfectly matched. The status transton restrcts the deformaton of certan vertces to guarantee the smoothness and contnuty of the deformaton. 5. Results and Valdaton We have tested the non-rgd regstraton algorthm usng a statstcal densty model of hempelvs. The statstcal model s computed from eght tranng mages (Secton 2). An addtonal pelvs CT mage was then acqured, and the non-rgd regstraton was appled between the statstcal model and the CT mage. Fgure 8 shows some vsual results of the regstraton process at dfference stages. The frst row s a 3D vsualzaton of the process n a volume renderng mode, and the second row shows one cross secton of the volume. The model s supermposed on the CT mage to llustrate the match between the model structure and the anatomcal structure. Fgure 8a s the ntal state of the model. Fgure 8b s the result after the affne transformaton. Fgure 8c s the result after the global deformaton by optmzng fve statstcal mode parameters. After ths stage, the shape of the model roughly matches the pelvs boundary, but some small dscrepances stll exst due to lmted varablty n the model. Fnally, Fgure 8d s the result after the local correspondence fndng and deformaton stage. The supermposng of the model and the mage demonstrates a very close match between the pelvs boundary and the deformed model. 5

16 Deformable Atlas/CT Regstraton Affne Transformaton Global Deformaton Local Deformaton Energy Functon Iteraton Fgure 9. Energy functon n non-rgd model/ct regstraton Fgure 9 plots the value of the energy functon between the statstcal model and the CT mage durng the regstraton procedure. X-axs denotes the teraton ndex, and Y-axs s the value of the energy functon (Equaton 4). The energy functon s normalzed to the range of [0 00], where 0 means the model and the mage are perfectly matched, and 00 means that they are totally unmatched. From Fgure 9, the ntal value of the energy functon s about 90. After the affne transformaton stage, t drops to around 20. And at the end of the global deformaton stage, t decreases to around 6. Fnally, the local deformaton stage further mproves the energy functon to about 3. There are several jumps n the plot; the jump at st teraton s the result of movng the centrod of the model to the center of the mage volume; other jumps are the result of changng mage resoluton n the mult-resoluton scheme. The total process takes about 0 mnutes on a Pentum III 850 PC. Among these, the ntalzaton stage takes about 2 mnutes, the affne transformaton stage takes about mnute, the global deformaton stage takes about 2 mnutes, and the local deformaton stage takes about 5 mnutes. Currently there are eght pelvs CT mages n the tranng set. A leave-one-out valdaton was conducted on the tranng data sets. One data set n the tranng set was selected as the testng data, and the other seven data sets were used to buld a statstcal model. Then the statstcal model was regstered wth the testng data set, and the regstraton results were valdated wth a ground truth model. The ground truth model was obtaned by a sem-automatc segmentaton followed by manually verfcaton and adjustment. Volume and surface-based error metrcs have been mplemented for the valdaton. The error metrcs measure the overlapped volume of two volumetrc 6

17 models and the surface dstance. Frst a scan of voxels nsde the ground truth model was obtaned. And a scan of voxels nsde the model produced by the non-rgd regstraton process was also generated. Assume model s the model produced by the regstraton procedure, and model 2 s the ground truth model. The percentage of the number of overlappng voxels n model and model 2 s computed, wrtten as Overlap = V V V 00% (3) 2 where V s the set of voxels n model, V 2 s the set of voxels n model 2, and represents the sze of a set; and the dstance between exteror surfaces of the two models s also evaluated, ncludng AvgD = avg µ V MaxD = max µ V ( d( µ, p( µ ))), ( d( µ, p( µ ))) where µ s the vertex on the exteror surface of V, and p(µ ) s closest pont of µ on the exteror surface of V 2, d(*,*) s the dstance between two ponts. AvgD measures the mean dstance, MaxD measures the maxmum dstance. The unt s mm. (4) Table. Leave-one-out valdaton of non-rgd model/ct regstraton Intal stage Affne transformaton Global deformaton Local deformaton (fnal result) Data set Overlap AvgD Overlap AvgD Overlap AvgD MaxD Overlap 0.0% % % % 2 5.4% % % % 3 0.0% % % % 4 0.0% % % % 5 0.0% % % % 6.3% %.5 9.6% % 7 3.8% % % % 8 0.0% % % % Avg.3% % % % Std Dev 2.% % % % Table lsts the results of the leave-one-out valdaton experment. The results were evaluated at the begnnng of the regstraton and at the end of each stage. The overlap percentage (Equaton 3) and the dstance between exteror surfaces (Equaton 4) of the transformed model and the ground truth model were reported. In the table, AvgD s the mean surface dstance, and MaxD s the maxmum surface dstance. Overlap s the percentage of overlappng voxels to all voxels wthn the model. In the average case, about 94% overlap between the ground truth model and the regstered model was acheved. The leave-one-out valdaton result n Table and the vsual result n Fgure 8 7

18 proved that our method based on statstcal models and flexble mesh templates s effectve on nonrgd regstraton of bony structures. The effect of several control parameters n the local deformaton algorthm was also evaluated. The parameters versus the regstraton accuracy (volume overlap) are lsted n Table 2. Among those parameters, w s and w d are the weghts for shape propertes and densty propertes; Template layer s the number of layers n the mesh template; Flexble search range s the search range assocated wth each node on the template; Gaussan morphng s the morphng parameter; and Transt_low and transt_h are thresholds used n the status transton. The detaled descrptons of the parameters can be found n Secton 4. For each parameter, we evaluated one value used n our current algorthm (n bold font, Tral 2) and two other values. We have roughly optmzed the parameters for our applcaton. However, the comparson also showed that usng other parameters doesn t degrade the results too much. Table 2. Control parameters vs. regstraton results Control Parameters Tral Overlap Tral 2 Overlap Tral3 Overlap w s and w d 0.3, % 0.7, % 0.8, % Template Layer 94.4% % % Flexble search range % % % Gaussan morphng % % % Transt_low, transt_h 0.3, % 0.5, % 0.6, % 6. Dscusson We have presented a new method for non-rgd medcal mage regstraton and model-based segmentaton usng a statstcal bone densty model. The model s represented as a tetrahedral mesh and contans both shape and densty propertes and ther varablty. We also proposed a multplelayer flexble mesh template to fnd the correspondence between vertces on the model and voxel locatons n the mage. The statstcal model s computed from a populaton of tranng models, so t characterzes the varablty nherent n the tranng set. To compensate for the varaton not exhbted n the tranng models, further local deformaton s performed usng a multple-layer flexble mesh template assocated wth each vertex on the model. Allowng local deformaton leads to refned local matches, and s more effectve than smply addng more models to the tranng set. The template tself s flexble and non-rgd so that t can be used to correlate approxmate features. Its multple-layer structure allows the potental to characterze dfferent sze of structures. 0-layer template descrbes the feature of a sngle pont. If suffcent layers are used, the mesh template can be used to 8

19 characterze the entre model. The mesh template also provdes a way to combne the shape feature and densty feature of an anatomcal structure n a sngle data structure. The experments of the mesh template matchng on bony anatomes have been proved successful. Snce the dea of the statstcal model and mesh template s generc, ths technque can be extended to other anatomcal structures ncludng soft tssues. In the experment wth pelvs mages, more weghts were put on shape propertes than on densty propertes snce the bone boundary s more promnent than densty dstrbuton nsde the bone. In other applcatons, such as bran regstraton where the boundary s not as clear as bones and the nformaton n densty dstrbuton s relatvely rch, more emphass can be put on the densty propertes. The regstraton process has three stages: affne transformaton, global deformaton, and local deformaton. Among these, global deformaton requres a statstcal model bult from a tranng set and provdes a start pont for further local deformaton. If the statstcs s not avalable, t s possble to skp the global deformaton stage. However, snce the local deformaton stage only has a lmted deformaton range and the template matchng tends to match local features, f the varablty of the structure s too bg, t may not be able to fnd matches for some vertces n ther neghborhoods. We may ncrease the flexblty of the template and the searchng range, but t would ntroduce a lot of ambguty and t s dffcult to determne the correct correspondence. We adopted a multple-resoluton and multple-stepsze scheme n the regstraton. We tred an experment usng just hghest resoluton mage and model. The optmzaton process stuck n the early stage of the regstraton. It s stll possble that regstraton converges to a local mnmum n the multple-resoluton scheme. But we haven t encountered ths problem n the applcaton of pelvs, partally because pelvs structure s not symmetrc and has dstngushable local features. In future works, we plan to buld a bone densty atlas based on the statstcal bone densty model to ncorporate more nformaton. In ths atlas, anatomcal landmarks are labeled and certan surgcal procedures are defned. Once the atlas s regstered to a specfc patent mage, the nformaton stored n the atlas can be transferred to the patent. We also plan to further extend ths model to other anatomes such as knees and vertebras. In our other work [36, 37], we have demonstrated 2D/3D nonrgd regstraton between the statstcal model and a set of X-ray mages. In general, the 2D/3D technque can be used to perform 3D patent specfc modelng and analyss wthout a patent specfc CT mage. Clncal tasks such as plannng from X-rays, ntra-operatve gudance, post-operatve analyss and retrospectve studes are also potental applcatons of anatomcal atlas and regstraton technques. 9

20 Acknowledgement Ths work was partally funded by NSF Engneerng Research Center grant EEC We thank Shadysde Hosptal for provdng the pelvs data. We specally thank Dnggang Shen for hs suggeston on deformable regstraton algorthm. We also thank Jerry Prnce, Chrstos Davatzkos, and Chengyang Xu for ther useful dscussons. Reference. Chen, M., et al., 3-D Deformable Regstraton of Medcal Images Usng a Statstcal Atlas. 998, Carnege Mellon Unversty: Pttsburgh, PA. 2. Cootes, T.F., et al. A Unfed Framework for Atlas Matchng usng Actve Appearance Models. n IPMI 999. Spnger. 3. Fleute, M. and S. Lavallee. Nonrgd 3-D/2-D Regstraton of Images Usng Statstcal Models. n MICCAI 999. Cambrdge, UK. 4. Lowe, D.G., Fttng Parameterzed Three-Dmensonal Models to Images. IEEE Transactons on Pattern Analyss and Machne Intellgence, 99. 3(5): p Maurer, C.R., et al., Regstraton of 3-D Images Usng Weghted Geometrcal Features. IEEE Transactons on Medcal Imagng, (6): p Shen, D. and C. Davatzkos. Adaptve-Focus Statstcal Shape Model for Segmentaton of 3D MR Structures. n MICCAI Pttsburgh, PA. 7. Leventon, M.E., Statstcal models n medcal mage analyss, n Electrcal Engneerng and Computer Scence. 2000, MIT. p Mller, M.I., et al., Mathematcal textbook of deformable neuroanatomes. Proceedngs of the Natonal Academy of Scence, (24): p Haller, J., et al., Hppocampal MR magng morphometry by means of general pattern matchng. Radology, : p Evans, A., et al., MRI-PET correlaton n three dmensons usng a volume-of-nterest (VOI) atlas. Journal of Cerebral Blood Flow and Metabolsm, 99. (2): p. A Woods, R., et al., Automated mage regstraton: II. Intersubject valdaton of lnear and nonlnear models. Journal of Computer Asssted Tomography, (): p Hawkes, D.J., Revew: Algorthms for radologcal mage regstraton and ther clncal applcaton. J. Anat., : p Rohde, G.K., A. Aldroub, and B.M. Dawant. Adaptve free-form deformaton for nter-patent medcal mage regstraton. n SPIE Medcal Imagng 200. San Dego, CA. 4. Rueckert, D., et al., Nonrgd regstraton usng free-form deformatons: Applcaton to breast MR Images. IEEE Transactons on Medcal Imagng, (8). 5. Feldmar, J. and N. Ayache, Rgd, Affne and Locally Affne Regstraton of Free-Form Surfaces. 994, INRIA. 6. Ferrant, M., et al., 3D Image Matchng Usng a Fnte Element Based Elastc Deformaton Model. 999, Radology, Brgham and Women Hosptal, Harvard Medcal School: Boston, USA. 7. Thompson, P. and A.W. Toga, A surface-based technque for warpng three dmensonal atlas to match anatomcal bran mages. IEEE Trans. Med. Imag., : p Collns, D., et al., Automatc 3D Intersubject Regstraton of MR Volumetrc Data nto Standardzed Talarach Space. Journal of Computer Asssted Tomography, (2): p Warfeld, S., et al. Automatc Identfcaton of Gray Matter Structures from MRI to Improve the Segmentaton of Whte Matter Lesons. n Medcal Robotcs & Computer Asssted Surgery (MRCAS) Warfeld, S., et al., Adaptve, template moderated, spatally varyng statstcal classfcaton. Medcal Image Analyss, (): p Booksten, F.L., Morphometrc tools for landmark data, Geometry and bology. 99: Cambrdge Unversty Press. 20

21 22. Bauer, A. Prmary THR Usng the ROBODOC System. n CAOS/USA Pttsburgh, Pennsylvana, USA. 23. Krettek, C., et al. Computer Asssted Femoral Nalng. n 4th Internatonal Symposum on CAOS, 999. Davos, Swtzerland. 24. Subsol, G., J.-P. Thron, and N. Ayache, A General Scheme for Automatcally Buldng 3D Morphometrc Anatomcal Atlas: Applcaton to a Skull Atlas. 995, INRIA: France. 25. Wells, W.M., et al., Adaptve segmentaton of MRI data. IEEE Transactons on Medcal Imagng, (4): p Van Den Elsen, P., et al. Grey value correlaton technques used for automatc matchng of CT and MR volume mages of the head. n SPIE Medcal Imagng Studholme, C., D. Hll, and D. Hawkes. Multresoluton voxel smlarty measures for MR-PET regstraton. n IPMI, Thron, J.-P., Fast Non-Rgd Matchng of 3D Medcal Images. 995, INRIA. 29. Kass, M., A. Wtkn, and D. Terzopoulos, Snakes: Actve Contour Models. Internatonal Journal of Computer Vson, 988: p Meyers, D., S. Sknner, and K. Sloan, Surfaces from Contours. ACM Transactons on Graphcs, 992. (3): p Trotts, I.J., B. Hamann, and K.I. Joy, Smplfcaton of Tetrahedral Meshes wth Error Bounds. IEEE Transactons On Vsualzaton and Computer Graphcs, (3): p. pp Yao, J. and R. Taylor. Tetrahedral Mesh Modelng of Densty Data for Anatomcal Atlases and Intensty-Based Regstraton. n MICCAI Pttsburgh, PA, USA: Sprnger. 33. Yao, J. and R. Taylor. Constructon and Smplfcaton of Bone Densty Models. n SPIE Medcal Imagng 200. San Dego, CA. 34. Press, W.H., et al., Numercal Recpes n C. Second ed. 992: Cambrdge Unversty Press. 35. Cootes, T.F. and C.J. Taylor, Statstcal Models of Appearance for Computer Vson Yao, J., A statstcal bone densty atlas and deformable medcal mage regstraton, n Computer Scence. 200, the Johns Hopkns Unversty: Baltmore, MD. p Yao, J. and R. Taylor. Deformable regstraton between a statstcal bone densty atlas and X-ray mages. n Internatonal CAOS