Point Similaity Measues Based on MRF Modeling of Diffeence Images fo Spline-Based D-D Rigid Registation of X-ay Fluooscopy to CT Images Guoyan Zheng, Xuan Zhang, Slavica Jonić,, Philippe Thévenaz, Michael Unse, and Lutz-Pete Nolte MEM Reseach Cente, Univesity of Ben, Stauffachestasse 78, CH-04, Ben, Switzeland Guoyan.Zheng@MEMcente.unibe.ch Biomedical Imaging Goup, École polytechnique fédéale de Lausanne (EPFL, CH-05, Lausanne VD, Switzeland Institut de Minéalogie et de Physique des Milieux Condensés, Univesité Piee et Maie Cuie, F-7505 Pais, Fance Abstact. One of the main factos that affect the accuacy of intensity-based egistation of two-dimensional (D X-ay fluooscopy to thee-dimensional (D CT data is the similaity measue, which is a citeion function that is used in the egistation pocedue fo measuing the quality of image match. This pape pesents a unifying famewok fo ationally deiving point similaity measues based on Makov andom field (MRF modeling of diffeence images which ae obtained by compaing the efeence fluooscopic images with thei associated digitally econstucted adiogaphs (DRR s. The optimal solution is defined as the maximum a posteio (MAP estimate of the MRF. Thee novel point similaity measues deived fom this famewok ae pesented. They ae evaluated using a phantom and a human cadaveic specimen. Combining any one of the newly poposed similaity measues with a peviously intoduced spline-based egistation scheme, we develop a fast and accuate egistation algoithm. We epot thei captue anges, conveging speeds, and egistation accuacies. Intoduction One of the main factos that affect the accuacy of intensity-based D-D egistation is the similaity measue, which is a citeion function that is used in the egistation pocedue fo measuing the quality of image match. An extensive study of six similaity measues applied specifically to D-D egistation has been pefomed by Penney et al. []. The similaity measues consideed by the authos wee: nomalized coss-coelation [], entopy of the diffeence image [], patten intensity [4], mutual infomation [5], gadient coelation [6], and gadient diffeence []. Using the fiducial makes to get the gold-standad egistation, the authos anked these measues based on thei accuacy and obustness. They found that patten intensity was one of the two similaity measues that wee able to egiste accuately and obustly, even when soft tissues and inteventional instuments wee pesent in the X-ay images. Unfotunately, patten intensity was designed by using some heuistic ules [4]. J.P.W. Pluim, B. Lika, and F.A. Geitsen (Eds.: WBIR 006, LNCS 4057, pp. 86 94, 006. Spinge-Velag Belin Heidelbeg 006
Point Similaity Measues Based on MRF Modeling of Diffeence Images 87 This wok fomulates a MRF model on the diffeence images obtained by compaing the input fluooscopic images with thei associated DRR s. The optimal solution is defined as the MAP estimated of the MRF. By using this unifying MAP-MRF famewok, we can deive new point similaity measue in a ational way. The optimization of each individual similaity measue deived fom this famewok leads to optimal egistation. We point out that two peviously published similaity measues, i.e., sum-of-squaed-diffeence (SSD [7] and patten intensity [4], can be also deived fom this famewok. The emainde of this pape is oganized as follows. Section biefly intoduces the D-D egistation scheme used in this pape. Section descibes the deivation of point similaity measues based on MRF modeling of the diffeence images. Section 4 pesents the expeimental esults, followed by conclusions in Section 5. Spline-Based D-D Registation Scheme The D-D egistation scheme used in this pape is based on a ecently intoduced spline-based multi-esolution D-D egistation scheme [7, 8]. This scheme follows the computation famewok of intensity-based methods. Given a set of X-ay images and a CT volume, it iteatively optimizes the six igid-body paametes descibing the oientation and the tanslation of the patient pose, by geneating and compaing floating DRR s with the efeence X-ay images using appopiate similaity measue. The diffeences between this method and othe intensity-based methods lie in [7]: a cubic-splines data model was used to compute the multi-esolution data pyamids fo both CT volume and X-ay images, the DRR s, as well as the gadient and the Hessian of the cost function; a Maquadt-Levenbeg non-linea least-squaes optimize was adapted to a multi-esolution context. The egistation was pefomed fom the coasest esolution until the finest one. The accuacy of this method depends on the chosen similaity measue. Peviously, accuacy of appoximately.4 ± 0. mm when SSD was used [7] has been epoted. Deiving Point Similaity Measues Based on MRF Modeling of Diffeent Images To find an optimal egistation tansfomation we cast the poblem into a Bayesian famewok of MAP-MRF estimate. We thus follow the fou steps of the MAP-MRF estimate [9].. Constuction of a pio pobability distibution p (T fo the egistation tansfomation T matching the efeence X-ay images to the floating DRR s.. Fomulation of an obsevation model p ( D T that descibes the distibution of the obseved diffeence images D by compaing the efeence X-ay images and the floating DRR s given any paticula ealization of the pio distibution.
88 G. Zheng et al.. Combination of the pio and the obsevation model into the posteio distibution by Bayes theoem p( T D p( D T p( T ( 4. Dawing infeence based on the posteio distibution.. Pio Distibution One advantage of fomulating D-D egistation accoding to Bayesian famewok is that we ae able to specify a pio distibution fo each configuation of egistation paamete space. In this pape, we don t take advantage of this popety. We teat all paamete configuations equally. Due to the Eule angle based paameteization of otation in ou appoach, p (T ae a unifom distibution. But it is possible to use this popety to favo cetain tansfomations when diffeent paameteization foms such as quatenion ae used.. Obsevation Model Given a ealization of the pio distibution, the obsevation model p(d T descibes the conditional distibution of the obseved diffeence images D. By specifying an obsevation model we may favo a tansfomation that establishes matching between egions of simila popeties. By modeling the diffeence image D as a MRF with espect to the th ode neighbohood system N = { N i } we can deive the enegy, function fo the obsevation model as: E( D T = Q [ α V ( d + ( α V ( d, d ' ' ] ( i, cad ( N ' ' q = ( i, N i, whee Q is the numbe of images and I J is the size of each image. The fist tem is the potential function fo single-pixel cliques and the second tem is the potential function fo all othe cliques. α [ 0 : ] weights the influence of these two tems. cad means to compute the numbe of pixels in neighbohood N,. ( N i, The selection of the potential functions in Eq. ( is a citical issue in MRF modeling [9]. As pointed out below, its selection decides the fom of similaity measue. The computation of the diffeence images also plays an impotant ole in the pesent famewok. In [4], an adustable scaling paamete was used to build the diffeence images. To eliminate this paamete, Jonić et al. [7] ties to nomalize the intensity ange of the input efeence fluooscopic images and that of the coesponding DRR s by emoving thei mean and then dividing by thei standad deviation. In this pape, we use a simila method. But unlike in [7], whee the mean and the standad deviation wee computed fom the complete egion of inteest (ROI, we com- pute them only using those pixels in the neighbohood N,. i i
Point Similaity Measues Based on MRF Modeling of Diffeence Images 89. MAP Estimate The posteio conditional pobability distibution is given by: In seach fo the MAP estimate: p( T D exp( E( D T T = ag max ( T p( T D To illustate how to deive similaity measues using the pesent famewok, two examples of peviously published similaity measues ae given as follows. Sum-of-Squaed-Diffeence (SSD: It can be deived fom Eq. ( by specifying α = and ( d i = d. V, Patten Intensity: the patten intensity poposed in [4] is witten in the fom: P, = cad( N ' ' ( i, N i, σ σ + ( d ' ' d σ (5 i, whee and σ ae two paametes to be expeimentally detemined. N i, is a neighbohood with adius. It can be deived fom the pesent famewok by specifying α = 0 and using following paiwise clique potential function: V ( d, d ' ' i, = ( d + whee d is a pixel in the neighbohood N,. i ', ' i ' ' i,.4 Deiving New Point Similaity Measues d σ Moe geneally, by choosing diffeent neighbohood system and by specifying diffeent clique potential functions that incopoates diffeent a pioi constaints, we can deive diffeent new similaity measues. Isotopic th ode neighbohood system and paiwise potential function with st ode smoothness constaint (INS: It is defined using following equation: α d + ( α ( d ' ' d i, (7 cad ( N ' ' ( i, N i, It is actually a combination of SSD and a modified fom of patten intensity [0]. Following the suggestion in [4], we also choose = pixels. Fom now on, we call this similaity measue INS. (4 (6
90 G. Zheng et al. Two anisotopic similaity measues can be deived using following equation: P, (, y = α di + ( α ( d(, x + d α (8 d x d y (, x ( (, y whee d = ; d = ( is the fist deivatives of the diffeence image D along X and Y diections, espectively. Anisotopic 4-neighbohood system and potential functions with fist ode smoothness constaint (AN4S: It computes the fist deivative in Eq. (8 using 4-neightbohood system with following convolution masks: [ 0 ] fo the detemination of d (, x and [ 0 ] T fo the detemination of d (, y Anisotopic 8-neighbohood system and potential functions with fist ode smoothness constaint (AN8S: It also computes the fist deivative in Eq. (8 using 4-neightbohood system but with following convolution masks: 0 0 fo the detemination of d (, x and 0 0 0 fo the detemination of 0 d (, y 4 Expeiments A phantom and a human cadaveic spine specimen togethe with thei gound tuths wee used in ou expeiments. Both phantom and cadaveic specimen wee scanned by a GE LightSpeed Ulta CT scanne (GE Healthcae, Chalfont St. Giles, United Kingdom with same inta-slice solution (0.6 mm x 0.6 mm but with diffeent inte-slice thickness,.5 mm fo the phantom and.5 mm fo the cadaveic specimen, which esulted in volume dataset of size 5x5x9 volxels fo phantom and 5x5x7 fo the cadaveic specimen, espectively. The D poection images of both phantom and cadaveic specimen wee acquied fom a Siemens ISO-C C-am (Siemens AG, Elangen, Gemany. They ae calibated and undistoted with custommade softwae with high accuacy. The phantom was custom-made to simulate a good
Point Similaity Measues Based on MRF Modeling of Diffeence Images 9 Fig.. Behavio of diffeent similaity measues. Cut though the minimum of diffeent similaity measues on the phantom data (the st and nd ows as well as on the cadaveic spine specimen (the d and 4 th ows. The odinate shows the value of diffeent similaity measues (they ae nomalized to the ange [0.0,.0], which ae given as functions of each igid tansfomation paamete in the ange of [-5 o, 5 o ] o [-5 mm, 5 mm] away fom the its gound tuth (( st column of the st and d ows: X otation; ( nd column of the st and d ows: Y otation; ( d column of the st and d ows: Z otation; (4 st column of the nd and 4 th ows: X tanslation; (5 nd column of the nd and 4 th ows: Y tanslation; (6 d column of the nd and 4 th ows: Z tanslation. Zeo in each abscissa means the gound tuth fo that individual paamete, obtained by paied point matching based on fiducial makes. condition. In contast, poections of inteventional instuments wee pesent in the X- ay images of the cadaveic specimen to simulate a pactical situation in image-guided theapy. The gound tuths wee obtained by implanting fiducial makes. Both phantom and cadaveic specimen wee equipped with infaed light emitting diodes (LEDs makes to establish a patient coodinate system (P-COS and was tacked using an optoelectonic position senso (OptoTak 00, Nothen Digital Inc., Wateloo, Canada. The actual locations of fiducial makes wee digitized in P-COS using an optoelectonically tacked pointe and wee matched to the coesponding points in CT volume dataset. The gound tuths wee then obtained using singula value decomposition with an accuacy of 0.5 mm fo phantom and 0.65 mm fo cadave, espectively.
9 G. Zheng et al. Fo all thee newly deived similaity measues, the paamete α was chosen as 0.5. Each time, two nealy othogonal C-am images fom the coesponding dataset wee used fo the expeiments descibed below. The fist expeiment was designed to compae the behavios of the newly deived similaity measues to those of the published similaity measues such as SSD and mutual infomation. Though mutual infomation was anked as least accuate in [], othe goup [, ] late found that it pefomed easonably well. The esults wee given in Figue. It was found that all similaity measues had simila behavio when tested on the phantom data but diffeent behavio when tested on the cadaveic data. Those similaity measues deived fom the pesent MAP-MRF famewok showed a supeio behavio compaed to othe two well-known similaity measues. Moe specially, the cuves fo the newly deived similaity measues have clea minima and ae smoothe, which is an impotant popety to take the advantage of ou D-D egistation scheme, which uses a gadient-based optimization technique. It is also evident that the behavio of mutual infomation is bette than that of SSD. Fig.. Expeimental esults of captue anges (left and conveging steps (ight Combining any one of the similaity measues with the D-D egistation scheme descibed in Section, we developed a D-D egistation algoithm. The second expeiment was designed to evaluate thei captue anges, conveging steps, and egistation accuacies of these egistation algoithms. Based on the investigation esults obtained in the fist expeiment, we only pefomed this expeiment on the human cadaveic specimen dataset to compae the thee newly deived similaity measues. Fo this pupose, we petubed the gound tuth tansfomation by andomly vaying each egistation paamete in the ange of [- o, o ] o [-mm, mm] to get 00 positions, and then anothe 00 positions in the ange of [-4 o, 4 o ] o [-4mm, 4mm], and so on until the final ange of [- o, o ] o [-mm, mm]. We then pefomed ou egistations and counted how many times they conveged fo each ange (when the taget egistation eo (TRE measued on those fiducial makes was less than.5 mm. The captue ange was defined when thee was at least 95% successful
Point Similaity Measues Based on MRF Modeling of Diffeence Images 9 Table. Results of egistation accuacies ate. The expeimental esults on captue anges and conveging steps ae given in Figue. The esults on egistation accuacies ae shown in Table. It was found that INS had lage captue ange than othe two similaity measues but it was also less accuate and equied moe steps to be conveged. 5 Conclusions In this pape, we intoduced a unifying MAP-MRF famewok to deive novel point similaity measues fo D-D egistation of X-ay fluooscopy to CT images. The deived novel point similaity measues had been evaluated using phantom and cadave and the esults showed that they povided satisfactoy D-D egistation accuacy, even when inteventional instuments wee pesent. Refeences. Penney G.P., Weese J., Little J.A., Desmedt P., Hill D.L.G., and Hawkes D.J.: A compaison of similaity measues fo use in D-D medical image egistation. IEEE T Med Imaging, Vol. 7, No. 4, (998 586-595. Lemieux L., Jagoe R., Fish D.R., Kitchen N.D., and Thomas D.G.T.: A patient-tocomputed-tomogaphy image egistation method based on digitally econstucted adiogaphs. Med Phys, Vol., No., (994 749-760. Buzug T.M., Weese J., Fassnacht C., and Loenz C.: Image egistation: convex weighting functions fo histogam-based similaity measues. Lectue Notes in Compute Science, Vol. 05, Spinge-Velag, Belin Heidelbeg New Yok (997 0-
94 G. Zheng et al. 4. Weese J., Buzug T.M., Loenz C., Fassnacht C.: An appoach to D/D egistation of a veteba in D x-ay fluooscopies with D CT images. Lectue Notes in Compute Science, Vol. 05, Spinge-Velag, Belin Heidelbeg New Yok (997 9-8 5. Maes F., Collignon A., Vandemeulen D., Machal G., and Suetens P. : Multi-modality image egistation by maximization of mutual infomation. IEEE T Med Imaging, Vol. 6, No., (997 87-98 6. Bown L.M.G. and Boult T.E.: Registation of plana film adiogaphs with computed tomogaphy. IEEE Poceedings of MMBIA (996 4-5 7. Jonić S., Thévenaz P., Zheng G., Nolte L.-P., and Unse M.: An Optimized Spline-based egistation of a D CT to a set of C-am images. Intenational Jounal of Biomedical Imaging (In Pess, 006. 8. Jonić S., Thévenaz P., and Unse M.: Multiesolution-based egistation of a volume to a set of its poection. Poceedings of the SPIE Intenational Symposium on Medical Imaging: Image Pocessing (MI 0, San Diego CA, USA, (00, Vol. 50, Pat II, pp. 049-05 9. Li S.Z.: Makov andom field modeling in compute vision. Spinge-Velag, Belin Heidelbeg New Yok, (995. 0. Zheng G., Zhang X., and Nolte L.-P.: Assessing spline-based multi-esolution D-D image egistation fo pactical use in sugical guidance. Lectue Notes in Compute Science, Vol. 50, (004 94-0. Zöllei L., Gimson E., Nobash A., and Wells W.: D-D igid egistation of X-ay fluooscopy and CT images using mutual infomation and spasely sampled histogam estimatos. Poceedings of CVPR 0, Volume, (00 679-70.. Russakoff D.B., Rohlfind T., and Maue C.R. J.: Fast intensity-based D-D image egistation of clinical data using lighting fields. Poceedings of ICCV 0, (00 46-4.