Proceedings 23rd Annual Conference IEEE/EMBS Oct.25-28, 2001, Istanbul, TURKEY

Transcription

1 Alignment of Serilly Aquire Slies using Glol Proeeings 23r Annul Conferene IEEE/EMBS Ot.25-28, 2001, Istnul, TURKEY Energy Funtion Stelios KRINIDIS, Christophoros NIKOU, Ionnis PITAS Deprtment of Informtis, Aristotle University of Thessloniki Box 451, Thessloniki, Greee Tel: , Fx: e-mil: Astrt An urte, omputtionlly effiient n fully-utomte lgorithm for the lignment of 2D serilly quire setions forming 3D volume is presente. The metho ounts for the min shortomings of 3D imge lignment: orrupte t (uts n ters), issimilrities or isontinuities etween slies, non prllel or missing slies. The pproh relies on the optimiztion of glol energy funtion, se on the ojet shpe, mesuring the similrity etween slie n its neighorhoo in the 3D volume. Slie similrity is ompute using the istne trnsform mesure in oth iretions. No prtiulr iretion is privilege in the metho voiing glol offsets, ises in the estimtion n error propgtion. The metho ws evlute on rel imges (meil n iologil 3D t) n the experimentl results emonstrte the metho's ury s reonstution errors re less thn 1 egree in rottion n less thn 1 pixel in trnsltion. Keywors: serilly quire imges, mislignment, imge registrtion, registrtion error, non-overlpping strutures, pixel similrity mesure, eterministi optimiztion. I. Introution Three-imensionl reonstrution of meil imges (tissue setions, CT n utoriogrphi slies) is now n integrl prt of iomeil reserh. Reonstrution of suh t sets into 3D volumes, vi the registrtions of 2D setions, hs gine n inresing interest. The registrtion of multiple slies is of utmost importne for the orret 3D visuliztion n morphometri nlysis (e.g. surfe n volume representtion) of the strutures of interest. Severl lignment lgorithms hve een propose in tht frmework. A review of generl meil imge registrtion methos is presente in [1], [2], [3]. The prinipl 3D lignment (reonstrution from 2D imges) methos my e lssifie in the following tegories: fiuil mrker-se methos [4], feture-se methos using ontours, rest lines or hrteristi points extrte from the imges [5], [6], n gry level-se registrtion tehniques using the intensities of the whole imge [7], [8], [9], [10]. Most of the ove mentione tehniques o not simultneously onsier the two mjor iffiulties involve in meil n CT snne t registrtion. At first, onseutive slies my iffer signifintly ue to istortions, isontinuities in ntomil strutures, uts n ters. These effets re more pronoune when istnt slies re involve in the registrtion. From this point of view, registrtion metho must e roust to missing t or outliers [7], [10]. Besies, registering the slies sequentilly (the seon with respet to the first, the thir with respet to the seon, et.) les to ifferent types of misregistrtion. If n error ours in the registrtion of slie with respet to the preeing slie, this error will propgte through the whole volume. Also, if the numer of slies to e registere is lrge, glol offset of the volume my e oserve, ue to error umultion [8]. In this pper, solution to the ove mentione shortomings is presente. A glol energy funtion hving s vriles the rigi trnsformtion prmeters (2D trnsltion n rottion) of given slie with respet to lol symmetri neighorhoo is propose. Glol energy funtions re powerful tool in omputer vision pplitions ut they hve not yet een onsiere for the registrtion of serilly quire slies. Our pprohws inspire y the tehnique presente in [11], whih onsists in minimizing glol energy funtion with the Itertive Closest Point lgorithm [12], to register multiple, prtilly overlpping views of 3D struture. The glol energy funtion implemente in our pproh is ssoite with pixel similrity metri se on the Eulien istne trnsform [13]. The reminer of the pper is orgnize s follows. The glol energy funtion formultion n the ssoite registrtion lgorithm is presente in setion II, experimentl results re presente in setion III n onlusions re rwn in setion IV. II. A glol energy funtion formultion Before presenting the lignment metho, the nottions use in our formultion re introue. A set of 2D serilly quire slies is represente y: V = fi i ji =1:::Ng (1) where I i is slie ( 2D imge) n N enotes the totl numer of slies. A pixel of 2D slie is represente y: p = (x; y) T, so tht I i (p) orrespons to the gry level (intensity) of pixel p of slie i. N x n N y esignte the numer ofpixelsofeh slie in the horizontl n vertil iretion respetively. Stnr two-imensionl rigi lignment onsists of estimting the rigi trnsformtion prmeters (trnsltion /01$ IEEE

2 Report Doumenttion Pge Report Dte 25 Ot 2002 Report Type N/A Dtes Covere (from... to) - Title n Sutitle Alignment of Serilly Aquire Slies Using Glol Energy Funtion Contrt Numer Grnt Numer Progrm Element Numer Author(s) Projet Numer Tsk Numer Work Unit Numer Performing Orgniztion Nme(s) n Aress(es) Deprtment of Informtis Aristotle University of Thessloniki Sponsoring/Monitoring Ageny Nme(s) n Aress(es) US Army Reserh, Development & Stnriztion Group (UK) PSC 802 Box 15 FPO AE Performing Orgniztion Report Numer Sponsor/Monitor s Aronym(s) Sponsor/Monitor s Report Numer(s) Distriution/Avilility Sttement Approve for puli relese, istriution unlimite Supplementry Notes Ppers from 23r Annul Interntionl Conferene of the IEEE Engineering in Meiine n Biology Soiety, Otoer 25-28, 2001, hel in Istnul, Turkey. See lso ADM for entire onferene on -rom. Astrt Sujet Terms Report Clssifition unlssifie Clssifition of Astrt unlssifie Clssifition of this pge unlssifie Limittion of Astrt UU Numer of Pges 4

3 t x, t y n rottion y ngle ) tht hve to e pplie to slies in the neighorhoo of i, y minimiztion the imge to e ligne (floting imge) in orer to mth of the following lol energy funtion (eq. 6). referene Proeeings imge. 23r Annul Conferene IEEE/EMBS Ot.25-28, 2001, Λ elre Istnul, slie TURKEY I i visite. In the pproh propose here, the lignment ofthe2d setions, within the 3D volume, is onsiere glolly y minimizing n energy funtion E( ), whih expresses the similrity etween the 2D setions: ffl en o en o E( ) = i=1 j=1 X N x N y f(i i (T i (p));i j (T j (p))) (2) where f( ) is similrity metri, I k enotes slie k n T k esigntes rigi trnsformtion with prmeters k = ft k x ;tk y ; k g. Eqution (2) inites tht for given set of rigi trnsformtion prmeters T i, pplie to the slie to e ligne I i, the similrity etween the trnsforme slie I i (T i (p)) n ll of the other lrey trnsforme slies I j (T j (p)) in the volume is umulte in the energy funtion. Assuming tht funtion f( ) is symmetri: f(i i (T i (p));i j (T j (p))) = f(i j (T j (p));i i (T i (p))) (3) whih is the se for the pixel similrity funtions onsiere here, yiels the following glol minimiztion prolem: ^ =rg min i=1 j=1 j<i N x N X y f(i i (T i (p));i j (T j (p))) (4) Without itionl onstrins, the optimiztion prolem (4) hs lerly n infinite numer of solutions (if the set of rigi trnsformtions ft ^ 1, T ^ 2 ;::: T ^ N g is solution, the sme hols true for ft ^ 1 ffit g, where ffit ;T^ 2 ffit ;:::T^ N T is n ritrry 2D rigi trnsformtion). To removethis miguity, the trnsformtion T ^ l pplie to n ritrry hosen slie k is onstrine to the ientity trnsformtion (we hvehosen k = 1 in our implementtion). As result, there re 3(N 1) prmeters to estimte. It is ommon sense tht istnt slies present very little similrity ue to ntomy n it woul e more pproprite to mesure the energy funtion only for slies presenting t lest some similrities. Therefore, the support region of funtion f( ) hs een limite to neighorhoo of rius R entere t eh slie n set: f(i i (T i (p));i j (T j (p))) = 0; 8 ji jj >R (5) Thus, the following lignment lgorithm is ssoite with the energy funtion (4): ffl o until onvergene. elre ll slies unvisite. o until ll slies re elre visite. Λ rnomly hose n unvisite slie I i 2 V. Λ upte the rigi trnsformtion prmeters T i ringing into lignment slie I i with the other E i ( i ) ef = i=1 j=1 ji jj»r N x N X y f(i i (T i (p));i j (T j (p))) (6) The minimiztion of the lol energy funtion (4) is onute y eterministi optimiztion lgorithm, known s Iterte Conitionl Moes (ICM) [14]. ICM is isrete Guss Seiel-like optimiztion tehnique, epting only onfigurtions eresing the ojetive funtion. Let us notie tht the prmeter ^ i orresponing to the minimum vlue of the lol energy funtion E i ( i ) (Equ. 6) lso orrespons to lol minimum vlue of the glol energy funtion E( ) with respet to i,keeping ll other prmeters j ;j 6= i fixe. It is thus esy to see tht the esrie lgorithm onverges towrs lol minimum of the initil energy funtion (2). This lol minimum orrespons to stisftory registrtion, sine the initil lignment of the 2D setions is generlly lose to the esire solution (if this is not the se, goo initiliztion my e otine y stnr orse lignment tehnique suh s prinipl xes registrtion). It is thus not neessry to resort here to greey glol optimiztion proeures, suh s simulte nneling or geneti lgorithms. Further improvement of the solution is otine y grient eent tehnique. To spee the lgorithm up multigri t proessing is lso implemente. The pixel similrity metri ssoite with the oveesrie glol energy funtion is se on istne trnsform ([13], [15]) (lso known s hmfer mthing tehnique [16]) n it is ompute from the 3D ojet ontours ([17]). A istne trnsformtion is n opertion tht onverts inry piture, onsisting of feture n non-feture elements (ontours), to piture where eh pixel hs vlue tht pproximtes its istne to the nerest ontour point. Thus, using the istne trnsform D(p) of the referene slie the metho ligns the floting slie y minimizing the istne etween the ontours of the imges. For further etils of the hmfer mthing metho the reer my refer to [16]. Consiering the slies per triplets, whih isvery ommon for stnr reonstrution prolems (i.e. setting R=1 in eq. 5), the estimtion of the lignment prmeters involves the non-liner similrity metri: f(t i (p)) = D i 1 (T i 1(p)) + D i+1 (T i+1 (p)); I i (T i (p)) 6= 0 where I i (T i (p)) 6= 0 mens tht only the ontour points of I i re involve. A lrge numer of interpoltions re involve in the lignment proess. The ury of estimtion of the rottion n trnsltion prmeters is iretly relte to the

4 ury of the unerlying interpoltion moel. Simple pprohes suh s the nerest neighor interpoltion re ommonly Proeeings use euse 23r Annul theyconferene re fst n IEEE/EMBS simple toot.25-28, implement, though they proue imges with notiele rti- 2001, Istnul, TURKEY fts. Besies, s the trnsltion n rottion prmeters shoul ompenste for ury y hving suvoxel vlues, this typeofinterpoltion woul not e pproprite. More stisftory results n e otine y smll-kernel ui onvolution tehniques, iliner, or onvolution-se interpoltion. Aoring to smpling theory, optiml results re otine using sinus rinl interpoltion, ut t the expense of high omputtionl ost. As ompromise, iliner interpoltion tehnique hs een use in the optimiztion steps. At the en of the lgorithm, the lignment prmeters re refine using sinus ril interpoltion tht preserves the qulity of the imge to e ligne. This tehnique hs proven to e fst n effiient. III. Experimentl Results To evlute our metho, we pplie the lgorithm to the reonstrution of n rtifiilly misligne 3D humn skull volume (figure 1). The slies of the originl CT volume were trnsforme using trnsltions vrying from -10 to +10 pixels n rottions vrying from -20 to +20 egrees. The trnsformtions for eh slie were rnom following uniform istriution in orer not to privilege ny slie (figures 1() n 1()). Tle I presents sttistis on the lignment errors. The lgorithm revele roust in ligning this type of imge proviing smll registrtion errors. Figures 1() n 1() present the reonstrute volume. Alignment error sttistis tx ty mein mximum men±s.ev 0.37± ± ±0.35 TABLE I A set of 140 slies of 3D CT humn skull volume were rtifiilly trnsforme using ifferent rigi trnsformtion prmeters. Eh slie ws rnomly trnsforme using trnsltions vrying from -10 to +10 pixels n rottions vrying from -20 to +20 egrees. Different sttistis on the errors for the rigi trnsformtion prmeters re presente. Trnsltion errors re expresse in pixels n rottion error in egrees. Moreover, we hve uniformly trnsforme 140 slies of the sme 3D volume y pplying to eh slie I i trnsltion of t i x = t i 1 x +0:15 pixels n t i y = t i 1 y + 0:15 pixels n rottion of i = i 1 +0:3 egrees. As the volume hs 140 slies, the lst slie is trnslte y 21 pixels in oth iretions n rotte y 42 egrees with respet to its initil position. Tle II presents the registrtion errors of the metho. It is illustrte tht our pproh hs supixel men n mein errors. Also mximum errors re slightly superior to 1 pixel n 1 egree respetively showing the roustness of the tehnique. Fig. 1. Reonstrution of 3D humn skull volume of 140 slies. () Multiplnr view of the volume efore registrtion. () Threeimensionl view of the volume efore registrtion. () Multiplnr view of the volume fter registrtion. () Three-imensionl view of the volume fter registrtion. Alignment error sttistis tx ty mein mximum men±s.ev 0.33± ± ±0.25 TABLE II A set of 140 slies of 3D CT humn skull volume were rtifiilly trnsforme using ifferent rigi trnsformtion prmeters. Eh slie ws trnslte y 0.15 pixels in oth iretions n rotte y 0.3 egrees with respet to its preeing slie. Different sttistis on the errors for the rigi trnsformtion prmeters re presente. Trnsltion errors re expresse in pixels n rottion error in egrees. Furthermore, the lgorithm ws pplie to the reonstrution of volumes (tooth germs, iologil tissues) with unknown groun truth. The metho's performne ws ompre with the mnul lignment omplishe y n expert physiin. Figure 2 shows the reonstrution of tooth germ y n expert entist (fig. 2() n 2()) n y ourmetho (fig. 2() n 2()). It is illustrte tht humn intervention fils to orretly lign the slies, whilst our metho is effiient n n hievelignment with high ury. Also, Figure 3 epits 3D tissue ontining lrge numer of vessels. Figures 3() n 3() show the volume ligne y n expert iologist n Figures 3() n 3() the tissue fter lignment y our metho. This volume presents uts n isontinuities n the tissues h een strethe uring the ut proeure. Despite these rwks, oring to the expert iologist, the lgorithm ligne orretly the slies. Finlly, let us notie tht the lgorithm hs omput-

5 proess. By these mens, the mjor prolems set y the registrtion of serilly quire slies re resse. With Proeeings 23r Annul Conferene IEEE/EMBS Ot.25-28, the2001, glol Istnul, (isotropi) TURKEY formultion of the registrtion prolem (rther thn stnr step y step, sequentil formultion), no glol offset nor error propgtions re oserve in the finl lignment. The pproh seems promising n its ssoition to more sophistite ut time onsuming pixel similrity metris (mutul informtion [18], roust estimtion-se mesures [19]) my improve its ury n is perspetive ofthiswork. Fig. 2. Reonstrution of 3D tooth volume of 265 slies. () Multiplnr view of the volume fter lignment y n expert entist. () Three-imensionl view of the volume fter lignment y n expert entist. () Multiplnr view of the volume fter registrtion. () Three-imensionl view of the volume fter registrtion. Fig. 3. Reonstrution of 3D tissue volume of 237 slies. () Multiplnr view of the volume fter lignment y n expert iologist. () Three-imensionl view of the volume fter lignment y n expert iologist. () Multiplnr view of the volume fter registrtion. () Three-imensionl view of the volume fter registrtion. tionl omplexity O(N x N y N) n requires pproximtely 10 min. to reonstrut volume on Pentium III (800 MHz) worksttion. IV. Conlusion The lignment metho esrie in this pper is kin to the glol energy funtion formultion propose in [11] to register multiple views of 3D surfe in omputer vision pplitions. The min ontriution of the pproh is to onsier the lignment prolem glolly on the 3D volume, y minimizing glol ojetive funtion expressing the similrity etween neighoring slies. The pproh oes not privilege ny prtiulr iretion in the registrtion Referenes [1] L. Gottesfel-Brown. A survey of imge registrtion tehniques. ACM Computing Surveys, 24(4): , [2] J. B. A. Mintz n M. A. Viergever. A survey of meil imge registrtion tehniques. Meil Imge Anlysis, 2(1):1 36, [3] P. Vn en Elsen, E. J. D. Pul, n M. A. Viergever. Meil imge mthing - review with lssifition. IEEE engineering in Meiine n Biology, 12(1):26 39, [4] A. F. Golszl, O. J. Tretik, P. J. Hn, S. Bhsin, n D. L. M Ehron. Three-imensionl reonstrution of tivte olumns from 2-[ 14 ] eoxy--gluose t. NeuroImge, 2:9 20, [5] L. Hir n R. Hwkins. Ojetive imge lignment for three-imensionl reonstrution of igitl utoriogrphs. Journl of Neurosiene Methos, 26:55 75, [6] A. Rngrjn, H. Chui, E. Mjolsness, S. Pppu, L. Dvhi, P. Golmn-Rki, n J. Dunn. A roust point-mthing lgorithm for utoriogrph lignment. Meil Imge Anlysis, 1(4): , [7] W. Zho, T. Young, n M. Ginserg. Registrtion n threeimensionl reonstrution of utoreiogrphi imges y the isprity nlysis metho. IEEE Trnstions on Meil Imging, 12(4): , [8] A. Anresen, A. M. Drewes, J.E. Assentoft, n N. E. Lrsen. Computer-ssiste lignment of stnr seril setions without use of rtifiil lnmrks. prtil pproh to the utiliztion of inomplete informtion of 3 reonstrution of the hippompl region. Journl of Neurosiene Methos, 45: , [9] B. kim, J. Boes, K. Frey, n C. Meyer. Mutul informtion for utomte unwrping of rt rin utoreiogrphs. NeuroImge, 5:31 40, [10] S. Ourselin, A. Rohe, G. Susol, X. Penne, n C. Sttonnet. Automti lignment of histologil setions for 3 reonstrution n nlysis. Sophi Anipolis, Frne, [11] R. Ben-Jem n F. Shmitt. A solution for the registrtion of multiple 3 points sets using unit quternions. In NotesinComputer siene. Proeeings of the 5 th Europen Conferene on Computer Vision (ECCV'98), volume 2, pges 34 50, Freiurg, Germny, June [12] M. J. Besl n N. MKy. A metho for the registrtion of 3 shpes. IEEE trnstions of Pttern Anlysis n Mhine Intelligene, 14(2): , [13] G. Borgefors. Distne trnsformtions in ritrry imensions. Computer Vision, Grphis, n Imge Proessing, 27: , [14] J. Besg. On the sttistil nlysis of irty pitures. Journl of the Royl Sttistil Soiety, 48(3): , [15] Per-Erik Dnielsson. Eulien istne trnsform. Computer Grphis n Imge Proessing, 14:227 28, [16] G. Borgefors. Hierrhil hmfer mthing: A prmetri ege mthing lgorithm. IEEE trnstions of Pttern Anlysis n Mhine Intelligene, 10: , Novemer [17] J. Cnny. A omputtionl pproh to ege etetion. IEEE Trnstions on Pttern Anlysis n Mhine Intelligene, pges , [18] W. WellsIII, P. Viol, H. Atsumi, S. Nkjim, n R. Kikinis. Multimol volume registrtion y mximiztion of mutul informtion. Meil Imge Anlysis, 1(1):33 51, [19] C. Nikou, J. P. Armsph, F. Heitz, I. J. Nmer, n D. Gruker. Mr/mr n mr/spet registrtion of rin imges y fst stohsti optimiztion of roust voxel similrity mesures. NeuroImge, 8(1):30 43, 1998.