Image Registration in Medical Imaging

Transcription

1 2/2/202 Image Regisraion in Medical Imaging BI260 VALERIE CARDENAS NICOLSON, PH.D ACKNOWLEDGEMENTS: COLIN STUDHOLME, PH.D. Medical Imaging Analysis Developing mahemaical algorihms o erac and relae informaion from medical images Two relaed aspecs of research Image Regisraion Finding spaial/emporal l correspondences beween image daa and/or models Image segmenaion Eracing/deecing specific feaures of ineres from image daa LUCAS AND KANADE, PROC IMAGE UNDERSTANDING WORKSHOP, 98 Medical Imaging Regsiraion: Overview Regisraion Wha is regisraion? Definiion Classificaions: Geomery, Transformaions, Measures Moivaion for work Where is image regisraion i used in medicine i and biomedical research? Measures of image alignmen Landmark/feaure based mehods Voel-based mehods Image inensiy difference and correlaion Muli-modaliy measures he deerminaion of a one-o-one mapping beween he coordinaes in one space and hose in anoher, such ha poins in he wo spaces ha correspond o he same anaomical poin are mapped o each oher. Calvin Maurer 93

2 2/2/202 Image Regisraion Key Applicaions I: Change deecion Esablishing correspondence, beween feaures in ses of images, and using a ransformaion model o infer correspondence away from hose feaures. Bill Crum 05 Look for differences in he same ype of images aken a differen imes, e.g.: Mapping pre and pos conras agen Digial subracion angiography (DSA) MRI wih gadolinium racer Mapping srucural changes Differen sages in umor growh (before and afer reamen) Neurodegeneraive disease-> quanifying issue loss paerns Deecing changes due o funcion Funcional MRI (fmri): before and afer brain simulus PET imaging: quaniaive racer upake measuremens Problem: Subjec scanned muliple imes-> removed from scanner We canno easily fi/know paien locaion and orienaion wih respec o imaging sysem Need o remove differences in paien posiioning o deec rue differences in paien images Key Applicaions II: Image Fusion Componens of Image Regisraion Algorihms Relae conrasing informaion from differen ypes of images Muli-modaliy imaging MRI-CT MRI-PET/SPECT Srucural MRI-funcional MRI Srucural MRI o srucural MRI Problem: Subjec scanned muliple imes-> differen scanners We canno easily fi/know paien locaion and orienaion wih respec o differen imaging sysems Need o remove differences in paien posiioning o relae informaion from differen ypes of images Image Daa Geomeries 2D-2D, 2D-3D, 3D-3D Transformaion ype Rigid/affine/non-rigid Correspondence crieria/measure Feaure based mehods voel based/dense field mehods Opimizaion mehod Maimizing/minimizing crieria wih respec o ransform 2

3 2/2/202 2D-2D Iner-modaliy image regisraion problem Eamples of Image Geomeries and Transformaion Models in Medical Applicaions Regisraion and display of he combined bone scan and radiograph in he diagnosis and managemen of wrie injuries, Hawkes e al., EJNM 99 2D-2D image ransformaions 2D-2D image ransformaions Simple case: parallel projecion 2 ranslaions and 2 (up-down/lef-righ) and one roaion (θ) This is a linear mapping from (, 2 ) o (y, y 2 ) y = cos θ- 2 sin θ+ y 2 = sin θ+ 2 cos θ+ 2 3

4 2/2/202 Roaing around a given locaion Image Scaling Scaling wih respec o a fied poin (,y) Scaling along an arbirary ais Influence of Affine Transformaions on Images Composing Transformaions Map lines o lines Map parallel lines o parallel lines Preserve raios of disances along a line Do NOT preserve absolue disances and angles 4

5 2/2/202 Radioherapy 2D-3D regisraion problems Treamen verificaion (Poral images) Treamen planning (simulaor sysem) Treamen guidance (Cyberknife sysem) Orhopaedic surgery Spinal surgery (Brain Lab, Medronic) Hip or Knee arhroplasy (ROBODOC) Verificaion of implan posiion Neuroinervenions Maching MRA o DSA Surgical microscope MRI/CT neurosurgery guidance Virual endoscopy 2D-3D Regisraion Geomery he deerminaion of a projecion mapping, from a 3D o a 2D coordinae sysem such ha poins in each space which correspond o he same anaomical poins are mapped o each oher. (A) Imaging/Acquisiion parameers (inrinsic) (B) Subjec Parameers (erinsic) 5

6 2/2/202 3D-3D image regisraion 3D Rigid Transformaions Many differen 3D clinical imaging modaliies MRI probably sill he leas common Images used in many differen clinical seings Diagnosis Treamen planning Treamen guidance Clinical research: sudying disease Transformaion ypes: Rigid posiioning of subjec: sill mos common Non-rigid deformaions o describe Tissue deformaion Imaging disorion Differences beween subjecs ' y' = z' y 34 z cos β 0 sin β 0 cosθ sinθ cosα sinα sinθ cosθ 0 0 R = = = R 0 sin cos 0 y R α α sin β 0 cos β 0 z z θ 0 0 T = 0 0 y α 0 0 β y z 0 Feaure Based Approaches Poin se -> Poin se (homologous) Poin se -> Poin se (non homologous so so need o find order) Poin se -> Surface Surface -> Surface (also Curve->Surface) 6

7 2/2/202 Manual poin landmark idenificaion (around 2 poins) in MR and CT Relaes sof issue srucures such as enhancing umor and blood flow o bone feaures in CT MR-CT Regisraion Homologous Feaure Based Alignmen General case of wo liss of corresponding feaure locaions [p, p 2,,p N ] and [q, q 2,,q N ] boh wih N poins We wan o find: Transformaion T(q) ha minimizes squared disance beween corresponding poins: E = p T ( r r q r Where one se of poins, q, is ransformed by T() -> Erapolae Transformaion for all image voels/piels ) 2 Procruses Poin Alignmen Golub and VanLoan, Mari Compuaions, 989 Procruses Poin Alignmen Remove ranslaional differences Calculae ranslaion ha brings each of he poin ses o he origin and subrac from each of he poin ses o creae cenered poin ses q' i = qi qi N i p' i = pi pi N Rewrie cenered poin liss as marices P = p' y z q' ' y' z' p' 2 = 2 y2 z2 = q' 2 = ' 2 y' 2 z' 2 Q M M M M M M M M p' N 3 y3 z3 q' N ' 3 y' 3 z' 3 i Scale (procruses normally includes esimae of scaling) Bu we can assume scanner voel dimensions are accurae I is possible o show ha he roaion soluion is: R=VU T where U and V are marices obained from he SVD of P T Q Recall ha (P T Q->(USV T ) Esimae roaions: find he roaion mari R such ha P T =RQ T The sysem P T =RQ T is over-deermined and here is noise, hus we wan o find he roaion mari R such ha minr= P T -RQ T 2 K.S. Arun, T.S. Huang, and S.D. Blosein, Leas square fiing of wo 3-d poin ses. IEEE PAMI, 9(5): , 987. S is a diagonal mari, U and V are marices conaining he lef and righ singular vecors. For 2D: T VU cosθ sinθ = sinθ cosθ 7

8 2/2/202 Alernaives o SVD alignmen Erapolaing Transformaions: Theoreical poin based regisraion error (poins on circle 50 mm radius seleced wih RMS error of 2mm) Quaernion mehods Horn, Closed-form soluion of absolue orienaion using uni quaernions. Journal of he Opical Sociey of America A, 4(4): , 987. Orhonormal marices Horn e al., Closed-form soluion of absolue orienaion using orhonormal marices. Journal of he Opical Sociey of America A, 5(7):27-35, July 988. Dual quaernions Walker e al., Esimaing 3-d locaion parameers using dual number quaernions. CVGIP: Image Undersanding, 54: , November 99. These algorihms generally show similar performance and sabiliy wih real world noisy daa Loruss e al., A comparison of four algorihms for esimaing 3-D rigid ransformaions. In Proceedings of he 4 h Briish machine conference (BVMC 95), pages , Birmingham, England, Sepember 995. Imporan facor: regisraion lowes where 3D landmarks are found The disribuion of arge regisraion error in rigid-body poin-based regisraion. Fizparick and Wes, IEEE TMI, 20(9): 200, Approaches o Landmark/Feaure Eracion and Maching Markers aached o subjec Need o be visible in differen ypes of images Hawkes e al., Regisraion and display of he combined bone scan and radiograph in he diagnosis and managemen of wris injuries, EJNM 99. Manual landmark idenificaion Time consuming, inroduce errors, difficul o find rue consisen 3D landmarks, bu VERY fleible and can adap o daa Hill e al., Accurae frameless regisraion of MR and CT images of he head: Applicaions in surgery and radioherapy planning, Radiology 994, Auomaed landmark idenificaion: geomeric models of local anaomy Need o be rue unique 3D poin in inensiy map: ip-like, saddle-like, and sphere like srucures Need o be consisen in differen subjecs and image ypes Worz and Rohr, Localizaion of anaomical poin landmarks in 3D medical images by fiing 3D parameric inensiy models, Medical Image Analysis 0():4-58, Non-homologous landmarks/3d srucures Easier o auomaically find general feaures: for eample poin on a surface using edge deecion Bu: which poin maps o which poin? Need o hen find correspondence and alignmen: poin cloud fiing Early feaure based brain/head maching Head-ha maching of head surfaces Rerospecive geomeric correlaion of MR, CT and PET images, Levin e al., Radiology

9 2/2/202 Alignmen of non-homologous feaure locaions: fuzzy correspondence Ieraive Closes Poin Algorihm General case of wo liss of poin locaions P=[p, p 2,,p N ] and Q=[q,q 2,,q M ] wih N and M poins respecively, and a lis of weighs [k ij ] o describe he fuzzy correspondence beween every possible pair: Regisraion error can hen be epressed as k ij pi ( Rqi + E ( R,) = ) Bu need o esimae R (roaion), (ranslaion) and correspondence [k ij ] i l Approimaes correspondence mari [k ij ] by assigning each poin o he curren closes poin. Applying curren ransformaion R and o Q=[q,q 2,,q M ], so ha Q =RQ i + Take each poin P=[p, p 2,,pp N ] and search lis Q i o find he neares poin Q ii o creae a new lis wih N poins Apply leas squares fi of curren neares poin (e.g. using Procruses poin maching) o esimae R and Repea unil change in ransformaion falls below hreshold Besl and McKay, A mehod for regisraion of 3-D shapes, IEEE PAMI, 4(2): , 99. ICP advanages and disadvanages Challenges in auomaing medical image regisraion Can be applied o boh discree landmarks, lines, surfaces ec. Bu: highly dependen on saring esimae! Only finds a local opima Can use muli-sar o improve search Search for closes poin in large poin liss or surfaces can be compuaionally epensive Finding suiable feaures True 3D landmarks Finding he same feaure in differen ypes of images No nice edges/corners and man made scenes Variable/limied anaomical coverage Truncaed Variable/low conras o noise 9

10 2/2/202 Piel/Voel based regisraion Hisory: emplae maching Deecing an objec or feaure based on piel/voel values Avoid he need o auomaically erac corresponding landmarks or surfaces Similariy measures for image regisraion can assume: Linear inensiy mapping Non-parameric -o- inensiy mapping Non-parameric many-o- inensiy mapping Simples: image inensiy difference Ieraive refinemen of ransformaion parameers: small displacemens Consider D case: for no noise, assume w() is some eac ranslaion of f() w()=f(+) and image mis-mach or error for given displacemen can simply be local difference in inensiy e()=f(+)-w() Lucas and Kanade, Proc Image Undersanding Workshop, 98, Ieraive refinemen of ransformaion parameers: small displacemens AT A POINT Ieraive refinemen of ransformaion parameers: small displacemens OVER ALL () [w()-f()]/f () A one poin, for small f () (f(+)-f())/ =(w()-f())/ So, ranslaion o align image f wih w a poin is hen [w()-f()]/f () so average over all is ( ) / n Bu: because of local approimaions, firs esimae of does no ge you o he opimal soluion, jus ges you nearer o i Need muliple seps: ieraive regisraion + if 0 = 0 hen n+ = n ( ) / n 0

11 2/2/202 Minimizing Sum of Squared Inensiy Difference Eension o N dimensional images We can use an alernaive form for he inensiy mis-mach or error: he squared inensiy difference E( ) = 2 [ f ( + ) w( )] To find he opimal ransformaion se: E 0 = 0 = 0 = Giving : 2 [ f ( + ) w( )] = 2 f '( )[ f ( ) + f '( ) w( )] f '( )( w( ) f ( )) so if 0 = 0 hen 2 n f '( ) 2 [ f ( ) + f '( ) w( )] + = + n f '( )( w( ) f ( )) 2 f '( ) The squared inensiy difference can be eended o he case where locaion and ranslaion are vecors of N dimensions: Alernaive image similariy measures for image alignmen Global Inensiy Variaions Many medical images have uncalibraed inensiies Gain or illuminaion changes Common issue: linear inensiy scaling and offse f =wk+b Absolue difference will no end o zero a correc mach: OR worse, minimum does no correspond o correc mach

12 2/2/202 Sum of Squared Difference and Correlaion D = D = J = s= J K K 2 [ f ( + s, y + ) w( s, )] This can be epanded o : w s 2 f ( s, y ) (, ) 2 s s s = = = = = = J K J K f ( + s, y + ) w( s, ) Correlaion Correlaion is measuring he residual errors beween he daa and a bes fi of a line o ha daa Allows relaionship o have a differen slope and offse Robus o global linear changes in brighness/conras D will be small when las erm is large or firs wo erms are small. Firs wo erms are he summed inensiies wihin he emplae region Neglecing he firs wo erms and he facor -2 gives us a simple commonly used measure of emplae mach o be MAXIMIZED c(, y) = J = s= f ( + s, y + ) w( s, ) This sum of producs of he inensiy pairs a each piel is he correlaion beween arge and emplae piel inensiies. K Normalized Correlaion: Boundary overlap Normalized Correlaion: sensiiviy o variance A image boundaries he overlap of emplae and arge image may be limied: mach will be reduced by number of piels no overlapping Can normalize measure by number of overlapping piels, N a emplae locaion, y c'(, y) = N J K = s= f ( + s, y + ) w( s, ) Correlaion is sill a funcion of overall energy or brighness of he image and emplae so can sill end up picking he brighes arge One opion: normalize by he summed brighness of he arge image J K c'(, y) = f ( + s, y + ) w( s, ) E = s= E = J K = s= f ( + s, y + ) 2

13 2/2/202 Variance? Correlaion Coefficien Mach may sill be biased by variance in Targe/Templae Wha abou non-linear inensiy mappings? Non-linear inensiy mapping 3

14 2/2/202 Muli Modaliy Similariy Measures Non-parameric -o- mapping Bu inensiy mapping is no usually smooh or easily parameerized (e.g., discree because of differen issue classes) Assume on -o- mapping of inensiies beween emplae and arge MATCHING IMAGES WITH DIFFERENT TISSUE CONTRAST PROPERTIES SPECT-MRI regisraion SPECT-MRI regisraion No easy o find 3D landmarks 4

15 2/2/202 The effecs of misregisraion in inensiy feaure space Pariioned Image Uniformiy [Woods e al. JCAT 93] Used o MRI-PET/SPECT regisraion MRI scan scalp edied-> only consider inra-cranial issues Gray-whie-CSF values in cranial region differ Non-monoonic mapping Approimaely -o- Pariioned Image Uniformiy [Woods e al. JCAT 93] Divide up he emplae image inensiies w ino bins b B The pariioned image uniformiy of he arge image f is given by he weighed summaion of normalized sandard deviaions of f in each bin: Many-o- Inensiy Mapping? 5

16 2/2/202 Changes in 2D Hisogram wih Alignmen How o Characerize Alignmen? Hisogram Sharpness Informaion and Enropy [Collignon, CVRMED 95] Informaion and Enropy [Collignon, CVRMED 95] 6

17 2/2/202 Muual Informaion [Viola and Wells: ICCV 95, Collignon e al.: IPMI 95] Join enropy, like correlaion, is influenced by he image srucure in he image overlap he changing ransformaion modifies he informaion provided by he images Insead: form a measure of he relaive informaion in he arge image wih respec o emplae using Muual informaion: Difference beween marginal and join enropies Normalized Muual Informaion [Sudholme e al., 998] When muual informaion is used o evaluae he alignmen of wo finie images: overlap sill has a confound Boh marginal enropies, H(F) and H(W) vary Alignmen can be driven by overlap ha has large H(F) and H(W). e.g., overlap wih equal area of background and foreground inensiies Raher han look for difference in join and marginal enropies, maimize heir raio (like correlaion coefficien): H ( F) + H ( W ) Y ( F, W ) = H ( F, W ) Bu his does no solve all problems! I s sill only overlaps of inensiies he bigges overlaps drive he regisraion 3D Rigid MR-CT Regisraion in he Skull Base 7

18 2/2/202 Image Fusion for Skull Base Surgery Planning Hawkes e al., 993 Subracion SPECT Imaging in Epilepsy CT: Bone MR Gadolinium: Tumor MRA: Blood Vessels Summary A range of medical alignmen measures have been developed in he las 5 yrs These vary in he assumpions hey make abou he relaionship beween inensiies in he wo images being mached Many oher crieria no covered! Many ways of modifying he crieria Evaluaion a muli resoluion/scale Edge/boundary/geomeric feaure eracion: modify conras Spaial windowing and encoding can localize he crieria Bes crieria will depend on he ype of daa you have How differen he informaion provided and wha conras is shared How much hey overlap Bibliography I Barnea and Silvermann, IEEE Transacions on Compuing 2(2), 79-86, 972. Woods e al., MRI-PET Regisraion wih Auomaed Algorihm, J. Compuer Assised Tomography, 7(4): , 993. Roche e al., The correlaion raio as a new similariy measure for mulimodal image regisraion, In Proc. Medical Image Compuing and Compuer Assised Inervenion, 998, 5-24, Spring LNCS. Hill e al., Voel similariy il i measures for auomaed image regisraion, i Proc. Visualizaion in Biomedical Compuing, ed. R.A. Robb, SPIE press, 994, Bellingham. Collignon e al., 3D mulimodaliy medical image regisraion using feaures space clusering, Proc. Of Compuer Vision, Virual Realiy and Roboics in Medicine, 995, , Springer LNCS. Collignon e al., Auomaed muli-modaliy image regisraion based on informaion heory, Proc. Of Informaion Processing in Medical Imaging, 995, Kluwer, Dordrech. 8

19 2/2/202 Bibliography I Viola and Well, Alignmen by maimizaion of muual informaion, Proc. Of Inernaion Conference on Compuer Vision, 995, Ed. Grimson, Schafer, Blake, Sugihara. Sudholme e al., A normalised enropy measure of 3D medical image alignmen, Proceedings of SPIE Medical Imaging 998, San Diego, SPIE Press, Sudholme e al., An overlap invarian enropy measure of 3D image alignmen, Paern Recogniion, 32(), 999, Maes e al., Mulimodaliy image regisraion by maimizaion of muual informaion, IEEE Transacions on Medical Imaging, Vol 6, 87-98,