Fundamentals of medical imaging registration I - II. Olivier Clatz Ph.D.

Transcription

1 Fundamentals of medical imaging registration I - II Olivier Clatz Ph.D. oclatz@bwh.harvard.edu 1

2 What is registration? The process of aligning a target image to a source image More generally, determining the transform that maps points in the target image to points in the source image Image 1 Image 2 X X =T(X) 2

3 Overview Why doing registration? What type of transformation? What type of similarity? How to estimate the transformation? Geometric registration Iconic registration How to resample the image? 3

4 Overview Why doing registration? What type of transformation? What type of similarity? How to estimate the transformation? Geometric registration Iconic registration How to resample the image? 4

5 Why doing registration? Analysis of temporal evolution Fusion of multimodal images Inter-patients comparison Atlas superposition Reconstruction of a 3D volume 5

6 Temporal Evolution coronal coronal Time 1 Time 2 sagittal axial sagittal axial David Rey, Gérard Subsol, Hervé Delingette, and Nicholas Ayache. Automatic Detection and Segmentation of Evolving Processes in 3D Medical Images: Application to Multiple Sclerosis. Medical Image Analysis, 6(2): , June

7 Temporal Evolution Preoperative MRI Intraoperative MRI Olivier Clatz, Hervé Delingette, Ion-Florin Talos, Alexandra J. Golby, Ron Kikinis, Ferenc A. Jolesz, Nicholas Ayache, and Simon K. Warfield. Robust Non-Rigid Registration to Capture Brain Shift from Intra-Operative MRI. IEEE Transactions on Medical Imaging, 24(11): , Nov

8 Fusion of multimodal images MRI PET CAT (Positron emission tomography) anatomical US Visible Man 8

9 Inter-patients comparaison W i = 1 N M n= 1 r D T n r D n Statistics on T i Compute brain variability Alzheimer's HIV/AIDS Schizophrenia Drug Abuse Development Vincent Arsigny, Pierre Fillard, Xavier Pennec, and Nicholas Ayache. Log-Euclidean Metrics for Fast and Simple Calculus on Diffusion Tensors. Magnetic Resonance in Medicine, 56(2): , August Paul M. Thompson, Christine N. Vidal, Jay N. Giedd, Peter Gochman, Jonathan Blumenthal, Rob Nicolson, Arthur W. Toga, Judith L. Rapoport (2001). Mapping Adolescent Brain Change Reveals Dynamic Wave of Accelerated Gray Matter Loss in Very Early-Onset Schizophrenia, Proceedings of the National Academy of Sciences of the USA, vol. 98, no. 20: , September 25,

10 Atlas Superposition MRI Label Pierre-Yves Bondiau, Gregoire Malandain, Stephane Chanalet, Pierre-Yves Marcy, Jean-Louis Habrand, Francois Fauchon, Philippe Paquis, Adel Courdi, Olivier Commowick, Isabelle Rutten, and Nicholas Ayache. Atlas-based automatic segmentation of MR images: validation study on the brainstem in radiotherapy context. Int J Radiat Oncol Biol Phys, 61(1):289-98, January

11 Reconstruction of a 3D volume Series of contiguous 2D slices (thickness ~60nm) from Electron Microscopy camera The aim: to build a 3D volume Series of successive 2D slices Orthogonal views of the reconstructed volume using affine transformations J. Dauguet, A. Dubois, A.-S. Herard, L. Besret, G. Bonvento, P. Hantraye and T. Delzescaux. Towards a Routine Analysis of Anatomical and Functional Post Mortem Slices in 3 Dimensions. XXIInd International Symposium on Cerebral Blood Flow, Metabolism, and Function, Amsterdam,

12 Different Classes of Problems Images: Mono or Multimodalities Intra- or Inter-subjects registration Transformation : Rigid or Non Rigid 12

13 Temporal Evolution coronal coronal Time 1 Time 2 sagittal axial sagittal axial Intra Patient Mono modality Rigid & Non Rigid 13

14 Temporal Evolution Preoperative MRI Intraoperative MRI Intra Patient Monomodal Non Rigid 14

15 Fusion of multimodal images MRI PET CAT (Positron emission tomography) anatomical US Intra Patient Multi modal Rigid Visible Man 15

16 Inter-patients comparaison Compute brain variability Alzheimer's HIV/AIDS Schizophrenia Drug Abuse Development Inter Patient Mono modal Non Rigid Paul M. Thompson, Christine N. Vidal, Jay N. Giedd, Peter Gochman, Jonathan Blumenthal, Rob Nicolson, Arthur W. Toga, Judith L. Rapoport (2001). Mapping Adolescent Brain Change Reveals Dynamic Wave of Accelerated Gray Matter Loss in Very Early-Onset Schizophrenia, Proceedings of the National Academy of Sciences of the USA, vol. 98, no. 20: , September 25,

17 Atlas Superposition MRI Label Inter Patient Multi modal Non Rigid 17

18 Reconstruction of a 3D volume Series of contiguous 2D slices (thickness ~60nm) from Electron Microscopy camera The aim: to build a 3D volume Series of successive 2D slices Orthogonal views of the reconstructed volume using affine transformations Intra Patient Mono modality Rigid & Non Rigid 18

19 Two Classes of solution Geometric Registration (or feature-based) Extract feature points Compute displacement of similar points Fit a transformation with or without regularization Iconic Registration (or intensity-based) Fit the transformation that optimizes similarity Need: Define Transformation Define Similarity Define regularization 19

20 Overview Why doing registration? What type of transformation? What type of similarity? How to estimate the transformation? Geometric registration Iconic registration How to resample the image? 20

21 Resampling Reference I r Floating I f Displacement Field: X =T(X) To resample: need for displacement field T -1 (X) I f (X)=I f (T -1 (X)) Resampled I f 21

22 How to resample the image? Value I(x,y)? I(x,y +1) I(x +1,y +1) x =int(x) y =int(y) Nearest neighbor I(x,y)=I(round(x), round(y)) I(x,y ) I(x +1,y ) Bilinear interpolation a=x-x b=y-y I(x,y +1) I(x +1,y +1) I(x,y)=a(1-b)I(x +1,y ) +(1-a)(1-b)I(x,y ) +abi(x +1,y +1) y a I(x,y) +b(1-a)i(x,y +1) Bicubic, splines. b I(x,y ) x I(x +1,y ) 22

23 23 Resampling Vectors and Tensors Resampling of a vector field: X =T(X) or U=F(X) Compute local affine transformation (Jacobian): J= T Update Vector: V f (X)=J(T -1 (X))V f (T -1 (X)) ( ) X X z z z y y y x x x T z T y T x T z T y T x T z T y T x T = Tensors: Recall: a tensor can be written: Theoretically: But what does the tensor and the transformation mean? T X V X V X W ) ( ) ( ) ( 3 1 r r = T f f X T J X T W X T J X W )) ( ( )) ( ( )) ( ( ) '( =

24 Resampling Don t forget the Nyquist Shannon theorem! To downsample the image by a factor N (each dimension) Remove the high frequency of the image: Fast Fourier Transform (FFT) high freq =0 FFT -1 Or Gaussian filter σ~n High freq = 0 FFT FFT -1 Downsample 24

25 Overview Why doing registration? What type of transformation? What type of similarity? How to estimate the transformation? Geometric registration Iconic registration How to resample the image? 25

26 Transformations Class T Rigid (displacement) Similitude Affine Polynomial Mesh-based Splines Free 26

27 Transformations Class T Rigid: T ( x) = Rx + Rotation (R) and translation (t) 6 parameters : (R : 3; t : 3) invariants: distances (isometric), orientation, curvature, angles, lines t Similitude: Adds a scaling factor 7 parameters T ( x) = s. Rx + invariants: distance ratio, orientation, angles, line t 27

28 Transformations Class T Affine : T ( x) = Bx + t B 3x3 matrix 12 parameters: (B : 9; t : 3) invariants: lines, parallelism Polynomial: Parameters: T ( x) = P( x) D N Γ O= 1 O D = D N C O= 1 O D+ O 1 D = dimension, O = order (or degree) Example, D=2, O=2 P x ( x, y) = a 1 + b 1 x + c 1 y + d 1 xx + e 1 xy + f 1 yy P y ( x, y) = a 2 + b 2 x + c 2 y + d 2 xx + e 2 xy + f 2 yy 28

29 29 Transformations Class T Mesh-based (piecewise affine): For X T (P 1,P 2,P 3 ) = = 3 1 ) ( i h i P i X T P 1 P 2 P 3 = y x P P P P P P h h h y y y x x x

30 Transformations Class T Splines: Local polynomials d, with global continuity of order C(d-1). Number of parameters : depends on the number of nodes Locally affine transformations Free Transformation: One displacement u(x) per voxel Parameters: 3 times number of voxels Need for regularization to ensure diffeomorphism T ( x) = x + u( x) 30

31 Transformations Class T More Transformations Cosine (SPM) J. Ashburner and K. J. Friston. Spatial normalization. In A.W. Toga, editor, Brain Warping, pages Academic Press, J. Ashburner and K. J. Friston. The role of registration and spatial normalization in detecting activations in functional imaging. Clinical MRI/Developments in MR, 7(1):26-28, Multi-affine Alain Pitiot, Eric Bardinet, Paul M. Thompson, and Grégoire Malandain. Piecewise Affine Registration of Biological Images for Volume Reconstruction. Medical Image Analysis, 10(3): , June Poly-affine Vincent Arsigny, Olivier Commowick, Xavier Pennec, and Nicholas Ayache. A Fast and Log-Euclidean Polyaffine Framework for Locally Affine Registration. Research Report RR-5865, INRIA Sophia-Antipolis, March Radial Basis Functions M. Fornefett, K. Rohr, and H.S. Stiehl. Radial Basis Functions with Compact Support for Elastic Registration of Medical Images. Image and Vision Computing 19:1-2 (2001) Wavelet A Wavelet Tour of Signal Processing. Stéphane Mallat. Academic Press,

32 Overview Why doing registration? What type of transformation? What type of similarity? How to estimate the transformation? Geometric registration Iconic registration How to resample the image? 32

33 Depends on the joint histograms: What type of similarity? Intensity = 80 V=V+1 Image 1 Intensity = 100 Image 2 33

34 Similarity measures Assumption: Same intensity Image I intensity Adapted measure Image J intensity Sum of squared difference Sum of absolute differences S( T ) S ( T ) = = k k ( i k j k i k j k 2 ) Interpolation: j k J ( T ( xk )) 34

35 Similarity measures Assumption: Linear relation Image I intensity Adapted measure Image J intensity Correlation coefficient ρ IJ ( T ) ( i I )( k = k ik jk j k IJ J ) 35

36 Similarity measures Assumption: Statistical relationship Image I intensity Adapted measures Image J intensity Joint Entropy (Hill, 95; Collignon, 95) Mutual Information (Collignon, 95; Viola, 95) Normalized Mutual Information (Studholme, 98) MI ( I, J ) = H ( I ) + H ( J ) H ( I, J ) = P ( i, j) log P ( i, ( ) j) ( i j P i P j ) 36

37 Similarity measures More measures: Roche, Alexis - Malandain, Grégoire - Ayache, Nicholas. Unifying Maximum Likelihood Approaches in Medical Image Registration. Rapport de recherche de l'inria - Sophia Antipolis, Equipe : EPIDAURE 21 pages - Juillet 1999 L. Zöllei: "A Unified Information Theoretic Framework for Pair- and Group-wise Registration of Medical Images", Ph.D. thesis, MIT; MIT-CSAIL 37

38 Overview Why doing registration? What type of transformation? What type of similarity? How to estimate the transformation? Geometric registration Iconic registration How to resample the image? 38

39 Geometric registration Simple example (1): 39

40 Simple example (1): Extract feature points (structure tensor) T = G * ( I)( I) T Select n largest T Geometric registration T. Hartkens, K. Rohr, and H. S. Stiehl "Evaluation of 3D Operators for the Detection of Anatomical Point Landmarks in MR and CT Images"Computer Vision and Image Understanding, Volume 86, Number 2, pp ,

41 Geometric registration Simple example (1): Find local correspondences Block matching 41

42 Geometric registration Find optimal transformation argmin T k T( x k ) y k 2 42

43 Geometric registration Simple example (2), Iterative Closest Point: Consider 2 sets of points I 1 ={P i } and I 2 ={Q j } Consider an initial estimate of the transformation =T (usually identity) 1. Each point P i of I 1 is paired with the closest point Q j of I 2 2. We look for T i that minimizes the squared distance between impaired points 3. Update the position of P i+1 =T i (P i ) Iteration until convergence 43

44 1.Optimization of Pairings Qj Pi 44

45 2. Optimization of T Qj Pi 45

46 1-bis Optimization of Pairings Qj Pi 46

47 2-bis. Optimization of T Qj Pi 47

48 Geometric registration More general case: Parametric Transformation T, parameters p i Regularization energy E(T). Mechanical energy Gradient based UKU U ( U ) T T Minimize the sum of the two: arg min p i E r ( T ) + k T ( x k ) y k 2 48

49 Overview Why doing registration? What type of transformation? What type of similarity? How to estimate the transformation? Geometric registration Iconic registration How to resample the image? 49

50 50 Iconic registration Basic idea: try to find the transformation parameters that directly minimize the similarity between the 2 images ( ) ), ( ) ( min arg T F s r p I I E T E i + = = = i j k k k k k k k k j P i P j i P j i P IJ j i J j I i j i ) ( ) ( ), ( )log, ( ) )( ( ) ( 2 T = UKU ( ) T = U U

51 Optimization How to optimize the functional? Can you compute analytical solution? Can you compute the derivative? No: Powell, downhill simplex, genetic algorithms Yes: gradient descent More details: numerical recipes in C 51