HST.582J / 6.555J / J Biomedical Signal and Image Processing Spring 2007

Transcription

1 MIT OpenCourseWare HST.582J / 6.555J / J Biomedical Signal and Image Processing Spring 2007 For information about citing these materials or our Terms of Use, visit:

2 Harvard-MIT Division of Health Sciences and Technology HST.582J: Biomedical Signal and Image Processing, Spring 2007 Course Director: Dr. Julie Greenberg Medical Image Registration I HST Lilla Zöllei and William Wells

3 The Registration Problem Spring 2007 HST

4 The Registration Problem Spring 2007 HST

5 The Registration Problem Spring 2007 HST

6 Applications multi-modality fusion (intra-subject) time-series processing e.g.: fmri experiments, cardiac ultrasound warping across patients (inter-subject, uni-modal) warping to / from atlas for anatomical labeling image-guided surgery: modeling tissue deformation, comparing pre- and intra-operative scans, Spring 2007 HST

7 The Registration Problem T init T final Spring 2007 HST

8 Medical Image Registration Medical image data sets Transform (move around) initial value Compare with objective function score Optimization algorithm motion parameters Spring 2007 HST

9 Roadmap Data representation Transformation types Objective functions Feature/surface-based Intensity-based Optimization methods Current research topics Spring 2007 HST

10 Data Representation z y object Object represented as a function example: coordinates Æ intensity x Spring 2007 HST

11 Data Representation f(x): space Æ intensity x 2 representation of moved object x 1 gx ( ') = ftx ( ( ')) x 2 x 1 Spring 2007 HST

12 Data Representation x 2 x 0 focus on a feature before motion: f ( x 0) = f0 after motion What value of x 0 : g( x 0 ') = f? x 1 f( T( x 0 ')) = f 0 1 x 0 T( x0 ')= x0 x 0 ' = T (x 0) x 2 0 Feature of f 0 now is located at T 1 ( x 0 ) motion of object is T 1 (.) x 1 Spring 2007 HST

13 Medical Image Registration Medical image data sets Transform (move around) initial value Compare with objective function score Optimization algorithm motion parameters Spring 2007 HST

14 Space of transformations Data dimensions 2D-2D 3D-3D 2D-3D Example cardiac ultrasound, x-ray patient repositioning, histology MRI, MR/MR, MR/CT, CT/CT, PET-MR, X-ray/CT, fluoroscopy-ct, surface model/video Spring 2007 HST

15 Class of transformations What motions or distortions are allowed to merge datasets? Rigid Transformations: displacement rotation & displacement Non-Rigid Transformations: parametric affine piecewise-affine. non-parametric Spring 2007 HST

16 Displacement only 2D: 2 parameters 3D: 3 parameters T( x) x+ D For example: x 1 D x x D x 1 T = + T x 2 = x 2 + D 2 x x D x3 x 3 D 3 in 2D in 3D Spring 2007 HST

17 Rigid Motion T( x ) R( x ) + D both rotations and displacements are allowed length-preserving transformation order of transformations does matter! 2D: 3 parameters; 3D: 6 parameters for example: in 2D (non-linear in θ) T( x) = Rx cosθ sin R = θ sin θ cos θ + D R: valid rotation matrix R T R = 1 and R = +1 Spring 2007 HST

18 in 3D: rotate by θ about axis Nˆ if q is a unit quaternion, such that q = cos θ ; Nˆ sin θ 2 2 q 2 + q 2 q 2 q 2 2( q q + qq ) 2( qq + qq ) R = 2(q q + q q ) q 2 q 2 + q 2 q 2 2( q q + q q ) ( q q q3 q ) 2( qq+ qq ) q q 1 q 2 + q 3 Horn, B.K.P., Closed Form Solution of Absolute Orientation using Unit Quaternions, Journal of the Optical Society A, Vol. 4, No. 4, pp , April Spring 2007 HST

19 Non-rigid transformations Parametric affine piecewise-affine others Non-parametric Spring 2007 HST

20 Affine Transformation T( x ) M( x) + D M: square matrix beyond rigid motion, allows shears and scaling preserves notion of parallel lines 2D: 6 parameters 3D: 12 parameters Spring 2007 HST

21 2D affine: x 1 m 11 m 12 x 1 D 1 T = + x m m x D OR x 1 m11 m 12 D 1 x1 T x2 = m 21 m 22 D 2 x 2 linear form T( X ) = QX homogeneous coordinate Spring 2007 HST

22 2D affine: determined by control points Y Y Y = Q X X X 1 = [ = Q Y Y Y X X X Y QX ] [ ] Y QX =[ ][ ] Y = QX 3 3 (if the inverse exists) Y 1 X 1 X 2 X 3 Y 2 Y 3 Spring 2007 HST

23 Affine transformation for other points: determined by motion of control points X y 1 x 1 y 2 = Q x Y Spring 2007 HST

24 Piecewise affine 2D example: subdivide space of X into triangles use different affine transformation for each triangle (determined by vertices ) -e.g.: some use in breast registration Spring 2007 HST

25 Other Parametric Transformations Finite element models engineering methods to simulate mechanical / electrical systems (discretization of the space; integral problem formulation turned into system of linear equations) Spline models Thin-plate splines, B-splines, Cubic splines Fred Bookstein, Morphometric Tools for Landmark Data : Geometry and Biology Spring 2007 HST

26 S. Timoner: Compact Representations for Fast Non-rigid Registration of Medical Images (MIT PhD`03) Spring 2007 HST

27 S. Timoner: Compact Representations for Fast Non-rigid Registration of Medical Images (PhD`03) Spring 2007 HST

28 Non-parametric models Continuum mechanics elastic solid models, fluid transport, 2000 IEEE. Courtesy of IEEE. Used with permission. G. E. Christensen* and H. J. Johnson: Consistent Image Registration. IEEE TMI, VOL. 20, NO. 7, JULY 2001 Spring 2007 HST

29 2000 IEEE. Courtesy of IEEE. Used with permission. Christensen, G. E., et al. Large-Deformation Image Registration using Fluid Landmarks. Image Analysis and Interpretation 2000, Proceedings of 4th IEEE Southwest Symposium, pp Spring 2007 HST

30 Medical Image Registration Medical image data sets Transform (move around) initial value Compare with objective function score Optimization algorithm motion parameters Spring 2007 HST

31 Objective functions measure how well things are lined up assumption: the input datasets (U,V) are related to each other by some transformation T define: energy function to be optimized E = f( U( x ), V( T( x ))) 2 main styles: feature- or surface-based intensity-based Spring 2007 HST

32 Feature-based measures compute alignment quality based upon the agreement of 2 sets of landmark features assumption: landmarks visible in both images they can be reliably located and they can guide the alignment of the whole image Spring 2007 HST

33 Surface-based measure a simple measure of alignment discretized curve points E = i 2 min x i y j = C 2 (x i ) j i x i C ( x ) y min x y j j y i chamfer function / distance transform Spring 2007 HST

34 Chamfer function C(x): distance to nearest point on curve curve fast way to calculate C(x) on a grid C(x) = G. Borgefors. Hierarchical chamfer matching: a parametric edge matching algorithm. IEEE Trans. on Pattern Analysis and Machine Intelligence, PAMI- 10(6): , November 1988 Spring 2007 HST

35 if structure is missing, not a robust measure of alignment; large penalties may swamp the measure x y Alternative solution: robust chamfer matching C '( x) min( d, C ( x )) y 0 y not close to any point on y (penalty saturates) Spring 2007 HST

36 feature-based techniques used at the BWH: laser data locater data patient skin in MR similar methods pioneered by Charles Pelizzari (Dept Radiation Oncology, U. Chicago) Head in Hat MR-SPECT/PET registration; extract surfaces of heads; register the surfaces Spring 2007 HST

37 Intensity-based measures compute alignment quality based upon the intensity profiles of the input images no landmark or feature selection is necessary Spring 2007 HST

38 SSE and correlation (I) SSE: sum of squared errors E = U ( x ) i V( T( x )) i 2 x i = 2 2 U ( x ) 2 U( x ) V( T( x )) + V ( T( x )) i i i i x i 1 st term: no dependency over T; 3 rd term contributes a constant* equivalent problem: E ' U ( x i ) V ( T ( x i )) classical correlation x i Spring 2007 HST

39 SSE and correlation (II) For translation only: T(x) = X + D E ' = U ( x i ) V ( x + D) i x i Convolution can use Fourier methods Spring 2007 HST

40 SSE and correlation (III) CAVEAT: problems can surface if some structure is missing or extra-large quadratic penalties can swamp the measure E = U ( ) x i x i V( T ( )) x i 2 Spring 2007 HST

41 More robust measures (I) SAV: summed absolute value Ex ( ) = Ux ( i ) V( Tx ( i )) x i Amy Gieffers, 6A HP Andover*: cardiac ultrasound registration *later Agilent, later Phillips Spring 2007 HST

42 More robust measures (II) Saturated SSE penalty saturates Ex ( ) = min (d, Ux ( i ) V( Tx ( i )) 2 ) x i Spring 2007 HST

43 Multimodal inputs Spring 2007 HST

44 Multimodal Inputs correlation fails for multi-modal registration when intensities are different; e.g.: MR-CT one solution: apply a special intensity transform to the MRI to make it look more like CT; then compute the correlation measure e.g.: Petra Van Den Elsen Spring 2007 HST

45 MR-CT situation bone white matter fat CT gray matter CSF air MR Spring 2007 HST

46 Histograms Data Bins or buckets Counts: Rel. frequency: Spring 2007 HST

47 Histogram Joint Intensity of Images Images: U V histogram Y Y V X X U intensities relative freq. Joint intensities: (1,2)(2,2)(1,2)(1,3)(2,3)(1,3)(1,2)(2,2)(1,2) Spring 2007 HST

48 MRI & CT pairs misaligned slightly misaligned aligned Spring 2007 HST

49 Joint histogram: MRI & CT registered MRI MRI CT CT Spring 2007 HST

50 Joint histogram: MRI & CT; slightly off MRI MRI CT Correct registration CT Slight mis-registration Spring 2007 HST

51 Joint histogram: MRI & CT; significantly off MRI MRI CT Correct registration CT Significant mis-registration Spring 2007 HST

52 entropy: H( p( x )) p( x ) log 2( p( x )) e.g.: predictable coin -- no uncertainty, biased coin -- lowest entropy 1 moderate uncertainty, moderate entropy x fair coin -- most uncertain, highest entropy p.5 0 H T H T H T Spring 2007 HST

53 Examples of joint intensity distributions V 1 4 V U U H = log (4) H = log (6) 2 2 Spring 2007 HST

54 Real CT-MR registration: 3D starting position Spring 2007 HST

55 CT-MR registration final result Spring 2007 HST

56 CT-MR registration movie From: Wells, W. M., et al. "Multi-modal Volume Registration by Maximization of Mutual Information." Medical Image Analysis 1, no. 1 (March 1996): Courtesy Elsevier, Inc., Used with permission. Spring 2007 HST

57 Registration by minimization of joint entropy Alignment by maximization of mutual information Viola, PhD 1995 Pluim, PhD 2001 many more papers (2002: more than 100) West Fitzpatrick et al 1998 JCAT clear winner MR/CT also good for MR/PET Spring 2007 HST

58 Chuck Meyer et al. (Radiology Dept, U Mich.) Non-rigid mutual information registration Thin-plate spline warp model Downhill simplex optimizer rat brain auto-radiograph CT breast MRI breast MRI Spring 2007 HST

59 END Spring 2007 HST