Where are we now? Structural MRI processing and analysis

Where are we now? Structural MRI processing and analysis Pierre-Louis Bazin bazin@cbs.mpg.de Leipzig, Germany

Structural MRI processing: why bother? Just use the standards? SPM FreeSurfer FSL However: - Software are application-specific - Medical image computing research is continuously active - Every method has limitations or biases - Multiple analysis approaches help validate measurements - Customized processing enhance analysis capabilities

Structural MRI processing applications fmri localization Group statistics Anatomy-guided signal averaging Morphometry Lesion and tumor quantification Plasticity Brain atrophy monitoring Surgical planning Micro-anatomy parcellation Anatomical atlases...

Overview Images Cortical mapping Segmentation Registration Dealing with pathology

Images: can you name this organ?

Images: can you name these modalities?

Images: basic definitions superior Sagittal 3D image Coronal posterior Axial anterior 3D Voxel inferior right left Resolution: voxel dimension A. Vesalius, on the Fabric of the Human Body

Images: basic definitions density 3D image volume intensity H ( I ) = card ({ x I ( x) ] I ε, I + ε [ } ) Multi-channel images Intensity histogram Longitudinal images

Structural image processing 1. Segmentation Applications: labeling, morphometry, cortical thickness, shape analysis, lesion detection

Structural image processing 2. Registration Applications: group statistics, atlasing, longitudinal processing

Structural image processing 3. Cortical mapping Applications: fmri processing, area parcellation, tissue quantification, cortical atrophy

Structural image processing Special cases: pathology

Image Segmentation Goal: label automatically brain structures Many methods depending on structures, applications

Image Segmentation: tissue classification Each voxel is assigned to WM, GM or CSF Basic assumption: tissue types cluster in separable intensity distributions Methods: unsupervised (EM, FCM, etc) or supervised (knn, SVM, etc)

Segmentation method: binary threshold Algorithm: 1. Choose a threshold value 2. Classify voxels above and below threshold to different classes

Segmentation method: K-means Algorithm: 1. Choose K mean values 2. Classify voxels to belong to the closest mean value 3. Recompute the means from the classified voxels

Segmentation method: Fuzzy C-means Algorithm: 1. Choose K mean values 2. Give a fuzzy membership to voxels based on distance to all means 3. Recompute the means from a weighted average of fuzzy voxels

Segmentation method: EM Algorithm Algorithm: 1. Choose K mean values 2. Give a membership probability to voxels based on distance 3. Recompute the means from the expectation on voxels

So, which to choose? Image slice K-means FCM EM Threshold: depends on how the threshold is determined K-means: very sensitive to initialization, oscillatory behavior FCM: stable convergence, lowest sensitivity EM: stable, probabilistic interpretation, sensitive to initialization

Problems: noise Pre-filtering to remove noise (anisotropic diffusion, total variation..) Post-filtering to smooth the classification Smoothness penalty functions/markov Random Fields

More problems: image artifacts Segmentation algorithms must also deal with inhomogeneities, partial volume effects

Intensity Modeling Model of choice: Mixture of Gaussian WM GM CSF Reality: Not so nice T1 T1 WM GM Vasculature Dura mater CSF

Spatial Structure Modeling vs. Statistical atlas prior Multi-atlas segmentation Important challenge: image registration

Higher-dimensional data Multi-modal segmentation DWI: segmentation of white matter tracts fmri: network identification

Recent trends in segmentation Multi-atlas segmentation Patch-based segmentation Surface evolution with boundary learning

How to compare? Validation Manual delineations Automated segmentations Overlap Distances Dice & Jaccard coefficients 2 A B A+ B A B A B Sensitivity, specificity,. Average surface distance Signed surface distance Hausdorff distance

How to compare? Validation Brain phantoms (Brainweb) - simulated MRI - ground truth anatomy Manual delineation databases - real MRI data - specific delineation goals Scan-rescan experiments - real data, ideal comparison - only measures robustness Grand Challenges (MICCAI) - real data, blind processing - direct comparison of mehods

Segmentation summary Segmentation extracts regions of interest from brain MRI - There are many segmentation methods & approaches with different assumptions and biases - Anatomical complexity, noise, inhomogeneities, pathology, require elaborate prior models for automated segmentation - Recent trends include non-linear registration, patch matching, machine learning, data fusion, - Validation performance is important but task-specific - Pathology makes many generic methods misinterpret data

Image Registration Registration is the application of a geometric transformation to the coordinate system of an image to bring it into correspondence with a second image. T(A) -1 T (B) Image A Image B

Image Registration Example ORIGINAL IMAGES

Image Registration Example ORIGINAL IMAGES REGISTERED IMAGES

Image Registration Basic Steps 1. Define class of transformations (rigid body, affine, nonlinear) 2. Define similarity index (homologous features, intensity, other more complex measures) 3. Optimize transformation parameters to maximize similarity index (a.k.a minimize disparity index, cost function) 4. Transform the images with interpolation How do we decide on the class of transformation or similarity index to be used?

Spatial transformation examples Rotation Shearing Translation Perspective Scaling Non-linear

Similarity index examples Intensity difference d ( A, B)= I A ( X ) I B (D ( X )) Landmark matching d ( A, B)= X L, A D ( X L, B ) 2 2 X L Normalized cross correlation (I A ( X ) I A)( I B (D( X )) I B ) s (A, B)= σ Aσ B X Normalized mutual information H ( A)+ H (B) s (A, B)= = H (A, B) p( I A )ln p( I A )+ p ( I B ) ln p ( I B ) I I I p (I A, I B ) ln p ( I A, I B )

Intramodal / Intersubject How do we decide on the class of transformation or similarity index to be used?

Intermodal / Intrasubject MRI PET How do we decide on the class of transformation or similarity index to be used?

Intramodal / Intrasubject Time 1 Time 2 2 years later How do we decide on the class of transformation or similarity index to be used?

Template registration How do we decide on the class of transformation or similarity index to be used?

Image Interpolation Interpolation the model-based recovery of continuous data from discrete data within a known range of abscissas f(x)?? x 1-D 2-D 3-D

Interpolation: Nearest Neighbors Choose the value from the closest sample point Advantages: - simple and fast - never modifies input values Disadvantages: - coarsest possible method - blocking artifact

Interpolation: Linear Choose a weighted average from the closest 2dim points Advantages: - still simple and fast - maintains intensity bounds Disadvantages: - still coarse - severe blurring artifact

Interpolation: Windowed Sinc Convolve the data with a sinc kernel Advantages: - ideal case for band-limited signals Disadvantages: - requires windowing (Lanczos) - ringing artifact at edges - somewhat slow

Interpolation artifacts - Ringing Oscillations occur at sharp boundaries

Interpolation artifacts - Aliasing Occurs when downsampling to a resolution that does not sufficiently represent structure

Interpolation artifacts - Blocking Occurs mainly with nearest-neighbor interpolation

Interpolation artifacts - Blurring Occurs with repeated interpolations Can be minimized by composing transformations

Non-linear Transformations How do we represent non-linear transformations? What is a diffeomorphic transformation? What is a symmetric transformation? Are there artifacts?

Registration Validation? Groupwise average quality manual delineations (NIREP, MindBoggle) Application-specific metrics?

Image Registration Trends ANTs / SyN Demons algorithm Symmetric diffeomorphism - conservative deformations - best on evaluation experiments - estimates both deformations - almost perfect? Fast registration - many variants: symmetric, diffeomorphic, spherical, diffusion... - easy to implement Are we done with registration research??

Image Registration Challenges Correspondences? Partial coverage Complex shapes

Registration Summary Registration bring brain images in correspondence - assumptions of meaningful correspondences - variable results for inter- intra- subject / modality - interpolation may introduce artifacts - best methods highly accurate, but still limited - validation tools are becoming available - challenges arise with pathologies, slab imaging, high resolution imaging of complex shapes

Cortical Mapping Segmentation of the cortex: cortical surfaces Inflated representations:

How are surfaces generated? Given a segmentation map: j, ps j [ 0,1] ps j 0.5 ps j < 0.5 inside the object outside the object Implicit representation: X Surface ps ( X ) = 0.5

How are surfaces generated? Contours defined by linear interpolation between voxels Marching squares Implicit resolution 6 or 18-C Marching cubes 26-C Voxel connectivity matters

Cortical Meshes & Resolution Mesh resolution ~ number of points [10,000 750,000]

Cortical Mapping Mapping of surface geometry & surface anatomy Curvedness Shape index Mapping of cortical thickness Mapping of volumetric data mean Quantitative T1 profiles

Cortical Thickness How to measure cortical thickness? [Yezzi & Prince, 2003]

Cortical Layer Geometry? CSF WM?

Cortical Layer Geometry CSF WM The bending of cortical layer changes their relative thickness in order to preserve their respective local volume [Bok, Z. ges. Neurol. Psychiat., 1929]

Cortical Layer Geometry 2 mm

Cortical Layer Mapping Incorrect layering model: curvature artifacts

Cortical Layer Mapping CSF T1 profiles oc r t i c la d e p ht WM

Cortical Mapping Summary Cortical mapping projects MR data onto the cortical sheet - Surface inflation enhances cortex visualization - Geometric shape information available, not very useful - Cortical thickness hard to define everywhere consistently - Defining cortical depth brings back the 3rd dimension, outlines laminar structure of the cortex - Cortical layer geometry impacts intra-cortical mapping

Pathologies For segmentation: - undefined tissue types - lower SNR - differ from priors For registration: - unusual shapes - non-matching lesions Dedicated methods?

Pathologies For segmentation: - undefined tissue types - lower SNR - differ from priors For registration: - unusual shapes - non-matching lesions Dedicated methods?

How to choose an algorithm? SPM The standards are good, but newer methods have improved upon them FreeSurfer FSL Where to find these new methods? NITRC.org MICCAI Grand Challenges How to evaluate these methods? Check validation experiments Evaluate relevance to your study How do I combine algorithms? Pipeline environments (LONI, JIST, Nipype)

NITRC: NeuroInformatics Tools & Resources

MICCAI Grand Challenges Principle: - a manually segmented database is created - competitors receive most of the data but only a subset of labels - some new data must be processed 'live' Since 2007 Many Grand Challenges are used afterwards for validation of new methods

Software developed at the Institute

Conclusions Structural image processing enables: - segmentation, measurement of brain structures - detection, quantification of pathology - grouping of multi-modal, multi-subject data Many methods are available: - search for what you want to do, not what's customary - interrogate validation & challenges - use integrative, modular systems to combine tools