Norbert Schuff VA Medical Center and UCSF

Transcription

1 Norbert Schuff Medical Center and UCSF Medical Imaging Informatics N.Schuff Course # Slide 1/67

2 Objective Learn the principle segmentation techniques Understand the role of segmentation in medical imaging

3 Overview Definitions Segmentation methods Deterministic/probabilistic Comparisons New trends Role of Segmentation Summary & Conclusion Literature

4 The Concept Of Segmentation Identify classes (features) that characterize this image! Intensity: Bright - dark Shape: Squares, spheres, triangles Texture: homogeneous speckled Connectivity: Isolated - connected Topology: Closed - open

5 More On The Concept Of Segmentation: Can you still identify classes in each image?

6 Segmentation Based On Intensity & Topology Gray matter segmentation topology Intensity

7 Segmentation Based On Texture & Structure Texture Tensors

8 Definitions Segmentation is the partitioning of an image into regions that are homogeneous with respect to some characteristic. In medical context: Segmentation is the delineation of anatomical structures and other features of interest, i.e. lesions, tumors, functional units.

9 Formal Definition If the domain of an image is, then the segmentation problem is to determine classes Z k, whose union represent the entire domain N k 1 Classes are connected by intersections : Z k k Z k Z j ; j;0

10 More Definitions When the constraint of connectivity is removed, then the determination of classes Z k is termed pixel classification. Determining the total number of classes K can be a challenging problem. In medical imaging, the number of classes is often based on a-priori knowledge of anatomy, e.g. K=4 (gray, white, CSF, other) for brain imaging.

11 Labeling Labeling is the process of assigning a meaningful designation to each image region Freesurfer Brain Labeling Template This process is often performed separately from segmentation. Generally, computer-automated labeling is desirable Labeling and classes may not necessarily share a one-to-one correspondence

12 Dimensionality Dimensionality refers to whether the segmentation operates in a 2D or 3D domain. Generally, 2D methods are applied to 2D images and 3D methods to 3D images. In some instances, 2D methods can be applied sequentially to 3D images.

13 Characteristic and Membership Functions A characteristic function is an indicator whether a pixel at location j belongs to a particular class Z k k j 1 if. element. of. class 0 otherwise This can be generalized to a membership function, which does not have to be binary valued. k 1 0 j 1, for. all. pixels.&. classes N k k j 1, for. all. pixels The characteristic function describes a deterministic segmentation process whereas the membership function describes a probabilistic one.

14 y Class Y y y Simple Membership Functions Binary Probabilistic b=50 b=25 b= Feature x (i.e. intensity x x x1 sigmoid. function 1 y 1 exp a x * b

15 Segmentation Has An Important Role Computational diagnostic Surgical planning Database storage/retrieval SEGM Image registration Atlases informatics Image transmission Quantification Partial volume correction Super resolution

16 Segmentation Methods

17 Threshold Method Angiogram showing a right MCA aneurysm Histogram (fictitious) Threshold (T A ) (T B ± ) Dr. Chris Ekong; a.0

18 Threshold Method Original Segmented

19 Threshold Method Applied To Brain MRI White matter segmentation Results in major anatomical errors

20 Threshold: Principle Limitations Works only for segmentation based on intensities/contrasts Robust only for images with global uniformity and high contrast to noise ratios Local variability causes distortions The intrinsic assumption is that the probability of features is uniformly distributed

21 Region Growing - Edge Detection Seed point Region growing groups pixels or subregions into larger regions. A simple procedure is pixel aggregation, It starts with a seed point and progresses to neighboring pixels that have similar properties. Region growing is better than edge detection in noisy images. Guided by e.g. energy potentials with a penalty on high energy Similarity: Ii I j V i, j z score i, j Edges: V i, r I I erf ( r) I i r r

22 Region Growing: Principle Limitations Segmentation results dependent on seed selection Local variability dominates the growth process Global features are ignored Generalizations: Unsupervised segmentation (i.e. insensitive to selection of seeds) Exploitation of both local and global variability The intrinsic assumption still is that the probability of features is uniformly distributed

23 Clustering Can be applied to pre-defined features, e.g. intensities, texture Usually requires transformation of image to a feature space Clustering seeks patterns in the feature space Two commonly used clustering algorithms K-mean Fuzzy C-mean

24 Definitions: Clustering Clustering is a process for classifying patterns in such a way that the samples within a class Z k are more similar to one another than samples belonging to the other classes Z m, m k; m = 1 K. The number of clusters is usually pre-defined

25 K-Means The k-means algorithm attempts to cluster n patterns based on attributes (e.g. intensity) into k classes k < n. The objective is to minimize total intra-cluster variance in the least-square sense: K k 1 jsk 2 j k µ k is the mean point (centroid) of cluster k

26 d2 K- means Hypothetical: Two features - three classes d1

27 d2 K - means Four classes d1

28 d2 K - means Original (3 classes) d1

29 d2 K - means 2 clusters Two classes d1

30 d2 K means TRAPPED! Three classes Three classes d1

31 Fuzzy Clustering The fuzzy C-means algorithm is a generalization of K- means. Rather than assigning a pattern to only one class, the fuzzy C-means assigns the pattern a number m, with 0 <= m <= 1.

32 Component Fuzzy C - means Four classes Component 1 These two components explain 100 % of the point variability.

33 Fuzzy C- Means Segmentation I Two classes Original Class I Class 2

34 Fuzzy Segmentation II Four classes Class I Class 2 Original Class 3 Class 4

35 Brain Segmentation With Fuzzy C- Means Bias field inhomogeneity contributes to the problem of poor segmentation

36 Clustering: Principle Limitations Convergence to the optimal configuration is not guaranteed. Outcome depends on the number of clusters chosen. No easy control over balancing global and local variability Intrinsic assumption of a uniform feature probability is still being made Generalization needed: Relax requirement to predetermine number of classes Balance influence of global and local variability Possibility to including a-priori information, such as non-uniform distribution of features.

37 Segmentation As Probabilistic Problem Treat both intensities Y and classes Z as random distributions The segmentation problems is to find the classes that maximize the likelihood to represent the image

38 Segmentation As A Probabilistic Problem Prior pz ( Y) p( Y Z) * p( Z) Likelihood Z is the segmented image Y is the observed image p(z): p(y Z): P(Z Y): prior probability likelihood posterior distribution that best represents the data Posterior

39 Probability In Spatial Context Model classes Z as Markov Random Fields (MRF) MRF Rules: Classes exist: p(z) > 0 for all z in Z Probability of z at a location depends on the spatial neighborhood Observed intensities in the neighborhood are a random process

40 MRF Based Segmentation 1 st and 2 nd order MRFs (p,q) (p,q+1) (p+1,q) (p=1,q+1)

41 MRF Based Segmentation 1 st and 2 nd order MRFs Step I: Define prior class distribution energy, i.e. using a random process (p,q) (p,q+1) zz, prior 1, z z 1 z z prior prior (p+1,q) (p=1,q+1)

42 MRF Based Segmentation 1 st and 2 nd order MRFs (p,q) (p,q+1) Step I: Define prior class distribution energy: zz, true 1, z z 1 z z true true (p+1,q) (p=1,q+1) Step II: Select distribution of conditional observation probability, e.g gaussian: E y 2 p, q p, q. yz 2 2 z y p,q is the pixel value at location (p,q) z and z are the mean value and variance of the class z z

43 MRF Based Segmentation 1 st and 2 nd order MRFs (p,q) (p+1,q) (p,q+1) (p=1,q+1) Step III: Solve (iteratively) for the minimal distribution energy arg min E p, q z, z y 2 p. q y z s true 2 sn 2 z s z assignment similarity energy

44 probability Iterative Segmentation Process 3 2 Update MRF classify Global Distribution 1 intensity Distribution model

45 MRF Based Segmentations 4T MRI, SPM2, priors for GM, WM based on 60 subjects

46 Generalization: Mixed Gaussian Distributions y 2 arg min E p, q z, z. p. q z y s t 2 tn 2 z s White matter z q z arg min E p,, z z y s t tn s Find solution iteratively using Expectation Maximization (EM) White matter White matter lesion y y p. q w w p. q wml wml

47 probability EM Based Segmentation UpdateMRF classify Global Distribution intensity Update distribution model

48 EM Segmentation Standard 1.5 MRI, SPM2, tissue classes: GM, WM, CSF, WM Lesions

49 Limitations Of Energy Minimization Case where energy minimization might not work: green: wrong segmentation red: correct segmentation Because the separation energy depends on mean and variance of the classes, separation can go wrong if red variance is much smaller than green variance, even when red mean is larger than green mean.

50 EM segmentation problems EMS EMS gray matter segmentation areas where EMS has problems.

51 Potential Solution: Gradient Based Algorithms The separation penalty is defined based on magnitude and direction of image gradient: E p, q I 1 p I q Small gradient magnitude: Large separation penalty (no change in class) in gradient direction Small separation penalty (class change) in other directions small Large gradient magnitude Small separation penalty in grad directions Large penalty in other directions. Large penalty

52 Local Structure Gradients (LSG) Case where energy minimization did not work: green: wrong segmentation red: correct segmentation Using LSG leads to: higher separation energy along grad direction Lower separation energy in orthogonal directions leads to a correct red segmentation

53 Energy versus Gradient Based Segmentations EMS gray matter Geo-Cuts gray matter

54 Other Segmentation Methods Edge detection Split and merge Level sets Graph partitioning Fractal (multiscale)

55 Edge Detection Original Edge filtered Image edges may be defined by: Zero-crossings of the Laplacian (differential operator) Maxima of gradient fields Maxima of coefficients of basis functions (e.g. fourier, wavelets, curvelets others (huge related literature)

56 Split and Merge Principle Example: lower spine Source:

57 Level Sets Principle: A level set of a function, e.g. representing shape, is the set were the function takes on a constant value c, e.g. zero at the plane intersection. Example: White Matter Segmentation Source:

58 Graph Partitioning Principle: Example: Liver Tumor on CT Source:

59 Pixel order MultiScale: Co-Occurrance Matrix (CM) Definition: The CM is a tabulation of how often different combinations of pixel brightness values (gray levels) occur in an image. kernel CM Image Sharp Blurred image Pixel that stores calculated CM value More Blurred Pixel order

60 Evaluation of CM CM MRI with spatially varying intensity bias MRI with spatially homogenous intensity bias Medical Imaging Informatics 2011, N.Schuff Course # Slide 60/67

61 Fractal Dimensions Intuitive Idea: Many natural objects have structures that are repeated regardless of scale. Repetitive structures can be quantified by fractal dimensions (FD). Sierpinski Triangle Medical Imaging Informatics 2011, N.Schuff Course # Slide 61/67

62 Fractal Dimensions Definition: Box counting Sierpinski Triangle Medical Imaging Informatics 2011, N.Schuff Course # Slide 62/67

63 Fractal Dimensions MEDICAL IMAGE SEGMENTATION USING MULTIFRACTAL ANALYSIS Soundararajan Ezekiel; Medical Imaging Informatics 2011, N.Schuff Course # Slide 63/67

64 Summary Automated Threshold MRF EMS Graph Partitioning Fuzzy C- Means Level Sets Split & Merge Initializing Semiautomated Region growing K-means Cooccurrence Edge Detection Manual Tracing Fractal Dimensions Medical Imaging Informatics 2011, N.Schuff Course # Slide 64/67

65 Literature 1. Segmentation Methods I and II; in Handbook of Biomedical Imaging; Ed. J. S. Suri; Kluwer Academic WIKI-Books 1. SPM: FSL-FAST: Medical Imaging Informatics 2011, N.Schuff Course # Slide 65/67