Lecture: Segmentation I FMAN30: Medical Image Analysis. Anders Heyden

Size: px

Start display at page:

Download "Lecture: Segmentation I FMAN30: Medical Image Analysis. Anders Heyden"

Steven Gerald McGee
5 years ago
Views:

1 Lecture: Segmentation I FMAN30: Medical Image Analysis Anders Heyden

2 Content What is segmentation? Motivation Segmentation methods Contour-based Voxel/pixel-based Discussion

3 What is segmentation? Segmentation is the process of dividing an image into different parts/segments that represent interesting parts of the image, e.g. different organs or anothomical structures. Usually the backgound constitute one segment The segmentation can either be represented as different countours that encloses the different parts or masks that are binary (or integer) images where 0 represents background pixels and 1, 2, etc represents different objects.

4 Example: Segmentation of leukocytes in stained smears of peripheral blood. Initial segmentation of nuclei Final segmentation of nuclei and cytoplasm

5 Motivation Segmentation is a link between low-level image processing and high-level methods, such as classification. It is often of vital importance to segment human organs in different image modalities. It is also necessary to segment cells in different types of smears and sections. Segmentation reveals the shape of objects and also makes it possible to investigate image intensities and texture within the objects.

6 Segmentation methods There are roughly two different classes of segmentation methods: - Find the perimeter of the object (contour tracking, active countours, level-set-methods, fast marching methods, watershed, variational methods) - Classify pixels as inside or outside (thresholding, split and merge, graph-based segmentation) - Some methods could be considered as a mixture of these two classes

7 Other characteristics The segmentation may or may not take into account neighbouring pixels, i.e. classify each pixel independently of the classification of surrounding pixels or the classification of each pixel depends on the classification of neighbouring pixels. The segmentation may or may not take into account any prior information about the object, i.e. the shape, size, texture, etc.

8 Segmentation methods Thresholding (Otsu) Split and merge Mathematical morphology (watershed) Active contours Variational methods Chan-Vese-segmentation Graph-based segmentation These lectures constitute a toolbox, where you (hopefully) can select an appropriate segmentation method for each application.

9 Criteria for complete segmentation Every pixel belongs to one region No pixel can belong to more than one region Every region is a connected collection of pixels Each region is uniform (according to some criterium) Each pair of neighboring regions is non-uniform The concept of connectivity will be defined in detail later!

10 Thresholding Given a (grayscale) input image I(x,y), construct a binary image B(x,y): Gray-scale image Thresholded image

11 Problems with thresholding The threshold T has to be chosen The resulting segmentation might not be connected Classification of one pixel independent of its neighbours Can produce irregular boundaries Acceptable results only when the object has a significant different gray-level than the background Advantage: Simple to use and implement!

12 How to select the threshold Look at the histogram of the image Assume that the intensities in the background and in the object are Gaussian distributed with different means and variances, estimate these parameters and select the threshold according to e.g. equal error rate Otsus method: Minimize inter-class variance, which is equivalent to maximizing intra-class variance

13 Effect of different thresholds

14 Histogram

15 Gaussian distributions Assume that the intensities within the object (foreground) are Gaussian distributed, i.e. Assume that the intensities for boundary pixels follows another Gaussian distribution Select the threshold according to equal error rate, i.e. the probability of misclassifying a background pixel is the same as the probability of missclassifying an object pixel Select the threshold according to

16 Gaussian distributions

17 Otsus method Introduce the following variables Minimize the intra-class variance: Equivalently maximize the inter-class variance:

18 Otsus method: Example Histogram 6x6 Image Set T=2, giving

Threshold T<0 T=0 T=1 T=2 T=3 T=4 Weight, Background W b = 0 W b = 0.222 W b = 0.4167 W b = 0.

2609 M b = 2.0313 Variance, Background σ 2 b = 0 σ 2 b = 0 σ 2 b = 0.2489 σ 2 b = 0.4637 σ 2 b = 1.

3611 W f = 0.1111 Mean, Foreground M f = 2.3611 M f = 3.0357 M f = 3.7143 M f = 3.8947 M f = 4.

5152 σ 2 f = 0.2130 σ 2 f = 0 Within Class Variance σ 2 W = 3.1196 σ 2 W = 1.5268 σ 2 W = 0.

19 Threshold T<0 T=0 T=1 T=2 T=3 T=4 Weight, Background W b = 0 W b = W b = W b = W b = W b = Mean, Background M b = 0 M b = 0 M b = M b = M b = M b = Variance, Background σ 2 b = 0 σ 2 b = 0 σ 2 b = σ 2 b = σ 2 b = σ 2 b = Weight, Foreground W f = 1 W f = W f = W f = W f = W f = Mean, Foreground M f = M f = M f = M f = M f = M f = Variance, Foreground σ 2 f = σ 2 f = σ 2 f = σ 2 f = σ 2 f = σ 2 f = 0 Within Class Variance σ 2 W = σ 2 W = σ 2 W = σ 2 W = σ 2 W = σ 2 W = Between Class Variance σ 2 B = 0 σ 2 B = σ 2 B = σ 2 B = σ 2 B = σ 2 B =

20 Thresholding color images Select one threshold for each channel (R,G,B) Select a plane in R-G-B-space and define object pixels according to Both the threshold T and the normal to the plane have to be selected Select a reference color and a distance and define object pixels according to Etc.

21 Example Tr=Tb=Tg=200 [50-100: : ]

22 Example Distance 50 from the reference color (80,100,50)

23 Region growing Goal: Divide the image R into regions R 1,, R n Means: Use some criterium P described as For instance P might be true if the pixel intensities are similar.

24 Criteria for region growing

25 Algorithm for region growing Start with a number of seed pixels. Usually pixels that are known to lie within the object of interest Add neighbouring pixels as long as Iterate until no further pixels could be added Continue with new seed pixels if needed, e.g. if the object is not covered or if a new object needs to be segmented

26 How to select the criterium P Depends on images and applications The difference between the darkest and brightest pixels within a region should be less than T The intensity of each pixel should not be more different than d from a pre-defined intensity m. Etc.

27 Example of region growing The difference between the brightest and the darkest pixel should be at most 3:

28 Example of region growing Region growing can be very sensitive to selected parameters: d = 41 d = 42

29 Split and merge Start with the whole image as one initial region In each step Merge neighboring regions that are similar Split regions that are not similar Similarity could be measured according to a criteria P in the same way as for region growing

30 Algorithm for split and merge Define a property P on each possible region, with values TRUE and FALSE (should resemble similarity). Start with R={R 1 } Iterate over all regions If P(R i )=FALSE, split R i into four smaller regions (2D) If P(R i +R j )=TRUE for two neighboring regions, merge them to one region Iterate until a stationary solution is found

31 Definition of connectivity Define which pixels that are adjecent (or connected) to a pixel. This could be done in several different ways, e.g. 4-connectivity 8-connectivity

32 Definition of paths A 4-connected path from one pixel p to another pixel s is defined as a sequence of pixels {p,q,.,r,s} such that q is a 4-neighbor to p etc. A 8-connected path is defined in the same way

33 Connected regions A 4-connected region is defined as a collection of pixels where there is a 4-connected path between each pair of pixels A 8-connected region is defined similarly

34 Example The dark area is 8-connected, but not 4-connected!

35 Mathematical morphology

36 Dilation

37 Erosion

38 Opening Definition: The opening av A with B is defined as Opening = first erosion then dilation Properties: Gives smoother contours Removes narrow structures (splits up objects) Eliminate thin parts

39 Closing Definition: The closing av A with B is defined as Closing = first dilation then erosion Properties: Gives smoother contours Fills up small parts Fills up holes

40 Combine Thresholding with Mathematical morphology Thresholding may produce irregular contours, small holes inside the object and small parts of the object outside the desired object. It may also produce an over-segmented object Use opening to remove spurious objects and smooth the contour Use closing to fill holes and smooth the contour Closing can also be used to get rid of oversegmentation

41 Watershed segmentation Based on mathematical morphology Gives closed contours Independent of shape and size Efficient and exact Analogy to where water flows in a landscape

42 Analogy with water flowing The gradient of the image is considered as a 3Dlandscape Each object starts from a local minimum in the gradient image and this is where water runs out. It starts to rain at the landscape and each object is defined as the set of pixels where a falling raindrop ends up in the corresponding local minimum.

43 Algorithm for watershed Fill water from below in the gradient image When two basins connects, build a (infinitely high) wall in between. These walls define the contours between the objects.

44 Watershed segmentation

45 Watershed segmentation (cont)

46 Building walls Walls are built when two catchment bases connects. This is done to identify pixels on the boundary between two different objects. This can effectively be computed using morphological operators

47 Practical aspects Low-pass-filter the image Find significant local minima Compute gradient image Apply Watershed Improve the result (e.g. by morphology)

48 Example I

49 Example II Oversegmentation due to too many local minima!

50 Internal and external markers Use internal markers to collect several local minima to a bigger collection of pixels that defines an object. Use external markers to define the background.

51 Distance transformation Introduce a metric (distance measure) in the image: d(p,q), p=(x,y), q=(s,t) The metric should fulfil: d(p,q) >0, p q; d(p,q)=0, p=q d(p,q)=d(q,p) (reflexivity) d(p,r) <= d(p,q)+d(q,r) (the triangle inequality)

52 Different metrics Euclidean distance Manhattan Chessboard The metric could also depend on pixel intensities!

53 More metrics Chamfer: approximation of Euclidean distance Cf real Euclidean:

54 Illustration Manhattan Chessboard

55 Definitions Given a region in an image The distance for a path between two points is defined as the sum of the distances between consecutive pixels. The distance between two points is defined as the distance for the shortest path between them. The distance between a region and a pixel is defined as the shortest distance between the pixel and any pixel in the region. The distance transform is defined as the distance from each pixel in the image to a pre-defined set.

56 Example Distance to the background: Chessboard

57 Example Binary image Distance transform

58 Usage of distance transform Important tool for image segmentation Input to snakes (next lecture) Could be used to extract skeletons Compact representation of objects

59 Shortest path segmentation Define the length between two neighbouring pixels (usually short if the intensities are similar and longer otherwise) Define a starting point and an end point Find the shortest path between the point This problem can be solved using Djikstras

60 Septum segmentation in ultrasound images

61 Conclusion Toolbox of different segmentation algorithms Each algorithm has to be adapted to the specific problem Usually critical parameters have to be selected or estimated Usually several different methods have to be tested More segmentation algorithms: Active countours, snakes, fast marching, level-set methods and variational methods Segmentation using shape models

Bioimage Informatics

Bioimage Informatics Lecture 14, Spring 2012 Bioimage Data Analysis (IV) Image Segmentation (part 3) Lecture 14 March 07, 2012 1 Outline Review: intensity thresholding based image segmentation Morphological