Digital Image Analysis and Processing CPE 0907544 Image Segmentation Part II Chapter 10 Sections : 10.3 10.4 Dr. Iyad Jafar
Outline Introduction Thresholdingh Fundamentals Basic Global Thresholding Optimal Thresholding Using Otsu s Method Variable Local Thresholding Region-based Segmentation Region growing Region Splitting and Merging
Introduction 3 Segmentation can be achieved by Dt Detectingti discontinuities iti (d (edge-based d segmentation) tti Assumes that the boundaries between the objects themselves and the background are sufficiently different Detecting similarities Assumes that the objects to be segmented are similar according to some criteria Examples Thresholding Region based segmentation Wt Watersheds hd
Thresholding Thresholding is based on partitioning the image into regions based on intensity values and/or properties p of these values It is simple and computationally cheap In its very simple form, thresholding attempts to find an intensity value that separates the objects of interest from remaining of the image 4 Usually this is performed by investigatingi i the image histogram which may contain modes that represent the object(s) and the background
Thresholding Background Background Objects Object Object 1 Histogram of an image with light objects and dark background. Segmenting g the objects can be achieved by specifying T such that More challenging situation where we need to specify two thresholds in order to segment the object from background (Multiple Thresholding) 5 g( x,y ) 1, f(x,y) > T 0, f(x,y) T a, f(x,y) > T g( x,y ) b, T f(x,y) T c, f(x,y) T 1 1
Thresholding 6 Thresholding Approaches Global specify one or more fixed threshold values based on histogram shape Local or regional thresholding threshold values T at any point (x,y) is dependent on the properties of neighborhood around (x,y) Dynamic or adaptive threshold value depends on neighborhood properties and spatial coordinates as well The success of intensity thresholding depends on the separation between the peaks in the histogram The noise content in the image (the histogram modes broaden as noise increases) The relative size of objects and background The uniformity of illumination source The uniformity of the reflectance properties of the image
Thresholding The Role of Noise in Thresholding Noise-free image Segmentation is trivial Image corrupted with Gaussian noise of zero mean and standard deviation of 10 Segmentation is can still be performed Image corrupted with Gaussian noise of zero mean and standard deviation of 50 Segmentation is impossibleibl 7
Thresholding The Role of Illumination in Thresholding Nonuniform illumination function that is multiplied by the image 8 Image corrupted with Gaussian noise of zero mean and standard deviation of 10 Segmentation can still be performed Result of nonuniform illumination Simple segmentation is impossible * Similar result would be obtained with uniform illumination and nonuniform reflectance
Basic Global Thresholding 9 When the intensity distribution of objects and background is sufficiently enough, it is possible to use a single global threshold value; a very common situation in practice To find such global threshold automatically 1. Select an initial estimate for the global threshold value,t. Segment the image using T to produce two groups of pixels; G1 containing pixels with intensity values > T and G with intensity values T 3. Compute the mean intensity value for each group, m 1 and m, and define the new threshold by T=(m 1 +m )/ 4. Repeat steps and 3 until the change intissmallerthan a specified value T (and/or N iterations were performed)
Basic Global Thresholding Example 10.9. Original Image Histogram Segmentation Result Initial threshold = mean intensity of image Optimal threshold = 15.4 Three iterations 10
Optimal Global Thresholding Views the thresholding problem as a statistical-decision problem with the objective minimizing the average error of miss-classifying pixels to two or more groups The method requires the knowledge of The probability density function of the intensity levels of each class 11 The probability that each class occurs in a given applications Such requirements are usually hard to obtain in practice! Alternatively, l we consider optimal global lthresholding h using Otsu s method The method is optimal in the sensethat itmaximizes i the betweenclass variance It is based on computations performed on the histogram of an image; an easily obtainable 1-D array
Otsu s Optimal Global Thresholding 1 Let {0,1,,..L-1} denote L distinct levels in MxN image. with normalized histogram whose components are p i = n i /MN Supposewethresholdthe image with a threshold T(k) = k, 0<k<L-1, to produce two classes C 1 and C, where C 1 is the set of pixels with intensity levels in [0,k] and C is the set of pixels with ihintensityi levelsl in [k+1,l-1] 1] Based on this threshold, the probability that a pixel belongs to C 1 (the probability of class C 1 occurring) is and for C is k P(k) 1 p i i 0 L 1 i 1 1 i k 1 P(k) p P(k)
Otsu s Optimal Global Thresholding 13 The mean intensity value m 1 of pixels assigned to C 1 is k k k 1 m 1( k ) i P( i / C 1) i P( C 1 / i )P( i ) / P(C 1) = i p P(k) i 0 i 0 1 i 0 Similarly, the mean intensity m value of pixels assigned to C is L 1 L 1 m(k) i P(i/C ) = i p i P(k) i k 1 i k 1 The average intensity of the entire image is given by L 1 m i p i 0 The cumulative mean up to level k is We can easily verify G Pm P m m 1 1 G 1 P 1 P i 1 k m( k ) i p i 0 i i
Otsu s Optimal Global Thresholding In order to evaluate the goodness of the threshold at level k we use the metric (separability measure) η(k) where σ G is the variance of the entire image and σ B is the between-class variance which is defined as B σg σ (k) B 1 1 G G σ (k) P(k)(m(k) m ) P(k)(m (k) m ) = P(k)P(k)(m(k) m (k)) = 1 1 ( m P( k ) m( k )) G 1 P(k)( 1 P(k)) 1 1 14 For optimal segmentation, we need to find the intensity level k that maximizes η as it implies larger separability between classes * η(k ) 0 k L 1 B max σ (k)
15 Otsu s Optimal Global Thresholding Notes As the difference between m 1 and m increases, σ B increases to indicate better seperability The definition of the metric assumes that σ G > 0. The variance of an image could be zero if it contains one intensity level only In order to find the optimal threshold k*, we have to compute for all integer values of k and then pick the value that maximizes the between-class variance Once the optimal threshold is found, we simply threshold the image into two classes using g( x,y ) η σ B 1, f(x,y) > k* 0, f(x,y) k* σ B
Otsu s Optimal Global Thresholding Summary of Otsu s Algorithm 1) Compute the normalized histogram of the input image and denote its components by p i, i =1 1,,,L-1 ) Compute the cumulative sum P i (k), for k = 0,1,,,L-1 using k P(k) 1 p i i 0 3) Compute the cumulative means, m(k), for k = 0,1,,,L-1 using 16 m( k ) i p 4) Compute the global intensity mean m G using 5) G k i 0 L 1 m i p 6) Compute the between-class variance σ B(k) for k = 0,1,,,L-1 using 7) Pick Otsu s s threshold k* that gives the maximum value for σ B (k) 8) Obtain the separability measure at k* using σ B(k*) i 0 G 1 B P 1(k)( 1 1 σ (k) i i (m P(k) m(k)) P(k)) η(k*) σ G
Otsu s Optimal Global Thresholding Example 10.10. Segmentation of the molecules from background Polymersome cells image Image Histogram 17 Segmentation using basic global thresholding T* = 169 Segmentation using Otsu ss global thresholding k* = 181
Otsu s Optimal Global Thresholding Example 10.11. 18
Enhancing Global Thresholding by Smoothing It was noted earlier that the presence of noise highly affects the result of thresholding One approach to improve thresholding result is to smooth the original image Segmentation Fails Noisy image, its histogram, and Otsu s segmentation result 19 Noisy image after smoothing by 5x5mask, its histogram, and Otsu s segmentation result
Effect of Relative Object Size on Thresholdingh As the size of the object decreases, its contribution to the image histogram decrease and it becomes harder to segment out Segmentation Fails Noisy image, its histogram, and Otsu s segmentation result Segmentation Fails 0 Noisy image after smoothing by 5x5mask, its histogram, and Otsu s segmentation result
The Use of Edges to Improve Thresholdingh Global thresholding can be improved by incorporating edge information, especially when segmenting unimodal histograms 1 Compute the gradient magnitude image and threshold it by T Noisy image with small object and its histogram 3 Thresholded Gradient Image T = 99.7% (binary image) 1 Segmentation result using Otsu s threshold computed from 3 (k* = 134) Histogram of 3 Logical AND between 1 and
Variable Thresholding Variable Thresholdingh Based on Image Partitioning i Subdivide the image into nonoverlapping rectangles to compensate in non-uniformities in illumination and/or reflectance. Threshold each subimage separately The method works well when the objects and the background occupy regions of reasonably comparable size Noisy image with nonuniform illumination Histogram Otsu s segmentation result Image partitioning Histogram of each subimage Segmentation result after applying Otsu s method to each subimage
Variable Thresholding Variable Thresholding Based on Local Image Properties Specify different threshold value at each pixel location (x,y) based on some statistical properties of the pixel s neighborhood Some useful statistical measures for a neighborhood S xy are the mean m xy and standard deviation σ xy Some common forms for specifying the variable threshold are T xy =aσ xy OR +bm xy T xy =aσ xy +bm G 3 We can also use predicate functions based on the neighborhood parameters 1, f(x,y) > aσ xy and f(x,y) > m xy g(x,y ) 0, otherwise
Variable Thresholding Example 10.1. 1 Variable Thresholdingh Based on Local Image Properties Original Image Otsu Results 4 Local Standard Deviation Image (3x3 neighborhood) g( x,y ) Variable Thresholding using g(x,y) 1, f(x,y) > 30σ and f(x,y) > 15. m 0, otherwise xy G
5 Segmentation by Region Growing The approach is based on grouping pixels or subregions into larger regions based on a predefined criteria for growth The basic approach is to start from a set of seed pixels and from these grow regions by appending to each seed those neighboring pixels that meet the predefined properties How to select the seeds? Nature of problem Random Interactively How to select the similarity properties? Nature of problem and image type (color, monochrome..) Use statistical measures of local neighborhoods May incorporate pixel location
Segmentation by Region Growing 6 A General Region Growing Algorithm 1. Define the similarity criteria (difference intensity, variance, ). Define the stopping criteria (properties of the region, size, shape, ) 3. Select a set of seed pixels; S 4. Define an empty array O(x,y) and initialize its elements to 0 s except at seed locations. Seed locations are assigned different intensity values 5. For each seed in S, check its neighbors (4-,8-, or d-neighbors) against the similarity criteria 6. If any of the neighboring pixels satisfy the criteria, then set the corresponding location in O to the same intensity level as its seed 7. Check if the stopping criteria is not met. If not, repeat steps 4 through 7 to the newly added pixels 8. Repeat steps 4 through 7 until all seed pixels are processed
Segmentation by Region Growing Example 10.13. Segment the cracks in the weld Original Image and its histogram Thresholding result to identify regions of high intensity 7 Result of region growing g( x,y ) 1, f(x,y)-n(x,y) > 16 0, otherwise Seeds specified by random selection from those in the thresholded image
8 Segmentation by Region Splitting and Merging Unlike region growing, region splitting and merging approaches the segmentation problem in a top-down approach In this method, the image region is split into a set of disjoint regions and then merge and/or split the regions to satisfy certain criteria i and the segmentation conditions i we discussedd earlier Let R represents the entire image region and Q be a given predicate function: Region splitting attempts to subdivide the image successively into smaller and smaller quadrant regions, R i,suchthat Q(R i )=TRUE. We start from the entire image R. If Q(R) =FALSE, subdivided it into 4 quadrants. Repeat the process for every quadrant and divide it into 4 quadrants if Q(R i )= FALSE Once no splitting is possible (according to some criteria), check adjacent quadrants. If the Q(R j U R k ) = TRUE, then merge these two regions
Segmentation by Region Splitting and dmerging Splitting into quadrants Quadtree 9 Algorithm 1. Split into four disjoint quadrants any region Ri for which Q(Ri) = FLASE. When no further splitting is possible based on some criteria (size of quadrant), merge any adjacent regions for which Q(R j U R k ) = TRUE 3. Stop when no further merging is possible
Segmentation by Region Splitting and dmerging Example 10.14 Segment the less dense matter which is of noisy nature (high standard deviation when compared to background and the dense Original Minimum quadrant size 3x3 region) and moderate intensity Use the predicate function g( x,y ) 1, σ > 10 and 0<m<16 0, otherwise Minimum quadrant size 16x16 Minimum quadrant size 8x8 30
Related Matlab Functions imbw kmeans graythresh (implements Otsu s method) 31