Binary Image Processing Tuesday, 14/02/2017 ntonis rgyros e-mail: argyros@csd.uoc.gr 1 Today From gray level to binary images Processing of binary images Mathematical morphology 2 Computer Vision, Spring 2017 1
Thresholding thresholded image T: threshold 1 if I( x, y) > T O( x, y) = 0 if I( x, y) T (objects) (background) Threshold selection: User-defined Αutomatic (e.g., Ηistogram partitioning) 3 Simple Global Thresholding To partition the image histogram by using a single threshold T. Then the image is scanned and labels are assigned. This technique is successful in highly controlled environments. Computer Vision, Spring 2017 2
Sometimes works fine Sometimes works fine When? The population of foreground and background pixels are well separated in the image histogram 6 Computer Vision, Spring 2017 3
...some times it doesn t When? global threshold does not separate well foreground from background 7 Idea to solve the problem: adaptive thresholding Key idea: The global histogram may not have distinctive peaks, but subimages might have. Split the image into subregions Very small subimages: Do not cover both foreground and background pixels Very large subimages: Violate the assumption of approximately uniform illumination Find a local threshold by statistically examining the intensity values of the local neighborhood of each pixel. The statistic which is most appropriate depends largely on the input image. Simple and fast functions include the mean of the local intensity distribution, the median value, or the mean of the minimum and maximum values 8 Computer Vision, Spring 2017 4
Computer Vision, Spring 2017 5
Thresholding: input Interactive demo: http://homepages.inf.ed.ac.uk/rbf/hipr2/adapthreshdemo.htm 11 Global thresholding 12 Computer Vision, Spring 2017 6
daptive thresholding, mean, 7x7 regions 13 daptive thresholding, mean, 17x17 regions 14 Computer Vision, Spring 2017 7
daptive thresholding, mean minus C", 7x7, C=7 15 daptive thresholding, median-c, 7x7 regions, C=4 16 Computer Vision, Spring 2017 8
Optimal Thresholding The histogram of an image containing two principal brightness regions can be considered an estimate of the brightness probability density function p(z): the sum (or mixture) of two unimodal densities (one for light, one for dark regions). Optimal Thresholding The mixture parameters are proportional to the areas of the picture of each brightness. If the form of the densities is known or assumed, determining an optimal threshold (in terms of minimum error) for segmenting the image is possible. Computer Vision, Spring 2017 9
Finding peak values in a histogram Not a trivial problem 20 Computer Vision, Spring 2017 10
Finding peak values: algorithm Find LOCL MXIM that have a MINIMUM DISTNCE PRT (g i and g j ) Find lowest point between them: (g k ) Measure peakiness : min(h(g i ),H(g j ))/H(g k ) Find (g i,g j,g k ) with highest peakiness 21 Other automatic thresholding methods 1. Otsu s method 2. K-means clustering Computer Vision, Spring 2017 11
Otsu s method Best choices for t. utomatic thresholding method automatically picks best threshold t given an image histogram ssumes 2 groups are present in the image: 1. Those that are <= t. 2. Those that are > t. Otsu s method For every possible t:. Calculate within group variances: 1. probability of being in group 1; probability of being in group 2 2. determine mean of group 1; determine mean of group 2 3. calculate variance for group 1; calculate variance for group 2 4. calculate weighted sum of group variances B. Remember which t gave rise to minimum. Computer Vision, Spring 2017 12
Otsu s method: probability of being in each group 1 t ( ) = ( ) q t p i i= 0 2 max ( ) = ( ) q t p i i= t+ 1 Otsu s method: mean of individual groups µ t ( t) = i p( i) / q ( t) 1 1 i= 0 µ max ( t) = i p( i) / q ( t) 2 2 i= t+ 1 Computer Vision, Spring 2017 13
Otsu s method: variance of individual groups t 2 1 1 1 i= 0 2 ( t) = i ( t) p( i) / q ( t) σ µ max 2 2 2 2 i= t+ 1 2 ( t) = i ( t) p( i) / q ( t) σ µ Otsu s method:weighted sum of group variances ( ) = ( ) ( ) + ( ) ( ) σ σ σ 2 t q t 2 t q t 2 t W 1 1 2 2 Calculate for all t s and minimize. { σ 2 ( t W ) t } min 0 max Computer Vision, Spring 2017 14
Otsu s method Iterative K-Means Clustering for thresholding Form 2 clusters from a set of pixel gray values. 1. Set ic=1 (iteration count). 2. Choose 2 random gray values as our initial K means, m 1 (1), and m 2 (1). 3. For each pixel gray value x i compute fabs(x i,m j (ic)) for each j=1,2. 4. ssign x i to the cluster C j with the nearest mean. 5. ic =ic+1; update the means to get a new set m 1 (ic), m 2 (ic). 6. Repeat 3..5 until Cj(ic+1) = Cj(ic) for all j. Computer Vision, Spring 2017 15
Iterative K-Means Clustering for thresholding Example. m1(1)=260.83, m2(1)=539.00 m1(2)=39.37, m2(2)=1045.65 m1(3)=52.29, m2(3)=1098.63 m1(4)=54.71, m2(4)=1106.28 m1(5)=55.04, m2(5)=1107.24 m1(6)=55.10, m2(6)=1107.44 m1(7)=55.10, m2(7)=1107.44... K Means method Computer Vision, Spring 2017 16
Otsu vs. K-Means What does Otsu s method determine? a single threshold, t What does K-Means determine (for K=2)? m1 and m2 are cluster means (centers) Once m1 and m2 are determined, how can they be used to determine the threshold? Otsu vs. K-Means final word: K-Means readily generalizes to: 1. arbitrary number of classes (K) 2. can easily be extended to many, many features (i.e., feature vectors instead of only gray values/higher dimensions) K-Means will find a local optimum, but that may not be the global optimum! Computer Vision, Spring 2017 17
Processing of binary (=two valued) images Images with only two values: 0 (usually background) and 1 (usually foreground) says HI 35 Morphological Filtering Main idea Examine the geometrical structure of an image by matching it with small patterns called structuring elements at various locations By varying the size and shape of the matching patterns, we can extract useful information about the shape of the different parts of the image and their interrelations. 36 Computer Vision, Spring 2017 18
Example: thresholded image of erythrocytes Several objects Some touching each other Some with noisy boundaries Can we improve this image? 37 Improving the thresholded image Removal of contour pixels for better object separation Filling of small gaps Removal of small objects and extrusions 38 Computer Vision, Spring 2017 19
Morphological filtering Noisy image will break down OCR systems Clean image Noisy image 39 Morphological filtering By applying MF, we increase the OCR accuracy! Restored image 40 Computer Vision, Spring 2017 20
Mathematical morphology n approach for processing a digital image based on shape mathematical tool for investigating geometric structure in image The language of morphology is set theory 41 Mathematical morphology Two basic operations (dual to each other) Dilation: enlarges foreground, shrinks background Erosion: shrinks foreground, enlarges background and several composite closing opening... 42 Computer Vision, Spring 2017 21
Shape Operators Shapes are usually combined by means of : Set Union X 1 X 2 X X 1 2 Set Difference X 1 X 2 c 2 1 1 2 X \ X = X X 43 Dilation Let and B are subsets in 2D space. : image undergoing analysis B: Structuring element denotes dilation The translation of by x is defined as ( ) x = { c Z 2 c= a+ x, for some a } The dilation of by B can be computed as the union of translation of by the elements of B (or vice versa) B = ( ) = ( B) b B b a a 44 Computer Vision, Spring 2017 22
Dilation, example B B 45 Dilation (0,0) (0,1 ) B B 46 Computer Vision, Spring 2017 23
Dilation B = ( ) = ( B) b B b a a (B) x B 47 Example of Dilation Structuring Element Pablo Picasso, Pass with the Cape, 1960 48 Computer Vision, Spring 2017 24
Computer Vision, Spring 2017 25 Properties of Dilation Commutative ssociative Dilation of a subset D B D B implies B B = C B C B = ) ( ) ( 49 Properties of Dilation Translation Invariance Linearity Containment Decomposition of structuring element x x B B ) ( ) ( = ) ( ) ( ) ( C B C C B = ) ( ) ( ) ( C B C C B ) ( ) ( ) ( C B C B = 50
Erosion Erosion is the morphological dual to dilation Let ΘB denote the erosion of by B. Then: Θ B = { x Z 2 ( B) x ) 51 2D example of erosion B ΘB 52 Computer Vision, Spring 2017 26
Erosion We defined erosion in terms of translation Θ B = { x Z 2 ( B) x ) but can also be defined in terms of intersection Θ B = ( ) b B b 53 Erosion Θ B = { x Z 2 ( B) x ) (B) x ΘB 54 Computer Vision, Spring 2017 27
Example of Erosion Structuring Element 55 Erosion Original image Eroded image 56 Computer Vision, Spring 2017 28
Erosion Eroded once Eroded twice 57 Counting Coins Counting coins is difficult because they touch each other! Solution: Thresholding and Erosion separates them! 22-Feb-17 58 Computer Vision, Spring 2017 29
Duality Relationship 2 B Z, the reflection of B, B is defined as B = { x x = b, b B } Erosion and Dilation Duality Theorem ( Θ B ) c = c B 59 Duality Relationship Dilation and Erosion: what one does to image foreground, the other does to the image background. 60 Computer Vision, Spring 2017 30
Duality Erosion of the foreground (white) results in dilation of the background (black) 61 Opening and Closing Opening and closing are operations composed of dilation and erosion Opening B = ( ΘB) B Closing B = ( B) ΘB 62 Computer Vision, Spring 2017 31
Example 63 Opening Example 1 Opening with a 11 pixel diameter disc 64 Computer Vision, Spring 2017 32
Opening Example 2 3x9 and 9x3 Structuring Element 3*9 9*3 65 Opening Example 3 Use large structuring element that fits into the big blobs Structuring Element: 11 pixel disc 66 Computer Vision, Spring 2017 33
Closing Example 1 Closing operation with a 22 pixel disc Closes small holes in the foreground 67 Closing Example 2 1. Threshold 2. Closing with disc of size 20 Thresholded closed 68 Computer Vision, Spring 2017 34
Opening and Closing Opening: Closing: 69 nother example OPENING: Removal of scratches CLOSING: Most of the intrusions have been covered 70 Computer Vision, Spring 2017 35
Opening and Closing Their re-application has not further effects to the previously transformed result B = ( B) B B = ( B) B 71 Opening and Closing Translation invariance ( B) x = B ( B) x = B B B Duality ( B) c = c B 72 Computer Vision, Spring 2017 36
Contour extraction 73 Contour extraction 74 Computer Vision, Spring 2017 37
Region filling 75 Connected pixels 4-connectivity 8-connectivity 4-connectivity : Pixel connected with itself and its N, W, E, S neighbors 8-connectivity: Pixel connected with itself and its N, W, S, E, NW, NE, SE, SW neighbors 76 Computer Vision, Spring 2017 38
Connected components labeling Input: binary image Output: Given the relation connectivity between two pixels, compute the transitive closure of this relation. The transitive closure of the connectivity relation is an equivalence relation (why?) and we are interested in the partition that induces to the set of foreground image pixels. 77 Connected components labeling 78 Computer Vision, Spring 2017 39
Connected components labeling 79 Blobs ( = connected components) Compute properties: rea (= #pixels) Bounding box (= minimum enclosing rectangle) spect ratio (= width/height of the bounding box) Orientation Circularity Perimeter (= # contour pixels) Various shape statistics first, primitive way of representing image content Compression of information 80 Computer Vision, Spring 2017 40
Binary image Binary image I(x,y) I(x,y) = 0 (background) I(x,y) = 1 (foreground) 81 Moments Moments Central moments (translation invariant) Normalized central moments (translation + scale invariant) Hu moments (translation, scale and rotation invariant) 82 Computer Vision, Spring 2017 41
Moments Moments M ij = x y i j x y I( x, y) rea: zero order moment = M 00 = I( x, y) x y 83 Centroid Defined as the point: ( x, y) M M, M10 xi ( x, y) x y 10 01 = M 00 M 00 = M 01 = yi ( x, y) x y Centroid 84 Computer Vision, Spring 2017 42
2 nd order moments M = 2 20 x I( x, y), x y M = 2 02 y I( x, y), x y = M11 xyi ( x, y) x y 85 Central moments i j U = ( x x) ( y y) I( x, y) ij x y ( ) 2 U20 = x x I( x, y), x y U11 = ( x x)( y y) I ( x, y) x y ( ) 2 U02 = y y I( x, y), x y 86 Computer Vision, Spring 2017 43
Computation of central moments U = M 02 02 M 2 01 U = M 20 20 M 2 10 U = M 11 11 M M 10 01 87 Inertia matrix U U U 20 11 = U11 U 02 88 Computer Vision, Spring 2017 44
Orientation 89 Normalized central moments U = ( M ) i j ij η ij + + 2 2 00 90 Computer Vision, Spring 2017 45
Hu moments invariant to translation, scaling and rotation 91 Our intention / first approximation of object recognition = 92 Computer Vision, Spring 2017 46
Circularity 93 Computer Vision, Spring 2017 47