Morphological Image Processing

Morphological Image Processing Binary dilation and erosion" Set-theoretic interpretation" Opening, closing, morphological edge detectors" Hit-miss filter" Morphological filters for gray-level images" Cascading dilations and erosions" Rank filters, median filters, majority filters" Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 1

Binary images are common" Binary image processing l Intermediate abstraction in a gray-scale/color image analysis system" Thresholding/segmentation" Presence/absence of some image property" l Text and line graphics, document image processing" Representation of individual pixels as 0 or 1, convention:" l foreground, object = 1 (white)" l background = 0 (black)" Processing by logical functions is fast and simple" Shift-invariant logical operations on binary images: morphological image processing! Morphological image processing has been generalized to gray-level images via level sets" Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 2

Shift-invariance Assume that digital images f [x,y] and g[x,y] have infinite support" " "" [ x, y] {, 2, 1,0,1,2, } {, 2, 1,0,1,2, }... then, for all integers a and b [ ] g[ x, y] f x, y Shift-" invariant" system" " [ ] g[ x a, y b] f x a, y b Shift-" invariant" system" Shift-invariance does not imply linearity (or vice versa)." Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 3

Structuring element Neighborhood window operator" { } = f x W f x, y { } x, y y : x, y Π xy " Π xy Example structuring elements :" structuring element " " 5x5 square" x x cross" y y Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 4

Binary dilation { } g x, y = OR W f x, y := dilate( f,w ) x f [x, y] Π y xy g[x, y] Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 5

Binary dilation with square structuring element { } g x, y = OR W f x, y := dilate( f,w ) Expands the size of 1-valued objects" Smoothes object boundaries" Closes holes and gaps" Original (701x781)" dilation with " 3x3 structuring element" dilation with " 7x7 structuring element" Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 6

Binary erosion { } g x, y = AND W f x, y := erode( f,w ) x f [x, y] Π y xy g[x, y] Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 7

Binary erosion with square structuring element { } g x, y = AND W f x, y := erode( f,w ) Original (701x781)" erosion with " 3x3 structuring element" erosion with " 7x7 structuring element" Shrinks the size of 1-valued objects" Smoothes object boundaries" Removes peninsulas, fingers, and small objects" Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 8

Relationship between dilation and erosion Duality: erosion is dilation of the background" ( ) = NOT erode NOT f dilate f,w erode f,w (,W ) (,W ) ( ) = NOT dilate NOT f But: erosion is not the inverse of dilation" f ( ) ( ) x, y erode dilate( f,w ),W dilate erode( f,w ),W Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 9

Example: blob separation/detection by erosion Original binary image" Circles (792x892)" Erosion by 30x30 " structuring element! Erosion by 70x70 " structuring element! Erosion by 96x96 " structuring element! Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 10

Example: blob separation/detection by erosion Original binary image" Circles (792x892)" Erosion by disk-shaped " structuring element" Diameter=15! Erosion by disk-shaped " structuring element" Diameter=35! Erosion by disk-shaped " structuring element" Diameter=48! Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 11

Example: chain link fence hole detection Original grayscale image Fence (1023 x 1173)! Fence thresholded using Otsu s method! Erosion with 151x151 cross structuring element! Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 12

Set-theoretic interpretation " Set of object pixels" F {( x, y): f ( x, y) = 1} Background: complement of foreground set" F c {( x, y): f ( x, y) = 0} Dilation is Minkowski set addition" G = F Π xy { ( ) F, ( p x, p ) y Π } xy = ( x + p x, y + p y ): x, y ( p x,p y ) Π xy = F + px,p y ( ) Continuous (x,y).! Works for discrete [x,y] in the same way.! Commutative and associative!! translation of F ( by vector "p x, p ) y Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 14

Set-theoretic interpretation: dilation G F y x Π xy G = F Π xy { ( ) F, ( p x, p ) y Π } xy = ( x + p x, y + p y ): x, y = F + px,p y ( p x,p y ) Π xy ( ) Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 15

Set-theoretic interpretation: erosion G G = F Not commutative!! Not associative!! Minkowski set subtraction" ( p x,p y ) Π xy F + ( px,p y ) = FΘΠ xy Reversed structuring" element" y x Π xy Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 16

Goal: smoothing without size change" Open filter " Close filter" open f,w Opening and closing ( ( ),W ) ( ),W ( ) = dilate erode f,w ( ) = erode dilate f,w close f,w ( ) Open filter and close filter are biased" l Open filter removes small 1-regions" l Close filter removes small 0-regions" l Bias is often desired for enhancement or detection!" Unbiased size-preserving smoothers" close open f,w open close f,w ( ( ),W ) ( ( ),W ) ( ) = close open f,w ( ) = open close f,w close-open and open-close are duals, but not inverses of each other. " Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 17

Small hole removal by closing Original binary mask" Dilation 10x10" Closing 10x10" Difference to original mask" Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 18

Morphological edge detectors f x, y dilate( f,w) f erode( f,w) f dilate( f,w) erode( f,w) Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 20

Binary image f Recognition by erosion ( ) open( NOT f,w ) = dilate erode( NOT f,w ),W 1400" 2000" Structuring element W 44" 34" Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 21

Recognition by erosion Structuring element W 44" 34" Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 22

Binary image f Recognition by erosion ( ) open( NOT f,w ) = dilate erode( NOT f,w ),W 1400" 2000" Structuring element W 62" 18" Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 23

Recognition by erosion Structuring element W 62" 18" Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 24

Binary image f Hit-miss filter ( ( ) & erode( f,w ),W ) dilate erode NOT f,v 1400" 2000" Structuring element V 62" 62" Structuring element W 18" 18" Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 25

Hit-miss filter Structuring element V 62" 18" Structuring element W 62" 18" Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 26

Morphological filters for gray-level images Threshold sets of a gray-level image f [x,y] ( ) = x, y Τ θ f x, y { : f x, y θ}, < θ < + Reconstruction of original image from threshold sets" f x, y { ( )} = sup θ : x, y Τ θ f x, y Idea of morphological operators for multi-level (or continuous-amplitude) signals" l Decompose into threshold sets" l Apply binary morphological operator to each threshold set" l Reconstruct via supremum operation" l Gray-level operators thus obtained: flat operators!! Flat morphological operators and thresholding are commutative! Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 27

Dilation/erosion for gray-level images " Explicit decomposition into threshold sets not required in practice" Flat dilation operator: local maximum over window W g x, y = max{ W { f x, y }}:= dilate f,w Flat erosion operator: local minimum over window W ( ) g x, y = min{ W { f x, y }}:= erode f,w ( ) Binary dilation/erosion operators contained as special case" Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 28

1-d illustration of erosion and dilation Structuring element length =" Dilation" Erosion" Original Structuring element: horizontal line Amplitude Sample no. Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 29

Image example Original Dilation Erosion Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 30

Flat dilation with different structuring elements Original Diamond Disk 20 degree line 9 points 2 horizontal lines Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 31

Example: counting coins thresholded 1 connected component Original 20 connected components dilation thresholded after dilation Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 32

Example: chain link fence hole detection Original grayscale image Fence (1023 x 1173)! Flat erosion with 151x151" cross structuring element! Binarized by Thresholding! Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 33

Morphological edge detector original f dilation g g f g-f g f thresholded Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 35

Beyond flat morphological operators " General dilation operator" g x, y = sup α,β Like linear convolution, with sup replacing summation, addition replacing multiplication" Dilation with unit impulse " d α,β does not change input signal:" f x, y = sup α,β { f x α, y β + w α,β } = sup = 0 α = β = 0 else α,β { f x α, y β + d α,β } { } w x α, y β + f α,β Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 36

Find such that" w α,β Answer:" f x, y Flat dilation as a special case = sup α,β { f x α, y β + w α,β } = dilate f,w w α,β = 0 α,β Π xy else ( ) Hence, write in general" g!" x, y# $ = sup{ f!" x α, y β # $ + w!" α,β # $ } α,β = dilate f,w ( ) = dilate( w, f ) Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 37

General erosion for gray-level images General erosion operator" g x, y = inf α,β { f x α, y β w α,β } = erode f,w Flat erosion contained as a special case" ( ) Dual of dilation" g x, y = inf { f x α, y β α,β w α,β } = sup{ f x α, y β α,β + w α,β } = dilate( f,w) Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 38

Cascaded dilations f dilate( f,w 1 ) dilate ( f,w,w 1 2 ) dilate ( f,w,w,w 1 2 3) dilate dilate( f,w 1 ),w 2 = dilate( f,w) where w = dilate( w 1,w ) 2 Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 39

Cascaded erosions Cascaded erosions can be lumped into single erosion" erode erode( f,w 1 ),w 2 = erode dilate ( f,w 1),w 2 = dilate dilate( f,w 1 ),w 2 ( ) ( ) ( ) = dilate f,w = erode f,w where w = dilate w 1,w 2 New structuring element (SE) is not the erosion of one SE by the other, but dilation." Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 40

Fast dilation and erosion Idea: build larger dilation and erosion operators by cascading simple, small operators" Example: binary erosion by 11x11 window" [ x y] f, [ x y] f, Erosion with 1x3 window" " Erosion with 1x3 window" Erosion with 11x11 window" 5 stages! [ x y] g, [ x y] g, Erosion with 3x1 window" " Erosion with 3x1 window" 5 stages! 120 AND per pixel" 2x10 = 20 AND per pixel" Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 41

Rank filters Generalisation of flat dilation/erosion: in lieu of min or max value in window, use the p-th ranked value" Increases robustness against noise" Best-known example: median filter for noise reduction" Concept useful for both gray-level and binary images" All rank filters are commutative with thresholding" Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 43

Median filter Gray-level median filter" Binary images: majority filter" Self-duality" g x, y = median W f x, y { } g x, y = MAJ W f x, y median f,w majority f,w ( ) = median( f,w ) { } ( ) = NOT majority NOT f := median( f,w ) := majority( f,w ) (,W ) " Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 44

Majority filter: example Binary image with 5% Salt&Pepper noise" 3x3 majority filter" 20% Salt&Pepper noise" 3x3 majority filter" Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 45

Median filter: example Original image" 5% Salt&Pepper noise" 3x3 median filtering" 7x7 median filtering" Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 46

Example: non-uniform lighting compensation Original image 1632x1216 pixels" Dilation (local max)" Rank filter 61x61 structuring element" 10st brightest pixel" 61x61 structuring element" Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 47

Example: non-uniform lighting compensation Background original image" After global thresholding" Digital Image Processing: Bernd Girod, 2013 Stanford University -- Morphological Image Processing 48