Topic 6 Representation and Description Background Segmentation divides the image into regions Each region should be represented and described in a form suitable for further processing/decision-making Representation of the regions using (i) External Characteristics its boundary (eg. Shape) (ii) Internal Characteristics pixels comprising the region (eg. Color, Texture) Description (i) External Characteristics eg. Shape, length (ii) Internal Characteristics eg. Color, texture, etc. Description shouldn t be sensitive to rotation, translation, size, etc. 1
Representation Schemes Segmentation yield raw data (sometimes segmentation can be useful representation and used directly to obtain descriptors) Chain Code Idea - Approximate the boundary by straight line segments - Represent the boundary by the direction of the line segments - Length and direction must be determined Representation Schemes Chain Code - Length determined by the distance between pixels - Direction quantized into 4 or 8 values 1 3 2 1 2 0 4 0 3 5 7 6 2
Chain Code Representation Schemes 4-directional Chain Code 8-directional Chain Code Representation Schemes Chain Code Normalization (for invariance to starting point ) Treat the chain code as a circular sequence Normalization (for invariance to rotations ) Use the first difference of the chain code by counting (counterclockwise) the number of directions that separate two adjacent elements in the code Example: 10103322 210103322 First Difference: 33133030 3
Representation Schemes Polygonal approximation Idea - Represent the boundary by straight line segments (polygon) - The goal is to capture the essence of the boundary shape using the fewest possible number of segments (i) Minimum Perimeter Polygons Representation Schemes Polygonal approximation (ii) Merging techniques Edge points are added to line segments as the edge list is gone through New line segments are started when the edge points deviate too far from the line segment (least-square error line fit is used to determine this) 4
Representation Schemes Polygonal approximation (ii) Merging techniques Merge points along the boundary using a leastsquare-error-line until the least square error line > Threshold then, a new line is started When the starting point is reached again (merging is completed), the intersections of the adjacent lines makes up the points of the polygon Least-square-error-line: use a straight line, y = a + bx to approximate a given set of data (x 1, y 1 ), (x 2, y 2 ),... (x n, y n ) and the least-square error: n 2 = [ yi ( a + bx i )] i= 1 Source of Image: Stina Svensson, Centre for Image Analysis Representation Schemes Polygonal approximation (ii) Splitting techniques Subdivide a segment into two parts until a given criterion is satisfied (e.g., least-square error line fit is less than a threshold) Boundary divided into segments based on extreme points Resulting Polygon 5
Representation Schemes Signature 1D representation of a boundary Simplest method: plot the distance from the centroid to the boundary as a function of angle Representation Schemes Skeleton of a region Important for structural shape of a plane region By finding Medial Axis Transformation (MAT) MAT of a region R with border B (proposed by Blum): For each point p in R, closest neighbor B is calculated If p has more than one neighbor then, it belongs to the Medial axis (skeleton) of R (Closest calculated using distance eg. Euclidean distance) 6
Representation Schemes Skeleton of a region Distance Transform Representation Schemes Skeleton of a region 7
Boundary Descriptor Length of contour Counting the number of pixel along the contour Diameter - maximum distance between pairs of pixels Curvature Rate of Change of slope computed using the difference between slopes of adjacent boundary segments Boundary Descriptor Shape Number Shape number of the boundary the first difference of the chain-coded boundary, having the smallest magnitude (normalized) first number of shape number = difference element (digit) with the lowest value subsequent number of shape number = following numbers after the lowest value in the difference list The number of digits of the shape number is called the order of the shape number. The order of a shape number limits the number of different shapes it can encode. 8
Boundary Descriptor Shape Number Boundary Descriptor Fourier Descriptor - Represent the boundary as a sequence of point coordinates by traversing the boundary, for example, in a counterclockwise order (x0, y0), (x1, y1),..., (xn-1, yn-1) (for simplicity, x(k) = xk and y(k) = yk ) - a point s - (x,y) is representation s(k) =x(k) + j y(k) - Discrete Fourier transform (DFT) of s(k) N 1 1 j2πuk a( u) = s( k) exp( ) N k = 0 N a (u) are called the fourier descriptors of the boundary - Inverse Discrete Fourier transform (IDFT) 9
Fourier Descriptor Boundary Descriptor Boundary Descriptor Shape of boundary can be quantitatively described by using moments k n n th moment of v about mean is µ n = ( vi m) p( vi ) i= 1 where m is mean m = k i= 1 v p( i v i ) - Second moments measures the spread of the curve about mean value - Third moments measures symmetry about mean - Both used to describe the boundary 10
Regional Descriptors Area of a region Perimeter of a region is length of its boundary Compactness = (perimeter) 2 / area Number of holes in the object Number of connected objects in an image Texture of a region Mean, Max, min gray level, number pixel above / below mean Topological Descriptors Useful for global description of image Number of holes in the image (H) Number of connected objects (C) Euler number E = C-H Euler Number (Letter A) = 0 (1 connected component & 1 hole) Euler Number (Letter B) = -1 (1 connected component & 2 holes) 11
Form, structure Morphology Used in pre post processing Language of morphology is set theory Mathematical morphology a tool for extracting image components that are useful in the representation and description of region shapes like boundaries and skeletons Morphological operations Neighbourhood operations carried out in spatial domain Based on mathematical morphology set theoretical framework originally for binary images extended for grey scale images Applications: extract info about forms and structures shaping and filtering of forms and structures 12
Morphological processing Consists essentially of two steps: 1. Probe a given object in x[m,n] with a structuring element (se) 2. Find how the se fits with the object MP...contd. Information about fit is used to extract info about the form of object OR change pixel values and shape objects Different {size, shape} of se yields different kinds of info about the object shapes the forms in different ways 13
Definitions Given sets a A, and b B and vector x Translation by x : A x = {a+x a A} Reflection: -B = {-b b B} Complement: A c = {d d A} Difference: A-B = {d d A, d B}=A B c Examples Set A A x x = (1,1) -A Origin (0,0) Set B New origin (1,1) A- B 14
Definitions Given sets a A, and b B and vector x Translation by x : A x = {a+x a A} Reflection: -B = {-b b B} Complement: A c = {d d A} Difference: A-B = {d d A, d B}=A B c Reflection of A = -A (180 clockwise direction) Set A Origin (0,0) Source: http://homepages.inf.ed.ac.uk/rbf/cvonline/local_copies/gasteratos/soft/2.htm -A 15
Basic operations in MP Erosion shrinks an object X B = {x B x X} Set of all locations x such that se is within (inside or contain in) X Dilation enlarges an object X B = {x -B x X φ} Set of all locations x such that -se has at least 1 pixel within (inside) X Examples Object A After erosion se = B Eroded pixel After dilation -B Dilated pixel 16
Examples Erosion Structuring Element Dilation Source: http://www.cee.hw.ac.uk/hipr/ht ml/erode.html http://www.cee.hw.ac.uk/hipr/ht ml/dilate.html Basic operations in MP..contd. Opening erosion followed by dilation (X B) B = XoB Smoothes contours, fills in small islands Closing dilation followed by erosion (X B) B = X B Blocks narrow channels and thin holes 17
Opening - Example Image X After erosion After dilation se B origin -B Output Result is on the origin pixel Morphological Image Processing 18
Structuring Element Opening Example All pixels which can be covered by the structuring element with the structuring element being entirely within the foreground region will be preserved. However, all foreground pixels which can not be reached by the structuring element without parts of it moving out of the foreground region will be eroded away. Source: http://www.cee.hw.ac.uk/hipr/html/open.html Closing -example Image X After dilation After erosion se B origin -B 19
Morphological Image Processing Closing Example Structuring Element If the structuring element can be made to touch that point, without any part of the element being inside a foreground region, then that point remains background If this is not possible, then the pixel is set to foreground Source: http://www.cee.hw.ac.uk/hipr/html/close.html 20
Morphological Image Processing Applications Morphological algorithms Boundary extraction:β(x) = X-(X B) Hit-Miss Transform: X * B = (X B1) (X B2) where B = (B1,B2); B1: object part and B2: local background Thinning: X B = X - X * B Thickening: Can be done by thinning the background 21
Hit-and-Miss Transform Hit-and-Miss Transform Template Matching SE to detect corners (run 4 times and OR all the output images to get the final image) If the foreground and background pixels in the structuring element exactly match foreground and background pixels in the image, then the pixel underneath the origin of the structuring element is set to the foreground colour. If it doesn't match, then that pixel is set to the background colour. http://www.cee.hw.ac.uk/hipr/html/hitmiss.html Boundary detection With B 1 β 1 (X) Image X With B 2 β 2 (X) Centre pixels are those retained after erosion of X by B 22
Purpose: Example Basic Morphological Algorithms: Boundary Extraction to extract image components that are useful in the representation and description of shape. Boundary Extraction: ( A) = A ( A B) β (9.5-1) Morphological Image Processing Boundary Extraction 23
Morphological Image Processing Boundary Extraction 2D Dilation Morphological Image Processing: Extending to Gray-level Image 24
Morphological Image Processing: Extending to Gray-level Image 2D Erosion Source: Mathematical Morphology by N.Young Other algos. - Skeletonisation A skeleton is a minimal representation for structures Provides topological and metric information End points, nodes, holes; branch length, angles, etc Several approaches exist 25
Methods for skeletonisation Basic approach conditional erosion remove pixels only if they don t split a region can use a fate table for implementation Removal of centre pixel Splits a region Does not split a region Methods for skeletonisation Using iterative erosion S( X ) = S S k : Skeleton subset Θ k : Erosion ( X ) = U k U p S k ( X ) ( XΘpB) (( XΘpB) ob) Drawback - very sensitive to change in shape 26
Skeletonisation - example B X Light red: X Red: XoB Lilac: S 0 B Methods for skeletonisation Fit a set of max balls inside the object Skeleton = {Centre points of the set of max balls} Max ball is a ball of max size that just fits within X Disadv. Can result in disjoint skeleton 27
Distance Transform A map which gives the distance from an object point to the nearest boundary different distance measure D can be used also called a Euclidean Distance Map Applications feature measurement, binary to grey/colour conversion, skeletons, cluster analysis, etc. EDM examples With D 4 With D 8 Skeleton using (D 4 ) 28
MP on Grey scale images Erosion and dilation use rank operations Simple defn.: replace centre pixel with min or max value in the window represented by the se B Variation: centre pixel = max{x[m-i,n-j] + B[i,j]; i,j D B } for dilation Similar to binary case for erosion, the se should fit completely inside X for dilation, se should overlap X by at least one pixel Applications Grey scale morphological processing can be used for image enhancement, analysis Examples smoothing shadow removal segmentation size distributions of objects 29