Representation REPRESENTATION & DESCRIPTION After image segmentation the resulting collection of regions is usually represented and described in a form suitable for higher level processing. Most important representations are based on shape or texture. Desirable property: descriptors should be insensitive to changes in size, translation or rotation. Actual measurement of features in digital images makes use of many of the techniques discussed earlier, such as linear or morphological image operators. 1 Descriptors Moore boundary tracking Boundary descriptors Put boundary points of a binary region R (or its boundary) in a clockwise-sorted order. Needed for computation of boundary descriptors. 2 3
Chain codes Chain codes Freeman chain codes: strings of integers representing a boundary by a connected sequence of straight-line segments of specified length and direction. The direction of each line segment is coded using a numbering scheme adapted to the connectivity. (a) 8-directional chain code. (b) Digital boundary with resampling grid. (c) Resampled boundary. (d) 8-directional chain-coded boundary. The accuracy of the straight-line representation depends on the spacing of the sampling grid. 4 5 Chain codes: first difference Polygon representation First difference of a chain code: count number of direction changes (e.g., counterclockwise) that separate two adjacent code elements. chain code: 10103322 (start lower left) first difference: 33133030 (circular sequence) A digital boundary can also be approximated by a polygon, possibly with minimum length (MPP: minimum-perimeter polygon). Such rubber band approximations directly within the grey value image are known as active contour models, or snakes. 6 7
Minimum-perimeter polygon (MPP) Signatures (a) Light grey: region containing the boundary curve. (b) Convex (white) and concave (black) corner vertices. (c) Concave vertices moved to diagonal mirror locations. The MPP is indicated. 1-D representation of a boundary Example: distance r(θ) of centroid to boundary as a function of angle θ. 8 Often the boundary is first smoothed 9 Signature Characteristic Profile For each angle, compute difference between radial distance from ellipse center to ellipse and to contour, respectively. (Ciobanu and Du Buf, 2002.) 10 11
Convex and non-convex objects Boundary segments An object (binary image) is convex if for every pair of points p and q within the object, every point on the straight line segment that joins p and q is also within the object. Otherwise, we call it non-convex. p p To reduce the complexity of the boundary, decompose it into segments q q If the boundary encloses a region S, compute the convex hull H of S, and the convex deficiency D = H \ S. convex object non-convex object Mark points on the boundary at which a transition in and out of the convex deficiency occurs. 12 13 Boundary descriptors Curvature Length: no. of pixels along contour, length of MPP. Diameter: Diam(B) = max [D(p i, p j )] with p i, p j points on the i,j boundary B and D a distance measure. Major axis: line segment connecting the extreme points comprising the diameter Minor axis: line segment perpendicular to major axis such that rectangle defined by major and minor axis tightly encloses the boundary Eccentricity: ratio of lengths of major and minor axes Curvature 0.06 0.04 0.02 0 0.02 0.04 0.06 0 0.25 0.5 0.75 1 Position along contour Curvature 0.12 0.1 0.08 0.06 0.04 0.02 0 0.02 0 0.25 0.5 0.75 1 Position along contour 0.06 0.05 0.04 0.03 0.02 0.01 0 0.01 0.02 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Position along contour dashed: Gaussian smoothing, continuous: Adaptive Gaussian smoothing Curvature 14 15
Shape numbers Fourier descriptors Given a chain-coded boundary, its shape number is that particular cyclic permutation of the first difference which is lexicographically smallest among all the cyclic permutations Order n of shape number: no. of digits in its representation Represent boundary as a sequence s(k) = x(k) + i y(k), k = 0, 1,... K 1, where (x(k), y(k)) are coordinates of points on the contour. DFT of the vector (s(0), s(1),..., s(k 1)) yields K complex coefficients: the Fourier descriptors. Approximation: only use P < K of the Fourier coefficients and perform an inverse DFT. 16 17 Fourier descriptors Statistical moments Represent boundary as a function g(r). Let p(v i ), i = 1, 2,..., A 1 be the amplitude histogram of g(r). Mean: m = A 1 i=0 v i p(v i ) (a) Input (2868 points). (b)-(h) Approximations using 1434, 286, 144, 72, 36, 18, 8 Fourier coefficients. n th moment: µ n = A 1 i=0 (v i m) n p(v i ) n = 2, 3,... 18 19
Descriptors Simple regional descriptors Mean, median, maximum, minimum grey level Regional descriptors Area A, perimeter P Compactness: Circularity ratio: For a circle, R c = 1 P 2 A R c = 4πA P 2 R c is invariant under translation, rotation, and scaling 20 21 Topological descriptors Genus via erosions V H B Invariant to a large class of local deformations. Examples: number of connected components; number of holes Genus or Euler number: E = C H (number of connected components minus the number of holes). Computable by erosions or hit-or-miss transforms. g 4 (I): binary image I with 4-connected 1-pixels and 8- connected 0-pixels: g 4 (I) = # I # I V # I H + # I B Similar formulas for g 8 (I), also via hit-or-miss transforms. # I=no. of pixels of I, etc. 22 23
Texture Texture descriptions Refers to the spatial distribution of discrete grey value variations, described in terms of: uniformity coarseness regularity directionality Three main approaches: 1. statistical: moments, co-occurrence matrix 2. structural, viewing a texture as an arrangement of texture primitives 3. spectral, using the Fourier transform to detect global periodicities Straw Raffia 24 25 Statistical texture description: co-occurrence matrix Co-occurrence matrix Let Q be a spatial predicate defined on pixel pairs (p, q), such as: Q(p, q) q is a right neighbour of p. For an image with L levels and a binary spatial predicate Q, the L L co-occurrence matrix G is defined by: g(i, j) ={no. of pixel pairs with grey levels (z i, z j ) satisfying predicate Q}, 1 i, j L Predicate: Q(p, q) q is a right neighbour of p. 26 27
Co-occurrence matrix Co-occurrence matrix: example Given a co-occurrence matrix G w.r.t. a spatial predicate Q n denotes the total number of pixels pairs satisfying Q (sum of the elements of G) texture 1 texture 2 texture 3 G1 G2 G3 The quantity pij = gij /n is an estimate of the probability that a pixel pair satisfying Q has values (zi, zj ). Properties of the distribution pij can be used to characterize the spatial properties represented by G 28 Structural texture description Textures generated from texture primitives (Brodatz texture collection, http://www.ux.uis.no/~tranden/brodatz.html). 29 Spectral texture description 30 Fourier spectra of random and ordered textures. 31
Spatial moment invariants Spatial moment invariants Central moments of an M N image f(x, y): µ pq = M 1 x=0 N 1 (x x) p (y y) q f(x, y), p, q = 1, 2,... y=0 with x = m 10 /m 00, y = m 01 /m 00, m pq = M 1 N 1 x=0 y=0 xp y q f(x, y). Normalized central moments: η pq = µ pq /µ γ 00 where γ = p+q 2 + 1. Invariant moments: combinations φ 1,..., φ 7 of η pq which are invariant to translation, rotation and scale-change. Values of each invariant moment φ k are the same for all pictures. 32 33