SUMMARY PART I. Variance, 2, is directly a measure of roughness. A bounded measure of smoothness is

Transcription

1 Digital Image Analsis SUMMARY PART I Fritz Albregtsen 4..6 Teture description of regions Remember: we estimate local properties (features) to be able to isolate regions which are similar in an image (segmentation), and possibl later identif these regions (classification), usuall with the final goal of object description One can describe the teture of a region b: smoothness, roughness, regularit, orientation... Problem: we want the local properties to be as local as possible Large region or window Precise estimate of features Imprecise estimate of location Small window Precise estimate of location Imprecise estimate t of feature values Using variance estimates Variance,, is directl a measure of roughness A bounded measure of smoothness is R is close to for homogenous areas R tends to as, roughness, increase. order statistics discussion. order statistics can separate two regions even if μ = μ, as long as The statistics of a piel (, ) is found in a local window Problems: Edges around objects are eaggerated Solution: use adaptive windows. order statistics does not describe geometr or contet Cannot discriminate between Solution: Calculate. order statistics with different resolutions, and obtain indirect information about. and higher order statistics. Simpl use. or higher order statistics. 3 4

2 GLCM Matri element P(i,j d,θ) in a GLCM is. order probabilit of changing from gralevel i to j when moving distance d in direction. Dimension of co-occurrence matri is GG (G = gra-levels in image) Reduce the number of gra-levels b re-quantization Choose a distance d and a direction In this eample, d= and = GLCM Usuall a good idea to reduce the number of (d,) variations evaluated Simple pairwise relations: P(d, ) = P t (d,8 ) P(d,45 ) = P t (d,5 ) P(d,9 ) = P t (d,7 ) P(d,35 ) = P t (d,35 ) Isotropic matri b averaging P(), { o, 45 o, 9 o, 35 o } Beware of differences in effective window size! Check all piel pairs with distance d and direction inside the window. Q(i,j d,) is the number of piel pairs where piel in the pair has piel value i and piel has piel value j. j An isotropic teture is equal in all directions If the teture has a clear orientation, we select according to this. 5 6 GLCM features Learning goals - teture A number of features are available (Haralick et al., ) Ma be seen as weight functions applied to probabilit matri: Weighting based on value of GLCM element Eample: Entrop W(i,j) = -log{p(i,j)} Weighting based on position in GLCM 5 Eample: 5 Inertia W(i,j) = (i-j) Understand what teture is, and the difference between first order and second order measures Understand the GLCM matri, and be able to describe algorithm Understand how we go from an image to a GLCM feature image Preprocessing, choosing d and, selecting some features that t are not too correlated There is no optimal teture features, it depends on the problem You should be able to discuss which feature will discriminate between two different tetures, t based on the distribution ib ti of their GLCM s. 7 8

3 Edge-based segmentation Thinning of edge in 33 neighborhood Two steps are needed:. Edge detection (to identif edgels -edge piels) - (Gradient, Laplacian, LoG, Cann filtering). Edge linking linking adjacent edgels into edges Local Processing magnitude of the gradient direction of the gradient vector edges in a predefined neighborhood are linked if both magnitude and direction criteria is satisfied Global Processing via Hough Transform Quantize the edge direction into 4-directions four (or eight) directions. If gradient magnitude G(,) >, inspect the two neighboring piels in the given direction. 3 If gradient magnitude of an of these 3 neighbors is higher than G(,), 4 mark kthe piel. 5 4 When all piels have been scanned, 6 7 delete or suppress the marked piels. Used iterativel in nonmaima suppression. 3 8-directions 9 Hough transform basic idea HT and polar representation of lines One point in (,) corresponds to a line in the (a,b)-plane (,)-space (a,b)-space Polar representation of lines

4 HT using the full gradient information Given a gradient magnitude image g(,) containing a line. Simple algorithm: for all g( i, i )>T do for all do = i cos + i sin find indees (m,n) corresponding to (ρ, θ) and increment A(m,n); Better algorithm if we have both The gradient magnitude g(,) And the gradient components g and g So we can compute gradient direction g g (, ) arctan The new algorithm: g for all g( i, i )>T do = i cos( g (,)) + i sin( g (,)) find indees (m, n) corresponding to (, g (,)), increment A(m,n); Hough transform for circles Acircleintheplane in -plane is given b ( c) c r So we have a 3D parameter space. Simple 3D accumulation procedure: set all A[ c, c,r]=; for ever (,) where g(,)>t for all c and c r=sqrt((- c ) +(- c ) ); A[ c, c,r] = A[ c, c,r]+; Better procedure? 3 4 Hough transform for ellipses A general ellipse in the -plane has 5 parameters: Position of centre ( c, c ), semi-aes (a,b), and orientation (θ). Thus, we have a 5D parameter space. For large images and full parameter resolution, straight forward HT ma easil overwhelm our computer! Reducing accumulator dimensionalit: - Pick piel pairs with opposite gradient directions - Midpoint of pair votes for centre of ellipse! - Reduces HT-accumulator from 5D to 3D. - Min and ma distance of piel pairs => b & a Reducing accumulator etent & resolution Using other tricks Learning goals - HT Understand the basic Hough Transform, and its limitations Understand that the normal representation is more general Be able to implement line detecting HT with accumulator matri Understand how this ma be improved b gradient direction information. Be able to implement simple HT for circles of given size, in order to find position of circular objects in image. Understand how this ma be improved. Understand simple HT to detect ellipses Understand how this ma be improved. Understand the basic random Hough transform. Do the eercises! 5 6

5 Shape representations vs. descriptors After the segmentation of an image, its regions or edges are represented and described in a manner appropriate for further processing. Shape representation: the was we store and represent the objects Perimeter (and all the methods based on perimeter) Interior (and all the methods ) Shape descriptors: recipes for features es characterizing acte ing object shape. The resulting feature values should be useful for discrimination between different object tpes. Chains Chains represent objects within the image or borders between an object and the background Object piel How do we define border piels? 4-neighbors 8-neighbors In which order do we check the neighbors? Chains represent the object piel, but not their spatial relationship directl. 8 Chain codes Start point & rotation Chain codes represent the boundar of a region Chain codes are formed b following the boundar in a given direction (e.g. clockwise) with 4-neighbors or 8-neighbors 3 A code is associated with each direction 3 A code is based on a starting 4 point, often the upper leftmost point of the object directional code 8-directional code The chain code depends on the starting point. It can be made invariant of start point b treating it as a circular/periodic sequence, and redefining the start point so that the resulting number is of minimum magnitude. We can also normalize for rotation b using the first difference of the chain code: (i.e., direction i changes between code elements) Code: 33 First difference (counterclockwise): To find first difference, look at the code and count counterclockwise directions. Treating the curve as circular, add the 3 for the first point. Minimum circular shift of first difference: This invariance is onl valid if the boundar itself is invariant i to rotation ti and scale. 9

6 Relative chain code Here, directions are defined in relation to a moving perspective. Eample: Orders given to a blind driver ("F, "B, "L, "R"). The directional code representing an section of the contour is relative to the directional code of the preceding section. 3 Wh is the relative code: R,F,F,R,F,R,R,L,L,R,R,F? 4 The absolute chain code for the triangles are 4,,7 and, 5, 3. The relative codes are 7, 7,. (Remember Forward is ) Invariant to rotation, as long as starting point remains the same. Start-point invariance b Minimum circular shift. Note: To find the first R, we look back to the end of the contour Note: rotate the code tbl table so thatt is forward from our position Recursive boundar splitting Draw straight line between contour points that are farthest apart. These two points are the initial breakpoints.. For each intermediate point:. Compute the point-to-line to distance 3. Find the point with the greatest distance from the line. 4. If this distance is greater than given threshold, we have a new breakpoint. 5. The previous line segment is replaced b two, and -4 above is repeated for each of them. The procedure is repeated until all contour points are within the threshold distance from a corresponding line segment. The resulting ordered set of breakpoints is then the set of vertices of a polgon approimating the original contour This is probabl the most frequentl used polgonization. Since it is recursive, the procedure is fairl slow. Recursive boundar splitting Draw line between initial breakpoints; boundar points farthest apart.. For each intermediate point:. If greatest distance from the line is lager than a given threshold, we have a new breakpoint. 3. The previous line segment is replaced b two line segments, and d - above is repeated for each of them. The procedure is repeated until all contour points are within the threshold distance from a corresponding line segment. The resulting ordered set of breakpoints is then the set of vertices of a polgon approimating the original contour. The set of breakpoints is a subset of the set of boundar points. 3 Sequential polgonization Start with an contour point as first breakpoint. Step through ordered sequence of contour points. Using previous breakpoint as the current origin, area between contour and approimating line is: Ai Ai i i i i, si i i Let previous point be new breakpoint if area deviation A per unit length s of approimating line segment eceeds a specified tolerance, T. If A i /s i < T, i is incremented and (A i, s i ) is recomputed. Otherwise, the previous point is stored as a new breakpoint, and the origin is moved to new breakpoint. This method is purel sequential and ver fast. Reference: K. Wall and P.E. Danielsson, A Fast Sequential Method for Polgonal Approimation of Digital Curves, Computer Vision, Graphics, and Image Processing, vol. 8, 984, pp

7 Sequential polgonization Signature representations = A signature is a D functional representation of a D boundar. It can be represented in several was. Simple choise: radius vs. angle: Ai i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i Invariant to translation. Not invariant to starting point, rotation or scaling. 5 6 Boundar segments from conve hull The boundar can be decomposed into segments. Useful to etract information from concave parts of the objects. Conve hull H of set S is the smallest conve set containing S. The set difference H-S is called the conve deficienc D. If we trace the boundar and identif the points where we go in and out of the conve deficienc, these points can represent important border points charaterizing the shape of the border. Border points are often nois, and smoothing can be applied first. Smooth the border b moving average of k boundar points. Use polgonal approimation to boundar. Simple algorithm to get conve hull from polgons. 7 Contour representation using D Fourier transform The coordinates (,) of these M points are Start then put into a comple vector s : s(k)=(k)+i(k), k[,m-] 7 We choose a direction (e.g. anti-clockwise) We view the -ais as the real ais and the -ais as the imaginar one for a sequence of comple numbers. The representation of the object contour is changed, but all the information is preserved. We have transformed the contour problem from D to D. = = s( ) 3 i s() i s(3) 3 3i s(4) 3 4i 8

8 Fourier-coefficients from f(k) We perform a D forward Fourier transform a( u) M M k iuk s( k)ep M M M k uk s( k) cos M uk isin, M u, M Comple coefficients a(u) are the Fourier representation of boundar. a() contains the center of mass of the object. Eclude a() as a feature for object recognition. a(), a(),...,a(m-) will describe the object in increasing detail. These depend on rotation, scaling and starting point of the contour. For object recognitions, use onl the N first coefficients (a(n), N<M) This corresponds to setting a(k)=, k>n- Fourier Smbol reconstruction Inverse Fourier transform gives an approimation to the original contour N ˆ( ) iuk s k a( u)ep, k, M k M We have onl used N features to reconstruct each component of s(k). ˆ( ) The number of points in the approimation is the same (M ), but the number of coefficients (features) used to reconstruct each point is smaller (N<M ). Use an even number of descriptors. The first -6 6 descriptors are found to be sufficient i for character description. The can be used as features for classification. The Fourier descriptors can be invariant to translation and rotation if the coordinate sstem is appropriatel chosen. All properties of D Fourier transform pairs (scaling, translation, rotation) can be applied. 9 3 Eamples from Trier et al. 996 Run Length Encoding of Objects Sequences of adjacent piels are represented as runs. Absolute notation of foreground in binar images: Run i = ;<row i, column i, runlength i >; Relative notation in gralevel images: ;(gralevel i, runlength i ); This is used as a lossless compression transform. Relative notation in binar images: Start value, length, length,, eol, Start value, length, length,, eol,eol. This is also useful for representation of image bit planes. RLE is found in TIFF, GIF, JPEG,, and in fa machines. 3 3

9 Topologic features Projections This is a group of invariant integer features Invariant to position, rotation, scaling, warping Features based on the object skeleton Number of terminations (one line from a point) Number of breakpoints or corners (two lines from a point) Number of branching points (three lines from a point) Number of crossings (> three lines from a point) Region with two holes Region features: Number of holes in the object (H) Number of components (C) Euler number, E = C - H Number of connected components number of holes Smmetr Regions with three connected components D horizontal projection of the region: p h( ) f (, ) D vertical projection of the region: p v ( ) f (, ) Can be made scale independent b using a fied number of bins and normalizing i the histograms. Radial projection in reference to centroid -> signature Moments Central moments Borrows ideas from phsics and statistics. For a given continuous intensit distribution g(, ) we define moments m pq b These are position invariant moments : where The total mass and the center of mass are given b For sampled (and bounded) intensit distributions f (, ) Amomentm m pq is of order p+q q. This moment is NOT invariant to position of object! 35 This corresponds to computing ordinar moments after having translated the object so that center of mass is in origo. Central moments are independent of position, but are not scaling or rotation invariant. What is for a binar object? 36

10 Object orientation - I Object orientation - II Orientation is defined as the angle, relative to the X-ais, of an ais through h the centre of mass that gives the lowest moment of inertia. Orientation θ relative to X-ais found b minimizing: I f, where the rotated coordinates are given b cos sin, sin cos The second order central moment of the object around the α-ais, epressed in terms of,, and the orientation angle θ of the object: cos sin f I, β Y θ α X Second order central moment around the α-ais: I cos sin f, Derivative w.r.t. Θ = => I f (, ) cos sin sin cos f (, ) cos sin f (, ) sin cos cos sin sin cos sin cos tan tan cos sin tan So the object orientation is given b: β Y θ α X We take the derivative of this epression with respect to the angle θ Set derivative equal to zero, and find a simple epression for θ : tan, where, if,, if Bounding bo - again Image-oriented bounding bo: The smallest rectangle around the object, having sides parallell to the edges of the image. Found b searching for min and ma and within the object (min, min, ma, ma) Object-oriented bounding bo: Smalles rectangle around the object, having one side parallell to the orientation of the object (). The transformation cos sin, cos sin is applied to all piels in the object (or its boundar). Then search for min, min, ma, ma What if we want scale-invariance? Changing the scale of f(,) b (,) gives a new image: f (, ) f /, / The transformed central moments p q q pq If =, scale-invariant central moments are given b the normalization: i pq p q pq,, p q ( ) pq 39 4

11 Moments as shape features The curse-of-dimensionalit The central moments are seldom used directl as shape descriptors. Major and minor ais are useful shape descriptors. Object orientation is normall not used directl, but to estimate rotation. The set of 7 Hu moments can be used as shape features. (Start with the first four, as the last half are often zero for simple objects). Also called peaking phenomenon. For a finite training sample size, the correct classification rate initiall increases when adding new features, attains a maimum and then begins to decrease. The implication is that: t For a high measurement compleit, we will need large amounts of training data in order to attain the best classification performance. => 5- samples per feature per class. Illustration from G.F. Hughes (968). Correct classification rate as function of feature dimensionalit, for different amounts of training data. Equal prior probabilities of the two classes is assumed. 4 4