Biomedical Image Analysis Point, Edge and Line Detection Contents: Point and line detection Advanced edge detection: Canny Local/regional edge processing Global processing: Hough transform BMIA 15 V. Roth & P. Cattin 152
Fundamentals BMIA 15 V. Roth & P. Cattin 153
Background Approximation of first-order derivative at x: Taylor expansion of f(x + x). Only linear terms: = f (x) = f(x + 1) f(x). f(x) x Second order: 2 f x = f (x) 2 x = f (x + 1) f (x) = f(x + 2) f(x + 1) f(x + 1) + f(x) = f(x + 2) 2f(x + 1) + f(x), subtracting 1 2 f = f(x + 1) 2f(x) + f(x 1) x2 BMIA 15 V. Roth & P. Cattin 154
Points, Edges, and Lines BMIA 15 V. Roth & P. Cattin 155
Points, Edges, and Lines (contd.) Ramp: First order derivatives produce thick edges. Point: Response magnitude of 2nd derivative is stronger enhances fine details (including noise). Line: Thin lines can be better detected with the 2nd derivative. Step: For the step as well as for the ramp, the 2nd derivative has a negative component double-edge effect. BMIA 15 V. Roth & P. Cattin 156
Spatial Filter Masks Computation of derivatives: 3 3 filter masks. Response at center: R = w 1 z 1 + w 2 z 2 + + w 9 z 9, where z k is the pixel intensity at location corresponding to k-th coefficient in the mask. BMIA 15 V. Roth & P. Cattin 157
Detection of isolated points Should be based on 2nd derivatives Laplacian: Digital functions: f = 2 f = 2 f x 2 + 2 f y 2. f = f(x+1, y)+f(x 1, y)+f(x, y+1)+f(x, y 1) 4f(x, y) Additional diagonal directions: ( f) = f + f(x 1, y 1) + f(x 1, y + 1) + f(x + 1, y 1) + f(x + 1, y + 1) 4f(x, y). Detect points by thresholding response: g = 1, if R T. BMIA 15 V. Roth & P. Cattin 158
Point Detection BMIA 15 V. Roth & P. Cattin 159
Line Detection: Fundamentals Use 2nd derivatives: stronger response and thinner lines than 1st derivatives. One approach: use Laplacian with proper handling of double-line effect Problem: what to do with the negative values? Taking the absolute value of Laplacian image doubles thickness of lines, see Fig. 10.5(c). Better: use only positive values (or values exceeding a threshold to remove noise) thinner lines, Fig. 10.5(d). BMIA 15 V. Roth & P. Cattin 160
Line Detection: Fundamentals Remaining problem: lines that are wide w.r.t. the size of the mask produce two response lines separated by zero valley. General assumption in line detection: lines are thin w.r.t. size of the mask. Thick lines are treated as regions and handled by edge detection. BMIA 15 V. Roth & P. Cattin 161
Line Detection with Laplacian BMIA 15 V. Roth & P. Cattin 162
Line Detection Masks Laplacian is isotropic (response independent of orientation). Often, interest lies in detecting lines with specified directions orientation-specific filter masks BMIA 15 V. Roth & P. Cattin 163
Diagonal Line Detection BMIA 15 V. Roth & P. Cattin 164
Steps, Ramps, Edges BMIA 15 V. Roth & P. Cattin 165
Steps, Ramps, Edges (cont d) BMIA 15 V. Roth & P. Cattin 166
Derivatives of Edges 1st derivative constant over ramp 2nd deriv. peaks at boarders of ramp Zero-crossing between extrema We conclude: 1st derivative detect edges 2nd derivative polarity of edges zero-crossing locate ramps BMIA 15 V. Roth & P. Cattin 167
Sensitivity to Noise (a) No noise (b) White noise,η = 0.004 (c) White noise, η = 0.02 (d) White noise, η = 0.1 1st and 2nd derivative look as expected (a). Noise derivatives more and more useless, although the noise in the image is almost invisible. BMIA 15 V. Roth & P. Cattin 168
Sensitivity to Noise (2) (a) White noise, η = 0.004 (b) filtered with Gaussian, σ = 1 (c) White noise, η = 0.02 (d) filtered with Gaussian, σ = 2 (a)+(c): derivatives without prior filtering; (b)+(d): filtered with a Gaussian low-pass filter of size 7. BMIA 15 V. Roth & P. Cattin 169
Basic Edge Detection A basic edge detection method should include the following three processing steps: 1. Smoothing: the Images must be smoothed prior to calculating the derivative, 2. Detection of edge points: A local operation to find edge candidates 3. Edge localization: A regional operation that groups edges candidates comprising a real edge BMIA 15 V. Roth & P. Cattin 170
Basic Edge Detection Gradient f(x, y) points in the direction of the greatest rate of change of f at location (x, y). [ ] [ ] f gx y f = grad(f) = = f g y y BMIA 15 V. Roth & P. Cattin 171
Basic Edge Detection Direction of f(x, y) is perpendicular to the edge: α(x, y) = tan 1 [g y /g x ] Magnitude (length) of f: rate of change in the direction of the gradient edge strength M(x, y) = mag( f) = gx 2 + gy 2 BMIA 15 V. Roth & P. Cattin 172
Spatial Masks for Gradients BMIA 15 V. Roth & P. Cattin 173
Sobel Examples BMIA 15 V. Roth & P. Cattin 174
Sobel Examples (gradient magnitudes) BMIA 15 V. Roth & P. Cattin 175
Sobel Examples (gradient angle) α(x, y) = tan 1 [g y /g x ] BMIA 15 V. Roth & P. Cattin 176
Sobel Examples (additional smoothing) BMIA 15 V. Roth & P. Cattin 177
Sobel Examples (thresholding) BMIA 15 V. Roth & P. Cattin 178
More advanced techniques for edge detection So far: simple spatial filtering operations, no provisions made for edge characteristics or noise content. Marr-Hildreth (1980): (i) intensity changes depend on scale detection requires operators of different sizes. (ii) sudden intensity changes lead to peak in 1st derivative, or, equivalently, to zero crossings in 2nd derivative. Idea: search for differential operator (1st and 2nd derivative) that can be tuned to act on desired scale. BMIA 15 V. Roth & P. Cattin 179
Marr-Hildreth Edge Detector Idea: use Laplacian of Gaussian LoG = G, with G being the 2D-Gaussian function G(x, y) = exp( x2 + y 2 2σ 2 ) G = x ( x σ G(x, y)) + 2 y ( y G(x, y)) σ2 = ( x2 σ 1 4 σ2)g(x, y) + (y2 σ 1 4 σ 2) = ( x2 + y 2 2σ 2 )G(x, y). σ 4 BMIA 15 V. Roth & P. Cattin 180
Marr-Hildreth: Conceptual Ideas Scale of operator is determined by σ: zero crossings at x 2 + y 2 = 2σ 2 circle with r = 2σ. Sometimes called Mexican hat operator. Two fundamental ideas: (i) Gaussian part blurs the image at scales much larger than σ. Gaussian is smooth in both the spatial and frequency domains no artifacts like ringing. (ii) Laplacian is isotropic, responds equally to changes in intensity in any mask direction (avoids need for several oriented masks). BMIA 15 V. Roth & P. Cattin 181
Marr-Hildreth Edge Detector BMIA 15 V. Roth & P. Cattin 182
Marr-Hildreth: Algorithm Running Marr-Hildreth means convolving the LoG filter with the input image f(x, y): g(x, y) = [ G(x, y)] f(x, y) = [G(x, y) f(x, y)] (linearity). 1. Filter input image with n n Gaussian lowpass filter. 2. Compute Laplacian of the filtered image based on appropriate mask. 3. Find zero crossings of the result BMIA 15 V. Roth & P. Cattin 183
Marr-Hildreth Algorithm (2) Specifying the size n of the Gaussian filter: 99.7% of the volume under a 2D Gaussian is within the 3σ region around the mean. Rule of thumb: choose size such that n is the smallest odd integer 6σ. Detecting zero crossings: use 3 3 neighborhood around any pixel p in filtered image g(x, y). A zero-crossing at p implies that the signs of at least two of its opposing neighboring pixels must differ test 4 cases (l/r, u/d, and two diagonals). Reduce influence of noise by additionally requiring that responses exceed threshold. BMIA 15 V. Roth & P. Cattin 184
Marr-Hildreth Example BMIA 15 V. Roth & P. Cattin 185
Marr-Hildreth: Approximations Idea: approximate LoG by a difference of Gaussians (DoG): DoG(x, y) = 1 2πσ1 2 G(x, y) 1 2πσ2 2 G(x, y), σ 1 > σ 2. BMIA 15 V. Roth & P. Cattin 186
Three basic objectives: Canny Edge Detector 1. Low error rate: All edges found and no spurious responses 2. Well localized edge points: Edge points should be as close as possible to the true edge 3. Single edge point response: Only one point should be returned per edge thin edges Canny managed to formulate these criteria mathematically and found optimal solutions to these formulations. BMIA 15 V. Roth & P. Cattin 187
Canny Edge Detector Canny [1986]: optimal 1D detector for edges distorted with white Gaussian noise: 1st derivative of Gaussian d dx G(x) = d x 2 dx e 2σ 2 = x σ 2e x2 2σ 2 Question: also in 2D? Yes, but only in the direction of the edge normal Problem: edge normals unknown! BMIA 15 V. Roth & P. Cattin 188
Optimal Step Edge Detector Solution 1: apply 1D edge detector in all possible directions impractical Sol. 2: Smooth image (Gaussian), compute gradient edge strength and direction BMIA 15 V. Roth & P. Cattin 189
Step 1: Gradient Magnitude and Direction Blurred image f b (x, y) by convolution with Gaussian: f b (x, y) = G(x, y) f(x, y) Gradient magnitude and direction are calculated: M(x, y) = gx 2 + gy 2 and α(x, y) = tan 1 [g y /g x ] Partial derivatives g x and g y : Sobel masks etc. Remember: M(x, y) and α(x, y) are of the same size as the image. BMIA 15 V. Roth & P. Cattin 190
Step 1: Gradient Magnitude and Direction g x = f/ x g y = f/ xy M(x, y) = g 2 x + g 2 y BMIA 15 V. Roth & P. Cattin 191
Step 2: non-maxima suppression Gradient wide ridges around local maxima thin those wide ridges non-maxima suppression: 1. Find all pixels along the edge normal α(x, y) 2. Only retain maximum magnitude value along normal BMIA 15 V. Roth & P. Cattin 192
Discrete Ridge Thinning In case of real images we have to specify a number of discrete orientations for the edge normals. In a 3 3 region there are 4 different edge orientations: BMIA 15 V. Roth & P. Cattin 193
Discrete Ridge Thinning (2) Depending on the direction of the edge normal α(x, y), define the pixels that have to be tested to suppress the non-maxima: BMIA 15 V. Roth & P. Cattin 194
Non-Maxima Suppression Algorithm 1. Find the discrete direction d k closest to α(x, y) 2. If M(central pix.) < M(one of its two neighbors along d k ), suppress it ( 0). BMIA 15 V. Roth & P. Cattin 195
Step3: Hysteresis Thresholding Final operation: use hysteresis thresholding to reduce false edge points. Canny suggested T H /T L 2.5 1. Select T H seed points 2. Inflate regions around seed points until value < T L BMIA 15 V. Roth & P. Cattin 196
Canny vs Marr-Hildreth BMIA 15 V. Roth & P. Cattin 197
Canny vs Marr-Hildreth BMIA 15 V. Roth & P. Cattin 198
Edge Linking and Boundary Detection Ideal case: edges... edge detection gives sets of pixels lying on In practice: edges not complete due to noise etc. Solution: linking algorithms designed to assemble edge pixels into meaningful edges and/or region boundaries Two main approaches: local (small neighborhoods) and global (whole image) BMIA 15 V. Roth & P. Cattin 199
Local processing Idea: similarity between points based on gradient magnitude and direction. Edge pixel at (s, t) is similar in magnitude to pixel at (x, y) if M(S, t) M(x, y) E. Similar in angle if α(s, t) α(x, y) A. Link two pixels if both criteria fulfilled. Computational expensive (all neighbors of every point) simplification desired BMIA 15 V. Roth & P. Cattin 200
Local processing: Efficient Algorithm Compute magnitude and angle arrays M(x, y), α(x, y) Define threshold E, direction A and acceptable band of directions around A binary image { 1 if M(x, y) > E AND α(x, y) = A ± T A g(x, y) = 0 otherwise Scan rows and fill all gaps with length < threshold K Rotate by angle θ, repeat horizontal scanning, rotate back. BMIA 15 V. Roth & P. Cattin 201
Local processing: Example BMIA 15 V. Roth & P. Cattin 202
Global Processing for Finding Lines Problem: Find all subsets of the known n edge points in an image that lie on a straight line. Idea: Find all lines determined by every possible combination of two edge points. Find the subset of points that are close to a particular line. Involves finding n(n 1)/2 n 2 lines and n(n(n 1))/2 n 3 comparisons of every point with every line. Intractable for almost all applications BMIA 15 V. Roth & P. Cattin 203
An Alternative Approach: Hough 1962 Point (x i, y i ) line y i = ax i + b. -many lines through (x i, y i ), all fulfill y i = ax i + b for some a and b. Rewrite b = x i a + y i one single line for every point (x i, y i ) in ab-plane. Two points (x i, y i ) and (x j, y j ) at (a, b ) the two lines intersect. Intersection slope a and the intercept b of the line passing through both (x i, y i ) and (x j, y j ). BMIA 15 V. Roth & P. Cattin 204
Hough Transform Problem with this approach: Slope a as line approaches vertical direction Also intercept b ab-plane must be infinitely large BMIA 15 V. Roth & P. Cattin 205
Hough Transform for Straight Lines Solution: normal representation x cos θ + y sin θ = ϱ, 90 θ 90, D ϱ D D: distance between opposite corners of the input image. Parameters ϱ and θ are bound and allow for a compact parameter space (accumulator) Intersection point (ϱ, θ ) corresponds to the line passing through both (x i, y i ) and (x j, y j ). BMIA 15 V. Roth & P. Cattin 206
Hough Transform Sub-divide the ϱθ-parameter space into m n cells, initialize with 0 s For θ = θ min... θmax: Evaluate ϱ = x k cos(θ) + y k sin(θ) Increase A(θ, ϱ) by 1 Value p in A(θ i, ϱ j ) means: p points in xy-plane lie on line ϱ j = x k cos(θ i ) + y k sin(θ i ) Peaks in A dominant lines in image BMIA 15 V. Roth & P. Cattin 207
Image with 5 points Hough Example Range of θ: 90... + 90, ϱ: ± 2D, each subdivided into 128 cells. The brightest point ( A ): line through 1, 3 and 5 A has coordinates ϱ = 0, θ = 45 : line passes origin, oriented at 45 Point B : line through 2, 3 and 4, oriented at θ = 45. BMIA 15 V. Roth & P. Cattin 208
Hough Example BMIA 15 V. Roth & P. Cattin 209
Hough Transform for Circles Hough transform applicable to any function f(v, c) = 0 with coordinates v and coefficients c. Circle: (x c 1 ) 2 + (y c 2 ) 2 = c 2 3 3 parameters (c 1, c 2, c 3 ) the accumulator is also 3-dimensional increased computational complexity BMIA 15 V. Roth & P. Cattin 210
Hough Transform for Circles Left: Canny edge detection result. Middle: Hough accumulator. Right: the strongest two circles found BMIA 15 V. Roth & P. Cattin 211