Lesson 6: Contours. 1. Introduction. 2. Image filtering: Convolution. 3. Edge Detection. 4. Contour segmentation

Transcription

1 . Introduction Lesson 6: Contours 2. Image filtering: Convolution 3. Edge Detection Gradient detectors: Sobel Canny... Zero crossings: Marr-Hildreth 4. Contour segmentation Local tracking Hough transform Computer Vision - J.D. Tardós

2 Contour Segmentation Obtain meaningful elements of the objects surface Contours summarize most of the image information Computer Vision - J.D. Tardós 2

3 Edge detection Contour Segmentation Edges: points with significant brightness change» Maxima of the image Gradient (st derivative of brightness)» Zero crossings of the image Laplacian (2nd derivative of brightness) Contour Segmentation Edge grouping Noise removal Line or curve fitting Convolution Masks Masks Computer Vision - J.D. Tardós 3

4 Example: Canny s method Canny operator: derivative of a Gaussian Maxima detection Gradient magnitude and orientation Local gradient maxima Contour segmentation Local tracking + hysteresis thresholding Contour chains Line or curve fitting Lines and curves Computer Vision - J.D. Tardós 4

5 2. Image filtering. Convolution Typical image noise: Salt and pepper noise: some random pixels that are completely white or black Gaussian noise: the grey level of all pixels is perturbed by a random gaussian value I[ i j] = I ideal [ i j] + p p 2 N( σ ) figura 4.5 Jain Computer Vision - J.D. Tardós 5

6 Computer Vision - J.D. Tardós Linear filters: Convolution Linear filters: Convolution Every pixel in the output image is a linear combination of several neighbouring pixels from the input image Convolution Mask (or Kernel): Masks are usually centred If not mark the central pixel [ ] [ ] [ ] [ ] [ ] [ ] n j m i f n m K j i f n m K j i g k k k k k k k k k n m K b a m d c n d b b c b d c d a a c a + + = = = = L L L L L L L L L L L L L L L L

7 Uncentredmask Convolution: Examples Forward difference K= g(ij)= f(ij +) f(ij) Centred mask Binomial filter 3x3 B 3x3 = g(i j) = 6 [f(i j ) + 2f(i j) + f(i j +) + 2f (i j ) + 4f(i j) + 2 f( j +) + f (i + j ) + 2f(i + j) + f(i + j +)] Computer Vision - J.D. Tardós 7

8 Convolution: Garbage Band Near the image borders the mask does not fit inside the actual image Garbage band For a n x n mask (with n odd) the size of the garbage bands is: (n-)/2 Either let the garbage enter the garbage band Or better fill the band with zeros Computer Vision - J.D. Tardós 8

9 Image Smoothing (noise removal) Box filter Arithmetic mean: K 3x 3 = 9 Binomial Filter Values from the Pascal triangle: Weight decrease far from the centre: B 3x 3 = Dividing by the sum of the mask weights the mean image brightness remains unchanged Computer Vision - J.D. Tardós 9

10 Gaussian Smoothing Gaussian Filter G σ (i j) = c e (i 2 + j 2 ) 2 σ 2 c is computed such that the weights sum up to For bigger σ bigger smoothing effect Mask size: 5σ -7σ Example: Gaussian mask 5 x 5 for σ = c = / Computer Vision - J.D. Tardós

11 Gaussian Smoothing Smoothing with σ = 2 3 and Computer Vision - J.D. Tardós

12 Separable Masks Separability condition: rank(k) = K( i j) = K y ( i) K x ( j) g( i j) = K( i j) f ( i j) = K y ( i) [ K x ( j) f ( i j)] = K x ( j) [ K y ( i) f ( i j)] Mask nxn Mask nx + Mask xn more more efficient Binomial Filter: B 3x 3 = = Gaussian Filter: G σ = = Computer Vision - J.D. Tardós 2

13 Computer Vision - J.D. Tardós Separable Masks: Implementation Separable Masks: Implementation Efficient programming: Only an intermediate image row is needed ) ( ) ( ) ( ) ( ) ( ) ( )] ( ) ( [ ) ( ) ( ) ( ) ( j i f j K j i g j i f i K j i f j i f i K j K j i f j i K j i g x y y x = = = = row i of f (ij) Kx row i of g(ij) Ky row i of f(ij)

14 It is a non-linear filter Median Filter Sort the nxn neighbours of pixel (ij) by grey level Assign to the pixel (ij) in the output image the central value (the median) Good for salt and pepper noise and for removing defects on analogic slides or negatives» The outliers do not influence the output value Computer Vision - J.D. Tardós 4

15 Median Filter Original image 64 (85) (525) Median filter 32 (85) (525) Linear filter Computer Vision - J.D. Tardós (85) (55) 5

16 3. Edge Detection Maxima of the Gradient Zero crossings of the Laplacian (st derivative) (2nd derivative) Brightness Digitalized image st derivative st difference - 2nd derivative -2 2nd difference Computer Vision - J.D. Tardós 6

17 Computer Vision - J.D. Tardós Gradient Gradient-Based Edge Detectors Based Edge Detectors Gradient: Operators: Divide by the sum of the positive mask coefficients Divide by the sum of the positive mask coefficients ( ) ( ) ( ) ( ) y x y x y x y x Orientatio n f Magnitude y f x f y x Gradient f = + + = = = atan 2 ) ( 2 2 θ Sobel Prewitt Roberts difference st y x

18 Was designed to optimize: Canny Edge Detector Detection robustness against noise Precision in edge location Single response Can be approximated by the derivative of a Gaussian: G σ (x) = x x2 2 exp σ 2σ σ= Mask size: 5σ7σ Example for σ = Divide by by the the sum sum of of the the positive mask mask coefficients Computer Vision - J.D. Tardós 8

19 Canny Edge Detector Gradient in x and y axis: It is a separable operator x = G σ (x) G σ (y) f (i j) y = G σ (x) G σ (y) f (i j) For σ = and n = 5 x = K f (i j) y = K f (i j) Computer Vision - J.D. Tardós 9

20 Non-Maximum Suppression Find Find point where the the gradient magnitude is is maximal in in the the direction of of the the gradient vector For every pixel with magnitude bigger that a threshold verify the maximum using a 3x3 window: A A2 A3 g B A8 A A4 C A7 A6 A5 Compute the gradient magnitude at points B and C by linear interpolation using the magnitudes at A3-A4 and A7-A8 and the distance between the points Point A is a maximum if Magnitude(A) > Magnitude (B) and Magnitude(A) >= Magnitude (C) Computer Vision - J.D. Tardós 2

21 Gradient Maxima σ = Computer Vision - J.D. Tardós 2

22 Edge Detection using the Laplacian Laplacian: 2 f (x y) = 2 x + 2 y = 2 f x f y 2 2 x 2ªdiferencia 2 2 y 2 Typical Laplacian operators: Zero crossings give continuous and closed contours (why?) The 2 nd derivative amplifies the noise twice: smoothing is mandatory Computer Vision - J.D. Tardós 22

23 Marr-Hildreth operator Gaussian Filter + Laplacian 2 G σ f (i j) = ( 2 G σ ) f (i j) 2 G σ (r) = r 2 2σ 2 σ 4 exp r 2 2σ σ σ= Inverted Mexican Hat Hat Computer Vision - J.D. Tardós 23

24 Finding Zero-Crossings Only keep crossings bigger than a threshold mag_y:= mag_x:= if if g(ij) >= >= then then if if g(i+j) < then then mag_y:= g(ij) - g(i+j) endif endif if if g(ij+) < then then mag_x:= g(ij) - g(ij+) endif endif else else if if g(i+j) >= >= then then mag_y:= g(i+j) - g(ij) endif endif if if g(ij+) >= >= then then mag_x:= g(ij+) - g(ij) endif endif endif endif magnitude(ij):= max(mag_ymag_x) Computer Vision - J.D. Tardós 24

25 Edges Obtained at Zero-Crossings Computer Vision - J.D. Tardós 25

26 Goal: 4. Contour segmentation Obtain straight lines or curves Remove noisy contours Main techniques: Local analysis:» Contour following Hysteresis thhresholding» Line or curve fitting Split and merge Total least squares Hough Transform Computer Vision - J.D. Tardós 26

27 Hysteresis thresholding Goal: obtain chains of edge points with a given intensity (magnitude of gradient or zero-crossing) Finding a good threshold may be difficult th th = 8 broken contours th th = 4 too too much much noise noise Computer Vision - J.D. Tardós 27

28 Hysteresis thresholding An contour chain is required to have: All pixels with magnitude >= th_inf At least one pixel with magnitude >= th_sup Remove short contour chains Th_inf=4 Th_sup=8 Th_inf=4 Th_sup=8 Length> Computer Vision - J.D. Tardós 28

29 Contour Following procedure procedure get_chains get_chains for for i:= i:= to to num_rows num_rows for for j:= j:= to to num_cols num_cols start:=(ij) start:=(ij) if if magnitude(start) magnitude(start) >= >= th_sup th_sup then then n:= n:= follow(startchainn) follow(startchainn) reverse(chain n) reverse(chain n) follow(next(start)chainn)) follow(next(start)chainn)) if n >= minimal_length then if n >= minimal_length then store(chainn) store(chainn) endif endif endif endif endfor endfor endfor endfor end end get_chains get_chains procedure procedure follow(current follow(current in in out out chain chain in in out out n) n) while while current current <> <> nil nil n:=n+ n:=n+ chain(n):= chain(n):= current current magnitude(current):= magnitude(current):= current:= current:= next(current) next(current) endwhile endwhile end end follow follow function function next(current) next(current) return return pixel pixel for for next next in in neighbours(current) neighbours(current) if if magnitude(next) magnitude(next) >= >= th_inf th_inf then then return return next next endif endif endfor endfor return return nil nil end end next next In which order? Computer Vision - J.D. Tardós 29

30 Contour Following: which order? Reading order Contour Broken 4-neighbours first Computer Vision - J.D. Tardós 3

31 . Split (recursive): Split-and and-merge Obtain the line passing thru both chain endpoints Find the pixel further away from the line If distance > threshold (e.g. 2 pixels) split at this point Recursion for both chain sides 2. Merge: Merge two neighbouring lines if the distances of all the intermediate points to the merged line is smaller that the threshold 3. Remove short lines and lines with small gradient magnitude 4. Obtain the line passing thru the chain endpoints or perform a total least-squares line fitting Computer Vision - J.D. Tardós 3

32 Split-and and-merge: Example Split Split Split Split Stop Stop splitting Merge Merge Computer Vision - J.D. Tardós 32

33 Computing distances Distance from point 3 to the line passing through points and 2: (x3y3) (x2y2) d (xy) d = x 3( y y 2 ) y 3 ( x x 2 )+ y 2 x y x 2 ( y y 2 ) 2 + ( x x 2 ) Computer Vision - J.D. Tardós 33

34 Least-Squares Line Fitting Total least-squares (total regression) Find the line that minimizes the squared sum of perpendicular distances from all the points to the line: y θ ρ di Line Equation : x x cosθ + y sinθ ρ = Distance of Minimize : pixel i : d i D 2 = = x cosθ + y i i sinθ ρ ( x cosθ + y sinθ ρ ) 2 i i i Solution: same that finding the axis of minimal inertia Compute the position and orientation of the blob Endpoints: compute the perpendicular projection of the first and last pixels on the line (how?) Computer Vision - J.D. Tardós 34

35 Contour chains and lines obtained Computer Vision - J.D. Tardós 35

36 4.2 Hough Transform Global detection of lines or curves Using parametric representation Parameters defining a line: ρ y θ ρ = x cosθ + y sinθ i j y x y x y 2 2 ρ x y 3 3 θ x x y Computer Vision - J.D. Tardós x = j num_cols/2 y = num_rows/2 - i 36

37 Hough Transform A pixel (xi yi) belongs to all lines with equation: ρ=xi cosθ + yi sinθ The space (ρθ) is divided in cells Each pixel votes for the lines passing through itself Find the most voted lines ρ x y x y x y 3 3 x y 4 4 θ ρ Computer Vision - J.D. Tardós 37

38 Hough Tr.: Using the Orientation Pixels only vote for lines with orientation similar to its gradient orientation Improves the computing time Y x y Gradient ρ x y 2 2 θ ρ x y θ x y 4 4 X Computer Vision - J.D. Tardós 38

39 Hough Tr.: Using the Orientation Without Orientación for for i:= i:= to to num_rows for for j:= j:= to to num_cols if if magnitude(ij)>=th x:= x:= j - num_cols /2 /2 y:= y:= num_rows/2 - i for for θ:=θmin to to θmax θmax ρ:= ρ:= x*cos(θ) + y*sin(θ) vote_to_line(ijρθ) endfor endif endif endfor endfor Using Using Orientación for for i:= i:= to to num_rows for for j:= j:= to to num_cols if if magnitude(ij)>=th x:= x:= j - num_cols /2 /2 y:= y:= num_rows/2 - i θ:= θ:= orientation(ij) ρ:= ρ:= x*cos(θ) + y*sin(θ) vote_to_line(ijρθ) endif endif endfor endfor Computer Vision - J.D. Tardós 39

40 Hough Transform for Circles Valid for curves but the number of parameters increases Example: circle Radius and centre (r a b) Parametric form (polar) Using gradient orientation: r 2 = ( x a) 2 + ( y b) 2 x = a + r cosθ y = b + rsinθ θgra (xy) θgra (xy) (ab) (ab) a = x r cosθ b = y rsinθ a = x + r cosθ b = y + rsinθ Computer Vision - J.D. Tardós 4

41 Hough Transform for Circles 3D voting table: (rab) Using Using Orientation for for i:= i:= to to num_rows for for j:= j:= to to num_cols if if magnitude(ij)>=th x:= x:= j - num_cols /2 /2 y:= y:= num_rows/2 - i θ:= θ:= orientation(ij) for for r:= r:= rmin rmin to to rmax rmax a:= a:= x - r*cos(θ) b:= b:= y - r*sin(θ) vote_circle(ijrab) endfor endif endif endfor endfor Computer Vision - J.D. Tardós 4

42 Finding the centre of a corridor Edge points vote for a point of the horizon (all?) Horizon line Δ Δ Computer Vision - J.D. Tardós 42