IMAGE PROCESSING >FILTERS AND EDGE DETECTION FOR COLOR IMAGES UTRECHT UNIVERSITY RONALD POPPE

Transcription

1 IMAGE PROCESSING >FILTERS AND EDGE DETECTION FOR COLOR IMAGES UTRECHT UNIVERSITY RONALD POPPE

2 OUTLINE Filters for color images Edge detection for color images Canny edge detection

3 FILTERS FOR COLOR IMAGES

4 FILTERS FOR COLOR IMAGES We have built filters for gray-valued/scalar images Extension to color images is not straightforward Three options: 1. Convert image to gray values and perform scalar filtering 2. Perform scalar filtering in each color channel and combine results 3. Treat color as a vector and perform filtering in vector space

5 FILTERS FOR COLOR IMAGES 2 Drawback of converting to grayscale Different colors map to the same intensity value Loss of contrast between color channels

6 LINEAR FILTERING Option 2: process each channel independently Recap: scalar filtering Scalar image I with pixels I(u,v) Discrete filter kernel H with elements H(i,j) (assumed rectangular)

7 LINEAR FILTERING 2 For a vector-valued image, we replace I by I: For the k th color channel, we thus apply the following filtering: For the three RGB channels, we thus obtain:

8 LINEAR FILTERING 3 Schematic representation of the use of monochromatic filters:

9 LINEAR FILTERING 4 Option 3: treat pixel as a vector For a linear smoothing filter: C u,v = (c 1,, c n ) the color pixel values in I at (u,v), with n the size of H Each c m ϵ R K (K the number of color channels) is positive H u,v = (α 1,, α n ) the filter kernel H at (u,v) The resulting color pixel Ī(u,v) is a linear combination of C u,v and H u,v : Ī(u,v) = α 1. c 1 + α 2. c α n. c n

10 LINEAR FILTERING 5 If all αα mm are positive and the coefficients are normalized such that αα mm = 1: Resulting color Ī(u,v) falls within convex hull of c 1 c n Special case: for two colors c 1 and c 2, the results lies on a line between them

11 LINEAR FILTERING 5 A color space is linear or non-linear A transformation T between two color spaces is linear or non-linear We can perform linear filtering in a linear space by transforming to and from it

12 LINEAR FILTERING 6 When the mapping from A to B with transformation T is non-linear: The interpolation in A and the back-transformed interpolation in B are not the same The interpolation is only correct in one of the spaces

13 LINEAR FILTERING 7 Differences in component/luminance values in relation to linear RGB Mapping from red to green, and from black to white (255, 255, 255)

14 LINEAR FILTERING 8 Due a non-linear mapping, distances between colors vary As a result: interpolated values are also different For srgb: note the dark bands as a result of the highly non-linear RG-curve

15 LINEAR FILTERING 9 When applying a linear smoothing filter, we want the brightness of the image (rather than that of each pixel) to be constant: For a scalar image, this is guaranteed when for all filter kernel values: αα mm = 1 For a vector image, we at least want the minimum and maximum brightness to be within the range of the original image:

16 LINEAR FILTERING 9 The brightness condition is satisfied when: Brightness is a linear combination of component (channel) values Brightness is an independent component (e.g., Y in CIEXYZ) Brightness in RGB is termed luminance Brightness in srgb is termed luma Both are linear combinations of their components Both are preserved in linear filtering in their respective color spaces

17 LINEAR FILTERING 10 In CIELUV and CIELAB, brightness is a function of lightness L*: L* = Y 1/2.38 Brightness condition is again satisfied in linear filtering In summary, filtering in any of the described color spaces preserves a particular brightness quantity: Luma for srgb Luminance (Y ) for linear RGB Lightness (L) for CIELUV and CIELAB

18 LINEAR FILTERING 11 Ideally, the perceived color difference is proportional to the distance in color space Expected for color spaces motivated by psychophysical measurements Unfortunately, not achieved by any of the (common) color spaces

19 LINEAR FILTERING 12 Applying a linear filter in RGB and srgb always yields colors in the gamut Converted to CIELAB, the colors of RGB and srgb are non-convex Linear filtering can lead to values outside the gamut

20 LINEAR FILTERING 13 In summary: Linear filtering on individual components is common On (non-linear) srgb, it is not justified Results on RGB are (perceptually) not better CIELUV and CIELAB are more suitable When transforming from RGB or srgb, linear filtering in these spaces can lead to out-of-gamut values

21 QUESTIONS?

22 NON-LINEAR COLOR FILTERS

23 NON-LINEAR COLOR FILTERS The result of a non-linear filter is not a linear combination of the original pixels values Median, min, max, etc. Scalar median filter takes the median value for each channel independently Typically different median value per channel Risk of introducing color

24 NON-LINEAR COLOR FILTERS 2 A median filter requires that the values can be sorted No natural ordering exists for vectors Median value of a sequence has smallest sum of distances to all other values For a sequence of scalar values P = (p 1,, p n ), p m = median(p) such that Extension from scalar to vector is straightforward: For P = (p 1,, p n ), p m = median(p) such that

25 NON-LINEAR COLOR FILTERS 3 We replaced scalar difference. with vector norm. D L (p,p) is the aggregate distance of p with respect to P under distance norm L: Some common distance norms: Manhattan distance Euclidian distance Chebyshev distance

26 NON-LINEAR COLOR FILTERS 4 We can thus define the vector median using aggregate distance: Vector median does not consider color channels to be independent No new color values introduced in filtering Scalar vs. vector median:

27 NON-LINEAR COLOR FILTERS 5 Drawback of a median filter is that it also smooths edges Addressed by sharpening vector median filter Key idea is not to consider all pixels but only the a most similar ones Excludes pixels of other color regions (considered outlier values) As a result: no blurring of contrast regions Modification of vector median filter by introducing additional sorting step As a consequence, added time complexity

28 QUESTIONS?

29 EDGE DETECTION IN COLOR IMAGES

30 EDGE DETECTION Contrast is essential in visual processing Edges between objects We have looked at edge detection in grayscale images More sophisticated to do in vector (color) images

31 MONOCHROMATIC TECHNIQUES Brief recap of single channel edge detection We can apply gradient filters such as Sobel in x and y direction: From the gradient vector we, we can calculate the edge s Strength: Orientation:

32 MONOCHROMATIC TECHNIQUES 2 The orientation gives the direction of the maximum change Normal to the edge tangent For the multidimensional case, we can apply the process for each channel:

33 MONOCHROMATIC TECHNIQUES 3 From these derivatives, we can determine the edge strength in each channel: Edge strength is measured as: Manhattan distance (L 1 norm) Euclidian distance (L 2 norm) Taking the maximum strength (L norm)

34 MONOCHROMATIC TECHNIQUES 4 Calculating the orientation is less straightforward Each channel has own orientation Simple solution is to select orientation of channel with strongest gradient

35 QUESTIONS?

36 EDGES IN VECTOR-VALUED IMAGES

37 EDGES IN VECTOR-VALUED IMAGES Monochromatic techniques treat color channels independently Potentially, information gets lost In the following, we treat an image as a three-dimensional vector field Coordinates x = (x,y) are 2D The color values (e.g. RGB) are 3D An image is thus a mapping I: R 2 R 3

38 EDGES IN VECTOR-VALUED IMAGES 2 For a (RGB) vector image I = (I R, I G, I B ), the gradient at point ẋ is: The Jacobian matrix J I (ẋ) combines the partial derivatives of a vector field: In rows, the color components In columns, the partial derivatives according to direction (x and y)

39 EDGES IN VECTOR-VALUED IMAGES 3 Norm of the column vector is the amount of change in x or y direction We now want to find the direction of maximum change We define the unit vector oriented at angle θ: The directional gradient of I is the product of this vector and the Jacobian:

40 EDGES IN VECTOR-VALUED IMAGES 4 We obtain the squared local contrast of I in direction θ by taking the square norm: We want to find the angle θ with the maximum gradient magnitude, so max S θ (I, ẋ) Root of first partial derivative of S with respect to θ Termed maximum local contrast

41 EDGES IN VECTOR-VALUED IMAGES 5 Alternative is to take the largest eigenvalue of The largest eigenvalue is: This is the same value as the maximum local contrast

42 EDGES IN VECTOR-VALUED IMAGES 6 One of eigenvectors associated with the maximum local contrast is Also, the rate of change is also maximal in the opposite direction x 1 So S θ (I, ẋ) for θ is the same as for θ+kπ This makes the orientation of the maximum change inherently ambiguous The orientation of the edge normal can be obtained using:

43 EDGES IN VECTOR-VALUED IMAGES 7 Overall, the differences between monochromatic edge detection and vectorbased edge detection are minimal Added computational cost not really justified Typically, monochromatic edge detection is applied

44 CANNY EDGE DETECTION

45 CANNY EDGE DETECTION To obtain edges, we threshold the gradient magnitude image Typically contains spurious edges Addressed with Canny edge detection

46 CANNY EDGE DETECTION 2 We first focus on grayscale images and then move to color Canny edge detection consists of three steps: Pre-processing Edge localization Edge tracing and hysteresis thresholding In pre-processing, the image is first smoothed with a Gaussian kernel Then, we calculate the local gradients in x- and y-direction

47 CANNY EDGE DETECTION 3 Edge localization aims to detect the pixels with the largest local change Non-maximum suppression algorithm Considered an edge thinning technique For each pixel, determine the orientation For efficiency, consider only four octants Compare E mag with neighboring pixels in direction Suppress pixel when not the maximum Result is edge image with many zeros

48 CANNY EDGE DETECTION 4 Calculation of orientation is facilitated by simple rotation over π/8: Octant can be derived by comparing sign and value of x and y Circumvents use of computationally costly trigonometric functions

49 CANNY EDGE DETECTION 5 Final step is to find sets of connected non-suppressed edge points Termed hysteresis thresholding Uses low and high thresholds t lo and t hi Loop over image pixels until you find an unprocessed pixel with E mag > t hi Add connected pixels (u,v ) when E mag (u,v ) > t lo Continue until there are no unprocessed pixels with E mag > t hi Result are sets ( curvy lines ) of pixels with relatively stable edges

50 CANNY EDGE DETECTION 6 Example: Different colors of different sets of pixels

51 CANNY EDGE DETECTION 7 For color images, we can merge the edges from individual channels Edges in different channels typically occur at different locations Many spurious (and less meaningful) edges We can use the multi-gradient concept Adapt only the pre-processing Pre-processing: Smoothing is performed per channel Color edge magnitude is calculated as the squared local contrast

52 CANNY EDGE DETECTION 8 Considerations of smoothing: Parameter σ determines the amount of smoothing Effect on the amount of located edges σ = 1.0 σ = 5.0

53 QUESTIONS?

54 NEXT LECTURE

55 NEXT LECTURE Next lecture is about: Exam Q&A Friday November 2, 11:00-12:45 (RUPPERT-042) Walk-in session: Friday November 2, 9:00-10:45 Deadline Sunday November 11, 23:00

56 EXAM Q&A Exam Q&A on Friday November 2, 11:00-12:45 Send me questions/topics so I can cover them No later than today

57 CONTENTS OF THIS LECTURE

58 CONTENTS OF THIS LECTURE Advanced methods (book III) Chapter 3: Filters for Color Images (not 3.3) Chapter 4: Edge Detection in Color Images (not )