Part 3: Image Processing

Transcription

1 Part 3: Image Processing Moving Window Transform Georgy Gimel farb COMPSCI 373 Computer Graphics and Image Processing 1 / 62

2 1 Examples of linear / non-linear filtering 2 Moving window transform 3 Gaussian linear filtering 4 Edge detection 5 Rank filter 2 / 62

3 Local Means, Standard Deviations, and Ratios Image f(x, y) Mean 3 3 µ(x, y) Std 3 3 σ(x, y) Ratio 3 3 ρ(x, y) 3 3 window: Mean µ(x, y) = ξ= 1 η= 1 f(x + ξ, y + η) Standard deviation (std): 1 1 σ(x, y) = 1 3 (f(x + ξ, y + η) µ(x, y)) 2 Ratio ρ(x, y) = ξ= 1 η= 1 µ(x,y) max{σ(x,y),1} 3 / 62

4 Local Means, Standard Deviations, and Ratios Means, standard deviations (std), and mean/std ratios of grey values in a rectangular window of size (2k + 1) (2l + 1): 1 k Mean µ(x, y) = l (2k+1)(2l+1) η= l f(x + ξ, y + η) Standard deviation (std): k σ(x, y) = Ratio ρ(x, y) = 1 (2k+1)(2l+1) µ(x,y) max{σ(x,y),1} ξ= k ξ= k η= l l (f(x + ξ, y + η) µ(x, y)) 2 The window centre (0, 0) is moving across an input image f and visits each location (x, y). 4 / 62

5 Local Means, Standard Deviations, and Ratios Image f(x, y) Mean 7 7 µ(x, y) Std 7 7 σ(x, y) Ratio 7 7 ρ(x, y) 7 7 window: Mean µ(x, y) = ξ= 3 η= 3 f(x + ξ, y + η) Standard deviation (std): 3 3 σ(x, y) = 1 7 (f(x + ξ, y + η) µ(x, y)) 2 Ratio ρ(x, y) = ξ= 3 η= 3 µ(x,y) max{σ(x,y),1} 5 / 62

6 Local Means, Standard Deviations, and Ratios Image f(x, y) Mean 3 15 µ(x, y) Std 3 15 σ(x, y) Ratio 3 15 ρ(x, y) 3 15 window: Mean µ(x, y) = ξ= 1 η= 7 f(x + ξ, y + η) Standard deviation (std): 1 7 σ(x, y) = (f(x + ξ, y + η) µ(x, y)) Ratio ρ(x, y) = ξ= 1 η= 7 µ(x,y) max{σ(x,y),1} 6 / 62

7 Moving Window Transform (MWT) Slides 3 6 exemplify moving window transform: Local linear filter: average (mean) grey value. Local non-linear filter: scaled standard deviation (std) of grey values. Local non-linear filter: scaled mean / std ratio. Transformed image g = MWT(f): Value g(x, y) at pixel location (x, y) is a certain linear or non-linear function of values of the original image f. The values of f are taken in a (2k + 1) (2l + 1) rectangle, being centred on pixel location (x, y). E.g. k = l = 1 for the 3 3 window and (k = 1, l = 7) for the 3 15 window. General linear MWT multiplies each kernel (filter) coefficient with the image value that lies directly beneath it. 7 / 62

8 MWT (Example 1): Weighted Mean in 3 3 Window µ(x, y) = 1 1 s(ξ,η)f(x+ξ,y+η) ξ= 1 η= s(ξ,η) ξ= 1 η= 1 8 / 62

9 MWT (Example 1): Weighted Mean in 3 3 Window µ(x, y) = 1 1 s(ξ,η)f(x+ξ,y+η) ξ= 1 η= s(ξ,η) ξ= 1 η= 1 9 / 62

10 MWT (Example 2): Linear x-derivative Filter 10 / 62

11 MWT (Example 2): Linear x-derivative Filter g = MWT(f) Finite difference approximation of the partial derivative g(x, y) = f(x,y) x 11 / 62

12 MWT (Example 2): Linear x-derivative Filter g = MWT(f) Finite difference approximation of the partial derivative g(x, y) = f(x,y) x 12 / 62

13 MWT (Example 3): Linear y-derivative Filter g = MWT(f) Finite difference approximation of the partial derivative g(x, y) = f(x,y) y 13 / 62

14 MWT (Example 4): Linear Laplacian Filter #1 g = MWT(f) Finite difference approximation of the Laplacian operator g(x, y) = 2 f(x,y) + 2 f(x,y) x 2 y 2 14 / 62

15 MWT (Example 5): Linear Laplacian Filter #2 g = MWT(f) Finite difference approximation of the Laplacian operator g(x, y) = 2 f(x,y) + 2 f(x,y) x 2 y 2 15 / 62

16 MWT: Linear Laplacian Filtering Initial image Laplacian #1 Laplacian #2 16 / 62

17 Gaussian Linear Filtering To blur images and remove noise and fine detail. One-dimensional Gaussian function (zero mean): G(x) = 1 σ 2π e σ standard deviation x 2 2σ 2 κ = x σ G(κ) G(κ) G(0) / 62

18 Gaussian Linear Filtering Standard deviation of the Gaussian probability density function guides its behaviour: 68% of the x-values are the range [mean σ, mean + σ]. 95% of the x-values are the range [mean 2σ, mean + 2σ]. 99.7% of the x-values are the range [mean 3σ, mean + 3σ]. Distribution of the Gaussian function values (Wikipedia) 18 / 62

19 1D Gaussian Linear Filter Built using the 1D Gaussian G(x) = 1 2πσ e x2 2σ 2 Filter kernel s: an integer-valued discrete approximation of the normed Gaussian; e.g. for the 1 5 filter with σ = 1: x G(x) Norming 10G(x) G(u) u= 2 Integer rounding Gaussian filter s(x) G(x) Norming G(u) u= 2 Integer rounding Gaussian filter s(x) Norming λ G(x) is not unique: its only goal to make the rounding sensible. 19 / 62

20 Gaussian Linear Filtering 2D Gaussian filter is built using the 2D Gaussian G(x, y) = G(x)G(y) = 1 x 2 +y 2 2πσ 2 e 2σ 2 20 / 62

21 Gaussian Linear Filtering Filter kernel s: an integer-valued approximation of digitised continuous 2D Gaussian; e.g., the 5 5 window with σ = or The larger the value of σ, the wider the peak of the Gaussian and the larger the blurring. Non-uniform averaging: low pass filtering. Rotational symmetry with no directional bias. Fast computations due to separability (2D = 1D 1D). Might not preserve image brightness. 21 / 62

22 Gaussian Filtering Versus Median Filtering Gaussian filter: blurred edges; residual noise. Salt-and-pepper noise Filtered image 22 / 62

23 Gaussian Filtering Versus Median Filtering Median filter: non-blurred edges; removed noise. Salt-and-pepper noise Filtered image 23 / 62

24 Gaussian Filtering Versus Median Filtering 3 3 GF 5 5 GF GF 3 3 MF 5 5 MF MF 24 / 62

25 Gaussian Linear Filtering Inefficient for removing salt-and-pepper noise. Averaging is not robust to outliers (large deviations). Median is much more robust with respect to outliers. Efficient for image smoothing to more accurate approximation of derivatives in edge detection. Edges in initial image Edges after Gaussian smoothing 25 / 62

26 Initial Image to Detect Edges 26 / 62

27 Edge Enhancement 27 / 62

28 Edges Enhanced after Smoothing 28 / 62

29 MWT Based Edge Detection One of major applications of image filtering. Formal definition of an edge: Locations of sudden grey level / colour changes. Transition between objects or between an object and background. Locations that attract visual attention. Problem: Image noise has similar properties. Conventional 3-step approach: 1 Noise reduction (preserving edges as much as possible). 2 Edge enhancement. 3 Edge localisation. 29 / 62

30 MWT Based Edge Detection Estimation of the grey level gradient at pixels: g x (x, y) = f(x,y) x Finite difference approximation {}}{ f(x + 1, y) f(x 1, y) 2 g y (x, y) = f(x,y) y f(x, y + 1) f(x, y 1) 2 }{{} Finite difference approximation Gradient magnitude g(x, y) and angle a(x, y): g(x, y) = gx(x, 2 y) + gy(x, 2 y) g x (x, y) + g y (x, y) ( ) a(x, y) = tan 1 gy(x,y) g x(x,y) 30 / 62

31 MWT Based Edge Detection Noise smoothing using a low-pass filter (mean, Gaussian, etc). Separable Prewitt kernels (for the 3 3 averaging): Separable Sobel kernels (for the 3 3 weighted mean): / 62

32 Edge Detection Using Gradients Sign-alternate kernels: Output images with positive and negative values. Display: by mapping zero gradient to mid-grey level. Positive gradient values appear brighter. Negative gradient values appear darker. If meaningful edges are supposed to be strong, thresholded gradients form a binary edge map. Problem 1: Non-sharp edges for gradual transitions. Problem 2: Strong edges produced by noise. Problem 3: Edge localisation at ridges in the map. 32 / 62

33 Edge Detection Using Gradients Initial image: 2 4/ html 33 / 62

34 Edge Detection Using Gradients Automatically detected edges [3 3 Laplacian]: 2 4/ html 34 / 62

35 Edge Detection Using Gradients Automatically detected edges [ImageJ]: 2 4/ html 35 / 62

36 Edge Detection Using Gradients Automatically detected edges [the Uni of California at Berkeley project]: 2 4/ html 36 / 62

37 Edge Detection Using Gradients Human sketch of the meaningful boundaries: 2 4/ html 37 / 62

38 Rank Filtering: Percentiles A α th -percentile (or centile): a threshold, such that α% of signals from a moving window are below it. The 25 th -percentile is called the first quartile. The 50 th -percentile is the median or second quartile. The 75 th -percentile is the third quartile % 8% % 28% % 48% % 68% % 90 88% 5 12% 7 32% 65 52% 75 72% 90 92% 5 16% 8 36% 70 56% 75 76% 90 96% 6 20% 8 40% 70 60% 80 80% % 38 / 62

39 Rank Filtering: Quantiles Percentiles and quartiles are specific types of quantiles, or signals taken at regular intervals from the cumulative histogram (CH) of the signals. Non-linear α th -percentile filter the output signal q α such that CH(q α ) α 100 CH(q max): q H(q) CH(q) α% q 4 q min = 0; q 25 q 28 = 6; q 50 q 52 = 65; q 75 q 76 = 75; q 100 q max = / 62

40 Rank Filtering: Quantiles, Quartiles, Percentiles CH(q) 100% % 15 50% 10 25% 5 0% Quartiles: q 40 / 62

41 Rank Filtering Rank k of a signal q: the position of this signal in the ordered signal sequence q [1] q [2]... q [K]. Non-linear k-rank filters: the output filter signal is the signal q [k] of the rank k in the ordered signal sequence. q H(q) CH(q) α [k 1] % : α [k] %) k q:min k q:max α [ ) α [k 1], α [k] Rank k q [k] / 62

42 Rank Filters: Minimum Filter (k = 1) 3 3 filter 5 5 filter filter 42 / 62

43 Rank Filters: Median Filter (k = K 2 ) 3 3 filter 5 5 filter filter 43 / 62

44 Rank Filters: Maximum Filter (k = K) 3 3 filter 5 5 filter filter 44 / 62

45 Rank Filtering: Initial Image 45 / 62

46 Rank Filtering: Minimum Filter / 62

47 Rank Filtering: Minimum Filter / 62

48 Rank Filtering: Minimum Filter / 62

49 Rank Filtering: Initial Image 49 / 62

50 Rank Filtering: Median Filter / 62

51 Rank Filtering: Median Filter / 62

52 Rank Filtering: Median Filter / 62

53 Rank Filtering: Initial Image 53 / 62

54 Rank Filtering: Maximum Filter / 62

55 Rank Filtering: Maximum Filter / 62

56 Rank Filtering: Maximum Filter / 62

57 Rank Filtering: Initial Image 57 / 62

58 Rank Filtering: Median Filter / 62

59 Median Filter and Edge Detection 59 / 62

60 Berkeley Edge Detection Project 60 / 62

61 Human Edge Sketch 61 / 62

62 Moving Window Transform: Conclusions The MWT is one of the most-used image processing tools. Processing of signals within a moving, or sliding window W on an input image: g(x, y) = MWT [f(x + ξ, y + η) : (ξ, η) W ]; e.g. g(x, y) = 1 f(x, y) f(x + 1, y) f(x, y + 1) + f(x + 1, y + 1) 2 Linear mean, Gaussian, and Laplacian filters and non-linear gradient and rank filters are in the most common use. Sequential image processing with different filters may help in noise suppression, edge detection, and image segmentation. Generally, the MWT is adaptive, i.e. the window shape and/or signal processing algorithm may vary in accord with the input data. 62 / 62