Edges, interpolation, templates. Nuno Vasconcelos ECE Department, UCSD (with thanks to David Forsyth)

Transcription

1 Edges, interpolation, templates Nuno Vasconcelos ECE Department, UCSD (with thanks to David Forsyth)

2 Edge detection edge detection has many applications in image processing an edge detector implements the following steps: compute gradient magnitude f f 0, y0) = ( x0, y0) + ( x0, y0) f ( x x y thin and follow edge points find locations of maximum gradient magnitude follow these maxima to form contours discard points that are not maxima declare maxima as edges

3 Derivatives to compute the derivatives f f f x ( x, y ), f y ( x, y ) = ( x 0, y 0), ( x 0, y 0) x y we rely on a sequence of smoothing with a Gaussian (to eliminate noise) convolution with difference filter f x : f y : ( ) n n n n 3

4 Derivatives accomplished in a single step by convolving image with two derivative of a Gaussian (DoG) filters h x h y where ( n, n) = g ( n +, n) g ( n, n) ( n, n ) = g ( n, n + ) g ( n, n ) g ( n, n ) πσ n + n exp σ = DoG along n DoG along n 4

5 Non-maximum suppression is there a maximum at q? yes, if value at q is larger than those at both p and r p and r are the pixels in the direction of the gradient that are pixel apart from q typically they do not fall in the pixel grid we need to interpolate, e.g. r = α b + ( α) a a α α b 5

6 Predicting the next edge point assume the marked point is an edge point we construct the tangent to the edge curve (which is normal to the gradient at that point) ( f ( x, y), f ( x, y ) T t ( x, y) = ) y use this to predict the next points (here either r or s). x 6

7 Cleaning up even when gradient is ~ zero, there are maxima due to noise check that maximum value of gradient value is large enough (threshold) once we are following an edge we must avoid gaps due to similarity with background use hysteresis use a high threshold to start edge curves and a low threshold to continue them. 7

8 roblem: various parameters, for all values we tried result was not perfect 8

9 Effects of noise Is there an alternative? recall we followed this path to overcome the noise problem are there other alternatives? 9

10 Solution: smooth first this is what we get with st order derivatives 0

11 Derivative theorem of convolution can we extend this idea?

12 Laplacian of Gaussian Consider Laplacian of Gaussian operator where is the edge? zero-crossings of bottom graph

13 The Laplacian of Gaussian another way to detect max of first derivative is to look for a zero second derivative D analogy is the Laplacian f f f ( x, y) = ( x, y) + x y with second-order derivatives, noise is even greater concern smoothing ( x, y) smooth with Gaussian, apply Laplacian this is the same as filtering with a Laplacian of Gaussian filter G σ ( x, y) 3

14 D edge detection filters Laplacian of Gaussian Gaussian derivative of Gaussian is the Laplacian operator: 4

15 The Laplacian of Gaussian this is very close to what the early stages of the brain seem to be doing recordings of retinal ganglion cells called centersurround cells two types: on-center off-center 5

16 Edge detection strategy filter with Laplacian of Gaussian detect zero crossings mark the zero points where: there is a sufficiently large derivative, and enough contrast once again we have parameters scale of Gaussian smoothing thresholds once again no set of universal parameters LoG ZD does not seem to be better than the strategy of looking for maxima of gradient magnitude. 6

17 sigma=4 contrast= LOG zero crossings contrast=4 sigma= 7

18 Non-maximum suppression we have seen that to find if q is a maximum we need to know what is the image value at r but this does not fall on the pixel grid this is called interpolation it is a very frequent operation in image processing a α α b 8

19 Interpolation the most obvious application is to improve the resolution image super-resolved note the increased detail, e.g. the reduced artifacts on the lines 9

20 Interpolation but there are many others e.g. the restoration of degraded movies 0

21 Interpolation image synthesis

22 Interpolation texture mapping

23 Interpolation how does one do this? the simplest method is nearest-neighbor interpolation we simply replicate the image intensity (or color) of the closest pixel e.g. in this case, because the desired location p is closest to (x,y+) we make I( p) = I( x, y + ) this is not very good because it generates artifacts one location replicated from one pixel (x,y+) (x,y) an infinitesimally close neighbor replicated from another p (x+,y+) (x+,y) 3

24 Interpolation much better is bilinear interpolation assume image varies linearly, weight each pixel according to their distance to p let a = p x x, b = p y yand make I( p) = ( a) b I( x, y + ) (x,y+) (x+,y+) + ( + a ( b) I( x + a b I( x a) ( b) I( x, y) +, y) +, y + ) b (x,y) a p (x+,y) works much better than nearest neighbor 4

25 Interpolation note that these can be implemented with filtering for nearest neighbors 5

26 Interpolation for bilinear interpolation 6

27 Interpolation and there are obviously many other filters the best method is frequently bi-cubic interpolation 7

28 Interpolation how do the three methods compare? image interpolated with nearest neighbor 8

29 Interpolation how do the three methods compare? image interpolated with bilinear method 9

30 Interpolation how do the three methods compare? image interpolated with bi-cubic method 30

31 Interpolation so, what method should I use? the higher order the filter, the more computation required the gains are diminishing after some point bilinear usually justified over nearest neighbor bi-cubic sometimes worth it, but judge on a case by case basis higher order than cubic is usually not worth it to play with this: the matlab interp function implements all the methods plus a spline-based method that we will not get into very good applet at rpolation/index.htm 3

32 Filters as templates applying a filter at some point can be seen as taking a dotproduct between the image and some vector filtering the image is a set of dot products insight filters look like the effects they are intended to find filters find effects they look like 3

33 Positive responses 33

34 34

35 The z transform once again, it is a straightforward extension of D Definition: the z-transform of the sequence x[n,n ] is X ( z the region of the (z,z ) plane where this sum is finite is called the Region of Convergence (ROC) it turns out that:, z) x[ n, n] n n = in D the ROC is much more complicated than in D while in D the ROC is bounded by poles (0D subspace of the D complex plane) in D is bounded by pole surfaces (D subspaces of the 4D space of two complex variables) z n z n 35

36 The z-transform computation is also much harder: as you might remember from D most useful tool in computing z-transforms is polynomial factorization z-transform is a ratio of two polynomials Y ( z) = N( z) D( z) we factor in to a sum of low order terms, e.g. Y ( z) = i a and then invert each of the terms to get y[n] iz 36

37 z-transform in D we only have one of two situations ) the sequence is separable, in which case everything reduces to the D case x[ n, n ] = x [ n ] x [ n ] X ( z, z ) = X ( z ) X ( z ) ROC : z ROC of X ( z ) and z ROC of X ( z ) the proof is identical to that of the DSFT ) the signal is not separable here our polynomials are of the form z m z n and, in general, it is not know how to factor them we can solve only if sequence is simple enough that we can do it by inspection (from the definition of the z-transform) 37

38 38 Example consider the sequence the z-transform is ], [ ], [ n n u b a n n x n n = ( ) ( ) ( ) ( ) b z a z bz az az az az az z z X n n n n n n n n > > = = = = = = = ,, ), ( ROC z z a b

39 Sampling in D consider an analog signal x c (t,t ) and let its analog Fourier transform be X c (Ω,Ω ) we use capital Ω to emphasize that this is analog frequency sample with period (T,T ) to obtain a discrete-space signal x [ n, n ] = x c ( t, t ) t t = n T = nt ; 39

40 Sampling in D relationship between the Discrete-Space FT of x[n,n ] and the FT of x c (t,t ) is simple extension of D result X ( ω, ω ) = T T DSFT of x[n,n ] FT of x c (ω,ω ) discrete spectrum analog spectrum Discrete Space spectrum is sum of replicas of analog spectrum in the base replica the analog frequency Ω (Ω ) is mapped into the digital frequency Ω T (Ω T ) discrete spectrum has periodicity (π,π) r = r = ω πr ω πr X, c T T 40

41 For example Ω Ω Ω ω π Ω Ω ω π Ω T Ω T Ω' α = Ω' T Ω'' β = Ω'' T π Ω T π ω no aliasing if π Ω T Ω' T π Ω' T Ω'' T π Ω' T π ω T π / Ω' T π / Ω'' 4

42 Aliasing the frequency (Ω /π,ω /π) is the critical sampling frequency below it we have aliasing ω this is just like the D case, but now there are more possibilities for overlap ω 4

43 Reconstruction if there is no aliasing we can recover the signal in a way similar to the D case y ( t, t ) = c ( t n T ) T T x [ n, n ] π π n = = ( ) n t nt T T note: in D there are many more possibilities than in D sin sin e.g. the sampling grid does not have to be rectangular, e.g. hexagonal sampling when T = T /sqrt(3) and x c ( t, t ) t = ; = [, ] = nt t x n n nt 0 in practice, however, one usually adopts the rectangular grid π π ( t n T ) ( t n T ) n, n both even or odd otherwise 43

44 a sequence of images obtained by downsampling without any filtering aliasing: the lowfrequency parts are replicated throughout the low-res image 44

45 The role of smoothing none some a lot too little leads to aliasing too much leads to loss of information 45

46 Aliasing in video video frames are the result of temporal sampling fast moving objects are above the critical frequency above a certain speed they are aliased and appear to move backwards this was common in old western movies and become known as the wagon wheel effect here is an example: super-resolution increases the frame rate and eliminates aliasing from Space-Time Resolution in Video by E. Shechtman, Y. Caspi and M. Irani (PAMI 005). 46

47 47