Edges, interpolation, templates. Nuno Vasconcelos ECE Department, UCSD (with thanks to David Forsyth)

Transcription

1 Edges, interpolation, templates Nuno Vasconcelos ECE Department, UCSD (with thanks to David Forsyth)

2 Gradients and edges edges are points of large gradient magnitude edge detection strategy 1. determine magnitude of image gradient 2 2 f f 0, y0) = ( x0, y0) + ( x0, y0) f ( x x y 2. mark points where gradient magnitude is particularly large wrt neighbours (ideally, curves of such points) to compute the derivatives we rely on finite difference kernels 2 large gradient magnitude small gradient magnitude 2

3 Finite difference kernels in two dimensions we have various possible kernels e.g., N 1 =2, N 2 =3, derivative along n 1, (line n 2 =k) (horizontal) n 2 n 1 derivative along n 2, (line n 1 =k) (vertical) derivative along line n 1 =n 2 (diagonal) n 2 n 2 n 1 n 1 3

4 Smoothing reduces noise these are high-frequency filters to avoid noisy derivative estimates: 1. start by low-pass filtering, to suppress noise 2. compute derivative on smoothed image i.e. for a smoothing filter g(n 1,n 2 ) compute ( g x) h * * note that, by associativity of convolution, this is equal to ( h * g) * x i.e. filter the image with the filter h*g g(n 1,n 2 ) h(n 1,n 2 ) smoothed derivatives 4

5 The derivative of a Gaussian let s consider, for example, h( n1, n2) = δ ( n1 + 1, n2) δ ( n1, n2) in which case h * g( n1, n2) = g( n1 + 1, n2) g( n1, n2) is a difference of two Gaussians this is the derivative of a Gaussian (DoG) filter for other definitions of h we have a similar result DoG along n 1 DoG along n 2 5

6 Detecting edge points know how to compute gradient, still three major issues: 1) gradient magnitude at different scales is different (see below); which should we choose? 2) gradient magnitude is large along thick trail; what are the significant points? 3) how do we link the relevant points up into curves? 6

7 Maxima of gradient magnitude let s leave scale open for now maxima of gradient magnitude: the point to remember is that the gradient is perpendicular to the edge we look for the maximum in the direction of the gradient this is called non-maximum suppression two algorithmic issues: at which point is the maximum? where is the next one? 7

8 Non-maximum suppression is there a maximum at q? yes, if value at q is larger than those at both p and r p and r are the pixels in the direction of the gradient that are 1 pixel apart from q typically they do not fall in the pixel grid we need to interpolate, e.g. r = α b + ( 1 α) a a α 1 α b 8

9 Predicting the next edge point assume the marked point is an edge point we construct the tangent to the edge curve (which is normal to the gradient at that point) ( f ( x, y), f ( x, y ) T t ( x, y) = ) y use this to predict the next points (here either r or s). x 9

10 Cleaning up even when gradient is ~ zero, there are maxima due to noise check that maximum value of gradient value is large enough (threshold) once we are following an edge we must avoid gaps due to similarity with background use hysteresis use a high threshold to start edge curves and a low threshold to continue them. 10

11 roblem: various parameters, for all values we tried result was not perfect 11

12 Effects of noise Is there an alternative? recall we followed this path to overcome the noise problem are there other alternatives? 12

13 Solution: smooth first this is what we get with 1 st order derivatives 13

14 Derivative theorem of convolution can we extend this idea? 14

15 Laplacian of Gaussian Consider Laplacian of Gaussian operator where is the edge? zero-crossings of bottom graph 15

16 The Laplacian of Gaussian another way to detect max of first derivative is to look for a zero second derivative 2D analogy is the Laplacian f f f ( x, y) = ( x, y) x y with second-order derivatives, noise is even greater concern smoothing ( x, y) smooth with Gaussian, apply Laplacian this is the same as filtering with a Laplacian of Gaussian filter 2 G σ ( x, y) 16

17 2D edge detection filters Laplacian of Gaussian Gaussian derivative of Gaussian is the Laplacian operator: 17

18 The Laplacian of Gaussian this is very close to what the early stages of the brain seem to be doing recordings of retinal ganglion cells called centersurround cells two types: on-center off-center 18

19 Edge detection strategy filter with Laplacian of Gaussian detect zero crossings mark the zero points where: there is a sufficiently large derivative, and enough contrast once again we have parameters scale of Gaussian smoothing thresholds once again no set of universal parameters LoG ZD does not seem to be better than the strategy of looking for maxima of gradient magnitude. 19

20 sigma=4 contrast=1 LOG zero crossings contrast=4 sigma=2 20

21 Non-maximum suppression we have seen that to find if q is a maximum we need to know what is the image value at r but this does not fall on the pixel grid this is called interpolation it is a very frequent operation in image processing a α 1 α b 21

22 Interpolation the most obvious application is to improve the resolution image super-resolved note the increased detail, e.g. the reduced artifacts on the lines 22

23 Interpolation but there are many others e.g. the restoration of degraded movies 23

24 Interpolation image synthesis 24

25 Interpolation texture mapping 25

26 Interpolation how does one do this? the simplest method is nearest-neighbor interpolation we simply replicate the image intensity (or color) of the closest pixel e.g. in this case, because the desired location p is closest to (x,y+1) we make I( p) = I( x, y + 1) this is not very good because it generates artifacts one location replicated from one pixel (x,y+1) (x,y) an infinitesimally close neighbor replicated from another p (x+1,y+1) (x+1,y) 26

27 Interpolation much better is bilinear interpolation assume image varies linearly, weight each pixel according to their distance to p let a = p x x, b = p y yand make I( p) = (1 a) b I( x, y + 1) (x,y+1) (x+1,y+1) + (1 + a (1 b) I( x + a b I( x a) (1 b) I( x, y) + 1, y) + 1, y + 1) b (x,y) a p (x+1,y) works much better than nearest neighbor 27

28 Interpolation note that these can be implemented with filtering for nearest neighbors 28

29 Interpolation for bilinear interpolation 29

30 Interpolation and there are obviously many other filters the best method is frequently bi-cubic interpolation 30

31 Interpolation how do the three methods compare? image interpolated with nearest neighbor 31

32 Interpolation how do the three methods compare? image interpolated with bilinear method 32

33 Interpolation how do the three methods compare? image interpolated with bi-cubic method 33

34 Interpolation so, what method should I use? the higher order the filter, the more computation required the gains are diminishing after some point bilinear usually justified over nearest neighbor bi-cubic sometimes worth it, but judge on a case by case basis higher order than cubic is usually not worth it to play with this: the matlab interp2 function implements all the methods plus a spline-based method that we will not get into very good applet at rpolation/index.htm 34

35 Filters as templates applying a filter at some point can be seen as taking a dotproduct between the image and some vector filtering the image is a set of dot products insight filters look like the effects they are intended to find filters find effects they look like 35

36 Positive responses 36

37 37