Announcements. Image Matching! Source & Destination Images. Image Transformation 2/ 3/ 16. Compare a big image to a small image

Transcription

1 2/3/ Announcements PA is due in week Image atching! Leave time to learn OpenCV Think of & implement something creative CS 50 Lecture #5 February 3 rd, 20 2/ 3/ 2 Compare a big image to a small image So far, we have been covering background material Geometric Image Transformations Fourier Analysis Issues: Geometric manipulation Differences in information content Pattern comparison We haven t done any vision yet! OK, lets consider two images... are they similar? 2/ 3/ Image Transformation I(x,y) = I (G [x,y] T) Simple for continuous, infinite images Problematic for discrete, finite images 3 2/ 3/ 4 Source & Destination Images We apply a transformation to a source image to produce a destination image The role of source & destination are not symmetric We need to know where every destination pixel came from in the source image Typically, a non-integer location We do not need to know where every source pixel went It could be off the image, or between pixels, or.

2 2/3/ Basic Transformation Algorithm For every (x,y) in dest { real (u,v) = G - (x,y) pixel p = Interpolate(Src, u, v) dest(x,y) = p } Applying Transformations I assume you can invert a 3x3 matrix So the trick is interpolation. 3 forms: Nearest Neighbor (fast, bad) Bilinear (less fast, good) Bicubic (slowest, best) OpenCV supports all three forms Nearest Neighbor Interpolation Bilinear Interpolation (u, v ) = G - (x, y, ) u = round(u ) v = round(v ) Interpolate(Src, x, y) = Src[u,v] In terms of Fourier Analysis, this is awful in the frequency domain (0,0) (u,0) (,0) (u,v) X (u,v) (0,) (u,) (,) Linearly Interpolate along row Linearly Interpolate along row Bilinear Interpolation (II) Bilinear interpolation is actually a product of two linear interpolations and therefore non-linear Equivalent to a least-squar es approximation if you put a plane through the four neighbori ng points Typical expression: Linear algebraic expression Bicubic Interpolation Product of two cubic interpolations in x, in y Based on a 4x4 grid of neighboring pixels In each dimension, create a cubic curve that exactly interpolates all four points Similar to Bezier curves in graphics Except curve passes through all 4 points 2

3 2/3/ Bicubic Interpolation (II) (-,-) (2,-) The equation of a cubic function is: This can be rewritten as: (u,v) X (-,2) Bicubic Interpolation (III) (2,2) We know the values of f at -,0,,2 Bicubic Interpolation (IV) Therefore: Bicubic Interpolation (V) To interpolate a value:. Interpolate along the four rows 2. Interpolate the results vertically And: Each interpolation is a matrix/vector multiply mults, 2 adds per interpolation 80 mults, 0 adds overall The Interpolation Anti-climax You don t need to implement geometric transformations of interpolati ons OpenCV supports geometric transformations warpaffine applies an affine transformation warpperspective applies a perspective transformation Both give you the option of interpolation technique Nearest Neighbor Bilinear Bicubic The point of the lecture is to know what is happening when you use them Back to this: Issues: Geometric manipulation Differences in information content Pattern comparison 2/ 3/ 8 3

4 2/3/ Will the pixels match? Given a pixel I s (x,y) in the small image And the interpolated pixel I b (G - (x,y)) Will they be the same? Given NN interpolation? Given bilinear interpolation? Given bicubic interpolation? Why or why not? Frequencies & Scale I " x,y = N ) ) F " u,v cos 2πuvx N ;9< 89:57 + isin N Contribution of terms below Nyquist rate Noise created by aliased energy above Nyquist rate +N " x,y 2/3/ 9 2/3/ 20 Frequencies & Scale (cont.) I = G :? x,y = ) ) F = u,v cos 2πuvx ;9< 89: + isin Note that > N, so more frequencies Slightly less aliased noise +N = x, y The difference is I = G :? x, y I " x, y = ) ) F = u,v cos 2πuvx + isin + ;C 57 8C 57 :5 7 + isin + ;C 57 8C : ) ) F = u, v cos 2πuvx N = G :? x, y N " x,y Frequencies between the Nyquist rates 2/3/ 2 2/3/ 22 So to compare across scales Remove frequencies from the higherscaled image Remove all frequencies above the lower Nyquist rate But how? Low-Pass Filtering Approach # Drop high frequency Fourier coefficients. Ø But there is a better way. To low-pass filter an image: ) convert to frequency domain 2) discard all values for u > thresh 3) Convert back to spatial domain Brainflux Fourier Applet 2/3/ 23 2/3/ 24 4

5 2/3/ Convolution We arrive at the fundamental idea of convolution. Slide a mask over an image. At each window position, multiply the mask values by the image value under them. Sum the results for every pixel. Think of this as a sliding dot product Convolution (II) Formally, convolution is often expressed as follows: Ø Of course, we are dealing with finite, discrete functions: 2/3/ 25 2/3/ 2 Convolution Examples Let F = [,2,3,4,5] Let F2 = [,2,,2,] Let G = [-,2,-] Let G2 = [/3,/3,/3] Then F*G=[0,0,0,0,] Then F2*G=[0,2,-2,2,0] Then F*G2=[,2,3,4,3] Then F2*G2=[,4/3,5/3,4/3,] Convolution (III) Why introduce convolution now? Because multiplying two Fourier transforms in the frequency domain is the same as convolving their inverse Fourier transforms in the spatial domain! (trust me) 2/3/ 27 2/3/ 28 Convolution Theorem In mathematics, the convolution theorem states that under suitable conditions the Fourier transform of a convolution is the pointwise product of Fourier transforms. 2/3/ 29 5