Texture. Outline. Image representations: spatial and frequency Fourier transform Frequency filtering Oriented pyramids Texture representation

Transcription

1 Texture Outline Image representations: spatial and frequency Fourier transform Frequency filtering Oriented pyramids Texture representation 1

2 Image Representation The standard basis for images is the set of unit vectors corresponding to each pixel. A toy example: Another Image Basis The standard basis is not the only one we can use to describe an image E.g., the Hadamard basis (basis images shown here for 2 x 2 images, where black = +1/2, white = -1/2) For the previous example, we can express the image with these new (normalized) basis vectors as: Coefficients of sum = projection of I onto new basis (dot product) These are the coordinates of the image in Hadamard space We can also say that I has undergone a Hadamard transform H: 2

3 Hadamard Basis for 8 x 8 images Basis images are patterns of rectangular waves with different frequencies courtesy of H. Hel-Or Note that the number of basis images = Image dimensions (w x h) Why Use Another Basis? The standard basis is convenient, but it yields little insight into the structure of the image In contrast, consider our example: The magnitudes of the entries in I H quantify the contribution of each basis image to I Observation: Dominant textures/edges correspond to higher weights on basis images that look like them This makes sense, because weights come from dot product, which is correlation Image compression: Leave out basis images with smallest weights when synthesizing I 3

4 Sinusoidal Bases Binary-valued, rectangular wave pattern of Hadamard basis doesn t capture real image gradients well Idea: Use smoothly-varying sinusoidal patterns at different frequencies, angles for basis images Approximating Arbitrary Functions with Sinusoidal Sums courtesy of H. Hel-Or 4

5 Sine & Cosine Functions: Review courtesy of H. Hel-Or Sine Function: Amplitude & Phase nonlinear operation courtesy of H. Hel-Or 5

6 Combining Sine & Cosine Waves for Linear Shifting Observation: Adding a sine wave to a cosine wave with the same frequency yields a scaled and shifted (co)-sine wave with the same frequency courtesy of H. Hel-Or Using this result, we can define a sinusoidal basis Fourier Basis The Fourier basis uses the following family of complex sinusoidal functions Real (cos) part (u, v) (1, 0) (1, 1) (0, 5) Imaginary (sin) part 6

7 Complex Numbers z = u + iv, where u, v are real and i is imaginary unit equal to r Representations θ Cartesian: (u, v) Polar: re iθ, where r =(u 2 + v 2 ) 1/2 is called the modulus or magnitude, and the angle θ =tan -1 (v/u) (using Euler formula e ix =cosx + i sin x) Real numbers are complex numbers with a 0 imaginary component (u, v) Fourier Basis Functions: Details Values are in range [-1, 1] Frequency: r = (u 2 + v 2 ) 1/2 Direction: θ = tan -1 (v/u) (x, y) = (-1, 1) θ r (1, -1) Basis function for (u, v) = (1, 1) 7

8 Fourier Basis (Imaginary part) v Fourier Transform Given a function f(x, y), its Fourier transform F(u, v) is defined by: F(u, v) is a complex-valued function More compactly: F(u, v) = F(f(x, y)) Invertible: F -1 (F(u, v)) = f(x, y) 8

9 Discrete Fourier Transform Intuitive Meaning of the Fourier Transform F Describe image as sum of periodic functions of different frequencies Coefficients of terms in sum proportional to prevalence in image of features with corresponding frequencies In this sense, we say that F takes an image from the spatial domain to the frequency domain 9

10 Example: Fourier Transform of Image I log( F(I) ) Phase of F(I) courtesy of R. Fisher et al. log of magnitude taken because of wide dynamic range Example: Fourier Transform of a Fourier Basis-like Function f I f F(I f ) courtesy of R. Fisher et al. Biggest weights are on DC component and frequencies corresponding to f and -f. Locations tell us the stripe width & angle 10

11 Example: Fourier Transform of another Fourier Basis-like Function f I f log F(I f ) Thresholded F(I f ) Fourier transform magnitudes DC peak give us angle here (The vertical axes are flipped relative to the Fourier basis shown earlier) Application: Unrotating Text Thresholding as before tells us orientation 11

12 Notes on Fourier Transform The convolution of two functions is the same as the product of their Fourier transforms Given, we have that Helpful way to convolve efficiently (less so for small kernels) because of: Fast Fourier Transform (FFT): Algorithm for computing Fourier transform in time n log n Frequency Filtering I log F(I) courtesy of P. Bourke 12

13 Low- and High-Pass Filters Low-pass I High-pass Masked log F(I) mask result courtesy of P. Bourke Band-Pass Filters courtesy of P. Bourke Masking a specific range of frequencies emphasizes features at that scale 13

14 Laplacian Pyramids as Band-Pass Filters courtesy of Wolfram from Forsyth & Ponce Each level is the difference of a more smoothed and less smoothed image It contains the band of frequencies in between Oriented Pyramids Laplacian pyramid + direction sensitivity from Forsyth & Ponce v 14

15 Oriented Pyramids from Forsyth & Ponce Gabor Filters Localized Fourier transforms : Make each kernel from product of Fourier basis image and Gaussian Frequency Odd Even Larger scale Smaller scale from Forsyth & Ponce 15

16 Texture Representation: Filter Responses Choose a group of filters Edge/Bar filters: Something like Gabor filters at different orientations, scales Spot filters: Center-surround filters like a Gaussian at multiple scales Run filters over image to get a set of response images Example: Filter Responses Filter bank Input image from Forsyth & Ponce Filter responses at one scale 16

17 Texture Similarity based on Response Statistics Collect statistics of responses over an image or sub-image Mean of squared response Mean and variance of squared response Euclidean distance between vectors of response statistics for two images is measure of texture similarity Example: Categorizing Textures Only vertical & horizontal filters in bank; response vector is (v, h) from Forsyth & Ponce squared responses Dark Grey = Horizontal Light Grey = Vertical White = Both Black = Neither 17

18 Application: Texture-based Image Matching Decreasing response vector similarity Query image Ordered list of best matches from Forsyth & Ponce 18