FOURIER TRANSFORM GABOR FILTERS. and some textons

Size: px

Start display at page:

Download "FOURIER TRANSFORM GABOR FILTERS. and some textons"

Myles Richards
5 years ago
Views:

1 FOURIER TRANSFORM GABOR FILTERS and some textons

2 Thank you for the slides. They come mostly from the following sources Alexei Efros CMU Martial Hebert CMU

3 Image sub-sampling 1/8 1/4 Throw away every other row and column to create a 1/2 size image - called image sub-sampling Slide by Steve Seitz Image sub-sampling 1/2 1/4 (2x zoom) 1/8 (4x zoom) Why does this look so crufty? Slide by Steve Seitz 2

4 Even worse for synthetic images Slide by Steve Seitz Really bad in video Slide by Paul Heckbert 3

5 Alias: n., an assumed name Input signal: Picket fence receding Into the distance will produce aliasing Matlab output: WHY? x = 0:.05:5; imagesc(sin((2.^x).*x)) Aj-aj-aj: Alias! Not enough samples Aliasing occurs when your sampling rate is not high enough to capture the amount of detail in your image Can give you the wrong signal/image an alias Where can it happen in graphics? During image synthesis: sampling continous singal into discrete signal e.g. ray tracing, line drawing, function plotting, etc. During image processing: resampling discrete signal at a different rate e.g. Image warping, zooming in, zooming out, etc. To do sampling right, need to understand the structure of your signal/image Enter Monsieur Fourier 4

6 Questions How do discrete images differ from continuous images? How do we avoid aliasing while sampling? Sampling an image Examples of GOOD sampling 10

7 Undersampling Examples of BAD sampling -> Aliasing Constructing a pyramid by taking every second pixel leads to layers that badly misrepresent the top layer 11

8 Fourier Transform We want to understand the frequency ω of our signal. So, let s reparametrize the signal by ω instead of x: f(x) Fourier Transform F(ω) For every ω from 0 to inf, F(ω) holds the amplitude A and phase φ of the corresponding sine How can F hold both? Complex number trick! F ( ω) = R( ω) + ii( ω) 2 2 I( ω) A = ± R( ω) + I( ω) φ = tan 1 R( ω) We can always go back: F(ω) Inverse Fourier Transform Asin(ωx + φ) f(x) Frequency Spectra Usually, amplitude is more interesting than phase: 6

9 Extension to 2D in Matlab, check out: imagesc(log(abs(fftshift(fft2(im))))); 9

10 This is the magnitude transform of the cheetah pic This is the phase transform of the cheetah pic 10

11 This is the magnitude transform of the zebra pic 11

This is the phase transform of the zebra pic Curious things about FT on images The magnitude spectra of all natural images quite similar Heavy on low-frequencies, falling off in high frequences

12 This is the phase transform of the zebra pic Curious things about FT on images The magnitude spectra of all natural images quite similar Heavy on low-frequencies, falling off in high frequences Will any image be like that, or is it a property of the world we live in? Most information in the image is carried in the phase, not the amplitude Seems to be a fact of life Not quite clear why 12

13 Reconstruction with zebra phase, cheetah magnitude Reconstruction with cheetah phase, zebra magnitude 13

14 Questions How can we represent images at multiple scales? Sampling without smoothing. Top row shows the images, sampled at every second pixel to get the next; bottom row shows the magnitude spectrum of these images. 14

15 Sampling with smoothing. Top row shows the images. We get the next image by smoothing the image with a Gaussian with sigma 1 pixel, then sampling at every second pixel to get the next; bottom row shows the magnitude spectrum of these images. Representation of scale 15

Representation of scale The Gaussian pyramid Smooth with gaussians, because a gaussian*gaussian=another gaussian Synthesis smooth and subsample Analysis Start with the top image (coarse) and move to

16 Representation of scale The Gaussian pyramid Smooth with gaussians, because a gaussian*gaussian=another gaussian Synthesis smooth and subsample Analysis Start with the top image (coarse) and move to lower (fine) image layers Applications Search for correspondence look at coarse scales, then refine with finer scales Lowers computational cost Edge tracking a good edge at a fine scale has parents at a coarser scale 16

an approximation of the Laplacian at scale number k Laplacian is a band-pass filter: Both

17 Given input I Laplacian Pyramids Construct Gaussian pyramid I G 1,..,I G n Take the difference between consecutive levels: I L k = I L k I L k-1 Image I L k is an approximation of the Laplacian at scale number k Laplacian is a band-pass filter: Both high frequencies (edges and noise) and low frequencies (slow variations of intensity across the image) Laplacian Pyramids 17

18 Odd Gabor filter First Derivative Even Gabor filter Laplacian 22

19 Example: Texture Classification Profound observation: Cows and buildings don t look the same! Basic idea: Model the distribution of texture over the image (or over a region) and classify in different classes based on the texture models learned from training examples. Cow Building The Concept of Texton Clustering Multiple training images of the same texture Texton Dictionary Filter responses over a bank of filters 28

20 Example of Filter Banks Isotropic Gabor S Gaussian derivatives at different scales and orientations LM MR8 Example Textons (LM) 29

21 Example: Visual words in photographs Images Word maps Visual dictionary Modeling Texton Distributions Training image Filter Responses Texton Map Model = Histogram of textons in the image 30

22 Classification Compare with Stored Models from Training Images Models of Plastic Input Image (or Region of an Input Image) Model Models of Grass Example Classification Input Region Textons 32

23 The original, e.g. Brodatz, CURet textures are relative simple. For example, the CUReT database was better when done with just raw intensities of an NxN (3,5,7) neighbourhood. So the textons may only have limited applications M. Varma and A. Zisserman.Texture classification: Are filter banks necessary? In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, volume 2, pages , Madison, Wisconsin, June In the reading list.

Frequency analysis, pyramids, texture analysis, applications (face detection, category recognition)

Frequency analysis, pyramids, texture analysis, applications (face detection, category recognition) Outline Measuring frequencies in images: Definitions, properties Sampling issues Relation with Gaussian