Computer Vision for VLFeat and more...

Size: px

Start display at page:

Download "Computer Vision for VLFeat and more..."

Godfrey Ford
5 years ago
Views:

1 Computer Vision for VLFeat and more... Holistic Methods Francisco Escolano, PhD Associate Professor University of Alicante, Spain

2 Contents PCA/Karhunen-Loeve (slides appart) GIST and Spatial Evelope Image Complexity

3 GIST and Spatial Envelopes

4 Discrete Fourier Transform Maps a signal (1D) to a new domain. The domain of frequencies. The inverse mapping returns the original signal: fft and ifft MATLAB commands Converts a sequence x(1)...x(n) of real numbers into a combination of (complex) sinusoids ordered by their frequencies. Euler s formula

continuous transform F(w1, w2). Magnitude and Phase.

5 Image Fourier Transform Maps a gray image (2D) to a domain of x and y frequencies: fft2 and ifft2 MATLAB commands F(p,q) is a sample of the continuous transform F(w1, w2). Magnitude and Phase. q MATLAB: F = fft2(f,256,256); F2 = fftshift(f); imshow(log(abs(f2)),[-1 5]); colormap(jet); colorbar p

6 Image Fourier Transform Binary images (gray=1) Hamming window Reduces boundary effects Amplitude vs Phase: Amplitude spectrum Phase function

7 Image Fourier Transform Amplitude gives unlocalized information about image structure. Represents spatial frequencies spread everywhere. Thus informs us about orientation, smoothness, length and width of the contours forming the image. The squared magnitude of the FT (energy spectrum) gives the distribution of the signal s energy among the different spatial frequencies.

8 Image Fourier Transform Phase represents information relative to local properties of the image. It contains information relative to the form and position of the image components. Windowed FT. Defined around a given (x,y) with a radius associated to the window. Since the energy spectrum captures global things this allows us to localize image properties: Spectrogram Computed at 8x8 overlapped blocks of diameter 64 pixels. Now more localized if the radius is small.

9 PCAs of FTs Energy spectrum can be decomposed in terms of PCA/KL basis and similarly with the spectrogram PCA of global energy spectrum N*N samples for PCA PCA of the spectrogram 64*N*N samples for PCA Rearrange the samples of enegy spectrum or spectrogram in a column vector Ni (training images) << N^2 Compute expectations along the image database and then Cov matrix and eigenvectors Reduce the number of samples of A (log-polar sampling)

10 PCAs of FTs Log-polar sampling: Sample A(fx,fy) with: Gaussians arranged in a log-polar array and calculated by rotating and scaling where f0=0.02, 0.04, 0.08, 0.16 and 0.32 (5 frequencies) and 12 orientations are considered. L<<Ni log-polar transform Compose from solving Eigenvectors Eigenvalues Approximations

11 PCAs of FTs First 6 principal components of the energy spectrum First 6 principal components of the spectrogram. 4x4 blocks used. Each sub-image to the local energy spectrum Redundant representations, spectrogram enough!

orientations per scale (from HF to LF) param.

fc_prefilt = 4; % Computing gist: [gist, param] = LMgist(img, '',

image') subplot(122) showgist(gist, param) title('descriptor')

12 % Load image img = imread('demo2.jpg'); GIST in MATLAB % GIST Parameters: clear param % number of orientations per scale (from HF to LF) param.orientationsperscale = [ ]; param.numberblocks = 4; param.fc_prefilt = 4; % Computing gist: [gist, param] = LMgist(img, '', param); % Visualization figure subplot(121) imshow(img) title('input image') subplot(122) showgist(gist, param) title('descriptor') Re-normalize image demogist #Filters x #Cells From only one image: 512=(8x4)x(4*4) is given by the filter bank BUT the code is ready for multiple images

However we must modify such space so that each dimension has a semantic meaning:

13 Spatial Envelope The PCs of Energy spectrum (ES) and spectrogram (SG) define a feature space into which scene can be projected. However we must modify such space so that each dimension has a semantic meaning: openness, roughness, naturalness, etc. Specific templates must be learnt. Skyscraper=low expansion, little bit of rouhgness and low openness

14 Spatial Envelope Estimation of spatial evelope attributes s from image-based featues (v and w) after PCA/KL can be done with linear regression. Global DST we must learn di Windowed DST

15 Spatial Envelope Learning the d parameters: 1. Training set: (link v, w with s values) 2. Assume a linear model: 3. Solve by least squares: Column Inverse, ill conditioned if NG or NL are too high!

16 Spatial Envelope What are the image features leading to a given spatial envelope propery? Look the templates! Separating positive and negative parts Opponent energy image

17 Spatial Envelope What is the contribution of each spatial location? Look the windowed templates! Localized filters x,y in 8x8 Opponent energy image Use Hanning window

18 Spatial Envelope DST h_(x,y) h+(x,y) WDST(x,y,fx,fy) at 4x4 Localized filters x,y in 8x8 Naturalness: Images (top), DST*global spectrum (middle), opposed image (bottom)

19 DST Openness for Natural (a) and Man-made (b) Spatial Envelope Ruggedness for Natural (c) Expansion for Man-made (d) Roughness for Natural (e) and Man-made (f) h_(x,y) h+(x,y)

20 Spatial Envelope WDST Openness Ruggedness Roughness Natural Man Made

21 Spatial Envelope Ruggedness: deviation of the ground wrt horizon (horizon covered) Roughness: Size of the major components

22 Spatial Envelope Distance between images: WDST=more detail

23 Properties Spatial Envelope Errors

24 To do: false alarms Spatial Envelope Too global! But Scene centered representations predict false alarms

Statistics of Natural Image Categories

Statistics of Natural Image Categories Authors: Antonio Torralba and Aude Oliva Presented by: Sebastian Scherer Experiment Please estimate the average depth from the camera viewpoint to all locations(pixels)