Descriptors for CV. Introduc)on:

Transcription

1 Descriptors for CV Content Introduction 2.Histograms 3.HOG 4.LBP 5.Haar Wavelets 6.Video based descriptor 7.How to compare descriptors 8.BoW paradigm Color RGB histogram Introduc)on: Image descriptors - Descriptor: a high level representation of an image or video. - Descriptor: a vector - Dense and sparse descriptors - Useful for: - geometric based matching - image retrieval, - object recognition and categorization,

2 Color HS+V histogram Histogram of oriented gradients (HOG) HS V Histogram of oriented gradients (HOG) Combining histograms

3 LBP: local binary pattern LBP: local binary pattern Haar Wavelets Haar Wavelets

4 Video based descriptors (HOG/HOF) Bag of Words (bag of features models) Origin 1: texture recogni)on Bag of Words (bag of features models) Origin 1: texture recogni)on Bag of Words (bag of features models) Origin 2: text analysis (Frequency of words of a dic)onary, Salton & McGill (1983))

5 Bag of Words Bag of Words 1. extract features 2. learn visual vocabulary Bag of Words 3. represent images by frequencies of «visual words» Given an image to classify: 1. Extract features 2. Quantize features using visual vocabulary 3. Represent images by frequencies of visual words 4. Estimate the class using a previously learn classifier (eg. SVM, KNN, AdaBoost, )

6 1. Feature extraction 1. Feature extraction Regular grid Vogel & Schiele, 2003 Fei-Fei & Perona, 2005 Regular grid Vogel & Schiele, 2003 Fei-Fei & Perona, 2005 Interest point detector Csurka et al Fei-Fei & Perona, 2005 Sivic et al Feature extraction 1. Feature extraction Slide credit: Josef Sivic Slide credit: Josef Sivic 24 24

7 Slide credit: Josef Sivic 25 Slide credit: Josef Sivic K-Means Clustering The codebook is used for quantizing features A vector quantizer takes a feature vector and maps it to the index of the nearest codevector in a codebook Codebook = visual vocabulary Codevector = visual word Slide credit: Josef Sivic

8 exemple of visual vocabulary The codebook is used for quantizing features A vector quantizer takes a feature vector and maps it to the index of the nearest codevector in a codebook Codebook = visual vocabulary Codevector = visual word Fei-Fei et al exemple of visual words 3. Image representation exemple of codewords Sivic et al

9 4. Image classification codewords space Minskowski-form distance: CHI2 distance: Codewords Codeword space We need a to define a distance between codewords We need a to define a distance between codewords

10 Kullback-Leiber divergence Battacharyya distance We need a to define a distance between codewords KNN: K- Nearest Neighbors linear classifiers

11 Results from Pascal VOC Challenge 2010 and much more! SVM, AR Applications AR Applications

12 AR Applications AR Applications