Object recognition. Methods for classification and image representation

Size: px

Start display at page:

Download "Object recognition. Methods for classification and image representation"

Estella Perry
5 years ago
Views:

1 Object recognition Methods for classification and image representation

2 Credits Slides by Pete Barnum Slides by FeiFei Li Paul Viola, Michael Jones, Robust Realtime Object Detection, IJCV 04 Navneet Dalal and Bill Triggs, Histograms of Oriented Gradients for Human Detection, CVPR05 Kristen Grauman, Gregory Shakhnarovich, and Trevor Darrell, Virtual Visual Hulls: ExampleBased 3D Shape Inference from Silhouettes S. Lazebnik, C. Schmid, and J. Ponce. Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories. Yoav Freund Robert E. Schapire, A Short Introduction to Boosting

3 Object recognition What is it? Instance Category Something with a tail Where is it? (CC) By Yannic Meyer Localization Segmentation How many are there? (CC) By Paul Godden (CC) By Peter Hellberg

4 Object recognition What is it? Instance Category Something with a tail Where is it? Localization Segmentation How many are there? (CC) By Dunechaser

5 Face detection??? features classify 1 face 1 not face x F(x) y We slide a window over the image Extract features for each window Classify each window into face/nonface

6 What is a face? Eyes are dark (eyebrowsshadows) Cheeks and forehead are bright. Nose is bright Paul Viola, Michael Jones, Robust Realtime Object Detection, IJCV 04

7 Basic feature extraction x 120 x 357 x 629 x 834 Information type: intensity Sum over: gray and white rectangles Output: graywhite Separate output value for Each type Each scale Each position in the window FEX(im)=x=[x 1,x 2,.,x n ] Paul Viola, Michael Jones, Robust Realtime Object Detection, IJCV 04

8 Face detection? features classify 1 face 1 not face x F(x) y We slide a window over the image Extract features for each window Classify each window into face/nonface

9 Classification w Examples are points in R n Positives are separated from negatives by the hyperplane w y=sign(w T xb)

10 Classification w x R n data points P(x) distribution of the data y(x) true value of y for each x F decision function: y=f(x, θ) θ parameters of F, e.g. θ=(w,b) We want F that makes few mistakes

11 Loss function POSSIBLE CANCER w ABSOLUTELY NO RISK OF CANCER Our decision may have severe implications L(y(x),F(x, θ)) loss function How much we pay for predicting F(x,θ), when the true value is y(x) Classification error: Hinge loss

12 Learning Total loss shows how good a function (F, θ) is: Learning is to find a function to minimize the loss: How can we see all possible x?

13 Datasets Dataset is a finite sample {x i } from P(x) Dataset has labels {(x i,y i )} Datasets today are big to ensure the sampling is fair #images #classes #instances Caltech Pascal VOC LabelMe ???

14 Overfitting A simple dataset. Two models Linear Nonlinear

15 Overfitting Let s get more data. Simple model has better generalization.

16 Overfitting Loss As complexity increases, the model overfits the data Real loss Training loss decreases Real loss increases Training loss We need to penalize model complexity = to regularize Model complexity

17 Overfitting Split the dataset Training set Validation set Test set Use training set to optimize model parameters Use validation test to choose the best model Use test set only to measure the expected loss Loss Stopping point Training set loss Model complexity Test set loss Validation set loss

18 Classification methods K Nearest Neighbors Decision Trees Linear SVMs Kernel SVMs Boosted classifiers

19 K Nearest Neighbors o Memorize all training data Find K closest points to the query The neighbors vote for the label: Vote()=2 Vote( )=1

20 KNearest Neighbors Nearest Neighbors (silhouettes) Kristen Grauman, Gregory Shakhnarovich, and Trevor Darrell, Virtual Visual Hulls: ExampleBased 3D Shape Inference from Silhouettes

21 KNearest Neighbors Silhouettes from other views 3D Visual hull Kristen Grauman, Gregory Shakhnarovich, and Trevor Darrell, Virtual Visual Hulls: ExampleBased 3D Shape Inference from Silhouettes

22 Decision tree V()=8 No X 1 >2 Yes V()=8 o V()=2 V()=8 No X 2 >1 Yes V()=4 V()=0 V()=4 V()=8 V()=2

23 Decision Tree Training V()=57% V()=80% V()=64% V()=80% V()=100% Partition data into pure chunks Find a good rule Split the training data Build left tree Build right tree Count the examples in the leaves to get the votes: V(), V() Stop when Purity is high Data size is small At fixed level

24 Decision trees x 120 x 357 x 629 x 834 Stump: 1 root 2 leaves If x i > a then positive else negative Very simple Weak classifier Paul Viola, Michael Jones, Robust Realtime Object Detection, IJCV 04

25 Support vector machines margin w Simple decision Good classification Good generalization

26 Support vector machines w Support vectors:

27 How do I solve the problem? It s a convex optimization problem Can solve in Matlab (don t) Download from the web SMO: Sequential Minimal Optimization SVMLight LibSVM LibLinear SVMPerf Pegasos

28 Linear SVM for pedestrian detection

29 Slides by Pete Barnum Navneet Dalal and Bill Triggs, Histograms of Oriented Gradients for Human Detection, CVPR05

30 centered diagonal uncentered cubiccorrected Sobel Slides by Pete Barnum Navneet Dalal and Bill Triggs, Histograms of Oriented Gradients for Human Detection, CVPR05

31 Histogram of gradient orientations Orientation Slides by Pete Barnum Navneet Dalal and Bill Triggs, Histograms of Oriented Gradients for Human Detection, CVPR05

32 8 orientations X= 15x7 cells Slides by Pete Barnum Navneet Dalal and Bill Triggs, Histograms of Oriented Gradients for Human Detection, CVPR05

33 pedestrian Slides by Pete Barnum Navneet Dalal and Bill Triggs, Histograms of Oriented Gradients for Human Detection, CVPR05

34 Kernel SVM Decision function is a linear combination of support vectors: Prediction is a dot product: Kernel is a function that computes the dot product of data points in some unknown space: We can compute the decision without knowing the space:

35 Useful kernels Linear! RBF Histogram intersection Pyramid match

36 Histogram intersection 1 Assign to texture cluster Count S. Lazebnik, C. Schmid, and J. Ponce. Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories.

37 (Spatial) Pyramid Match S. Lazebnik, C. Schmid, and J. Ponce. Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories.

38 Boosting Weak classifier Classifier that is slightly better than random guessing Weak learner builds weak classifiers

39 Boosting Start with uniform distribution Iterate: 1. Get a weak classifier f k 2. Compute it s 01 error 3. Take 4. Update distribution Output the final strong classifier Yoav Freund Robert E. Schapire, A Short Introduction to Boosting

40 Face detection? features classify 1 face 1 not face x F(x) y We slide a window over the image Extract features for each window Classify each window into face/nonface

41 Face detection Use haarlike features Use decision stumps as week classifiers Use boosting to build a strong classifier Use sliding window to detect the face X234>1.3 No x120 1 Face x629 x357 Yes 1 Nonface x834

Beyond Bags of features Spatial information & Shape models

Beyond Bags of features Spatial information & Shape models Jana Kosecka Many slides adapted from S. Lazebnik, FeiFei Li, Rob Fergus, and Antonio Torralba Detection, recognition (so far )! Bags of features