Syllabus. 1. Visual classification Intro 2. SVM 3. Datasets and evaluation 4. Shallow / Deep architectures

Transcription

1 Syllabus 1. Visual classification Intro 2. SVM 3. Datasets and evaluation 4. Shallow / Deep architectures

2 Image classification How to define a category? Bicycle Paintings with women Portraits Concepts, semantics, ontologies ÞComputer Vision meets Machine Learning

3 Image classification based on BoW Training set BoW histogram vector space bird Decision boundary dog Learn a classification model to determine the decision boundary

4 Image/video datasets for training/testing Choice of the categories (objects, concepts) Number of categories Number of images per category Hierarchical structure? Mono-label/multi-labels Pre-processing Color, resolution, centered

5 Image/video datasets for training/testing Training Set A Training classifiers on A Testing on B: error evaluation A and B disjoints! Testing Set B

6 ImageNet dataset! Large Scale Visual Recognition Challenge (ILSVRC) 1,2 Million images, 1000 classes Paper: ImageNet: A Large-Scale Hierarchical Image Database, Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li and Li Fei-Fei, CVPR 2009

7 } Based on WordNet } Each node is depicted by images } A knowledge ontology } Taxonomy } Partonomy ImageNet } Website:

8 Constructing ImageNet 2-step process Step 1 : Collect candidate images Via the Internet Step 2 : Clean up candidate Images by humans

9 Step 1: Collect Candidate Images from the Internet } For each synset, the queries are the set of WordNet synonyms } Accuracy of Internet Image search results: 10 % (in 2010) - For clean images, needs 10K images } Query expansion - Synonyms: German police dog, German shepherd dog - Appending words form ancestors: sheepdog, dog } Multiple Languages - Italian, Dutch, Spanish, Chinese } More engines: Yahoo!, flickr, Google } Parallel downloading

10 Step 2: Clean up the candidate Images by humans Rely on humans to verify each candidate image collected for a given synset Amazon Mechanical Turk (AMT) Present the users with a set of candidate images and the definition of the target synset let users select the best match ones

11 Users Enhancement - Provide wiki and links for definitions - Make sure workers read the definition - Definition quiz Human users make mistakes (3% wrong annotations) Not all users follow the instructions Ensure Accuracy Users do not always agree with each other Subtle or confusing synsets, e.g. Burmese cat Quality Control System Randomly sample an initial subset of images to users Have multiple users independently label same image Obtain a confidence score table, indicating the probability of an image being a good image given the user votes Different categories requires different levels of consensus Proceed until a pre-determined confidence score threshold reached

12 Properties of ImageNet Accuracy: clean dataset at all level Diversity: variable appearances, positions, view points, poses, background clutter, occlusions. Limitations: Crawling bias, Improve algorithm: PageRank AMT: hierarchical users based on their ability Only one tag per image

13 Scientific challenges

14 Scientific challenges

15 Classification pipeline Deep CNN End-to-end learning

16 Binary Classification Φ Classifier f / f(φ(x)) = Face/notFace x? => knn, linear classifiers

17 Classification pipeline Questions about: Linear/ non linear decision function Learning from examples: training/validation/test sets Unsupervised / Supervised learning (Kmeans,PCA /LDA) Learning binary / Multiclass classifiers Theory: Risk minimization, empirical risk Minimizing classifier criteria Instantiations: (lin/kernel) SVM classifiers, deep architectures

18 Syllabus 1. Visual classification Intro 2. SVM 3. Datasets and evaluation 4. Shallow / Deep architectures

19 Linear and Kernel classifiers Notations: Image/Patterns x 2 X : function transforming the patterns into feature vectors (x) <, > dot product in the feature space endowed by ( ) Classes y = ±1 Early kernel classifiers derived from the perceptron [Rosenblatt58]: taking the sign of a linear discriminant function: f(x) =< w, (x) > +b Classifiers called -machines

20 Linear and Kernel classifiers Question: how to find/estimate f? Feature function usually hand-chosen for each problem Several for image processing like BoW w and b: parameters to be determined f(x) = w, (x) + b Learning algorithm on a set of training examples: A =(x 1,y 1 ) (x n,y n )

21 Which hyperplane? w? b?

22 SVM SVM optimization: maximizing the margin between + and - Def.: Margin = distance between the hyperplanes f(x) = 1and f(x) = 1 (dashed lines in Figure). Intuitively, a classifier with a larger margin is more robust to fluctuations Hard Margin, details in course... Final expression for the Hard Margin SVM optimization: min w,b P (w, b) = 1 2 kwk2 with 8 i y i f(x i ) 1

23 SVM Hard Margin: OK if data are linearly separated Otherwise: noisy data (in red) disrupt the optim. Solution: Soft SVM 23

24 SVM: Soft Margin Introducing the slack variables i, one usually gets rid of the inconvenient max of the loss and rewrite the problem as min w,b P (w, b) = 1 2 kwk2 + C nx i=1 i with 8 i yi f(x i ) 1 i 8 i i 0 For very large values of the hyper-parameter C, Hard Margin case: Minimization of w (ie margin maximization) under the constraint that all training examples are correctly classified with a loss equal to zero. Smaller values of C relax this constraint: Soft Margin case SVMs that produces markedly better results on noisy problems. 24

25 SVM learning scheme Equivalently, minimizing the following objective function in feature space with the hinge loss function: `(y i f(x i )) = max (0, 1 y i f(x i )) Regularization min w,b P (w, b) = 1 2 kwk2 + C nx `(y i f(x i )) i=1 Data fitting Margin Maximization Constraint satisfaction

26 Learning SVMs: Primal/Dual

27 SVM optimization Standard equivalent formulation without enforcing i to be positive: Optimization on coe cients i of the SVM kernel expansion f(x) = P n i=1 ik(x, x i )+b by defining the dual objective function: D( ) = X i i y i 1 2 X i j k(x i,x j ) i,j and solving the SVM dual Quadratic Programming (QP) problem. 8 >< max D( ) with >: P i i =0 A i apple i apple B i A i =min(0,cy i ) B i = max(0,cy i )

28 Classification pipeline To summarize on SVM :

29 Solving equation: SVM Support Vector Machines (SVM) defined by three incremental steps: 1. [Vapnik63]: linear classifier / separates the training examples with the widest margin => Optimal Hyperplane 29

30 Solving equation: SVM Support Vector Machines (SVM) defined by three incremental steps: 1. [Vapnik63]: linear classifier / separates the training examples with the widest margin =>Optimal Hyperplane 2. [Guyon93] Optimal Hyperplane built in the feature space induced by a kernel function 30

31 Solving equation: SVM Support Vector Machines (SVM) defined by three incremental steps: 1. [Vapnik63]: linear classifier / separates the training examples with the widest margin =>Optimal Hyperplane 2. [Guyon93] Optimal Hyperplane built in the feature space induced by a kernel function 3. [Cortes95] soft version: noisy problems addressed by allowing some examples to violate the margin constraint 31

32 ~ Slide 32 of 47 Image Feature extraction Local descriptors Feature coding Visual codes Pooling Image signature Classification Class label Dense SIFT K-Means Clustering Spatial Pyramids & Pooling SVM Classifier NN Noyaux Fusion

33 Appendix: Solving SVM Min P or Max D => QP (Quadratic programing) family optimization Good news: efficient batch numerical algorithms have been developed to solve the specific SVM QP problem (PhD Labbe hinge loss, convex objective, ) Some strategies (exploiting specif.): Conjugate Gradient method [Vapnik] Sequential Minimal Optimization (SMO) [platt]. In both methods successive searches along well chosen directions Some famous SVM solvers like SVMLight [Joachims] or SVMTorch propose to use decomposition algorithms to define such directions SVMstruct (for structured outputs) State-of-the-art implementation of SMO: [libsvm] => used in tutorials LibLinear bib for primal optim (with MATLAB) 33

34 SMO algo for SVM optimization 1. Set 0 and compute the initial gradient g of D( ) 2. Choose a -violating pair(*) (i, j) Stop if no such pair exists g i g j 3. min, B i i, j A j k ii + k jj 2k ij 4. i i +, j j 5. g s g s (k is k js ) s {1...n} 6. Return to step 2 (*) pairs in +1/-1 with significant diff of gradients A ways to easily satisfy the null sum coeff constraint 34