Metric Learning for Large-Scale Image Classification:
|
|
- Constance Brown
- 5 years ago
- Views:
Transcription
1 Metric Learning for Large-Scale Image Classification: Generalizing to New Classes at Near-Zero Cost Florent Perronnin 1 work published at ECCV 2012 with: Thomas Mensink 1,2 Jakob Verbeek 2 Gabriela Csurka 1 1 Xerox Research Centre Europe, 2 INRIA NIPS BigVision Workshop December 7,
2 Motivation Real-life image datasets are always evolving: new images are added every second new labels, tags, faces and products appear over time for example: Facebook, Flickr, Twitter, Amazon... Need to annotate these items for indexing and retrieval Therefore, we are interested in methods for large-scale visual classification where we can add new images and new classes at near-zero cost on the fly 2
3 Outline 1. Introduction 2. Distance Based Classifiers 3. Metric learning for NCM Classifier 4. Experimental Evaluation 5. Conclusion 3
4 Introduction Recent focus on large-scale image classification ImageNet data set [1] Currently over 14 million images, and 20 thousand classes Standard large-scale classification pipeline: High dim. features: Super Vector [3] & Fisher Vector [4] Linear 1-vs-Rest SVM classifiers [2,3,4] Stochastic Gradient Descent (SGD) training [3,4] In this work, we take features for granted and focus on the learning problem. 1. Deng et al., ImageNet: A large-scale hierarchical image database, CVPR Deng et al., What does classifying 10,000 image categories tell us?, ECCV Lin et al., Large-scale image classification: Fast feature extraction, CVPR Sánchez and Perronnin, High-dimensional signature compression for large-scale image classification, CVPR 11 4
5 Challenges of open-ended datasets 1-vs-Rest + SGD might look ideal for our problem: 1-vs-Rest: classes are trained independently SGD: online algorithm can accomodate new data Still several issues need to be addressed: Given a new sample, feed it to all classifiers? costly and suboptimal [1] How to balance the negatives and positives? How to regularize (and choose the step-size)? We turn to distance-based classifiers. 1. Perronnin et al., Towards good practice in large-scale learning for image classification, CVPR 12 5
6 Outline 1. Introduction 2. Distance Based Classifiers 3. Metric learning for NCM Classifier 4. Experimental Evaluation 5. Conclusion 6
7 Distance Based Classifiers Classify based on the distance between images, or between image and class-representatives: k-nearest Neighbors Nearest Class Mean Classification Trivial addition of new images or new classes Critically depends on the distance function 7
8 k-nearest Neighbor Classifier Assign an image i to the most common class among the k closest images from the training set Very flexible non-linear model Easy to integrate new images Easy to integrate new classes Expensive at test time! 8
9 k-nearest Neighbor Classifier Assign an image i to the most common class among the k closest images from the training set Very flexible non-linear model Easy to integrate new images Easy to integrate new classes Expensive at test time! 8
10 k-nearest Neighbor Classifier Assign an image i to the most common class among the k closest images from the training set Very flexible non-linear model Easy to integrate new images Easy to integrate new classes Expensive at test time! Metric Learning: Large Margin Nearest Neighbors [1] 1. Weinberger et al., Distance Metric Learning for LMNN Classification, NIPS 06 8
11 Nearest Class Mean Classifier Assign an image i to the class with the closest class mean µ c = 1 N c i:y i =c x i c = argmin d(x, µ c ) c Very fast at test time: linear model Easy to integrate new images Easy to integrate new classes Class only represented with mean, not flexible enough? 9
12 Nearest Class Mean Classifier Assign an image i to the class with the closest class mean µ c = 1 N c i:y i =c x i c = argmin d(x, µ c ) c Very fast at test time: linear model Easy to integrate new images Easy to integrate new classes Class only represented with mean, not flexible enough? 9
13 Nearest Class Mean Classifier Assign an image i to the class with the closest class mean µ c = 1 N c i:y i =c x i c = argmin d(x, µ c ) c Very fast at test time: linear model Easy to integrate new images Easy to integrate new classes Class only represented with mean, not flexible enough? We introduce metric learning 9
14 Outline 1. Introduction 2. Distance Based Classifiers 3. Metric learning for NCM Classifier 4. Experimental Evaluation 5. Conclusion 10
15 Mahalanobis Distance Learning d(x, x ) = (x x ) M(x x ) d W (x, x ) = W x W x M = I Euclidean distance Likely to be suboptimal 2. M : D D Full Mahalanobis distance Huge number of parameters for large D Expensive to compute distances in O ( D 2) 3. M = W W Low-Rank Projection W : m D Controllable number of parameters: m D Allows for compression of images to only m dimensions Cheap computation of distances in O ( m 2) 11
16 Mahalanobis Distance Learning d(x, x ) = (x x ) M(x x ) d W (x, x ) = W x W x M = I Euclidean distance Likely to be suboptimal 2. M : D D Full Mahalanobis distance Huge number of parameters for large D Expensive to compute distances in O ( D 2) 3. M = W W Low-Rank Projection W : m D Controllable number of parameters: m D Allows for compression of images to only m dimensions Cheap computation of distances in O ( m 2) 11
17 Mahalanobis Distance Learning d(x, x ) = (x x ) M(x x ) d W (x, x ) = W x W x M = I Euclidean distance Likely to be suboptimal 2. M : D D Full Mahalanobis distance Huge number of parameters for large D Expensive to compute distances in O ( D 2) 3. M = W W Low-Rank Projection W : m D Controllable number of parameters: m D Allows for compression of images to only m dimensions Cheap computation of distances in O ( m 2) 11
18 NCM Metric Learning (NCMML) Probabilistic formulation using the soft-min function: p(c x) = exp d W (x, µ c ) C c =1 exp d W (x, µ c ) Corresponds to class posterior in generative model: p(x c) = N (x; µ c, Σ), with shared covariance matrix Crucial point: parameters W and {µ c, c = 1,..., C} can be learned independently on different data subsets. 12
19 NCM Metric Learning (NCMML) Discriminative maximum likelihood training: We maximize with respect to W : L(W ) = N ln p(y i x i ) i=1 Implicit regularization through the rank of W Stochastic Gradient Descent (SGD): at time t Pick a random sample (x t, y t ) Update: W (t) = W (t 1) + η t W =W (t 1) ln p(y t x t ) mini-batch more efficient 13
20 Illustration of Learned Distances 14
21 Illustration of Learned Distances 14
22 Relationship to FDA Three non-linearly separable classes 15
23 Relationship to FDA Fisher Discriminant Analysis: maximizes variance between all class means 15
24 Relationship to FDA NCMML: maximizes variance between nearby class means 15
25 Relation to other linear classifiers f c (x) = b c + w c x Linear SVM Learn {b c, w c } per class WSABIE [1] w c = v c W W R d D Learn {v c } per class and shared W Nearest Class Mean b c = W µ c 2 2, w c = 2 ( µ c W W ) Learn shared W 1. Weston et al., Scaling up to large vocabulary image annotation, IJCAI 11 16
26 Outline 1. Introduction 2. Distance Based Classifiers 3. Metric learning for NCM Classifier 4. Experimental Evaluation 5. Conclusion 17
27 Experimental Evaluation Data sets: ILSVRC 10: classes = 1,000, images = 1.2M training + 50K validation + 150K test INET10K: classes 10K, images = 4.5M training + 50K validation + 4.5M test Features: 4K and 64K dimensional Fisher Vectors [1] PQ Compression on 64K features [2] 1. Perronnin et al., Improving the Fisher kernel for image classification, ECCV Jégou et al., Product quantization for nearest neighbor search, PAMI 11 18
28 Evaluation: ILSVRC 10 (Top 5 acc.) k-nn & NCM improve with metric learning NCM outperforms more flexible k-nn 4K Fisher Vectors Projection dimensionality l 2 k-nn, LMNN [1] - dynamic NCM, learned metric Weinberger et al., Distance Metric Learning for LMNN Classification, NIPS 06 19
29 Evaluation: ILSVRC 10 (Top 5 acc.) k-nn & NCM improve with metric learning NCM outperforms more flexible k-nn NCM competitive with SVM and WSABIE 4K Fisher Vectors Projection dimensionality l 2 k-nn, LMNN [1] - dynamic NCM, learned metric WSABIE [2] Baseline: 1-vs-Rest SVM Weinberger et al., Distance Metric Learning for LMNN Classification, NIPS Weston et al., Scaling up to large vocabulary image annotation, IJCAI 11 19
30 Generalization on INET10K (Top 1 acc.) Nearest Class Mean Classifier Compute means of 10K classes, in about 1 CPU hour Re-use metric learned on ILSVRC 10 1-vs-Rest SVM baseline Train 10K SVM classifiers, in about 280 CPU days 20
31 Generalization on INET10K (Top 1 acc.) Nearest Class Mean Classifier Compute means of 10K classes, in about 1 CPU hour Re-use metric learned on ILSVRC 10 1-vs-Rest SVM baseline Train 10K SVM classifiers, in about 280 CPU days Feat. dim. 64K 21K 128K 60K Method NCM SVM SVM [1] SVM [2] DL [3] Flat top Deng et al., What does classifying 10,000 image categories tell us?, ECCV Perronnin et al., Good practice in large-scale image classification, CVPR Le et al., Building high-level features using large scale unsupervised learning, ICML 12 20
32 Transfer Learning - Zero-Shot Prior Use ImageNet class hiearchy to estimate a mean, [1] Internal nodes Training nodes New class 1. Rohrbach et al., Evaluating knowledge transfer and zero-shot learning in a large-scale setting, CVPR 11 21
33 Transfer Learning - Zero-Shot Prior Use ImageNet class hiearchy to estimate a mean, [1] Internal nodes Training nodes New class 1. Rohrbach et al., Evaluating knowledge transfer and zero-shot learning in a large-scale setting, CVPR 11 21
34 Transfer Learning - Zero-Shot Prior Use ImageNet class hiearchy to estimate a mean, [1] Internal nodes Training nodes New class 1. Rohrbach et al., Evaluating knowledge transfer and zero-shot learning in a large-scale setting, CVPR 11 21
35 Transfer Learning - Results ILSVRC 10 Step 1 Metric learning on 800 classes Step 2 Estimate means for remaining 200 for evaluation: Data mean (Maximum Likelihood) Zero-Shot prior + data mean (Maximum a Posteriori) 80 Top-5 accuracy Number of samples per class 22
36 Outline 1. Introduction 2. Distance Based Classifiers 3. Metric learning for NCM Classifier 4. Experimental Evaluation 5. Conclusion 23
37 Conclusion Nearest Class Mean (NCM) Classification We proposed NCM Metric Learning Outperforms k-nn, on par with SVM and WSABIE Advantages of NCM over alternatives: Allows adding new images and classes at near zero cost Shows competitive results on unseen classes Can benefit from class priors for small sample sizes Further improvements Extension using multiple class centroids [1] 1. Mensink et al., Large Scale Metric Learning for Distance-Based Image Classification, Tech-report,
38 Metric Learning for Large-Scale Image Classification: Generalizing to New Classes at Near-Zero Cost Florent Perronnin 1 work published at ECCV 2012 with: Thomas Mensink 1,2 Jakob Verbeek 2 Gabriela Csurka 1 1 Xerox Research Centre Europe, 2 INRIA NIPS BigVision Workshop December 7,
Metric Learning for Large Scale Image Classification:
Metric Learning for Large Scale Image Classification: Generalizing to New Classes at Near-Zero Cost Thomas Mensink 1,2 Jakob Verbeek 2 Florent Perronnin 1 Gabriela Csurka 1 1 TVPA - Xerox Research Centre
More informationDistance-Based Image Classification: Generalizing to new classes at near-zero cost
Distance-Based Image Classification: Generalizing to new classes at near-zero cost Thomas Mensink, Jakob Verbeek, Florent Perronnin, Gabriela Csurka To cite this version: Thomas Mensink, Jakob Verbeek,
More informationImproved Fisher Vector for Large Scale Image Classification XRCE's participation for ILSVRC
Improved Fisher Vector for Large Scale Image Classification XRCE's participation for ILSVRC Jorge Sánchez, Florent Perronnin and Thomas Mensink Xerox Research Centre Europe (XRCE) Overview Fisher Vector
More informationon learned visual embedding patrick pérez Allegro Workshop Inria Rhônes-Alpes 22 July 2015
on learned visual embedding patrick pérez Allegro Workshop Inria Rhônes-Alpes 22 July 2015 Vector visual representation Fixed-size image representation High-dim (100 100,000) Generic, unsupervised: BoW,
More informationLarge-scale visual recognition Efficient matching
Large-scale visual recognition Efficient matching Florent Perronnin, XRCE Hervé Jégou, INRIA CVPR tutorial June 16, 2012 Outline!! Preliminary!! Locality Sensitive Hashing: the two modes!! Hashing!! Embedding!!
More informationAggregating Descriptors with Local Gaussian Metrics
Aggregating Descriptors with Local Gaussian Metrics Hideki Nakayama Grad. School of Information Science and Technology The University of Tokyo Tokyo, JAPAN nakayama@ci.i.u-tokyo.ac.jp Abstract Recently,
More informationMetric learning approaches! for image annotation! and face recognition!
Metric learning approaches! for image annotation! and face recognition! Jakob Verbeek" LEAR Team, INRIA Grenoble, France! Joint work with :"!Matthieu Guillaumin"!!Thomas Mensink"!!!Cordelia Schmid! 1 2
More informationFisher vector image representation
Fisher vector image representation Jakob Verbeek January 13, 2012 Course website: http://lear.inrialpes.fr/~verbeek/mlcr.11.12.php Fisher vector representation Alternative to bag-of-words image representation
More informationFace2Face Comparing faces with applications Patrick Pérez. Inria, Rennes 2 Oct. 2014
Face2Face Comparing faces with applications Patrick Pérez Inria, Rennes 2 Oct. 2014 Outline Metric learning for face comparison Expandable parts model and occlusions Face sets comparison Identity-based
More information3D Object Recognition using Multiclass SVM-KNN
3D Object Recognition using Multiclass SVM-KNN R. Muralidharan, C. Chandradekar April 29, 2014 Presented by: Tasadduk Chowdhury Problem We address the problem of recognizing 3D objects based on various
More informationMixtures of Gaussians and Advanced Feature Encoding
Mixtures of Gaussians and Advanced Feature Encoding Computer Vision Ali Borji UWM Many slides from James Hayes, Derek Hoiem, Florent Perronnin, and Hervé Why do good recognition systems go bad? E.g. Why
More informationInstance-based Learning
Instance-based Learning Machine Learning 10701/15781 Carlos Guestrin Carnegie Mellon University February 19 th, 2007 2005-2007 Carlos Guestrin 1 Why not just use Linear Regression? 2005-2007 Carlos Guestrin
More informationTagProp: Discriminative Metric Learning in Nearest Neighbor Models for Image Annotation
TagProp: Discriminative Metric Learning in Nearest Neighbor Models for Image Annotation Matthieu Guillaumin, Thomas Mensink, Jakob Verbeek, Cordelia Schmid LEAR team, INRIA Rhône-Alpes, Grenoble, France
More informationGlobal Metric Learning by Gradient Descent
Global Metric Learning by Gradient Descent Jens Hocke and Thomas Martinetz University of Lübeck - Institute for Neuro- and Bioinformatics Ratzeburger Allee 160, 23538 Lübeck, Germany hocke@inb.uni-luebeck.de
More informationECG782: Multidimensional Digital Signal Processing
ECG782: Multidimensional Digital Signal Processing Object Recognition http://www.ee.unlv.edu/~b1morris/ecg782/ 2 Outline Knowledge Representation Statistical Pattern Recognition Neural Networks Boosting
More informationNetwork Traffic Measurements and Analysis
DEIB - Politecnico di Milano Fall, 2017 Introduction Often, we have only a set of features x = x 1, x 2,, x n, but no associated response y. Therefore we are not interested in prediction nor classification,
More informationPattern Spotting in Historical Document Image
Pattern Spotting in historical document images Sovann EN, Caroline Petitjean, Stéphane Nicolas, Frédéric Jurie, Laurent Heutte LITIS, University of Rouen, France 1 Outline Introduction Commons Pipeline
More informationILSVRC on a Smartphone
[DOI: 10.2197/ipsjtcva.6.83] Express Paper ILSVRC on a Smartphone Yoshiyuki Kawano 1,a) Keiji Yanai 1,b) Received: March 14, 2014, Accepted: April 24, 2014, Released: July 25, 2014 Abstract: In this work,
More informationSearch Engines. Information Retrieval in Practice
Search Engines Information Retrieval in Practice All slides Addison Wesley, 2008 Classification and Clustering Classification and clustering are classical pattern recognition / machine learning problems
More informationUnsupervised Learning
Networks for Pattern Recognition, 2014 Networks for Single Linkage K-Means Soft DBSCAN PCA Networks for Kohonen Maps Linear Vector Quantization Networks for Problems/Approaches in Machine Learning Supervised
More informationCombine the PA Algorithm with a Proximal Classifier
Combine the Passive and Aggressive Algorithm with a Proximal Classifier Yuh-Jye Lee Joint work with Y.-C. Tseng Dept. of Computer Science & Information Engineering TaiwanTech. Dept. of Statistics@NCKU
More informationA Taxonomy of Semi-Supervised Learning Algorithms
A Taxonomy of Semi-Supervised Learning Algorithms Olivier Chapelle Max Planck Institute for Biological Cybernetics December 2005 Outline 1 Introduction 2 Generative models 3 Low density separation 4 Graph
More informationFace detection and recognition. Detection Recognition Sally
Face detection and recognition Detection Recognition Sally Face detection & recognition Viola & Jones detector Available in open CV Face recognition Eigenfaces for face recognition Metric learning identification
More informationGenerative and discriminative classification techniques
Generative and discriminative classification techniques Machine Learning and Category Representation 013-014 Jakob Verbeek, December 13+0, 013 Course website: http://lear.inrialpes.fr/~verbeek/mlcr.13.14
More informationMachine Learning Basics: Stochastic Gradient Descent. Sargur N. Srihari
Machine Learning Basics: Stochastic Gradient Descent Sargur N. srihari@cedar.buffalo.edu 1 Topics 1. Learning Algorithms 2. Capacity, Overfitting and Underfitting 3. Hyperparameters and Validation Sets
More informationClass 6 Large-Scale Image Classification
Class 6 Large-Scale Image Classification Liangliang Cao, March 7, 2013 EECS 6890 Topics in Information Processing Spring 2013, Columbia University http://rogerioferis.com/visualrecognitionandsearch Visual
More informationContent-based image and video analysis. Machine learning
Content-based image and video analysis Machine learning for multimedia retrieval 04.05.2009 What is machine learning? Some problems are very hard to solve by writing a computer program by hand Almost all
More informationInstance-based Learning CE-717: Machine Learning Sharif University of Technology. M. Soleymani Fall 2015
Instance-based Learning CE-717: Machine Learning Sharif University of Technology M. Soleymani Fall 2015 Outline Non-parametric approach Unsupervised: Non-parametric density estimation Parzen Windows K-Nearest
More informationCompressed local descriptors for fast image and video search in large databases
Compressed local descriptors for fast image and video search in large databases Matthijs Douze2 joint work with Hervé Jégou1, Cordelia Schmid2 and Patrick Pérez3 1: INRIA Rennes, TEXMEX team, France 2:
More informationMachine Learning. Topic 5: Linear Discriminants. Bryan Pardo, EECS 349 Machine Learning, 2013
Machine Learning Topic 5: Linear Discriminants Bryan Pardo, EECS 349 Machine Learning, 2013 Thanks to Mark Cartwright for his extensive contributions to these slides Thanks to Alpaydin, Bishop, and Duda/Hart/Stork
More informationSupervised Learning: Nearest Neighbors
CS 2750: Machine Learning Supervised Learning: Nearest Neighbors Prof. Adriana Kovashka University of Pittsburgh February 1, 2016 Today: Supervised Learning Part I Basic formulation of the simplest classifier:
More informationMulti-label Classification. Jingzhou Liu Dec
Multi-label Classification Jingzhou Liu Dec. 6 2016 Introduction Multi-class problem, Training data (x $, y $ ) ( ), x $ X R., y $ Y = 1,2,, L Learn a mapping f: X Y Each instance x $ is associated with
More informationComputational Statistics The basics of maximum likelihood estimation, Bayesian estimation, object recognitions
Computational Statistics The basics of maximum likelihood estimation, Bayesian estimation, object recognitions Thomas Giraud Simon Chabot October 12, 2013 Contents 1 Discriminant analysis 3 1.1 Main idea................................
More informationAll lecture slides will be available at CSC2515_Winter15.html
CSC2515 Fall 2015 Introduc3on to Machine Learning Lecture 9: Support Vector Machines All lecture slides will be available at http://www.cs.toronto.edu/~urtasun/courses/csc2515/ CSC2515_Winter15.html Many
More informationINF 4300 Classification III Anne Solberg The agenda today:
INF 4300 Classification III Anne Solberg 28.10.15 The agenda today: More on estimating classifier accuracy Curse of dimensionality and simple feature selection knn-classification K-means clustering 28.10.15
More informationA Family of Contextual Measures of Similarity between Distributions with Application to Image Retrieval
A Family of Contextual Measures of Similarity between Distributions with Application to Image Retrieval Florent Perronnin, Yan Liu and Jean-Michel Renders Xerox Research Centre Europe (XRCE) Textual and
More informationLarge-scale visual recognition The bag-of-words representation
Large-scale visual recognition The bag-of-words representation Florent Perronnin, XRCE Hervé Jégou, INRIA CVPR tutorial June 16, 2012 Outline Bag-of-words Large or small vocabularies? Extensions for instance-level
More informationSupport Vector Machines
Support Vector Machines About the Name... A Support Vector A training sample used to define classification boundaries in SVMs located near class boundaries Support Vector Machines Binary classifiers whose
More informationSupervised vs unsupervised clustering
Classification Supervised vs unsupervised clustering Cluster analysis: Classes are not known a- priori. Classification: Classes are defined a-priori Sometimes called supervised clustering Extract useful
More informationMachine Learning. Nonparametric methods for Classification. Eric Xing , Fall Lecture 2, September 12, 2016
Machine Learning 10-701, Fall 2016 Nonparametric methods for Classification Eric Xing Lecture 2, September 12, 2016 Reading: 1 Classification Representing data: Hypothesis (classifier) 2 Clustering 3 Supervised
More informationBasic Problem Addressed. The Approach I: Training. Main Idea. The Approach II: Testing. Why a set of vocabularies?
Visual Categorization With Bags of Keypoints. ECCV,. G. Csurka, C. Bray, C. Dance, and L. Fan. Shilpa Gulati //7 Basic Problem Addressed Find a method for Generic Visual Categorization Visual Categorization:
More informationLarge scale object/scene recognition
Large scale object/scene recognition Image dataset: > 1 million images query Image search system ranked image list Each image described by approximately 2000 descriptors 2 10 9 descriptors to index! Database
More informationDeep Learning for Computer Vision
Deep Learning for Computer Vision Spring 2018 http://vllab.ee.ntu.edu.tw/dlcv.html (primary) https://ceiba.ntu.edu.tw/1062dlcv (grade, etc.) FB: DLCV Spring 2018 Yu Chiang Frank Wang 王鈺強, Associate Professor
More informationProceedings of the International MultiConference of Engineers and Computer Scientists 2018 Vol I IMECS 2018, March 14-16, 2018, Hong Kong
, March 14-16, 2018, Hong Kong , March 14-16, 2018, Hong Kong , March 14-16, 2018, Hong Kong , March 14-16, 2018, Hong Kong TABLE I CLASSIFICATION ACCURACY OF DIFFERENT PRE-TRAINED MODELS ON THE TEST DATA
More informationFaceNet. Florian Schroff, Dmitry Kalenichenko, James Philbin Google Inc. Presentation by Ignacio Aranguren and Rahul Rana
FaceNet Florian Schroff, Dmitry Kalenichenko, James Philbin Google Inc. Presentation by Ignacio Aranguren and Rahul Rana Introduction FaceNet learns a mapping from face images to a compact Euclidean Space
More informationInstance-based Learning
Instance-based Learning Machine Learning 10701/15781 Carlos Guestrin Carnegie Mellon University October 15 th, 2007 2005-2007 Carlos Guestrin 1 1-Nearest Neighbor Four things make a memory based learner:
More informationPart-based and local feature models for generic object recognition
Part-based and local feature models for generic object recognition May 28 th, 2015 Yong Jae Lee UC Davis Announcements PS2 grades up on SmartSite PS2 stats: Mean: 80.15 Standard Dev: 22.77 Vote on piazza
More informationObject Classification Problem
HIERARCHICAL OBJECT CATEGORIZATION" Gregory Griffin and Pietro Perona. Learning and Using Taxonomies For Fast Visual Categorization. CVPR 2008 Marcin Marszalek and Cordelia Schmid. Constructing Category
More informationNon-Bayesian Classifiers Part I: k-nearest Neighbor Classifier and Distance Functions
Non-Bayesian Classifiers Part I: k-nearest Neighbor Classifier and Distance Functions Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr CS 551, Fall 2017 CS 551,
More informationAdaptive Binary Quantization for Fast Nearest Neighbor Search
IBM Research Adaptive Binary Quantization for Fast Nearest Neighbor Search Zhujin Li 1, Xianglong Liu 1*, Junjie Wu 1, and Hao Su 2 1 Beihang University, Beijing, China 2 Stanford University, Stanford,
More informationClassification. Vladimir Curic. Centre for Image Analysis Swedish University of Agricultural Sciences Uppsala University
Classification Vladimir Curic Centre for Image Analysis Swedish University of Agricultural Sciences Uppsala University Outline An overview on classification Basics of classification How to choose appropriate
More informationA Dendrogram. Bioinformatics (Lec 17)
A Dendrogram 3/15/05 1 Hierarchical Clustering [Johnson, SC, 1967] Given n points in R d, compute the distance between every pair of points While (not done) Pick closest pair of points s i and s j and
More informationMetric Learning Applied for Automatic Large Image Classification
September, 2014 UPC Metric Learning Applied for Automatic Large Image Classification Supervisors SAHILU WENDESON / IT4BI TOON CALDERS (PhD)/ULB SALIM JOUILI (PhD)/EuraNova Image Database Classification
More informationCOMP 551 Applied Machine Learning Lecture 13: Unsupervised learning
COMP 551 Applied Machine Learning Lecture 13: Unsupervised learning Associate Instructor: Herke van Hoof (herke.vanhoof@mail.mcgill.ca) Slides mostly by: (jpineau@cs.mcgill.ca) Class web page: www.cs.mcgill.ca/~jpineau/comp551
More informationECE 5424: Introduction to Machine Learning
ECE 5424: Introduction to Machine Learning Topics: Unsupervised Learning: Kmeans, GMM, EM Readings: Barber 20.1-20.3 Stefan Lee Virginia Tech Tasks Supervised Learning x Classification y Discrete x Regression
More informationClassification. Vladimir Curic. Centre for Image Analysis Swedish University of Agricultural Sciences Uppsala University
Classification Vladimir Curic Centre for Image Analysis Swedish University of Agricultural Sciences Uppsala University Outline An overview on classification Basics of classification How to choose appropriate
More information1 Case study of SVM (Rob)
DRAFT a final version will be posted shortly COS 424: Interacting with Data Lecturer: Rob Schapire and David Blei Lecture # 8 Scribe: Indraneel Mukherjee March 1, 2007 In the previous lecture we saw how
More informationCSE 573: Artificial Intelligence Autumn 2010
CSE 573: Artificial Intelligence Autumn 2010 Lecture 16: Machine Learning Topics 12/7/2010 Luke Zettlemoyer Most slides over the course adapted from Dan Klein. 1 Announcements Syllabus revised Machine
More informationCS 179 Lecture 16. Logistic Regression & Parallel SGD
CS 179 Lecture 16 Logistic Regression & Parallel SGD 1 Outline logistic regression (stochastic) gradient descent parallelizing SGD for neural nets (with emphasis on Google s distributed neural net implementation)
More informationDiscriminate Analysis
Discriminate Analysis Outline Introduction Linear Discriminant Analysis Examples 1 Introduction What is Discriminant Analysis? Statistical technique to classify objects into mutually exclusive and exhaustive
More informationEfficient Algorithms may not be those we think
Efficient Algorithms may not be those we think Yann LeCun, Computational and Biological Learning Lab The Courant Institute of Mathematical Sciences New York University http://yann.lecun.com http://www.cs.nyu.edu/~yann
More informationMachine Learning Basics. Sargur N. Srihari
Machine Learning Basics Sargur N. srihari@cedar.buffalo.edu 1 Overview Deep learning is a specific type of ML Necessary to have a solid understanding of the basic principles of ML 2 Topics Stochastic Gradient
More informationClustering Lecture 5: Mixture Model
Clustering Lecture 5: Mixture Model Jing Gao SUNY Buffalo 1 Outline Basics Motivation, definition, evaluation Methods Partitional Hierarchical Density-based Mixture model Spectral methods Advanced topics
More informationSupport vector machines
Support vector machines When the data is linearly separable, which of the many possible solutions should we prefer? SVM criterion: maximize the margin, or distance between the hyperplane and the closest
More informationThe exam is closed book, closed notes except your one-page (two-sided) cheat sheet.
CS 189 Spring 2015 Introduction to Machine Learning Final You have 2 hours 50 minutes for the exam. The exam is closed book, closed notes except your one-page (two-sided) cheat sheet. No calculators or
More informationProblem 1: Complexity of Update Rules for Logistic Regression
Case Study 1: Estimating Click Probabilities Tackling an Unknown Number of Features with Sketching Machine Learning for Big Data CSE547/STAT548, University of Washington Emily Fox January 16 th, 2014 1
More informationMachine Learning. Chao Lan
Machine Learning Chao Lan Machine Learning Prediction Models Regression Model - linear regression (least square, ridge regression, Lasso) Classification Model - naive Bayes, logistic regression, Gaussian
More informationTrans Media Relevance Feedback for Image Autoannotation
MENSINK et al.: TMRF FOR IMAGE AUTOANNOTATION 1 Trans Media Relevance Feedback for Image Autoannotation Thomas Mensink 12 thomas.mensink@xrce.xerox.com Jakob Verbeek 2 jakob.verbeek@inrialpes.fr Gabriela
More informationBased on Raymond J. Mooney s slides
Instance Based Learning Based on Raymond J. Mooney s slides University of Texas at Austin 1 Example 2 Instance-Based Learning Unlike other learning algorithms, does not involve construction of an explicit
More informationImageCLEF 2011
SZTAKI @ ImageCLEF 2011 Bálint Daróczy joint work with András Benczúr, Róbert Pethes Data Mining and Web Search Group Computer and Automation Research Institute Hungarian Academy of Sciences Training/test
More informationLTI Thesis Defense: Riemannian Geometry and Statistical Machine Learning
Outline LTI Thesis Defense: Riemannian Geometry and Statistical Machine Learning Guy Lebanon Motivation Introduction Previous Work Committee:John Lafferty Geoff Gordon, Michael I. Jordan, Larry Wasserman
More informationLogistic Regression. Abstract
Logistic Regression Tsung-Yi Lin, Chen-Yu Lee Department of Electrical and Computer Engineering University of California, San Diego {tsl008, chl60}@ucsd.edu January 4, 013 Abstract Logistic regression
More informationSegmentation: Clustering, Graph Cut and EM
Segmentation: Clustering, Graph Cut and EM Ying Wu Electrical Engineering and Computer Science Northwestern University, Evanston, IL 60208 yingwu@northwestern.edu http://www.eecs.northwestern.edu/~yingwu
More informationLast week. Multi-Frame Structure from Motion: Multi-View Stereo. Unknown camera viewpoints
Last week Multi-Frame Structure from Motion: Multi-View Stereo Unknown camera viewpoints Last week PCA Today Recognition Today Recognition Recognition problems What is it? Object detection Who is it? Recognizing
More informationarxiv: v1 [cs.cv] 20 Dec 2013
Occupancy Detection in Vehicles Using Fisher Vector Image Representation arxiv:1312.6024v1 [cs.cv] 20 Dec 2013 Yusuf Artan Xerox Research Center Webster, NY 14580 Yusuf.Artan@xerox.com Peter Paul Xerox
More informationClassification: Linear Discriminant Functions
Classification: Linear Discriminant Functions CE-725: Statistical Pattern Recognition Sharif University of Technology Spring 2013 Soleymani Outline Discriminant functions Linear Discriminant functions
More informationPartitioning Data. IRDS: Evaluation, Debugging, and Diagnostics. Cross-Validation. Cross-Validation for parameter tuning
Partitioning Data IRDS: Evaluation, Debugging, and Diagnostics Charles Sutton University of Edinburgh Training Validation Test Training : Running learning algorithms Validation : Tuning parameters of learning
More informationShifting from Naming to Describing: Semantic Attribute Models. Rogerio Feris, June 2014
Shifting from Naming to Describing: Semantic Attribute Models Rogerio Feris, June 2014 Recap Large-Scale Semantic Modeling Feature Coding and Pooling Low-Level Feature Extraction Training Data Slide credit:
More informationNearest Neighbor with KD Trees
Case Study 2: Document Retrieval Finding Similar Documents Using Nearest Neighbors Machine Learning/Statistics for Big Data CSE599C1/STAT592, University of Washington Emily Fox January 22 nd, 2013 1 Nearest
More informationIntroduction to Artificial Intelligence
Introduction to Artificial Intelligence COMP307 Machine Learning 2: 3-K Techniques Yi Mei yi.mei@ecs.vuw.ac.nz 1 Outline K-Nearest Neighbour method Classification (Supervised learning) Basic NN (1-NN)
More informationLearning Visual Semantics: Models, Massive Computation, and Innovative Applications
Learning Visual Semantics: Models, Massive Computation, and Innovative Applications Part II: Visual Features and Representations Liangliang Cao, IBM Watson Research Center Evolvement of Visual Features
More informationClustering. Mihaela van der Schaar. January 27, Department of Engineering Science University of Oxford
Department of Engineering Science University of Oxford January 27, 2017 Many datasets consist of multiple heterogeneous subsets. Cluster analysis: Given an unlabelled data, want algorithms that automatically
More informationGenerative and discriminative classification
Generative and discriminative classification Machine Learning and Object Recognition 2017-2018 Jakob Verbeek Classification in its simplest form Given training data labeled for two or more classes Classification
More informationMTTS1 Dimensionality Reduction and Visualization Spring 2014, 5op Jaakko Peltonen
MTTS1 Dimensionality Reduction and Visualization Spring 2014, 5op Jaakko Peltonen Lecture 9: Metric Learning Motivation metric learning Metric learning means learning a better metric (better distance function)
More informationRecursive Similarity-Based Algorithm for Deep Learning
Recursive Similarity-Based Algorithm for R Tomasz Maszczyk & W lodzis law Duch Nicolaus Copernicus University Toruń, Poland ICONIP 2012 {tmaszczyk,wduch}@is.umk.pl 1 / 21 R Similarity-Based Learning ()
More informationMachine Learning in Biology
Università degli studi di Padova Machine Learning in Biology Luca Silvestrin (Dottorando, XXIII ciclo) Supervised learning Contents Class-conditional probability density Linear and quadratic discriminant
More informationDeep Generative Models Variational Autoencoders
Deep Generative Models Variational Autoencoders Sudeshna Sarkar 5 April 2017 Generative Nets Generative models that represent probability distributions over multiple variables in some way. Directed Generative
More informationECS289: Scalable Machine Learning
ECS289: Scalable Machine Learning Cho-Jui Hsieh UC Davis Sept 22, 2016 Course Information Website: http://www.stat.ucdavis.edu/~chohsieh/teaching/ ECS289G_Fall2016/main.html My office: Mathematical Sciences
More informationOptimizing 1-Nearest Prototype Classifiers
2013 IEEE Conference on Computer Vision and Pattern Recognition Optimizing 1-Nearest Prototype Classifiers Paul Wohlhart, Martin Köstinger, Michael Donoser, Peter M. Roth, Horst Bischof Institute for Computer
More informationUnsupervised Learning
Unsupervised Learning Learning without Class Labels (or correct outputs) Density Estimation Learn P(X) given training data for X Clustering Partition data into clusters Dimensionality Reduction Discover
More informationMachine Learning. B. Unsupervised Learning B.1 Cluster Analysis. Lars Schmidt-Thieme, Nicolas Schilling
Machine Learning B. Unsupervised Learning B.1 Cluster Analysis Lars Schmidt-Thieme, Nicolas Schilling Information Systems and Machine Learning Lab (ISMLL) Institute for Computer Science University of Hildesheim,
More informationIntroduction to object recognition. Slides adapted from Fei-Fei Li, Rob Fergus, Antonio Torralba, and others
Introduction to object recognition Slides adapted from Fei-Fei Li, Rob Fergus, Antonio Torralba, and others Overview Basic recognition tasks A statistical learning approach Traditional or shallow recognition
More informationQuasi Cosine Similarity Metric Learning
Quasi Cosine Similarity Metric Learning Xiang Wu, Zhi-Guo Shi and Lei Liu School of Computer and Communication Engineering, University of Science and Technology Beijing, No.30 Xueyuan Road, Haidian District,
More informationCS246: Mining Massive Datasets Jure Leskovec, Stanford University
CS246: Mining Massive Datasets Jure Leskovec, Stanford University http://cs246.stanford.edu [Kumar et al. 99] 2/13/2013 Jure Leskovec, Stanford CS246: Mining Massive Datasets, http://cs246.stanford.edu
More informationK-Means Clustering. Sargur Srihari
K-Means Clustering Sargur srihari@cedar.buffalo.edu 1 Topics in Mixture Models and EM Mixture models K-means Clustering Mixtures of Gaussians Maximum Likelihood EM for Gaussian mistures EM Algorithm Gaussian
More informationCSE255 Assignment 1 Improved image-based recommendations for what not to wear dataset
CSE255 Assignment 1 Improved image-based recommendations for what not to wear dataset Prabhav Agrawal and Soham Shah 23 February 2015 1 Introduction We are interested in modeling the human perception of
More informationFacial Expression Classification with Random Filters Feature Extraction
Facial Expression Classification with Random Filters Feature Extraction Mengye Ren Facial Monkey mren@cs.toronto.edu Zhi Hao Luo It s Me lzh@cs.toronto.edu I. ABSTRACT In our work, we attempted to tackle
More informationDeep Learning and Its Applications
Convolutional Neural Network and Its Application in Image Recognition Oct 28, 2016 Outline 1 A Motivating Example 2 The Convolutional Neural Network (CNN) Model 3 Training the CNN Model 4 Issues and Recent
More information27: Hybrid Graphical Models and Neural Networks
10-708: Probabilistic Graphical Models 10-708 Spring 2016 27: Hybrid Graphical Models and Neural Networks Lecturer: Matt Gormley Scribes: Jakob Bauer Otilia Stretcu Rohan Varma 1 Motivation We first look
More informationPerceptron as a graph
Neural Networks Machine Learning 10701/15781 Carlos Guestrin Carnegie Mellon University October 10 th, 2007 2005-2007 Carlos Guestrin 1 Perceptron as a graph 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0-6 -4-2
More information