Supervised Distance Metric Learning
|
|
|
- Anissa Martin
- 5 years ago
- Views:
Transcription
1 Supervised Distance Metric Learning A Retrospective Nan Xiao Stat. Dept., Central South Univ. Q4 2013
2 Outline Theory Algorithms Applications
3 What is a metric?
4 A metric is a function.
5 Function satisfies:
6 A metric is a similarity measure.
7
8 Euclidean Distance Minkowski Distance Mahalanobis Distance Levenshtein (Edit) Distance... Kernel Function
9 Distance Metrics as an Essential Building Block in PR & ML Unsupervised Learning (K-means) Supervised Learning (k-nn)
10 Distance Metrics as an Essential Building Block in PR & ML Computer Vision Natural Language Processing Information Retrieval (Text, Picture, Music, Video) Bioinformatics (DNA / Protein Sequence Alignment)
11 Metric Selection Priori Euclidean Distance / Cosine Distance Gaussian Kernel / Linear Kernel Experiment Cross Validation
12 Why we need customized distance metrics?
13 Cougar Face Category from Caltech101 Dataset! Frome, A et al., (2007).Image Retrieval and Classification Using Local Distance Functions.
14 α-alanine / β-alanine
15 Similarity Baseline Effect in Large Scale Chemical DB
16 Rulers vs. Metrics
17 The Unsupervised Approach
18 Manifold Learning PCA / MDS (Not Essentially) Isomap (Science, 2000) LLE / Hessian LLE LTSA Spectral Embedding and many more variants.
19 Real Swiss Roll
20 Swiss Roll Data
21 The Problem of Unsupervised Distance Metric Learning A Demo
22 The Supervised Approach
23 Data Sample Learned Metric Learned Predictor Underlying Distribution Metric Learning Algorithm Metric- Based Algorithm Prediction Learning Pipeline
24 How to apply the learned metric?
25 Metric vs. Transformation Let is positive semi-definite (PSD),
26 Metric vs. Transformation Using a new metric in the original space is equivalent to first applying linear transformation then using Euclidean distance in new space of.
27 Transformation Types Identity Matrix No Transformation Diagonal Matrix Scaling Only, No Rotation Orthogonal Matrix Rotation Only, No Scaling Full Square Matrix Scaling + Rotation Non-square Full Matrix Scaling + Rotation + Dim Reduction
28 Dimensionality Reduction Take eigenvector decomposition and then
29 Supervised Approach Early Approach Nearest-Neighbor Driven Approach Information Theoretic Driven Approach
30 Early Approaches
31 Introducing Side-Information Must-Link / Cannot Link Constrains:
32 s1 s2 s3 d1 d2 Metric Learning s4 s5 Intuition Behind Early Approach
33 General Form: Learning Form
34 GDMLCP (Xing et al., 2002)
35 Original Data and Projected Data
36 GDMLCP Convex Optimization Very Slow, Iterative, Not Parallelizable Global Optimum, Intractable for Multimodal Data
37 J. Fan et al., (Nov 2013) QUADRO: A Supervised Dimension Reduction Method via Rayleigh Quotient Optimization! arxiv Exactly the same idea, but for high-dim classification
38 Nearest Neighbor Driven Approaches
39 NCA (Hinton et al., 2004) Minimize the expected LOO error of a stochastic NN classifier in the projection space introduced by M.
40 NCA The probability is a neighbor of : The probability is correctly classified: then learn the distance metric by solving:
41
42 NCA Summary Non-Convex Optimization Local Maxima
43 LMNN (Weinberger, 2005) Most widely used, many variants Local constrains inspired by SVM: The k-nearest Euclidean neighbors ( target neighbors ) of any training samples should belong to the correct class while keeping away samples of other classes.
44 LMNN
45 LMNN The metric is learned by the convex optimization:
46 LMNN Summary! Convex Optimization Special Purpose Solver by Subgradient Descent Parallelizable, Scales to Billions of Constrains Constrained by Euclidean NN Selection
47 Information Theoretic Driven Approaches
48 RCA (Bar-Hillel, 2003) Chunklets The inverse of is a Mahalanobis distance metric.
49 RCA It is a optimal solution to a mutual information measure. It is a optimal solution to the optimization problem consisting in minimizing the within-class distances.
50 RCA (cont d) DCA (Discriminant Component Analysis) for incorporating cannot link constrains Kernel RCA
51 ITML (Davis, 2007) Define Bregman Divergence on a PSD matrix: is often set to thus the regularization aims at keeping the learned distance close to Euclidean distance.
52 ITML
53 ITML Summary The information-theoretic interpretation behind minimizing Dld is equivalent to minimizing the KL divergence between two multivariate Gaussian distributions parameterized by M and M0.
54 ITML Summary Efficient Optimization Algorithm & Global Optimum Hand-picked M0 may have a unknown influence
55 Applications
56 Music Classification
57 Task Briefings MP3s downloaded from a set of music blogs Thousands of songs 319 blogs, 164 artists, 74 albums Slaney, M., Weinberger, K. Q., & White, W. (2008). Learning a Metric for Music Similarity.
58 Task Briefings Extract 18 features for each song: Songs broken up into segments (80ms to a few seconds) Mean segment duration Track tempo estimate Regularity of the beat Estimation of the time signature overall loudness estimate of the track Training done via labels based on blog, artist, and album (separately)
59 Learned Distance Matrix
60 Performance Comparison
61 In-Silico Toxicity Prediction
62 Task Briefings Five toxicity datasets from FDA, EPA, etc. Toxic / Non-Toxic, 100 / 100 samples FP4 / MACCS 0-1 valued molecular fingerprints
63 A Glimpse of Data
64 Sensitivity Gaussian RBF Kernel FP4 CPDBAS No RCA (0.7962) CPDBAS 20% RCA (0.8324) DBPCAN No RCA (0.9467) DBPCAN 30% RCA (0.9666) EPAFHM No RCA (0.7175) EPAFHM 30% RCA (0.7275) FDAMDD No RCA (0.8913) FDAMDD 30% RCA (0.8970) NCTRER No RCA (0.8873) NCTRER 20% RCA (0.8994) Specificity
65 Sensitivity Gaussian RBF Kernel MACCS CPDBAS No RCA (0.8482) CPDBAS 30% RCA (0.8627) DBPCAN No RCA (0.9780) DBPCAN 30% RCA (0.9924) EPAFHM No RCA (0.7982) EPAFHM 20% RCA (0.8061) FDAMDD No RCA (0.8545) FDAMDD 30% RCA (0.8738) NCTRER No RCA (0.8839) NCTRER 30% RCA (0.8990) Specificity
66 Model Summary Fully- / Weakly- / Semi-Supervised Metric Form: Linear (Kernelization) / Non-Linear / Local Optimum: Global / Local Dim Reduction / Non-Dim Reduction Regularization / No Regularization
67 Metric Learning Learning paradigm Form of metric Scalability Optimality of the solution Dimensionality reduction Fully supervised Linear w.r.t. number of examples Local Yes Weakly supervised Nonlinear w.r.t. dimension Global No Semi supervised Local 5 Key Properties of Supervised Distance Metric Learning Algorithms
68 Name Year Source Form of Scalability Supervision Optimum Dimension Additional Regularizer Code Metric w.r.t. n w.r.t. d Reduction Information MMC 2002 Yes Weak Linear Global No None S&J 2003 No Weak Linear Global No Frobenius NCA 2004 Yes Full Linear Local Yes None For k-nn MCML 2005 Yes Full Linear Global No None For k-nn LMNN 2005 Yes Full Linear Global No None For k-nn RCA 2003 Yes Weak Linear Global No None ITML 2007 Yes Weak Linear Global Yes LogDet Online version SDML 2009 No Weak Linear Global Yes LogDet+L 1 n d POLA 2004 No Weak Linear Global No None Online LEGO 2008 No Weak Linear Global Yes LogDet Online RDML 2009 No Weak Linear Global No Frobenius Online MDML 2012 No Weak Linear Global Yes Nuclear norm Online mt-lmnn 2010 Yes Full Linear Global No Frobenius Multi-task MLCS 2011 No Weak Linear Local Yes N/A Multi-task GPML 2012 No Weak Linear Global Yes von Neumann Multi-task TML 2010 Yes Weak Linear Global No Frobenius Transfer learning LPML 2006 No Weak Linear Global Yes L 1 SML 2009 No Weak Linear Global Yes L 2,1 BoostMetric 2009 Yes Weak Linear Global Yes None DML-p 2012 No Weak Linear Global No None RML 2010 No Weak Linear Global No Frobenius Noisy constraints MLR 2010 Yes Full Linear Global Yes Nuclear norm For ranking SiLA 2008 No Full Linear N/A No None Online gcosla 2009 No Weak Linear Global No None Online OASIS 2009 Yes Weak Linear Global No Frobenius Online SLLC 2012 No Full Linear Global No Frobenius For linear classif. RSL 2013 No Full Linear Local No Frobenius Rectangular matrix LSMD 2005 No Weak Nonlinear Local Yes None SVML 2012 No Full Nonlinear Local Yes Frobenius For SVM GB-LMNN 2012 No Full Nonlinear Local Yes None M 2 -LMNN 2008 Yes Full Local Global No None GLML 2010 No Full Local Global No Diagonal Generative Bk-means 2009 No Weak Local Global No RKHS norm Bregman dist. PLML 2012 Yes Weak Local Global No Manifold+Frob RFD 2012 Yes Weak Local N/A No None Random forests χ 2 -LMNN 2012 No Full Nonlinear Local Yes None Histogram data GML 2011 No Weak Linear Local No None Histogram data EMDL 2012 No Weak Linear Local No Frobenius Histogram data LRML 2008 Yes Semi Linear Global No Laplacian M-DML 2009 No Semi Linear Local No Laplacian Auxiliary metrics SERAPH 2012 Yes Semi Linear Local Yes Trace+entropy Probabilistic CDML 2011 No Semi N/A N/A N/A N/A N/A N/A Domain adaptation DAML 2011 No Semi Nonlinear Global No MMD Domain adaptation
69 Future Problems Scalability with both p and n High Dim and Sparse Metric Learning (J-L Lemma) Learning Richer Metrics Similarity Degree? Metric Learning for Structured Data (Char/Tree/Graph) CV / NLP / Bio- / Chem- / -Informatics / -Metrics
70 References (Survey) A. Bellet. A. Habrard. M. Sebban. (2013). A Survey on Metric Learning for Feature Vectors and Structured Data. (Survey) Brian Kulis. (2012). Metric Learning: A Survey. (Survey) Yang Liu. (2007). Distance Metric Learning: A Comprehensive Survey.
71 (GDMLCP) Eric P. Xing, Andrew Y. Ng, Michael I. Jordan, Stuart Russell. (2003). Distance Metric Learning with Application to Clustering with Sideinformation. NIPS'15.
72 (NCA) J. Goldberger, S. Roweis, G. Hinton, R. Salakhutdinov. (2004). Neighborhood Components Analysis. NIPS'17. (LMNN) K. Q. Weinberger, J. Blitzer, L. K. Saul (2006). Distance Metric Learning for Large Margin Nearest Neighbor Classification. NIPS 18.
73 (RCA) Aharon Bar-Hillel, Tomer Hertz, Noam Shental, Daphna Weinshall. (2003). Learning Distance Functions Using Equivalence Relations. ICML 03. (ITML) Jason V. Davis, Brian Kulis, Prateek Jain, Suvrit Sra, Inderjit S. Dhillon. (2007). Informationtheoretic Metric Learning. ICML 07.
74 Q & A
MTTS1 Dimensionality Reduction and Visualization Spring 2014, 5op Jaakko Peltonen
MTTS1 Dimensionality Reduction and Visualization Spring 2014, 5op Jaakko Peltonen Lecture 9: Metric Learning Motivation metric learning Metric learning means learning a better metric (better distance function)
Constrained Metric Learning via Distance Gap Maximization
Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence (AAAI-10) Constrained Metric Learning via Distance Gap Maximization Wei Liu, Xinmei Tian, Dacheng Tao School of Computer Engineering
Metric Learning Applied for Automatic Large Image Classification
September, 2014 UPC Metric Learning Applied for Automatic Large Image Classification Supervisors SAHILU WENDESON / IT4BI TOON CALDERS (PhD)/ULB SALIM JOUILI (PhD)/EuraNova Image Database Classification
Learning Models of Similarity: Metric and Kernel Learning. Eric Heim, University of Pittsburgh
Learning Models of Similarity: Metric and Kernel Learning Eric Heim, University of Pittsburgh Standard Machine Learning Pipeline Manually-Tuned Features Machine Learning Model Desired Output for Task Features
Distance metric learning: A two-phase approach
ESANN 07 proceedings, European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning. Bruges (Belgium), 6-8 April 07, i6doc.com publ., ISBN 978-8758709-. Distance metric
Learning Better Data Representation using Inference-Driven Metric Learning
Learning Better Data Representation using Inference-Driven Metric Learning Paramveer S. Dhillon CIS Deptt., Univ. of Penn. Philadelphia, PA, U.S.A dhillon@cis.upenn.edu Partha Pratim Talukdar Search Labs,
Semi-Supervised Distance Metric Learning for Collaborative Image Retrieval
Semi-Supervised Distance Metric Learning for Collaborative Image Retrieval Steven C.H. Hoi School of Computer Engineering Nanyang Technological University chhoi@ntu.edu.sg Wei Liu and Shih-Fu Chang Department
on learned visual embedding patrick pérez Allegro Workshop Inria Rhônes-Alpes 22 July 2015
on learned visual embedding patrick pérez Allegro Workshop Inria Rhônes-Alpes 22 July 2015 Vector visual representation Fixed-size image representation High-dim (100 100,000) Generic, unsupervised: BoW,
Information-Theoretic Metric Learning
Jason V. Davis Brian Kulis Prateek Jain Suvrit Sra Inderjit S. Dhillon Dept. of Computer Science, University of Texas at Austin, Austin, TX 7872 Abstract In this paper, we present an information-theoretic
A Taxonomy of Semi-Supervised Learning Algorithms
A Taxonomy of Semi-Supervised Learning Algorithms Olivier Chapelle Max Planck Institute for Biological Cybernetics December 2005 Outline 1 Introduction 2 Generative models 3 Low density separation 4 Graph
Global Metric Learning by Gradient Descent
Global Metric Learning by Gradient Descent Jens Hocke and Thomas Martinetz University of Lübeck - Institute for Neuro- and Bioinformatics Ratzeburger Allee 160, 23538 Lübeck, Germany hocke@inb.uni-luebeck.de
Neighbourhood Components Analysis
Neighbourhood Components Analysis Jacob Goldberger, Sam Roweis, Geoff Hinton, Ruslan Salakhutdinov Department of Computer Science, University of Toronto {jacob,rsalakhu,hinton,roweis}@cs.toronto.edu Abstract
Large-Scale Face Manifold Learning
Large-Scale Face Manifold Learning Sanjiv Kumar Google Research New York, NY * Joint work with A. Talwalkar, H. Rowley and M. Mohri 1 Face Manifold Learning 50 x 50 pixel faces R 2500 50 x 50 pixel random
Parametric Local Metric Learning for Nearest Neighbor Classification
Parametric Local Metric Learning for Nearest Neighbor Classification Jun Wang Department of Computer Science University of Geneva Switzerland Jun.Wang@unige.ch Adam Woznica Department of Computer Science
Similarity Estimation using Bayes Ensembles
Similarity Estimation using Bayes Ensembles Tobias Emrich, Franz Graf, Hans-Peter Kriegel, Matthias Schubert, and Marisa Thoma Ludwig-Maximilians-Universität München Oettingenstr. 67, Munich, Germany {emrich,graf,kriegel,schubert,thoma}@dbs.ifi.lmu.de
Technical Report. Title: Manifold learning and Random Projections for multi-view object recognition
Technical Report Title: Manifold learning and Random Projections for multi-view object recognition Authors: Grigorios Tsagkatakis 1 and Andreas Savakis 2 1 Center for Imaging Science, Rochester Institute
Efficient Algorithm for Distance Metric Learning
Efficient lgorithm for Distance Metric Learning Yipei Wang Language Technologies Institute, Carnegie Mellon University Pittsburgh, P 15232 yipeiw@cmu.edu bstract Distance metric learning provides an approach
A Gradient-based Metric Learning Algorithm for k-nn Classifiers
A Gradient-based Metric Learning Algorithm for k-nn Classifiers Nayyar A. Zaidi 1, David McG. Squire 1, and David Suter 2 1 Clayton School of Information Technology, Monash University, VIC 38, Australia,
CSE 6242 A / CS 4803 DVA. Feb 12, Dimension Reduction. Guest Lecturer: Jaegul Choo
CSE 6242 A / CS 4803 DVA Feb 12, 2013 Dimension Reduction Guest Lecturer: Jaegul Choo CSE 6242 A / CS 4803 DVA Feb 12, 2013 Dimension Reduction Guest Lecturer: Jaegul Choo Data is Too Big To Do Something..
Metric Learning with Two-Dimensional Smoothness for Visual Analysis
Metric Learning with Two-Dimensional Smoothness for Visual Analysis Xinlei Chen Zifei Tong Haifeng Liu Deng Cai The State Key Lab of CAD&CG, College of Computer Science, Zhejiang University 388 Yu Hang
An Efficient Algorithm for Local Distance Metric Learning
An Efficient Algorithm for Local Distance Metric Learning Liu Yang and Rong Jin Michigan State University Dept. of Computer Science & Engineering East Lansing, MI 48824 {yangliu1, rongjin}@cse.msu.edu
Learning Compact and Effective Distance Metrics with Diversity Regularization. Pengtao Xie. Carnegie Mellon University
Learning Compact and Effective Distance Metrics with Diversity Regularization Pengtao Xie Carnegie Mellon University 1 Distance Metric Learning Similar Dissimilar Distance Metric Wide applications in retrieval,
Locally Smooth Metric Learning with Application to Image Retrieval
Locally Smooth Metric Learning with Application to Image Retrieval Hong Chang Xerox Research Centre Europe 6 chemin de Maupertuis, Meylan, France hong.chang@xrce.xerox.com Dit-Yan Yeung Hong Kong University
Regularized Large Margin Distance Metric Learning
2016 IEEE 16th International Conference on Data Mining Regularized Large Margin Distance Metric Learning Ya Li, Xinmei Tian CAS Key Laboratory of Technology in Geo-spatial Information Processing and Application
Fast Solvers and Efficient Implementations for Distance Metric Learning
Fast Solvers and Efficient Implementations for Distance Metric Learning Kilian Q. Weinberger kilian@yahoo-inc.com Yahoo! Research, 2821 Mission College Blvd, Santa Clara, CA 9505 Lawrence K. Saul saul@cs.ucsd.edu
Metric Learning for Large-Scale Image Classification:
Metric Learning for Large-Scale Image Classification: Generalizing to New Classes at Near-Zero Cost Florent Perronnin 1 work published at ECCV 2012 with: Thomas Mensink 1,2 Jakob Verbeek 2 Gabriela Csurka
Non-linear dimension reduction
Sta306b May 23, 2011 Dimension Reduction: 1 Non-linear dimension reduction ISOMAP: Tenenbaum, de Silva & Langford (2000) Local linear embedding: Roweis & Saul (2000) Local MDS: Chen (2006) all three methods
Dimension Reduction CS534
Dimension Reduction CS534 Why dimension reduction? High dimensionality large number of features E.g., documents represented by thousands of words, millions of bigrams Images represented by thousands of
CSE 6242 A / CX 4242 DVA. March 6, Dimension Reduction. Guest Lecturer: Jaegul Choo
CSE 6242 A / CX 4242 DVA March 6, 2014 Dimension Reduction Guest Lecturer: Jaegul Choo Data is Too Big To Analyze! Limited memory size! Data may not be fitted to the memory of your machine! Slow computation!
Stepwise Metric Adaptation Based on Semi-Supervised Learning for Boosting Image Retrieval Performance
Stepwise Metric Adaptation Based on Semi-Supervised Learning for Boosting Image Retrieval Performance Hong Chang & Dit-Yan Yeung Department of Computer Science Hong Kong University of Science and Technology
Thorsten Joachims Then: Universität Dortmund, Germany Now: Cornell University, USA
Retrospective ICML99 Transductive Inference for Text Classification using Support Vector Machines Thorsten Joachims Then: Universität Dortmund, Germany Now: Cornell University, USA Outline The paper in
Learning a Distance Metric for Structured Network Prediction. Stuart Andrews and Tony Jebara Columbia University
Learning a Distance Metric for Structured Network Prediction Stuart Andrews and Tony Jebara Columbia University Outline Introduction Context, motivation & problem definition Contributions Structured network
Semi-supervised Data Representation via Affinity Graph Learning
1 Semi-supervised Data Representation via Affinity Graph Learning Weiya Ren 1 1 College of Information System and Management, National University of Defense Technology, Changsha, Hunan, P.R China, 410073
Semi-Supervised Clustering via Learnt Codeword Distances
Semi-Supervised Clustering via Learnt Codeword Distances Dhruv Batra 1 Rahul Sukthankar 2,1 Tsuhan Chen 1 www.ece.cmu.edu/~dbatra rahuls@cs.cmu.edu tsuhan@cmu.edu 1 Carnegie Mellon University 2 Intel Research
Instance-based Learning
Instance-based Learning Machine Learning 10701/15781 Carlos Guestrin Carnegie Mellon University February 19 th, 2007 2005-2007 Carlos Guestrin 1 Why not just use Linear Regression? 2005-2007 Carlos Guestrin
Inference Driven Metric Learning (IDML) for Graph Construction
University of Pennsylvania ScholarlyCommons Technical Reports (CIS) Department of Computer & Information Science 1-1-2010 Inference Driven Metric Learning (IDML) for Graph Construction Paramveer S. Dhillon
Online Metric Learning and Fast Similarity Search
Online Metric Learning and Fast Similarity Search Prateek Jain, Brian Kulis, Inderjit S. Dhillon, and Kristen Grauman Department of Computer Sciences University of Texas at Austin Austin, TX 78712 {pjain,kulis,inderjit,grauman}@cs.utexas.edu
Locally Linear Landmarks for large-scale manifold learning
Locally Linear Landmarks for large-scale manifold learning Max Vladymyrov and Miguel Á. Carreira-Perpiñán Electrical Engineering and Computer Science University of California, Merced http://eecs.ucmerced.edu
Clustering with Multiple Graphs
Clustering with Multiple Graphs Wei Tang Department of Computer Sciences The University of Texas at Austin Austin, U.S.A wtang@cs.utexas.edu Zhengdong Lu Inst. for Computational Engineering & Sciences
Quasi Cosine Similarity Metric Learning
Quasi Cosine Similarity Metric Learning Xiang Wu, Zhi-Guo Shi and Lei Liu School of Computer and Communication Engineering, University of Science and Technology Beijing, No.30 Xueyuan Road, Haidian District,
Fast Indexing and Search. Lida Huang, Ph.D. Senior Member of Consulting Staff Magma Design Automation
Fast Indexing and Search Lida Huang, Ph.D. Senior Member of Consulting Staff Magma Design Automation Motivation Object categorization? http://www.cs.utexas.edu/~grauman/slides/jain_et_al_cvpr2008.ppt Motivation
ECS289: Scalable Machine Learning
ECS289: Scalable Machine Learning Cho-Jui Hsieh UC Davis Sept 22, 2016 Course Information Website: http://www.stat.ucdavis.edu/~chohsieh/teaching/ ECS289G_Fall2016/main.html My office: Mathematical Sciences
arxiv: v1 [stat.ml] 3 Jan 2012
![arxiv: v1 [stat.ml] 3 Jan 2012](/thumbs/84/89183585.jpg "arxiv: v1 [stat.ml] 3 Jan 2012")
Random Forests for Metric Learning with Implicit Pairwise Position Dependence arxiv:1201.0610v1 [stat.ml] 3 Jan 2012 Caiming Xiong, David Johnson, Ran Xu and Jason J. Corso Department of Computer Science
CSE 6242 / CX October 9, Dimension Reduction. Guest Lecturer: Jaegul Choo
CSE 6242 / CX 4242 October 9, 2014 Dimension Reduction Guest Lecturer: Jaegul Choo Volume Variety Big Data Era 2 Velocity Veracity 3 Big Data are High-Dimensional Examples of High-Dimensional Data Image
Image Classification from Small Sample, with Distance Learning and Feature Selection
Image Classification from Small Sample, with Distance Learning and Feature Selection Daphna Weinshall and Lior Zamir School of Computer Science and Engineering, The Hebrew University of Jerusalem, Jerusalem,
SEMI-SUPERVISED NONLINEAR DISTANCE METRIC LEARNING VIA RANDOM FOREST AND RELATIVE SIMILARITY ALGORITHM
SEMI-SUPERVISED NONLINEAR DISTANCE METRIC LEARNING VIA RANDOM FOREST AND RELATIVE SIMILARITY ALGORITHM N. Saranya 1, C. Usha Nandhini 2 1 Research Scholar, Department of Computer Science, Vellalar College
Metric Learning for Large Scale Image Classification:
Metric Learning for Large Scale Image Classification: Generalizing to New Classes at Near-Zero Cost Thomas Mensink 1,2 Jakob Verbeek 2 Florent Perronnin 1 Gabriela Csurka 1 1 TVPA - Xerox Research Centre
Large Margin Component Analysis
DRAFT (November 2, 26) Final version to appear in Proc. NIPS 26 Large Margin Component Analysis Lorenzo Torresani Riya, Inc. lorenzo@riya.com Kuang-chih Lee Riya, Inc. kclee@riya.com Abstract Metric learning
Facial Expression Recognition Using Non-negative Matrix Factorization
Facial Expression Recognition Using Non-negative Matrix Factorization Symeon Nikitidis, Anastasios Tefas and Ioannis Pitas Artificial Intelligence & Information Analysis Lab Department of Informatics Aristotle,
Semi-Crowdsourced Clustering: Generalizing Crowd Labeling by Robust Distance Metric Learning
Semi-Crowdsourced Clustering: Generalizing Crowd Labeling by Robust Distance Metric Learning Jinfeng Yi, Rong Jin, Anil K. Jain, Shaili Jain, Tianbao Yang Michigan State University, East Lansing, MI 48824,
Modelling and Visualization of High Dimensional Data. Sample Examination Paper
Duration not specified UNIVERSITY OF MANCHESTER SCHOOL OF COMPUTER SCIENCE Modelling and Visualization of High Dimensional Data Sample Examination Paper Examination date not specified Time: Examination
Learning Anisotropic RBF Kernels
Learning Anisotropic RBF Kernels Fabio Aiolli and Michele Donini University of Padova - Department of Mathematics Via Trieste, 63, 35121 Padova - Italy {aiolli,mdonini}@math.unipd.it Abstract. We present
CPSC 340: Machine Learning and Data Mining. Principal Component Analysis Fall 2016
CPSC 340: Machine Learning and Data Mining Principal Component Analysis Fall 2016 A2/Midterm: Admin Grades/solutions will be posted after class. Assignment 4: Posted, due November 14. Extra office hours:
Face2Face Comparing faces with applications Patrick Pérez. Inria, Rennes 2 Oct. 2014
Face2Face Comparing faces with applications Patrick Pérez Inria, Rennes 2 Oct. 2014 Outline Metric learning for face comparison Expandable parts model and occlusions Face sets comparison Identity-based
Hashing with Graphs. Sanjiv Kumar (Google), and Shih Fu Chang (Columbia) June, 2011
Hashing with Graphs Wei Liu (Columbia Columbia), Jun Wang (IBM IBM), Sanjiv Kumar (Google), and Shih Fu Chang (Columbia) June, 2011 Overview Graph Hashing Outline Anchor Graph Hashing Experiments Conclusions
Deep Learning for Computer Vision
Deep Learning for Computer Vision Spring 2018 http://vllab.ee.ntu.edu.tw/dlcv.html (primary) https://ceiba.ntu.edu.tw/1062dlcv (grade, etc.) FB: DLCV Spring 2018 Yu Chiang Frank Wang 王鈺強, Associate Professor
Data fusion and multi-cue data matching using diffusion maps
Data fusion and multi-cue data matching using diffusion maps Stéphane Lafon Collaborators: Raphy Coifman, Andreas Glaser, Yosi Keller, Steven Zucker (Yale University) Part of this work was supported by
A kernel-based active learning strategy for content-based image retrieval
A kernel-based active learning strategy for content-based image retrieval I. Daoudi, K. Idrissi Université de Lyon, CNRS, INSA-Lyon, LIRIS, UMR505, F-696, France {dimane, kidrissi@liris.cnrs.fr Abstract
Learning a Manifold as an Atlas Supplementary Material
Learning a Manifold as an Atlas Supplementary Material Nikolaos Pitelis Chris Russell School of EECS, Queen Mary, University of London [nikolaos.pitelis,chrisr,lourdes]@eecs.qmul.ac.uk Lourdes Agapito
Dimension reduction : PCA and Clustering
Dimension reduction : PCA and Clustering By Hanne Jarmer Slides by Christopher Workman Center for Biological Sequence Analysis DTU The DNA Array Analysis Pipeline Array design Probe design Question Experimental
IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING 1
IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING 1 Semi-Supervised Nonlinear Distance Metric Learning via Forests of Max-Margin Cluster Hierarchies David M. Johnson, Caiming Xiong, and Jason J. Corso
Sensitivity to parameter and data variations in dimensionality reduction techniques
Sensitivity to parameter and data variations in dimensionality reduction techniques Francisco J. García-Fernández 1,2,MichelVerleysen 2, John A. Lee 3 and Ignacio Díaz 1 1- Univ. of Oviedo - Department
Dimension Reduction of Image Manifolds
Dimension Reduction of Image Manifolds Arian Maleki Department of Electrical Engineering Stanford University Stanford, CA, 9435, USA E-mail: arianm@stanford.edu I. INTRODUCTION Dimension reduction of datasets
Data Preprocessing. Javier Béjar. URL - Spring 2018 CS - MAI 1/78 BY: $\
![Data Preprocessing. Javier Béjar. URL - Spring 2018 CS - MAI 1/78 BY: $\](/thumbs/89/99985995.jpg "Data Preprocessing. Javier Béjar. URL - Spring 2018 CS - MAI 1/78 BY: $\")
Data Preprocessing Javier Béjar BY: $\ URL - Spring 2018 C CS - MAI 1/78 Introduction Data representation Unstructured datasets: Examples described by a flat set of attributes: attribute-value matrix Structured
Efficient Algorithms may not be those we think
Efficient Algorithms may not be those we think Yann LeCun, Computational and Biological Learning Lab The Courant Institute of Mathematical Sciences New York University http://yann.lecun.com http://www.cs.nyu.edu/~yann
Learning Visual Similarity Measures for Comparing Never Seen Objects
Learning Visual Similarity Measures for Comparing Never Seen Objects Eric Nowak INRIA - INPG - Bertin Technologies 155 r Louis Armand 13290 Aix en Provence - France eric.nowak@inrialpes.fr Frédéric Jurie
PCCA: A New Approach for Distance Learning from Sparse Pairwise Constraints
PCCA: A New Approach for Distance Learning from Sparse Pairwise Constraints Alexis Mignon and Frédéric Jurie GREYC, CNRS UMR 6072, Université de Caen, France lastname.firstname@unicaen.fr Abstract This
Classification with Partial Labels
Classification with Partial Labels Nam Nguyen, Rich Caruana Cornell University Department of Computer Science Ithaca, New York 14853 {nhnguyen, caruana}@cs.cornell.edu ABSTRACT In this paper, we address
Traditional clustering fails if:
Traditional clustering fails if: -- in the input space the clusters are not linearly separable; -- the distance measure is not adequate; -- the assumptions limit the shape or the number of the clusters.
无监督学习中的选代表和被代表问题 - AP & LLE 张响亮. Xiangliang Zhang. King Abdullah University of Science and Technology. CNCC, Oct 25, 2018 Hangzhou, China
无监督学习中的选代表和被代表问题 - AP & LLE 张响亮 Xiangliang Zhang King Abdullah University of Science and Technology CNCC, Oct 25, 2018 Hangzhou, China Outline Affinity Propagation (AP) [Frey and Dueck, Science, 2007]
Perceptron as a graph
Neural Networks Machine Learning 10701/15781 Carlos Guestrin Carnegie Mellon University October 10 th, 2007 2005-2007 Carlos Guestrin 1 Perceptron as a graph 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0-6 -4-2
Clustering will not be satisfactory if:
Clustering will not be satisfactory if: -- in the input space the clusters are not linearly separable; -- the distance measure is not adequate; -- the assumptions limit the shape or the number of the clusters.
K-Nearest Neighbors. Jia-Bin Huang. Virginia Tech Spring 2019 ECE-5424G / CS-5824
K-Nearest Neighbors Jia-Bin Huang ECE-5424G / CS-5824 Virginia Tech Spring 2019 Administrative Check out review materials Probability Linear algebra Python and NumPy Start your HW 0 On your Local machine:
Evgeny Maksakov Advantages and disadvantages: Advantages and disadvantages: Advantages and disadvantages: Advantages and disadvantages:
Today Problems with visualizing high dimensional data Problem Overview Direct Visualization Approaches High dimensionality Visual cluttering Clarity of representation Visualization is time consuming Dimensional
Applied Statistics for Neuroscientists Part IIa: Machine Learning
Applied Statistics for Neuroscientists Part IIa: Machine Learning Dr. Seyed-Ahmad Ahmadi 04.04.2017 16.11.2017 Outline Machine Learning Difference between statistics and machine learning Modeling the problem
On Information-Maximization Clustering: Tuning Parameter Selection and Analytic Solution
ICML2011 Jun. 28-Jul. 2, 2011 On Information-Maximization Clustering: Tuning Parameter Selection and Analytic Solution Masashi Sugiyama, Makoto Yamada, Manabu Kimura, and Hirotaka Hachiya Department of
Data-driven Kernels via Semi-Supervised Clustering on the Manifold
Kernel methods, Semi-supervised learning, Manifolds, Transduction, Clustering, Instance-based learning Abstract We present an approach to transductive learning that employs semi-supervised clustering of
A Kernelized Information-Theoretic Metric Learning for Cancer Diagnosis using High-Dimensional Molecular Profiling Data
A Kernelized Information-Theoretic Metric Learning for Cancer Diagnosis using High-Dimensional Molecular Profiling Data FEIYU XIONG, Drexel University MOSHE KAM, New Jersey Institute of Technology LEONID
A Data-dependent Distance Measure for Transductive Instance-based Learning
A Data-dependent Distance Measure for Transductive Instance-based Learning Jared Lundell and DanVentura Abstract We consider learning in a transductive setting using instance-based learning (k-nn) and
SVMs and Data Dependent Distance Metric
SVMs and Data Dependent Distance Metric N. Zaidi, D. Squire Clayton School of Information Technology, Monash University, Clayton, VIC 38, Australia Email: {nayyar.zaidi,david.squire}@monash.edu Abstract
ADVANCED MACHINE LEARNING. Mini-Project Overview
11 Mini-Project Overview Lecture : Prof. Aude Billard (aude.billard@epfl.ch) Teaching Assistants : Nadia Figueroa, Ilaria Lauzana, Brice Platerrier 22 Deadlines for projects / surveys Sign up for lit.
Clustering. CS294 Practical Machine Learning Junming Yin 10/09/06
Clustering CS294 Practical Machine Learning Junming Yin 10/09/06 Outline Introduction Unsupervised learning What is clustering? Application Dissimilarity (similarity) of objects Clustering algorithm K-means,
CSE 573: Artificial Intelligence Autumn 2010
CSE 573: Artificial Intelligence Autumn 2010 Lecture 16: Machine Learning Topics 12/7/2010 Luke Zettlemoyer Most slides over the course adapted from Dan Klein. 1 Announcements Syllabus revised Machine
Semi-supervised Clustering by Input Pattern Assisted Pairwise Similarity Matrix Completion
Semi-supervised Clustering by Input Pattern Assisted Pairwise Similarity Matrix Completion Jinfeng Yi yijinfen@msu.edu Lijun Zhang zhanglij@msu.edu Rong Jin rongjin@cse.msu.edu Qi Qian qianqi@msu.edu Anil
Isometric Mapping Hashing
Isometric Mapping Hashing Yanzhen Liu, Xiao Bai, Haichuan Yang, Zhou Jun, and Zhihong Zhang Springer-Verlag, Computer Science Editorial, Tiergartenstr. 7, 692 Heidelberg, Germany {alfred.hofmann,ursula.barth,ingrid.haas,frank.holzwarth,
Stochastic k-neighborhood Selection for Supervised and Unsupervised Learning
for Supervised and Unsupervised Learning Daniel Tarlow Microsoft Research Cambridge Kevin Swersky, Laurent Charlin University of Toronto Ilya Sutskever Google Inc. Richard S. Zemel University of Toronto
Random Forests for Metric Learning with Implicit Pairwise Position Dependence
Random Forests for Metric Learning with Implicit Pairwise Position Dependence ABSTRACT Caiming Xiong Department of Computer Science and Engineering SUNY at Buffalo cxiong@buffalo.edu Ran Xu Department
Convergence of Multi-Pass Large Margin Nearest Neighbor Metric Learning
Convergence of Multi-Pass Large Margin Nearest Neighbor Metric Learning Christina Göpfert Benjamin Paassen Barbara Hammer CITEC center of excellence Bielefeld University - Germany (This is a preprint of
Robot Learning. There are generally three types of robot learning: Learning from data. Learning by demonstration. Reinforcement learning
Robot Learning 1 General Pipeline 1. Data acquisition (e.g., from 3D sensors) 2. Feature extraction and representation construction 3. Robot learning: e.g., classification (recognition) or clustering (knowledge
Fast Discriminative Component Analysis for Comparing Examples
NIPS 2006 LCE workshop Fast Discriminative Component Analysis for Comparing Examples Jaakko Peltonen 1, Jacob Goldberger 2, and Samuel Kaski 1 1 Helsinki Institute for Information Technology & Adaptive
Case-Based Reasoning. CS 188: Artificial Intelligence Fall Nearest-Neighbor Classification. Parametric / Non-parametric.
CS 188: Artificial Intelligence Fall 2008 Lecture 25: Kernels and Clustering 12/2/2008 Dan Klein UC Berkeley Case-Based Reasoning Similarity for classification Case-based reasoning Predict an instance
CS 188: Artificial Intelligence Fall 2008
CS 188: Artificial Intelligence Fall 2008 Lecture 25: Kernels and Clustering 12/2/2008 Dan Klein UC Berkeley 1 1 Case-Based Reasoning Similarity for classification Case-based reasoning Predict an instance
Applying Supervised Learning
Applying Supervised Learning When to Consider Supervised Learning A supervised learning algorithm takes a known set of input data (the training set) and known responses to the data (output), and trains
Is that you? Metric learning approaches for face identification
Is that you? Metric learning approaches for face identification Matthieu Guillaumin, Jakob Verbeek, Cordelia Schmid To cite this version: Matthieu Guillaumin, Jakob Verbeek, Cordelia Schmid. Is that you?
Machine Learning / Jan 27, 2010
Revisiting Logistic Regression & Naïve Bayes Aarti Singh Machine Learning 10-701/15-781 Jan 27, 2010 Generative and Discriminative Classifiers Training classifiers involves learning a mapping f: X -> Y,
Overview Citation. ML Introduction. Overview Schedule. ML Intro Dataset. Introduction to Semi-Supervised Learning Review 10/4/2010
INFORMATICS SEMINAR SEPT. 27 & OCT. 4, 2010 Introduction to Semi-Supervised Learning Review 2 Overview Citation X. Zhu and A.B. Goldberg, Introduction to Semi- Supervised Learning, Morgan & Claypool Publishers,
Unsupervised Learning
Unsupervised Learning Learning without Class Labels (or correct outputs) Density Estimation Learn P(X) given training data for X Clustering Partition data into clusters Dimensionality Reduction Discover
Instances on a Budget
Retrieving Similar or Informative Instances on a Budget Kristen Grauman Dept. of Computer Science University of Texas at Austin Work with Sudheendra Vijayanarasimham Work with Sudheendra Vijayanarasimham,
Preface to the Second Edition. Preface to the First Edition. 1 Introduction 1
Preface to the Second Edition Preface to the First Edition vii xi 1 Introduction 1 2 Overview of Supervised Learning 9 2.1 Introduction... 9 2.2 Variable Types and Terminology... 9 2.3 Two Simple Approaches
Unsupervised learning in Vision
Chapter 7 Unsupervised learning in Vision The fields of Computer Vision and Machine Learning complement each other in a very natural way: the aim of the former is to extract useful information from visual