On Information-Maximization Clustering: Tuning Parameter Selection and Analytic Solution
|
|
|
- Christina Harmon
- 6 years ago
- Views:
Transcription
1 ICML2011 Jun. 28-Jul. 2, 2011 On Information-Maximization Clustering: Tuning Parameter Selection and Analytic Solution Masashi Sugiyama, Makoto Yamada, Manabu Kimura, and Hirotaka Hachiya Department of Computer Science Tokyo Institute of Technology
2 Goal of Clustering 2 Given unlabeled samples, assign cluster labels so that Samples in the same cluster are similar. Samples in other clusters are dissimilar. Throughout this talk, we assume is known.
3 Contents 3 1. Problem formulation 2. Review of existing approaches 3. Proposed method A) Clustering B) Tuning parameter optimization 4. Experiments
4 Model-based Clustering 4 Learn a mixture model by maximum-likelihood or Bayes estimation: K-means Dirichlet process mixture Pros and cons: (MacQueen, 1967) (Ferguson, 1973) No tuning parameters. Cluster shape depends on pre-defined cluster models ( Gaussian). Initialization is difficult.
5 Model-free Clustering 5 No parametric assumption on clusters: Spectral clustering: K-means after non-linear manifold embedding (Shi & Malik, 2000; Ng et al., 2002) Discriminative clustering: Learn a classifier and cluster labels simultaneously (Xu et al., 2005; Bach & Harchaoui, 2008) Dependence maximization: Determine labels so that dependence on samples is maximized (Song et al., 2007; Faivishevsky & Goldberger, 2010) Information maximization: Learn a classifier so that some information measure is maximized (Agakov & Barberu, 2006; Gomes et al., 2010)
6 Model-free Clustering (cont.) 6 Pros and cons: Cluster shape is flexible. Kernel/similarity parameter choice is difficult. Initialization is difficult.
7 Contents 7 1. Problem formulation 2. Review of existing approaches 3. Proposed method A) Clustering B) Tuning parameter optimization 4. Experiments
8 Goal of Our Research 8 We propose a new information-maximization clustering method: Global analytic solution is available. Objective tuning-parameter choice is possible. In the proposed method: A non-parametric kernel classifier is learned so that an information measure is maximized. Tuning parameters are chosen so that an information measure is maximized.
9 Squared-loss Mutual Information 9 (SMI) As an information measure, we use SMI: Ordinary MI is the KL divergence. SMI is the Pearson (PE) divergence. Both KL and PE are f-divergences (thus they have similar properties). Indeed, as ordinary MI, SMI satisfies
10 Contents Problem formulation 2. Review of existing approaches 3. Proposed method A) Clustering B) Tuning parameter optimization 4. Experiments
11 Kernel Probabilistic Classifier 11 Kernel probabilistic classifier: Learn the classifier so that SMI is maximized. Challenge: only is available for training
12 SMI Approximation 12 Approximate cluster-posterior by kernel model: Approximate expectation by sample average: Assume cluster-prior is uniform: : # clusters Then we obtain the following SMI approximator:
13 Maximizing SMI Approximator 13 Under mutual orthonormality of, a solution is given by principal components of kernel matrix. Similar to Ding & He (ICML2004)
14 SMI-based Clustering (SMIC) 14 Post-processing: Adjusting sign of principal components : :Vector with all ones Normalization according to. Rounding-up negative probability estimates to 0. Final solution (analytically computable): : -th element of a vector :Vector with all zeros
15 Contents Problem formulation 2. Review of existing approaches 3. Proposed method A) Clustering B) Tuning parameter optimization 4. Experiments
16 Tuning Parameter Choice 16 Solution of SMIC depends on kernel functions. We determine kernels so that SMI is maximized. We may use the same for this purpose. However, is not accurate enough since it is an unsupervised estimator of SMI. In the phase of tuning parameter choice, estimated labels are available!
17 Supervised SMI Estimator 17 Least-squares mutual information (LSMI): Directly estimate the density ratio Suzuki & Sugiyama (AISTATS2010) without going through density estimation. Density-ratio estimation is substantially easier than density estimation (à lavapnik). Knowing Knowing
18 Density-Ratio Estimation 18 Kernel density-ratio model: Least-squares fitting: : Kernel function (We use Gaussian kernel)
19 Density-Ratio Estimation (cont.) 19 Empirical and regularized training criterion: Global solution can be obtained analytically: Kernel and regularization parameter can be determined by cross-validation.
20 20 Least-Squares Mutual Information (LSMI) SMI approximator is given analytically as LSMI achieves a fast non-parametric convergence rate! Suzuki & Sugiyama (AISTATS2010) We determine the kernel function in SMIC so that LSMI is maximized.
21 Summary of Proposed Method 21 SMI Clustering with LSMI: Input: Unlabeled samples Kernel candidates Output: Cluster labels SMIC LSMI
22 Contents Problem formulation 2. Review of existing approaches 3. Proposed method A) Clustering B) Tuning parameter optimization 4. Experiments
23 Experimental Setup 23 For SMIC, we use a sparse variant of the local scaling kernel: (Zelnik-Manor & Perona, NIPS2004) : -th neighbor of Tuning parameter is determined by LSMI maximization.
24 Illustration of SMIC SMI estimate SMI estimate SMI estimate SMI estimate t t t t SMIC with model selection by LSMI works well!
25 Performance Comparison 25 KM: K-means clustering SC: Self-tuning spectral clustering (MacQueen, 1967) (Zelnik-Manor & Perona, NIPS2004) MNN: Dependence-maximization clustering based on mean nearest neighbor approximation (Faivishevsky & Goldberger, ICML2010) MIC: Information-maximization clustering for kernel logistic models with model selection by maximum-likelihood mutual information (Gomes, Krause & Perona, NIPS2010) (Suzuki, Sugiyama, Sese & Kanamori, FSDM2008)
26 Experimental Results 26 Digit (d =256,n=5000, and c =10) KM SC MNN MIC SMIC ARI 0.42(0.01) 0.24(0.02) 0.44(0.03) 0.63(0.08) 0.63(0.05) Time [3631.7] 14.4[359.5] Face (d = 4096,n=100, and c = 10) KM SC MNN MIC SMIC ARI 0.60(0.11) 0.62(0.11) 0.47(0.10) 0.64(0.12) 0.65(0.11) Time [30.8] 0.0[19.3] 20News Group Document (d =50,n= 700, and c =7) KM SC MNN MIC SMIC ARI 0.00(0.00) 0.09(0.02) 0.09(0.02) 0.01(0.02) 0.19(0.03) Time [530.5] 0.3[115.3] Sens eval2 Word (d =50,n =300, and c =3) KM SC MNN MIC SMIC ARI 0.04(0.05) 0.02(0.01) 0.02(0.02) 0.04(0.04) 0.08(0.05) Time [369.6] 0.2[203.9] Accelerometry (d = 5,n = 300, and c = 3) KM SC MNN MIC SMIC ARI 0.49(0.04) 0.58(0.14) 0.71(0.05) 0.57(0.23) 0.68(0.12) Time [410.6] 0.2[92.6] Speech (d =50,n=400, and c =2) KM SC MNN MIC SMIC ARI 0.00(0.00) 0.00(0.00) 0.04(0.15) 0.18(0.16) 0.21(0.25) Time [413.4] 0.3[179.7] Adjusted Rand index (ARI): larger is better Red: Best or comparable by 1%t-test SMIC works well and computationally efficient!
27 Conclusions 27 Weaknesses of existing clustering methods: Cluster initialization is difficult. Tuning parameter choice is difficult. SMIC: A new information-maximization clustering method based on squared-loss mutual information (SMI): Analytic global solution is available. Objective tuning parameter choice is possible. MATLAB code is available from
28 Other Usage of SMI 28 Feature selection Suzuki, Sugiyama, Sese & Kanamori (BMC Bioinfo. 2009) Dimensionality reduction Suzuki & Sugiyama (AISTATS2010) Yamada, Niu, Takagi & Sugiyama (ArXiv2011) Independent component analysis Independence test Causal inference Suzuki & Sugiyama (Neural Comp. 2011) Sugiyama & Suzuki (IEICE-ED2011) Yamada & Sugiyama (AAAI2010)
Semi-Supervised Information-Maximization Clustering
Neural Networks, vol.57, pp.103 111, 2014. 1 Semi-Supervised Information-Maximization Clustering Daniele Calandriello Politecnico di Milano, Milano, Italy daniele.calandriello@mail.polimi.it Gang Niu Baidu
Dependence-Maximization Clustering with Least-Squares Mutual Information
Journal of Advanced Computational Intelligence and Intelligent Informatics. vol.5, no.7, pp.800-805, 20. Dependence-Maximization Clustering with Least-Squares Mutual Information Manabu Kimura Tokyo Institute
Clustering via Mode Seeking by Direct Estimation of the Gradient of a Log-Density
Clustering via Mode Seeking by Direct Estimation of the Gradient of a Log-Density Hiroaki Sasaki 1, Aapo Hyvärinen 2,3, and Masashi Sugiyama 1 1 Graduate School of Information Science and Engineering,
Contents. Preface to the Second Edition
Preface to the Second Edition v 1 Introduction 1 1.1 What Is Data Mining?....................... 4 1.2 Motivating Challenges....................... 5 1.3 The Origins of Data Mining....................
On Order-Constrained Transitive Distance
On Order-Constrained Transitive Distance Author: Zhiding Yu and B. V. K. Vijaya Kumar, et al. Dept of Electrical and Computer Engineering Carnegie Mellon University 1 Clustering Problem Important Issues:
A Taxonomy of Semi-Supervised Learning Algorithms
A Taxonomy of Semi-Supervised Learning Algorithms Olivier Chapelle Max Planck Institute for Biological Cybernetics December 2005 Outline 1 Introduction 2 Generative models 3 Low density separation 4 Graph
Machine Learning. Unsupervised Learning. Manfred Huber
Machine Learning Unsupervised Learning Manfred Huber 2015 1 Unsupervised Learning In supervised learning the training data provides desired target output for learning In unsupervised learning the training
Intro to Artificial Intelligence
Intro to Artificial Intelligence Ahmed Sallam { Lecture 5: Machine Learning ://. } ://.. 2 Review Probabilistic inference Enumeration Approximate inference 3 Today What is machine learning? Supervised
Overview Citation. ML Introduction. Overview Schedule. ML Intro Dataset. Introduction to Semi-Supervised Learning Review 10/4/2010
INFORMATICS SEMINAR SEPT. 27 & OCT. 4, 2010 Introduction to Semi-Supervised Learning Review 2 Overview Citation X. Zhu and A.B. Goldberg, Introduction to Semi- Supervised Learning, Morgan & Claypool Publishers,
COMP 551 Applied Machine Learning Lecture 13: Unsupervised learning
COMP 551 Applied Machine Learning Lecture 13: Unsupervised learning Associate Instructor: Herke van Hoof (herke.vanhoof@mail.mcgill.ca) Slides mostly by: (jpineau@cs.mcgill.ca) Class web page: www.cs.mcgill.ca/~jpineau/comp551
The K-modes and Laplacian K-modes algorithms for clustering
The K-modes and Laplacian K-modes algorithms for clustering Miguel Á. Carreira-Perpiñán Electrical Engineering and Computer Science University of California, Merced http://faculty.ucmerced.edu/mcarreira-perpinan
MTTS1 Dimensionality Reduction and Visualization Spring 2014, 5op Jaakko Peltonen
MTTS1 Dimensionality Reduction and Visualization Spring 2014, 5op Jaakko Peltonen Lecture 9: Metric Learning Motivation metric learning Metric learning means learning a better metric (better distance function)
ECG782: Multidimensional Digital Signal Processing
ECG782: Multidimensional Digital Signal Processing Object Recognition http://www.ee.unlv.edu/~b1morris/ecg782/ 2 Outline Knowledge Representation Statistical Pattern Recognition Neural Networks Boosting
Machine Learning and Data Mining. Clustering (1): Basics. Kalev Kask
Machine Learning and Data Mining Clustering (1): Basics Kalev Kask Unsupervised learning Supervised learning Predict target value ( y ) given features ( x ) Unsupervised learning Understand patterns of
MTTTS17 Dimensionality Reduction and Visualization. Spring 2018 Jaakko Peltonen. Lecture 11: Neighbor Embedding Methods continued
MTTTS17 Dimensionality Reduction and Visualization Spring 2018 Jaakko Peltonen Lecture 11: Neighbor Embedding Methods continued This Lecture Neighbor embedding by generative modeling Some supervised neighbor
Object Segmentation and Tracking in 3D Video With Sparse Depth Information Using a Fully Connected CRF Model
Object Segmentation and Tracking in 3D Video With Sparse Depth Information Using a Fully Connected CRF Model Ido Ofir Computer Science Department Stanford University December 17, 2011 Abstract This project
Lecture 11: Clustering Introduction and Projects Machine Learning
Lecture 11: Clustering Introduction and Projects Machine Learning Andrew Rosenberg March 12, 2010 1/1 Last Time Junction Tree Algorithm Efficient Marginals in Graphical Models 2/1 Today Clustering Project
Machine Learning (BSMC-GA 4439) Wenke Liu
Machine Learning (BSMC-GA 4439) Wenke Liu 01-31-017 Outline Background Defining proximity Clustering methods Determining number of clusters Comparing two solutions Cluster analysis as unsupervised Learning
Visual Representations for Machine Learning
Visual Representations for Machine Learning Spectral Clustering and Channel Representations Lecture 1 Spectral Clustering: introduction and confusion Michael Felsberg Klas Nordberg The Spectral Clustering
Supervised vs. Unsupervised Learning
Clustering Supervised vs. Unsupervised Learning So far we have assumed that the training samples used to design the classifier were labeled by their class membership (supervised learning) We assume now
Generative and discriminative classification techniques
Generative and discriminative classification techniques Machine Learning and Category Representation 2014-2015 Jakob Verbeek, November 28, 2014 Course website: http://lear.inrialpes.fr/~verbeek/mlcr.14.15
Thorsten Joachims Then: Universität Dortmund, Germany Now: Cornell University, USA
Retrospective ICML99 Transductive Inference for Text Classification using Support Vector Machines Thorsten Joachims Then: Universität Dortmund, Germany Now: Cornell University, USA Outline The paper in
Clustering Lecture 5: Mixture Model
Clustering Lecture 5: Mixture Model Jing Gao SUNY Buffalo 1 Outline Basics Motivation, definition, evaluation Methods Partitional Hierarchical Density-based Mixture model Spectral methods Advanced topics
Machine Learning. Supervised Learning. Manfred Huber
Machine Learning Supervised Learning Manfred Huber 2015 1 Supervised Learning Supervised learning is learning where the training data contains the target output of the learning system. Training data D
Cluster Analysis (b) Lijun Zhang
Cluster Analysis (b) Lijun Zhang zlj@nju.edu.cn http://cs.nju.edu.cn/zlj Outline Grid-Based and Density-Based Algorithms Graph-Based Algorithms Non-negative Matrix Factorization Cluster Validation Summary
IBL and clustering. Relationship of IBL with CBR
IBL and clustering Distance based methods IBL and knn Clustering Distance based and hierarchical Probability-based Expectation Maximization (EM) Relationship of IBL with CBR + uses previously processed
Generative and discriminative classification techniques
Generative and discriminative classification techniques Machine Learning and Category Representation 013-014 Jakob Verbeek, December 13+0, 013 Course website: http://lear.inrialpes.fr/~verbeek/mlcr.13.14
Unsupervised Learning
Unsupervised Learning Pierre Gaillard ENS Paris September 28, 2018 1 Supervised vs unsupervised learning Two main categories of machine learning algorithms: - Supervised learning: predict output Y from
Preface to the Second Edition. Preface to the First Edition. 1 Introduction 1
Preface to the Second Edition Preface to the First Edition vii xi 1 Introduction 1 2 Overview of Supervised Learning 9 2.1 Introduction... 9 2.2 Variable Types and Terminology... 9 2.3 Two Simple Approaches
Regularized Multi-Task Learning for Multi-Dimensional Log-Density Gradient Estimation
1 arxiv:1508.00085v1 [stat.ml] 1 Aug 2015 Regularized Multi-Task Learning for Multi-Dimensional Log-Density Gradient Estimation Ikko Yamane, Hiroaki Sasaki, Masashi Sugiyama Graduate School of Frontier
Aarti Singh. Machine Learning / Slides Courtesy: Eric Xing, M. Hein & U.V. Luxburg
Spectral Clustering Aarti Singh Machine Learning 10-701/15-781 Apr 7, 2010 Slides Courtesy: Eric Xing, M. Hein & U.V. Luxburg 1 Data Clustering Graph Clustering Goal: Given data points X1,, Xn and similarities
Transductive Phoneme Classification Using Local Scaling And Confidence
202 IEEE 27-th Convention of Electrical and Electronics Engineers in Israel Transductive Phoneme Classification Using Local Scaling And Confidence Matan Orbach Dept. of Electrical Engineering Technion
Performance Evaluation of Various Classification Algorithms
Performance Evaluation of Various Classification Algorithms Shafali Deora Amritsar College of Engineering & Technology, Punjab Technical University -----------------------------------------------------------***----------------------------------------------------------
Machine Learning. Semi-Supervised Learning. Manfred Huber
Machine Learning Semi-Supervised Learning Manfred Huber 2015 1 Semi-Supervised Learning Semi-supervised learning refers to learning from data where part contains desired output information and the other
Cross-Domain Object Matching with Model Selection
Makoto Yamada Tokyo Institute of Technology yamada@sg.cs.titech.ac.jp Masashi Sugiyama Tokyo Institute of Technology sugi@cs.titech.ac.jp Abstract The goal of cross-domain object matching (CDOM) is to
Machine Learning : Clustering, Self-Organizing Maps
Machine Learning Clustering, Self-Organizing Maps 12/12/2013 Machine Learning : Clustering, Self-Organizing Maps Clustering The task: partition a set of objects into meaningful subsets (clusters). The
Pattern Recognition. Kjell Elenius. Speech, Music and Hearing KTH. March 29, 2007 Speech recognition
Pattern Recognition Kjell Elenius Speech, Music and Hearing KTH March 29, 2007 Speech recognition 2007 1 Ch 4. Pattern Recognition 1(3) Bayes Decision Theory Minimum-Error-Rate Decision Rules Discriminant
Bioinformatics - Lecture 07
Bioinformatics - Lecture 07 Bioinformatics Clusters and networks Martin Saturka http://www.bioplexity.org/lectures/ EBI version 0.4 Creative Commons Attribution-Share Alike 2.5 License Learning on profiles
COSC160: Detection and Classification. Jeremy Bolton, PhD Assistant Teaching Professor
COSC160: Detection and Classification Jeremy Bolton, PhD Assistant Teaching Professor Outline I. Problem I. Strategies II. Features for training III. Using spatial information? IV. Reducing dimensionality
Unsupervised Learning
Unsupervised Learning Learning without Class Labels (or correct outputs) Density Estimation Learn P(X) given training data for X Clustering Partition data into clusters Dimensionality Reduction Discover
Introduction to Pattern Recognition Part II. Selim Aksoy Bilkent University Department of Computer Engineering
Introduction to Pattern Recognition Part II Selim Aksoy Bilkent University Department of Computer Engineering saksoy@cs.bilkent.edu.tr RETINA Pattern Recognition Tutorial, Summer 2005 Overview Statistical
Machine Learning. Topic 5: Linear Discriminants. Bryan Pardo, EECS 349 Machine Learning, 2013
Machine Learning Topic 5: Linear Discriminants Bryan Pardo, EECS 349 Machine Learning, 2013 Thanks to Mark Cartwright for his extensive contributions to these slides Thanks to Alpaydin, Bishop, and Duda/Hart/Stork
Computing Gaussian Mixture Models with EM using Equivalence Constraints
Computing Gaussian Mixture Models with EM using Equivalence Constraints Noam Shental, Aharon Bar-Hillel, Tomer Hertz and Daphna Weinshall email: tomboy,fenoam,aharonbh,daphna@cs.huji.ac.il School of Computer
Probabilistic Graphical Models
Overview of Part Two Probabilistic Graphical Models Part Two: Inference and Learning Christopher M. Bishop Exact inference and the junction tree MCMC Variational methods and EM Example General variational
On Classification: An Empirical Study of Existing Algorithms Based on Two Kaggle Competitions
On Classification: An Empirical Study of Existing Algorithms Based on Two Kaggle Competitions CAMCOS Report Day December 9th, 2015 San Jose State University Project Theme: Classification The Kaggle Competition
Unsupervised learning in Vision
Chapter 7 Unsupervised learning in Vision The fields of Computer Vision and Machine Learning complement each other in a very natural way: the aim of the former is to extract useful information from visual
10-701/15-781, Fall 2006, Final
-7/-78, Fall 6, Final Dec, :pm-8:pm There are 9 questions in this exam ( pages including this cover sheet). If you need more room to work out your answer to a question, use the back of the page and clearly
Machine Learning. Nonparametric methods for Classification. Eric Xing , Fall Lecture 2, September 12, 2016
Machine Learning 10-701, Fall 2016 Nonparametric methods for Classification Eric Xing Lecture 2, September 12, 2016 Reading: 1 Classification Representing data: Hypothesis (classifier) 2 Clustering 3 Supervised
Globally and Locally Consistent Unsupervised Projection
Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence Globally and Locally Consistent Unsupervised Projection Hua Wang, Feiping Nie, Heng Huang Department of Electrical Engineering
Density estimation. In density estimation problems, we are given a random from an unknown density. Our objective is to estimate
Density estimation In density estimation problems, we are given a random sample from an unknown density Our objective is to estimate? Applications Classification If we estimate the density for each class,
Clustering K-means. Machine Learning CSEP546 Carlos Guestrin University of Washington February 18, Carlos Guestrin
Clustering K-means Machine Learning CSEP546 Carlos Guestrin University of Washington February 18, 2014 Carlos Guestrin 2005-2014 1 Clustering images Set of Images [Goldberger et al.] Carlos Guestrin 2005-2014
Clustering. CS294 Practical Machine Learning Junming Yin 10/09/06
Clustering CS294 Practical Machine Learning Junming Yin 10/09/06 Outline Introduction Unsupervised learning What is clustering? Application Dissimilarity (similarity) of objects Clustering algorithm K-means,
Clustering. Supervised vs. Unsupervised Learning
Clustering Supervised vs. Unsupervised Learning So far we have assumed that the training samples used to design the classifier were labeled by their class membership (supervised learning) We assume now
Pattern Recognition Letters
Pattern Recognition Letters 33 () 6 6 Contents lists available at SciVerse ScienceDirect Pattern Recognition Letters journal homepage: www.elsevier.com/locate/patrec An unsupervised data projection that
Grundlagen der Künstlichen Intelligenz
Grundlagen der Künstlichen Intelligenz Unsupervised learning Daniel Hennes 29.01.2018 (WS 2017/18) University Stuttgart - IPVS - Machine Learning & Robotics 1 Today Supervised learning Regression (linear
Machine Learning Lecture 3
Course Outline Machine Learning Lecture 3 Fundamentals (2 weeks) Bayes Decision Theory Probability Density Estimation Probability Density Estimation II 26.04.206 Discriminative Approaches (5 weeks) Linear
Introduction to spectral clustering
Introduction to spectral clustering Denis Hamad LASL ULCO Denis.Hamad@lasl.univ-littoral.fr Philippe Biela HEI LAGIS Philippe.Biela@hei.fr Data Clustering Data clustering Data clustering is an important
Supervised Learning for Image Segmentation
Supervised Learning for Image Segmentation Raphael Meier 06.10.2016 Raphael Meier MIA 2016 06.10.2016 1 / 52 References A. Ng, Machine Learning lecture, Stanford University. A. Criminisi, J. Shotton, E.
MULTIVARIATE TEXTURE DISCRIMINATION USING A PRINCIPAL GEODESIC CLASSIFIER
MULTIVARIATE TEXTURE DISCRIMINATION USING A PRINCIPAL GEODESIC CLASSIFIER A.Shabbir 1, 2 and G.Verdoolaege 1, 3 1 Department of Applied Physics, Ghent University, B-9000 Ghent, Belgium 2 Max Planck Institute
CS 195-5: Machine Learning Problem Set 5
CS 195-5: Machine Learning Problem Set 5 Douglas Lanman dlanman@brown.edu 26 November 26 1 Clustering and Vector Quantization Problem 1 Part 1: In this problem we will apply Vector Quantization (VQ) to
Cross-Domain Object Matching with Model Selection
Cross-Domain Object Matching with Model Selection Makoto Yamada Tokyo Institute of Technology yamada@sg.cs.titech.ac.jp Masashi Sugiyama Tokyo Institute of Technology sugi@cs.titech.ac.jp arxiv:2.46v [stat.ml]
MACHINE LEARNING: CLUSTERING, AND CLASSIFICATION. Steve Tjoa June 25, 2014
MACHINE LEARNING: CLUSTERING, AND CLASSIFICATION Steve Tjoa kiemyang@gmail.com June 25, 2014 Review from Day 2 Supervised vs. Unsupervised Unsupervised - clustering Supervised binary classifiers (2 classes)
Machine Learning (BSMC-GA 4439) Wenke Liu
Machine Learning (BSMC-GA 4439) Wenke Liu 01-25-2018 Outline Background Defining proximity Clustering methods Determining number of clusters Other approaches Cluster analysis as unsupervised Learning Unsupervised
Information-Theoretic Feature Selection Algorithms for Text Classification
Proceedings of International Joint Conference on Neural Networks, Montreal, Canada, July 31 - August 4, 5 Information-Theoretic Feature Selection Algorithms for Text Classification Jana Novovičová Institute
Homework. Gaussian, Bishop 2.3 Non-parametric, Bishop 2.5 Linear regression Pod-cast lecture on-line. Next lectures:
Homework Gaussian, Bishop 2.3 Non-parametric, Bishop 2.5 Linear regression 3.0-3.2 Pod-cast lecture on-line Next lectures: I posted a rough plan. It is flexible though so please come with suggestions Bayes
ADVANCED MACHINE LEARNING. Mini-Project Overview
11 Mini-Project Overview Lecture : Prof. Aude Billard (aude.billard@epfl.ch) Teaching Assistants : Nadia Figueroa, Ilaria Lauzana, Brice Platerrier 22 Deadlines for projects / surveys Sign up for lit.
IMAGE ANALYSIS, CLASSIFICATION, and CHANGE DETECTION in REMOTE SENSING
SECOND EDITION IMAGE ANALYSIS, CLASSIFICATION, and CHANGE DETECTION in REMOTE SENSING ith Algorithms for ENVI/IDL Morton J. Canty с*' Q\ CRC Press Taylor &. Francis Group Boca Raton London New York CRC
Kernel-based Transductive Learning with Nearest Neighbors
Kernel-based Transductive Learning with Nearest Neighbors Liangcai Shu, Jinhui Wu, Lei Yu, and Weiyi Meng Dept. of Computer Science, SUNY at Binghamton Binghamton, New York 13902, U. S. A. {lshu,jwu6,lyu,meng}@cs.binghamton.edu
Feature Selection. CE-725: Statistical Pattern Recognition Sharif University of Technology Spring Soleymani
Feature Selection CE-725: Statistical Pattern Recognition Sharif University of Technology Spring 2013 Soleymani Outline Dimensionality reduction Feature selection vs. feature extraction Filter univariate
CAMCOS Report Day. December 9 th, 2015 San Jose State University Project Theme: Classification
CAMCOS Report Day December 9 th, 2015 San Jose State University Project Theme: Classification On Classification: An Empirical Study of Existing Algorithms based on two Kaggle Competitions Team 1 Team 2
Generalized Transitive Distance with Minimum Spanning Random Forest
Generalized Transitive Distance with Minimum Spanning Random Forest Author: Zhiding Yu and B. V. K. Vijaya Kumar, et al. Dept of Electrical and Computer Engineering Carnegie Mellon University 1 Clustering
Data Mining Chapter 9: Descriptive Modeling Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University
Data Mining Chapter 9: Descriptive Modeling Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University Descriptive model A descriptive model presents the main features of the data
Using Machine Learning to Optimize Storage Systems
Using Machine Learning to Optimize Storage Systems Dr. Kiran Gunnam 1 Outline 1. Overview 2. Building Flash Models using Logistic Regression. 3. Storage Object classification 4. Storage Allocation recommendation
CPSC 340: Machine Learning and Data Mining. Principal Component Analysis Fall 2016
CPSC 340: Machine Learning and Data Mining Principal Component Analysis Fall 2016 A2/Midterm: Admin Grades/solutions will be posted after class. Assignment 4: Posted, due November 14. Extra office hours:
http://www.xkcd.com/233/ Text Clustering David Kauchak cs160 Fall 2009 adapted from: http://www.stanford.edu/class/cs276/handouts/lecture17-clustering.ppt Administrative 2 nd status reports Paper review
ISyE 6416 Basic Statistical Methods Spring 2016 Bonus Project: Big Data Report
ISyE 6416 Basic Statistical Methods Spring 2016 Bonus Project: Big Data Report Team Member Names: Caroline Roeger, Damon Frezza Project Title: Clustering and Classification of Handwritten Digits Responsibilities:
Laplacian Eigenmaps and Bayesian Clustering Based Layout Pattern Sampling and Its Applications to Hotspot Detection and OPC
Laplacian Eigenmaps and Bayesian Clustering Based Layout Pattern Sampling and Its Applications to Hotspot Detection and OPC Tetsuaki Matsunawa 1, Bei Yu 2 and David Z. Pan 3 1 Toshiba Corporation 2 The
Deep Generative Models Variational Autoencoders
Deep Generative Models Variational Autoencoders Sudeshna Sarkar 5 April 2017 Generative Nets Generative models that represent probability distributions over multiple variables in some way. Directed Generative
Deep Learning for Computer Vision
Deep Learning for Computer Vision Spring 2018 http://vllab.ee.ntu.edu.tw/dlcv.html (primary) https://ceiba.ntu.edu.tw/1062dlcv (grade, etc.) FB: DLCV Spring 2018 Yu Chiang Frank Wang 王鈺強, Associate Professor
CSE 6242 A / CX 4242 DVA. March 6, Dimension Reduction. Guest Lecturer: Jaegul Choo
CSE 6242 A / CX 4242 DVA March 6, 2014 Dimension Reduction Guest Lecturer: Jaegul Choo Data is Too Big To Analyze! Limited memory size! Data may not be fitted to the memory of your machine! Slow computation!
Machine Learning Lecture 3
Many slides adapted from B. Schiele Machine Learning Lecture 3 Probability Density Estimation II 26.04.2016 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Course
CS6375: Machine Learning Gautam Kunapuli. Mid-Term Review
Gautam Kunapuli Machine Learning Data is identically and independently distributed Goal is to learn a function that maps to Data is generated using an unknown function Learn a hypothesis that minimizes
MULTIVARIATE ANALYSES WITH fmri DATA
MULTIVARIATE ANALYSES WITH fmri DATA Sudhir Shankar Raman Translational Neuromodeling Unit (TNU) Institute for Biomedical Engineering University of Zurich & ETH Zurich Motivation Modelling Concepts Learning
CS 229 Midterm Review
CS 229 Midterm Review Course Staff Fall 2018 11/2/2018 Outline Today: SVMs Kernels Tree Ensembles EM Algorithm / Mixture Models [ Focus on building intuition, less so on solving specific problems. Ask
Feature Selection for Image Retrieval and Object Recognition
Feature Selection for Image Retrieval and Object Recognition Nuno Vasconcelos et al. Statistical Visual Computing Lab ECE, UCSD Presented by Dashan Gao Scalable Discriminant Feature Selection for Image
Unsupervised Learning and Clustering
Unsupervised Learning and Clustering Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr CS 551, Spring 2009 CS 551, Spring 2009 c 2009, Selim Aksoy (Bilkent University)
Latent Domains Modeling for Visual Domain Adaptation
Latent Domains Modeling for Visual Domain Adaptation Caiming Xiong SUNY at Buffalo cxiong@buffalo.edu Scott McCloskey Honeywell ACS Scott.McCloskey@honeywell.com Shao-Hang Hsieh SUNY at Buffalo shaohang@buffalo.edu
Pairwise Clustering based on the Mutual-Information Criterion
Pairwise Clustering based on the Mutual-Information Criterion Amir Alush, Avishay Friedman, Jacob Goldberger Faculty of Engineering, Bar-Ilan University, Ramat-Gan 52900, Israel Abstract Pairwise clustering
Supervised vs unsupervised clustering
Classification Supervised vs unsupervised clustering Cluster analysis: Classes are not known a- priori. Classification: Classes are defined a-priori Sometimes called supervised clustering Extract useful
What is machine learning?
Machine learning, pattern recognition and statistical data modelling Lecture 12. The last lecture Coryn Bailer-Jones 1 What is machine learning? Data description and interpretation finding simpler relationship
Structured Models in. Dan Huttenlocher. June 2010
Structured Models in Computer Vision i Dan Huttenlocher June 2010 Structured Models Problems where output variables are mutually dependent or constrained E.g., spatial or temporal relations Such dependencies
Machine Learning / Jan 27, 2010
Revisiting Logistic Regression & Naïve Bayes Aarti Singh Machine Learning 10-701/15-781 Jan 27, 2010 Generative and Discriminative Classifiers Training classifiers involves learning a mapping f: X -> Y,
1 Introduction. 3 Data Preprocessing. 2 Literature Review
Rock or not? This sure does. [Category] Audio & Music CS 229 Project Report Anand Venkatesan(anand95), Arjun Parthipan(arjun777), Lakshmi Manoharan(mlakshmi) 1 Introduction Music Genre Classification continues
Discriminative Clustering for Image Co-segmentation
Discriminative Clustering for Image Co-segmentation Armand Joulin Francis Bach Jean Ponce INRIA Ecole Normale Supérieure, Paris January 2010 Introduction Introduction Task: dividing simultaneously q images
Recap: Gaussian (or Normal) Distribution. Recap: Minimizing the Expected Loss. Topics of This Lecture. Recap: Maximum Likelihood Approach
Truth Course Outline Machine Learning Lecture 3 Fundamentals (2 weeks) Bayes Decision Theory Probability Density Estimation Probability Density Estimation II 2.04.205 Discriminative Approaches (5 weeks)
Unsupervised Learning
Deep Learning for Graphics Unsupervised Learning Niloy Mitra Iasonas Kokkinos Paul Guerrero Vladimir Kim Kostas Rematas Tobias Ritschel UCL UCL/Facebook UCL Adobe Research U Washington UCL Timetable Niloy
CSE 6242 A / CS 4803 DVA. Feb 12, Dimension Reduction. Guest Lecturer: Jaegul Choo
CSE 6242 A / CS 4803 DVA Feb 12, 2013 Dimension Reduction Guest Lecturer: Jaegul Choo CSE 6242 A / CS 4803 DVA Feb 12, 2013 Dimension Reduction Guest Lecturer: Jaegul Choo Data is Too Big To Do Something..
Cross-Instance Tuning of Unsupervised Document Clustering Algorithms
Cross-Instance Tuning of Unsupervised Document Clustering Algorithms Damianos Karakos, Jason Eisner, and Sanjeev Khudanpur Center for Language and Speech Processing Johns Hopkins University Carey E. Priebe
Content-based image and video analysis. Machine learning
Content-based image and video analysis Machine learning for multimedia retrieval 04.05.2009 What is machine learning? Some problems are very hard to solve by writing a computer program by hand Almost all
Object Classification Problem
HIERARCHICAL OBJECT CATEGORIZATION" Gregory Griffin and Pietro Perona. Learning and Using Taxonomies For Fast Visual Categorization. CVPR 2008 Marcin Marszalek and Cordelia Schmid. Constructing Category