The Pre-Image Problem in Kernel Methods
|
|
- Shauna Meryl Fox
- 6 years ago
- Views:
Transcription
1 The Pre-Image Problem in Kernel Methods James Kwok Ivor Tsang Department of Computer Science Hong Kong University of Science and Technology Hong Kong The Pre-Image Problem in Kernel Methods ICML
2 Outline Introduction Proposed Method Experimental Results Conclusion The Pre-Image Problem in Kernel Methods ICML
3 Kernel Principal Component Analysis Kernel PCA: linear PCA in the feature space Given {x 1,..., x N }, construct K = [K ij ] K = UΛU (assume that {ϕ(x 1 ),..., ϕ(x N )} has been centered) U = [α 1,..., α N ] with α i = [α i1,..., α in ] Λ = diag(λ 1,..., λ N ) kth orthonormal eigenvector: V k = N i=1 (α ki/ λ k )ϕ(x i ) Projection of ϕ(x) onto V k : β k = ϕ(x) V k Projection P ϕ(x) of ϕ(x): P ϕ(x) = K k=1 β kv k The Pre-Image Problem in Kernel Methods ICML
4 The Pre-Image Problem Given noisy x, can we recover an ˆx such that ϕ(ˆx) = P ϕ(x)? Non-trivial as ϕ is not invertible in general Moreover, the exact pre-image may not exist Only an approximate solution can be obtained The Pre-Image Problem in Kernel Methods ICML
5 Solution of Mika et al. (NIPS11) Minimize the feature-space distance ϕ(ˆx) P ϕ(x) nonlinear optimization iteration scheme for Gaussian kernels k(x, y) = exp( x y 2 /c): ˆx t+1 = N i=1 γ i exp( ˆx t x i 2 /c)x i N i=1 γ i exp( ˆx t x i 2 /c), γ i = K β k α ki k=1 Problems: numerical instability local minimum The Pre-Image Problem in Kernel Methods ICML
6 Basic Idea Neighbors are most important in determining the location find the distances between P ϕ(x) and its neighboring ϕ(x i ) s Obtain the corresponding input-space distances Use these distances to constrain the location of the pre-image The Pre-Image Problem in Kernel Methods ICML
7 Input- and Feature-Space Distances Feature-space distance between P ϕ(x) and its neighbor ϕ(x i ) d 2 (P ϕ(x), ϕ(x i )) = P ϕ(x) ϕ(x i ) 2 = P ϕ(x) 2 + ϕ(x i ) 2 2P ϕ(x) ϕ(x i ) recall that P ϕ(x) = K k=1 β kv k d 2 (P ϕ(x), ϕ(x i )) can be computed based on K Obtain the corresponding input-space distance d ij The Pre-Image Problem in Kernel Methods ICML
8 From d ij to d ij Isotropic kernels: k(x i, x j ) = κ( x i x j 2 ) d 2 ij = k(x i, x i ) + k(x j, x j ) 2k(x i, x j ) = K ii + K jj 2κ( x i x j 2 ) = K ii + K jj 2κ(d 2 ij) κ(d 2 ij) = 1 2 (K ii + K jj d 2 ij) typically, κ is invertible example: Gaussian kernel κ(z) = exp( βz) d 2 ij = 1 β log(1 2 (K ii + K jj d 2 ij)) The Pre-Image Problem in Kernel Methods ICML
9 Dot product kernels: k(x i, x j ) = κ(x i x j) K ij = k(x i, x j ) = κ(x ix j ) = κ(s ij ) κ is often invertible example: polynomial kernel κ(z) = z p s ij = K 1 p ij when p is odd corresponding input-space distance: d 2 ij = s2 ii + s2 jj 2s ij The Pre-Image Problem in Kernel Methods ICML
10 Back to Input Space Assume that the pre-image ˆx is in the span of the x i s Mika et al. also obtains the same assumption Maintain the distances between ˆx and its neighbors (in the leastsquare sense) similar to the ideas in classical scaling and metric multidimensional scaling (MDS) reduces to finding the least-square solution for a system of linear equations The Pre-Image Problem in Kernel Methods ICML
11 Details After kernel PCA, we obtain P ϕ(x) Identify the n neighbors (ϕ(x 1 ),..., ϕ(x n )) of P ϕ(x) X d n = [x 1, x 2,..., x n ] (input space) assume that they span a q-dimensional space (rank(x) = q) Obtain the desired input-space distances between the unknown pre-image and these neighbors d 2 = [d 2 (x, x 1 ), d 2 (x, x 2 ),..., d 2 (x, x n )] Obtain a basis for these centered neighbors center these neighbors: HX H is the centering matrix I 1 n 11 The Pre-Image Problem in Kernel Methods ICML
12 use SVD on (HX ) : XH = UΛV = UZ U d q = [e 1,..., e q ] (e i s orthonormal) Z q n = [z 1,..., z n ] (z i is the projections of x i onto e j s) distances of x i s to the centroid: d 2 0 = [ z 1 2,..., z n 2 ] embed the desired pre-image ˆx in the span of these neighbors d 2 (ˆx, x i ) d 2 i, i = 1,..., n. if exact pre-image exists, distances exactly preserved least-square solution ẑ = 1 2 Λ 1 V (d 2 d 2 0) expressed in terms of the coordinate system defined by the e j s transform back to the original coordinate system in the input space ˆx = Uẑ + x The Pre-Image Problem in Kernel Methods ICML
13 Experiment: Pre-Images in Kernel PCA images from USPS dataset, 100 test images ten neighbors are used in locating the pre-image Gaussian noise (σ 2 = 0.25, 0.3, 0.4, 0.5) similar results on salt-and-pepper noise The Pre-Image Problem in Kernel Methods ICML
14 Results (Gaussian kernel) noisy image; (300 training images) Mika et al. ; proposed method; (60 training images) Mika et al. ; proposed method number of σ 2 SNR training images noisy images our method Mika et al The Pre-Image Problem in Kernel Methods ICML
15 Experiment on Polynomial Kernel (Order=3) Mika et al. we derive a similar iteration scheme as for Gaussian kernels however, the iteration does not converge Proposed method Noisy image; 300 training images; 60 training images The Pre-Image Problem in Kernel Methods ICML
16 number of training images σ 2 SNR from our method The Pre-Image Problem in Kernel Methods ICML
17 Pre-Images in Kernel k-means Clustering Kernelized version of k-means clustering 3000 images randomly selected from the USPS data set k = 10 (number of digits) Gaussian kernel Use the proposed method to find pre-images of cluster centroids For comparison, also obtain cluster centroids by averaging in the input space over all patterns belonging to the same cluster The Pre-Image Problem in Kernel Methods ICML
18 Results Input space averaging Proposed method yields centroids that are visually more appealing The Pre-Image Problem in Kernel Methods ICML
19 Conclusion Finding the pre-image of a feature vector in the kernel-induced feature space Advantages: non-iterative involves only linear algebra does not suffer from numerical instabilities or the local minimum problem Can be applied equally well to both isotropic kernels and dot product kernels Experimental results show significant improvements over the traditional method The Pre-Image Problem in Kernel Methods ICML
20 Future Directions Can be applied to other kernel applications that also require computation of the pre-images Extension to other classes of kernels only addressed isotropic kernels and dot product kernels in this paper kernel dependency estimation: pre-images for structured objects The Pre-Image Problem in Kernel Methods ICML
SOLVING THE PRE-IMAGE PROBLEM IN KERNEL MACHINES: A DIRECT METHOD. Paul Honeine, Cédric Richard
SOLVING THE PRE-IMAGE PROBLEM IN KERNEL MACHINES: A DIRECT METHOD Paul Honeine, Cédric Richard Institut Charles Delaunay (FRE CNRS 2848), LM2S, Université de technologie de Troyes, Troyes, France ABSTRACT
More informationThe Pre-Image Problem and Kernel PCA for Speech Enhancement
The Pre-Image Problem and Kernel PCA for Speech Enhancement Christina Leitner and Franz Pernkopf Signal Processing and Speech Communication Laboratory, Graz University of Technology, Inffeldgasse 6c, 8
More informationLarge-Scale Face Manifold Learning
Large-Scale Face Manifold Learning Sanjiv Kumar Google Research New York, NY * Joint work with A. Talwalkar, H. Rowley and M. Mohri 1 Face Manifold Learning 50 x 50 pixel faces R 2500 50 x 50 pixel random
More informationUnsupervised Learning
Unsupervised Learning Learning without Class Labels (or correct outputs) Density Estimation Learn P(X) given training data for X Clustering Partition data into clusters Dimensionality Reduction Discover
More informationIteratively Re-weighted Least Squares for Sums of Convex Functions
Iteratively Re-weighted Least Squares for Sums of Convex Functions James Burke University of Washington Jiashan Wang LinkedIn Frank Curtis Lehigh University Hao Wang Shanghai Tech University Daiwei He
More informationKernel Principal Component Analysis: Applications and Implementation
Kernel Principal Component Analysis: Applications and Daniel Olsson Royal Institute of Technology Stockholm, Sweden Examiner: Prof. Ulf Jönsson Supervisor: Prof. Pando Georgiev Master s Thesis Presentation
More informationUnsupervised Learning
Networks for Pattern Recognition, 2014 Networks for Single Linkage K-Means Soft DBSCAN PCA Networks for Kohonen Maps Linear Vector Quantization Networks for Problems/Approaches in Machine Learning Supervised
More informationDivide and Conquer Kernel Ridge Regression
Divide and Conquer Kernel Ridge Regression Yuchen Zhang John Duchi Martin Wainwright University of California, Berkeley COLT 2013 Yuchen Zhang (UC Berkeley) Divide and Conquer KRR COLT 2013 1 / 15 Problem
More informationKernel Methods in Machine Learning
Outline Department of Computer Science and Engineering Hong Kong University of Science and Technology Hong Kong Joint work with Ivor Tsang, Pakming Cheung, Andras Kocsor, Jacek Zurada, Kimo Lai November
More informationRobust Kernel Methods in Clustering and Dimensionality Reduction Problems
Robust Kernel Methods in Clustering and Dimensionality Reduction Problems Jian Guo, Debadyuti Roy, Jing Wang University of Michigan, Department of Statistics Introduction In this report we propose robust
More information3D Surface Recovery via Deterministic Annealing based Piecewise Linear Surface Fitting Algorithm
3D Surface Recovery via Deterministic Annealing based Piecewise Linear Surface Fitting Algorithm Bing Han, Chris Paulson, and Dapeng Wu Department of Electrical and Computer Engineering University of Florida
More informationGeneral Instructions. Questions
CS246: Mining Massive Data Sets Winter 2018 Problem Set 2 Due 11:59pm February 8, 2018 Only one late period is allowed for this homework (11:59pm 2/13). General Instructions Submission instructions: These
More informationLecture 19: Convex Non-Smooth Optimization. April 2, 2007
: Convex Non-Smooth Optimization April 2, 2007 Outline Lecture 19 Convex non-smooth problems Examples Subgradients and subdifferentials Subgradient properties Operations with subgradients and subdifferentials
More informationData Analysis 3. Support Vector Machines. Jan Platoš October 30, 2017
Data Analysis 3 Support Vector Machines Jan Platoš October 30, 2017 Department of Computer Science Faculty of Electrical Engineering and Computer Science VŠB - Technical University of Ostrava Table of
More informationGraph Laplacian Kernels for Object Classification from a Single Example
Graph Laplacian Kernels for Object Classification from a Single Example Hong Chang & Dit-Yan Yeung Department of Computer Science, Hong Kong University of Science and Technology {hongch,dyyeung}@cs.ust.hk
More informationChapter 7: Computation of the Camera Matrix P
Chapter 7: Computation of the Camera Matrix P Arco Nederveen Eagle Vision March 18, 2008 Arco Nederveen (Eagle Vision) The Camera Matrix P March 18, 2008 1 / 25 1 Chapter 7: Computation of the camera Matrix
More informationUnsupervised learning in Vision
Chapter 7 Unsupervised learning in Vision The fields of Computer Vision and Machine Learning complement each other in a very natural way: the aim of the former is to extract useful information from visual
More informationELEG Compressive Sensing and Sparse Signal Representations
ELEG 867 - Compressive Sensing and Sparse Signal Representations Gonzalo R. Arce Depart. of Electrical and Computer Engineering University of Delaware Fall 211 Compressive Sensing G. Arce Fall, 211 1 /
More informationCOMP 558 lecture 19 Nov. 17, 2010
COMP 558 lecture 9 Nov. 7, 2 Camera calibration To estimate the geometry of 3D scenes, it helps to know the camera parameters, both external and internal. The problem of finding all these parameters is
More informationNon-linear dimension reduction
Sta306b May 23, 2011 Dimension Reduction: 1 Non-linear dimension reduction ISOMAP: Tenenbaum, de Silva & Langford (2000) Local linear embedding: Roweis & Saul (2000) Local MDS: Chen (2006) all three methods
More informationDiffusion Wavelets for Natural Image Analysis
Diffusion Wavelets for Natural Image Analysis Tyrus Berry December 16, 2011 Contents 1 Project Description 2 2 Introduction to Diffusion Wavelets 2 2.1 Diffusion Multiresolution............................
More informationSummarizing Inter-Query Learning in Content-Based Image Retrieval via Incremental Semantic Clustering
Summarizing Inter-Query Learning in Content-Based Image Retrieval via Incremental Semantic Clustering Iker Gondra, Douglas R. Heisterkamp Department of Computer Science Oklahoma State University Stillwater,
More informationKernel spectral clustering: model representations, sparsity and out-of-sample extensions
Kernel spectral clustering: model representations, sparsity and out-of-sample extensions Johan Suykens and Carlos Alzate K.U. Leuven, ESAT-SCD/SISTA Kasteelpark Arenberg B-3 Leuven (Heverlee), Belgium
More informationMultidimensional scaling Based in part on slides from textbook, slides of Susan Holmes. October 10, Statistics 202: Data Mining
Multidimensional scaling Based in part on slides from textbook, slides of Susan Holmes October 10, 2012 1 / 1 Multidimensional scaling A visual tool Recall the PCA scores were X V = U where X = HX S 1/2
More informationLaplacian Eigenmaps and Bayesian Clustering Based Layout Pattern Sampling and Its Applications to Hotspot Detection and OPC
Laplacian Eigenmaps and Bayesian Clustering Based Layout Pattern Sampling and Its Applications to Hotspot Detection and OPC Tetsuaki Matsunawa 1, Bei Yu 2 and David Z. Pan 3 1 Toshiba Corporation 2 The
More informationFilters. Advanced and Special Topics: Filters. Filters
Filters Advanced and Special Topics: Filters Dr. Edmund Lam Department of Electrical and Electronic Engineering The University of Hong Kong ELEC4245: Digital Image Processing (Second Semester, 2016 17)
More informationMachine Learning for Signal Processing Clustering. Bhiksha Raj Class Oct 2016
Machine Learning for Signal Processing Clustering Bhiksha Raj Class 11. 13 Oct 2016 1 Statistical Modelling and Latent Structure Much of statistical modelling attempts to identify latent structure in the
More informationData-Dependent Kernels for High-Dimensional Data Classification
Proceedings of International Joint Conference on Neural Networks, Montreal, Canada, July 31 - August 4, 2005 Data-Dependent Kernels for High-Dimensional Data Classification Jingdong Wang James T. Kwok
More informationAlternative Statistical Methods for Bone Atlas Modelling
Alternative Statistical Methods for Bone Atlas Modelling Sharmishtaa Seshamani, Gouthami Chintalapani, Russell Taylor Department of Computer Science, Johns Hopkins University, Baltimore, MD Traditional
More informationClustering K-means. Machine Learning CSEP546 Carlos Guestrin University of Washington February 18, Carlos Guestrin
Clustering K-means Machine Learning CSEP546 Carlos Guestrin University of Washington February 18, 2014 Carlos Guestrin 2005-2014 1 Clustering images Set of Images [Goldberger et al.] Carlos Guestrin 2005-2014
More informationSupport Vector Machines
Support Vector Machines RBF-networks Support Vector Machines Good Decision Boundary Optimization Problem Soft margin Hyperplane Non-linear Decision Boundary Kernel-Trick Approximation Accurancy Overtraining
More informationFeature scaling in support vector data description
Feature scaling in support vector data description P. Juszczak, D.M.J. Tax, R.P.W. Duin Pattern Recognition Group, Department of Applied Physics, Faculty of Applied Sciences, Delft University of Technology,
More informationKernels and representation
Kernels and representation Corso di AA, anno 2017/18, Padova Fabio Aiolli 20 Dicembre 2017 Fabio Aiolli Kernels and representation 20 Dicembre 2017 1 / 19 (Hierarchical) Representation Learning Hierarchical
More informationClustering: Classic Methods and Modern Views
Clustering: Classic Methods and Modern Views Marina Meilă University of Washington mmp@stat.washington.edu June 22, 2015 Lorentz Center Workshop on Clusters, Games and Axioms Outline Paradigms for clustering
More informationAdvanced Machine Learning Practical 1: Manifold Learning (PCA and Kernel PCA)
Advanced Machine Learning Practical : Manifold Learning (PCA and Kernel PCA) Professor: Aude Billard Assistants: Nadia Figueroa, Ilaria Lauzana and Brice Platerrier E-mails: aude.billard@epfl.ch, nadia.figueroafernandez@epfl.ch
More informationSelf-Organized Similarity based Kernel Fuzzy Clustering Model and Its Applications
Fifth International Workshop on Computational Intelligence & Applications IEEE SMC Hiroshima Chapter, Hiroshima University, Japan, November 10, 11 & 12, 2009 Self-Organized Similarity based Kernel Fuzzy
More informationCIS 520, Machine Learning, Fall 2015: Assignment 7 Due: Mon, Nov 16, :59pm, PDF to Canvas [100 points]
CIS 520, Machine Learning, Fall 2015: Assignment 7 Due: Mon, Nov 16, 2015. 11:59pm, PDF to Canvas [100 points] Instructions. Please write up your responses to the following problems clearly and concisely.
More informationMultiresponse Sparse Regression with Application to Multidimensional Scaling
Multiresponse Sparse Regression with Application to Multidimensional Scaling Timo Similä and Jarkko Tikka Helsinki University of Technology, Laboratory of Computer and Information Science P.O. Box 54,
More informationSUPPORT VECTOR MACHINES
SUPPORT VECTOR MACHINES Today Reading AIMA 8.9 (SVMs) Goals Finish Backpropagation Support vector machines Backpropagation. Begin with randomly initialized weights 2. Apply the neural network to each training
More informationKernels for Structured Data
T-122.102 Special Course in Information Science VI: Co-occurence methods in analysis of discrete data Kernels for Structured Data Based on article: A Survey of Kernels for Structured Data by Thomas Gärtner
More informationMay 1, CODY, Error Backpropagation, Bischop 5.3, and Support Vector Machines (SVM) Bishop Ch 7. May 3, Class HW SVM, PCA, and K-means, Bishop Ch
May 1, CODY, Error Backpropagation, Bischop 5.3, and Support Vector Machines (SVM) Bishop Ch 7. May 3, Class HW SVM, PCA, and K-means, Bishop Ch 12.1, 9.1 May 8, CODY Machine Learning for finding oil,
More informationCS 195-5: Machine Learning Problem Set 5
CS 195-5: Machine Learning Problem Set 5 Douglas Lanman dlanman@brown.edu 26 November 26 1 Clustering and Vector Quantization Problem 1 Part 1: In this problem we will apply Vector Quantization (VQ) to
More informationCSE 547: Machine Learning for Big Data Spring Problem Set 2. Please read the homework submission policies.
CSE 547: Machine Learning for Big Data Spring 2019 Problem Set 2 Please read the homework submission policies. 1 Principal Component Analysis and Reconstruction (25 points) Let s do PCA and reconstruct
More informationA Weighted Kernel PCA Approach to Graph-Based Image Segmentation
A Weighted Kernel PCA Approach to Graph-Based Image Segmentation Carlos Alzate Johan A. K. Suykens ESAT-SCD-SISTA Katholieke Universiteit Leuven Leuven, Belgium January 25, 2007 International Conference
More informationCOSC160: Detection and Classification. Jeremy Bolton, PhD Assistant Teaching Professor
COSC160: Detection and Classification Jeremy Bolton, PhD Assistant Teaching Professor Outline I. Problem I. Strategies II. Features for training III. Using spatial information? IV. Reducing dimensionality
More informationChapter 7: Competitive learning, clustering, and self-organizing maps
Chapter 7: Competitive learning, clustering, and self-organizing maps António R. C. Paiva EEL 6814 Spring 2008 Outline Competitive learning Clustering Self-Organizing Maps What is competition in neural
More informationData Preprocessing. Javier Béjar. URL - Spring 2018 CS - MAI 1/78 BY: $\
Data Preprocessing Javier Béjar BY: $\ URL - Spring 2018 C CS - MAI 1/78 Introduction Data representation Unstructured datasets: Examples described by a flat set of attributes: attribute-value matrix Structured
More informationChap.12 Kernel methods [Book, Chap.7]
Chap.12 Kernel methods [Book, Chap.7] Neural network methods became popular in the mid to late 1980s, but by the mid to late 1990s, kernel methods have also become popular in machine learning. The first
More informationarxiv: v3 [cs.cv] 31 Aug 2014
Kernel Principal Component Analysis and its Applications in Face Recognition and Active Shape Models Quan Wang Rensselaer Polytechnic Institute, Eighth Street, Troy, Y 28 USA wangq@rpi.edu arxiv:27.3538v3
More informationNon-Bayesian Classifiers Part II: Linear Discriminants and Support Vector Machines
Non-Bayesian Classifiers Part II: Linear Discriminants and Support Vector Machines Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr CS 551, Spring 2007 c 2007,
More informationSensor Network Localization Using Sensor Perturbation
Sensor Network Localization Using Sensor Perturbation YUANCHEN ZHU and STEVEN J. GORTLER Harvard University and DYLAN THURSTON Columbia University Sensor network localization is an instance of the NP-HARD
More informationGeneralized Principal Component Analysis CVPR 2007
Generalized Principal Component Analysis Tutorial @ CVPR 2007 Yi Ma ECE Department University of Illinois Urbana Champaign René Vidal Center for Imaging Science Institute for Computational Medicine Johns
More informationNetwork Traffic Measurements and Analysis
DEIB - Politecnico di Milano Fall, 2017 Introduction Often, we have only a set of features x = x 1, x 2,, x n, but no associated response y. Therefore we are not interested in prediction nor classification,
More informationHomework 4: Clustering, Recommenders, Dim. Reduction, ML and Graph Mining (due November 19 th, 2014, 2:30pm, in class hard-copy please)
Virginia Tech. Computer Science CS 5614 (Big) Data Management Systems Fall 2014, Prakash Homework 4: Clustering, Recommenders, Dim. Reduction, ML and Graph Mining (due November 19 th, 2014, 2:30pm, in
More informationKernel Methods. Chapter 9 of A Course in Machine Learning by Hal Daumé III. Conversion to beamer by Fabrizio Riguzzi
Kernel Methods Chapter 9 of A Course in Machine Learning by Hal Daumé III http://ciml.info Conversion to beamer by Fabrizio Riguzzi Kernel Methods 1 / 66 Kernel Methods Linear models are great because
More informationConnecting the Out-of-Sample and Pre-Image Problems in Kernel Methods
Connecting the Out-of-Sample and Pre-Image Problems in Kernel Methods Pablo Arias Universidad de la República parias@fing.edu.uy Gregory Randall Universidad de la República randall@fing.edu.uy Guillermo
More informationLocal Isomorphism to Solve the Pre-image Problem in Kernel Methods
Local Isomorphism to Solve the Pre-image Problem in Kernel Methods Dong Huang 1,2, Yuandong Tian 1 and Fernando De la Torre 1 1 Robotics Institute, Carnegie Mellon University, USA 2 University of Electronic
More informationNon-exhaustive, Overlapping k-means
Non-exhaustive, Overlapping k-means J. J. Whang, I. S. Dhilon, and D. F. Gleich Teresa Lebair University of Maryland, Baltimore County October 29th, 2015 Teresa Lebair UMBC 1/38 Outline Introduction NEO-K-Means
More informationBasis Functions. Volker Tresp Summer 2017
Basis Functions Volker Tresp Summer 2017 1 Nonlinear Mappings and Nonlinear Classifiers Regression: Linearity is often a good assumption when many inputs influence the output Some natural laws are (approximately)
More informationModule 4. Non-linear machine learning econometrics: Support Vector Machine
Module 4. Non-linear machine learning econometrics: Support Vector Machine THE CONTRACTOR IS ACTING UNDER A FRAMEWORK CONTRACT CONCLUDED WITH THE COMMISSION Introduction When the assumption of linearity
More informationMachine Learning / Jan 27, 2010
Revisiting Logistic Regression & Naïve Bayes Aarti Singh Machine Learning 10-701/15-781 Jan 27, 2010 Generative and Discriminative Classifiers Training classifiers involves learning a mapping f: X -> Y,
More informationA survey of kernel and spectral methods for clustering
A survey of kernel and spectral methods for clustering Maurizio Filippone a Francesco Camastra b Francesco Masulli a Stefano Rovetta a a Department of Computer and Information Science, University of Genova,
More informationCS Introduction to Data Mining Instructor: Abdullah Mueen
CS 591.03 Introduction to Data Mining Instructor: Abdullah Mueen LECTURE 8: ADVANCED CLUSTERING (FUZZY AND CO -CLUSTERING) Review: Basic Cluster Analysis Methods (Chap. 10) Cluster Analysis: Basic Concepts
More informationA Dendrogram. Bioinformatics (Lec 17)
A Dendrogram 3/15/05 1 Hierarchical Clustering [Johnson, SC, 1967] Given n points in R d, compute the distance between every pair of points While (not done) Pick closest pair of points s i and s j and
More informationClustering. Department Biosysteme Karsten Borgwardt Data Mining Course Basel Fall Semester / 238
Clustering Department Biosysteme Karsten Borgwardt Data Mining Course Basel Fall Semester 2015 163 / 238 What is Clustering? Department Biosysteme Karsten Borgwardt Data Mining Course Basel Fall Semester
More informationSecond Order SMO Improves SVM Online and Active Learning
Second Order SMO Improves SVM Online and Active Learning Tobias Glasmachers and Christian Igel Institut für Neuroinformatik, Ruhr-Universität Bochum 4478 Bochum, Germany Abstract Iterative learning algorithms
More informationFeature selection. Term 2011/2012 LSI - FIB. Javier Béjar cbea (LSI - FIB) Feature selection Term 2011/ / 22
Feature selection Javier Béjar cbea LSI - FIB Term 2011/2012 Javier Béjar cbea (LSI - FIB) Feature selection Term 2011/2012 1 / 22 Outline 1 Dimensionality reduction 2 Projections 3 Attribute selection
More informationMarkov Networks in Computer Vision
Markov Networks in Computer Vision Sargur Srihari srihari@cedar.buffalo.edu 1 Markov Networks for Computer Vision Some applications: 1. Image segmentation 2. Removal of blur/noise 3. Stereo reconstruction
More informationFace Recognition Using Wavelet Based Kernel Locally Discriminating Projection
Face Recognition Using Wavelet Based Kernel Locally Discriminating Projection Venkatrama Phani Kumar S 1, KVK Kishore 2 and K Hemantha Kumar 3 Abstract Locality Preserving Projection(LPP) aims to preserve
More informationSparse multidimensional scaling for blind tracking in sensor networks
Sparse multidimensional scaling for blind tracking in sensor networks R. Rangarajan 1, R. Raich 2, and A. O. Hero III 3 1 University of Michigan rangaraj@eecs.umich.edu 2 University of Michigan ravivr@eecs.umich.edu
More informationMachine Learning for OR & FE
Machine Learning for OR & FE Unsupervised Learning: Clustering Martin Haugh Department of Industrial Engineering and Operations Research Columbia University Email: martin.b.haugh@gmail.com (Some material
More informationNearest Neighbor with KD Trees
Case Study 2: Document Retrieval Finding Similar Documents Using Nearest Neighbors Machine Learning/Statistics for Big Data CSE599C1/STAT592, University of Washington Emily Fox January 22 nd, 2013 1 Nearest
More informationCluster Analysis. Mu-Chun Su. Department of Computer Science and Information Engineering National Central University 2003/3/11 1
Cluster Analysis Mu-Chun Su Department of Computer Science and Information Engineering National Central University 2003/3/11 1 Introduction Cluster analysis is the formal study of algorithms and methods
More informationVisualizing pairwise similarity via semidefinite programming
Visualizing pairwise similarity via semidefinite programming Amir Globerson Computer Science and Artificial Intelligence Laboratory MIT Cambridge, MA 02139 gamir@csail.mit.edu Sam Roweis Department of
More informationA Weighted Majority Voting based on Normalized Mutual Information for Cluster Analysis
A Weighted Majority Voting based on Normalized Mutual Information for Cluster Analysis Meshal Shutaywi and Nezamoddin N. Kachouie Department of Mathematical Sciences, Florida Institute of Technology Abstract
More informationRobust Kernel Principal Component Analysis
Robust Kernel Principal Component Analysis Minh Hoai Nguyen & Fernando De la Torre Carnegie Mellon University, Pittsburgh, PA 15213, USA. Abstract Kernel Principal Component Analysis (KPCA) is a popular
More informationMarkov Networks in Computer Vision. Sargur Srihari
Markov Networks in Computer Vision Sargur srihari@cedar.buffalo.edu 1 Markov Networks for Computer Vision Important application area for MNs 1. Image segmentation 2. Removal of blur/noise 3. Stereo reconstruction
More informationConcept of Curve Fitting Difference with Interpolation
Curve Fitting Content Concept of Curve Fitting Difference with Interpolation Estimation of Linear Parameters by Least Squares Curve Fitting by Polynomial Least Squares Estimation of Non-linear Parameters
More informationDigital Image Processing Laboratory: MAP Image Restoration
Purdue University: Digital Image Processing Laboratories 1 Digital Image Processing Laboratory: MAP Image Restoration October, 015 1 Introduction This laboratory explores the use of maximum a posteriori
More informationAdaptive Hash Retrieval with Kernel Based Similarity
Adaptive Hash Retrieval with Kernel Based Similarity Xiao Bai a, Cheng Yan a, Haichuan Yang a, Lu Bai b, Jun Zhou c, Edwin Robert Hancock d a School of Computer Science and Engineering, Beihang University,
More informationUsing Machine Learning to Optimize Storage Systems
Using Machine Learning to Optimize Storage Systems Dr. Kiran Gunnam 1 Outline 1. Overview 2. Building Flash Models using Logistic Regression. 3. Storage Object classification 4. Storage Allocation recommendation
More informationNonlinear Noise Reduction
Chapter 3 Nonlinear Noise Reduction Many natural and engineered systems generate nonlinear deterministic time series that are contaminated by random measurement or dynamic noise. Even clean time series
More informationTracking system. Danica Kragic. Object Recognition & Model Based Tracking
Tracking system Object Recognition & Model Based Tracking Motivation Manipulating objects in domestic environments Localization / Navigation Object Recognition Servoing Tracking Grasping Pose estimation
More informationRecent Developments in Model-based Derivative-free Optimization
Recent Developments in Model-based Derivative-free Optimization Seppo Pulkkinen April 23, 2010 Introduction Problem definition The problem we are considering is a nonlinear optimization problem with constraints:
More informationCPSC 340: Machine Learning and Data Mining. Principal Component Analysis Fall 2016
CPSC 340: Machine Learning and Data Mining Principal Component Analysis Fall 2016 A2/Midterm: Admin Grades/solutions will be posted after class. Assignment 4: Posted, due November 14. Extra office hours:
More informationLocally Weighted Least Squares Regression for Image Denoising, Reconstruction and Up-sampling
Locally Weighted Least Squares Regression for Image Denoising, Reconstruction and Up-sampling Moritz Baecher May 15, 29 1 Introduction Edge-preserving smoothing and super-resolution are classic and important
More information(University Improving of Montreal) Generative Adversarial Networks with Denoising Feature Matching / 17
Improving Generative Adversarial Networks with Denoising Feature Matching David Warde-Farley 1 Yoshua Bengio 1 1 University of Montreal, ICLR,2017 Presenter: Bargav Jayaraman Outline 1 Introduction 2 Background
More informationKernel-Based Metric Adaptation with Pairwise Constraints
Kernel-Based Metric Adaptation with Pairwise Constraints Hong Chang and Dit-Yan Yeung Department of Computer Science Hong Kong University of Science and Technology Clear Water Bay, Kowloon, Hong Kong {hongch,dyyeung}@cs.ust.hk
More informationAdaptive Sparse Kernel Principal Component Analysis for Computation and Store Space Constrained-based Feature Extraction
Journal of Information Hiding and Multimedia Signal Processing c 2015 ISSN 2073-4212 Ubiquitous International Volume 6, Number 4, July 2015 Adaptive Sparse Kernel Principal Component Analysis for Computation
More informationSTATS306B STATS306B. Clustering. Jonathan Taylor Department of Statistics Stanford University. June 3, 2010
STATS306B Jonathan Taylor Department of Statistics Stanford University June 3, 2010 Spring 2010 Outline K-means, K-medoids, EM algorithm choosing number of clusters: Gap test hierarchical clustering spectral
More informationRobust Subspace Computation Using L1 Norm
Robust Subspace Computation Using L1 Norm Qifa Ke and Takeo Kanade August 2003 CMU-CS-03-172 School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213 Abstract Linear subspace has many
More informationGeometric Registration for Deformable Shapes 3.3 Advanced Global Matching
Geometric Registration for Deformable Shapes 3.3 Advanced Global Matching Correlated Correspondences [ASP*04] A Complete Registration System [HAW*08] In this session Advanced Global Matching Some practical
More informationarxiv: v1 [cs.lg] 3 Jan 2018
CLUSTERING OF DATA WITH MISSING ENTRIES Sunrita Poddar, Mathews Jacob Department of Electrical and Computer Engineering, University of Iowa, IA, USA arxiv:1801.01455v1 [cs.lg] 3 Jan 018 ABSTRACT The analysis
More informationData Preprocessing. Javier Béjar AMLT /2017 CS - MAI. (CS - MAI) Data Preprocessing AMLT / / 71 BY: $\
Data Preprocessing S - MAI AMLT - 2016/2017 (S - MAI) Data Preprocessing AMLT - 2016/2017 1 / 71 Outline 1 Introduction Data Representation 2 Data Preprocessing Outliers Missing Values Normalization Discretization
More informationModel Criticism for Regression on the Grassmannian
Model Criticism for Regression on the Grassmannian Yi Hong, Roland Kwitt 3, and Marc Niethammer,2 University of North Carolina (UNC) at Chapel Hill, NC, US 2 Biomedical Research Imaging Center, UNC-Chapel
More informationUnsupervised Outlier Detection and Semi-Supervised Learning
Unsupervised Outlier Detection and Semi-Supervised Learning Adam Vinueza Department of Computer Science University of Colorado Boulder, Colorado 832 vinueza@colorado.edu Gregory Z. Grudic Department of
More informationIntroduction to Machine Learning
Introduction to Machine Learning Isabelle Guyon Notes written by: Johann Leithon. Introduction The process of Machine Learning consist of having a big training data base, which is the input to some learning
More informationSubspace Clustering. Weiwei Feng. December 11, 2015
Subspace Clustering Weiwei Feng December 11, 2015 Abstract Data structure analysis is an important basis of machine learning and data science, which is now widely used in computational visualization problems,
More informationInterferogram Analysis using Active Instance-Based Learning
Interferogram Analysis using Active Instance-Based Learning Olac Fuentes and Thamar Solorio Instituto Nacional de Astrofísica, Óptica y Electrónica Luis Enrique Erro 1 Santa María Tonantzintla, Puebla,
More informationCPSC 340: Machine Learning and Data Mining. Kernel Trick Fall 2017
CPSC 340: Machine Learning and Data Mining Kernel Trick Fall 2017 Admin Assignment 3: Due Friday. Midterm: Can view your exam during instructor office hours or after class this week. Digression: the other
More information