CSE 6242 A / CX 4242 DVA. March 6, Dimension Reduction. Guest Lecturer: Jaegul Choo
|
|
- Annabel Amice Poole
- 5 years ago
- Views:
Transcription
1 CSE 6242 A / CX 4242 DVA March 6, 2014 Dimension Reduction Guest Lecturer: Jaegul Choo
2 Data is Too Big To Analyze! Limited memory size! Data may not be fitted to the memory of your machine! Slow computation! dim vs. 10-dim vectors for Euclidean distance computation 2
3 Two Axes of Data Set! No. of data items! How many data items?! No. of dimensions! How many dimensions representing each item? Data item index Dimension index Columns as data items vs. Rows as data items We will use this during lecture 3
4 Dimension Reduction Let s Reduce Data (along Dimension Axis) High-dim data No. of dimensions Dimension Reduction low-dim data Additional info about data Other parameters Dim-reducing Transformer for new data 4 : user-specified
5 What You Get from DR Obviously,! Less storage! Faster computation More importantly,! Noise removal (improving quality of data)! Leads better performance for tasks! 2D/3D representation! Enables visual data exploration 5
6 Applications Traditionally,! Microarray data analysis! Information retrieval! Face recognition! Protein disorder prediction! Network intrusion detection! Document categorization! Speech recognition More interestingly,! Interactive visualization of high-dimensional data 6
7 Visualizing Map of Science Visualization 7
8 Two Main Techniques 1. Feature selection! Selects a subset of the original variables as reduced dimensions! For example, the number of genes responsible for a particular disease may be small 2. Feature extraction! Each reduced dimension involves multiple original dimensions! Active research area 8 Note that Feature = Variable = Dimension
9 Feature Selection What are the optimal subset of m features to maximize a given criterion?! Widely-used criteria! Information gain, correlation,! Typically combinatorial optimization problems! Therefore, greedy methods are popular! Forward selection: Empty set add one variable at a time! Backward elimination: Entire set remove one variable at a time 9
10 From now on, we will only discuss about feature extraction
11 Aspects of DR! Linear vs. Nonlinear! Unsupervised vs. Supervised! Global vs. Local! Feature vectors vs. Similarity (as an input) 11
12 Aspects of DR Linear vs. Nonlinear Linear! Represents each reduced dimension as a linear combination of original dimensions! e.g., Y1 = 3*X1 4*X *X3 1.5*X4, Y2 = 2*X *X2 X3 + 2*X4! Naturally capable of mapping new data to the same space D1 D2 X1 1 1 X2 1 0 X3 0 2 Dimension Reduction D1 D2 Y Y X4 1 1
13 Aspects of DR Linear vs. Nonlinear Linear! Represents each reduced dimension as a linear combination of original dimensions! e.g., Y1 = 3*X1 4*X *X3 1.5*X4, Y2 = 2*X *X2 X3 + 2*X4! Naturally capable of mapping new data to the same space Nonlinear! More complicated, but generally more powerful! Recently popular topics 13
14 Aspects of DR Unsupervised vs. Supervised Unsupervised! Uses only the input data High-dim data No. of dimensions Dimension Reduction low-dim data 14 Additional info about data Other parameters Dim-reducing Transformer for a new data
15 Aspects of DR Unsupervised vs. Supervised Supervised! Uses the input data + additional info High-dim data No. of dimensions Dimension Reduction low-dim data 15 Additional info about data Other parameters Dim-reducing Transformer for a new data
16 Aspects of DR Unsupervised vs. Supervised Supervised! Uses the input data + additional info! e.g., grouping label High-dim data No. of dimensions Dimension Reduction low-dim data 16 Additional info about data Other parameters Dim-reducing Transformer for a new data
17 Aspects of DR Global vs. Local Dimension reduction typically tries to preserve all the relationships/distances in data! Information loss is unavoidable! Then, what would you care about? Global! Treats all pairwise distances equally important! Tends to care larger distances more Local! Focuses on small distances, neighborhood relationships! Active research area a.k.a. manifold learning 17
18 Aspects of DR Feature vectors vs. Similarity (as an input)! Typical setup (feature vectors as an input) High-dim data No. of dimensions Dimension Reduction low-dim data 18 Additional info about data Other parameters Dim-reducing Transformer for a new data
19 Aspects of DR Feature vectors vs. Similarity (as an input)! Typical setup (feature vectors as an input)! Some methods take similarity matrix instead! (i,j)-th component indicates similarity between i-th and j-th data Similarity matrix No. of dimensions Dimension Reduction low-dim data 19 Additional info about data Other parameters Dim-reducing Transformer for a new data
20 Aspects of DR Feature vectors vs. Similarity (as an input)! Typical setup (feature vectors as an input)! Some methods take similarity matrix instead! Some methods internally convert feature vectors to similarity matrix before performing No. of dimension reduction Similarity dimensions matrix Dimension Reduction High-dim data Dimension Reduction low-dim data low-dim data 20 Additional info about data Other parameters a.k.a. Graph Embedding Dim-reducing Transformer for a new data
21 Aspects of DR Feature vectors vs. Similarity (as an input) Why called graph embedding?! Similarity matrix can be viewed as a graph where similarity represents edge weight High-dim data Dimension Reduction Similarity matrix low-dim data 21 a.k.a. Graph Embedding
22 Methods! Traditional! Principal component analysis (PCA)! Multidimensional scaling (MDS)! Linear discriminant analysis (LDA)! Advanced (nonlinear, kernel, manifold learning)! Isometric feature mapping (Isomap)! t-distributed stochastic neighborhood embedding (t-sne) 22 * Matlab codes are available at
23 Principal Component Analysis! Finds the axis showing the greatest variation, and project all points into this axis! Reduced dimensions are orthogonal! Algorithm: Eigen-decomposition! Pros: Fast! Cons: Limited performances Linear Unsupervised Global Feature vectors PC2 PC1 23
24 24 Principal Component Analysis Document Visualization
25 Multidimensional Scaling (MDS) Intuition! Tries to preserve given ideal pairwise distances in lowdimensional space! Metric MDS actual distance! Preserves given ideal distance values! Nonmetric MDS ideal distance! When you only know/care about ordering of distances! Preserves only the orderings of distance values Nonlinear Unsupervised Global Similarity input 25! Algorithm: gradient-decent type c.f. classical MDS is the same as PCA
26 Multidimensional Scaling Sammon s mapping Sammon s mapping! Local version of MDS! Down-weights errors in large distances! Algorithm: gradient-decent type Nonlinear Unsupervised Local Similarity input 26
27 Multidimensional Scaling Force-directed graph layout Force-directed graph layout! Rooted from graph visualization, but essentially variant of metric MDS! Spring-like attractive + repulsive forces between nodes! Algorithm: gradient-decent type Nonlinear Unsupervised Global! Widely-used in visualization Similarity input! Aesthetically pleasing results! Simple and intuitive! Interactivity 27
28 Demos! Prefuse Multidimensional Scaling Force-directed graph layout! D3:
29 Multidimensional Scaling In all variants,! Pros: widely-used (works well in general)! Cons: slow! Nonmetric MDS is even much slower than metric MDS 29
30 Linear Discriminant Analysis What if clustering information is available? LDA tries to separate clusters by! Putting different cluster as far as possible! Putting each cluster as compact as possible 30 (a) (b)
31 Aspects of DR Unsupervised vs. Supervised Supervised! Uses the input data + additional info! e.g., grouping label High-dim data No. of dimensions Dimension Reduction low-dim data 31 Additional info about data Other parameters Dim-reducing Transformer for a new data
32 Linear Discriminant Analysis vs. Principal Component Analysis 2D visualization of 7 Gaussian mixture of 1000 dimensions Linear discriminant analysis (Supervised) Principal component analysis (Unsupervised)
33 Linear Discriminant Analysis Maximally separates clusters by! Putting different cluster as far as possible! Putting each cluster as compact as possible! Algorithm: generalized eigendecomposition! Pros: better show cluster structure! Cons: may distort original relationship of data Linear Supervised Global Feature vectors 33
34 Methods! Traditional! Principal component analysis (PCA)! Multidimensional scaling (MDS)! Linear discriminant analysis (LDA)! Advanced (nonlinear, kernel, manifold learning)! Isometric feature mapping (Isomap)! t-distributed stochastic neighborhood embedding (t-sne) 34 * Matlab codes are available at
35 Manifold Learning Swiss Roll Data Swiss roll data! Originally in 3D! What is the intrinsic dimensionality? (allowing flattening) intrinsic semantic 35
36 Manifold Learning Swiss Roll Data Swiss roll data! Originally in 3D! What is the intrinsic dimensionality? (allowing flattening) 2D intrinsic semantic 36 What if your data has low intrinsic dimensionality but resides in highdimensional space?
37 Manifold Learning Goal and Approach 37 Manifold! Curvi-linear low-dimensional structure of your data based on intrinsic dimensionality Manifold learning! Match intrinsic dimensions to axes of dimension-reduced output space How?! Each piece of manifold is appox. linear! Utilize local neighborhood information! e.g. for a particular point,! Who are my neighbors?! How closely am I related to neighbors? Demo available at
38 Isomap (Isometric Feature Mapping) Let s preserve pairwise geodesic distance (along manifold)! Compute geodesic distance as the shortest path length from k-nearest neighbor (k-nn) graph! *Eigen-decomposition on pairwise geodesic distance matrix to obtain embedding that best preserves given distances 38 * Recall eigen-decomposition is the main algorithm of PCA
39 Isomap (Isometric Feature Mapping) 39! Algorithm: all-pair shortest path computation + eigendecomposition! Pros: performs well in general! Cons: slow (shortest path), sensitive to parameters Nonlinear Unsupervised Global: all pairwise distances are considered Feature vectors
40 Isomap Facial Data Example Angle Person (k is the value in k-nn graph) k=8 Cluster structure 40 k=22 k=49 Which one do you think is the best?
41 t-sne (t-distributed Stochastic Neighborhood Embedding) Made specifically for visualization! (in very low dimension)! Can reveal clusters without any supervision! e.g., spoken letter data PCA t-sne 41 Official website:
42 t-sne (t-distributed Stochastic Neighborhood Embedding) How it works! Converts distance into probability! Farther distance gets lower probability! Then, minimize differences in probability distribution between high- and low-dimensional spaces! KL divergence naturally focuses on neighborhood relationships! Difference from SNE! t-sne uses heavy-tailed t-distribution instead of Gaussian.! Suitable for dimension reduction to a very low dimension 42
43 t-sne (t-distributed Stochastic Neighborhood Embedding)! Algorithm: gradient-decent type! Pros: works surprisingly well in 2D/3D visualization! Cons: very slow Nonlinear Unsupervised Local Similarity input 43
44 DR in Interactive Visualization What can you do from visualization via dimension reduction?! e.g., Multidimensional scaling applied to document data 44
45 DR in Interactive Visualization As many data items involve, it s harder to analyze! For n data items, users are given O(n 2 ) relations spatially encoded in visualization! Too many to understand in general 45
46 DR in Interactive Visualization What to first look at? Thus, people tend to look for a small number of objects that perceptually/visually stand out, e.g.,! Outliers (if any) 46
47 DR in Interactive Visualization What to first look at? Thus, people tend to look for a small number of objects that perceptually/visually stand out, e.g.,! Outliers (if any) More commonly,! Subgroups/clusters! However, it is hard to expect for DR to always reveal clusters 47
48 DR in Interactive Visualization What to first look at? What if DR cannot reveal subgroups/clusters clearly? Or even worse, what if our data do not originally have any?! Often, pre-defined grouping information is injected and color-coded.! Such grouping information is usually obtained as! Pre-given labels along with data! Computed labels by clustering 48
49 Dimension Reduction in Action Handwritten Digit Data Visualization Now we can obtain Cluster/data relationship Subcluster/outlier Visualization of handwritten digit data Subcluster #1 in 5 Subcluster #2 in 5 49 Major data in 7 Minor group #1 in 7 Minor group #2 in 7! Treating two subclusters in digit 5 as separate clusters! Classification accuracy improved from 89% to 93% (LDA+k-NN)
50 Practitioner s Guide Caveats Can you trust dimension reduction results?! Expect significant distortion/information loss in 2D/3D! What algorithm think is the best may not be what we think is the best, e.g., PCA visualization of facial image data (1, 2)-dimension (3, 4)-dimension 50
51 Practitioner s Guide Caveats How would you determine the best method and its parameters for your needs?! Unlike typical data mining problems where only one shot is allowed, you can freely try out different methods with different parameters! Basic understanding of methods will greatly help applying them properly! What is a particular method trying to achieve? And how suitable is it to your needs?! What are the effects of increasing/decreasing parameters? 51
52 Practitioner s Guide General Recommendation Want something simple and fast to visualize data?! PCA, force-directed layout Want to first try some manifold learning methods?! Isomap! if it doesn t show any good, probably neither will anything else. Have cluster label to use? (pre-given or computed)! LDA (supervised)! Supervised approach is sometimes the only viable option when your data do not have clearly separable clusters No labels, but still want some clusters to be revealed? Or simply, want some state-of-the-art method for visualization?! t-sne (but, may be slow) 52
53 Practitioner s Guide Results Still Not Good? Pre-process data properly as needed! Data centering! Subtract the global mean from each vector! Normalization! Make each vector have unit Euclidean norm! Otherwise, a few outlier can affect dimension reduction significantly! Application-specific pre-processing! Document: TF-IDF weighting, remove too rare and/or short terms! Image: histogram normalization 53
54 Practitioner s Guide Too Slow?! Apply PCA to reduce to an intermediate dimensions before the main dimension reduction step! t-sne does it by default! The results may even be improved due to noise removed by PCA! See if there is any approximated but faster version! Landmarked versions (only using a subset of data items)! e.g., landmarked Isomap! Linearized versions (the same criterion, but only allow linear mapping)! e.g., Laplacian Eigenmaps Locality preserving projection 54
55 Practitioner s Guide Still need more? Be creative! And feel free to tweak dimension reduction! Play with its algorithm, convergence criteria, etc.! See if you can impose label information 55 Original t-sne t-sne with simple modification
56 Practitioner s Guide Still need more? Be creative! And feel free to tweak dimension reduction! Play with its algorithm, convergence criteria, etc.! See if you can impose label information! Restrict the number of iterations to save computational time. 56 The raison d etre of DR is to serve us in exploring data and solving complicated real-world problems
57 Useful Resource Nice review article by L.J.P. van der Maaten et al.! dimensionality_reduction_a_comparative_review.pdf Matlab toolbox for dimension reduction! Matlab_Toolbox_for_Dimensionality_Reduction.html Matlab manifold learning demo! 57
58 Useful Resource FODAVA Testbed Software 58 Available at For a recent version, contact me at jaegul.choo@cc.gatech.edu
59 Thank you for your attention! Jaegul Choo
CSE 6242 / CX October 9, Dimension Reduction. Guest Lecturer: Jaegul Choo
CSE 6242 / CX 4242 October 9, 2014 Dimension Reduction Guest Lecturer: Jaegul Choo Volume Variety Big Data Era 2 Velocity Veracity 3 Big Data are High-Dimensional Examples of High-Dimensional Data Image
More informationCSE 6242 A / CS 4803 DVA. Feb 12, Dimension Reduction. Guest Lecturer: Jaegul Choo
CSE 6242 A / CS 4803 DVA Feb 12, 2013 Dimension Reduction Guest Lecturer: Jaegul Choo CSE 6242 A / CS 4803 DVA Feb 12, 2013 Dimension Reduction Guest Lecturer: Jaegul Choo Data is Too Big To Do Something..
More informationDimension Reduction CS534
Dimension Reduction CS534 Why dimension reduction? High dimensionality large number of features E.g., documents represented by thousands of words, millions of bigrams Images represented by thousands of
More informationNon-linear dimension reduction
Sta306b May 23, 2011 Dimension Reduction: 1 Non-linear dimension reduction ISOMAP: Tenenbaum, de Silva & Langford (2000) Local linear embedding: Roweis & Saul (2000) Local MDS: Chen (2006) all three methods
More informationData Mining Chapter 3: Visualizing and Exploring Data Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University
Data Mining Chapter 3: Visualizing and Exploring Data Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University Exploratory data analysis tasks Examine the data, in search of structures
More informationCPSC 340: Machine Learning and Data Mining. Multi-Dimensional Scaling Fall 2017
CPSC 340: Machine Learning and Data Mining Multi-Dimensional Scaling Fall 2017 Assignment 4: Admin 1 late day for tonight, 2 late days for Wednesday. Assignment 5: Due Monday of next week. Final: Details
More informationADVANCED MACHINE LEARNING. Mini-Project Overview
11 Mini-Project Overview Lecture : Prof. Aude Billard (aude.billard@epfl.ch) Teaching Assistants : Nadia Figueroa, Ilaria Lauzana, Brice Platerrier 22 Deadlines for projects / surveys Sign up for lit.
More informationCIE L*a*b* color model
CIE L*a*b* color model To further strengthen the correlation between the color model and human perception, we apply the following non-linear transformation: with where (X n,y n,z n ) are the tristimulus
More informationData Preprocessing. Javier Béjar. URL - Spring 2018 CS - MAI 1/78 BY: $\
Data Preprocessing Javier Béjar BY: $\ URL - Spring 2018 C CS - MAI 1/78 Introduction Data representation Unstructured datasets: Examples described by a flat set of attributes: attribute-value matrix Structured
More informationInterAxis: Steering Scatterplot Axes via Observation-Level Interaction
Interactive Axis InterAxis: Steering Scatterplot Axes via Observation-Level Interaction IEEE VAST 2015 Hannah Kim 1, Jaegul Choo 2, Haesun Park 1, Alex Endert 1 Georgia Tech 1, Korea University 2 October
More informationCourtesy of Prof. Shixia University
Courtesy of Prof. Shixia Liu @Tsinghua University Outline Introduction Classification of Techniques Table Scatter Plot Matrices Projections Parallel Coordinates Summary Motivation Real world data contain
More informationCPSC 340: Machine Learning and Data Mining. Deep Learning Fall 2018
CPSC 340: Machine Learning and Data Mining Deep Learning Fall 2018 Last Time: Multi-Dimensional Scaling Multi-dimensional scaling (MDS): Non-parametric visualization: directly optimize the z i locations.
More informationLocality Preserving Projections (LPP) Abstract
Locality Preserving Projections (LPP) Xiaofei He Partha Niyogi Computer Science Department Computer Science Department The University of Chicago The University of Chicago Chicago, IL 60615 Chicago, IL
More informationAn Interactive Visual Testbed System for Dimension Reduction and Clustering of Large-scale High-dimensional Data
An Interactive Visual Testbed System for Dimension Reduction and Clustering of Large-scale High-dimensional Data Jaegul Choo, Hanseung Lee, Zhicheng Liu, John Stasko, and Haesun Park Georgia Institute
More informationDimension reduction : PCA and Clustering
Dimension reduction : PCA and Clustering By Hanne Jarmer Slides by Christopher Workman Center for Biological Sequence Analysis DTU The DNA Array Analysis Pipeline Array design Probe design Question Experimental
More informationLarge-Scale Face Manifold Learning
Large-Scale Face Manifold Learning Sanjiv Kumar Google Research New York, NY * Joint work with A. Talwalkar, H. Rowley and M. Mohri 1 Face Manifold Learning 50 x 50 pixel faces R 2500 50 x 50 pixel random
More informationModelling and Visualization of High Dimensional Data. Sample Examination Paper
Duration not specified UNIVERSITY OF MANCHESTER SCHOOL OF COMPUTER SCIENCE Modelling and Visualization of High Dimensional Data Sample Examination Paper Examination date not specified Time: Examination
More informationLecture Topic Projects
Lecture Topic Projects 1 Intro, schedule, and logistics 2 Applications of visual analytics, basic tasks, data types 3 Introduction to D3, basic vis techniques for non-spatial data Project #1 out 4 Data
More informationMTTTS17 Dimensionality Reduction and Visualization. Spring 2018 Jaakko Peltonen. Lecture 11: Neighbor Embedding Methods continued
MTTTS17 Dimensionality Reduction and Visualization Spring 2018 Jaakko Peltonen Lecture 11: Neighbor Embedding Methods continued This Lecture Neighbor embedding by generative modeling Some supervised neighbor
More informationLocality Preserving Projections (LPP) Abstract
Locality Preserving Projections (LPP) Xiaofei He Partha Niyogi Computer Science Department Computer Science Department The University of Chicago The University of Chicago Chicago, IL 60615 Chicago, IL
More informationCPSC 340: Machine Learning and Data Mining. Principal Component Analysis Fall 2016
CPSC 340: Machine Learning and Data Mining Principal Component Analysis Fall 2016 A2/Midterm: Admin Grades/solutions will be posted after class. Assignment 4: Posted, due November 14. Extra office hours:
More informationUnsupervised Learning
Unsupervised Learning Learning without Class Labels (or correct outputs) Density Estimation Learn P(X) given training data for X Clustering Partition data into clusters Dimensionality Reduction Discover
More informationDeep Learning for Computer Vision
Deep Learning for Computer Vision Spring 2018 http://vllab.ee.ntu.edu.tw/dlcv.html (primary) https://ceiba.ntu.edu.tw/1062dlcv (grade, etc.) FB: DLCV Spring 2018 Yu Chiang Frank Wang 王鈺強, Associate Professor
More informationCluster Analysis. Mu-Chun Su. Department of Computer Science and Information Engineering National Central University 2003/3/11 1
Cluster Analysis Mu-Chun Su Department of Computer Science and Information Engineering National Central University 2003/3/11 1 Introduction Cluster analysis is the formal study of algorithms and methods
More informationEmotion Classification
Emotion Classification Shai Savir 038052395 Gil Sadeh 026511469 1. Abstract Automated facial expression recognition has received increased attention over the past two decades. Facial expressions convey
More informationSelecting Models from Videos for Appearance-Based Face Recognition
Selecting Models from Videos for Appearance-Based Face Recognition Abdenour Hadid and Matti Pietikäinen Machine Vision Group Infotech Oulu and Department of Electrical and Information Engineering P.O.
More informationData Preprocessing. Javier Béjar AMLT /2017 CS - MAI. (CS - MAI) Data Preprocessing AMLT / / 71 BY: $\
Data Preprocessing S - MAI AMLT - 2016/2017 (S - MAI) Data Preprocessing AMLT - 2016/2017 1 / 71 Outline 1 Introduction Data Representation 2 Data Preprocessing Outliers Missing Values Normalization Discretization
More informationIsometric Mapping Hashing
Isometric Mapping Hashing Yanzhen Liu, Xiao Bai, Haichuan Yang, Zhou Jun, and Zhihong Zhang Springer-Verlag, Computer Science Editorial, Tiergartenstr. 7, 692 Heidelberg, Germany {alfred.hofmann,ursula.barth,ingrid.haas,frank.holzwarth,
More informationLinear and Non-linear Dimentionality Reduction Applied to Gene Expression Data of Cancer Tissue Samples
Linear and Non-linear Dimentionality Reduction Applied to Gene Expression Data of Cancer Tissue Samples Franck Olivier Ndjakou Njeunje Applied Mathematics, Statistics, and Scientific Computation University
More informationAdvanced Data Visualization
Advanced Data Visualization CS 6965 Spring 2018 Prof. Bei Wang Phillips University of Utah Lecture 03 Dim Reduction & Vis t-sne HD Announcement Project 1 has been posted on the schedule webpage: http://www.sci.utah.edu/~beiwang/teaching/cs6965-spring-2018/
More informationLearning a Manifold as an Atlas Supplementary Material
Learning a Manifold as an Atlas Supplementary Material Nikolaos Pitelis Chris Russell School of EECS, Queen Mary, University of London [nikolaos.pitelis,chrisr,lourdes]@eecs.qmul.ac.uk Lourdes Agapito
More informationMSA220 - Statistical Learning for Big Data
MSA220 - Statistical Learning for Big Data Lecture 13 Rebecka Jörnsten Mathematical Sciences University of Gothenburg and Chalmers University of Technology Clustering Explorative analysis - finding groups
More informationData fusion and multi-cue data matching using diffusion maps
Data fusion and multi-cue data matching using diffusion maps Stéphane Lafon Collaborators: Raphy Coifman, Andreas Glaser, Yosi Keller, Steven Zucker (Yale University) Part of this work was supported by
More informationToday. Gradient descent for minimization of functions of real variables. Multi-dimensional scaling. Self-organizing maps
Today Gradient descent for minimization of functions of real variables. Multi-dimensional scaling Self-organizing maps Gradient Descent Derivatives Consider function f(x) : R R. The derivative w.r.t. x
More informationMultidimensional scaling Based in part on slides from textbook, slides of Susan Holmes. October 10, Statistics 202: Data Mining
Multidimensional scaling Based in part on slides from textbook, slides of Susan Holmes October 10, 2012 1 / 1 Multidimensional scaling A visual tool Recall the PCA scores were X V = U where X = HX S 1/2
More informationClustering CS 550: Machine Learning
Clustering CS 550: Machine Learning This slide set mainly uses the slides given in the following links: http://www-users.cs.umn.edu/~kumar/dmbook/ch8.pdf http://www-users.cs.umn.edu/~kumar/dmbook/dmslides/chap8_basic_cluster_analysis.pdf
More informationDimension Reduction of Image Manifolds
Dimension Reduction of Image Manifolds Arian Maleki Department of Electrical Engineering Stanford University Stanford, CA, 9435, USA E-mail: arianm@stanford.edu I. INTRODUCTION Dimension reduction of datasets
More informationChapter DM:II. II. Cluster Analysis
Chapter DM:II II. Cluster Analysis Cluster Analysis Basics Hierarchical Cluster Analysis Iterative Cluster Analysis Density-Based Cluster Analysis Cluster Evaluation Constrained Cluster Analysis DM:II-1
More informationApplying Supervised Learning
Applying Supervised Learning When to Consider Supervised Learning A supervised learning algorithm takes a known set of input data (the training set) and known responses to the data (output), and trains
More informationClustering K-means. Machine Learning CSEP546 Carlos Guestrin University of Washington February 18, Carlos Guestrin
Clustering K-means Machine Learning CSEP546 Carlos Guestrin University of Washington February 18, 2014 Carlos Guestrin 2005-2014 1 Clustering images Set of Images [Goldberger et al.] Carlos Guestrin 2005-2014
More informationVisualizing the quality of dimensionality reduction
Visualizing the quality of dimensionality reduction Bassam Mokbel 1, Wouter Lueks 2, Andrej Gisbrecht 1, Michael Biehl 2, Barbara Hammer 1 1) Bielefeld University - CITEC Centre of Excellence, Germany
More informationSensitivity to parameter and data variations in dimensionality reduction techniques
Sensitivity to parameter and data variations in dimensionality reduction techniques Francisco J. García-Fernández 1,2,MichelVerleysen 2, John A. Lee 3 and Ignacio Díaz 1 1- Univ. of Oviedo - Department
More informationNonlinear projections. Motivation. High-dimensional. data are. Perceptron) ) or RBFN. Multi-Layer. Example: : MLP (Multi(
Nonlinear projections Université catholique de Louvain (Belgium) Machine Learning Group http://www.dice.ucl ucl.ac.be/.ac.be/mlg/ 1 Motivation High-dimensional data are difficult to represent difficult
More informationManifold Learning for Video-to-Video Face Recognition
Manifold Learning for Video-to-Video Face Recognition Abstract. We look in this work at the problem of video-based face recognition in which both training and test sets are video sequences, and propose
More informationSlides for Data Mining by I. H. Witten and E. Frank
Slides for Data Mining by I. H. Witten and E. Frank 7 Engineering the input and output Attribute selection Scheme-independent, scheme-specific Attribute discretization Unsupervised, supervised, error-
More informationCSE 547: Machine Learning for Big Data Spring Problem Set 2. Please read the homework submission policies.
CSE 547: Machine Learning for Big Data Spring 2019 Problem Set 2 Please read the homework submission policies. 1 Principal Component Analysis and Reconstruction (25 points) Let s do PCA and reconstruct
More informationDimension reduction for hyperspectral imaging using laplacian eigenmaps and randomized principal component analysis
Dimension reduction for hyperspectral imaging using laplacian eigenmaps and randomized principal component analysis Yiran Li yl534@math.umd.edu Advisor: Wojtek Czaja wojtek@math.umd.edu 10/17/2014 Abstract
More informationMultiresponse Sparse Regression with Application to Multidimensional Scaling
Multiresponse Sparse Regression with Application to Multidimensional Scaling Timo Similä and Jarkko Tikka Helsinki University of Technology, Laboratory of Computer and Information Science P.O. Box 54,
More informationClustering. CE-717: Machine Learning Sharif University of Technology Spring Soleymani
Clustering CE-717: Machine Learning Sharif University of Technology Spring 2016 Soleymani Outline Clustering Definition Clustering main approaches Partitional (flat) Hierarchical Clustering validation
More informationRobust Pose Estimation using the SwissRanger SR-3000 Camera
Robust Pose Estimation using the SwissRanger SR- Camera Sigurjón Árni Guðmundsson, Rasmus Larsen and Bjarne K. Ersbøll Technical University of Denmark, Informatics and Mathematical Modelling. Building,
More informationRecognizing Handwritten Digits Using the LLE Algorithm with Back Propagation
Recognizing Handwritten Digits Using the LLE Algorithm with Back Propagation Lori Cillo, Attebury Honors Program Dr. Rajan Alex, Mentor West Texas A&M University Canyon, Texas 1 ABSTRACT. This work is
More informationFacial Expression Recognition Using Non-negative Matrix Factorization
Facial Expression Recognition Using Non-negative Matrix Factorization Symeon Nikitidis, Anastasios Tefas and Ioannis Pitas Artificial Intelligence & Information Analysis Lab Department of Informatics Aristotle,
More informationVisualizing Data using t-sne
Journal of Machine Learning Research 9 (2008) Submitted 5/08; Published Visualizing Data using t-sne Laurens van der Maaten MICC-IKAT Maastricht University P.O. Box 616, 6200 MD Maastricht, The Netherlands
More informationClustering. CS294 Practical Machine Learning Junming Yin 10/09/06
Clustering CS294 Practical Machine Learning Junming Yin 10/09/06 Outline Introduction Unsupervised learning What is clustering? Application Dissimilarity (similarity) of objects Clustering algorithm K-means,
More informationLocally Linear Landmarks for large-scale manifold learning
Locally Linear Landmarks for large-scale manifold learning Max Vladymyrov and Miguel Á. Carreira-Perpiñán Electrical Engineering and Computer Science University of California, Merced http://eecs.ucmerced.edu
More informationCase-Based Reasoning. CS 188: Artificial Intelligence Fall Nearest-Neighbor Classification. Parametric / Non-parametric.
CS 188: Artificial Intelligence Fall 2008 Lecture 25: Kernels and Clustering 12/2/2008 Dan Klein UC Berkeley Case-Based Reasoning Similarity for classification Case-based reasoning Predict an instance
More informationCS 188: Artificial Intelligence Fall 2008
CS 188: Artificial Intelligence Fall 2008 Lecture 25: Kernels and Clustering 12/2/2008 Dan Klein UC Berkeley 1 1 Case-Based Reasoning Similarity for classification Case-based reasoning Predict an instance
More informationSYDE Winter 2011 Introduction to Pattern Recognition. Clustering
SYDE 372 - Winter 2011 Introduction to Pattern Recognition Clustering Alexander Wong Department of Systems Design Engineering University of Waterloo Outline 1 2 3 4 5 All the approaches we have learned
More informationFeature Selection. CE-725: Statistical Pattern Recognition Sharif University of Technology Spring Soleymani
Feature Selection CE-725: Statistical Pattern Recognition Sharif University of Technology Spring 2013 Soleymani Outline Dimensionality reduction Feature selection vs. feature extraction Filter univariate
More informationPouya Kousha Fall 2018 CSE 5194 Prof. DK Panda
Pouya Kousha Fall 2018 CSE 5194 Prof. DK Panda 1 Observe novel applicability of DL techniques in Big Data Analytics. Applications of DL techniques for common Big Data Analytics problems. Semantic indexing
More informationAN IMPROVED HYBRIDIZED K- MEANS CLUSTERING ALGORITHM (IHKMCA) FOR HIGHDIMENSIONAL DATASET & IT S PERFORMANCE ANALYSIS
AN IMPROVED HYBRIDIZED K- MEANS CLUSTERING ALGORITHM (IHKMCA) FOR HIGHDIMENSIONAL DATASET & IT S PERFORMANCE ANALYSIS H.S Behera Department of Computer Science and Engineering, Veer Surendra Sai University
More informationDimension reduction for hyperspectral imaging using laplacian eigenmaps and randomized principal component analysis:midyear Report
Dimension reduction for hyperspectral imaging using laplacian eigenmaps and randomized principal component analysis:midyear Report Yiran Li yl534@math.umd.edu Advisor: Wojtek Czaja wojtek@math.umd.edu
More informationDimensionality Reduction and Visualization
MTTS1 Dimensionality Reduction and Visualization! Spring 2014 Jaakko Peltonen Lecture 6: Nonlinear dimensionality reduction, part 1 Dimension reduction Dimension reduction methods What to do if... I have
More informationSupport Vector Machines for visualization and dimensionality reduction
Support Vector Machines for visualization and dimensionality reduction Tomasz Maszczyk and W lodzis law Duch Department of Informatics, Nicolaus Copernicus University, Toruń, Poland tmaszczyk@is.umk.pl;google:w.duch
More informationComputational Statistics and Mathematics for Cyber Security
and Mathematics for Cyber Security David J. Marchette Sept, 0 Acknowledgment: This work funded in part by the NSWC In-House Laboratory Independent Research (ILIR) program. NSWCDD-PN--00 Topics NSWCDD-PN--00
More informationSchool of Computer and Communication, Lanzhou University of Technology, Gansu, Lanzhou,730050,P.R. China
Send Orders for Reprints to reprints@benthamscienceae The Open Automation and Control Systems Journal, 2015, 7, 253-258 253 Open Access An Adaptive Neighborhood Choosing of the Local Sensitive Discriminant
More informationContent-based image and video analysis. Machine learning
Content-based image and video analysis Machine learning for multimedia retrieval 04.05.2009 What is machine learning? Some problems are very hard to solve by writing a computer program by hand Almost all
More informationGene Clustering & Classification
BINF, Introduction to Computational Biology Gene Clustering & Classification Young-Rae Cho Associate Professor Department of Computer Science Baylor University Overview Introduction to Gene Clustering
More informationClustering and Visualisation of Data
Clustering and Visualisation of Data Hiroshi Shimodaira January-March 28 Cluster analysis aims to partition a data set into meaningful or useful groups, based on distances between data points. In some
More informationPart I. Graphical exploratory data analysis. Graphical summaries of data. Graphical summaries of data
Week 3 Based in part on slides from textbook, slides of Susan Holmes Part I Graphical exploratory data analysis October 10, 2012 1 / 1 2 / 1 Graphical summaries of data Graphical summaries of data Exploratory
More informationCorrespondence. CS 468 Geometry Processing Algorithms. Maks Ovsjanikov
Shape Matching & Correspondence CS 468 Geometry Processing Algorithms Maks Ovsjanikov Wednesday, October 27 th 2010 Overall Goal Given two shapes, find correspondences between them. Overall Goal Given
More informationCurvilinear Distance Analysis versus Isomap
Curvilinear Distance Analysis versus Isomap John Aldo Lee, Amaury Lendasse, Michel Verleysen Université catholique de Louvain Place du Levant, 3, B-1348 Louvain-la-Neuve, Belgium {lee,verleysen}@dice.ucl.ac.be,
More information1 Case study of SVM (Rob)
DRAFT a final version will be posted shortly COS 424: Interacting with Data Lecturer: Rob Schapire and David Blei Lecture # 8 Scribe: Indraneel Mukherjee March 1, 2007 In the previous lecture we saw how
More informationVisual Representations for Machine Learning
Visual Representations for Machine Learning Spectral Clustering and Channel Representations Lecture 1 Spectral Clustering: introduction and confusion Michael Felsberg Klas Nordberg The Spectral Clustering
More informationCPSC 340: Machine Learning and Data Mining. Recommender Systems Fall 2017
CPSC 340: Machine Learning and Data Mining Recommender Systems Fall 2017 Assignment 4: Admin Due tonight, 1 late day for Monday, 2 late days for Wednesday. Assignment 5: Posted, due Monday of last week
More informationTechnical Report. Title: Manifold learning and Random Projections for multi-view object recognition
Technical Report Title: Manifold learning and Random Projections for multi-view object recognition Authors: Grigorios Tsagkatakis 1 and Andreas Savakis 2 1 Center for Imaging Science, Rochester Institute
More informationLecture 19: Generative Adversarial Networks
Lecture 19: Generative Adversarial Networks Roger Grosse 1 Introduction Generative modeling is a type of machine learning where the aim is to model the distribution that a given set of data (e.g. images,
More informationOn the Effect of Clustering on Quality Assessment Measures for Dimensionality Reduction
On the Effect of Clustering on Quality Assessment Measures for Dimensionality Reduction Bassam Mokbel, Andrej Gisbrecht, Barbara Hammer CITEC Center of Excellence Bielefeld University D-335 Bielefeld {bmokbel
More informationSupervised vs. Unsupervised Learning
Clustering Supervised vs. Unsupervised Learning So far we have assumed that the training samples used to design the classifier were labeled by their class membership (supervised learning) We assume now
More informationMachine Learning : Clustering, Self-Organizing Maps
Machine Learning Clustering, Self-Organizing Maps 12/12/2013 Machine Learning : Clustering, Self-Organizing Maps Clustering The task: partition a set of objects into meaningful subsets (clusters). The
More informationImage Processing. Image Features
Image Processing Image Features Preliminaries 2 What are Image Features? Anything. What they are used for? Some statements about image fragments (patches) recognition Search for similar patches matching
More informationClustering Web Documents using Hierarchical Method for Efficient Cluster Formation
Clustering Web Documents using Hierarchical Method for Efficient Cluster Formation I.Ceema *1, M.Kavitha *2, G.Renukadevi *3, G.sripriya *4, S. RajeshKumar #5 * Assistant Professor, Bon Secourse College
More informationBehavioral Data Mining. Lecture 18 Clustering
Behavioral Data Mining Lecture 18 Clustering Outline Why? Cluster quality K-means Spectral clustering Generative Models Rationale Given a set {X i } for i = 1,,n, a clustering is a partition of the X i
More informationMachine Learning using MapReduce
Machine Learning using MapReduce What is Machine Learning Machine learning is a subfield of artificial intelligence concerned with techniques that allow computers to improve their outputs based on previous
More informationA Survey on Pre-processing and Post-processing Techniques in Data Mining
, pp. 99-128 http://dx.doi.org/10.14257/ijdta.2014.7.4.09 A Survey on Pre-processing and Post-processing Techniques in Data Mining Divya Tomar and Sonali Agarwal Indian Institute of Information Technology,
More informationSubspace Clustering. Weiwei Feng. December 11, 2015
Subspace Clustering Weiwei Feng December 11, 2015 Abstract Data structure analysis is an important basis of machine learning and data science, which is now widely used in computational visualization problems,
More informationCOSC160: Detection and Classification. Jeremy Bolton, PhD Assistant Teaching Professor
COSC160: Detection and Classification Jeremy Bolton, PhD Assistant Teaching Professor Outline I. Problem I. Strategies II. Features for training III. Using spatial information? IV. Reducing dimensionality
More informationFeature selection. LING 572 Fei Xia
Feature selection LING 572 Fei Xia 1 Creating attribute-value table x 1 x 2 f 1 f 2 f K y Choose features: Define feature templates Instantiate the feature templates Dimensionality reduction: feature selection
More informationMachine Learning Techniques for Data Mining
Machine Learning Techniques for Data Mining Eibe Frank University of Waikato New Zealand 10/25/2000 1 PART VII Moving on: Engineering the input and output 10/25/2000 2 Applying a learner is not all Already
More informationAkarsh Pokkunuru EECS Department Contractive Auto-Encoders: Explicit Invariance During Feature Extraction
Akarsh Pokkunuru EECS Department 03-16-2017 Contractive Auto-Encoders: Explicit Invariance During Feature Extraction 1 AGENDA Introduction to Auto-encoders Types of Auto-encoders Analysis of different
More informationEfficient Algorithms may not be those we think
Efficient Algorithms may not be those we think Yann LeCun, Computational and Biological Learning Lab The Courant Institute of Mathematical Sciences New York University http://yann.lecun.com http://www.cs.nyu.edu/~yann
More informationFace Recognition using Tensor Analysis. Prahlad R. Enuganti
Face Recognition using Tensor Analysis Prahlad R. Enuganti The University of Texas at Austin Final Report EE381K 14 Multidimensional Digital Signal Processing May 16, 2005 Submitted to Prof. Brian Evans
More informationNeural Networks for Machine Learning. Lecture 15a From Principal Components Analysis to Autoencoders
Neural Networks for Machine Learning Lecture 15a From Principal Components Analysis to Autoencoders Geoffrey Hinton Nitish Srivastava, Kevin Swersky Tijmen Tieleman Abdel-rahman Mohamed Principal Components
More informationExploratory Data Analysis using Self-Organizing Maps. Madhumanti Ray
Exploratory Data Analysis using Self-Organizing Maps Madhumanti Ray Content Introduction Data Analysis methods Self-Organizing Maps Conclusion Visualization of high-dimensional data items Exploratory data
More informationThe Curse of Dimensionality. Panagiotis Parchas Advanced Data Management Spring 2012 CSE HKUST
The Curse of Dimensionality Panagiotis Parchas Advanced Data Management Spring 2012 CSE HKUST Multiple Dimensions As we discussed in the lectures, many times it is convenient to transform a signal(time
More informationCIS 520, Machine Learning, Fall 2015: Assignment 7 Due: Mon, Nov 16, :59pm, PDF to Canvas [100 points]
CIS 520, Machine Learning, Fall 2015: Assignment 7 Due: Mon, Nov 16, 2015. 11:59pm, PDF to Canvas [100 points] Instructions. Please write up your responses to the following problems clearly and concisely.
More informationFinding Structure in CyTOF Data
Finding Structure in CyTOF Data Or, how to visualize low dimensional embedded manifolds. Panagiotis Achlioptas panos@cs.stanford.edu General Terms Algorithms, Experimentation, Measurement Keywords Manifold
More informationRDRToolbox A package for nonlinear dimension reduction with Isomap and LLE.
RDRToolbox A package for nonlinear dimension reduction with Isomap and LLE. Christoph Bartenhagen October 30, 2017 Contents 1 Introduction 1 1.1 Loading the package......................................
More informationVisual Encoding Design
CSE 442 - Data Visualization Visual Encoding Design Jeffrey Heer University of Washington Review: Expressiveness & Effectiveness / APT Choosing Visual Encodings Assume k visual encodings and n data attributes.
More informationECG782: Multidimensional Digital Signal Processing
ECG782: Multidimensional Digital Signal Processing Object Recognition http://www.ee.unlv.edu/~b1morris/ecg782/ 2 Outline Knowledge Representation Statistical Pattern Recognition Neural Networks Boosting
More information