Advanced Applied Multivariate Analysis
|
|
|
- Alisha Smith
- 6 years ago
- Views:
Transcription
1 Advanced Applied Multivariate Analysis STAT, Fall 3 Sungkyu Jung Department of Statistics University of Pittsburgh sungkyu@pitt.edu / 3
2 General Information Course Webpage: Prerequisite: None (officially), but probability and inference theory linear algebra R, SAS or Matlab programing. / 3
3 What is multivariate analysis? First course of statistics: numbers random variables Second course of statistics: vectors of numbers random vectors Basis for analysis of more complex objects, e.g. functions, matrices, tensors, images, networks. 3 Data Exploration: visualization of relationships between observations. 4 Discovering and modeling patterns from dataset: Visualization, Clustering, Multivariate distributions. 5 Confirming patterns: Inference. 6 Dimension Reduction: PCA, CCA, SVD. 7 Predictions: Regression, Classification. 3 / 3
4 What is multivariate dataset? Multivariate statistical analysis concerns multivariate data where each observation consisting of many measurements on the same subject. We suppose the dataset X = {X,..., X n } has n observations (Here, n is called the sample size), and each observation X i = (x i,..., x ip ) is a vector in R p (Here, p is called the dimension). These are often recorded in a p n matrix: X = ( x x n ) X,, X n =..... x p x np 4 / 3
5 Data Exploration - Visualization D example Hidalgo Stamp Data n = 485 observed values of thickness for Mexico stamps over > 7 years During 98s Stamp papers produced in several factories? No records. Can we guess by looking at the data? Izenman and Sommer (988), here we use data stamp in R package BSDA. 5 / 3
6 Data Exploration - D Hidalgo Stamp Histograms with different bin widths. How many factories? Histogram of thickness Histogram of thickness Frequency 5 5 Frequency h = h =. Histogram of thickness Histogram of thickness Frequency 4 8 Frequency h = h =. 6 / 3
7 Data Exploration - D Hidalgo Stamp Histograms with different bin locations. (Fixed bin width.) Histogram of thickness Histogram of thickness Frequency 5 Frequency thickness thickness Histogram of thickness Histogram of thickness Frequency Frequency thickness thickness 7 / 3
8 Data Exploration - Kernel density estimate Histograms are dependent on binwidth and bin location. Smaller binwidth preferred for exploration. Different bin locations can obscure important underlying patterns; A solution is to average out the effect on different bin location Averaged histogram A more elegant solution is kernel density estimate. For X,..., X n i.i.d. f (x) (continuous pdf), a kernel density estimator of f is obtained as ˆf h (x) = n ( ) x n h K Xi, h i= where the kernel K( ) is a function satisfying K(x)dx =. 8 / 3
9 Kernel density estimate - illustration Toy data: x i = 3, 5, 9,,, 4. Consider a kernel density estimate with Gaussian kernel K(t) = π e t /, with bandwidth h =. For each i, h K ( ) x xi = h (x x πh e i ) h. 9 / 3
10 Kernel density estimate - illustration.8 toy data density observations Toy data: x i = 3, 5, 9,,, 4. / 3
11 Kernel density estimate - illustration.8 toy data density observations overlaid is, for h =, ( ) x h K x = h πh e (x x ) h. / 3
12 Kernel density estimate - illustration.8 toy data density observations For all i =,..., 6, ( ) x h K xi = h (x x πh e i ) h. / 3
13 Kernel density estimate - illustration.8 toy data density observations Kernel Density Estimate (KDE) of the pdf: ˆf h (x) = n n i= h K ( x Xi h ), 3 / 3
14 KDE for Hidalgo Stamp KDE KDE Density Density N = 485 Bandwidth = N = 485 Bandwidth =.5 KDE KDE Density 3 4 Density N = 485 Bandwidth = N = 485 Bandwidth =. 4 / 3
15 KDE for Hidalgo Stamp Small bandwidth leads undersmooth; Large bandwidth leads oversmooth Is it unimodal? Bi-modal? Or several modes? Which modes are really there? Choice of bandwidth Important in practice. Controversial issue. Many recommendations (Silverman s rule of thumb, Sheather-Jones Plug-In.) The consensus is that there is never a consensus. 5 / 3
16 KDE for Hidalgo Stamp Bandwidth, by Silverman s rule-of-thumb, is (sample size) /5 9% of minimum of two population standard deviation estimates: i) the sample standard deviation, ii) IQR/.34 KDE Density N = 485 Bandwidth =.39 6 / 3
17 Data Exploration - Multivariate data Dimension p = 6 example Swiss bank notes n = Swiss bank notes (See Fig.., Härdle and Simar) Each note (obs.) has p = 6 measurements (variables). Additional information: first half are genuine; the other half are counterfeit. Visualization of 6-dim l data? Can use 6 KDEs overlaided with jitterplot for each measurements (variables) 7 / 3
18 Swiss bank notes - Marginal KDEs Marginal KDEs overlaided with jitterplot for each of 6 variables. Informative, realistic when p is small No information about association between variables kde.5 kde kde x x x3.7.8 kde x4 kde x5 kde x6 8 / 3
19 Swiss bank notes - Scatterplot Pairs of variables are best visualized by scatterplot, e.g. X 4 vs X 6 below. Understood as point clouds (representing the empirical distributions) ( ) 6 many pairs to choose from. 4.5 Swiss bank notes: blue o = genuine, green x = fake X X 4 9 / 3
20 Swiss bank notes - Scatterplot Scatters of three variables can also be informative, if software allows to rotate the axes. Otherwise, the 3D scatterplot is a scatterplot of linear combinations of the three variables. Swiss bank notes: blue o = genuine, green x = fake X X X 4 3 / 3
21 Swiss bank notes - scatterplot matrix A traditional, yet powerful, tool is to construct a matrix of scatterplots. - Too busy with p = 6..5 X X.5.5 X X X X 4 X 3 X 5 X X 6 X.5.5 X X X.5.5 X X X.5.5 X 4 X 4 X X 5 3 X 5 X.5.5 X 6 X 6 X 4 X X 4. X 4 3 X 4 X X X 4 X 5 X 6 X 5 3 X X X X 5 3 X 5 3 X X 5 X 5 3 X 6 X 6 X X X X 6 X 6 X 4 X 6 3 X 5.4. X 6 / 3
22 Swiss bank notes - scatterplot matrix Better to visualize with principal component scores. Direction 5 Direction 6 Direction Direction 4. Direction Direction Direction Direction.5.5 Direction.5.5 Direction 5 Direction 6 Direction Direction Direction 4 Direction.4. Direction Direction Direction.5.5 Direction.5.5 Direction 5 Direction 6 Direction Direction Direction Direction Direction 5 Direction 6 Direction Direction Direction 4 Direction 4 Direction 4.5 Direction Direction Direction 6 Direction 4 Direction Direction Direction Direction Direction Direction Direction 5 Direction Direction Direction 5 Direction Direction Direction Direction Direction Direction Direction Direction Direction 6 / 3
23 Swiss bank notes - scatterplot matrix With principal component scores, we can focus on fewer combinations; Dimension Reduction.3.. Direction Direction Direction Direction.4 Direction.3.. Direction Direction Direction.8 Direction Direction / 3
24 Swiss bank notes - related methods If obtaining succint representation of data is of interest, then using the first two principal component scores appeared in the first block of the scatterplot matrix would do the job (Principal Component Analysis) Parametric modeling: Are distributions of Swiss bank note measurements normal (Gaussian)? (Multivariate Normal Distribution) 3 Without the information on genuine and counterfate notes, can we classify n = notes into two distinct groups? (Clustering) 4 With the information on genuine and counterfate notes, Are the means and covariances of genuine and counterfate notes different? (Statistical inference) When there is a new bank note, how to predict whether the new note is genuine? (Classification) 4 / 3
25 High dimensional data Modern challenges Gene expression data in Golub et al. (999) p = 79 gene expression levels (numeric) for n = 47 subjects with acute lymphoblastic leukemia (ALL) and n = 5 subjects with acute myeloid leukemia (AML). 3 Scientific task is to identify new cancer class and/or to assign tumors to known classes (ALL or AML). 5 / 3
26 Golub data Taking a subgroup of data with 7 ALLs and AMLs. A visualization of the matrix X p n. Is it helpful? Processed gene expressions (Golub) genes subject index: 7 = ALL, 8 37 = AML 6 / 3
27 Golub data Scatterplot matrix for the first three genes (variables). x 3 4 ALL AML X X X x 3 X X 4 X 3 X X x X X 7 / 3
28 Golub data Scatterplot matrix using the first three Principal Component Scores. A pattern there? x 5.5 ALL.5 AML 5 5 PC Direction x 4 x 4 PC Direction x PC Direction x 4.5 x 5 PC Direction x PC3 Direction x 4 x 4 PC Direction PC3 Direction PC Direction 5 x x PC Direction x PC3 Direction PC Direction x 4 x PC Direction x 4 PC Direction PC3 Direction x 4 x PC3 Direction x 4 8 / 3
29 Golub data Scatterplot matrix using the a liniear combination of all variables (that leads a good separation of two groups) and two Principal Component Scores. Better pattern? PC Direction x 4 3 ALL AML Discriminant Dir. x 4 x Discriminant Dir. x 4 x 4 Discriminant Dir. 4 x PC Direction x x PC Direction x x 4 PC Direction Discriminant Dir. x PC Direction x 4 x PC Direction x 4.5 x 5 PC Direction 5 5 Discriminant Dir. x 4 PC Direction PC Direction 5 x PC Direction x 4 9 / 3
30 Next: review on matrix algebra. 3 / 3
Nonparametric Density Estimation
Nonparametric Estimation Data: X 1,..., X n iid P where P is a distribution with density f(x). Aim: Estimation of density f(x) Parametric density estimation: Fit parametric model {f(x θ) θ Θ} to data parameter
Quiz Section Week 8 May 17, Machine learning and Support Vector Machines
Quiz Section Week 8 May 17, 2016 Machine learning and Support Vector Machines Another definition of supervised machine learning Given N training examples (objects) {(x 1,y 1 ), (x 2,y 2 ),, (x N,y N )}
Exploratory data analysis for microarrays
Exploratory data analysis for microarrays Jörg Rahnenführer Computational Biology and Applied Algorithmics Max Planck Institute for Informatics D-66123 Saarbrücken Germany NGFN - Courses in Practical DNA
Expectation Maximization (EM) and Gaussian Mixture Models
Expectation Maximization (EM) and Gaussian Mixture Models Reference: The Elements of Statistical Learning, by T. Hastie, R. Tibshirani, J. Friedman, Springer 1 2 3 4 5 6 7 8 Unsupervised Learning Motivation
Density estimation. In density estimation problems, we are given a random from an unknown density. Our objective is to estimate
Density estimation In density estimation problems, we are given a random sample from an unknown density Our objective is to estimate? Applications Classification If we estimate the density for each class,
Density estimation. In density estimation problems, we are given a random from an unknown density. Our objective is to estimate
Density estimation In density estimation problems, we are given a random sample from an unknown density Our objective is to estimate? Applications Classification If we estimate the density for each class,
9/29/13. Outline Data mining tasks. Clustering algorithms. Applications of clustering in biology
9/9/ I9 Introduction to Bioinformatics, Clustering algorithms Yuzhen Ye (yye@indiana.edu) School of Informatics & Computing, IUB Outline Data mining tasks Predictive tasks vs descriptive tasks Example
Instance-based Learning CE-717: Machine Learning Sharif University of Technology. M. Soleymani Fall 2015
Instance-based Learning CE-717: Machine Learning Sharif University of Technology M. Soleymani Fall 2015 Outline Non-parametric approach Unsupervised: Non-parametric density estimation Parzen Windows K-Nearest
Nature Methods: doi: /nmeth Supplementary Figure 1
Supplementary Figure 1 Schematic representation of the Workflow window in Perseus All data matrices uploaded in the running session of Perseus and all processing steps are displayed in the order of execution.
Introduction The problem of cancer classication has clear implications on cancer treatment. Additionally, the advent of DNA microarrays introduces a w
MASSACHUSETTS INSTITUTE OF TECHNOLOGY ARTIFICIAL INTELLIGENCE LABORATORY and CENTER FOR BIOLOGICAL AND COMPUTATIONAL LEARNING DEPARTMENT OF BRAIN AND COGNITIVE SCIENCES A.I. Memo No.677 C.B.C.L Paper No.8
Introduction to machine learning, pattern recognition and statistical data modelling Coryn Bailer-Jones
Introduction to machine learning, pattern recognition and statistical data modelling Coryn Bailer-Jones What is machine learning? Data interpretation describing relationship between predictors and responses
An Introduction to PDF Estimation and Clustering
Sigmedia, Electronic Engineering Dept., Trinity College, Dublin. 1 An Introduction to PDF Estimation and Clustering David Corrigan corrigad@tcd.ie Electrical and Electronic Engineering Dept., University
CANCER PREDICTION USING PATTERN CLASSIFICATION OF MICROARRAY DATA. By: Sudhir Madhav Rao &Vinod Jayakumar Instructor: Dr.
CANCER PREDICTION USING PATTERN CLASSIFICATION OF MICROARRAY DATA By: Sudhir Madhav Rao &Vinod Jayakumar Instructor: Dr. Michael Nechyba 1. Abstract The objective of this project is to apply well known
Data Mining with SPSS Modeler
Tilo Wendler Soren Grottrup Data Mining with SPSS Modeler Theory, Exercises and Solutions Springer 1 Introduction 1 1.1 The Concept of the SPSS Modeler 2 1.2 Structure and Features of This Book 5 1.2.1
Supervised vs unsupervised clustering
Classification Supervised vs unsupervised clustering Cluster analysis: Classes are not known a- priori. Classification: Classes are defined a-priori Sometimes called supervised clustering Extract useful
Last time... Coryn Bailer-Jones. check and if appropriate remove outliers, errors etc. linear regression
Machine learning, pattern recognition and statistical data modelling Lecture 3. Linear Methods (part 1) Coryn Bailer-Jones Last time... curse of dimensionality local methods quickly become nonlocal as
Data Mining Chapter 3: Visualizing and Exploring Data Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University
Data Mining Chapter 3: Visualizing and Exploring Data Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University Exploratory data analysis tasks Examine the data, in search of structures
COMPUTATIONAL STATISTICS UNSUPERVISED LEARNING
COMPUTATIONAL STATISTICS UNSUPERVISED LEARNING Luca Bortolussi Department of Mathematics and Geosciences University of Trieste Office 238, third floor, H2bis luca@dmi.units.it Trieste, Winter Semester
Data Mining. CS57300 Purdue University. Bruno Ribeiro. February 1st, 2018
Data Mining CS57300 Purdue University Bruno Ribeiro February 1st, 2018 1 Exploratory Data Analysis & Feature Construction How to explore a dataset Understanding the variables (values, ranges, and empirical
Clustering. Mihaela van der Schaar. January 27, Department of Engineering Science University of Oxford
Department of Engineering Science University of Oxford January 27, 2017 Many datasets consist of multiple heterogeneous subsets. Cluster analysis: Given an unlabelled data, want algorithms that automatically
Acquisition Description Exploration Examination Understanding what data is collected. Characterizing properties of data.
Summary Statistics Acquisition Description Exploration Examination what data is collected Characterizing properties of data. Exploring the data distribution(s). Identifying data quality problems. Selecting
Network Traffic Measurements and Analysis
DEIB - Politecnico di Milano Fall, 2017 Introduction Often, we have only a set of features x = x 1, x 2,, x n, but no associated response y. Therefore we are not interested in prediction nor classification,
Machine Learning in Biology
Università degli studi di Padova Machine Learning in Biology Luca Silvestrin (Dottorando, XXIII ciclo) Supervised learning Contents Class-conditional probability density Linear and quadratic discriminant
Data Mining and Analytics
Data Mining and Analytics Aik Choon Tan, Ph.D. Associate Professor of Bioinformatics Division of Medical Oncology Department of Medicine aikchoon.tan@ucdenver.edu 9/22/2017 http://tanlab.ucdenver.edu/labhomepage/teaching/bsbt6111/
Homework. Gaussian, Bishop 2.3 Non-parametric, Bishop 2.5 Linear regression Pod-cast lecture on-line. Next lectures:
Homework Gaussian, Bishop 2.3 Non-parametric, Bishop 2.5 Linear regression 3.0-3.2 Pod-cast lecture on-line Next lectures: I posted a rough plan. It is flexible though so please come with suggestions Bayes
Applying Supervised Learning
Applying Supervised Learning When to Consider Supervised Learning A supervised learning algorithm takes a known set of input data (the training set) and known responses to the data (output), and trains
Non-Parametric Modeling
Non-Parametric Modeling CE-725: Statistical Pattern Recognition Sharif University of Technology Spring 2013 Soleymani Outline Introduction Non-Parametric Density Estimation Parzen Windows Kn-Nearest Neighbor
Support Vector Machines: Brief Overview" November 2011 CPSC 352
Support Vector Machines: Brief Overview" Outline Microarray Example Support Vector Machines (SVMs) Software: libsvm A Baseball Example with libsvm Classifying Cancer Tissue: The ALL/AML Dataset Golub et
10-701/15-781, Fall 2006, Final
-7/-78, Fall 6, Final Dec, :pm-8:pm There are 9 questions in this exam ( pages including this cover sheet). If you need more room to work out your answer to a question, use the back of the page and clearly
CS6220: DATA MINING TECHNIQUES
CS6220: DATA MINING TECHNIQUES 2: Data Pre-Processing Instructor: Yizhou Sun yzsun@ccs.neu.edu September 10, 2013 2: Data Pre-Processing Getting to know your data Basic Statistical Descriptions of Data
CPSC 340: Machine Learning and Data Mining. Multi-Dimensional Scaling Fall 2017
CPSC 340: Machine Learning and Data Mining Multi-Dimensional Scaling Fall 2017 Assignment 4: Admin 1 late day for tonight, 2 late days for Wednesday. Assignment 5: Due Monday of next week. Final: Details
Kernel Density Estimation (KDE)
Kernel Density Estimation (KDE) Previously, we ve seen how to use the histogram method to infer the probability density function (PDF) of a random variable (population) using a finite data sample. In this
1 Overview XploRe is an interactive computational environment for statistics. The aim of XploRe is to provide a full, high-level programming language
Teaching Statistics with XploRe Marlene Muller Institute for Statistics and Econometrics, Humboldt University Berlin Spandauer Str. 1, D{10178 Berlin, Germany marlene@wiwi.hu-berlin.de, http://www.wiwi.hu-berlin.de/marlene
Bayes Classifiers and Generative Methods
Bayes Classifiers and Generative Methods CSE 4309 Machine Learning Vassilis Athitsos Computer Science and Engineering Department University of Texas at Arlington 1 The Stages of Supervised Learning To
COMS 4771 Clustering. Nakul Verma
COMS 4771 Clustering Nakul Verma Supervised Learning Data: Supervised learning Assumption: there is a (relatively simple) function such that for most i Learning task: given n examples from the data, find
Adaptive Learning of an Accurate Skin-Color Model
Adaptive Learning of an Accurate Skin-Color Model Q. Zhu K.T. Cheng C. T. Wu Y. L. Wu Electrical & Computer Engineering University of California, Santa Barbara Presented by: H.T Wang Outline Generic Skin
CS 229 Midterm Review
CS 229 Midterm Review Course Staff Fall 2018 11/2/2018 Outline Today: SVMs Kernels Tree Ensembles EM Algorithm / Mixture Models [ Focus on building intuition, less so on solving specific problems. Ask
Data Preprocessing. Data Preprocessing
Data Preprocessing Dr. Sanjay Ranka Professor Computer and Information Science and Engineering University of Florida, Gainesville ranka@cise.ufl.edu Data Preprocessing What preprocessing step can or should
Regression III: Advanced Methods
Lecture 3: Distributions Regression III: Advanced Methods William G. Jacoby Michigan State University Goals of the lecture Examine data in graphical form Graphs for looking at univariate distributions
Sparse and large-scale learning with heterogeneous data
Sparse and large-scale learning with heterogeneous data February 15, 2007 Gert Lanckriet (gert@ece.ucsd.edu) IEEE-SDCIS In this talk Statistical machine learning Techniques: roots in classical statistics
Clustering and The Expectation-Maximization Algorithm
Clustering and The Expectation-Maximization Algorithm Unsupervised Learning Marek Petrik 3/7 Some of the figures in this presentation are taken from An Introduction to Statistical Learning, with applications
SD 372 Pattern Recognition
SD 372 Pattern Recognition Lab 2: Model Estimation and Discriminant Functions 1 Purpose This lab examines the areas of statistical model estimation and classifier aggregation. Model estimation will be
Generative and discriminative classification techniques
Generative and discriminative classification techniques Machine Learning and Category Representation 2014-2015 Jakob Verbeek, November 28, 2014 Course website: http://lear.inrialpes.fr/~verbeek/mlcr.14.15
Dynamic Thresholding for Image Analysis
Dynamic Thresholding for Image Analysis Statistical Consulting Report for Edward Chan Clean Energy Research Center University of British Columbia by Libo Lu Department of Statistics University of British
Feature extraction techniques to use in cereal classification
Feature extraction techniques to use in cereal classification Ole Mathis Kruse, IMT 2111 2005 1 Problem Is it possible to discriminate between different species- or varieties of cereal grains - using image
Distance Measures for Gene Expression (and other high-dimensional) Data
Distance Measures for Gene Expression (and other high-dimensional) Data Utah State University Spring 04 STAT 5570: Statistical Bioinformatics Notes. References Chapter of Bioconductor Monograph (course
SVM Classification in -Arrays
SVM Classification in -Arrays SVM classification and validation of cancer tissue samples using microarray expression data Furey et al, 2000 Special Topics in Bioinformatics, SS10 A. Regl, 7055213 What
Image Processing. Image Features
Image Processing Image Features Preliminaries 2 What are Image Features? Anything. What they are used for? Some statements about image fragments (patches) recognition Search for similar patches matching
Dimension reduction : PCA and Clustering
Dimension reduction : PCA and Clustering By Hanne Jarmer Slides by Christopher Workman Center for Biological Sequence Analysis DTU The DNA Array Analysis Pipeline Array design Probe design Question Experimental
Multiclass Classifiers Based on Dimension Reduction
Multiclass Classifiers Based on Dimension Reduction with Generalized LDA Hyunsoo Kim Barry L Drake Haesun Park College of Computing, Georgia Institute of Technology, Atlanta, GA 30332, USA Abstract Linear
These slides follow closely the (English) course textbook Pattern Recognition and Machine Learning by Christopher Bishop
Machine Learning Algorithms (IFT6266 A7) Prof. Douglas Eck, Université de Montréal These slides follow closely the (English) course textbook Pattern Recognition and Machine Learning by Christopher Bishop
Grundlagen der Künstlichen Intelligenz
Grundlagen der Künstlichen Intelligenz Unsupervised learning Daniel Hennes 29.01.2018 (WS 2017/18) University Stuttgart - IPVS - Machine Learning & Robotics 1 Today Supervised learning Regression (linear
Overview Citation. ML Introduction. Overview Schedule. ML Intro Dataset. Introduction to Semi-Supervised Learning Review 10/4/2010
INFORMATICS SEMINAR SEPT. 27 & OCT. 4, 2010 Introduction to Semi-Supervised Learning Review 2 Overview Citation X. Zhu and A.B. Goldberg, Introduction to Semi- Supervised Learning, Morgan & Claypool Publishers,
CIS 520, Machine Learning, Fall 2015: Assignment 7 Due: Mon, Nov 16, :59pm, PDF to Canvas [100 points]
![CIS 520, Machine Learning, Fall 2015: Assignment 7 Due: Mon, Nov 16, :59pm, PDF to Canvas [100 points]](/thumbs/89/100746783.jpg "CIS 520, Machine Learning, Fall 2015: Assignment 7 Due: Mon, Nov 16, :59pm, PDF to Canvas [100 points]")
CIS 520, Machine Learning, Fall 2015: Assignment 7 Due: Mon, Nov 16, 2015. 11:59pm, PDF to Canvas [100 points] Instructions. Please write up your responses to the following problems clearly and concisely.
MSA220 - Statistical Learning for Big Data
MSA220 - Statistical Learning for Big Data Lecture 13 Rebecka Jörnsten Mathematical Sciences University of Gothenburg and Chalmers University of Technology Clustering Explorative analysis - finding groups
Stat 602X Exam 2 Spring 2011
Stat 60X Exam Spring 0 I have neither given nor received unauthorized assistance on this exam. Name Signed Date Name Printed . Below is a small p classification training set (for classes) displayed in
CPSC 340: Machine Learning and Data Mining. Outlier Detection Fall 2016
CPSC 340: Machine Learning and Data Mining Outlier Detection Fall 2016 Admin Assignment 1 solutions will be posted after class. Assignment 2 is out: Due next Friday, but start early! Calculus and linear
Generative and discriminative classification techniques
Generative and discriminative classification techniques Machine Learning and Category Representation 013-014 Jakob Verbeek, December 13+0, 013 Course website: http://lear.inrialpes.fr/~verbeek/mlcr.13.14
The Curse of Dimensionality
The Curse of Dimensionality ACAS 2002 p1/66 Curse of Dimensionality The basic idea of the curse of dimensionality is that high dimensional data is difficult to work with for several reasons: Adding more
Unsupervised learning in Vision
Chapter 7 Unsupervised learning in Vision The fields of Computer Vision and Machine Learning complement each other in a very natural way: the aim of the former is to extract useful information from visual
Clustering K-means. Machine Learning CSEP546 Carlos Guestrin University of Washington February 18, Carlos Guestrin
Clustering K-means Machine Learning CSEP546 Carlos Guestrin University of Washington February 18, 2014 Carlos Guestrin 2005-2014 1 Clustering images Set of Images [Goldberger et al.] Carlos Guestrin 2005-2014
STAT 1291: Data Science
STAT 1291: Data Science Lecture 18 - Statistical modeling II: Machine learning Sungkyu Jung Where are we? data visualization data wrangling professional ethics statistical foundation Statistical modeling:
Statistical Pattern Recognition
Statistical Pattern Recognition Features and Feature Selection Hamid R. Rabiee Jafar Muhammadi Spring 2014 http://ce.sharif.edu/courses/92-93/2/ce725-2/ Agenda Features and Patterns The Curse of Size and
RDRToolbox A package for nonlinear dimension reduction with Isomap and LLE.
RDRToolbox A package for nonlinear dimension reduction with Isomap and LLE. Christoph Bartenhagen October 30, 2017 Contents 1 Introduction 1 1.1 Loading the package......................................
Statistical Pattern Recognition
Statistical Pattern Recognition Features and Feature Selection Hamid R. Rabiee Jafar Muhammadi Spring 2013 http://ce.sharif.edu/courses/91-92/2/ce725-1/ Agenda Features and Patterns The Curse of Size and
Gene selection through Switched Neural Networks
Gene selection through Switched Neural Networks Marco Muselli Istituto di Elettronica e di Ingegneria dell Informazione e delle Telecomunicazioni Consiglio Nazionale delle Ricerche Email: Marco.Muselli@ieiit.cnr.it
The exam is closed book, closed notes except your one-page cheat sheet.
CS 189 Fall 2015 Introduction to Machine Learning Final Please do not turn over the page before you are instructed to do so. You have 2 hours and 50 minutes. Please write your initials on the top-right
Stat 342 Exam 3 Fall 2014
Stat 34 Exam 3 Fall 04 I have neither given nor received unauthorized assistance on this exam. Name Signed Date Name Printed There are questions on the following 6 pages. Do as many of them as you can
Lecture Topic Projects
Lecture Topic Projects 1 Intro, schedule, and logistics 2 Applications of visual analytics, basic tasks, data types 3 Introduction to D3, basic vis techniques for non-spatial data Project #1 out 4 Data
University of Florida CISE department Gator Engineering. Data Preprocessing. Dr. Sanjay Ranka
Data Preprocessing Dr. Sanjay Ranka Professor Computer and Information Science and Engineering University of Florida, Gainesville ranka@cise.ufl.edu Data Preprocessing What preprocessing step can or should
Kernel Methods and Visualization for Interval Data Mining
Kernel Methods and Visualization for Interval Data Mining Thanh-Nghi Do 1 and François Poulet 2 1 College of Information Technology, Can Tho University, 1 Ly Tu Trong Street, Can Tho, VietNam (e-mail:
Clustering and Visualisation of Data
Clustering and Visualisation of Data Hiroshi Shimodaira January-March 28 Cluster analysis aims to partition a data set into meaningful or useful groups, based on distances between data points. In some
A problem - too many features. TDA 231 Dimension Reduction: PCA. Features. Making new features
A problem - too many features TDA 1 Dimension Reduction: Aim: To build a classifier that can diagnose leukaemia using Gene expression data. Data: 7 healthy samples,11 leukaemia samples (N = 8). Each sample
Training-Free, Generic Object Detection Using Locally Adaptive Regression Kernels
Training-Free, Generic Object Detection Using Locally Adaptive Regression Kernels IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIENCE, VOL.32, NO.9, SEPTEMBER 2010 Hae Jong Seo, Student Member,
Title: Optimized multilayer perceptrons for molecular classification and diagnosis using genomic data
Supplementary material for Manuscript BIOINF-2005-1602 Title: Optimized multilayer perceptrons for molecular classification and diagnosis using genomic data Appendix A. Testing K-Nearest Neighbor and Support
A Comparative Study of Locality Preserving Projection and Principle Component Analysis on Classification Performance Using Logistic Regression
Journal of Data Analysis and Information Processing, 2016, 4, 55-63 Published Online May 2016 in SciRes. http://www.scirp.org/journal/jdaip http://dx.doi.org/10.4236/jdaip.2016.42005 A Comparative Study
Modelling Bivariate Distributions Using Kernel Density Estimation
Modelling Bivariate Distributions Using Kernel Density Estimation Alexander Bilock, Carl Jidling and Ylva Rydin Project in Computational Science 6 January 6 Department of information technology Abstract
Fisher vector image representation
Fisher vector image representation Jakob Verbeek January 13, 2012 Course website: http://lear.inrialpes.fr/~verbeek/mlcr.11.12.php Fisher vector representation Alternative to bag-of-words image representation
Discriminate Analysis
Discriminate Analysis Outline Introduction Linear Discriminant Analysis Examples 1 Introduction What is Discriminant Analysis? Statistical technique to classify objects into mutually exclusive and exhaustive
Bandwidth Selection for Kernel Density Estimation Using Total Variation with Fourier Domain Constraints
IEEE SIGNAL PROCESSING LETTERS 1 Bandwidth Selection for Kernel Density Estimation Using Total Variation with Fourier Domain Constraints Alexander Suhre, Orhan Arikan, Member, IEEE, and A. Enis Cetin,
Note Set 4: Finite Mixture Models and the EM Algorithm
Note Set 4: Finite Mixture Models and the EM Algorithm Padhraic Smyth, Department of Computer Science University of California, Irvine Finite Mixture Models A finite mixture model with K components, for
JMP Book Descriptions
JMP Book Descriptions The collection of JMP documentation is available in the JMP Help > Books menu. This document describes each title to help you decide which book to explore. Each book title is linked
What is clustering. Organizing data into clusters such that there is high intra- cluster similarity low inter- cluster similarity
Clustering What is clustering Organizing data into clusters such that there is high intra- cluster similarity low inter- cluster similarity Informally, finding natural groupings among objects. High dimensional
Use of Extreme Value Statistics in Modeling Biometric Systems
Use of Extreme Value Statistics in Modeling Biometric Systems Similarity Scores Two types of matching: Genuine sample Imposter sample Matching scores Enrolled sample 0.95 0.32 Probability Density Decision
Contents Machine Learning concepts 4 Learning Algorithm 4 Predictive Model (Model) 4 Model, Classification 4 Model, Regression 4 Representation
Contents Machine Learning concepts 4 Learning Algorithm 4 Predictive Model (Model) 4 Model, Classification 4 Model, Regression 4 Representation Learning 4 Supervised Learning 4 Unsupervised Learning 4
Statistical Pattern Recognition
Statistical Pattern Recognition Features and Feature Selection Hamid R. Rabiee Jafar Muhammadi Spring 2012 http://ce.sharif.edu/courses/90-91/2/ce725-1/ Agenda Features and Patterns The Curse of Size and
Linear and Non-linear Dimentionality Reduction Applied to Gene Expression Data of Cancer Tissue Samples
Linear and Non-linear Dimentionality Reduction Applied to Gene Expression Data of Cancer Tissue Samples Franck Olivier Ndjakou Njeunje Applied Mathematics, Statistics, and Scientific Computation University
Modelling and Visualization of High Dimensional Data. Sample Examination Paper
Duration not specified UNIVERSITY OF MANCHESTER SCHOOL OF COMPUTER SCIENCE Modelling and Visualization of High Dimensional Data Sample Examination Paper Examination date not specified Time: Examination
Non-linear dimension reduction
Sta306b May 23, 2011 Dimension Reduction: 1 Non-linear dimension reduction ISOMAP: Tenenbaum, de Silva & Langford (2000) Local linear embedding: Roweis & Saul (2000) Local MDS: Chen (2006) all three methods
CLASSIFICATION AND CHANGE DETECTION
IMAGE ANALYSIS, CLASSIFICATION AND CHANGE DETECTION IN REMOTE SENSING With Algorithms for ENVI/IDL and Python THIRD EDITION Morton J. Canty CRC Press Taylor & Francis Group Boca Raton London NewYork CRC
CPSC 340: Machine Learning and Data Mining. Probabilistic Classification Fall 2017
CPSC 340: Machine Learning and Data Mining Probabilistic Classification Fall 2017 Admin Assignment 0 is due tonight: you should be almost done. 1 late day to hand it in Monday, 2 late days for Wednesday.
University of Cambridge Engineering Part IIB Paper 4F10: Statistical Pattern Processing Handout 11: Non-Parametric Techniques.
. Non-Parameteric Techniques University of Cambridge Engineering Part IIB Paper 4F: Statistical Pattern Processing Handout : Non-Parametric Techniques Mark Gales mjfg@eng.cam.ac.uk Michaelmas 23 Introduction
Machine Learning : Clustering, Self-Organizing Maps
Machine Learning Clustering, Self-Organizing Maps 12/12/2013 Machine Learning : Clustering, Self-Organizing Maps Clustering The task: partition a set of objects into meaningful subsets (clusters). The
Data mining with Support Vector Machine
Data mining with Support Vector Machine Ms. Arti Patle IES, IPS Academy Indore (M.P.) artipatle@gmail.com Mr. Deepak Singh Chouhan IES, IPS Academy Indore (M.P.) deepak.schouhan@yahoo.com Abstract: Machine
Statistics of Natural Image Categories
Statistics of Natural Image Categories Authors: Antonio Torralba and Aude Oliva Presented by: Sebastian Scherer Experiment Please estimate the average depth from the camera viewpoint to all locations(pixels)
1 Introduction Motivation and Aims Functional Imaging Computational Neuroanatomy... 12
Contents 1 Introduction 10 1.1 Motivation and Aims....... 10 1.1.1 Functional Imaging.... 10 1.1.2 Computational Neuroanatomy... 12 1.2 Overview of Chapters... 14 2 Rigid Body Registration 18 2.1 Introduction.....
University of Cambridge Engineering Part IIB Paper 4F10: Statistical Pattern Processing Handout 11: Non-Parametric Techniques
University of Cambridge Engineering Part IIB Paper 4F10: Statistical Pattern Processing Handout 11: Non-Parametric Techniques Mark Gales mjfg@eng.cam.ac.uk Michaelmas 2011 11. Non-Parameteric Techniques
Image Analysis, Classification and Change Detection in Remote Sensing
Image Analysis, Classification and Change Detection in Remote Sensing WITH ALGORITHMS FOR ENVI/IDL Morton J. Canty Taylor &. Francis Taylor & Francis Group Boca Raton London New York CRC is an imprint
Recap: Gaussian (or Normal) Distribution. Recap: Minimizing the Expected Loss. Topics of This Lecture. Recap: Maximum Likelihood Approach
Truth Course Outline Machine Learning Lecture 3 Fundamentals (2 weeks) Bayes Decision Theory Probability Density Estimation Probability Density Estimation II 2.04.205 Discriminative Approaches (5 weeks)
MATLAB Routines for Kernel Density Estimation and the Graphical Representation of Archaeological Data
Christian C. Beardah Mike J. Baxter MATLAB Routines for Kernel Density Estimation and the Graphical Representation of Archaeological Data 1 Introduction Histograms are widely used for data presentation