Experimental Evaluation of Latent Variable Models. for Dimensionality Reduction

Save this PDF as:

Size: px
Start display at page:

Download "Experimental Evaluation of Latent Variable Models. for Dimensionality Reduction"

Transcription

1 Experimental Evaluation of Latent Variable Models for Dimensionality Reduction Miguel Á. Carreira-Perpiñán and Steve Renals a Dept. of Computer Science, University of Sheffield th IEEE Signal Processing Society Workshop on Neural Networks for Signal Processing (NNSP8) Aug. Sep., 8, Cambridge, UK a This work has been supported by a scholarship from the Spanish Ministry of Education and Science, by a ESPRIT Long Term Research Project SPRACH (77) and by an award from the Nuffield Foundation.

2 Electropalatography (EPG) A plastic pseudopalate fitted to a person s mouth detects the presence or absence of contact between the tongue and the palate in 6 different locations during an utterance (sampled at Hz). Result: sequence of 6-dimensional binary EPG frames. Data reduction necessary, traditionally via fixed linear indices. ACCOR-II database: synchronised data (EPG, acoustic, etc.) for different utterances and speakers. The mapping phoneme-to-epg is not one-to-one, e.g. / or /. Miguel Á. Carreira-Perpiñán and Steve Renals Experimental Evaluation of Latent Variable Models for Dimensionality Reduction

3 Electropalatography (cont.) wires to PC lips palate teeth teeth wires to PC teeth teeth lips velum palate electrodes velum electrodes Pseudopalate and representative EPGs for the typical stable phase of different phonemes. Miguel Á. Carreira-Perpiñán and Steve Renals Experimental Evaluation of Latent Variable Models for Dimensionality Reduction

4 NNSP8, AUG. SEP., 8, CARIDGE, UK - The Reading pseudopalate. Sfrag replacements EXPERIMENTAL EVALUATION OF LATENT VARIABLE MODELS FOR DIMENSIONALITY REDUCTION

5 Latent variable models Prior p(x) Induced p(t Θ) t x x f f(x; Θ) t Manifold M t t x Latent space of dimension L = Data space of dimension D = Marginalisation in latent space: p(t) = p(t x)p(x) dx. Maximum likelihood parameter estimation: l(θ) = N n= log p(t n Θ). Inverse mapping given by informative point (mean, mode) of posterior: p(x t) = p(t x)p(x) p(t). Miguel Á. Carreira-Perpiñán and Steve Renals Experimental Evaluation of Latent Variable Models for Dimensionality Reduction

6 Examples of latent variable models Factor analysis: prior is normal N (, I), mapping is linear, noise model is normal with diagonal covariance matrix. : like but noise model has isotropic covariance. : prior is uniform over discrete latent grid, mapping is a generalised linear model, noise model is normal with isotropic covariance matrix. Mixtures of factor analysers (one mean parameter and one factor per analyser, noise model covariance matrix common to all analysers). We also tried mixtures of multivariate Bernoulli distributions (not really a latent variable model). All these models can be trained via an EM algorithm. Miguel Á. Carreira-Perpiñán and Steve Renals Experimental Evaluation of Latent Variable Models for Dimensionality Reduction

7 Factors / prototypes (speaker RK) λ λ λ λ λ 5 λ 6 λ 7 λ 8 λ λ λ λ λ λ 5 λ 6 λ 7 λ 8 λ λ λ λ λ λ 5 λ 6 λ 7 λ 8 λ λ λ λ λ 5 λ 6 λ 7 λ 8 λ M Λ µ π =. Λ µ π =. Λ µ π =. Λ µ π =.7 p π =.5 p π =. p π =.7 p π =. p 5 π 5 =.5 p 6 π 6 =. p 7 π 7 =. p 8 π 8 =.5 p π =. Miguel Á. Carreira-Perpiñán and Steve Renals Experimental Evaluation of Latent Variable Models for Dimensionality Reduction 5

8 and reconstruction error (speaker RK) Training set Test set x 5 5 x M M Squared reconstruction error M M Miguel Á. Carreira-Perpiñán and Steve Renals Experimental Evaluation of Latent Variable Models for Dimensionality Reduction 6

9 ! " #,. # ) *,+ - %, Two-dimensional representation (speaker RK) Factor analysis frag replacements Factor Factor Factor Factor Trajectory in latent space of the highlighted utterance fragment I prefer Kant to Hobbes for a good bedtime book ( #(! " $&% /. ). Miguel Á. Carreira-Perpiñán and Steve Renals Experimental Evaluation of Latent Variable Models for Dimensionality Reduction 7

10 Conclusions Adaptive methods outperform fixed data reduction indices. and performed similarly in terms of likelihood. Mixtures of factor analysers and multivariate Bernoulli distributions did not perform well. Two-dimensional outperformed all other methods in terms of likelihood and error reconstruction and reveals nonlinear structure in the data. This suggests a low intrinsic dimensionality for the EPG data. Additional results available via the web at miguel/research/epg.html Miguel Á. Carreira-Perpiñán and Steve Renals Experimental Evaluation of Latent Variable Models for Dimensionality Reduction 8

11 Factors / prototypes (speaker HD) λ λ λ λ λ 5 λ 6 λ 7 λ 8 λ λ λ λ λ λ 5 λ 6 λ 7 λ 8 λ λ λ λ λ λ 5 λ 6 λ 7 λ 8 λ λ λ λ λ 5 λ 6 λ 7 λ 8 λ M Λ µ π =. Λ µ π =.6 Λ µ π =. Λ µ π =.6 p π =. p π =. p π =. p π =. p 5 π 5 =. p 6 π 6 =.7 p 7 π 7 =. p 8 π 8 =.7 p π =. Miguel Á. Carreira-Perpiñán and Steve Renals Experimental Evaluation of Latent Variable Models for Dimensionality Reduction

12 and reconstruction error (speaker HD) Training set Test set Squared reconstruction error x M M 6 6 x M M Miguel Á. Carreira-Perpiñán and Steve Renals Experimental Evaluation of Latent Variable Models for Dimensionality Reduction

13 ! " #,. # ) *,+ - %, Two-dimensional representation (speaker HD) Factor analysis 7 frag replacements Factor Factor Factor Factor Trajectory in latent space of the highlighted utterance fragment I prefer Kant to Hobbes for a good bedtime book ( #(! " $&% /. ). Miguel Á. Carreira-Perpiñán and Steve Renals Experimental Evaluation of Latent Variable Models for Dimensionality Reduction

14 !!! # ) *,+ - %,!! " #,. Discontinuities in latent space (speaker RK) Factor analysis.5.8 Factor Frame 5 Frame Factor Frame 7 Selected subsequence of the utterance fragment I prefer Kant to Hobbes for a good bedtime book ( #! " $ % /. ). The abrupt transition from to (frames 5 6) produces a discontinuity in latent space. Miguel Á. Carreira-Perpiñán and Steve Renals Experimental Evaluation of Latent Variable Models for Dimensionality Reduction

15 factors before and after varimax rotation (speaker HD) BEFORE λ λ λ λ λ 5 λ 6 λ 7 λ 8 λ AFTER λ λ λ λ λ 5 λ 6 λ 7 λ 8 λ Both sets of components span the same linear subspace, but the varimax-rotated one is more easily interpretable. Miguel Á. Carreira-Perpiñán and Steve Renals Experimental Evaluation of Latent Variable Models for Dimensionality Reduction

Experimental Evaluation of Latent Variable Models for Dimensionality Reduction

Experimental Evaluation of Latent Variable Models for Dimensionality Reduction In: Proc. of the 18 IEEE Signal Processing Society Workshop on Neural Networks for Signal Processing (NNSP8), pp.5-17, Cambridge, UK. URL: http://www.dcs.shef.ac.uk/ miguel/papers/nnsp8.html Experimental

More information

Experimental Evaluation of Latent Variable Models for Dimensionality Reduction

Experimental Evaluation of Latent Variable Models for Dimensionality Reduction Experimental Evaluation of Latent Variable Models for Dimensionality Reduction Miguel A. Carreira-Perpiiian Steve Renals Dept. of Computer Science, University of Sheffield, Sheffield S1 4DP, UK {M.Carreira,S.Renals}@dcs.shef.ac.uk

More information

Speech Recognition Lecture 8: Acoustic Models. Eugene Weinstein Google, NYU Courant Institute Slide Credit: Mehryar Mohri

Speech Recognition Lecture 8: Acoustic Models. Eugene Weinstein Google, NYU Courant Institute Slide Credit: Mehryar Mohri Speech Recognition Lecture 8: Acoustic Models. Eugene Weinstein Google, NYU Courant Institute eugenew@cs.nyu.edu Slide Credit: Mehryar Mohri Speech Recognition Components Acoustic and pronunciation model:

More information

The Laplacian Eigenmaps Latent Variable Model

The Laplacian Eigenmaps Latent Variable Model The Laplacian Eigenmaps Latent Variable Model with applications to articulated pose tracking Miguel Á. Carreira-Perpiñán EECS, UC Merced http://faculty.ucmerced.edu/mcarreira-perpinan Articulated pose

More information

Clustering Lecture 5: Mixture Model

Clustering Lecture 5: Mixture Model Clustering Lecture 5: Mixture Model Jing Gao SUNY Buffalo 1 Outline Basics Motivation, definition, evaluation Methods Partitional Hierarchical Density-based Mixture model Spectral methods Advanced topics

More information

Monocular Human Motion Capture with a Mixture of Regressors. Ankur Agarwal and Bill Triggs GRAVIR-INRIA-CNRS, Grenoble, France

Monocular Human Motion Capture with a Mixture of Regressors. Ankur Agarwal and Bill Triggs GRAVIR-INRIA-CNRS, Grenoble, France Monocular Human Motion Capture with a Mixture of Regressors Ankur Agarwal and Bill Triggs GRAVIR-INRIA-CNRS, Grenoble, France IEEE Workshop on Vision for Human-Computer Interaction, 21 June 2005 Visual

More information

Trajectory Inverse Kinematics By Conditional Density Models

Trajectory Inverse Kinematics By Conditional Density Models Trajectory Inverse Kinematics By Conditional Density Models Chao Qin and Miguel Á. Carreira-Perpiñán EECS, School of Engineering, UC Merced ICRA 08, Pasadena 1 Introduction Robot arm inverse kinematics

More information

Note Set 4: Finite Mixture Models and the EM Algorithm

Note Set 4: Finite Mixture Models and the EM Algorithm Note Set 4: Finite Mixture Models and the EM Algorithm Padhraic Smyth, Department of Computer Science University of California, Irvine Finite Mixture Models A finite mixture model with K components, for

More information

Clustering K-means. Machine Learning CSEP546 Carlos Guestrin University of Washington February 18, Carlos Guestrin

Clustering K-means. Machine Learning CSEP546 Carlos Guestrin University of Washington February 18, Carlos Guestrin Clustering K-means Machine Learning CSEP546 Carlos Guestrin University of Washington February 18, 2014 Carlos Guestrin 2005-2014 1 Clustering images Set of Images [Goldberger et al.] Carlos Guestrin 2005-2014

More information

Problem 1 (20 pt) Answer the following questions, and provide an explanation for each question.

Problem 1 (20 pt) Answer the following questions, and provide an explanation for each question. Problem 1 Answer the following questions, and provide an explanation for each question. (5 pt) Can linear regression work when all X values are the same? When all Y values are the same? (5 pt) Can linear

More information

The K-modes and Laplacian K-modes algorithms for clustering

The K-modes and Laplacian K-modes algorithms for clustering The K-modes and Laplacian K-modes algorithms for clustering Miguel Á. Carreira-Perpiñán Electrical Engineering and Computer Science University of California, Merced http://faculty.ucmerced.edu/mcarreira-perpinan

More information

Pattern Recognition. Kjell Elenius. Speech, Music and Hearing KTH. March 29, 2007 Speech recognition

Pattern Recognition. Kjell Elenius. Speech, Music and Hearing KTH. March 29, 2007 Speech recognition Pattern Recognition Kjell Elenius Speech, Music and Hearing KTH March 29, 2007 Speech recognition 2007 1 Ch 4. Pattern Recognition 1(3) Bayes Decision Theory Minimum-Error-Rate Decision Rules Discriminant

More information

Time Series Analysis by State Space Methods

Time Series Analysis by State Space Methods Time Series Analysis by State Space Methods Second Edition J. Durbin London School of Economics and Political Science and University College London S. J. Koopman Vrije Universiteit Amsterdam OXFORD UNIVERSITY

More information

Machine Learning. B. Unsupervised Learning B.1 Cluster Analysis. Lars Schmidt-Thieme, Nicolas Schilling

Machine Learning. B. Unsupervised Learning B.1 Cluster Analysis. Lars Schmidt-Thieme, Nicolas Schilling Machine Learning B. Unsupervised Learning B.1 Cluster Analysis Lars Schmidt-Thieme, Nicolas Schilling Information Systems and Machine Learning Lab (ISMLL) Institute for Computer Science University of Hildesheim,

More information

22 October, 2012 MVA ENS Cachan. Lecture 5: Introduction to generative models Iasonas Kokkinos

22 October, 2012 MVA ENS Cachan. Lecture 5: Introduction to generative models Iasonas Kokkinos Machine Learning for Computer Vision 1 22 October, 2012 MVA ENS Cachan Lecture 5: Introduction to generative models Iasonas Kokkinos Iasonas.kokkinos@ecp.fr Center for Visual Computing Ecole Centrale Paris

More information

Locally Linear Landmarks for large-scale manifold learning

Locally Linear Landmarks for large-scale manifold learning Locally Linear Landmarks for large-scale manifold learning Max Vladymyrov and Miguel Á. Carreira-Perpiñán Electrical Engineering and Computer Science University of California, Merced http://eecs.ucmerced.edu

More information

Machine Learning A W 1sst KU. b) [1 P] Give an example for a probability distributions P (A, B, C) that disproves

Machine Learning A W 1sst KU. b) [1 P] Give an example for a probability distributions P (A, B, C) that disproves Machine Learning A 708.064 11W 1sst KU Exercises Problems marked with * are optional. 1 Conditional Independence I [2 P] a) [1 P] Give an example for a probability distribution P (A, B, C) that disproves

More information

Semi-Supervised Construction of General Visualization Hierarchies

Semi-Supervised Construction of General Visualization Hierarchies Semi-Supervised Construction of General Visualization Hierarchies Peter Tiňo Yi Sun Ian Nabney Aston University, Aston Triangle, Birmingham, B4 7ET United Kingdom Abstract We have recently developed a

More information

Self-organizing mixture models

Self-organizing mixture models Self-organizing mixture models Jakob Verbeek, Nikos Vlassis, Ben Krose To cite this version: Jakob Verbeek, Nikos Vlassis, Ben Krose. Self-organizing mixture models. Neurocomputing / EEG Neurocomputing,

More information

Generative and discriminative classification techniques

Generative and discriminative classification techniques Generative and discriminative classification techniques Machine Learning and Category Representation 2014-2015 Jakob Verbeek, November 28, 2014 Course website: http://lear.inrialpes.fr/~verbeek/mlcr.14.15

More information

CSC 411: Lecture 14: Principal Components Analysis & Autoencoders

CSC 411: Lecture 14: Principal Components Analysis & Autoencoders CSC 411: Lecture 14: Principal Components Analysis & Autoencoders Raquel Urtasun & Rich Zemel University of Toronto Nov 4, 2015 Urtasun & Zemel (UofT) CSC 411: 14-PCA & Autoencoders Nov 4, 2015 1 / 18

More information

Learning a Manifold as an Atlas Supplementary Material

Learning a Manifold as an Atlas Supplementary Material Learning a Manifold as an Atlas Supplementary Material Nikolaos Pitelis Chris Russell School of EECS, Queen Mary, University of London [nikolaos.pitelis,chrisr,lourdes]@eecs.qmul.ac.uk Lourdes Agapito

More information

Constrained Hidden Markov Models

Constrained Hidden Markov Models Constrained Hidden Markov Models Sam Roweis roweis@gatsby.ucl.ac.uk Gatsby Unit, University College London Abstract By thinking of each state in a hidden Markov model as corresponding to some spatial region

More information

Clustering algorithms

Clustering algorithms Clustering algorithms Machine Learning Hamid Beigy Sharif University of Technology Fall 1393 Hamid Beigy (Sharif University of Technology) Clustering algorithms Fall 1393 1 / 22 Table of contents 1 Supervised

More information

CSC 411: Lecture 14: Principal Components Analysis & Autoencoders

CSC 411: Lecture 14: Principal Components Analysis & Autoencoders CSC 411: Lecture 14: Principal Components Analysis & Autoencoders Richard Zemel, Raquel Urtasun and Sanja Fidler University of Toronto Zemel, Urtasun, Fidler (UofT) CSC 411: 14-PCA & Autoencoders 1 / 18

More information

Gaussian Process Latent Variable Models for Visualisation of High Dimensional Data

Gaussian Process Latent Variable Models for Visualisation of High Dimensional Data Gaussian Process Latent Variable Models for Visualisation of High Dimensional Data Neil D. Lawrence Department of Computer Science University of Sheffield Regent Court, 211 Portobello Street, Sheffield,

More information

Robust cartogram visualization of outliers in manifold learning

Robust cartogram visualization of outliers in manifold learning Robust cartogram visualization of outliers in manifold learning Alessandra Tosi 1 and Alfredo Vellido 1 1- Llenguatges i Sistemes Informàtics, Universitat Politècnica de Catalunya, Edifici Omega, Campus

More information

10-701/15-781, Fall 2006, Final

10-701/15-781, Fall 2006, Final -7/-78, Fall 6, Final Dec, :pm-8:pm There are 9 questions in this exam ( pages including this cover sheet). If you need more room to work out your answer to a question, use the back of the page and clearly

More information

Variational Autoencoders. Sargur N. Srihari

Variational Autoencoders. Sargur N. Srihari Variational Autoencoders Sargur N. srihari@cedar.buffalo.edu Topics 1. Generative Model 2. Standard Autoencoder 3. Variational autoencoders (VAE) 2 Generative Model A variational autoencoder (VAE) is a

More information

Deep Generative Models Variational Autoencoders

Deep Generative Models Variational Autoencoders Deep Generative Models Variational Autoencoders Sudeshna Sarkar 5 April 2017 Generative Nets Generative models that represent probability distributions over multiple variables in some way. Directed Generative

More information

Mixture Models and the EM Algorithm

Mixture Models and the EM Algorithm Mixture Models and the EM Algorithm Padhraic Smyth, Department of Computer Science University of California, Irvine c 2017 1 Finite Mixture Models Say we have a data set D = {x 1,..., x N } where x i is

More information

Hierarchical Gaussian Process Latent Variable Models

Hierarchical Gaussian Process Latent Variable Models Neil D. Lawrence neill@cs.man.ac.uk School of Computer Science, University of Manchester, Kilburn Building, Oxford Road, Manchester, M13 9PL, U.K. Andrew J. Moore A.Moore@dcs.shef.ac.uk Dept of Computer

More information

Audio-Visual Speech Activity Detection

Audio-Visual Speech Activity Detection Institut für Technische Informatik und Kommunikationsnetze Semester Thesis at the Department of Information Technology and Electrical Engineering Audio-Visual Speech Activity Detection Salome Mannale Advisors:

More information

( ) =cov X Y = W PRINCIPAL COMPONENT ANALYSIS. Eigenvectors of the covariance matrix are the principal components

( ) =cov X Y = W PRINCIPAL COMPONENT ANALYSIS. Eigenvectors of the covariance matrix are the principal components Review Lecture 14 ! PRINCIPAL COMPONENT ANALYSIS Eigenvectors of the covariance matrix are the principal components 1. =cov X Top K principal components are the eigenvectors with K largest eigenvalues

More information

Discriminative training and Feature combination

Discriminative training and Feature combination Discriminative training and Feature combination Steve Renals Automatic Speech Recognition ASR Lecture 13 16 March 2009 Steve Renals Discriminative training and Feature combination 1 Overview Hot topics

More information

Spatial Outlier Detection

Spatial Outlier Detection Spatial Outlier Detection Chang-Tien Lu Department of Computer Science Northern Virginia Center Virginia Tech Joint work with Dechang Chen, Yufeng Kou, Jiang Zhao 1 Spatial Outlier A spatial data point

More information

A New Manifold Representation for Visual Speech Recognition

A New Manifold Representation for Visual Speech Recognition A New Manifold Representation for Visual Speech Recognition Dahai Yu, Ovidiu Ghita, Alistair Sutherland, Paul F. Whelan School of Computing & Electronic Engineering, Vision Systems Group Dublin City University,

More information

Segmentation: Clustering, Graph Cut and EM

Segmentation: Clustering, Graph Cut and EM Segmentation: Clustering, Graph Cut and EM Ying Wu Electrical Engineering and Computer Science Northwestern University, Evanston, IL 60208 yingwu@northwestern.edu http://www.eecs.northwestern.edu/~yingwu

More information

An Efficient Model Selection for Gaussian Mixture Model in a Bayesian Framework

An Efficient Model Selection for Gaussian Mixture Model in a Bayesian Framework IEEE SIGNAL PROCESSING LETTERS, VOL. XX, NO. XX, XXX 23 An Efficient Model Selection for Gaussian Mixture Model in a Bayesian Framework Ji Won Yoon arxiv:37.99v [cs.lg] 3 Jul 23 Abstract In order to cluster

More information

Hybrid Quasi-Monte Carlo Method for the Simulation of State Space Models

Hybrid Quasi-Monte Carlo Method for the Simulation of State Space Models The Tenth International Symposium on Operations Research and Its Applications (ISORA 211) Dunhuang, China, August 28 31, 211 Copyright 211 ORSC & APORC, pp. 83 88 Hybrid Quasi-Monte Carlo Method for the

More information

Bo#leneck Features from SNR- Adap9ve Denoising Deep Classifier for Speaker Iden9fica9on

Bo#leneck Features from SNR- Adap9ve Denoising Deep Classifier for Speaker Iden9fica9on Bo#leneck Features from SNR- Adap9ve Denoising Deep Classifier for Speaker Iden9fica9on TAN Zhili & MAK Man-Wai APSIPA 2015 Department of Electronic and Informa2on Engineering The Hong Kong Polytechnic

More information

Statistical Techniques in Robotics (16-831, F12) Lecture#05 (Wednesday, September 12) Mapping

Statistical Techniques in Robotics (16-831, F12) Lecture#05 (Wednesday, September 12) Mapping Statistical Techniques in Robotics (16-831, F12) Lecture#05 (Wednesday, September 12) Mapping Lecturer: Alex Styler (in for Drew Bagnell) Scribe: Victor Hwang 1 1 Occupancy Mapping When solving the localization

More information

SMEM Algorithm for Mixture Models

SMEM Algorithm for Mixture Models LETTER Communicated by Christopher Bishop SMEM Algorithm for Mixture Models Naonori Ueda Ryohei Nakano NTT Communication Science Laboratories, Hikaridai, Seika-cho, Soraku-gun, Kyoto 619-0237 Japan Zoubin

More information

Machine Learning. B. Unsupervised Learning B.1 Cluster Analysis. Lars Schmidt-Thieme

Machine Learning. B. Unsupervised Learning B.1 Cluster Analysis. Lars Schmidt-Thieme Machine Learning B. Unsupervised Learning B.1 Cluster Analysis Lars Schmidt-Thieme Information Systems and Machine Learning Lab (ISMLL) Institute for Computer Science University of Hildesheim, Germany

More information

CIS 520, Machine Learning, Fall 2015: Assignment 7 Due: Mon, Nov 16, :59pm, PDF to Canvas [100 points]

CIS 520, Machine Learning, Fall 2015: Assignment 7 Due: Mon, Nov 16, :59pm, PDF to Canvas [100 points] CIS 520, Machine Learning, Fall 2015: Assignment 7 Due: Mon, Nov 16, 2015. 11:59pm, PDF to Canvas [100 points] Instructions. Please write up your responses to the following problems clearly and concisely.

More information

Tight Clusters and Smooth Manifolds with the Harmonic Topographic Map.

Tight Clusters and Smooth Manifolds with the Harmonic Topographic Map. Proceedings of the th WSEAS Int. Conf. on SIMULATION, MODELING AND OPTIMIZATION, Corfu, Greece, August -9, (pp8-) Tight Clusters and Smooth Manifolds with the Harmonic Topographic Map. MARIAN PEÑA AND

More information

ECE521: Week 11, Lecture March 2017: HMM learning/inference. With thanks to Russ Salakhutdinov

ECE521: Week 11, Lecture March 2017: HMM learning/inference. With thanks to Russ Salakhutdinov ECE521: Week 11, Lecture 20 27 March 2017: HMM learning/inference With thanks to Russ Salakhutdinov Examples of other perspectives Murphy 17.4 End of Russell & Norvig 15.2 (Artificial Intelligence: A Modern

More information

Grundlagen der Künstlichen Intelligenz

Grundlagen der Künstlichen Intelligenz Grundlagen der Künstlichen Intelligenz Unsupervised learning Daniel Hennes 29.01.2018 (WS 2017/18) University Stuttgart - IPVS - Machine Learning & Robotics 1 Today Supervised learning Regression (linear

More information

Pictorial Structures for Object Recognition

Pictorial Structures for Object Recognition Pictorial Structures for Object Recognition Felzenszwalb and Huttenlocher Presented by Stephen Krotosky Pictorial Structures Introduced by Fischler and Elschlager in 1973 Objects are modeled by a collection

More information

METRIC PLANE RECTIFICATION USING SYMMETRIC VANISHING POINTS

METRIC PLANE RECTIFICATION USING SYMMETRIC VANISHING POINTS METRIC PLANE RECTIFICATION USING SYMMETRIC VANISHING POINTS M. Lefler, H. Hel-Or Dept. of CS, University of Haifa, Israel Y. Hel-Or School of CS, IDC, Herzliya, Israel ABSTRACT Video analysis often requires

More information

Split Merge Incremental LEarning (SMILE) of Mixture Models

Split Merge Incremental LEarning (SMILE) of Mixture Models Split Merge Incremental LEarning (SMILE of Mixture Models Konstantinos Blekas and Isaac E. Lagaris Department of Computer Science, University of Ioannina, 45 Ioannina, Greece {kblekas,lagaris}@cs.uoi.gr

More information

Random projection for non-gaussian mixture models

Random projection for non-gaussian mixture models Random projection for non-gaussian mixture models Győző Gidófalvi Department of Computer Science and Engineering University of California, San Diego La Jolla, CA 92037 gyozo@cs.ucsd.edu Abstract Recently,

More information

Clustering K-means. Machine Learning CSEP546 Carlos Guestrin University of Washington February 18, Carlos Guestrin

Clustering K-means. Machine Learning CSEP546 Carlos Guestrin University of Washington February 18, Carlos Guestrin Clustering K-means Machine Learning CSEP546 Carlos Guestrin University of Washington February 18, 2014 Carlos Guestrin 2005-2014 1 Clustering images Set of Images [Goldberger et al.] Carlos Guestrin 2005-2014

More information

Nonlinear Image Interpolation using Manifold Learning

Nonlinear Image Interpolation using Manifold Learning Nonlinear Image Interpolation using Manifold Learning Christoph Bregler Computer Science Division University of California Berkeley, CA 94720 bregler@cs.berkeley.edu Stephen M. Omohundro'" Int. Computer

More information

Assignment 2. Unsupervised & Probabilistic Learning. Maneesh Sahani Due: Monday Nov 5, 2018

Assignment 2. Unsupervised & Probabilistic Learning. Maneesh Sahani Due: Monday Nov 5, 2018 Assignment 2 Unsupervised & Probabilistic Learning Maneesh Sahani Due: Monday Nov 5, 2018 Note: Assignments are due at 11:00 AM (the start of lecture) on the date above. he usual College late assignments

More information

Automatic Singular Spectrum Analysis for Time-Series Decomposition

Automatic Singular Spectrum Analysis for Time-Series Decomposition Automatic Singular Spectrum Analysis for Time-Series Decomposition A.M. Álvarez-Meza and C.D. Acosta-Medina and G. Castellanos-Domínguez Universidad Nacional de Colombia, Signal Processing and Recognition

More information

Lec 08 Feature Aggregation II: Fisher Vector, Super Vector and AKULA

Lec 08 Feature Aggregation II: Fisher Vector, Super Vector and AKULA Image Analysis & Retrieval CS/EE 5590 Special Topics (Class Ids: 44873, 44874) Fall 2016, M/W 4-5:15pm@Bloch 0012 Lec 08 Feature Aggregation II: Fisher Vector, Super Vector and AKULA Zhu Li Dept of CSEE,

More information

Probabilistic Facial Feature Extraction Using Joint Distribution of Location and Texture Information

Probabilistic Facial Feature Extraction Using Joint Distribution of Location and Texture Information Probabilistic Facial Feature Extraction Using Joint Distribution of Location and Texture Information Mustafa Berkay Yilmaz, Hakan Erdogan, Mustafa Unel Sabanci University, Faculty of Engineering and Natural

More information

Warped Mixture Models

Warped Mixture Models Warped Mixture Models Tomoharu Iwata, David Duvenaud, Zoubin Ghahramani Cambridge University Computational and Biological Learning Lab March 11, 2013 OUTLINE Motivation Gaussian Process Latent Variable

More information

Experimental Analysis of GTM

Experimental Analysis of GTM Experimental Analysis of GTM Elias Pampalk In the past years many different data mining techniques have been developed. The goal of the seminar Kosice-Vienna is to compare some of them to determine which

More information

Last week. Multi-Frame Structure from Motion: Multi-View Stereo. Unknown camera viewpoints

Last week. Multi-Frame Structure from Motion: Multi-View Stereo. Unknown camera viewpoints Last week Multi-Frame Structure from Motion: Multi-View Stereo Unknown camera viewpoints Last week PCA Today Recognition Today Recognition Recognition problems What is it? Object detection Who is it? Recognizing

More information

Part I. Hierarchical clustering. Hierarchical Clustering. Hierarchical clustering. Produces a set of nested clusters organized as a

Part I. Hierarchical clustering. Hierarchical Clustering. Hierarchical clustering. Produces a set of nested clusters organized as a Week 9 Based in part on slides from textbook, slides of Susan Holmes Part I December 2, 2012 Hierarchical Clustering 1 / 1 Produces a set of nested clusters organized as a Hierarchical hierarchical clustering

More information

Deep Mixtures of Factor Analysers

Deep Mixtures of Factor Analysers Yichuan Tang tang@cs.toronto.edu Ruslan Salakhutdinov rsalakhu@cs.toronto.edu Geoffrey Hinton hinton@cs.toronto.edu Department of Computer Science, University of Toronto, Toronto, Ontario, CANADA Abstract

More information

ESTIMATING HEAD POSE WITH AN RGBD SENSOR: A COMPARISON OF APPEARANCE-BASED AND POSE-BASED LOCAL SUBSPACE METHODS

ESTIMATING HEAD POSE WITH AN RGBD SENSOR: A COMPARISON OF APPEARANCE-BASED AND POSE-BASED LOCAL SUBSPACE METHODS ESTIMATING HEAD POSE WITH AN RGBD SENSOR: A COMPARISON OF APPEARANCE-BASED AND POSE-BASED LOCAL SUBSPACE METHODS Donghun Kim, Johnny Park, and Avinash C. Kak Robot Vision Lab, School of Electrical and

More information

CS839: Probabilistic Graphical Models. Lecture 10: Learning with Partially Observed Data. Theo Rekatsinas

CS839: Probabilistic Graphical Models. Lecture 10: Learning with Partially Observed Data. Theo Rekatsinas CS839: Probabilistic Graphical Models Lecture 10: Learning with Partially Observed Data Theo Rekatsinas 1 Partially Observed GMs Speech recognition 2 Partially Observed GMs Evolution 3 Partially Observed

More information

Unsupervised Learning

Unsupervised Learning Unsupervised Learning Learning without Class Labels (or correct outputs) Density Estimation Learn P(X) given training data for X Clustering Partition data into clusters Dimensionality Reduction Discover

More information

FACE RECOGNITION USING INDEPENDENT COMPONENT

FACE RECOGNITION USING INDEPENDENT COMPONENT Chapter 5 FACE RECOGNITION USING INDEPENDENT COMPONENT ANALYSIS OF GABORJET (GABORJET-ICA) 5.1 INTRODUCTION PCA is probably the most widely used subspace projection technique for face recognition. A major

More information

08 An Introduction to Dense Continuous Robotic Mapping

08 An Introduction to Dense Continuous Robotic Mapping NAVARCH/EECS 568, ROB 530 - Winter 2018 08 An Introduction to Dense Continuous Robotic Mapping Maani Ghaffari March 14, 2018 Previously: Occupancy Grid Maps Pose SLAM graph and its associated dense occupancy

More information

Multi-pose lipreading and audio-visual speech recognition

Multi-pose lipreading and audio-visual speech recognition RESEARCH Open Access Multi-pose lipreading and audio-visual speech recognition Virginia Estellers * and Jean-Philippe Thiran Abstract In this article, we study the adaptation of visual and audio-visual

More information

arxiv: v1 [cond-mat.dis-nn] 30 Dec 2018

arxiv: v1 [cond-mat.dis-nn] 30 Dec 2018 A General Deep Learning Framework for Structure and Dynamics Reconstruction from Time Series Data arxiv:1812.11482v1 [cond-mat.dis-nn] 30 Dec 2018 Zhang Zhang, Jing Liu, Shuo Wang, Ruyue Xin, Jiang Zhang

More information

Application of Principal Components Analysis and Gaussian Mixture Models to Printer Identification

Application of Principal Components Analysis and Gaussian Mixture Models to Printer Identification Application of Principal Components Analysis and Gaussian Mixture Models to Printer Identification Gazi. Ali, Pei-Ju Chiang Aravind K. Mikkilineni, George T. Chiu Edward J. Delp, and Jan P. Allebach School

More information

Energy Based Models, Restricted Boltzmann Machines and Deep Networks. Jesse Eickholt

Energy Based Models, Restricted Boltzmann Machines and Deep Networks. Jesse Eickholt Energy Based Models, Restricted Boltzmann Machines and Deep Networks Jesse Eickholt ???? Who s heard of Energy Based Models (EBMs) Restricted Boltzmann Machines (RBMs) Deep Belief Networks Auto-encoders

More information

Passive Differential Matched-field Depth Estimation of Moving Acoustic Sources

Passive Differential Matched-field Depth Estimation of Moving Acoustic Sources Lincoln Laboratory ASAP-2001 Workshop Passive Differential Matched-field Depth Estimation of Moving Acoustic Sources Shawn Kraut and Jeffrey Krolik Duke University Department of Electrical and Computer

More information

The Multi Stage Gibbs Sampling: Data Augmentation Dutch Example

The Multi Stage Gibbs Sampling: Data Augmentation Dutch Example The Multi Stage Gibbs Sampling: Data Augmentation Dutch Example Rebecca C. Steorts Bayesian Methods and Modern Statistics: STA 360/601 Module 8 1 Example: Data augmentation / Auxiliary variables A commonly-used

More information

Developing a Data Driven System for Computational Neuroscience

Developing a Data Driven System for Computational Neuroscience Developing a Data Driven System for Computational Neuroscience Ross Snider and Yongming Zhu Montana State University, Bozeman MT 59717, USA Abstract. A data driven system implies the need to integrate

More information

Overview of machine learning

Overview of machine learning Overview of machine learning Kevin P. Murphy Last updated November 26, 2007 1 Introduction In this Chapter, we provide a brief overview of the most commonly studied problems and solution methods within

More information

Thesis Proposal : Switching Linear Dynamic Systems with Higher-order Temporal Structure. Sang Min Oh

Thesis Proposal : Switching Linear Dynamic Systems with Higher-order Temporal Structure. Sang Min Oh Thesis Proposal : Switching Linear Dynamic Systems with Higher-order Temporal Structure Sang Min Oh sangmin@cc.gatech.edu 28th May 2008 Contents 1 Introduction 1 1.1 Automated Temporal Sequence Analysis.................................

More information

SGN (4 cr) Chapter 11

SGN (4 cr) Chapter 11 SGN-41006 (4 cr) Chapter 11 Clustering Jussi Tohka & Jari Niemi Department of Signal Processing Tampere University of Technology February 25, 2014 J. Tohka & J. Niemi (TUT-SGN) SGN-41006 (4 cr) Chapter

More information

Straight Lines and Hough

Straight Lines and Hough 09/30/11 Straight Lines and Hough Computer Vision CS 143, Brown James Hays Many slides from Derek Hoiem, Lana Lazebnik, Steve Seitz, David Forsyth, David Lowe, Fei-Fei Li Project 1 A few project highlights

More information

DATA MINING LECTURE 7. Hierarchical Clustering, DBSCAN The EM Algorithm

DATA MINING LECTURE 7. Hierarchical Clustering, DBSCAN The EM Algorithm DATA MINING LECTURE 7 Hierarchical Clustering, DBSCAN The EM Algorithm CLUSTERING What is a Clustering? In general a grouping of objects such that the objects in a group (cluster) are similar (or related)

More information

Recognition: Face Recognition. Linda Shapiro EE/CSE 576

Recognition: Face Recognition. Linda Shapiro EE/CSE 576 Recognition: Face Recognition Linda Shapiro EE/CSE 576 1 Face recognition: once you ve detected and cropped a face, try to recognize it Detection Recognition Sally 2 Face recognition: overview Typical

More information

Data Mining Chapter 3: Visualizing and Exploring Data Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University

Data Mining Chapter 3: Visualizing and Exploring Data Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University Data Mining Chapter 3: Visualizing and Exploring Data Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University Exploratory data analysis tasks Examine the data, in search of structures

More information

Statistical Techniques in Robotics (STR, S15) Lecture#05 (Monday, January 26) Lecturer: Byron Boots

Statistical Techniques in Robotics (STR, S15) Lecture#05 (Monday, January 26) Lecturer: Byron Boots Statistical Techniques in Robotics (STR, S15) Lecture#05 (Monday, January 26) Lecturer: Byron Boots Mapping 1 Occupancy Mapping When solving the localization problem, we had a map of the world and tried

More information

Mixture Models and EM

Mixture Models and EM Table of Content Chapter 9 Mixture Models and EM -means Clustering Gaussian Mixture Models (GMM) Expectation Maximiation (EM) for Mixture Parameter Estimation Introduction Mixture models allows Complex

More information

MAXIMUM LIKELIHOOD ESTIMATION USING ACCELERATED GENETIC ALGORITHMS

MAXIMUM LIKELIHOOD ESTIMATION USING ACCELERATED GENETIC ALGORITHMS In: Journal of Applied Statistical Science Volume 18, Number 3, pp. 1 7 ISSN: 1067-5817 c 2011 Nova Science Publishers, Inc. MAXIMUM LIKELIHOOD ESTIMATION USING ACCELERATED GENETIC ALGORITHMS Füsun Akman

More information

Neural Networks for Machine Learning. Lecture 15a From Principal Components Analysis to Autoencoders

Neural Networks for Machine Learning. Lecture 15a From Principal Components Analysis to Autoencoders Neural Networks for Machine Learning Lecture 15a From Principal Components Analysis to Autoencoders Geoffrey Hinton Nitish Srivastava, Kevin Swersky Tijmen Tieleman Abdel-rahman Mohamed Principal Components

More information

Local Linear Embedding. Katelyn Stringer ASTR 689 December 1, 2015

Local Linear Embedding. Katelyn Stringer ASTR 689 December 1, 2015 Local Linear Embedding Katelyn Stringer ASTR 689 December 1, 2015 Idea Behind LLE Good at making nonlinear high-dimensional data easier for computers to analyze Example: A high-dimensional surface Think

More information

Adaptation of a mixture of multivariate Bernoulli distributions

Adaptation of a mixture of multivariate Bernoulli distributions Adaptation of a mixture of multivariate Bernoulli distributions Content areas: Transfer, Adaptation, Multi-task Learning; Sensor Networks Abstract The mixture of multivariate Bernoulli distributions (MMB)

More information

SPARSE COMPONENT ANALYSIS FOR BLIND SOURCE SEPARATION WITH LESS SENSORS THAN SOURCES. Yuanqing Li, Andrzej Cichocki and Shun-ichi Amari

SPARSE COMPONENT ANALYSIS FOR BLIND SOURCE SEPARATION WITH LESS SENSORS THAN SOURCES. Yuanqing Li, Andrzej Cichocki and Shun-ichi Amari SPARSE COMPONENT ANALYSIS FOR BLIND SOURCE SEPARATION WITH LESS SENSORS THAN SOURCES Yuanqing Li, Andrzej Cichocki and Shun-ichi Amari Laboratory for Advanced Brain Signal Processing Laboratory for Mathematical

More information

Tree-based Cluster Weighted Modeling: Towards A Massively Parallel Real- Time Digital Stradivarius

Tree-based Cluster Weighted Modeling: Towards A Massively Parallel Real- Time Digital Stradivarius Tree-based Cluster Weighted Modeling: Towards A Massively Parallel Real- Time Digital Stradivarius Edward S. Boyden III e@media.mit.edu Physics and Media Group MIT Media Lab 0 Ames St. Cambridge, MA 039

More information

MSA220 - Statistical Learning for Big Data

MSA220 - Statistical Learning for Big Data MSA220 - Statistical Learning for Big Data Lecture 13 Rebecka Jörnsten Mathematical Sciences University of Gothenburg and Chalmers University of Technology Clustering Explorative analysis - finding groups

More information

Facial Expression Detection Using Implemented (PCA) Algorithm

Facial Expression Detection Using Implemented (PCA) Algorithm Facial Expression Detection Using Implemented (PCA) Algorithm Dileep Gautam (M.Tech Cse) Iftm University Moradabad Up India Abstract: Facial expression plays very important role in the communication with

More information

3D Human Motion Analysis and Manifolds

3D Human Motion Analysis and Manifolds D E P A R T M E N T O F C O M P U T E R S C I E N C E U N I V E R S I T Y O F C O P E N H A G E N 3D Human Motion Analysis and Manifolds Kim Steenstrup Pedersen DIKU Image group and E-Science center Motivation

More information

Data Preprocessing. Javier Béjar. URL - Spring 2018 CS - MAI 1/78 BY: $\

Data Preprocessing. Javier Béjar. URL - Spring 2018 CS - MAI 1/78 BY: $\ Data Preprocessing Javier Béjar BY: $\ URL - Spring 2018 C CS - MAI 1/78 Introduction Data representation Unstructured datasets: Examples described by a flat set of attributes: attribute-value matrix Structured

More information

Rectification and Distortion Correction

Rectification and Distortion Correction Rectification and Distortion Correction Hagen Spies March 12, 2003 Computer Vision Laboratory Department of Electrical Engineering Linköping University, Sweden Contents Distortion Correction Rectification

More information

STA 4273H: Sta-s-cal Machine Learning

STA 4273H: Sta-s-cal Machine Learning STA 4273H: Sta-s-cal Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! h0p://www.cs.toronto.edu/~rsalakhu/ Lecture 3 Parametric Distribu>ons We want model the probability

More information

Single Particle Reconstruction Techniques

Single Particle Reconstruction Techniques T H E U N I V E R S I T Y of T E X A S S C H O O L O F H E A L T H I N F O R M A T I O N S C I E N C E S A T H O U S T O N Single Particle Reconstruction Techniques For students of HI 6001-125 Computational

More information

Guide for inversion of noisy magnetic field using FFT

Guide for inversion of noisy magnetic field using FFT Guide for inversion of noisy magnetic field using FFT Eitan Levin Alexander Y. Meltzer August 29, 216 In this note, we explain how to use the code packages MagInverter2D and MagInverter1D to invert noisy

More information

This leads to our algorithm which is outlined in Section III, along with a tabular summary of it's performance on several benchmarks. The last section

This leads to our algorithm which is outlined in Section III, along with a tabular summary of it's performance on several benchmarks. The last section An Algorithm for Incremental Construction of Feedforward Networks of Threshold Units with Real Valued Inputs Dhananjay S. Phatak Electrical Engineering Department State University of New York, Binghamton,

More information

t 1 y(x;w) x 2 t 2 t 3 x 1

t 1 y(x;w) x 2 t 2 t 3 x 1 Neural Computing Research Group Dept of Computer Science & Applied Mathematics Aston University Birmingham B4 7ET United Kingdom Tel: +44 (0)121 333 4631 Fax: +44 (0)121 333 4586 http://www.ncrg.aston.ac.uk/

More information