Variational Methods for Discrete-Data Latent Gaussian Models
|
|
|
- Opal Davidson
- 5 years ago
- Views:
Transcription
1 Variational Methods for Discrete-Data Latent Gaussian Models University of British Columbia Vancouver, Canada March 6, 2012
2 The Big Picture Joint density models for data with mixed data types Bayesian models principled and robust approach Algorithms that are not only accurate and fast, but are also easy to tune, implement, and intuitive (speed- accuracy tradeoffs) Variational Methods for Discrete-Data Latent Gaussian Models Slide 2 of 46
3 Sources of Discrete Data User rating data Survey/voting data and blogs for sentiment analysis Health data Consumer choice data Sports/game data tag correlation. Slide 3 of 46
4 Motivation: Recommendation system Movie rating dataset Missing values Different types of data User1 User2 User3 User4 User5 User6. Movie Movie Movie Movie Movie Movie Slide 4 of 46
5 Missing Ratings From Wikipedia on Netflix-prize dataset The training set is such that the average user rated over 200 movies, and the average movie was rated by over 5000 users. But there is wide variance in the data some movies in the training set have as few as 3 ratings, while one user rated over 17,000 movies. Movielens Dataset Slide 6 of 46
6 Sources of Discrete Data User rating data Survey/voting data and blogs for sentiment analysis Health data Consumer choice data Sports/game data tag correlation. Slide 8 of 46
7 What we need! For these datasets, we need a method of analysis which Handles missing values efficiently Makes efficient use of the data by weighting reliable data vectors more than the unreliable ones Makes efficient use of the data by fusing different types of data efficiently (binary, ordinal, categorical, count, text) Slide 9 of 46
8 Factor Model movies D Y D movies W L factors N users Z N users L factors ExpFamily: Gaussian: Slide 10 of 46 Collins et. al. 2002, Khan et. al.2010, Yu et.al. 2009
9 Bayesian Learning µ, Σ x z n y n n=1:n W This talk: Lower bound maximization Slide 11 of 46
10 Variational Methods Design tractable bounds to reduce approximation error Efficient optimization since lower bounds are concave : good convergence rates and easy convergence diagnostics Efficient expectation-maximization (EM) algorithms for parameter leaning Comparable performance to MCMC, but much faster Algorithms with a wide range of speed-accuracy trade-offs Slide 12 of 46
11 Outline Latent Gaussian models Bounds for binary data Bounds for categorical data Results Future work and conclusions Slide 13 of 46
12 Outline Latent Gaussian models Definition and examples Problem with parameter learning Bounds for binary data Bounds for categorical data Results Future work and conclusions Slide 14 of 46
13 Latent Gaussian Model (LGM) y 11 y 12 y 13 η 11 η 12 η 13 z 11 z 12 z 13 y 21 y 23 y 32 y 32 η 21 η 23 w η 32 η 32 N L z 21 z 22 z 23 D y D1 y D2 y D3 N η D1 η D2 η D3 D L µ, Σ z n W y n n=1:n Slide 15 of 46
14 Likelihood Examples Data type Real Count Binary Distribution Gaussian Poisson Bernoulli-Logit 0.9 Categorical Ordinal Multinomial-Logit Proportional-odds 2 Slide 16 of 46
15 Parameter Estimation µ, Σ z n W y n Bernoulli-logit n=1:n x Slide 19 of 46
16 Jensen s Lower Bound Slide 20 of 46
17 Variational Lower Bound Generalized EM algorithm E-step involves minimizing convex function Early stopping in E-step (Almost) no tuning parameters Easy convergence diagnostics Slide 21 of 46
18 Outline Latent Gaussian models Bounds for binary data Bernoulli-logistic likelihood The Bohning bound (Khan, Marlin, Bouchard, Murphy, NIPS 2010) Piecewise bounds (Marlin, Khan, Murphy, ICML 2011) Bounds for categorical data Results Future work and conclusions Slide 22 of 46
19 Bernoulli-Logit Likelihood some other tractable terms in m and V Slide 23 of 46
20 Local Variational Bounds + some other tractable terms in m and V x Bohning s bound (Khan, Marlin, Bouchard, Murphy 2010) Jaakola s bound (Jaakkola and Jordan1996) Piecewise quadratic bounds (Marlin, Khan, Murphy 2011) Slide 24 of 46
21 Bohning Bound is Faster Slide 25 of 46 O(L 3 ND) For n = 1:N V n = (W T A n W + I) -1 m n =.. end O(L 2 ND) V = (W T AW + I) -1 For n = 1:N m n =.. end
22 Piecewise bounds are more accurate Bohning Jaakkola Piecewise Q 1 (x) Q 3 (x) Q 2 (x) Slide 27 of 46
23 Details of Piecewise bounds Find cut points and parameters of each piece by minimizing maximum error Linear pieces (Hsiung, Kim and Boyd, 2008) Quadratic Pieces (Nelder- Mead method) Fixed Piecewise Bounds! Increase accuracy by increasing the number of pieces Slide 28 of 46
24 Outline Latent Gaussian models Bounds for binary data Bounds for categorical data Multinomial-logistic likelihood and local variational bounds Stick-breaking likelihood (Khan, Mohamed, Marlin, Murphy, AI-Stats 2012) Results Future work and conclusions Slide 29 of 46
25 Multinomial-Logit Likelihood Slide 30 of 46
26 Local Variational bounds The Bohning bound Fast and closed form updates The log bound (Blei and Lafferty 2006) More accurate than the Bohning bound, but slower The product of sigmoid bound (Bouchard 2007) Slide 31 of 46
27 Stick-Breaking Likelihood 0 1 Slide 32 of 46
28 Outline Latent Gaussian models Bounds for binary data Bounds for categorical data Results Future work and conclusions Slide 33 of 46
29 Speed Accuracy Trade-offs Binary FA : UCI voting dataset (D=15, N=435) Bohning Jaakkola Piecewise Linear with 3 pieces Piecewise Quad with 3 pieces Piecewise Quad with 10 pieces Slide 34 of 46
30 Comparison with EP Binary Gaussian Process : Ionosphere dataset (D=200) Σ ij =σ exp[- x i -x j 2 /s] µ, Σ z n W y n n=1:n Slide 35 of 46
31 EP vs PW : Posterior Distribution (Neg) KL-Lower Bound to MargLik Approximation To MargLik Pred Error EP Piece ewise Bound Slide 38 of 46
32 Comparison with EP Both methods give very similar results for GPs Our approach can be easily extended to factor models Variational EM objective function is well-defined and can be obtained by solving minimization of convex functions Numerically stable Slide 39 of 46
33 MultiClass Gaussian Process MCMC Glass dataset (D=143, K=6) Bohning Log VB-probit Stick-PW Prediction Error Ne egloglik Slide 40 of 46
34 Categorical Factor Analysis Glass dataset (D = 10, N = 958, sum of K = 29) Logit-log Stick-PW Slide 41 of 46
35 Outline Latent Gaussian models Bounds for binary data Bounds for categorical data Results Future work and conclusions Slide 42 of 46
36 Future Work Large-scale collaborative filtering Use convexity to design approximate gradient methods Sparse Gaussian Posterior Distribution Tuning HMC using Bayesian optimization methods Latent Sparse-factor model Conditional models (e.g. to model for tag-image correlation) Slide 43 of 46
37 Conclusions Variational methods show comparable performance with existing approaches The main sources of errors is the bounding error Design of piecewise bounds to control these errors A good control over speed-accuracy trade-offs can be obtained Variational lower bounds can be optimized efficiently Use of convex optimization methods to get fast convergence rates and easy convergence diagnostics Design of efficient expectation-maximization (EM) algorithms Slide 44 of 46
38 Collaborators Kevin Murphy UBC Benjamin Marlin U. Mass-Amherst Guillaume Bouchard XRCE, France Shakir Mohamed U. Cambridge, now at UBC Slide 45 of 46
39 Thank You Slide 46 of 46
08 An Introduction to Dense Continuous Robotic Mapping
NAVARCH/EECS 568, ROB 530 - Winter 2018 08 An Introduction to Dense Continuous Robotic Mapping Maani Ghaffari March 14, 2018 Previously: Occupancy Grid Maps Pose SLAM graph and its associated dense occupancy
ECE521: Week 11, Lecture March 2017: HMM learning/inference. With thanks to Russ Salakhutdinov
ECE521: Week 11, Lecture 20 27 March 2017: HMM learning/inference With thanks to Russ Salakhutdinov Examples of other perspectives Murphy 17.4 End of Russell & Norvig 15.2 (Artificial Intelligence: A Modern
Deep Generative Models Variational Autoencoders
Deep Generative Models Variational Autoencoders Sudeshna Sarkar 5 April 2017 Generative Nets Generative models that represent probability distributions over multiple variables in some way. Directed Generative
Warped Mixture Models
Warped Mixture Models Tomoharu Iwata, David Duvenaud, Zoubin Ghahramani Cambridge University Computational and Biological Learning Lab March 11, 2013 OUTLINE Motivation Gaussian Process Latent Variable
Applied Bayesian Nonparametrics 5. Spatial Models via Gaussian Processes, not MRFs Tutorial at CVPR 2012 Erik Sudderth Brown University
Applied Bayesian Nonparametrics 5. Spatial Models via Gaussian Processes, not MRFs Tutorial at CVPR 2012 Erik Sudderth Brown University NIPS 2008: E. Sudderth & M. Jordan, Shared Segmentation of Natural
Clustering Lecture 5: Mixture Model
Clustering Lecture 5: Mixture Model Jing Gao SUNY Buffalo 1 Outline Basics Motivation, definition, evaluation Methods Partitional Hierarchical Density-based Mixture model Spectral methods Advanced topics
Mini-project 2 CMPSCI 689 Spring 2015 Due: Tuesday, April 07, in class
Mini-project 2 CMPSCI 689 Spring 2015 Due: Tuesday, April 07, in class Guidelines Submission. Submit a hardcopy of the report containing all the figures and printouts of code in class. For readability
BINARY PRINCIPAL COMPONENT ANALYSIS IN THE NETFLIX COLLABORATIVE FILTERING TASK
BINARY PRINCIPAL COMPONENT ANALYSIS IN THE NETFLIX COLLABORATIVE FILTERING TASK László Kozma, Alexander Ilin, Tapani Raiko Helsinki University of Technology Adaptive Informatics Research Center P.O. Box
Conjugate-Computation Variational Inference : Converting Variational Inference in Non-Conjugate Models to Inferences in Conjugate Models
Conjugate-Computation Variational Inference : Converting Variational Inference in Non-Conjugate Models to Inferences in Conjugate Models Mohammad Emtiyaz Khan Center for Advanced Intelligence Project (AIP)
Logis&c Regression. Aar$ Singh & Barnabas Poczos. Machine Learning / Jan 28, 2014
Logis&c Regression Aar$ Singh & Barnabas Poczos Machine Learning 10-701/15-781 Jan 28, 2014 Linear Regression & Linear Classifica&on Weight Height Linear fit Linear decision boundary 2 Naïve Bayes Recap
Ludwig Fahrmeir Gerhard Tute. Statistical odelling Based on Generalized Linear Model. íecond Edition. . Springer
Ludwig Fahrmeir Gerhard Tute Statistical odelling Based on Generalized Linear Model íecond Edition. Springer Preface to the Second Edition Preface to the First Edition List of Examples List of Figures
CS 229 Midterm Review
CS 229 Midterm Review Course Staff Fall 2018 11/2/2018 Outline Today: SVMs Kernels Tree Ensembles EM Algorithm / Mixture Models [ Focus on building intuition, less so on solving specific problems. Ask
Adaptive Dropout Training for SVMs
Department of Computer Science and Technology Adaptive Dropout Training for SVMs Jun Zhu Joint with Ning Chen, Jingwei Zhuo, Jianfei Chen, Bo Zhang Tsinghua University ShanghaiTech Symposium on Data Science,
ECE 5424: Introduction to Machine Learning
ECE 5424: Introduction to Machine Learning Topics: Unsupervised Learning: Kmeans, GMM, EM Readings: Barber 20.1-20.3 Stefan Lee Virginia Tech Tasks Supervised Learning x Classification y Discrete x Regression
Introduction to Machine Learning CMU-10701
Introduction to Machine Learning CMU-10701 Clustering and EM Barnabás Póczos & Aarti Singh Contents Clustering K-means Mixture of Gaussians Expectation Maximization Variational Methods 2 Clustering 3 K-
Learning the Structure of Sum-Product Networks. Robert Gens Pedro Domingos
Learning the Structure of Sum-Product Networks Robert Gens Pedro Domingos w 20 10x O(n) X Y LL PLL CLL CMLL Motivation SPN Structure Experiments Review Learning Graphical Models Representation Inference
CRFs for Image Classification
CRFs for Image Classification Devi Parikh and Dhruv Batra Carnegie Mellon University Pittsburgh, PA 15213 {dparikh,dbatra}@ece.cmu.edu Abstract We use Conditional Random Fields (CRFs) to classify regions
Computationally Efficient M-Estimation of Log-Linear Structure Models
Computationally Efficient M-Estimation of Log-Linear Structure Models Noah Smith, Doug Vail, and John Lafferty School of Computer Science Carnegie Mellon University {nasmith,dvail2,lafferty}@cs.cmu.edu
Clustering and The Expectation-Maximization Algorithm
Clustering and The Expectation-Maximization Algorithm Unsupervised Learning Marek Petrik 3/7 Some of the figures in this presentation are taken from An Introduction to Statistical Learning, with applications
Comparison of Variational Bayes and Gibbs Sampling in Reconstruction of Missing Values with Probabilistic Principal Component Analysis
Comparison of Variational Bayes and Gibbs Sampling in Reconstruction of Missing Values with Probabilistic Principal Component Analysis Luis Gabriel De Alba Rivera Aalto University School of Science and
Sum-Product Networks. STAT946 Deep Learning Guest Lecture by Pascal Poupart University of Waterloo October 15, 2015
Sum-Product Networks STAT946 Deep Learning Guest Lecture by Pascal Poupart University of Waterloo October 15, 2015 Introduction Outline What is a Sum-Product Network? Inference Applications In more depth
Sampling PCA, enhancing recovered missing values in large scale matrices. Luis Gabriel De Alba Rivera 80555S
Sampling PCA, enhancing recovered missing values in large scale matrices. Luis Gabriel De Alba Rivera 80555S May 2, 2009 Introduction Human preferences (the quality tags we put on things) are language
Network Traffic Measurements and Analysis
DEIB - Politecnico di Milano Fall, 2017 Sources Hastie, Tibshirani, Friedman: The Elements of Statistical Learning James, Witten, Hastie, Tibshirani: An Introduction to Statistical Learning Andrew Ng:
Clustering K-means. Machine Learning CSEP546 Carlos Guestrin University of Washington February 18, Carlos Guestrin
Clustering K-means Machine Learning CSEP546 Carlos Guestrin University of Washington February 18, 2014 Carlos Guestrin 2005-2014 1 Clustering images Set of Images [Goldberger et al.] Carlos Guestrin 2005-2014
Probabilistic Graphical Models Part III: Example Applications
Probabilistic Graphical Models Part III: Example Applications Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr CS 551, Fall 2014 CS 551, Fall 2014 c 2014, Selim
Using Machine Learning to Optimize Storage Systems
Using Machine Learning to Optimize Storage Systems Dr. Kiran Gunnam 1 Outline 1. Overview 2. Building Flash Models using Logistic Regression. 3. Storage Object classification 4. Storage Allocation recommendation
10-701/15-781, Fall 2006, Final
-7/-78, Fall 6, Final Dec, :pm-8:pm There are 9 questions in this exam ( pages including this cover sheet). If you need more room to work out your answer to a question, use the back of the page and clearly
Missing Data Analysis for the Employee Dataset
Missing Data Analysis for the Employee Dataset 67% of the observations have missing values! Modeling Setup Random Variables: Y i =(Y i1,...,y ip ) 0 =(Y i,obs, Y i,miss ) 0 R i =(R i1,...,r ip ) 0 ( 1
K-Means Clustering. Sargur Srihari
K-Means Clustering Sargur srihari@cedar.buffalo.edu 1 Topics in Mixture Models and EM Mixture models K-means Clustering Mixtures of Gaussians Maximum Likelihood EM for Gaussian mistures EM Algorithm Gaussian
Latent Regression Bayesian Network for Data Representation
2016 23rd International Conference on Pattern Recognition (ICPR) Cancún Center, Cancún, México, December 4-8, 2016 Latent Regression Bayesian Network for Data Representation Siqi Nie Department of Electrical,
Lecture 8: The EM algorithm
10-708: Probabilistic Graphical Models 10-708, Spring 2017 Lecture 8: The EM algorithm Lecturer: Manuela M. Veloso, Eric P. Xing Scribes: Huiting Liu, Yifan Yang 1 Introduction Previous lecture discusses
Conditional Random Fields - A probabilistic graphical model. Yen-Chin Lee 指導老師 : 鮑興國
Conditional Random Fields - A probabilistic graphical model Yen-Chin Lee 指導老師 : 鮑興國 Outline Labeling sequence data problem Introduction conditional random field (CRF) Different views on building a conditional
Generative and discriminative classification techniques
Generative and discriminative classification techniques Machine Learning and Category Representation 013-014 Jakob Verbeek, December 13+0, 013 Course website: http://lear.inrialpes.fr/~verbeek/mlcr.13.14
Modeling and Monitoring Crop Disease in Developing Countries
Modeling and Monitoring Crop Disease in Developing Countries John Quinn 1, Kevin Leyton-Brown 2, Ernest Mwebaze 1 1 Department of Computer Science 2 Department of Computer Science Makerere University,
Weighted Alternating Least Squares (WALS) for Movie Recommendations) Drew Hodun SCPD. Abstract
Weighted Alternating Least Squares (WALS) for Movie Recommendations) Drew Hodun SCPD Abstract There are two common main approaches to ML recommender systems, feedback-based systems and content-based systems.
Unsupervised: no target value to predict
Clustering Unsupervised: no target value to predict Differences between models/algorithms: Exclusive vs. overlapping Deterministic vs. probabilistic Hierarchical vs. flat Incremental vs. batch learning
CPSC 340: Machine Learning and Data Mining. Principal Component Analysis Fall 2016
CPSC 340: Machine Learning and Data Mining Principal Component Analysis Fall 2016 A2/Midterm: Admin Grades/solutions will be posted after class. Assignment 4: Posted, due November 14. Extra office hours:
STA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 12 Combining
Expectation-Maximization. Nuno Vasconcelos ECE Department, UCSD
Expectation-Maximization Nuno Vasconcelos ECE Department, UCSD Plan for today last time we started talking about mixture models we introduced the main ideas behind EM to motivate EM, we looked at classification-maximization
Case Study IV: Bayesian clustering of Alzheimer patients
Case Study IV: Bayesian clustering of Alzheimer patients Mike Wiper and Conchi Ausín Department of Statistics Universidad Carlos III de Madrid Advanced Statistics and Data Mining Summer School 2nd - 6th
Implicit Mixtures of Restricted Boltzmann Machines
Implicit Mixtures of Restricted Boltzmann Machines Vinod Nair and Geoffrey Hinton Department of Computer Science, University of Toronto 10 King s College Road, Toronto, M5S 3G5 Canada {vnair,hinton}@cs.toronto.edu
DS Machine Learning and Data Mining I. Alina Oprea Associate Professor, CCIS Northeastern University
DS 4400 Machine Learning and Data Mining I Alina Oprea Associate Professor, CCIS Northeastern University September 20 2018 Review Solution for multiple linear regression can be computed in closed form
Structured Learning. Jun Zhu
Structured Learning Jun Zhu Supervised learning Given a set of I.I.D. training samples Learn a prediction function b r a c e Supervised learning (cont d) Many different choices Logistic Regression Maximum
Probabilistic Matrix Factorization
DRAFT. A revised version will appear in NIPS 007. Probabilistic Matrix Factorization Ruslan Salakhutdinov and Andriy Mnih Department of Computer Science, University of Toronto 6 King s College Rd, M5S
Machine Learning Classifiers and Boosting
Machine Learning Classifiers and Boosting Reading Ch 18.6-18.12, 20.1-20.3.2 Outline Different types of learning problems Different types of learning algorithms Supervised learning Decision trees Naïve
DS Machine Learning and Data Mining I. Alina Oprea Associate Professor, CCIS Northeastern University
DS 4400 Machine Learning and Data Mining I Alina Oprea Associate Professor, CCIS Northeastern University January 22 2019 Outline Practical issues in Linear Regression Outliers Categorical variables Lab
Machine Learning / Jan 27, 2010
Revisiting Logistic Regression & Naïve Bayes Aarti Singh Machine Learning 10-701/15-781 Jan 27, 2010 Generative and Discriminative Classifiers Training classifiers involves learning a mapping f: X -> Y,
DS Machine Learning and Data Mining I. Alina Oprea Associate Professor, CCIS Northeastern University
DS 4400 Machine Learning and Data Mining I Alina Oprea Associate Professor, CCIS Northeastern University January 24 2019 Logistics HW 1 is due on Friday 01/25 Project proposal: due Feb 21 1 page description
Expectation Propagation
Expectation Propagation Erik Sudderth 6.975 Week 11 Presentation November 20, 2002 Introduction Goal: Efficiently approximate intractable distributions Features of Expectation Propagation (EP): Deterministic,
732A54/TDDE31 Big Data Analytics
732A54/TDDE31 Big Data Analytics Lecture 10: Machine Learning with MapReduce Jose M. Peña IDA, Linköping University, Sweden 1/27 Contents MapReduce Framework Machine Learning with MapReduce Neural Networks
Distributed Training of Large-scale Logistic models
Distributed Training of Large-scale Logistic models Siddharth Gopal 5000 Forbes Ave, Carnegie Mellon University, Pittsburgh PA 15217 USA Yig Yang 5000 Forbes Ave, Carnegie Mellon University, Pittsburgh
Alternating Minimization. Jun Wang, Tony Jebara, and Shih-fu Chang
Graph Transduction via Alternating Minimization Jun Wang, Tony Jebara, and Shih-fu Chang 1 Outline of the presentation Brief introduction and related work Problems with Graph Labeling Imbalanced labels
Score function estimator and variance reduction techniques
and variance reduction techniques Wilker Aziz University of Amsterdam May 24, 2018 Wilker Aziz Discrete variables 1 Outline 1 2 3 Wilker Aziz Discrete variables 1 Variational inference for belief networks
Machine Learning using Matlab. Lecture 3 Logistic regression and regularization
Machine Learning using Matlab Lecture 3 Logistic regression and regularization Presentation Date (correction) 10.07.2017 11.07.2017 17.07.2017 18.07.2017 24.07.2017 25.07.2017 Project proposals 13 submissions,
Tree-structured approximations by expectation propagation
Tree-structured approximations by expectation propagation Thomas Minka Department of Statistics Carnegie Mellon University Pittsburgh, PA 15213 USA minka@stat.cmu.edu Yuan Qi Media Laboratory Massachusetts
Building Classifiers using Bayesian Networks
Building Classifiers using Bayesian Networks Nir Friedman and Moises Goldszmidt 1997 Presented by Brian Collins and Lukas Seitlinger Paper Summary The Naive Bayes classifier has reasonable performance
Probabilistic Graphical Models
Overview of Part Two Probabilistic Graphical Models Part Two: Inference and Learning Christopher M. Bishop Exact inference and the junction tree MCMC Variational methods and EM Example General variational
Variational Autoencoders. Sargur N. Srihari
Variational Autoencoders Sargur N. srihari@cedar.buffalo.edu Topics 1. Generative Model 2. Standard Autoencoder 3. Variational autoencoders (VAE) 2 Generative Model A variational autoencoder (VAE) is a
Quantitative Biology II!
Quantitative Biology II! Lecture 3: Markov Chain Monte Carlo! March 9, 2015! 2! Plan for Today!! Introduction to Sampling!! Introduction to MCMC!! Metropolis Algorithm!! Metropolis-Hastings Algorithm!!
Estimating Human Pose in Images. Navraj Singh December 11, 2009
Estimating Human Pose in Images Navraj Singh December 11, 2009 Introduction This project attempts to improve the performance of an existing method of estimating the pose of humans in still images. Tasks
Logistic Regression. Abstract
Logistic Regression Tsung-Yi Lin, Chen-Yu Lee Department of Electrical and Computer Engineering University of California, San Diego {tsl008, chl60}@ucsd.edu January 4, 013 Abstract Logistic regression
Bayes Net Learning. EECS 474 Fall 2016
Bayes Net Learning EECS 474 Fall 2016 Homework Remaining Homework #3 assigned Homework #4 will be about semi-supervised learning and expectation-maximization Homeworks #3-#4: the how of Graphical Models
Supervised Learning for Image Segmentation
Supervised Learning for Image Segmentation Raphael Meier 06.10.2016 Raphael Meier MIA 2016 06.10.2016 1 / 52 References A. Ng, Machine Learning lecture, Stanford University. A. Criminisi, J. Shotton, E.
Clustering K-means. Machine Learning CSEP546 Carlos Guestrin University of Washington February 18, Carlos Guestrin
Clustering K-means Machine Learning CSEP546 Carlos Guestrin University of Washington February 18, 2014 Carlos Guestrin 2005-2014 1 Clustering images Set of Images [Goldberger et al.] Carlos Guestrin 2005-2014
Simulation Calibration with Correlated Knowledge-Gradients
Simulation Calibration with Correlated Knowledge-Gradients Peter Frazier Warren Powell Hugo Simão Operations Research & Information Engineering, Cornell University Operations Research & Financial Engineering,
Information Filtering for arxiv.org
Information Filtering for arxiv.org Bandits, Exploration vs. Exploitation, & the Cold Start Problem Peter Frazier School of Operations Research & Information Engineering Cornell University with Xiaoting
Liangjie Hong*, Dawei Yin*, Jian Guo, Brian D. Davison*
Tracking Trends: Incorporating Term Volume into Temporal Topic Models Liangjie Hong*, Dawei Yin*, Jian Guo, Brian D. Davison* Dept. of Computer Science and Engineering, Lehigh University, Bethlehem, PA,
CHAPTER 1 INTRODUCTION
Introduction CHAPTER 1 INTRODUCTION Mplus is a statistical modeling program that provides researchers with a flexible tool to analyze their data. Mplus offers researchers a wide choice of models, estimators,
CS839: Probabilistic Graphical Models. Lecture 10: Learning with Partially Observed Data. Theo Rekatsinas
CS839: Probabilistic Graphical Models Lecture 10: Learning with Partially Observed Data Theo Rekatsinas 1 Partially Observed GMs Speech recognition 2 Partially Observed GMs Evolution 3 Partially Observed
CSC412: Stochastic Variational Inference. David Duvenaud
CSC412: Stochastic Variational Inference David Duvenaud Admin A3 will be released this week and will be shorter Motivation for REINFORCE Class projects Class Project ideas Develop a generative model for
Dynamic Bayesian network (DBN)
Readings: K&F: 18.1, 18.2, 18.3, 18.4 ynamic Bayesian Networks Beyond 10708 Graphical Models 10708 Carlos Guestrin Carnegie Mellon University ecember 1 st, 2006 1 ynamic Bayesian network (BN) HMM defined
CPSC 340: Machine Learning and Data Mining. Principal Component Analysis Fall 2017
CPSC 340: Machine Learning and Data Mining Principal Component Analysis Fall 2017 Assignment 3: 2 late days to hand in tonight. Admin Assignment 4: Due Friday of next week. Last Time: MAP Estimation MAP
Variational Methods for Graphical Models
Chapter 2 Variational Methods for Graphical Models 2.1 Introduction The problem of probabb1istic inference in graphical models is the problem of computing a conditional probability distribution over the
FMA901F: Machine Learning Lecture 3: Linear Models for Regression. Cristian Sminchisescu
FMA901F: Machine Learning Lecture 3: Linear Models for Regression Cristian Sminchisescu Machine Learning: Frequentist vs. Bayesian In the frequentist setting, we seek a fixed parameter (vector), with value(s)
TagProp: Discriminative Metric Learning in Nearest Neighbor Models for Image Annotation
TagProp: Discriminative Metric Learning in Nearest Neighbor Models for Image Annotation Matthieu Guillaumin, Thomas Mensink, Jakob Verbeek, Cordelia Schmid LEAR team, INRIA Rhône-Alpes, Grenoble, France
Feature Subset Selection for Logistic Regression via Mixed Integer Optimization
Feature Subset Selection for Logistic Regression via Mixed Integer Optimization Yuichi TAKANO (Senshu University, Japan) Toshiki SATO (University of Tsukuba) Ryuhei MIYASHIRO (Tokyo University of Agriculture
Gradient Descent. Wed Sept 20th, James McInenrey Adapted from slides by Francisco J. R. Ruiz
Gradient Descent Wed Sept 20th, 2017 James McInenrey Adapted from slides by Francisco J. R. Ruiz Housekeeping A few clarifications of and adjustments to the course schedule: No more breaks at the midpoint
Neural Networks. Single-layer neural network. CSE 446: Machine Learning Emily Fox University of Washington March 10, /10/2017
3/0/207 Neural Networks Emily Fox University of Washington March 0, 207 Slides adapted from Ali Farhadi (via Carlos Guestrin and Luke Zettlemoyer) Single-layer neural network 3/0/207 Perceptron as a neural
CPSC 340: Machine Learning and Data Mining. Logistic Regression Fall 2016
CPSC 340: Machine Learning and Data Mining Logistic Regression Fall 2016 Admin Assignment 1: Marks visible on UBC Connect. Assignment 2: Solution posted after class. Assignment 3: Due Wednesday (at any
Convexization in Markov Chain Monte Carlo
in Markov Chain Monte Carlo 1 IBM T. J. Watson Yorktown Heights, NY 2 Department of Aerospace Engineering Technion, Israel August 23, 2011 Problem Statement MCMC processes in general are governed by non
Analysis and Synthesis of 3D Shape Families via Deep Learned Generative Models of Surfaces
Analysis and Synthesis of 3D Shape Families via Deep Learned Generative Models of Surfaces Haibin Huang, Evangelos Kalogerakis, Benjamin Marlin University of Massachusetts Amherst Given an input 3D shape
Multi-Class Logistic Regression and Perceptron
Multi-Class Logistic Regression and Perceptron Instructor: Wei Xu Some slides adapted from Dan Jurfasky, Brendan O Connor and Marine Carpuat MultiClass Classification Q: what if we have more than 2 categories?
Introduction to Machine Learning
Department of Computer Science, University of Helsinki Autumn 2009, second term Session 8, November 27 th, 2009 1 2 3 Multiplicative Updates for L1-Regularized Linear and Logistic Last time I gave you
Class 6 Large-Scale Image Classification
Class 6 Large-Scale Image Classification Liangliang Cao, March 7, 2013 EECS 6890 Topics in Information Processing Spring 2013, Columbia University http://rogerioferis.com/visualrecognitionandsearch Visual
Shape from Silhouettes I CV book Szelisky
Shape from Silhouettes I CV book Szelisky 11.6.2 Guido Gerig CS 6320, Spring 2012 (slides modified from Marc Pollefeys UNC Chapel Hill, some of the figures and slides are adapted from M. Pollefeys, J.S.
Behavioral Data Mining. Lecture 18 Clustering
Behavioral Data Mining Lecture 18 Clustering Outline Why? Cluster quality K-means Spectral clustering Generative Models Rationale Given a set {X i } for i = 1,,n, a clustering is a partition of the X i
Simulation Calibration with Correlated Knowledge-Gradients
Simulation Calibration with Correlated Knowledge-Gradients Peter Frazier Warren Powell Hugo Simão Operations Research & Information Engineering, Cornell University Operations Research & Financial Engineering,
Multi-task Multi-modal Models for Collective Anomaly Detection
Multi-task Multi-modal Models for Collective Anomaly Detection Tsuyoshi Ide ( Ide-san ), Dzung T. Phan, J. Kalagnanam PhD, Senior Technical Staff Member IBM Thomas J. Watson Research Center This slides
Search Engines. Information Retrieval in Practice
Search Engines Information Retrieval in Practice All slides Addison Wesley, 2008 Classification and Clustering Classification and clustering are classical pattern recognition / machine learning problems
Graphical Models, Bayesian Method, Sampling, and Variational Inference
Graphical Models, Bayesian Method, Sampling, and Variational Inference With Application in Function MRI Analysis and Other Imaging Problems Wei Liu Scientific Computing and Imaging Institute University
Markov Chain Monte Carlo (part 1)
Markov Chain Monte Carlo (part 1) Edps 590BAY Carolyn J. Anderson Department of Educational Psychology c Board of Trustees, University of Illinois Spring 2018 Depending on the book that you select for
Univariate Extreme Value Analysis. 1 Block Maxima. Practice problems using the extremes ( 2.0 5) package. 1. Pearson Type III distribution
Univariate Extreme Value Analysis Practice problems using the extremes ( 2.0 5) package. 1 Block Maxima 1. Pearson Type III distribution (a) Simulate 100 maxima from samples of size 1000 from the gamma
Multiple Imputation for Missing Data. Benjamin Cooper, MPH Public Health Data & Training Center Institute for Public Health
Multiple Imputation for Missing Data Benjamin Cooper, MPH Public Health Data & Training Center Institute for Public Health Outline Missing data mechanisms What is Multiple Imputation? Software Options
Improving the way neural networks learn Srikumar Ramalingam School of Computing University of Utah
Improving the way neural networks learn Srikumar Ramalingam School of Computing University of Utah Reference Most of the slides are taken from the third chapter of the online book by Michael Nielson: neuralnetworksanddeeplearning.com
Pattern Recognition. Kjell Elenius. Speech, Music and Hearing KTH. March 29, 2007 Speech recognition
Pattern Recognition Kjell Elenius Speech, Music and Hearing KTH March 29, 2007 Speech recognition 2007 1 Ch 4. Pattern Recognition 1(3) Bayes Decision Theory Minimum-Error-Rate Decision Rules Discriminant
CS489/698: Intro to ML
CS489/698: Intro to ML Lecture 14: Training of Deep NNs Instructor: Sun Sun 1 Outline Activation functions Regularization Gradient-based optimization 2 Examples of activation functions 3 5/28/18 Sun Sun
Virtual Vector Machine for Bayesian Online Classification
Virtual Vector Machine for Bayesian Online Classification Thomas P. Minka, Rongjing Xiang, Yuan (Alan) Qi Appeared in UAI 2009 Presented by Lingbo Li Introduction Online Learning Update model parameters
K-Means and Gaussian Mixture Models
K-Means and Gaussian Mixture Models David Rosenberg New York University June 15, 2015 David Rosenberg (New York University) DS-GA 1003 June 15, 2015 1 / 43 K-Means Clustering Example: Old Faithful Geyser
Linear Regression & Gradient Descent
Linear Regression & Gradient Descent These slides were assembled by Byron Boots, with grateful acknowledgement to Eric Eaton and the many others who made their course materials freely available online.
Modelling Proportions and Count Data
Modelling Proportions and Count Data Rick White May 4, 2016 Outline Analysis of Count Data Binary Data Analysis Categorical Data Analysis Generalized Linear Models Questions Types of Data Continuous data: