Clustering Relational Data using the Infinite Relational Model
|
|
|
- Alban Griffin
- 5 years ago
- Views:
Transcription
1 Clustering Relational Data using the Infinite Relational Model Ana Daglis Supervised by: Matthew Ludkin September 4, 2015 Ana Daglis Clustering Data using the Infinite Relational Model September 4, / 29
2 Outline 1 Clustering 2 Model 3 Gibbs Sampling Methodology Results 4 Methodology Results 5 Future Work Ana Daglis Clustering Data using the Infinite Relational Model September 4, / 29
3 Clustering Clustering Cluster Analysis: Given an unlabelled data, want algorithms that automatically group the datapoints into coherent subsets/clusters. Applications: recommendation engines (Netflix, itunes, Quora,...) image compression targeted marketing Google News Ana Daglis Clustering Data using the Infinite Relational Model September 4, / 29
4 Model Infinite Relational Model Infinite Relational Model (IRM) is a model, in which each node is assigned to a cluster. The number of clusters is not known initially and is learned from the data as part of the statistical inference. IRM is represented by the following parameters: z i - cluster, containing node i, for i = 1,..., n. φ i,j - probability of an edge between i-th and j-th clusters. Ana Daglis Clustering Data using the Infinite Relational Model September 4, / 29
5 Model Assumptions Given the adjacency matrix of the graph, X, as our data, we assume that X i,j Bernoulli(φ zi,z j ). Since z and φ are not known, hierarchical and beta priors respectively are imposed: { z CRP(A) φ i,j Beta(a, b). Ana Daglis Clustering Data using the Infinite Relational Model September 4, / 29
6 Model Chinese Restaurant Process (CRP(A)) The Chinese restaurant process is a discrete process, whose value at time n is the partition of 1, 2,..., n. At time n = 1, have trivial partition {{1}}. At time n + 1, element n + 1 is either: 1 added to an existing block with probability b /(n + A), where b is the size of the block, or 2 creates a completely new block with probability A/(n + A). Ana Daglis Clustering Data using the Infinite Relational Model September 4, / 29
7 Model Chinese Restaurant Process (CRP(A)) Ana Daglis Clustering Data using the Infinite Relational Model September 4, / 29
8 Model Chinese Restaurant Process (CRP(A)) 1 1+A A 1+A 0 Ana Daglis Clustering Data using the Infinite Relational Model September 4, / 29
9 Model Chinese Restaurant Process (CRP(A)) 1 2+A 1 2+A A 2+A Ana Daglis Clustering Data using the Infinite Relational Model September 4, / 29
10 Model Chinese Restaurant Process (CRP(A)) 1 3+A 2 3+A A 3+A Ana Daglis Clustering Data using the Infinite Relational Model September 4, / 29
11 Gibbs Sampling Methodology Gibbs Sampling Want: a sample from a multivariate distribution θ = (θ 1, θ 2,..., θ d ). Algorithm: 1 Initialize with θ = (θ (0) 1, θ(0) 2,..., θ(0) d ). 2 For i = 1, 2,..., n, Simulate θ (i) 1 from the conditional θ 1 (θ (i 1) 2,..., θ (i 1) d ) Simulate θ (i) 2 from the conditional θ 2 (θ (i) 1, θ(i 1) 3,..., θ (i 1) d )... Simulate θ (i) d from the conditional θ d (θ (i) 1, θ(i) 2,..., θ(i) d 1 ). 3 Discard the first k iterations and estimate the posterior distribution using (θ (k+1) 1, θ (k+1) 2,..., θ (k+1) ),..., (θ (n) 1, θ(n) 2,..., θ(n) d ). d Ana Daglis Clustering Data using the Infinite Relational Model September 4, / 29
12 Gibbs Sampling Methodology Gibbs Sampling We use the Gibbs sampling to infer the posterior distribution of z. The cluster assignments, z i, are iteratively sampled from their conditional distribution, P(z i = k z \i, X ) P(X z)p(z i = k z \i ), where z \i denotes all cluster assignments except z i. Ana Daglis Clustering Data using the Infinite Relational Model September 4, / 29
13 Gibbs Sampling Methodology Simulated Data We applied the Gibbs sampling algorithm to a simulated network with the following parameters: 96 nodes split into 6 blocks φ i,i = 0.85, for i = 1,...n φ i,j = 0.05, for i j a = b = 1 for uniform prior A = 1. (a) Simulated network Ana Daglis Clustering Data using the Infinite Relational Model September 4, / 29
14 Gibbs Sampling Results Simulated Data We applied the Gibbs sampling algorithm to a simulated network with the following parameters: 96 nodes split into 6 blocks φ i,i = 0.85, for i = 1,...n φ i,j = 0.05, for i j a = b = 1 for uniform prior A = 1. (b) Supplied network Ana Daglis Clustering Data using the Infinite Relational Model September 4, / 29
15 Gibbs Sampling Results Block structure obtained Ana Daglis Clustering Data using the Infinite Relational Model September 4, / 29
16 Gibbs Sampling Results Trace-plot of the number of blocks 6 Number of blocks Iteration Ana Daglis Clustering Data using the Infinite Relational Model September 4, / 29
17 Gibbs Sampling Results Gibbs Sampling Summary The algorithm fails to split the data into 6 clusters within iterations, and is stuck in five-cluster configuration for a long time. The main problem with the Gibbs sampler is that it is slow to converge, and it often becomes trapped in a local mode (5 blocks in this case). A possible improvement is the split-merge algorithm, which updates simultaneously a group of nodes and avoids these problems. Ana Daglis Clustering Data using the Infinite Relational Model September 4, / 29
18 Methodology Algorithm: 1 Select two distinct nodes, i and j, uniformly at random. 2 If i and j belong to the same cluster, split that cluster into two by assigning elements to either of the two clusters independently with equal probability. 3 If i and j belong to different clusters, merge those clusters. 4 Evaluate Metropolis-Hastings acceptance probability. If accepted, the new cluster assignment becomes the next step of the algorithm. Otherwise, the initial cluster assignment remains as the next state. [ a(z, z) = min 1, q(z z )P(z )L(X z ] ) q(z, z)p(z)l(x z) where q is proposal probability, P(z) prior, L(X z) likelihood. Ana Daglis Clustering Data using the Infinite Relational Model September 4, / 29
19 Methodology Ana Daglis Clustering Data using the Infinite Relational Model September 4, / 29
20 Methodology Ana Daglis Clustering Data using the Infinite Relational Model September 4, / 29
21 Methodology Ana Daglis Clustering Data using the Infinite Relational Model September 4, / 29
22 Methodology Ana Daglis Clustering Data using the Infinite Relational Model September 4, / 29
23 Methodology Ana Daglis Clustering Data using the Infinite Relational Model September 4, / 29
24 Methodology Ana Daglis Clustering Data using the Infinite Relational Model September 4, / 29
25 Results Gibbs Sampler + Split-Merge We applied the Gibbs sampler together with the split-merge algorithm to the earlier network. For every nine full Gibbs sampling scans, one split-merge step was used. The algorithm appropriately splits the data into six clusters, has short burn-in time and mixes well. Ana Daglis Clustering Data using the Infinite Relational Model September 4, / 29
26 Results Block structure obtained Ana Daglis Clustering Data using the Infinite Relational Model September 4, / 29
27 Results Trace-plot of the number of blocks 7 Number of blocks Iteration Ana Daglis Clustering Data using the Infinite Relational Model September 4, / 29
28 Future Work Future Work Assess the performance of the algorithms when the blocks significantly vary in size. Evaluate the complexities of the algorithms. Explore more advanced algorithms (such as the Restricted Gibbs Sampling Split-Merge). Ana Daglis Clustering Data using the Infinite Relational Model September 4, / 29
29 Future Work References Schmidt, M. N. and Mørup, M. (2013). Non-parametric Bayesian modeling of complex networks. IEEE Signal Processing Magazine, 30: Jain, S. and Neal, R. M. (2004). A split-merge Markov chain Monte Carlo procedure for the Dirichlet process mixture model. Journal of Computational and Graphical Statistics, 13: Ana Daglis Clustering Data using the Infinite Relational Model September 4, / 29
Bayesian Statistics Group 8th March Slice samplers. (A very brief introduction) The basic idea
Bayesian Statistics Group 8th March 2000 Slice samplers (A very brief introduction) The basic idea lacements To sample from a distribution, simply sample uniformly from the region under the density function
MCMC Methods for data modeling
MCMC Methods for data modeling Kenneth Scerri Department of Automatic Control and Systems Engineering Introduction 1. Symposium on Data Modelling 2. Outline: a. Definition and uses of MCMC b. MCMC algorithms
K-means and Hierarchical Clustering
K-means and Hierarchical Clustering Xiaohui Xie University of California, Irvine K-means and Hierarchical Clustering p.1/18 Clustering Given n data points X = {x 1, x 2,, x n }. Clustering is the partitioning
Quantitative Biology II!
Quantitative Biology II! Lecture 3: Markov Chain Monte Carlo! March 9, 2015! 2! Plan for Today!! Introduction to Sampling!! Introduction to MCMC!! Metropolis Algorithm!! Metropolis-Hastings Algorithm!!
Model-Based Clustering for Online Crisis Identification in Distributed Computing
Model-Based Clustering for Crisis Identification in Distributed Computing Dawn Woodard Operations Research and Information Engineering Cornell University with Moises Goldszmidt Microsoft Research 1 Outline
Convexization in Markov Chain Monte Carlo
in Markov Chain Monte Carlo 1 IBM T. J. Watson Yorktown Heights, NY 2 Department of Aerospace Engineering Technion, Israel August 23, 2011 Problem Statement MCMC processes in general are governed by non
An Efficient Model Selection for Gaussian Mixture Model in a Bayesian Framework
IEEE SIGNAL PROCESSING LETTERS, VOL. XX, NO. XX, XXX 23 An Efficient Model Selection for Gaussian Mixture Model in a Bayesian Framework Ji Won Yoon arxiv:37.99v [cs.lg] 3 Jul 23 Abstract In order to cluster
Semiparametric Mixed Effecs with Hierarchical DP Mixture
Semiparametric Mixed Effecs with Hierarchical DP Mixture R topics documented: April 21, 2007 hdpm-package........................................ 1 hdpm............................................ 2 hdpmfitsetup........................................
A Short History of Markov Chain Monte Carlo
A Short History of Markov Chain Monte Carlo Christian Robert and George Casella 2010 Introduction Lack of computing machinery, or background on Markov chains, or hesitation to trust in the practicality
Case Study IV: Bayesian clustering of Alzheimer patients
Case Study IV: Bayesian clustering of Alzheimer patients Mike Wiper and Conchi Ausín Department of Statistics Universidad Carlos III de Madrid Advanced Statistics and Data Mining Summer School 2nd - 6th
Nested Sampling: Introduction and Implementation
UNIVERSITY OF TEXAS AT SAN ANTONIO Nested Sampling: Introduction and Implementation Liang Jing May 2009 1 1 ABSTRACT Nested Sampling is a new technique to calculate the evidence, Z = P(D M) = p(d θ, M)p(θ
Markov chain Monte Carlo methods
Markov chain Monte Carlo methods (supplementary material) see also the applet http://www.lbreyer.com/classic.html February 9 6 Independent Hastings Metropolis Sampler Outline Independent Hastings Metropolis
Linear Modeling with Bayesian Statistics
Linear Modeling with Bayesian Statistics Bayesian Approach I I I I I Estimate probability of a parameter State degree of believe in specific parameter values Evaluate probability of hypothesis given the
Machine Learning. Unsupervised Learning. Manfred Huber
Machine Learning Unsupervised Learning Manfred Huber 2015 1 Unsupervised Learning In supervised learning the training data provides desired target output for learning In unsupervised learning the training
Analysis of Incomplete Multivariate Data
Analysis of Incomplete Multivariate Data J. L. Schafer Department of Statistics The Pennsylvania State University USA CHAPMAN & HALL/CRC A CR.C Press Company Boca Raton London New York Washington, D.C.
A Nonparametric Bayesian Approach to Detecting Spatial Activation Patterns in fmri Data
A Nonparametric Bayesian Approach to Detecting Spatial Activation Patterns in fmri Data Seyoung Kim, Padhraic Smyth, and Hal Stern Bren School of Information and Computer Sciences University of California,
CS281 Section 9: Graph Models and Practical MCMC
CS281 Section 9: Graph Models and Practical MCMC Scott Linderman November 11, 213 Now that we have a few MCMC inference algorithms in our toolbox, let s try them out on some random graph models. Graphs
Machine Learning and Data Mining. Clustering (1): Basics. Kalev Kask
Machine Learning and Data Mining Clustering (1): Basics Kalev Kask Unsupervised learning Supervised learning Predict target value ( y ) given features ( x ) Unsupervised learning Understand patterns of
Jing Gao 1, Feng Liang 1, Wei Fan 2, Chi Wang 1, Yizhou Sun 1, Jiawei i Han 1 University of Illinois, IBM TJ Watson.
Jing Gao 1, Feng Liang 1, Wei Fan 2, Chi Wang 1, Yizhou Sun 1, Jiawei i Han 1 University of Illinois, IBM TJ Watson Debapriya Basu Determine outliers in information networks Compare various algorithms
Unsupervised Learning: Clustering
Unsupervised Learning: Clustering Vibhav Gogate The University of Texas at Dallas Slides adapted from Carlos Guestrin, Dan Klein & Luke Zettlemoyer Machine Learning Supervised Learning Unsupervised Learning
Graphical Models, Bayesian Method, Sampling, and Variational Inference
Graphical Models, Bayesian Method, Sampling, and Variational Inference With Application in Function MRI Analysis and Other Imaging Problems Wei Liu Scientific Computing and Imaging Institute University
The Multi Stage Gibbs Sampling: Data Augmentation Dutch Example
The Multi Stage Gibbs Sampling: Data Augmentation Dutch Example Rebecca C. Steorts Bayesian Methods and Modern Statistics: STA 360/601 Module 8 1 Example: Data augmentation / Auxiliary variables A commonly-used
Approximate Bayesian Computation. Alireza Shafaei - April 2016
Approximate Bayesian Computation Alireza Shafaei - April 2016 The Problem Given a dataset, we are interested in. The Problem Given a dataset, we are interested in. The Problem Given a dataset, we are interested
Overview. Monte Carlo Methods. Statistics & Bayesian Inference Lecture 3. Situation At End Of Last Week
Statistics & Bayesian Inference Lecture 3 Joe Zuntz Overview Overview & Motivation Metropolis Hastings Monte Carlo Methods Importance sampling Direct sampling Gibbs sampling Monte-Carlo Markov Chains Emcee
SYDE Winter 2011 Introduction to Pattern Recognition. Clustering
SYDE 372 - Winter 2011 Introduction to Pattern Recognition Clustering Alexander Wong Department of Systems Design Engineering University of Waterloo Outline 1 2 3 4 5 All the approaches we have learned
Image analysis. Computer Vision and Classification Image Segmentation. 7 Image analysis
7 Computer Vision and Classification 413 / 458 Computer Vision and Classification The k-nearest-neighbor method The k-nearest-neighbor (knn) procedure has been used in data analysis and machine learning
A noninformative Bayesian approach to small area estimation
A noninformative Bayesian approach to small area estimation Glen Meeden School of Statistics University of Minnesota Minneapolis, MN 55455 glen@stat.umn.edu September 2001 Revised May 2002 Research supported
MCMC Diagnostics. Yingbo Li MATH Clemson University. Yingbo Li (Clemson) MCMC Diagnostics MATH / 24
MCMC Diagnostics Yingbo Li Clemson University MATH 9810 Yingbo Li (Clemson) MCMC Diagnostics MATH 9810 1 / 24 Convergence to Posterior Distribution Theory proves that if a Gibbs sampler iterates enough,
Machine Learning (BSMC-GA 4439) Wenke Liu
Machine Learning (BSMC-GA 4439) Wenke Liu 01-25-2018 Outline Background Defining proximity Clustering methods Determining number of clusters Other approaches Cluster analysis as unsupervised Learning Unsupervised
Clustering algorithms
Clustering algorithms Machine Learning Hamid Beigy Sharif University of Technology Fall 1393 Hamid Beigy (Sharif University of Technology) Clustering algorithms Fall 1393 1 / 22 Table of contents 1 Supervised
PIANOS requirements specifications
PIANOS requirements specifications Group Linja Helsinki 7th September 2005 Software Engineering Project UNIVERSITY OF HELSINKI Department of Computer Science Course 581260 Software Engineering Project
Clustering. Mihaela van der Schaar. January 27, Department of Engineering Science University of Oxford
Department of Engineering Science University of Oxford January 27, 2017 Many datasets consist of multiple heterogeneous subsets. Cluster analysis: Given an unlabelled data, want algorithms that automatically
Machine Learning (BSMC-GA 4439) Wenke Liu
Machine Learning (BSMC-GA 4439) Wenke Liu 01-31-017 Outline Background Defining proximity Clustering methods Determining number of clusters Comparing two solutions Cluster analysis as unsupervised Learning
Statistical Matching using Fractional Imputation
Statistical Matching using Fractional Imputation Jae-Kwang Kim 1 Iowa State University 1 Joint work with Emily Berg and Taesung Park 1 Introduction 2 Classical Approaches 3 Proposed method 4 Application:
Supervised vs. Unsupervised Learning
Clustering Supervised vs. Unsupervised Learning So far we have assumed that the training samples used to design the classifier were labeled by their class membership (supervised learning) We assume now
Sampling Table Configulations for the Hierarchical Poisson-Dirichlet Process
Sampling Table Configulations for the Hierarchical Poisson-Dirichlet Process Changyou Chen,, Lan Du,, Wray Buntine, ANU College of Engineering and Computer Science The Australian National University National
Modeling time series with hidden Markov models
Modeling time series with hidden Markov models Advanced Machine learning 2017 Nadia Figueroa, Jose Medina and Aude Billard Time series data Barometric pressure Temperature Data Humidity Time What s going
Mesh segmentation. Florent Lafarge Inria Sophia Antipolis - Mediterranee
Mesh segmentation Florent Lafarge Inria Sophia Antipolis - Mediterranee Outline What is mesh segmentation? M = {V,E,F} is a mesh S is either V, E or F (usually F) A Segmentation is a set of sub-meshes
Summary: A Tutorial on Learning With Bayesian Networks
Summary: A Tutorial on Learning With Bayesian Networks Markus Kalisch May 5, 2006 We primarily summarize [4]. When we think that it is appropriate, we comment on additional facts and more recent developments.
Computer vision: models, learning and inference. Chapter 10 Graphical Models
Computer vision: models, learning and inference Chapter 10 Graphical Models Independence Two variables x 1 and x 2 are independent if their joint probability distribution factorizes as Pr(x 1, x 2 )=Pr(x
ADAPTIVE METROPOLIS-HASTINGS SAMPLING, OR MONTE CARLO KERNEL ESTIMATION
ADAPTIVE METROPOLIS-HASTINGS SAMPLING, OR MONTE CARLO KERNEL ESTIMATION CHRISTOPHER A. SIMS Abstract. A new algorithm for sampling from an arbitrary pdf. 1. Introduction Consider the standard problem of
A Nonparametric Bayesian Approach to Modeling Overlapping Clusters
A Nonparametric Bayesian Approach to Modeling Overlapping Clusters Katherine A. Heller Gatsby Computational Neuroscience Unit University College London Zoubin Ghahramani Engineering Department University
Dynamic Thresholding for Image Analysis
Dynamic Thresholding for Image Analysis Statistical Consulting Report for Edward Chan Clean Energy Research Center University of British Columbia by Libo Lu Department of Statistics University of British
Digital Image Processing Laboratory: Markov Random Fields and MAP Image Segmentation
Purdue University: Digital Image Processing Laboratories Digital Image Processing Laboratory: Markov Random Fields and MAP Image Segmentation December, 205 Introduction This laboratory explores the use
Unsupervised Learning
Networks for Pattern Recognition, 2014 Networks for Single Linkage K-Means Soft DBSCAN PCA Networks for Kohonen Maps Linear Vector Quantization Networks for Problems/Approaches in Machine Learning Supervised
Hidden Markov Models in the context of genetic analysis
Hidden Markov Models in the context of genetic analysis Vincent Plagnol UCL Genetics Institute November 22, 2012 Outline 1 Introduction 2 Two basic problems Forward/backward Baum-Welch algorithm Viterbi
Clustering Lecture 5: Mixture Model
Clustering Lecture 5: Mixture Model Jing Gao SUNY Buffalo 1 Outline Basics Motivation, definition, evaluation Methods Partitional Hierarchical Density-based Mixture model Spectral methods Advanced topics
Monte Carlo for Spatial Models
Monte Carlo for Spatial Models Murali Haran Department of Statistics Penn State University Penn State Computational Science Lectures April 2007 Spatial Models Lots of scientific questions involve analyzing
Expectation Maximization. Machine Learning 10701/15781 Carlos Guestrin Carnegie Mellon University
Expectation Maximization Machine Learning 10701/15781 Carlos Guestrin Carnegie Mellon University April 10 th, 2006 1 Announcements Reminder: Project milestone due Wednesday beginning of class 2 Coordinate
ECE521: Week 11, Lecture March 2017: HMM learning/inference. With thanks to Russ Salakhutdinov
ECE521: Week 11, Lecture 20 27 March 2017: HMM learning/inference With thanks to Russ Salakhutdinov Examples of other perspectives Murphy 17.4 End of Russell & Norvig 15.2 (Artificial Intelligence: A Modern
Probabilistic Graphical Models
Probabilistic Graphical Models Lecture 17 EM CS/CNS/EE 155 Andreas Krause Announcements Project poster session on Thursday Dec 3, 4-6pm in Annenberg 2 nd floor atrium! Easels, poster boards and cookies
GiRaF: a toolbox for Gibbs Random Fields analysis
GiRaF: a toolbox for Gibbs Random Fields analysis Julien Stoehr *1, Pierre Pudlo 2, and Nial Friel 1 1 University College Dublin 2 Aix-Marseille Université February 24, 2016 Abstract GiRaF package offers
Issues in MCMC use for Bayesian model fitting. Practical Considerations for WinBUGS Users
Practical Considerations for WinBUGS Users Kate Cowles, Ph.D. Department of Statistics and Actuarial Science University of Iowa 22S:138 Lecture 12 Oct. 3, 2003 Issues in MCMC use for Bayesian model fitting
Markov Chain Monte Carlo (part 1)
Markov Chain Monte Carlo (part 1) Edps 590BAY Carolyn J. Anderson Department of Educational Psychology c Board of Trustees, University of Illinois Spring 2018 Depending on the book that you select for
Latent Variable Models and Expectation Maximization
Latent Variable Models and Expectation Maximization Oliver Schulte - CMPT 726 Bishop PRML Ch. 9 2 4 6 8 1 12 14 16 18 2 4 6 8 1 12 14 16 18 5 1 15 2 25 5 1 15 2 25 2 4 6 8 1 12 14 2 4 6 8 1 12 14 5 1 15
1 Methods for Posterior Simulation
1 Methods for Posterior Simulation Let p(θ y) be the posterior. simulation. Koop presents four methods for (posterior) 1. Monte Carlo integration: draw from p(θ y). 2. Gibbs sampler: sequentially drawing
RJaCGH, a package for analysis of
RJaCGH, a package for analysis of CGH arrays with Reversible Jump MCMC 1. CGH Arrays: Biological problem: Changes in number of DNA copies are associated to cancer activity. Microarray technology: Oscar
Clustering K-means. Machine Learning CSEP546 Carlos Guestrin University of Washington February 18, Carlos Guestrin
Clustering K-means Machine Learning CSEP546 Carlos Guestrin University of Washington February 18, 2014 Carlos Guestrin 2005-2014 1 Clustering images Set of Images [Goldberger et al.] Carlos Guestrin 2005-2014
A GENERAL GIBBS SAMPLING ALGORITHM FOR ANALYZING LINEAR MODELS USING THE SAS SYSTEM
A GENERAL GIBBS SAMPLING ALGORITHM FOR ANALYZING LINEAR MODELS USING THE SAS SYSTEM Jayawant Mandrekar, Daniel J. Sargent, Paul J. Novotny, Jeff A. Sloan Mayo Clinic, Rochester, MN 55905 ABSTRACT A general
Generative and discriminative classification techniques
Generative and discriminative classification techniques Machine Learning and Category Representation 2014-2015 Jakob Verbeek, November 28, 2014 Course website: http://lear.inrialpes.fr/~verbeek/mlcr.14.15
Machine Learning A W 1sst KU. b) [1 P] Give an example for a probability distributions P (A, B, C) that disproves
![Machine Learning A W 1sst KU. b) [1 P] Give an example for a probability distributions P (A, B, C) that disproves](/thumbs/93/111796408.jpg "Machine Learning A W 1sst KU. b) [1 P] Give an example for a probability distributions P (A, B, C) that disproves")
Machine Learning A 708.064 11W 1sst KU Exercises Problems marked with * are optional. 1 Conditional Independence I [2 P] a) [1 P] Give an example for a probability distribution P (A, B, C) that disproves
Bayesian Modelling with JAGS and R
Bayesian Modelling with JAGS and R Martyn Plummer International Agency for Research on Cancer Rencontres R, 3 July 2012 CRAN Task View Bayesian Inference The CRAN Task View Bayesian Inference is maintained
Estimation of Bilateral Connections in a Network: Copula vs. Maximum Entropy
Estimation of Bilateral Connections in a Network: Copula vs. Maximum Entropy Pallavi Baral and Jose Pedro Fique Department of Economics Indiana University at Bloomington 1st Annual CIRANO Workshop on Networks
Machine Learning. B. Unsupervised Learning B.1 Cluster Analysis. Lars Schmidt-Thieme
Machine Learning B. Unsupervised Learning B.1 Cluster Analysis Lars Schmidt-Thieme Information Systems and Machine Learning Lab (ISMLL) Institute for Computer Science University of Hildesheim, Germany
From Bayesian Analysis of Item Response Theory Models Using SAS. Full book available for purchase here.
From Bayesian Analysis of Item Response Theory Models Using SAS. Full book available for purchase here. Contents About this Book...ix About the Authors... xiii Acknowledgments... xv Chapter 1: Item Response
Learning stick-figure models using nonparametric Bayesian priors over trees
Learning stick-figure models using nonparametric Bayesian priors over trees Edward W. Meeds, David A. Ross, Richard S. Zemel, and Sam T. Roweis Department of Computer Science University of Toronto {ewm,
Machine Learning
Machine Learning 10-601 Tom M. Mitchell Machine Learning Department Carnegie Mellon University April 1, 2019 Today: Inference in graphical models Learning graphical models Readings: Bishop chapter 8 Bayesian
Bayesian Segmentation and Motion Estimation in Video Sequences using a Markov-Potts Model
Bayesian Segmentation and Motion Estimation in Video Sequences using a Markov-Potts Model Patrice BRAULT and Ali MOHAMMAD-DJAFARI LSS, Laboratoire des Signaux et Systemes, CNRS UMR 8506, Supelec, Plateau
Unsupervised Learning. Clustering and the EM Algorithm. Unsupervised Learning is Model Learning
Unsupervised Learning Clustering and the EM Algorithm Susanna Ricco Supervised Learning Given data in the form < x, y >, y is the target to learn. Good news: Easy to tell if our algorithm is giving the
Fitting Social Network Models Using the Varying Truncation S. Truncation Stochastic Approximation MCMC Algorithm
Fitting Social Network Models Using the Varying Truncation Stochastic Approximation MCMC Algorithm May. 17, 2012 1 This talk is based on a joint work with Dr. Ick Hoon Jin Abstract The exponential random
Mixture Models and the EM Algorithm
Mixture Models and the EM Algorithm Padhraic Smyth, Department of Computer Science University of California, Irvine c 2017 1 Finite Mixture Models Say we have a data set D = {x 1,..., x N } where x i is
Bayesian Estimation for Skew Normal Distributions Using Data Augmentation
The Korean Communications in Statistics Vol. 12 No. 2, 2005 pp. 323-333 Bayesian Estimation for Skew Normal Distributions Using Data Augmentation Hea-Jung Kim 1) Abstract In this paper, we develop a MCMC
Clustering K-means. Machine Learning CSEP546 Carlos Guestrin University of Washington February 18, Carlos Guestrin
Clustering K-means Machine Learning CSEP546 Carlos Guestrin University of Washington February 18, 2014 Carlos Guestrin 2005-2014 1 Clustering images Set of Images [Goldberger et al.] Carlos Guestrin 2005-2014
Network-based auto-probit modeling for protein function prediction
Network-based auto-probit modeling for protein function prediction Supplementary material Xiaoyu Jiang, David Gold, Eric D. Kolaczyk Derivation of Markov Chain Monte Carlo algorithm with the GO annotation
Discussion on Bayesian Model Selection and Parameter Estimation in Extragalactic Astronomy by Martin Weinberg
Discussion on Bayesian Model Selection and Parameter Estimation in Extragalactic Astronomy by Martin Weinberg Phil Gregory Physics and Astronomy Univ. of British Columbia Introduction Martin Weinberg reported
Clustering. Supervised vs. Unsupervised Learning
Clustering Supervised vs. Unsupervised Learning So far we have assumed that the training samples used to design the classifier were labeled by their class membership (supervised learning) We assume now
Clustering web search results
Clustering K-means Machine Learning CSE546 Emily Fox University of Washington November 4, 2013 1 Clustering images Set of Images [Goldberger et al.] 2 1 Clustering web search results 3 Some Data 4 2 K-means
Introduction to Machine Learning CMU-10701
Introduction to Machine Learning CMU-10701 Clustering and EM Barnabás Póczos & Aarti Singh Contents Clustering K-means Mixture of Gaussians Expectation Maximization Variational Methods 2 Clustering 3 K-
GT "Calcul Ensembliste"
GT "Calcul Ensembliste" Beyond the bounded error framework for non linear state estimation Fahed Abdallah Université de Technologie de Compiègne 9 Décembre 2010 Fahed Abdallah GT "Calcul Ensembliste" 9
Package beast. March 16, 2018
Type Package Package beast March 16, 2018 Title Bayesian Estimation of Change-Points in the Slope of Multivariate Time-Series Version 1.1 Date 2018-03-16 Author Maintainer Assume that
Slice sampler algorithm for generalized Pareto distribution
Slice sampler algorithm for generalized Pareto distribution Mohammad Rostami, Mohd Bakri Adam Yahya, Mohamed Hisham Yahya, Noor Akma Ibrahim Abstract In this paper, we developed the slice sampler algorithm
SGN (4 cr) Chapter 11
SGN-41006 (4 cr) Chapter 11 Clustering Jussi Tohka & Jari Niemi Department of Signal Processing Tampere University of Technology February 25, 2014 J. Tohka & J. Niemi (TUT-SGN) SGN-41006 (4 cr) Chapter
Modeling Dyadic Data with Binary Latent Factors
Modeling Dyadic Data with Binary Latent Factors Edward Meeds Department of Computer Science University of Toronto ewm@cs.toronto.edu Radford Neal Department of Computer Science University of Toronto radford@cs.toronto.edu
Training Restricted Boltzmann Machines with Multi-Tempering: Harnessing Parallelization
Training Restricted Boltzmann Machines with Multi-Tempering: Harnessing Parallelization Philemon Brakel, Sander Dieleman, and Benjamin Schrauwen Department of Electronics and Information Systems, Ghent
Recap: The E-M algorithm. Biostatistics 615/815 Lecture 22: Gibbs Sampling. Recap - Local minimization methods
Recap: The E-M algorithm Biostatistics 615/815 Lecture 22: Gibbs Sampling Expectation step (E-step) Given the current estimates of parameters λ (t), calculate the conditional distribution of latent variable
Sampling informative/complex a priori probability distributions using Gibbs sampling assisted by sequential simulation
Sampling informative/complex a priori probability distributions using Gibbs sampling assisted by sequential simulation Thomas Mejer Hansen, Klaus Mosegaard, and Knud Skou Cordua 1 1 Center for Energy Resources
Package hergm. R topics documented: January 10, Version Date
Version 3.1-0 Date 2016-09-22 Package hergm January 10, 2017 Title Hierarchical Exponential-Family Random Graph Models Author Michael Schweinberger [aut, cre], Mark S.
Clustering: Classic Methods and Modern Views
Clustering: Classic Methods and Modern Views Marina Meilă University of Washington mmp@stat.washington.edu June 22, 2015 Lorentz Center Workshop on Clusters, Games and Axioms Outline Paradigms for clustering
10601 Machine Learning. Hierarchical clustering. Reading: Bishop: 9-9.2
161 Machine Learning Hierarchical clustering Reading: Bishop: 9-9.2 Second half: Overview Clustering - Hierarchical, semi-supervised learning Graphical models - Bayesian networks, HMMs, Reasoning under
Warped Mixture Models
Warped Mixture Models Tomoharu Iwata, David Duvenaud, Zoubin Ghahramani Cambridge University Computational and Biological Learning Lab March 11, 2013 OUTLINE Motivation Gaussian Process Latent Variable
Image Restoration using Markov Random Fields
Image Restoration using Markov Random Fields Based on the paper Stochastic Relaxation, Gibbs Distributions and Bayesian Restoration of Images, PAMI, 1984, Geman and Geman. and the book Markov Random Field
Probabilistic Graphical Models
10-708 Probabilistic Graphical Models Homework 4 Due Apr 27, 12:00 noon Submission: Homework is due on the due date at 12:00 noon. Please see course website for policy on late submission. You must submit
Chapter 1. Introduction
Chapter 1 Introduction A Monte Carlo method is a compuational method that uses random numbers to compute (estimate) some quantity of interest. Very often the quantity we want to compute is the mean of
Coherent Motion Segmentation in Moving Camera Videos using Optical Flow Orientations - Supplementary material
Coherent Motion Segmentation in Moving Camera Videos using Optical Flow Orientations - Supplementary material Manjunath Narayana narayana@cs.umass.edu Allen Hanson hanson@cs.umass.edu University of Massachusetts,
Debugging MCMC Code. Charles J. Geyer. April 15, 2017
Debugging MCMC Code Charles J. Geyer April 15, 2017 1 Introduction This document discusses debugging Markov chain Monte Carlo code using the R contributed package mcmc (Version 0.9-5) for examples. It
ALTERNATIVE METHODS FOR CLUSTERING
ALTERNATIVE METHODS FOR CLUSTERING K-Means Algorithm Termination conditions Several possibilities, e.g., A fixed number of iterations Objects partition unchanged Centroid positions don t change Convergence
Clustering: K-means and Kernel K-means
Clustering: K-means and Kernel K-means Piyush Rai Machine Learning (CS771A) Aug 31, 2016 Machine Learning (CS771A) Clustering: K-means and Kernel K-means 1 Clustering Usually an unsupervised learning problem
10701 Machine Learning. Clustering
171 Machine Learning Clustering What is Clustering? Organizing data into clusters such that there is high intra-cluster similarity low inter-cluster similarity Informally, finding natural groupings among
Efficient Markov Chain Monte Carlo Algorithms For MIMO and ISI channels
Efficient Markov Chain Monte Carlo Algorithms For MIMO and ISI channels Rong-Hui Peng Department of Electrical and Computer Engineering University of Utah /7/6 Summary of PhD work Efficient MCMC algorithms
Deep Generative Models Variational Autoencoders
Deep Generative Models Variational Autoencoders Sudeshna Sarkar 5 April 2017 Generative Nets Generative models that represent probability distributions over multiple variables in some way. Directed Generative