STAT 598L Probabilistic Graphical Models. Instructor: Sergey Kirshner. Introduction

Transcription

1 STAT 598L Probabilistic Graphical Models Instructor: Sergey Kirshner Introduction

2 You = 2

3 General Reasoning Scheme Data Conclusion Observations 3

4 General Reasoning Scheme Inference Learning αβγ,,,... Data Model Conclusion Observations 4

5 Curse of Dimensionality for Binary Data d-variate binary multinomial distribution Need 2 d -1 free parameters for the general case 5

6 Dimensionality reduction The true process has (hopefully sparse) structure! Need to find it or use a computationally efficient approximation Examples Principal component analysis (PCA) Independent component analysis (ICA) Manifold learning Sparse learning (LASSO) 6

7 Fundamental Questions Representation How to encode the domain knowledge? What does the model capture? Inference How to answer queries with the model? Unknown given observed Learning What model is the right one? Structure (functional form) Parameters 7

8 Graphical Models in a Nutshell Graphical models are a marriage between probability theory and graph theory. Michael I. Jordan Language to represent complex systems 8

9 Marriage Between Probability and Graph Theories Graph serves as the skeleton for the underlying probabilistic model Structure corresponds to conditional independence relations Probability theory describes the relationship between the units of the model, random variables Key idea: modularity Complex system described with smaller parts Graphs describe how the parts connect Probability theory describes how basic units interact within parts and how parts interact between themselves 9

10 Avoiding the Curse 2 6-1=31 free parameters X 1 X 2 X 3 X 4 X 5 X 6 P

11 Avoiding the Curse Use compact representations 2 6-1=31 free parameters Product of conditional distributions Product of potential functions 11

12 Directed Graphical Models (Bayesian Networks) x 1 x 2 Impose conditional independence relations x 6 x 3 Represented by a directed acyclic graph (DAG) x 5 x =18 free parameters 12

13 Undirected Graphical Models (MRFs) x 1 x 2 Impose conditional independence relations x 6 x 3 Represented by an undirected graph x 5 x =14 free parameters STAT 598L: Probabilistic Graphical Models (Introduction) 13

14 Graph Separation = Conditional Independence 14

15 Advantages Conditional independence relations modeled with a graph More conditional independence relations captured = fewer free parameters Intuitive way to introduce prior knowledge Inference and learning algorithms are performed as algorithms on the underlying dependence graph Complexity of the algorithms depends on the structure of the graph 15

16 Speech Recognition 16

17 Modeling Sensor Data [Guestrin et al] Slide adopted from Carlos Guestrin s PGM course 17

18 Multi-Site Rainfall Time Series Modeling 18

19 Multi-Site Rainfall Time Series Modeling 19

20 Rain Generating Process day 1 day 2 day 3 day T R 1 R 2 R 3 R T March 2,

21 Fundamental Questions Representation How to encode the domain knowledge? What does the model capture? Inference How to answer queries with the model? Unknown given observed Learning What model is the right one? Structure (functional form) Parameters 21

22 Administrative Part Course page: Schedule, announcements, homeworks Blackboard Vista Grades, solutions, slides (for now), discussion (?) Office Hours, Mondays 2:30-3:30pm, HAAS 118 works too No TA, so feedback is limited by my available time Talk to your classmates 22

23 Grading Homeworks (4-6) (25%) Fairly long Midterm (25%) Covers predetermined topics Project (40%) Course-long, lets you get your feet wet Class participation (10%) Ask questions, present in class Suggestions? 23

24 Lectures Roughly 2/3 of a course devoted to essential topics for graphical models Directed graphical models, undirected graphical models, exact inference, structure and parameter learning, several well-known specific instances of such models, a few applications Additional topics presented by the students or by the instructor Approximate inference, influence diagrams, etc. Project presentations 24

25 Questions Questions, concerns, feedback (please!) Sergey Kirshner, 25