18 October, 2013 MVA ENS Cachan. Lecture 6: Introduction to graphical models Iasonas Kokkinos
|
|
- Emerald Austin
- 5 years ago
- Views:
Transcription
1 Machine Learning for Computer Vision 1 18 October, 2013 MVA ENS Cachan Lecture 6: Introduction to graphical models Iasonas Kokkinos Iasonas.kokkinos@ecp.fr Center for Visual Computing Ecole Centrale Paris Galen Group INRIA-Saclay
2 2 Lecture outline Introduction Graphs and computation Graphical models Chain-structured graphical models Tree-structured graphical models
3 Problem 1: measuring the length of a queue 3 while only complaining to your closest neighbors
4 Problem 2: two queues 4
5 Problem N: N queues 5
6 6 Lecture outline Introduction Computation on graphs Graphical models Chain-structured graphical models Tree-structured graphical models
7 Graphical models 7 Blend of graph theory and probability Graph nodes: random variables Graph edges: relationships Directed graphs: Bayesian Networks Undirected graphs: Markov Random Fields
8 Bayesian Networks 8 Directed Acyclic Graph Nodes - : random variables Edges - : dependencies Leave from `parents, arrive at children Distribution of each child conditioned on parent: Joint probability distribution: Bayesian network for Brownian motion:
9 Conditional independencies 9 We can trivially obtain a bayesian network for a probability distribution Fully connected graph! Bayesian net Non-trivial cases: some conditional independence Main interest: exploit independencies for training & inference
10 A Bayesian Network-based Expert System 10 `Asia Network A: trip to Asia T: turbeculosis S: smoking L: Lung Cancer B: Bronchitis E: Turbeculosis/Lung Cancer X-ray results D: Dyspnea Given: X-rays, Dyspnea, patient went to Asia, patient smokes Wanted: posterior probability of Bronchitis
11 Reading Conditional Independence from a BN Graph 11 Big Ben Clock 1 Clock 2 State 1 State 2 State 3 Bicycle Motorcycle Cycle `Explaining away
12 12 Lecture outline Introduction Chain-structured graphical models Hidden Markov Models Kalman Filter Tree-structured graphical models
13 Hidden variables 13 Mixture models: combine simple distributions into complex one Mixture Models Data clustering: side-product of parameter estimation
14 Image clustering - segmentation 14 Observations are not independent (Consecutive points: common region)
15 Dependent Hidden variables 15 Mixture Models Hiden Markov Models Introduce dependencies between hidden variables First-order Markov Model M x M transition matrix
16 Hidden Markov Models 16 hidden states Dependent sequence of observations observed process Conditioned on state sequence, observations become independent Parameters Observation probabilities discrete continuous Transition probabilities Starting probabilities Estimation: Expectation Maximization
17 HMMs & Speech Recognition 17 Input: Air pressure signal Output: words/phrases Probabilistic model: signal given word Hidden Variables: phonemes (/ae/,/ey/,/z/ ) phone subunits Hidden variables: not independent Show,shoe,she,sell,sea, Sv? (Sven,svelte,..) Sb?
18 18 Lecture outline Introduction Chain-structured graphical models Hidden Markov Models Kalman Filter Tree-structured graphical models Loopy graphical models
19 Detection vs. Tracking 19 t=1 t=2 t=20 t=21
20 Detection vs. Tracking 20 t=1 t=2 t=20 t=21 Detection: each frame independently
21 Detection vs. Tracking 21 t=1 t=2 t=20 t=21 Tracking with dynamics: combine image measurements with our expectation of the object s motion. Restrict search for the object Measurement noise is reduced by trajectory smoothness.
22 Tracking 22 Input: Series of Images Output: Estimate of object s motion/action/gesture Hidden Variables (`State ) Time-dependent Assume linear dynamics State, X Observations, Y
23 Simple Linear Dynamics 23 Brownian Motion Constant Velocity Time Time Slide credit: E. Sudderth
24 Linear Dynamical Systems 24 Noise in State, noise in Observations hidden states observed process Slide credit: E. Sudderth
25 Inference Tasks 25 Prediction Filtering (Kalman Filter) Smoothing Slide credit: E. Sudderth
26 Lecture outline 26 Introduction Chain-structured graphical models Tree-structured graphical models Pictorial Structures Max/Sum-Product algorithm
27 Problem N: N queues 27
28 28 Machine Learning for Computer Vision Lecture 6 Part-based Models x1 x2 x3 x4 x5
29 29 Machine Learning for Computer Vision Lecture 6 Part-based Models x1 x2 x3 x4 x5
30 Unary terms (appearance model) 30 Consider a patch of the image around the location of the part Construct statistical model for appearance at part location Filterbank responses SIFT features
31 Components of a part-based model: structure Pairwise terms (deformation model) 31 Deformable objects as physical systems Masses: driven towards good image locations Springs: enforce relations among parts Star models (tree-structured models) Pick single part as root (e.g. the nose) Model other parts (eyes, mouth corners) w.r.t. root Probability of configuration Unary Terms Pairwise Terms How can we maximize over L?
32 32 Machine Learning for Computer Vision Lecture 6 Deformable Part Models (DPMs) S(x) = P X x1 x2 x3 Up (xp ) + Bp (x, xp ) p=1 x5 x4 Local appearance Up (xp ) = hwp, H(xp )i Pairwise compatibility Bp (x, xp ) = Object score P X S(x) = p=1 (h hp h p )2 (v 0 0 max [U (x ) + B (x, x )] p p 0 x vp v p )2
33 Lecture outline 33 Introduction Chain-structured graphical models Tree-structured graphical models Pictorial Structures Max/Sum-Product algorithm
34 Inference on tree-structured Graphs 34 Assume we want to find the most likely value of Brute-force maximization: Exploit factorization
35 Exploiting the factorization 35
36 Exploiting the factorization 36
37 Max-Product algorithm 37 Distributed computation on a graph `message-passing Message from node i to node j `Given all I know from the rest, this is where you should be Beliefs: At convergence
38 Sum-Product algorithm 38 Distributed computation on a graph `message-passing Message from node i to node j `Given all I know from the rest, this is where you should be Beliefs: At convergence
39 Max/Sum-Product algorithms 39 Asychronous inference Local computations Guaranteed optimality for tree-structured graphs Equivalent algorithms: Tracking: Kalman Filtering (Sum-Product) HMMs: Alpha-Beta (Sum-Product) Coding/HMMs: Viterbi Algorithm (Max-Product) Bayesian Nets: Belief Propagation (Sum-Product) Max Product: Dynamic Programming
40 40 Machine Learning for Computer Vision Lecture 6 Deformable Part Models (DPMs) S(x) = P X x1 x2 x3 Up (xp ) + Bp (x, xp ) p=1 x5 x4 Local appearance Up (xp ) = hwp, H(xp )i Pairwise compatibility Bp (x, xp ) = Object score P X S(x) = p=1 (h hp h p )2 (v 0 0 max [U (x ) + B (x, x )] p p 0 x vp v p )2
41 Pictorial Structures 41 Graphical model Nodes: part locations Edges: dependencies - relative locations Object localization: maximization with respect to Unary terms Pairwise terms Message Passing: Efficient Implementation: Felzenszwalb & Huttenlocher, CVPR 2001/IJCV 05
42 42
43 Generalized Distance Transforms- 1D 43 Plot min x1 {m 1 (x 1 )+ (x 1,x 2 )} as function of x 2 (x 1 = a, x 2 ) = 2 (x 2 a) 2 2 (x 2 b) 2 m 1 (x 1 = a) a b m 1 (x 1 = b) x 1 x 2
44 GDT- 1D 44 Plot min x1 {m 1 (x 1 )+ (x 1,x 2 )} as function of x 2 x 1 x 2 For each x 2 Finding min over x 1 is equivalent finding minimum over set of offset parabolas Lower envelope computed in O(h) rather than O(h 2 ) via gen. distance transform
45 GDT- 1D 45 Plot min x1 {m 1 (x 1 )+ (x 1,x 2 )} as function of x 2 x 1 x 2 For each x 2 Finding min over x 1 is equivalent finding minimum over set of offset parabolas Lower envelope computed in O(h) rather than O(h 2 ) via gen. distance transform
46 Detection Results 46
47 Part-based models + discriminative framework 47 P. Felzenszwalb, D. McAllester, D. Ramanan`A Discriminatively Trained, Multiscale, Deformable Part Model CVPR 2008 Linear classifier gives individual part cost Gaussian-like cost for spatial offsets Learn cost for parts and deformations Discriminative framework State-of-the-art, 2008
48 Discriminative Training of DPMs Training data consists of images with labeled bounding boxes. Need to learn the model structure, filters and deformation costs. 48 Training
49 Multiple Instance Learning 49 Typical Learning Multiple Instance Learning Positive bag: at least one instance should be positive Negative bag: no instance should be positive Slide Credit: K. Grauman
50 50 Score function for DPM detection data term spatial prior score(p 0,...,p n )= n F i (H, p i ) n d i (dx 2 i,dy 2 i ) i=0 i=1 displacements filters deformation parameters score(z) = (H, z) concatenation filters and deformation parameters concatenation of HOG features and part displacement features
51 51 SVM training for DPMs Classifiers that score an example x using f (x) = max z Z(x) (x, z) β are model parameters z are latent values Training data D =(x 1,y 1,..., x n,y n ) y i { 1, 1} We would like to find β such that: y i f (x i ) > 0 Minimize L D ( )= C n i=1 max(0, 1 y i f (x i ))
52 52 Reduce to Latent SVM training problem Annotation & Latent Variables Positive example specifies some z should have high score Bounding box defines range of root locations - Parts can be anywhere - This defines Z(x) P.F. Felzenszwalb, R. B. Girshick, D. Ramanan and D. A. McAllester, a discriminatively trained multiscale deformable part model, IEEE PAMI, 2010
53 53 Latent SVM training L D ( )= C n i=1 max(0, 1 y i f (x i )) Convex if we fix z for positive examples Optimization: - Initialize β and iterate: - Pick best z for each positive example - Optimize β via gradient descent with data-mining
54 Appendix: Inference & Learning for Chain-structured GMs 54 Discrete State: Hidden Markov Models Forward-Backward Viterbi Learning Continuous State Kalman Filter Particle Filters
55 Hidden Markov Models 55 hidden states observed process Dynamics take simple form based on hidden states Process: word utterance Hidden States: phonemes Observed: speech signal Mixture Models HMMs
56 Three Main problems 56 Likelihood Evaluation Inference Parameter Estimation
57 Three Algorithms 57 Likelihood Evaluation (Forward-Backward) Inference (Viterbi) Parameter Estimation (EM/Baum-Welch)
58 Likelihood Evaluation 58 Using Marginalization Complexity: where N is the number of states
59 Exploiting the independencies 59 Rewrite likelihood as Introduce Observe that
60 Recursion for α 60
61 Recursion for α 61 Forward in time
62 Rewrite likelihood 62 Gain in computation: Brute force: Recursive:
63 Backward Variables β 63 Similar recursion holds: Backward in time:
64 64 Discrete State: Hidden Markov Models Forward-Backward Viterbi Learning Continuous State Kalman Filter Particle Filters
65 Three Main problems 65 Likelihood Evaluation (Forward-backward) Inference (Viterbi) Parameter Estimation (EM)
66 Posterior for a single state 66 Bayes optimal decision.. Introduce Based on Bayes rule Can rewrite using α,β
67 γ=αβ 67 So
68 Optimal State Sequence 68 Different from concatenation of optimal states Viterbi Algorithm Exploit Decomposable cost function Dynamic Programming
69 69 Exploit conditional independence
70 Viterbi Algorithm: recursion 70 Introduce Express recursively:
71 Viterbi Algorithm 71 Initialize Recurse Finalize Backtrack
72 72 Discrete State: Hidden Markov Models Forward-Backward Viterbi Learning Continuous State Kalman Filter Particle Filters
73 Three Main problems 73 Likelihood Evaluation (Forward-backward) Inference (Viterbi) Parameter Estimation (EM)
74 Full observation log-likelihood 74 Hidden Variables: State Sequence Full observations: Observations + HVs. Full observation likelihood Full Observation log-likelihood:
75 Parameter Estimates for Full Observations 75
76 Expectation Maximization 76 E-step: Expectation of Full-observation log-likelihood M-step: Maximization w.r.t parameters
77 E-step 77 Use current parameter estimate Run forward-backward algorithm Form γ Plug in full-observation log-likelihood
78 Posterior on joint states 78
79 M-step 79 Replace hidden variables with their expectations
80 5 th Lecture Layout 80 Discrete State: Hidden Markov Models Forward-Backward Viterbi Learning Continuous State Kalman Filter Particle Filters
81 Linear Dynamical Systems 81 Noise in State, noise in Observations hidden states observed process Identical dependencies among random variables But now integrals instead of summations All distributions are Gaussian
82 Kalman Filter Recursion 82 State estimate As in HMMs, come up with a recursion State estimate = prediction x correction All distributions are Gaussian: analytic expressions
83 Prediction distribution 83 Write in terms of previous state posterior
84 Posterior distribution 84 Posterior distribution Consider joint distribution of state and observation
85 Joint distribution of state and observation 85 We have distribution for x, need to consider y So
86 Posterior distribution on state 86 Condition on new observation y
87 1D Discrete-time Brownian motion 87 Dynamical model: Deterministic system driven by random excitation Input: White Gaussian Noise Output: Brownian motion Realization of input: Realization of output:
88 Example: Randomly Drifting Points 88
89 2D Brownian motion as a model for natural images 89 2D White Gaussian Noise 2D BM (stochastic fractal)
90 Generative model for Brownian motion 90 Synthesis equation: Coefficients Basis 2D: Fourier Synthesis Equation: use sinusoidals as signal basis PCA: `Customized basis for stochastic process (Karhunen-Loeve) PCA on Brownian motion: Sinusoidal basis
91 Random Variable Dependencies 91 Stochastic process: sequence of RVs Independent RVs (WGN) Dependent RVs (Brownian Motion) (1 st -order Markov) 2D:
22 October, 2012 MVA ENS Cachan. Lecture 5: Introduction to generative models Iasonas Kokkinos
Machine Learning for Computer Vision 1 22 October, 2012 MVA ENS Cachan Lecture 5: Introduction to generative models Iasonas Kokkinos Iasonas.kokkinos@ecp.fr Center for Visual Computing Ecole Centrale Paris
More informationPart-Based Models for Object Class Recognition Part 3
High Level Computer Vision! Part-Based Models for Object Class Recognition Part 3 Bernt Schiele - schiele@mpi-inf.mpg.de Mario Fritz - mfritz@mpi-inf.mpg.de! http://www.d2.mpi-inf.mpg.de/cv ! State-of-the-Art
More informationObject Detection with Discriminatively Trained Part Based Models
Object Detection with Discriminatively Trained Part Based Models Pedro F. Felzenszwelb, Ross B. Girshick, David McAllester and Deva Ramanan Presented by Fabricio Santolin da Silva Kaustav Basu Some slides
More informationECE521: Week 11, Lecture March 2017: HMM learning/inference. With thanks to Russ Salakhutdinov
ECE521: Week 11, Lecture 20 27 March 2017: HMM learning/inference With thanks to Russ Salakhutdinov Examples of other perspectives Murphy 17.4 End of Russell & Norvig 15.2 (Artificial Intelligence: A Modern
More informationD-Separation. b) the arrows meet head-to-head at the node, and neither the node, nor any of its descendants, are in the set C.
D-Separation Say: A, B, and C are non-intersecting subsets of nodes in a directed graph. A path from A to B is blocked by C if it contains a node such that either a) the arrows on the path meet either
More informationObject Recognition Using Pictorial Structures. Daniel Huttenlocher Computer Science Department. In This Talk. Object recognition in computer vision
Object Recognition Using Pictorial Structures Daniel Huttenlocher Computer Science Department Joint work with Pedro Felzenszwalb, MIT AI Lab In This Talk Object recognition in computer vision Brief definition
More informationPart-Based Models for Object Class Recognition Part 2
High Level Computer Vision Part-Based Models for Object Class Recognition Part 2 Bernt Schiele - schiele@mpi-inf.mpg.de Mario Fritz - mfritz@mpi-inf.mpg.de https://www.mpi-inf.mpg.de/hlcv Class of Object
More informationPart-Based Models for Object Class Recognition Part 2
High Level Computer Vision Part-Based Models for Object Class Recognition Part 2 Bernt Schiele - schiele@mpi-inf.mpg.de Mario Fritz - mfritz@mpi-inf.mpg.de https://www.mpi-inf.mpg.de/hlcv Class of Object
More informationLecture 5: Exact inference
Lecture 5: Exact inference Queries Inference in chains Variable elimination Without evidence With evidence Complexity of variable elimination which has the highest probability: instantiation of all other
More information3 October, 2013 MVA ENS Cachan. Lecture 2: Logistic regression & intro to MIL Iasonas Kokkinos
Machine Learning for Computer Vision 1 3 October, 2013 MVA ENS Cachan Lecture 2: Logistic regression & intro to MIL Iasonas Kokkinos Iasonas.kokkinos@ecp.fr Department of Applied Mathematics Ecole Centrale
More informationDevelopment in Object Detection. Junyuan Lin May 4th
Development in Object Detection Junyuan Lin May 4th Line of Research [1] N. Dalal and B. Triggs. Histograms of oriented gradients for human detection, CVPR 2005. HOG Feature template [2] P. Felzenszwalb,
More informationGraphical Models for Computer Vision
Graphical Models for Computer Vision Pedro F Felzenszwalb Brown University Joint work with Dan Huttenlocher, Joshua Schwartz, Ross Girshick, David McAllester, Deva Ramanan, Allie Shapiro, John Oberlin
More informationSTA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 5 Inference
More informationMotivation: Shortcomings of Hidden Markov Model. Ko, Youngjoong. Solution: Maximum Entropy Markov Model (MEMM)
Motivation: Shortcomings of Hidden Markov Model Maximum Entropy Markov Models and Conditional Random Fields Ko, Youngjoong Dept. of Computer Engineering, Dong-A University Intelligent System Laboratory,
More informationPart II. C. M. Bishop PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 8: GRAPHICAL MODELS
Part II C. M. Bishop PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 8: GRAPHICAL MODELS Converting Directed to Undirected Graphs (1) Converting Directed to Undirected Graphs (2) Add extra links between
More informationIntroduction to Graphical Models
Robert Collins CSE586 Introduction to Graphical Models Readings in Prince textbook: Chapters 10 and 11 but mainly only on directed graphs at this time Credits: Several slides are from: Review: Probability
More informationSum-Product Networks. STAT946 Deep Learning Guest Lecture by Pascal Poupart University of Waterloo October 15, 2015
Sum-Product Networks STAT946 Deep Learning Guest Lecture by Pascal Poupart University of Waterloo October 15, 2015 Introduction Outline What is a Sum-Product Network? Inference Applications In more depth
More informationComputer Vision Group Prof. Daniel Cremers. 4. Probabilistic Graphical Models Directed Models
Prof. Daniel Cremers 4. Probabilistic Graphical Models Directed Models The Bayes Filter (Rep.) (Bayes) (Markov) (Tot. prob.) (Markov) (Markov) 2 Graphical Representation (Rep.) We can describe the overall
More informationPATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 8: GRAPHICAL MODELS
PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 8: GRAPHICAL MODELS Bayesian Networks Directed Acyclic Graph (DAG) Bayesian Networks General Factorization Bayesian Curve Fitting (1) Polynomial Bayesian
More informationCS839: Probabilistic Graphical Models. Lecture 10: Learning with Partially Observed Data. Theo Rekatsinas
CS839: Probabilistic Graphical Models Lecture 10: Learning with Partially Observed Data Theo Rekatsinas 1 Partially Observed GMs Speech recognition 2 Partially Observed GMs Evolution 3 Partially Observed
More informationConditional Random Fields and beyond D A N I E L K H A S H A B I C S U I U C,
Conditional Random Fields and beyond D A N I E L K H A S H A B I C S 5 4 6 U I U C, 2 0 1 3 Outline Modeling Inference Training Applications Outline Modeling Problem definition Discriminative vs. Generative
More informationLearning and Recognizing Visual Object Categories Without First Detecting Features
Learning and Recognizing Visual Object Categories Without First Detecting Features Daniel Huttenlocher 2007 Joint work with D. Crandall and P. Felzenszwalb Object Category Recognition Generic classes rather
More information10-701/15-781, Fall 2006, Final
-7/-78, Fall 6, Final Dec, :pm-8:pm There are 9 questions in this exam ( pages including this cover sheet). If you need more room to work out your answer to a question, use the back of the page and clearly
More informationHidden Markov Models. Slides adapted from Joyce Ho, David Sontag, Geoffrey Hinton, Eric Xing, and Nicholas Ruozzi
Hidden Markov Models Slides adapted from Joyce Ho, David Sontag, Geoffrey Hinton, Eric Xing, and Nicholas Ruozzi Sequential Data Time-series: Stock market, weather, speech, video Ordered: Text, genes Sequential
More informationCS 664 Flexible Templates. Daniel Huttenlocher
CS 664 Flexible Templates Daniel Huttenlocher Flexible Template Matching Pictorial structures Parts connected by springs and appearance models for each part Used for human bodies, faces Fischler&Elschlager,
More informationComputer Vision Group Prof. Daniel Cremers. 4a. Inference in Graphical Models
Group Prof. Daniel Cremers 4a. Inference in Graphical Models Inference on a Chain (Rep.) The first values of µ α and µ β are: The partition function can be computed at any node: Overall, we have O(NK 2
More informationSegmentation. Bottom up Segmentation Semantic Segmentation
Segmentation Bottom up Segmentation Semantic Segmentation Semantic Labeling of Street Scenes Ground Truth Labels 11 classes, almost all occur simultaneously, large changes in viewpoint, scale sky, road,
More informationAlgorithms for Markov Random Fields in Computer Vision
Algorithms for Markov Random Fields in Computer Vision Dan Huttenlocher November, 2003 (Joint work with Pedro Felzenszwalb) Random Field Broadly applicable stochastic model Collection of n sites S Hidden
More informationIntroduction to SLAM Part II. Paul Robertson
Introduction to SLAM Part II Paul Robertson Localization Review Tracking, Global Localization, Kidnapping Problem. Kalman Filter Quadratic Linear (unless EKF) SLAM Loop closing Scaling: Partition space
More informationMachine Learning. Sourangshu Bhattacharya
Machine Learning Sourangshu Bhattacharya Bayesian Networks Directed Acyclic Graph (DAG) Bayesian Networks General Factorization Curve Fitting Re-visited Maximum Likelihood Determine by minimizing sum-of-squares
More informationPictorial Structures for Object Recognition
Pictorial Structures for Object Recognition Felzenszwalb and Huttenlocher Presented by Stephen Krotosky Pictorial Structures Introduced by Fischler and Elschlager in 1973 Objects are modeled by a collection
More informationAssignment 2. Unsupervised & Probabilistic Learning. Maneesh Sahani Due: Monday Nov 5, 2018
Assignment 2 Unsupervised & Probabilistic Learning Maneesh Sahani Due: Monday Nov 5, 2018 Note: Assignments are due at 11:00 AM (the start of lecture) on the date above. he usual College late assignments
More informationA Discriminatively Trained, Multiscale, Deformable Part Model
A Discriminatively Trained, Multiscale, Deformable Part Model by Pedro Felzenszwalb, David McAllester, and Deva Ramanan CS381V Visual Recognition - Paper Presentation Slide credit: Duan Tran Slide credit:
More informationStructured Models in. Dan Huttenlocher. June 2010
Structured Models in Computer Vision i Dan Huttenlocher June 2010 Structured Models Problems where output variables are mutually dependent or constrained E.g., spatial or temporal relations Such dependencies
More informationGraphical Models & HMMs
Graphical Models & HMMs Henrik I. Christensen Robotics & Intelligent Machines @ GT Georgia Institute of Technology, Atlanta, GA 30332-0280 hic@cc.gatech.edu Henrik I. Christensen (RIM@GT) Graphical Models
More informationMassachusetts Institute of Technology Department of Electrical Engineering and Computer Science Algorithms for Inference Fall 2014
Massachusetts Institute of Technology Department of Electrical Engineering and Computer Science 6.438 Algorithms for Inference Fall 2014 1 Course Overview This course is about performing inference in complex
More informationCOMP90051 Statistical Machine Learning
COMP90051 Statistical Machine Learning Semester 2, 2016 Lecturer: Trevor Cohn 20. PGM Representation Next Lectures Representation of joint distributions Conditional/marginal independence * Directed vs
More informationStructured Learning. Jun Zhu
Structured Learning Jun Zhu Supervised learning Given a set of I.I.D. training samples Learn a prediction function b r a c e Supervised learning (cont d) Many different choices Logistic Regression Maximum
More informationFMA901F: Machine Learning Lecture 6: Graphical Models. Cristian Sminchisescu
FMA901F: Machine Learning Lecture 6: Graphical Models Cristian Sminchisescu Graphical Models Provide a simple way to visualize the structure of a probabilistic model and can be used to design and motivate
More informationMachine Learning A WS15/16 1sst KU Version: January 11, b) [1 P] For the probability distribution P (A, B, C, D) with the factorization
Machine Learning A 708.064 WS15/16 1sst KU Version: January 11, 2016 Exercises Problems marked with * are optional. 1 Conditional Independence I [3 P] a) [1 P] For the probability distribution P (A, B,
More informationRegularization and Markov Random Fields (MRF) CS 664 Spring 2008
Regularization and Markov Random Fields (MRF) CS 664 Spring 2008 Regularization in Low Level Vision Low level vision problems concerned with estimating some quantity at each pixel Visual motion (u(x,y),v(x,y))
More informationDeformable Part Models
CS 1674: Intro to Computer Vision Deformable Part Models Prof. Adriana Kovashka University of Pittsburgh November 9, 2016 Today: Object category detection Window-based approaches: Last time: Viola-Jones
More informationMarkov Random Fields and Segmentation with Graph Cuts
Markov Random Fields and Segmentation with Graph Cuts Computer Vision Jia-Bin Huang, Virginia Tech Many slides from D. Hoiem Administrative stuffs Final project Proposal due Oct 27 (Thursday) HW 4 is out
More informationLoopy Belief Propagation
Loopy Belief Propagation Research Exam Kristin Branson September 29, 2003 Loopy Belief Propagation p.1/73 Problem Formalization Reasoning about any real-world problem requires assumptions about the structure
More informationCS242: Probabilistic Graphical Models Lecture 3: Factor Graphs & Variable Elimination
CS242: Probabilistic Graphical Models Lecture 3: Factor Graphs & Variable Elimination Instructor: Erik Sudderth Brown University Computer Science September 11, 2014 Some figures and materials courtesy
More informationPart-based models. Lecture 10
Part-based models Lecture 10 Overview Representation Location Appearance Generative interpretation Learning Distance transforms Other approaches using parts Felzenszwalb, Girshick, McAllester, Ramanan
More informationExpectation Propagation
Expectation Propagation Erik Sudderth 6.975 Week 11 Presentation November 20, 2002 Introduction Goal: Efficiently approximate intractable distributions Features of Expectation Propagation (EP): Deterministic,
More informationFMA901F: Machine Learning Lecture 3: Linear Models for Regression. Cristian Sminchisescu
FMA901F: Machine Learning Lecture 3: Linear Models for Regression Cristian Sminchisescu Machine Learning: Frequentist vs. Bayesian In the frequentist setting, we seek a fixed parameter (vector), with value(s)
More informationMachine Learning A W 1sst KU. b) [1 P] For the probability distribution P (A, B, C, D) with the factorization
Machine Learning A 708.064 13W 1sst KU Exercises Problems marked with * are optional. 1 Conditional Independence a) [1 P] For the probability distribution P (A, B, C, D) with the factorization P (A, B,
More informationScene Grammars, Factor Graphs, and Belief Propagation
Scene Grammars, Factor Graphs, and Belief Propagation Pedro Felzenszwalb Brown University Joint work with Jeroen Chua Probabilistic Scene Grammars General purpose framework for image understanding and
More informationProbabilistic Graphical Models
Overview of Part Two Probabilistic Graphical Models Part Two: Inference and Learning Christopher M. Bishop Exact inference and the junction tree MCMC Variational methods and EM Example General variational
More informationComputer vision: models, learning and inference. Chapter 10 Graphical Models
Computer vision: models, learning and inference Chapter 10 Graphical Models Independence Two variables x 1 and x 2 are independent if their joint probability distribution factorizes as Pr(x 1, x 2 )=Pr(x
More information27: Hybrid Graphical Models and Neural Networks
10-708: Probabilistic Graphical Models 10-708 Spring 2016 27: Hybrid Graphical Models and Neural Networks Lecturer: Matt Gormley Scribes: Jakob Bauer Otilia Stretcu Rohan Varma 1 Motivation We first look
More informationScene Grammars, Factor Graphs, and Belief Propagation
Scene Grammars, Factor Graphs, and Belief Propagation Pedro Felzenszwalb Brown University Joint work with Jeroen Chua Probabilistic Scene Grammars General purpose framework for image understanding and
More informationInference and Representation
Inference and Representation Rachel Hodos New York University Lecture 5, October 6, 2015 Rachel Hodos Lecture 5: Inference and Representation Today: Learning with hidden variables Outline: Unsupervised
More informationComputer Vision Group Prof. Daniel Cremers. 4. Probabilistic Graphical Models Directed Models
Prof. Daniel Cremers 4. Probabilistic Graphical Models Directed Models The Bayes Filter (Rep.) (Bayes) (Markov) (Tot. prob.) (Markov) (Markov) 2 Graphical Representation (Rep.) We can describe the overall
More informationFast, Accurate Detection of 100,000 Object Classes on a Single Machine
Fast, Accurate Detection of 100,000 Object Classes on a Single Machine Thomas Dean etal. Google, Mountain View, CA CVPR 2013 best paper award Presented by: Zhenhua Wang 2013.12.10 Outline Background This
More informationObject Category Detection. Slides mostly from Derek Hoiem
Object Category Detection Slides mostly from Derek Hoiem Today s class: Object Category Detection Overview of object category detection Statistical template matching with sliding window Part-based Models
More informationModern Object Detection. Most slides from Ali Farhadi
Modern Object Detection Most slides from Ali Farhadi Comparison of Classifiers assuming x in {0 1} Learning Objective Training Inference Naïve Bayes maximize j i logp + logp ( x y ; θ ) ( y ; θ ) i ij
More informationPart based models for recognition. Kristen Grauman
Part based models for recognition Kristen Grauman UT Austin Limitations of window-based models Not all objects are box-shaped Assuming specific 2d view of object Local components themselves do not necessarily
More informationExtracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields
Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields Lin Liao Dieter Fox Henry Kautz Department of Computer Science & Engineering University of Washington Seattle,
More informationCPSC 340: Machine Learning and Data Mining. Principal Component Analysis Fall 2016
CPSC 340: Machine Learning and Data Mining Principal Component Analysis Fall 2016 A2/Midterm: Admin Grades/solutions will be posted after class. Assignment 4: Posted, due November 14. Extra office hours:
More informationObject Recognition with Deformable Models
Object Recognition with Deformable Models Pedro F. Felzenszwalb Department of Computer Science University of Chicago Joint work with: Dan Huttenlocher, Joshua Schwartz, David McAllester, Deva Ramanan.
More informationCSEP 573: Artificial Intelligence
CSEP 573: Artificial Intelligence Machine Learning: Perceptron Ali Farhadi Many slides over the course adapted from Luke Zettlemoyer and Dan Klein. 1 Generative vs. Discriminative Generative classifiers:
More informationMachine Learning A W 1sst KU. b) [1 P] Give an example for a probability distributions P (A, B, C) that disproves
Machine Learning A 708.064 11W 1sst KU Exercises Problems marked with * are optional. 1 Conditional Independence I [2 P] a) [1 P] Give an example for a probability distribution P (A, B, C) that disproves
More informationExam Topics. Search in Discrete State Spaces. What is intelligence? Adversarial Search. Which Algorithm? 6/1/2012
Exam Topics Artificial Intelligence Recap & Expectation Maximization CSE 473 Dan Weld BFS, DFS, UCS, A* (tree and graph) Completeness and Optimality Heuristics: admissibility and consistency CSPs Constraint
More informationCategory-level localization
Category-level localization Cordelia Schmid Recognition Classification Object present/absent in an image Often presence of a significant amount of background clutter Localization / Detection Localize object
More informationECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale Learning
ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale Learning Topics Bayes Nets: Inference (Finish) Variable Elimination Graph-view of VE: Fill-edges, induced width
More informationDeformable Part Models
Deformable Part Models References: Felzenszwalb, Girshick, McAllester and Ramanan, Object Detec@on with Discrimina@vely Trained Part Based Models, PAMI 2010 Code available at hkp://www.cs.berkeley.edu/~rbg/latent/
More informationLecture 5: Exact inference. Queries. Complexity of inference. Queries (continued) Bayesian networks can answer questions about the underlying
given that Maximum a posteriori (MAP query: given evidence 2 which has the highest probability: instantiation of all other variables in the network,, Most probable evidence (MPE: given evidence, find an
More informationOSU CS 536 Probabilistic Graphical Models. Loopy Belief Propagation and Clique Trees / Join Trees
OSU CS 536 Probabilistic Graphical Models Loopy Belief Propagation and Clique Trees / Join Trees Slides from Kevin Murphy s Graphical Model Tutorial (with minor changes) Reading: Koller and Friedman Ch
More informationLecture 21 : A Hybrid: Deep Learning and Graphical Models
10-708: Probabilistic Graphical Models, Spring 2018 Lecture 21 : A Hybrid: Deep Learning and Graphical Models Lecturer: Kayhan Batmanghelich Scribes: Paul Liang, Anirudha Rayasam 1 Introduction and Motivation
More informationProbabilistic Graphical Models
Probabilistic Graphical Models Lecture 17 EM CS/CNS/EE 155 Andreas Krause Announcements Project poster session on Thursday Dec 3, 4-6pm in Annenberg 2 nd floor atrium! Easels, poster boards and cookies
More informationTracking Algorithms. Lecture16: Visual Tracking I. Probabilistic Tracking. Joint Probability and Graphical Model. Deterministic methods
Tracking Algorithms CSED441:Introduction to Computer Vision (2017F) Lecture16: Visual Tracking I Bohyung Han CSE, POSTECH bhhan@postech.ac.kr Deterministic methods Given input video and current state,
More informationECE521 W17 Tutorial 10
ECE521 W17 Tutorial 10 Shenlong Wang and Renjie Liao *Some of materials are credited to Jimmy Ba, Eric Sudderth, Chris Bishop Introduction to A4 1, Graphical Models 2, Message Passing 3, HMM Introduction
More informationConditional Random Fields - A probabilistic graphical model. Yen-Chin Lee 指導老師 : 鮑興國
Conditional Random Fields - A probabilistic graphical model Yen-Chin Lee 指導老師 : 鮑興國 Outline Labeling sequence data problem Introduction conditional random field (CRF) Different views on building a conditional
More informationPattern Recognition. Kjell Elenius. Speech, Music and Hearing KTH. March 29, 2007 Speech recognition
Pattern Recognition Kjell Elenius Speech, Music and Hearing KTH March 29, 2007 Speech recognition 2007 1 Ch 4. Pattern Recognition 1(3) Bayes Decision Theory Minimum-Error-Rate Decision Rules Discriminant
More informationLecture 13 Segmentation and Scene Understanding Chris Choy, Ph.D. candidate Stanford Vision and Learning Lab (SVL)
Lecture 13 Segmentation and Scene Understanding Chris Choy, Ph.D. candidate Stanford Vision and Learning Lab (SVL) http://chrischoy.org Stanford CS231A 1 Understanding a Scene Objects Chairs, Cups, Tables,
More information2/15/2009. Part-Based Models. Andrew Harp. Part Based Models. Detect object from physical arrangement of individual features
Part-Based Models Andrew Harp Part Based Models Detect object from physical arrangement of individual features 1 Implementation Based on the Simple Parts and Structure Object Detector by R. Fergus Allows
More informationGenerative and discriminative classification techniques
Generative and discriminative classification techniques Machine Learning and Category Representation 013-014 Jakob Verbeek, December 13+0, 013 Course website: http://lear.inrialpes.fr/~verbeek/mlcr.13.14
More informationModeling time series with hidden Markov models
Modeling time series with hidden Markov models Advanced Machine learning 2017 Nadia Figueroa, Jose Medina and Aude Billard Time series data Barometric pressure Temperature Data Humidity Time What s going
More informationFinal Exam. Introduction to Artificial Intelligence. CS 188 Spring 2010 INSTRUCTIONS. You have 3 hours.
CS 188 Spring 2010 Introduction to Artificial Intelligence Final Exam INSTRUCTIONS You have 3 hours. The exam is closed book, closed notes except a two-page crib sheet. Please use non-programmable calculators
More informationCSE 586 Final Programming Project Spring 2011 Due date: Tuesday, May 3
CSE 586 Final Programming Project Spring 2011 Due date: Tuesday, May 3 What I have in mind for our last programming project is to do something with either graphical models or random sampling. A few ideas
More informationDynamic Bayesian network (DBN)
Readings: K&F: 18.1, 18.2, 18.3, 18.4 ynamic Bayesian Networks Beyond 10708 Graphical Models 10708 Carlos Guestrin Carnegie Mellon University ecember 1 st, 2006 1 ynamic Bayesian network (BN) HMM defined
More informationProbabilistic Graphical Models
School of Computer Science Probabilistic Graphical Models Theory of Variational Inference: Inner and Outer Approximation Eric Xing Lecture 14, February 29, 2016 Reading: W & J Book Chapters Eric Xing @
More informationUsing the Deformable Part Model with Autoencoded Feature Descriptors for Object Detection
Using the Deformable Part Model with Autoencoded Feature Descriptors for Object Detection Hyunghoon Cho and David Wu December 10, 2010 1 Introduction Given its performance in recent years' PASCAL Visual
More informationParticle Filters for Visual Tracking
Particle Filters for Visual Tracking T. Chateau, Pascal Institute, Clermont-Ferrand 1 Content Particle filtering: a probabilistic framework SIR particle filter MCMC particle filter RJMCMC particle filter
More informationPreviously. Part-based and local feature models for generic object recognition. Bag-of-words model 4/20/2011
Previously Part-based and local feature models for generic object recognition Wed, April 20 UT-Austin Discriminative classifiers Boosting Nearest neighbors Support vector machines Useful for object recognition
More informationDetection III: Analyzing and Debugging Detection Methods
CS 1699: Intro to Computer Vision Detection III: Analyzing and Debugging Detection Methods Prof. Adriana Kovashka University of Pittsburgh November 17, 2015 Today Review: Deformable part models How can
More informationBelief propagation and MRF s
Belief propagation and MRF s Bill Freeman 6.869 March 7, 2011 1 1 Undirected graphical models A set of nodes joined by undirected edges. The graph makes conditional independencies explicit: If two nodes
More informationExact Inference: Elimination and Sum Product (and hidden Markov models)
Exact Inference: Elimination and Sum Product (and hidden Markov models) David M. Blei Columbia University October 13, 2015 The first sections of these lecture notes follow the ideas in Chapters 3 and 4
More informationFacial Landmark Tracking by Tree-based Deformable Part Model Based Detector
Facial Landmark Tracking by Tree-based Deformable Part Model Based Detector Michal Uřičář, Vojtěch Franc, and Václav Hlaváč Department of Cybernetics, Faculty Electrical Engineering Czech Technical University
More informationHidden Markov Models. Gabriela Tavares and Juri Minxha Mentor: Taehwan Kim CS159 04/25/2017
Hidden Markov Models Gabriela Tavares and Juri Minxha Mentor: Taehwan Kim CS159 04/25/2017 1 Outline 1. 2. 3. 4. Brief review of HMMs Hidden Markov Support Vector Machines Large Margin Hidden Markov Models
More informationK-Means and Gaussian Mixture Models
K-Means and Gaussian Mixture Models David Rosenberg New York University June 15, 2015 David Rosenberg (New York University) DS-GA 1003 June 15, 2015 1 / 43 K-Means Clustering Example: Old Faithful Geyser
More informationWorkshop report 1. Daniels report is on website 2. Don t expect to write it based on listening to one project (we had 6 only 2 was sufficient
Workshop report 1. Daniels report is on website 2. Don t expect to write it based on listening to one project (we had 6 only 2 was sufficient quality) 3. I suggest writing it on one presentation. 4. Include
More informationTime series, HMMs, Kalman Filters
Classic HMM tutorial see class website: *L. R. Rabiner, "A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition," Proc. of the IEEE, Vol.77, No.2, pp.257--286, 1989. Time series,
More informationECE 5424: Introduction to Machine Learning
ECE 5424: Introduction to Machine Learning Topics: Unsupervised Learning: Kmeans, GMM, EM Readings: Barber 20.1-20.3 Stefan Lee Virginia Tech Tasks Supervised Learning x Classification y Discrete x Regression
More informationPreface to the Second Edition. Preface to the First Edition. 1 Introduction 1
Preface to the Second Edition Preface to the First Edition vii xi 1 Introduction 1 2 Overview of Supervised Learning 9 2.1 Introduction... 9 2.2 Variable Types and Terminology... 9 2.3 Two Simple Approaches
More informationObject Detection with Partial Occlusion Based on a Deformable Parts-Based Model
Object Detection with Partial Occlusion Based on a Deformable Parts-Based Model Johnson Hsieh (johnsonhsieh@gmail.com), Alexander Chia (alexchia@stanford.edu) Abstract -- Object occlusion presents a major
More informationBuilding Classifiers using Bayesian Networks
Building Classifiers using Bayesian Networks Nir Friedman and Moises Goldszmidt 1997 Presented by Brian Collins and Lukas Seitlinger Paper Summary The Naive Bayes classifier has reasonable performance
More information