CS649 Sensor Networks IP Track Lecture 6: Graphical Models

Transcription

1 CS649 Sensor Networks IP Track Lecture 6: Grahical Models I-Jeng Wang htt://hinrg.cs.jhu.edu/wsn06/ Sring 2006 CS 649 1

2 Sring 2006 CS Grahical Models Grahical Model: grahical reresentation of joint robability distribution that encodes the conditional indeendency structure of the random variables Tyes of grahical models Directed grahical models: ayesian networks Undirected grahical models: factor grahs junction trees Markov random fields = V v v v V a M J J M J M J = = ayesian Networks chain rule alying conditional indeendency

3 Sring 2006 CS Undirected Grahical Models: Markov Random Fields Given an undirected grah G = V each node v has an associated random variable v Markov roerty: is conditional indeendent of given S if S searates and where S V Factorization: The joint distribution can be factorized into roducts of functions defined over all cliques Hammersley-Clifford Theorem: For strictly osition the Markov and the Factorization roerties are equivalent = C C C C V Z 1 ψ S Z V ψ ψ ψ ψ ψ = = CC C C V Z 1 ψ comatibility function

4 Imortance of Grahical Structures in Probability Models Provide a structure to organize imortant and difficult calculations in robability models to achieve efficient comutations Key comutation with robability models Marginalization robabilistic inference : vidence observed F V = U F U H F: Inference variables H: Hidden variables e.g. P = true = true Modes or most robable configuration MP x * F = arg max Normalization constant Z x F F Sring 2006 CS 649 4

5 Significance of Structure for Marginalization Direct comutation lead to high comlexity r 6! xloiting the distributive law and factorization structure can significantly reduce comlexity r 3! Variable limination Pick an order to eliminate For each variable: a. Push in its sum as far as ossible b. Comute the sum Sring 2006 CS 649 5

6 Message Passing lgorithms on Trees Cliques in an undirected tree are airs and singleton i V = ψ ψ V i i j Sum-roduct algorithm given below: Comute all marginals Max-roduct: Comute MP configurations relacing sums with maximizations Variants with the same algebraic structure commutative semi-ring i j Sring 2006 CS 649 6

7 Proerties of Message Passing lgorithms synchronous imlementation is straightforward Only require local message assing on the grah Guaranteed convergence in finite number of iterations on trees exact algorithms Need not converge on grahs with cycles however have been alied with successes Performance under non-stationary environments questionable Sring 2006 CS 649 7

8 Grahical Models and Message Passing lgorithms for Sensor Networks? The distributed and asynchronous nature of the messaging assing seems well suited for information rocessing in sensor networks However key questions need to be answered: Maing constructing a grahical model onto the sensor network How do we construct a relevant robability grah given a sensor network alication? The robability grah is NOT the communication grah of the sensor network Dealing with the unreliable wireless medium and low cost sensor nodes Convergence issues with message assing on looy grahs Sring 2006 CS 649 8

9 Robust rchitecture for Distributed Inference in Sensor Networks Mark. Paskin Carlos Guestrin Jim McFadden IPSN 2005 Sring 2006 CS 649 9

10 asic Ideas Using junction tree structure for inference to ensure convergence of message assing algorithms three-layer architecture The sanning tree layer: nsure the reliability of communications to suort necessary information exchange in message assing algorithms; Provide a sace of stable sanning tree to enable otimization of junction tree construction The junction tree layer: Construct a tree structure for inference taking into account the underlying robability model the grah and the communication constraints the sanning trees The inference layer: Perform aroriate message assing algorithms over the junction tree for robabilistic inference Sring 2006 CS

11 Undirected Grahical Models and Junction Trees cluster grah T is a junction tree for a grah G if it satisfies the following: Single connected: exactly one ath between two clusters Covering: each clique in G is a subset of some cluster node in T Running intersection: If two clusters and C contain a variable then all clusters on the ath between and C also contains searator Sring 2006 CS

12 Junction Tree Formation t each sensor node initialize a cluster node with its local variables D i to ensure the covering roerty Udate the clusters by running a message assing algorithms with reachable variable messages to ensure the running intersection roerty Otimize the junction tree by minimizing the comuting sizes of clusters and communication sizes of the searators costs Sring 2006 CS

13 From a Sanning Tree to a Junction Tree Sanning Tree with Initial Local Variables ssignment Constructed Junction Tree fter Message Passing Sring 2006 CS

14 Some Oen Research Issues How do we obtain the grahical model the comatibility functions or factors? How do we comute and communicate the inference results to where it is needed? not necessarily colocated with the node that contains the relevant variables Is it ossible to derive a more robust message assing algorithm that can take advantage of the wireless local wireless broadcast caability in sensor network? Sring 2006 CS