Bayesian Networks: Independencies and Inference. What Independencies does a Bayes Net Model?

Transcription

1 Bayesan Networks: Indeendences and Inference Scott Daves and Andrew Moore Note to other teachers and users of these sldes. Andrew and Scott would be delghted f you found ths source materal useful n gvng your own lectures. Feel free to use these sldes verbatm or to modfy them to ft your own needs. oweront orgnals are avalable. If you make use of a sgnfcant orton of these sldes n your own lecture lease nclude ths message or the followng lnk to the source reostory of Andrew s tutorals: htt:// omments and correctons gratefully receved. What Indeendences does a Bayes Net Model? In order for a Bayesan network to model a robablty dstrbuton the followng must be true by defnton: ach varable s condtonally ndeendent of all ts nondescendants n the grah gven the value of all ts arents. Ths mles 1 K n arents 1 But what else does t mly? n 1

2 2 What Indeendences does a Bayes Net Model? xamle: Gven does learnng the value of tell us nothng new about? I.e. s equal to? es. Snce we know the value of all of s arents namely and s not a descendant of s condtonally ndeendent of. Also snce ndeendence s symmetrc. Quck roof that ndeendence s symmetrc Assume: Then: Bayes s Rule han Rule By Assumton Bayes s Rule

3 What Indeendences does a Bayes Net Model? Let I<> reresent and beng condtonally ndeendent gven. I<>? es ust as n revous examle: All s arents gven and s not a descendant. What Indeendences does a Bayes Net Model? U V I<{U}>? No. I<{UV}>? es. Maybe I< S > ff S acts a cutset between and n an undrected verson of the grah? 3

4 Thngs get a lttle more confusng has no arents so we re know all ts arents values trvally s not a descendant of So I<{}> even though there s a undrected ath from to through an unknown varable. What f we do know the value of though? Or one of ts descendants? The Burglar Alarm examle Burglar arthquake Alarm hone all our house has a twtchy burglar alarm that s also sometmes trggered by earthquakes. arth arguably doesn t care whether your house s currently beng burgled Whle you are on vacaton one of your neghbors calls and tells you your home s burglar alarm s rngng. Uh oh! 4

5 Thngs get a lot more confusng Burglar arthquake Alarm hone all But now suose you learn that there was a medum-szed earthquake n your neghborhood. Oh whew! robably not a burglar after all. arthquake exlans away the hyothetcal burglar. But then t must not be the case that I<Burglar{hone all} arthquake> even though I<Burglar{} arthquake>! d-searaton to the rescue Fortunately there s a relatvely smle algorthm for determnng whether two varables n a Bayesan network are condtonally ndeendent: d-searaton. Defnton: and are d-searated by a set of evdence varables ff every undrected ath from to s blocked where a ath s blocked ff one or more of the followng condtons s true:... 5

6 A ath s blocked when... There exsts a varable V on the ath such that t s n the evdence set the arcs uttng V n the ath are tal-to-tal Or there exsts a varable V on the ath such that t s n the evdence set the arcs uttng V n the ath are tal-to-head Or... V V A ath s blocked when the funky case Or there exsts a varable V on the ath such that t s NOT n the evdence set nether are any of ts descendants the arcs uttng V on the ath are head-to-head V 6

7 d-searaton to the rescue cont d Theorem [Verma & earl 1998]: If a set of evdence varables d-searates and n a Bayesan network s grah then I< >. d-searaton can be comuted n lnear tme usng a deth-frst-search-lke algorthm. Great! We now have a fast algorthm for automatcally nferrng whether learnng the value of one varable mght gve us any addtonal hnts about some other varable gven what we already know. Mght : Varables may actually be ndeendent when they re not d- searated deendng on the actual robabltes nvolved d-searaton examle A G B D F H I< {} D>? I< {A} D>? I< {A B} D>? I< {A B J} D>? I< {A B J} D>? I J 7

8 Bayesan Network Inference Inference: calculatng for some varables or sets of varables and. Inference n Bayesan networks s #-hard! Inuts: ror robabltes of.5 Reduces to I1 I2 I3 I4 I5 How many satsfyng assgnments? O O must be #sat. assgn.*.5^#nuts Bayesan Network Inference But nference s stll tractable n some cases. Let s look a secal class of networks: trees / forests n whch each node has at most one arent. 8

9 Decomosng the robabltes Suose we want where s some set of evdence varables. Let s slt nto two arts: - s the art consstng of assgnments to varables n the subtree rooted at s the rest of t Decomosng the robabltes cont d 9

10 10 Decomosng the robabltes cont d Decomosng the robabltes cont d

11 11 Decomosng the robabltes cont d? a Where: α s a constant ndeendent of π λ - Usng the decomoston for nference We can use ths decomoston to do nference as follows. Frst comute λ - for all recursvely usng the leaves of the tree as the base case. If s a leaf: If s n : λ 1 f matches 0 otherwse If s not n : - s the null set so - 1 constant

12 Quck asde: Vrtual evdence For theoretcal smlcty but wthout loss of generalty let s assume that all varables n the evdence set are leaves n the tree. Why can we do ths WLOG: quvalent to Observe Observe Where 1 f 0 otherwse alculatng λ for non-leaves Suose has one chld c. Then:? c 12

13 13 alculatng λ for non-leaves Suose has one chld c. Then: c? alculatng λ for non-leaves Suose has one chld c. Then: c?

14 14 alculatng λ for non-leaves Suose has one chld c. Then: c?? alculatng λ for non-leaves Now suose has a set of chldren. Snce d-searates each of ts subtrees the contrbuton of each subtree to λ s ndeendent:??? where λ s the contrbuton to - of the art of the evdence lyng n the subtree rooted at one of s chldren.

15 We are now λ-hay So now we have a way to recursvely comute all the λ s startng from the root and usng the leaves as the base case. If we want we can thnk of each node n the network as an autonomous rocessor that asses a lttle λ message to ts arent. λ λ λ λ λ λ The other half of the roblem Remember απ λ. Now that we have all the λ s what about the π s? π. What about the root of the tree r? In that case r s the null set so π r r. No sweat. Snce we also know λ r we can comute the fnal r. So for an arbtrary wth arent let s nductvely assume we know π and/or. How do we get π? 15

16 16 omutng π omutng π

17 17 omutng π omutng π

18 18 omutng π? omutng π Where π s defned as??

19 We re done. ay! Thus we can comute all the π s and n turn all the s. an thnk of nodes as autonomous rocessors assng λ and π messages to ther neghbors λ π π λ λ λ λ λ π π π π onunctve queres What f we want e.g. A B nstead of ust margnal dstrbutons A and B? Just use chan rule: A B A B A ach of the latter robabltes can be comuted usng the technque ust dscussed. 19

20 olytrees Technque can be generalzed to olytrees: undrected versons of the grahs are stll trees but nodes can have more than one arent Dealng wth cycles an deal wth undrected cycles n grah by clusterng varables together A A B D B D ondtonng Set to 0 Set to 1 20

21 Jon trees Arbtrary Bayesan network can be transformed va some evl grah-theoretc magc nto a on tree n whch a smlar method can be emloyed. B A D F G BD AB BD In the worst case the on tree nodes must take on exonentally many combnatons of values but often works well n ractce DF 21