TESTING AND IMPROVING LOCAL ADAPTIVE IMPORTANCE SAMPLING IN LJF LOCAL-JT IN MULTIPLY SECTIONED BAYESIAN NETWORKS

Transcription

1 TESTING AND IMPROVING LOCAL ADAPTIVE IMPORTANCE SAMPLING IN LJF LOCAL-JT IN MULTIPLY SECTIONED BAYESIAN NETWORKS Dan Wu 1 and Sona Bhatt 2 1 School of Computer Scence Unversty of Wndsor, Wndsor, Ontaro Canada danwu@uwndsor.ca 2 School of Computer Scence Unversty of Wndsor, Wndsor, Ontaro Canada bhattf@uwndsor.ca ABSTRACT Multply Sectoned Bayesan Network (MSBN) provdes a model for probablstc reasonng n mult-agent systems. The exact nference s costly and dffcult to be appled n the context of MSBNs as the sze of problem doman becomes larger and complex. So the approxmate technques are used as an alternatve n such cases. Recently, for reasonng n MSBNs, LJFbased Local Adaptve Importance Sampler (LLAIS) has been developed for approxmate reasonng n MSBNs. However, the prototype of LLAIS s tested only on Alarm Network (37 nodes). But further testng on larger networks has not been reported yet, so the scalablty and relablty of algorthm remans questonable. Hence, we tested LLAIS on three large networks (treated as local JTs) namely Halfnder (56 nodes), Wn95pts (76 nodes) and PathFnder(109 nodes). From the experments done, t s seen that LLAIS wthout parameters tuned shows good convergence for Halfnder and Wn95pts but not for Pathfnder network. Further when these parameters are tuned the algorthm shows consderable mprovement n ts accuracy and convergence for all the three networks tested. KEYWORDS MSBN, LJF, Adaptve Importance samplng, Tunable parameters 1. INTRODUCTION Multply Sectoned Bayesan Networks (MSBN) s the model grounded on the dea of cooperatve mult-agent probablstc reasonng, s an extenson of the tradtonal Bayesan Network model and t provde us wth soluton to the probablstc Reasonng under cooperatve agents. The Multple agents [1] collectvely and cooperatvely reason about ther respectve problem doman on the bass of ther local knowledge, local observaton and lmted nter-agent communcaton. Typcally the nference n MSBN s generally carred out n some secondary structure known as lnked Juncton tree forest (LJF). The LJF provdes a coherent framework for exact nference wth MSBN [2], LJF consttutes local Juncton trees (JT) and lnkage trees for makng connectons between the neghbourng agents to communcate among themselves. Agents communcate through the messages passed over the LJF lnkage trees and belef updates n each LJF local juncton tree (JT) are performed upon the arrval of a new nter-agent message. Jan Zzka et al. (Eds) : CCSEIT, AIAP, DMDB, MoWN, CoSIT, CRIS, SIGL, ICBB, CNSA-2016 pp , CS & IT-CSCP 2016 DOI : /cst

2 68 Computer Scence & Informaton Technology (CS & IT) However the computatonal cost of exact nference makes t mpractcal for larger and complex domans. So the approxmate nference algorthms are beng used to estmate the posteror belefs. Hence, t s very mportant to study the practcablty and convergence propertes of samplng algorthms on large Bayesan networks. To date there are many stochastc samplng algorthms proposed for Bayesan Networks and are wdely used n BN approxmaton but ths area s qute problematc, snce many attempts have been made n developng MSBN approxmaton algorthms but all of these forgo the LJF structure and sample MSBN drectly n global context. Also t has been shown that such type of approxmaton requres more nter-agent message passng and also leaks the prvacy of local subnet [3]. So, samplng MSBN n global context s not good dea as t analyses only small part of entre mult-agent doman space. So n order to examne local approxmaton and to mantan LJF framework, the samplng process s to be done at each agent s subnet. The LJFbased Local adaptve Importance Sampler (LLAIS) [3] s an example of extenson of BN Importance samplng technques to JT s. An mportant aspect of ths algorthm s that t facltates nter-agent message calculaton along wth the approxmaton of the posteror probabltes. So far the applcaton of LLAIS s done on smaller network consstng of 37 nodes whch s treated as local JT n LJF. LLAIS produced good estmates of local posteror belefs for ths smaller network but ts further testng on larger szes of local JTs s not reported yet. We tested LLAIS for ts scalablty and relablty on the three larger networks treatng them as local JTs n LJF. It s mportant to test the algorthm snce the sze of local JT can vary and can go beyond 37 nodes network, on whch prelmnary testng has been done. Our testng demonstrated that wthout tunng of parameters, LLAIS s qute scalable for Halfnder (56 nodes) and Wn95pts (76 nodes) but once t s appled to Pathfnder (109 nodes) network ts performance deterorates. Further, when these parameters are tuned properly t resulted n sgnfcant mprovement n the performance of algorthm, now t requres less number of samples and less updates than requred by the orgnal algorthm to gve better results. 2. BACKGROUND 2.1 Multply Sectoned Bayesan Networks (MSBNs) In ths paper, we assume that the reader s famlar wth Bayesan networks (BNs) and basc probablty theory [4]. The Multply Sectoned Bayesan Networks (MSBNs) [2] extend the tradtonal BN model from a sngle agent orented paradgm to the dstrbuted mult-agent paradgm and provdes a framework to apply probablstc nference n dstrbuted mult-agent systems. Under MSBNs, a large doman can be modelled modularly and the nference task can be performed n coherent and dstrbuted fashon. The MSBN model s based on the followng fve assumptons: 1. Agent s belef s represented as probablty. 2. Agents communcate ther belefs based on a small set of shared varables. 3. A smpler agent organzaton s preferred. 4. A DAG s used to structure each agent s knowledge. 5. An agent s local JPD admts the agent s belef of ts local varables and the shared varables wth other agents.

3 Computer Scence & Informaton Technology (CS & IT) 69 Fgure 1: (a) A BN (b) A small MSBN wth three subnets (c) the correspondng MSBN hypertree. Fgure 2. An MSBN LJF shown wth ntal potentals assgned to all the three subnets. MSBN consst of set of BN subnets where each subnet represents the partal vew of a larger problem doman. The unon of all subnet DAGs must also be DAG, denoted byg. These subnets are organsed nto a tree structure called a hypertree [2] denoted byψ. Each hypertree node, known as hypernode, corresponds to a subnet; each hypertree lnk, known as hyperlnk, corresponds to a d-sepset, whch s set of shared varables between the adjacent subnets. A hypertree ψ s purposely structured so that (1) for any varable x contaned n more than one subnet wth ts parents ( x) varables between two subnets π n G, there must exst a subnet contanng π ( x) N and ; (2) shared N j are contaned n each subnet on the path between N and N j n ψ. A hyperlnk renders two sdes of the network condtonally ndependent smlar to the separator n a juncton tree (JT). Fg. 1 (a) shows BN whch s sectoned nto MSBN wth three subnets n Fg. 1(b) and Fg. 1(c) shows the correspondng hypertree structure. A derved secondary structure called lnked juncton tree forest(ljf) s used for nference n MSBNs; t s constructed through a process of cooperatve and dstrbuted complaton where each hypernode n hypertree ψ s transformed nto local JT, and each hyperlnk s transformed nto a lnkage tree, whch s a JT constructed from d- sepset. Each cluster of a lnkage tree s called a lnkage, and each separator, a lnkage separator. The cluster n a local JT that contans a lnkage s called a lnkage host. Fg. 2 shows the LJF constructed from the MSBN n Fg.1 (b) and (c). Local JTs, T 0, T1 and T2 are constructed from BN subnets G 0, G 1 and G respectvely, are enclosed by boxes wth sold edges. The lnkage 2 trees; L 20( L 02 ) and L ( ) 21 L 12, are enclosed by boxes wth dotted edges. The lnkage tree L20 contans two lnkages { a, b, c} and { b, c, d} wth lnkage separator bc (not shown n the fgure). The lnkage hosts of T0 for L02 are clusters { a, b, c} and { b, c, d}.

4 70 Computer Scence & Informaton Technology (CS & IT) 3. BASIC IMPORTANCE SAMPLING FOR LJF Here we assume that readers are aware of basc mportance samplng for LJF local JT. The research done so far has hghlghted the dffcultes n applyng stochastc samplng to MSBNs at a global level [5]. Drect local samplng s also not feasble due to the absence of a vald BN structure [3]. However, an LJF local JT can be calbrated wth a margnal over all the varables [6] makng local samplng possble. Algorthms proposed earler combne samplng wth JT belef propagaton but do not support effcent nter-agent message calculatons n context of MSBNs. The [3] ntroduced a JT-based mportance sampler by defnng an explct form of the mportance functon so that t facltates the learnng of the optmal mportance functon. The JPD over all the varables n a calbrated local JT can be obtaned smlar to Bayesan network DAG factorzaton. Let C C Κ, C 1, 2 Κ ntersecton property. The separator m be the m JT clusters gven n the orderng whch satsfes the runnng S = for = 1 and S C C C Κ Κ C ) for = ( = 2,3, Κ, m. Snce S C, the resduals are defned as R = C \ S. The juncton tree runnng ntersecton property guarantees that the separator S separates the resdual R from the set ( C C Κ Κ C ) 1 \ S n JT. 1 2 Thus applyng the chan rule to partton the resdues gven by the separators and have JPD m expressed as P( C, Κ Κ, C ) ( ) 1 m = P R = 1 S. The man dea s to select the root from the JT clusters and then drectng all the separators away from the root formng a drected samplng JT. It s analogous to BN snce both follow recursve form of factorzaton. Once the JPD has been defned for LJF local JT, the mportance functon defned as: m P '( X \ E) = P R = ( \ E S) 1 E= e (1) P' n basc sampler s The vertcal bar n P ( R \ E S ) E= e ndcates the substtuton of e for E n P ( R \ E S ). Ths mportance functon s factored nto set of local components each correspondng to the JT clusters. It means when the calbrated potental s gven on each JT cluster C we can easly compute for every cluster the value of P R S ) drectly. For the root cluster: P( R S ) = P( R ) = P( C ), = 0. ( We traverse a samplng JT and sample varables of the resdue set n each cluster correspondng to the local condtonal dstrbuton. Ths samplng s smlar to the BN samplng except now group of nodes are beng sampled and not the ndvdual nodes. Whenever cluster s encountered wth the node n the evdence set E, t wll be assgned value whch s gven by evdence assgnment. A complete sample consst of the assgnment to all the non- evdence nodes accordng to the local JT s pror dstrbuton. The score for each sample can be computed as: P( S, E) Score = (2) P'( S ) The score so computed n Equaton 2 wll be used n LLAIS algorthm for adaptve mportance samplng. It s proven that the optmal mportance functon for BN mportance samplng s the posteror dstrbuton P ( X E = e) [7]. Applyng ths result to JTs, we can defne the optmal mportance functon as:

5 Computer Scence & Informaton Technology (CS & IT) 71 m ρ ( X \ E) = = P( R \ E E = ) (3) e 1 The above Equaton 3 takes nto account the nfluence of all the evdences from all clusters n the sample of current cluster. 3.1 LJF-Based Local Adaptve Importance Sampler (LLAIS) In 2010, LJF local JT mportance sampler called LLAIS [3] was desgned that follows the prncple of adaptve mportance samplng for learnng factors of mportance functon. Ths algorthm was specfcally proposed for the approxmaton of posterors n case of local JT n LJF provdng the framework for calculaton of nter-agent messages between the adjacent local JTs. The sub-optmal mportance functon used for LJF Local Adaptve Importance Samplng s as follows, m ρ ( X \ E) = = P( R \ E S, E = e) (4) 1 Ths mportance functon s represented n the form of set of local tables. Ths mportance functon s learned to approach the optmal samplng dstrbuton. These local tables are called the Clustered Importance Condtonal Probablty Table (CICPT). These CICPT tables are created for each local JT cluster consstng of the probabltes ndexed by the separator to the precedent cluster (based on the cluster orderng n the samplng tree) and condtoned by the evdence. For non-root JT clusters, CICPT table are defned n the form of P( R S, E), and for the JT root cluster, CICPT table are of the form of P( R S, E) = P( C E). The learnng strategy s to learn these CICPT tables on the bass of most recent batch of samples and hence the nfluence of all evdences s counted through the current sample set. These CICPT tables have the structure smlar to the factored mportance functon and are alke to an ICPT table of Adaptve Importance Samplng of BN n the prevous secton 4.1 and are updated perodcally by the scores of samples generated from the prevous tables. Algorthm for LLAIS Step 1. Specfy the total number of samples M, total updates K and update nterval L, Intalze the CICPT tables as n Equaton 4. Step 2. Generate L samples wth the scores accordng to the current CICPT tables. Estmate P' ( R S, e) by normalzng the scores for each resdue set gven the states of separator set. Step 3. Update the CICPT tables based on the followng learnng functon [45]: P K + 1 ( R where η(k) s the learnng rate. k S, e) = (1 η ( k)) P ( R S, e) +η( k) P'( R S, e), Step 4. Modfy the mportance functon f necessary, wth the heurstc of Є-cutoff. For the next update, go to Step 2. Step 5. Generate the samples from the learned mportance functon and calculate scores as n Equaton 2. Step 6. Output the posteror dstrbuton for each node.

6 72 Computer Scence & Informaton Technology (CS & IT) In LLAIS the mportance functon s dynamcally tuned from the ntal pror dstrbuton and samples obtaned from the current mportance functon are used to refne gradually the samplng dstrbuton. It s well known that thck tals are desrable for mportance samplng n BNs. The reason behnd t s that the qualty of approxmaton deterorates n the presence of probabltes due to generaton of large number of samples havng zero weghts [3]. Ths ssue s solved usng the heurstc Є-cutoff [7], the small probabltes are replaced wth Є f less than a threshold Є, and the change s compensated by subtractng the dfference from the largest probablty. 4. IMPROVING LLAIS BY TUNING THE TUNEABLE PARAMETERS The tuneable parameters plays vtal role n the performance of samplng algorthm. There are many tuneable parameters n LLAIS such as the heurstc value of threshold -cutoff, updatng ntervals, number of updates, number of samples and learnng rate dscussed as follows: 1. Threshold -cutoff t s used for handlng very small probabltes n the network. The proper tunng helps the tal of mportance functon not to decay faster, the optmal value for -cutoff s dependent upon the network and plays key role n gettng better precson these experments wth dfferent cut-off values are motvated from [8]. 2. Number of updates and updatng nterval - the number of updates plays an mportant role n the sense that t denotes how many tmes the CICPT table has to be updated so that t wll result n optmal output and updatng nterval denotes the number of samples that have to be updated. 3. Number of samples - plays very mportant role n the stochastc samplng algorthm as the performance of samplng ncreases wth the number of samples. It s always good to have mnmum number of samples that can help you reach better output for t wll be tme and cost effcent 4. Learnng Rate - n [7] s defned as the rate at whch optmal mportance functon wll be a k / k max learned as per the formula η ( k ) = a( ), where a = ntal learnng rate, b = learnng b rate n the last step, k = number of updates and k max = total number of updates. These tuneable parameters are tuned after many experments n whch they were gven heurstcally dfferent values and then checked for performance. Table 1 shows the comparson of values of varous tuneable parameters for orgnal and mproved LLAIS. Table 1: Shows the comparson of values of varous tuneable parameters for orgnal LLAIS and mproved LLAIS. Tunable parameters Orgnal LLAIS Improved LLAIS Number of samp les Number of up dates 5 3 Up datng nterval Nodes wth outcomes <5 Nodes wth outcomes < 5 Threshold value Nodes wth outcomes < 8 Nodes wth outcomes < Else = Else =

7 5. EXPERIMENT RESULTS Computer Scence & Informaton Technology (CS & IT) 73 We used Kevn Murphy s Bayesan Network toolbox n MATLAB for expermentng wth LLAIS. For testng of LLAIS algorthm, the exact mportance functon s computed, whch s consdered to be the optmal one and then ts performance of samplng s compared wth that of approxmate mportance functon n LLAIS. The testng s done on Halfnder (56 nodes), Wn95pts (76 nodes) and Pathfnder (109 nodes), whch are treated as local JT n LJF. The approxmaton accuracy s measured n terms of Hellnger s dstance whch s consdered to be perfect n handlng zero probabltes whch are common n case of BN. From [8], The Hellnger s dstance between two dstrbutons F1 and F2 whch have the probabltes P ( x ) and P ( x ) for state j j = 1,2, Κ Κ, n ) of node respectvely, such that X E 1 j s defned as: 2 j N \ E n ( 2 { P x j= j P x X 1 1( ) 2( j )} H ( F1, F2 ) = (5) n X N \ E where N s the set of all nodes n the network, E s the set of evdence nodes and n s the number of states for node. P ( x ) and P ( x ) are sampled and exact margnal probablty of state j of node. 1 j 2 j 5.1 Experment Results for Testng LLAIS For each of the three networks we generated n total 30 test cases consstng of the three sequences of 10 test cases each. The three sequences nclude 9, 11 and 13 evdence nodes respectvely. For each of the three networks, LLAIS wth exact and approxmate mportance functon s evaluated usng M = 5000samples. Wth LLAIS usng approxmate mportance a k / k max functon, the learnng functon used s η( k ) = a( ) and set a = 0. 4 and b = 0. 14, total b updates K = 5 and each updatng step, L = The exact mportance functon s optmal hence t does not requre updatng and learnng. Fg.4 shows the results for all the 30 test cases generated for Halfnder network. Each test case was run for 10 tmes and average Hellnger s dstance was recorded as a functon of P(E) to measure the performance of LLAIS as P(E) goes more and more unlkely. It can be seen that LLAIS usng approxmate mportance functon performs qute well and shows good scalablty for ths network.

8 74 Computer Scence & Informaton Technology (CS & IT) Fgure 4: Performance comparson of approxmate and exact mportance functon combnng all the 30 test cases generated n terms of Hellnger s dstance for Halfnder network. Fg. 5 shows the results generated for all the 30 test cases generated from Wn95pts network. It can be concluded that for ths network too LLAIS usng approxmate mportance functon shows good scalablty and ts performance s qute comparable wth that usng exact mportance functon. Fg. 6 shows the results generated for all the 30 test cases generated from Pathfnder networkit s seen that for ths network LLAIS performed poor, the reason s the presence of extreme probabltes whch needs to deal wth. Hence LLAIS doesn t prove to be scalable and relable for ths network. Table 2 below shows the comparson of the statstcal results for all the 30 test cases generated usng approxmate and exact mportance functon n LLAIS. Fgure 5: Performance comparson of approxmate and exact mportance functon combnng all the 30 test cases generated n terms of Hellnger s dstance for Wn95pts network.

9 Computer Scence & Informaton Technology (CS & IT) 75 Fgure 6: Performance comparson of approxmate and exact mportance functon combnng all the 30 test cases generated n terms of Hellnger s dstance for Pathfnder network. Table 2: Comparng the statstcal results for all 30 test cases generated for testng LLAIS for all the three networks. Name of ne Halfnde r ne twork Hellnger's Approx. mp Exact mp Mnmum Error Maxmum Error Mean Medan Varance 1.99E E-06 Name of ne Wn95pts ne twork Hellnger's Approx. mp Exact mp Mnmum Error Maxmum Error Mean Medan Varance 3.18E E-05 Name of ne Pathfnde r ne twork Hellnger's Approx. mp Exact mp Mnmum Error Maxmum Error Mean Medan Varance 6.05E E Experment Results for Improved LLAIS After tunng the parameters as dscussed n secton 4, LLAIS shows consderable mprovement n ts accuracy and scalablty wth proper tunng of tunable parameters. Now the Improved LLAIS uses less number of samples and less updates n comparson to the Orgnal LLAIS for gvng posteror belefs. Fg. 7 shows the comparson of performance of Orgnal LLAIS wth Improved LLAIS and t can be seen that Improved LLAIS performs qute well showng good scalablty on Halfnder network.

10 76 Computer Scence & Informaton Technology (CS & IT) Fgure 7: Performance comparson of Orgnal LLAIS and Improved LLAIS for Halfnder network. Hellnger s dstance Fg. 8 shows the comparson of performance of Orgnal LLAIS wth Improved LLAIS for Wn95pts network and t can be seen n the graph that here also Improved LLAIS performed qute well wth less errors as compared to the Orgnal LLAIS. Fgure 8: Performance comparson of Orgnal LLAIS and Improved LLAIS for Wn95pts network. Hellnger s dstance for each of the 30 test cases plotted aganst P (E) Fg 9 shows the comparson of performance of Improved LLAIS wth Orgnal LLAIS. The most extreme probabltes are found n ths network, hence adjustments wth threshold values played a key role n mprovng the performance; hence after tunng the parameters Improved LLAIS showed better performance n comparson to the orgnal one for ths network. Table 3 shows the comparson of statstcal results from all 30 test cases generated for Improved LLAIS and Orgnal LLAIS.

11 Computer Scence & Informaton Technology (CS & IT) 77 Fgure 9: Performance comparson of orgnal LLAIS and mproved LLAIS for Pathfnder network. Hellnger s dstance for each of the 30 test cases plotted aganst P (E) Table 3: shows the comparson of results for orgnal LLAIS and Improved LLAIS from all 30 test cases generated. Name of ne Halfnde r ne twork Hellnger's Orgnal Improve d Mnmum Error Maxmum Error Mean Medan Varance 7.08E E-06 Name of ne Wn95pts ne twork Hellnger's Orgnal Improve d Mnmum Error Maxmum Error Mean Medan Varance 6.45E E-06 Name of ne Pathfnde r ne twork Hellnger's Orgnal Improve d Mnmum Error Maxmum Error Mean Medan Varance 7.80E E CONCLUSION AND FUTURE WORKS LLAIS s the extenson of BN mportance samplng to JTs. Snce the prelmnary testng of the algorthm was done only on smaller local-jt n LJF of 37 nodes, hence the scalablty and relablty of the algorthm was questonable as the sze of local-jts may vary. From the experments done, t can be concluded that LLAIS wthout parameters tuned performs qute well on local-jt of sze 56 and 76 nodes but ts performance deterorates on 109 nodes network due to presence of extreme probabltes, once the parameters are tuned algorthm shows consderable mprovement n ts accuracy. It has been seen that learnng tme of the optmal mportance functon takes too long, so the choce of ntal mportance functon Pr 0 ( X \ E) close to the

12 78 Computer Scence & Informaton Technology (CS & IT) optmal mportance functon can greatly affect the accuracy and convergence n the algorthm. As mentoned n [3], there s stll one mportant queston that remans unanswered how the local accuracy wll affect the overall performance of the entre network. Further experments are stll to be done on the full scale MSBNs. REFERENCES [1] Karen H. Jn, Effcent probablstc nference algorthms for cooperatve Mult-agent Systemsǁ, Ph.D. dssertaton, Unversty of Wndsor (Canada), [2] Y.Xang, Probablstc Reasonng n Multagent Systems: A Graphcal Models Approachǁ. Cambrdge Unversty Press, [3] Karen H. Jn and Dan Wu, Local Importance Samplng n Multply Sectoned Bayesan Networksǁ, Florda Artfcal Intellgence Research Socety Conference, North Amerca, May [4] Daphne Koller and Nr Fredman, Probablstc Graphcal Models-Prncples and Technques, MIT Press, [5] Y.Xang, Comparson of multagent nference methods n Multply Sectoned Bayesan Networksǁ, Internatonal journal of approxmate reasonng, vol. 33, pp , [6] K.H.Jn and D.Wu, Margnal calbraton n mult-agent probablstc systemsǁ, In Proceedngs of the 20th IEEE Internatonal conference on Tools wth AI, [7] J. Cheng and M. J. Druzdzel, BN-AIS: An adaptve mportance samplng algorthm for evdental reasonng n large Bayesan networksǁ, Artfcal Intellgence Research, vol.13, pp , [8] C. Yuan, Importance Samplng for Bayesan Networks: Prncples, Algorthms, and Performanceǁ, Ph.D. dssertaton, Unversty of Pttsburgh, 2006.