Evaluating Influence Diagrams

Transcription

1 Evalating Inflence Diagrams Where we ve been and where we re going Mark Crowley Department of Compter Science University of British Colmbia Agst 31, 2004 Abstract In this paper we will srvey the history of Inflence Diagrams from their origin in Decision Theory to modern AI ses. We will compare the varios methods that have been sed to find an optimal strategy for a given inflence diagram. These methods have varios advantages nder different assmptions and there is a progression to the present of richer, more general soltions. We also look at an abstract algorithm sed as an informal tool and determine whether it is eqivalent to other formal methods in the literatre. 1 Introdction This paper will look at the development of inflence diagrams from their beginnings in decision analysis to their crrent important place in many areas of compter science inclding artificial intelligence. We will layot the different methods sed to optimize decisions sing inflence diagrams by compting them directly or via conversions to other models sch as decision graphs and bayesian networks. The latter type in particlar will be looked at in depth and it will be contrasted against the performance of varios algorithms. We will also explain an intitive algorithm sed informally by some researchers and analyze its advantages or similarities with methods described in the literatre. The main contribtion of this paper is to identify trends throgh all these soltions over time and help focs work on open qestions and to find new directions being pointed to by the existing literatre. 2 Inflence Diagrams Inflence diagrams (IDs) were proposed by Howard and Matheson [HM03] as a tool to simplify modelling and analysis of decision trees. Decision trees represent each decision or chance variable as a new level in a tree. The leaves of the tree are tilities that express which ending configrations are more desirable. Solving a decision problem reqires finding the optimal path throgh this tree that maximizes expected tility. 1

2 Now instead of representing each level of that tree we represent each variable as single nodes in a graph. An inflence diagram is a directed acyclic graph N containing nodes representing the variables of the decision problem, V = X D U. Each variable has its own domain of vales, dom(v). The set of parents of a node v i is denoted π vi. These variables are of three types, see Figre 1: chance nodes, x i X, represent random variables in the analysis. They have an associated conditional probability table (CPT) representing a distribtion P(x i π xi ). decision nodes, d i D, represent decisions to be made. We se decision rles δ i, to represent the mapping of each permtation of its parents to exactly one decision, δ i (π di ) dom(d i ) tility nodes, denoted i U, represent fnctions that map each permtation of its parents to exactly one tility vale val i (π i ) dom( i ). No other variables are allowed to depend on a tility directly so tility nodes do not have children. An optimal decision rle δ i is one that maximizes E δi [val i (π i )], the expected vale of the tility nodes it effects. The goal in decision analysis is to find an optimal policy, which is the set of all optimal decision rles = {δ 1,...,δ k }, one for each tility node. The expected vale of a tility node given is denoted E [val i (π i )]. Ths, we are seeking: and = {δ i D i U, argmax δi E δi [val i (π i )]} (1) E [N ]= E [val i (π i )] (2) i U Finding these is what we mean by evalating an inflence diagram. x 1 d 1 x 2 d 2 x3 Figre 1: A simple Inflence Diagram 2

3 Definition 1 An inflence diagram is reglar if there is a definite ordering of the decision nodes sch that there is a path from each decision to the next decision. Frther, the ID is no-forgetting if each decision has access to all of the information the previos decisions had. This means that every element of d i π di is a parent of every decision following d i. IDs have several well known advantages over decision trees. They simplify modelling by allowing the analyst to specify single nodes that represent entire probability distribtions over nearly arbitrary relationships with other variables. We still limit orselves to reglarity as defined above and no loops bt this still provides a level of expression not possible with trees. In order to express conditional independence relationships that are natral in an ID we wold need to resort to the se of complicated information sets amongst siblings in a tree. This may help comptationally bt not in terms of constrcting models. We also mst consider that as the nmber of variables in the problem grows, the size of a symmetric tree grows exponentially, which is not tre in the case of inflence diagrams and their CPTs since they need only represent the probabilities associated with their parents in the graph. So inflence diagrams have mch to offer if they can be evalated efficiently. 3 Using Bayesian Networks to Evalate Inflence Diagrams This approach was first pt forward by Cooper [Coo88] who showed primarily how to redce inflence diagrams to bayesian networks. Shachter and Peot [SP92] soon after showed an improved algorithm that was mch more efficient. Later, Zhang [Zha98] introdced a modified method that greatly redced the nmber of nodes being considered in each part of the evalation. Recently Xiang [XY01] has proposed an algorithm that claims to be as efficient as Zhang s bt simpler. We will briefly present the approaches of these algorithms. 3.1 Cooper s Redction Cooper [Coo88] converted the ID problem to a BN problem in the following way. An inflence diagram is essentially very similar to a bayesian network already, all that is reqired is to ensre that all nodes have proper probability distribtions associated with them to allow s to perform inference. We ths proceed by essentially converting all node types to chance nodes. Decision nodes are converted to chance nodes with an even distribtion: α di dom(d i ),P(α di π di )= 1 dom(d i ) We also need to assign some probability to tility nodes to represent their payoff. Cooper assigned the probability of the new binary-valed tility nodes given its parents 3

4 in the BN a probability proportional to the vale of the tility fnction in the ID for the same parents 1 : val (π ) P( = 1 π )= max π val (π ) P( = 0 π )=1 val (π ) max π val (π ) And obviosly no change is needed for the chance nodes from the ID itself. With the constrction complete they then showed that the problem of finding an optimal decision in the ID redces to a problem in the BN 2 : E δi [val (π )] = P( = 1 π,d i ) Their sggested method of solving then is to maximize these δ i for each permtation of π. This can reqire an exponential nmber of inference steps. Therefore later work based themselves on this redction method bt went frther. 3.2 Shachter and Peot s Algorithm This method [SP92] took advantage of the independence strctre of the BN and the reglarity constraint that says there is a defined order of decision nodes. They noted that instead of maximizing P( = 1 π,d i ) they cold instead maximize P(π,d i = 1). This is preferable since the comptations can now be done locally sing standard BN techniqes of belief propagation. We can recrsively optimize each δ i starting from the last contine optimizing δ i 1 ntil we reach the beginning. This method seems very intitive and is actally nearly eqivalent to Shachter s original techniqe from [Sha86] except that now the bayesian network is handling all of the graph maniplations which previosly had to be specified separately. 3.3 Zhang s Algorithm This method was based on previos advances made by Zhang and Poole regarding the notion of stepwise decomposable IDs [ZP92] and bayesian inference techniqes [ZP94]. The intition for this method is that in a reglar, no-forgetting ID the parents of each decision divide the graph into two independent partitions, see Figre 3.3. They show that if we consider the last decision in the ordering d k, called the tail decision node, then it partitions the network in an interesting way. Call the partition containing the tail decision node T. This set of nodes is separated from nodes previos to those in π di. It also only incldes nodes in π dk and V 2 that can effect the tilities in V 2, denoted U i. The optimal decision rle for d i can then be compted as follows: δ i (π d i )=arg max δi P T ( = 1 π di,d i ) (3) U 2 1 In this and the following section the algorithms limit the ID to only one tility node, therefore sbscripts on s are not necessary at this point 2 we inclde d i simply for emphasize bt it is not necessary since d i π 4

5 V 1 π di d i V 2 Figre 2: A simple Inflence Diagram Note that P T ( ) indicates that the probability is compted relative to nodes only in T since all other nodes earlier in the graph are irrelevant to this decision. This greatly redces the comptational task for compting the optimal tail decision. Previos methods always inclded the entire graph in their calclations even thogh they had no impact. The algorithm then proceeds to recrsively divide the BN into a body and a tail, optimizing the tail decision, setting that decision to have decision rle δ i and repeating on the remaining body. Dring this process more nodes can be prned from the redced tail to improve comptation frther. It trns ot that after we consider this we can see that the reqirement of noforgetting can now be dropped as long as reglarity is maintained. That is, a decision d i need not have incoming arcs from all d 1...d i 1 previos to it as long as there is still a rote going throgh d 1...d i in order. This means that previos decisions can now be hidden from s with chance nodes as shown in Figre 3. The information is not really hidden since d i is still conditionally dependent on d i 1 as long as c i 1 is not observed so the decision will still inflence ftre ones. Using Zhang s algorithm d i can still be optimized. So this opens p inflence diagram evalation to less restricted forms. V1 πdi V2 xi 2 xi 1 xi di Figre 3: A inflence diagram with no-forgetting rle dropped 5

6 3.4 Other algorithms Xiang and Ye [XY01] have recently pblished an algorithm sing BNs that claims to be as efficient as other described here and simpler than Zhang s in particlar. Frther analysis is reqired to determine if these claims are significant as the paper does not state any conclsions directly. Mch work has also occrred arond evalating inflence diagrams withot sing BNs bt sing other methods sch as Shenoy s valation based networks [She92] and some recent work by Dechter sing BNs bt with bcket elimination [Dec00]. 4 An Abstract Algorithm The following method has been proposed informally by David Poole and follows from several intitive observations. Assme we have inflence diagrams as described already which are represented as bayesian networks in the standard way as described by Cooper [Coo88]. Also assme that we have the variable elimination (VE) algorithm [ZP94]. Using VE we can specify to eliminate a node and a factor will be prodced to distribte away that node s probability to effected nodes in the graph. With these tools in hand we can simply specify a method for evalating inflence diagrams. We define U d to be the set of all U sch that d an(). This is the set of tility nodes that d needs to maximize. We note that as long as the parents of these tility nodes, π Ud, are all accessible to d then d can be optimized easily, as in Figre 4. Ths the algorithm eliminates any nodes that are parents of a node in U d bt not d. for all d D do {chosen in reverse order} while π Ud (π d d) do Let x be any node in (π Ud (π d d)) eliminate x sing Variable Elimination algorithm add all nodes in π x to π Ud end while for all instantions of variables in π d do δ (π d )=arg max d (P( = 1 π d,d)) {choose vale for d that maximizes U d } end for make d a chance node by setting its CPT to δ end for We see in Consider the ID N shown in Figre 4. In this sitation we can clearly see that the decision d has all the necessary information to decide optimally. For each permtation of π d choose each decision α dom(d) sch that α = arg max α (val (π,α)) After this we will have δ so we assign that to d s CPT to trn it into another chance node Then we eliminate d k which pdates the table of to take this decision rle into accont 6

7 x 1 x 2 d x 3 Figre 4: Base case of redction sing the algorithm Now we can eliminate x 1,x 2,x 3 to give s the tility node which is really jst E δ [N ] With this as or base case the rest of the method involves being able to redce more complicated IDs to ones containing the form in Figre 4. In Figre 5 we can see another simple case. Here the only node inhibiting optimizing d is x 4 since d does not observe it. With VE its easy to fix this, we simply eliminate x 4 which will pdate the CPTs for,x 1,d k. x 4 x 1 x 2 d x 3 Figre 5: Another simple case of redction sing the algorithm Let s now consider the ID shown previosly for Zhang s algorithm, Figre 3. Can or intitive algorithm evalate this sitation? If we consider the steps shown in Figre 6 it wold seem the answer is yes. Any information arc that wold come from d i 1 wold be irrelevant since x i 1 already provides information to s and it does not effect the actal tility. Also, since we are converting d i to a chance node once it is optimized, the no-forgetting constraint does not apply and d i can be eliminated sing VE withot complication. It is not srprising this method performs as well as Zhang s algorithm. The definition of stepwise-decomposability partitions nodes in the graph sing the same fndamental rles that govern conditional independence on which VE is based. The VE algorithm for marginalizing the probabilities inherently does comptations locally and 7

8 xi 2 xi 1 xi xi xi 2 xi 1 di (i) (ii) xi 2 xi 1 xi 2 (iii) (iv) Figre 6: Evalating a forgetfl ID. (i)tail decision node optimized (ii)-(iv) removing other chance nodes before next decision node need not rely on nodes frther back in the graph that are irrelevant given the parents of a node. Zhang s algorithm sets p this framework independent of VE however and ths is general enogh to se any bayesian inference techniqe that works. Or intitive algorithm is basically a special case of Zhang s where we have hardcoded the se of the inference techniqe and ths mch of the other complexity of the algorithm is taken care of already. 5 Conclsion We have shown a brief overview of the vast and still growing field of inflence diagram research. We have discssed their basis in decision analysis and shown some of the major algorithms for evalating them in order to come to an optimal policy that maximizes tility. Many of these algorithms tilize bayesian networks and we have also shown an informal algorithm from intition sed by researchers that is a special case of the algorithm by Zhang [Zha98]. There are many open areas of research being actively prsed inclding ways to frther improve evalation efficiency and extend inflence diagrams to ever more general realms. 8

9 References [Coo88] G.F. Cooper. A method for sing belief networks as inflence diagrams. In Procceedings of the Twelfth Conference on Uncertainty in Artificial Intelligence, pages 55 63, [Dec00] Rina Dechter. A new perspective on algorithms for optimizing policies nder ncertainty. American Association for Artificial Intelligence, [HM03] R. A. Howard and J. E. Matheson. Inflence diagrams. In Readings on the Principles and Applications of Decision Analysis, pages Strategic Decisions Grop, [Sha86] R.D. Shachter. Evaltating inflence diagrams. Operations Research, [She92] P.P. Shenoy. Valation-based systems for bayesian decision analysis. Operations Research, pages , [SP92] R.D. Shachter and Peot. Decision making sing probabilistic inference methods. In Procceedings of the Eigth Conference on Uncertainty in Artificial Intelligence, pages , [XY01] Y. Xiang and C. Ye. A simple method to evalate inflence diagrams. In Third International Conference on Cognitive Science, [Zha98] Nevin Zhang. Probabilistic inference in inflence diagrams. In Proceedings of the Forteenth Conference on Unvertainty in Artificial Intelligence, pages , [ZP92] [ZP94] Nevin Zhang and David Poole. Stepwise decomposable inflence diagrams. In Proceedings of the Forth International Conference on Knowledge Representation and Reasoning, pages , Nevin Zhang and David Poole. A simple approach to bayesian network comptations. In Proceedings of the Tenth Canadian Conference on Artificial Intelligence, pages ,