AND/OR MultiValued Decision Diagrams for Constraint Networks


 Opal Haynes
 1 years ago
 Views:
Transcription
1 N/OR MultiValued ecision iagrams for onstraint Networks Robert Mateescu, and Rina echter 2 lectrical ngineering epartment, alifornia Institute of Technology Pasadena, onald ren School of Information and omputer Science, University of alifornia, Irvine Irvine, bstract. The paper is an overview of a recently developed compilation data structure for graphical models, with specific application to constraint networks. The N/OR MultiValued ecision iagram (OM) augments well known decision diagrams (Os, Ms) with N nodes, in order to capture function decomposition structure. The OM is based on a pseudo tree of the network, rather than a linear ordering of its variables. The OM of a constraint network is a canonical form given a pseudo tree. We describe two main approaches for compiling the OM of a constraint network. The first is a top down, searchbased procedure, that works by applying reduction rules to the trace of the memory intensive N/OR search algorithm. The second is a bottom up, inferencebased procedure, that uses a ucket limination schedule. or both algorithms, the compilation time and the size of the OM are, in the worst case, exponential in the treewidth of the constraint graph, rather than pathwidth as is known for ordered binary decision diagrams (Os). Introduction The paper is an overview of N/OR MultiValued ecision iagrams (OMs) as a compiled data structure for constraint networks. We present here an extension of the work in [], while still maintaining the focus on constraint networks. OMs for weighted graphical models and for constraint optimization were presented in [2, 3]. The OM is based on two existing frameworks: () N/OR search spaces for graphical models; (2) decision diagrams. N/OR search spaces [4 6] have proven to be a unifying framework for various classes of search algorithms for graphical models. The main novelty is the exploitation of independencies between variables during search, which can provide exponential speedups over traditional search methods that can be viewed as traversing an OR structure. The N nodes capture problem decomposition into independent subproblems, and the OR nodes represent branching according to variable values. ecision diagrams are widely used in many areas of research, especially in software and hardware verification [7, 8]. represents a oolean function by a directed This work was done while at the University of alifornia, Irvine.
2 acyclic graph with two sink nodes (labeled 0 and ), and every internal node is labeled with a variable and has exactly two children: low for 0 and high for. If isomorphic nodes were not merged, we would have the full search tree explored by the backtracking algorithm. The tree is ordered if variables are encountered in the same order along every branch. It can then be compressed by merging isomorphic nodes (i.e., with the same label and identical children), and by eliminating redundant nodes (i.e., whose low and high children are identical). The result is the celebrated reduced ordered binary decision diagram, or O for short, introduced by ryant [9]. owever, the underlying structure is OR (i.e., linear structure rather than tree). If N/OR search trees are reduced by node merging and redundant nodes elimination we get a compact search graph that can be viewed as a representation augmented with N nodes. This paper shows how to combine the two ideas, creating a decision diagram that has an N/OR structure, thus exploiting problem decomposition. s a detail, the number of values is also increased from two to any constant. In the context of constraint networks, decision diagrams can be used to represent the whole set of solutions, facilitating solutions count, solution enumeration and queries on equivalence of constraint networks. The benefit of moving from OR structure to N/OR is in a lower complexity of the algorithms and size of the compiled structure. It typically moves from being bounded exponentially in pathwidth pw, which is characteristic to chain decompositions or linear structures, to being exponentially bounded in treewidth w, which is characteristic of tree structures (it always holds that w pw and pw w log n). In both cases, the compiled structure achieved in practice is often far smaller than what the bounds suggest. decision diagram offers a compilation of a propositional knowledgebase. multivalued N/OR decision diagram extends compilation to general graphical models. The knowledge compilation approach has become an important research direction in automated reasoning in the past decades [0 2]. Typically, a knowledge representation language is compiled into a compact data structure on which various queries can be answered quickly. ccordingly, the computational effort can be divided between an offline and an online phase where most of the work is pushed offline. ompilation can also be used to generate compact building blocks to be used by online algorithms multiple times. Macrooperators compiled during or prior to search can be viewed in this light [3], while in graphical models the building blocks are the functions whose compact, compiled, representation can be used effectively across many tasks. s one example, consider product configuration tasks and imagine a user that chooses sequential options to configure a product. In a naive system, the user would be allowed to choose any valid option at the current level based only on the initial constraints, until either the product is configured, or else, when a deadend is encountered, the system would backtrack to some previous state and continue from there. This would in fact be a search through the space of possible partial configurations. Needless to say, it would be very unpractical, and would offer the user no guarantee of finishing in a limited time. system based on compilation would actually build the backtrackfree search space in the offline phase, and represent it as compactly as possible. In the online phase, only valid partial configurations (i.e., that can be extended to a full valid
3 configuration) are allowed, and depending on the query type, response time guarantees can be offered in terms of the size of the compiled structure. Numerous other examples, such as diagnosis and planning problems can be formulated as graphical models, and for which compilation would be useful. ompilation in diagnosis can facilitate fast detection of possible faults or explanations for some unusual behavior. In planning, compilation would allow swift adjustments according to changes in the environment. Probabilistic models are one of the most used types of graphical models, and the basic query is to compute conditional probabilities of some variables given the evidence. compact compilation of a probabilistic model would allow fast response for any change in the evidence along time. ormal verification is another example where compilation is heavily used to compare equivalence of circuit design, or to check the behavior of a circuit. inary ecision iagram () [9] are arguably the most widely known and used compiled structure. Our OM proposal is related to two earlier research lines within the literature. The first is the work on isjoint Support ecompositions (S) [4], investigated within the area of design automation [5], that were proposed as enhancements for s aimed at exploiting function decomposition. The second is the work on trees [6]. nother related proposal is the recent work by argier and Vilarem [7] on compiling SPs into treedriven automata. We will comment more on the relationship between these work and OM in the related work section. The structure of the paper is as follows. Section 2 provides the preliminaries. Section 3 gives and overview of N/OR search spaces. Section 4 introduces the OM and Section 5 shows that it is a canonical form for constraint networks. Section 6 describes a search based algorithm for compiling the OM. Section 7 presents a compilation algorithm based on a ucket limination schedule and the PPLY operation. Section 8 presents related work and Section 9 concludes. 2 Preliminaries constraint network and its associated graph are defined in the usual way: efinition (constraint network). constraint network is a 3tuple R = X,,, where: X = {X,...,X n } is a set of variables; = {,..., n } is the set of their finite domains of values, with cardinalities k i = i and k = max n i= k i ; = {,..., r } is a set of constraints over subsets of X. ach constraint is defined as = (S i,r i ), where S i is the set of variables on which the constraint is defined, called its scope, and R i is the relation defined on S i. efinition 2 (constraint graph). The constraint graph (or primal graph) of a constraint network is an undirected graph, = (X, ), that has variables as its vertices and an edge connecting any two variables that appear in the scope (set of arguments) of the same constraint. pseudo tree resembles the tree rearrangements introduced in [8]: efinition 3 (pseudo tree). pseudo tree of a graph = (X,) is a rooted tree T having the same set of nodes X, such that every arc in is a backarc in T (i.e., it connects nodes on the same path from root).
4 f() (a) Table 0 0 (b) Ordered tree (c) Isomorphic nodes (d) Redundant nodes (e) O ig.. oolean function representation and reduction rules efinition 4 (induced graph, induced width, treewidth, pathwidth). n ordered graph is a pair (,d), where is an undirected graph, and d = (X,...,X n ) is an ordering of the nodes. The width of a node in an ordered graph is the number of neighbors that precede it in the ordering. The width of an ordering d, denoted w(d), is the maximum width over all nodes. The induced width of an ordered graph, w (d), is the width of the induced ordered graph obtained as follows: for each node, from last to first in d, its preceding neighbors are connected in a clique. The induced width of a graph, w, is the minimal induced width over all orderings. The induced width is also equal to the treewidth of a graph. The pathwidth pw of a graph is the treewidth over the restricted class of orderings that correspond to chain decompositions. 2. inary ecision iagrams Review ecision diagrams are widely used in many areas of research to represent decision processes. In particular, they can be used to represent functions. ue to the fundamental importance of oolean functions, a lot of effort has been dedicated to the study of inary ecision iagrams (s), which are extensively used in software and hardware verification [7, 8]. ryant [9] introduced the Ordered inary ecision iagram (O). The order of variables along any path of an O is the same. Os provide a compact representation and efficient operations (the apply procedure, that combines two Os by an operation is at most quadratic in the sizes of the input diagrams). xample. igure (a) shows a table representation of a oolean function. binary tree representation is shown in igure (b). The internal round nodes represent the variables, the solid edges are the (or high) value, and the dotted edges are the 0 (or low) value. The leaf square nodes show the value of the function for each assignment along a path. The tree is ordered, because variables appear in the same order along each path. There are two reduction rules that transform a decision diagram into an equivalent one: () isomorphism: merge nodes that have the same label and the same children; (2)
5 () 2 () 3 () 4 () (a) onstraint network : h 4 () : 2 () h 3 () h 2 () : 3 () : () h () : 4 () (b) execution bucket bucket bucket bucket bucket (c) ucket tree ig. 2. ucket limination redundancy: eliminate nodes whose low and high edges point to the same node, and connect parent of removed node directly to child of removed node. pplying the two reduction rules exhaustively yields a reduced O, sometimes denoted ro. We will just use O and assume that it is completely reduced. xample 2. igure (c) shows the binary tree from (b) after the isomorphic terminal nodes have been merged. The highlighted nodes, labeled with, are isomorphic, and igure (d) shows the result after they are merged. Now, the highlighted nodes labeled with and are redundant, and removing them gives the O in igure (e). 2.2 ucket limination Review ucket limination () [9] is a well known variable elimination algorithm for inference in graphical models. n ordering d = (X,X 2,...,X n ) of the variables guides the execution of. ach variable is associated with a bucket. ach constraint from is placed in the bucket of its latest variable in d. uckets are processed from X n to X by eliminating the bucket variable (the constraints residing in the bucket are joined together, and the bucket variable is projected out) and placing the resulting constraint (also called message) in the bucket of its latest variable in d. fter its execution, renders the network backtrack free, and a solution can be produced by assigning variables along d. can also produce the solutions count if marginalization is done by summation (rather than projection) over the functional representation of the constraints, and join is substituted by multiplication. also constructs a bucket tree, by linking the bucket of each X i to the destination bucket of its message (called the parent bucket). node in the bucket tree typically has a bucket variable, a collection of constraints, and a scope (the union of the scopes of its constraints). If the nodes of the bucket tree are replaced by their respective bucket variables, it is easy to see that we obtain a pseudo tree of the constraint graph. xample 3. igure 2(a) shows a network with four constraints. igure2(b) shows the execution of ucket limination along d = (,,,,). The buckets are processed from to 3. igure 2(c) shows the bucket tree. The pseudo tree corresponding to the order d is given in ig. 3(a). 3 igure 2 reverses the top down bucket processing described in earlier papers.
6 [ ] 0 0 [] [] [] [] (a) Pseudo tree (b) Search tree (c) ontext minimal graph ig. 3. N/OR search tree 3 Overview of N/OR Search Space for onstraint Networks The N/OR search space is a recently introduced [4 6] unifying framework for advanced algorithmic schemes for graphical models. Its main virtue consists in exploiting independencies between variables during search, which can provide exponential speedups over traditional search methods oblivious to problem structure. Since N/OR MultiValued ecision iagrams are based on N/OR search spaces, we provide a comprehensive overview for completeness sake. 3. N/OR Search Tree The N/OR search tree is guided by a pseudo tree of the constraint graph. The idea is to exploit the problem decomposition into independent subproblems during search. ssigning a value to a variable (conditioning) is equivalent in graph terms to removing that variable (and its incident edges) from the constraint graph. partial assignment can therefore lead to the decomposition of the residual constraint graph into independent components, each of which can be searched (or solved) separately. The pseudo tree captures precisely all these decompositions, given an order of variable instantiation. efinition 5 (N/OR search tree of a constraint network). iven a constraint network R = X,,, its constraint graph and a pseudo tree T of, the associated N/OR search tree has alternating levels of OR and N nodes. The OR nodes are labeled X i and correspond to variables. The N nodes are labeled X i,x i and correspond to value assignments. The structure of the N/OR search tree is based on T. The root is an OR node labeled with the root of T. The children of an OR node X i are N nodes labeled with assignments X i,x i that are consistent with the assignments along the path from the root. The children of an N node X i,x i are OR nodes labeled with the children of variable X i in the pseudo tree T. The leaves of N nodes are labeled with. There is a one to one correspondence between solution subtrees of the N/OR search graph and solutions of the constraint network [4]. xample 4. igure 3 shows an example of an N/OR search tree for the constraint network given in igure 2(a), assuming all tuples are consistent, and variables are binary valued. When some tuples are inconsistent, some of the paths in the tree do not exists. igure 3(a) gives the pseudo tree that guides the search, from top to bottom, as indicated
7 by the arrows. The dotted arcs are backarcs from the primal graph. igure 3(b) shows the N/OR search tree, with the alternating levels of OR (circle) and N (square) nodes, and having the structure indicated by the pseudo tree. The N/OR search tree can be traversed by a depth first search algorithm, thus using linear space. It was already shown [8, 20, 2, 4, 6] that: Theorem. iven a constraint network R and a pseudo tree T of depth m, the size of the N/OR search tree based on T is O(n k m ), where k bounds the domains of variables. constraint network of treewidth w has a pseudo tree of depth at most w log n, therefore it has an N/OR search tree of size O(n k w log n ). The N/OR search tree expresses the set of all possible assignments to the problem variables (all solutions). The difference from the traditional OR search space is that a solution is no longer a path from root to a leaf, but rather a subtree, defined as follows: efinition 6 (solution subtree). solution subtree of an N/OR search tree contains the root node. or every OR nodes it contains one of its child nodes and for each of its N nodes it contains all its child nodes, and all its leaf nodes are consistent. 3.2 N/OR Search raph The N/OR search tree may contain nodes that root identical conditioned (on the current path assignment) subproblems. These nodes are said to be unifiable. When unifiable nodes are merged, the search space becomes a graph. Its size becomes smaller at the expense of using additional memory by the search algorithm. The depth first search algorithm can therefore be modified to cache previously computed results, and retrieve them when the same nodes are encountered again. The notion of unifiable nodes is defined formally next. efinition 7 (minimal N/OR graph, isomorphism). Two N/OR search graphs and are isomorphic if there exists a one to one mapping σ from the vertices of to the vertices of such that for any vertex v, if σ(v) = v, then v and v root identical subgraphs relative to σ. The minimal N/OR graph is such that all the isomorphic subgraphs are merged. Isomorphic nodes (that root isomorphic subgraphs) are also said to be unifiable. Theorem 2 ([6]). The minimal N/OR search graph of a constraint network R relative to a pseudotree T is unique. Note that the definition of minimality in [6] is based only on isomorphism reduction. We extend it by also eliminating the redundant nodes. The previous theorem only shows that given an N/OR graph, the merge operator has a fixed point, which is the minimal N/OR graph. It can be shown that the OM is a canonical representation (given a pseudo tree), namely that any two equivalent constraint networks can be represented by the same unique OM, and the OM is minimal in terms of number of nodes. Some unifiable nodes can be identified based on their contexts. We can define graph based contexts for both OR nodes and N nodes, just by expressing the set of ancestor
8 variables in T that completely determine a conditioned subproblem. owever, it can be shown that using caching based on OR contexts makes caching based on N contexts redundant and vice versa, so we will only use OR caching. ny value assignment to the context of X separates the subproblem below X from the rest of the network. efinition 8 (OR context). iven a pseudo tree T of an N/OR search space, context(x) = [X...X p ] is the set of ancestors of X in T, ordered descendingly, that are connected in the primal graph to X or to descendants of X. efinition 9 (context unifiable OR nodes). iven an N/OR search graph, two OR nodes n and n 2 are context unifiable if they have the same variable label X and the assignments of their contexts is identical. Namely, if π is the partial assignment of variables along the path to n, and π 2 is the partial assignment of variables along the path to n 2, then their restriction to the context of X is the same: π context(x) = π 2 context(x). The depth first search algorithm that traverses the N/OR search tree, can be modified to traverse a graph, if enough memory is available. We could allocate a cache table for each variable X, the scope of the table being context(x). The size of the cache table for X is therefore the product of the domains of variables in its context. or each variable X, and for each possible assignment to its context, the corresponding conditioned subproblem is solved only once and the computed value is saved in the cache table, and whenever the same context assignment is encountered again, the value of the subproblem is retrieved from the cache table. Such an algorithm traverses what is called the context minimal N/OR graph. efinition 0 (context minimal N/OR graph). The context minimal N/OR graph is obtained from the N/OR search tree by merging all the context unifiable OR nodes. It was already shown that [20, 4, 6]: Theorem 3. iven a constraint network R, its primal graph and a pseudo tree T, the size of the context minimal N/OR search graph based on T, and therefore the size of its minimal N/OR search graph, is O(n k w T () ), where wt () is the induced width of over the depth first traversal of T, and k bounds the domain size. xample 5. We refer again to igure 3. igure 3(a) shows the pseudo tree, where the (OR) context of each node appears in square brackets. Notice that the context of a node is identical to the message scope from its bucket in ig. 2. igure 3(c) shows the context minimal N/OR graph. 4 N/OR MultiValued ecision iagram (OM) The context minimal N/OR graph (efinition 0) offers an effective way of identifying some unifiable nodes during the execution of the search algorithm. Namely, context unifiable nodes are discovered based only on their paths from the root, without actually
9 solving their corresponding subproblems. owever, merging based on context is not complete, which means that there may still exist unifiable nodes in the search graph that do not have identical contexts. Moreover, some of the nodes in the context minimal N/OR graph may be redundant, for example when the set of solutions rooted at variable X i does not depend on the specific value assigned to X i (this situation is not detectable based on context). This is sometimes termed as interchangeable values or symmetrical values. We propose to augment the minimal N/OR search graph with removing redundant variables as is common in O representation, as well as to adopt notational conventions common in this community. This yields a data structure that we call N/OR, that exploits decomposition by using N nodes. We present the extension over multivalued variables yielding N/OR M or OM. Subsequently we present two algorithms for compiling the canonical OM of a constraint network: the first is search based, and uses the memory intensive N/OR graph search to generate the context minimal N/OR graph, and then reduces it bottom up by applying reduction rules; the second is inference based, and uses a ucket limination schedule to combine the OMs of initial functions by PPLY operations (similar to the apply for Os). oth approaches have the same worst case complexity as the N/OR graph search with context based caching, and also the same complexity as ucket limination, namely time and space exponential in the treewidth of the problem, O(n k w ). 4. rom N/OR Search raphs to ecision iagrams We will now show how we can process an N/OR search graph by reduction rules similar to the case of Os, in order to obtain a representation of minimal size. In the case of Os, a node is labeled with a variable name, for example, and the low (dotted line) and high (solid line) outgoing arcs capture the restriction of the function to the assignments = 0 or =. To determine the value of the function, one needs to follow either one or the other (but not both) of the outgoing arcs from. The straightforward extension of Os to multivalued variables (multivalued decision diagrams, or Ms) was presented in [22]. We generalize the O and M representations by allowing each outgoing arc to be an N arc. n N arc connects a node to a set of nodes, and captures the decomposition of the problem into independent components. We define the N/OR ecision iagram representation based on N/OR search graphs. We find it useful to introduce the metanode data structure, which defines small portions of any N/OR graph, based on an OR node and its N children: efinition (metanode). metanode u in an N/OR search graph of a constraint network consists of an OR node labeled X (therefore var(u) = X) and its k N children labeled x,...,x k that correspond to the value assignments of X. ach N node labeled x i stores a list of pointers to child metanodes, denoted by u.children i. We also define two special metanodes, that will play the role of the terminal nodes in Os. The terminal metanode 0 indicates the inconsistent assignments, while the terminal metanode indicates the consistent ones.
10 ny N/OR search graph can now be viewed as a diagram of metanodes, simply by grouping OR nodes with their N children, and adding the terminal metanodes appropriately. It is now easy to see when a variable is redundant with respect to the outcome of the function based on the current partial assignment. Intuitively, any assignment to a redundant variable should lead to the same set of solutions. efinition 2 (redundant metanode). iven an N/OR search graph represented with metanodes, a metanode u with var(u) = X and (X) = k is redundant iff u.children =... = u.children k. n N/OR graph, that contains a redundant metanode u, can be transformed into an equivalent graph by replacing any incoming arc into u with its common list of children u.children (joined in an N arc), and then removing u and its outgoing arcs from. The notion of isomorphism is extended naturally from N/OR graphs to metanodes. efinition 3 (isomorphic metanodes). iven an N/OR search graph represented with metanodes, two metanodes u and v having var(u) = var(v) = X and (X) = k are isomorphic iff u.children i = v.children i, i {,...,k}. Naturally, the N/OR graph obtained by merging isomorphic metanodes is equivalent to the original one. We can now define the N/OR MultiValued ecision iagram: efinition 4 (OM). n N/OR MultiValued ecision iagram (OM) is a weighted N/OR search graph that is completely reduced by isomorphic merging and redundancy removal, namely: () it contains no isomorphic metanodes; and (2) it contains no redundant metanodes. 5 OMs for onstraint Networks re anonical orms It is well known that Os are canonical representations of oolean functions given an ordering of the variables [9], and this property extends to Ms [22]. In the case of Os and OMs, the canonicity is with respect to a pseudo tree, following the transition from total orders (that correspond to a linear ordering) to partial orders (that correspond to a pseudo tree ordering). Proposition. Let f be a function, not always zero, defined by a constraint network over X. iven a partition {X,...,X m } of the set of variables X (namely, X i X j = φ, i j, and X = m i= Xi ), if f = f... f m and f = g... g m, such that scope(f i ) = scope(g i ) = X i for all i {,...,m}, then f i = g i for all i {,...,m}. Namely, if f can be decomposed over the given partition, then the decomposition is unique.
11 ased on the previous proposition, in can be shown that OMs for constraint networks are canonical representations given a pseudo tree. Theorem 4 (OMs are canonical for a given pseudo tree). iven a constraint network, and a pseudo tree T of its constraint graph, there is a unique (up to isomorphism) OM that represents it, and it has the minimal number of metanodes. The proof, omitted for space reasons, is by structural induction over the depth of the pseudo tree. constraint network is defined by its relations (or functions). There exist equivalent constraint networks that are defined by different sets of functions, even having different scope signatures. owever, equivalent constraint networks define the same function, and we can ask if the OM of different equivalent constraint networks is the same. The following theorem can be derived immediately from Theorem 4. Theorem 5. Two equivalent constraint networks that admit the same pseudo tree T have the same OM based on T. 6 Using N/OR Search to enerate OMs In Section 4. we described how we can transform an N/OR graph into an OM by applying reduction rules. In Section 6. we describe the explicit algorithm that takes as input a constraint network, performs N/OR search with contextbased caching to obtain the context minimal N/OR graph, and in Section 6.2 we give the procedure that applies the reduction rules bottom up to obtain the OM. The reduction procedure can actually be incorporated in the search algorithm, but we present it separately for clarity. 6. lgorithm N/ORSROM lgorithm, called N/ORSROM, compiles a constraint network into an OM. memory intensive (with contextbased caching) N/OR search is used to create the context minimal N/OR graph (see efinition 0). The input to N/ORSROM is a constraint network R and a pseudo tree T, that also defines the ORcontext of each variable. ach variable X i has an associated cache table, whose scope is the context of X i in T. This ensures that the trace of the search is the context minimal N/OR graph. list denoted by L Xi (see line 34), is used for each variable X i to save pointers to metanodes labeled with X i. These lists are used by the procedure that performs the bottom up reduction, per layers of the N/OR graph (one layer contains all the nodes labeled with one given variable). The fringe of the search is maintained on a stack calledopn. The current node (either OR or N node) is denoted by n, its parent by p, and the current path by π n. The children of the current node are denoted by successors(n). or each node n, the oolean attribute consistent(n) indicates if the current path can be extended to a solution. This information is useful for pruning the search space.
12 lgorithm : N/OR SR  OM 2 input : R = X,, ; pseudo tree T rooted at X ; parents pa i (ORcontext) for every variable X i. output : OM of R. forall X i X do Initialize contextbased cache table ache Xi (pa i) withnull entries 3 reate new OR nodet, labeled with X i; consistent(t) true; pushton top ofopn 4 whileopn φ do 5 n top(opn); removenfromopn // orward 6 successors(n) φ 7 ifnis an OR node labeled with X i then // ORexpand 8 if ache Xi (asgn(π n)[pa i]) null then 9 onnect parent ofnto ache Xi (asgn(π n)[pa i]) // Use the cached pointer 0 else forall x i i do 2 reate new N nodet, labeled with X i, x i 3 if X i, x i is consistent with π n then // onstraint Propagation 4 consistent(t) true 5 addtto successors(n) 6 else 7 consistent(t) false 8 make terminal 0 the only child oft ifnis an N node labeled with X i, x i then if children T (X i) == φ then make terminal the only child ofn else forall Y children T (X i) do reate new OR nodet, labeled with Y consistent(t) false addtto successors(n) // Nexpand dd successors(n) to top ofopn while successors(n) == φ do // acktrack letpbe the parent ofn ifnis an OR node labeled with X i then if X i == X then // Search is complete all ottomupreduction procedure // begin reduction to OM ache(asgn(π n)[pa i]) n // Save in cache dd metanode ofnto the list L X i consistent(p) consistent(p) consistent(n) if consistent(p) == false then // heck if p is deadend remove successors(p) fromopn successors(p) φ ifnis an N node labeled with X i, x i then consistent(p) consistent(p) consistent(n); removenfrom successors(p) n p The algorithm is based on two mutually recursive steps: orward (beginning at line 5) and acktrack (beginning at line 28), which call each other (or themselves) until the search terminates. In the forward phase, the N/OR graph is expanded top down. The two types of nodes, N and OR, are treated differently according to their semantics. efore an OR node is expanded, the cache table of its variable is checked (line 8). If the entry is not null, a link is created to the already existing OR node that roots the graph equivalent to the current subproblem. Otherwise, the OR node is expanded by
13 generating its N descendants. ach value x i of X i is checked for consistency (line 3). ny level of constraint propagation can be performed in this step (e.g., look ahead, arc consistency, path consistency, iconsistency etc.). The computational overhead can increase, in the hope of pruning the search space more aggressively. We should note that constraint propagation is not crucial for the algorithm, and the complexity guarantees are maintained even without it. The consistent N nodes are added to the list of successors of n (line 5), while the inconsistent ones are linked to the terminal 0 metanode (line 8). n N node n labeled with X i,x i is expanded (line 9) based on the structure of the pseudo tree. If X i is a leaf in T, thennis linked to the terminal metanode (line 2). Otherwise, an OR node is created for each child of X i in T (line 23). The forward step continues as long as the current node is not a deadend and still has unevaluated successors. The backtrack phase is triggered when a node has an empty set of successors (line 28). Note that, as each successor is processed, it is removed from the set of successors in line 4. When the backtrack reaches the root (line 3), the search is complete, the context minimal N/OR graph has been generated, and the Procedure OTTOMUPRUTION is called. When the backtrack step processes an OR node (line 30), it saves a pointer to it in cache, and also adds a pointer to the corresponding metanode to the list L Xi. The consistent attribute of the N parent p is updated by conjunction with consistent(n). If the N parent p becomes inconsistent, it is not necessary to check its remaining OR successors (line 37). When the backtrack step processes an N node (line 39), the consistent attribute of the OR parent p is updated by disjunction with consistent(n). The N/OR search algorithm usually maintains a value for each node, corresponding to a task that is solved (e.g., counting solutions, or cost of the optimal solution). We did not include values in our description because an OM is just an equivalent representation of the original constraint network R. ny task over R can be solved by a traversal of the OM. It is however up to the user to include more information in the metanodes (e.g., number of solutions for a subproblem). 6.2 Reducing the ontext Minimal N/OR raph to an OM Procedure ottomupreduction processes the variables bottom up relative to the pseudo tree T. We use the depth first traversal ordering of T (line ), but any other bottom up ordering is as good. The outer for loop (starting at line 2) goes through each level of the context minimal N/OR graph (where a level contains all the OR and N nodes labeled with the same variable, in other words it contains all the metanodes of that variable). or efficiency, and to ensure the complexity guarantees, a hash table, initially empty, is used for each level. The inner for loop (starting at line 4) goes through all the metanodes of a level, that are also saved (or pointers to them are saved) in the list L Xi. or each new metanode n in the list L Xi, in line 5 the hash table is checked to verify if a node isomorphic withnalready exists. If the hash table already contains a nodepcorresponding to the hash key (X i,n.children,...,n.children ki ), thenpandnare isomorphic and should be merged. Otherwise, if the new metanoden
14 Procedure ottomupreduction input : constraint network R = X,, ; a pseudo tree T of the primal graph, rooted at X ; ontext minimal N/OR graph, and lists L X i of metanodes for each level X i. output : OM of R. Let d = {X,..., X n} be the depth first traversal ordering of T 2 for i n down to do 3 Let be a hash table, initially empty 4 forall metanodesnin L X i do 5 if (X i, n.children,..., n.children ki ) returns a metanodepthen 6 mergenwithpin the N/OR graph else ifnis redundant then eliminatenfrom the N/OR graph else hashninto the table : (X i, n.children,..., n.children ki ) n 2 return reduced N/OR graph is redundant, then it is eliminated from the N/OR graph. If none of the previous two conditions is met, then the new metanodenis hashed into the table. Proposition 2. The output of Procedure ottomupreduction is the OM of R along the pseudo tree T, namely the resulting N/OR graph is completely reduced. Note that we explicated Procedure ottomupreduction separately only for clarity. In practice, it can actually be included in lgorithm N/ORSR OM, and the reduction rules can be applied whenever the search backtracks. We can maintain a hash table for each variable, during the N/OR search, to store pointers to metanodes. When the search backtracks out of an OR node, it can already check the redundancy of that metanode, and also look up in the hash table to check for isomorphism. Therefore, the reduction of the N/OR graph can be done during the N/OR search, and the output will be the OM of R. rom Theorem 3 and Proposition 2 we can conclude: Theorem 6. iven a constraint network R and a pseudo tree T of its constraint graph, the OM of R corresponding to T has size bounded by O(n k w T () ) and it can be computed by lgorithm N/ORSROM in time O(n k w T () ), where wt () is the induced width of over the depth first traversal of T, and k bounds the domain size. 7 Using ucket limination to enerate OMs In this section we propose to use a ucket limination () type algorithm to guide the compilation of a constraint network into an OM. The idea is to express the constraints as OMs, and then combine them via the PPLY operator by following a schedule. The PPLY is a procedure very similar to that from Os [9], but it is adapted to N/OR search graphs. It takes as input two constraints represented as OMs based on the same pseudo tree, and outputs their join, also represented as an OM based on the same pseudo tree.
15 (a) (b) ig. 4. (a) onstraint graph for = {,..., 9}, where =, 2 =, 3 =, 4 =, 5 =, 6 =, 7 =, 8 =, 9 = ; (b) Pseudo tree (bucket tree) for ordering d = (,,,,,,, ) m 7 m 7 m 6 m 3 m 3 9 m 5 m 6 m 4 5 m 2 m m 5 m 4 m 2 m ig. 5. xecution of V with OMs xample 6. onsider the constraint network defined by X = {,,...,}, =... = = {0,} and the constraints (where denotes XOR): =, 2 =, 3 =, 4 =, 5 =, 6 =, 7 =, 8 =, 9 =. The constraint graph is shown in igure 4(a). onsider the ordering d = (,,,,,,,). The pseudo tree (or bucket tree) induced by d is given in ig. 4(b). igure 5 shows the execution of with OMs along ordering d. Initially, the constraints through 9 are represented as OMs and placed in the bucket of their latest variable in d. The scope of any original constraint always appears on a path from root to a leaf in the pseudo tree. Therefore, each original constraint is represented by an OM based on a chain. (i.e. there is no branching into independent components at any point). The chain is just the scope of the constraint, ordered according to d. or bivalued variables, the original constraints are represented by Os, for multiplevalued variables they are Ms. Note that we depict meta
16 0 (a) 0 (b) ig. 6. (a) The final OM; (b) The O corresponding to d nodes: one OR node and its two N children, that appear inside each gray node. The dotted edge corresponds to the 0 value (the low edge in Os), the solid edge to the value (the high edge). We have some redundancy in our notation, keeping both N value nodes and arctypes (doted arcs from 0 and solid arcs from ). The scheduling is used to process the buckets in reverse order of d. bucket is processed by joining all the OMs inside it, using the PPLY operator. owever, the step of elimination of the bucket variable is omitted because we want to generate the full OM. In our example, the messages m = 2 and m 2 = 3 4 are still based on chains, so they are still Os. Note that they still contain the variables and, which have not been eliminated. owever, the message m 3 = 5 m m 2 is not an O anymore. We can see that it follows the structure of the pseudo tree, where has two children, and. Some of the nodes corresponding to have two outgoing edges for value. The processing continues in the same manner. The final output of the algorithm, which coincides with m 7, is shown in igure 6(a). The O based on the same ordering d is shown in ig. 6(b). Notice that the OM has 8 nonterminal nodes and 47 edges, while the O has 27 nonterminal nodes and 54 edges. 7. lgorithm OM lgorithm 2, called OM, creates the OM of a constraint network by using a schedule for PPLY operations. iven an order d of the variables, a pseudo tree is created based on the constraint graph (this is just the bucket tree, or elimination tree of d). ach initial constraint i is then represented as an OM, denoted by, and placed in its bucket. To obtain the OM of a constraint, its scope is ordered according to d, a search tree (based on a chain) that represents i is generated, and then reduced by Procedure ottomupreduction. Then, the algorithm proceeds exactly like, with the only difference that the join of constraints (represented as OMs) is realized by the PPLY algorithm, and variables are not eliminated but aomdd i carried around to the destination bucket. The messages between buckets are initialized with the dummy OM of, denoted by aomdd, which is neutral for join.
17 lgorithm 2: OM input : onstraint network R = X,,, where X = {X,..., X n}, = {,..., r} ; order d = (X,..., X n) output : OM representing i i Let T be the pseudo tree (bucket tree) corresponding to d; for i to r do // place constraints in buckets 2 place aomdd in the bucket of its latest variable in d i 3 for i n down to do // process buckets 4 message(x i) aomdd // initialize with OM of ; 5 while bucket(x i) φ do // combine OMs in bucket of X i 6 pick aomdd f from bucket(x i); 7 bucket(x i) bucket(x i) \ { aomdd f }; 8 message(x i) PPLY(message(X i), aomdd f ) 9 0 add message(x i) to the bucket of the parent of X i in T return message(x ) * = ig. 7. xample of PPLY operation 7.2 The OM PPLY Operation The apply operator takes as input two OMs representing constraints and 2 and returns an OM representing their join 2. In Os the apply operator combines two input diagrams based on the same variable ordering. Likewise, in order to combine two OMs we assume that they are based on the same pseudo tree. This restriction is satisfied when we use PPLY to combine the constraints in the same bucket of the based algorithm. There are also more relaxed version of PPLY, when the pseudo trees of the two input constraints need only be compatible, rather than identical. Intuitively, this means that the pseudo trees generate partial orders (based on descendance relation) that are not in conflict. or space reasons, we will not present the details of the PPLY algorithm here, but refer the reader to []. We will just briefly describe it by an example. xample 7. igure 7 shows the result of combining two oolean functions by an N operation (or product). The input functions f and g are represented by OMs based on chain pseudo trees, while the results is based on the pseudo tree that expresses the decomposition after variables and are instantiated. The PPLY operator performs a depth first traversal of the two input OMs, and generates the resulting OM based on the output pseudo tree. Similar to the case of Os, a function or an OM can be identified by its root metanode. In this example the input metanodes have labels (, 2,, 2, etc.). The output metanode labeled by 2 2 is
18 the root of a diagram that represents the function obtained by combining the functions rooted by 2 and 2. The complexity of OM and the output size are similar to those of the search based algorithm: Theorem 7. The space complexity of OM and the size of the output OM are O(n k w ), where n is the number of variables, k is the maximum domain size and w is the treewidth of the bucket tree. The time complexity is bounded by O(r k w ), where r is the number of initial functions. 8 Related Work There are various lines of related research. The formal verification literature, beginning with [9] contains a very large number of papers dedicated to the study of s. owever, s are in fact OR structures (the underlying pseudo tree is a chain) and do not take advantage of the problem decomposition in an explicit way. The complexity bounds for Os are based on pathwidth rather than treewidth. s noted earlier, the work on isjoint Support ecomposition (S) is related to N/OR s in various ways [4]. The main common aspect is that both approaches show how structure decomposition can be exploited in a like representation. S is focused on oolean functions and can exploit more refined structural information that is inherent to oolean functions. In contrast, N/OR s assumes only the structure conveyed in the constraint graph. They are therefore more broadly applicable to any constraint expression and also to graphical models in general. They allow a simpler and higher level exposition that yields graphbased bounds on the overall size of the generated OM. McMillan introduced the trees [6], along with the operations for combining them. or circuits of bounded tree width, trees have linear space upper bound O( g 2 w22w ), where g is the size of the circuit g (typically linear in the number of variables) and w is the treewidth. This bound hides some very large constants to claim the linear dependence on g when w is bounded. owever, McMillan maintains that when the input function is a N expression trees have the same bounds as N/OR s, namely they are exponential in the treewidth only. The N/OR structure restricted to propositional theories is very similar to deterministic decomposable negation normal form (dnn) []. More recently, in [23], the trace of the PLL algorithm is used to generate an O, and compared with the typical formal verification approach of combining the Os of the input function according to some schedule. The structures that were investigated are still OR. Mcllester [24] introduced the case factor diagrams () which subsume Markov random fields of bounded tree width and probabilistic context free grammars (P). s are very much related to the N/OR graphs. The s target the minimal representation, by exploiting decomposition (similar to N nodes) but also by exploiting context sensitive information and allowing dynamic ordering of variables based on context. s do not eliminate the redundant nodes, and part of the cause is that they use
19 zero suppression. There is no claim about s being a canonical form, and also there is no description of how to combine two s. More recently, independently and in parallel to our work on N/OR graphs [4, 5], argier and Vilarem [7] proposed the compilation of SPs into treedriven automata, which have many similarities to our work. Their main focus is the transition from linear automata to tree automata (similar to that from OR to N/OR), and the possible savings for treestructured networks and hypertrees of constraints due to decomposition. Their compilation approach is guided by a treedecomposition while ours is guided by a variableelimination based algorithms. nd, it is well known that ucket limination and clustertree decomposition are in principle, the same [25]. 9 onclusion This paper gives an overview of a new compilation data structure for constraint networks. The N/OR Multivalued ecision iagram (OM) [ 3] emerges from the study of N/OR search spaces for graphical models [4, 5, 26, 6] and ordered binary decision diagrams (Os) [9]. raphical models algorithms that are searchbased and compiled datastructures such as s differ primarily by their choices of time vs memory. When we move from regular OR to an N/OR search space, the spectrum of algorithms available is improved for all time vs memory decisions. We believe that the N/OR search space clarifies the available choices and helps guide the user into making an informed selection of the algorithm that would fit best the particular query asked, the specific input function and the available computational resources. We presented the two main algorithmic approaches for compiling an OM for constraint networks. The first is a top down procedure, that uses memory intensive N/OR search, and applies reduction rules to the trace of the search. The second is a bottom up procedure that uses a ucket limination schedule to combine the constraints via the PPLY operator. s part of our current and future work, we are implementing, and experimenting with, the algorithms described here in order to provide an empirical evaluation. cknowledgments This work was supported in part by the NS grants IIS and IIS References. Mateescu, R., echter, R.: ompiling constraint networks into N/OR multivalued decision diagrams (OMs). In: Proceedings of the Twelfth International onference on Principles and Practice of onstraint Programming (P 06). (2006) Mateescu, R., echter, R.: nd/or multivalued decision diagrams (aomdds) for weighted graphical models. In: Proceedings of the Twenty Third onference on Uncertainty in rtificial Intelligence (UI 07). (2007) 3. Mateescu, R., Marinescu, R., echter, R.: N/OR multivalued decision diagrams for constraint optimization. In: Proceedings of the Thirteenth International onference on Principles and Practice of onstraint Programming (P 07). (2007)
20 4. echter, R., Mateescu, R.: Mixtures of deterministicprobabilistic networks and their N/OR search space. In: Proceedings of the Twentieth onference on Uncertainty in rtificial Intelligence (UI 04). (2004) echter, R., Mateescu, R.: The impact of N/OR search spaces on constraint satisfaction and counting. In: Proceedings of the Tenth International onference on Principles and Practice of onstraint Programming (P 04). (2004) echter, R., Mateescu, R.: N/OR search spaces for graphical models. rtificial Intelligence 7 (2007) larke,., rumberg, O., Peled,.: Model hecking. MIT Press (999) 8. McMillan, K.L.: Symbolic Model hecking. Kluwer cademic (993) 9. ryant, R..: raphbased algorithms for boolean function manipulation. I Transaction on omputers 35 (986) Selman,., Kautz,.: Knowledge compilation and theory approximation. Journal of the M 43 (996) arwiche,., Marquis, P.: knowledge compilation map. Journal of rtificial Intelligence Research (JIR) 7 (2002) adoli, M., onini,.m.: survey on knowledge compilation. I ommunications 0 (997) Korf, R., elner,.: isjoint pattern database heuristics. rtificial Intelligence 34 (2002) ertacco, V., amiani, M.: The disjunctive decomposition of logic functions. In: International onference on omputerided esign (I). (997) rayton, R., McMullen,.: The decomposition and factorization of boolean expressions. In: ISS, Proceedings of the International Symposium on ircuits and Systems. (982) McMillan, K.L.: ierarchical representation of discrete functions with application to model checking. In: omputer ided Verification. (994) argier,., Vilarem, M.: ompiling csps into treedriven automata for interactive solving. onstraints 9 (2004) reuder,.., Quinn, M.J.: Taking advantage of stable sets of variables in constraint satisfaction problems. In: Proceedings of the Ninth International Joint onference on rtificial Intelligence (IJI 85). (985) echter, R.: ucket elimination: unifying framework for reasoning. rtificial Intelligence 3 (999) ayardo, R., Miranker,.: complexity analysis of spacebound learning algorithms for the constraint satisfaction problem. In: Proceedings of the Thirteenth National onference on rtificial Intelligence (I 96). (996) arwiche,.: Recursive conditioning. rtificial Intelligence 25 (200) Srinivasan,., Kam, T., Malik, S., rayton, R.K.: lgorithms for discrete function manipulation. In: International onference on omputerided esign (I). (990) uang, J., arwiche,.: pll with a trace: rom sat to knowledge compilation. In: Proceedings of the Nineteenth International Joint onference on rtificial Intelligence (IJI 05). (2005) Mcllester,., ollins, M., Pereira,.: asefactor diagrams for structured probabilistic modeling. In: Proceedings of the Twentieth onference on Uncertainty in rtificial Intelligence (UI 04). (2004) echter, R., Pearl, J.: Tree clustering for constraint networks. rtificial Intelligence 38 (989) Mateescu, R., echter, R.: The relationship between N/OR search and variable elimination. In: Proceedings of the Twenty irst onference on Uncertainty in rtificial Intelligence (UI 05). (2005)
Compiling Constraint Networks into AND/OR MultiValued Decision Diagrams (AOMDDs)
ompiling onstraint Networks into N/OR MultiValued ecision iagrams (OMs) Robert Mateescu and Rina echter onald ren School of Information and omputer Science University of alifornia, Irvine, 926973425
More informationAND/OR MultiValued Decision Diagrams (AOMDDs) for Graphical Models
Journal of rtificial Intelligence Research 33 (28) 46559 Submitted 5/8; published 2/8 N/OR MultiValued ecision iagrams (OMs) for Graphical Models Robert Mateescu lectrical ngineering epartment alifornia
More informationAND/OR Cutset Conditioning
ND/OR utset onditioning Robert Mateescu and Rina Dechter School of Information and omputer Science University of alifornia, Irvine, 92697 {mateescu, dechter}@ics.uci.edu bstract utset conditioning is one
More informationThe Relationship Between AND/OR Search and Variable Elimination
The Relationship etween N/OR Search and Variable limination Robert Mateescu and Rina echter School of Information and omputer Science University of alifornia, Irvine, 926973425 {mateescu,dechter}@ics.uci.edu
More informationModel Checking I Binary Decision Diagrams
/42 Model Checking I Binary Decision Diagrams Edmund M. Clarke, Jr. School of Computer Science Carnegie Mellon University Pittsburgh, PA 523 2/42 Binary Decision Diagrams Ordered binary decision diagrams
More informationBinary Decision Diagrams
Logic and roof Hilary 2016 James Worrell Binary Decision Diagrams A propositional formula is determined up to logical equivalence by its truth table. If the formula has n variables then its truth table
More informationAND/OR Search Spaces for Graphical Models
AND/OR Search Spaces for Graphical Models Rina Dechter Department of Information and Computer Science University of California, Irvine, CA 969745 {dechter }@ics.uci.edu Abstract The paper introduces an
More informationAdvancing AND/OR Search for Optimization Using Diverse Principles
Advancing AN/OR Search for Optimization Using iverse Principles Radu Marinescu and Rina echter School of Information and omputer Science University of alifornia, Irvine, A 969745 {radum,dechter}@ics.uci.edu
More informationBoolean Representations and Combinatorial Equivalence
Chapter 2 Boolean Representations and Combinatorial Equivalence This chapter introduces different representations of Boolean functions. It then discusses the applications of these representations for proving
More informationBestFirst AND/OR Search for Graphical Models
estirst N/ Search for Graphical Models Radu Marinescu and Rina echter School of Information and omputer Science University of alifornia, Irvine, 92627 {radum,dechter}@ics.uci.edu bstract The paper presents
More informationBehavior models and verification Lecture 6
Behavior models and verification Lecture 6 http://d3s.mff.cuni.cz Jan Kofroň, František Plášil Model checking For a Kripke structure M = (S, I, R, L) over AP and a (state based) temporal logic formula
More informationJoint Entity Resolution
Joint Entity Resolution Steven Euijong Whang, Hector GarciaMolina Computer Science Department, Stanford University 353 Serra Mall, Stanford, CA 94305, USA {swhang, hector}@cs.stanford.edu No Institute
More informationEvaluating the Impact of AND/OR Search on 01 Integer Linear Programming
Evaluating the Impact of AND/OR Search on 01 Integer Linear Programming Radu Marinescu, Rina Dechter Donald Bren School of Information and Computer Science University of California, Irvine, CA 92697,
More informationTrees. 3. (Minimally Connected) G is connected and deleting any of its edges gives rise to a disconnected graph.
Trees 1 Introduction Trees are very special kind of (undirected) graphs. Formally speaking, a tree is a connected graph that is acyclic. 1 This definition has some drawbacks: given a graph it is not trivial
More informationConstraint Satisfaction Problems
Constraint Satisfaction Problems Search and Lookahead Bernhard Nebel, Julien Hué, and Stefan Wölfl AlbertLudwigsUniversität Freiburg June 4/6, 2012 Nebel, Hué and Wölfl (Universität Freiburg) Constraint
More informationSome Applications of Graph Bandwidth to Constraint Satisfaction Problems
Some Applications of Graph Bandwidth to Constraint Satisfaction Problems Ramin Zabih Computer Science Department Stanford University Stanford, California 94305 Abstract Bandwidth is a fundamental concept
More informationSearch Algorithms for Solving Queries on Graphical Models & the Importance of Pseudotrees in their Complexity.
Search Algorithms for Solving Queries on Graphical Models & the Importance of Pseudotrees in their Complexity. University of California, Irvine CS199: Individual Study with Rina Dechter Héctor Otero Mediero
More informationOn the SpaceTime Tradeoff in Solving Constraint Satisfaction Problems*
Appeared in Proc of the 14th Int l Joint Conf on Artificial Intelligence, 55856, 1995 On the SpaceTime Tradeoff in Solving Constraint Satisfaction Problems* Roberto J Bayardo Jr and Daniel P Miranker
More informationDistributed minimum spanning tree problem
Distributed minimum spanning tree problem JuhoKustaa Kangas 24th November 2012 Abstract Given a connected weighted undirected graph, the minimum spanning tree problem asks for a spanning subtree with
More informationA SchedulabilityPreserving Transformation Scheme from Boolean Controlled Dataflow Networks to Petri Nets
SchedulabilityPreserving ransformation Scheme from oolean ontrolled Dataflow Networks to Petri Nets ong Liu Edward. Lee University of alifornia at erkeley erkeley,, 94720, US {congliu,eal}@eecs. berkeley.edu
More informationOn the Practical Significance of Hypertree vs Tree Width
On the Practical Significance of Hypertree vs Tree Width Rina Dechter 1, Lars Otten 1, and Radu Marinescu 2 bstract. In 2000, [4] presented a new graph parameter, the hypertree width, and showed that it
More informationIntegrating Probabilistic Reasoning with Constraint Satisfaction
Integrating Probabilistic Reasoning with Constraint Satisfaction IJCAI Tutorial #7 Instructor: Eric I. Hsu July 17, 2011 http://www.cs.toronto.edu/~eihsu/tutorial7 Getting Started Discursive Remarks. Organizational
More informationMixed deterministic and probabilistic networks
nn Math rtif Intell (28) 54:3 5 OI.7/s4729932y Mixed deterministic and probabilistic networks Robert Mateescu Rina echter Published online: 25 pril 29 Springer Science + usiness Media.V. 29 bstract
More informationComputer Vision Group Prof. Daniel Cremers. 4. Probabilistic Graphical Models Directed Models
Prof. Daniel Cremers 4. Probabilistic Graphical Models Directed Models The Bayes Filter (Rep.) (Bayes) (Markov) (Tot. prob.) (Markov) (Markov) 2 Graphical Representation (Rep.) We can describe the overall
More informationKnowledge Compilation Properties of TreeofBDDs
Knowledge Compilation Properties of TreeofBDDs Sathiamoorthy Subbarayan IT University of Copenhagen, Denmark sathi@itu.dk Lucas Bordeaux and Youssef Hamadi Microsoft Research, Cambridge, UK lucasb,youssefh@microsoft.com
More informationCore Membership Computation for Succinct Representations of Coalitional Games
Core Membership Computation for Succinct Representations of Coalitional Games Xi Alice Gao May 11, 2009 Abstract In this paper, I compare and contrast two formal results on the computational complexity
More informationComputer Vision Group Prof. Daniel Cremers. 4. Probabilistic Graphical Models Directed Models
Prof. Daniel Cremers 4. Probabilistic Graphical Models Directed Models The Bayes Filter (Rep.) (Bayes) (Markov) (Tot. prob.) (Markov) (Markov) 2 Graphical Representation (Rep.) We can describe the overall
More informationTreewidth and graph minors
Treewidth and graph minors Lectures 9 and 10, December 29, 2011, January 5, 2012 We shall touch upon the theory of Graph Minors by Robertson and Seymour. This theory gives a very general condition under
More informationAn Effective Upperbound on Treewidth Using Partial Fillin of Separators
An Effective Upperbound on Treewidth Using Partial Fillin of Separators Boi Faltings Martin Charles Golumbic June 28, 2009 Abstract Partitioning a graph using graph separators, and particularly clique
More informationMotivation. CS389L: Automated Logical Reasoning. Lecture 5: Binary Decision Diagrams. Historical Context. Binary Decision Trees
Motivation CS389L: Automated Logical Reasoning Lecture 5: Binary Decision Diagrams Işıl Dillig Previous lectures: How to determine satisfiability of propositional formulas Sometimes need to efficiently
More informationMachine Learning Lecture 16
ourse Outline Machine Learning Lecture 16 undamentals (2 weeks) ayes ecision Theory Probability ensity stimation Undirected raphical Models & Inference 28.06.2016 iscriminative pproaches (5 weeks) Linear
More informationA General Scheme for Automatic Generation of Search Heuristics from Specification Dependencies. Kalev Kask and Rina Dechter
A General Scheme for Automatic Generation of Search Heuristics from Specification Dependencies Kalev Kask and Rina Dechter Department of Information and Computer Science University of California, Irvine,
More informationCS521 \ Notes for the Final Exam
CS521 \ Notes for final exam 1 Ariel Stolerman Asymptotic Notations: CS521 \ Notes for the Final Exam Notation Definition Limit BigO ( ) Smallo ( ) Big ( ) Small ( ) Big ( ) Notes: ( ) ( ) ( ) ( )
More informationBestFirst AND/OR Search for Most Probable Explanations
MARINSU & DHTR 259 estfirst / Search for Most Probable xplanations Radu Marinescu and Rina Dechter School of Information and omputer Science University of alifornia, Irvine, A 926973425 {radum,dechter}@ics.uci.edu
More informationThe Encoding Complexity of Network Coding
The Encoding Complexity of Network Coding Michael Langberg Alexander Sprintson Jehoshua Bruck California Institute of Technology Email: mikel,spalex,bruck @caltech.edu Abstract In the multicast network
More informationMiniBuckets: A General Scheme for Generating Approximations in Automated Reasoning
MiniBuckets: A General Scheme for Generating Approximations in Automated Reasoning Rina Dechter* Department of Information and Computer Science University of California, Irvine dechter@ics. uci. edu Abstract
More informationRecoloring kdegenerate graphs
Recoloring kdegenerate graphs Jozef Jirásek jirasekjozef@gmailcom Pavel Klavík pavel@klavikcz May 2, 2008 bstract This article presents several methods of transforming a given correct coloring of a kdegenerate
More informationTowards Parallel Search for Optimization in Graphical Models
Towards Parallel Search for Optimization in Graphical Models Lars Otten and Rina Dechter Bren School of Information and Computer Sciences University of California, Irvine {lotten,dechter}@ics.uci.edu Abstract
More informationCh9: Exact Inference: Variable Elimination. Shimi Salant, Barak Sternberg
Ch9: Exact Inference: Variable Elimination Shimi Salant Barak Sternberg Part 1 Reminder introduction (1/3) We saw two ways to represent (finite discrete) distributions via graphical data structures: Bayesian
More informationBinary Decision Diagrams (BDD)
Binary Decision Diagrams (BDD) Contents Motivation for Decision diagrams Binary Decision Diagrams ROBDD Effect of Variable Ordering on BDD size BDD operations Encoding state machines Reachability Analysis
More informationConstraint Satisfaction Problems
Constraint Satisfaction Problems CE417: Introduction to Artificial Intelligence Sharif University of Technology Spring 2013 Soleymani Course material: Artificial Intelligence: A Modern Approach, 3 rd Edition,
More information3.1 Basic Definitions and Applications
Graphs hapter hapter Graphs. Basic efinitions and Applications Graph. G = (V, ) n V = nodes. n = edges between pairs of nodes. n aptures pairwise relationship between objects: Undirected graph represents
More informationMarkov Random Fields
3750 Machine earning ecture 4 Markov Random ields Milos auskrecht milos@cs.pitt.edu 5329 ennott quare 3750 dvanced Machine earning Markov random fields Probabilistic models with symmetric dependences.
More informationLecture 2  Graph Theory Fundamentals  Reachability and Exploration 1
CME 305: Discrete Mathematics and Algorithms Instructor: Professor Aaron Sidford (sidford@stanford.edu) January 11, 2018 Lecture 2  Graph Theory Fundamentals  Reachability and Exploration 1 In this lecture
More informationCS301  Data Structures Glossary By
CS301  Data Structures Glossary By Abstract Data Type : A set of data values and associated operations that are precisely specified independent of any particular implementation. Also known as ADT Algorithm
More informationOptimization I : Brute force and Greedy strategy
Chapter 3 Optimization I : Brute force and Greedy strategy A generic definition of an optimization problem involves a set of constraints that defines a subset in some underlying space (like the Euclidean
More informationSTAT 598L Probabilistic Graphical Models. Instructor: Sergey Kirshner. Exact Inference
STAT 598L Probabilistic Graphical Models Instructor: Sergey Kirshner Exact Inference What To Do With Bayesian/Markov Network? Compact representation of a complex model, but Goal: efficient extraction of
More informationEvaluating the impact of AND/OR search on 01 integer linear programming
Constraints (2010) 15:29 63 DOI 10.1007/s1060100990707 Evaluating the impact of AND/OR search on 01 integer linear programming Radu Marinescu Rina Dechter Published online: 18 March 2009 The Author(s)
More informationFoundations of Computer Science Spring Mathematical Preliminaries
Foundations of Computer Science Spring 2017 Equivalence Relation, Recursive Definition, and Mathematical Induction Mathematical Preliminaries Mohammad Ashiqur Rahman Department of Computer Science College
More informationA CSP Search Algorithm with Reduced Branching Factor
A CSP Search Algorithm with Reduced Branching Factor Igor Razgon and Amnon Meisels Department of Computer Science, BenGurion University of the Negev, BeerSheva, 84105, Israel {irazgon,am}@cs.bgu.ac.il
More informationInteraction Between Input and OutputSensitive
Interaction Between Input and OutputSensitive Really? Mamadou M. Kanté Université Blaise Pascal  LIMOS, CNRS Enumeration Algorithms Using Structure, Lorentz Institute, August 26 th, 2015 1 Introduction
More informationDSeparation. b) the arrows meet headtohead at the node, and neither the node, nor any of its descendants, are in the set C.
DSeparation Say: A, B, and C are nonintersecting subsets of nodes in a directed graph. A path from A to B is blocked by C if it contains a node such that either a) the arrows on the path meet either
More informationOn the SpaceTime Tradeoff in Solving Constraint Satisfaction Problems*
On the SpaceTime Tradeoff in Solving Constraint Satisfaction Problems* Roberto J. Bayardo Jr. and Daniel P. Miranker Department of Computer Sciences and Applied Research Laboratories The University of
More informationMaximal Prime Subgraph Decomposition of Bayesian Networks: A Relational Database Perspective
Maximal Prime ubgraph ecomposition of ayesian Networks: Relational atabase Perspective an Wu chool of omputer cience University of Windsor Windsor Ontario anada N9 3P4 Michael Wong epartment of omputer
More informationUtilizing Device Behavior in StructureBased Diagnosis
Utilizing Device Behavior in StructureBased Diagnosis Adnan Darwiche Cognitive Systems Laboratory Department of Computer Science University of California Los Angeles, CA 90024 darwiche @cs. ucla. edu
More informationBayesian Networks and Decision Graphs
ayesian Networks and ecision raphs hapter 7 hapter 7 p. 1/27 Learning the structure of a ayesian network We have: complete database of cases over a set of variables. We want: ayesian network structure
More informationGraph and Digraph Glossary
1 of 15 31.1.2004 14:45 Graph and Digraph Glossary A B C D E F G H IJ K L M N O PQ R S T U V WZ Acyclic Graph A graph is acyclic if it contains no cycles. Adjacency Matrix A 01 square matrix whose
More informationByzantine Consensus in Directed Graphs
Byzantine Consensus in Directed Graphs Lewis Tseng 1,3, and Nitin Vaidya 2,3 1 Department of Computer Science, 2 Department of Electrical and Computer Engineering, and 3 Coordinated Science Laboratory
More informationModule 5 Graph Algorithms
Module 5 Graph lgorithms Dr. Natarajan Meghanathan Professor of Computer Science Jackson State University Jackson, MS 97 Email: natarajan.meghanathan@jsums.edu 5. Graph Traversal lgorithms Depth First
More informationIntroduction III. Graphs. Motivations I. Introduction IV
Introduction I Graphs Computer Science & Engineering 235: Discrete Mathematics Christopher M. Bourke cbourke@cse.unl.edu Graph theory was introduced in the 18th century by Leonhard Euler via the Königsberg
More informationModern Exact and Approximate Combinatorial Optimization Algorithms: Max Product and Max Sum product
Modern xact and pproximate ombinatorial Optimization lgorithms: Max Product and Max Sum product In the pursuit of universal solver Rina echter onald ren school of IS, University of alifornia, Irvine Main
More informationCMTreeMiner: Mining Both Closed and Maximal Frequent Subtrees
MTreeMiner: Mining oth losed and Maximal Frequent Subtrees Yun hi, Yirong Yang, Yi Xia, and Richard R. Muntz University of alifornia, Los ngeles, 90095, US {ychi,yyr,xiayi,muntz}@cs.ucla.edu bstract. Tree
More informationProcessing Probabilistic and Deterministic Graphical Models. Rina Dechter DRAFT
Processing Probabilistic and Deterministic Graphical Models Rina Dechter DRAFT May 1, 2013 c Rina Dechter 2 Contents 1 Introduction 7 1.1 Probabilistic vs Deterministic Models..................... 7 1.2
More informationDistributed Algorithms 6.046J, Spring, 2015 Part 2. Nancy Lynch
Distributed Algorithms 6.046J, Spring, 2015 Part 2 Nancy Lynch 1 This Week Synchronous distributed algorithms: Leader Election Maximal Independent Set BreadthFirst Spanning Trees Shortest Paths Trees
More informationReference Sheet for CO142.2 Discrete Mathematics II
Reference Sheet for CO14. Discrete Mathematics II Spring 017 1 Graphs Defintions 1. Graph: set of N nodes and A arcs such that each a A is associated with an unordered pair of nodes.. Simple graph: no
More informationBACKGROUND: A BRIEF INTRODUCTION TO GRAPH THEORY
BACKGROUND: A BRIEF INTRODUCTION TO GRAPH THEORY General definitions; Representations; Graph Traversals; Topological sort; Graphs definitions & representations Graph theory is a fundamental tool in sparse
More informationAdvanced Algorithms Class Notes for Monday, October 23, 2012 Min Ye, Mingfu Shao, and Bernard Moret
Advanced Algorithms Class Notes for Monday, October 23, 2012 Min Ye, Mingfu Shao, and Bernard Moret Greedy Algorithms (continued) The best known application where the greedy algorithm is optimal is surely
More informationProgramming Languages Third Edition
Programming Languages Third Edition Chapter 12 Formal Semantics Objectives Become familiar with a sample small language for the purpose of semantic specification Understand operational semantics Understand
More informationAlgorithm Design (8) Graph Algorithms 1/2
Graph Algorithm Design (8) Graph Algorithms / Graph:, : A finite set of vertices (or nodes) : A finite set of edges (or arcs or branches) each of which connect two vertices Takashi Chikayama School of
More informationAssignment 4 Solutions of graph problems
Assignment 4 Solutions of graph problems 1. Let us assume that G is not a cycle. Consider the maximal path in the graph. Let the end points of the path be denoted as v 1, v k respectively. If either of
More informationSolving 3SAT. Radboud University Nijmegen. Bachelor Thesis. Supervisors: Henk Barendregt Alexandra Silva. Author: Peter Maandag s
Solving 3SAT Radboud University Nijmegen Bachelor Thesis Author: Peter Maandag s3047121 Supervisors: Henk Barendregt Alexandra Silva July 2, 2012 Contents 1 Introduction 2 1.1 Problem context............................
More informationSTA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 5 Inference
More informationLOGIC AND DISCRETE MATHEMATICS
LOGIC AND DISCRETE MATHEMATICS A Computer Science Perspective WINFRIED KARL GRASSMANN Department of Computer Science University of Saskatchewan JEANPAUL TREMBLAY Department of Computer Science University
More informationThese definitions cover some basic terms and concepts of graph theory and how they have been implemented as C++ classes.
hapter 5. raph Theory *raph Theory, *Mathematics raph theory is an area of mathematics which has been incorporated into IS to solve some specific problems in oolean operations and sweeping. It may be also
More informationAcyclic Network. Tree Based Clustering. Tree Decomposition Methods
Summary s Join Tree Importance of s Solving Topological structure defines key features for a wide class of problems CSP: Inference in acyclic network is extremely efficient (polynomial) Idea: remove cycles
More informationEfficient Generation of Evolutionary Trees
fficient Generation of volutionary Trees MUHMM ULLH NN 1 M. SIUR RHMN 2 epartment of omputer Science and ngineering angladesh University of ngineering and Technology (UT) haka1000, angladesh 1 adnan@cse.buet.ac.bd
More informationFMA901F: Machine Learning Lecture 6: Graphical Models. Cristian Sminchisescu
FMA901F: Machine Learning Lecture 6: Graphical Models Cristian Sminchisescu Graphical Models Provide a simple way to visualize the structure of a probabilistic model and can be used to design and motivate
More informationConsistency and Set Intersection
Consistency and Set Intersection Yuanlin Zhang and Roland H.C. Yap National University of Singapore 3 Science Drive 2, Singapore {zhangyl,ryap}@comp.nus.edu.sg Abstract We propose a new framework to study
More informationTopological parameters for timespace tradeoff
Artificial Intelligence 125 (2001) 93 118 Topological parameters for timespace tradeoff Rina Dechter a,, Yousri El Fattah b a Information & Computer Science, University of California, Irvine, CA 92717,
More informationLecture Notes on Binary Decision Diagrams
Lecture Notes on Binary Decision Diagrams 15122: Principles of Imperative Computation William Lovas Notes by Frank Pfenning Lecture 25 April 21, 2011 1 Introduction In this lecture we revisit the important
More informationstorientations September 29, 2005
storientations September 29, 2005 Introduction Let G = (V, E) be an undirected biconnected graph of n nodes and m edges. The main problem this chapter deals with is different algorithms for orienting
More informationDefinition of Graphs and Trees. Representation of Trees.
Definition of Graphs and Trees. Representation of Trees. Chapter 6 Definition of graphs (I) A directed graph or digraph is a pair G = (V,E) s.t.: V is a finite set called the set of vertices of G. E V
More informationMAL 376: Graph Algorithms I Semester Lecture 1: July 24
MAL 376: Graph Algorithms I Semester 20142015 Lecture 1: July 24 Course Coordinator: Prof. B. S. Panda Scribes: Raghuvansh, Himanshu, Mohit, Abhishek Disclaimer: These notes have not been subjected to
More informationConflict Graphs for Combinatorial Optimization Problems
Conflict Graphs for Combinatorial Optimization Problems Ulrich Pferschy joint work with Andreas Darmann and Joachim Schauer University of Graz, Austria Introduction Combinatorial Optimization Problem CO
More informationCHAPTER 5 GENERATING TEST SCENARIOS AND TEST CASES FROM AN EVENTFLOW MODEL
CHAPTER 5 GENERATING TEST SCENARIOS AND TEST CASES FROM AN EVENTFLOW MODEL 5.1 INTRODUCTION The survey presented in Chapter 1 has shown that Model based testing approach for automatic generation of test
More informationTCOM 501: Networking Theory & Fundamentals. Lecture 11 April 16, 2003 Prof. Yannis A. Korilis
TOM 50: Networking Theory & undamentals Lecture pril 6, 2003 Prof. Yannis. Korilis 2 Topics Routing in ata Network Graph Representation of a Network Undirected Graphs Spanning Trees and Minimum Weight
More informationAn algorithm for Performance Analysis of SingleSource Acyclic graphs
An algorithm for Performance Analysis of SingleSource Acyclic graphs Gabriele Mencagli September 26, 2011 In this document we face with the problem of exploiting the performance analysis of acyclic graphs
More informationAn Inference Mechanism for PointInterval Logic
Proceedings of the TwentyFirst International FLIRS onference (2008) n Inference Mechanism for PointInterval Logic Mashhood Ishaque omputer Science epartment Tufts University, Medford, M mishaq01@cs.tufts.edu
More informationAND/OR Graph Search for Genetic Linkage Analysis
/OR Graph Search for Genetic Linkage Analysis Radu Marinescu and Rina Dechter School of Information and omputer Science University of alifornia, Irvine, A 926973425 {radum,dechter}@ics.uci.edu Abstract
More informationChapter 8: Data Abstractions
Chapter 8: Data Abstractions Computer Science: An Overview Tenth Edition by J. Glenn Brookshear Presentation files modified by Farn Wang Copyright 28 Pearson Education, Inc. Publishing as Pearson AddisonWesley
More informationFaster parameterized algorithms for Minimum FillIn
Faster parameterized algorithms for Minimum FillIn Hans L. Bodlaender Pinar Heggernes Yngve Villanger Abstract We present two parameterized algorithms for the Minimum FillIn problem, also known as Chordal
More informationmywbut.com Uninformed Search
Uninformed Search 1 2.4 Search Searching through a state space involves the following: set of states Operators and their costs Start state test to check for goal state We will now outline the basic search
More informationQuery Processing & Optimization
Query Processing & Optimization 1 Roadmap of This Lecture Overview of query processing Measures of Query Cost Selection Operation Sorting Join Operation Other Operations Evaluation of Expressions Introduction
More informationOrganizing Spatial Data
Organizing Spatial Data Spatial data records include a sense of location as an attribute. Typically location is represented by coordinate data (in 2D or 3D). 1 If we are to search spatial data using the
More informationCSE 473: Artificial Intelligence
CSE 473: Artificial Intelligence Constraint Satisfaction Luke Zettlemoyer Multiple slides adapted from Dan Klein, Stuart Russell or Andrew Moore What is Search For? Models of the world: single agent, deterministic
More informationIndex. C cmp;l, 126. C rq;l, 127. A A sp. B BANDWIDTH, 32, 218 complexity of, 32 kbandwidth, 32, 218
, 24, 96, 116 compatible, 125 ++, 38 [], 125 P;l, 25 Q;l, 127 A A sp I, 166, 173 A sp R, 166, 173 174 A tw I, 189, 205 A tw R, 189, 205 210 adjacency list representation, 13 adjacent, 9 algorithm, 11
More information15.4 Longest common subsequence
15.4 Longest common subsequence Biological applications often need to compare the DNA of two (or more) different organisms A strand of DNA consists of a string of molecules called bases, where the possible
More informationThese notes present some properties of chordal graphs, a set of undirected graphs that are important for undirected graphical models.
Undirected Graphical Models: Chordal Graphs, Decomposable Graphs, Junction Trees, and Factorizations Peter Bartlett. October 2003. These notes present some properties of chordal graphs, a set of undirected
More informationExact Algorithms Lecture 7: FPT Hardness and the ETH
Exact Algorithms Lecture 7: FPT Hardness and the ETH February 12, 2016 Lecturer: Michael Lampis 1 Reminder: FPT algorithms Definition 1. A parameterized problem is a function from (χ, k) {0, 1} N to {0,
More informationOn Structural Parameterizations of the Matching Cut Problem
On Structural Parameterizations of the Matching Cut Problem N. R. Aravind, Subrahmanyam Kalyanasundaram, and Anjeneya Swami Kare Department of Computer Science and Engineering, IIT Hyderabad, Hyderabad,
More information