AND/OR MultiValued Decision Diagrams for Constraint Networks


 Opal Haynes
 1 years ago
 Views:
Transcription
1 N/OR MultiValued ecision iagrams for onstraint Networks Robert Mateescu, and Rina echter 2 lectrical ngineering epartment, alifornia Institute of Technology Pasadena, onald ren School of Information and omputer Science, University of alifornia, Irvine Irvine, bstract. The paper is an overview of a recently developed compilation data structure for graphical models, with specific application to constraint networks. The N/OR MultiValued ecision iagram (OM) augments well known decision diagrams (Os, Ms) with N nodes, in order to capture function decomposition structure. The OM is based on a pseudo tree of the network, rather than a linear ordering of its variables. The OM of a constraint network is a canonical form given a pseudo tree. We describe two main approaches for compiling the OM of a constraint network. The first is a top down, searchbased procedure, that works by applying reduction rules to the trace of the memory intensive N/OR search algorithm. The second is a bottom up, inferencebased procedure, that uses a ucket limination schedule. or both algorithms, the compilation time and the size of the OM are, in the worst case, exponential in the treewidth of the constraint graph, rather than pathwidth as is known for ordered binary decision diagrams (Os). Introduction The paper is an overview of N/OR MultiValued ecision iagrams (OMs) as a compiled data structure for constraint networks. We present here an extension of the work in [], while still maintaining the focus on constraint networks. OMs for weighted graphical models and for constraint optimization were presented in [2, 3]. The OM is based on two existing frameworks: () N/OR search spaces for graphical models; (2) decision diagrams. N/OR search spaces [4 6] have proven to be a unifying framework for various classes of search algorithms for graphical models. The main novelty is the exploitation of independencies between variables during search, which can provide exponential speedups over traditional search methods that can be viewed as traversing an OR structure. The N nodes capture problem decomposition into independent subproblems, and the OR nodes represent branching according to variable values. ecision diagrams are widely used in many areas of research, especially in software and hardware verification [7, 8]. represents a oolean function by a directed This work was done while at the University of alifornia, Irvine.
2 acyclic graph with two sink nodes (labeled 0 and ), and every internal node is labeled with a variable and has exactly two children: low for 0 and high for. If isomorphic nodes were not merged, we would have the full search tree explored by the backtracking algorithm. The tree is ordered if variables are encountered in the same order along every branch. It can then be compressed by merging isomorphic nodes (i.e., with the same label and identical children), and by eliminating redundant nodes (i.e., whose low and high children are identical). The result is the celebrated reduced ordered binary decision diagram, or O for short, introduced by ryant [9]. owever, the underlying structure is OR (i.e., linear structure rather than tree). If N/OR search trees are reduced by node merging and redundant nodes elimination we get a compact search graph that can be viewed as a representation augmented with N nodes. This paper shows how to combine the two ideas, creating a decision diagram that has an N/OR structure, thus exploiting problem decomposition. s a detail, the number of values is also increased from two to any constant. In the context of constraint networks, decision diagrams can be used to represent the whole set of solutions, facilitating solutions count, solution enumeration and queries on equivalence of constraint networks. The benefit of moving from OR structure to N/OR is in a lower complexity of the algorithms and size of the compiled structure. It typically moves from being bounded exponentially in pathwidth pw, which is characteristic to chain decompositions or linear structures, to being exponentially bounded in treewidth w, which is characteristic of tree structures (it always holds that w pw and pw w log n). In both cases, the compiled structure achieved in practice is often far smaller than what the bounds suggest. decision diagram offers a compilation of a propositional knowledgebase. multivalued N/OR decision diagram extends compilation to general graphical models. The knowledge compilation approach has become an important research direction in automated reasoning in the past decades [0 2]. Typically, a knowledge representation language is compiled into a compact data structure on which various queries can be answered quickly. ccordingly, the computational effort can be divided between an offline and an online phase where most of the work is pushed offline. ompilation can also be used to generate compact building blocks to be used by online algorithms multiple times. Macrooperators compiled during or prior to search can be viewed in this light [3], while in graphical models the building blocks are the functions whose compact, compiled, representation can be used effectively across many tasks. s one example, consider product configuration tasks and imagine a user that chooses sequential options to configure a product. In a naive system, the user would be allowed to choose any valid option at the current level based only on the initial constraints, until either the product is configured, or else, when a deadend is encountered, the system would backtrack to some previous state and continue from there. This would in fact be a search through the space of possible partial configurations. Needless to say, it would be very unpractical, and would offer the user no guarantee of finishing in a limited time. system based on compilation would actually build the backtrackfree search space in the offline phase, and represent it as compactly as possible. In the online phase, only valid partial configurations (i.e., that can be extended to a full valid
3 configuration) are allowed, and depending on the query type, response time guarantees can be offered in terms of the size of the compiled structure. Numerous other examples, such as diagnosis and planning problems can be formulated as graphical models, and for which compilation would be useful. ompilation in diagnosis can facilitate fast detection of possible faults or explanations for some unusual behavior. In planning, compilation would allow swift adjustments according to changes in the environment. Probabilistic models are one of the most used types of graphical models, and the basic query is to compute conditional probabilities of some variables given the evidence. compact compilation of a probabilistic model would allow fast response for any change in the evidence along time. ormal verification is another example where compilation is heavily used to compare equivalence of circuit design, or to check the behavior of a circuit. inary ecision iagram () [9] are arguably the most widely known and used compiled structure. Our OM proposal is related to two earlier research lines within the literature. The first is the work on isjoint Support ecompositions (S) [4], investigated within the area of design automation [5], that were proposed as enhancements for s aimed at exploiting function decomposition. The second is the work on trees [6]. nother related proposal is the recent work by argier and Vilarem [7] on compiling SPs into treedriven automata. We will comment more on the relationship between these work and OM in the related work section. The structure of the paper is as follows. Section 2 provides the preliminaries. Section 3 gives and overview of N/OR search spaces. Section 4 introduces the OM and Section 5 shows that it is a canonical form for constraint networks. Section 6 describes a search based algorithm for compiling the OM. Section 7 presents a compilation algorithm based on a ucket limination schedule and the PPLY operation. Section 8 presents related work and Section 9 concludes. 2 Preliminaries constraint network and its associated graph are defined in the usual way: efinition (constraint network). constraint network is a 3tuple R = X,,, where: X = {X,...,X n } is a set of variables; = {,..., n } is the set of their finite domains of values, with cardinalities k i = i and k = max n i= k i ; = {,..., r } is a set of constraints over subsets of X. ach constraint is defined as = (S i,r i ), where S i is the set of variables on which the constraint is defined, called its scope, and R i is the relation defined on S i. efinition 2 (constraint graph). The constraint graph (or primal graph) of a constraint network is an undirected graph, = (X, ), that has variables as its vertices and an edge connecting any two variables that appear in the scope (set of arguments) of the same constraint. pseudo tree resembles the tree rearrangements introduced in [8]: efinition 3 (pseudo tree). pseudo tree of a graph = (X,) is a rooted tree T having the same set of nodes X, such that every arc in is a backarc in T (i.e., it connects nodes on the same path from root).
4 f() (a) Table 0 0 (b) Ordered tree (c) Isomorphic nodes (d) Redundant nodes (e) O ig.. oolean function representation and reduction rules efinition 4 (induced graph, induced width, treewidth, pathwidth). n ordered graph is a pair (,d), where is an undirected graph, and d = (X,...,X n ) is an ordering of the nodes. The width of a node in an ordered graph is the number of neighbors that precede it in the ordering. The width of an ordering d, denoted w(d), is the maximum width over all nodes. The induced width of an ordered graph, w (d), is the width of the induced ordered graph obtained as follows: for each node, from last to first in d, its preceding neighbors are connected in a clique. The induced width of a graph, w, is the minimal induced width over all orderings. The induced width is also equal to the treewidth of a graph. The pathwidth pw of a graph is the treewidth over the restricted class of orderings that correspond to chain decompositions. 2. inary ecision iagrams Review ecision diagrams are widely used in many areas of research to represent decision processes. In particular, they can be used to represent functions. ue to the fundamental importance of oolean functions, a lot of effort has been dedicated to the study of inary ecision iagrams (s), which are extensively used in software and hardware verification [7, 8]. ryant [9] introduced the Ordered inary ecision iagram (O). The order of variables along any path of an O is the same. Os provide a compact representation and efficient operations (the apply procedure, that combines two Os by an operation is at most quadratic in the sizes of the input diagrams). xample. igure (a) shows a table representation of a oolean function. binary tree representation is shown in igure (b). The internal round nodes represent the variables, the solid edges are the (or high) value, and the dotted edges are the 0 (or low) value. The leaf square nodes show the value of the function for each assignment along a path. The tree is ordered, because variables appear in the same order along each path. There are two reduction rules that transform a decision diagram into an equivalent one: () isomorphism: merge nodes that have the same label and the same children; (2)
5 () 2 () 3 () 4 () (a) onstraint network : h 4 () : 2 () h 3 () h 2 () : 3 () : () h () : 4 () (b) execution bucket bucket bucket bucket bucket (c) ucket tree ig. 2. ucket limination redundancy: eliminate nodes whose low and high edges point to the same node, and connect parent of removed node directly to child of removed node. pplying the two reduction rules exhaustively yields a reduced O, sometimes denoted ro. We will just use O and assume that it is completely reduced. xample 2. igure (c) shows the binary tree from (b) after the isomorphic terminal nodes have been merged. The highlighted nodes, labeled with, are isomorphic, and igure (d) shows the result after they are merged. Now, the highlighted nodes labeled with and are redundant, and removing them gives the O in igure (e). 2.2 ucket limination Review ucket limination () [9] is a well known variable elimination algorithm for inference in graphical models. n ordering d = (X,X 2,...,X n ) of the variables guides the execution of. ach variable is associated with a bucket. ach constraint from is placed in the bucket of its latest variable in d. uckets are processed from X n to X by eliminating the bucket variable (the constraints residing in the bucket are joined together, and the bucket variable is projected out) and placing the resulting constraint (also called message) in the bucket of its latest variable in d. fter its execution, renders the network backtrack free, and a solution can be produced by assigning variables along d. can also produce the solutions count if marginalization is done by summation (rather than projection) over the functional representation of the constraints, and join is substituted by multiplication. also constructs a bucket tree, by linking the bucket of each X i to the destination bucket of its message (called the parent bucket). node in the bucket tree typically has a bucket variable, a collection of constraints, and a scope (the union of the scopes of its constraints). If the nodes of the bucket tree are replaced by their respective bucket variables, it is easy to see that we obtain a pseudo tree of the constraint graph. xample 3. igure 2(a) shows a network with four constraints. igure2(b) shows the execution of ucket limination along d = (,,,,). The buckets are processed from to 3. igure 2(c) shows the bucket tree. The pseudo tree corresponding to the order d is given in ig. 3(a). 3 igure 2 reverses the top down bucket processing described in earlier papers.
6 [ ] 0 0 [] [] [] [] (a) Pseudo tree (b) Search tree (c) ontext minimal graph ig. 3. N/OR search tree 3 Overview of N/OR Search Space for onstraint Networks The N/OR search space is a recently introduced [4 6] unifying framework for advanced algorithmic schemes for graphical models. Its main virtue consists in exploiting independencies between variables during search, which can provide exponential speedups over traditional search methods oblivious to problem structure. Since N/OR MultiValued ecision iagrams are based on N/OR search spaces, we provide a comprehensive overview for completeness sake. 3. N/OR Search Tree The N/OR search tree is guided by a pseudo tree of the constraint graph. The idea is to exploit the problem decomposition into independent subproblems during search. ssigning a value to a variable (conditioning) is equivalent in graph terms to removing that variable (and its incident edges) from the constraint graph. partial assignment can therefore lead to the decomposition of the residual constraint graph into independent components, each of which can be searched (or solved) separately. The pseudo tree captures precisely all these decompositions, given an order of variable instantiation. efinition 5 (N/OR search tree of a constraint network). iven a constraint network R = X,,, its constraint graph and a pseudo tree T of, the associated N/OR search tree has alternating levels of OR and N nodes. The OR nodes are labeled X i and correspond to variables. The N nodes are labeled X i,x i and correspond to value assignments. The structure of the N/OR search tree is based on T. The root is an OR node labeled with the root of T. The children of an OR node X i are N nodes labeled with assignments X i,x i that are consistent with the assignments along the path from the root. The children of an N node X i,x i are OR nodes labeled with the children of variable X i in the pseudo tree T. The leaves of N nodes are labeled with. There is a one to one correspondence between solution subtrees of the N/OR search graph and solutions of the constraint network [4]. xample 4. igure 3 shows an example of an N/OR search tree for the constraint network given in igure 2(a), assuming all tuples are consistent, and variables are binary valued. When some tuples are inconsistent, some of the paths in the tree do not exists. igure 3(a) gives the pseudo tree that guides the search, from top to bottom, as indicated
7 by the arrows. The dotted arcs are backarcs from the primal graph. igure 3(b) shows the N/OR search tree, with the alternating levels of OR (circle) and N (square) nodes, and having the structure indicated by the pseudo tree. The N/OR search tree can be traversed by a depth first search algorithm, thus using linear space. It was already shown [8, 20, 2, 4, 6] that: Theorem. iven a constraint network R and a pseudo tree T of depth m, the size of the N/OR search tree based on T is O(n k m ), where k bounds the domains of variables. constraint network of treewidth w has a pseudo tree of depth at most w log n, therefore it has an N/OR search tree of size O(n k w log n ). The N/OR search tree expresses the set of all possible assignments to the problem variables (all solutions). The difference from the traditional OR search space is that a solution is no longer a path from root to a leaf, but rather a subtree, defined as follows: efinition 6 (solution subtree). solution subtree of an N/OR search tree contains the root node. or every OR nodes it contains one of its child nodes and for each of its N nodes it contains all its child nodes, and all its leaf nodes are consistent. 3.2 N/OR Search raph The N/OR search tree may contain nodes that root identical conditioned (on the current path assignment) subproblems. These nodes are said to be unifiable. When unifiable nodes are merged, the search space becomes a graph. Its size becomes smaller at the expense of using additional memory by the search algorithm. The depth first search algorithm can therefore be modified to cache previously computed results, and retrieve them when the same nodes are encountered again. The notion of unifiable nodes is defined formally next. efinition 7 (minimal N/OR graph, isomorphism). Two N/OR search graphs and are isomorphic if there exists a one to one mapping σ from the vertices of to the vertices of such that for any vertex v, if σ(v) = v, then v and v root identical subgraphs relative to σ. The minimal N/OR graph is such that all the isomorphic subgraphs are merged. Isomorphic nodes (that root isomorphic subgraphs) are also said to be unifiable. Theorem 2 ([6]). The minimal N/OR search graph of a constraint network R relative to a pseudotree T is unique. Note that the definition of minimality in [6] is based only on isomorphism reduction. We extend it by also eliminating the redundant nodes. The previous theorem only shows that given an N/OR graph, the merge operator has a fixed point, which is the minimal N/OR graph. It can be shown that the OM is a canonical representation (given a pseudo tree), namely that any two equivalent constraint networks can be represented by the same unique OM, and the OM is minimal in terms of number of nodes. Some unifiable nodes can be identified based on their contexts. We can define graph based contexts for both OR nodes and N nodes, just by expressing the set of ancestor
8 variables in T that completely determine a conditioned subproblem. owever, it can be shown that using caching based on OR contexts makes caching based on N contexts redundant and vice versa, so we will only use OR caching. ny value assignment to the context of X separates the subproblem below X from the rest of the network. efinition 8 (OR context). iven a pseudo tree T of an N/OR search space, context(x) = [X...X p ] is the set of ancestors of X in T, ordered descendingly, that are connected in the primal graph to X or to descendants of X. efinition 9 (context unifiable OR nodes). iven an N/OR search graph, two OR nodes n and n 2 are context unifiable if they have the same variable label X and the assignments of their contexts is identical. Namely, if π is the partial assignment of variables along the path to n, and π 2 is the partial assignment of variables along the path to n 2, then their restriction to the context of X is the same: π context(x) = π 2 context(x). The depth first search algorithm that traverses the N/OR search tree, can be modified to traverse a graph, if enough memory is available. We could allocate a cache table for each variable X, the scope of the table being context(x). The size of the cache table for X is therefore the product of the domains of variables in its context. or each variable X, and for each possible assignment to its context, the corresponding conditioned subproblem is solved only once and the computed value is saved in the cache table, and whenever the same context assignment is encountered again, the value of the subproblem is retrieved from the cache table. Such an algorithm traverses what is called the context minimal N/OR graph. efinition 0 (context minimal N/OR graph). The context minimal N/OR graph is obtained from the N/OR search tree by merging all the context unifiable OR nodes. It was already shown that [20, 4, 6]: Theorem 3. iven a constraint network R, its primal graph and a pseudo tree T, the size of the context minimal N/OR search graph based on T, and therefore the size of its minimal N/OR search graph, is O(n k w T () ), where wt () is the induced width of over the depth first traversal of T, and k bounds the domain size. xample 5. We refer again to igure 3. igure 3(a) shows the pseudo tree, where the (OR) context of each node appears in square brackets. Notice that the context of a node is identical to the message scope from its bucket in ig. 2. igure 3(c) shows the context minimal N/OR graph. 4 N/OR MultiValued ecision iagram (OM) The context minimal N/OR graph (efinition 0) offers an effective way of identifying some unifiable nodes during the execution of the search algorithm. Namely, context unifiable nodes are discovered based only on their paths from the root, without actually
9 solving their corresponding subproblems. owever, merging based on context is not complete, which means that there may still exist unifiable nodes in the search graph that do not have identical contexts. Moreover, some of the nodes in the context minimal N/OR graph may be redundant, for example when the set of solutions rooted at variable X i does not depend on the specific value assigned to X i (this situation is not detectable based on context). This is sometimes termed as interchangeable values or symmetrical values. We propose to augment the minimal N/OR search graph with removing redundant variables as is common in O representation, as well as to adopt notational conventions common in this community. This yields a data structure that we call N/OR, that exploits decomposition by using N nodes. We present the extension over multivalued variables yielding N/OR M or OM. Subsequently we present two algorithms for compiling the canonical OM of a constraint network: the first is search based, and uses the memory intensive N/OR graph search to generate the context minimal N/OR graph, and then reduces it bottom up by applying reduction rules; the second is inference based, and uses a ucket limination schedule to combine the OMs of initial functions by PPLY operations (similar to the apply for Os). oth approaches have the same worst case complexity as the N/OR graph search with context based caching, and also the same complexity as ucket limination, namely time and space exponential in the treewidth of the problem, O(n k w ). 4. rom N/OR Search raphs to ecision iagrams We will now show how we can process an N/OR search graph by reduction rules similar to the case of Os, in order to obtain a representation of minimal size. In the case of Os, a node is labeled with a variable name, for example, and the low (dotted line) and high (solid line) outgoing arcs capture the restriction of the function to the assignments = 0 or =. To determine the value of the function, one needs to follow either one or the other (but not both) of the outgoing arcs from. The straightforward extension of Os to multivalued variables (multivalued decision diagrams, or Ms) was presented in [22]. We generalize the O and M representations by allowing each outgoing arc to be an N arc. n N arc connects a node to a set of nodes, and captures the decomposition of the problem into independent components. We define the N/OR ecision iagram representation based on N/OR search graphs. We find it useful to introduce the metanode data structure, which defines small portions of any N/OR graph, based on an OR node and its N children: efinition (metanode). metanode u in an N/OR search graph of a constraint network consists of an OR node labeled X (therefore var(u) = X) and its k N children labeled x,...,x k that correspond to the value assignments of X. ach N node labeled x i stores a list of pointers to child metanodes, denoted by u.children i. We also define two special metanodes, that will play the role of the terminal nodes in Os. The terminal metanode 0 indicates the inconsistent assignments, while the terminal metanode indicates the consistent ones.
10 ny N/OR search graph can now be viewed as a diagram of metanodes, simply by grouping OR nodes with their N children, and adding the terminal metanodes appropriately. It is now easy to see when a variable is redundant with respect to the outcome of the function based on the current partial assignment. Intuitively, any assignment to a redundant variable should lead to the same set of solutions. efinition 2 (redundant metanode). iven an N/OR search graph represented with metanodes, a metanode u with var(u) = X and (X) = k is redundant iff u.children =... = u.children k. n N/OR graph, that contains a redundant metanode u, can be transformed into an equivalent graph by replacing any incoming arc into u with its common list of children u.children (joined in an N arc), and then removing u and its outgoing arcs from. The notion of isomorphism is extended naturally from N/OR graphs to metanodes. efinition 3 (isomorphic metanodes). iven an N/OR search graph represented with metanodes, two metanodes u and v having var(u) = var(v) = X and (X) = k are isomorphic iff u.children i = v.children i, i {,...,k}. Naturally, the N/OR graph obtained by merging isomorphic metanodes is equivalent to the original one. We can now define the N/OR MultiValued ecision iagram: efinition 4 (OM). n N/OR MultiValued ecision iagram (OM) is a weighted N/OR search graph that is completely reduced by isomorphic merging and redundancy removal, namely: () it contains no isomorphic metanodes; and (2) it contains no redundant metanodes. 5 OMs for onstraint Networks re anonical orms It is well known that Os are canonical representations of oolean functions given an ordering of the variables [9], and this property extends to Ms [22]. In the case of Os and OMs, the canonicity is with respect to a pseudo tree, following the transition from total orders (that correspond to a linear ordering) to partial orders (that correspond to a pseudo tree ordering). Proposition. Let f be a function, not always zero, defined by a constraint network over X. iven a partition {X,...,X m } of the set of variables X (namely, X i X j = φ, i j, and X = m i= Xi ), if f = f... f m and f = g... g m, such that scope(f i ) = scope(g i ) = X i for all i {,...,m}, then f i = g i for all i {,...,m}. Namely, if f can be decomposed over the given partition, then the decomposition is unique.
11 ased on the previous proposition, in can be shown that OMs for constraint networks are canonical representations given a pseudo tree. Theorem 4 (OMs are canonical for a given pseudo tree). iven a constraint network, and a pseudo tree T of its constraint graph, there is a unique (up to isomorphism) OM that represents it, and it has the minimal number of metanodes. The proof, omitted for space reasons, is by structural induction over the depth of the pseudo tree. constraint network is defined by its relations (or functions). There exist equivalent constraint networks that are defined by different sets of functions, even having different scope signatures. owever, equivalent constraint networks define the same function, and we can ask if the OM of different equivalent constraint networks is the same. The following theorem can be derived immediately from Theorem 4. Theorem 5. Two equivalent constraint networks that admit the same pseudo tree T have the same OM based on T. 6 Using N/OR Search to enerate OMs In Section 4. we described how we can transform an N/OR graph into an OM by applying reduction rules. In Section 6. we describe the explicit algorithm that takes as input a constraint network, performs N/OR search with contextbased caching to obtain the context minimal N/OR graph, and in Section 6.2 we give the procedure that applies the reduction rules bottom up to obtain the OM. The reduction procedure can actually be incorporated in the search algorithm, but we present it separately for clarity. 6. lgorithm N/ORSROM lgorithm, called N/ORSROM, compiles a constraint network into an OM. memory intensive (with contextbased caching) N/OR search is used to create the context minimal N/OR graph (see efinition 0). The input to N/ORSROM is a constraint network R and a pseudo tree T, that also defines the ORcontext of each variable. ach variable X i has an associated cache table, whose scope is the context of X i in T. This ensures that the trace of the search is the context minimal N/OR graph. list denoted by L Xi (see line 34), is used for each variable X i to save pointers to metanodes labeled with X i. These lists are used by the procedure that performs the bottom up reduction, per layers of the N/OR graph (one layer contains all the nodes labeled with one given variable). The fringe of the search is maintained on a stack calledopn. The current node (either OR or N node) is denoted by n, its parent by p, and the current path by π n. The children of the current node are denoted by successors(n). or each node n, the oolean attribute consistent(n) indicates if the current path can be extended to a solution. This information is useful for pruning the search space.
12 lgorithm : N/OR SR  OM 2 input : R = X,, ; pseudo tree T rooted at X ; parents pa i (ORcontext) for every variable X i. output : OM of R. forall X i X do Initialize contextbased cache table ache Xi (pa i) withnull entries 3 reate new OR nodet, labeled with X i; consistent(t) true; pushton top ofopn 4 whileopn φ do 5 n top(opn); removenfromopn // orward 6 successors(n) φ 7 ifnis an OR node labeled with X i then // ORexpand 8 if ache Xi (asgn(π n)[pa i]) null then 9 onnect parent ofnto ache Xi (asgn(π n)[pa i]) // Use the cached pointer 0 else forall x i i do 2 reate new N nodet, labeled with X i, x i 3 if X i, x i is consistent with π n then // onstraint Propagation 4 consistent(t) true 5 addtto successors(n) 6 else 7 consistent(t) false 8 make terminal 0 the only child oft ifnis an N node labeled with X i, x i then if children T (X i) == φ then make terminal the only child ofn else forall Y children T (X i) do reate new OR nodet, labeled with Y consistent(t) false addtto successors(n) // Nexpand dd successors(n) to top ofopn while successors(n) == φ do // acktrack letpbe the parent ofn ifnis an OR node labeled with X i then if X i == X then // Search is complete all ottomupreduction procedure // begin reduction to OM ache(asgn(π n)[pa i]) n // Save in cache dd metanode ofnto the list L X i consistent(p) consistent(p) consistent(n) if consistent(p) == false then // heck if p is deadend remove successors(p) fromopn successors(p) φ ifnis an N node labeled with X i, x i then consistent(p) consistent(p) consistent(n); removenfrom successors(p) n p The algorithm is based on two mutually recursive steps: orward (beginning at line 5) and acktrack (beginning at line 28), which call each other (or themselves) until the search terminates. In the forward phase, the N/OR graph is expanded top down. The two types of nodes, N and OR, are treated differently according to their semantics. efore an OR node is expanded, the cache table of its variable is checked (line 8). If the entry is not null, a link is created to the already existing OR node that roots the graph equivalent to the current subproblem. Otherwise, the OR node is expanded by
13 generating its N descendants. ach value x i of X i is checked for consistency (line 3). ny level of constraint propagation can be performed in this step (e.g., look ahead, arc consistency, path consistency, iconsistency etc.). The computational overhead can increase, in the hope of pruning the search space more aggressively. We should note that constraint propagation is not crucial for the algorithm, and the complexity guarantees are maintained even without it. The consistent N nodes are added to the list of successors of n (line 5), while the inconsistent ones are linked to the terminal 0 metanode (line 8). n N node n labeled with X i,x i is expanded (line 9) based on the structure of the pseudo tree. If X i is a leaf in T, thennis linked to the terminal metanode (line 2). Otherwise, an OR node is created for each child of X i in T (line 23). The forward step continues as long as the current node is not a deadend and still has unevaluated successors. The backtrack phase is triggered when a node has an empty set of successors (line 28). Note that, as each successor is processed, it is removed from the set of successors in line 4. When the backtrack reaches the root (line 3), the search is complete, the context minimal N/OR graph has been generated, and the Procedure OTTOMUPRUTION is called. When the backtrack step processes an OR node (line 30), it saves a pointer to it in cache, and also adds a pointer to the corresponding metanode to the list L Xi. The consistent attribute of the N parent p is updated by conjunction with consistent(n). If the N parent p becomes inconsistent, it is not necessary to check its remaining OR successors (line 37). When the backtrack step processes an N node (line 39), the consistent attribute of the OR parent p is updated by disjunction with consistent(n). The N/OR search algorithm usually maintains a value for each node, corresponding to a task that is solved (e.g., counting solutions, or cost of the optimal solution). We did not include values in our description because an OM is just an equivalent representation of the original constraint network R. ny task over R can be solved by a traversal of the OM. It is however up to the user to include more information in the metanodes (e.g., number of solutions for a subproblem). 6.2 Reducing the ontext Minimal N/OR raph to an OM Procedure ottomupreduction processes the variables bottom up relative to the pseudo tree T. We use the depth first traversal ordering of T (line ), but any other bottom up ordering is as good. The outer for loop (starting at line 2) goes through each level of the context minimal N/OR graph (where a level contains all the OR and N nodes labeled with the same variable, in other words it contains all the metanodes of that variable). or efficiency, and to ensure the complexity guarantees, a hash table, initially empty, is used for each level. The inner for loop (starting at line 4) goes through all the metanodes of a level, that are also saved (or pointers to them are saved) in the list L Xi. or each new metanode n in the list L Xi, in line 5 the hash table is checked to verify if a node isomorphic withnalready exists. If the hash table already contains a nodepcorresponding to the hash key (X i,n.children,...,n.children ki ), thenpandnare isomorphic and should be merged. Otherwise, if the new metanoden
14 Procedure ottomupreduction input : constraint network R = X,, ; a pseudo tree T of the primal graph, rooted at X ; ontext minimal N/OR graph, and lists L X i of metanodes for each level X i. output : OM of R. Let d = {X,..., X n} be the depth first traversal ordering of T 2 for i n down to do 3 Let be a hash table, initially empty 4 forall metanodesnin L X i do 5 if (X i, n.children,..., n.children ki ) returns a metanodepthen 6 mergenwithpin the N/OR graph else ifnis redundant then eliminatenfrom the N/OR graph else hashninto the table : (X i, n.children,..., n.children ki ) n 2 return reduced N/OR graph is redundant, then it is eliminated from the N/OR graph. If none of the previous two conditions is met, then the new metanodenis hashed into the table. Proposition 2. The output of Procedure ottomupreduction is the OM of R along the pseudo tree T, namely the resulting N/OR graph is completely reduced. Note that we explicated Procedure ottomupreduction separately only for clarity. In practice, it can actually be included in lgorithm N/ORSR OM, and the reduction rules can be applied whenever the search backtracks. We can maintain a hash table for each variable, during the N/OR search, to store pointers to metanodes. When the search backtracks out of an OR node, it can already check the redundancy of that metanode, and also look up in the hash table to check for isomorphism. Therefore, the reduction of the N/OR graph can be done during the N/OR search, and the output will be the OM of R. rom Theorem 3 and Proposition 2 we can conclude: Theorem 6. iven a constraint network R and a pseudo tree T of its constraint graph, the OM of R corresponding to T has size bounded by O(n k w T () ) and it can be computed by lgorithm N/ORSROM in time O(n k w T () ), where wt () is the induced width of over the depth first traversal of T, and k bounds the domain size. 7 Using ucket limination to enerate OMs In this section we propose to use a ucket limination () type algorithm to guide the compilation of a constraint network into an OM. The idea is to express the constraints as OMs, and then combine them via the PPLY operator by following a schedule. The PPLY is a procedure very similar to that from Os [9], but it is adapted to N/OR search graphs. It takes as input two constraints represented as OMs based on the same pseudo tree, and outputs their join, also represented as an OM based on the same pseudo tree.
15 (a) (b) ig. 4. (a) onstraint graph for = {,..., 9}, where =, 2 =, 3 =, 4 =, 5 =, 6 =, 7 =, 8 =, 9 = ; (b) Pseudo tree (bucket tree) for ordering d = (,,,,,,, ) m 7 m 7 m 6 m 3 m 3 9 m 5 m 6 m 4 5 m 2 m m 5 m 4 m 2 m ig. 5. xecution of V with OMs xample 6. onsider the constraint network defined by X = {,,...,}, =... = = {0,} and the constraints (where denotes XOR): =, 2 =, 3 =, 4 =, 5 =, 6 =, 7 =, 8 =, 9 =. The constraint graph is shown in igure 4(a). onsider the ordering d = (,,,,,,,). The pseudo tree (or bucket tree) induced by d is given in ig. 4(b). igure 5 shows the execution of with OMs along ordering d. Initially, the constraints through 9 are represented as OMs and placed in the bucket of their latest variable in d. The scope of any original constraint always appears on a path from root to a leaf in the pseudo tree. Therefore, each original constraint is represented by an OM based on a chain. (i.e. there is no branching into independent components at any point). The chain is just the scope of the constraint, ordered according to d. or bivalued variables, the original constraints are represented by Os, for multiplevalued variables they are Ms. Note that we depict meta
16 0 (a) 0 (b) ig. 6. (a) The final OM; (b) The O corresponding to d nodes: one OR node and its two N children, that appear inside each gray node. The dotted edge corresponds to the 0 value (the low edge in Os), the solid edge to the value (the high edge). We have some redundancy in our notation, keeping both N value nodes and arctypes (doted arcs from 0 and solid arcs from ). The scheduling is used to process the buckets in reverse order of d. bucket is processed by joining all the OMs inside it, using the PPLY operator. owever, the step of elimination of the bucket variable is omitted because we want to generate the full OM. In our example, the messages m = 2 and m 2 = 3 4 are still based on chains, so they are still Os. Note that they still contain the variables and, which have not been eliminated. owever, the message m 3 = 5 m m 2 is not an O anymore. We can see that it follows the structure of the pseudo tree, where has two children, and. Some of the nodes corresponding to have two outgoing edges for value. The processing continues in the same manner. The final output of the algorithm, which coincides with m 7, is shown in igure 6(a). The O based on the same ordering d is shown in ig. 6(b). Notice that the OM has 8 nonterminal nodes and 47 edges, while the O has 27 nonterminal nodes and 54 edges. 7. lgorithm OM lgorithm 2, called OM, creates the OM of a constraint network by using a schedule for PPLY operations. iven an order d of the variables, a pseudo tree is created based on the constraint graph (this is just the bucket tree, or elimination tree of d). ach initial constraint i is then represented as an OM, denoted by, and placed in its bucket. To obtain the OM of a constraint, its scope is ordered according to d, a search tree (based on a chain) that represents i is generated, and then reduced by Procedure ottomupreduction. Then, the algorithm proceeds exactly like, with the only difference that the join of constraints (represented as OMs) is realized by the PPLY algorithm, and variables are not eliminated but aomdd i carried around to the destination bucket. The messages between buckets are initialized with the dummy OM of, denoted by aomdd, which is neutral for join.
17 lgorithm 2: OM input : onstraint network R = X,,, where X = {X,..., X n}, = {,..., r} ; order d = (X,..., X n) output : OM representing i i Let T be the pseudo tree (bucket tree) corresponding to d; for i to r do // place constraints in buckets 2 place aomdd in the bucket of its latest variable in d i 3 for i n down to do // process buckets 4 message(x i) aomdd // initialize with OM of ; 5 while bucket(x i) φ do // combine OMs in bucket of X i 6 pick aomdd f from bucket(x i); 7 bucket(x i) bucket(x i) \ { aomdd f }; 8 message(x i) PPLY(message(X i), aomdd f ) 9 0 add message(x i) to the bucket of the parent of X i in T return message(x ) * = ig. 7. xample of PPLY operation 7.2 The OM PPLY Operation The apply operator takes as input two OMs representing constraints and 2 and returns an OM representing their join 2. In Os the apply operator combines two input diagrams based on the same variable ordering. Likewise, in order to combine two OMs we assume that they are based on the same pseudo tree. This restriction is satisfied when we use PPLY to combine the constraints in the same bucket of the based algorithm. There are also more relaxed version of PPLY, when the pseudo trees of the two input constraints need only be compatible, rather than identical. Intuitively, this means that the pseudo trees generate partial orders (based on descendance relation) that are not in conflict. or space reasons, we will not present the details of the PPLY algorithm here, but refer the reader to []. We will just briefly describe it by an example. xample 7. igure 7 shows the result of combining two oolean functions by an N operation (or product). The input functions f and g are represented by OMs based on chain pseudo trees, while the results is based on the pseudo tree that expresses the decomposition after variables and are instantiated. The PPLY operator performs a depth first traversal of the two input OMs, and generates the resulting OM based on the output pseudo tree. Similar to the case of Os, a function or an OM can be identified by its root metanode. In this example the input metanodes have labels (, 2,, 2, etc.). The output metanode labeled by 2 2 is
18 the root of a diagram that represents the function obtained by combining the functions rooted by 2 and 2. The complexity of OM and the output size are similar to those of the search based algorithm: Theorem 7. The space complexity of OM and the size of the output OM are O(n k w ), where n is the number of variables, k is the maximum domain size and w is the treewidth of the bucket tree. The time complexity is bounded by O(r k w ), where r is the number of initial functions. 8 Related Work There are various lines of related research. The formal verification literature, beginning with [9] contains a very large number of papers dedicated to the study of s. owever, s are in fact OR structures (the underlying pseudo tree is a chain) and do not take advantage of the problem decomposition in an explicit way. The complexity bounds for Os are based on pathwidth rather than treewidth. s noted earlier, the work on isjoint Support ecomposition (S) is related to N/OR s in various ways [4]. The main common aspect is that both approaches show how structure decomposition can be exploited in a like representation. S is focused on oolean functions and can exploit more refined structural information that is inherent to oolean functions. In contrast, N/OR s assumes only the structure conveyed in the constraint graph. They are therefore more broadly applicable to any constraint expression and also to graphical models in general. They allow a simpler and higher level exposition that yields graphbased bounds on the overall size of the generated OM. McMillan introduced the trees [6], along with the operations for combining them. or circuits of bounded tree width, trees have linear space upper bound O( g 2 w22w ), where g is the size of the circuit g (typically linear in the number of variables) and w is the treewidth. This bound hides some very large constants to claim the linear dependence on g when w is bounded. owever, McMillan maintains that when the input function is a N expression trees have the same bounds as N/OR s, namely they are exponential in the treewidth only. The N/OR structure restricted to propositional theories is very similar to deterministic decomposable negation normal form (dnn) []. More recently, in [23], the trace of the PLL algorithm is used to generate an O, and compared with the typical formal verification approach of combining the Os of the input function according to some schedule. The structures that were investigated are still OR. Mcllester [24] introduced the case factor diagrams () which subsume Markov random fields of bounded tree width and probabilistic context free grammars (P). s are very much related to the N/OR graphs. The s target the minimal representation, by exploiting decomposition (similar to N nodes) but also by exploiting context sensitive information and allowing dynamic ordering of variables based on context. s do not eliminate the redundant nodes, and part of the cause is that they use
19 zero suppression. There is no claim about s being a canonical form, and also there is no description of how to combine two s. More recently, independently and in parallel to our work on N/OR graphs [4, 5], argier and Vilarem [7] proposed the compilation of SPs into treedriven automata, which have many similarities to our work. Their main focus is the transition from linear automata to tree automata (similar to that from OR to N/OR), and the possible savings for treestructured networks and hypertrees of constraints due to decomposition. Their compilation approach is guided by a treedecomposition while ours is guided by a variableelimination based algorithms. nd, it is well known that ucket limination and clustertree decomposition are in principle, the same [25]. 9 onclusion This paper gives an overview of a new compilation data structure for constraint networks. The N/OR Multivalued ecision iagram (OM) [ 3] emerges from the study of N/OR search spaces for graphical models [4, 5, 26, 6] and ordered binary decision diagrams (Os) [9]. raphical models algorithms that are searchbased and compiled datastructures such as s differ primarily by their choices of time vs memory. When we move from regular OR to an N/OR search space, the spectrum of algorithms available is improved for all time vs memory decisions. We believe that the N/OR search space clarifies the available choices and helps guide the user into making an informed selection of the algorithm that would fit best the particular query asked, the specific input function and the available computational resources. We presented the two main algorithmic approaches for compiling an OM for constraint networks. The first is a top down procedure, that uses memory intensive N/OR search, and applies reduction rules to the trace of the search. The second is a bottom up procedure that uses a ucket limination schedule to combine the constraints via the PPLY operator. s part of our current and future work, we are implementing, and experimenting with, the algorithms described here in order to provide an empirical evaluation. cknowledgments This work was supported in part by the NS grants IIS and IIS References. Mateescu, R., echter, R.: ompiling constraint networks into N/OR multivalued decision diagrams (OMs). In: Proceedings of the Twelfth International onference on Principles and Practice of onstraint Programming (P 06). (2006) Mateescu, R., echter, R.: nd/or multivalued decision diagrams (aomdds) for weighted graphical models. In: Proceedings of the Twenty Third onference on Uncertainty in rtificial Intelligence (UI 07). (2007) 3. Mateescu, R., Marinescu, R., echter, R.: N/OR multivalued decision diagrams for constraint optimization. In: Proceedings of the Thirteenth International onference on Principles and Practice of onstraint Programming (P 07). (2007)
20 4. echter, R., Mateescu, R.: Mixtures of deterministicprobabilistic networks and their N/OR search space. In: Proceedings of the Twentieth onference on Uncertainty in rtificial Intelligence (UI 04). (2004) echter, R., Mateescu, R.: The impact of N/OR search spaces on constraint satisfaction and counting. In: Proceedings of the Tenth International onference on Principles and Practice of onstraint Programming (P 04). (2004) echter, R., Mateescu, R.: N/OR search spaces for graphical models. rtificial Intelligence 7 (2007) larke,., rumberg, O., Peled,.: Model hecking. MIT Press (999) 8. McMillan, K.L.: Symbolic Model hecking. Kluwer cademic (993) 9. ryant, R..: raphbased algorithms for boolean function manipulation. I Transaction on omputers 35 (986) Selman,., Kautz,.: Knowledge compilation and theory approximation. Journal of the M 43 (996) arwiche,., Marquis, P.: knowledge compilation map. Journal of rtificial Intelligence Research (JIR) 7 (2002) adoli, M., onini,.m.: survey on knowledge compilation. I ommunications 0 (997) Korf, R., elner,.: isjoint pattern database heuristics. rtificial Intelligence 34 (2002) ertacco, V., amiani, M.: The disjunctive decomposition of logic functions. In: International onference on omputerided esign (I). (997) rayton, R., McMullen,.: The decomposition and factorization of boolean expressions. In: ISS, Proceedings of the International Symposium on ircuits and Systems. (982) McMillan, K.L.: ierarchical representation of discrete functions with application to model checking. In: omputer ided Verification. (994) argier,., Vilarem, M.: ompiling csps into treedriven automata for interactive solving. onstraints 9 (2004) reuder,.., Quinn, M.J.: Taking advantage of stable sets of variables in constraint satisfaction problems. In: Proceedings of the Ninth International Joint onference on rtificial Intelligence (IJI 85). (985) echter, R.: ucket elimination: unifying framework for reasoning. rtificial Intelligence 3 (999) ayardo, R., Miranker,.: complexity analysis of spacebound learning algorithms for the constraint satisfaction problem. In: Proceedings of the Thirteenth National onference on rtificial Intelligence (I 96). (996) arwiche,.: Recursive conditioning. rtificial Intelligence 25 (200) Srinivasan,., Kam, T., Malik, S., rayton, R.K.: lgorithms for discrete function manipulation. In: International onference on omputerided esign (I). (990) uang, J., arwiche,.: pll with a trace: rom sat to knowledge compilation. In: Proceedings of the Nineteenth International Joint onference on rtificial Intelligence (IJI 05). (2005) Mcllester,., ollins, M., Pereira,.: asefactor diagrams for structured probabilistic modeling. In: Proceedings of the Twentieth onference on Uncertainty in rtificial Intelligence (UI 04). (2004) echter, R., Pearl, J.: Tree clustering for constraint networks. rtificial Intelligence 38 (989) Mateescu, R., echter, R.: The relationship between N/OR search and variable elimination. In: Proceedings of the Twenty irst onference on Uncertainty in rtificial Intelligence (UI 05). (2005)
AND/OR MultiValued Decision Diagrams (AOMDDs) for Graphical Models
Journal of rtificial Intelligence Research 33 (28) 46559 Submitted 5/8; published 2/8 N/OR MultiValued ecision iagrams (OMs) for Graphical Models Robert Mateescu lectrical ngineering epartment alifornia
More informationAND/OR Cutset Conditioning
ND/OR utset onditioning Robert Mateescu and Rina Dechter School of Information and omputer Science University of alifornia, Irvine, 92697 {mateescu, dechter}@ics.uci.edu bstract utset conditioning is one
More informationThe Relationship Between AND/OR Search and Variable Elimination
The Relationship etween N/OR Search and Variable limination Robert Mateescu and Rina echter School of Information and omputer Science University of alifornia, Irvine, 926973425 {mateescu,dechter}@ics.uci.edu
More informationBoolean Representations and Combinatorial Equivalence
Chapter 2 Boolean Representations and Combinatorial Equivalence This chapter introduces different representations of Boolean functions. It then discusses the applications of these representations for proving
More informationBehavior models and verification Lecture 6
Behavior models and verification Lecture 6 http://d3s.mff.cuni.cz Jan Kofroň, František Plášil Model checking For a Kripke structure M = (S, I, R, L) over AP and a (state based) temporal logic formula
More informationSome Applications of Graph Bandwidth to Constraint Satisfaction Problems
Some Applications of Graph Bandwidth to Constraint Satisfaction Problems Ramin Zabih Computer Science Department Stanford University Stanford, California 94305 Abstract Bandwidth is a fundamental concept
More informationMixed deterministic and probabilistic networks
nn Math rtif Intell (28) 54:3 5 OI.7/s4729932y Mixed deterministic and probabilistic networks Robert Mateescu Rina echter Published online: 25 pril 29 Springer Science + usiness Media.V. 29 bstract
More informationConstraint Satisfaction Problems
Constraint Satisfaction Problems Search and Lookahead Bernhard Nebel, Julien Hué, and Stefan Wölfl AlbertLudwigsUniversität Freiburg June 4/6, 2012 Nebel, Hué and Wölfl (Universität Freiburg) Constraint
More informationBestFirst AND/OR Search for Most Probable Explanations
MARINSU & DHTR 259 estfirst / Search for Most Probable xplanations Radu Marinescu and Rina Dechter School of Information and omputer Science University of alifornia, Irvine, A 926973425 {radum,dechter}@ics.uci.edu
More informationTowards Parallel Search for Optimization in Graphical Models
Towards Parallel Search for Optimization in Graphical Models Lars Otten and Rina Dechter Bren School of Information and Computer Sciences University of California, Irvine {lotten,dechter}@ics.uci.edu Abstract
More informationThe Encoding Complexity of Network Coding
The Encoding Complexity of Network Coding Michael Langberg Alexander Sprintson Jehoshua Bruck California Institute of Technology Email: mikel,spalex,bruck @caltech.edu Abstract In the multicast network
More informationOn the SpaceTime Tradeoff in Solving Constraint Satisfaction Problems*
On the SpaceTime Tradeoff in Solving Constraint Satisfaction Problems* Roberto J. Bayardo Jr. and Daniel P. Miranker Department of Computer Sciences and Applied Research Laboratories The University of
More informationMaximal Prime Subgraph Decomposition of Bayesian Networks: A Relational Database Perspective
Maximal Prime ubgraph ecomposition of ayesian Networks: Relational atabase Perspective an Wu chool of omputer cience University of Windsor Windsor Ontario anada N9 3P4 Michael Wong epartment of omputer
More information3.1 Basic Definitions and Applications
Graphs hapter hapter Graphs. Basic efinitions and Applications Graph. G = (V, ) n V = nodes. n = edges between pairs of nodes. n aptures pairwise relationship between objects: Undirected graph represents
More informationCS301  Data Structures Glossary By
CS301  Data Structures Glossary By Abstract Data Type : A set of data values and associated operations that are precisely specified independent of any particular implementation. Also known as ADT Algorithm
More informationBayesian Networks and Decision Graphs
ayesian Networks and ecision raphs hapter 7 hapter 7 p. 1/27 Learning the structure of a ayesian network We have: complete database of cases over a set of variables. We want: ayesian network structure
More informationProcessing Probabilistic and Deterministic Graphical Models. Rina Dechter DRAFT
Processing Probabilistic and Deterministic Graphical Models Rina Dechter DRAFT May 1, 2013 c Rina Dechter 2 Contents 1 Introduction 7 1.1 Probabilistic vs Deterministic Models..................... 7 1.2
More informationModule 5 Graph Algorithms
Module 5 Graph lgorithms Dr. Natarajan Meghanathan Professor of Computer Science Jackson State University Jackson, MS 97 Email: natarajan.meghanathan@jsums.edu 5. Graph Traversal lgorithms Depth First
More informationThese notes present some properties of chordal graphs, a set of undirected graphs that are important for undirected graphical models.
Undirected Graphical Models: Chordal Graphs, Decomposable Graphs, Junction Trees, and Factorizations Peter Bartlett. October 2003. These notes present some properties of chordal graphs, a set of undirected
More informationSolving 3SAT. Radboud University Nijmegen. Bachelor Thesis. Supervisors: Henk Barendregt Alexandra Silva. Author: Peter Maandag s
Solving 3SAT Radboud University Nijmegen Bachelor Thesis Author: Peter Maandag s3047121 Supervisors: Henk Barendregt Alexandra Silva July 2, 2012 Contents 1 Introduction 2 1.1 Problem context............................
More informationBACKGROUND: A BRIEF INTRODUCTION TO GRAPH THEORY
BACKGROUND: A BRIEF INTRODUCTION TO GRAPH THEORY General definitions; Representations; Graph Traversals; Topological sort; Graphs definitions & representations Graph theory is a fundamental tool in sparse
More informationTCOM 501: Networking Theory & Fundamentals. Lecture 11 April 16, 2003 Prof. Yannis A. Korilis
TOM 50: Networking Theory & undamentals Lecture pril 6, 2003 Prof. Yannis. Korilis 2 Topics Routing in ata Network Graph Representation of a Network Undirected Graphs Spanning Trees and Minimum Weight
More informationAdvanced Algorithms Class Notes for Monday, October 23, 2012 Min Ye, Mingfu Shao, and Bernard Moret
Advanced Algorithms Class Notes for Monday, October 23, 2012 Min Ye, Mingfu Shao, and Bernard Moret Greedy Algorithms (continued) The best known application where the greedy algorithm is optimal is surely
More informationFMA901F: Machine Learning Lecture 6: Graphical Models. Cristian Sminchisescu
FMA901F: Machine Learning Lecture 6: Graphical Models Cristian Sminchisescu Graphical Models Provide a simple way to visualize the structure of a probabilistic model and can be used to design and motivate
More informationCHAPTER 5 GENERATING TEST SCENARIOS AND TEST CASES FROM AN EVENTFLOW MODEL
CHAPTER 5 GENERATING TEST SCENARIOS AND TEST CASES FROM AN EVENTFLOW MODEL 5.1 INTRODUCTION The survey presented in Chapter 1 has shown that Model based testing approach for automatic generation of test
More informationSTA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 5 Inference
More informationDefinition of Graphs and Trees. Representation of Trees.
Definition of Graphs and Trees. Representation of Trees. Chapter 6 Definition of graphs (I) A directed graph or digraph is a pair G = (V,E) s.t.: V is a finite set called the set of vertices of G. E V
More informationAdvances in Parallel Branch and Bound
ECAI 2012 Advances in Parallel Branch and Bound Lars Otten and Rina Dechter Dept. of Computer Science University of California, Irvine Summary Parallelizing AND/OR Branch and Bound:  Advanced optimization
More informationSDD AdvancedUser Manual Version 1.1
SDD AdvancedUser Manual Version 1.1 Arthur Choi and Adnan Darwiche Automated Reasoning Group Computer Science Department University of California, Los Angeles Email: sdd@cs.ucla.edu Download: http://reasoning.cs.ucla.edu/sdd
More informationArtificial Intelligence (part 4a) Problem Solving Using Search: Structures and Strategies for State Space Search
Artificial Intelligence (part 4a) Problem Solving Using Search: Structures and Strategies for State Space Search Course Contents Again..Selected topics for our course. Covering all of AI is impossible!
More information3 NoWait Job Shops with Variable Processing Times
3 NoWait Job Shops with Variable Processing Times In this chapter we assume that, on top of the classical nowait job shop setting, we are given a set of processing times for each operation. We may select
More informationOrganizing Spatial Data
Organizing Spatial Data Spatial data records include a sense of location as an attribute. Typically location is represented by coordinate data (in 2D or 3D). 1 If we are to search spatial data using the
More informationDistributed Algorithms 6.046J, Spring, Nancy Lynch
Distributed Algorithms 6.046J, Spring, 205 Nancy Lynch What are Distributed Algorithms? Algorithms that run on networked processors, or on multiprocessors that share memory. They solve many kinds of problems:
More informationCS 188: Artificial Intelligence Fall 2011
CS 188: Artificial Intelligence Fall 2011 Lecture 5: CSPs II 9/8/2011 Dan Klein UC Berkeley Multiple slides over the course adapted from either Stuart Russell or Andrew Moore 1 Today Efficient Solution
More informationAnytime AND/OR Depthfirst Search for Combinatorial Optimization
Anytime AND/OR Depthfirst Search for Combinatorial Optimization Lars Otten and Rina Dechter Dept. of Computer Science University of California, Irvine Outline AND/OR Search Spaces. Conflict: Decomposition
More information!!" '!" Fall 2003 ICS 275A  Constraint Networks 2
chapter 10 1 !"!!" # $ %&!" '!" Fall 2003 ICS 275A  Constraint Networks 2 ( Complexity : O(exp(n)) Fall 2003 ICS 275A  Constraint Networks 3 * bucket i = O(exp( w )) DR time and space : O( n exp( w *
More informationGenerating Random Solutions for Constraint Satisfaction Problems
From: I02 Proceedings. opyright 2002, I (www.aaai.org). ll rights reserved. Generating Random Solutions for onstraint Satisfaction Problems Rina echter Kalev Kask University of alifornia, Irvine {dechter,
More informationNPHardness. We start by defining types of problem, and then move on to defining the polynomialtime reductions.
CS 787: Advanced Algorithms NPHardness Instructor: Dieter van Melkebeek We review the concept of polynomialtime reductions, define various classes of problems including NPcomplete, and show that 3SAT
More information15.4 Longest common subsequence
15.4 Longest common subsequence Biological applications often need to compare the DNA of two (or more) different organisms A strand of DNA consists of a string of molecules called bases, where the possible
More informationSolution for Homework set 3
TTIC 300 and CMSC 37000 Algorithms Winter 07 Solution for Homework set 3 Question (0 points) We are given a directed graph G = (V, E), with two special vertices s and t, and nonnegative integral capacities
More informationThe Data Locality of Work Stealing
The Data Locality of Work Stealing Umut A. Acar School of Computer Science Carnegie Mellon University umut@cs.cmu.edu Guy E. Blelloch School of Computer Science Carnegie Mellon University guyb@cs.cmu.edu
More informationGraph Theory. Probabilistic Graphical Models. L. Enrique Sucar, INAOE. Definitions. Types of Graphs. Trajectories and Circuits.
Theory Probabilistic ical Models L. Enrique Sucar, INAOE and (INAOE) 1 / 32 Outline and 1 2 3 4 5 6 7 8 and 9 (INAOE) 2 / 32 A graph provides a compact way to represent binary relations between a set of
More informationPart II. C. M. Bishop PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 8: GRAPHICAL MODELS
Part II C. M. Bishop PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 8: GRAPHICAL MODELS Converting Directed to Undirected Graphs (1) Converting Directed to Undirected Graphs (2) Add extra links between
More informationECE 5775 (Fall 17) HighLevel Digital Design Automation. Binary Decision Diagrams Static Timing Analysis
ECE 5775 (Fall 17) HighLevel Digital Design Automation Binary Decision Diagrams Static Timing Analysis Announcements Start early on Lab 1 (CORDIC design) Fixedpoint design should not have usage of DSP48s
More informationStackless BVH Collision Detection for Physical Simulation
Stackless BVH Collision Detection for Physical Simulation Jesper Damkjær Department of Computer Science, University of Copenhagen Universitetsparken 1, DK2100, Copenhagen Ø, Denmark damkjaer@diku.dk February
More information6. Lecture notes on matroid intersection
Massachusetts Institute of Technology 18.453: Combinatorial Optimization Michel X. Goemans May 2, 2017 6. Lecture notes on matroid intersection One nice feature about matroids is that a simple greedy algorithm
More informationMaximal Monochromatic Geodesics in an Antipodal Coloring of Hypercube
Maximal Monochromatic Geodesics in an Antipodal Coloring of Hypercube Kavish Gandhi April 4, 2015 Abstract A geodesic in the hypercube is the shortest possible path between two vertices. Leader and Long
More informationSymbolic Manipulation of Boolean Functions Using a Graphical Representation. Abstract
Symbolic Manipulation of Boolean Functions Using a Graphical Representation Randal E. Bryant 1 Dept. of Computer Science CarnegieMellon University Abstract In this paper we describe a data structure for
More informationModeling and Reasoning with Bayesian Networks. Adnan Darwiche University of California Los Angeles, CA
Modeling and Reasoning with Bayesian Networks Adnan Darwiche University of California Los Angeles, CA darwiche@cs.ucla.edu June 24, 2008 Contents Preface 1 1 Introduction 1 1.1 Automated Reasoning........................
More informationAv. Prof. Mello Moraes, 2231, , São Paulo, SP  Brazil
" Generalizing Variable Elimination in Bayesian Networks FABIO GAGLIARDI COZMAN Escola Politécnica, University of São Paulo Av Prof Mello Moraes, 31, 05508900, São Paulo, SP  Brazil fgcozman@uspbr Abstract
More informationDiscrete Optimization with Decision Diagrams
Discrete Optimization with Decision Diagrams J. N. Hooker Joint work with David Bergman, André Ciré, Willem van Hoeve Carnegie Mellon University Australian OR Society, May 2014 Goal Find an alternative
More informationLOGIC SYNTHESIS AND VERIFICATION ALGORITHMS. Gary D. Hachtel University of Colorado. Fabio Somenzi University of Colorado.
LOGIC SYNTHESIS AND VERIFICATION ALGORITHMS by Gary D. Hachtel University of Colorado Fabio Somenzi University of Colorado Springer Contents I Introduction 1 1 Introduction 5 1.1 VLSI: Opportunity and
More informationAlgorithms for Finding Dominators in Directed Graphs
Department of Computer Science Aarhus University Master s Thesis Algorithms for Finding Dominators in Directed Graphs Author: Henrik Knakkegaard Christensen 20082178 Supervisor: Gerth Støling Brodal January
More informationTerm Algebras with Length Function and Bounded Quantifier Elimination
with Length Function and Bounded Ting Zhang, Henny B Sipma, Zohar Manna Stanford University tingz,sipma,zm@csstanfordedu STeP Group, September 3, 2004 TPHOLs 2004  p 1/37 Motivation: Program Verification
More informationAdvanced Database Systems
Lecture IV Query Processing Kyumars Sheykh Esmaili Basic Steps in Query Processing 2 Query Optimization Many equivalent execution plans Choosing the best one Based on Heuristics, Cost Will be discussed
More informationarxiv: v2 [cs.ds] 18 May 2015
Optimal Shuffle Code with Permutation Instructions Sebastian Buchwald, Manuel Mohr, and Ignaz Rutter Karlsruhe Institute of Technology {sebastian.buchwald, manuel.mohr, rutter}@kit.edu arxiv:1504.07073v2
More informationChapter 3. Set Theory. 3.1 What is a Set?
Chapter 3 Set Theory 3.1 What is a Set? A set is a welldefined collection of objects called elements or members of the set. Here, welldefined means accurately and unambiguously stated or described. Any
More informationCopyright 2000, Kevin Wayne 1
Chapter 3  Graphs Undirected Graphs Undirected graph. G = (V, E) V = nodes. E = edges between pairs of nodes. Captures pairwise relationship between objects. Graph size parameters: n = V, m = E. Directed
More informationAbout the Tutorial. Audience. Prerequisites. Copyright & Disclaimer. Compiler Design
i About the Tutorial A compiler translates the codes written in one language to some other language without changing the meaning of the program. It is also expected that a compiler should make the target
More informationRelations and Graphs
s and are Pictures of (Binary) s E. Wenderholm Department of Computer Science SUNY Oswego c 2016 Elaine Wenderholm All rights Reserved Outline 1 A Function that returns a boolean Special Properties of
More informationEECS 219C: Formal Methods Binary Decision Diagrams (BDDs) Sanjit A. Seshia EECS, UC Berkeley
EECS 219C: Formal Methods Binary Decision Diagrams (BDDs) Sanjit A. Seshia EECS, UC Berkeley Boolean Function Representations Syntactic: e.g.: CNF, DNF (SOP), Circuit Semantic: e.g.: Truth table, Binary
More information12/5/17. trees. CS 220: Discrete Structures and their Applications. Trees Chapter 11 in zybooks. rooted trees. rooted trees
trees CS 220: Discrete Structures and their Applications A tree is an undirected graph that is connected and has no cycles. Trees Chapter 11 in zybooks rooted trees Rooted trees. Given a tree T, choose
More informationGraphBased Algorithms for Boolean Function Manipulation
GraphBased Algorithms for Boolean Function Manipulation Introduction Boolean algebra forms a corner stone of Computer system & Digital design. So, we need efficient algorithms to represent & manipulate
More informationA New Approach to Graph Recognition and Applications to DistanceHereditary Graphs
Nakano Si, Uehara R, Uno T. A new approach to graph recognition and applications to distancehereditary graphs. JOUR NAL OF COMPUTER SCIENCE AND TECHNOLOGY 24(3): 517 533 May 2009 A New Approach to Graph
More informationQuery Optimization. Query Optimization. Optimization considerations. Example. Interaction of algorithm choice and tree arrangement.
COS 597: Principles of Database and Information Systems Query Optimization Query Optimization Query as expression over relational algebraic operations Get evaluation (parse) tree Leaves: base relations
More informationScribes: Romil Verma, Juliana Cook (2015), Virginia Williams, Date: May 1, 2017 and Seth HildickSmith (2016), G. Valiant (2017), M.
Lecture 9 Graphs Scribes: Romil Verma, Juliana Cook (2015), Virginia Williams, Date: May 1, 2017 and Seth HildickSmith (2016), G. Valiant (2017), M. Wootters (2017) 1 Graphs A graph is a set of vertices
More informationDiscrete Mathematics and Probability Theory Fall 2009 Satish Rao,David Tse Note 8
CS 70 Discrete Mathematics and Probability Theory Fall 2009 Satish Rao,David Tse Note 8 An Introduction to Graphs Formulating a simple, precise specification of a computational problem is often a prerequisite
More information15.4 Longest common subsequence
15.4 Longest common subsequence Biological applications often need to compare the DNA of two (or more) different organisms A strand of DNA consists of a string of molecules called bases, where the possible
More informationThe Structure of BullFree Perfect Graphs
The Structure of BullFree Perfect Graphs Maria Chudnovsky and Irena Penev Columbia University, New York, NY 10027 USA May 18, 2012 Abstract The bull is a graph consisting of a triangle and two vertexdisjoint
More informationSemantic Subtyping. Alain Frisch (ENS Paris) Giuseppe Castagna (ENS Paris) Véronique Benzaken (LRI U Paris Sud)
Semantic Subtyping Alain Frisch (ENS Paris) Giuseppe Castagna (ENS Paris) Véronique Benzaken (LRI U Paris Sud) http://www.cduce.org/ Semantic Subtyping  Groupe de travail BD LRI p.1/28 CDuce A functional
More informationConstraint Satisfaction. AI Slides (5e) c Lin
Constraint Satisfaction 4 AI Slides (5e) c Lin Zuoquan@PKU 20032018 4 1 4 Constraint Satisfaction 4.1 Constraint satisfaction problems 4.2 Backtracking search 4.3 Constraint propagation 4.4 Local search
More informationAnytime Anyspace AND/OR Bestfirst Search for Bounding Marginal MAP
Anytime Anyspace AN/OR Bestfirst Search for Bounding Marginal MAP Qi Lou University of alifornia, Irvine Irvine, A 92697, USA qlou@ics.uci.edu Rina echter University of alifornia, Irvine Irvine, A 92697,
More informationLecture1: Symbolic Model Checking with BDDs. Edmund M. Clarke, Jr. Computer Science Department Carnegie Mellon University Pittsburgh, PA 15213
Lecture: Symbolic Model Checking with BDDs Edmund M Clarke, Jr Computer Science Department Carnegie Mellon University Pittsburgh, PA 523 Temporal Logic Model Checking Specification Language: A propositional
More informationP Is Not Equal to NP. ScholarlyCommons. University of Pennsylvania. Jon Freeman University of Pennsylvania. October 1989
University of Pennsylvania ScholarlyCommons Technical Reports (CIS) Department of Computer & Information Science October 1989 P Is Not Equal to NP Jon Freeman University of Pennsylvania Follow this and
More informationTHE BRAID INDEX OF ALTERNATING LINKS
THE BRAID INDEX OF ALTERNATING LINKS YUANAN DIAO, GÁBOR HETYEI AND PENGYU LIU Abstract. It is well known that the minimum crossing number of an alternating link equals the number of crossings in any reduced
More informationMULTINODE STATIC LOGIC IMPLICATIONS FOR REDUNDANCY IDENTIFICATION
MULTINODE STTI LOGI IMPLITIONS FOR REDUNDNY IDENTIFITION Kabir Gulrajani and Michael S. Hsiao Intel orporation, Dupont, W Department of Electrical and omputer Engineering, Rutgers University, Piscataway,
More informationDatabase System Concepts
Chapter 13: Query Processing s Departamento de Engenharia Informática Instituto Superior Técnico 1 st Semester 2008/2009 Slides (fortemente) baseados nos slides oficiais do livro c Silberschatz, Korth
More informationContextFree Grammars and Parsers. Peter S. Housel January 2001
ContextFree Grammars and Parsers Peter S. Housel January 2001 Copyright This is the Monday grammar library, a set of facilities for representing contextfree grammars and dynamically creating parser automata
More informationChapter 13: Query Processing
Chapter 13: Query Processing! Overview! Measures of Query Cost! Selection Operation! Sorting! Join Operation! Other Operations! Evaluation of Expressions 13.1 Basic Steps in Query Processing 1. Parsing
More informationGraph Representations and Traversal
COMPSCI 330: Design and Analysis of Algorithms 02/08/201702/20/2017 Graph Representations and Traversal Lecturer: Debmalya Panigrahi Scribe: Tianqi Song, Fred Zhang, Tianyu Wang 1 Overview This lecture
More information4.1 Review  the DPLL procedure
Applied Logic Lecture 4: Efficient SAT solving CS 4860 Spring 2009 Thursday, January 29, 2009 The main purpose of these notes is to help me organize the material that I used to teach today s lecture. They
More informationUNIT IV NONLINEAR DATA STRUCTURES 4.1 Trees TREE: A tree is a finite set of one or more nodes such that there is a specially designated node called the Root, and zero or more non empty sub trees T1,
More informationReducing Directed Max Flow to Undirected Max Flow and Bipartite Matching
Reducing Directed Max Flow to Undirected Max Flow and Bipartite Matching Henry Lin Division of Computer Science University of California, Berkeley Berkeley, CA 94720 Email: henrylin@eecs.berkeley.edu Abstract
More information6.001 Notes: Section 31.1
6.001 Notes: Section 31.1 Slide 31.1.1 In previous lectures we have seen a number of important themes, which relate to designing code for complex systems. One was the idea of proof by induction, meaning
More informationGraph Algorithms Using Depth First Search
Graph Algorithms Using Depth First Search Analysis of Algorithms Week 8, Lecture 1 Prepared by John Reif, Ph.D. Distinguished Professor of Computer Science Duke University Graph Algorithms Using Depth
More informationIntroduction to Computers and Programming. Concept Question
Introduction to Computers and Programming Prof. I. K. Lundqvist Lecture 7 April 2 2004 Concept Question G1(V1,E1) A graph G(V, where E) is V1 a finite = {}, nonempty E1 = {} set of G2(V2,E2) vertices and
More informationA Semantics to Generate the Contextsensitive Synchronized ControlFlow Graph (extended)
A Semantics to Generate the Contextsensitive Synchronized ControlFlow Graph (extended) Marisa Llorens, Javier Oliver, Josep Silva, and Salvador Tamarit Universidad Politécnica de Valencia, Camino de
More informationSimpler, Lineartime Transitive Orientation via Lexicographic BreadthFirst Search
Simpler, Lineartime Transitive Orientation via Lexicographic BreadthFirst Search Marc Tedder University of Toronto arxiv:1503.02773v1 [cs.ds] 10 Mar 2015 Abstract Comparability graphs are the undirected
More informationFrom Exact to Anytime Solutions for Marginal MAP
From Exact to Anytime Solutions for Marginal MAP Junkyu Lee University of California, Irvine Radu Marinescu IBM Research Ireland Rina Dechter University of California, Irvine Irvine, CA 92697, USA radu.marinescu@ie.ibm.com
More informationCSI 604 Elementary Graph Algorithms
CSI 604 Elementary Graph Algorithms Ref: Chapter 22 of the text by Cormen et al. (Second edition) 1 / 25 Graphs: Basic Definitions Undirected Graph G(V, E): V is set of nodes (or vertices) and E is the
More informationConstraint Satisfaction Problems
Constraint Satisfaction Problems Berlin Chen Department of Computer Science & Information Engineering National Taiwan Normal University References: 1. S. Russell and P. Norvig. Artificial Intelligence:
More informationDepartment of Computer Science and Technology
UNIT : Stack & Queue Short Questions 1 1 1 1 1 1 1 1 20) 2 What is the difference between Data and Information? Define Data, Information, and Data Structure. List the primitive data structure. List the
More informationToday. CS 188: Artificial Intelligence Fall Example: Boolean Satisfiability. Reminder: CSPs. Example: 3SAT. CSPs: Queries.
CS 188: Artificial Intelligence Fall 2007 Lecture 5: CSPs II 9/11/2007 More CSPs Applications Tree Algorithms Cutset Conditioning Today Dan Klein UC Berkeley Many slides over the course adapted from either
More informationSolutions. Suppose we insert all elements of U into the table, and let n(b) be the number of elements of U that hash to bucket b. Then.
Assignment 3 1. Exercise [11.23 on p. 229] Modify hashing by chaining (i.e., bucketvector with BucketType = List) so that BucketType = OrderedList. How is the runtime of search, insert, and remove affected?
More informationConstraint Satisfaction Problems
Constraint Satisfaction Problems LookBack Malte Helmert and Stefan Wölfl AlbertLudwigsUniversität Freiburg June 5, 2007 S. Wölfl, M. Helmert (Universität Freiburg) Constraint Satisfaction Problems June
More informationConstraint Satisfaction Problems. slides from: Padhraic Smyth, Bryan Low, S. Russell and P. Norvig, JeanClaude Latombe
Constraint Satisfaction Problems slides from: Padhraic Smyth, Bryan Low, S. Russell and P. Norvig, JeanClaude Latombe Standard search problems: State is a black box : arbitrary data structure Goal test
More informationHardness of Subgraph and Supergraph Problems in ctournaments
Hardness of Subgraph and Supergraph Problems in ctournaments Kanthi K Sarpatwar 1 and N.S. Narayanaswamy 1 Department of Computer Science and Engineering, IIT madras, Chennai 600036, India kanthik@gmail.com,swamy@cse.iitm.ac.in
More informationChapter 12: Indexing and Hashing
Chapter 12: Indexing and Hashing Basic Concepts Ordered Indices B+Tree Index Files BTree Index Files Static Hashing Dynamic Hashing Comparison of Ordered Indexing and Hashing Index Definition in SQL
More informationCS 161 Lecture 11 BFS, Dijkstra s algorithm Jessica Su (some parts copied from CLRS) 1 Review
1 Review 1 Something I did not emphasize enough last time is that during the execution of depthfirstsearch, we construct depthfirstsearch trees. One graph may have multiple depthfirstsearch trees,
More information15451/651: Design & Analysis of Algorithms October 5, 2017 Lecture #11: Depth First Search and Strong Components last changed: October 17, 2017
15451/651: Design & Analysis of Algorithms October 5, 2017 Lecture #11: Depth First Search and Strong Components last changed: October 17, 2017 1 Introduction Depth first search is a very useful technique
More information