Question. Lecture 11: CSPs I. Course plan. Review: paradigm

Transcription

1 Lecture 11: CSPs I cs221.stanford.edu/q uestion Find two neighboring countries, one that begins with an A and the other that speaks Hungarian. CS221 / Autumn 2016 / Liang CS221 / Autumn 2016 / Liang 1 Course plan We ve finished our tour of machine learning and state-based models, which brings us to the midpoint of this course. Let s reflect a bit on what we ve covered so far. Search problems Markov decision processes Constraint satisfaction problems Adversarial games Bayesian networks Reflex States ariables Logic Low-level intelligence Machine learning High-level intelligence CS221 / Autumn 2016 / Liang 2 Review: paradigm Recall the paradigm that we ve been working in: separate modeling (what to compute) from algorithms (how to compute it). Real-world task Modeling Formal task (model) Algorithms Program CS221 / Autumn 2016 / Liang 4

2 State-based models [Modeling] Framework search problems MDPs/games Objective minimum cost paths maximum value policies [Algorithms] Modeling: In the context of state-based models, we seek to find minimum cost paths (for search problems) or maximum value policies (for MDPs and games). Algorithms: o compute these solutions, we can either work on the search/game tree or on the state graph. In the former case, we end up with recursive procedures which take exponential time but require very little memory (generally linear in the size of the solution). In the latter case, where we are fortunate to have few enough states to fit into memory, we can work directly on the graph, which can often yield an exponential savings in time. Given that we can find the optimal solution with respect to a fixed model, the final question is where this model actually comes from. Learning provides the answer: from data. You should think of machine learning as not just a way to do binary classification, but more as a way of life, which can be used to support a variety of different models. In the rest of the course, modeling, algorithms, and learning will continue to be the three structural supports of the techniques we will develop. ree-based backtracking minimax/expectimax Graph-based DP, UCS, A* value/policy iteration [Learning] Methods structured Perceptron -learning, D learning CS221 / Autumn 2016 / Liang 6 State-based models: takeaway 1 B S G A One high-level takeaway is the motto: specify locally, optimize globally. When we re building a search problem, we only need to specify how the states are connected through actions and what the local action costs are; we need not specify the long-term consequences of taking an action. It is the job of the algorithms to take all of this local information into account and produce globally optimal solutions (minimum cost paths). his separation is quite powerful in light of modeling and algorithms: having to worry only about local interactions makes modeling easier, but we still get the benefits of a globally optimal solution via algorithms which are constructed independent of the domain-specific details. We will see this local specification + global optimization pattern again in the context of variable-based models. Key idea: specify locally, optimize globally Modeling: specifies local interactions Algorithms: find globally optimal solutions CS221 / Autumn 2016 / Liang 8 State-based models: takeaway 2 B S G he second high-level takeaway which is core to state-based models is the notion of state. he state, which summarizes previous actions, is one of the key tools that allows us to manage the exponential search problems frequently encountered in AI. We will see the notion of state appear again in the context of conditional independence in variable-based models. With states, we were in the mindset of thinking about taking a sequence of actions (where order is important) to reach a goal. However, in some tasks, order is irrelevant. In these cases, maybe search isn t the best way to model the task. Let s see an example. A Key idea: state A state is a summary of all the past actions sufficient to choose future actions optimally. Mindset: move through states (nodes) via actions (edges) CS221 / Autumn 2016 / Liang 10

3 Map coloring uestion: how can we color each of the 7 provinces {red,green,blue} so that no two neighboring provinces have the same color? (one possible solution) CS221 / Autumn 2016 / Liang 12 CS221 / Autumn 2016 / Liang 13 Search,,,,,, As a search problem State: partial assignment of colors to provinces Action: assign next uncolored province a compatible color What s next? Exploit the problem structure! ariable ordering doesn t affect correctness ariables are interdependent in a local way CS221 / Autumn 2016 / Liang 14 CS221 / Autumn 2016 / Liang 15 We can certainly use search to find an assignment of colors to the provinces of Australia. Let s fix an arbitrary ordering of the provinces. Each state contains an assignment of colors to a subset of the provinces (a partial assignment), and each action chooses a color for the next unassigned province as long as the color isn t already assigned to one of its neighbors. In this way, all the leaves of the search tree are solutions (18 of them). (In the slide, in the interest of space, we ve only shown the subtree rooted at a partial assignment to 3 variables.) his is a fine way to solve this problem, and in general, it shows how powerful search is: we don t actually need any new machinery to solve this problem. But the question is: can we do better? First, the particular search tree that we drew had several dead ends; it would be better if we could detect these earlier. We will see in this lecture that the fact that the order in which we assign variables doesn t matter for correctness gives us the flexibility to dynamically choose a better ordering of the variables. hat, with a bit of lookahead will allow us to dramatically improve the efficiency over naive tree search. Second, it s clear that asmania s color can be any of the three colors regardless of the colors on the mainland. his is an instance of independence, and next time, we ll see how to exploit these observations systematically. A new framework... Key idea: variables ariable-based models Solutions to problems assignments to variables (modeling). Decisions about variable ordering, etc. chosen by algorithms. CS221 / Autumn 2016 / Liang 17

4 With that motivation in mind, we now embark on our journey into variable-based models. ariablebased models is an umbrella term that includes constraint satisfaction problems (CSPs), Markov networks, Bayesian networks, hidden Markov models (HMMs), conditional random fields (CRFs), etc., which we ll get to later in the course. he term graphical models can be used interchangeably with variable-based models, and the term probabilistic graphical models (PGMs) generally encompasses both Markov networks (also called undirected graphical models) and Bayesian networks (directed graphical models). he unifying theme is the idea of thinking about solutions to problems as assignments of values to variables (this is the modeling part). All the details about how to find the assignment (in particular, which variables to try first) are delegated to the algorithms. So the advantage of using variable-based models over statebased models is that it s making the algorithms do more of the work, freeing up more time for modeling. State-based models: An analogy ariable-based models: CS221 / Autumn 2016 / Liang 19 An apt analogy are programming languages. Solving a problem directly by implementing an ad-hoc program is like using assembly language. Solving a problem using state-based models is like using C. Solving a problem using variable-based models is like using Python. By moving to a higher language, you might forgo some amount of ability to optimize manually, but the advantage is that (i) you can think at a higher level and (ii) there are more opportunities for optimizing automatically. How to describe your solution uestion: How are you solving the problem? Bad answer: I m using MDPs. Good answer: he states capture position and velocity. CS221 / Autumn 2016 / Liang 21 While we re on the topic of programming languages and tools, it s worth making one final general remark: What is the best way to describe a solution (e.g., in writing up your final project)? It s not very informative to simply say that you used SMs, because you could have used SMs in a thousand different ways, just like you could have used C++ in a thousand different ways. SMs, MDPs, and C++ are just tools and frameworks that help you get the job done, but should only play a supporting role in your description. It is much more useful to describe the model that you used to represent your real-world task. What are the features trying to capture about the problem (in the case of machine learning)? What are the states, actions, costs, etc. capturing (in the case of state-based models)? Once the tools (MDPs, etc.) become second nature, it is almost as if they are invisible. It s like when you master a language, you can think in it without constantly referring to the framework. Roadmap Factor graphs Dynamic ordering Arc consistency Modeling CS221 / Autumn 2016 / Liang 23

5 B or R? Factor graph (example) must agree B or R? tend to agree X 1 X 2 X 3 B or R? he most important concept for the next three weeks will be that of a factor graph. But before we define it formally, let us consider a simple example. Suppose there are three people, each of which will vote for a color, red or blue. We know that Person 1 is leaning pretty set on blue, and Person 3 is leaning red. Person 1 and Person 2 must have the same color, while Person 2 and Person 3 would weakly prefer to have the same color. We can model this as a factor graph consisting of three variables, X 1, X 2, X 3, each of which must be assigned red (R) or blue (B). We encode each of the constraint/preference as a factor, which assigns a non-negative number based on the assignment to a subset of the variables. We can either describe the factor as an explicit table, or via a function (e.g., [x 1 = x 2]). f 1 f 2 f 3 f 4 x 1 f 1(x 1) R 0 B 1 x 1 x 2 f 2(x 1, x 2) R R 1 R B 0 B R 0 B B 1 x 2 x 3 f 3(x 2, x 3) R R 3 R B 2 B R 2 B B 3 x 3 f 4(x 3) R 2 B 1 f 2(x 1, x 2) = [x 1 = x 2] f 3(x 2, x 3) = [x 2 = x 3] + 2 [demo] CS221 / Autumn 2016 / Liang 24 Factor graph X 1 X 2 X 3 Now we proceed to the general definition. a factor graph consists of a set of variables and a set of factors: (i) n variables X 1,..., X n, which are represented as circular nodes in the graphical notation; and (ii) m factors (also known as potentials) f 1,..., f m, which are represented as square nodes in the graphical notation. Each variable X i can take on values in its domain Domain i. Each factor f j is a function that takes an assignment x to all the variables and returns a non-negative number representing how good that assignment is (from the factor s point of view). Usually, each factor will depend only on a small subset of the variables. f 1 f 2 f 3 f 4 Definition: factor graph ariables: X = (X 1,..., X n ), where X i Domain i Factors (potentials): f 1,..., f m, with each f j (X) 0 CS221 / Autumn 2016 / Liang 26 Example: map coloring Notation: we use [condition] to represent the indicator function which is equal to 1 if the condition is true and 0 if not. Normally, this is written 1[condition], but we drop the 1 for succinctness. ariables: X = (,,,,,, ) Domain i {R, G, B} Factors: f 1 (X) = [ ] f 2 (X) = [ ]... CS221 / Autumn 2016 / Liang 28

6 Factors he key aspect that makes factor graphs useful is that each factor f j only depends on a subset of variables, called the scope. he arity of the factors is generally small (think 1 or 2). Definition: scope and arity Scope of a factor f j : set of variables it depends on. Arity of f j is the number of variables in the scope. Unary factors (arity 1); Binary factors (arity 2). Example: map coloring Scope of f 1 (X) = [ ] is {, } f 1 is a binary factor CS221 / Autumn 2016 / Liang 30 Assignment weights (example) x 1 x 2 f 2(x 1, x 2) x 2 x 3 f 3(x 2, x 3) x 1 f 1(x 1) R R 1 R R 3 x 3 f 4(x 3) R 0 R B 0 R B 2 R 2 B 1 B R 0 B R 2 B 1 B B 1 B B 3 A factor graph specifies all the local interactions between variables. We wish to find a global solution. A solution is called an assignment, which specifies a value for each variable. Each assignment is associated with a weight, which is just the product over each factor evaluated on that assignment. Intuitively, each factor contributes to the weight. Note that any factor has veto power: if it returns zero, then the entire weight is irrecoverably zero. In this setting, the maximum weight assignment is (B, B, R), which has a weight of 4. You can think of this is the optimal configuration or the most likely outcome. x 1 x 2 x 3 Weight R R R = 0 R R B = 0 R B R = 0 R B B = 0 B R R = 0 B R B = 0 B B R = 4 B B B = 3 CS221 / Autumn 2016 / Liang 32 [demo] Assignment weights Definition: assignment weight Formally, the weight of an assignment x is the product of all the factors applied to that assignment ( m j=1 fj(x)). hink of all the factors chiming in on their opinion of x. We multiply all these opinions together to get the global opinion. Our objective will be to find the maximum weight assignment. Note: do not confuse the term weight in the context of factor graphs with the weight vector in machine learning. Each assignment x = (x 1,..., x n ) has a weight: m Weight(x) = f j (x) j=1 Objective: find the maximum weight assignment arg max Weight(x) x CS221 / Autumn 2016 / Liang 34

7 Example: map coloring In the map coloring example, each factor only looks at the variables of two adjacent countries and checks if the colors are different (returning 1) or the same (returning 0). From a modeling perspective, this allows us to specify local interactions in a modular way. A global notion of consistency is achieved by multiplying together all the factors. Again note that the factors are multiplied (not added), which means that any factor has veto power: a single zero causes the entire weight to be zero. Assignment: x = { : R, : G, : B, : R, : G, : R, : G} Weight: Weight(x) = = 1 Assignment: x = { : R, : R, : B, : R, : G, : R, : G} Weight: Weight(x ) = = 0 CS221 / Autumn 2016 / Liang 36 Constraint satisfaction problems Definition: constraint satisfaction problem (CSP) Constraint satisfaction problems are just a special case of factor graphs where each of the factors returns either 0 or 1. Such a factor is a constraint, where 1 means the constraint is satisfied and 0 means that it is not. In a CSP, all assignments have either weight 1 or 0. Assignments with weight 1 are called consistent (they satisfy all the constraints), and the assignments with weight 0 are called inconsistent. Our goal is to find any consistent assignment (if one exists). A CSP is a factor graph where all factors are constraints: f j (x) {0, 1} for all j = 1,..., m he constraint is satisfied iff f j (x) = 1. Definition: consistent assignments An assignment x is consistent iff Weight(x) = 1 (i.e., all constraints are satisfied). CS221 / Autumn 2016 / Liang 38 Summary so far Roadmap X 1 X 2 X 3 Factor graphs f 1 f 2 f 3 f 4 Dynamic ordering Factor graph (general) variables factors assignment weight CSP (all or nothing) variables constraints consistent or inconsistent Arc consistency Modeling CS221 / Autumn 2016 / Liang 40 CS221 / Autumn 2016 / Liang 41

8 Extending partial assignments he general idea, as we ve already seen in our search-based solution is to work with partial assignments. We ve defined the weight of a full assignment to be the product of all the factors applied to that assignment. We extend this definition to partial assignments: he weight of a partial assignment is defined to be the product of all the factors whose scope includes only assigned variables. For example, if only and are assigned, the weight is just value of the single factor between them. When we assign a new variable a value, the weight of the new extended assignment is defined to be the original weight times all the factors that depend on the new variable and only previously assigned variables. [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] CS221 / Autumn 2016 / Liang 42 Dependent factors Formally, we will use D(x, X i) to denote this set of these factors, which we will call dependent factors. For example, if we assign, then D(x, ) contains two factors: the one between and and the one between and. Partial assignment (e.g., x = { : R, : G}) Definition: dependent factors Let D(x, X i ) be set of factors depending on X i but not on unassigned variables. D({ : R, : G}, ) = {[ ], [ ]} CS221 / Autumn 2016 / Liang 44 Backtracking search Algorithm: backtracking search Backtrack(x, w, Domains): If x is complete assignment: update best and return Choose unassigned ARIABLE X i Order ALUES Domain i of chosen X i For each value v in that order: δ f j (x {X i : v}) f j D(x,X i) If δ = 0: continue Domains Domains via LOOKAHEAD Backtrack(x {X i : v}, wδ, Domains ) Now we are ready to present the full backtracking search, which is a recursive procedure that takes in a partial assignment x, its weight w, and the domains of all the variables Domains = (Domain 1,..., Domain n). If the assignment x is complete (all variables are assigned), then we update our statistics based on what we re trying to compute: We can increment the total number of assignments seen so far, check to see if x is better than the current best assignment that we ve seen so far (based on w), etc. (For CSPs where all the weights are 0 or 1, we can stop as soon as we find one consistent assignment, just as in DFS for search problems.) Otherwise, we choose an unassigned variable X i. Given the choice of X i, we choose an ordering of the values of that variable X i. Next, we iterate through all the values v Domain i in that order. For each value v, we compute δ, which is the product of the dependent factors D(x, X i); recall this is the multiplicative change in weight from assignment x to the new assignment x {X i : v}. If δ = 0, that means a constraint is violated, and we can ignore this partial assignment completely, because multiplying more factors later on cannot make the weight non-zero. We then perform lookahead, removing values from the domains Domains to produce Domains. his is not required (we can just use Domains = Domains), but it can make our algorithm run faster. (We ll see one type of lookahead in the next slide.) Finally, we recurse on the new partial assignment x {X i : v}, the new weight wδ, and the new domain Domains. If we choose an unassigned variable according to an arbitrary fixed ordering, order the values arbitrarily, and do not perform lookahead, we get the basic tree search algorithm that we would have used if we were thinking in terms of a search problem. We will next start to improve the efficiency by exploiting properties of the CSP. CS221 / Autumn 2016 / Liang 46

9 Lookahead: forward checking (example) First, we will look at forward checking, which is a way to perform a one-step lookahead. he idea is that as soon as we assign a variable (e.g., = R), we can pre-emptively remove inconsistent values from the domains of neighboring variables (i.e., those that share a factor). If we keep on doing this and get to a point where some variable has an empty domain, then we can stop and backtrack immediately, since there s no possible way to assign a value to that variable which is consistent with the previous partial assignment. In this example, after is assigned blue, we remove inconsistent values (blue) from s domain, emptying it. At this point, we need not even recurse further, since there s no way to extend the current assignment. We would then instead try assigning to red. Inconsistent - prune! CS221 / Autumn 2016 / Liang 48 Lookahead: forward checking Key idea: forward checking (one-step lookahead) After assigning a variable X i, eliminate inconsistent values from the domains of X i s neighbors. If any domain becomes empty, don t recurse. When unassign X i, restore neighbors domains. When unassigning a variable, remember to restore the domains of its neighboring variables! he simplest way to implement this is to make a copy of the domains of the variables before performing forward checking. his is foolproof, but can be quite slow. A fancier solution is to keep a counter (initialized to be zero) c iv for each variable X i and value v in its domain. When we remove a value v from the domain of X i, we increment c iv. An element is deemed to be removed when c iv > 0. When we want to un-remove a value, we decrement c iv. his way, the remove operation is reversible, which is important since a value might get removed multiple times due to multiple neighboring variables. Later, we will look at arc consistency, which will allow us to lookahead even more. CS221 / Autumn 2016 / Liang 50 Choosing an unassigned variable Now let us look at the problem of choosing an unassigned variable. Intuitively, we want to choose the variable which is most constrained, that is, the variable whose domain has the fewest number of remaining valid values (based on forward checking), because those variables yield smaller branching factors. Which variable to assign next? Key idea: most constrained variable Choose variable that has the fewest consistent values. his example: (has only one value) CS221 / Autumn 2016 / Liang 52

10 Order values of a selected variable What values to try for? Once we ve selected an unassigned variable X i, we need to figure out which order to try the different values in. Here the principle we will follow is to first try values which are less constrained. here are several ways we can think about measuring how constrained a variable is, but for the sake of concreteness, here is the heuristic we ll use: just count the number of values in the domains of all neighboring variables (those that share a factor with X i). If we color red, then we have 2 valid values for, 2 for, and 2 for. If we color blue, then we have only 1 for, 1 for, and 2 for. herefore, red is preferable (6 total valid values versus 4). he intuition is that we want values which impose the fewest number of constraints on the neighbors, so that we are more likely to find a consistent assignment = 6 consistent values = 4 consistent values Key idea: least constrained value Order values of selected X i by decreasing number of consistent values of neighboring variables. CS221 / Autumn 2016 / Liang 54 When to fail? Most constrained variable (MC): Must assign every variable If going to fail, fail early more pruning he most constrained variable and the least constrained value heuristics might seem conflicting, but there is a good reason for this superficial difference. An assignment involves every variable whereas for each variable we only need to choose some value. herefore, for variables, we want to try to detect failures early on if possible (because we ll have to confront those variables sooner or later), but for values we want to steer away from possible failures because we might not have to consider those other values. Least constrained value (LC): Need to choose some value Choosing value most likely to lead to solution CS221 / Autumn 2016 / Liang 56 When do these heuristics help? Most constrained variable: useful when at least some factors are constraints Most constrained variable is useful for finding maximum weight assignments in any factor graph as long as there are some factors which are constraints, because we only save work if we can prune away assignments with zero weight, and this only happens with violated constraints (weight 0). On the other hand, least constrained value only makes sense if all the factors are constraints (CSPs). Ordering the values makes sense if we re going to just find the first consistent assignment. If there are any non-constraint factors, then would need to look at all consistent assignments to see which one has the maximum weight. Analogy: think about when depth-first search is guaranteed to find the minimum cost path. Least constrained value: useful when all factors are constraints Need lookahead (e.g., forward checking) to actually prune domains! CS221 / Autumn 2016 / Liang 58

11 Review: backtracking search Roadmap Algorithm: backtracking search Backtrack(x, w, Domains): If x is complete assignment: update best and return Choose unassigned ARIABLE X i Order ALUES Domain i of chosen X i For each value v in that order: δ f j (x {X i : v}) f j D(x,X i) If δ = 0: continue Domains Domains via LOOKAHEAD Backtrack(x {X i : v}, wδ, Domains ) Factor graphs Dynamic ordering Arc consistency Modeling CS221 / Autumn 2016 / Liang 60 CS221 / Autumn 2016 / Liang 61 Arc consistency Idea: eliminate values from domains reduce branching Example: numbers Before enforcing arc consistency on X i : X i Domain i = {1, 2, 3, 4, 5} X j Domain j = {1, 2, 3, 4, 5} f 1 (X) = [X i + X j = 4] f 2 (X) = [X j 2] After enforcing arc consistency on X i : X i Domain i = {2, 3} Now let us return to the issue of using lookahead to eliminate values from domains of unassigned variables. One motivation is that smaller domains lead to smaller branching factors, which makes search faster. A second motivation is that since the domain sizes are used in the context of the dynamic ordering heuristics (most constrained variable and least constrained value), we can hope to choose better orderings with domains that more accurately reflect what values are actually possible. We ve already seen forward checking as a simple way of using lookahead to prune the domains of unassigned variables. Shortly, we will introduce AC-3, which is forward checking without brakes. o build up to that, we need to introduce the idea of arc consistency. he idea behind enforcing arc consistency is to look at all the factors that involve just two variables X i and X j and rule out any values in the domain of X i which are obviously bad without even looking at another variables. o enforce arc consistency on X i with respect to X j, we go through each of the values in the domain of X i and remove it if there is no value in the domain of X j that is consistent with X i. For example, X i = 4 is ruled out because no value X j {1, 2, 3, 4, 5} satisfies X i + X j = 4. [whiteboard: bipartite graph] CS221 / Autumn 2016 / Liang 62 Arc consistency AC-3 (example) Definition: arc consistency A variable X i is arc consistent with respect to X j if for each x i Domain i, there exists x j Domain j such that f({x i : x i, X j : x j }) 0 for all factors f whose scope contains X i and X j. Algorithm: enforce arc consistency EnforceArcConsistency(X i, X j ): Remove values from Domain i to make X i arc consistent with respect to X j. CS221 / Autumn 2016 / Liang 64 CS221 / Autumn 2016 / Liang 65

12 AC-3 Forward checking: when assign X j : x j, set Domain j = {x j } and enforce arc consistency on all neighbors X i with respect to X j AC-3: repeatedly enforce arc consistency on all variables Algorithm: AC-3 Add X j to set. While set is non-empty: Remove any X j from set. For all neighbors X i of X j : Enforce arc consistency on X i w.r.t. X j. If Domain i changed, add X i to set. In fact, we already saw a limited version of arc consistency. In forward checking, when we assign a variable X i to a value, we are actually enforcing arc consistency on the neighbors of X i with respect to X i. Why stop there? AC-3 doesn t. In AC-3, we start by enforcing arc consistency on the neighbors of X i (forward checking). But then, if the domains of any neighbor X j changes, then we enforce arc consistency on the neighbors of X j, etc. In the example, after we assign : R, performing AC-3 is the same as forward checking. But after the assignment : G, AC-3 goes wild and eliminates all but one value from each of the variables on the mainland. Note that unlike BFS graph search, a variable could get added to the set multiple times because its domain can get updated more than once. More specifically, we might enforce arc consistency on (X i, X j) up to D times in the worst case, where D = max 1 i n Domain i is the size of the largest domain. here are at most m different pairs (X i, X j) and each call to enforce arc consistency takes O(D 2 ) time. herefore, the running time of this algorithm is O(ED 3 ) in the very worst case where E is the number of edges (usually, it s much better than this). CS221 / Autumn 2016 / Liang 66 Limitations of AC-3 Ideally, if no solutions, AC-3 would remove all values from a domain AC-3 isn t always effective: In the best case, if there is no way to consistently assign values all the variables, then running AC-3 will detect that there is no solution by emptying out a domain. However, this is not always the case, as the example above shows. Locally, everything looks fine, even though there s no global solution. Advanced: We could generalize arc consistency to fix this problem. Instead of looking at every 2 variables and the factors between them, we could look at every subset of k variables, and check that there s a way to consistently assign values to all k, taking into account all the factors involving those k variables. However, there is a substantial cost to doing this (the running time is exponential in k in the worst case), so generally arc consistency (k = 2) is good enough. No actual solutions, but AC-3 doesn t do anything! Intuition: if we look locally at the graph, nothing blatantly wrong... CS221 / Autumn 2016 / Liang 68 Summary Roadmap Basic template: backtracking search on partial assignments Factor graphs Dynamic ordering: most constrained variable (fail early), least constrained value (try to succeed) Dynamic ordering Lookahead: forward checking (enforces arc consistency on neighbors), AC-3 (enforces arc consistency on neighbors and their neighbors, etc.) Arc consistency Modeling CS221 / Autumn 2016 / Liang 70 CS221 / Autumn 2016 / Liang 71

13 Example: L question hree sculptures (A, B, C) are to be exhibited in rooms 1, 2 of an art gallery. he exhibition must satisfy the following conditions: Sculptures A and B cannot be in the same room. Sculptures B and C must be in the same room. Room 2 can only hold one sculpture. [demo] Example: event scheduling (section) event e time slot t Setup: Have E events and time slots Each event e must be put in exactly one time slot Each time slot t can have at most one event Event e allowed in time slot t only if (e, t) A CS221 / Autumn 2016 / Liang 72 CS221 / Autumn 2016 / Liang 73 Consider a simple scheduling problem, where we have E events that we want to schedule into time slots. here are three families of requirements: (i) every event must be scheduled into a time slot; (ii) every time slot can have at most one event (zero is possible); and (iii) we are given a fixed set A of (event, time slot) pairs which are allowed. here are in general multiple ways to cast a problem as a CSP, and the purpose of this example is to show two reasonable ways to do it. Example: event scheduling (section) event e time slot t CSP formulation 1: ariables: for each event e, X e {1,..., } Constraints (only one event per time slot): for each pair of events e e, enforce [X e X e ] Constraints (only scheduled allowed times): for each event e, enforce [(e, X e ) A] CS221 / Autumn 2016 / Liang 75 he first formulation is perhaps the more natural one. We make a variable X e for each event, whose value will be the time slot that the event is scheduled into. Since each variable can only take on one value, we automatically satisfy the requirement that every event must be put in exactly one time slot. However, we need to make sure no two events end up in the same time slot. o do this, we can create a binary constraint between every pair of distinct event variables X e and X e that enforces their values to be different (X e X e ). Finally, to deal with the requirement that an event is scheduled only in allowed time slots, we just need to add a unary constraint for each variable saying that the time slot X e that s chosen for that event is allowed. Note that we end up with E variables with domain size, and O(E 2 ) binary constraints. Example: event scheduling (section) event e time slot t CSP formulation 2: ariables: for each time slot t, Y t {1,..., E} { } Constraints (each event is scheduled exactly once): for each event e, enforce [Y t = e for exactly one t] Constraints (only schedule allowed times): for each time slot t, enforce [Y t = or (Y t, t) A] CS221 / Autumn 2016 / Liang 77

14 Alternatively, we can take the perspective of the time slots and ask which event was scheduled in each time slot. So we introduce a variable Y t for each time slot t which takes on a value equal to one of the events or none ( ). Unlike the first formulation, we don t get for free the requirement that each event is put in exactly one time slot. o add it, we introduce E constraints, one for each event. Each constraint needs to depend on all variables and check that the number of time slots t which have event e assigned to that slot (Y t = e) is exactly 1. On the other hand, the requirement that each time slot has at most one event assigned to it we get for free, since each variable takes on exactly one value. Finally, we add constraints, one for each time slot t enforcing that if there was an event scheduled there (Y t ), then it better be allowed according to A. With this formulation, we have variables with domain size E +1, and E -ary constraints. We will show shortly that each -ary constraints can be converted into O( ) binary constraints with O( ) variables. herefore, the resulting formulation has variables with domain size E + 1, O( 2 ) variables with domain size 2 and O( 2 ) binary constraints. Which one is better? Since E is required for the existence of a consistent solution, the first formulation is better. If the problem were modified so that not all events had to be scheduled and E, then the second formulation would be better. Arity Modeling: allow arity of factors to be arbitrary X 1 X 2 X 3 X 4 X 5 X 6 Algorithms: assume arity of all factors is 1 or 2 X 1 X 2 Later: reduction from n to 2 with auxiliary variables CS221 / Autumn 2016 / Liang 79 When we are modeling with factor graphs, we would like the factors to have any arity (depend on any number of variables). his allows us to, for example, require that at least one of the provinces is colored red. However, from an algorithms and implementation perspective, it is often useful to just think about unary and binary factors. For example, arc consistency is defined with respect to binary factors. It appears that there is a tradeoff between modeling expressivity and algorithmic efficiency, but this is actual not a real tradeoff, since we can reduce the general arity case to the unary-binary case. N-ary constraints ariables: X 1, X 2, X 3, X 4 {0, 1} Factor: [X 1 X 2 X 3 X 4 ] Examples: X 1 X 2 X 3 X 4 Weight({X 1 : 0, X 2 : 0, X 3 : 0, X 4 : 0}) = 0 Weight({X 1 : 0, X 2 : 1, X 3 : 0, X 4 : 0}) = 1 Weight({X 1 : 0, X 2 : 1, X 3 : 0, X 4 : 1}) = 1 Problem: algorithms so far only take unary/binary factors... CS221 / Autumn 2016 / Liang 81 Consider the simple problem: given n variables X 1,..., X n, where each X i {0, 1}, impose the requirement that at least one X i = 1. he case of n = 4 is shown in the slide. N-ary constraints: first attempt X 1 X 2 X 3 X 4 Key idea: auxiliary variable Auxiliary variables hold intermediate computation. Factors: Initialization: [A 0 = 0] Processing: [A i = A i 1 X i ] Final output: [A 4 = 1] Still have factors involving 3 variables... i X i : A i : CS221 / Autumn 2016 / Liang 83

15 he key idea is to break down the computation of the n-ary constraint into n simple steps. As a first attempt, let s introduce an auxiliary variable A i for i = 1,..., n which represents the OR of variables X 1,..., X i. hen we can write a simple recurrence that updates A i with A i 1. he constraint [A n = 1] enforces that the OR of all the variables is 1. It is important to note that while our intuitions are based on procedurally computing A i s, one after the other, these computations are actually represented declaratively as constraints in the CSP. We have eliminated the massive n-ary constraint with ternary constraints (depending on A i, A i 1, X i). Can we replace the ternary constraint with unary and binary constraints? N-ary constraints: second attempt Key idea: pack A i 1 and A i into one variable B i (0, 0) (0, 1) (1, 1) (1, 1) B 1 B 2 B 3 B 4 X 1 X 2 X 3 X ariables: B i is (pre, post) pair from processing X i Factors: Initialization: [B 1 [1] = 0] Processing: [B i [2] = B i [1] X i ] Final output: [B 4 [2] = 1] Consistency: [B i 1 [2] = B i [1]] CS221 / Autumn 2016 / Liang 85 he key idea to turn the ternary constraint [A i = A i 1 X i] into a binary constraint is to merge A i 1 and A i into one variable, represented as one variable B i. he variable B i will represent a pair of booleans, where B i[1] represents A i 1 and B i[2] represents A i. Now, the ternary constraint is just a binary constraint: [B i[2] = B i[1] X i]! However, note that A i 1 is represented twice, both in B i and B i 1. So we need to add another binary constraint to enforce that the two are equal: [B i 1[2] = B i[1]]. he initialization and final output factors are the same as before. Example: relation extraction Motivation: build a question-answering system Which US presidents played the guitar? Prerequisite: learn knowledge by reading the web Systems: [NELL (CMU)] [OpenIE (UW)] CS221 / Autumn 2016 / Liang 87 Now let s look at a different problem. Some background which is unrelated to CSPs: A major area of research in natural language processing is relation extraction, the task of building systems that can process the enormous amount of unstructured text on the web, and populate a structured knowledge base, so that we can answer complex questions by querying the knowledge base. Example: relation extraction Input (hundreds of millions of web pages): Barack Obama is the 44th and current President of the United States... Output (database of relations): EmployedBy(BarackObama, UnitedStates) Profession(BarackObama, President)... CS221 / Autumn 2016 / Liang 89

16 Example: relation extraction ypical predictions of classifiers: State-of-the-art methods typically use machine learning, casting it as a classification problem. However, relation extraction is a very difficult problem, and even the best systems today often fail, producing nonsensical facts. A key observation is that these classification decisions are not independent, and we have some prior knowledge on how they should be related. For example, you can t be born in two places, and you also can t be born in an instrument (not usually, anyway). BornIn(BarackObama,UnitedStates) 0.9 BornIn(BarackObama,Kenya) 0.6 BornIn(JohnLennon,guitar) 0.7 ype(guitar,instrument) How do reconcile conflicting predictions? CS221 / Autumn 2016 / Liang 90 Example: relation extraction BornIn(BarackObama,UnitedStates) can t be born in two places BornIn(BarackObama,Kenya) BornIn(JohnLennon,guitar) can t be born in an instrument ype(guitar,instrument) General framework Classification decisions are generally related: Y 1 Y 2 Y 3 Y 4 Unary factors: local classifiers (provide evidence) exp(w φ(x i )Y i ) Binary factors: enforce that outputs are consistent [Y i consistent with Y i ] CS221 / Autumn 2016 / Liang 92 CS221 / Autumn 2016 / Liang 93 o operationalize this intuition, we can leverage factor graphs. hink about each of the classification decisions as a variable, which can take on 1 or 0 (assume binary classification for now). We have a unary factor which specifies the contribution of the classifier. Recall that linear classifiers return a score w φ(x i), which is a real number. Factors must be non-negative, so it s typical to exponentiate the score. We can add binary factors between pairs of classification decisions which are related in some way (e.g., [BornIn(BarackObama,UnitedStates)+BornIn(BarackObama,Kenya) 1]). he factors do not have to be hard constraints, but rather general preferences that encode soft preferences (e.g., returning weight 0.01 instead of 0). Once we have a CSP, we can ask for the maximum weight assignment, which takes into account all the information available and reasons about it globally. Summary Factor graphs: modeling framework (variables, factors) Key property: ordering decisions pushed to algorithms Algorithms: backtracking search + dynamic ordering + lookahead Modeling: lots of possibilities! CS221 / Autumn 2016 / Liang 95