Constraint Satisfaction Problems

Transcription

1 Constraint Satisfaction Problems Constraint satisfaction problems Backtracking algorithms for CSP Heuristics Local search for CSP Problem structure and difficulty of solving

2 Search Problems The formalism of search problems we have discussed so far is a very powerful formalism that depends on the notion of state. From the point of view of a search algorithm, a state is a black box with no discernible internal structure. A state can be represented by an arbitrary data structure and can be accessed only by problem-specific functions: successor, goal test, heuristics etc.

3 Constraint Satisfaction Problems The framework we will now present (constraint satisfaction problems) admits a very simple standard representation. This allows us to define search algorithms that take advantage of this very simple representation and use general purpose heuristics to enable solution of large problems. The simple structure also allows us to define methods for problem decomposition and offers us an intimate connection between the structure of a problem and the difficulty of solving it.

4 Constraint Satisfaction Problems - Definitions A constraint satisfaction problem (CSP) is defined by: A set of variables X 1,..., X n. Each variable has a domain D i of possible values. A set of constraints C 1,..., C m. Each constraint involves some subset of the variables and specifies the allowable combinations of values for that subset. Formally, an (k-ary) constraint C on a set of variables X 1,..., X k is a subset of the Cartesian product D 1 D k.

5 Definitions (cont d) A solution to a CSP is a complete assignment of values to variables such that all the constraints are satisfied. A CSP is called consistent it has a solution, otherwise it is called inconsistent.

6 Example: Map Coloring Western Australia Northern Territory Queensland South Australia New South Wales Victoria Tasmania

7 Example: Formal Definition Variables: W A, NT, SA, Q, NSW, V, T Domain (same for all variables): { red, green, blue } Constraints: C(W A, NT ) = { (red, green), (red, blue), (green, red), (blue, red), (blue, green) } More succinctly W A NT. Similarly for the other pairs of variables.

8 Constraint Graphs WA NT Q SA NSW V Victoria T

9 Example:The 8-queens problem

10 Example: Formal Definition Variables: Let variable X i (i = 1,..., 8) represent the column that the i-th queen occupies in the i-th row. If columns are represented by numbers 1 to 8 then the domain of every variable X i is D i = {1, 2,..., 8}.

11 Example: Formal Definition Constraints: There is a binary constraint C(X i, X j ) for each pair of variables. These constraints can be specified succinctly as follows: For all variables X i and X j, X i X j. For all variables X i and X j, if X i = a and X j = b then i j a b and i j b a.

12 Example: Cryptarithmetic T W O F T U W R O + T W O F O U R (a) X 3 X 2 X 1 (b)

13 Example: Formal Definition Variables and domains: F, T, U, W, R, O {0, 1, 2, 3, 4, 5, 6, 7, 8, 9} X 1, X 2, X 3 {0, 1} Constraints: alldiff(f, T, U, W, R, O) O + O = R + 10X 1 X 1 + W + W = U + 10X 2 X 2 + T + T = O + 10X 3 X 3 = F

14 Constraints in Real Life In many practical problems there are soft constraints in addition to hard constraints. Soft constraints encode preferences. Example: Timetabling for a University. What are the hard constraints? What are the soft constraints?

15 Other Examples of CSPs The satisfiability problem in Boolean logic (also a CSP with finite domains). Temporal reasoning (infinite domains). Timetabling Job-shop scheduling, airline crew scheduling Spatial reasoning Integer, linear and non-linear programming (operations research).

16 CSP Technology: Practical, Successful and AI! CSPs are certainly a very successful example of ideas from AI with many of applications. There are currently several companies marketing such technology:

17 A Taxonomy of CSPs Discrete vs. continuous variables Finite vs. infinite domains Explicit enumeration of allowed combinations of values vs. constraint languages Linear vs. non-linear constraints Unary vs. binary vs.... constraints In the rest of this presentation, we will concentrate on search algorithms for binary, finite domain CSPs.

18 Binary vs. Non-Binary Constraints It is possible to reduce every non-binary finite-domain CSP to an equivalent binary CSP. Two general methods for doing this are: the dual transformation and the hidden variables transformation. See the paper Fahiem Bacchus, Xinguang Chen, Peter van Beek, and Toby Walsh. Binary vs. non-binary constraints. Artificial Intelligence, 140:1-37, for more details.

19 Search Algorithms for CSPs Let us apply a general search algorithm to CSP: Initial state: all variables are unassigned. Actions: Assign to any unassigned variable X i any value from D i. Goal test: All variables are assigned and the constraints are satisfied. The branching factor in this case is n i=1 D i where D i is the cardinality of domain D i. The last level of the tree has ( n i=1 D i ) n nodes.

20 Search Algorithms for CSPs (cont d) Better approach: Order the variables! CSPs are commutative search problems i.e., the order of application of actions does not matter. Characteristics: The size of the search space is finite. If the variables are ordered as X 1,..., X n the number of nodes in the search tree is 1 + n i=1 ( D 1 D i ). When is the number of nodes in the search tree maximal? Minimal? The depth of the search tree is fixed. There are similar subtrees.

21 Search Algorithms for CSPs (cont d) Which of the search algorithms presented so far is appropriate for solving CSPs: BFS? No! BFS will not be effective because goal states are located at the leaves of the search tree. DFS? Better than BFS. But it wastes time searching when constraints are already violated.

22 Search Algorithms for CSPs (cont d) We will present variants of DFS for CSPs. These algorithms are based on the idea of backtracking search: choose values for one variable at a time, and backtrack when there are no more legal values to assign. We will see the following algorithms and their variants/hybrids: Simple or chronological backtracking (BT) Forward checking (FC) Backjumping (BJ) Conflict-directed Backjumping (CBJ) Maintaining Arc Consistency (MAC)

23 BT in Operation: Example WA=red WA=green WA=blue WA=red NT=green WA=red NT=blue NT=green WA=red NT=green Q=blue

24 Chronological Backtracking (BT) The basic idea in any backtracking algorithm is to start with a partial solution and to extend it until we reach a complete solution. BT follows this general method. Additionally, when it reaches a dead-end, it always backtracks to the last decision made (hence its name!). function Backtracking-Search(csp) returns a solution or failure return Recursive-Backtracking({}, csp)

25 BT (cont d) function Recursive-Backtracking(assignment, csp) returns a solution or failure if assignment is complete then return assignment var Select-Unassigned-Variable(Variables[csp], assignment, csp) for each value in Order-Domain-Values(var, assignment, csp) do if value is consistent with assignment according to Constraints(csp) then add {var = value} to assignment result Recursive-Backtracking(assignment, csp) if result failure then return result remove {var = value} from assignment return failure

26 BT (cont d) Evaluation: Complete? Yes Time: O(d n e) where d is the maximum domain size, n is the number of variables, and e is the number of constraints. Space: O(nd). This is the amount of space needed for storing the domains of the variables. The above time and space complexity bounds are based on the assumption that constraints can be stored using a constant amount of space, and constraint checks can be done in constant time.

27 BT (cont d) Backtracking can be improved if we give clever answers to the following questions: 1. Which variable should be assigned next, and in what order should its values be tried? 2. What are the implications of the current variable assignments to the other unassigned variables? 3. When a path fails, can the search avoid this failure in subsequent paths?

28 Variable Ordering Heuristics By default the function Select-Unassigned-Variable selects the next unassigned variable from the list Variables[csp]. This is static variable assignment and seldom results in efficient search. The minimum remaining values (MRV) heuristic: choose the variable with the fewest remaining legal values i.e., minimize the branching factor (dynamic test). Question: In the map coloring problem for Australia, what is the variable to be assigned after the assignments W A = red, NT = green have been made? Answer: SA because only the value blue is possible.

29 Example: Map Coloring Western Australia Northern Territory Queensland South Australia New South Wales Victoria Tasmania

30 Variable Ordering Heuristics (cont d) For coloring the map of Australia, how do we start our search? The degree heuristic: choose the variable involved in the largest number of constraints with other unassigned variables (static test). (Intuition: increase future elimination of values, to decrease future branching factors.) Question: In the map coloring problem for Australia, what is the variable to be assigned first? Answer: SA that has degree 5. All the other variables have degree 2 or 3 except of T that has degree 0.

31 Variable Ordering Heuristics (cont d) MRV is usually more powerful than the degree. They can be used together with the latter one playing the role of a tie-breaker when the first one cannot make a distinction. We will use just MRV to refer to this combination of heuristics.

32 Evaluation Problem BT BT+MRV FC FC+MRV Min-con USA (>1000K) (>1000K) 2K n-queens (>40000K) 13500K (>40000K) 817K 4K Zebra 3859K 1K 35K 0.5K 2K

33 Value Ordering Heuristics Once a variable has been selected, the algorithm must decide on the order in which to examine values. The least-constraining-value heuristic (LCV): prefer the value that rules out the fewest choices for neighbouring variables in the constraint graph. In other words, leave maximum flexibility for subsequent variable assignments. LCV is not useful if we are looking for all solutions or the problem has no solution.

34 Value Ordering Heuristics (cont d) Question: In the map coloring problem for Australia, what is the value to be assigned to variable Q after the assignments have been made? W A = red, NT = green Answer: We can only have blue or red. The value blue would be a bad choice according to LCV: it has two conflicts (with NSW and SA; in fact, it eliminates the last legal value for SA). red is better because it has only 1 conflict: with value red for NSW.

35 Example: Map Coloring Western Australia Northern Territory Queensland South Australia New South Wales Victoria Tasmania

36 Constraint Propagation The main idea behind constraint propagation is to consider the given constraints early in the search or even before the search has started! For example, we can prune the search space by examining the consequences of partial assignments. The algorithms that have been proposed to achieve this are sometimes referred to as look-ahead algorithms.

37 Forward Checking Forward Checking (FC) belongs to the family of backtracking algorithms based on constraint propagation and maintains the following invariant: For every unassigned variable, there exists at least one value in its domain which is compatible with the values that have been assigned to other variables.

38 Forward Checking FC works as follows: every time a value v is assigned to a variable, FC will remove all values which are inconsistent with v from the domains of the unassigned variables. If the domain of any of the unassigned variables is reduced to an empty set, then v will be rejected. FC is typically used together with the MRV heuristic since all the machinery required for the implementation of MRV is used by FC as well.

39 FC in Operation (Without MRV) WA NT Q NSW V SA T Initial domains After WA=red After Q=green After V=blue R G B R G B R G B R G B R G B R G B R G B R G B R G B R G B R G B G B R G B R B G R B R G B B R G B R B G R B R G B

40 Evaluation Problem BT BT+MRV FC FC+MRV Min-con USA (>1000K) (>1000K) 2K n-queens (>40000K) 13500K (>40000K) 817K 4K Zebra 3859K 1K 35K 0.5K 2K

41 Example: Map Coloring with FC Western Australia Northern Territory Queensland South Australia New South Wales Victoria Tasmania W A = red, Q = green implies NT = blue, SA = blue. This inconsistency is not detected by FC.

42 Example (cont d) The partial assignment W A = red, Q = green together with the problem constraints W A NT, W A SA, Q NT, Q SA imply W A, Q {red, blue, green} NT = blue, SA = blue. These implied constraints together with the problem constraint NT SA are inconsistent. FC s machinery do not allow it to make this second constraint propagation step.

43 Arc Consistency We can improve this behavior of FC by devising more sophisticated propagation steps. For example, the constraint propagation step of forward checking can actually become stronger by using the concept of arc consistency. Definition. Let X, Y be variables of a CSP P and (X, Y ) be a directed arc in the constraint graph for P. The arc (X, Y ) is called consistent if for every value x of X, there is some value y of Y such that x is consistent with y. Definition. A CSP is called arc consistent if all its arcs (i.e., the arcs of its constraint graph) are consistent.

44 The Example Revisited WA NT Q NSW V SA T Initial domains After WA=red After Q=green After V=blue R G B R G B R G B R G B R G B R G B R G B R G B R G B R G B R G B G B R G B R B G R B R G B B R G B R B G R B R G B

45 Example (cont d) Consider the third row of the previous table for the problem of coloring the map of Australia using FC. At this stage of the algorithm, let us look at a few arcs to see whether they are consistent: Arc (SA, NSW ): The current domains of nodes SA and NSW are { blue } and { red, blue }, thus arc (SA, NSW ) is consistent. Arc (N SW, SA): This one is inconsistent, because assignment NSW = blue does not have a consistent assignment for SA. In this case, we should delete the value blue from the domain of NSW to make the arc consistent.

46 Example (cont d) Arc (SA, NT ): This arc is inconsistent because the current domain for SA and NT is {blue}. If we try make this arc consistent, we remove the value blue from the domain of SA, leaving this domain empty. Thus applying arc consistency resulted in earlier detection of an inconsistency during search (a path that would not lead to a solution).

47 Arc Consistency (cont d) In a backtracking algorithm, arc consistency can be applied as: Preprocessing step before the search starts. Constraint propagation step after each assignment repeatedly, until no more arc inconsistencies remain. This algorithm is known as MAC which stands for maintaining arc consistency. Example: If we use MAC after the assignment W A = red, Q = green in our map coloring example, we immediately discover the impossibility of extending this assignment because arc (SA, N T ) is inconsistent.

48 The Algorithm AC-3 function AC-3( csp) returns the CSP, possibly with reduced domains inputs: csp, a binary CSP with variables {X 1, X 2,..., X n } local variables: queue, a queue of arcs, initially all the arcs in csp while queue is not empty do (X i, X j ) Remove-First(queue) if Remove-Inconsistent-Values(X i, X j ) then for each X k in Neighbors[X i ] do add (X k, X i ) to queue

49 The Algorithm AC-3 (cont d) function Remove-Inconsistent-Values( X i, X j ) returns true iff we remove a value removed false for each x in Domain[X i ] do if no value y in Domain[X j ] allows (x,y) to satisfy the constraint between X i and X j then delete x from Domain[X i ]; removed true return removed

50 AC-3 AC-3 runs in O(n 2 d 3 ) time where n is the number of variables and d the maximum number of domain values: A binary CSP has at most O(n 2 ) arcs. An arc can enter the queue at most d times. Checking consistency of an arc can be done in O(d 2 ) time. There are faster arc-consistency algorithms.

51 Other Notions of Consistency 1-consistency or node consistency: A unary constraint c on variable X is 1-consistent if for each value v domain(x), v is a solution of c. If a constraint involves more than one variable then it is trivially 1-consistent. 2-consistency is a synonym of arc consistency. Stronger notions of consistency: 3-consistency or path consistency, 4-consistency,..., k-consistency, global consistency (k-consistency for all 1 k n in a CSP with n variables).

52 k-consistency Definition. A CSP is k-consistent if each variable assignment to k 1 variables {X 1 = v 1,..., X k 1 = v k 1 }, that satisfies all the constraints involving variables X 1,..., X k 1, can be extended to an assignment {X 1 = v 1,..., X k 1 = v k 1, X k = v k } that satisfies all the constraints involving variables X 1,..., X k 1, X k.

53 US02 Teqnht NohmosÔnh Example The CSP X 1 X 2 2, X 2 X 3 5, X 1 X 3 10, X 1, X 2, X 3 R is 1- and 2-consistent but not 3-consistent. For example, the variable assignment {X 1 = 10, X 3 = 0} that satisfies X 1 X 3 10 cannot be extended to a variable assignment that satisfies all three constraints (try it!). The CSP X 1 X 2 2, X 2 X 3 5, X 1 X 3 7, X 1, X 2, X 3 R which has a stricter constraint on X 1, X 3 is 3-consistent. The constraint X 1 X 3 7 is implied by the other two constraints and can be obtained by constraint propagation.

54 Other Notions of Consistency (cont d) Consistency notions stronger than arc consistency can be employed in backtracking algorithms in a similar way. At each step of the search process, these consistency steps bring into the light the implications of the problem constraints by examining 3 variables, 4 variables,..., n variables at a time. Important: The cost of constraint propagation steps needs to be carefully balanced with their benefits. One way to determine this is by carrying out experiments.

55 Intelligent Backtracking Backtracking algorithms can become more effective by backtracking in a more sophisticated way (intelligently!). The techniques proposed to achieve this are sometimes referred to as look-back techniques. The idea here is to avoid backtracking chronologically like BT, but rather backtrack in a clever way to the actual cause of the failure.

56 Chronological Backtracking Western Australia Northern Territory Queensland South Australia New South Wales Victoria Tasmania

57 Chronological Backtracking (cont d) Let us consider BT in the problem of map coloring with a fixed variable ordering: Q, NSW, V, T, SA, W A, NT. Suppose we have generated the following partial assignment: Q = red, NSW = green, V = blue, T = red When we try the next variable, SA, we see that every value violates a constraint. Now BT tell us to backtrack and try a new color for Tasmania! This is not a good idea!

58 Backjumping Backjumping (BJ) is an intelligent backtracking algorithm. When a dead-end occurs at variable x, BJ does not backtrack to the previous variable like BT. Instead, it backtracks to the deepest variable in the search tree (also called the most recent variable) which caused a value in the domain of x to be eliminated. The set of variables that caused a value in the domain of x to be eliminated is called the conflict set of x.

59 Example with BJ Western Australia Northern Territory Queensland South Australia New South Wales Victoria Tasmania

60 BJ (cont d) Let us consider BJ in the problem of map coloring with a fixed variable ordering: Q, NSW, V, T, SA, W A, NT Suppose we have generated the following partial assignment: Q = red, NSW = green, V = blue, T = red When we try the next variable, SA, we see that every value violates a constraint. The variables that cause elimination of all possible values for SA are {Q, NSW, V }. Now BJ tells us to backtrack to V.

61 BJ (cont d) To implement BJ, we need to remember the deepest variable that caused a value of the current variable to be rejected (the deepest variable in the conflict set).

62 BJ vs. FC When backjumping occurs, all values of a domain are in conflict with the current assignment. This would have already been detected by FC! Proposition. Every branch of a search tree pruned by BJ is also pruned by FC. Thus BJ is redundant in a search using FC or a stronger constraint propagation algorithm such as MAC.

63 Example Western Australia Northern Territory Queensland South Australia New South Wales Victoria Tasmania

64 From BJ to CBJ Let us consider again BJ in the problem of map coloring with the fixed variable ordering W A, NSW, T, NT, Q, V, SA. Suppose we have generated the following partial assignment: W A = red, NSW = red This assignment cannot lead us to a solution. But let us assign T = red, and continue with NT, Q, V, SA. This is not going to work and, eventually, we run out of values at NT. Where should we backtrack?

65 From BJ to CBJ (cont d) BJ cannot tell us anything useful because the conflict set of NT is empty (i.e., NT has values that are consistent with all previous variables). In this case, it is really the variables NT, Q, V, SA taken together that conflict with the previous variables. This leads to a deeper notion of a conflict set of a variable x: it is that set of preceding variables that caused x, together with any subsequent variables, to lead to failure. Under this definition, the conflict set for NT is {W A, NSW } and we should backtrack to NSW. This is the idea behind conflict-directed backjumping.

66 Conflict-Directed Backjumping (CBJ) In CBJ every variable has a conflict set as in BJ. When a consistency check between an instantiation v i of the current variable x i and an instantiation v k of a previously assigned variable x k fails, then x k is added to the conflict set of x i. When there are no more values to be tried for x i, CBJ backtracks to the deepest variable x h in the conflict set of x i. At the same time, the variables in the conflict set of x i (except of x h ) are added to the conflict set of x h so that no information about conflicts is lost. This is the important difference with BJ.

67 US02 Teqnht NohmosÔnh Running CBJ on our Example Order of variables: W A, NSW, T, NT, Q, V, SA Order of values: red, blue, green CBJ proceeds as follows (se show values assigned to variables and conflict sets): W A = red NSW = red NT = blue, CS(NT ) = {W A} Q = green, CS(Q) = {NSW, NT } V = blue, CS(V ) = {NSW } SA cannot be assigned any value and CS(SA) = {NSW, V, Q}. We backtrack to V (deepest variable in the conflict set). CS(V ) is set to {NSW, Q}.

68 Running CBJ on our Example (cont d) Order of variables: W A, NSW, T, NT, Q, V, SA Order of values: red, blue, green CBJ continues as follows: V = green, CS(V ) = {NSW, Q} SA cannot be assigned any value and CS(SA) = {NSW, NT, V }. We backtrack to V (deepest variable in the conflict set). CS(V ) is set to {NSW, NT }. V cannot be assigned any value and CS(V ) = {NSW, NT, Q}. We backtrack to Q (deepest variable in the conflict set). CS(Q) is set to {NSW, NT }.

69 Running CBJ on our Example (cont d) Order of variables: W A, NSW, T, NT, Q, V, SA Order of values: red, blue, green CBJ continues as follows: There are no other values to be tried for Q and CS(Q) = {NSW, NT }. We backtrack to NT (deepest variable in the conflict set). CS(NT ) is set to {NSW, W A}. NT = green, CS(NT ) = {NSW, W A}. Now variables Q, V, SA will be tried in a similar way and no solution will be found. When we are back at NT, we backtrack to NSW (the deepest variable in the conflict set) and not to T.

70 Hybrid Algorithms The ideas presented in the previous algorithms can be combined to create hybrid algorithms e.g., FC-CBJ or MAC-CBJ. Heuristics are usually combined with these hybrid algorithms as well.

71 Evaluating Backtracking Algorithms Criteria: 1. Worst case time/space complexity 2. Run time 3. Number of nodes visited in the search tree 4. Number of consistency checks performed

72 Evaluating Backtracking Algorithms (cont d) Results: BT, FC, BJ, CBJ, MAC and their variants have exponential worst case time complexity. Run time can vary depending on implementation details. Number of visited nodes FC-CBJ FC-BJ FC BJ BT CBJ BJ MAC-CBJ MAC-BJ MAC MAC FC

73 Evaluating Backtracking Algorithms (cont d) Number of consistency checks performed CBJ BJ BT FC-CBJ FC-BJ FC FC may perform more or less consistency checks than BJ and BT depending on the problem. Experimental results have shown that in most cases a good constraint propagation algorithm (like MAC or FC) with a good set of heuristics (like MRV and LCV) can go a long way in solving difficult CSP problems.

74 Related Publications and Software Grzegorz Kondrak and Peter van Beek. A theoretical evaluation of selected backtracking algorithms. Artificial Intelligence, 89: , Xinguang Chen and Peter van Beek. Conflict-directed backjumping revisited. Journal of Artificial Intelligence Research, 14:53-81, csplib: A library of C functions for solving binary constraint satisfaction problems. The above are available from the Web site of Peter van Beek:

75 Local Search Algorithms All the search algorithms we have presented up to now (for general search problems or CSPs) are systematic: they explore the search space carefully keeping track of each path explored until they find a solution. We have already seen how to solve n-queens problems using local search. Can we solve arbitrary CSPs using local search algorithms?

76 Local Search Algorithms (cont d) Idea: Start with a solution and make modifications until you reach a solution. Graphically: evaluation current state

77 Local Search Algorithms for CSPs Local search is particularly useful for CSPs. A local search algorithm starts with a random assignment of values to variables and then modifies (repairs) this assignment until it becomes a solution. These algorithms are also referred to as heuristic repair algorithms in the literature. We have already seen the application of hill-climbing to the 8-queens problem.

78 The Min-Conflicts Heuristic In choosing a new value for a variable, a useful heuristic is to choose the one that would cause the minimum number of conflicts with the current assignment to the other variables. The above heuristic is called the min-conflicts heuristic and it is surprisingly powerful for many CSPs.

79 Min-Conflicts function Min-Conflicts(csp, max-steps) returns a solution or failure inputs: csp, a constraint satisfaction problem max-steps, the number of steps allowed before giving up local variables: current, a complete assignment var, a variable value, a value for a variable current an initial complete assignment for csp for i = 1 to max-steps do if current is a solution for csp then return current var a randomly chosen, conflicted variable from Variables[csp] value the value v for var that minimizes Conflicts(var, v, current, csp) set var=value in current return failure

80 Min-Conflicts (cont d) The function Conflicts counts the number of constraints violated by a particular value, given the rest of the assignment.

81 Min-Conflicts and N-Queens When given a reasonable initial state, it can solve problems with millions of queens in around 50 steps (in fact, its running time is roughly independent of the problem size). We can compute an initial state by iterating through the columns (variables), placing each queen in the row (value) where it conflicts with the least number of previously placed queens. The repair can be accomplished by keeping a list of all queens that are in conflict (i.e., are attacked by others) together with counters showing the number of attacking queens for each alternative position of these queens. Note: The N-queens problem is easy for local search because solutions are densely distributed throughout the state space.

82 Example

83 Evaluation Problem BT BT+MRV FC FC+MRV Min-con USA (>1000K) (>1000K) 2K n-queens (>40000K) 13500K (>40000K) 817K 4K Zebra 3859K 1K 35K 0.5K 2K

84 Min-Conflicts and Scheduling Problems The min-conflicts heuristic has been used in observation scheduling algorithms for the Hubble telescope reducing the scheduling time from 3 weeks to 10 minutes! For details see the paper Steven Minton, Mark D. Johnston, Andrew B. Philips and Philip Laird. Minimizing Conflicts: A Heuristic Repair Method for Constraint Satisfaction and Scheduling Problems. Artificial Intelligence 58(1-3), pages (1992). The importance of local search algorithms in problems such as scheduling is that on-line re-scheduling is an important operation. This can be gracefully carried out by local search as well. The local search techniques we have discussed (hill climbing, simulated annealing etc.) can also be applied to constraint optimization problems.

85 The Structure of Problems There are ways to exploit the structure of a CSP to find solutions quickly. For example: Independent subproblems Tree CSPs and directed arc consistency Reducing general constraint graphs to trees: Fixing the values of some variables so that the remaining variables form a tree. Constructing a tree decomposition of the graph into a set of connected subproblems.

86 Independent Subproblems Look for connected components in the constraint graph that give subproblems CSP i. Example: Coloring Tasmania is a subproblem of the map coloring problem we presented earlier which is independent of the problem of coloring the rest of Australia. If each CSP i has c variables, then solving each subproblem separately and then combining the solutions takes O(d c n/c) time (linear in n), while solving the original problem takes O(d n ) time (exponential in n).

87 Tree CSPs and Directed Arc Consistency A E B D A B C D E F C (a) F (b)

88 Tree CSPs and Directed Arc Consistency Tree CSPs can be solved in time linear in the number of variables (O(nd 2 )) as follows: Choose any variable as the root of the tree, and order the variables as X 1,..., X n from the root to the leaves in such a way that every node s parent in the tree precedes it in the ordering. For j from n down to 2, apply arc consistency to arc (X i, X j ), where X i is the parent of X j, removing values from Domain(X i ) as necessary. This is called directed arc consistency. For j from 1 to n, assign any value for X j consistent with the value assigned for X i, where X i is the parent of X j.

89 Reducing Constraint Graphs to Trees WA NT Q SA NSW V Victoria T Step1: Assign a value to variable SA, remove inconsistent values from the other variables and then remove node SA from the graph.

90 Reducing Constraint Graphs to Trees (cont d) WA NT Q NSW V Victoria T Step2: Solve the remaining subproblem using tree algorithms.

91 Tree Decomposition NT Q WA NT SA Q SA SA NSW SA NSW V T

92 Readings Chapter 5 of AIMA.