Chronological Backtracking Conflict Directed Backjumping Dynamic Backtracking Branching Strategies Branching Heuristics Heavy Tail Behavior

Transcription

1 PART III: Search

2 Outline Depth-first Search Chronological Backtracking Conflict Directed Backjumping Dynamic Backtracking Branching Strategies Branching Heuristics Heavy Tail Behavior Best-First Search Limited Discrepancy Search

3 Depth-first Search Algorithms Backtracking tree (BT) search algorithms essentially perform depth-first traversal of a search tree. Every node represents a decision made on a variable X. At each node: check constraints between X and the already assigned variables; if consistent continue down in the tree; otherwise prune the underlying subtree and backtrack to an unassigned variable that still has alternative values. Systematic search Eventually finds a solution or proves unsatisfiability. Complexity O(d n ), exponential!

4 Chronological BT Backtracks to the most recent variable. Suffers from trashing. The same failure can be remade an exponential number of times. Consistency is checked only between the current and the past variables.

5 CBT for 4-Queens

6 CBT + Forward Checking Propagation

7 CBT + Arc Consistency Propagation

8 Non-Chronological Backtracking Backtrack on a culprit variable. Ignore decisions that didn t contribute to the failure. E.g., N-Queens Backtracking to X 5 is pointless. Better to backtrack on X 4.

9 Conflict Sets For each variable, maintain a conflict set CS(X i ). CS(X i ): set of variables which ruled out values from the domain of X i.

10 Conflict Directed Backjumping Backtracks to the last variable in the conflict set. The conflict set is backed up. Intermediate decisions are removed. We can also do CBJ + FC and CBJ + MAC.

11 No-goods Subset of incompatible assignments. Implied constraints discovered during search after a dead end. E.g., [X 1, X 2, X 3 ] with domain {0, 1} where X i X j for 1 i <j 3 (X 1 = 0 and X 3 = 0) or equivalently (X 1 = 0 X 3 0) is a nogood. No-good resolution: discovery of new no-goods. X 1 = 0 X 3 0 X 2 = 1 X 3 1 X 1 = 0 X 2 1

12 Dynamic Backtracking One no-good for each incompatible value in the domain of X i is maintained. Empty domain: new no-good by no-good resolution. Backtrack to the variable in the right hand side of the no-good.

13 Dynamic Backtracking Backtracks to the last decision responsible for the dead-end. Intermediate decisions are not removed.

14 Branching Strategies The method of extending a node in the search tree. Usually consists of posting a unary constraint on a chosen variable X i. X i & the ordering of the branches are chosen by the heuristics. D-way branching: One branch is generated for each v j D(X i ) by X i v j. 2-way branching: 2 branches are generated for each v j D(X i ) by X i v j and X i \ v j. Domain splitting: k branches are generated by X i D j where D 1 D k are partitions of D i.

15 Branching Heuristics Guide the search. Given a branching strategy, which branch to consider first? Which variable next? Which value(s) next? Problem specific vs generic heuristics. Static vs dynamic heuristics.

16 Static Variable Ordering Heuristics A variable is associated with each level. Branches are generated in the same order all over the tree. Calculated once and for all before search starts, hence cheap to evaluate.

17 Popular Static Generic Heuristics The order of definition in case of a sequence of variables: X 1, X 2,, X n Top down, left to right, row by row in case of a matrix of variables: X 11, X 12,, X 1m X 21, X 22,, X 2m X n1, X n2,, X nm

18 Dynamic Variable Ordering Heuristics At any node, any variable & branch can be considered. Decided dynamically during search, hence costly. Takes into account the current state of the search tree.

19 Popular Dynamic Generic Heuristics Fail-first principle: to succeed, try first where you are most likely to fail. Minimum domain (dom) Choose next the variable with minimum domain. Idea: minimize the search tree size. Most constrained (deg) Choose next the variable involved in most number of constraints. Idea: maximize constraint propagation. Combination Minimize dom / deg

20 Map Coloring

21 Map Coloring

22 Map Coloring

23 Map Coloring CBT+ AC with 2-way branching using various heuristics.

24 Lexicographic Ordering

25 Lexicographic Ordering

26 Lexicographic Ordering

27 Lexicographic Ordering

28 Maximum Degree

29 Maximum Degree

30 Maximum Degree

31 Maximum Degree

32 Minimum Domain

33 Minimum Domain

34 Minimum Domain

35 Minimum Domain / Degree

36 Minimum Domain / Degree

37 Minimum Domain / Degree

38 Minimum Domain / Degree

39 Minimum Domain / Degree

40 Minimum Domain / Degree

41 Minimum Domain / Degree

42 Weighted Degree Heuristic Constraints are weighted. Initially set to 1. During propagation, the weight w is incremented by 1 if the constraint fails. The weighted degree of a variable X i : w(x i ) = " c s.t. X i!x (c) w(c) Domain over weighted degree heuristic: Choose the variable X i with minimum dom(x i ) / w(x i ).

43 Value Ordering Heuristics Succeed-first principle: choose next the value most likely to be part of a solution. Approximating the number of solutions. Looking at the remaining domain sizes when a value is assigned to a variable.

44 Heavy Tail Behavior Given a collection of instances of a problem, we often observe a few exceptionally hard instances. These instances are rare, but take exceptionally longer to solve. Large impact on the mean runtime for a given set. As opposed to normal distributions, the mean does not stabilize when the size of the sample grows. When the sample grows, the mean runtime is skewed up. Heavy tail behavior!

45 Heavy Tail Behavior Not a characteristic of the instance! The same behavior is observed if we run several times the same instance while varying some parameter of the solver. For some combination instance + solver parameters, we get trapped into an exponential subtree.

46 Heavy Tail Behavior Randomization Add some randomized parameter in variable or value selection (for instance to break ties). Given the same random seed the solver will explore the same tree, however it will never explore two identical subproblems in the same way. Restarting After a given limit, for instance in number of explored nodes: restart from scratch. Randomization + restarts eliminates the huge variance in solver performance. And therefore reduces the mean runtime when a heavy tail behavior could be observed.

47 Problems with Depth-first Search The branches out of a node, ordered by a value ordering heuristic, are explored in left-to-right order, the left-most branch being the most promising. For many problems, heuristics are more accurate at deep nodes. - Often first decision is wrong. - Takes too long time to undo this decision. - Remember the puzzle example!

48 Problems with Depth-first Search Depth-first search: - puts tremendous burden on the heuristics early in the search and light burden deep in the search; - consequently mistakes made near the root of the tree can be costly to discover and undo. Best-first search strategy is of interest.

49 Limited Discrepancy Search A discrepancy is the case where the search does not follow the left-most branch out of a node. LDS Trusts the value ordering heuristic and gives priority to the left branches. Iteratively searches the tree by increasing number of discrepancies, preferring discrepancies that occur near the root of the tree. The search recovers from mistakes made early in the search.

50 Limited Discrepancy Search LDS 0 th iteration, explore the leftmost branch. 1 sth iteration, explore all left branches except 1 branch. 2 nd iteration, explore all left branches except 2 branches.