Constructive Search Algorithms

Transcription

1 Constructive Search Algorithms! Introduction Historically the major search method for CSPs Reference: S.W.Golomb & L.D.Baumert (1965) Backtrack Programming, JACM 12: Extended for Intelligent Backtracking techniques Plus Constraint Propagation techniques Used in CSPs, tree-search and Constraint Logic Programming (CLP)! Hybrid Algorithms for CSPS Reference: P. Prosser (1993) Hybrid Algorithms for the Constraint Satisfaction Problem, Computational Intelligence 9(3), pp Paper unifies important constructive search + constraint propagation algorithms in a common framework Also defines 4 new variations September 16, 2007 Copyright 2003 by Bill Havens 1 of 24

2 Assumptions in paper 1. Tree search using a stack-based architecture (eg- Prolog, CLP) Efficient but limiting search architecture Elegant implementation in the Warren Abstract Machine (WAM) for Prolog Reference: H. Ait-Kaci, Warren s Abstract Machine - A tutorial reconstruction, MIT Press, Variables instantiated in some arbitrary (fixed) order Variable ordering is an important research issue 3. Strict boundary between past/future variables in ordering (cf- dynamic backtracking) Past Vars = variables already assigned values in tree Future Vars = variables currently unassigned Current Var = variable being instantiated or backtracked 4. All constraints are binary (unnecessary) September 16, 2007 Copyright 2003 by Bill Havens 2 of 24

3 Family of Backtrack Algorithms Horizontal axis = backtracking dimension Vertical axis = constraint propagation dimension 9 possible algorithms in family Caveat: we will cover a subset here September 16, 2007 Copyright 2003 by Bill Havens 3 of 24

4 Top-Level Search Algorithm Given a set of variables v[] of size n, find a first solution such that every constraint c[i,j] in C holds on variables v[i] and v[j]. procedure bcssp(n, status) Note unusal iterative coding style (from Nadel[89]) September 16, 2007 Copyright 2003 by Bill Havens 4 of 24

5 Procedure label() Procedure label(i, consistent) finds an assignment for v[i]. on success, sets reference variable consistent = true and returns the next var, i+1, to instantiate. on failure, sets consistent = false and returns var i (for backtracking)! Cases on label() return values 1. consistent = true and 1< i < n bcssp() will iterate on labelling the next variable i 2. consistent = true and i = n+1 bcssp() will terminate with status = solution 3. consistent = false bcssp() will call procedure unlabel(i, consistent) on var i September 16, 2007 Copyright 2003 by Bill Havens 5 of 24

6 Procedure unlabel() procedure unlabel(i, consistent) performs backtracking on some variable h < i in the static ordering. Resets every domain D j for variable v[j] such that h < j " i Then the current value of v[h] is removed from domain D h Sets consistent = true iff D h is not empty and returns var h Otherwise, D h = {} then consistent = false and returns var h! Cases on unlabel() return values 1. consistent = true and 1 " h " i bcssp() will iterate on labelling variable h 2. consistent = false and 1 " h < i bcssp() will call unlabel() again on the new backtrack point h Deep backtracks recursively 3. consistent = false and h = 0 bcssp() will terminate with status = impossible (no backtrack points left) September 16, 2007 Copyright 2003 by Bill Havens 6 of 24

7 Parmeterizing the Algorithms Forward moves use method x-label for various instances of x Backward moves use method x-unlabel for various instances of x! Traditional chronological backtracking (BT) Earliest and simplest constructive tree search algorithm Still used in Prolog!!! September 16, 2007 Copyright 2003 by Bill Havens 7 of 24

8 procedure bt-label() Consistency checking is performed ONLY backwards against previously instantiated variables When assigning v[i] only predicates v[i,h] for h < i are checked On failure: 1. Algorithm shallow backtracks by selecting another value in D i if possbile 2. Otherwise deep backtracks by return consistent = false and current var i to bcssp() September 16, 2007 Copyright 2003 by Bill Havens 8 of 24

9 procedure bt-unlabel() Definition: the culprit = the variable h selected for backtracking For BT, the culprit is always the previous variable, h = i - 1 The domain D i of var i is reset to full domain (stack architecture again) Current value of v[h] removed from D h consistent = true iff Dh # {} September 16, 2007 Copyright 2003 by Bill Havens 9 of 24

10 BackJumping Algorithm Major problem with BT Culprit may NOT be the actual cause of inconsistency Example CSP C 13 X 1 C 12 C 35 X 3 X 2 C 24 Failure X 5 X 4 Why backtrack from X5 to X4? X4 will choose a new assignment BUT X5 will inevitably fail again? Why? Actual culprit = X3 Called thrashing [Mackworth77] How do you know the real culprit? September 16, 2007 Copyright 2003 by Bill Havens 10 of 24

11 BackJumping method Attempts to backtrack over irrelevant variables Identifies culprit = greatest var h < i such that there exists c[h,i] in C which removes a value from domain Di Method: define a variable maxcheck[i] for each var i which contains the index of the greatest variable constraining var i (as above) maxcheck updated during consistency checking otherwise same method as bt-label() September 16, 2007 Copyright 2003 by Bill Havens 11 of 24

12 procedure bj-unlabel() maxcheck provides culprit for bj-label() resets domains D j, for j = h+1,..., i resets maxcheck[i] = 0! Analysis BJ is an approximation to the real culprit Consider example Prosser figure 3 Suppose no constraint c[1,2] in C What happens when v[2] must backtrack? Thus BJ backtracks chronologically September 16, 2007 Copyright 2003 by Bill Havens 12 of 24

13 Induced Graphs Need to understand the concept of the induced graph Reference: Bayardo&Mirankar (1996) A Complexity Analysis of Space-Bounded Learning Algorithms for the CSP, proc. AAAI-96, pp Explicit representation of culprit dependencies Computed indirectly by CBJ and other Intelligent Backtracking methods Only approximated by BJ Definition: given a constraint graph G = (X, C) plus a total ordering of the variables in X, the induced graph G = (X, C ) contains the following set of edges C : 1. all edges (i,j) $ C are also edges in C 2. if (i,j) $ C and (h,j) $ C such that i<j and h<j then edge (i,h) $ C 3. no other edges are in C September 16, 2007 Copyright 2003 by Bill Havens 13 of 24

14 Examples of Induced Graphs X 1 X 2 X 1 X 2 X 3 X 3 X 4 X 4 X 5 X 5 Directed arcs given by variable ordering New arcs added whenever a node has 2 parents Pathological example Who is the culprit for variable X 2? Why? 5 September 16, 2007 Copyright 2003 by Bill Havens 14 of 24

15 Conflict-Directed Backjumping Method (CBJ) Definition: the conflict set of a variable X i are those past vars which failed consistency with X i. Culprit Identification Rule: the culprit of the conflict set for var X i is Xj where j % h for all X h in the conflict set. Required for complete search CBJ maintains a conflict set conf-set[i] for each variable v[i]. Initially all conf-set[i] = {0} Whenever a constraint c(i,h), h < i, fails then index h is added to confset[i] Thus the conflict set represents those previous variables whose values are in conflict with v[i]. If v[i] fails (via D i = {}) then h = max(conflict-set[i]) according to the culprit identification rule. Plus remaining vars in conflict-set[i] are transferred to conflict-set[h] State of v[i] represented by pair: current-domain[i] and conflict-set[i] September 16, 2007 Copyright 2003 by Bill Havens 15 of 24

16 procedure cbj-label() Same as bj-label() except for lines 9-13 Note: algorithm discovers inconsistency AFTER for-loop exits (yeech!) conf-set[i] maintained as a set of candidate culprits (no duplicates) Why is 0 in the conflict set? September 16, 2007 Copyright 2003 by Bill Havens 16 of 24

17 procedure cbj-unlabel() Again similar to bj-unlabel() Line 3 identifies culprit h for backtracking Line 4 updates conf-set[h] = conf-set[h] & conf-set[i] - {h} September 16, 2007 Copyright 2003 by Bill Havens 17 of 24

18 Analysis of CBJ Smarter than either BT or BJ Computes culprit on failure exactly Algorithm is complete Overhead is still low (polynomial pain for exponential gain) Backtrack method of choice for stack-based (tree) search! Comparison with Induced Graph Note that CBJ is computing the induced graph indirectly See line 4 of cbj-unlabel() But only if a constraint actually removes a value from the current domain ' i. Basis for more sophisticated Intelligent Backtracking scheme (egdynamic backtracking) September 16, 2007 Copyright 2003 by Bill Havens 18 of 24

19 Forward Checking (FC)! Overview Simplest form of constraint propagation in backtrack search Note: we will not cover Backmarking (BM) here Reference: R. Haralick & G. Elliott (1980) Increasing tree search efficiency for CSPs, Aritificial Intelligence 14: Prunes the live domains ' j of future variables v[j] given a particular assignment to variable v[i] for every v[j] connected by a constraint c[i,j]. Thus every future variable v[j] will be consistent with all past variables (but perhaps not with each other) Realizes the basic Arc Consistency condition between pairs of past and future variables. Powerful technique - on domain wipeout of ' j then assignment v[i] is known inconsistent. Original method uses BT as the search control structure September 16, 2007 Copyright 2003 by Bill Havens 19 of 24

20 Implementing Forward Checking First we require the machinery for recording domain reductions Called from v[i] to prune values from the current domain ' j of v[j], i<j and there exists a c[i,j] in C Returns true iff ' j # {} Remember deleted values in reduction Line 9 updates ' j Lines maintains sufficient info for undoing assignment to v[i] September 16, 2007 Copyright 2003 by Bill Havens 20 of 24

21 procedure undo-reduction(i) Undoes the domain reductions on backtracking assignment v[i] Line 5 removes the trail stack entry reduction from last variable to prune ' i Line 6 restores ' i given reduction Ignore line 7 temporarily September 16, 2007 Copyright 2003 by Bill Havens 21 of 24

22 procedure fc-label(i, consistent) Main procedure for FC backtracking Attempts to instantiate v[i] while pruning the lives domains of all future vars v[j] sharing a constraint c[i,j] On domain wipeout, then reductions are reversed and a new assignment for v[i] is attempted. Line 8 calls check-forward(i,j) to do FC Line 12 undoes reductions if assignment is not consistent September 16, 2007 Copyright 2003 by Bill Havens 22 of 24

23 procedure fc-unlabel(i, consistent) Again analogous to bt-unlabel() except Line 4 undoes the reductions to the live domains of future variables Line 5 restores the current live domain of v[i] before backtracking (from only those last reductions on the trail stack) Caveat: undo machinery here is very opaque September 16, 2007 Copyright 2003 by Bill Havens 23 of 24

24 Example: Thrashing in FC FC does NOT avoid thrashing Why? Because control structure is BT v[5] is current variable v[3] has pruned values from domain fo v[6] v[6] has domain wipeout v[5] backtracks to v[4] by mistake. The real culprit = v[3] September 16, 2007 Copyright 2003 by Bill Havens 24 of 24