Introduction. 2D1380 Constraint Satisfaction Problems. Outline. Outline. Earlier lectures have considered search The goal is a final state

Transcription

1 CS definition 2D1380 Constraint Satisfaction roblems CS definition Earlier lectures have considered search The goal is a final state CSs CSs There was no internal structure to the problem or states States could be considered any data-structure (say, a java class) Strategies were developed to be problem specific. It would be desirable to have generic methods for solve problems. Kungl Tekniska Högskolan hic@kth.se One such type of problems is studied in this lecture September 6, CS definition 2 CS definition CS definition 2 CS definition CSs 3 4 CSs CSs 5 CSs 6 Summary 6 Summary

2 Constraint Satisfaction roblems CS Example: Graph coloring (cont.) CS definition CSs A CS problem is defined by: A set of variables: 1, 2,..., n A set of constraints: C 1, C 2,..., C m Each variable has a domain D i that is non-empty An solution is an assignment across i such that the constraints C j are satisfied. Some CS also have an objective function that must be maximized. CS definition CSs Solutions are assignments that satisfy all constraints: { WA = red, NT = green, Q = red, NSW = green, V = red, SA = blue, T = green } CS Example: Graph coloring Constraint Graph CS definition CSs Western Australia Northern Territory South Australia Tasmania Queensland New South Wales Victoria CS definition CSs Binary CS: each constraint relates at most two variables Constraint graph: nodes are variables, arcs show constraints WA NT SA V Q NSW Assign a color to each region WA, SA, NT, Q, NSW, V, T Neighbouring regions must have different colors C( i ) C( j ) iff j N( i ) General purpose CS algorithms exploit the graph structure Independent graph components are individual problems (Tasmania) T

3 Types of CSs Real world examples of CSs CS definition CSs Discrete variables (e.g.colors) Finite domains: size d O(d n ) Infinite domains: e.g., job scheduling, flights, routing,... Linear solvable constraints, non-linear undecidable Continuous variables St / End time: train schedules, Linear constraints olynomial complexity CS definition CSs Assignment problems Which crew on which flight (no over-time) Time-table problems Which plane to which destination? Hardware configuration roduction scheduling Floor planning for supermarkets Many real-world problems have continuous variables Types of constraints CS as a search problem CS definition CSs Unary constraints involve a single variable e.g., SA green Binary constraints involve pairs of variables e.g., SA WA N-ary/Higher order involves 3 or more variables For example used in crypto-analysis (see book) referential constraints (soft) Can be integrated into a cost metric for each variable Constrained optimization problem CS definition CSs States are defined by values assigned this far Initial state: empty assignment {} Successor function: assign value to yet unassigned variable that does not conflict with current assignments fail if no legal assignment possible Goal test: the problem is complete 1 Algorithms can be used for any CS! 2 Every solution appears at depth n use depth-first search 3 Complexity is n!d n (ups!)

4 CS 1 CS definition CSs 2 CS definition CSs CS definition CSs function -Search(csp) returns solution/failure return Recursive-({ }, csp) function Recursive-(assignment, csp) returns soln/failure if assignment is complete then return assignment var Select-Unassigned-Variable(Variables[csp], assignment, csp) for each value in Order-Domain-Values(var, assignment, csp) do if value is consistent with assignment given Constraints[csp] then add {var = value} to assignment result Recursive-(assignment, csp) if result failure then return result remove {var = value} from assignment return failure 6 Summary search example CS definition CSs Variable assignments are commutative, i.e.: [SA=blue then WA=red] same as [WA=red then SA=blue] One variable assignment to be considered in each step branch factor b = d d n leaves Search for CSs with single assigments - backtrack when fails CS definition CSs search is the uninformed search for CSs Can solve relatively complex problems such as n-queens, for n

5 Improving performance Degree heuristic CS definition CSs General purpose methods can give significant speed gains 1 Which variable should be assigned next? 2 What order should values be tested in? 3 Can inevitable failures be detected early? 4 Does problem structure provide hints/constraints? CS definition CSs Selection of nodes for initial assignment? Degree heuristic. Select the variable with the most constraints Minimum remaining values Least constraining value CS definition Minimum remaining values Choose the variable with the fewest legal values Significantly reduces the search in backtracking CS definition Given a variable: choose the least constraining value the value that rules out the fewest for remaining variables CSs CSs These heuristics allow handling of problems such as 1000 queens

6 Forward chaining Arc consistency CS definition CSs This far only single variables have been considered. Idea: Keep track of legal values for unassigned variables If a situation arrives with no legal values for a variable failure Terminate search when any variable has no legal values CS definition CSs Ensure that arc relations are consistens Y x > y for value x of there is allowed y As changes it neighbours must be rechecked Arc consistency detects failures earlier than forward checking Constraint propagation Arc Consistency Algorithms (Machworth 1977) CS definition CSs Consideration of constraints across variables How does assignment of one influence connected variables? A number of different approaches Arc-consistency - pairwise constriants k-consistency - k-wise constriants Node consistency (1-consistency) Resource constraints - (x+y < N) CS definition CSs function AC-3( csp) returns the CS, possibly with reduced domains inputs: csp, a binary CS with variables { 1, 2,..., n} local variables: queue, a queue of arcs, initially all the arcs in csp while queue is not empty do ( i, j ) Remove-First(queue) if Remove-Inconsistent-Values( i, j ) then for each k in Neighbors[ i ] do add ( k, i ) to queue function Remove-Inconsistent-Values( i, j ) returns true iff succeeds removed false for each x in Domain[ i ] do if no value y in Domain[ j ] allows (x,y) to satisfy the constraint i j then delete x from Domain[ i ]; removed true return removed

7 Minimum conflicts algorithm 1 CS definition CSs 2 CS definition CSs CS definition CSs function Min-Conflicts(csp, max-steps) returns a solution or failure inputs: csp, a constraint satisfaction problem max-steps, the number of steps allowed before giving up local variables: current, a complete assignment var, a variable value, a value for a variable current an initial complete assignment for csp for i = 1 to max-steps do var a randomly chosen, conflicted variable from Variables[csp] value the value v for var that minimizes Conflicts(var, v, current, csp) set var=value in current if current is a solution for csp then return current end return failure 6 Summary Local search for CSs Minimum conflict example - 8-queens CS definition CSs Assign a state to all variables Fix inconsistencies by random changes in a variable Use minimum conflict as a heuristic for value selection CS definition CSs

8 Comparison of search / solution strategies roblem Structure for CSs CS definition CSs roblem Backtrack BT+MRV Forward FC+MRV Min Con USA (>1000K) (>1000K) 2K n-queens (>40000K) 13500K (>40000K) 817K 4K Zebra 3859K 1K 35K 500 2K Random 1 415K 3K 26K 2K Random 2 942K 27K 77K 15K CS definition CSs How does one structure the problems to make it tractable? Can the problem be decomposed into subproblems? Does the graph can several independent components? Can the problem be organised as a tree structure? Tree problems have linear complexity (O(n)) Decomposition of problem into a tree structure Australia graph colouring problem CS definition CSs 1 2 CS definition 3 4 CS definition CSs WA NT SA V Q NSW 5 CSs T 6 Summary Two independent components

9 Tree-structured CSs Doing a tree conversion CS definition CSs CS definition CSs For nearly tree structured domains. Assign Variable, introduce constraints and apply tree algorithms Tree structured CSs can be solved in O(nd 2 ) compared to the general CS - O(d n ) Efficient for organisation of problems Algorithm for tree-structured CSs Decomposition into a tree structure CS definition Choose a root for the problem Order variables from root to leaves CS definition Design systems as small CSs within a tree structure NT NT Q WA SA SA CSs CSs SA Q NSW for j from n to 2 apply RemoveInconsistent (arent( j ), j ) For j from 1 to n, assign j consistently with arent( j ) SA NSW T V

10 1 CS definition CSs 2 CS definition 3 4 CS definition CSs THE END! 5 CSs 6 Summary Summary CS definition CSs CS a special type of problems - many real world applications Organised as a constraint graph is commonly used as a strategy A number of heuristics can be applied to speed-up search Local search is efficient for a number of problems The problem can often be decomposed into subproblems to speed up the overall search

11 2D1380 Adversarial Search - Zero-Sum Some problems are multi-agent (Lecture 1) Cooperative vs competitive interaction Competitive environments influences strategies for planning Typically such environments are termed games Initially we will consider zero-sum games Consideration of basic strategies in terms of search Kungl Tekniska Högskolan hic@kth.se September 7, 2005 The basis for adversarial search 1 2 in that include an element of chance 6 in games Computers considers possible lines of play (Babbage, 1846) Algorithms for perfect play (Zermelo, 1912; Von Neumann, 1944) Finite horizon, approximate evaluation (Zuze, 1945; Wiener, 1948; Shannon, 1950) First chess program (Turing, 1951) Machines learn to improve accuracy (Samuel, ) runing to allow deeper search (McChy, 1956) 7 Summary

12 Types of The components of a game Deterministic Chance erfect Chess, Checkers Backgammon Information go, othello monopoly Imperfect battleships bridge,poker Information tic-tac-toe scrabble, paper-scissor-... Two players MA and MIN An initial state (the st situation) A successor function (move, state) A terminal state - when is the game over? A utility function: (+1, -1, 0) typically The progression can be modelled as a (game-) tree Game Tree (2-player, deterministic, turns) 1 MA () 2 in 3 MIN (O) 4 MA () O O O... 5 that include an element of chance 6 in games MIN (O) O O O Summary TERMINAL Utility O O O O O O O O O O

13 Strategies The Minimax Algorithm In normal search the sequence is optimized for all actions In a game the next move by the opponent cannot aways be predicted. Some assumption can be put forward Our agents are assumed to be rational: Self-interested (maximize ones own profit) Will always choose the best possible option (could be the worst for the opponent) Analysis of the full game tree can be demanding or impossible function Minimax-Decision(state) returns an action inputs: state, current state in game return the a in Actions(state) maximizing Min-Value(Result(a, state)) function Max-Value(state) returns a utility value if Terminal-Test(state) then return Utility(state) v for a, s in Successors(state) do v Max(v, Min-Value(s)) return v function Min-Value(state) returns a utility value if Terminal-Test(state) then return Utility(state) v for a, s in Successors(state) do v Min(v, Max-Value(s)) return v The minimax strategy roperties of minimax? Consider a 2-ply game The strategy of best achievable payoff against best play by opponent MA has actions a 1, a 2, a 3 and MIN has actions b 1, b 2, b 3. The best strategy? MA MIN 3 3 B 2 C 2 D A a 1 a 2 a 3 Complete Only if tree is finite For special games like chess specific rules can be applied Yes, against an optimal opponent Time complexity - O(b m ) Space Complexity - O(bm) - depth first recursion For chess b 35, m 100 for reasonable games Exact solution with minimax is not realistic b 1 b 2 b 3 c 1 c 2 c 3 d 1 d 2 d

14 Re-analysis of the 2-ply game 1 (a) [!", +"] A (b) [!", +"] [!", 3] B [!", 3] B A 2 in (c) [3, +"] [3, 3] B A (d) [3, 3] B [3, +"] [!", 2] A C 5 that include an element of chance 6 in games (e) [3, 14] A (f) [3, 3] A 7 Summary [3, 3] [!", 2] [!", 14] B C D [3, 3] [!", 2] [2, 2] B C D What is this α and β thing? layer A problem with minimax is the exponential search/recursion Do we really need to analyse all nodes? Desirable to cut off unrealistic plays early Opponent layer Opponent m n α is the best value found off current path If m is better than n we will never get to play n β is defined similarly for MIN

15 The algorithm 1 function Alpha-Beta-Decision(state) returns an action return the a in Actions(state) maximizing Min-Value(Result(a, state)) function Max-Value(state, α, β) returns a utility value inputs: state, current state in game α, the value of the best alternative for max along the path to state β, the value of the best alternative for min along the path to state if Terminal-Test(state) then return Utility(state) v for a, s in Successors(state) do v Max(v, Min-Value(s, α, β)) if v β then return v α Max(α, v) return v function Min-Value(state, α, β) returns a utility value same as Max-Value but with roles of α, β reversed 2 in that include an element of chance 6 in games 7 Summary roperties of α beta? Handling of resource limitations? Completeness - pruning does not affect the final result! Complexity - depends heavily on order of evaluation. With perfect ordering O(b m/2 ) Unfortunately is still a large number! Standard approaches Cutoff-Test rather than Terminal-Test i.e. depth limited search Eval rather than Utility, i.e. an evaluation function that estimate the value of a picular strategy/position Suppose we have 60 seconds and can explore 10 4 nodes / second nodes per move 35 7/2 α β reaches 7 moves ahead which is a pretty decent chess programme

16 Evaluation functions Deterministic games in the real world Include both value so far + estimate of chance of winning Backgammon Go players refuse to play against computers that are too good (a) White to move (b) White to move In chess the evaluation is a linear combination n Eval(s) = w 1 f 1 (s) + w 2 f 2 (s) w n f n (s) = w i f i (s) i=1 players refuse to play against computers as they are too poor. b > 300 so most computers cannot perform reasonable search I.e. f 1 (s) = #white queens #black queens Deterministic games in the real world Checkers: Chess: Chinook ended 40-years of reign of human world champion Marion Wesley in Used an endgame database definign perfect play for positions 8 or few pieces on the board positions DeepBlue defeated human world champion Gary Kasparov in a six game match DeepBlue searches 200 million positions / second. Upto depth 40 in search. 1 2 in that include an element of chance 6 in games 7 Summary

17 Nondeterministic games - Backgammon Algorithm for nondeterministic games Expectiminimax gives perfect play Just like Minimax, except we must also handle chance nodes:... if state is a Max node then return the highest ExpectiMinimax-Value of Successors(state) if state is a Min node then return the lowest ExpectiMinimax-Value of Successors(state) if state is a chance node then return average of ExpectiMinimax-Value of Successors(state) Nondeterministic games in general of imperfect information In nondeterministic games an element of chance is introduced could as simple as a coin-flip E.g., card games, where opponent s initial cards are unknown Typically one can compute a probability for each possible deal Seems just like having one big dice roll at the beginning of the game Idea compute the minimax value of each action in each deal, choose the action with highest expected value over all deals GIB, current best bridge program, approximates this idea by 1 generating 100 deals consistent with bidding information 2 picking the action that wins most tricks on average

18 1 1 2 in 3 2 in that include an element of chance 6 in games 5 that include an element of chance 6 in games 7 Summary 7 Summary How good are computer today? Summary Chess computers as good as humans! Checkers can check up to 262 moves ahead! Othello Humans are no match for computers Backgammon Self-learning today top 3 in the world Go today as good as a weak amateur Bridge won world championship yr 2000! Competitive environment Only considered simple cases Four components: initial state, actions, terminal test, and utility function erfect information allow use of minimax to optimize search Use of evaluation functions to allow depth limited search With a component of chance the expected value can be used similar to minimax (expectimax) Imperfect information requires use of belief state. Today computers are as good as humans in many different game situations

19 THE END!