Binary Trees, Constraint Satisfaction Problems, and Depth First Search with Backtracking

Transcription

1 Binary Trees, Constraint Satisfaction Problems, and Depth First Search with Backtracking 1. Binary Trees and Depth-First Search I hope you remember from CSE205 what a basic tree data structure is. There are many different types of tree data struc - tures, e.g., you studied binary trees and binary search trees in CSE205. Depending on the type of tree being used, there are different ways of representing the tree in a program. A common technique for implementing a binary tree is to create a tree node structure for each node of the binary tree, tree_node is structure(data, left_child, right_child) where tree_node stores three references to: (1) the data stored in the node, or a null reference if the node does not store data; (2) the left child of the node, or a null reference if the node does not have a left child; (3) the right child of the node, or a null reference if the node does not have a right child. If required, a reference to the parent node can also be stored, tree_node is structure(data, parent, left_child, right_child) The parent node would be required in situations where we wish to traverse a path starting at a node lower in the tree and ending at a node higher up in the tree. The binary tree then consists of zero (in the case of an empty tree) or more tree_node objects, with one special node designated as the root node or simply the root. When implemented in a programming language, an variable T would be a reference to the root tree_node object. Given a binary tree T, we can discuss tree traversals, i.e., methods for visiting the various nodes of T in a systematic manner. The four most common binary tree traversals are: prefix traversal, infix traversal, postfix traversal, and breadthfirst traversal. For example, consider tree T 1 shown below, 7 \ 3 8 \ \ 2 For prefix, infix, and postfix traversals, the prefix of each word specifies when (during the traversal) we visit the root node of a subtree. In a prefix traversal, we visit the root node first (pre means before), then the left child, and then the right child. Of course, this happens recursively, so when we begin traversing the subtree rooted at the left child, we first visit the left child node, then we traverse the subtree rooted at the left child of the left child, and finally, we traverse the subtree rooted at the right child of the left child. For T 1 we will visit the nodes in this order: 7, 3, 2, 5, 8, 4, 2. In a postfix traversal of a subtree rooted at node r, we first traverse the left subtree of r, then the right subtree of r, and then we visit node r (post means after). For T 1 we will visit the nodes in this order: 2, 5, 3, 2, 4, 8, 7. In an infix traversal of a subtree rooted at node r, we first traverse the left subtree of r, then we visit r, and then we traverse the right subtree of r. For T 1 we will visit the nodes in this order: 2, 3, 5, 7, 4, 2, 8. Prefix, infix, and postfix traversals are examples of depth-first traversals, because we follow complete paths from upperlevel nodes down to lowest-level nodes (leaves), before we then move back up and visit upper-level nodes, and then back down to lower-level nodes, and so on. Given a binary tree, a common operation is to search the tree to locate a particular datum. A prefix, infix, or postfix traversal during a search is referred to as a depth-first search. The fourth traversal method is breadth-first, where we visit all of the nodes at level l (in a left-to-right manner) before beginning to traverse the nodes at level l + 1. For T 1 we will visit the node in this order: 7, 3, 8, 2, 5, 4, 2. Searching a tree using a breadth-first traversal is referred to as breadth-first search. Breadth-first traversal is implemented using a queue to store the nodes to be visited. Depth-first traversals do not require a queue. (c) Kevin R. Burger :: Computer Science & Engineering :: Arizona State University Page 1

2 2. Constraint Satisfaction Problems A constraint satisfaction problem (CSP) is one where values from one or more input domains I 1, I 2, I 3,..., are assigned to one or more problem variables, v 1, v 2, v 3,..., subject to one or more constraints C 1, C 2, C 3,..., (constraints impose limits on the values from the domains that may be assigned to the variables). CSP's in which the values are discrete and the input domains finite are often easier to solve than CSP's in which the values are continuous and one or more of the input domains is infinite. We will focus on CSP's involving discrete values and finite domains. For example, suppose in problem csp, that we have two problem variables v 1 and v 2. Let both variables have the same input domain, which is I 1 = I 2 = N 10 (the set of natural numbers 1, 2, 3,..., 10). Let the problem csp be: find all solutions to v v < 5 such that v 1 is odd (a constraint C 1 on the values from I that can be assigned to v 1 ) and v 2 > v 1 (another constraint C 2 on the values that can be assigned to v 2 once a value for v 1 is assigned; or conversely, a constraint on the value that can be assigned to v 1 once a value for v 2 has been assigned). A third constraint C 3 is that v v < 5 which we can also write as C 3 : v v 2 2 < 4. A state of the problem is defined to be the assignment of values from the input domains to some or all of the problem variables, e.g., S 1 : {v 1 = 1} is one state (note that v 2 has not been assigned), and S 2 : {v 1 = 1, v 2 = 2} is another state. A complete assignment (or we could say complete state) is a state where each problem variable is assigned, e.g., S 1 is an incomplete assignment and S 2 is a complete assignment. An assignment (or state) is consistent or legal if the assignments to the problem variables do not violate any of the constraints, e.g., S 3 : {v 1 = 4, v 2 = 2} is not a legal state (or is inconsistent) because v 1 is not odd (violating C 1 ), v 2 is not greater than v 1 (violating C 2 ), and v v 2 4 (violating C 3 ). A solution to a CSP is a complete assignment that does not violate any of the constraints, i.e., a solution is a complete and consistent state. CSP's can be formulated as a search problem: we have a finite search space consisting of complete states (all problem variables are assigned a value), some of which are solutions (the states that are consistent) and some of which are not. The CSP consists of, An initial state: S 1 : {} where all variables are unassigned. A successor function: a new state is generated from the previous state by assigning a value from I n to an unassigned variable v n, without violating any constraints. A goal test: all variables are assigned values, forming a complete assignment or complete state, and the state is checked to see if it is a solution to csp (it will be a solution if the state is consistent). As the search space is searched, we can represent the set of generated states by a tree which will be of maximum height n, since there are n variables to be assigned. The leaves of the tree (complete assignments) will be checked to see if they are solutions to csp. For our example problem, the initial state is represented by S 1 and the successor function will create a new state S 2 from S 1 by assigning a value from I 1 to unassigned problem variable v 1, S 1 : {} S 2 : {v 1 = 1} S 2 is not a complete assignment, so next, the successor function assigns a value from I 2 to unassigned variable v 2. The first value of I 2 which has not already been assigned to v 2 is 1, so we make the assignment, S 1 : {} S 2 : {v 1 = 1} S 3 : {v 1 = 1, v 2 = 1} (c) Kevin R. Burger :: Computer Science & Engineering :: Arizona State University Page 2

3 We now check to see if the assignment of 1 to v 2 violates a constraint: C 2 is violated because v 2 > v 1 is false; C 3 is not violated because < 4 is true. Since a constraint was violated, S 3 (although it is a complete state) is inconsistent, which means, of course, that S 3 is not a solution. At this time, we backtrack to the previous state S 2, S 1 : {} S 2 : {v 1 = 1} and the successor function assigns the next value from I 2 (which is 2) to v 2, S 1 : {} S 2 : {v 1 = 1} S 3 : {v 1 = 1, v 2 = 2} We now check to see if the assignment of 2 to v 2 violates a constraint: C 2 is not violated because v 2 > v 1 is true and C 3 is not violated because < 4 is true. The goal test identifies S 3 as a solution because S 3 is a complete and consistent state. If our problem was to find any solution, then we can stop now, but often times we are interested in finding all solutions. In the latter case, we would again backtrack to the previous state, S 1 : {} S 2 : {v 1 = 1} and the successor function would then assign the next value from I 2 (which is 3) to v 2, S 1 : {} S 2 : {v 1 = 1} S 3 : {v 1 = 1, v 2 = 3} We now check to see if the assignment of 3 to v 2 violates a constraint: C 2 is not violated because v 2 > v 1 is true and C 3 is not violated because < 4 is true. The goal test identifies S 3 as a second solution because S 3 is a complete and consistent state. Next, the algorithm would backtrack to the previous state, the successor function would assign the next value from I 2 to v 2, it would check for any constraint violations and if there are none, it would mark {v 1 = 1, v 2 = 4} as a solution. This process continues until {v 1 = 1, v 2 = 10} is identified as a solution, but this time, when we backtrack to make the next assignment to v 2, there is no assignment to be made because all values from I 2 have been assigned to v 2. So, we backtrack again back to the initial state, S 1 : {} where the successor function will now assign the next value from I 1 (2) to v 1, S 1 : {} S 2 : {v 1 = 2} and the process of assigning 1, 2, 3,..., 10 to v 2 will commence once again. This algorithm performs a depth-first search of the search space and is known as depth-first search with backtracking. DFS with backtracking is one of many methods for solving CSP's. The general algorithm follows. (c) Kevin R. Burger :: Computer Science & Engineering :: Arizona State University Page 3

4 function solve_csp (csp) returns either the solution(s) to the problem or no_solution notes: csp is the data structure formulation of the constraint satisfaction problem. We assume csp is a structure type which includes these data members: (1) csp.n is the number of problem variables. (2) The CSP has one or more problem variables (csp.v 1, csp.v 2, csp.v 3,..., csp.v n ) and we need to keep track of which variables have been assigned to and which have not. We can let csp.unassigned_vars and csp.assigned_vars be the two lists. Note that csp.assigned_vars is initialized to the empty list and csp.unassigned_vars must be initialized to [csp.v 1, csp.v 2, csp.v 3,..., csp.v n ]. (3) Each variable has an input domain of values that can be assigned to the variable, and we need to keep track of which values have been assigned and which have not. We can let csp.v x.assigned_vals and csp.v x.unassigned_vals be the two lists for variable csp.v x;. Note that each of csp.v x.assigned_vals is initialized to the empty list and each of csp.v x.unassigned_vals must be initialized to the values in the input domain for the variable. (4) csp.constraints = {C 1, C 2, C 3,..., C c } is the finite set of c constraints, of which there may be more than, or fewer than, the number of variables. Note that in practice, constraints are often not stored as data members, but rather, are actually implemented as code which checks to see if the constraints are violated. (5) The algorithm operates by selecting a variable from the list of unassigned variables to be assigned each of the values in the selected variable's list of unassigned values. Let csp.curr_var represent the selected variable. (6) The algorithm operates by starting with an initial state (often, no problem variables have yet been assigned) and making assignments to create new states. Let csp.assignment be the current state. This function is simple. It initializes some of the csp structure data members and then calls dfs_backtrack(csp) which performs the depth-first search with backtracking. We initialize csp.assignment so the so the initial state is {}, i.e., no problem variables have yet been assigned. csp.n number-of-problem-variables -- csp.n will vary from problem to problem csp.assigned_vars [] -- Initially, no variables have been assigned values csp.unassigned_vars [list-of problem-vars] -- Initially, all variables are unassigned for each var v x csp.unassignd_vars do -- Initialize the lists of values for each problem variable csp.v x.assigned_vals [] -- Initially empty since no variables have been assigned values csp.v x.unassigned_vals [list-of-values] -- Initialized to be all of the values in the input domain of var v x csp.curr_var nothing -- The current variable will be selected in dfs_backtrack() csp.assignment {} -- The initial assignment (state) is empty when result dfs_backtrack (csp) -- call dfs_backtrack() which returns success or fail = success then return csp.solution -- dfs_backtrack() stores the solution in csp.solution = fail then return no_solution function dfs_backtrack (csp) returns either success (when a solution is found) or fail (no solution found) notes: This version of the algorithm finds and returns only the first solution that is discovered. How to modify the algorithm to find and return all solutions is discussed below. -- dfs_backtrack() is a recursive search. We do not actually build the tree and then search it; rather, we build small -- parts of the tree on the fly during the search. The base case occurs when csp reflects a complete assignment. Since -- this function is never called when a constraint is violated (see below), the assignment will be consistent, meaning -- the current csp assignment is a solution. In this case, we return success indicating we found a solution. when csp.assignment is complete then csp.solution csp.assignment return success -- We reach this point if we do not have a complete assignment. The search process starts by choosing one of the -- unassigned variables to be the next variable that is assigned a value from the variable's input domain. For this, -- we can assume the existence of a select_curr_var(csp) function which selects one of the unassigned variables -- (see below). Upon return, csp.curr_var will be the selected variable. select_curr_variable(csp) (c) Kevin R. Burger :: Computer Science & Engineering :: Arizona State University Page 4

5 -- We now must try to assign every value from csp.curr_var's list of unassigned values to csp.curr_var. Some or all -- of these assignments may violate constraints and when a constraint is violated, we must undo the assignment and -- continue the loop with the next value. In situations where a constraint is not violated, it means we may be on to -- a solution, so we recursively call the function to assign values to the remaining unassigned variables. for value csp.curr_var.unassigned_vals that has not yet been assigned do -- Make the assignment to create the new state. Next, check to see if any constraints were violated. csp.curr_var value -- note: we assume this modifies csp.assignment (the current state; creates successor state) when csp.assignment violates a constraint then -- We must undo the assignment to revert back to the previous assignment (state) and then continue -- trying the remaining values in csp.curr_var's list of unassigned values. csp.curr_var nothing -- note: we assume this modifies csp.assignment, reverting to the prior state continue -- looping otherwise -- No constraints were violated in making the assignment. We are now ready to make the recursive call -- which will select the next unassigned variable for assignment. However, before doing that, we must -- move value from csp.curr_var's list of unassigned values to its assigned values list. csp.curr_var.unassigned_vals.remove(value) csp.curr_var.assigned_values.add(value) result dfs_backtrack(csp) -- When dfs_backtrack() returns success, the current state (in csp.assignment) was a solution. Since -- this algorithm only finds one solution, at this time we just return success (note: csp.solution was -- written with the solution in the previous recursive call). We will continue to return success all the -- way up the calling tree where we finally return success back to solve_csp(). when result = success then return success -- On the other hand, when dfs_backtrack() returns fail, it means that we reached an inconsistent -- state, and this means that the value we assigned above to csp.curr_var will always lead to a non- -- solution. Consequently, we must undo the assignment of value to csp.curr_var, move value back -- from csp.curr_var's assigned values list to its unassigned values list, and then continue the loop -- to assign the remaining values. = fail then csp.curr_var nothing -- note: we assume this modifies csp.assignment, reverts to prior state csp.curr_var.assigned_values.remove(value) csp.curr_var.unassigned_vals.add(value) -- When we drop out of the for loop above, it means that we tried assigning every value from csp.curr_var's list -- of unassigned values to csp.curr_var and none of them led to a solution (we would not be here if it did). -- Consequently, we backtrack (by returning from this recursive function call), returning fail to indicate that no -- solution was found. However, before doing that we must move csp.curr_var back from the list of assigned -- variables to the list of unassigned variables. Note that csp.curr_var was set to nothing above. csp.assigned_vars.remove(csp.curr_var) csp.unassigned_vars.add(csp.curr_var) return fail (c) Kevin R. Burger :: Computer Science & Engineering :: Arizona State University Page 5

6 select_curr_var() can select any unassigned variable in csp.unassigned_vars. In a simple backtracking algorithm, we can assume that csp.unassigned_var is ordered and we can simply select the first unassigned variable in order. function select_curr_var (csp) -- csp maintains two lists of variables: a list of variables which have been assigned values and a list of variables which -- have not yet been assigned a value. The simplest selection function is to simple select and return the first variable -- from the list of unassigned variables. Note that we need to move the selected variable from the list of unassigned -- variables to the list of assigned variables. csp.curr_var csp.unassigned_vars 0 csp.assigned_vars.add(csp.curr_var) csp.unassigned_vars.remove(csp.curr_var) If the CSP has a solution, solve_csp() and dfs_backtrack() are guaranteed to find it, however long it may take. The basic algorithm may be altered with various techniques which are designed to speed up the search. For example, one method is to choose for assignment the variable which has the fewest number of legal assignments, i.e., choose the variable which leads to the most constraints being violated. This strategy is called the minimum remaining values heuristic and the hope is that this will lead more quickly to incomplete states during the depth-first search. To implement the MRV heuristic, we modify select_curr_var(), function select_curr_variable (csp) -- This is reasonably high-level pseudocode. How we determine which variable has the fewest number of legal -- assignments would need to be expanded upon. csp.curr_var the variable from csp.unassigned_vars that has the fewest number of legal assignments csp.assigned_vars.add(csp.curr_var) csp.unassigned_vars.remove(csp.curr_var) Other enhancements can be found in [3] or any good artificial intelligence textbook. This version of the algorithm will find and return only the first solution it discovers. If it is known that multiple solutions exist, then relatively minor changes can be made to make the algorithm return all solutions. Modifications are underlined. function solve_csp (csp) returns a list of solutions to the problem or no_solution csp.n number-of-problem-variables -- csp.n will vary from problem to problem csp.assigned_vars [] -- Initially, no variables have been assigned values csp.unassigned_vars [list-of problem-vars] -- Initially, all variables are unassigned for each var v x csp.unassignd_vars do -- Initialize the lists of values for each problem variable csp.v x.assigned_vals [] -- Initially empty since no variables have been assigned values csp.v x.unassigned_vals [list-of-values] -- Initialized to be all of the values in the input domain of var v x csp.curr_var nothing -- The current variable will be selected in dfs_backtrack() csp.assignment {} -- The initial assignment (state) is empty csp.solutions [] -- Initially, no solutions have been found dfs_backtrack (csp) -- Call dfs_backtrack() which no longer returns a value when csp.solutions -- If csp.solutions is empty, the problem has no solutions. is nonempty then return csp.solutions is empty then return no_solution function dfs_backtrack (csp) note: dfs_backtrack() no longer needs to return a value. When we find a solution the base case will be triggered, and we will add the solution to the list of solutions, csp.solutions. After that, we just return, undo the current assignment, reverting back to the prior state, and then continue searching for more solutions. (c) Kevin R. Burger :: Computer Science & Engineering :: Arizona State University Page 6

7 when csp.assignment is complete then csp.solutions.add(csp.assignment) return select_curr_variable(csp) for value csp.curr_var.unassigned_vals that has not yet been assigned do csp.curr_var value when csp.assignment violates a constraint then csp.curr_var nothing continue -- looping otherwise csp.curr_var.unassigned_vals.remove(value) csp.curr_var.assigned_values.add(value) dfs_backtrack(csp) -- Now, when dfs_backtrack() returns after a solution is found, we do not want to return. Basically, we -- just need to continue the search at the state we were in prior to creating the current (solution) state. -- To revert back to the prior state, we undo the assignment of value to csp.curr_var and continue -- the loop. Look back at dfs_backtrack() on p. 5 and you should note that what we must do when we -- succeed is the same thing we must do when dfs_backtrack() returns false. csp.curr_var nothing csp.curr_var.unassigned_vals.remove(value) csp.curr_var.assigned_values.add(value) csp.assigned_vars.remove(csp.curr_var) csp.unassigned_vars.add(csp.curr_var) 3. Example CSP: Solving Futoshiki Puzzles Futoshiki is a popular puzzle that involves placing digits the digits 1 through n into an n n grid (n can vary from 4 to 9) such that the digits in each row and column are unique [4, 5]. Some grid elements may already be initialized with the correct digit for that element. Additionally, some pairs of grid elements have inequality constraints that must be satisfied. For example, in the simple 4 4 puzzle below, the second digit on the first row must be less than the third digit of that row. Each Futoshiki puzzle has only one solution. Remember, a CSP consists of: (1) a set of variables; (2) a set of input domains (values) for each variable; and (3) a set of constraints. A Futoshiki puzzle can be formulated as a CSP, with the set of n 2 variables being the n 2 grid elements. The input domain for each variable is [1, 4]. There are three categories of constraints: (1) n row constraints which specify that the digits in each row must all be different; (2) n column constraints which specify that the digits in each column must all be different; (3) zero or more inequality constraints involving pairs of variables. Let csp be a data structure which stores the relevant for the problem. Let csp.n be the size of the grid and let csp.grid be the two dimensional array of digits. Then, in general for an n n puzzle, the row and column constraints are, (c) Kevin R. Burger :: Computer Science & Engineering :: Arizona State University Page 7

8 C row:0 : csp.grid 0,0 = csp.grid 0,1 =... = csp.grid 0,n-2 = csp.grid 0,n-1 C row:1 : csp.grid 1,0 = csp.grid 1,1 =... = csp.grid 1,n-2 = csp.grid 1,n-1... C row:n-1 : csp.grid n-1,0 = csp.grid n-1,1 =... = csp.grid n-1,n-2 = csp.grid n-1,n-1 C col:0 : csp.grid 0,0 = csp.grid 1,0 =... = csp.grid n-2,0 = csp.grid n-1,0 C col:1 : csp.grid 0,1 = csp.grid 1,1 =... = csp.grid n-2,1 = csp.grid n-1,1... C col:n-1 : csp.grid 0,n-1 = csp.grid 1,,n-1 =... = csp.grid n-2,n-1 = csp.grid n-1,n-1 All digits on row 0 must be different All digits on row 1 must be different All digits on row n - 1 must be different All digits in col 0 must be different All digits in col 1 must be different All digits in col n - 1 must be different Each inequality symbol in the puzzle introduces a constraint as well. For example, in the 4 4 puzzle shown above, we have, C ineq:1 : csp.grid 0,1 < csp.grid 0,2 Inequality constraint on row 0 C ineq:2 : csp.grid 2,0 < csp.grid 3,0 First inequality constraint on row 2 C ineq:3 : csp.grid 2,2 < csp.grid 3,2 Second inequality constraint on row 2 C ineq:4 : csp.grid 3,0 < csp.grid 3,1 Inequality constraint on row 3 Constraints are not data but rather, constraints are program code that is executed to check if a constraint is violated or not. Therefore, we need a way to generically represent the inequality constraints because they will vary from puzzle to puzzle. To figure out how to do this, let us consider the program input, i.e., how will we format the input data to represent a specific problem? Obviously, we need to read n, the size of the grid, so we can make that the first input value. Next, we need to be able to specify which grid elements are already initialized with a digit. There are two immediately obvious ways: (1) input the values of the entire grid, with zero's being placed in empty grid elements; or (2) specify a list of 3-tuples, with the first two elements in the tuple being the row and column of the grid element being filled with a value, and the third element being the initial value. I like the second method better because the first method would increase the size of the input and it would also make creating the input file a bit more work. At this time, our input file for the puzzle on the previous page will be, 4 3, 2, 3 whereas with method 1 it would be (note: requires a lot more typing, especially for 9 9 puzzles), To specify an inequality constraint, we could represent it as a pair of 2-tuples, with the first 2-tuple of the pair being the row and column indices of the element that is less than the second element, which is also specified with a 2-tuple containing the row and column indices. Using method 1 to specify initial grid elements, the format of the input file for the example puzzle would be, 4 3, 2, 3 0, 1, 0, 2 2, 0, 3, 0 2, 2, 3, 2 3, 0, 3, 1 (c) Kevin R. Burger :: Computer Science & Engineering :: Arizona State University Page 8

9 However, since the number of initialized grid elements and inequality constraints will vary from puzzle to puzzle, we need a way to specify in the input file when we have reached the end of the list of initialized grid elements. There are a variety of ways to do this. One method which will make the code fairly simple is to uniquely identify each each line of input with a 1-character "code" of some sort, e.g, n 4 i 3, 2, 3 < 0, 1, 0, 2 < 2, 0, 3, 0 < 2, 2, 3, 2 < 3, 0, 3, 1 Now, a line starting with n contains the value of n, a line starting with i specifies a 3-tuple for a grid element to be initialized, and the lines starting with < represent inequality constraints. One advantage of this format is that the lines now can be in any order, e.g., we could represent the same puzzle with this input, < 3, 0, 3, 1 < 2, 0, 3, 0 i 3, 2, 3 < 0, 1, 0, 2 < 2, 2, 3, 2 n 4 Now we have a generic way to represent the input so the program will be able to solve any puzzle. In the csp data structure, inequality constraints can be represented as a list of sublists, where each sublist is the pair of 2-tuples specifying the row and column indices of the involved grid elements, e.g., after reading the input, csp.ineq_constraints would be the list [[(0, 1), (0, 2)], [(2, 0), (3, 0)], [(2, 2), (3, 2)], [(3, 0), (3, 1)]]. To check these constraints, we iterate over this list, e.g., ineq_constraint_violated (csp) -- Return true when a constraint is violated, otherwise return false. 1. For each constraint in csp.ineq_constraints do the following, a. row1, col1 the row and column indices of the first grid element in constraint b. row2, col2 the row and column indicesof the second grid element in constraint c. If the grid element at (row1, col1) is greater than or equal to the grid element at (row2, col2) then, 1. Return true, indicating an inequality constraint was violated. 2. If we drop out of the loop, then no constraints were violated so return false, indicating success. The next issue to be addressed concerns the input domains for each variable. In the magic square problem, the input do - main for each variable was [1, 9], but in keeping track of which values from the input domain that are assigned and unas - signed, we required only one list of values because assignments involving duplicate values from [1, 9] were not made. If we apply that same approach to this problem, i.e., if we have only one list of unassigned values, then once we assign the digit 1 to a grid element in row 0, there will not be a 1 in the list of unassigned values when we reach row 1. There are two obvious ways to handle this: (1) Each of the n 2 variables maintains its own list of assigned and unassigned values, with each of the unassigned values lists being initialized to [1, 2, 3, 4]; or (2) Create just n variables, one for each row, and each variable maintains its own list of assigned and unassigned values, with each of the unassigned values lists being initialized to all of the n! permutations of the digits [1, n]. Using method 1, the algorithm would start by placing 1 in the grid element at (0, 0), moving 1 from csp.grid 0,0 's unassigned values list to its assigned values list. Then it will make a recursive call, with csp.grid 0,1 being the next variable to be assigned. csp.grid 0,1 's unassigned values list is [1, 2, 3, 4], so the algorithm would assign the first value in this list (1) to csp.grid 0,1 and then when it checks the constraints, it would determine that constraint C row:0 has been violated. So, it would undo that assignment. Next, it would assign 2 to csp.grid 0,1, determine no constraint was violated, and recursively call the function. During that call, csp.grid 0,2 will be selected as the next variable to be assigned, and since its unassigned values list is [1, 2, 3, 4], the algorithm will proceed by first assigning 1 to csp.grid 0,2 (which violates a constraint because 1 was (c) Kevin R. Burger :: Computer Science & Engineering :: Arizona State University Page 9

10 assigned to csp.grid 0,0, so that assignment will be undone), and then 2 (which violates a constraint again because 2 was assigned to csp.grid 0,1, so again, the assignment will be undone), and finally it will assign 3, discovering that no constraints were violated. I hope you see that this method will lead to a lot of assignments that violate constraints. It would be much faster if once we assign 1 to csp.grid 0,0, we remove 1 from the list of digits that can be assigned to variables on row 0. Similarly, once we assign 2 to csp.grid 0,1, we should remove that digit from the list of digits that can be assigned to row 0. This observation leads us to method 2, where the n = 4 variables would be csp.grid row:0, csp.grid row:1, csp.grid row:2, csp.grid row:3. We would maintain four unassigned values lists, one per variable, with each list being initialized to all of the 4! = 24 permutations of [1, 2, 3, 4], specifically: [[1, 2, 3, 4], [1, 2, 4, 3], [1, 3, 2, 4], [1, 3, 4, 2], [1, 4, 2, 3], [1, 4, 3, 2], [2, 1, 3, 4], [2, 1, 4, 3], [2, 3, 1, 4], [2, 3, 4, 1], [2, 4, 1, 3], [2, 4, 3, 1], [3, 1, 2, 4], [3, 1, 4, 2], [3, 2, 1, 4], [3, 2, 4, 1], [3, 4, 1, 2], [3, 4, 2, 1], [4, 1, 2, 3], [4, 1, 3, 2], [4, 2, 1, 3], [4, 2, 3, 1], [4, 3, 1, 2], [4, 3, 2, 1]]. Method 2 will definitely find the solution more quickly; we would make substantially fewer variable assignments that violate constraints because we would be assigning values to entire rows of grid elements at a time rather than to one grid element at a time. Another advantage of the latter formulation is that it would reduce the number of constraints that must be checked after each assignment. For example, since each assignment would assign a permutation of [1, n] to a row, it would be impossible for the same digit to be repeated on a row, thus eliminating the need for the n row constraints. However, we would still need to check the column and inequality constraints. Here is the 4 4 puzzle problem formulation, csp.n 4 csp.grid [[0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 3, 0]] csp.unassigned_vars [csp.grid row:0, csp.grid row:1, csp.grid row:2, csp.grid row:3 ] csp.assigned_vars [] csp.grid row:0.unassigned_vals [[1,2,3,4], [1,2,4,3], [1,3,2,4],..., [4,3,1,2], [4,3,2,1]] csp.grid row:0.assigned_vals [] csp.grid row:1.unassigned_vals [[1,2,3,4], [1,2,4,3], [1,3,2,4],..., [4,3,1,2], [4,3,2,1]] csp.grid row:1.assigned_vals [] csp.grid row:2.unassigned_vals [[1,2,3,4], [1,2,4,3], [1,3,2,4],..., [4,3,1,2], [4,3,2,1]] csp.grid row:2.assigned_vals [] csp.grid row:3.unassigned_vals [[1,2,3,4], [1,2,4,3], [1,3,2,4],..., [4,3,1,2], [4,3,2,1]] csp.grid row:3.assigned_vals [] csp.ineq_constraints [[(0, 1), (0, 2)], [(2, 0), (3, 0)], [(2, 2), (3, 2)], [(3, 0), (3, 1)]] csp.col_constraints [C col:0, C col:1, C col:2, C col:3 ] C col:0 : csp.grid 0,0 = csp.grid 1,0 = csp.grid 2,0 = csp.grid 3,0 C col:1 : csp.grid 0,1 = csp.grid 1,1 = csp.grid 2,1 = csp.grid 3,1 C col:2 : csp.grid 0,2 = csp.grid 1,2 = csp.grid 2,2 = csp.grid 3,2 C col:3 : csp.grid 0,3 = csp.grid 1,3 = csp.grid 2,3 = csp.grid 3,3 We can represent each of the n problem variables as a simple integer which specifies the row number so when implemented, csp.unassigned_vars would actually be initialized to the list [0, 1, 2,..., n-1]. We need the ability to create the structure and initialize the data members, which we can do in an OO programming language by creating a simple class with only an initialization function. class CSP function init() self.n nothing self.grid nothing self.curr_var nothing self.unassigned_vars nothing self.assigned_vars list() (this is csp.assignment) -- Initialized when dfs_backtrack() is called the first time -- Initially no problem variables have been assigned (c) Kevin R. Burger :: Computer Science & Engineering :: Arizona State University Page 10

11 self.v x.unassigned_vals nothing self.v x.assigned_vals nothing self.ineq_constraints list() self.solution nothing -- Initially there are no inequality constraints, initialized when reading input -- Initialized in dfs_backtrack() when the solution is found Note that the following csp data members cannot be completely initialized until the input is read: csp.n, csp.grid, csp. curr_var, csp.unassigned_vars, csp.v x.unassigned_vals, csp.v x.assigned_vals, and csp.ineq_constraints. We are now ready to design the algorithm. Version 1 is a very high-level, mostly English description, Version 1 - High-Level Description in Outline Form 1. Read the puzzle input, initializing required problem variables. 2. Use the standard DFS with backtracking algorithm to find the solution to the puzzle. 3. Output the solution. The next step is to identify how steps 1-3 will be partitioned into functions. Obviously, we need a main-like function, which will be solve_problem(). solve_problem() - Create the csp data structure to store the relevant problem information. Call read_input(csp) to read the puzzle from stdin. read_input() will read the input, initializing the following csp data members as it does: csp.n, csp.grid, csp.unassigned_vars, csp.v x.unassigned_vals, csp.v x.assigned_vals, and csp.ineq_constraints. Next, call solve_ problem(csp) which will use the DFS with backtracking method to solve the puzzle. Upon return, csp.solution will contain the solution to the puzzle, which will be output when solve_problem() calls output_solution(csp.solution). read_input(csp) - Will read input from stdin until the end-of-file is reached. When reading a line of input, first, read the first string of the line, which will be "n" (specifies the size of the puzzle), "i" (specifies an initial value for a grid element), or "<" (specifies an inequality constraint). Then based on the type of data on the line, read the rest of the line, initializing any required csp data members. See comments in solve_problem() for a list of csp data members that are initialized in read_input(). select_next_var(csp) - Each time dfs_backtrack() is recursively called we must choose a new variable from the list of unassigned variables to be the current variable to be assigned values. This function simply chooses the first variable from csp.unassigned_vars to be the current variable. It will move the current variable from the list of unassigned variables to the list of assigned variables before returning. dfs_backtrack(csp) - Implements the standard DFS with backtracking algorithm, see dfs_backtrack() on pp Checking constraints can be done either in this function or in separate functions. Since including the constraints in this function would only make it lengthier and more complicated than it already is, it is better to check constraints in separate functions. With our formulation, we do not need to check row constraints, but we do need to check the column and inequality constraints. We will check the column constraints by calling col_constraint_violated(csp), which will return true when a column constraint is violated and false otherwise. We will check the inequality constraints by calling ineq_constraint_ violated(csp), which will return true when an inequality constraint is violated and false otherwise. When the solution is found, return it in csp.solution. col_constraint_violated(csp) - Iterates over each column of csp.grid (which is the current assignment or state) checking to see if a digit is duplicated in the column. If so, returns true, indicating a column constraint was violated. Otherwise, returns false. ineq_constraint_violated(csp) - Iterates over csp.ineq_constraints checking each of the inequality constraints. Returns true if an inequality constraint is violated, otherwise returns false. output_solution(csp) - Outputs the completed grid in a format which shall be discussed later. (c) Kevin R. Burger :: Computer Science & Engineering :: Arizona State University Page 11

12 We are now ready to document the design of the second version of the algorithm. Version 2 - Mid-Level Algorithm in Outline Form class CSP function init() self.n nothing self.grid nothing self.curr_var nothing self.unassigned_vars nothing self.assigned_vars list() self.v x.unassigned_vals nothing self.v x.assigned_vals nothing self.ineq_constraints list() self.solution nothing (this is csp.assignment) -- Initialized when dfs_backtrack() is called the first time -- Initially no problem variables have been assigned -- Initially there are no inequality constraints, initialized when reading input -- Initialized in dfs_backtrack() when the solution is found col_constraint_violated (csp) -- Return true when a constraint is violated, otherwise return false. 1. Let col vary from 0 to csp.n - 1 and do the following for each value of col, a. If column col of csp.grid contains duplicate values then return true, indicating a constraint was violated. 2. If we drop out of the loop, then no constraints were violated so return false. ineq_constraint_violated (csp) -- Return true when a constraint is violated, otherwise return false. 1. For each constraint in csp.ineq_constraints do the following, a. row1, col1 the row and column indices of the first grid element in constraint b. row2, col2 the row and column indicesof the second grid element in constraint c. If the grid element at (row1, col1) is greater than or equal to the grid element at (row2, col2) then, 1. Return true, indicating an inequality constraint was violated. 2. If we drop out of the loop, then no constraints were violated so return false. output_solution (csp) -- csp.solution is the complete and consistent 2D grid. 1. Let row vary from 0 to csp.n - 1, a. Let col vary from 0 to csp.n - 1, 1. Send csp.solution row,col to stdout, followed by a space. b. Send a newline to stdout. read_input (csp) 1. For each line of text that is read from stdin do the following, a. type read first string from line b. When type is n then, 1. Read the integer from line and assign it to csp.n. 2. Create 2D array csp.grid with csp.n rows and csp.n columns. Initialize all elements to Initialize csp.unassigned_vars to [0, 1, 2,..., csp.n - 1]. 4. Let x vary from 0 to csp.n - 1 and do the following for each value of x, a. Create csp.v x data member. b. Initialize csp.v x.unassigned_vals to all the permutations of the list [1, 2, 3,..., csp.n]. c. Initialize csp.v x.assigned_vals to the empty list. c. Else, when type is i then, 1. Read and let the three integers on the line be r, c, and v. 2. Assign v to csp.grid r,c. (c) Kevin R. Burger :: Computer Science & Engineering :: Arizona State University Page 12

13 d. Else, when type is < then, 1. Read and let the four integers on the line be r1, c1, r2, c2. 2. Create two 2-tuples, t1 (r1, c1), t2 (r2, c2). 3. Create a list, l [t1, t2]. 4. Append l to csp.ineq_constraints. select_curr_var (csp) -- selects the first variable in csp.unassigned_vars to be the current variable 1. Let csp.curr_var be csp.unssigned_vars csp.assigned_vars.add(csp.curr_var). 3. csp.unassigned_vars.remove(csp.curr_var). solve_problem () -- this is the main function 1. Create csp data structure, initializing some of the data members (see the text for the list). 2. Call read_input(csp), which will initialize the remaining csp data members. 4. Call dfs_backtrack(csp). 5. Call output_solution(csp). dfs_backtrack (csp) -- Recursive function. Upon return, the puzzle solution will be stored in csp.solution. 1. If csp.unassigned_vars is non-empty then we have a complete assignment (state). Since we do not call this function when a constraint is violataed (see below), this means the current assignment is consistent. An assignment that is complete and consistent is a solution, so, a. Copy csp.grid to csp.solution. b. Return success 2. Otherwise, we have an incomplete state so call select_curr_var(csp). Upon return, csp.curr_var will be the selected variable. 3. For each value in csp.curr_var.unassigned_vals do the following, a. Copy the digits from the permutation value to row csp.curr_var of csp.grid (make the assignment). b. If col_constraint_violated(csp) returns true or if ineq_constraint_violated(csp) returns true then, 1. Copy 0's to row csp.curr_var of csp.grid (undo the assignment). 2. Continue with Step 3. c. Remove value from csp.curr_var.unassigned_vals. d. Add value to csp.curr_var.assigned_vals. e. Call dfs_backtrack(csp) assigning the return value to result. f. If result is success then, 1. Return success. This will cause all of the recursive calls to return, ultimately returning back to solve_problem(). g. Else if result is fail then, 1. Copy 0's to row csp.curr_var of csp.grid (undo the assignment). 2. Remove value from csp.curr_var.assigned_vals. 3. Add value to csp.curr_var.unassigned_vals. 4. If we drop out of the loop, we have reached a dead-end for the current assignment. Remove csp.curr_var from csp.assigned_vars. 5. Add csp.curr_var to csp.unassigned_vars. 6. Return fail. We are now ready to design Version 3 of the algorithm. However, there are some low-level implementation issues that are not addressed in Version 2, which is still a bit too high-level to directly translate into Python. One of those issues concerns how we represent csp.v x.unassigned_vals and csp.v x.assigned_vals. There are a plethora of ways to do that. Suppose in a CSP that we have csp.n = 3 variables. Each variable can be identified by an integer, so the first variable is 0, the second variable is 1, and the third variable is 2, making csp.unassigned_vars initialized to [0, 1, 2]. Suppose the input (c) Kevin R. Burger :: Computer Science & Engineering :: Arizona State University Page 13

14 domains for the three variables are [1, 2], [3, 4, 5], and [6, 7, 8, 9], respectively. We can create this list: csp.values = [([1, 2], []), ([3, 4, 5], []), ([6, 7, 8, 9], [])], where each of the three list elements is a 2-tuple; the first tuple is associated with variable 0, the second tuple with variable 1, and the third tuple with variable 2. Each 2-tuple contains two lists: the first list is the list of unassigned values for the variable and the second list is the list of assigned values. Initially, all of the values are in the unassigned values lists, but as assignments are made, assigned values will be moved to the corresponding variable's assigned values list. In Python, tuple elements are commonly accessed with a subscript, e.g., if t is the 2-tuple ([1, 2], []) then t[0] is the list [1, 2] and t[1] is the empty list []. It is possible to create named tuples [see p. 29 in reference 6], which permits us to reference the tuple elements by name rather than number. For example, >>> from collections import namedtuple >>> Values = namedtuple("values", "unassigned_vals", "assigned_vals") creates a class type named Values >>> csp.values = [Values([1, 2], []), Values([3, 4, 5], []), Values([6, 7, 8, 9], [])] >>> print(values[0], values[1], values[2], sep="\n") Values(unassigned_vals=[1, 2], assigned_vals=[]) Values(unassigned_vals=[3, 4, 5], assigned_vals=[]) Values(unassigned_vals=[6, 7, 8, 9], assigned_vals=[]) >>> print(values[0].unassigned_vals, values[0].assigned_vals) [1, 2] [] >>> var0_first_unassigned_val = values[0].unassigned_vals[0] >>> print(var0_first_unassigned_val) 1 >>> curr_var = 1 >>> curr_val = values[curr_var].unassigned_vals[0] >>> print(curr_val) 3 >>> values[curr_var].unassigned_vals.remove(curr_val) removes curr_val from the unassigned values list >>> print(values[curr_var]) Values(unassigned_vals=[4, 5], assigned_vals=[]) >>> values[curr_var].assigned_vals.append(curr_val) adds curr_val to the assigned values list >>> print(values[curr_var]) Values(unassigned_vals=[4, 5], assigned_vals=[3]) Named tuples are simply syntactic sugar that improve the readability of the code. If you wish to learn Python, you should learn how to use them. For larger more complex CSP's, representing the values lists for each variable in this way may not be the most efficient way to do it, but I'm in a hurry, so it will work for now. The second issue concerns col_constraint_violated(). In that function, we need to check each of the columns of csp.grid for duplicate values. To do that, we can create a new function contains_duplicates(list) that will return true if the list list contains duplicate values or false, otherwise. Note that grid elements corresponding to unassigned variables are assigned the value 0 (empty) to indicate an empty grid element, so when checking for duplicates in a column, we do not consider 0's to be duplicated. The second new function is get_col() which will create a new 1D list containing the values in the specified column of csp.grid. The third issue involves creating the 2D array csp.grid. For that, I have created a new function alloc_2darray(n, m) which will allocate and return a 2D array (as a list of sublists) with n rows and m columns. For representing inequality constraints, I have created a new namedtuple class IneqConstraint which assigns the names x and y to represent the two variables being compared. Each element of IneqConstraint is a named 2-tuple (Index) containing the rowcol indices of the associated variable. (c) Kevin R. Burger :: Computer Science & Engineering :: Arizona State University Page 14

15 In dfs_backtrack(), making the assignment of value to csp.curr_var will involve writing each of the digits in the permutation value to csp.grid in the row specified by csp.curr_var. We also need to undo assignments by writing 0's into csp.grid. I have created a new function assign_row() which will do this. The final issue concerns the for loop in dfs_backtrack() which iterates over the values of csp.curr_var's unassigned values list. Each time through this loop, we assign the iterated value to csp.curr_var (by writing the value into csp.grid). If the assignment does not violate a constraint, then we need to make a recursive call, but before doing that, we have to remove the value from csp.curr_var's list of unassigned values and add it to csp.curr_var's list of assigned values. Upon return, if no solution was found, we must assign the next value to csp.curr_var to continue searching. This requires us to move the assigned value from csp.curr_var's assigned values list back to csp.curr_var's unassigned values list. However, the problem is that in our implementation language (Python), we cannot modify the elements of a list that is being iterated over (trust me, bugs happen). To handle this, in dfs_backtrack() before beginning the for loop, we make a copy of csp.curr_ var's unassigned values list, and iterate over the copy. This allows us to modify csp.curr_var's unassigned values list without screwing up the iterator. Note: in the implementation language we must make a deep copy of the list. Version 3 - Low-Level Algorithm in Semi-Formal Pseudocode Form enumerated values {empty 0, success 0, fail 1} class Index namedtuple (row, col) -- represents a rowcol index in a 2darray class IneqConstraint namedtuple (x, y) -- x and y are 2-tuples containing the rowcol indices of variables x and y class Values namedtuple (unassigned_vals, assigned_vals) class CSP function init () self.n nothing self.grid nothing self.curr_var nothing self.unassigned_vars nothing self.assigned_vars list() self.values nothing self.ineq_constraints list() self.solution nothing (this is csp.assignment) -- Initialized when dfs_backtrack() is called the first time -- Initially no problem variables have been assigned -- Initially there are no inequality constraints, initialized when reading input -- Initialized in dfs_backtrack() when the solution is found function alloc_2darray (n, m, value) -- Allocates an n m 2D array, initializing all elements to value. array 2darray(n, m) -- Assume 2darray is a fundamental data type. for row 0 to n - 1 do for col 0 to m - 1 do array row,col value return array function assign_row (n, row, array : 2darray, value : list) -- Copies each value from value to array row row. n is the number of columns in the row. for col 0 to n - 1 do array row,col value col function col_constraint_violated (csp) -- Return true when a constraint is violated, otherwise return false. for col 0 to csp.n - 1 do l get_col(col, csp.n, csp.grid) -- Initialize list to the values in column col of csp.grid when contains_duplicates(l) = true then return true return false (c) Kevin R. Burger :: Computer Science & Engineering :: Arizona State University Page 15