Scientific Programming. Backtracking

Size: px

Start display at page:

Download "Scientific Programming. Backtracking"

Julia Lewis
5 years ago
Views:

1 Scientific Programming Backtracking Alberto Montresor Università di Trento 2018/12/15 This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

2 Table of contents 1 Introduction 2 Enumeration Subsets Permutations Games

3 Introduction Introduction Problem classes (decisional, search, optimization) Definition bases on the concept of admissible solution: a solution that satisfies a given set of criteria Typical problems Build one or all admissible solution Counting the admissible solutions Find the admissible solution "largest", "smallest", in general "optimal" Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 1 / 37

4 Introduction Typical problems Enumeration List algorithmically all possible solutions (search space) Example: list all the permutations of a set Build at least a solution We use the algorithm for enumeration, stopping at the first solution found Example: identify a sequence of steps in the Fifteen game Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 2 / 37

5 Introduction Typical problems Count the solutions In some cases, it is possible to count in analytical way Example: counting the number of subsets of k elements taken by a set of n elements n! k! (n k)! In other cases, we build the solutions and we count them Example: number of subsets of a integer set S whose sum is equal to a prime number Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 3 / 37

6 Introduction Typical problems Find optimal solutions We enumerate all possible solutions and evaluate them through a cost function Only if other techniques are not possible: Dynamic programming Greedy Example: Hamiltonian circuit (Traveling salesman) Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 4 / 37

7 Introduction Build all solutions To build all the solutions, we use a "brute-force" approach Sometimes, it is the only possible way The power of modern computer makes possible to deal with problems medium-small problems 10! = (permutation of 10 elements) 2 20 = (subsets of 20 elements) Sometimes, the space of all possible solutions does not need to be analyzed entirely Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 5 / 37

8 Introduction Backtracking Approach "Prova a fare qualcosa, e se non va bene, disfalo e prova qualcos altro" "Ritenta, e sarai più fortunato" How it works? A systematic methods to iterate all possible instances of a research space It is an algorithm technique that, as the others, need to be "customized" for the specific problem to be solved. Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 6 / 37

9 Introduction General organization General organization A solution is represented by a list S The content of element S[i] is taken from a set of choices C that depends on the problem Examples C generic set, possible solutions permutations of C C generic set, possible solutions subsets of C C game moves, possible solutions a sequence of moves C edges of a graph, possible solutions paths Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 7 / 37

10 Introduction Partial solutions At each step, we start from a partial solution S where k 0 choices have been already taken If S[0 : k] is an admissible solution, we "process" it E.g., we can print it We can then decide to stop here or keep going by listing/printing all solutions If S[0 : k] is not a complete solution: If possible, we extended solution S[0 : k] with one of the possible choices to get a solution S[0 : k + 1] Otherwise, we "cancel" the element S[k] (backtrack) and we go back to to solution S[0 : k 1] Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 8 / 37

11 Introduction Decision trees Decision tree Search space Root Empty solution Internal nodes Partial solutions Leaves Admissible solutions S[0] S[1] S[2] S[3] S[3] Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 9 / 37

12 Introduction Pruning "Branches" of the trees that do not bring to admissible solutions can be "pruned" The evaluation is done in the partial solutions corresponding to internal nodes Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 10 / 37

13 Enumeration boolean enumeration(object[ ] S, int n, int i,... ) Set C = choices(s, n, i,...) % Compute C based on S[0 : i 1] foreach c C do S[i] = c if isadmissible(s, n, i) then if processsolution(s, n, i,...) then return True if enumeration(s, n, i + 1,...) then return True return False Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 11 / 37

14 Enumeration S: list containing the partial solutions i: current index...: additional information C: the set of possible candidates to extend the current solution isadmissible(): returns True if S[0 : i] is an admissible solution processsolution(): returns True to stop the execution at the first admissible solution False to explore the entire tree Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 12 / 37

15 Subsets Example 1 List all subsets of {0,..., n 1} def subsets(s,n,i): C = [1, 0] if i<n else [] for c in C: S[i] = c if i==n-1: if (processsolution(s)): return True else: subsets(s,n,i+1) return False def processsolution(s): for i in range(len(s)): if (S[i]==1): print(i, "", end="") print("") return False n = 4 S = [0]*n subsets(s,4,0) Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 13 / 37

16 Subsets Example 1 (Clean-up version) List all possible subsets of {0,..., n 1} def subsets(s,n,i): for c in [1, 0]: S[i] = c if i == n-1: processsolution(s) else: subsets(s,n,i+1) Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 14 / 37

17 Subsets Example 1 There is no pruning. All the possible space is explored. But this is required by the definition of the problem Computational complexity O(n2 n ) In which order sets are printed? Is it possible to think to an iterative version, ad-hoc for this problem? (non-backtracking) Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 15 / 37

18 Subsets Example 1 subsets(int n) for j = 0 to 2 n 1 do print { for i = 0 to n 1 do if (j and 2 i ) 0 then print i print } Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 16 / 37

19 Subsets Example 2 List all possible subsets of size k of a set {0,..., n 1} Iterative version What is the cost? Solution based on backtracking Can we prune? Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 17 / 37

20 Subsets Example 2 - Attempt 1 def subsets(s, k, n,i): C = [1,0] for c in C: S[i] = c if i == n-1: if admissible(s, k): processsolution(s) else: subsets(s, k, n, i+1): def admissible(s, k): return sum(s) == k def processsolution(s): for i in range(len(s)): if (S[i]==1): print(i, "", end="") print("") return False n = 4 k = 2 S = [0]*n subsets(s, 2, 4, 0) Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 18 / 37

21 Subsets Example 2 - Attempt 2 def subsets(s, k, n, i, count): C = [1, 0] for c in C: S[i] = c count = count + c if i == n-1: if count == k: processsolution(s) else: subsets(s,k, n, i+1, count) count = count - c def admissible(s, k): tot = sum(s) return tot == k def processsolution(s): for i in range(len(s)): if (S[i]==1): print(i, "", end="") print("") return False n = 4 k = 2 S = [0]*n subsets(s, 2, 4, 0, 0) Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 19 / 37

22 Subsets Example 2 - Attempt 3 def subsets(s, k, n, i, count): if count < k and count + (n-i) >= k: C = [1,0] else C = [] for c in C: S[i] = c count = count + c if count == k: processsolution(s) else: subsets(s, k, n, i+1, count) count = count - c S[i] = 0 def processsolution(s): for i in range(len(s)): if (S[i]==1): print(i, "", end="") print("") return False n = 4 k = 2 S = [0]*n subsets(s, 2, 4, 0, 0) Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 20 / 37

23 Subsets Example Esempio 2: vantaggi 2: advantages Alberto Montresor Montresor (UniTN) SP - Backtracking 2018/12/15 21 / 3720

24 Subsets Example 2 - Summary Cosa abbiamo imparato? By "customizing" the generic algorithm, we can get a more efficient solution It is efficient for small values of k (close to 1) large values of k (close to n) No great saving for n/2 It is difficult to obtain the same efficiency with an iterative algorithm Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 22 / 37

25 Permutations Example 3 Print all permutations of a set A def permutations(s, i, original): for c in original: S[i] = c original.remove(c) if (len(original)==0): print(s) else: permutations(s, i+1, original) original.add(c) original = { 1,2,3,4 } S = [0]*len(original) permutations(s, 0, original) Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 23 / 37

Games Eight queens puzzle Problem The eight queens puzzle is the problem of placing eight chess queens on an 8 8 chessboard so that no two queens threaten each other History: Introduced by Max

26 Games Eight queens puzzle Problem The eight queens puzzle is the problem of placing eight chess queens on an 8 8 chessboard so that no two queens threaten each other History: Introduced by Max Bezzel (1848) Gauss found 72 of the 92 solutions Let s start from the stupidest approach, and let s refine the solution step by step Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 24 / 37

27 Games Eight queens puzzle Idea: There are n 2 possible positions where a queen may be located S[0 : n 2 ] binary array S[i] = True "queen in i" isadmissible() i == n 2 choices(s, n, i) pruning {True, False} # Solutions for n = Comments if the new queen threaten one of the existing queens, returns Maybe we have a representation problem? Binary matrix, really sparse Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 25 / 37

28 Games Better, but the space is still huge... Problem: how to distinguish between a solution " " and a solution " "? Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 26 / 37 Eight queens puzzle Idea: For each queen, there are n 2 possible positions S[0 : n] coordinates in {0... n 2 1} isadmissible() S[i] coordinate of queen i i == n choices(s, n, i) {0... n 2 1} pruning returns the subset of legal moves # Solutions for n = 8 (n 2 ) n = 64 8 = Comments

29 Games Eight queens puzzle Idea: do not place queens in positions that occur "before" the ones already taken S[0 : n] coordinates in {0... n 2 1} isadmissible() S[i] coordinate of queen i i == n choices(s, n, i) {0... n 2 1} pruning returns legal moves, S[i] > S[i 1] # Solutions for n = 8 (n 2 ) n /n! = 2 48 / Comments Very good, but there is still space for improvement Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 27 / 37

30 Games Eight queens puzzle Idea: each row of the board must contain exactly one queen S[0 : n] coordinates in {0... n 1} isadmissible() S[i] column of queen i, where row = i i == n choices(s, n, i) {0... n 1} pruning returns only legal columns # Solutions for n = 8 n n = Comments Almost there Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 28 / 37

31 Games Eight queens puzzle Idea: every column must contain exactly one queen S[0 : n] coordinates in {0... n 1} isadmissible() permutations of {1... n} i == n choices(s, n, i) {0... n 1} pruning removes diagonals # Solutions for n = 8 n! = 8! = Comments Solutions actually visited = Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 29 / 37

32 Games Eight queens puzzle def queens(s, i, columns): for c in columns: S[i] = c columns.remove(c) if (diagonalsok(s,i)): if len(columns)==0: printboard(s) else: queens(s,i+1,columns) columns.add(c) Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 30 / 37

lo Knight s tour Enumeration Games Problem A Knight s tour is a sequence of moves of a knight on a chessboard such that the knight visits every square only once.

33 lo Knight s tour Enumeration Games Problem A Knight s tour is a sequence of moves of a knight on a chessboard such that the knight visits every square only once. deri ancora una scacchiera n n; lo scopo è trovare un giro di cavallo, n percorso di mosse valide del cavallo in modo che ogni casella venga al più una volta backtrack Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 31 / 37 30

34 Games Knight s tour Solution Board n n whose cells contains: 0 if the cell has not been visited i if the cell has been visited at step i # Solutions: 64! But: at each step, there are at most 8 possible cells, so the maximum number of visits is In reality, much less thank to pruning Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 32 / 37

35 Games Knight s tour def knight(s, i, x, y): n = len(s) C = moves(s, x, y) if (i == n*n): for k in range(n): print(s[k]) return True else: S[x][y] = i for c in C: if knight(s, i+1, x+dx[c], y+dy[c]): return True S[x][y] = 0 return False Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 33 / 37

36 Games Knight s tour dx = [-1, +1, +2, +2, +1, -1, -2, -2] dy = [-2, -2, -1, +1, +2, +2, +1, -1] def moves(s, x, y): res = set() for i in range(8): nx = x + dx[i] ny = y + dy[i] if (0 <= nx < 8) and (0 <= ny < 8) and S[nx][ny] ==0 : res.add(i) return res Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 34 / 37

37 Games Sudoku - "Suuji wa dokushin ni kagiru" Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 35 / 37

38 Games Sudoku boolean sudoku(int[ ][ ] S, int i) int x = i mod 9 int y = i/9 Set C = Set() if i 80 then if S[x, y] 0 then C.insert(S[x, y]) else for c = 1 to 9 do if check(s, x, y, c) then C.insert(c) int old = S[x, y] foreach c C do S[x, y] = c if i = 80 then processsolution(s, n) return True if sudoku(s, i + 1) then return True S[x, y] = old return False Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 36 / 37

39 Games Sudoku boolean check(int[ ][ ] S, int x, int y, int c) for j = 0 to 8 do if S[x, j] = c then return False if S[j, y] = c then return False int b x = x/3 int b y = y/3 for i x = 0 to 2 do for int i y = 0 to 2 do if S[b x 3 + i x, b y 3 + i y ] = c then return False % Column check % Row check % Subtable check return True Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 37 / 37

Backtracking and Branch-and-Bound

Backtracking and Branch-and-Bound Usually for problems with high complexity Exhaustive Search is too time consuming Cut down on some search using special methods Idea: Construct partial solutions and extend