ECE250: Algorithms and Data Structures Dynamic Programming Part B

Transcription

1 ECE250: Algorithms and Data Structures Dynamic Programming Part B Ladan Tahvildari, PEng, SMIEEE Associate Professor Software Technologies Applied Research (STAR) Group Dept. of Elect. & Comp. Eng. University of Waterloo Materials from CLRS: Chapter 15.4

2 Acknowledgements v The following resources have been used to prepare materials for this course: Ø MIT OpenCourseWare Ø Introduction To Algorithms (CLRS Book) Ø Data Structures and Algorithm Analysis in C++ (M. Wiess) Ø Data Structures and Algorithms in C++ (M. Goodrich) v Thanks to many people for pointing out mistakes, providing suggestions, or helping to improve the quality of this course over the last ten years: Ø Lecture 25 ECE250 2

3 Multiplying Matrices v Two matrices, A n m matrix and B m k matrix, can be multiplied to get C with dimensions n k, using nmk scalar multiplications a11 a b11 b12 b13 a21 a22 =... c22... b21 b22 b 23 a31 a m c = a b i, j i, l l, j l= 1 v Problem: Compute a product of many matrices efficiently v Matrix multiplication is associative Ø (AB)C = A(BC) ECE250 3

4 Multiplying Matrices (cont ) v Let M(i,j) be the minimum number of multiplications necessary to compute j k= i A k v Key Observations Ø The outermost parenthesis partition the chain of matrices (i,j) at some k, (i k<j): (A i A k )(A k+1 A j ) Ø The optimal parenthesization of matrices (i,j) has optimal parenthesizations on either side of k: for matrices (i,k) and (k+1,j) ECE250 4

5 Multiplying Matrices (cont ) v Recurrence: Mii (,) = 0 { } 1 Mij (, ) = min Mik (, ) + Mk ( + 1, j) + d dd i k< j i k j v It requires only Θ(n 2 ) space to store the optimal cost M(i,j) for each of the sub-problems: half of a 2d array M[1..n,1..n] Ø Thus, we have to solve sub-problems in the increasing length of sub-problems: first sub-problems of length 2, then of length 3 and so on. v To reconstruct an optimal parenthesization for each pair (i,j) we record in c[i, j]=k the optimal split into two sub-problems (i, k) and (k+1, j) ECE250 5

6 Multiplying Matrices (cont ) v After the execution: M [1,n] contains the value of an optimal solution and c contains optimal subdivisions (choices of k) of any sub-problem into two sub-sub-problems v We ran the algorithm on the six matrices: A 1 is a 30x35 matrix, A 2 is a 35x15 matrix, A 3 is a 15x5 matrix, A 4 is a 5x10 matrix, A 5 is a 10x20 matrix, and A 6 is a 20x25 matrix. ECE250 6

7 Multiplying Matrices (cont ) Matrix-Chain-Order(d 0 d n ) 01 for i 1 to n do 02 M[i,i] 0 03 for L 2 to n do 04 for i 1 to n-l+1 do 05 j i+l-1 06 M[i,j] 07 for k i to j-1 do 08 q M[i,k]+M[k+1,j]+d i-1 d k d j 09 if q < M[i,j] then 10 M[i,j] q 11 c[i,j] k 12 return M, c Running time: O(n 3 ); is also Ω(n 3 ) From exponential time to polynomial ECE250 7

8 Dynamic Programming v A sequence of four steps: 1. Show optimal substructure an optimal solution to the problem contains within it optimal solutions to sub-problems Ø Solution to a problem: ü ü Making a choice out of a number of possibilities (look what possible choices there can be) Solving one or more sub-problems that are the result of a choice (characterize the space of sub-problems) Ø Show that solutions to sub-problems must themselves be optimal for the whole solution to be optimal (use cut-andpaste argument) ECE250 8

9 Dynamic Programming (cont ) 2. Write a recurrence for the value of an optimal solution Ø M opt = Min over all choices k {(Combination (e.g., sum) of M opt of all sub-problems, resulting from choice k) + (the cost associated with making the choice k)} Ø Show that the number of different instances of subproblems is bounded by a polynomial ECE250 9

10 Dynamic Programming (cont ) 3. Compute the value of an optimal solution in a bottom-up fashion, so that you always have the necessary sub-results pre-computed 4. Construct an optimal solution from computed information (which records a sequence of choices made that lead to an optimal solution) ECE250 10

11 Longest Common Subsequence v Two text strings are given: X and Y v There is a need to quantify how similar they are v Applications: Ø Comparing DNA sequences in studies of evolution of different species Ø Spell checkers v One of the measures of similarity is the length of a Longest Common Subsequence (LCS) ECE250 11

12 LCS: Definition v Z is a subsequence of X, if it is possible to generate Z by skipping some (possibly none) characters from X v For example: X = ACGGTTA, Y= CGTAT, LCS(X,Y) = CGTA or CGTT v To solve LCS problem we have to find skips that generate LCS(X,Y) from X, and skips that generate LCS(X,Y) from Y v The longest-common-subsequence problem: find a maximum-length common subsequence ECE250 12

13 Step1 - LCS: Optimal Substructure v We make Z to be empty and proceed from the ends of X m = x 1 x 2 x m and Y n = y 1 y 2 y n Ø If x m =y n, append this symbol to the beginning of Z, and find optimally LCS(X m-1, Y n-1 ) Ø If x m y n, Skip either a letter from X or a letter from Y Decide which decision to do by comparing LCS(X m, Y n-1 ) and LCS(X m-1, Y n ) Ø Cut-and-paste argument ECE250 13

14 Step 2 - LCS: Recurrence v The algorithm could be easily extended by allowing more editing operations in addition to copying and skipping (e.g., changing a letter) v Let c[i,j] = LCS(X i, Y j ) 0 if i= 0 or j = 0 ci [, j] = ci [ 1, j 1] + 1 if i, j> 0 and xi = y max{ ci [, j 1], ci [ 1, j]} if i, j> 0 and xi y j j ECE250 14

15 Step 3 - LCS: Compute the Optimum LCS-Length(X, Y, m, n) 1 for i 1 to m do 2 c[i,0] 0 3 for j 0 to n do 4 c[0,j] 0 5 for i 1 to m do 6 for j 1 to n do 7 if x i = y j then 8 c[i,j] c[i-1,j-1]+1 9 b[i,j] copy 10 else if c[i-1,j] c[i,j-1] then 11 c[i,j] c[i-1,j] 12 b[i,j] skipx 13 else 14 c[i,j] c[i,j-1] 15 b[i,j] skipy 16 return c, b ECE250 15

16 Step 4: Constructing an LCS PRINT-LCS(b, X, i, j) 1 if i==0 and j==0 2 return 3 if b[i,j] == copy 5 PRINT-LCS(b, X, i-1, j-1) 6 print x i 7 elseif b[i,j] == skipx 8 PRINT-LCS(b, X, i-1, j) 9 else PRINT-LCS(b, X, i, j-1) ECE250 16

17 LCS: Examples v Let s run: X= ABCBDAB, Y= BDCABA v Another one: X = GGTTCAT, Y= GTATCT ECE250 17