Dynamic Programming Matrix-chain Multiplication

Transcription

1 1 / 32 Dynamic Programming Matrix-chain Multiplication CS 584: Algorithm Design and Analysis Daniel Leblanc 1 1 Senior Adjunct Instructor Portland State University Maseeh College of Engineering and Computer Science Winter 2018

2 2 / 32 Table of Contents Dynamic Programming Overview Matrix-chain Multiplication Optimal Substructure Recursive Solution Overlapping Sub-problems Final Solution

3 3 / 32 Table of Contents Dynamic Programming Overview Matrix-chain Multiplication Optimal Substructure Recursive Solution Overlapping Sub-problems Final Solution

4 4 / 32 Dynamic Programming The phrase Dynamic Programming was coined by Richard Bellman in the 1950s while he was doing research for the RAND corporation. It was chosen because It was something not even a Congressman could object to.

5 5 / 32 Dynamic Programming Many problems where a divide-and-conquer solution seems applicable have very high running times if they are implemented in a straightforward recursive way, because they end up solving the same sub-problem multiple times. Dynamic Programming is the name for the technique of avoiding recomputation of sub-problems. This can be done by either memoizing the results of each recursive call, or by reorganizing the computation to be bottom up so that sub-problems are always solved just once before they are needed.

6 6 / 32 Dynamic Programming Fundamentally, Dynamic Programming is a technique that trades time for space. In return for not recomputing sub-problems, we must store their results. Unlike most of the design techniques we talk about in this class, dynamic programming can often reduce running times from exponential to polynomial.

7 7 / 32 Dynamic Programming In order for dynamic programming to apply the problem must have two properties: 1. Optimal Substructure: The optimal solution can be obtained by combining the optimal solution to sub-problems. 2. Overlapping Sub-problems: Any recursive algorithm solving the problem must encounter the same sub-problem more than once.

8 Optimal Substructure Not all optimization problems have an optimal substructure. 8 / 32

9

10 Overlapping Sub-problems A typical divide-and-conquer algorithm generates a recursion tree, in which each sub-problem feeds a result to a single parent. 9 / 32

11 Overlapping Sub-problems We can visualize dynamic programming as generating a Directed Acyclic Graph in which sub-problem results can be fed to multiple parents. 10 / 32

12 11 / 32 Memoization Memoization is a top-down approach that directly falls out of the recursive formulation of a problem. We simple modify a recursive algorithm to store the solutions to sub-problems in a table. Prior to attempting to solve a sub-problem we first check the table to see if we already have a solution If we do it is returned directly, otherwise we solve the sub-problem and store the results before returning.

13 Tabulation Tabulation is a bottom-up approach where we reformulate the problem so that we can solve the sub-problems first. At each step we build on the sub-solutions to arrive at solutions to bigger sub-problems. 12 / 32

14 13 / 32 Table of Contents Dynamic Programming Overview Matrix-chain Multiplication Optimal Substructure Recursive Solution Overlapping Sub-problems Final Solution

15 Matrix Multiplication If we multiply a p q matrix A with a q r matrix B, we produce a p r matrix C where each Cij is the dot product of row i of A with column j of B. (Note that in order for this to be well defined, the number of columns of A must equal the number of rows of B.) 14 / 32

16 Matrix-chain Multiplication 15 / 32

17 16 / 32 Matrix-chain Multiplication Problem: Give a sequence of n matrices A 1, A 2,...A n to be multiplied together, where A i has dimensions p i 1 p i, determine how to fully parenthesize the sequence to minimize the total number of scalar multiplications required.

18 Full Parenthesization A full parenthesization of the sequence is either a single matrix or the product of two fully parenthesized sub-sequences surrounded by parenthesization. 17 / 32

19 Optimal Substructure 18 / 32

20 Optimal Substructure 19 / 32

21 20 / 32 Recursive Solution While we have an optimal substructure we still need to find the optimal split point k. Unfortunately there isn t any heuristic that we can use to get to the correct k. To find it we ll need to search all the possibilities.

22 21 / 32 Recursive Solution RECURSIVEMATRIXCHAIN(p, i, j) 1 if i == j 2 return 0 3 m = 4 for k = i to j 1 5 q = RECURSIVEMATRIXCHAIN(p, i, k) +RECURSIVEMATRIXCHAIN(p, k + 1, j) + p i 1 p k p j 6 m = min(m, q) 7 return m

23 Complexity 22 / 32

24 23 / 32 Recursion Tree (CLRS figure 15.7)

25 24 / 32 Overlapping Sub-problems While our recursive solution is extremely inefficient we can see from the recursion tree that it does a significant amount of repeated work.

26 DAG 25 / 32

27 Estimated Running Time 26 / 32

28 Memoized Matrix-chain Order 27 / 32

29 28 / 32 Memoized Matrix-chain Order MEMOIZEDMATRIXCHAIN(p, i, j) 1 if i == j 2 return 0 3 if m[i, j] == NIL 4 m[i, j] = 5 for k = i to j 1 6 q = RECURSIVEMATRIXCHAIN(p, i, k) +RECURSIVEMATRIXCHAIN(p, k + 1, j) + p i 1 p k p j 7 m[i, j] = min(m[i, j], q) 8 return m[i,j]

30 29 / 32 Bottom-up Matrix-chain Order Since every subproblem is required to find the optimal solution we can improve performance by switching to a bottom-up method.

31 30 / 32 Example matrix A 1 A 2 A 3 A 4 A 5 A 6 dimension

32 31 / 32 Bottom-up Matrix-chain Order MATRIXCHAINORDER(p) 1 n = p.length 1 2 let m[1..n, 1..n] be a new Table 3 for i = 1 to n 4 m[i, i] = 0 5 for l = 2 to n // l is the chain length 6 for i = 1 to n l j = i + l 1 8 m[i, j] = 9 for k = i to j 1 10 q = m[i, k] + m[k + 1, j] + p i 1 p k p j 11 if q < m[i, j] 12 m[i, j] = q 13 return m

33 32 / 32