Dynamic Programming (Part #2)

Size: px
Start display at page:

Download "Dynamic Programming (Part #2)"

Transcription

1 Dynamic Programming (Part #) Introduction to Algorithms MIT Press (Chapter 5)

2 Matrix-Chain Multiplication Problem: given a sequence A, A,, A n, compute the product: A A A n Matrix compatibility: C = A B C=A A A i A i+ A n col A = row B col i = row i+ row C = row A col C = col B row C = row A col C = col An

3 MATRIX-MULTIPLY(A, B) if columns[a] rows[b] then error incompatible dimensions else for i to rows[a] do for j to columns[b] k do C[i, j] = for k to columns[a] do C[i, j] C[i, j] + A[i, k] B[k, j] j cols[b] rows[a] cols[a] cols[b] multiplications j cols[b] i * k = i rows[a] A B rows[a] C 3

4 Matrix-Chain Multiplication In what order should we multiply the matrices? A A A n Parenthesize the product to get the order in which matrices are multiplied E.g.: A A A 3 = ((A A ) A 3 ) = (A (A A 3 )) Which one of these orderings should we choose? The order in which we multiply the matrices has a significant impact on the cost of evaluating the product 4

5 Example A A A 3 A : x A : x 5 A 3 : 5 x 5. ((A A ) A 3 ): A A = x x 5 = 5, ( x 5) ((A A ) A 3 ) = x 5 x 5 =,5 Total: 7,5 scalar multiplications. (A (A A 3 )): A A 3 = x 5 x 5 = 5, ( x 5) (A (A A 3 )) = x x 5 = 5, Total: 75, scalar multiplications one order of magnitude difference!! 5

6 Matrix-Chain Multiplication: Problem Statement Given a chain of matrices A, A,, A n, where A i has dimensions p i- x p i, fully parenthesize the product A A A n in a way that minimizes the number of scalar multiplications. A A A i A i+ A n p x p p x p p i- x p i p i x p i+ p n- x p n 6

7 What is the number of possible parenthesizations? Exhaustively checking all possible parenthesizations is not efficient! It can be shown that the number of parenthesizations grows as Ω(4 n /n 3/ ) (see page 333 in your textbook) 7

8 . The Structure of an Optimal Parenthesization Notation: A i j = A i A i+ A j, i j Suppose that an optimal parenthesization of A i j splits the product between A k and A k+, where i k < j A i j = A i A i+ A j = A i A i+ A k A k+ A j = A i k A k+ j 8

9 Optimal Substructure A i j = A i k A k+ j The parenthesization of the prefix A i k must be an optimal parenthesization If there were a less costly way to parenthesize A i k, we could substitute that one in the parenthesization of A i j and produce a parenthesization with a lower cost than the optimum contradiction! An optimal solution to an instance of the matrix-chain multiplication contains within it optimal solutions to subproblems 9

10 . A Recursive Solution Subproblem: determine the minimum cost of parenthesizing A i j = A i A i+ A j for i j n Let m[i, j] = the minimum number of multiplications needed to compute A i j full problem (A..n ): m[, n] i = j: A i i = A i m[i, i] =, for i =,,, n

11 . A Recursive Solution Consider the subproblem of parenthesizing A i j = A i A i+ A j for i j n = A i k A k+ j for i k < j m[i, k] p i- p k p j m[k+,j] Assume that the optimal parenthesization splits the product A i A i+ A j at k (i k < j) m[i, j] = m[i, k] + m[k+, j] + p i- p k p j min # of multiplications to compute A i k min # of multiplications to compute A k+ j # of multiplications to compute A i k A k j

12 . A Recursive Solution (cont.) m[i, j] = m[i, k] + m[k+, j] + p i- p k p j We do not know the value of k There are j i possible values for k: k = i, i+,, j- Minimizing the cost of parenthesizing the product A i A i+ A j becomes: if i = j m[i, j] = min {m[i, k] + m[k+, j] + p i- p k p j } ik<j if i < j

13 3. Computing the Optimal Costs if i = j m[i, j] = min {m[i, k] + m[k+, j] + p i- p k p j } ik<j if i < j Computing the optimal solution recursively takes exponential time! How many subproblems? (n ) n 3 n Parenthesize A i j for i j n One problem for each choice of i and j 3 i 3 j

14 3. Computing the Optimal Costs (cont.) if i = j m[i, j] = min {m[i, k] + m[k+, j] + p i- p k p j } ik<j How do we fill in the tables m[..n,..n]? if i < j Determine which entries of the table are used in computing m[i, j] A i j = A i k A k+ j Subproblems size is one less than the original size Idea: fill in m such that it corresponds to solving problems of increasing length 4

15 3. Computing the Optimal Costs (cont.) if i = j m[i, j] = min {m[i, k] + m[k+, j] + p i- p k p j } ik<j Length = : i = j, i =,,, n Length = : j = i +, i =,,, n- m[, n] gives the optimal solution to the problem n if i < j 3 n Compute rows from bottom to top and from left to right 3 j i 5

16 Example: min {m[i, k] + m[k+, j] + p i- p k p j } m[, 5] = min m[, ] + m[3, 5] + p p p 5 m[, 3] + m[4, 5] + p p 3 p 5 m[, 4] + m[5, 5] + p p 4 p 5 k = k = 3 k = i j Values m[i, j] depend only on values that have been previously computed 6

17 Example min {m[i, k] + m[k+, j] + p i- p k p j } Compute A A A 3 A : x (p x p ) A : x 5 (p x p ) A 3 : 5 x 5 (p x p 3 ) m[i, i] = for i =,, 3 m[, ] = m[, ] + m[, ] + p p p (A A ) = + + ** 5 = 5, m[, 3] = m[, ] + m[3, 3] + p p p 3 (A A 3 ) = + + * 5 * 5 = 5, m[, 3] = min m[, ] + m[, 3] + p p p 3 = 75, (A (A A 3 )) 3 m[, ] + m[3, 3] + p p p 3 = 7,5 ((A A )A 3 )

18 Matrix-Chain-Order(p) O(N 3 ) 8

19 4. Construct the Optimal Solution In a similar matrix s we keep the optimal values of k s[i, j] = a value of k such that an optimal parenthesization of A i..j splits the product between A k and A k+ n 3 3 n k j 9

20 4. Construct the Optimal Solution s[, n] is associated with the entire product A..n The final matrix multiplication will be split at k = s[, n] n 3 n A..n = A..s[, n] A s[, n]+..n For each subproduct recursively find the corresponding value of k that results in an optimal parenthesization 3 j

21 4. Construct the Optimal Solution s[i, j] = value of k such that the optimal parenthesization of A i A i+ A j splits the product between A k and A k i j s[, n] = 3 A..6 = A..3 A 4..6 s[, 3] = A..3 = A.. A..3 s[4, 6] = 5 A 4..6 = A 4..5 A 6..6

22 4. Construct the Optimal Solution (cont.) PRINT-OPT-PARENS(s, i, j) if i = j then print A i else print ( PRINT-OPT-PARENS(s, i, s[i, j]) PRINT-OPT-PARENS(s, s[i, j] +, j) print ) i j

23 Example: A A 6 ( ( A ( A A 3 ) ) ( ( A 4 A 5 ) A 6 ) ) PRINT-OPT-PARENS(s, i, j) if i = j then print A i else print ( PRINT-OPT-PARENS(s, i, s[i, j]) PRINT-OPT-PARENS(s, s[i, j] +, j) print ) P-O-P(s,, 6) s[, 6] = 3 i =, j = 6 ( P-O-P (s,, 3) s[, 3] = i =, j = 3 ( P-O-P(s,, ) A ) s[..6,..6] P-O-P(s,, 3) s[, 3] = i =, j = 3 ( P-O-P (s,, ) A ) i P-O-P (s, 3, 3) A 3 3 j

24 Longest Common Subsequence Given two sequences X = x, x,, x m Y = y, y,, y n find a maximum length common subsequence (LCS) of X and Y E.g.: X = A, B, C, B, D, A, B Subsequences of X: A subset of elements in the sequence taken in order A, B, D, B, C, D, B, etc. 4

25 Example X = A, B, C, B, D, A, B X = A, B, C, B, D, A, B Y = B, D, C, A, B, A Y = B, D, C, A, B, A B, C, B, A and B, D, A, B are longest common subsequences of X and Y (length = 4) B, C, A, however is not a LCS of X and Y 5

26 Brute-Force Solution For every subsequence of X, check whether it s a subsequence of Y There are m subsequences of X to check Each subsequence takes (n) time to check scan Y for first letter, from there scan for second, and so on Running time: (n m ) 6

27 Making the choice X = A, B, D, E Y = Z, B, E Choice: include one element into the common sequence (E) and solve the resulting subproblem X = A, B, D, G Y = Z, B, D Choice: exclude an element from a string and solve the resulting subproblem 7

28 Notations Given a sequence X = x, x,, x m we define the i-th prefix of X, for i =,,,, m X i = x, x,, x i c[i, j] = the length of a LCS of the sequences X i = x, x,, x i and Y j = y, y,, y j 8

29 A Recursive Solution Case : x i = y j e.g.: X i = A, B, D, E Y j = Z, B, E c[i, j] = c[i -, j - ] + Append x i = y j to the LCS of X i- and Y j- Must find a LCS of X i- and Y j- optimal solution to a problem includes optimal solutions to subproblems 9

30 A Recursive Solution Case : x i y j e.g.: X i = A, B, D, G Y j = Z, B, D c[i, j] = max { c[i -, j], c[i, j-] } Must solve two problems find a LCS of X i- and Y j : X i- = A, B, D and Y j = Z, B, D find a LCS of X i and Y j- : X i = A, B, D, G and Y j = Z, B Optimal solution to a problem includes optimal solutions to subproblems 3

31 Overlapping Subproblems To find a LCS of X and Y we may need to find the LCS between X and Y n- and that of X m- and Y Both the above subproblems has the subproblem of finding the LCS of X m- and Y n- Subproblems share subsubproblems 3

32 3. Computing the Length of the LCS if i = or j = c[i, j] = c[i-, j-] + if x i = y j max(c[i, j-], c[i-, j]) if x i y j n y j: y y y n x i x x first second i m x m j 3

33 Additional Information if i,j = c[i, j] = c[i-, j-] + if x i = y j max(c[i, j-], c[i-, j]) if x i y j b & c: x i A B 3 C m D 3 n A C D F y j: c[i-,j] i c[i,j-] j A matrix b[i, j]: For a subproblem [i, j] it tells us what choice was made to obtain the optimal value If x i = y j b[i, j] = Else, if c[i -, j] c[i, j-] else b[i, j] = b[i, j] = 33

34 LCS-LENGTH(X, Y, m, n). for i to m. do c[i, ] 3. for j to n 4. do c[, j] 5. for i to m 6. do for j to n 7. do if x i = y j The length of the LCS if one of the sequences is empty is zero 8. then c[i, j] c[i -, j - ] + 9. b[i, j ]. else if c[i -, j] c[i, j - ]. then c[i, j] c[i -, j]. b[i, j] 3. else c[i, j] c[i, j - ] 4. b[i, j] 5.return c and b Case : x i = y j Case : x i y j Running time: (mn) 34

35 Example X = A, B, C, B, D, A Y = B, D, C, A, B, A If x i = y j b[i, j] = Else if c[i -, j] c[i, j-] else b[i, j] = b[i, j] = if i = or j = c[i, j] = c[i-, j-] x i A B C B D A B if x i = y j max(c[i, j-], c[i-, j]) if x i y j y j B D C A B A

36 4. Constructing a LCS Start at b[m, n] and follow the arrows When we encounter a in b[i, j] x i = y j is an element of the LCS x i A B C B D A B y j B D C A B A

37 PRINT-LCS(b, X, i, j). if i = or j =. then return 3. if b[i, j] = 4. then PRINT-LCS(b, X, i -, j - ) 5. print x i 6. elseif b[i, j] = Running time: (m + n) 7. then PRINT-LCS(b, X, i -, j) 8. else PRINT-LCS(b, X, i, j - ) Initial call: PRINT-LCS(b, X, length[x], length[y]) 37

CS473-Algorithms I. Lecture 10. Dynamic Programming. Cevdet Aykanat - Bilkent University Computer Engineering Department

CS473-Algorithms I. Lecture 10. Dynamic Programming. Cevdet Aykanat - Bilkent University Computer Engineering Department CS473-Algorithms I Lecture 1 Dynamic Programming 1 Introduction An algorithm design paradigm like divide-and-conquer Programming : A tabular method (not writing computer code) Divide-and-Conquer (DAC):

More information

Dynamic Programming II

Dynamic Programming II June 9, 214 DP: Longest common subsequence biologists often need to find out how similar are 2 DNA sequences DNA sequences are strings of bases: A, C, T and G how to define similarity? DP: Longest common

More information

Elements of Dynamic Programming. COSC 3101A - Design and Analysis of Algorithms 8. Discovering Optimal Substructure. Optimal Substructure - Examples

Elements of Dynamic Programming. COSC 3101A - Design and Analysis of Algorithms 8. Discovering Optimal Substructure. Optimal Substructure - Examples Elements of Dynamic Programming COSC 3A - Design and Analysis of Algorithms 8 Elements of DP Memoization Longest Common Subsequence Greedy Algorithms Many of these slides are taken from Monica Nicolescu,

More information

Lecture 8. Dynamic Programming

Lecture 8. Dynamic Programming Lecture 8. Dynamic Programming T. H. Cormen, C. E. Leiserson and R. L. Rivest Introduction to Algorithms, 3rd Edition, MIT Press, 2009 Sungkyunkwan University Hyunseung Choo choo@skku.edu Copyright 2000-2018

More information

Chapter 3 Dynamic programming

Chapter 3 Dynamic programming Chapter 3 Dynamic programming 1 Dynamic programming also solve a problem by combining the solutions to subproblems. But dynamic programming considers the situation that some subproblems will be called

More information

15.Dynamic Programming

15.Dynamic Programming 15.Dynamic Programming Dynamic Programming is an algorithm design technique for optimization problems: often minimizing or maximizing. Like divide and conquer, DP solves problems by combining solutions

More information

Introduction to Algorithms

Introduction to Algorithms Introduction to Algorithms Dynamic Programming Well known algorithm design techniques: Brute-Force (iterative) ti algorithms Divide-and-conquer algorithms Another strategy for designing algorithms is dynamic

More information

We ve done. Now. Next

We ve done. Now. Next We ve done Matroid Theory Task scheduling problem (another matroid example) Dijkstra s algorithm (another greedy example) Dynamic Programming Now Matrix Chain Multiplication Longest Common Subsequence

More information

Efficient Sequential Algorithms, Comp309. Problems. Part 1: Algorithmic Paradigms

Efficient Sequential Algorithms, Comp309. Problems. Part 1: Algorithmic Paradigms Efficient Sequential Algorithms, Comp309 Part 1: Algorithmic Paradigms University of Liverpool References: T. H. Cormen, C. E. Leiserson, R. L. Rivest Introduction to Algorithms, Second Edition. MIT Press

More information

12 Dynamic Programming (2) Matrix-chain Multiplication Segmented Least Squares

12 Dynamic Programming (2) Matrix-chain Multiplication Segmented Least Squares 12 Dynamic Programming (2) Matrix-chain Multiplication Segmented Least Squares Optimal substructure Dynamic programming is typically applied to optimization problems. An optimal solution to the original

More information

ECE250: Algorithms and Data Structures Dynamic Programming Part B

ECE250: Algorithms and Data Structures Dynamic Programming Part B ECE250: Algorithms and Data Structures Dynamic Programming Part B Ladan Tahvildari, PEng, SMIEEE Associate Professor Software Technologies Applied Research (STAR) Group Dept. of Elect. & Comp. Eng. University

More information

14 Dynamic. Matrix-chain multiplication. P.D. Dr. Alexander Souza. Winter term 11/12

14 Dynamic. Matrix-chain multiplication. P.D. Dr. Alexander Souza. Winter term 11/12 Algorithms Theory 14 Dynamic Programming (2) Matrix-chain multiplication P.D. Dr. Alexander Souza Optimal substructure Dynamic programming is typically applied to optimization problems. An optimal solution

More information

/463 Algorithms - Fall 2013 Solution to Assignment 3

/463 Algorithms - Fall 2013 Solution to Assignment 3 600.363/463 Algorithms - Fall 2013 Solution to Assignment 3 (120 points) I (30 points) (Hint: This problem is similar to parenthesization in matrix-chain multiplication, except the special treatment on

More information

Dynamic Programming. Design and Analysis of Algorithms. Entwurf und Analyse von Algorithmen. Irene Parada. Design and Analysis of Algorithms

Dynamic Programming. Design and Analysis of Algorithms. Entwurf und Analyse von Algorithmen. Irene Parada. Design and Analysis of Algorithms Entwurf und Analyse von Algorithmen Dynamic Programming Overview Introduction Example 1 When and how to apply this method Example 2 Final remarks Introduction: when recursion is inefficient Example: Calculation

More information

Lecture 13: Chain Matrix Multiplication

Lecture 13: Chain Matrix Multiplication Lecture 3: Chain Matrix Multiplication CLRS Section 5.2 Revised April 7, 2003 Outline of this Lecture Recalling matrix multiplication. The chain matrix multiplication problem. A dynamic programming algorithm

More information

Algorithm Design Techniques part I

Algorithm Design Techniques part I Algorithm Design Techniques part I Divide-and-Conquer. Dynamic Programming DSA - lecture 8 - T.U.Cluj-Napoca - M. Joldos 1 Some Algorithm Design Techniques Top-Down Algorithms: Divide-and-Conquer Bottom-Up

More information

Data Structures and Algorithms Week 8

Data Structures and Algorithms Week 8 Data Structures and Algorithms Week 8 Dynamic programming Fibonacci numbers Optimization problems Matrix multiplication optimization Principles of dynamic programming Longest Common Subsequence Algorithm

More information

Chain Matrix Multiplication

Chain Matrix Multiplication Chain Matrix Multiplication Version of November 5, 2014 Version of November 5, 2014 Chain Matrix Multiplication 1 / 27 Outline Outline Review of matrix multiplication. The chain matrix multiplication problem.

More information

Unit 4: Dynamic Programming

Unit 4: Dynamic Programming Unit 4: Dynamic Programming Course contents: Assembly-line scheduling Matrix-chain multiplication Longest common subsequence Optimal binary search trees Applications: Cell flipping, rod cutting, optimal

More information

y j LCS-Length(X,Y) Running time: O(st) set c[i,0] s and c[0,j] s to 0 for i=1 to s for j=1 to t if x i =y j then else if

y j LCS-Length(X,Y) Running time: O(st) set c[i,0] s and c[0,j] s to 0 for i=1 to s for j=1 to t if x i =y j then else if Recursive solution for finding LCS of X and Y if x s =y t, then find an LCS of X s-1 and Y t-1, and then append x s =y t to this LCS if x s y t, then solve two subproblems: (1) find an LCS of X s-1 and

More information

We augment RBTs to support operations on dynamic sets of intervals A closed interval is an ordered pair of real

We augment RBTs to support operations on dynamic sets of intervals A closed interval is an ordered pair of real 14.3 Interval trees We augment RBTs to support operations on dynamic sets of intervals A closed interval is an ordered pair of real numbers ], with Interval ]represents the set Open and half-open intervals

More information

CMPS 2200 Fall Dynamic Programming. Carola Wenk. Slides courtesy of Charles Leiserson with changes and additions by Carola Wenk

CMPS 2200 Fall Dynamic Programming. Carola Wenk. Slides courtesy of Charles Leiserson with changes and additions by Carola Wenk CMPS 00 Fall 04 Dynamic Programming Carola Wenk Slides courtesy of Charles Leiserson with changes and additions by Carola Wenk 9/30/4 CMPS 00 Intro. to Algorithms Dynamic programming Algorithm design technique

More information

Introduction to Algorithms

Introduction to Algorithms Introduction to Algorithms 6.046J/18.401J LECTURE 12 Dynamic programming Longest common subsequence Optimal substructure Overlapping subproblems Prof. Charles E. Leiserson Dynamic programming Design technique,

More information

ECE608 - Chapter 15 answers

ECE608 - Chapter 15 answers ¼ À ÈÌ Ê ½ ÈÊÇ Ä ÅË ½µ ½ º¾¹¾ ¾µ ½ º¾¹ µ ½ º¾¹ µ ½ º ¹½ µ ½ º ¹¾ µ ½ º ¹ µ ½ º ¹¾ µ ½ º ¹ µ ½ º ¹ ½¼µ ½ º ¹ ½½µ ½ ¹ ½ ECE608 - Chapter 15 answers (1) CLR 15.2-2 MATRIX CHAIN MULTIPLY(A, s, i, j) 1. if

More information

So far... Finished looking at lower bounds and linear sorts.

So far... Finished looking at lower bounds and linear sorts. So far... Finished looking at lower bounds and linear sorts. Next: Memoization -- Optimization problems - Dynamic programming A scheduling problem Matrix multiplication optimization Longest Common Subsequence

More information

Data Structure and Algorithm II Homework #2 Due: 13pm, Monday, October 31, === Homework submission instructions ===

Data Structure and Algorithm II Homework #2 Due: 13pm, Monday, October 31, === Homework submission instructions === Data Structure and Algorithm II Homework #2 Due: 13pm, Monday, October 31, 2011 === Homework submission instructions === Submit the answers for writing problems (including your programming report) through

More information

Algorithms: Dynamic Programming

Algorithms: Dynamic Programming Algorithms: Dynamic Programming Amotz Bar-Noy CUNY Spring 2012 Amotz Bar-Noy (CUNY) Dynamic Programming Spring 2012 1 / 58 Dynamic Programming General Strategy: Solve recursively the problem top-down based

More information

Matrix Multiplication. (Dynamic Programming)

Matrix Multiplication. (Dynamic Programming) Matrix Multiplication (Dynamic Programming) Matrix Multiplication: Review Suppose that A 1 is of size S 1 x S 2, and A 2 is of size S 2 x S 3. What is the time complexity of computing A 1 * A 2? What is

More information

Algorithm Design and Analysis

Algorithm Design and Analysis Algorithm Design and Analysis LECTURE 16 Dynamic Programming Least Common Subsequence Saving space Adam Smith Least Common Subsequence A.k.a. sequence alignment edit distance Longest Common Subsequence

More information

CS60020: Foundations of Algorithm Design and Machine Learning. Sourangshu Bhattacharya

CS60020: Foundations of Algorithm Design and Machine Learning. Sourangshu Bhattacharya CS60020: Foundations of Algorithm Design and Machine Learning Sourangshu Bhattacharya Dynamic programming Design technique, like divide-and-conquer. Example: Longest Common Subsequence (LCS) Given two

More information

15.4 Longest common subsequence

15.4 Longest common subsequence 15.4 Longest common subsequence Biological applications often need to compare the DNA of two (or more) different organisms A strand of DNA consists of a string of molecules called bases, where the possible

More information

CS Algorithms. Dynamic programming and memoization. (Based on slides by Luebke, Lim, Wenbin)

CS Algorithms. Dynamic programming and memoization. (Based on slides by Luebke, Lim, Wenbin) CS 7 - lgorithms Dynamic programming and memoization (ased on slides by Luebke, Lim, Wenbin) When to Use Dynamic Programming Usually used to solve Optimization problems Usually the problem can be formulated

More information

CMSC351 - Fall 2014, Homework #4

CMSC351 - Fall 2014, Homework #4 CMSC351 - Fall 2014, Homework #4 Due: November 14th at the start of class PRINT Name: Grades depend on neatness and clarity. Write your answers with enough detail about your approach and concepts used,

More information

Dynamic Programming Matrix-chain Multiplication

Dynamic Programming Matrix-chain Multiplication 1 / 32 Dynamic Programming Matrix-chain Multiplication CS 584: Algorithm Design and Analysis Daniel Leblanc 1 1 Senior Adjunct Instructor Portland State University Maseeh College of Engineering and Computer

More information

Dynamic Programming part 2

Dynamic Programming part 2 Dynamic Programming part 2 Week 7 Objectives More dynamic programming examples - Matrix Multiplication Parenthesis - Longest Common Subsequence Subproblem Optimal structure Defining the dynamic recurrence

More information

CS 231: Algorithmic Problem Solving

CS 231: Algorithmic Problem Solving CS 231: Algorithmic Problem Solving Naomi Nishimura Module 5 Date of this version: June 14, 2018 WARNING: Drafts of slides are made available prior to lecture for your convenience. After lecture, slides

More information

Module 27: Chained Matrix Multiplication and Bellman-Ford Shortest Path Algorithm

Module 27: Chained Matrix Multiplication and Bellman-Ford Shortest Path Algorithm Module 27: Chained Matrix Multiplication and Bellman-Ford Shortest Path Algorithm This module 27 focuses on introducing dynamic programming design strategy and applying it to problems like chained matrix

More information

Tutorial 6-7. Dynamic Programming and Greedy

Tutorial 6-7. Dynamic Programming and Greedy Tutorial 6-7 Dynamic Programming and Greedy Dynamic Programming Why DP? Natural Recursion may be expensive. For example, the Fibonacci: F(n)=F(n-1)+F(n-2) Recursive implementation memoryless : time= 1

More information

Dynamic Programming Group Exercises

Dynamic Programming Group Exercises Name: Name: Name: Dynamic Programming Group Exercises Adapted from material by Cole Frederick Please work the following problems in groups of 2 or 3. Use additional paper as needed, and staple the sheets

More information

Computer Sciences Department 1

Computer Sciences Department 1 1 Advanced Design and Analysis Techniques (15.1, 15.2, 15.3, 15.4 and 15.5) 3 Objectives Problem Formulation Examples The Basic Problem Principle of optimality Important techniques: dynamic programming

More information

Unit-5 Dynamic Programming 2016

Unit-5 Dynamic Programming 2016 5 Dynamic programming Overview, Applications - shortest path in graph, matrix multiplication, travelling salesman problem, Fibonacci Series. 20% 12 Origin: Richard Bellman, 1957 Programming referred to

More information

15.4 Longest common subsequence

15.4 Longest common subsequence 15.4 Longest common subsequence Biological applications often need to compare the DNA of two (or more) different organisms A strand of DNA consists of a string of molecules called bases, where the possible

More information

Homework3: Dynamic Programming - Answers

Homework3: Dynamic Programming - Answers Most Exercises are from your textbook: Homework3: Dynamic Programming - Answers 1. For the Rod Cutting problem (covered in lecture) modify the given top-down memoized algorithm (includes two procedures)

More information

Lecture 57 Dynamic Programming. (Refer Slide Time: 00:31)

Lecture 57 Dynamic Programming. (Refer Slide Time: 00:31) Programming, Data Structures and Algorithms Prof. N.S. Narayanaswamy Department of Computer Science and Engineering Indian Institution Technology, Madras Lecture 57 Dynamic Programming (Refer Slide Time:

More information

Dynamic Programming. Nothing to do with dynamic and nothing to do with programming.

Dynamic Programming. Nothing to do with dynamic and nothing to do with programming. Dynamic Programming Deliverables Dynamic Programming basics Binomial Coefficients Weighted Interval Scheduling Matrix Multiplication /1 Knapsack Longest Common Subsequence 6/12/212 6:56 PM copyright @

More information

Lecture 4: Dynamic programming I

Lecture 4: Dynamic programming I Lecture : Dynamic programming I Dynamic programming is a powerful, tabular method that solves problems by combining solutions to subproblems. It was introduced by Bellman in the 950 s (when programming

More information

Longest Common Subsequence. Definitions

Longest Common Subsequence. Definitions Longest Common Subsequence LCS is an interesting variation on the classical string matching problem: the task is that of finding the common portion of two strings (more precise definition in a couple of

More information

Partha Sarathi Manal

Partha Sarathi Manal MA 515: Introduction to Algorithms & MA353 : Design and Analysis of Algorithms [3-0-0-6] Lecture 29 http://www.iitg.ernet.in/psm/indexing_ma353/y09/index.html Partha Sarathi Manal psm@iitg.ernet.in Dept.

More information

Subsequence Definition. CS 461, Lecture 8. Today s Outline. Example. Assume given sequence X = x 1, x 2,..., x m. Jared Saia University of New Mexico

Subsequence Definition. CS 461, Lecture 8. Today s Outline. Example. Assume given sequence X = x 1, x 2,..., x m. Jared Saia University of New Mexico Subsequence Definition CS 461, Lecture 8 Jared Saia University of New Mexico Assume given sequence X = x 1, x 2,..., x m Let Z = z 1, z 2,..., z l Then Z is a subsequence of X if there exists a strictly

More information

Algorithms: COMP3121/3821/9101/9801

Algorithms: COMP3121/3821/9101/9801 NEW SOUTH WALES Algorithms: COMP3121/3821/9101/9801 Aleks Ignjatović School of Computer Science and Engineering University of New South Wales TOPIC 5: DYNAMIC PROGRAMMING COMP3121/3821/9101/9801 1 / 38

More information

CSE 373 Analysis of Algorithms, Fall Homework #3 Solutions Due Monday, October 18, 2003

CSE 373 Analysis of Algorithms, Fall Homework #3 Solutions Due Monday, October 18, 2003 Piyush Kumar CSE 373 Analysis of Algorithms, Fall 2003 Homework #3 Solutions Due Monday, October 18, 2003 Problem 1 Find an optimal parenthesization of a matrix chain product whose sequence of dimensions

More information

CMPS 102 Solutions to Homework 7

CMPS 102 Solutions to Homework 7 CMPS 102 Solutions to Homework 7 Kuzmin, Cormen, Brown, lbrown@soe.ucsc.edu November 17, 2005 Problem 1. 15.4-1 p.355 LCS Determine an LCS of x = (1, 0, 0, 1, 0, 1, 0, 1) and y = (0, 1, 0, 1, 1, 0, 1,

More information

CSE 101, Winter Design and Analysis of Algorithms. Lecture 11: Dynamic Programming, Part 2

CSE 101, Winter Design and Analysis of Algorithms. Lecture 11: Dynamic Programming, Part 2 CSE 101, Winter 2018 Design and Analysis of Algorithms Lecture 11: Dynamic Programming, Part 2 Class URL: http://vlsicad.ucsd.edu/courses/cse101-w18/ Goal: continue with DP (Knapsack, All-Pairs SPs, )

More information

Write an algorithm to find the maximum value that can be obtained by an appropriate placement of parentheses in the expression

Write an algorithm to find the maximum value that can be obtained by an appropriate placement of parentheses in the expression Chapter 5 Dynamic Programming Exercise 5.1 Write an algorithm to find the maximum value that can be obtained by an appropriate placement of parentheses in the expression x 1 /x /x 3 /... x n 1 /x n, where

More information

Algorithmic Paradigms. Chapter 6 Dynamic Programming. Steps in Dynamic Programming. Dynamic Programming. Dynamic Programming Applications

Algorithmic Paradigms. Chapter 6 Dynamic Programming. Steps in Dynamic Programming. Dynamic Programming. Dynamic Programming Applications lgorithmic Paradigms reed. Build up a solution incrementally, only optimizing some local criterion. hapter Dynamic Programming Divide-and-conquer. Break up a problem into two sub-problems, solve each sub-problem

More information

Dynamic Programming 1

Dynamic Programming 1 Dynamic Programming 1 Jie Wang University of Massachusetts Lowell Department of Computer Science 1 I thank Prof. Zachary Kissel of Merrimack College for sharing his lecture notes with me; some of the examples

More information

Longest Common Subsequences and Substrings

Longest Common Subsequences and Substrings Longest Common Subsequences and Substrings Version November 5, 2014 Version November 5, 2014 Longest Common Subsequences and Substrings 1 / 16 Longest Common Subsequence Given two sequences X = (x 1, x

More information

Presentation for use with the textbook, Algorithm Design and Applications, by M. T. Goodrich and R. Tamassia, Wiley, Dynamic Programming

Presentation for use with the textbook, Algorithm Design and Applications, by M. T. Goodrich and R. Tamassia, Wiley, Dynamic Programming Presentation for use with the textbook, Algorithm Design and Applications, by M. T. Goodrich and R. Tamassia, Wiley, 25 Dynamic Programming Terrible Fibonacci Computation Fibonacci sequence: f = f(n) 2

More information

CSE 417 Dynamic Programming (pt 4) Sub-problems on Trees

CSE 417 Dynamic Programming (pt 4) Sub-problems on Trees CSE 417 Dynamic Programming (pt 4) Sub-problems on Trees Reminders > HW4 is due today > HW5 will be posted shortly Dynamic Programming Review > Apply the steps... 1. Describe solution in terms of solution

More information

F(0)=0 F(1)=1 F(n)=F(n-1)+F(n-2)

F(0)=0 F(1)=1 F(n)=F(n-1)+F(n-2) Algorithms Dana Shapira Lesson #4: Dynamic programming Fibonacci Series F()= F()= F(n)=F(n-)+F(n-) Write a Divide and Conquer Algorithm! What is its running time? Binomial Coefficients n! n = ( )! n! Recursive

More information

Dynamic Programming in Haskell

Dynamic Programming in Haskell Dynamic Programming in Haskell Thomas Sutton, Anchor 2015-05-27 Introduction Introduction This is a talk in two parts: 1. First I ll introduce dynamic programming and a framework for implementing DP algorithms

More information

Introduction to Algorithms

Introduction to Algorithms Thomas H. Cormen Charles E. Leiserson Ronald L. Rivest Clifford Stein Introduction to Algorithms Second Edition The MIT Press Cambridge, Massachusetts London, England McGraw-Hill Book Company Boston Burr

More information

Design and Analysis of Algorithms 演算法設計與分析. Lecture 7 April 6, 2016 洪國寶

Design and Analysis of Algorithms 演算法設計與分析. Lecture 7 April 6, 2016 洪國寶 Design and Analysis of Algorithms 演算法設計與分析 Lecture 7 April 6, 2016 洪國寶 1 Course information (5/5) Grading (Tentative) Homework 25% (You may collaborate when solving the homework, however when writing up

More information

Dynamic Programming Shabsi Walfish NYU - Fundamental Algorithms Summer 2006

Dynamic Programming Shabsi Walfish NYU - Fundamental Algorithms Summer 2006 Dynamic Programming What is Dynamic Programming? Technique for avoiding redundant work in recursive algorithms Works best with optimization problems that have a nice underlying structure Can often be used

More information

Dynamic Programming CS 445. Example: Floyd Warshll Algorithm: Computing all pairs shortest paths

Dynamic Programming CS 445. Example: Floyd Warshll Algorithm: Computing all pairs shortest paths CS 44 Dynamic Programming Some of the slides are courtesy of Charles Leiserson with small changes by Carola Wenk Example: Floyd Warshll lgorithm: Computing all pairs shortest paths Given G(V,E), with weight

More information

Efficient Sequential Algorithms, Comp309. Motivation. Longest Common Subsequence. Part 3. String Algorithms

Efficient Sequential Algorithms, Comp309. Motivation. Longest Common Subsequence. Part 3. String Algorithms Efficient Sequential Algorithms, Comp39 Part 3. String Algorithms University of Liverpool References: T. H. Cormen, C. E. Leiserson, R. L. Rivest Introduction to Algorithms, Second Edition. MIT Press (21).

More information

10/24/ Rotations. 2. // s left subtree s right subtree 3. if // link s parent to elseif == else 11. // put x on s left

10/24/ Rotations. 2. // s left subtree s right subtree 3. if // link s parent to elseif == else 11. // put x on s left 13.2 Rotations MAT-72006 AA+DS, Fall 2013 24-Oct-13 368 LEFT-ROTATE(, ) 1. // set 2. // s left subtree s right subtree 3. if 4. 5. // link s parent to 6. if == 7. 8. elseif == 9. 10. else 11. // put x

More information

Dynamic Programmming: Activity Selection

Dynamic Programmming: Activity Selection Dynamic Programmming: Activity Selection Select the maximum number of non-overlapping activities from a set of n activities A = {a 1,, a n } (sorted by finish times). Identify easier subproblems to solve.

More information

15-451/651: Design & Analysis of Algorithms January 26, 2015 Dynamic Programming I last changed: January 28, 2015

15-451/651: Design & Analysis of Algorithms January 26, 2015 Dynamic Programming I last changed: January 28, 2015 15-451/651: Design & Analysis of Algorithms January 26, 2015 Dynamic Programming I last changed: January 28, 2015 Dynamic Programming is a powerful technique that allows one to solve many different types

More information

Dynamic Programming. An Enumeration Approach. Matrix Chain-Products. Matrix Chain-Products (not in book)

Dynamic Programming. An Enumeration Approach. Matrix Chain-Products. Matrix Chain-Products (not in book) Matrix Chain-Products (not in book) is a general algorithm design paradigm. Rather than give the general structure, let us first give a motivating example: Matrix Chain-Products Review: Matrix Multiplication.

More information

(Feodor F. Dragan) Department of Computer Science Kent State University. Advanced Algorithms, Feodor F. Dragan, Kent State University 1

(Feodor F. Dragan) Department of Computer Science Kent State University. Advanced Algorithms, Feodor F. Dragan, Kent State University 1 $GYDQFH $OJRULWKPV (Feodor F. Dragan) Department of Computer Science Kent State University Advanced Algorithms, Feodor F. Dragan, Kent State University Textbook: Thomas Cormen, Charles Leisterson, Ronald

More information

IN101: Algorithmic techniques Vladimir-Alexandru Paun ENSTA ParisTech

IN101: Algorithmic techniques Vladimir-Alexandru Paun ENSTA ParisTech IN101: Algorithmic techniques Vladimir-Alexandru Paun ENSTA ParisTech License CC BY-NC-SA 2.0 http://creativecommons.org/licenses/by-nc-sa/2.0/fr/ Outline Previously on IN101 Python s anatomy Functions,

More information

Greedy Algorithms. Algorithms

Greedy Algorithms. Algorithms Greedy Algorithms Algorithms Greedy Algorithms Many algorithms run from stage to stage At each stage, they make a decision based on the information available A Greedy algorithm makes decisions At each

More information

Dynamic Programming. Lecture Overview Introduction

Dynamic Programming. Lecture Overview Introduction Lecture 12 Dynamic Programming 12.1 Overview Dynamic Programming is a powerful technique that allows one to solve many different types of problems in time O(n 2 ) or O(n 3 ) for which a naive approach

More information

Lecture 22: Dynamic Programming

Lecture 22: Dynamic Programming Lecture 22: Dynamic Programming COSC242: Algorithms and Data Structures Brendan McCane Department of Computer Science, University of Otago Dynamic programming The iterative and memoised algorithms for

More information

Dynamic Programming. Outline and Reading. Computing Fibonacci

Dynamic Programming. Outline and Reading. Computing Fibonacci Dynamic Programming Dynamic Programming version 1.2 1 Outline and Reading Matrix Chain-Product ( 5.3.1) The General Technique ( 5.3.2) -1 Knapsac Problem ( 5.3.3) Dynamic Programming version 1.2 2 Computing

More information

Design and Analysis of Algorithms

Design and Analysis of Algorithms Design and Analysis of Algorithms CSE 5311 Lecture 16 Greedy algorithms Junzhou Huang, Ph.D. Department of Computer Science and Engineering CSE5311 Design and Analysis of Algorithms 1 Overview A greedy

More information

Dynamic Programming: 1D Optimization. Dynamic Programming: 2D Optimization. Fibonacci Sequence. Crazy 8 s. Edit Distance

Dynamic Programming: 1D Optimization. Dynamic Programming: 2D Optimization. Fibonacci Sequence. Crazy 8 s. Edit Distance Dynamic Programming: 1D Optimization Fibonacci Sequence To efficiently calculate F [x], the xth element of the Fibonacci sequence, we can construct the array F from left to right (or bottom up ). We start

More information

Dynamic Programming. CIS 110, Fall University of Pennsylvania

Dynamic Programming. CIS 110, Fall University of Pennsylvania Dynamic Programming CIS 110, Fall 2012 University of Pennsylvania Dynamic Programming Dynamic programming records saves computation for reuse later. Programming: in the optimization sense ( Linear Programming

More information

CSC 505, Spring 2005 Week 6 Lectures page 1 of 9

CSC 505, Spring 2005 Week 6 Lectures page 1 of 9 CSC 505, Spring 2005 Week 6 Lectures page 1 of 9 Objectives: learn general strategies for problems about order statistics learn how to find the median (or k-th largest) in linear average-case number of

More information

DESIGN AND ANALYSIS OF ALGORITHMS (DAA 2017)

DESIGN AND ANALYSIS OF ALGORITHMS (DAA 2017) DESIGN AND ANALYSIS OF ALGORITHMS (DAA 2017) Veli Mäkinen Design and Analysis of Algorithms 2017 week 4 11.8.2017 1 Dynamic Programming Week 4 2 Design and Analysis of Algorithms 2017 week 4 11.8.2017

More information

Algorithms IV. Dynamic Programming. Guoqiang Li. School of Software, Shanghai Jiao Tong University

Algorithms IV. Dynamic Programming. Guoqiang Li. School of Software, Shanghai Jiao Tong University Algorithms IV Dynamic Programming Guoqiang Li School of Software, Shanghai Jiao Tong University Dynamic Programming Shortest Paths in Dags, Revisited Shortest Paths in Dags, Revisited The special distinguishing

More information

Algorithms for Data Science

Algorithms for Data Science Algorithms for Data Science CSOR W4246 Eleni Drinea Computer Science Department Columbia University Thursday, October 1, 2015 Outline 1 Recap 2 Shortest paths in graphs with non-negative edge weights (Dijkstra

More information

CS 170 DISCUSSION 8 DYNAMIC PROGRAMMING. Raymond Chan raychan3.github.io/cs170/fa17.html UC Berkeley Fall 17

CS 170 DISCUSSION 8 DYNAMIC PROGRAMMING. Raymond Chan raychan3.github.io/cs170/fa17.html UC Berkeley Fall 17 CS 170 DISCUSSION 8 DYNAMIC PROGRAMMING Raymond Chan raychan3.github.io/cs170/fa17.html UC Berkeley Fall 17 DYNAMIC PROGRAMMING Recursive problems uses the subproblem(s) solve the current one. Dynamic

More information

Algorithms Dr. Haim Levkowitz

Algorithms Dr. Haim Levkowitz 91.503 Algorithms Dr. Haim Levkowitz Fall 2007 Lecture 4 Tuesday, 25 Sep 2007 Design Patterns for Optimization Problems Greedy Algorithms 1 Greedy Algorithms 2 What is Greedy Algorithm? Similar to dynamic

More information

Dynamic Programming. December 15, CMPE 250 Dynamic Programming December 15, / 60

Dynamic Programming. December 15, CMPE 250 Dynamic Programming December 15, / 60 Dynamic Programming December 15, 2016 CMPE 250 Dynamic Programming December 15, 2016 1 / 60 Why Dynamic Programming Often recursive algorithms solve fairly difficult problems efficiently BUT in other cases

More information

Longest Common Subsequence, Knapsack, Independent Set Scribe: Wilbur Yang (2016), Mary Wootters (2017) Date: November 6, 2017

Longest Common Subsequence, Knapsack, Independent Set Scribe: Wilbur Yang (2016), Mary Wootters (2017) Date: November 6, 2017 CS161 Lecture 13 Longest Common Subsequence, Knapsack, Independent Set Scribe: Wilbur Yang (2016), Mary Wootters (2017) Date: November 6, 2017 1 Overview Last lecture, we talked about dynamic programming

More information

Outline. CS38 Introduction to Algorithms. Fast Fourier Transform (FFT) Fast Fourier Transform (FFT) Fast Fourier Transform (FFT)

Outline. CS38 Introduction to Algorithms. Fast Fourier Transform (FFT) Fast Fourier Transform (FFT) Fast Fourier Transform (FFT) Outline CS8 Introduction to Algorithms Lecture 9 April 9, 0 Divide and Conquer design paradigm matrix multiplication Dynamic programming design paradigm Fibonacci numbers weighted interval scheduling knapsack

More information

CMSC 451: Dynamic Programming

CMSC 451: Dynamic Programming CMSC 41: Dynamic Programming Slides By: Carl Kingsford Department of Computer Science University of Maryland, College Park Based on Sections 6.1&6.2 of Algorithm Design by Kleinberg & Tardos. Dynamic Programming

More information

CSE 4/531 Solution 3

CSE 4/531 Solution 3 CSE 4/531 Solution 3 Edited by Le Fang November 7, 2017 Problem 1 M is a given n n matrix and we want to find a longest sequence S from elements of M such that the indexes of elements in M increase and

More information

CS141: Intermediate Data Structures and Algorithms Greedy Algorithms

CS141: Intermediate Data Structures and Algorithms Greedy Algorithms CS141: Intermediate Data Structures and Algorithms Greedy Algorithms Amr Magdy Activity Selection Problem Given a set of activities S = {a 1, a 2,, a n } where each activity i has a start time s i and

More information

Analysis of Algorithms. Unit 4 - Analysis of well known Algorithms

Analysis of Algorithms. Unit 4 - Analysis of well known Algorithms Analysis of Algorithms Unit 4 - Analysis of well known Algorithms 1 Analysis of well known Algorithms Brute Force Algorithms Greedy Algorithms Divide and Conquer Algorithms Decrease and Conquer Algorithms

More information

Week 11: Minimum Spanning trees

Week 11: Minimum Spanning trees Week 11: Minimum Spanning trees Agenda: Minimum Spanning Trees Prim s Algorithm Reading: Textbook : 61-7 1 Week 11: Minimum Spanning trees Minimum spanning tree (MST) problem: Input: edge-weighted (simple,

More information

COMP251: Greedy algorithms

COMP251: Greedy algorithms COMP251: Greedy algorithms Jérôme Waldispühl School of Computer Science McGill University Based on (Cormen et al., 2002) Based on slides from D. Plaisted (UNC) & (goodrich & Tamassia, 2009) Disjoint sets

More information

Optimization II: Dynamic Programming

Optimization II: Dynamic Programming Chapter 12 Optimization II: Dynamic Programming In the last chapter, we saw that greedy algorithms are efficient solutions to certain optimization problems. However, there are optimization problems for

More information

Framework for Design of Dynamic Programming Algorithms

Framework for Design of Dynamic Programming Algorithms CSE 441T/541T Advanced Algorithms September 22, 2010 Framework for Design of Dynamic Programming Algorithms Dynamic programming algorithms for combinatorial optimization generalize the strategy we studied

More information

CSE 417 Dynamic Programming (pt 5) Multiple Inputs

CSE 417 Dynamic Programming (pt 5) Multiple Inputs CSE 417 Dynamic Programming (pt 5) Multiple Inputs Reminders > HW5 due Wednesday Dynamic Programming Review > Apply the steps... optimal substructure: (small) set of solutions, constructed from solutions

More information

UC Berkeley CS 170: Efficient Algorithms and Intractable Problems Handout 15 Lecturer: Michael Jordan October 26, Notes 15 for CS 170

UC Berkeley CS 170: Efficient Algorithms and Intractable Problems Handout 15 Lecturer: Michael Jordan October 26, Notes 15 for CS 170 UC Berkeley CS 170: Efficient Algorithms and Intractable Problems Handout 15 Lecturer: Michael Jordan October 26, 2005 Notes 15 for CS 170 1 Introduction to Dynamic Programming Consider the following algorithm

More information

CS141: Intermediate Data Structures and Algorithms Dynamic Programming

CS141: Intermediate Data Structures and Algorithms Dynamic Programming CS141: Intermediate Data Structures and Algorithms Dynamic Programming Amr Magdy Programming? In this context, programming is a tabular method Other examples: Linear programing Integer programming 2 Rod

More information

More Dynamic Programming

More Dynamic Programming CS 374: Algorithms & Models of Computation, Fall 2015 More Dynamic Programming Lecture 12 October 8, 2015 Chandra & Manoj (UIUC) CS374 1 Fall 2015 1 / 43 What is the running time of the following? Consider

More information