CS141: Intermediate Data Structures and Algorithms Dynamic Programming
|
|
|
- Adrian Parker
- 5 years ago
- Views:
Transcription
1 CS141: Intermediate Data Structures and Algorithms Dynamic Programming Amr Magdy
2 Programming? In this context, programming is a tabular method Other examples: Linear programing Integer programming 2
3 Rod Cutting Problem 3
4 Rod Cutting Problem 4
5 Rod Cutting Problem 5
6 Rod Cutting Problem 6
7 Rod Cutting Problem 7
8 Rod Cutting Problem 8
9 Rod Cutting Problem 9
10 Rod Cutting Problem Given a rod of length n and prices p i, find the cutting strategy that makes the maximum revenue In the example: (2+2) cutting makes r=5+5=10 10
11 Rod Cutting Problem Naïve: try all combinations 11
12 Rod Cutting Problem Naïve: try all combinations How many? 12
13 Rod Cutting Problem Naïve: try all combinations How many? 0 cut: 1 1 cut: (n-1) 2 cuts: n-1 C 2 = Θ(n 2 ) 3 cuts: n-1 C 3 = Θ(n 3 ) n cuts: n-1 C n-1 = Θ(1) 13
14 Rod Cutting Problem Naïve: try all combinations How many? 0 cut: 1 1 cut: (n-1) 2 cuts: n-1 C 2 = Θ(n 2 ) 3 cuts: n-1 C 3 = Θ(n 3 ) n cuts: n-1 C n-1 = Θ(1) Total: Θ(1+n+n 2 +n 3 + +n n/2 +.+n 3 +n 2 +n+1) Total: O(n n ) 14
15 Rod Cutting Problem Naïve: try all combinations How many? 0 cut: 1 1 cut: (n-1) 2 cuts: n-1 C 2 = Θ(n 2 ) 3 cuts: n-1 C 3 = Θ(n 3 ) n cuts: n-1 C n-1 = Θ(1) Total: Θ(1+n+n 2 +n 3 + +n n/2 +.+n 3 +n 2 +n+1) Total: O(n n ) Better solution? Can I divide and conquer? 15
16 Rod Cutting Problem Naïve: try all combinations How many? 0 cut: 1 1 cut: (n-1) 2 cuts: n-1 C 2 = Θ(n 2 ) 3 cuts: n-1 C 3 = Θ(n 3 ) n cuts: n-1 C n-1 = Θ(1) Total: Θ(1+n+n 2 +n 3 + +n n/2 +.+n 3 +n 2 +n+1) Total: O(n n ) Better solution? Can I divide and conquer? divide 16
17 Rod Cutting Problem Naïve: try all combinations How many? 0 cut: 1 1 cut: (n-1) 2 cuts: n-1 C 2 = Θ(n 2 ) 3 cuts: n-1 C 3 = Θ(n 3 ) n cuts: n-1 C n-1 = Θ(1) Total: Θ(1+n+n 2 +n 3 + +n n/2 +.+n 3 +n 2 +n+1) Total: O(n n ) Better solution? Can I divide and conquer? divide conquer 17
18 Rod Cutting Problem Naïve: try all combinations How many? 0 cut: 1 1 cut: (n-1) 2 cuts: n-1 C 2 = Θ(n 2 ) 3 cuts: n-1 C 3 = Θ(n 3 ) n cuts: n-1 C n-1 = Θ(1) Total: Θ(1+n+n 2 +n 3 + +n n/2 +.+n 3 +n 2 +n+1) Total: O(n n ) Better solution? Can I divide and conquer? But I don t really know the best way to divide divide conquer 18
19 Rod Cutting Problem Naïve: try all combinations How many? 0 cut: 1 1 cut: (n-1) 2 cuts: n-1 C 2 = Θ(n 2 ) 3 cuts: n-1 C 3 = Θ(n 3 ) n cuts: n-1 C n-1 = Θ(1) Total: Θ(1+n+n 2 +n 3 + +n n/2 +.+n 3 +n 2 +n+1) Total: O(n n ) Better solution? Can I divide and conquer? But I don t really know the best way to divide 19
20 Rod Cutting Problem Naïve: try all combinations How many? 0 cut: 1 1 cut: (n-1) 2 cuts: n-1 C 2 = Θ(n 2 ) 3 cuts: n-1 C 3 = Θ(n 3 ) n cuts: n-1 C n-1 = Θ(1) Total: Θ(1+n+n 2 +n 3 + +n n/2 +.+n 3 +n 2 +n+1) Total: O(n n ) Better solution? Can I divide and conquer? But I don t really know the best way to divide 20
21 Rod Cutting Problem Naïve: try all combinations How many? 0 cut: 1 1 cut: (n-1) 2 cuts: n-1 C 2 = Θ(n 2 ) 3 cuts: n-1 C 3 = Θ(n 3 ) n cuts: n-1 C n-1 = Θ(1) Total: Θ(1+n+n 2 +n 3 + +n n/2 +.+n 3 +n 2 +n+1) Total: O(n n ) Better solution? Can I divide and conquer? But I don t really know the best way to divide 21
22 Rod Cutting Problem Recursive top-down algorithm 22
23 Rod Cutting Problem Better solution? Can I divide and conquer? But I don t really know the best way to divide How many subproblems (recursive calls)? 23
24 Rod Cutting Problem Better solution? Can I divide and conquer? But I don t really know the best way to divide How many subproblems (recursive calls)? 24
25 Rod Cutting Problem Better solution? Can I divide and conquer? But I don t really know the best way to divide How many subproblems (recursive calls)? (Prove by induction) 25
26 Then, T n = 2 n 26 Rod Cutting Recursive Complexity Find the complexity of T n = 1 + σ n 1 j=0 T(j) Proof by induction: Assume the solution is some function X(n) Show that X(n) is true for the smallest n (the base case), e.g., n=0 Prove that X(n+1) is a solution for T(n+1) given X(n) You are done Given T n = 1 + σ n 1 j=0 T(j) Assume T n = 2 n T 0 = 1 + σ 1 j=0 T j = 1 = 2 0 (base case) n T n + 1 = 1 + σ j=0 T j = 1 + σ n 1 j=0 T j + T n = T n + T n = 2T n = 2 2 n = 2 n+1
27 Rod Cutting Problem Better solution? Can I divide and conquer? But I don t really know the best way to divide How many subproblems (recursive calls)? (Prove by induction) Can we do better? 27
28 Rod Cutting Problem Better solution? Can I divide and conquer? But I don t really know the best way to divide How many subproblems (recursive calls)? Can we do better? 28
29 Rod Cutting Problem Subproblem overlapping No need to re-solve the same problem 29
30 Rod Cutting Problem Subproblem overlapping Idea: No need to re-solve the same problem Solve each subproblem once Write down the solution in a lookup table (array, hashtable, etc) When needed again, look it up in Θ(1) 30
31 Rod Cutting Problem Subproblem overlapping Idea: No need to re-solve the same problem Dynamic Programming Solve each subproblem once Write down the solution in a lookup table (array, hashtable, etc) When needed again, look it up in Θ(1) 31
32 Rod Cutting Problem Recursive top-down dynamic programming algorithm 32
33 Rod Cutting Problem Recursive top-down dynamic programming algorithm Θ(n 2 ) 33
34 Rod Cutting Problem Bottom-up dynamic programming algorithm I know I will need the smaller problems solve them first Solve problem of size 0, then 1, then 2, then 3, then n 34
35 Rod Cutting Problem Bottom-up dynamic programming algorithm I know I will need the smaller problems solve them first Solve problem of size 0, then 1, then 2, then 3, then n 35
36 Rod Cutting Problem Bottom-up dynamic programming algorithm I know I will need the smaller problems solve them first Solve problem of size 0, then 1, then 2, then 3, then n Θ(n 2 ) 36
37 Elements of a Dynamic Programming Problem Optimal substructure Optimal solution of a larger problem comes from optimal solutions of smaller problems Subproblem overlapping Same exact sub-problems are solved again and again 37
38 Dynamic Programming vs. D&C How different? 38
39 Dynamic Programming vs. D&C How different? No subproblem overlapping Each subproblem with distinct input is a new problem Not necessarily optimization problems, i.e., no objective function 39
40 Reconstructing Solution Rod cutting problem: What are the actual cuts? Not only the best revenue (the optimal objective function value) 40
41 Reconstructing Solution Rod cutting problem: What are the actual cuts? Not only the best revenue (the optimal objective function value) 41
42 Reconstructing Solution Rod cutting problem: What are the actual cuts? Not only the best revenue (the optimal objective function value) Let s trace examples 42
43 Matrix Chain Multiplication How to multiply a chain of four matrices? 43
44 Matrix Chain Multiplication How to multiply a chain of four matrices? 44
45 Matrix Chain Multiplication How to multiply a chain of four matrices? Does it really make a difference? 45
46 Matrix Chain Multiplication How to multiply a chain of four matrices? Does it really make a difference? # of multiplications: A.rows*B.cols*A.cols 46
47 Matrix Chain Multiplication Does it really make a difference? # of multiplications: A.rows*B.cols*A.cols Example: A1*A2*A3 Dimensions: 10x100x5x50 # of multiplications in ((A1*A2)*A3)=10*100*5+10*5*50=7.5K # of multiplications in (A1*(A2*A3))=100*5*50+10*100*50=75K 47
48 Matrix Chain Multiplication Given n matrices A 1 A 2 A n of dimensions p 0 p 1 p n, find the optimal parentheses to multiply the matrix chain 48
49 Matrix Chain Multiplication Given n matrices A 1 A 2 A n of dimensions p 0 p 1 p n, find the optimal parentheses to multiply the matrix chain A 1 A 2 A 3 A 4 A 5 A n 49
50 Matrix Chain Multiplication Given n matrices A 1 A 2 A n of dimensions p 0 p 1 p n, find the optimal parentheses to multiply the matrix chain A 1 A 2 A 3 A 4 A 5 A n (A 1 A 2 A 3 )(A 4 A 5 A n ) 50
51 Matrix Chain Multiplication Given n matrices A 1 A 2 A n of dimensions p 0 p 1 p n, find the optimal parentheses to multiply the matrix chain A 1 A 2 A 3 A 4 A 5 A n (A 1 A 2 A 3 )(A 4 A 5 A n ) Sub-chains C1 = (A 1 A 2 A 3 ), C2 = (A 4 A 5 A n ) 51
52 Matrix Chain Multiplication Given n matrices A 1 A 2 A n of dimensions p 0 p 1 p n, find the optimal parentheses to multiply the matrix chain A 1 A 2 A 3 A 4 A 5 A n (A 1 A 2 A 3 )(A 4 A 5 A n ) Sub-chains C1 = (A 1 A 2 A 3 ), C2 = (A 4 A 5 A n ) Total Cost C = cost(c1)+cost(c2)+p 0 p 3 p n 52
53 Matrix Chain Multiplication Given n matrices A 1 A 2 A n of dimensions p 0 p 1 p n, find the optimal parentheses to multiply the matrix chain A 1 A 2 A 3 A 4 A 5 A n (A 1 A 2 A 3 )(A 4 A 5 A n ) Sub-chains C1 = (A 1 A 2 A 3 ), C2 = (A 4 A 5 A n ) Total Cost C = cost(c1)+cost(c2)+p 0 p 3 p n Then, if cost(c1) and cost(c2) are minimal (i.e., optimal), then C is optimal (optimal substructure holds) 53
54 Matrix Chain Multiplication Given n matrices A 1 A 2 A n of dimensions p 0 p 1 p n, find the optimal parentheses to multiply the matrix chain A 1 A 2 A 3 A 4 A 5 A n (A 1 A 2 A 3 )(A 4 A 5 A n ) Sub-chains C1 = (A 1 A 2 A 3 ), C2 = (A 4 A 5 A n ) Total Cost C = cost(c1)+cost(c2)+p 0 p 3 p n Then, if cost(c1) and cost(c2) are minimal (i.e., optimal), then C is optimal (optimal substructure holds) Proof by contradiction: Given C is optimal, are cost(c1)=c1 and cost(c2)=c2 optimal? Assume c1 is NOT optimal, then Ǝ an optimal solution of cost c1 < c1 Then c1 +c2+p < c1+c2+p C < C Then C is not optimal contradiction! Then C1 has to be optimal optimal substructure holds 54
55 Matrix Chain Multiplication Given n matrices A 1 A 2 A n of dimensions p 0 p 1 p n, find the optimal parentheses to multiply the matrix chain A 1 A 2 A 3 A 4 A 5 A n (A 1 A 2 A 3 )(A 4 A 5 A n ) Sub-chains C1 = (A 1 A 2 A 3 ), C2 = (A 4 A 5 A n ) Total Cost C = cost(c1)+cost(c2)+p 0 p 3 p n Then, if cost(c1) and cost(c2) are minimal (i.e., optimal), then C is optimal (optimal substructure holds) Optimal C1, C2 might be one of different options C1 = (A 1 A 2 ), C2 = (A 3 A 4 A 5 A n ) C1 = (A 1 ), C2 = (A 2 A 3 A 4 A 5 A n ) C1 = (A 1 A 2 A 3 A 4 ), C2 = (A 5 A n ). 55
56 Matrix Chain Multiplication Generally: A i A k A j of dimensions p i p k p j (A i A k )(A k+1 A j ), where k=i,i+1, j-1 Then, solve each sub-chains recursively 56
57 Matrix Chain Multiplication Generally: A i A k A j of dimensions p i p k p j (A i A k )(A k+1 A j ), where k=i,i+1, j-1 Then, solve each sub-chains recursively 57
58 Matrix Chain Multiplication Generally: A i A k A j of dimensions p i p k p j (A i A k )(A k+1 A j ), where k=i,i+1, j-1 Then, solve each sub-chains recursively A 1 A 2 A 3 A 4 A 5 k=1 k=2 k=3 k=4 (A 1 ) (A 2 A 3 A 4 A 5 ) (A 1 A 2 ) ( A 3 A 4 A 5 ) (A 1 A 2 A 3 )(A 4 A 5 ) (A 1 A 2 A 3 A 4 )(A 5 ).. k=1 k=2 k=3 k=1 k=2 (A 2 )(A 3 A 4 A 5 ) (A 2 A 3 )(A 4 A 5 ) (A 2 A 3 A 4 )(A 5 ) (A 1 )(A 2 A 3 ) (A 1 A 2 )(A 3 ). k=1 k=2 (A 2 )(A 3 A 4 ) (A 2 A 3 )(A 4 ) 58
59 Matrix Chain Multiplication Generally: A i A k A j of dimensions p i p k p j (A i A k )(A k+1 A j ), where k=i,i+1, j-1 Then, solve each sub-chains recursively A 1 A 2 A 3 A 4 A 5 k=1 k=2 k=3 k=4 (A 1 ) (A 2 A 3 A 4 A 5 ) (A 1 A 2 ) ( A 3 A 4 A 5 ) (A 1 A 2 A 3 )(A 4 A 5 ) (A 1 A 2 A 3 A 4 )(A 5 ).. k=1 k=2 k=3 k=1 k=2 (A 2 )(A 3 A 4 A 5 ) (A 2 A 3 )(A 4 A 5 ) (A 2 A 3 A 4 )(A 5 ) (A 1 )(A 2 A 3 ) (A 1 A 2 )(A 3 ). k=1 k=2 (A 2 )(A 3 A 4 ) (A 2 A 3 )(A 4 ) 59
60 Matrix Chain Multiplication Generally: A i A k A j of dimensions p i p k p j (A i A k )(A k+1 A j ), where k=i,i+1, j-1 Then, solve each sub-chains recursively Obviously, a lot of overlapping subproblems appear 60
61 Matrix Chain Multiplication Generally: A i A k A j of dimensions p i p k p j (A i A k )(A k+1 A j ), where k=i,i+1, j-1 Then, solve each sub-chains recursively Obviously, a lot of overlapping subproblems appear Optimal substructure + subproblem overlapping = dynamic programming 61
62 Matrix Chain Multiplication: Designing Algorithm What is the smallest subproblem? 62
63 Matrix Chain Multiplication: Designing Algorithm What is the smallest subproblem? A chain of length 2 63
64 Matrix Chain Multiplication: Designing Algorithm What is the smallest subproblem? A chain of length 2 Solve all chains of length 2, then 3, then 4, n 64
65 Matrix Chain Multiplication: Designing Algorithm What is the smallest subproblem? A chain of length 2 Solve all chains of length 2, then 3, then 4, n A1: 30x35 A2: 35x15 A3: 15x5 A4: 5x10 A5: 10x20 A6: 20x25 65
66 Matrix Chain Multiplication: Designing Algorithm What is the smallest subproblem? A chain of length 2 Solve all chains of length 2, then 3, then 4, n A1: 30x35 A2: 35x15 A3: 15x5 A4: 5x10 A5: 10x20 A6: 20x25 66
67 Matrix Chain Multiplication: Designing Algorithm What is the smallest subproblem? A chain of length 2 Solve all chains of length 2, then 3, then 4, n A1: 30x35 A2: 35x15 A3: 15x5 A4: 5x10 A5: 10x20 A6: 20x25 67
68 Matrix Chain Multiplication: Designing Algorithm What is the smallest subproblem? A chain of length 2 Solve all chains of length 2, then 3, then 4, n A 1 A 2 A 3 = (A 1 A 2 )A 3 Or A 1 (A 2 A 3 ) A1: 30x35 A2: 35x15 A3: 15x5 A4: 5x10 A5: 10x20 A6: 20x25 68
69 Matrix Chain Multiplication: Designing Algorithm What is the smallest subproblem? A chain of length 2 Solve all chains of length 2, then 3, then 4, n A 2 A 3 A 4 = (A 2 A 3 )A 4 Or A 2 (A 3 A 4 ) A1: 30x35 A2: 35x15 A3: 15x5 A4: 5x10 A5: 10x20 A6: 20x25 69
70 Matrix Chain Multiplication: Designing Algorithm What is the smallest subproblem? A chain of length 2 Solve all chains of length 2, then 3, then 4, n A 3 A 4 A 5 = (A 3 A 4 )A 5 Or A 3 (A 4 A 5 ) A1: 30x35 A2: 35x15 A3: 15x5 A4: 5x10 A5: 10x20 A6: 20x25 70
71 Matrix Chain Multiplication: Designing Algorithm What is the smallest subproblem? A chain of length 2 Solve all chains of length 2, then 3, then 4, n A 4 A 5 A 6 = (A 4 A 5 )A 6 Or A 4 (A 5 A 6 ) A1: 30x35 A2: 35x15 A3: 15x5 A4: 5x10 A5: 10x20 A6: 20x25 71
72 Matrix Chain Multiplication: Designing Algorithm What is the smallest subproblem? A chain of length 2 Solve all chains of length 2, then 3, then 4, n A 1 A 2 A 3 A 4 = A 1 (A 2 A 3 A 4 ) Or (A 1 A 2 )(A 3 A 4 ) Or (A 1 A 2 A 3 ) A 4 A1: 30x35 A2: 35x15 A3: 15x5 A4: 5x10 A5: 10x20 A6: 20x25 72
73 Matrix Chain Multiplication: Designing Algorithm What is the smallest subproblem? A chain of length 2 Solve all chains of length 2, then 3, then 4, n A1: 30x35 A2: 35x15 A3: 15x5 A4: 5x10 A5: 10x20 A6: 20x25 73
74 Matrix Chain Multiplication: Designing Algorithm What is the smallest subproblem? A chain of length 2 Solve all chains of length 2, then 3, then 4, n A1: 30x35 A2: 35x15 A3: 15x5 A4: 5x10 A5: 10x20 A6: 20x25 74
75 Matrix Chain Multiplication: Designing Algorithm What is the smallest subproblem? A chain of length 2 Solve all chains of length 2, then 3, then 4, n A1: 30x35 A2: 35x15 A3: 15x5 A4: 5x10 A5: 10x20 A6: 20x25 75
76 Matrix Chain Multiplication 76
77 Longest Common Subsequence Required reading: Book section
78 Book Readings Ch. 15:
Dynamic Programming Group Exercises
Name: Name: Name: Dynamic Programming Group Exercises Adapted from material by Cole Frederick Please work the following problems in groups of 2 or 3. Use additional paper as needed, and staple the sheets
Lecture 4: Dynamic programming I
Lecture : Dynamic programming I Dynamic programming is a powerful, tabular method that solves problems by combining solutions to subproblems. It was introduced by Bellman in the 950 s (when programming
Algorithms. Ch.15 Dynamic Programming
Algorithms Ch.15 Dynamic Programming Dynamic Programming Not a specific algorithm, but a technique (like divide-and-conquer). Developed back in the day when programming meant tabular method (like linear
1 i n (p i + r n i ) (Note that by allowing i to be n, we handle the case where the rod is not cut at all.)
Dynamic programming is a problem solving method that is applicable to many different types of problems. I think it is best learned by example, so we will mostly do examples today. 1 Rod cutting Suppose
CS 380 ALGORITHM DESIGN AND ANALYSIS
CS 380 ALGORITHM DESIGN AND ANALYSIS Lecture 14: Dynamic Programming Text Reference: Chapter 15 Dynamic Programming We know that we can use the divide-and-conquer technique to obtain efficient algorithms
CS141: Intermediate Data Structures and Algorithms Greedy Algorithms
CS141: Intermediate Data Structures and Algorithms Greedy Algorithms Amr Magdy Activity Selection Problem Given a set of activities S = {a 1, a 2,, a n } where each activity i has a start time s i and
Lecture 8. Dynamic Programming
Lecture 8. Dynamic Programming T. H. Cormen, C. E. Leiserson and R. L. Rivest Introduction to Algorithms, 3rd Edition, MIT Press, 2009 Sungkyunkwan University Hyunseung Choo choo@skku.edu Copyright 2000-2018
Dynamic Programming. Design and Analysis of Algorithms. Entwurf und Analyse von Algorithmen. Irene Parada. Design and Analysis of Algorithms
Entwurf und Analyse von Algorithmen Dynamic Programming Overview Introduction Example 1 When and how to apply this method Example 2 Final remarks Introduction: when recursion is inefficient Example: Calculation
Introduction to Algorithms
Introduction to Algorithms 6.046J/18.401J LECTURE 12 Dynamic programming Longest common subsequence Optimal substructure Overlapping subproblems Prof. Charles E. Leiserson Dynamic programming Design technique,
Dynamic programming 4/9/18
Dynamic programming 4/9/18 Administrivia HW 3 due Wednesday night Exam out Thursday, due next week Multi-day takehome, open book, closed web, written problems Induction, AVL trees, recurrences, D&C, multithreaded
CS473-Algorithms I. Lecture 10. Dynamic Programming. Cevdet Aykanat - Bilkent University Computer Engineering Department
CS473-Algorithms I Lecture 1 Dynamic Programming 1 Introduction An algorithm design paradigm like divide-and-conquer Programming : A tabular method (not writing computer code) Divide-and-Conquer (DAC):
Chapter 3 Dynamic programming
Chapter 3 Dynamic programming 1 Dynamic programming also solve a problem by combining the solutions to subproblems. But dynamic programming considers the situation that some subproblems will be called
Framework for Design of Dynamic Programming Algorithms
CSE 441T/541T Advanced Algorithms September 22, 2010 Framework for Design of Dynamic Programming Algorithms Dynamic programming algorithms for combinatorial optimization generalize the strategy we studied
Computer Sciences Department 1
1 Advanced Design and Analysis Techniques (15.1, 15.2, 15.3, 15.4 and 15.5) 3 Objectives Problem Formulation Examples The Basic Problem Principle of optimality Important techniques: dynamic programming
CS60020: Foundations of Algorithm Design and Machine Learning. Sourangshu Bhattacharya
CS60020: Foundations of Algorithm Design and Machine Learning Sourangshu Bhattacharya Dynamic programming Design technique, like divide-and-conquer. Example: Longest Common Subsequence (LCS) Given two
We augment RBTs to support operations on dynamic sets of intervals A closed interval is an ordered pair of real
14.3 Interval trees We augment RBTs to support operations on dynamic sets of intervals A closed interval is an ordered pair of real numbers ], with Interval ]represents the set Open and half-open intervals
15.Dynamic Programming
15.Dynamic Programming Dynamic Programming is an algorithm design technique for optimization problems: often minimizing or maximizing. Like divide and conquer, DP solves problems by combining solutions
Homework3: Dynamic Programming - Answers
Most Exercises are from your textbook: Homework3: Dynamic Programming - Answers 1. For the Rod Cutting problem (covered in lecture) modify the given top-down memoized algorithm (includes two procedures)
Problem Strategies. 320 Greedy Strategies 6
Problem Strategies Weighted interval scheduling: 2 subproblems (include the interval or don t) Have to check out all the possibilities in either case, so lots of subproblem overlap dynamic programming:
Algorithm Design Techniques part I
Algorithm Design Techniques part I Divide-and-Conquer. Dynamic Programming DSA - lecture 8 - T.U.Cluj-Napoca - M. Joldos 1 Some Algorithm Design Techniques Top-Down Algorithms: Divide-and-Conquer Bottom-Up
10/24/ Rotations. 2. // s left subtree s right subtree 3. if // link s parent to elseif == else 11. // put x on s left
13.2 Rotations MAT-72006 AA+DS, Fall 2013 24-Oct-13 368 LEFT-ROTATE(, ) 1. // set 2. // s left subtree s right subtree 3. if 4. 5. // link s parent to 6. if == 7. 8. elseif == 9. 10. else 11. // put x
CS 231: Algorithmic Problem Solving
CS 231: Algorithmic Problem Solving Naomi Nishimura Module 5 Date of this version: June 14, 2018 WARNING: Drafts of slides are made available prior to lecture for your convenience. After lecture, slides
/463 Algorithms - Fall 2013 Solution to Assignment 3
600.363/463 Algorithms - Fall 2013 Solution to Assignment 3 (120 points) I (30 points) (Hint: This problem is similar to parenthesization in matrix-chain multiplication, except the special treatment on
Dynamic Programming II
June 9, 214 DP: Longest common subsequence biologists often need to find out how similar are 2 DNA sequences DNA sequences are strings of bases: A, C, T and G how to define similarity? DP: Longest common
Write an algorithm to find the maximum value that can be obtained by an appropriate placement of parentheses in the expression
Chapter 5 Dynamic Programming Exercise 5.1 Write an algorithm to find the maximum value that can be obtained by an appropriate placement of parentheses in the expression x 1 /x /x 3 /... x n 1 /x n, where
ALGORITHM DESIGN DYNAMIC PROGRAMMING. University of Waterloo
ALGORITHM DESIGN DYNAMIC PROGRAMMING University of Waterloo LIST OF SLIDES 1-1 List of Slides 1 2 Dynamic Programming Approach 3 Fibonacci Sequence (cont.) 4 Fibonacci Sequence (cont.) 5 Bottom-Up vs.
CS473 - Algorithms I
CS473 - Algorithms I Lecture 4 The Divide-and-Conquer Design Paradigm View in slide-show mode 1 Reminder: Merge Sort Input array A sort this half sort this half Divide Conquer merge two sorted halves Combine
So far... Finished looking at lower bounds and linear sorts.
So far... Finished looking at lower bounds and linear sorts. Next: Memoization -- Optimization problems - Dynamic programming A scheduling problem Matrix multiplication optimization Longest Common Subsequence
IN101: Algorithmic techniques Vladimir-Alexandru Paun ENSTA ParisTech
IN101: Algorithmic techniques Vladimir-Alexandru Paun ENSTA ParisTech License CC BY-NC-SA 2.0 http://creativecommons.org/licenses/by-nc-sa/2.0/fr/ Outline Previously on IN101 Python s anatomy Functions,
Design and Analysis of Algorithms 演算法設計與分析. Lecture 7 April 15, 2015 洪國寶
Design and Analysis of Algorithms 演算法設計與分析 Lecture 7 April 15, 2015 洪國寶 1 Course information (5/5) Grading (Tentative) Homework 25% (You may collaborate when solving the homework, however when writing
Today: Matrix Subarray (Divide & Conquer) Intro to Dynamic Programming (Rod cutting) COSC 581, Algorithms January 21, 2014
Today: Matrix Subarray (Divide & Conquer) Intro to Dynamic Programming (Rod cutting) COSC 581, Algorithms January 21, 2014 Reading Assignments Today s class: Chapter 4.1, 15.1 Reading assignment for next
ECE608 - Chapter 15 answers
¼ À ÈÌ Ê ½ ÈÊÇ Ä ÅË ½µ ½ º¾¹¾ ¾µ ½ º¾¹ µ ½ º¾¹ µ ½ º ¹½ µ ½ º ¹¾ µ ½ º ¹ µ ½ º ¹¾ µ ½ º ¹ µ ½ º ¹ ½¼µ ½ º ¹ ½½µ ½ ¹ ½ ECE608 - Chapter 15 answers (1) CLR 15.2-2 MATRIX CHAIN MULTIPLY(A, s, i, j) 1. if
Elements of Dynamic Programming. COSC 3101A - Design and Analysis of Algorithms 8. Discovering Optimal Substructure. Optimal Substructure - Examples
Elements of Dynamic Programming COSC 3A - Design and Analysis of Algorithms 8 Elements of DP Memoization Longest Common Subsequence Greedy Algorithms Many of these slides are taken from Monica Nicolescu,
To illustrate what is intended the following are three write ups by students. Diagonalization
General guidelines: You may work with other people, as long as you write up your solution in your own words and understand everything you turn in. Make sure to justify your answers they should be clear
CMPS 2200 Fall Dynamic Programming. Carola Wenk. Slides courtesy of Charles Leiserson with changes and additions by Carola Wenk
CMPS 00 Fall 04 Dynamic Programming Carola Wenk Slides courtesy of Charles Leiserson with changes and additions by Carola Wenk 9/30/4 CMPS 00 Intro. to Algorithms Dynamic programming Algorithm design technique
Greedy Algorithms. Informal Definition A greedy algorithm makes its next step based only on the current state and simple calculations on the input.
Greedy Algorithms Informal Definition A greedy algorithm makes its next step based only on the current state and simple calculations on the input. easy to design not always correct challenge is to identify
15.4 Longest common subsequence
15.4 Longest common subsequence Biological applications often need to compare the DNA of two (or more) different organisms A strand of DNA consists of a string of molecules called bases, where the possible
CS303 (Spring 2008) Solutions to Assignment 7
CS303 (Spring 008) Solutions to Assignment 7 Problem 1 Here is our implementation of a class for long numbers. It includes addition, subtraction, and both ways of multiplying. The rest of the code, not
Midterm solutions. n f 3 (n) = 3
Introduction to Computer Science 1, SE361 DGIST April 20, 2016 Professors Min-Soo Kim and Taesup Moon Midterm solutions Midterm solutions The midterm is a 1.5 hour exam (4:30pm 6:00pm). This is a closed
Lecture 13: Chain Matrix Multiplication
Lecture 3: Chain Matrix Multiplication CLRS Section 5.2 Revised April 7, 2003 Outline of this Lecture Recalling matrix multiplication. The chain matrix multiplication problem. A dynamic programming algorithm
Jana Kosecka. Linear Time Sorting, Median, Order Statistics. Many slides here are based on E. Demaine, D. Luebke slides
Jana Kosecka Linear Time Sorting, Median, Order Statistics Many slides here are based on E. Demaine, D. Luebke slides Insertion sort: Easy to code Fast on small inputs (less than ~50 elements) Fast on
Chapter 1 Divide and Conquer Algorithm Theory WS 2013/14 Fabian Kuhn
Chapter 1 Divide and Conquer Algorithm Theory WS 2013/14 Fabian Kuhn Divide And Conquer Principle Important algorithm design method Examples from Informatik 2: Sorting: Mergesort, Quicksort Binary search
y j LCS-Length(X,Y) Running time: O(st) set c[i,0] s and c[0,j] s to 0 for i=1 to s for j=1 to t if x i =y j then else if
![y j LCS-Length(X,Y) Running time: O(st) set c[i,0] s and c[0,j] s to 0 for i=1 to s for j=1 to t if x i =y j then else if](/thumbs/87/97232383.jpg "y j LCS-Length(X,Y) Running time: O(st) set c[i,0] s and c[0,j] s to 0 for i=1 to s for j=1 to t if x i =y j then else if")
Recursive solution for finding LCS of X and Y if x s =y t, then find an LCS of X s-1 and Y t-1, and then append x s =y t to this LCS if x s y t, then solve two subproblems: (1) find an LCS of X s-1 and
Data Structures and Algorithms CSE 465
Data Structures and Algorithms CSE 465 LECTURE 4 More Divide and Conquer Binary Search Exponentiation Multiplication Sofya Raskhodnikova and Adam Smith Review questions How long does Merge Sort take on
Single Source Shortest Path (SSSP) Problem
Single Source Shortest Path (SSSP) Problem Single Source Shortest Path Problem Input: A directed graph G = (V, E); an edge weight function w : E R, and a start vertex s V. Find: for each vertex u V, δ(s,
15.4 Longest common subsequence
15.4 Longest common subsequence Biological applications often need to compare the DNA of two (or more) different organisms A strand of DNA consists of a string of molecules called bases, where the possible
L.J. Institute of Engineering & Technology Semester: VIII (2016)
Subject Name: Design & Analysis of Algorithm Subject Code:1810 Faculties: Mitesh Thakkar Sr. UNIT-1 Basics of Algorithms and Mathematics No 1 What is an algorithm? What do you mean by correct algorithm?
1 Definition of Reduction
1 Definition of Reduction Problem A is reducible, or more technically Turing reducible, to problem B, denoted A B if there a main program M to solve problem A that lacks only a procedure to solve problem
1 Non greedy algorithms (which we should have covered
1 Non greedy algorithms (which we should have covered earlier) 1.1 Floyd Warshall algorithm This algorithm solves the all-pairs shortest paths problem, which is a problem where we want to find the shortest
How many leaves on the decision tree? There are n! leaves, because every permutation appears at least once.
Chapter 8. Sorting in Linear Time Types of Sort Algorithms The only operation that may be used to gain order information about a sequence is comparison of pairs of elements. Quick Sort -- comparison-based
12 Dynamic Programming (2) Matrix-chain Multiplication Segmented Least Squares
12 Dynamic Programming (2) Matrix-chain Multiplication Segmented Least Squares Optimal substructure Dynamic programming is typically applied to optimization problems. An optimal solution to the original
HOMEWORK 4 CS5381 Analysis of Algorithm
HOMEWORK 4 CS5381 Analysis of Algorithm 1. QUESTION 1 Why do we want the loop index i in line 2 of BUILD-MAX-HEAP as shown below to decrease from length[a]/2 to 2 rather than increase from 1 to length[a]/2?
Algorithms for Data Science
Algorithms for Data Science CSOR W4246 Eleni Drinea Computer Science Department Columbia University Thursday, October 1, 2015 Outline 1 Recap 2 Shortest paths in graphs with non-negative edge weights (Dijkstra
Chain Matrix Multiplication
Chain Matrix Multiplication Version of November 5, 2014 Version of November 5, 2014 Chain Matrix Multiplication 1 / 27 Outline Outline Review of matrix multiplication. The chain matrix multiplication problem.
Dynamic Programming. Lecture Overview Introduction
Lecture 12 Dynamic Programming 12.1 Overview Dynamic Programming is a powerful technique that allows one to solve many different types of problems in time O(n 2 ) or O(n 3 ) for which a naive approach
Dynamic Programming Matrix-chain Multiplication
1 / 32 Dynamic Programming Matrix-chain Multiplication CS 584: Algorithm Design and Analysis Daniel Leblanc 1 1 Senior Adjunct Instructor Portland State University Maseeh College of Engineering and Computer
We ve done. Now. Next
We ve done Matroid Theory Task scheduling problem (another matroid example) Dijkstra s algorithm (another greedy example) Dynamic Programming Now Matrix Chain Multiplication Longest Common Subsequence
PRAM Divide and Conquer Algorithms
PRAM Divide and Conquer Algorithms (Chapter Five) Introduction: Really three fundamental operations: Divide is the partitioning process Conquer the the process of (eventually) solving the eventual base
Design and Analysis of Algorithms 演算法設計與分析. Lecture 7 April 6, 2016 洪國寶
Design and Analysis of Algorithms 演算法設計與分析 Lecture 7 April 6, 2016 洪國寶 1 Course information (5/5) Grading (Tentative) Homework 25% (You may collaborate when solving the homework, however when writing up
Dynamic Programming. CIS 110, Fall University of Pennsylvania
Dynamic Programming CIS 110, Fall 2012 University of Pennsylvania Dynamic Programming Dynamic programming records saves computation for reuse later. Programming: in the optimization sense ( Linear Programming
Greedy algorithms is another useful way for solving optimization problems.
Greedy Algorithms Greedy algorithms is another useful way for solving optimization problems. Optimization Problems For the given input, we are seeking solutions that must satisfy certain conditions. These
Dynamic Programming: 1D Optimization. Dynamic Programming: 2D Optimization. Fibonacci Sequence. Crazy 8 s. Edit Distance
Dynamic Programming: 1D Optimization Fibonacci Sequence To efficiently calculate F [x], the xth element of the Fibonacci sequence, we can construct the array F from left to right (or bottom up ). We start
Introduction to Algorithms
Introduction to Algorithms Dynamic Programming Well known algorithm design techniques: Brute-Force (iterative) ti algorithms Divide-and-conquer algorithms Another strategy for designing algorithms is dynamic
Design and Analysis of Algorithms
Design and Analysis of Algorithms CSE 5311 Lecture 8 Sorting in Linear Time Junzhou Huang, Ph.D. Department of Computer Science and Engineering CSE5311 Design and Analysis of Algorithms 1 Sorting So Far
Chapter 1 Divide and Conquer Algorithm Theory WS 2014/15 Fabian Kuhn
Chapter 1 Divide and Conquer Algorithm Theory WS 2014/15 Fabian Kuhn Divide And Conquer Principle Important algorithm design method Examples from Informatik 2: Sorting: Mergesort, Quicksort Binary search
CSC Design and Analysis of Algorithms
CSC 8301- Design and Analysis of Algorithms Lecture 6 Divide and Conquer Algorithm Design Technique Divide-and-Conquer The most-well known algorithm design strategy: 1. Divide a problem instance into two
Chapter 1 Divide and Conquer Algorithm Theory WS 2015/16 Fabian Kuhn
Chapter 1 Divide and Conquer Algorithm Theory WS 2015/16 Fabian Kuhn Divide And Conquer Principle Important algorithm design method Examples from Informatik 2: Sorting: Mergesort, Quicksort Binary search
CSC Design and Analysis of Algorithms. Lecture 6. Divide and Conquer Algorithm Design Technique. Divide-and-Conquer
CSC 8301- Design and Analysis of Algorithms Lecture 6 Divide and Conquer Algorithm Design Technique Divide-and-Conquer The most-well known algorithm design strategy: 1. Divide a problem instance into two
Recitation 12. Dynamic Programming Announcements. SegmentLab has been released and is due Apr 14.
Recitation 12 Dynamic Programming 12.1 Announcements SegmentLab has been released and is due Apr 14. 73 74 RECITATION 12. DYNAMIC PROGRAMMING 12.2 Matrix Chain Product Definition 12.1. In the matrix chain
Algorithms: COMP3121/3821/9101/9801
NEW SOUTH WALES Algorithms: COMP3121/3821/9101/9801 Aleks Ignjatović School of Computer Science and Engineering University of New South Wales TOPIC 5: DYNAMIC PROGRAMMING COMP3121/3821/9101/9801 1 / 38
Algorithms Dr. Haim Levkowitz
91.503 Algorithms Dr. Haim Levkowitz Fall 2007 Lecture 4 Tuesday, 25 Sep 2007 Design Patterns for Optimization Problems Greedy Algorithms 1 Greedy Algorithms 2 What is Greedy Algorithm? Similar to dynamic
CS4311 Design and Analysis of Algorithms. Lecture 9: Dynamic Programming I
CS4311 Design and Analysis of Algorithms Lecture 9: Dynamic Programming I 1 About this lecture Divide-and-conquer strategy allows us to solve a big problem by handling only smaller sub-problems Some problems
Divide and Conquer Algorithms
CSE341T 09/13/2017 Lecture 5 Divide and Conquer Algorithms We have already seen a couple of divide and conquer algorithms in this lecture. The reduce algorithm and the algorithm to copy elements of the
CS141: Intermediate Data Structures and Algorithms Analysis of Algorithms
CS141: Intermediate Data Structures and Algorithms Analysis of Algorithms Amr Magdy Analyzing Algorithms 1. Algorithm Correctness a. Termination b. Produces the correct output for all possible input. 2.
Recursive-Fib(n) if n=1 or n=2 then return 1 else return Recursive-Fib(n-1)+Recursive-Fib(n-2)
Dynamic Programming Any recursive formula can be directly translated into recursive algorithms. However, sometimes the compiler will not implement the recursive algorithm very efficiently. When this is
Efficient Sequential Algorithms, Comp309. Problems. Part 1: Algorithmic Paradigms
Efficient Sequential Algorithms, Comp309 Part 1: Algorithmic Paradigms University of Liverpool References: T. H. Cormen, C. E. Leiserson, R. L. Rivest Introduction to Algorithms, Second Edition. MIT Press
Chapter 4. Divide-and-Conquer. Copyright 2007 Pearson Addison-Wesley. All rights reserved.
Chapter 4 Divide-and-Conquer Copyright 2007 Pearson Addison-Wesley. All rights reserved. Divide-and-Conquer The most-well known algorithm design strategy: 2. Divide instance of problem into two or more
Sankalchand Patel College of Engineering - Visnagar Department of Computer Engineering and Information Technology. Assignment
Class: V - CE Sankalchand Patel College of Engineering - Visnagar Department of Computer Engineering and Information Technology Sub: Design and Analysis of Algorithms Analysis of Algorithm: Assignment
Data Structures and Algorithms Week 8
Data Structures and Algorithms Week 8 Dynamic programming Fibonacci numbers Optimization problems Matrix multiplication optimization Principles of dynamic programming Longest Common Subsequence Algorithm
4 Dynamic Programming
4 Dynamic Programming Dynamic Programming is a form of recursion. In Computer Science, you have probably heard the tradeoff between Time and Space. There is a trade off between the space complexity and
Dynamic Programming (Part #2)
Dynamic Programming (Part #) Introduction to Algorithms MIT Press (Chapter 5) Matrix-Chain Multiplication Problem: given a sequence A, A,, A n, compute the product: A A A n Matrix compatibility: C = A
CSE 417 Dynamic Programming (pt 4) Sub-problems on Trees
CSE 417 Dynamic Programming (pt 4) Sub-problems on Trees Reminders > HW4 is due today > HW5 will be posted shortly Dynamic Programming Review > Apply the steps... 1. Describe solution in terms of solution
Computer Science 236 Fall Nov. 11, 2010
Computer Science 26 Fall Nov 11, 2010 St George Campus University of Toronto Assignment Due Date: 2nd December, 2010 1 (10 marks) Assume that you are given a file of arbitrary length that contains student
What does this print?
public class Test_Static { int a; static int b; public Test_Static(int av, int bv) { a= av; b= bv; } public void print() { System.out.println ("a= " + a + " b= " + b); } public static void main (String
CS141: Intermediate Data Structures and Algorithms Analysis of Algorithms
CS141: Intermediate Data Structures and Algorithms Analysis of Algorithms Amr Magdy Analyzing Algorithms 1. Algorithm Correctness a. Termination b. Produces the correct output for all possible input. 2.
CS 506, Sect 002 Homework 5 Dr. David Nassimi Foundations of CS Due: Week 11, Mon. Apr. 7 Spring 2014
CS 506, Sect 002 Homework 5 Dr. David Nassimi Foundations of CS Due: Week 11, Mon. Apr. 7 Spring 2014 Study: Chapter 4 Analysis of Algorithms, Recursive Algorithms, and Recurrence Equations 1. Prove the
Greedy Algorithms CLRS Laura Toma, csci2200, Bowdoin College
Greedy Algorithms CLRS 16.1-16.2 Laura Toma, csci2200, Bowdoin College Overview. Sometimes we can solve optimization problems with a technique called greedy. A greedy algorithm picks the option that looks
Dynamic Programming 1
Dynamic Programming 1 Jie Wang University of Massachusetts Lowell Department of Computer Science 1 I thank Prof. Zachary Kissel of Merrimack College for sharing his lecture notes with me; some of the examples
14 Dynamic. Matrix-chain multiplication. P.D. Dr. Alexander Souza. Winter term 11/12
Algorithms Theory 14 Dynamic Programming (2) Matrix-chain multiplication P.D. Dr. Alexander Souza Optimal substructure Dynamic programming is typically applied to optimization problems. An optimal solution
Chapter 1 Divide and Conquer Algorithm Theory WS 2013/14 Fabian Kuhn
Chapter 1 Divide and Conquer Algorithm Theory WS 2013/14 Fabian Kuhn Number of Inversions Formal problem: Given: array,,,, of distinct elements Objective: Compute number of inversions 0 Example: 4, 1,
Unit 4: Dynamic Programming
Unit 4: Dynamic Programming Course contents: Assembly-line scheduling Matrix-chain multiplication Longest common subsequence Optimal binary search trees Applications: Cell flipping, rod cutting, optimal
ECE608 - Chapter 16 answers
¼ À ÈÌ Ê ½ ÈÊÇ Ä ÅË ½µ ½ º½¹ ¾µ ½ º½¹ µ ½ º¾¹½ µ ½ º¾¹¾ µ ½ º¾¹ µ ½ º ¹ µ ½ º ¹ µ ½ ¹½ ½ ECE68 - Chapter 6 answers () CLR 6.-4 Let S be the set of n activities. The obvious solution of using Greedy-Activity-
Dynamic Programming. An Enumeration Approach. Matrix Chain-Products. Matrix Chain-Products (not in book)
Matrix Chain-Products (not in book) is a general algorithm design paradigm. Rather than give the general structure, let us first give a motivating example: Matrix Chain-Products Review: Matrix Multiplication.
CMSC351 - Fall 2014, Homework #4
CMSC351 - Fall 2014, Homework #4 Due: November 14th at the start of class PRINT Name: Grades depend on neatness and clarity. Write your answers with enough detail about your approach and concepts used,
1 Greedy algorithms and dynamic programming
TIE-20106 1 1 Greedy algorithms and dynamic programming This chapter covers two malgorithm design principles more: greedy algorithms and dynamic programming A greedy algorithm is often the most natural
Outline. CS38 Introduction to Algorithms. Fast Fourier Transform (FFT) Fast Fourier Transform (FFT) Fast Fourier Transform (FFT)
Outline CS8 Introduction to Algorithms Lecture 9 April 9, 0 Divide and Conquer design paradigm matrix multiplication Dynamic programming design paradigm Fibonacci numbers weighted interval scheduling knapsack
CS483 Analysis of Algorithms Lecture 03 Divide-n-Conquer
CS483 Analysis of Algorithms Lecture 03 Divide-n-Conquer Jyh-Ming Lien February 05, 2009 this lecture note is based on Algorithms by S. Dasgupta, C.H. Papadimitriou, and U.V. Vazirani and to the Design
Efficient Sequential Algorithms, Comp309. Motivation. Longest Common Subsequence. Part 3. String Algorithms
Efficient Sequential Algorithms, Comp39 Part 3. String Algorithms University of Liverpool References: T. H. Cormen, C. E. Leiserson, R. L. Rivest Introduction to Algorithms, Second Edition. MIT Press (21).
Subsequence Definition. CS 461, Lecture 8. Today s Outline. Example. Assume given sequence X = x 1, x 2,..., x m. Jared Saia University of New Mexico
Subsequence Definition CS 461, Lecture 8 Jared Saia University of New Mexico Assume given sequence X = x 1, x 2,..., x m Let Z = z 1, z 2,..., z l Then Z is a subsequence of X if there exists a strictly
5.1 The String reconstruction problem
CS125 Lecture 5 Fall 2014 5.1 The String reconstruction problem The greedy approach doesn t always work, as we have seen. It lacks flexibility; if at some point, it makes a wrong choice, it becomes stuck.