Optimization II: Dynamic programming Ricardo Fukasawa rfukasawa@uwaterloo.ca Department of Combinatorics and Optimization Faculty of Mathematics University of Waterloo Nov 2, 2016 R. Fukasawa (C&O) Optimization II 1 / 21
Picking items Question Suppose you have a list of n items with integer weights (a 1,...,a n ) and with values (c 1,...,c n ) and you can carry at most an integer b amount of weight. Which items would you choose to carry to maximize the total value you carry? Example: n = 5, (a 1,a 2,a 3,a 4,a 5 ) = (5,3,6,4,7) (c 1,c 2,c 3,c 4,c 5 ) = (6,4,5,7,6) b = 11 Examples of feasible solutions: R. Fukasawa (C&O) Optimization II 2 / 21
Picking items Question Suppose you have a list of n items with integer weights (a 1,...,a n ) and with values (c 1,...,c n ) and you can carry at most an integer b amount of weight. Which items would you choose to carry to maximize the total value you carry? Example: n = 5, (a 1,a 2,a 3,a 4,a 5 ) = (5,3,6,4,7) (c 1,c 2,c 3,c 4,c 5 ) = (6,4,5,7,6) b = 11 Examples of feasible solutions: Carry 1 and 3, with value 11 R. Fukasawa (C&O) Optimization II 2 / 21
Picking items Question Suppose you have a list of n items with integer weights (a 1,...,a n ) and with values (c 1,...,c n ) and you can carry at most an integer b amount of weight. Which items would you choose to carry to maximize the total value you carry? Example: n = 5, (a 1,a 2,a 3,a 4,a 5 ) = (5,3,6,4,7) (c 1,c 2,c 3,c 4,c 5 ) = (6,4,5,7,6) b = 11 Examples of feasible solutions: Carry 1 and 3, with value 11 Carry 4 and 5, with value 13 R. Fukasawa (C&O) Optimization II 2 / 21
Picking items Question Suppose you have a list of n items with integer weights (a 1,...,a n ) and with values (c 1,...,c n ) and you can carry at most an integer b amount of weight. Which items would you choose to carry to maximize the total value you carry? Example: n = 5, (a 1,a 2,a 3,a 4,a 5 ) = (5,3,6,4,7) (c 1,c 2,c 3,c 4,c 5 ) = (6,4,5,7,6) b = 11 Examples of feasible solutions: Carry 1 and 3, with value 11 Carry 4 and 5, with value 13 Carry 4 and 6, with value 12 R. Fukasawa (C&O) Optimization II 2 / 21
Picking items Question Suppose you have a list of n items with integer weights (a 1,...,a n ) and with values (c 1,...,c n ) and you can carry at most an integer b amount of weight. Which items would you choose to carry to maximize the total value you carry? Proposed greedy algorithms: R. Fukasawa (C&O) Optimization II 3 / 21
Picking items Question Suppose you have a list of n items with integer weights (a 1,...,a n ) and with values (c 1,...,c n ) and you can carry at most an integer b amount of weight. Which items would you choose to carry to maximize the total value you carry? Proposed greedy algorithms: Pick smallest weight item, until weight limit is exceeded. R. Fukasawa (C&O) Optimization II 3 / 21
Picking items Question Suppose you have a list of n items with integer weights (a 1,...,a n ) and with values (c 1,...,c n ) and you can carry at most an integer b amount of weight. Which items would you choose to carry to maximize the total value you carry? Proposed greedy algorithms: Pick smallest weight item, until weight limit is exceeded. Pick largest value item that does not exceed weight limit. R. Fukasawa (C&O) Optimization II 3 / 21
Picking items Question Suppose you have a list of n items with integer weights (a 1,...,a n ) and with values (c 1,...,c n ) and you can carry at most an integer b amount of weight. Which items would you choose to carry to maximize the total value you carry? Proposed greedy algorithms: Pick smallest weight item, until weight limit is exceeded. Pick largest value item that does not exceed weight limit. Order items from highest to lowest value / weight ratio ( c j a j ). Pick the items in that order if possible. R. Fukasawa (C&O) Optimization II 3 / 21
Picking items Question Suppose you have a list of n items with integer weights (a 1,...,a n ) and with values (c 1,...,c n ) and you can carry at most an integer b amount of weight. Which items would you choose to carry to maximize the total value you carry? Proposed greedy algorithms: Pick smallest weight item, until weight limit is exceeded. Pick largest value item that does not exceed weight limit. Order items from highest to lowest value / weight ratio ( c j a j ). Pick the items in that order if possible. Suggestions? R. Fukasawa (C&O) Optimization II 3 / 21
Alternative approach: Suppose I am considering just the last item (item n). Then either: I pick item n, get a value of c n and reduce my capacity by a n or I don t pick item n and still have capacity b Example: n = 3, (a 1,a 2,a 3 ) = (2,3,1) (c 1,c 2,c 3 ) = (5,7,6) b = 4 If I pick item 3, then I have already a value of 6, with capacity of 3 left for items 1,2. If I don t pick item 3, then I have already a value of 0, with capacity of 4 left for items 1,2. R. Fukasawa (C&O) Optimization II 4 / 21
Example: n = 3, (a 1,a 2,a 3 ) = (2,3,1) (c 1,c 2,c 3 ) = (5,7,6) b = 4 Conclusion: In my example, I should pick the best of: R. Fukasawa (C&O) Optimization II 5 / 21
Example: n = 3, (a 1,a 2,a 3 ) = (2,3,1) (c 1,c 2,c 3 ) = (5,7,6) b = 4 Conclusion: In my example, I should pick the best of: Picking item 3: 6+ optimal value of problem with just items 1,2 and capacity 3 R. Fukasawa (C&O) Optimization II 5 / 21
Example: n = 3, (a 1,a 2,a 3 ) = (2,3,1) (c 1,c 2,c 3 ) = (5,7,6) b = 4 Conclusion: In my example, I should pick the best of: Picking item 3: 6+ optimal value of problem with just items 1,2 and capacity 3 Not picking item 3: 0+ optimal value of problem with just items 1,2 and capacity 4 R. Fukasawa (C&O) Optimization II 5 / 21
Idea: (DYNAMIC PROGRAMMING) Optimal solution to a problem can be constructed from optimal solution of simpler versions of the same problem R. Fukasawa (C&O) Optimization II 6 / 21
Idea: (DYNAMIC PROGRAMMING) Optimal solution to a problem can be constructed from optimal solution of simpler versions of the same problem Notation R. Fukasawa (C&O) Optimization II 6 / 21
Idea: (DYNAMIC PROGRAMMING) Optimal solution to a problem can be constructed from optimal solution of simpler versions of the same problem Notation v(i,d) - Optimal value for problem with items 1,...,i and capacity d R. Fukasawa (C&O) Optimization II 6 / 21
Idea: (DYNAMIC PROGRAMMING) Optimal solution to a problem can be constructed from optimal solution of simpler versions of the same problem Notation v(i,d) - Optimal value for problem with items 1,...,i and capacity d What we want: v(n,b) R. Fukasawa (C&O) Optimization II 6 / 21
Idea: (DYNAMIC PROGRAMMING) Optimal solution to a problem can be constructed from optimal solution of simpler versions of the same problem Notation v(i,d) - Optimal value for problem with items 1,...,i and capacity d What we want: v(n,b) Back to example: R. Fukasawa (C&O) Optimization II 6 / 21
Idea: (DYNAMIC PROGRAMMING) Optimal solution to a problem can be constructed from optimal solution of simpler versions of the same problem Notation v(i,d) - Optimal value for problem with items 1,...,i and capacity d What we want: Back to example: Picking item 3: v(n,b) R. Fukasawa (C&O) Optimization II 6 / 21
Idea: (DYNAMIC PROGRAMMING) Optimal solution to a problem can be constructed from optimal solution of simpler versions of the same problem Notation v(i,d) - Optimal value for problem with items 1,...,i and capacity d What we want: v(n,b) Back to example: Picking item 3: 6+ optimal value of problem with just items 1,2 and capacity 3 R. Fukasawa (C&O) Optimization II 6 / 21
Idea: (DYNAMIC PROGRAMMING) Optimal solution to a problem can be constructed from optimal solution of simpler versions of the same problem Notation v(i,d) - Optimal value for problem with items 1,...,i and capacity d What we want: Back to example: Picking item 3: v(n,b) v(3,4) = 6+v(2,3) R. Fukasawa (C&O) Optimization II 6 / 21
Idea: (DYNAMIC PROGRAMMING) Optimal solution to a problem can be constructed from optimal solution of simpler versions of the same problem Notation v(i,d) - Optimal value for problem with items 1,...,i and capacity d What we want: Back to example: Picking item 3: v(n,b) v(3,4) = 6+v(2,3) Not picking item 3: R. Fukasawa (C&O) Optimization II 6 / 21
Idea: (DYNAMIC PROGRAMMING) Optimal solution to a problem can be constructed from optimal solution of simpler versions of the same problem Notation v(i,d) - Optimal value for problem with items 1,...,i and capacity d What we want: v(n,b) Back to example: Picking item 3: Not picking item 3: v(3,4) = 6+v(2,3) 0+ optimal value of problem with just items 1,2 and capacity 4 R. Fukasawa (C&O) Optimization II 6 / 21
Idea: (DYNAMIC PROGRAMMING) Optimal solution to a problem can be constructed from optimal solution of simpler versions of the same problem Notation v(i,d) - Optimal value for problem with items 1,...,i and capacity d What we want: v(n,b) Back to example: Picking item 3: Not picking item 3: v(3,4) = 6+v(2,3) v(3,4) = 0+v(2,4) R. Fukasawa (C&O) Optimization II 6 / 21
Idea: (DYNAMIC PROGRAMMING) Optimal solution to a problem can be constructed from optimal solution of simpler versions of the same problem Notation v(i,d) - Optimal value for problem with items 1,...,i and capacity d What we want: v(n,b) Back to example: v(3,4) = max{6+v(2,3),0+v(2,4)} R. Fukasawa (C&O) Optimization II 6 / 21
In general, given: integer weights (a 1,...,a n ) values (c 1,...,c n ) integer capacity b v(i,d) = R. Fukasawa (C&O) Optimization II 7 / 21
In general, given: integer weights (a 1,...,a n ) values (c 1,...,c n ) integer capacity b v(i,d) = max{c i +v(i 1,d a i ),0+v(i 1,d)} R. Fukasawa (C&O) Optimization II 7 / 21
In general, given: integer weights (a 1,...,a n ) values (c 1,...,c n ) integer capacity b v(i,d) = max{c i +v(i 1,d a i ),0+v(i 1,d)} (only if i > 1 and d a i ) R. Fukasawa (C&O) Optimization II 7 / 21
In general, given: integer weights (a 1,...,a n ) values (c 1,...,c n ) integer capacity b v(i,d) = max{c i +v(i 1,d a i ),0+v(i 1,d)} (only if i > 1 and d a i ) Otherwise: Easy cases: i = 1 d < a i R. Fukasawa (C&O) Optimization II 7 / 21
Example n = 3, (a 1,a 2,a 3 ) = (2,3,1) (c 1,c 2,c 3 ) = (5,7,6) b = 4 R. Fukasawa (C&O) Optimization II 8 / 21
Dynamic programming Key ingredient: Optimal substructure: Optimal solution to a problem can be constructed from optimal solution of simpler versions of the same problem R. Fukasawa (C&O) Optimization II 9 / 21
Google maps (shortest path) Suppose circles represent intersections, lines are streets, numbers are distances. Goal: Find shortest path from A E R. Fukasawa (C&O) Optimization II 10 / 21
Google maps (shortest path) Suppose circles represent intersections, lines are streets, numbers are distances. Goal: Find shortest path from A E Nomenclature: Vertices (circles): {A,B,C,D,E,F,G} Edges (lines): {A,B}, {B,F}, {A,F}, {B,C},... Vertices are adjacent if there is an edge connecting them Path is a sequence of adjacent non-repeated vertices Example: (A,B,F,G,E) is an A E path Cost of path is sum of cost of edges Example: Path (A,B,F,G,E) has cost 18 R. Fukasawa (C&O) Optimization II 10 / 21
Shortest path Given G = (V,E) with costs c e > 0 for all e E, and two vertices s and t, find shortest s t path. Key observation Shortest path from A to E is shortest of: 5 + Shortest path from A to D 4 + Shortest path from A to G R. Fukasawa (C&O) Optimization II 11 / 21
Example Dijkstra s algorithm starting with A R. Fukasawa (C&O) Optimization II 12 / 21
Shortest path Given G = (V,E) with costs c e > 0 for all e E, and initial vertex s, find shortest s v path for all v V. Dijkstra s algorithm: Maintain a set S for which we know the shortest s v path. 1 Circle the beginning vertex. 2 List the uncircled vertices that are adjacent to circled ones, the paths, and the total time. 3 Identify the uncircled vertex with shortest total time and circle the corresponding adjacent vertex. 4 Repeat from step 2 until no uncircled vertices exist. R. Fukasawa (C&O) Optimization II 13 / 21