EDIT DISTANCE. Given two words (strings), how can we define a notion of closeness. For example: Is PELICAN closer to PENGUIN or POLITICIAN?

Transcription

1 CSE 101 Algorithm Design and Analysis Miles Jones Office 4208 CSE Building Lecture 22: Dynamic Programming Examples (Edit Distance/Knapsack)

2 EDIT DISTANCE Given two words (strings), how can we define a notion of closeness For example: Is PELICAN closer to PENGUIN or POLITICIAN?

3 EDIT DISTANCE (DEFINITION) We can keep track of how many changes we need to change one word into another. The changes can be insertion, deletion, or substitution. For example, if we line up the words PELICAN and OSTRICH P E L I C A N O S T R I C H s s s s s s s

4 EDIT DISTANCE (DEFINITION) P E L I C A N O S T R I C H s s s s s s s Is 7 the cheapest cost?

5 EDIT DISTANCE (DEFINITION) P E L I C A N O S T R I C H s s s s s s s Is 7 the cheapest cost? P E L - I C A N O S T R I C - H s s s i d s

6 EDIT DISTANCE (DEFINITION) P E L I C A N P E N G U I N s s s s Is 4 the cheapest?

7 EDIT DISTANCE (DEFINITION) P E L I C A N P O L I T I C I A N s s s s i i i Is 7 the cheapest?

8 EDIT DISTANCE (BRUTE FORCE) Brute force: try all possible combinations and find the minimum cost of all of them. What is the lower bound of the number of possible combinations if the size of the words are n,m, n m?

9 EDIT DISTANCE (BRUTE FORCE) Brute force: try all possible combinations and find the minimum cost of all of them. Lower bound of the number of possible combinations if the size of the words are n,m, n m. Each column could be one of three things (at least for the first n columns.) So there are at least 3 n different combinations. And really there are much more than that!!!!!

10 EDIT DISTANCE (DYNAMIC PROGRAMMING) Find the minimum cost of two words x[1 n] and y[1 m] Step 1: Define subproblems: Step 2: base case Step 3: express recursively Step 4: order the subproblems

11 EDIT DISTANCE (DYNAMIC PROGRAMMING) Find the minimum cost of two words x[1 n] and y[1 m] Step 1: Define subproblems: Let E(i,j) be the minimum cost of the two words x[1 i] and y[1 j]. Step 2: base case Step 3: express recursively Step 4: order the subproblems

12 EDIT DISTANCE (DYNAMIC PROGRAMMING) What are the base cases?

13 EDIT DISTANCE (DYNAMIC PROGRAMMING) What are the base cases? When the first word is empty then the edit distance is the length of the second word and when the second word is empty the edit distance is the length of the first word. E(i,0)=i E(0,j)=j

14 EDIT DISTANCE (DYNAMIC PROGRAMMING) Find the minimum cost of two words x[1 n] and y[1 m] Step 1: Define subproblems: Let E(i,j) be the minimum cost of the two words x[1 i] and y[1 j]. Step 2: base case E(i,0) = i E(0,j) = j Step 3: express recursively Step 4: order the subproblems

15 EDIT DISTANCE (DYNAMIC PROGRAMMING) Step 3: express recursively: Split into cases depending on the last column of the alignment of x[1 i] and y[1 j]. Case 1: the last column looks like x[i] Case 2: the last column looks like Case 3: the last column looks like - - y[j] x[i] y[j]

16 EDIT DISTANCE (DYNAMIC PROGRAMMING) Step 3: express recursively: Case 1: the last column looks like x[i] - This is a deletion with a cost of 1 so if the minimum cost of x[1 i] and y[1 j] has this in the last column then..

17 EDIT DISTANCE (DYNAMIC PROGRAMMING) Step 3: express recursively: Case 1: the last column looks like x[i] This is a deletion with a cost of 1 so if the minimum cost of x[1 i] and y[1 j] has this in the last column then E(i,j)=1+E(i-1,j) -

18 EDIT DISTANCE (DYNAMIC PROGRAMMING) Step 3: express recursively: Case 2: the last column looks like - y[j] This is an insertion with a cost of 1 so if the minimum cost of x[1 i] and y[1 j] has this in the last column then.

19 EDIT DISTANCE (DYNAMIC PROGRAMMING) Step 3: express recursively: Case 2: the last column looks like - This is an insertion with a cost of 1 so if the minimum cost of x[1 i] and y[1 j] has this in the last column then E(i,j)=1+E(i,j-1) y[j]

20 EDIT DISTANCE (DYNAMIC PROGRAMMING) Step 3: express recursively: Case 3: the last column looks like x[i] y[j]

21 EDIT DISTANCE (DYNAMIC PROGRAMMING) Step 3: express recursively: Case 3: the last column looks like x[i] Case 3.1: x[i]=y[j] E(i,j)=E(i-1,j-1) (no cost) Case 3.2: x[i] y[j] (substitution cost of 1) E(i,j)=1+E(i-1,j-1) y[j]

22 EDIT DISTANCE (DYNAMIC PROGRAMMING) Step 3: express recursively: So take the minimum of all three cases E(i,j)=min( 1 + E(i-1,j), 1 + E(i,j-1), (1-δ x i,y[j] )+E(i-1,j-1)) (delta function δ a,b = ቊ 0 if a b 1 if a = b )

23 EDIT DISTANCE (DYNAMIC PROGRAMMING) Step 4: ordering.. To calculate E(i,j), we need to know E(i-1,j), E(i,j-1) and E(i-1,j-1) Think of a 2-d array and where are the indices in relation to (i,j)? (i-1,j-1) (i-1,j) (i,j-1) (i,j) So, order them in such a way to visit all the necessary entries before you visit (i,j). One way to do this is left to right through rows going from top to bottom.

24 EDIT DISTANCE (DYNAMIC PROGRAMMING) EditDist(x[1 n],y[1 m]) Initialize for i from 1 to n, E(i,0)=i and for j from 1 to m, E(0,j)=j for i from 1 to n for j from 1 to m E(i,j)=min( 1 + E(i-1,j), 1 + E(i,j-1), (1-δ x i,y[j] )+E(i-1,j-1))

25 P O L I T I C I A N P E L I C A N EDIT DISTANCE (EXAMPLE)

26 P O L I T I C I A N P E L I C A N EDIT DISTANCE (EXAMPLE)

27 TABULATION/MEMOIZATION

28 THE KNAPSACK PROBLEM Suppose you are a burglar who breaks into a store and you want to leave with the maximum value of items. Your knapsack can only hold 13 lbs and the items in the store are: Value Weight

29 THE KNAPSACK PROBLEM What is the maximum value you can have from a list of items a[1],,a[n] where each item has a value v[i] and a weight w[i] given that you cannot have more weight than W. Step 1: subproblems:

30 THE KNAPSACK PROBLEM What is the maximum value you can have from a list of items a[1],,a[n] where each item has a value v[i] and a weight w[i] given that you cannot have more weight than W. Step 1: subproblems: Let K(w) be the maximum value you can have in a w-capacity knapsack.

31 THE KNAPSACK PROBLEM What is the maximum value you can have from a list of items a[1],,a[n] where each item has a value v[i] and a weight w[i] given that you cannot have more weight than W. Step 2: base cases: Let K(w) be the maximum value you can have in a w-capacity knapsack.

32 THE KNAPSACK PROBLEM What is the maximum value you can have from a list of items a[1],,a[n] where each item has a value v[i] and a weight w[i] given that you cannot have more weight than W. Let K(w) be the maximum value you can have in a w-capacity knapsack. Step 3: express recursively

33 THE KNAPSACK PROBLEM What is the maximum value you can have from a list of items a[1],,a[n] where each item has a value v[i] and a weight w[i] given that you cannot have more weight than W. Let K(w) be the maximum value you can have in a w-capacity knapsack. Step 3: express recursively What is K(w)? Take away the weight of each item and see what value it is if you add that item. K w = max i:w[i] w K w w i + v i

34 THE KNAPSACK PROBLEM What is the maximum value you can have from a list of items a[1],,a[n] where each item has a value v[i] and a weight w[i] given that you cannot have more weight than W. Let K(w) be the maximum value you can have in a w-capacity knapsack. K w = max i:w[i] w K w w i + v i Step 4: order. Then order the subproblems from 1 to W.

35 THE KNAPSACK PROBLEM pseudocode: Knapsack(v[1 n],w[1 n],w) K(0):=0 prev(0):=nil for w from 1 to W K(w):=0 for j from 0 to n if K(w)<K(w-w[j])+v[j] then K(w):= K(w-w[j])+v[j] prev(w):=j return K(W) Runtime???

36 THE KNAPSACK PROBLEM Value Weight v[i] w[i]

37 THE KNAPSACK PROBLEM (NO REPEATS) Suppose you are a burglar who breaks into somebody s house where there is only one item of each. You want to leave with the maximum value of items but you can t take more than one of each thing. Your knapsack can only hold 13 lbs and the items in the house are: Value Weight

38 THE KNAPSACK PROBLEM (NO REPEATS) What is the maximum value you can have from a list of items a[1],,a[n] where each item has a value v[i] and a weight w[i] given that you cannot have more weight than W. Step 1: subproblems:

39 THE KNAPSACK PROBLEM (NO REPEATS) What is the maximum value you can have from a list of items a[1],,a[n] where each item has a value v[i] and a weight w[i] given that you cannot have more weight than W. Step 1: subproblems: Let K(w,j) be the maximum value you can have in a w-capacity knapsack using only the items a[1], a[j]

40 THE KNAPSACK PROBLEM What is the maximum value you can have from a list of items a[1],,a[n] where each item has a value v[i] and a weight w[i] given that you cannot have more weight than W. Let K(w,j) be the maximum value you can have in a w-capacity knapsack using only the items a[1], a[j] Step 2: express recursively

41 THE KNAPSACK PROBLEM Let K(w) be the maximum value you can have in a w-capacity knapsack. Step 2: express recursively What is K(w,j)? Case 1: taking a[j] is better value: add the item a[j] to a knapsack with max cap w-w[j] and add the value of j. K w, j = K w w j, j 1 + v[j] Case 2: taking a[j] is not better value. K w, j = K w, j 1 So take the maximum of these two scenarios. K w, j = max {K w w j, j 1 + v j, K w, j 1 }

42 THE KNAPSACK PROBLEM K w, j = max {K w w j, j 1 + v j, K w, j 1 } Step 3: order and base cases. Base cases: K(0,j)=0 for all j K(w,0)=0 for all w (w w[j], j 1) We need to know (w w j, j 1) and K w, j 1 K(w,j) Order the problems left to right top to bottom. (w, j 1) (w, j) before computing

43 THE KNAPSACK PROBLEM Value Weight v[i] w[i]

44 TABULATION/MEMOIZATION

45 THE KNAPSACK PROBLEM Value Weight v[i] w[i]

46 THE KNAPSACK PROBLEM (NO REPEATS) pseudocode: Runtime??? Knapsack(v[1 n],w[1 n],w) K(0,j):=0 for all j K(w,0):=0 for all w prev(0):=nil for w from 1 to W for j from 0 to n K w, j = max {K w w j, j 1 + v j, K w, j 1 } return K(w,n)