Data Structures and Algorithms CSE 465

Transcription

1 Data Structures and Algorithms CSE 465 LECTURE 4 More Divide and Conquer Binary Search Exponentiation Multiplication Sofya Raskhodnikova and Adam Smith

2 Review questions How long does Merge Sort take on a sorted input? (Answer: Θ(n log n). The exact input only affects the constants in the running time of the merge operation.) True or False: n 2 + 5n = Ω(n 3 ) (False.) If f(n)=θ(h(n)) and g(n)=θ(h(n)) (True.) then f(n) + g(n) = Θ(h(n))

3 The divide-and-conquer design paradigm 1. Divide the problem (instance) into subproblems. 2. Conquer the subproblems by solving them recursively. 3. Combine subproblem solutions.

4 Binary search Find an element in a sorted array: 1. Divide: Check middle element. 2. Conquer: Recursively search 1 subarray. 3. Combine: Trivial. Example: Find

5 Binary search Find an element in a sorted array: 1. Divide: Check middle element. 2. Conquer: Recursively search 1 subarray. 3. Combine: Trivial. Example: Find

6 Binary search Find an element in a sorted array: 1. Divide: Check middle element. 2. Conquer: Recursively search 1 subarray. 3. Combine: Trivial. Example: Find

7 Binary search Find an element in a sorted array: 1. Divide: Check middle element. 2. Conquer: Recursively search 1 subarray. 3. Combine: Trivial. Example: Find

8 Binary search Find an element in a sorted array: 1. Divide: Check middle element. 2. Conquer: Recursively search 1 subarray. 3. Combine: Trivial. Example: Find

9 Binary search Find an element in a sorted array: 1. Divide: Check middle element. 2. Conquer: Recursively search 1 subarray. 3. Combine: Trivial. Example: Find

10 Binary Search BINARYSEARCH(b, A[1.. n]) find b in sorted array A 1. If n=0 then return not found 2. If A[n/2] = b then returnn/2 3. If A[n/2] < b then 4. return BINARYSEARCH(A[1.. n/2]) 5. Else 6. return n/2 + BINARYSEARCH(A[n/2+1.. n])

11 Recurrence for binary search T(n) = 1 T(n/2) + Θ(1) # subproblems subproblem size work dividing and combining

12 Recurrence for binary search T(n) = 1 T(n/2) + Θ(1) # subproblems subproblem size work dividing and combining T(n) = T(n/2) + c = T(n/4) + 2c = clog n= Θ(lg n).

13 Exponentiation Problem: Compute a b, where b is n bits long Question: How many multiplications? Naive algorithm: Θ(b) = Θ(2 n ) (exponential in the input length!) Divide-and-conquer algorithm: a b = a b/2 a b/2 if b is even; a (b 1)/2 a (b 1)/2 a if b is odd. T(b) = T(b/2) + Θ(1) T(b) = Θ(log b) = Θ(n).

14 Multiplying large integers Given n-bit integers a, b (in binary), compute c=ab a n-1 a n-2 a 0 Naive (grade-school) algorithm: b n-1 b n-2 b 0 Write a,b in binary Compute n intermediate products Do n additions Total work: Θ(n 2 ) n bits n bits n bits 2n bit output

15 Multiplying large integers Divide and Conquer (Attempt #1): Write a = A 1 2 n /2 + A 0 b = B 1 2 n /2 + B 0 We want ab = A 1 B 1 2 n + (A 1 B 0 + B 1 A 0 ) 2 n /2 + A 0 B 0 Multiply n/2 bit integers recursively T(n) = 4T(n/2) + Θ(n) Alas! this is still Θ(n 2 ). (Exercise: write out the recursion tree.)

16 Multiplying large integers Divide and Conquer (Attempt #1): Write a = A 1 2 n /2 + A 0 b = B 1 2 n /2 + B 0 We want ab = A 1 B 1 2 n + (A 1 B 0 + B 1 A 0 ) 2 n /2 + A 0 B 0 Multiply Consider n/2 the bit expression integers recursively (A T(n) 0 +A= 1 4T(n/2) ) (B 0 + B+ 1 ) Θ(n) = A 0 B 0 + A 1 B 1 + (A 0 B 1 + B 1 A 0 ) Alas! We can this get is away still Θ(n with 2 ) 3. multiplications! (in yellow) (Exercise: write out the recursion tree.) x = A 1 B 1 y = A 0 B 0 z = (A 0 +A 1 )(B 0 +B 1 ) Now we use ab = A 1 B 1 2 n + (A 1 B 0 + B 1 A 0 ) 2 n /2 + A 0 B 0 = x 2 n + (z x y) 2 n /2 + y

17 Multiplying large integers MULTIPLY (n, a, b) a and b are n-bit integers Assume n is a power of 2 for simplicity 1. If n 2 then use grade-school algorithm else 2. A 1 a div 2 n /2 ; B 1 b div 2 n /2 ; 3. A 0 a mod 2 n /2 ; B 0 b mod 2 n /2. 4. xmultiply(n/2, A 1, B 1 ) 5. ymultiply(n/2, A 0, B 0 ) 6. zmultiply(n/2, A 1 +A 0, B 1 +B 0 ) 7. Output x 2 n + (z x y)2 n/2 + y

18 Multiplying large integers The resulting recurrence T(n) = 3T(n/2) + Θ(n) We will see that T(n) = Θ (n log 23 ) = Θ(n 1.59 ) Note: There is a Θ(n log n) algorithm for multiplication.

19 Reminder: recursion trees Technique for guessing solution to recurrences Write out tree of recursive calls Each node gets assigned the work done during that call to the procedure (dividing and combining) Total work is sum of work at all nodes

20 Recursion tree for multiplication Solve T(n) = 3T(n/2) + cn, where c > 0 is constant. cn cn lg 2 n cn/2 cn/4 cn/4 Θ(1) cn/4 cn/2 cn/2 At bottom level: (3/2)cn (9/4)cn At level k: (3/2) k cn (3/2) log n cn = c n log 2 3

21 Recursion tree for multiplication Solve T(n) = 3T(n/2) + cn, where c > 0 is constant. lg 2 n Θ(1) cn/2 cn/4 cn/4 cn/4 Total work log n = 2 k = 0 cn cn/2 3 2 k cn cn/2 = Θ(3 log 2 n ) = Θ( n log cn (3/2)cn Geometric series: When base of exponent is constant (e.g. 3/2), the largest term is a constant fraction of total (9/4)cn 2 3 )

22 Appendix: geometric series x x x = for x < x x x x x n n = for x 1

23 So far: 3 recurrences Merge Sort T(n) = 2 T(n/2) + Θ(n) = Θ(n log n) Binary Search / Exponentiation T(n) = 1 T(n/2) + Θ(1) = Θ(log n) Multiplication T(n) = 3 T(n/2) + Θ(n) = Θ(n log 2 3 ) Coming soon: systematic ways to solve these.