Outline. CS38 Introduction to Algorithms. Fast Fourier Transform (FFT) Fast Fourier Transform (FFT) Fast Fourier Transform (FFT)

Transcription

1 Outline CS8 Introduction to Algorithms Lecture 9 April 9, 0 Divide and Conquer design paradigm matrix multiplication Dynamic programming design paradigm Fibonacci numbers weighted interval scheduling knapsack matrix-chain multiplication longest common subsequence April 9, 0 CS8 Lecture 9 Discrete Fourier Transform (DFT) Given n-th root of unity!, DFT n is a linear map from C n to C n : Fast Fourier Transform (FFT) DFT n has special structure (assume n= k ) reorder columns: first even, then odd consider exponents on! along rows: multiples of: same multiples plus: (i,j) entry is! ij rows repeat twice since! n = April 9, 0 CS8 Lecture 9 April 9, 0 CS8 Lecture 9 Fast Fourier Transform (FFT) so we are actually computing: so to compute DFT n x FFT(n:power of ; x). let! be a n-th root of unity. compute a = FFT(n/, x even ). compute b = FFT(n/, x odd ). y even = a + D b and y odd = a +! n/ D b 5. return vector y! is (n/)-th root of unity D = diagonal matrix diag(! 0,!,!,,! n/- ) April 9, 0 CS8 Lecture 9 5 Fast Fourier Transform (FFT) FFT(n:power of ; x). let! be a n-th root of unity. compute a = FFT(n/, x even ). compute b = FFT(n/, x odd ). y even = a + D b and y odd = a +! n/ D b 5. return vector y Running time? T() = T(n) = T(n/) + O(n) solution: T(n) = O(n log n) April 9, 0 CS8 Lecture 9 6

2 matrix multiplication A X B = C given n x n matrices A, B compute C = AB standard method: O(n ) operations Strassen: O(n log 7) = O(n.8 ) April 9, 0 CS8 Lecture 9 7 a a a a b b b b a b + a b + a b a b a b + a b + a b a b how many product operations? 8 Strassen: it is possible with 7 (!!) 7 products of form: (linear combos of a entries) x (linear combos of b entries) result is linear combos of these 7 products April 9, 0 CS8 Lecture 9 8 a a a a b b b b a b + a b + a b a b a b + a b + a b a b 7 products of form: (linear combos of a entries) x (linear combos of b entries) result is linear combos of these 7 products Key: identity holds when entries above are n/ x n/ matrices rather than scalars April 9, 0 CS8 Lecture 9 9 Strassen-matrix-mult(A, B: n x n matrices). p = A (B B ). p = (A + A ) B. p = (A + A ) B. p = A (B B ) 5. p 5 = (A + A )(B + B ) 6. p 6 = (A A )(B + B ) 7. p 7 = (A A )(B + B ) 8. C = P 5 + P P + P 6 ; C = P + P 9. C = P + P ; C = P 5 + P P P 7 0. return C 7 recursive calls additions/subtractions are entrywise: O(n ) a a a a b b b b c c c c running time recurrence? T(n) = 7T(n/) + O(n ) Solution: T(n) = O(n log 7) = O(n.8 ) a a a a b b b b April 9, 0 CS8 Lecture 9 April 9, 0 CS8 Lecture 9

3 a a a a b b b b c c c c a a a a b b b b c c c c b b b b b b b b a a a a a a a a April 9, 0 CS8 Lecture 9 April 9, 0 CS8 Lecture 9 express these a a a a b b b b c c c c a a a a b b b b as linear combinations of rank- matrices a a a a b b b b e.g.: April 9, 0 CS8 Lecture 9 5 April 9, 0 CS8 Lecture 9 6 Strassen s example Dynamic programming programming = planning dynamic = over time basic idea: identify subproblems express solution to subproblem in terms of other smaller subproblems build solution bottom-up by filling in a table defining subproblem is the hardest part April 9, 0 CS8 Lecture 9 8

4 Dynamic programming Simple example: computing Fibonacci #s f() = f() = f(i) = f(i-) + f(i-) recursive algorithm: Fibonacci(n). if n = or n = return(). else return(fibonacci(n-) + Fibonacci (n-)) running time? Dynamic programming Fibonacci(n). if n = or n = return(). else return(fibonacci(n-) + Fibonacci (n-)) better idea: -dimensional table; entry i contains f(i) build table bottom-up Fibonacci-table(n). T() = T() =. for i = to n do T(i) = T(i-) + T(i-). return(t(n)) April 9, 0 CS8 Lecture 9 9 April 9, 0 CS8 Lecture 9 0 job j starts at s j, finishes at f j, weight v j jobs compatible if they don't overlap a b c d Goal: find maximum weight subset of mutually compatible jobs. recall: greedy by earliest finishing time worked when weights were all counterexample with general weights: weight = 999 b e f weight = a h time g h time April 9, 0 CS8 Lecture 9 label jobs by finishing time f j Definition: p(j) = largest index i < j such that job i is compatible with j. 7 8 e.g. p(8) = 5 time subproblem j: jobs j OPT(j) = value achieved by optimum schedule relate to smaller subproblems case : use job j can t use jobs p(j)+,, j- p(j) = largest index i such that job i is compatible with j. must use optimal schedule for p(j) = OPT(p(j)) case : don t use job j must use optimal schedule for j- = OPT(j-) April 9, 0 CS8 Lecture 9 April 9, 0 CS8 Lecture 9

5 job j starts at s j, finishes at f j, weight v j OPT(j) = max {v j + OPT(p(j)), OPT(j-)} recursive solution? wtd-interval-schedule ((s, f, v ),, (s n, f n, v n )). a = v j + wtd-interval-schedule(first p(n) jobs). b = wtd-interval-schedule(first j- jobs). return(max(a,b)) running time? p(j) = largest index i such that job i is compatible with j. April 9, 0 CS8 Lecture 9 5 job j starts at s j, finishes at f j, weight v j OPT(j) = max {v j + OPT(p(j)), OPT(j-))} Wtd-interval-schedule((s, f, v ),, (s n, f n, v n )). OPT(0) = 0. sort by finish times f_i; compute p(i) for all i. for i = to n. OPT(i) = max {v i + OPT(p(i)), OPT(i-)} 5. return(opt(n)) running time? p(j) = largest index i such that job i is compatible with j. April 9, 0 CS8 Lecture 9 6 Store extra info:. was job i picked?. which table cell has solution to resulting subproblem? Wtd-interval-schedule((s, f, v ),, (s n, f n, v n )). OPT(0) = 0. sort by finish times f_i; compute p(i) for all i. for i = to n. OPT(i) = max {v i + OPT(p(i)), OPT(i-))} 5. return(opt(n)) OPT(n) gives value of optimal schedule how do we actually find schedule? April 9, 0 CS8 Lecture 9 7 Knapsack item i has weight w i and value v i goal: pack knapsack of capacity W with maximum value set of items greedy by weight, value, or ratio of weight/value all fail subproblems: optimum among items i-? April 9, 0 CS8 Lecture 9 8 Knapsack subproblems: optimum among items i-? case : don t use item i OPT(i) = OPT(i-) case : do use item i OPT(i) =? [what is weight used by subproblem?] subproblems, second attempt: optimum among items i-, with total weight w April 9, 0 CS8 Lecture 9 9 Knapsack subproblems: optimum among items i-, with total weight w case : don t use item i OPT(i, w) = OPT(i-, w) case : do use item i OPT(i, w) = OPT(i-, w w i ) OPT(i, w) = OPT(i-, w) if w i > w else: max {v i + OPT(i-, w-w i ), OPT(i-, w))} order to fill in the table? April 9, 0 CS8 Lecture 9 0 5

6 Knapsack Knapsack(v, w,, v n, w n, W). OPT(i, 0) = 0 for all i. for i = to n. for w = to W. if w i > w then OPT(i,w) = OPT(i-, w) 5. else OPT(i,w) = {v i + OPT(i-, w-w i ), OPT(i-, w))} 6. return(opt(n, W)) Running time? O(nW) space: O(nW) can improve to O(W) (how?) how do we actually find items? April 9, 0 CS8 Lecture 9 Sequence of matrices to multiply e.g. 0 A B goal: find best parenthesization e.g.: ((A B) C) D) = = 0 e.g. (A (B (C D)) = = 6 April 9, 0 CS8 Lecture 9 9 C 9 D = Sequence of n matrices to multiply, given by a, a,, a n+ Goal: output fully parenthesized expression with minimum cost fully parenthesized = single matrix: (A) or product of two fully parenthesized: (.)(.) try subproblems for ranges: OPT(,n) = min k OPT(,k) + OPT(k+,n) + a a k+ a n+ Sequence of n matrices to multiply, given by a, a,, a n+ OPT(i,j) = cost to multiply matrices i j optimally OPT(i,j) = 0 if i = j OPT(i,j) = min k OPT(i,k) + OPT(k+,j) + a i a k+ a j+ what order to fill in the table? April 9, 0 CS8 Lecture 9 April 9, 0 CS8 Lecture 9 Matrix-Chain(a, a,, a n+ ). OPT(i, i) = 0 for all i. for r = to n. for i = to n r ; j = i + r. OPT(i,j) = min i k < j OPT(i,k) + OPT(k+,j) + a i a k+ a j+ 5. return(opt(, n)) running time? O(n ) print out the optimal parenthesization? store chosen k in each cell April 9, 0 CS8 Lecture 9 5 6