Divide & Conquer Algorithms

Transcription

1 Divide & Conquer Algorithms

2 Calculating Powers of a Number Problem: Compute a n, where n N. Naive algorithm: Θ(n).

3 Calculating Powers of a Number Problem: Compute a n, where n N. Naive algorithm: Θ(n). Divide-and-conquer algorithm: a n = a n/2 a n/2 a (n 1)/2 a (n 1)/2 a if n is even; if n is odd.

4 Calculating Powers of a Number Problem: Compute a n, where n N. Naive algorithm: Θ(n). Divide-and-conquer algorithm: a n = a n/2 a n/2 a (n 1)/2 a (n 1)/2 a if n is even; if n is odd. T(n) = T(n/2) + Θ(1) T(n) = Θ(lgn).

5 Fibonacci Numbers Recursive definition: F n = 0 if n = 0 1 if n = 1 F n 1 + F n 2 if n

6 Fibonacci Numbers Recursive definition: F n = 0 if n = 0 1 if n = 1 F n 1 + F n 2 if n Naive recursive algorithm: Ω(φ n ) (exp. time) where φ =(1+ 5)/ 2 is the golden ratio.

7 Fibonacci Numbers Recursive definition: F n = 0 if n = 0 1 if n = 1 F n 1 + F n 2 if n Naive recursive algorithm: Ω(φ n ) (exp. time) where φ =(1+ 5)/ 2 is the golden ratio. Appendix B

8 Computing Fibonacci Numbers Bottom-up: Compute F 0, F 1, F 2,, F n in order, forming each number by summing the two previous. Running time: Θ(n). Naive recursive squaring: F n = φ n / 5 rounded to the nearest integer. Recursive squaring: Θ(lg n) time. This method is unreliable, since floating-point arithmetic is prone to round-off errors. Appendix B

9 Strassen algorithm - Wikipedia, the free encyclopedia 9/28/16, 9:42 PM The left column represents 2x2 matrix multiplication. Naïve matrix multiplication requires one multiplication for each "1" of the left column. Each of the other columns represents a single one of the 7 multiplications in the algorithm, and the sum of2x2 thematrix columns gives thenaïve full matrix The left column represents multiplication. matrix multiplication requires one multiplication for each "1" of the left co multiplication on the left. Each of the other columns represents a single one of the 7 multiplications in the algorithm, and the sum of the columns gives the full m multiplication on the left. Matrix Multiplication Let A, B be two square matrices over a ring R. We want to calculate the matrix product C as A, B be two square a ring R. We want to calculate the matrix product C as The left column represents 2x2 matrix multiplication. Naïve matrix multiplication requires Let one multiplication for eachmatrices "1" of theover left column. Each of the other columns represents a single one of the 7 multiplications in the algorithm, and the sum of the columns gives the full matrix multiplication onrepresents the left. 2x2 matrix multiplication. Naïve matrix multiplication requires one multiplication for each "1" of the left column. The left column Each of the other columns represents a single one of the 7 multiplications in the algorithm, and the sum of the columns gives the full matrix n 2n we fill the missing rows and If the matrices B are not of type 2n 2n we fill the missing rows and columns zeros. IfLet the are not ofover typea 2ring columns multiplication ona, thebleft. A,matrices B be two square matrices R. We want to calculate the matrix product C asa,zeros. We partition A, B and C into equally sized block matrices We A, B and matrices C into equally sized block Letpartition A, B be two square over a ring R. We wantmatrices to calculate the matrix product C as If the matrices A, B are not of type 2n 2n we fill the missing rows and columns zeros. We partition A, A, B and intoofequally If the matrices B arec not type 2nsized 2n block we fillmatrices the missing rows and columns zeros. We partition A, B and C into equally sized block matrices then then then then With this construction we have not reduced the number of multiplications. We still need 8 multiplications to calculate t matrices, the same number of multiplications we need when using standard matrix multiplication. NowWe comes important part. We define matrices With this construction we have not reduced the number of multiplications. stillthe need 8 multiplications to new calculate the Ci,j matrices, same number multiplications we need when using We standard matrix multiplication. With this the construction we haveofnot reduced the number of multiplications. still need 8 multiplications to calculate the Ci,j matrices, the same number of multiplications we need when using standard matrix multiplication. With this construction we have not reduced the number of multiplications. We still need 8 multiplications to calculate the Ci,j Now comes the important part. We define new matrices matrices, thethe same numberpart. of multiplications need when using standard matrix multiplication. Now comes important We define newwe matrices Now comes the important part. We define new matrices P Page 2 of 6 Page 2 of 6 Page 2 of 6

10 Matrix Multiplication SQUARE-MATRIX-MULTIPLY-RECURSIVE.A; B/ 1 n D A:rows 2 letc be a new n n matrix 3 if n == 1 4 c 11 D a 11 b 11 5 else partition A, B,andC as in equations (4.9) 6 C 11 D SQUARE-MATRIX-MULTIPLY-RECURSIVE.A 11 ;B 11 / C SQUARE-MATRIX-MULTIPLY-RECURSIVE.A 12 ;B 21 / 7 C 12 D SQUARE-MATRIX-MULTIPLY-RECURSIVE.A 11 ;B 12 / C SQUARE-MATRIX-MULTIPLY-RECURSIVE.A 12 ;B 22 / 8 C 21 D SQUARE-MATRIX-MULTIPLY-RECURSIVE.A 21 ;B 11 / C SQUARE-MATRIX-MULTIPLY-RECURSIVE.A 22 ;B 21 / 9 C 22 D SQUARE-MATRIX-MULTIPLY-RECURSIVE.A 21 ;B 12 / C SQUARE-MATRIX-MULTIPLY-RECURSIVE.A 22 ;B 22 / 10 return C This pseudocode glosses over one subtle but important implementation ( time of SQUARE-MATRIX-MULTIPLY-RE (.1/ if n D 1; T.n/ D 8T.n=2/ C.n 2 / if n>1: As we shall see from the master method i solution. Thus, this sim hall see from the T.n/ D.n 3 / an the straightfor e we continue on

11 the factor of 8 in equation (4.16) or the factor of 2 in recurrence (4.1), the recursion tree would just be linear, rather than bushy, and each level would contribute only one term to the sum. Bear in mind, therefore, that although asymptotic notation subsumes constant multiplicative factors, recursive notation such as T.n=2/ does not. Strassen s method The key to Strassen s method is to make the recursion tree slightly less bushy. That is, instead of performing eight recursive multiplications of n=2! n=2 matrices, it performs only seven. The cost of eliminating one matrix multiplication will be several new additions of n=2! n=2 matrices, but still only a constant number of additions. As before, the constant number of matrix additions will be subsumed by -notation when we set up the recurrence equation to characterize the running Strassen algorithm - Wikipedia, the free encyclopedia time. Strassen s method is not at all obvious. (This might be the biggest understatewith this construction webook.) have not reduced the number of multiplications. We still need 8 multiplications to calculate the Ci,j ment in this It has four steps: then Strassen algorithm 9/28/16, 9:42 PM matrices, the same number of multiplications we need when using standard matrix multiplication. 1. Divide the input matrices A and B and output matrix C into n=2! n=2 subma- trices, as inpart. equation (4.9).new Thismatrices step takes.1/ time by index calculation, just Now comes the important We define as in S QUARE -M ATRIX -M ULTIPLY-R ECURSIVE. only using 7 multiplications (one for each Mk) instead of 8. We may now express the Ci,j in terms of Mk, like this: 2. Create 10 matrices S1 ; S2 ; : : : ; S10, each of which is n=2! n=2 and is the sum or difference of two matrices created in step 1. We can create all 10 matrices in.n2 / time. Strassen algorithm - Wikipedia, the free encyclopedia 3. Using the submatrices created in step 1 and the 10 matrices created in step 2, recursively compute seven matrix products P1 ; P2 ; : : : ; P7. Each matrix Pi is n=2! n=2. 9/28/16, 9:42 PM Page 2 of 6 4. Compute the desired submatrices C11 ; C12 ; C21 ; C22 of the result matrix C by We iterate this division process n times (recursively) until the submatrices degenerate into numbers (elements of the ring R). adding and subtracting various combinations of the Pi matrices. We can comresulting product will be padded zeroes just like A and B, and should be stripped of the corresponding rows and 2 pute all four (one submatrices.n / time. The only using 7 multiplications for eachinm We may now express the Ci,j in terms of Mk, like this: k) instead of 8. columns. We shall see the details of steps 2 4 in a moment, but we already have enough information to set up a recurrence for the running time of Strassen s method.oflet us Practical implementations Strassen's algorithm switch to standard methods of matrix multiplication for small enough assume that once the matrix size n gets down tosubmatrices, 1, we performfor a simple which scalar those mulalgorithms are more efficient. The particular crossover point for which Strassen's algorithm is 80 Chapter 4 Divide-and-Conquer tiplication, just as in line 4 of S QUARE -M ATRIX -M ULTIPLY ECURSIVE When more efficient-r depends on.the specific implementation and hardware. Earlier authors had estimated that Strassen's algorithm is n > 1, steps 1, 2, and 4 take a total of.n2 / time, and step 3 requires us to perfaster for matrices widths from 32 to 128 for optimized implementations.[1] However, it has been observed that this form seven multiplications of n=2off! n=2 Hence, we obtain the following We have traded onematrices. matrix multiplication for a constant number matrix ad- and a 2010 study found that even a single step of Strassen's algorithm is crossover point has been increasing in of recent years, for theditions. running time T.n/ of Strassen s Once we understand recurrences and their solutions, we shall see that this We iterate thisrecurrence division process n times (recursively) until often the algorithm: submatrices degenerate into numbers (elements of the R). optimized traditional multiplication, until matrix sizes not beneficial on current architectures, compared to aring highly ( actually a lower asymptotic running time. Byof thethe master method rows and The resulting product will be tradeoff padded zeroes A and B,1000 and or should stripped corresponding.1/ if nleads D just 1 ;tolike exceed more,beand even for matrix sizes of several thousand the benefit is typically marginal at best (around 10% or lg 7 (4.18) T.n/ D in Section 4.5, recurrence (4.18) has the solution T.n/ D.n /. columns. [2] 7T.n=2/ C.n2 / if n > 1 : less). We now proceed to describe the details. In step 2, we create the following 10 Practical implementations of Strassen's matrices: algorithm switch to standard methods of matrix multiplication for small enough submatrices, for which those algorithms are more efficient. The particular crossover point for which Strassen's algorithm is Asymptotic complexity D implementation B12! B22 ; 1 more efficient depends on thesspecific and hardware. Earlier authors had estimated that Strassen's algorithm is C A12 ; faster for matrices widthss2fromd32 A to11128 for optimized implementations.[1] However, it has been observed that this The standard matrix multiplication takes approximately 2N3 (where N = 2n) arithmetic operations (additions and S3 Din recent A21 Cyears, A22 ; and a 2010 crossover point has been increasing study found that even a single step of Strassen's algorithm is 3). thetraditional asymptotic complexity isuntil Θ(Nmatrix often not beneficial on currentsarchitectures, to amultiplications); highly optimized multiplication, sizes D B21!compared B11 ; 4 exceed 1000 or more, and even matrix of ;several thousand the benefit is typically marginal at best (around 10% or S5for D A11sizes C A22 The number of additions and multiplications required in the Strassen algorithm can be calculated as follows: let f(n) be the S6 D B11 C B22 ; number of operations for a 2n 2n matrix. Then by recursive application of the Strassen algorithm, we see that S7 D A12! A22 ; f(n) = 7f(n 1) + ℓ4n, for some constant ℓ that depends on the number of additions performed at each application of the Asymptotic complexity S8 D B21 C B22 ; algorithm. Hence f(n) = (7 + o(1))n, i.e., the asymptotic complexity for multiplying matrices of size N = 2n using the S9 D A11! A21 ; Strassen algorithmn is 3 (where The standard matrix multiplication takesb11 approximately 2N N = 2 ) arithmetic operations (additions and S10 D C B12 : 3 multiplications); the asymptotic complexity is Θ(N ).. Since we must add or subtract n=2 " n=2 matrices 10 times, this step does indeed take.n2 / time. The number of additions and multiplications required in the Strassen algorithm can be calculated as follows: let f(n) be the The reduction in matrices the number of times arithmetic operations In step 3, we recursively multiply n=2"n=2 seven to compute the however comes at the price of a somewhat reduced numerical n n less).[2]