2/26/2016. Divide and Conquer. Chapter 6. The Divide and Conquer Paradigm

Transcription

1 Divide and Conquer Chapter 6 Divide and Conquer A divide and conquer algorithm divides the problem instance into a number of subinstances (in most cases 2), recursively solves each subinsance separately, and then combines the solutions to the subinstances to obtain the solution to the original problem instance. CS3330: Algorithms Divide and Conquer Consider the problem of finding both the minimum and maximum in an array of integers A[ n] and assume for simplicity that n is a power of 2. Divide the input array into two halves A[ n2] and A[(n2)+ n], find the minimum and maximum in each half and return the minimum of the two minima and the maximum of the two maxima. Divide and Conquer Input: An array A[ n] of n integers; Output: (x, y): the minimum and maximum integers in A;. minmax(, n); minmax(low, high). if high=low there is only number 2. return (A[low], A[low]); 3. if high=low+ there are only 2 numbers 4. if A[low]<A[high] return (A[low], A[high]); 5. else return(a[high], A[low]); 6. mid(low+high)2 ; more than two numbers 7. (x, y ) minmax(low, mid); 8. (x 2, y 2 ) minmax(mid+, high); 9. return(min{x, x 2 }, max{y, y 2 }); Divide and Conquer How many comparisons does this algorithm need? Given an array A[ n] of n elements, where n is a power of 2, it is possible to find both the minimum and maximum of the elements in A using only.5n 2 element comparisons. Let C(n) be the maximum number of comparisons the algorithm needs for n numbers. C() = 0, C(2) = C(n) = 2C(n2) + 2. Then C(n) =.5n 2 for (power of 2) n >. The Divide and Conquer Paradigm The divide step: the input is partitioned into p parts, each of size strictly less than n. The conquer step: performing p recursive call(s) if the problem size is greater than some predefined threshold n 0. The combine step: the solutions to the p recursive call(s) are combined to obtain the desired output.

2 Sorting algorithms Insertion, selection and bubble sort have quadratic worst case performance The faster comparison based algorithm? O(nlogn) Heapsort, Mergesort, Quicksort Merge Sort An example of divide and conquer technique Problem: Given n elements, sort elements into nondecreasing order Divide and Conquer: If n= terminate (every one element list is already sorted) If n>, partition elements into two or more subcollections; sort each; combine into a single sorted list How do we partition? Partitioning Choice First n elements into set A, last element set B Sort A using this partitioning scheme recursively B already sorted Combine A and B using method Insert() (= insertion into sorted array) Leads to recursive version of InsertionSort() Number of comparisons: O(n 2 ) Best case = n Worst case = c n i 2 n( n ) i 2 Divide Conquer View of Insertion Sort insertion_sort(int [] A, int n) { Pre: 0 <= n < A.length Post: A[0..n] is sorted if (n == 0) return; } divide: do nothing Now we have two parts: A[0..n-2] and A[n-] conquer: insertion_sort(a, n-); combine: insert(a, n-, A[n-]); insert(a, m, x) inserts x into A[0..m-] to keep it sorted. Partitioning Choice 2 Put element with largest key in B, remaining elements in A Sort A recursively To combine sorted A and B, append B to sorted A Use Max() to find largest element recursive SelectionSort() Use bubbling process to find and move largest element to right most position recursive BubbleSort() All O(n 2 ) Divide Conquer View of Selection Sort selection_sort(int [] A, int n) { Pre: 0 <= n < A.length Post: A[0..n] is sorted if (n == 0) return; } divide: int max = maxposition(a, n) ; it returns the position of max element exchange(a, n, max); Now we have two parts: A[0..n-2] and A[n-] conquer: selection_sort(a, n-); combine: do nothing. 2

3 Partitioning Choice 3 Let s try to achieve balanced partitioning A gets n2 elements, B gets the rest half Sort A and B recursively Combine sorted A and B using a process called merge, which combines two sorted lists into one How? We will see soon Mergesort Divide the array into two equally sized halves Recursively sort the halves Merge the sorted halves Example Partition into lists of size n2 [0, 4, 6, 3, 8, 2, 5, 7] Merge Example Cont d [2, 3, 4, 5, 6, 7, 8, 0 ] [0, 4, 6, 3] [8, 2, 5, 7] [3, 4, 6, 0] [2, 5, 7, 8] [0, 4] [6, 3] [8, 2] [5, 7] [4] [0] [3][6] [2][8] [5][7] [4, 0] [3, 6] [2, 8] [5, 7] [4] [0] [3][6] [2][8] [5][7] Merging The key to Merge Sort is merging two sorted lists into one, such that if you have two lists X (x x 2 x m ) and Y(y y 2 y n ) the resulting list is Z(z z 2 z m+n ) Example: X = { } Y = { 5 7 } merge(x, Y) = { } Merging X: Y: Result: 3

4 Merging (cont.) Merging (cont.) X: Y: X: Y: Result: Result: 3 Merging (cont.) Merging (cont.) X: Y: X: Y: Result: 3 5 Result: Merging (cont.) Merging (cont.) X: 54 Y: X: 54 Y: 75 Result: Result:

5 Merging (cont.) Merging (cont.) X: Y: 75 X: Y: Result: Result: Static Method mergesort() Evaluation Public static void mergesort(comparable []a, int left, int right) { sort a[left:right] using b[left:right] if (left < right) { at least two elements int mid = (left+right)2; midpoint mergesort(a, left, mid); mergesort(a, mid +, right); merge(a, b, left, mid, right); merge from a to b copy(b, a, left, right); copy result back to a } } Recurrence equation: Assume n is a power of 2 c if n= T(n) = 2T(n2) + c 2 n if n>, n=2 k By Substitution: T(n) = 2T(n2) + c 2 n T(n2) = 2T(n4) + c 2 n2 T(n) = 4T(n4) + 2 c 2 n T(n) = 8T(n8) + 3 c 2 n T(n) = 2 i T(n2 i ) + ic 2 n Solution Assuming n = 2 k, expansion halts when we get T() on right side; this happens when i=k T(n) = 2 k T() + kc 2 n Since 2 k =n, we know k=logn; since T() = c, we get T(n) = c n + c 2 nlogn; thus an upper bound for T mergesort (n) is O(nlogn) Implementing Merge Sort There are two basic ways to implement merge sort: In Place: Merging is done with only the input array Pro: Requires only the space needed to hold the array Con: Takes longer to merge because if the next element is in the right side then all of the elements must be moved down. Double Storage: Merging is done with a temporary array of the same size as the input array. Pro: Faster than In Place since the temp array holds the resulting array until both left and right sides are merged into the temp array, then the temp array is appended over the input array. Con: The memory requirement is doubled. 5

6 Variants and Applications There are other variants of Merge Sorts including bottom-up merge sort, k-way merge sorting, but the common variant is the Double Memory Merge Sort. Though the running time is O(n log n) and runs much faster than insertion sort and bubble sort, merge sort s large memory demands makes it not very practical for main memory sorting. Merge Sort is the major method for external sorting, parallel algorithms, and sorting circuits. Bottom up Merge Sort Sublists are always of size 2 k, except the last [2, 3, 4, 5, 6, 7, 8, 0 ] [3, 4, 6, 0] [2, 5, 7, 8] [4, 0] [3, 6] [2, 8] [5, 7] [4] [0] [3][6] [2][8] [5][7] Bottom up Merge Sort Sublists are always of size 2 k, except the last Bottom up Merge Sort Sublists are always of size 2 k, except the last [2, 3, 4, 5, 6, 8, 0 ] [2, 3, 4, 6, 8, 0 ] [3, 4, 6, 0] [2, 5, 8] [3, 4, 6, 0] [4, 0] [3, 6] [2, 8] [4] [0] [3][6] [2][8] [5] [4, 0] [3, 6] [2, 8] [4] [0] [3][6] [2][8] Bottom up Merge Sort Sublists are always of size 2 k, except the last [2, 3, 4, 6, 0 ] [3, 4, 6, 0] [4, 0] [3, 6] [4] [0] [3][6] [2] Quicksort Algorithm Given an array of n elements (e.g., integers): If array only contains one element, return Else pick one element to use as pivot. Partition elements into two sub arrays: Elements less than or equal to pivot Elements greater than pivot Quicksort two sub arrays Return results 6

7 Example We are given array of n integers to sort: Pick Pivot Element There are a number of ways to pick the pivot element. In this example, we will use the first element in the array: Partitioning Array 0. = left+; = right; Given a pivot, partition the elements of the array such that the resulting array consists of:. One sub array that contains elements >= pivot 2. Another sub array that contains elements < pivot The sub arrays are stored in the original data array. Partitioning loops through, swapping elements belowabove pivot = left+; = right;. While data[] <= data[pivot] = left+; = right;. While data[] <= data[pivot]

8 0. = left+; = right;. While data[] <= data[pivot] = left+; = right;. While data[] <= data[pivot] While data[] > data[pivot] = left+; = right;. While data[] <= data[pivot] While data[] > data[pivot] = left+; = right;. While data[] <= data[pivot] While data[] > data[pivot] If < swap data[] and data[] = left+; = right;. While data[] <= data[pivot] While data[] > data[pivot] If < swap data[] and data[] 0. = left+; = right;. While data[] <= data[pivot] While data[] > data[pivot] If < swap data[] and data[] 4. If >, go to

9 0. = left+; = right;. While data[] <= data[pivot] While data[] > data[pivot] If < swap data[] and data[] 4. If >, go to. 5. Swap data[] and data[pivot_index] 0. = left+; = right;. While data[] <= data[pivot] While data[] > data[pivot] If < swap data[] and data[] 4. If >, go to. 5. Swap data[] and data[pivot_index] 6. Return pivot_index = Partition Result What is best case running time? <= data[pivot] > data[pivot] Recursive calls on two sides to get a sorted array. What is best case running time? Recursion:. Partition splits array in two sub arrays of size n2 2. Quicksort each sub array What is best case running time? Recursion:. Partition splits array in two sub arrays of size n2 2. Quicksort each sub array Depth of recursion tree? 9

10 What is best case running time? Recursion:. Partition splits array in two sub arrays of size n2 2. Quicksort each sub array Depth of recursion tree? O(log 2 n) What is best case running time? Recursion:. Partition splits array in two sub arrays of size n2 2. Quicksort each sub array Depth of recursion tree? O(log 2 n) Number of accesses in partition? What is best case running time? Recursion:. Partition splits array in two sub arrays of size n2 2. Quicksort each sub array Depth of recursion tree? O(log 2 n) Number of accesses in partition? O(n) Best case running time: O(n log 2 n) Best case running time: O(n log 2 n) Worst case running time? Best case running time: O(n log 2 n) Worst case running time? Recursion:. Partition splits array in two sub arrays: one sub array of size 0 the other sub array of size n 2. Quicksort each sub array Depth of recursion tree? 0

11 Best case running time: O(n log 2 n) Worst case running time? Recursion:. Partition splits array in two sub arrays: one sub array of size 0 the other sub array of size n 2. Quicksort each sub array Depth of recursion tree? O(n) Best case running time: O(n log 2 n) Worst case running time? Recursion:. Partition splits array in two sub arrays: one sub array of size 0 the other sub array of size n 2. Quicksort each sub array Depth of recursion tree? O(n) Number of accesses per partition? Best case running time: O(n log 2 n) Worst case running time? Recursion:. Partition splits array in two sub arrays: one sub array of size 0 the other sub array of size n 2. Quicksort each sub array Depth of recursion tree? O(n) Number of accesses per partition? O(n) Best case running time: O(n log 2 n) Worst case running time: O(n 2 )!!! Best case running time: O(n log 2 n) Worst case running time: O(n 2 )!!! What can we do to avoid worst case? Randomly pick a pivot Bad divide: T(n) = T() + T(n ) + n O(n 2 ) Good divide: T(n) = T(n2) + T(n2) + n O(n log 2 n) Random divide: Suppose on average one bad divide followed by one good divide. T(n) = T() + T(n ) + n = T() + 2T((n )2) + 2n T(n) = 2n + 2T((n )2) is still O(n log 2 n)

12 Improved Pivot Selection Pick median value of three elements from data array: data[0], data[n2], and data[n ]. Use this median value as pivot. For large arrays, use the median of three medians from {data[0], data[], data[2]}, {data[n2 ], data[n2], data[n2+]}, and {data[n 3], data[n 2], data[n ]}. Improving Performance of Quicksort Improved selection of pivot. For sub arrays of size 00 or less, apply brute force search, such as insert sort. Sub array of size : trivial Sub array of size 2: if(data[first] > data[second]) swap them Sub array of size 3: left as an exercise. Improving Performance of Quicksort Improved selection of pivot. For sub arrays of size 00 or less, apply brute force search, such as insert sort. Test if the sub array is already sorted before recursive calls. Don t pick out the pivot (avoiding the last swap). 0. = left+; = right;. while (data[] <= data[pivot] ++; 2. while (data[] > data[pivot]) --; 3. if ( < ) swap data[] and data[]; 4. if ( > ) go to. 5. swap data[] and data[pivot_index]; not needed 6. return ; pivot_index = Divide and Conquer Simple Divide Fancy Divide Uneven Divide vs n Insert Sort Selection Sort Even Divide n2 vs n2 Merge Sort Quick Sort Stable Sorting Maintains the original relative positioning of equivalent values InsertSort, SelectSort, BubbleSort are stable Heapsort is not stable Mergesort is stable Quicksort is not stable 2

13 How Fast Can We Sort? Selection Sort, Bubble Sort, Insertion Sort: O(n 2 ) Heap Sort, Merge sort: O(nlogn) Quicksort: O(nlogn) - average What is common to all these algorithms? Make comparisons between input elements a i < a j, a i a j, a i = a j, a i a j, or a i >a j Comparison based Sorting Comparison sort Only comparison of pairs of elements may be used to gain order information about a sequence. Hence, a lower bound on the number of comparisons will be a lower bound on the complexity of any comparison based sorting algorithm. All our sorts have been comparison sorts The best worst case complexity so far is (n lg n) (e.g., merge sort). We prove a lower bound of (n lg n) for any comparison sort: merge sort and heapsort are optimal. The idea is simple: there are n! outcomes, so we need a tree with n! leaves, and therefore log(n!) = n log n. 73 Decision Tree For insertion sort operating on three elements. :2 > 2:3 :3 >,2,3 :3 2,,3 2:3 > >,3,2 3,,2 2,3, 3,2, Contains 3! = 6 leaves. Simply unroll all loops for all possible inputs. Node i:j means compare A[i] to A[j]. Leaves show outputs; No two paths go to same leaf! Decision Tree (Contd.) Execution of sorting algorithm corresponds to tracing a path from root to leaf. The tree models all possible execution traces. At each internal node, a comparison a i a j is made. If a i a j, follow left subtree, else follow right subtree. View the tree as if the algorithm splits in two at each node, based on information it has determined up to that point. When we come to a leaf, ordering a () a (2) a (n) is established. A correct sorting algorithm must be able to produce any permutation of its input. Hence, each of the n! permutations must appear at one or more of the leaves of the decision tree. A Lower Bound for Worst Case Worst case no. of comparisons for a sorting algorithm is Length of the longest path from root to any of the leaves in the decision tree for the algorithm. Which is the height of its decision tree. A lower bound on the running time of any comparison sort is given by A lower bound on the heights of all decision trees in which each permutation appears as a reachable leaf. Optimal sorting for three elements Any sort of six elements has 5 internal nodes. :2 > 2:3 :3 >,2,3 :3 2,,3 2:3 > >,3,2 3,,2 2,3, 3,2, There must be a worst-case path of length 3. 3

14 A Lower Bound for Worst Case Theorem: Any comparison sort algorithm requires (n lg n) comparisons in the worst case. Proof: Suffices to determine the height of a decision tree. The number of leaves is at least n!(# outputs) The number of internal nodes n! The height is at least log (n! ) = (n lg n) Can we do better? Linear time sorting algorithms Counting Sort Bucket sort Radix Sort Make certain assumptions about the data Linear sorts are NOT comparison sorts 80 Counting Sort Assumptions: n integers which are in the range [0... r] r is in the order of n, that is, r=o(n) Idea: For each element x, find the number of occurrences of x and store it in the counter Place x into its correct position in the output array using the counter. Step (i.e., frequencies) (r=6) 8 82 Step 2. int index = 0; 2. for i= to max_value 3. for j = to counter[i] 4. values[index++] = i; Copy value iinto the array counts[i] times When an element has other info Step : Create a queue for each key. When an element is inserted, the key counter increases by one and the element is stored in the queue. Step 2: For each key ifrom small to big, the elements in the queue iis inserted into the array.. int index = 0; 2. for i= 0 to max_value 3. for j = to counter[i] 4. values[index++] = removefirst(queue[i]);

15 Analysis of Counting Sort Overall time: O(n + r) Step : O(n) Step 2: O(n + r) Space: O(n+r) In practice we use COUNTING sort when r = O(n) running time is O(n) 85 Bucket Sort Assumption: the input is generated by a random process that distributes elements uniformly over a range R = [a..b] Idea: Divide R into k equal sized buckets (k=θ(n)) Distribute the n input values into the buckets Sort each bucket (e.g., using quicksort or bucket sort) Go through the buckets in order, listing elements in each one Similar idea to sort post office mails, where k = 0. Input: A[0.. (n-)], where a A[i] b for all i Output: elements A[i] sorted 86 Example Bucket Sort R = [ ] Example Bucket Sort A B Distribute Into buckets Sort within each bucket Example Bucket Sort Concatenate the lists from 0 to k together, in order Alg.: BUCKET SORT(A, n) Analysis of Bucket Sort for i to n do insert A[i] into list B[ k*a[i] ] for i 0 to k - do sort list B[i] with quicksort sort concatenate lists B[0], B[],..., B[n -] together in order return the concatenated lists O(n) k O(nk log(nk)) =O(nlog(nk) O(k) O(n) (if k=θ(n)) 5

16 Counting Sort vs Bucket Sort Counting sort uses queues while Bucket Sort uses buckets similar data structures. The queues used by Counting sort store the elements of the same key they are already sorted. The buckets used by Bucket Sort store the elements sharing one feature of the keys they need be further sorted. Both are stable sorting algorithms if Bucket Sort uses stable sorting for each bucket. Radix Sort Represents keys as d digit numbers in some base k key = x x 2...x d where 0 x i k Example: key=54 key 0 = 5, d=3, k=0 where 0 x i 9 92 Assumptions d=o() and k =O(n) Radix Sort Sorting looks at one column at a time For a d digit number, sort the least significant digit first Continue sorting on the next least significant digit, until all digits have been sorted Requires only d passes through the list RADIX SORT Alg.: RADIX SORT(A, d) for i to d do use a stable sort to sort array A on digit i (stable sort: preserves order of identical elements) Analysis of Radix Sort Given n numbers of d digits each, where each digit may take Integer Addition Addition. Given two n-bit integers a and b, compute a + b. Grade-school. (n) bit operations. up to k possible values, RADIX SORT correctly sorts the numbers in O(d(n+k)), or O(n) if d = O() and k = O(n) One pass of sorting per digit takes O(n+k) assuming that we use counting sort There are d passes (for each digit) Remark. Grade-school addition algorithm is optimal

17 Integer Multiplication Divide-and-Conquer Multiplication: Warmup Multiplication. Given two n-bit integers a and b, compute a b. Grade-school. (n 2 ) bit operations. To multiply two n-bit integers a and b: Multiply four ½n-bit integers, recursively. Add and shift to obtain result Ex. a 2 n 2 a a 0 b 2 n 2 b b 0 ab 2 n 2 a a 0 2 n 2 b b 0 2 n a b 2 n 2 a b 0 a 0 b a 0 b 0 a = 0000 b = 0000 a a 0 b b 0 T(n) 4Tn2 (n) T(n) (n2 ) recursive calls add, shift Q. Is grade-school multiplication algorithm optimal? Recursion Tree Karatsuba Multiplication 0 if n 0 T(n) 4T(n2) n otherwise T(n) lg n T(n) n 2 k n 2 lg n 2n 2 n k 0 2 n To multiply two n-bit integers a and b: Add two ½n bit integers. Multiply three ½n-bit integers, recursively. Add, subtract, and shift to obtain result. a 2 n 2 a a 0 T(n2) T(n2) T(n2) T(n2) 4(n2) b 2 n 2 b b 0 ab 2 n a b 2 n 2 a b 0 a 0 b a 0 b 0 2 n a b 2 n 2 (a a 0 )(b b 0 ) a b a 0 b 0 a 0 b T(n4) T(n4) T(n4) T(n4)... T(n4) T(n4) T(n4) T(n4) 6(n4) T(n 2 k ) 4 k (n 2 k ) T(2) T(2) T(2) T(2)... T(2) T(2) T(2) T(2) 4 lg n () Karatsuba Multiplication Karatsuba: Recursion Tree To multiply two n-bit integers a and b: Add two ½n bit integers. Multiply three ½n-bit integers, recursively. Add, subtract, and shift to obtain result. 0 if n 0 T(n) 3T(n2) n otherwise T(n) lg n T(n) n 3 k 3 lg n 2 n 2 3 k 0 3nlg 3 2n 2 n a 2 n 2 a a 0 b 2 n 2 b b 0 ab 2 n a b 2 n 2 a b 0 a 0 b a 0 b 0 T(n2) T(n2) T(n2) 3(n2) 2 n a b 2 n 2 (a a 0 )(b b 0 ) a b a 0 b 0 a 0 b T(n4) T(n4) T(n4) T(n4) T(n4) T(n4) T(n4) T(n4) T(n4) 9(n4) Theorem. [Karatsuba-Ofman 962] Can multiply two n-bit integers in O(n.585 ) bit operations.... T(n 2 k )... 3 k (n 2 k ) T n2 T(n) Tn2 T n2 (n) T (n) O(n lg 3 ) O(n.585 ) recursive calls add, subtract, shift... T(2) T(2) T(2) T(2)... T(2) T(2) T(2) T(2)... 3 lg n ()

18 Dot Product Matrix Multiplication Dot product. Given two length n vectors a and b, compute c = a b. Grade-school. (n) arithmetic operations. n a b a i b i a b a b (.70.30) (.20.40) (.0.30).32 i Matrix multiplication. Given two n-by-n matrices A and B, compute C = AB. Grade-school. (n 3 ) arithmetic operations. c c 2 c n a a 2 a n b b 2 b n c 2 c 22 c 2n a 2 a 22 a 2n b 2 b 22 b 2n c n c n2 c nn a n a n2 a nn b n b n2 b nn n c ij a ik b kj k Remark. Grade-school dot product algorithm is optimal. Q. Is grade-school matrix multiplication algorithm optimal? Block Matrix Multiplication Matrix Multiplication: Warmup C A A 2 B B To multiply two n-by-n matrices A and B: Divide: partition A and B into ½n-by-½n blocks. Conquer: multiply 8 pairs of ½n-by-½n matrices, recursively. Combine: add appropriate products using 4 matrix additions. C C 2 A C A B A 2 B 2 A 2 B B 2 C C 2 A B 2 A 2 B 2 C 22 A 2 A 22 B 2 B C 2 A 2 B A 22 B 2 C 22 A 2 B 2 A 22 B 22 C A B A 2 B T(n) 8T n2 (n2 ) T(n) (n 3 ) recursive calls add, form submatrices Fast Matrix Multiplication Fast Matrix Multiplication Key idea. multiply 2-by-2 blocks with only 7 multiplications. C C 2 A A 2 B B 2 C 2 C 22 A 2 A 22 B 2 B 22 P A (B 2 B 22 ) P 2 ( A A 2 ) B 22 To multiply two n-by-n matrices A and B: [Strassen 969] Divide: partition A and B into ½n-by-½n blocks. Compute: 4 ½n-by-½n matrices via 0 matrix additions. Conquer: multiply 7 pairs of ½n-by-½n matrices, recursively. Combine: 7 products into 4 terms using 8 matrix additions. C P 5 P 4 P 2 P 6 C 2 P P 2 C 2 P 3 P 4 C 22 P 5 P P 3 P 7 7 multiplications. 8 = additions and subtractions. P 3 ( A 2 A 22 ) B P 4 A 22 (B 2 B ) P 5 ( A A 22 ) (B B 22 ) P 6 ( A 2 A 22 ) (B 2 B 22 ) P 7 ( A A 2 ) (B B 2 ) Analysis. Assume n is a power of 2. T(n) = # arithmetic operations. T(n) 7T n2 (n2 ) T(n) (n log 2 7 ) O(n 2.8 ) recursive calls add, subtract

19 Fast Matrix Multiplication Fast Matrix Multiplication: Practice To multiply two n-by-n matrices A and B: [Strassen 969] T(n) 7T n2 (n2 ) T(n) (n log 2 7 ) O(n 2.8 ) recursive calls add, subtract Implementation issues. Sparsity. Caching effects. Numerical stability. Odd matrix dimensions. Crossover to classical algorithm around n = 28. Common misperception. Strassen is only a theoretical curiosity. Apple reports 8x speedup on G4 Velocity Engine when n 2,500. Range of instances where it's useful is a subject of controversy. Remark. Can "Strassenize" Ax = b, determinant, eigenvalues, SVD, Fast Matrix Multiplication: Theory Fast Matrix Multiplication: Theory Q. Multiply two 2-by-2 matrices with 7 scalar multiplications? A. Yes! [Strassen 969] (n log 2 7 ) O(n ) Q. Multiply two 2-by-2 matrices with 6 scalar multiplications? A. Impossible. [Hopcroft and Kerr 97] (n log 2 6 ) O(n 2.59 ) Q. Two 3-by-3 matrices with 2 scalar multiplications? A. Also impossible. (n log 3 2 ) O(n 2.77 ) Begun, the decimal wars have. [Pan, Bini et al, Schönhage, ] Two 20-by-20 matrices with 4,460 scalar multiplications. O(n ) O(n Two 48-by-48 matrices with 47,27 scalar multiplications. ) A year later. O(n ) December, 979. O(n ) January, 980. O(n ) Best known. O(n ) [Coppersmith-Winograd, 987] Conjecture. O(n 2+ ) for any > 0. Caveat. Theoretical improvements to Strassen are progressively less practical. 2 9