2 Divide and conquer Overview, Structure of divide-and-conquer algorithms, binary search, quick sort, Strassen multiplication. 13% 05 Divide-and- conquer The Divide and Conquer Paradigm, is a method of designing algorithms, a general approach to construct an efficient solution to a problem. The Divide and Conquer Paradigm is an algorithm design paradigm which uses this simple process: It Divides the problem into smaller sub-parts until these sub-parts become simple enough to be solved, and then the sub parts are solved recursively, and then the solutions to these sub-parts can be combined to give a solution to the original problem. In other words, this paradigm follows three basic steps; Step 1: Break down the problem into smaller sub-problems, Step 2: Solve these separate sub-problems, and Step 3: Combine the solutions of the sub-problems. Advantages: Solving difficult problems : Divide and conquer is a powerful tool for solving conceptually difficult problems: all it requires is a way of breaking the problem into sub-problems, of solving the trivial cases and of combining sub-problems to the original problem. Similarly, decrease and conquer only requires reducing the problem to a single smaller problem, such as the classic Tower of Hanoi puzzle, which reduces moving a tower of height n to moving a tower of height n 1. Algorithm efficiency : The divide-and-conquer paradigm often helps in the discovery of efficient algorithms. It was the key, for example, to Karatsuba's fast multiplication method, the quicksort and mergesort algorithms, the Strassen algorithm for matrix multiplication, and fast Fourier transforms. Parallelism : Divide and conquer algorithms are naturally adapted for execution in multiprocessor machines, especially shared-memory systems where the communication of data between processors does not need to be planned in advance, because distinct sub-problems can be executed on different processors. D & C mainly have three parts Divide : divide the problem into smaller sub problems. Conquer : the sub problem solving them recursively. If they are small enough solve the sub problem as base case. Combine : merge the solution of sub problem to find solution of actual problem. Analysis and Design of Algorithm Page 1
If we expand out two more recursive steps, it looks like this: Because divide-and-conquer creates at least two subproblems, a divide-and-conquer algorithm makes multiple recursive calls. Analysis and Design of Algorithm Page 2
Applications: Merge Sort Quick Sort Binary Search Strassen's Matrix Multiplication Closest pair (points) Binary Search: Binary search is a fast search algorithm with run-time complexity of Ο(log n). This search algorithm works on the principle of divide and conquer. For this algorithm to work properly the data collection should be in sorted form. Binary search search a particular item by comparing the middle most item of the collection. If match occurs then index of item is returned. If middle item is greater than item then item is searched in sub-array to the right of the middle item other wise item is search in sub-array to the left of the middle item. This process continues on sub-array as well until the size of subarray reduces to zero. How binary search works? For a binary search to work, it is mandatory for the target array to be sorted. Here is the process of binary search with an pictorial example. The below given is our sorted array and assume that we need to search location of value 31 using binary search. First, we shall determine the half of the array by using this formula mid = low + (high - low) / 2 Here it is, 0 + (9-0 ) / 2 = 4 (integer value of 4.5). So 4 is the mid of array. Now we compare the value stored at location 4, with the value being searched i.e. 31. We find that value at location 4 is 27, which is not a match. Because value is greater than 27 and Analysis and Design of Algorithm Page 3
we have a sorted array so we also know that target value must be in upper portion of the array. We change our low to mid + 1 and find the new mid value again. low = mid + 1 mid = low + (high - low) / 2 Our new mid is 7 now. We compare the value stored at location 7 with our target value 31. The value stored at location 7 is not a match, rather it is less that what we are looking for. So the value must be in lower part from this location. So we calculate the mid again. This time it is 5. We compare the value stored ad location 5 with our target value. We find that it is a match. We conclude that the target value 31 is stored at location 5. Binary search halves the searchable items and thus reduces the count of comparisons to be made to very less numbers. Pseudo code The pseudo code of binary search algorithm should look like this Analysis and Design of Algorithm Page 4
Procedure binary_search A sorted array n size of array x value ot be searched Set lowerbound = 1 Set upperbound = n while x not found if upperbound < lowerbound EXIT: x does not exists. set midpoint = lowerbound + ( upperbound - lowerbound ) / 2 if A[midPoint] < x set lowerbound = midpoint + 1 if A[midPoint] > x set upperbound = midpoint - 1 if A[midPoint] = x EXIT: x found at location midpoint end while end procedure Time Complexity is Ο(log n). Analysis and Design of Algorithm Page 5
Quick Sort: Quick sort is a highly efficient sorting algorithm and is based on partitioning of array of data into smaller arrays. A large array is partitioned into two arrays one of which holds values smaller than specified value say pivot based on which the partition is made and another array holds values greater than pivot value. The quick sort partitions an array and then calls itself recursively twice to sort the resulting two subarray. This algorithm is quite efficient for large sized data sets as its average and worst case complexity are of O(nlogn) where n are no. of items. The three steps of Quicksort are as follows: Divide: Rearrange the elements and split the array into two subarrays and an element in between such that so that each element in the left subarray is less than or equal the middle element and each element in the right subarray is greater than the middle element. Conquer: Recursively sort the two subarrays. Combine: None 1. Features o Divide-and-conquer recursive algorithm o One of the fastest sorting algorithms o Average running time O(NlogN) o Worst-case running time O(N 2 ) 2. Basic idea 1. Pick one element in the array, which will be the pivot. 2. Make one pass through the array, called a partition step, re-arranging the entries so that: the pivot is in its proper place. entries smaller than the pivot are to the left of the pivot. entries larger than the pivot are to its right. 3. Recursively apply quicksort to the part of the array that is to the left of the pivot, and to the right part of the array. Here we don't have the merge step, at the end all the elements are in the proper order. Algorithm STEP 1. Choosing the pivot Analysis and Design of Algorithm Page 6
Choosing the pivot is an essential step. Depending on the pivot the algorithm may run very fast, or in quadric time.: Example: 1. Some fixed element: e.g. the first, the last, the one in the middle This is a bad choice - the pivot may turn to be the smallest or the largest element, then one of the partitions will be empty. 2. Randomly chosen (by random generator ) - still a bad choice. 3. The median of the array (if the array has N numbers, the median is the [N/2] largest number. This is difficult to compute - increases the complexity. 4. The median-of-three choice: take the first, the last and the middle element. Choose the median of these three elements. 8, 3, 25, 6, 10, 17, 1, 2, 18, 5 The first element is 8, the middle - 10, the last - 5. The median of [8, 10, 5] is 8 STEP 2. Partitioning Partitioning is illustrated on the above example. 1. The first action is to get the pivot out of the way - swap it with the last element 5, 3, 25, 6, 10, 17, 1, 2, 18, 8 2. We want larger elements to go to the right and smaller elements to go to the left. Two "fingers" are used to scan the elements from left to right and from right to left: [5, 3, 25, 6, 10, 17, 1, 2, 18, 8] ^ ^ i j o While i is to the left of j, we move i right, skipping all the elements less than the pivot. If an element is found greater then the pivot, i stops Analysis and Design of Algorithm Page 7
o o o While j is to the right of i, we move j left, skipping all the elements greater than the pivot. If an element is found less then the pivot, j stops When both i and j have stopped, the elements are swapped. When i and j have crossed, no swap is performed, scanning stops, and the element pointed to by i is swapped with the pivot. In the example the first swapping will be between 25 and 2, the second between 10 and 1. 3. Restore the pivot. After restoring the pivot we obtain the following partitioning into three groups: [5, 3, 2, 6, 1] [ 8 ] [10, 25, 18, 17] STEP 3. Recursively quicksort the left and the right parts Partition in Quicksort The pivot value divides the list in to two parts. And recursively we find pivot for each sublists until all lists contains only one element. QuickSort Pivot Algorithm Based on our understanding of partitioning in quicksort, we should now try to write an algorithm for it here. Step 1 Choose the highest index value as pivot Step 2 Take two variables to point left and right of the list excluding pivot Step 3 left points to the low index Step 4 right points to the high Step 5 while value at left is less than pivot move right Step 6 while value at right is greater than pivot move left Step 7 if both step 5 and step 6 does not match swap left and right Step 8 if left right, the point where they met is new pivot QuickSort Pivot Pseudocode The pseudocode for the above algorithm can be derived as function partitionfunc(left, right, pivot) leftpointer = left -1 Analysis and Design of Algorithm Page 8
rightpointer = right while True do while A[++leftPointer] < pivot do //do-nothing end while while rightpointer > 0 && A[--rightPointer] > pivot do //do-nothing end while if leftpointer >= rightpointer break else swap leftpointer,rightpointer end if end while swap leftpointer,right return leftpointer end function QuickSort Algorithm Using pivot algorithm recursively we end-up with smaller possible partitions. Each partition then processed for quick sort. We define recursive algorithm for quicksort as below Step 1 Make the right-most index value pivot Step 2 partition the array using pivot value Step 3 quicksort left partition recursively Step 4 quicksort right partition recursively QuickSort Pseudocode To get more into it, let see the pseudocode for quick sort algorithm procedure quicksort(left, right) Analysis and Design of Algorithm Page 9
if right-left <= 0 return else pivot = A[right] partition = partitionfunc(left, right, pivot) quicksort(left,partition-1) quicksort(partition+1,right) end if end procedure Example: Analysis and Design of Algorithm Page 10
Analysis and Design of Algorithm Page 11
Analysis and Design of Algorithm Page 12
Analysis and Design of Algorithm Page 13
Analysis and Design of Algorithm Page 14
Analysis and Design of Algorithm Page 15
Analysis and Design of Algorithm Page 16
Analysis and Design of Algorithm Page 17
Analysis and Design of Algorithm Page 18
Strassen multiplication: Naive Method Following is a simple way to multiply two matrices. void multiply(int A[][N], int B[][N], int C[][N]) { for (int i = 0; i < N; i++) { for (int j = 0; j < N; j++) { C[i][j] = 0; for (int k = 0; k < N; k++) { C[i][j] += A[i][k]*B[k][j]; } } } } Time Complexity of above method is O(N 3 ). Let A, B be two square matrices. Calculate the matrix product C as Analysis and Design of Algorithm Page 19
If the matrices A, B are not of type 2 n 2 n we fill the missing rows and columns with zeros. We partition A, B and C into equally sized block matrices with then C i, j N k 1 a i, k b ThusT ( N) k, j N N N i1 j1 k 1 c cn 3 O( N 3 ) Strassen showed that 2x2 matrix multiplication can be accomplished in 7 multiplication and 18 additions or subtractions..(2 log 2 7 =2 2.807 ) This reduction can be done by Divide and Conquer Approach. A B = R Divide matrices into sub-matrices: A 0, A 1, A 2 etc Use blocked matrix multiply equations Recursively multiply sub-matrices Analysis and Design of Algorithm Page 20
Strassan Algorithm: C 11 = P 1 + P 4 - P 5 + P 7 = (A 11 + A 22 )(B 11 +B 22 ) + A 22 * (B 21 - B 11 ) - (A 11 + A 12 ) * B 22 + (A 12 - A 22 ) * (B 21 + B 22 ) = A 11 B 11 + A 11 B 22 + A 22 B 11 + A 22 B 22 + A 22 B 21 A 22 B 11 - A 11 B 22 -A 12 B 22 + A 12 B 21 + A 12 B 22 A 22 B 21 A 22 B 22 = A 11 B 11 + A 12 B 21 Complexity: Where N is the size of matrix. Analysis and Design of Algorithm Page 21