Chapter 3:- Divide and Conquer. Compiled By:- Sanjay Patel Assistant Professor, SVBIT.

Transcription

1 Chapter 3:- Divide and Conquer Compiled By:- Assistant Professor, SVBIT.

2 Outline Introduction Multiplying large Integers Problem Problem Solving using divide and conquer algorithm - Binary Search Sorting (Merge Sort, Quick Sort), Matrix Multiplication, Exponential.

3 What is divide and conquer A general paradigm for algorithm design Three-step process: 1. Divide the problem into smaller problems. 2. Conquer by solving these problems. 3. Combine these results together.

4 Cont d. Many useful algorithms are recursive in structure: to solve a given problem, they call themselves recursively one or more times to deal with closely related sub problems. These algorithms typically follow a divide-andconquer approach: they break the problem into several sub problems that are similar to the original problem but smaller in size, solve the sub problems recursively, and then combine these solutions to create a solution to the original problem.

5 Cont d The divide-and-conquer paradigm involves three steps at each level of the recursion. Divide the problem into a number of sub problems. Conquer the sub problems by solving them recursively. If the sub problem sizes are small enough, however, just solve the sub problems in a straightforward manner. Combine the solutions to the sub problems into the solution for the original problem.

6 Cont d.. a problem of size n subproblem 1 of size n/2 subproblem 2 of size n/2 Solution to Sub problem 1 Solution to Sub problem 2 Solution to the original problem It general leads to a recursive algorithm!

7 Multiplying large integers numbers A small example, A B where A = 2135 and B = 4014 A = ( ), B = ( ) So, A B = ( ) ( ) = ( )

8 Generalize the Formula We can generalize the formula for multiplication of large number, C = a * b C = c2 10 n + c1 10 n/2 + c0 Where n is total number of digits in the integer c2 = a1 * b1 C0 = a0 * b0 c1 = (a1 + a0 ) * (b1+ b0 ) - ( c2+ c0 )

9 Example.. Perform multiplication with the formula 42 * 34. C= a * b so, 42 * 34 a1= 4,, a0 = 2 b1 = 3, b0= 4 Now, we have to find, c0, C1, c2.. c2 = a1 * b1 =4 * 3 = 12 C0 = a0 * b0 =2 * 4 = 8 c1 = (a1 + a0 ) * (b1+ b0 ) - ( c2+ c0 ) = (4+2) * (3+4) (12+8) = 22

10 Cont d C = c c1 10 2/2 + c0 = 12 * * = = 1428 C = a * b 1428= a * b

11 Another example Perform multiplication 0981 * 1234 C= a * b = 0981 * 1234 a1=09, a0=81, b1=12, b0= 34

12 Cont d So, c2 = 108 c0 = 2754 c1 = 1278 C = c c1 10 4/2 + c0 = 108 * * =

13 Another method Multiply by solve by divide and conquer method * 0987 = * 6543 = * 0987 = * 6543 =

14 Cont d

15 à la russe method 63 * 123 by à la russe rule Answer = = 7749

16 Another Example 64 * 125 by à la russe rule Answer= 8000

17 Another example Answer = =

18 Analysis In this method there are 3 multiplication operation This is the case for two digit multiplication, now if there are n digit to be multiplied, then multiplication of n digits require 3 multiplication of n/2 digit numbers. Hence, recurrence relation C(n) = 3 C(n/2) where n>1 C(n)= 1 where n=1 After solving this recurrence equation with backward substitution method, C(n) = 3 log 2 n = n log 2 3 so, we will get C(n) ~ n 1.5

19 Sorting Merge Quick

20 Merge Sort Divide into two halves A FirstPart SecondPart Recursively sort FirstPart SecondPart Merge A is sorted!

21 Cont d The merge sort algorithm closely follows the divideand-conquer paradigm. Intuitively, it operates as follows. Divide: Divide the n-element sequence to be sorted into two subsequences of n/2 elements each. Conquer: Sort the two subsequences recursively using merge sort. Combine: Merge the two sorted subsequences to produce the sorted answer.

22 Example

23 Cont d

24 Cont d

25 Cont d

26 Cont d

27 Cont d

28 Cont d

29 Cont d

30 Cont d

31 Cont d

32 Cont d

33 Cont d

34 Cont d

35 Cont d

36 Cont d

37 Cont d

38 Cont d

39 Cont d

40 Cont d

41 Cont d

42 Cont d

43 Cont d

44 Another approach

45 Cont d Merge

46 Cont d Merge

47 Cont d Merge

48 Cont d Merge

49 Cont d Merge

50 Cont d Merge

51 Cont d Merge

52 Cont d Merge

53 Cont d Merge

54 Another example

55 Cont d

56 Cont d

57 Cont d

58 Cont d

59 Cont d

60 Cont d

61 Cont d

62 Cont d

63 Cont d

64 Analysis We can obtain the time complexity of merge sort. T(n) = T(n/2) + T(n/2) + cn Time taken by left sub list to get sorted Time taken by right sub list to get sorted Time taken for combining two sub list T(n)= 2 T(n/2) + cn with T(1)= 0 As per master theorem. a=2, b=2 and d=1 So, T(n)= (n d log n) Hence, the time complexity of the merge sort is (nlog n)

65 Algorithm.. If(low < high) then { mid ( low + high ) / 2 Merge sort(a,low,mid) Merge sort(a,mid+1,high) Combine(A,low,mid,high) } // spilt the list //first sub list //second sub list // merging of two sub list

66 Cont d.. Algorithm Combine( A[0.n-1]) K low // k as index for array temp i low // i as index for left sublist of array A J mid+1 // J as index for right sublist of array A

67 Cont d.. While(I <= mid and J<= high) do If (A[i] <= A[J]) then // if smaller element is present in left sub list { // copy that smaller element to temp array temp[k] A[i] i i+1 k k+1 } else // smaller element is present in right sub list { // copy that smaller element in temp array temp[k] A[J] J J+1 k k+1 } }

68 Cont d.. // copy remaining element of left sub list to temp While(I <= mid) do { temp[k] A[i] i i+1 k k+1 } // copy remaining element of right sub list to temp While(J <= high) do { temp[k] A[J] J J+1 k k+1 }

69 Time complexity ( Merge sort ) Best case Average case Worst case Θ( n log2n) Θ( n log2n) Θ(n log2n)

70 Implementation There are two basic ways to implement merge sort: In Place: Merging is done with only the input array Pro: Requires only the space needed to hold the array Con: Takes longer to merge because if the next element is in the right side then all of the elements must be moved down. Double Storage: Merging is done with a temporary array of the same size as the input array. Pro: Faster than In Place since the temp array holds the resulting array until both left and right sides are merged into the temp array, then the temp array is appended over the input array. Con: The memory requirement is doubled.

71 Cont d.. There are other variants of Merge Sorts including k-way merge sorting, but the common variant is the Double Memory Merge Sort. Though the running time is O(N logn) and runs much faster than insertion sort and bubble sort, merge sort s large memory demands makes it not very practical for main memory sorting. Important things to remember for the Midterm: Best Case, Average Case, and Worst Case = O(N logn) Storage Requirement: Double that needed to hold the array to be sorted.

72 Quick sort Quick Sort is an algorithm based on the DIVIDE- AND-CONQUER paradigm that selects a pivot element and reorders the given list in such a way that all elements smaller to it are on one side and those bigger than it are on the other. Then the sub lists are recursively sorted until the list gets completely sorted. The time complexity of this algorithm is O (n log n).

73 Advantages of quick sort Pros: 1) One advantage of parallel quick sort over other parallel sort algorithms is that no synchronization is required. A new thread is started as soon as a sub list is available for it to work on and it does not communicate with other threads. When all threads complete, the sort is done. 2) All comparisons are being done with a single pivot value, which can be stored in a register. 3) The list is being traversed sequentially, which produces very good locality of reference and cache behavior for arrays.

74 Disadvantages of quick sort Cons: 1) Auxiliary space used in the average case for implementing recursive function calls is O (log n) and hence proves to be a bit space costly, especially when it comes to large data sets. 2) Its worst case has a time complexity of O (n2) which can prove very fatal for large data sets. Competitive sorting algorithms

75 Example Lets understand this algorithm with the help of some example Consider pivot element= 50

76 Example If an unsorted list (represented as vector a) originally contains: we might select the item in the middle position, a[5], as the pivot, which is 8 in our illustration. Our process would then put all values less than 8 on the left side and all values greater than 8 on the right side. This first subdivision produces Now, each sublist is subdivided in exactly the same manner. This process continues until all sublists are in order. The list is then sorted. This is a recursive process.

77 Cont d.. We choose the value in the middle for the pivot. As in binary search, the index of this value is found by (first + last) / 2, where first and last are the indices of the initial and final items in the vector representing the list. We then identify a left_arrow and right_arrow on the far left and far right, respectively. This can be envisioned as: where left_arrow and right_arrow initially represent the lowest and highest indices of the vector items.

78 Cont d.. Starting on the right, the right_arrow is moved left until a value less than or equal to the pivot is encountered. This produces In a similar manner, left_arrow is moved right until a value greater than or equal to the pivot is encountered. This is the situation just encountered. Now the contents of the two vector items are swapped to produce Mr. Dave Clausen 78

79 Cont d.. We continue by moving right_arrow left to produce and moving left_arrow right yields These values are exchanged to produce

80 Cont d.. This process stops when left_arrow > right_arrow is TRUE. Since this is still FALSE at this point, the next right_arrow move produces and the left_arrow move to the right yields

81 Cont d.. Because we are looking for a value greater than or equal to pivot when moving left, left_arrow stops moving and an exchange is made to produce Notice that the pivot, 8, has been exchanged to occupy a new position. This is acceptable because pivot is the value of the item, not the index. As before, right_arrow is moved left and left_arrow is moved right to produce

82 Cont d.. Since right_arrow < left_arrow is TRUE, the first subdivision is complete. At this stage, numbers smaller than pivot are on the left side and numbers larger than pivot are on the right side. This produces two sublists that can be envisioned as Each sublist can now be sorted by the same function. This would require a recursive call to the sorting function. In each case, the vector is passed as a parameter together with the right and left indices for the appropriate sublist.

83 Algorithm Pivot A[low] i low j high While(i<=j) do { while(a[i] <= pivot) do i= i+1 while ( A[J] >= pivot) do j= j-1 } Swap(A[low],A[j]) //when i crosses j swap A[low] and A[J]

84 Quick-Sort: Best Case Even Partition n cn n/2 n/2 2 cn/2 = cn n/4 n/4 n/4 n/4 4 c/4 = cn log n levels n/3 3c = cn Total time: (nlogn)

85 Analysis We can obtain the time complexity of quick sort. C(n) = C(n/2) + C(n/2) + n Time required to sort left sub array Time required to sort right sub array Time taken for partitioning the sub array C(n)= 2 C(n/2) + n with T(1)= 0 As per master theorem. a=2, b=2 and d=1 So, T(n)= (n d log n) Hence, the time complexity of the quick sort is (n log n)

86 Analysis with substitution method C(n)= 2 C(n/2) + n put n=2 k C(2 k ) = 2C(2 k -1 ) + 2 k.(1) Now, k= k-1 C(2 k-1 ) = 2C(2 k -2 ) + 2 k -1 Put the result of C(2 k-1 ) in equation (1) C(2 k ) = 2[2C(2 k -2 ) + 2 k -1 ] + 2 k = 2 2 C(2 k -2 ) + 2 * 2 k k = 2 2 C(2 k -2 ) + 2 k + 2 k = 2 2 C(2 k -2 ) + 2* 2 k ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

87 Cont d.. Hence above equation becomes, with C(1)= 0 C(2 k ) = 2 k C(2 k -k ) + K * 2 k = 2 k C(2 0 ) + K * 2 k = 2 k C(1) + K * 2 k = 2 k * 0 + K * 2 k As we have assumed n= 2 k Apply logarithm (base 2 ) on both side log2n = log22 k = k log22 = k (1)

88 Cont d.. K= log2n, n= 2 k C(2 k ) = 2 k * 0 + K * 2 k C(n)= n * 0 + log2n * n = n log2n thus, it is provide the best case time complexity of quick sort Θ(n log2n)

89 Quick-Sort: Worst Case Unbalanced Partition n cn n-1 c(n-1) n-2 c(n-2) Happens only if input is sortd input is reversely sorted 3 2 3c 2c Total time: (n 2 )

90 Cont d.. C(n)= C(n-1)+ n C(n)= n + (n-1) + (n-2) As we know, n= n(n+1)/2 = n(n+1)/2 = n 2 The time complexity of worst case of the quick sort is (n 2 )

91 Time complexity (quick sort) Best case Average case Worst case Θ(n log2n) Θ(n log2n) Θ(n 2 )

92 Binary Search Binary search is an efficient searching method. While searching the elements using this method the most essential thing is that the elements in the array should be sorted one. Let A[m] be the mid element of array A. Then there are three condition to be tested.(key =searched element). If KEY=A[m] then desired element in the present list. Otherwise if KEY<A[m] then search the left sub list. Otherwise if KEY>A[m] then search the right sub list

93 Example The key element is 60. Apply the formula m = (low + high)/2 so A[m] = KEY check the condition = (0+6)/2 = 3 so, A[3] = KEY. A[3] = 40

94 Cont d Now, the right sub list is Now we will again divide the list and check the mid element. m = (low + high)/2 = (4+6)/2 = 5 A[5] = KEY. A[5] = 60

95 Algorithm Low 0 High n-1 while(low<high) do { m (low + high)/2 // mid of the array is obtained if(key=a[m]) then return m else if (KEY<A[m]) then high m-1 // search left sub list else low m+1 // right sub list } return -1 // if not present in the list

96 Analysis(worst case) C(n) = C(n/2) + 1 Time required to compare left sub list or right sub list One comparison made with middle element So Recurrence relation C(n)= C(n/2) +1 Now put n= 2 k C(2 k ) = C(2 k /2) + 1 = C(2 k -1 ) + 1 Solve this equation with backward substitution

97 Cont d.. C(2 k ) = C(2 k -1 ) + 1.(1) Now, k= k-1 C(2 k-1 ) = C(2 k -2 ) + 1 Put the result of C(2 k-1 ) in equation (1) C(2 k ) = [C(2 k -2 ) + 1 ] (2) Now, k= k-2 C(2 k-2 ) = C(2 k -3 ) + 1 Put the result of C(2 k-2 ) in equation (2) C(2 k ) = [C(2 k -3 ) +1] ;;;;;;;;;;;;;;;;;;;;;;; C(2 k ) = C(2 k -k ) + k

98 Cont d C(2 k ) = C(2 k -k ) + k = C(2 0 ) + k = C(1) + k C(1)= 1 = 1 + k As we have assumed n= 2 k Apply logarithm (base 2 ) on both side log2n = log22 k = k log22 = k (1)

99 Cont d K= log2n Now, C(n) = 1 + k = 1 + log2n The worst case time complexity of binary search is (log2n)

100 Time complexity Best case Average case Worst case Θ(1) Θ(log2N) Θ(log2N)

101 pros and cons Binary search Advantage :- Binary search is an optimal searching algorithm using which we can search the desired element very efficiently. Disadvantage:- This algorithm requires the list to be sorted. Then only this method is applicable. Application:- used to search desired record from database and for solving nonlinear equation with one unknown this method is used

102 Sequential vs. Binary search technique Sequential Technique This is the simple technique of searching an element This technique does not require the list to be sorted Every element of the list may get compared with the key element The worst case time complexity of this technique is O(n) Binary search technique This is the efficient technique of searching an element This technique requires the list to be sorted. Then only this method is applicable Only the mid element of the list is compared with key element The worst case time complexity of this technique is O(log n)

103 Matrix multiplication Suppose, we want to multiply two matrices A and B each size of N. There are total 8 multiplication and 4 addition.

104 Matrix multiplication Brute-force algorithm c 00 c 01 a 00 a 01 b 00 b 01 = * c 10 c 11 a 10 a 11 b 10 b 11 = a 00 * b 00 + a 01 * b 10 a 00 * b 01 + a 01 * b 11 a 10 * b 00 + a 11 * b 10 a 10 * b 01 + a 11 * b 11 8 multiplications 4 additions

105 Strassen matrix multiplication Strassen showed that 2 * 2 matrix multiplication can be accomplished in 7 multiplication and 18 addition or subtraction. Divide:- Divide matrices into sub matrices Conquer:- Use group of matrix multiplication equation Combine:- Recursively multiply sub matrices and get the final result of multiplication after performing required addition and subtraction.

106 Cont d.. P 1 = (A 11 + A 22 )(B 11 +B 22 ) P 2 = (A 21 + A 22 ) * B 11 P 3 = A 11 * (B 12 - B 22 ) P 4 = A 22 * (B 21 - B 11 ) P 5 = (A 11 + A 12 ) * B 22 P 6 = (A 21 - A 11 ) * (B 11 + B 12 ) P 7 = (A 12 - A 22 ) * (B 21 + B 22 ) C 11 = P 1 + P 4 - P 5 + P 7 C 12 = P 3 + P 5 C 21 = P 2 + P 4 C 22 = P 1 + P 3 - P 2 + P 6

107 Cont d.. C 11 = P 1 + P 4 - P 5 + P 7 = (A 11 + A 22 )(B 11 +B 22 ) + A 22 * (B 21 - B 11 ) - (A 11 + A 12 ) * B 22 + (A 12 - A 22 ) * (B 21 + B 22 ) = A 11 B 11 + A 11 B 22 + A 22 B 11 + A 22 B 22 + A 22 B 21 A 22 B 11 - A 11 B 22 -A 12 B 22 + A 12 B 21 + A 12 B 22 A 22 B 21 A 22 B 22 = A 11 B 11 + A 12 B 21

108 Example A= B= Implement Strassen's matrix multiplication on A & B

109 Cont d A11 A12 B11 B12 A21 A22 B21 B

110 Cont d.. P 1 = (A 11 + A 22 )(B 11 +B 22 ) P 2 = (A 21 + A 22 ) * B

111 Cont d.. P 3 = A 11 * (B 12 - B 22 ) P 4 = A 22 * (B 21 - B 11 )

112 Cont d.. P 5 = (A 11 + A 12 ) * B P 6 = (A 21 - A 11 ) * (B 11 + B 12 )

113 Cont d.. P 7 = (A 12 - A 22 ) * (B 21 + B 22 )

114 Cont d.. C 11 = P 1 + P 4 - P 5 + P C 12 = P 2 + P

115 Cont d.. C 21 = P 2 + P C 22 = P 1 + P 3 - P 2 + P

116 Cont d.. Thus the final product matrix C will be