O(n): printing a list of n items to the screen, looking at each item once.

Transcription

1 UNIT IV Sorting: O notation efficiency of sorting bubble sort quick sort selection sort heap sort insertion sort shell sort merge sort radix sort. O NOTATION BIG OH (O) NOTATION Big oh : the function f(n)=o(g(n)) iff there exist positive constants c and no such that f(n) c*g(n) for all n, n no.the Big oh (O) notation is used to give the upper bound for a function t(n) with a constant factor. t(n) is bounded above by a constant multiple of g(n).the upper bound on t(n) indicates that the function t(n) will be the worst-case that it does not consume more than this computing time. Example: Let's take an example of Big-O. Say that f(n) = 2n + 8, and g(n) =. Find a constant, so that 2n + 8 <=. The number 4 works here, giving us 16 <= 16. For any number n greater than 4, this will still work. Since we're trying to generalize this for large values of n, and small values (1, 2, 3). we can say that f(n) is generally faster than g(n); that is, f(n) is bound by g(n), and will always be less than it. To find the upper bound - the Big-O time - assuming we know that f(n) is equal to (exactly) 2n + 8, we can take a few shortcuts. For example, we can remove all constants from the runtime; eventually, at some value of c, they become irrelevant. This makes f(n) = 2n. Also, for convenience of comparison, we remove constant multipliers; in this case, the 2. This makes f(n) = n. It could also be said that f(n) runs in O(n) time; that lets us put a tighter (closer) upper bound onto the estimate. O(n): printing a list of n items to the screen, looking at each item once. Prepared by Mrs.D.Maladhy (AP/IT/RGCET) Page 1 1

2 O(ln n): also "log n", taking a list of items, cutting it in half repeatedly until there's only one item left. O( ): taking a list of n items, and comparing every item to every other item. OMEGA (Ω) NOTATION Omega: the function f(n)=ω(g(n)) iff there exist positive constants c and no such that f(n) c*g(n) for all n, n no. The Omega (Ω) notation is used to give the lower bound for a function t(n) within a constant factor. t(n) is bounded below by a constant multiple of g(n).the lower bound on t(n) indicates that the function t(n) will be the best-case that it does not consume more than this computing time. Theta (Θ) Notation Theta: the function f(n)=ө(g(n)) iff there exist positive constants c1,c2 and no such that c1 g(n) f(n) c2 g(n) for all n, n no.the Theta (Θ) notation is used to give the lower and upper bound for a function t(n) within a constant factor. t(n) is bounded above and below by constant multiples of g(n). The lower bound on t(n) indicates that the function t(n) will be the worst-case that it does not consume more than this computing time. Some polynomial running times: Prepared by Mrs.D.Maladhy (AP/IT/RGCET) Page 2 2

3 O(1) An algorithm with this running time is said to have "constant" running time. Basically, this means the algorithm always take about the same amount of time, regardless of the size of the input. To state it technically, if an algorithm will never perform more than a certain number of steps, no matter how large the input gets, then that algorithm is considered to have a constant running time. For example, an algorithm which consists of performing exactly 7 multiplications has a constant running time. Although constant time is the best running time an algorithm can have, that algorithm could still be considered bad if the total amount of time to run the algorithm were too large. Some examples of O(1) algorithms include: inserting an element onto the front of a linked list, popping from or pushing onto a stack, and retrieving the nth element of an array. O(n) An algorithm which runs in O(n) is said to have a "linear" running time. This basically means that the amount of time to run the algorithm is proportional to the size of the input. To be technical, an algorithm which never performs more than certain number of steps for each element in the input has a linear running time. For example, an algorithm which sums the total of a list of numbers has a linear running time, because the number of additions required is the same as the number of elements. Some examples of O(n) algorithms include searching through an unordered list, incrementing every element of an array, and calculating fibonacci numbers using dynamic programming. O(n 2 ) An algorithm with this running time is said to have "quadratic" running time. This means that whenever you increase the size of the input by a factor of n, the running time increases by a factor of n 2. For example, if you double the size of the input of a quadratic algorithm, then the running time will quadruple. Some sorting algorithms, such as insertion sort and bubble sort, have quadratic running times. O(lgn) An algorithm with O(lgn) running time is said to have "logarithmic" running time. This means that as the size of the input increases by a factor of n, the running time increases by a factor of the logarithm of n. For example, if you increase the input size of a O(lgn) algorithm by a factor of 1024, the running time will increase by a factor of 10. This running time is better than O(n), but not as good as O(1). As the input size gets large, however, the behavior becomes comparable to O(1) in many circumstances. Algorithms which search through ordered lists or binary trees, as well as operations on heaps generally have logarithmic running times. O(nlgn) An algorithm which has this order, will in increase in running time proportionate to the size of the input times the logarithm of the size of the input. Technically speaking, an algorithm which when given an input of size n never performs more than Prepared by Mrs.D.Maladhy (AP/IT/RGCET) Page 3 3

4 cnlgn steps has a running time of O(nlgn). This running time is better than O(n 2 ) but not quite as good as O(n). The fastest sorting algorithms, including mergesort and quicksort, have O(nlgn) running times. SORTING The two main classifications of sorting based on the source of data are a. Internal sorting b. External sorting c. Stable Sorting External sorting External sorting is a process of sorting in which large blocks of data stored in storage devices are moved to the main memory and then sorted.i.e A sort can be External if the records that it is sorting are in Auxiliary storage. Internal sorting Internal sorting is a process of sorting the data in the main memory. i.e A sort can be Internal if the records that it is sorting are in main memory. Stable sort A sorting technique is called stable if for all records i and j such that k[i] equals k[j], if r[i] precedes r[j] in the original file, r[i] precedes r[j] in the sorted file. That is, a stable sort keeps records with the same key in the same relative order that they were in before the sort. Various factors to be considered in deciding a sorting algorithm a. Programming time b. Execution time of the program c. Memory needed for program environment Bubble Sort: In this sorting method, to arrange elements in ascending order, we begin with the 0 th element an compare it with the 1 st element. If it is found to be greater than the 1 st element, then they can be interchanged. Prepared by Mrs.D.Maladhy (AP/IT/RGCET) Page 4 4

5 In this way all the elements are compared (excluding last) with their next element and are interchanged if required. On completing the first iteration, the largest element gets placed at the last position. After all the iterations, the list becomes a sorted list Prepared by Mrs.D.Maladhy (AP/IT/RGCET) Page 5 5

6 Time and space complexity of bubble sorting: The best case involves performing one pass which requires n-1 comparisons. Consequently, the best case is O(n). The worst case performance of the bubble sort is comparisons and exchanges. The average case is more difficult to analyze than the other cases. It can be shown that the average case analysis is O(n 2 ). the average no. of passes is approximately. for n=10 the average number of passes is 6. The average no. of comparisons and exchanges are both O(n 2 ). Selection Sort Selection sort is the easiest method of sorting. To sort the data in ascending order, the 0 th element is compared with all other elements. If the 0 th element then they are interchanged. In this way after the first iteration, the smallest element is placed at 0 th position. The procedure is repeated for 1 st element. Eg. Array before sorting: Prepared by Mrs.D.Maladhy (AP/IT/RGCET) Page 6 6

7 Array before sorting: exchange No exchange exchange exchange First iteration No exchange exchange Second iteration No exchange Prepared by Mrs.D.Maladhy (AP/IT/RGCET) Page 7 7

8 Third iteration Fourth iteration Sorted Array: Time and space complexity of selection sorting: The algorithm, the search for the record with the next smallest key is called a pass. There are n-1 such passes required in order to perform the sort. This is because each pass places one record into its proper location. During the first pass, in which the record with the smallest key is found, n-1 records are compared. In general, for the i th pass of the sort, n-i comparisons are required. The total number of comparisons is therefore, the sum Therefore, the no. of comparisons is proportional to n 2. i.e., O(n 2 ). INSERTION SORT Insertion sort is performed by inserting a particular element at the appropriate position. In insertion sort, the first iteration starts with comparison of 1 st element with the 0 th element. In the second iteration 2 nd element is compared with the 0 th and 1 st. In general in every iteration an element is compared with all elements. If at some point it is found that the element can be inserted at a position then space is created for it by shifting the other elements one position to the right and inserting the element at the suitable position. Prepared by Mrs.D.Maladhy (AP/IT/RGCET) Page 8 8

9 This procedure is repeated for all the elements in the array. Eg: unsorted array Third 11 iteration 1 No exchange Sorted Array: Time and space complexity of selection sorting: In the worst case algorithm makes (i+1) comparisons before making the insertion. Hence the computing time for the insertion is O(i) overall worst case time is O(n 2 ). The average case time is also O(n 2 ). SHELL SORT (DIMINSHING INCREMENT SORT): The shell sort, named after its developer Dona/d. L. shell in 1959 is an extension of the Insertion sort,which has the limitation, that it compares only the consecutive elements and The interchanges the elements by only one space. The smaller elements that are far away require Many passes through the sort, to properly insert them in its correct position. The shell sort overcomes this limitation, gains speed than insertion sort, by comparing elements that are at a specific distance from each other, and interchanges them if necessary.the shell sort divides the list into smaller sub lists, and then sorts the sub lists seperately using the insertion sort. This is done by considering the input list being n-sorted. This method splits the input list into h-independent sorted files. The procedure of h-sort is insertion sort considering only the h th element (starting any where).the value of h will be initially high and is repeatedly Decremented until it reaches 1.When h is equal to 1, a regular insertion sort is performed on the list, but Prepared by Mrs.D.Maladhy (AP/IT/RGCET) Page 9 9

10 by then the 1ist of data is guaranteed to be almost sorted. Using the above Procedure for any sequence values of h, always ending in 1 will produce a sorted list. ALGORITHM: Void shell sort (int a [ ], int n) d=n/2; for (i=0;i<n;i++) for (j=0;j<n-d;j++) if (a[ j ] > a[ j+d] ) temp= a[ j ]; a[ j ] = a[ j+d]; a[ j+d ] = temp; } } d=(d+1)/2; }} Shell sort is the method of choice for many sorting applications because it has acceptable running time for moderately large arrays (containing more than 5000 elements) and requires only a very small amount of code for its operation. The shell sort is also referred as diminishing increment sort, because the number of elements compared in a group continuously decreases in each pass. Prepared by Mrs.D.Maladhy (AP/IT/RGCET) Page 10 10

11 MERGE SORT Merging is the process of combining two sorted lists into merge sorted list. To perform the merge sort both the sorted lists are compared. The smaller of both the elements from both the lists are placed in the third list. Given an unsorted array, the array is divided into two array say x and y and can be sorted with any of the sorting algorithm. The 0 th element from the first array is compared with 0 th of the second array y. If it is smaller than it is moved to the third array. Now the 0 th element from the first array is compared with 1st element from the second array. Then the 1 st element from the first array is compared with the 1 st element from the second array. Now the1 st element from the first array in compared with the 2 nd element in the second array and place it in the third array. The same procedure is repeated till the end of one of the arrays is reached. Now the remaining elements from the other array are placed directly into the third list as are already in sorted order Prepared by Mrs.D.Maladhy (AP/IT/RGCET) Page 11 11

12 The merge algorithm also uses divide and conquer rule for its operation.the most popular method for sorting on external storage device is Merge sort. This method consists of two distinct phase. i) Fist segment of input list are sorted using a good internal sorting method. There sorted segment known as run s are written on to external storage as they are generated. ii) Second. the run generated in phase one are merged together in a merge_ tree Pattern. Until only one run is left , 26 1, 77 11, 61 15, 59 19, 48 1, 5, 26, 77 11, 15, 59, 61 19,48 1, 5, 11, 15, 26, 59, 61, 77 19, 48 1, 5, 11, 15, 19, 26, 48, 59, 61, 77 Because the simple merge functions merges only the loading records of the two runs being merged to be present in memory at one time. It is possible to merge large run s together. Time and space complexity of merge sort: The timing performance of this algorithm is O(n) where n denotes the num of the sizes of the two sub tables to be merged. RADIX SORT Radix sort is a small method that many people intuitively use when alphabetizing a large list of names. Specifically, the list of names is first sorted according to the first letter of each names, that is, the names are arranged in 26 classes Prepared by Mrs.D.Maladhy (AP/IT/RGCET) Page 12 12

13 Intuitively, one might want to sort numbers on the most significant digit. But Radix sort do counter-intuitively by sorting on the least significant digits first. On the first pass entire numbers sort on the least significant digit and combine in a array. Then on the second pass, the entire numbers are sorted again on the second least-significant digits and combine in a array and so on. Following example shows how Radix sort operates on seven 3-digits number. INPUT Buckets 1 st Pass 2 nd Pass 3 rd Pass Prepared by Mrs.D.Maladhy (AP/IT/RGCET) Page In the above example, the first column is the input. The remaining shows the list after successive sorts on increasingly significant digits position. The code for Radix sort assumes that each element in the n-element array A has d digits, where digit 1 is the lowest-order digit and d is the highestorder digit. Time and space complexity of radix sort: The average case for radix sort is O(m+n), the worst case is also O(m+n).

14 QUICK SORT The divide-and-conquer approach can be used to arrive at an efficient sorting method different from merge sort. In merge sort, the file a[1:n] was divided at its midpoint into sub arrays which were independently sorted & later merged. In Quick sort, the division into 2 sub arrays is made so that the sorted sub arrays do not need to be merged later. This is accomplished by rearranging the elements in a[1:n] such that a[i]<=a[j] for all i between 1 & n and all j between (m+1) & n for some m, 1<=m<=n. Thus the elements in a[1:m] & a[m+1:n] can be independently sorted. No merge is needed. This rearranging is referred to as partitioning. Function partition( ) of algorithm accomplishes an in-place partitioning of the elements of the elements a[m:p-1]. It is assumed that a[p] a[m] and that a[m] is the partitioning element. If m=1 & p-1=n, then a[n+1] must be defined and must be greater than or equal to all elements in a[1:n] The assumption that a[m] is the partition element is merely for convenience, other choices for the partitioning element than the first item in the set are better in practice. As its name implies, quicksort is the fastest known sorting algorithm in practice. Its average running time is O(n log n). It is very fast, mainly due to a very tight and highly optimized inner loop. It has O(n 2 ) worst-case performance, but this can be made exponentially unlikely with a little effort. Divide: Spilt the array into two sub arrays that each element in the left sub array is less than or equal to the middle element and each element in the right sub array is greater than the middle element. The splitting of the array into two sub arrays is based on the pivot element all the elements that are more than pivot should be in the right sub array. Conquer: Recursively sort the two sub arrays Prepared by Mrs.D.Maladhy (AP/IT/RGCET) Page 14 14

15 Combine: Combine the sorted elements in a group to form a list of sorted elements. STEPS: 1. If the number of elements in S is 0 or 1, then return. 2. Compare the first two elements. Choose the largest as the pivot. In case the first two elements are the same, compare the second and the third and choose the largest. Continue this till the last element is reached. In case all the elements have the same value, a pivot cannot be chosen. Return. 3. Swap A[l] and A[r]. The function interchange (a, i, j) does this. 4. Advance the l pointer till an element greater than the pivot element is found. 5. Decrement r pointer until an element less than or equal to the pivot is found. 6. Check whether l<r. If so, swap a[l] and a[r]. 7. If l>r, partition the list into 2 parts. (upto A[r] and after A[r]). 8. Follow the above mentioned steps for each partition. Time and space complexity of quick sorting: The worst case behaviour of this algorithm is O(n 2 ). The time required to position a record in a file of size n is O(n). if T(n) is the time taken to sort a file of n records, then when the file splits roughly into two equal parts each time a record is positioned correctly we have T(n) cn + 2T(n/2),for some constant c cn + 2(cn/2+2T(n/4)) 2cn + 4T(n/4).. cn log2 n +nt(1) = O(n log2 n) The better computing time for quick sort is O(n log2 n) HEAP SORT The binary heap data structure is an array that can be viewed as a complete binary tree. Prepared by Mrs.D.Maladhy (AP/IT/RGCET) Page 15 15

16 Each node of the binary tree corresponds to an element of the array. The array is completely filled on all levels except possibly lowest. Heaps are represented in level order, going from left to right. The array corresponding to the heap above is [25, 13, 17, 5, 8, 3]. The root of the tree A[1] and given index i of a node, the indices of its parent, left child and right child can be computed PARENT (i) return floor(i/2) LEFT (i) return 2i RIGHT (i) return 2i + 1 is represented by the array [20, 14, 17, 8, 6, 9, 4, 1]. Go from the 20 to the 6 first. The index of the 20 is 1. To find the index of the left child, calculate 1 * 2 = 2. This takes us to 14. Now, to go right, calculate 2 * = 5. This takes us to the 6. Try going from 4 to 20. 4's index is 7. To go to the parent, calculate 7 / 2 = 3, which takes us to the 17. Prepared by Mrs.D.Maladhy (AP/IT/RGCET) Page 16 16

17 Now, to get 17's parent, calculate 3 / 2 = 1, which takes us to the 20. A heap is a binary tree that is completely filled, with the Structure Property possible exception of the bottom level, which is filled from left to right. Such a tree is known as a complete binary tree. Figure shows an example. This property allows operations to be performed Heap Order Property quickly is the heap order property. Min-heap: If the minimum element has to be found quickly, the smallest element should be at the root. If the subtrees should also be min-heaps, then all the sub root nodes should be smaller than all of their descendants. Max-heap : If the maximum has to be found quickly, the largest element should be at the root. If the subtrees should also be max-heaps, then all the sub root nodes should be larger than all of their descendants. A heap of height h has the minimum number of elements when it has just one node at the lowest level. The levels above the lowest level form a complete binary tree of height h -1 and 2 h -1 nodes. Hence the minimum number of nodes possible in a heap of height h is 2 h. Clearly a heap of height h, has the maximum number of elements when its lowest level is completely filled. In this case the heap is a complete binary tree of height h and hence has 2 h+1-1 nodes. Following is not a heap, because the heap order property holds but the structure property does not hold. Algorithm void heapsort ( ) for (int i = n/2; i>=1; i- -) pushdown (i,n); Prepared by Mrs.D.Maladhy (AP/IT/RGCET) Page 17 17

18 for (int i = n; i>=2; i - -) swap (A[1], A[i]); pushdown(1, i-1); }} void pushdown (int first, int last) int t, i; int r = first; while (r <= last/2) if (last = = 2r) if ( A[r] < A[2*r] ) swap ( A[r], A[2 * r]); r = last; }} else if (A[r] < A[2*r] && A[2*r] >= A[2*r + 1] ) swap ( A[r], A[2 * r ]); r = 2 * r; } else if (A[r] < A[2*r+1] && A[2*r+1] >= A[2*r] ) swap ( A[r], A[2 * r + 1]); r = 2 * r+1; } else r = last; } Time and space complexity of heap sort: The worst case analysis is easier than average case. The depth of a complete binary tree of n nodes is [ ]. We must first create the heap and then sort the heap. The worst case at each step involves performing a no. of comparisons which is given by the depth of the tree. No. of comparisons is O( ). Average case shows as also O( ) Prepared by Mrs.D.Maladhy (AP/IT/RGCET) Page 18 18