Walls & Mirrors Chapter 9. Algorithm Efficiency and Sorting

Transcription

1 Walls & Mirrors Chapter 9 Algorithm Efficiency and Sorting

2 The Execution Time of Algorithms Counting an algorithm s operations int sum = item[0]; int j = 1; while (j < n) { sum += item[j]; ++j; } <- 1 assignment <- 1 assignment <- n comparisons <- n-1 plus/assignments <- n-1 plus/assignments Total: 5n-2 operations

3 The Execution Time of Algorithms Count the number of basic operations Read (get), write (put), load, store, compare, assignment, jump, arithmetic operations (increment, decrement, add, subtract, multiply, divide), shift, open, close, logical operations (not/complement, AND, OR, XOR),

4 Algorithm Growth Rates Measure an algorithm s time requirement as a function of the problem size Number of elements in an array Algorithm A requires n 2 /5 time units Algorithm B requires 5n time units Algorithm efficiency is a concern for large problems only

5 Common Growth-Rate Functions - I

6 Common Growth-Rate Functions - II

7 Big O Notation Algorithm A is order f(n)-denoted O(f(n))-if constants k and n 0 exist such that A requires <= k*f(n) time units to solve a problem of size n>=n 0 n 2 /5 O(n 2 ): k=1/5, n 0 =0 5*n O(n): k=5, n 0 =0

8 More Examples How about n 2-3n+10? O(n 2 ) if there exist k and n 0 such that kn 2 n 2-3n+10 for all n n 0 3n 2 n 2-3n+10 for all n 2; so k=3, n 0 =2

9 Properties of big-oh Ignore low-order terms O(n 3 +4n 2 +3n)=O(n 3 ) Ignore multiplicative constant O(5n 3 )=O(n 3 ) Combine growth-rate functions O(f(n)) + O(g(n)) = O(f(n)+g(n))

10 Worst-case vs. Average-case Analyses An algorithm can require different times to solve different problems of the same size. Worst-case analysis (find the maximum number of operations an algorithm can execute in all situations) is easier to calculate and is more common Average-case (enumerate all possible situations, find the time of each of the m possible cases, total and dive by m) is harder to compute but yields a more realistic expected behavior

11 Selection Sort values [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] Divides the array into two parts: already sorted, and not yet sorted. On each pass, finds the smallest of the unsorted elements, and swap it into its correct place, thereby increasing the number of sorted elements by one.

12 Selection Sort: Pass One values [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] U N S O R T E D

13 Selection Sort: End Pass One values [ 0 ] 6 SORTED [ 1 ] [ 2 ] [ 3 ] [ 4 ] U N S O R T E D

14 Selection Sort: Pass Two values [ 0 ] 6 SORTED [ 1 ] [ 2 ] [ 3 ] [ 4 ] U N S O R T E D

15 Selection Sort: End Pass Two values [ 0 ] [ 1 ] 6 10 SORTED [ 2 ] [ 3 ] [ 4 ] U N S O R T E D

16 Selection Sort: Pass Three values [ 0 ] [ 1 ] 6 10 SORTED [ 2 ] [ 3 ] [ 4 ] U N S O R T E D

17 Selection Sort: End Pass Three values [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] UNSORTED S O R T E D

18 Selection Sort: Pass Four values [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] UNSORTED S O R T E D

19 Selection Sort: End Pass Four values [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] S O R T E D

20 Selection Sort: How many comparisons? values [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] compares for values[0] 3 compares for values[1] 2 compares for values[2] 1 compare for values[3] =

21 For selection sort in general The number of comparisons when the array contains N elements is Sum = (N-1)+(N-2)+ +2+1

22 Notice that... Sum = (N-1) + (N-2) Sum = (N-2) + (N-1) 2* Sum = N + N N + N 2 * Sum = N * (N-1) Sum = N * (N-1)/2=.5 N N How many number of swaps? O(N) Therefore, the complexity is O(N 2 )

23 Code for Selection Sort void SelectionSort (int values[], int numvalues) // Post: Sorts array values[0.. numvalues-1 ] // into ascending order by key { int endindex = numvalues - 1 ; } for (int current = 0; current < endindex; current++) Swap (values[current], values[minindex(values,current, endindex)]);

24 Selection Sort code (contd) int MinIndex(int values[ ], int start, int end) // Post: Function value = index of the smallest value // in values [start].. values [end]. { int indexofmin = start ; } for(int index = start + 1 ; index <= end ; index++) if (values[ index] < values [indexofmin]) indexofmin = index ; return indexofmin;

25 Bubble Sort values [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] Compares neighboring pairs of array elements, starting with the last array element, and swaps neighbors whenever they are not in correct order. On each pass, this causes the smallest element to bubble up to its correct place in the array.

26 Bubble Sort: Pass One

27 Bubble Sort: Pass Two

28 Snapshot of BubbleSort

29 Code for Bubble Sort void BubbleSort(int values[], int numvalues) { int current = 0; while (current < numvalues - 1) { BubbleUp(values, current, numvalues-1); current++; } }

30 Bubble Sort code (contd.) void BubbleUp(int values[], int startindex, int endindex) // Post: Adjacent pairs that are out of // order have been switched between // values[startindex]..values[endindex] // beginning at values[endindex]. { } for (int index = endindex; index > startindex; index--) if (values[index] < values[index-1]) Swap(values[index], values[index-1]);

31 Observations on Bubble Sort There can be a large number of intermediate swaps. Can this algorithm be improved? What are the best/worst cases? This algorithm is O(N 2 ) in the worst-case

32 Insertion Sort values [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] One by one, each as yet unsorted array element is inserted into its proper place with respect to the already sorted elements. On each pass, this causes the number of already sorted elements to increase by one.

33 Insertion Sort Works like someone who inserts one more card at a time into a hand of cards that are already sorted. To insert 12, we need to make room for it by moving first 36 and then 24.

34 Insertion Sort Works like someone who inserts one more card at a time into a hand of cards that are already sorted. To insert 12, we need to make room for it by moving first 36 and then 24.

35 Insertion Sort Works like someone who inserts one more card at a time into a hand of cards that are already sorted. To insert 12, we need to make room for it by moving first 36 and then 24.

36 Insertion Sort Works like someone who inserts one more card at a time into a hand of cards that are already sorted. To insert 12, we need to make room for it by moving first 36 and then 24.

37 A Snapshot of the Insertion Sort Algorithm

38 Insertion Sort

39 Code for Insertion Sort void InsertionSort(int values[ ],int numvalues) // Post: Sorts array values[0.. numvalues-1] into // ascending order by key { for (int cursort = 0 ; cursort < numvalues; cursort++) InsertItem ( values, 0, cursort ) ; }

40 Insertion Sort code (contd.) void InsertItem (int values[ ], int start, int end ) // Post: Elements between values[start] and values [end] // have been sorted into ascending order by key. { bool finished = false ; int current = end ; bool moretosearch = (current!= start); } while (moretosearch &&!finished ) { if (values[current] < values[current - 1]) { Swap(values[current], values[current - 1); current--; moretosearch = ( current!= start ); } else finished = true ; }

41 Radix Sort

42 Radix Sort Pass One

43 Radix Sort End Pass One

44 Radix Sort Pass Two

45 Radix Sort End Pass Two

46 Radix Sort Pass Three

47 Radix Sort End Pass Three

48 Radix Sort

49 Radix Sort Cost: Each pass: n moves to form groups Each pass: n moves to combine them into one group Number of passes: d 2*d*n moves, 0 comparison However, demands substantial memory not a comparison sort

50 Sorting Algorithms and Average Case Number of Comparisons Simple Sorts Straight Selection Sort Bubble Sort Insertion Sort More Complex Sorts Quick Sort Merge Sort Heap Sort O(N 2 ) O(N*log N)