Simple Sorting Algorithms

Transcription

1 Simple Sorting Algorithms

2 Outline We are going to look at three simple sorting techniques: Bubble Sort, Selection Sort, and Insertion Sort We are going to develop the notion of a loop invariant We will write code for Bubble Sort and Selection Sort, then derive their loop invariants We will start with a loop invariant for Insertion Sort, and derive the code for it We will analyze the running time for each of the above 2

3 Bubble Sort Compare each element (except the last one) with its neighbor to the right Compare each element (except the last two) with its neighbor to the right If they are out of order, swap them This puts the largest element at the very end The last element is now in the correct and final place If they are out of order, swap them This puts the second largest element next to last The last two elements are now in their correct and final places Compare each element (except the last three) with its neighbor to the right Continue as above until you have no unsorted elements on the left 3

4 Example of Bubble Sort (done)

5 Code for Bubble Sort public static void bubblesort(int[] a) { int outer, inner; for (outer = a.length - 1; outer > 0; outer--) { // counting down for (inner = 0; inner < outer; inner++) { // bubbling up if (a[inner] > a[inner + 1]) { // if out of order... int temp = a[inner]; //...then swap a[inner] = a[inner + 1]; a[inner + 1] = temp; } } } } 5

6 Analysis of Bubble Sort for (outer = a.length - 1; outer > 0; outer--) { for (inner = 0; inner < outer; inner++) { if (a[inner] > a[inner + 1]) { // code for swap omitted } } } Let n = a.length = size of the array The outer loop is executed n-1 times (call it n, that s close enough) Each time the outer loop is executed, the inner loop is executed Inner loop executes n-1 times at first, linearly dropping to just once On average, inner loop executes about n/2 times for each execution of the outer loop In the inner loop, the comparison is always done (constant time), the swap might be done (also constant time) Result is n * n/2 * k, that is, O(n2/2 + k) = O(n2) 6

7 Loop invariants You run a loop in order to change things Oddly enough, what is usually most important in understanding a loop is finding an invariant: that is, a condition that doesn t change In Bubble Sort, we put the largest elements at the end, and once we put them there, we don t move them again The variable outer starts at the last index in the array and decreases to 0 Our invariant is: Every element to the right of outer is in the correct place That is, for all j > outer, if i < j, then a[i] <= a[j] When this is combined with the loop exit test, outer == 0, we know that all elements of the array are in the correct place 7

8 Another sort: Selection Sort Given an array of length n, Search elements 0 through n-1 and select the smallest Search elements 1 through n-1 and select the smallest Swap it with the element in location 2 Search elements 3 through n-1 and select the smallest Swap it with the element in location 1 Search elements 2 through n-1 and select the smallest Swap it with the element in location 0 Swap it with the element in location 3 Continue in this fashion until there s nothing left to search 8

9 Example and analysis of Selection Sort The Selection Sort might swap an array element with itself--this is harmless, and not worth checking for Analysis: The outer loop executes n-1 times The inner loop executes about n/2 times on average (from n to 2 times) Work done in the inner loop is constant (swap two array elements) Time required is roughly (n-1)*(n/2) You should recognize this as O(n2) 9

10 Code for Selection Sort public static void selectionsort(int[] a) { int outer, inner, min; for (outer = 0; outer < a.length - 1; outer++) { // outer counts down min = outer; for (inner = outer + 1; inner < a.length; inner++) { if (a[inner] < a[min]) { min = inner; } // Invariant: for all i, if outer <= i <= inner, then a[min] <= a[i] } // a[min] is least among a[outer]..a[a.length - 1] int temp = a[outer]; a[outer] = a[min]; a[min] = temp; // Invariant: for all i <= outer, if i < j then a[i] <= a[j] } } 10

11 Invariants for Selection Sort For the inner loop: This loop searches through the array, incrementing inner from its initial value of outer+1 up to a.length-1 As the loop proceeds, min is set to the index of the smallest number found so far Our invariant is: for all i such that outer <= i <= inner, a[min] <= a[i] For the outer (enclosing) loop: The loop counts up from outer = 0 Each time through the loop, the minimum remaining value is put in a[outer] Our invariant is: for all i <= outer, if i < j then a[i] <= a[j] 11

12 Summary so far We ve looked at code for Bubble Sort and Selection Sort We ve figured out their loop invariants We ve figured out their running time Next, we are going to start with a loop invariant that ought to result in a sort method We will derive the sort method We will figure out its running time 12

13 Insertion sort The outer loop of insertion sort is: for (outer = 1; outer < a.length; outer++) {...} The invariant is that all the elements to the left of outer are sorted with respect to one another For all i < outer, j < outer, if i < j then a[i] <= a[j] This does not mean they are all in their final correct place; the remaining array elements may need to be inserted When we increase outer, a[outer-1] becomes to its left; we must keep the invariant true by inserting a[outer-1] into its proper place This means: Finding the element s proper place Making room for the inserted element (by shifting over other elements) Inserting the element 13

14 One step of insertion sort sorted next to be inserted temp 10 less than sorted 14

15 Code for Insertion Sort public static void insertionsort(int[] array) { int inner, outer; for (outer = 1; outer < array.length; outer++) { int temp = array[outer]; inner = outer; while (inner > 0 && array[inner - 1] >= temp) { array[inner] = array[inner - 1]; inner--; } array[inner] = temp; // Invariant: For all i < outer, j < outer, if i < j then a[i] <= a[j] } } 15

16 Analysis of insertion sort We run once through the outer loop, inserting each of n elements; this is a factor of n On average, there are n/2 elements already sorted The inner loop looks at (and moves) half of these This gives a second factor of n/4 Hence, the time required for an insertion sort of an array of n elements is proportional to n2/4 Discarding constants, we find that insertion sort is O(n2) 16

17 Summary Bubble Sort, Selection Sort, and Insertion Sort are all O(n2) As we will see later, we can do much better than this with somewhat more complicated sorting algorithms Within O(n2), Bubble Sort is very slow, and should probably never be used for anything Selection Sort is intermediate in speed Insertion Sort is usually the fastest of the three--in fact, for small arrays (say, 10 or 15 elements), insertion sort is faster than more complicated sorting algorithms Selection Sort and Insertion Sort are good enough for small arrays Use of a Bubble Sort tends to elicit derision from your colleagues 17

18 The End 18

19 Recurrences Lecture 3

20 Sorting Iterative methods: Insertion sort Bubble sort Selection sort 2, 3, 4, 5, 6, 7, 8, 9, 10, J, Q, K, A Divide and conquer Merge sort Quicksort

21 Merge Sort uses the Divide & Conquer Approach of Algorithm Design Steps are Divide :- In case of mergesort Divide into two subarrays of size n/2 Conquer :- Sort recursively using mergesort Combine :- merge the subarrays.

22 Merge Sort: Idea Divide into two halves A: FirstPart SecondPart Recursively sort SecondPart FirstPart Merge A is sorted!

23 Merge Sort Algorithm MergeSort(A, left, right) { if (left < right) { mid = floor((left + right) / 2); MergeSort(A, left, mid); MergeSort(A, mid+1, right); Merge(A, left, mid, right); } } // Merge() takes two sorted subarrays of A and // merges them into a single sorted subarray of A // (how long should this take?)

24 Merge-Sort: Merge Sorted A: merge Sorted L: Sorted R:

25 Merge-Sort: Merge Example A: L: R:

26 Merge-Sort: Merge Example A: k=0 R: L: 3 1 i= j=

27 Merge-Sort: Merge Example A: k=1 R: L: i= j=

28 Merge-Sort: Merge Example A: k=2 R: L: i= j=

29 Merge-Sort: Merge Example A: k=3 R: L: i= j=1

30 Merge-Sort: Merge Example A: k=4 R: L: i= j=2

31 Merge-Sort: Merge Example A: k=5 R: L: i= j=3

32 Merge-Sort: Merge Example A: k=6 R: L: i= j=3

33 Merge-Sort: Merge Example A: k=7 R: L: i= j=4

34 Merge-Sort: Merge Example A: k=8 R: L: i= j=4

35 Merge-Sort Execution Example Divide

36 Merge-Sort Execution Example Recursive call, divide

37 Merge-Sort Execution Example Recursive call, divide

38 Merge-Sort Execution Example Recursive call, base case

39 Merge-Sort Execution Example Recursive call return

40 Merge-Sort Execution Example Recursive call, base case

41 Merge-Sort Execution Example Recursive call return

42 Merge-Sort Execution Example Merge

43 Merge-Sort Execution Example Recursive call return

44 Merge-Sort Execution Example Recursive call, divide

45 Merge-Sort Execution Example Recursive call, base case

46 Merge-Sort Execution Example Recursive call return

47 Merge-Sort Execution Example Recursive call, base case

48 Merge-Sort Execution Example Recursive call return

49 Merge-Sort Execution Example merge

50 Merge-Sort Execution Example Recursive call return

51 Merge-Sort Execution Example merge

52 Merge-Sort Execution Example Recursive call return

53 Merge-Sort Execution Example Recursive call

54 Merge-Sort Execution Example

55 Merge-Sort Execution Example Recursive call return

56 Merge-Sort Execution Example merge

57 Analysis of Merge Sort Statement Effort MergeSort(A, left, right) { if (left < right) { mid = floor((left + right) / 2); MergeSort(A, left, mid); MergeSort(A, mid+1, right); Merge(A, left, mid, right); } } So T(n) = (1) when n = 1, and 2T(n/2) + (n) when n > 1 So what (more succinctly) is T(n)? T(n) (1) (1) T(n/2) T(n/2) (n)

58 Recurrences The expression: c n 1 T ( n) n 2T cn n 1 2 is a recurrence. Recurrence: an equation that describes a function in terms of its value on smaller functions

59 Formulating recurrences In general, a recurrence is based on the three steps of Divide & Conquer paradigm T(n) = (1) if n<=c = a T(n/b) + C(n) + D(n) otherwise where C(n) is time to combine & D(n) is time to Divide

60 Example Recurrences T(n) = T(n-1) + n Θ(n) Recursive algorithm that halves the input but must examine every item in the input T(n) = 2T(n/2) + 1 Θ(lg n) Recursive algorithm that halves the input in one step T(n) = T(n/2) + n Recursive algorithm that loops through the input to eliminate one item T(n) = T(n/2) + c Θ(n2) Θ(n) Recursive algorithm that splits the input into 2 halves and does a constant amount of other work

61 Recurrence Examples 0 n 0 s ( n) c s(n 1) n 0 0 n 0 s ( n) n s(n 1) n 0 c n 1 T ( n) n 2T c n 1 2 c n 1 T ( n) n at cn n 1 b

62 Solving Recurrences Assumptions n is an integer Boundary conditions are ignored Omit floors & ceilings Thus, the recurrence for merge sort becomes T(n) = 2T(n/2) + (n) or T(n) = 2T(n/2) + cn

63 Methods of Solving Recurrences Iteration method Substitution method Recursion Tree Master method

64 Iteration Method Expand the recurrence Work some algebra to express as a summation Evaluate the summation

65 0 n 0 T ( n) c T (n 1) n 0 T(n) = c + T(n-1) c + c + T(n-2) 2c + T(n-2) 2c + c + T(n-3) 3c + T(n-3) kc + T(n-k) = ck + T(n-k)

66 0 n 0 T ( n) c T (n 1) n 0 So far for n >= k we have What if k = n? T(n) = ck + T(n-k) T(n) = cn + T(0) = cn Thus in general T(n) = cn

67 0 n 0 T ( n) n T (n 1) n 0 = = = = = = T(n) n + T(n-1) n + n-1 + T(n-2) n + n-1 + n-2 + T(n-3) n + n-1 + n-2 + n-3 + T(n-4) n + n-1 + n-2 + n n-(k-1) + T(n-k) n i i n k 1 T (n k )

68 0 n 0 T ( n) n T (n 1) n 0 So far for n >= k we have n i i n k 1 T (n k )

69 0 n 0 T ( n) n T (n 1) n 0 So far for n >= k we have n i T (n k ) i n k 1 What if k = n?

70 0 n 0 T ( n) n T (n 1) n 0 So far for n >= k we have n i T (n k ) i n k 1 What if k = n? n 1 i T (0) i 0 n 2 i 1 i 1 n n

71 0 n 0 T ( n) n T (n 1) n 0 So far for n >= k we have n i T (n k ) i n k 1 What if k = n? n 1 i T (0) i 0 n 2 i 1 i 1 n Thus in general n 1 T ( n) n 2 n

72 c n 1 n T (n) 2T c n 1 2 T(n) = 2T(n/2) + c 2(2T(n/2/2) + c) + c 22T(n/22) + 2c + c 22(2T(n/22/2) + c) + 3c 23T(n/23) + 4c + 3c 23T(n/23) + 7c 23(2T(n/23/2) + c) + 7c 24T(n/24) + 15c 2kT(n/2k) + (2k - 1)c

73 c n 1 n T (n) 2T c n 1 2 So far for n > 2k we have T(n) = 2kT(n/2k) + (2k - 1)c What if k = lg n? T(n) = 2lg n T(n/2lg n) + (2lg n - 1)c = n T(n/n) + (n - 1)c = n T(1) + (n-1)c = nc + (n-1)c = (2n - 1)c T(n) = O (n)

74 A Useful Result If c n 1 n T (n) at cn n 1 b n T (n) n log b n n logb a a b a b a b

75 Home Assignment Solve using iteration method T(n) = 3 T (n/4) +n T(n) = 2T(n/2) + n

76 Up Next Substitution Method Recursion Tree Method Master s theorem

77 Quicksort Lecture 7

78 Quicksort Another divide-and-conquer algorithm The array A[p..r] is partitioned into two possibly empty subarrays A[p..q-1] and A[q+1..r] Invariant: All elements in A[p..q] are less than all elements in A[q+1..r] The subarrays are recursively sorted by calls to quicksort Unlike merge sort, no combining step: two subarrays form an already-sorted array

79 Quicksort Quicksort(A, p, r) { if (p < r) { q = Partition(A, p, r); Quicksort(A, p, q-1); Quicksort(A, q+1, r); } } Initial call is quicksort(a, 1, length[a])

80 Partition Clearly, all the action takes place in the partition() function Rearranges the subarray in place End result: Two subarrays All values in first subarray all values in second Returns the index of the pivot element separating the two subarrays How do you suppose we implement this?

81 Partition (in Book) PARTITION(A, p, r ) { x A[r ] i p 1 for j p to r 1 { if A[ j ] x then {i i + 1 exchange A[i ] A[ j ] } } exchange A[i + 1] A[r ] return i + 1 }

82 Partition In Words (Alternative Approach) Partition(A, p, r): Select an element to act as the pivot (which?) Grow two regions, A[p..i] and A[j..r] All elements in A[p..i] <= pivot All elements in A[j..r] >= pivot Increment i until A[i] >= pivot Decrement j until A[j] <= pivot Swap A[i] and A[j] Note: slightly different from book s partition() Repeat until i >= j Return j

83 Partition Algorithm Partition(A, p, r) {x = A[p]; Illustrate on i = p - 1; A = {5, 3, 2, 6, 4, 1, 3, 7}; j = r + 1; while (TRUE) repeat j--; until A[j] <= x; repeat What is the running time of i++; partition()? until A[i] >= x; if (i < j) Swap(A, i, j); else return j; }

84 Partition Code Partition(A, p, r) x = A[p]; i = p - 1; j = r + 1; while (TRUE) repeat j--; until A[j] <= x; repeat i++; until A[i] >= x; if (i < j) Swap(A, i, j); else return j; partition() runs in O(n) time

85 Analyzing Quicksort What will be the worst case for the algorithm? What will be the best case for the algorithm? Partition is perfectly balanced Which is more likely? Partition is always unbalanced The latter, by far Will any particular input elicit the worst case? Yes: Already-sorted input

86 Analyzing Quicksort In the worst case: T(n) = T(n - 1) + T(0) + (n) = T(n - 1) + (n) Works out to T(n) = (n2)

87 Analyzing Quicksort In the best case: T(n) = 2T(n/2) + (n) What does this work out to? T(n) = (n lg n)

88 Analyzing Quicksort: Balanced Partitoning Assuming random input, average-case running time is much closer to O(n lg n) than O(n2) First, a more intuitive explanation/example: Suppose that partition() always produces a 9-to-1 split. This looks quite unbalanced! The recurrence is thus: Use n instead of O(n) for convenience (how?) T(n) = T(9n/10) + T(n/10) + cn How deep will the recursion go? (draw it)

89 Analyzing Quicksort: Average Case Intuitively, a real-life run of quicksort will produce a mix of bad and good splits Randomly distributed among the recursion tree Pretend for intuition that they alternate between best-case ((n/2 : n/2) and worst-case (n-1 : 0) What happens if we bad-split root node, then goodsplit the resulting size (n-1) node?

90 Analyzing Quicksort: Average Case Intuitively, a real-life run of quicksort will produce a mix of bad and good splits Randomly distributed among the recursion tree Pretend for intuition that they alternate between best-case (n/2 : n/2) and worst-case (n-1 : 1) What happens if we bad-split root node, then good-split the resulting size (n-1) node? We fail English

91 Analyzing Quicksort: Average Case Intuitively, a real-life run of quicksort will produce a mix of bad and good splits Randomly distributed among the recursion tree Pretend for intuition that they alternate between best-case (n/2 : n/2) and worst-case (n-1 : 1) What happens if we bad-split root node, then good-split the resulting size (n-1) node? We end up with three subarrays, size 0, (n-1)/2-1, (n-1)/2 Combined cost of splits = (n) + ( n 1) = (n) No worse than if we had good-split the root node!

92 Analyzing Quicksort: Average Case Intuitively, the (n) cost of a bad split (or 2 or 3 bad splits) can be absorbed into the O(n) cost of each good split Thus running time of alternating bad and good splits is still (n lg n), with slightly higher constants How can we be more rigorous?

93 Improving Quicksort The real liability of quicksort is that it runs in O(n2) on already-sorted input Book discusses two solutions: Randomize the input array, OR Pick a random pivot element How will these solve the problem? By insuring that no particular input can be chosen to make quicksort run in O(n2) time

94 Randomize Quicksort Random Sampling:- instead of always using A[r] as pivot, use a randomly chosen element from the subarray A[p..r] as pivot Randomized_Partition( A, p, r) { i = random(p,r) exchange(a[r], A[i]) return partion(a, p, r) }

95 Randomized Quicksort Randomized_Quicksort(A, p, r) { if (p < r) { q = Randomized_Partition(A, p, r); Randomized_Quicksort(A, p, q-1); Randomized_Quicksort(A, q+1, r); } }

96 Up Next Sorting in Linear Time Counting Sort Radix Sort Bucket Sort

97 Divide and Conquer Algorithms Sorting Algorithms (Already Covered) Searching Algorithms (Already Covered) Matrix Multiplication Problem Convex Hull Problem

98 Matrix Multiplication

99 Basic Matrix Multiplication Suppose we want to multiply two matrices of size N x N: for example A x B = C. C11 = a11b11 + a12b21 C12 = a11b12 + a12b22 C21 = a21b11 + a22b21 C22 = a21b12 + a22b22

100 Basic Matrix Multiplication matrix_mult (A, B){ for i= 1 to N { algorithm for j = 1 to N { Ci,j = 0 for k = 1 to N { Ci,j = Cij + Aik * Bkj }}}} N Ci, j ai,k bk, j k 1 Time analysis N N N Thus T ( N ) c cn 3 O( N 3 ) i 1 j 1 k 1

101 Divide and Conquer Matrix Multiply A A0 A1 A2 A3 B = B0 B1 B2 B3 = R A0 B0+A1 B2 A0 B1+A1 B3 A2 B0+A3 B2 A2 B1+A3 B3 Divide matrices into sub-matrices: A0, A1, A2 etc Use blocked matrix multiply equations Recursively multiply sub-matrices

102 Divide and Conquer Matrix Multiply A B = R a0 b0 = a0 b0 Terminate recursion with a simple base case If n is not a power of 2, matrices can be padded with zeros.

103 Analysis Divide and Conquer Matrix Multiply This method is no faster than the basic one. Each of the four equations specifies two multiplications of n/2 X n/2 matrices and the addition of their n/2 X n/2 products. Using these equation to define a straigntforward divide-and-conquer strategy, we derive the following recurrence for the time T(n) to multiply two n X n matirces: T(n) = 8T(n/2) + Θ(n2) = Θ(n3),

104 Strassens s Matrix Multiplication Strassen discovered a different recursive approach that requires only 7 recursive multiplications of n/2 X n/2 matrices and 18 Θ(n2) scalar additions and subtractions, yielding the recurrence T(n) = 7 T(n/2) + n2 = Θ(n lg7 ) = O(n2.81)

105 Strassens s Matrix Multiplication P1 = (A11+ A22)(B11+B22) P2 = (A21 + A22) * B11 P3 = A11 * (B12 - B22) P4 = A22 * (B21 - B11) P5 = (A11 + A12) * B22 P6 = (A21 - A11) * (B11 + B12) P7 = (A12 - A22) * (B21 + B22) C11 = P1 + P4 - P5 + P7 C12 = P3 + P5 C21 = P2 + P4 C22 = P1 + P3 - P2 + P6

106 C11 = P1 + P4 - P5 + P7 = (A11+ A22)(B11+B22) + A22 * (B21 - B11) - (A11 + A12) * B22+ (A12 - A22) * (B21 + B22) = A11 B11 + A11 B22 + A22 B11 + A22 B22 + A22 B21 A22 B11 A11 B22 -A12 B22 + A12 B21 + A12 B22 A22 B21 A22 B22 = A11 B11 + A12 B21

107 Convex Hull

108 Convex Hull The convex Hull of a set Q of points is the smallest convex polygon P for which each point in Q is either on the boundary of P or its interior. A simple polygon is convex if, given any two points on its boundary or in its interior, all points in the line segment drawn between them are contained in the polygon s boundary or interior

109 Convex Hull An informal definition that has a little more precision but is still easy to understand might say that the convex hull meets the following properties: The hull is a cycle graph whose vertices are composed of a subset of the points in set Q. No points in Q lie outside the graph. All interior angles in the graph are less than 180 degrees

110 Convex Hull Given a set of pins on a pinboard And a rubber band around them How does the rubber band look when it snaps tight? We may represent the convex hull as the sequence of points on the convex hull polygon, in counterclockwise or clockwise order

111 Some Applications Collision Avoidance robot motion planning Finding Smallest Box collision detection Shape Analysis

112 Brute Force Algorithm for finding Convex Hull for all points p in Q for all points s in Q if p!= s draw a line from p to s if all points in Q except p and s lie to the left of the line add the directed vector ps to the solution set After running this algorithm, we get a list of point pairs that compose the solution set, and we simply have to put them together in the correct order. Running Time - a triply nested loop that runs over the magnitude of N, making this an O(n3) algorithm. There are algorithms that calculate a convex hull in a 2D space considerably faster - in O(n lg n) time

113 Approaches to finding Convex Hull Several methods exist for finding the convex hull in O(n lg n) time Some methods are Incremental Method QuickHull Graham s Scan Jarvis March or gift wrapping, etc, We focus on the Divide and Conquer Approach The only technique known to extend to 3D and still achieve O(n lg n) time complexity

114 Convex Hull: Divide & Conquer Preprocessing: sort the points by xcoordinate Divide the set of points into two sets A and B: A contains the left n/2 points, B contains the right n/2 points Recursively compute the convex hull of A Recursively compute the convex hull of B Merge the two convex hulls A B

115 Convex Hull: Runtime Preprocessing: sort the points by xcoordinate Divide the set of points into two sets A and B: O(n log n) O(1) A contains the left n/2 points, B contains the right n/2 points Recursively compute the convex hull of A Recursively compute the convex hull of B Merge the two convex hulls T(n/2) T(n/2) O(n) just once

116 Convex Hull: Runtime Runtime Recurrence: T(n) = 2 T(n/2) + cn Solves to T(n) = (n log n)

117 Merging in O(n) time Find upper and lower tangents in O(n) time Compute the convex hull of A B: walk anti-clockwise around the convex hull of B, starting with right endpoint of lower tangent when hitting the right endpoint of the 1 5 upper tangent, cross over to the convex hull of A walk anti-clockwise around the convex hull of A when hitting left endpoint of the lower tangent we re done This takes O(n) time 0 6 A B

118 Finding the lower tangent in O(n) time 3 a = rightmost point of A b = leftmost point of B 4=b 4 while T=ab not lower tangent to both convex hulls of A and B do{ } while T not lower tangent to convex hull of A do{ a=a-1 } while T not lower tangent to convex hull of B do{ b=b+1 } can be checked in constant time a= A B right turn or left turn?

119 Convex Hull: Runtime Preprocessing: sort the points by xcoordinate Divide the set of points into two sets A and B: O(n log n) O(1) A contains the left n/2 points, B contains the right n/2 points Recursively compute the convex hull of A Recursively compute the convex hull of B Merge the two convex hulls T(n/2) T(n/2) O(n) just once

120 Convex Hull: Runtime Runtime Recurrence: T(n) = 2 T(n/2) + cn Solves to T(n) = (n log n)

121 Convex Hulls in Higher Dimensions Unfortunately, convex hull in d dimensions can have (n d/2 ) facets (proved by Klee, 1980) therefore, 4D hull can have quadratic size No O(n lg n) algorithm possible for d > 3 These approaches can extend to d > 3 gift wrapping QuickHull divide & conquer incremental

122 Searching and Sorting Linear Search Binary Search -Reading p Pearson Addison-Wesley. All rights reserved 1

123 Linear Search Searching is the process of determining whether or not a given value exists in a data structure or a storage media. We discuss two searching methods on one-dimensional arrays: linear search and binary search. The linear (or sequential) search algorithm on an array is: Sequentially scan the array, comparing each array item with the searched value. If a match is found; return the index of the matched element; otherwise return 1. Note: linear search can be applied to both sorted and unsorted arrays Pearson Addison-Wesley. All rights reserved 2

124 Linear Search The algorithm translates to the following Java method: public static int linearsearch(object[] array, Object key) { for(int k = 0; k < array.length; k++) if(array[k].equals(key)) return k; return -1; } 2006 Pearson Addison-Wesley. All rights reserved 3

125 Binary Search Binary search uses a recursive method to search an array to find a specified value The array must be a sorted array: a[0] a[1] a[2]... a[finalindex] If the value is found, its index is returned If the value is not found, -1 is returned Note: Each execution of the recursive method reduces the search space by about a half 2006 Pearson Addison-Wesley. All rights reserved 4

126 Binary Search An algorithm to solve this task looks at the middle of the array or array segment first If the value looked for is smaller than the value in the middle of the array Then the second half of the array or array segment can be ignored This strategy is then applied to the first half of the array or array segment 2006 Pearson Addison-Wesley. All rights reserved 5

127 Binary Search If the value looked for is larger than the value in the middle of the array or array segment Then the first half of the array or array segment can be ignored This strategy is then applied to the second half of the array or array segment If the value looked for is at the middle of the array or array segment, then it has been found If the entire array (or array segment) has been searched in this way without finding the value, then it is not in the array 2006 Pearson Addison-Wesley. All rights reserved 6

128 Pseudocode for Binary Search 2006 Pearson Addison-Wesley. All rights reserved 7

129 Recursive Method for Binary Search 2006 Pearson Addison-Wesley. All rights reserved 8

130 Execution of the Method search (Part 1 of 2) 2006 Pearson Addison-Wesley. All rights reserved 9

131 Execution of the Method search (Part 1 of 2) 2006 Pearson Addison-Wesley. All rights reserved 10

132 Checking the search Method 1. There is no infinite recursion On each recursive call, the value of first is increased, or the value of last is decreased If the chain of recursive calls does not end in some other way, then eventually the method will be called with first larger than last 2006 Pearson Addison-Wesley. All rights reserved 11

133 Checking the search Method 2. Each stopping case performs the correct action for that case If first > last, there are no array elements between a[first] and a[last], so key is not in this segment of the array, and result is correctly set to 1 If key == a[mid], result is correctly set to mid 2006 Pearson Addison-Wesley. All rights reserved 12

134 Checking the search Method 3. For each of the cases that involve recursion, if all recursive calls perform their actions correctly, then the entire case performs correctly If key < a[mid], then key must be one of the elements a[first] through a[mid-1], or it is not in the array The method should then search only those elements, which it does The recursive call is correct, therefore the entire action is correct 2006 Pearson Addison-Wesley. All rights reserved 13

135 Checking the search Method If key > a[mid], then key must be one of the elements a[mid+1] through a[last], or it is not in the array The method should then search only those elements, which it does The recursive call is correct, therefore the entire action is correct The method search passes all three tests: Therefore, it is a good recursive method definition 2006 Pearson Addison-Wesley. All rights reserved 14

136 Efficiency of Binary Search The binary search algorithm is extremely fast compared to an algorithm that tries all array elements in order About half the array is eliminated from consideration right at the start Then a quarter of the array, then an eighth of the array, and so forth 2006 Pearson Addison-Wesley. All rights reserved 15

137 Efficiency of Binary Search Given an array with 1,000 elements, the binary search will only need to compare about 10 array elements to the key value, as compared to an average of 500 for a serial search algorithm The binary search algorithm has a worst-case running time that is logarithmic: O(log n) A serial search algorithm is linear: O(n) If desired, the recursive version of the method search can be converted to an iterative version that will run more efficiently 2006 Pearson Addison-Wesley. All rights reserved 16

138 Iterative Version of Binary Search (Part 1 of 2) 2006 Pearson Addison-Wesley. All rights reserved 17

139 Iterative Version of Binary Search (Part 2 of 2) 2006 Pearson Addison-Wesley. All rights reserved 18