Mergesort. ! mergesort! sorting complexity! comparators. ! bottom-up mergesort. Two classic sorting algorithms

Size: px

Start display at page:

Download "Mergesort. ! mergesort! sorting complexity! comparators. ! bottom-up mergesort. Two classic sorting algorithms"

Jessica Hensley
5 years ago
Views:

! mergesort! sorting complexity! comparators ergesort. Java sort for objects. erl, ython stable sort. today eference: lgorithms in Java. th dition, ection. http://www.cs.princeton.

1 wo classic sorting algorithms ergesort Critical components in the world s computational infrastructure. Full scientific understanding of their properties has enabled us to develop them into practical system sorts. Quicksort honored as one of top algorithms of th century in science and engineering.! mergesort! sorting complexity! comparators ergesort. Java sort for objects. erl, ython stable sort. today eference: lgorithms in Java. th dition, ection. xcept as otherwise noted, the content of this presentation is licensed under the Creative Commons ttribution.5 icense. Quicksort. Java sort for primitive types. C qsort, Unix, g++, Visual C++, ython. next lecture lgorithms in Java, th dition obert edgewick and Kevin Wayne Copyright February 5, 9 :5: ergesort asic plan. Divide array into two halves. ecursively sort each half. erge two halves.! mergesort! bottom-up mergesort! sorting complexity! comparators input sort left half sort right half merge results G G G G ergesort overview

2 ergesort trace erging Goal. Combine two sorted subarrays into a sorted whole. lo hi merge(a,,, ) merge(a,,, ) merge(a,,, ) merge(a,,, 5) merge(a, 6, 6, ) merge(a,, 5, ) merge(a,,, ) merge(a,,, 9) merge(a,,, ) merge(a,, 9, ) merge(a,,, ) merge(a,,, 5) merge(a,,, 5) merge(a,,, 5) merge(a,,, 5) G G G G G G G G G G G G G G G G race of merge results for top-down mergesort result after recursive call Q. How to merge efficiently?. Use an auxiliary array. aux[] k i j input G C copy G C G C 5 6 G C C G C C G C G C G 5 C G G 6 C G C G 5 C G C G 6 merged result C G bstract in-place merge trace 5 6 erging: Java implementation ergesort: Java implementation public static void merge(comparable[] a, int lo, int mid, int hi) // erge a[lo..m] with a[m+..hi]. for (int k = lo; k < hi; k++) aux[k] = a[k]; int i = lo, j = mid; for (int k = lo; k < hi; k++) if (i == mid) a[k] = aux[j++]; else if (j == hi ) else if (less(aux[j], aux[i])) a[k] = aux[j++]; else copy merge public class erge private static Comparable[] aux; private static void merge(comparable[] a, int lo, int mid, int hi) /* as before */ private static void sort(comparable[] a, int lo, int hi) if (hi <= lo) return; int mid = lo + (hi - lo) / ; sort(a, lo, mid); sort(a, mid+, hi); merge(a, lo, mid, hi); aux[] lo i m j hi G H I k G H I public static void sort(comparable[] a) aux = new Comparable[a.length]; sort(a,, a.length - ); lo m hi 5 6 9

ergesort visualization ergesort animation first subfile done untouched merge in progress input second subfile first merge first half sorted second half sorted result

estimates: Home pc executes comparisons/second. upercomputer executes comparisons/second.

3 ergesort visualization ergesort animation first subfile done untouched merge in progress input second subfile first merge first half sorted second half sorted result Visual trace of top-down mergesort with cutoff for small subfiles 9 merge in progress output auxiliary array ergesort animation ergesort: empirical analysis unning time estimates: Home pc executes comparisons/second. upercomputer executes comparisons/second. insertion sort (N ) mergesort (N log N) computer thousand million billion thousand million billion home instant. hours years instant second min super instant second week instant instant instant ottom line. Good algorithms are better than supercomputers.

4 ergesort: mathematical analysis ergesort recurrence: proof roposition. ergesort uses ~ N lg N compares to sort any array of size N. ergesort recurrence. (N) = (N / ) + N for N >, with () =. Def. (N) = number of compares to mergesort an array of size N. = (N / ) + (N / ) + N roposition. If N is a power of, then (N) = N lg N. f. left half right half merge (N) N = N ergesort recurrence. (N) = (N / ) + N for N >, with () =. Not quite right for odd N. ame recurrence holds for many divide-and-conquer algorithms. (N/) (N/) (N/) (N/) (N/) (N/) (N/) = N (N/) = N... lg N (N / k ) k (N/ k ) = N olution. (N) ~ N lg N. For simplicity, we'll prove when N is a power of. rue for all N. [see C ] () () () () () () () ()... N/ () = N N lg N ergesort recurrence: proof ergesort recurrence: proof ergesort recurrence. (N) = (N / ) + N for N >, with () =. ergesort recurrence. (N) = (N / ) + N for N >, with () =. roposition. If N is a power of, then (N) = N lg N. f. (N) = (N/) + N given (N) / N = (N/) / N + = (N/) / (N/) + divide both sides by N algebra roposition. If N is a power of, then (N) = N lg N. f. [by induction on N] ase case: N =. Inductive hypothesis: (N) = N lg N. Goal: show that (N) = N lg (N). = (N/) / (N/) + + apply to first term (N) = (N) + N given = (N/) / (N/) apply to first term again = N lg N + N inductive hypothesis... = N (lg (N) - ) + N algebra = (N/N) / (N/N) stop applying, () = = N lg (N) QD = lg N 5 6

5 ergesort analysis: memory ergesort: practical improvements roposition G. ergesort uses extra space proportional to N. f. he array aux[] needs to be of size N for the last merge. two sorted subarrays C D G H I N U V F J Q Use insertion sort for small subarrays. ergesort has too much overhead for tiny subarrays. Cutoff to insertion sort for! elements. top if already sorted. Is biggest element in first half! smallest element in second half? Helps for nearly ordered lists. C D F G H I J N Q U V C D F G H I J N Q U V merged result C D F G H I J N Q U V Def. sorting algorithm is in-place if it uses (log N) extra memory. x. Insertion sort, selection sort, shellsort. liminate the copy to the auxiliary array. ave time (but not space) by switching the role of the input and auxiliary array in each recursive call. Challenge for the bored. In-place merge. [Kronrud, 969] x. ee rrays.sort(). ottom-up mergesort asic plan. ass through array, merging subarrays of size. epeat for subarrays of size,,, 6,...! mergesort! bottom-up mergesort! sorting complexity! comparators lo m hi merge(a,,, ) merge(a,,, ) merge(a,,, 5) merge(a, 6, 6, ) merge(a,,, 9) merge(a,,, ) merge(a,,, ) merge(a,,, 5) merge(a,,, ) merge(a,, 5, ) merge(a,, 9, ) merge(a,,, 5) merge(a,,, ) merge(a,,, 5) merge(a,,, 5) a[i] G G G G G G G G G G G G G G G G race of merge results for bottom-up mergesort ottom line. No recursion needed! 9

ottom-up mergesort: Java implementation ottom-up mergesort: visual trace public class ergeu private static

static void sort(comparable[] a) int N = a.

i, i+m, ath.min(i+m+m, N)); 6 ottom line. Concise industrial-strength code, if you have the space.

6 ottom-up mergesort: Java implementation ottom-up mergesort: visual trace public class ergeu private static Comparable[] aux; private static void merge(comparable[] a, int lo, int m, int hi) /* as before */ public static void sort(comparable[] a) int N = a.length; aux = new Comparable[N]; for (int m = ; m < N; m = m+m) for (int i = ; i < N-m; i += m+m) merge(a, i, i+m, ath.min(i+m+m, N)); 6 ottom line. Concise industrial-strength code, if you have the space. Visual trace of bottom-up mergesort otom-up mergesort animation otom-up mergesort animation this pass last pass merge in progress input merge in progress output auxiliary array

7 Complexity of sorting Computational complexity. Framework to study efficiency of algorithms for solving a particular problem.! mergesort! bottom-up mergesort! sorting complexity! comparators achine model. Focus on fundamental operations. Upper bound. Cost guarantee provided by some algorithm for. ower bound. roven limit on cost guarantee of all algorithms for. ptimal algorithm. lgorithm with best cost guarantee for. xample: sorting. achine model = # compares. Upper bound = ~ N lg N from mergesort. ower bound = ~ N lg N? ptimal algorithm = mergesort? lower bound ~ upper bound access information only through compares 5 6 Decision tree (for distinct elements) Compare-based lower bound for sorting yes a < b no worst-case number of compares roposition. ny compare-based sorting algorithm must use more than N lg N -. N comparisons in the worst-case. yes a b c b < c no a < c code between comparisons (e.g., sequence of exchanges) yes b a c a < c no b < c f. ssume input consists of N distinct values a through an. Worst case dictated by height h of decision tree. inary tree of height h has at most h leaves. N! different orderings " at least N! leaves. yes no yes no a c b c a b b c a c b a h at least N! leaves no more than h leaves

8 Compare-based lower bound for sorting Complexity of sorting roposition. ny compare-based sorting algorithm must use more than N lg N -. N comparisons in the worst-case. f. ssume input consists of N distinct values a through an. Worst case dictated by height h of decision tree. inary tree of height h has at most h leaves. N! different orderings " at least N! leaves. h " N! h " lg N! " lg (N / e) N = N lg N - N lg e " N lg N -. N tirling's formula achine model. Focus on fundamental operations. Upper bound. Cost guarantee provided by some algorithm for. ower bound. roven limit on cost guarantee of all algorithms for. ptimal algorithm. lgorithm with best cost guarantee for. xample: sorting. achine model = # compares. Upper bound = ~ N lg N from mergesort. ower bound = ~ N lg N. ptimal algorithm = mergesort. First goal of algorithm design: optimal algorithms. 9 Complexity results in context Complexity results in context (continued) ther operations? ergesort optimality is only about number of compares. pace? ergesort is not optimal with respect to space usage. Insertion sort, selection sort, and shellsort are space-optimal. Challenge. Find an algorithm that is both time- and space-optimal. ower bound may not hold if the algorithm has information about: he key values. heir initial arrangement. artially ordered arrays. Depending on the initial order of the input, we may not need N lg N compares. insertion sort requires (N) compares on an already sorted array essons. Use theory as a guide. x. Don't try to design sorting algorithm that uses! N lg N compares. Duplicate keys. Depending on the input distribution of duplicates, we may not need N lg N compares. Digital properties of keys. We can use digit/character compares instead of key compares for numbers and strings. stay tuned for -way quicksort stay tuned for radix sorts

9 Natural order Comparable interface: sort uses type s natural order. public class Date implements Comparable<Date> private final int month, day, year;! mergesort! bottom-up mergesort! sorting complexity! comparators public Date(int m, int d, int y) month = m; day = d; year = y; public int compareo(date that) if (this.year < that.year ) return -; if (this.year > that.year ) return +; if (this.month < that.month) return -; if (this.month > that.month) return +; if (this.day < that.day ) return -; if (this.day > that.day ) return +; return ; natural order Generalized compare Comparators Comparable interface: sort uses type s natural order. olution. Use Java's Comparator interface. roblem. ay want to use a non-natural order. roblem. Desired data type may not come with a natural order. x. ort strings by: Natural order. Case insensitive. panish. ritish phone book. Now is the time is Now the time café cafetero cuarto churro nube ñoño ckinley ackintosh pre-99 order for digraphs ch and ll and rr tring[] a;... rrays.sort(a); rrays.sort(a, tring.c_inniiv_d); rrays.sort(a, Collator.getInstance(ocale.NIH)); public interface Comparator<Key> public int compare(key v, Key w); emark. he compare() method implements a total order like compareo(). dvantages. Decouples the definition of the data type from the definition of what it means to compare two objects of that type. Can add any number of new orders to a data type. Can add an order to a library data type with no natural order. import java.text.collator; 5 6

10 Comparator example ort implementation with comparators everse order. ort an array of strings in reverse order. public class everserder implements Comparator<tring> public int compare(tring a, tring b) return b.compareo(a); comparator implementation client... rrays.sort(a, new everserder());... o support comparators in our sort implementations: ass Comparator to sort() and less(). Use it in less(). x. Insertion sort. pedantic Java code (see book for simpler version) public static <Key> void sort(key[] a, Comparator<Key> comparator) int N = a.length; for (int i = ; i < N; i++) for (int j = i; j > ; j--) if (less(comparator, a[j], a[j-])) exch(a, j, j-); else break; private static <Key> boolean less(comparator<key> c, Key v, Key w) return c.compare(v, w) < ; private static <Key> void exch(key[] a, int i, int j) Key swap = a[i]; a[i] = a[j]; a[j] = swap; Generalized compare Generalized compare Comparators enable multiple sorts of a single file (by different keys). x. nable sorting students by name or by section. x. ort students by name or by section. rrays.sort(students, tudent.y_n); rrays.sort(students, tudent.y_c); public class tudent public static final Comparator<tudent> Y_N = new yname(); public static final Comparator<tudent> Y_C = new yect(); sort by name ndrews attle Chen C ittle Whitman lair then sort by section Fox --5 Chen ndrews 66-- Dickinson lair 9 ittle private final tring name; private final int section;... private static class yname implements Comparator<tudent> public int compare(tudent a, tudent b) return a.name.compareo(b.name); Fox Furia Gazsi Kanaga ohde Dickinson rown rown rown Forbes Furia Kanaga ohde attle Gazsi C rown rown Forbes Whitman rown private static class yect implements Comparator<tudent> public int compare(tudent a, tudent b) return a.section - b.section; only use this trick if no danger of overflow 9

Generalized compare problem orting challenge 5 typical application. First, sort by name; then sort by section. Which sorts are stable? rrays.sort(students, tudent.y_n); ndrews attle Chen rrays.

11 Generalized compare problem orting challenge 5 typical application. First, sort by name; then sort by section. Which sorts are stable? rrays.sort(students, tudent.y_n); ndrews attle Chen rrays.sort(students, tudent.y_c); Fox --5 Whitman Chen lair lair Kanaga rown Dickinson 9 ittle ittle C Dickinson Insertion sort? election sort? hellsort? ergesort? sorted by time Fox --5 ndrews 66-- Furia rown Furia rown Gazsi rown ohde Forbes Kanaga rown attle C -- Whitman ohde Forbes Gazsi rown hoenix Houston Houston eattle eattle hoenix eattle eattle eattle tudents in section no longer in order by name. stable sort preserves the relative order of records with equal keys. 9:: 9:: 9:: 9::59 9:: 9:: 9:: 9::5 9::5 9:9: 9:9:6 9::5 9:: 9::5 9:5:5 9:5: 9:6: 9:: sorted by location (not stable) Houston Houston hoenix hoenix hoenix eattle eattle eattle eattle eattle 9:5:5 9:: 9::5 9:9:6 9:9: 9:: 9:5: 9::59 9:: 9:: 9:: 9:: 9::5 9::5 9:6: 9:: 9:: 9::5 sorted by location (stable) no longer sorted by time Houston Houston hoenix hoenix hoenix eattle eattle eattle eattle eattle 9:: 9::59 9:: 9:9: 9:9:6 9::5 9:5:5 9:5: 9:: 9:: 9:: 9::5 9:: 9:: 9::5 9:: 9::5 9:6: still sorted by time tability when sorting on a second key orting challenge 5 orting challenge 5 Is insertion sort stable? Is selection sort stable? public class Insertion public static void sort(comparable[] a) int N = a.length; for (int i = ; i < N; i++) for (int j = i; j > ; j--) if (less(a[j], a[j-])) exch(a, j, j-); else break; i j 5 9 entries in gray do not move entry in red is a[j] entries in black moved one position right for insertion insertion) race of insertion sort (array contents just after each public class election public static void sort(comparable[] a) int N = a.length; for (int i = ; i < N; i++) int min = i; for (int j = i+; j < N; j++) if (less(a[j], a[min])) min = j; exch(a, i, min); i min entries in blac are examined to the minimum entries in red are a[min] entries in gray a in final position race of selection sort (array contents just after each exchange)

12 orting challenge 5C orting challenge 5D Is shellsort stable? Is mergesort stable? public class hell // hellsort. public static void sort(comparable[] a) // ort into increasing order. int N = a.length; int h = ; while (h < N/) h = *h + ; //,,,,, 6, 9,... while (h >= ) // h-sort the file. for (int i = h; i < N; i++) // Insert a[i] among a[i-h], a[i-*h], a[i-*h].... for (int j = i; j > && less(a[j], a[j-h]); j -= h) exch(a, j, j-h); h = h/; public class erge private static Comparable[] aux; private static void merge(comparable[] a, int lo, int mid, int hi) /* as before */ private static void sort(comparable[] a, int lo, int hi) if (hi <= lo) return; int mid = lo + (hi - lo) / ; sort(a, lo, mid); sort(a, mid+, hi); merge(a, lo, mid, hi); public static void sort(comparable[] a) aux = new Comparable[a.length]; sort(a,, a.length - ); 5 6 orting challenge 5D (continued) ostscript: ptimizing mergesort (a short history) Is merge stable? Goal: remove instructions from the inner loop. public static void merge(comparable[] a, int lo, int mid, int hi) // erge a[lo..m] with a[m+..hi]. for (int k = lo; k < hi; k++) aux[k] = a[k]; int i = lo, j = mid; for (int k = lo; k < hi; k++) if (i == mid) a[k] = aux[j++]; else if (j == hi ) else if (less(aux[j], aux[i])) a[k] = aux[j++]; else public static void merge(comparable[] a, int lo, int mid, int hi) // erge a[lo..m] with a[m+..hi]. for (int k = lo; k < hi; k++) aux[k] = a[k]; int i = lo, j = mid; for (int k = lo; k < hi; k++) if (i == mid) a[k] = aux[j++]; else if (j == hi ) else if (less(aux[j], aux[i])) a[k] = aux[j++]; else aux[] lo i m j hi G H I G H I k

ostscript: ptimizing mergesort (a short history) ostscript: ptimizing mergesort (a short history) Idea (96s): Use sentinels Idea (9s): everse copy

=, j =, k = ; k < +; k++) if (less(aux[j], aux[i])) aux[k] = a[i++]; aux[k] = b[j++]; i G for (int j = m+; j <= hi; j++) aux[j] = a[hi-j+m+];!

merge k aux[] G H I lo aux[] roblem : till need copy G i m j hi I H k roblem : No good place to put sentinels G H I roblem : Complicates data-type

ostscript: ptimizing mergesort (a short history) orting challenge 6 Idea (99s): liminate copy with recursive argument switch roblem: Choose mergesort

5 ecursive argument switch is out (recommended only for pros) int m = (r+l)/; mergesortr(b, a, l, m); mergesortr(b, a, m+, r); merge(a, l, b, l, m, b,

13 ostscript: ptimizing mergesort (a short history) ostscript: ptimizing mergesort (a short history) Idea (96s): Use sentinels Idea (9s): everse copy public static void merge(comparable[] a, int lo, int m, int hi) for (int i = lo; i <= m; i++) copy aux[i] = a[i]; := maxint; b[n] := maxint; for (int i =, j =, k = ; k < +; k++) if (less(aux[j], aux[i])) aux[k] = a[i++]; aux[k] = b[j++]; i G for (int j = m+; j <= hi; j++) aux[j] = a[hi-j+m+];! H b[] I int i = lo, j = hi; for (int k = lo; k <= hi; k++) if (less(aux[j], aux[i])) a[k] = aux[j--]; else N j reverse copy! merge k aux[] G H I lo aux[] roblem : till need copy G i m j hi I H k roblem : No good place to put sentinels G H I roblem : Complicates data-type interface (what is infinity for your type?) 9 roblem: Copy still in inner loop. ostscript: ptimizing mergesort (a short history) orting challenge 6 Idea (99s): liminate copy with recursive argument switch roblem: Choose mergesort for lgs th edition. 5 ecursive argument switch is out (recommended only for pros) int m = (r+l)/; mergesortr(b, a, l, m); mergesortr(b, a, m+, r); merge(a, l, b, l, m, b, m+, r); Q. Why not use reverse array copy? public static void merge(comparable[] a, int lo, int m, int hi) for (int i = lo; i <= m; i++) aux[i] = a[i]; roblem: Complex interactions with reverse copy olution: Go back to sentinels). for (int j = m+; j <= hi; j++) aux[j] = a[hi-j+m+]; int i = lo, j = hi; for (int k = lo; k <= hi; k++) if (less(aux[j], aux[i])) a[k] = aux[j--]; else rrays.sort() 5 5

Algorithms. Algorithms 2.2 MERGESORT. mergesort bottom-up mergesort sorting complexity comparators stability ROBERT SEDGEWICK KEVIN WAYNE

Algorithms. Algorithms 2.2 MERGESORT. mergesort bottom-up mergesort sorting complexity comparators stability ROBERT SEDGEWICK KEVIN WAYNE Algorithms ROBERT SEDGEWICK KEVIN WAYNE 2.2 MERGESORT Algorithms F O U R T H E D I T I O N ROBERT SEDGEWICK KEVIN WAYNE mergesort bottom-up mergesort sorting complexity comparators stability http://algs4.cs.princeton.edu