How to Win Coding Competitions: Secrets of Champions. Week 3: Sorting and Search Algorithms Lecture 7: Lower bound. Stable sorting.

Transcription

1 How to Win Coding Competitions: Secrets of Champions Week 3: Sorting and Search Algorithms Lecture 7: Lower bound. Stable sorting. Comparators Maxim Buzdalov Saint Petersburg 2016

2 Lower bound for comparison sorting Theorem. For every comparison-based sorting algorithm, there exists an input with N elements which requires at least b = Ω(N log N) comparisons. 2 / 6

3 Lower bound for comparison sorting Theorem. For every comparison-based sorting algorithm, there exists an input with N elements which requires at least b = Ω(N log N) comparisons. Proof Assume inputs are permutations. There are N! different permutations 2 / 6

4 Lower bound for comparison sorting Theorem. For every comparison-based sorting algorithm, there exists an input with N elements which requires at least b = Ω(N log N) comparisons. Proof Assume inputs are permutations. There are N! different permutations From a single comparison, any algorithm receives at most one bit of information 2 / 6

5 Lower bound for comparison sorting Theorem. For every comparison-based sorting algorithm, there exists an input with N elements which requires at least b = Ω(N log N) comparisons. Proof Assume inputs are permutations. There are N! different permutations From a single comparison, any algorithm receives at most one bit of information After making t comparisons, an algorithm can distinguish only 2 t cases 2 / 6

6 Lower bound for comparison sorting Theorem. For every comparison-based sorting algorithm, there exists an input with N elements which requires at least b = Ω(N log N) comparisons. Proof Assume inputs are permutations. There are N! different permutations From a single comparison, any algorithm receives at most one bit of information After making t comparisons, an algorithm can distinguish only 2 t cases Thus, for the lower bound b, it holds that 2 b N!, or b log 2 (N!) 2 / 6

7 Lower bound for comparison sorting Theorem. For every comparison-based sorting algorithm, there exists an input with N elements which requires at least b = Ω(N log N) comparisons. Proof Assume inputs are permutations. There are N! different permutations From a single comparison, any algorithm receives at most one bit of information After making t comparisons, an algorithm can distinguish only 2 t cases Thus, for the lower bound b, it holds that 2 b N!, or b log 2 (N!) By Stirling s approximation: ( ( n ) n ) log 2 (N!) log 2 2πn = n log e 2 n Θ(n) + O(log n) = Θ(n log n) 2 / 6

8 Lower bound for comparison sorting Theorem. For every comparison-based sorting algorithm, there exists an input with N elements which requires at least b = Ω(N log N) comparisons. Proof Assume inputs are permutations. There are N! different permutations From a single comparison, any algorithm receives at most one bit of information After making t comparisons, an algorithm can distinguish only 2 t cases Thus, for the lower bound b, it holds that 2 b N!, or b log 2 (N!) By Stirling s approximation: ( ( n ) n ) log 2 (N!) log 2 2πn = n log e 2 n Θ(n) + O(log n) = Θ(n log n) Mergesort is asymptotically optimal. 2 / 6

9 Lower bound for comparison sorting Theorem. For every comparison-based sorting algorithm, there exists an input with N elements which requires at least b = Ω(N log N) comparisons. Proof Assume inputs are permutations. There are N! different permutations From a single comparison, any algorithm receives at most one bit of information After making t comparisons, an algorithm can distinguish only 2 t cases Thus, for the lower bound b, it holds that 2 b N!, or b log 2 (N!) By Stirling s approximation: ( ( n ) n ) log 2 (N!) log 2 2πn = n log e 2 n Θ(n) + O(log n) = Θ(n log n) Mergesort is asymptotically optimal. Quicksort is asymptotically optimal on average. 2 / 6

10 Stable sorting I What is stable sorting? 3 / 6

11 Stable sorting I What is stable sorting? A sorting algorithm is stable if it preserves the order of equal elements 3 / 6

12 Stable sorting I What is stable sorting? A sorting algorithm is stable if it preserves the order of equal elements Why is it useful? 3 / 6

13 Stable sorting I What is stable sorting? A sorting algorithm is stable if it preserves the order of equal elements Why is it useful? Assume elements are already ordered by 1 You wish to sort them by 2, but keep equal elements ordered by 1 3 / 6

14 Stable sorting I What is stable sorting? A sorting algorithm is stable if it preserves the order of equal elements Why is it useful? Assume elements are already ordered by 1 You wish to sort them by 2, but keep equal elements ordered by 1 Solution: just use a stable sorting 3 / 6

15 Stable sorting I What is stable sorting? A sorting algorithm is stable if it preserves the order of equal elements Why is it useful? Assume elements are already ordered by 1 You wish to sort them by 2, but keep equal elements ordered by 1 Solution: just use a stable sorting Which sorting algorithms are stable? 3 / 6

16 Stable sorting I What is stable sorting? A sorting algorithm is stable if it preserves the order of equal elements Why is it useful? Assume elements are already ordered by 1 You wish to sort them by 2, but keep equal elements ordered by 1 Solution: just use a stable sorting Which sorting algorithms are stable? Insertion sort: 3 / 6

17 Stable sorting I What is stable sorting? A sorting algorithm is stable if it preserves the order of equal elements Why is it useful? Assume elements are already ordered by 1 You wish to sort them by 2, but keep equal elements ordered by 1 Solution: just use a stable sorting Which sorting algorithms are stable? Insertion sort: stable, as it never swaps A[i] and A[i + 1] if they are equal 3 / 6

18 Stable sorting I What is stable sorting? A sorting algorithm is stable if it preserves the order of equal elements Why is it useful? Assume elements are already ordered by 1 You wish to sort them by 2, but keep equal elements ordered by 1 Solution: just use a stable sorting Which sorting algorithms are stable? Insertion sort: stable, as it never swaps A[i] and A[i + 1] if they are equal Mergesort: 3 / 6

19 Stable sorting I What is stable sorting? A sorting algorithm is stable if it preserves the order of equal elements Why is it useful? Assume elements are already ordered by 1 You wish to sort them by 2, but keep equal elements ordered by 1 Solution: just use a stable sorting Which sorting algorithms are stable? Insertion sort: stable, as it never swaps A[i] and A[i + 1] if they are equal Mergesort: stable, as merge always picks the left element from two equal ones 3 / 6

20 Stable sorting I What is stable sorting? A sorting algorithm is stable if it preserves the order of equal elements Why is it useful? Assume elements are already ordered by 1 You wish to sort them by 2, but keep equal elements ordered by 1 Solution: just use a stable sorting Which sorting algorithms are stable? Insertion sort: stable, as it never swaps A[i] and A[i + 1] if they are equal Mergesort: stable, as merge always picks the left element from two equal ones Quicksort: 3 / 6

21 Stable sorting I What is stable sorting? A sorting algorithm is stable if it preserves the order of equal elements Why is it useful? Assume elements are already ordered by 1 You wish to sort them by 2, but keep equal elements ordered by 1 Solution: just use a stable sorting Which sorting algorithms are stable? Insertion sort: stable, as it never swaps A[i] and A[i + 1] if they are equal Mergesort: stable, as merge always picks the left element from two equal ones Quicksort: not stable. 3 / 6

22 Stable sorting I What is stable sorting? A sorting algorithm is stable if it preserves the order of equal elements Why is it useful? Assume elements are already ordered by 1 You wish to sort them by 2, but keep equal elements ordered by 1 Solution: just use a stable sorting Which sorting algorithms are stable? Insertion sort: stable, as it never swaps A[i] and A[i + 1] if they are equal Mergesort: stable, as merge always picks the left element from two equal ones Quicksort: not stable. In an array of two equal items, it will swap them 3 / 6

23 Stable sorting II How to make every sorting stable? 4 / 6

24 Stable sorting II How to make every sorting stable? Keep original indices with the elements 4 / 6

25 Stable sorting II How to make every sorting stable? Keep original indices with the elements Before: After: 5; 1 2; 2 4; 3 4; 4 5; 5 2; 6 4 / 6

26 Stable sorting II How to make every sorting stable? Keep original indices with the elements Before: After: 5; 1 2; 2 4; 3 4; 4 5; 5 2; 6 In the sorting, use a modified ordering : Given x; a and y; b If x y, return x y If x = y, return a b 4 / 6

27 Stable sorting II How to make every sorting stable? Keep original indices with the elements Before: After: 5; 1 2; 2 4; 3 4; 4 5; 5 2; 6 In the sorting, use a modified ordering : Given x; a and y; b If x y, return x y original ordering here If x = y, return a b 4 / 6

28 Stable sorting II How to make every sorting stable? Keep original indices with the elements Before: After: 5; 1 2; 2 4; 3 4; 4 5; 5 2; 6 In the sorting, use a modified ordering : Given x; a and y; b If x y, return x y original ordering here If x = y, return a b Pros: Can use any sorting, don t need to care of implementation 4 / 6

29 Stable sorting II How to make every sorting stable? Keep original indices with the elements Before: After: 5; 1 2; 2 4; 3 4; 4 5; 5 2; 6 In the sorting, use a modified ordering : Given x; a and y; b If x y, return x y original ordering here If x = y, return a b Pros: Can use any sorting, don t need to care of implementation Cons: Additional memory needed for storing the indices More complicated ordering, may further decrease performance 4 / 6

30 Comparators A comparator (as in java.util.comparator<t>) is a custom ordering for sorting certain objects for specific needs 5 / 6

31 Comparators A comparator (as in java.util.comparator<t>) is a custom ordering for sorting certain objects for specific needs Requirements for a comparator: Total (every two elements should be reported as either <, =, or > ) If it reports a = b, then a and b must be equal (in a problem-dependent sense) More specifically, if a = b, then b = a If a < b, then it should be that b > a (and vice versa) 5 / 6

32 Comparators A comparator (as in java.util.comparator<t>) is a custom ordering for sorting certain objects for specific needs Requirements for a comparator: Total (every two elements should be reported as either <, =, or > ) If it reports a = b, then a and b must be equal (in a problem-dependent sense) More specifically, if a = b, then b = a If a < b, then it should be that b > a (and vice versa) Implementation of a comparator: Java-way: return 0 if equal, negative if less, positive if greater C++-way: return true if less, false otherwise 5 / 6

33 A non-trivial comparator (0; 3) Sort the points counterclockwise around the red dot ( 2; 2) (1; 2) ( 1; 1) (0; 0) (2; 0) ( 2; 2) (1; 2) 6 / 6

34 A non-trivial comparator ( 2; 2) (0; 3) (1; 2) Sort the points counterclockwise around the red dot Need to break a circle: start at Ox ( 1; 1) (0; 0) (2; 0) ( 2; 2) (1; 2) 6 / 6

35 A non-trivial comparator ( 2; 2) (0; 3) (1; 2) Sort the points counterclockwise around the red dot Need to break a circle: start at Ox Idea: compare using oriented triangle area ( 1; 1) (0; 0) (2; 0) ( 2; 2) (1; 2) 6 / 6

36 A non-trivial comparator ( 2; 2) (0; 3) (1; 2) Sort the points counterclockwise around the red dot Need to break a circle: start at Ox Idea: compare using oriented triangle area ( 1; 1) (0; 0) (2; 0) ( 2; 2) (1; 2) function LessThan((x 1, y 1 ), (x 2, y 2 )) return x 1 y 2 x 2 y 1 > 0 end function 6 / 6

37 A non-trivial comparator (0; 3) ( 2; 2) (1; 2) ( 1; 1) Sort the points counterclockwise around the red dot Need to break a circle: start at Ox Idea: compare using oriented triangle area Wait, what happens there? (0; 0) (2; 0) ( 2; 2) (1; 2) function LessThan((x 1, y 1 ), (x 2, y 2 )) return x 1 y 2 x 2 y 1 > 0 end function 6 / 6

38 A non-trivial comparator (0; 3) ( 2; 2) (1; 2) ( 1; 1) Sort the points counterclockwise around the red dot Need to break a circle: start at Ox Idea: compare using oriented triangle area... but first check the halfplanes ( 2; 2) (0; 0) (2; 0) (1; 2) function LessThan((x 1, y 1 ), (x 2, y 2 )) h 1 0, h 2 0 if y 1 < 0 or (y 1 = 0 and x 1 < 0) then h 1 1 end if if y 2 < 0 or (y 2 = 0 and x 2 < 0) then h 2 1 end if if h 1 h 2 then return h 1 < h 2 end if return x 1 y 2 x 2 y 1 > 0 end function 6 / 6

39 A non-trivial comparator (0; 3) ( 2; 2) (1; 2) ( 1; 1) Sort the points counterclockwise around the red dot Need to break a circle: start at Ox Idea: compare using oriented triangle area... but first check the halfplanes Okay, but what happens for colinear points? ( 2; 2) (0; 0) (2; 0) (1; 2) function LessThan((x 1, y 1 ), (x 2, y 2 )) h 1 0, h 2 0 if y 1 < 0 or (y 1 = 0 and x 1 < 0) then h 1 1 end if if y 2 < 0 or (y 2 = 0 and x 2 < 0) then h 2 1 end if if h 1 h 2 then return h 1 < h 2 end if return x 1 y 2 x 2 y 1 > 0 end function 6 / 6

40 A non-trivial comparator (0; 3) ( 2; 2) (1; 2) ( 1; 1) Sort the points counterclockwise around the red dot Need to break a circle: start at Ox Idea: compare using oriented triangle area... but first check the halfplanes... and, from colinear points, favor closer ones. ( 2; 2) (0; 0) (2; 0) (1; 2) function LessThan((x 1, y 1 ), (x 2, y 2 )) h 1 0, h 2 0 if y 1 < 0 or (y 1 = 0 and x 1 < 0) then h 1 1 end if if y 2 < 0 or (y 2 = 0 and x 2 < 0) then h 2 1 end if if h 1 h 2 then return h 1 < h 2 end if z x 1 y 2 x 2 y 1 if z = 0 then return x1 2 + y 1 2 < x y 2 2 end if return z > 0 end function 6 / 6