CS Algorithms and Complexity

Transcription

1 CS Algorithms and Complexity Basic Sorting Sean Anderson 1/18/18 Portland State University

2 Table of contents 1. Core Concepts of Sort 2. Selection Sort 3. Insertion Sort 4. Non-Comparison Sorts 5. Sorting Cases 6. R&C and Binary Search 1

3 Core Concepts of Sort

4 Sort Sequence Sorting Instance: A sequence s of elements drawn from a total-ordered set S (with an operation ) Problem: Find a sequence t drawn from S s.t. t perm(s) i, j 1.. S, i < j t[i] t[j] 2

5 Total Ordering What kind of objects can we sort? Need to be defined - total ordering Transitive - a b b c a c Anti-symmetric - a b b a a = b Total a b b a In practice, we often sort by fields of record types, or by some relevant function. 3

6 Stable Sorts Sort pairs (p 0, p 1 ) by p 0 : (4, a) (1, b) (1, c) (4, d) Unstable solution: (1, c) (1, b) (4, d) (4, a) Stable solution: (1, b) (1, c) (4, a) (4, d) Stable sort: equal (by our ) elements remain in the order they began in. 4

7 Representations Array: Constant time index for read, write Linear time insertion/deletion Linked List: Linear time indexing Constant access to front, maybe back Constant time insertion/deletion* Assorted others Regardless: swapping values is always O(1) 5

8 Selection Sort

9 Selection Sort Selection Sort is arguably the most basic sort (competes with bubble sort) Basic idea: find lowest, move to front; repeat Repeated Exhaustive Search 6

10 Selection Sort Pseudocode SelectionSort(A) for i 1.. A 1: min i for j i A : if A[j] < A[min]: min j A[i] A[min] 7

11 Selection Sort Animation Selection Sort Animation 8

12 Analyzing Selection Sort How many comparisons does selection sort make? n n 1 O(n 2 ) How many swaps? i=1 j=i+1 Is it stable? n 1 O(n) i=1 9

13 Bubble Sort Idea: local sort global sort Sequence Sorting (Local) Instance: A sequence s of elements drawn from a total-ordered set S (with an operation ) Problem: Find a sequence t drawn from S s.t. t perm(s) i 2.. S, i < j t[i 1] t[i] Local sort provably equivalent to global sort 10

14 Bubble Sort Animation Bubble Sort Animation 11

15 Bubble Sort Pseudocode BubbleSort(A) for i 1.. A : for j 1.. A i: if A[j + 1] < A[j]: A[j + 1] A[j] 12

16 Analyzing Bubble Sort Every comparison is potentially a swap: n i=1 j=i+1 n 1 O(n 2 ) Correctness: after a full inner loop, ith highest elements are correctly sorted Stability: equal valued elements will not be swapped 13

17 Insertion Sort

18 Insertion Sort Insertion Sort is the R&C approach to sort Basic Idea: sort first element, then first and second, then first through third, etc. Called insertion because we keep a growing sorted section and insert elements into it 14

19 Insertion Sort Pseudocode InsertionSort(A) for i 2.. A : v A[i] for j i..1: if j > 1 and v < A[j 1]: A[j] A[j 1] else: A[j] v 15

20 Animation Insertion Sort Animation 16

21 Analyzing Insertion Sort Let n = A Correctness: if A[1 : i] was sorted before iteration i, A[1 : i 1] is sorted after. A[1 : 1] is sorted by definition, so by induction, after n iterations, the array is sorted. Comparisons: ith element compared to at most i 1 elements. n i=1 i 1 = n2 + n 2 O(n 2 ) n = n2 n 2 17

22 More Analysis Swaps: worst case, entire sorted segment needs to be shifted (same as comparisons, O(n 2 ). Stable: yes. 18

23 Non-Comparison Sorts

24 NCS Comparison Sorts: algorithms that work for any totally ordered domain, because they compare using only the domain s. Non-comparison sorts use some other information about the contents of the array, often limiting them to specific domain 19

25 Bucket Sort The various hash-based algorithms from last week s PotD generalize to bucket sort. Idea: create constant sized buckets for contiguous chunks of array elements, then sort within buckets with a comparison sort. 20

26 Bucket Sort Animation Bucket Sort Animation 21

27 Bucket Sort Analysis Worst case complexity: O(n 2 ) (when most elements are in one bucket) Best case complexity: O(n + k), where k is the number of buckets (Why? ko( n k ) O(n + k)) Average case: assuming uniform distribution, also O(n + k) Worst case space: O(nk) 22

28 Problem of the Day Problem of the Day: Sorting AP Integers Arbitrary-Precision integers have an unbounded size We will represent AP integers as arrays of bits in binary in big-endian order. So when a =3 and a[0]=1 and a[1]=1 and a[2]=0 we re looking at a 6. Give an algorithm for comparing two AP integers, and give its big-o Given this, describe the big-o complexity of insertion sort of an array B of AP integers 23

29 Radix Sort What if we sort buckets with another bucket sort? One version: bucket by least significant digit, then next least, etc. 24

30 Radix Sort Pseudocode LSDRadix(A) d = find_longest_length(a) for i d..1: A bit_bucket_sort(a, d) 25

31 Radix Sort Animation Radix Sort Animation 26

32 Radix Sort Analysis Let d be the number of digits in the longest element and n be the number of elements. Finding longest length: O(n + d) Each pass: O(n) 27

33 Sorting Cases

34 Best, Worst, Average Consider selection, bubble, and insertion sort when a list is already sorted. Do they change? What if they re partially sorted? 28

35 R&C and Binary Search

36 Binary Search Classic: search an ordered list better than exhaustive! Search Instance: a sequence S and element s Problem: find an index i s.t. S[i] = s iff s S 29

37 Binary Search Pseudocode BinarySearch(A, x) m A /2 if A[m] = x: return m if A = 1: return false if x < A[m]: return BinarySearch(A[1 : m], x) if A[m] < x: return BinarySearch(A[m + 1 : A ], x) 30

38 Binary Search Analysis Classification: R&C by a constant factor We know it s O(log n) - is that true of all constant factor reductions? 31

39 Master Theorem Recurrence relationship: T(n) = at( n b ) + Θ(nd ) If a < b d : Θ(n d ) If a = b d : Θ(n d log d) If a > b d : Θ(n log b a ) 32

40 End Next time: more Master Theorem (and proof). Divide and Conquer Sorts! And other D&C. 33

41 References i