data structures and algorithms lecture 6

Transcription

1 data structures and algorithms lecture 6

2 overview lower bound counting sort radix sort stacks queues material

3 question we have seen for example insertion sort: worst-case in O(n 2 ) merge sort: worst-case O(n log n) heapsort: worst-case O(n log n) quicksort: on average in O(n log n) is O(n log n) the best worst-case that we have?

4 comparison based the crucial tests are a < b, a b, a = b, a b, a > b which are all assumed to be decidable in elementary time usually we assume a total order

5 decision tree we assume all elements of an input array to be different we retrict attention to tests of the form a b we consider a sorting algorithm and an input array of length n we consider a decision tree an internal node is labeled i : j with i, j indices from the input array a leaf corresponds to a permutation of {1,....n} a left subtree contains further comparisons if on the node is true a right rubtree contains further comparisons if on the node is false

6 decision tree: example

7 decision tree: observations for every comparison based sorting algorithm, for every input size n there is a decision tree a path represents an execution of the algorithm running time of an execution is length of the path modelling the execution worst-case running time is height of the decision tree every possible permutation of total n! must occur every permutation must be reachable by a path from the root

8 lowerbound on worst-case running time: theorem we have at least n! and at most 2 h leaves n! 2 h, so h log n!, so omitting ceilings and/or floors: h log(n!) = log( n 2... n) log( n 2... n 2 ) = log(( n 2 ) n 2 ) = n 2 log( n 2 ) hence h Ω(n log n) (can also be done using Stirling s formula)

9 summary: results Theorem: any comparison-based sorting algorithm is in Ω(n log n) Consequence: heapsort and merge sort are asymptotically optimal

10 questions puzzle from Knuth can we sort 5 elements in log 5! = 7 comparisons? what is the smallest depth of a leaf in a decision tree for some sorting algorithms for an input of size n? (for how many leafs can we achieve this depth?)

11 overview lower bound counting sort radix sort stacks queues material

12 lower bound asymptotic lower bound on worst-case running time of comparison based sorting algorithms is Ω(n log n) we will see some (not comparison based) linear sorting algorithms with a better lower bound

13 counting sort assumption: numbers in input come from fixed range {0,..., k}. algorithm idea: count the number of occurrences of each i from 0 to k time complexity: in Θ(n + k) for a input-array of length n drawback: fixed range, and requires additional arrays of length m

14 counting sort: pseudo-code input array A, output array B, range from 0 up to k Algorithm countingsort(a, B, k): new array C[0... k] for i := 0 to k do C[i] := 0 for j := 1 to A.length do C[A[j]] := C[A[j]] + 1 for i := 1 to k do C[i] := C[i] + C[i 1] for j := A.length downto 1 do B[C[A[j]]] := A[j] C[A[j]] := C[A[j]] 1

15 lexicographic ordering like in a dictionary for vectors: (x 1,..., x d ) < (y 1,..., y d ) if (x 1 < y 1 ) or ((x 1 = y 1 ) and (x 2,..., x n ) < (y 2,..., y n ))

16 radix sort for card-sorting machine

17 radix sort: example (7, 4, 6) (5, 1, 5) (2, 0, 6) (5, 1, 4) (2, 1, 4) (5, 1, 4) (2, 1, 4) (5, 1, 5) (7, 4, 6) (2, 0, 6) dimension 3 (2, 0, 6) (5, 1, 4) (2, 1, 4) (5, 1, 5) (7, 4, 6) dimension 2 (2, 0, 6) (2, 1, 4) (5, 1, 4) (5, 1, 5) (7, 4, 6) dimension 1

18 radix sort: pseudo-code sort per dimension using a stable sorting algorithm Algorithm radixsort(a, d): for i := 1 to d do use stable sort on digit d 1 is the lowest-order digit and d is the highest-order digit

19 when using wrong order not correct then goes to

20 radix sort: time complexity if the stable sorting algorithm (counting) per dimension is in Θ(n + k) then radix sort is in Θ(d(n + k)) discussion: is radex sort preferable to comparison-based sorting?

21 radix sort: application sort n integers in range {0,..., k} in representation with d digits use radix sort with dimension d example: range {0,..., 8} in representation with basis 3: 00, 01, 02, 10, 11, 12, 20, 21, 22

22 overview lower bound counting sort radix sort stacks queues material

23 data structures data structure: a systemactic way of storing and organizing data in a computer so that it can be used efficiently different data structures for different applications

24 data structures for sets of information where is x? add x remove x what is the smallest? what is the largest? given x, what is the next one? given x, what is the previous one?

25 abstract data type an abstract data type (ADT) specifies what are the operations and what are the errors / exceptions

26 stacks: properties LIFO: last-in first-out examples: stack of plates, memory of the undo button in an editor any data can be on a stack operations: push an element onto the stack pop the top element from the stack and return it example: 1 2 pop() returns push(3)

27 other example: method stack in the Java Virtual Machine active methods, with local variables and a program counter main() { int i = 5; bar foo(i); PC = 1 } m = 6 foo(int j) { int k; k = j+1; bar(k); } bar(int m) {... } foo PC = 3 j = 5 k = 6 main PC = 2 i = 5

28 ADT for stacks main operations: push(s, o) pushes element o onto the top of the stack S pop(s) removes the top element from the stap and returns it some auxiliary operations: size(s) returns the number of elements of the stack isempty(s) returns a boolean indicating whether the stack is empty top(s) returns the top element of the stack (without removing it) exceptions or errors for: calling pop or top to an empty stack

29 stacks: implementation using an array the maximum size of the stack is determined beforehand elements are added from left to right in the array if the stack is empty then S.top = 0 below S.top = 6 for overflow we need an exception (which is not in the ADT) S

30 stacks: pseudo-code for push N maximum size of the stack Algorithm push(s, o): if S.top = N then throw FullStackException else S.top := S.top + 1 S[S.top] := o

31 stacks: pseudo-code for pop Algorithm pop(s): if isempty(s) then throw EmptyStackException else o := S[S.top] S.top := S.top 1 return o

32 stacks: varia question: give pseudo-code for size(s), isempty(s), top(s) all operations in O(1) (why?) maximum size is determined beforehand (not according to ADT) note the errors / exceptions

33 overview lower bound counting sort radix sort stacks queues material

34 queues: properties queues will be discussed also in lecture 7! FIFO: first-in first-out examples: post office, printer queue any data can be on a queue operations: enqueue an element at the tail of the queue dequeue the element at the head and return it example: 1 2 dequeue() returns enqueue(3)

35 ADT for queues main operations: enqueue(q, o) adds element o at the tail of the queue dequeue(q) removes element at the head and returns it auxiliary operations: size(q) returns the number of elements of the queue isempty(q) returns a boolean indicating whether the queue is empty front(q) returns the head element of the queue (without removing it) exceptions / errors: call of dequeue() or front() to an empty queue

36 queues: implementation using circular array the maximum size of the queue is determined beforehand Q.head indicates position of the head (front) Q.tail indicates position right of the tail (rear) if the queue is empty then Q.head = Q.tail initially, Q.head = Q.tail = 1 in case of overflow, we need an exception

37 queues: implementation using circular array normal configuration Q h r wrapped configuration Q r h Q empty queue (head == tail) head tail

38 pseudo-code for enqueue size of the array fixed elements are at indices Q.head, Q.head + 1,..., Q.tail 1 check on overflow omitted Algorithm enqueue(q, x): Q[Q.tail] := x if Q.tail = Q.length then Q.tail := 1 else Q.taill := Q.tail + 1

39 pseudo-code for dequeue size of the array N = Q.length fixed check on empty queue omitted Algorithm dequeue(q): x := Q[Q.head] if Q.head = Q.length then Q.head := 1 else Q.head := Q.head + 1

40 queues: varia give pseudo-code for size(q), isempty(q), front(q) operations are all in O(1) maximum size determined beforehand (not according to ADT) note the exceptions / errors

41 overview lower bound counting sort radix sort stacks queues material

42 material book 8, 10.1 Donald Knuth: The art of computer programming, volume Sorting and Searching paper by Knuth on big-oh