Suggested Study Strategy

Size: px

Start display at page:

Download "Suggested Study Strategy"

Owen Byrd
5 years ago
Views:

1 Final Exam Thursday, 7 August 2014,19:00 22:00 Closed Book Will cover whole course, with emphasis on material after midterm (hash tables, binary search trees, sorting, graphs)

2 Suggested Study Strategy Review and understand the slides. Read the textbook, especially where concepts and methods are not yet clear to you. Do all of the practice problems provided. Do extra practice problems from the textbook. Review the midterm and solutions for practice writing this kind of exam. Practice writing clear, succinct pseudocode! Review the assignments See me or one of the TAs if there is anything that is still not clear.

3 Assistance Regular office hours will not be held You may me by / Skype

4 End of Term Review

5 1. Binary Search Trees 2. Sorting 3. Graphs Summary of Topics

6 Topic 1. Binary Search Trees

7 Binary Search Trees Insertion Deletion AVL Trees Splay Trees

8 Binary Search Trees A binary search tree is a binary tree storing key-value entries at its internal nodes and satisfying the following property: Let u, v, and w be three nodes such that u is in the left subtree of v and w is in the right subtree of v. We have key(u) key(v) key(w) The textbook assumes that external nodes are placeholders : they do not store entries (makes algorithms a little simpler) An inorder traversal of a binary search trees visits the keys in increasing order Binary search trees are ideal for maps or dictionaries with ordered keys

9 Binary Search Tree All nodes in left subtree Any node All nodes in right subtree

10 Search: Define Step Cut sub-tree in half. Determine which half the key would be in. Keep that half. key If key < root, then key is in left half. If key = root, then key is found If key > root, then key is in right half.

11 Insertion (For Dictionary) To perform operation insert(k, o), we search for key k (using TreeSearch) Suppose k is not already in the tree, and let w be the leaf reached by the search We insert k at node w and expand w into an internal node Example: insert 5 w < > > w 9

12 Insertion Suppose k is already in the tree, at node v. We continue the downward search through v, and let w be the leaf reached by the search Note that it would be correct to go either left or right at v. We go left by convention. We insert k at node w and expand w into an internal node Example: insert 6 2 < > > 9 2 w w 6 9

13 Deletion To perform operation remove(k), we search for key k Suppose key k is in the tree, and let v be the node storing k If node v has a leaf child w, we remove v and w from the tree with operation removeexternal(w), which removes w and its parent Example: remove 4 < w > v

14 Deletion (cont.) Now consider the case where the key k to be removed is stored at a node v whose children are both internal we find the internal node w that follows v in an inorder traversal we copy the entry stored at w into node v we remove node w and its left child z (which must be a leaf) by means of operation removeexternal(z) Example: remove v 1 5 v z w

15 Performance Consider a dictionary with n items implemented by means of a binary search tree of height h the space used is O(n) methods find, insert and remove take O(h) time The height h is O(n) in the worst case and O(log n) in the best case It is thus worthwhile to balance the tree (next topic)!

16 AVL Trees AVL trees are balanced. An AVL Tree is a binary search tree in which the heights of siblings can differ by at most height

17 Height of an AVL Tree Claim: The height of an AVL tree storing n keys is O(log n).

18 Insertion Imbalance may occur at any ancestor of the inserted node. height = 3 7 height = Insert(2) 3 4 Problem!

19 Insertion: Rebalancing Strategy Step 1: Search Starting at the inserted node, traverse toward the root until an imbalance is discovered. height = Problem!

20 Insertion: Rebalancing Strategy Step 2: Repair The repair strategy is called trinode restructuring. 3 nodes x, y and z are distinguished: z = the parent of the high sibling y = the high sibling x = the high child of the high sibling We can now think of the subtree rooted at z as consisting of these 3 nodes plus their 4 subtrees height = Problem!

21 Insertion: Trinode Restructuring Example Note that y is the middle value. height = h z h-1 y Restructure y h-1 h-3 T 3 h-2 x h-2 z h-2 x h-3 T 2 h-3 h-3 T 0 T 1 T 2 T 3 T 0 T 1 one is h-3 & one is h-4 one is h-3 & one is h-4

22 Insertion: Trinode Restructuring - 4 Cases There are 4 different possible relationships between the three nodes x, y and z before restructuring: x y z z y x y x z z x y height = h z height = h z height = h z height = h z y h-1 h-3 T 3 y h-3 h-1 T 0 y h-1 h-3 T 3 y h-3 h-1 T 0 h-2 x h-3 h-3 x h-2 h-3 h-2 x x h-2 h-3 T 2 T 1 T 0 T 3 T 0 T 1 T 2 T 3 T 1 T 2 T 1 T 2 one is h-3 & one is h- 4 one is h-3 & one is h- 4 one is h-3 & one is h- 4 one is h-3 & one is h- 4

23 Insertion: Trinode Restructuring - The Whole Tree Do we have to repeat this process further up the tree? No! The tree was balanced before the insertion. Insertion raised the height of the subtree by 1. Rebalancing lowered the height of the subtree by 1. Thus the whole tree is still balanced. height = h z h-1 y Restructure h-1 y h-3 T 3 h-2 x h-2 z h-2 x h-3 T 2 T 0 T 1 h-3 T 2 h-3 T 3 T 0 T 1 one is h-3 & one is h-4 one is h-3 & one is h-4

24 Removal Imbalance may occur at an ancestor of the removed node. height = 3 7 height = Remove(8) 2 4 Problem!

25 Step 1: Search Removal: Rebalancing Strategy Starting at the location of the removed node, traverse toward the root until an imbalance is discovered. height = Problem!

26 Removal: Rebalancing Strategy Step 2: Repair We again use trinode restructuring. 3 nodes x, y and z are distinguished: height = 3 7 z = the parent of the high sibling y = the high sibling x = the high child of the high sibling (if children are equally high, keep chain linear) Problem!

Removal: Rebalancing Strategy Step 2: Repair The idea is to rearrange these 3 nodes so that the middle value becomes the root and the other two becomes its children.

Note that z must be either bigger than both x and y or smaller than both x and y. Thus either x or y is made the root of this subtree, and z is lowered by 1.

27 Removal: Rebalancing Strategy Step 2: Repair The idea is to rearrange these 3 nodes so that the middle value becomes the root and the other two becomes its children. Thus the linear grandparent parent child structure becomes a triangular parent two children structure. Note that z must be either bigger than both x and y or smaller than both x and y. Thus either x or y is made the root of this subtree, and z is lowered by 1. Then the subtrees T 0 T 3 are attached at the appropriate places. Although the subtrees T 0 T 3 can differ in height by up to 2, after restructuring, sibling subtrees will differ by at most 1. h-2 x T 0 T 1 h-3 or h-3 & h- 4 height = h y T 2 z h-1 h-3 h-2 or h-3 T 3

Removal: Trinode Restructuring - 4 Cases There are 4 different possible relationships between the three nodes x, y and z before restructuring: x y z z y x y x z z x y height = h z height = h z height

28 Removal: Trinode Restructuring - 4 Cases There are 4 different possible relationships between the three nodes x, y and z before restructuring: x y z z y x y x z z x y height = h z height = h z height = h z height = h z y h-1 h-3 T 3 y h-3 h-1 T 0 y h-1 h-3 T 3 y h-3 h-1 T 0 h-2 x h-2 or h- 3 T 2 T 1 h-2 or h- 3 x h-2 h-2 or h- 3 T 0 h-2 x x h-2 T 3 h-2 or h- 3 T 0 T 1 T 2 T 3 T 1 T 2 T 1 T 2 h-3 or h-3 & h-4 h-3 or h-3 & h-4 h-3 or h-3 & h-4 h-3 or h-3 & h-4

29 Removal: Trinode Restructuring - Case 1 Note that y is the middle value. height = h y z h-1 h-3 T 3 Restructure h-2 x h or h-1 y h-1 or h-2 z h-2 x T 0 T 1 h-2 or h-3 T 2 T 0 T 1 T 2 T 3 h-3 or h-3 & h- 4 h-2 or h-3 h-3 h-3 or h-3 & h- 4

30 Step 2: Repair Removal: Rebalancing Strategy Unfortunately, trinode restructuring may reduce the height of the subtree, causing another imbalance further up the tree. Thus this search and repair process must be repeated until we reach the root.

31 Topic 2. Sorting

32 Comparison Sorting Selection Sort Bubble Sort Insertion Sort Merge Sort Heap Sort Quick Sort Linear Sorting Counting Sort Radix Sort Bucket Sort Sorting Algorithms

33 Comparison Sorts Comparison Sort algorithms sort the input by successive comparison of pairs of input elements. Comparison Sort algorithms are very general: they make no assumptions about the values of the input elements e.g.,3 11?

34 Sorting Algorithms and Memory Some algorithms sort by swapping elements within the input array Such algorithms are said to sort in place, and require only O(1) additional memory. Other algorithms require allocation of an output array into which values are copied. These algorithms do not sort in place, and require O(n) additional memory swap

35 Stable Sort A sorting algorithm is said to be stable if the ordering of identical keys in the input is preserved in the output. The stable sort property is important, for example, when entries with identical keys are already ordered by another criterion. (Remember that stored with each key is a record containing some useful information.)

36 Selection Sort Selection Sort operates by first finding the smallest element in the input list, and moving it to the output list. It then finds the next smallest value and does the same. It continues in this way until all the input elements have been selected and placed in the output list in the correct order. Note that every selection requires a search through the input list. Thus the algorithm has a nested loop structure Selection Sort Example

37 Bubble Sort Bubble Sort operates by successively comparing adjacent elements, swapping them if they are out of order. At the end of the first pass, the largest element is in the correct position. A total of n passes are required to sort the entire array. Thus bubble sort also has a nested loop structure Bubble Sort Example

38 Example: Insertion Sort

39 Merge Sort Get one friend to sort the first half Split Set into Two (no real work) Get one friend to sort the second half. 25,31,52,88,98 14,23,30,62,79

40 Merge Sort Merge two sorted lists into one 25,31,52,88,98 14,23,30,62,79 14,23,25,30,31,52,62,79,88,98

41 Analysis of Merge-Sort The height h of the merge-sort tree is O(log n) at each recursive call we divide in half the sequence, The overall amount or work done at the nodes of depth i is O(n) we partition and merge 2 i sequences of size n/2 i we make 2 i+1 recursive calls Thus, the total running time of merge-sort is O(n log n) depth #seqs size 0 1 n T(n) = 2T(n / 2) + O(n) 1 2 n/2 i 2 i n/2 i

42 Heap-Sort Algorithm Build an array-based (max) heap Iteratively call removemax() to extract the keys in descending order Store the keys as they are extracted in the unused tail portion of the array

43 Heap-Sort Running Time The heap can be built bottom-up in O(n) time Extraction of the ith element takes O(log(n - i+1)) time (for downheaping) Thus total run time is T(n) = O(n) + log(n - i + 1) n å i =1 = O(n) + log i n å i =1 O(n) + logn n å i =1 = O(nlogn)

44 Quick-Sort Quick-sort is a divide-andconquer algorithm: Divide: pick a random element x (called a pivot) and partition S into x L elements less than x E elements equal to x x G elements greater than x L E G Recur: Quick-sort L and G Conquer: join L, E and G x

45 The Quick-Sort Algorithm Algorithm QuickSort(S) if S.size() > 1 (L, E, G) = Partition(S) QuickSort(L) QuickSort(G) S = (L, E, G)

46 In-Place Quick-Sort Note: Use the lecture slides here instead of the textbook implementation (Section ) Partition set into two using randomly chosen pivot

47 Maintaining Loop Invariant

48 The In-Place Quick-Sort Algorithm Algorithm QuickSort(A, p, r) if p < r q = Partition(A, p, r) QuickSort(A, p, q - 1) QuickSort(A, q + 1, r)

49 Summary of Comparison Sorts Algorithm Best Case Worst Case Average Case In Place Stable Selection n 2 n 2 Yes Yes Comments Bubble n n 2 Yes Yes Insertion n n 2 Yes Yes Good if often almost sorted Merge n log n n log n No Yes Good for very large datasets that require swapping to disk Heap n log n n log n Yes No Best if guaranteed n log n required Quick n log n n 2 n log n Yes No Usually fastest in practice

50 Comparison Sort: Decision Trees For a 3-element array, there are 6 external nodes. For an n-element array, there are n! external nodes.

51 Comparison Sort To store n! external nodes, a decision tree must have a height of at least é ê logn! ù ú Worst-case time is equal to the height of the binary decision tree. Thus T(n) ÎW( logn! ) wherelogn! = n å log i ³ log ê ë n / 2ú û i =1 Thus T(n) ÎW(nlogn) ê ë n/ 2ú û å i =1 ÎW(nlogn) Thus MergeSort & HeapSort are asymptotically optimal.

52 Linear Sorts? Comparison sorts are very general, but are ( nlog n) Faster sorting may be possible if we can constrain the nature of the input.

53 CountingSort Input: Output: Index: Value v: Location of next record with digit v Algorithm: Go through the records in order putting them where they go.

54 CountingSort Input: Output: 0 1 Index: Value v: Location of next record with digit v Algorithm: Go through the records in order putting them where they go.

55 RadixSort Sort wrt which digit first? The least significant Sort wrt which digit Second? The next least significant Is sorted wrt least sig. 2 digits.

56 RadixSort i+1 Is sorted wrt first i digits. Sort wrt i+1st digit Is sorted wrt first i+1 digits. These are in the correct order because sorted wrt high order digit

57 RadixSort i+1 Is sorted wrt first i digits. Sort wrt i+1st digit Is sorted wrt first i+1 digits. These are in the correct order because was sorted & stable sort left sorted

58 Example 3. Bucket Sort Applicable if input is constrained to finite interval, e.g., [0 1). If input is random and uniformly distributed, expected run time is Θ(n).

59 Bucket Sort

60 Topic 3. Graphs

61 Definitions & Properties Implementations Depth-First Search Topological Sort Breadth-First Search Graphs

62 Properties Property 1 v deg(v) = 2 E Proof: each edge is counted twice Property 2 In an undirected graph with no self-loops and no multiple edges E V ( V - 1)/2 Proof: each vertex has degree at most ( V 1) Q: What is the bound for a digraph? A: E V (V -1) Notation V E deg(v) number of vertices number of edges degree of vertex v Example V = 4 E = 6 deg(v) = 3

63 Main Methods of the (Undirected) Graph ADT Vertices and edges are positions store elements Accessor methods endvertices(e): an array of the two endvertices of e opposite(v, e): the vertex opposite to v on e areadjacent(v, w): true iff v and w are adjacent replace(v, x): replace element at vertex v with x replace(e, x): replace element at edge e with x Update methods insertvertex(o): insert a vertex storing element o insertedge(v, w, o): insert an edge (v,w) storing element o removevertex(v): remove vertex v (and its incident edges) removeedge(e): remove edge e Iterator methods incidentedges(v): edges incident to v vertices(): all vertices in the graph edges(): all edges in the graph

64 Running Time of Graph Algorithms Running time often a function of both V and E. For convenience, we sometimes drop the. in asymptotic notation, e.g. O(V+E).

65 Implementing a Graph (Simplified) Space complexity: Time to find all neighbours of vertex u : Adjacency List ( V + E) (degree( u)) Adjacency Matrix 2 ( V ) ( V ) Time to determine if ( u, v) E : (degree( u)) (1)

66 DFS Example on Undirected Graph A A unexplored being explored A A finished unexplored edge B D E discovery edge back edge C A A B D E B D E C C

67 Example (cont.) A A B D E B D E C C A A B D E B D E C C

68 DFS Algorithm Pattern DFS(G) Precondition: G is a graph Postcondition: all vertices in G have been visited for each vertex u ÎV[G] color[u] = BLACK //initialize vertex for each vertex u ÎV[G] if color[u] = BLACK //as yet unexplored DFS-Visit(u) total work = q(v )

69 DFS Algorithm Pattern DFS-Visit (u) Precondition: vertex u is undiscovered Postcondition: all vertices reachable from u have been processed colour[u] RED for each v ÎAdj[u] //explore edge (u,v) if color[v] = BLACK DFS-Visit(v) colour[u] GRAY Thus running time = q(v + E) (assuming adjacency list structure) total work å = Adj[v] = q(e) v ÎV

70 Other Variants of Depth-First Search The DFS Pattern can also be used to Compute a forest of spanning trees (one for each call to DFSvisit) encoded in a predecessor list π[u] Label edges in the graph according to their role in the search (see textbook) Tree edges, traversed to an undiscovered vertex Forward edges, traversed to a descendent vertex on the current spanning tree Back edges, traversed to an ancestor vertex on the current spanning tree Cross edges, traversed to a vertex that has already been discovered, but is not an ancestor or a descendent

71 DAGs and Topological Ordering A directed acyclic graph (DAG) is a digraph that has no directed cycles A topological ordering of a digraph is a numbering D E v 1,, v n of the vertices such that for every edge (v i, v j ), we have i < j Example: in a task scheduling digraph, a topological ordering is a task sequence that satisfies the precedence constraints Theorem A digraph admits a topological ordering if and only if it is a DAG v 2 v 1 B A B A C D C DAG G v 4 v 5 v 3 E Topological ordering of G

72 b c d e f g a Linear Order h i j k l Alg: DFS Found Not Handled Stack f g e d.. f

73 b c d e f g a Linear Order h i j k l Alg: DFS Found Not Handled Stack When node is popped off stack, insert at front of linearly-ordered to do list. Linear Order:.. f l g e d

74 b c d e f g a Linear Order h i j k l Alg: DFS Found Not Handled Stack g e d Linear Order: l,f

75 BFS Example A A A undiscovered discovered (on Queue) finished unexplored edge L 1 B A C D discovery edge cross edge E F L 0 A L 0 A L 1 B C D L 1 B C D E F E F

76 BFS Example (cont.) L 0 A L 0 A L 1 B C D L 1 B C D E F L 2 E F L 0 A L 0 A L 1 B C D L 1 B C D L 2 E F L 2 E F

77 BFS Example (cont.) L 0 A L 0 A L 1 B C D L 1 B C D L 2 E F L 2 E F L 0 A L 1 B C D L 2 E F

78 Analysis Setting/getting a vertex/edge label takes O(1) time Each vertex is labeled three times once as BLACK (undiscovered) once as RED (discovered, on queue) once as GRAY (finished) Each edge is considered twice (for an undirected graph) Thus BFS runs in O( V + E ) time provided the graph is represented by an adjacency list structure

79 BFS Algorithm with Distances and Predecessors BFS(G,s) Precondition: G is a graph, s is a vertex in G Postcondition: d[u] = shortest distance d[u] and p[u] = predecessor of u on shortest paths from s to each vertex u in G for each vertex u ÎV[G] d[u] p[u] null color[u] = BLACK //initialize vertex colour[s] RED d[s] 0 Q.enqueue(s) while Q ¹ Æ u Q.dequeue() for each v ÎAdj[u] //explore edge (u,v) if color[v] = BLACK colour[v] RED d[v] d[u] + 1 p[v] u colour[u] GRAY Q.enqueue(v)

80 1. Binary Search Trees 2. Sorting 3. Graphs Summary of Topics

Final Exam. CSE 2011 Prof. J. Elder Last Updated: :41 PM

Final Exam. CSE 2011 Prof. J. Elder Last Updated: :41 PM Final Exam Ø Tue, 17 Apr 2012 19:00 22:00 LAS B Ø Closed Book Ø Format similar to midterm Ø Will cover whole course, with emphasis on material after midterm (maps, hashing, binary search trees, sorting,