CSCI 04 Binary Trees / Priority Queues / Heaps Mark Redekopp Michael Crowley
Trees Definition: A connected, acyclic (no cycles) graph with: A root node, r, that has 0 or more subtrees Exactly one path between any two nodes In general: Nodes have exactly one parent (except for the root) and 0 or more children N-ary tree Tree where each node has at most n children Binary tree = N-ary Tree with n=2 Left child root siblings parent Right child Ancestor Terms: Parent(i): Node directly above node i Child(i): Node directly below node i Siblings: Children of the same parent Root: Only node with no parent Leaf: Node with 0 children Height: Length of largest path from root to any leaf Subtree(n): Tree rooted at node n Ancestor(n): Any node on the path from n to the root Descendant(n): Any node in the subtree rooted at n Leaf subtree Descendant 2
Trees Full binary tree: Binary tree, T, where If height h>0 and both subtrees are full binary trees of height, h- If height h==0, then it is full by definition (Tree where all leaves are at level h and all other nodes have 2 children) Complete binary tree Tree where levels 0 to h- are full and level h is filled from left to right Balanced binary tree Tree where subtrees from any node differ in height by at most Full Complete, but not full Full DAPS, 6 th Ed. Figure 5-8 Complete 3
Tree Height A full binary tree of n nodes has height, log 2 (n + ) This implies the minimum height of any tree with n nodes is log 2 (n + ) The maximum height of a tree with n nodes is, n 5 nodes => height log 2 (6) = 4 5 nodes => height = 5 4
Tree Traversals A traversal iterates over all nodes of the tree Usually using a depth-first, recursive approach Three general traversal orderings Pre-order [Process root then visit subtrees] In-order [Visit left subtree, process root, visit right subtree] Post-order [Visit left subtree, visit right subtree, process root] 20 60 80 Preorder(TreeNode* t) { if t == NULL return process(t) // print val. Preorder(t->left) Preorder(t->right) } 60 20 0 30 25 50 80 Inorder(TreeNode* t) { if t == NULL return Inorder(t->left) process(t) // print val. Inorder(t->right) } 0 20 25 30 50 60 80 0 30 25 50 Postorder(TreeNode* t) { if t == NULL return Postorder(t->left) Postorder(t->right) process(t) // print val. } 0 25 50 30 20 80 60 5
Array-based and Link-based BINARY TREE IMPLEMENTATIONS 6
Array-Based Complete Binary Tree Binary tree that is complete (i.e. only the lowest-level contains empty locations and items added left to right) can be stored nicely in an array (let s say it starts at index and index 0 is empty) Can you find the mathematical relation for finding the index of node i's parent, left, and right child? Parent(i) = Left_child(i) = Right_child(i) = 0 4 5 6 8 0 8 em 8 35 4 0 28 3 36 43 6 35 4 0 28 3 36 43 6 parent(5) = Left_child(5) = Right_child(5) =
Array-Based Complete Binary Tree Binary tree that is complete (i.e. only the lowest-level contains empty locations and items added left to right) can be stored nicely in an array (let s say it starts at index and index 0 is empty) Can you find the mathematical relation for finding node i's parent, left, and right child? Parent(i) = i/2 Left_child(i) = 2*i Right_child(i) = 2*i + 0 4 em 8 5 6 8 35 4 0 28 3 0 36 43 6 8 parent(5) = 5/2 = 2 Left_child(5) = 2*5 = 0 Right_child(5) = 2*5+ = 35 4 0 28 3 36 43 6 Non-complete binary trees require much more bookeeping to store in arrays usually link-based approaches are preferred 8
Link-Based Approaches Much like a linked list but now with two pointers per Item Use NULL pointers to indicate no child Dynamically allocate and free items when you add/remove them #include<iostream> using namespace std; template <typename T> struct BTItem { T val; BTItem<T>* left,right; BTItem<T>* parent; }; // Bin. Search Tree template <typename T> class LinkedBST : public BST { public: BTree(); ~BTree(); void add(const T& v);... private: BTItem<T>* root_; }; BTItem<T> blueprint: Item<T>* left Item<T>* parent T val Item<T>* right class LinkedBST: 0x0 root_
Link-Based Approaches Add(5) Add(6) class LinkedBST: 0x0 root_ Add() 2 root_ 3 root_ 4 root_ 0xc0 Left Parent val right 0
Link-Based Approaches Add(5) Add(6) Add() class LinkedBST: 0x0 root_ 2 0xc0 root_ 3 4 0xc0 root_ 0xc0 root_ 0xc0 Left NULL parent NULL val 5 right NULL 0xc0 Left NULL parent NULL val 5 right 0x2a0 0xc0 Left NULL parent NULL val 5 right 0x2a0 0x2a0 Left NULL parent 0xc0 val 6 right NULL 0x2a0 Left NULL parent 0xc0 val 6 right 0x0e0 0x0e0 Left NULL parent 0x2a0 val right NULL
PRIORITY QUEUES 2
Traditional Queue Traditional Queues Accesses/orders items based on POSITION (front/back) Did not care about item's VALUE Priority Queue (pop_front) (push_back) 5 33 62 8 4 Orders items based on VALUE Either minimum or maximum Traditional Queue Leads to a "sorted" list Think hospital ER, air-traffic control, etc. Items arrive in some arbitrary order (push) 4 When removing an item, we always want the minimum or maximum depending on the implementation (pop) 5 33 62 8 Heaps that always yield the min value are called min-heaps Priority Queue Heaps that always yield the max value are called max-heaps 3
Priority Queue What member functions does a Priority Queue have? push(item) Add an item to the appropriate location of the PQ top() Return the min./max. value pop() - Remove the front (min. or max) item from the PQ size() - Number of items in the PQ empty() - Check if the PQ is empty [Optional]: changepriority(item, new_priority) Useful in many algorithms (especially AI and search algorithms) Implementations Priority is based upon intrinsic data-type being stored (i.e. operator<() of type T) Priority is passed in separately from data type, T, Allows the same object to have different priorities based on the programmer's desire (i.e. same object can be assigned different priorities) (pop) (push) (top) 5 33 62 8 Priority Queue 4 4
STL Priority Queue Implements a max-pq by default Operations: push(new_item) pop(): removes but does not return top item top() return top item (item at back/end of the container) size() empty() http://www.cplusplus.com/refer ence/stl/priority_queue/push/ // priority_queue::push/pop #include <iostream> #include <queue> using namespace std; int main () { priority_queue<int> mypq; mypq.push(30); mypq.push(00); mypq.push(25); mypq.push(40); cout << "Popping out elements..."; while (!mypq.empty()) { cout<< " " << mypq.top(); mypq.pop(); } cout<< endl; return 0; } Code here will print 00 40 30 25 5
Priority Queue Efficiency If implemented as a sorted array list Insert() = Top() = Pop() = If implemented as an unsorted array list Insert() = Top() = Pop() = 6
Priority Queue Efficiency If implemented as a sorted array list [Use back of array as location of top element] Insert() = O(n) Top() = O() Pop() = O() If implemented as an unsorted array list Insert() = O() Top() = O(n) Pop() = O(n)
STL Priority Queue Template Template that allows type of element, container class, and comparison operation for ordering to be provided First template parameter should be type of element stored Second template parameter should be the container class you want to use to store the items (usually vector<type_of_elem>) Third template parameters should be comparison function object/class that will define the order from first to last in the container // priority_queue::push/pop #include <iostream> #include <queue> using namespace std; int main () { priority_queue<int, vector<int>, greater<int>> mypq; mypq.push(30); mypq.push(00); mypq.push(25); cout<< "Popping out elements..."; while (!mypq.empty()) { cout<< " " << mypq.top(); mypq.pop(); } } Code here will print 25, 30, 00 greater<int> will yield a min-heap less<int> will yield a max-heap Push(30) Push(00) Push(25) 0 30 0 00 0 00 30 2 30 25 Push(n): walk while (item[i] > n), then insert Top(): Return last item Pop(): Remove last item 8
STL Priority Queue Template For user defined classes, must implement operator<() for maxheap or operator>() for min-heap Code here will pop in order: Jane Charlie Bill // priority_queue::push/pop #include <iostream> #include <queue> #include <string> using namespace std; class Item { public: int score; string name; Item(int s, string n) { score = s; name = n;} bool operator>(const Item &lhs, const Item &rhs) const { if(lhs.score > rhs.score) return true; else return false; } }; int main () { priority_queue<item, vector<item>, greater<item> > mypq; Item i(25, Bill ); mypq.push(i); Item i2(5, Jane ); mypq.push(i2); Item i3(0, Charlie ); mypq.push(i3); cout<< "Popping out elements..."; while (!mypq.empty()) { cout<< " " << mypq.top().name; mypq.pop(); } }
HEAPS 20
Heap Data Structure Provides an efficient implementation for a priority queue Can think of heap as a complete binary tree with the property that every parent is less-than (if min-heap) or greater-than (if max-heap) both children But no ordering property between children Minimum/Maximum value is always the top element 8 35 4 0 28 3 36 43 6 25 Min-Heap 2
Heap Operations Push: Add a new item to the heap and modify heap as necessary Pop: Remove min/max item and modify heap as necessary Top: Returns min/max Since heaps are complete binary trees we can use an array/vector as the container template <typename T> class MinHeap { public: virtual void push(const T& item) = 0; virtual T& top() = 0; virtual void pop() = 0; virtual int size() const = 0; virtual bool empty() const = 0; } template <typename T> class ArrayMinHeap : public MinHeap { public: ArrayMinHeap(int init_capacity); ~ArrayMinHeap() void push(const T& item); T& top(); void pop(); int size() const; bool empty() const; private: void heapify(int idx); vector<t> items_; } 22
Array/Vector Storage for Heap Recall: Full binary tree (i.e. only the lowest-level contains empty locations and items added left to right) can be modeled as an array (let s say it starts at index ) where: Parent(i) = i/2 Left_child(p) = 2*p Right_child(p) = 2*p + 0 4 5 6 8 0 em 8 35 4 0 28 3 36 43 6 8 4 5 6 35 4 0 8 0 4 28 3 36 43 6 parent(5) = 5/2 = 2 Left_child(5) = 2*5 = 0 Right_child(5) = 2*5+ = 23
Push Heap Add item to first free location at bottom of tree Recursively promote it up while it is less than its parent Remember valid heap all parents < children so we need to promote it up until that property is satisfied Push_Heap(8) void ArrayMinHeap<T>::push(const T& item) { items_.push_back(item); trickleup(items_.size()-); } void trickleup(int loc) { // could be implemented recursively int parent = loc/2; while(parent >= && items_[loc] < items_[parent] ) { swap(items_[parent], items_[loc]); loc = parent; parent = loc/2; } } 8 8 8 4 5 6 35 4 0 35 4 8 0 4 28 3 36 43 6 25 8 28 3 36 43 6 25 0 8 24
Top() Top() simply needs to return first item T& ArrayMinHeap<T>::top() { if( empty() ) throw(std::out_of_range()); return items_[]; } Top() returns 8 4 5 6 35 4 0 8 0 28 3 36 43 6 25 25
Pop Heap Takes last (greatest) node puts it in the top location and then recursively swaps it for the smallest child until it is in its right place Original void ArrayMinHeap<T>::pop() { items_[] = items_.back(); items_.pop_back() heapify(); // a.k.a. trickledown() } void ArrayMinHeap<T>::heapify(int idx) { if(idx == leaf node) return; int smallerchild = 2*idx; // start w/ left if(right child exists) { int rchild = smallerchild+; if(items_[rchild] < items_[smallerchild]) smallerchild = rchild; } } if(items_[idx] > items_[smallerchild]){ swap(items_[idx], items_[smallerchild]); heapify(smallerchild); } } 25 8 8 8 0 4 5 6 35 4 0 4 5 6 35 4 0 35 4 25 8 0 28 3 36 43 6 25 8 0 2 28 3 36 43 6 28 3 36 43 6 26
Practice Push() Push(23) 28 3 8 4 5 6 8 0 36 35 4 0 28 3 8 2 4 5 6 35 26 24 0 36 43 2 50 Pop() Pop() 4 28 3 8 2 4 5 6 35 26 24 0 36 43 2 50 8 28 3 4 5 6 0 36 8 35 26 24 2
HEAPSORT 28
Using a Heap to Sort If we could make a valid heap out of an arbitrary array, could we use that heap to sort our data? Sure, just call top() and pop() n times You'll get your data out in sorted order How long would that take? n calls to top() and pop() top() = O() pop = O(lg n) Thus total time = O(n * lg n) But how long does it take to convert the array to a valid heap? 0 4 5 6 8 em 28 8 0 35 4 Arbitrary Array 8 0 4 4 5 6 35 28 8 0 4 5 6 8 em 4 0 35 28 8 Array Converted to Valid Heap 0 4 5 6 8 em 0 4 8 28 35 Array after top/popping the heap n times 2
Converting An Array to a Heap If we have two heaps can we unify them with some arbitrary value If we put an arbitrary value in the top spot how can we make a heap? Heapify!! (we did this in pop() ) 0 4 5 6 8 em 28 8 0 35 4 Original Array 28 8 4 5 6 0 35 4 8 Tree View of Array 30
Converting An Array to a Heap To convert an array to a heap we can use the idea of first making heaps of both sub-trees and then combining the sub-trees (a.k.a. semi heaps) into one unified heap by calling heapify() on their parent() First consider all leaf nodes, are they valid heaps if you think of them as the root of a tree? Yes!! So just start at the first non-leaf 0 4 5 6 8 em 28 8 0 35 4 Original Array 8 0 28 8 4 5 6 35 4 Tree View of Array 3
Converting An Array to a Heap First consider all leaf nodes, are they valid heaps if you think of them as the root of a tree? Yes!! So just start at the first non-leaf Heapify(Loc. 4) 8 0 28 8 4 5 6 35 4 Leafs are valid heaps by definition heapify(4) heapify(3) heapify(2) heapify() 8 4 Already in the right order 0 8 4 0 8 3 6 4 Swap 8 & 3 6 4 8 Already a heap 8 4 0 2 35 0 Swap 28 <-> Swap 28 <-> 4 28 4 5 6 35 4 8 8 32
Converting An Array to a Heap Now that we have a valid heap, we can sort by top and popping Can we do it in place? Yes, Break the array into "heap" and "sorted" areas, iteratively adding to the "sorted" area Swap top & last heapify() 0 4 4 5 6 35 28 8 0 4 4 5 6 35 28 8 8 0 4 5 6 8 0 4 5 6 8 em 4 0 35 28 8 em 0 4 35 28 8 Swap top & last heapify() 0 4 4 5 6 0 4 em 8 0 4 35 28 8 5 6 8 35 28 8 0 4 5 6 35 28 0 4 em 0 8 4 4 5 6 8 35 28 33
Converting An Array to a Heap Now that we have a valid heap, we can sort by top and popping Can we do it in place? Yes, Break the array into "heap" and "sorted" areas, iteratively adding to the "sorted" area Swap top & last heapify() 8 0 4 5 6 35 28 0 4 em 28 8 4 4 5 6 8 35 0 8 35 4 4 5 0 4 em 4 8 28 28 5 6 8 35 0 Swap top & last heapify() 8 35 4 28 4 5 0 4 em 35 8 28 5 6 8 4 0 4 35 8 0 4 em 8 28 35 28 5 6 8 4 0 34
Converting An Array to a Heap 4 35 8 0 4 em 8 28 35 28 5 6 8 4 0 0 4 em 35 28 8 5 6 8 4 0 Notice the result is in descending order. How could we make it ascending order? Create a max heap rather than min heap. 35