Greedy Approach: Intro

Transcription

1 Greedy Approach: Intro Applies to optimization problems only Problem solving consists of a series of actions/steps Each action must be 1. Feasible 2. Locally optimal 3. Irrevocable Motivation: If always select best action at a given point, will hopefully result in overall optimal solution Greedy algorithms not guaranteed to produce optimal solutions 1

2 Greedy Approach: Prim s Algorithm The problem: Given n points, connect them in the cheapest way so that there is a path between every pair of points If represent points as nodes in a graph, paths as edges, and costs as edge weights, problem reduces to finding a minimum spanning tree Spanning tree of connected graph Tree that connects all vertices Minimum spanning tree is spanning tree of least cost Exhaustive search problematic: Algorithm grows exponentially Not simple Prim s algorithm more efficient than exhaustive search Basic approach: On each iteration, expand tree constructed so far by adding closest vertex not yet in tree 2

3 Greedy Approach: Prim s Algorithm (2) Algorithm: Alg prim(g) { //Input: weighted connected graph G = <V, E> //Output: set of edges in minimum spanning tree of G, E_t V_T <- {V_0 E_T <- 0 for (i <- 1 to V - 1) { find minimum weight edge e* = (v*, u*), where v in V_T, u in V - V_T V_T <- V_T union u* E_T <- E_T union e* return $E_T$ Implementation issues: For each vertex not in tree, must represent cheapest edge linking it to tree Use 2 fields: name of nearest vertex in tree, edge cost (node(nearestneighbor-in-tree, cost)) Nodes with no edges to nodes in tree have cost of Finding next node is simply search of weights When node to be added id d, need to Analysis: 1. Move u to tree 2. For each node not in tree, update edge linking to node in tree if u provides cheaper link to tree than old link Uses n 1 iterations Efficiency depends on structures used to represent G and priority queue (Q) If represent G as weight matrix, Q as unordered list, t(n) Θ( V 2 ) If use minheap for Q V 1 deletions from Q E verifications Heap size V Each operation O(log V ), so ( V 1 + E )O(log V ) t(n) O( E log V ) 3

4 Different approach from Prim Basic approach: Greedy Approach: Kruskal s Algorithm Given list of edges sorted by weight (in non-decreasing order) Add next edge from list, providing it does not create a cycle At any point, may have many unconnected components End result is single spanning tree Kruskal s algorithm: Alg kruskal(g) { //Input: weighted connected graph G = <V, E> //Output: set of edges in minimum spanning tree of G, E_t sort E in non-decreasing order by weight E_T <- 0 ecounter <- 0 k <- 0 while (ecounter < V - 1) { k++ if (E_T cup {e_k is acyclic { find minimum weight edge e* = (v*, u*), where v in V_T, u in V - V_T E_T <- E_T union e_k ecounter++ return E_T More complex than Prim: Must check whether addition of edge creates cycle Cycle occurs if vertices being connected lie in same component Can consider construction from following approach: Initially have forest of n trees - each is single vertex of G Solution is single tree of those vertices Each iteration connects 2 subtrees of forest into one tree by shortest edge Union-find algorithms make above approach efficient, dominated only by initial sort 4

5 Greedy Approach: Subset and Union-find - Intro Concerned with disjoint subsets of distinct elements This abstract data type has following operations makeset(x) Creates a singleton subset of x Can only be applied once to x find(x) Returns subset containing x union(x, y) Creates union of subsets containing x and y Original subsets deleted Subsets usually identified by a representative element Could be first element added to subset Could be smallest element in subset... Elements usually mapped to integers 1..n Two approaches to implementation: 1. Quick find Optimizes the find operation 2. Quick union Optimizes the union operation 5

6 Greedy Approach: Subset and Union-find - Quick Find Subsets represented as linked lists List header contains 1. Size of subset 2. Pointer to first element 3. Pointer to last element Have one list/bucket for each element (n lists) find implemented via representative array Has n slots Slot i contains representative element for element indexed by i α... β... δ... n... α... α... α... Operations: makeset(x) Create node for x and link to header Θ(1) find(x) Look up representative element of x s subset in representative array Θ(1) 6

7 union(x, y) Greedy Approach: Subset and Union-find - Quick Find (2) find(y) Append y s list to x s list Update elements in y s subset in representative array Eliminate y s subset (set pointer to NULL) Θ w (n) for one application; Θ w (n 2 ) for n 1 To improve efficiency: Append shorter list to longer list Series of unions improves to O(nlogn) 7

8 Greedy Approach: Subset and Union-find - Quick Union Subsets represented as trees Root is representative element Links point from nodes to parent Have auxialiary array of pointers to elements nodes in trees Operations: 1. makeset(x) Create single-node tree for x Θ(1) 2. union(x, y) find(y) Append root of y s tree to root of x s tree Eliminate y s tree (set pointer to root NULL) Θ w (1) for one application; Θ w (n) for n 1 3. find(x) Follow pointers from x to root of tree Θ w (n) 8

9 Greedy Approach: Subset and Union-find - Quick Union (2) To improve efficiency: 1. Append shorter tree to larger tree Tree can be measured by (a) Number of nodes (union by size) (b) Height of tree (union by rank) Both require additional storage at each node of tree: (a) Number of descendants, or (b) Height Efficiency improves to O(log n) 2. Path compression On every find, change link of all traversed nodes to point to root 9

10 Greedy Approach: Dijkstra s Algorithm Concerned with single-source, shortest-path problem: Like all-pairs shortest-path problem, but only considers paths from a single node General approach: 1. Given source, find node v closest to it 2. Find node closest to v 3. etc. At end of ith iteration, will have created tree T i with 1. Source as root 2. i 1 nodes closest to source At any point, nodes divided into 3 disjoint sets: 1. Those in the tree 2. Those with an edge connecting them to a node in the tree (the fringe) 3. Those with no links to the tree At any point, the next node u to be added will come from the fringe Let v be the node in the tree to which u has the cheapest edge d v be the cost of the cheapest path from the source to v Then, u is node with min(w(u, v) + d v ), where u fringe To implement, label each node with 2 values: 1. d: shortest distance from source to the node (through a node in the tree) 2. v: its parent node On each iteration, simply choose the node with smallest d Once u selected, update non-tree nodes u: For every node u with edge e(u, u) and weight w(u, u) If d u > d u + w(u, u), then let d u = d u + w(u, u) v u = u This does not work for negative weights 10

11 Greedy Approach: Dijkstra s Algorithm (3) Algorithm: Analysis Dijkstra { //Input: Weighted graph G = <V, E>, source node s //Output: du, vu for every node u in V initialize q //priority queue set to null for (each u in V) { du <- maxval vu <- null insert (q, u, du) ds <- 0 decrease (q, s, ds) vt <- 0 for (i <- 0 to V -1) { u* <- deletemin(q) Vt <- Vt U {u* for (each u in (V - Vt) adjacent to u*) if (du* + w(u*, u) < du) { du <- du* + w(u*, u) vu <- u* decrease (q, u, du) Efficiency depends on structures used to represent G and priority queue (q) as in Prim s algorithm If represent G as weight matrix, Q as unordered list, t(n) Θ( V 2 ) If use adjacency linked list for G and minheap for q, t(n) O( E log V ) 11

12 Greedy Approach: Huffman Trees Problem: Want to encode a string with minimal number of bits Fixed-length encoding uses same number of bits for each character Variable-length encoding uses different bit lengths for different characters Use shorter bit strings for more frequently occurring characters Problem: How to determine when one encoded bit string ends and next begins? Solution: Prefix-free code Prefix-free encoding (or just prefix encoding) is one in which no bitstring code is valid prefix of any other When scan a prefix-free encoded string, as soon as recognize a code, start the next Principle for creating prefix-free encoding: Represent code as a binary tree Each character of alphabet located at leaf Path from root to leaf represents code for character at leaf Since every path is unique, no code will be duplicated Binary encoding determined by labeling all left branches with 1 s, all right with 0 s (or vice-versa) 12

13 Greedy Approach: Huffman Trees (2) Algorithm: Huffman{ //Input: Alphabet A and frequencies F of occurence //Output: Huffman tree for (each c in A) { create singleton tree label node with c lable node with corresponding frequency f while (more than one tree remains) { id 2 trees with smallest weights create a new node labeled with sum of weights make one tree left subtree make other tree right subtree To generate codes, label branches as discussed above Expected number of encoded bits per character = n i=1 l i f i where l i is length of path from root to leaf c i Compression ratio = log 2(n) expected bits per char log 2 (n) Huffman trees solve more general problem of constructing tree of minimal weighted path length Consider binary tree with n leaves Each leaf labeled with weight w i Weighted path length of tree = n i=1 l i w i, where l i is length of path from root to leaf i This arises in decision making, using decision trees (see C10) 13