Graph Algorithm Design (8) Graph Algorithms / Graph:, : A finite set of vertices (or nodes) : A finite set of edges (or arcs or branches) each of which connect two vertices Takashi Chikayama School of Engineering The University of Tokyo Things Best Represented as Graphs Concrete existences and what connect them Stations connected via railroad lines Liquid or gas tanks connected with pipes Computers and/or switches connected by cables Relations between: Things: on, under, above, below, adjacent-to, Events: precedence, causality Notions: subsumption, equivalence, Multiple and Simple Edges Edges are multiple if there is another edge connecting the same two vertices Otherwise, an edge is simple A graph is simple if all the edges are simple An edge of a simple graph is a vertex pair,
Subgraphs Subgraph: A graph both of its vertices and edges are subsets of the original graph Spanning subgraph: A connected subgraph containing all the vertices of the original graph Directed Graphs and Weighted Graphs When edges of a graph have No direction: undirected graph Directions: directed graph or digraph Graphs with values associated with their edges are called weighted graphs Weighted digraphs are sometime called networks 6 Walks, Trails and Paths Loops and Cycles Walk: An alternating sequence of adjacent vertices and edges connecting them For a digraph, the order of the vertices must agree with the edge direction Trail: A walk without duplicated edges Path: A walk without duplicated vertices Loop: An edge with that links to itself Cycle: A trail that starts and ends at the same vertex Acyclic Graph: A graph without cycles 8
Connectivity A graph is connected if all nodes are directly or indirectly linked Otherwise the graph is disconnected Complete graph: a graph in which all pairs of vertices have a connecting edge Connectivity in Digraphs A directed graphs is weakly connected if an undirected graph obtained by removing directions of its edges is connected It is strongly connected if there exists a directed path from all of its nodes to all of its nodes. A strong component of a graph is a maximal strongly connected subgraph of the graph. 9 0 Degree or Valence Degree (or valence) of a vertex: the number of edges connected to a vertex In-degree and out-degree are distinguished in digraphs in-degree = degree = out-degree = Connectivity of a Graph Given a graph, if at least k vertices have to be removed to make the graph disconnected, the graph is called k-vertex-connected If at least k edges have to be removed to make a graph it disconnected, the graph is called to be k-edge-connected Cut Set: A set of edges removing which will make the graph disconnected When one single edge forms a cut set, that edge is called a bridge
Tree Tree: A connected acyclic simple graph All the edges in a tree are bridges A vertex of a tree is a leaf if it has only one edge connected to it Note that most of the trees that are used in algorithms are rooted trees, that are simple acyclic digraphs with certain restriction on edge directions Representation of Graphs A variety of representations are possible Chosen considering the characteristics of the graph and the kind of processing required Points are: Memory amount required Costs to obtain information such as: Whether given two vertices are directly linked Which vertices connect to which Enumerate all the vertices or all the edges Adjacency Matrix A square matrix that has the same number of rows and columns as the number of vertices Element (i, j) being /0 tells whether an edge from vertex i to vertex j exists or not For weighted graphs, the edge weights are stored The matrix is symmetric for an undirected graph 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Merits and Demerits of Adjacency Matrix Dense graphs (with many edges) can be compactly represented Efficient to tell whether two vertices are directly connected For sparse graphs, redundant with many zeros For graphs with n vertices, the size is O(n ), irrespective of the number of edges For sparse graphs, enumerating vertices directly connected a given vertex is inefficient: O(n) Only simple graphs can be represented; No way to represent mulitple edges 6
Incidence Matrix A matrix in which rows correspond to vertices and columns correspond to edges Element (i, j) being /0 tells whether the vertex i is one end of the edge j or not For digraphs, different values for starts and ends can be used; yet another value for loops A B C D E F G 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 D E B C A G F Merits and Demerits Any graphs can be represented Sparse graphs can be represented compactly With vertices and edges, the size is Edge weights can be stored separately ( ) Inefficient to tell whether two vertices have a direct edge connecting them For sparse graphs, enumerating vertices adjacent to a given vertex is inefficient Not as efficient as adjacency matrix for dense graphs 8 Adjacency List For each vertex, a list of adjacent vertices is kept A list representation of the information in an adjacency matrix Merits and Demerits of Adjacency List Compact for sparse graphs For vertices and edges Enumeration of adjacent vertices is efficient for sparse graphs Inefficient for dense graphs One pointer instead of one bit per edge If edges are to be registered for both ends, the representation is redundant (doubles the memory) 9 0
Sparse Adjacency Matrix Non-zero elements of an adjacency matrix are linked with pointers, both row-wise and column-wise 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0,,,,,,, Merits and Demerits of Sparse Adjacency Matrix Compact for sparse graphs For vertices and edges Inefficient for dense graphs For each non-zero elements, its row number, column number, and two pointers are required Only one bit in an adjacency matrix Cost of telling whether two vertices are adjacent is proportional to the number of edges from (or to) the same vertex Hash Table Hash table of edges indexed with its two ends h(i, j) Efficient to tell whether two vertices are adjacent or not Compact for sparse graphs; for edges Enumerating edges from a given vertex is quite inefficient Hash collisions may degrade the performance Hash Index in Addition to Sparse Adjacency Matrix A hash table can be added to a sparse adjacency matrix to allow efficient accesses h(, ),,,,,,, 6
Graph Traversal Graph Traversal Starting from a given vertex, find all the vertices connected to it by tracing edges Depth-first traversal Visit a vertex adjacent to the most recently visited vertex next Breadth-first traversal Visit a vertex adjacent to the least recently visited vertex next Data structures: A flag to tell whether a vertex is already visited To decide the vertex to visit next A LIFO stack for depth-first traversal Depth-first Stack Breadth-first Queue A FIFO queue for breadth-first traversal 6 Edge Weights Priority Queues Weights given to edges can represent information such as distance, time, costs, capacities, etc Locations as vertices and distances as weights System states as vertices and time of transition as weights Tanks as vertices, maximum flow of connecting pipes as weights Computers or switches as vertices, bandwidths as weights If weights depend on directions, digraphs are appropriate Many of the algorithms for weighted graphs require the following Keeping a subset of edges and/or vertices each associated some value Inspecting edges in the order of their values Efficiency in finding the edge/vertex with the largest (or the smallest) value is often the key Priority queues fit this purpose 8
Heap Property Heap is a structure with the following properties A heap is a tree A parent node has its associated value less than or equal to those of their children nodes This is for min-heaps; for max-heaps the order relation is reversed As a consequence, the item at the root is with the minimum value 8 8 9 Priority Queue Represented as a Binary Heap Binary heap (or sometime simply heap) A balanced binary tree with key values at nodes No child node has a value less than parent Note that the value may be the same The heap is not unique for the same set of keys 0 0 0 0 Representing a Binary Heap as an Array A binary heap can be represented as an array compactly with the same number of elements Two children of A[ k ] are A[ k+ ] and A[ k+ ] Condition of being a heap is A[ k ] A[ k+ ] and A[ k ] A[ k+ ] 0 0 Heap Operation: Extracting the Least Item. The least item is at the root, so take it off. The last item in the heap is moved to the root. If the lesser of the values of its children is less than that of the moved item, swap them. Repeat the step until a leaf is reached Repeated at most as many times as the tree depth O(log n) 0 8
Heap Operation: Adding a New Item. Insert the new item as the last item. If it has a key less than its parent, swap them. Repeat the step until the key is greater than or equal to the parent, or the root has been reached Repeated at most as many times as the tree depth O(log n) Heap Operation: Removing an Item. Remove the item and place the last item there. If its key is less than the parent, or greater than the lesser of the children, swap them. Repeat until reaching the root or a leaf Repeated at most as many times as the tree depth O(log n) 0 0 Binomial Tree Binomial Heap Binomial tree is a tree structure used for heaps A binomial tree of order 0 is a single node A binomial tree of order k has a root node that has k binomial trees of orders k, k,...,,, 0 (in this order) as its children A binomial tree of order k has k nodes A collection of binomial trees satisfying: All binomial trees that have heap property, and No two binomial trees have the same order From the above, the following can be said The smallest item is at one of the roots of trees At most log n trees for n-elements k=0 k= k= k= k=0 6 9 k= k= 6 9
Binomial Heap: Merging Two Heaps Merging two binomial heaps can be done in the same way as adding two binary numbers Merge trees of the same order of both heaps When one has a tree of order k and the other doesn t, the resultant heap will have that tree When both have a tree of order k, they are merged to a tree of order k+, which should be considered in merging trees of order k+ (carry ) Merging three trees of order k, from both heaps and a carry tree, one will be in the result and the rest are merged into a carry tree of order k+ Binomial Heap: A Merger Example k=0 k= k= k= + 0 = 00 k= 8 Binomial Heap: Complexity of Merger The worst case complexity of merging two binomial heaps is proportional to the number of trees in the heap, that is O(log n) Adding a new item to a binomial heap can be done by: Making a heap consisting only of the added item Merging it with the original heap The worst case complexity of adding an element is O(log n) The worst case is caused by carry propagation 9 Binomial Heap: Retrieving the Minimum Item The minimum item is at one of the roots of at most log n trees; linear search costs O(log n) When the minimum is at the root of order k tree, removing it leaves trees with orders 0 thru k The binomial heap is reconstructed by merging: The original heap without the order k tree, and A heap of trees with orders 0 thru k 0000 = 00000 + 0000 = 00000 + + = 00 + The total cost is O(log n) 0 0
Skew Binary Numbers Common binary representation of an integer is: n = { k A[k]} Weights of digits are,,, 8, 6,, 6, e.g. 9 = +8+6+6 is, starting from LSB, 000 An alternative representation: n = { ( k+ ) A[k]} Weights of digits are,,,,, 6,, The first non-zero digit is allowed to be e.g. 9 = ++6 is represented as 000 Natural numbers can be uniquely represented in this system Skew Binary Numbers: Examples Decimal Skew binary 0 0 0 6 0 00 8 0 9 0 Decimal Skew binary 0 0 0 00 000 6 00 00 8 00 9 0 +0 + + Decimal Skew binary 0 0 00 00 0 0 0 6 8 0 9 00 Weights of digits are,,,,, 6,, Skew Binary Numbers: Incrementing/Decrementing When the first non-zero digit is not, simply incrementing the very first digit will do e.g. 8+ = ++6 + = +++6 000 00 When the first non-zero digit is, observing that ( k+ ) + = ( k+ ) incrementing can be by setting that digit to 0 and incrementing the next digit e.g. 9+ = ( ++6)+ = ( +)++6 = ++6 = +6 000 0000 Note that no more than two digits are changed Skew Binary Numbers: Examples Skew Decimal binary 0 0 0 6 0 00 8 0 9 0 Decimal Skew binary 0 0 0 00 000 6 00 00 8 00 9 0 Decimal Skew binary 0 0 00 00 0 0 0 6 8 0 9 00
Skew Binary Numbers: Sparse Representation A skew binary number represented as a list consisting only of non-zero elements 9: 000 6 On incrementing it, the first non-zero digit is at the top and no more than two items are changed 9: 000 9: 0000 6 The worst case complexity is O() The same applies to decrementing Skew Binomial Heap A collection of binomial heap trees, just like normal binomial heaps There can be two smallest trees, instead of one Operations are almost the same as for binomial heaps, but when a new element is added, at most one place of carry propagation is needed 6 Minimum Spanning Trees Minimum spanning tree (MST) For a connected weighted graph with positive weights, MST is the spanning subgraph with the least sum of edge weights MST is a tree If it is not, there must be a cycle Removing one edge from the cycle will not lose connectivity while decreasing the weight sum Applications: Minimizing cost for connecting all the items with roads, railroads, computer networks, etc. Kruskal s Algorithm Joseph Kruskal, 96 Adding edges in increasing order of weights. Sort the list of edges in the increasing order of their weights. Initiate the graph as an empty graph. Add edges to the graph one by one from the list, making multiple trees grow up gradually If adding an edge would make a cycle, ignore it. Finish when the graph is connected With m edges, sorting cost of O(m log m) is dominant 8
Kruskal s Algorithm: An Example 6 8 0 9 9 Prim s Algorithm (aka Jarnik s, Jarnik-Prim, or DJP) Jarnik 90, Prim 9, Dijkstra 99 Adding nodes one by one to a connected graph. An arbitrary vertex is selected as the initial tree. Initialize the set of edges outgoing from the tree. Select the edge from the set that will not form a cycle with the least weight and add to the tree. Add new outgoing edges, if any, to the set. Repeat and until the tree spans all the nodes The set of outgoing edges is maintained as a heap Heap operations in and are dominant The total cost is log 0 Prim s Algorithm: An Example 6 8 0 9 Finding the Shortest Paths Among paths from one vertex to another in a weighted graph, the shortest path has the minimum weight sum of its edges (distance) Variations: Either directed or undirected graph Variations on start and end vertices Single pair/all pairs Single source or single destination (inverting edge meanings, these are the same) Whether negative or zero weights are allowed
Dijkstra s Algorithm: Policy Edsger W. Dijkstra, 96 An algorithm based on dynamic programming for single-source positive weight problems Vertices are associate them with information of their distances from the source The information is either tentative or final Tentative distances are initially set to infinity and gradually decreased as computation proceeds When no further decrease is possible, the distance information becomes final, which gives the minimum distance from the source Dijkstra s Algorithm: Procedure. Make all vertices give distances of infinity and make them tentative, except for the starting vertex, which is set to zero and made final. For tentative vertices adjacent to one just made final, add it to the frontier set, update their distances. Choose the vertex with the least distance among frontier vertices and make it final. If that made all the vertices final, stop. Otherwise, go back to and repeat until all the vertices become final 6 8 0 6 9 0 8 Dijkstra s Algorithm: Computational Complexity With n vertices and m edges All the vertices except for finalized one can be kept in a heap in the order of tentative distances All the following have complexities of log Taking out the vertex with least distance Removing a vertex Registering a new vertex These heap operations are repeated O(n) times Initialization cost is Thus, the total complexity is log Other Representative Shortest Path Algorithms Bellman-Ford Algorithm Richard Bellman and Lester Ford, Jr., 98 Allows negative weights of worst case complexity Dijkstra s is better if weights are positive Warshall-Floyd Algorithm Robert Floyd, 96 (Bernard Roy, 99 and Stephen Warshall, 96) Shortest paths between all the vertex pairs are computed simultaneously of worst case complexity 6
Finding Connected Subgraphs Finding connected subgraphs and labeling vertices with IDs of the subgraph it belongs to Essentially, finding an equivalence class Edges represent equivalence between directly connected vertices Aka union-find problem A Naïve Algorithm: Using List Structures Vertices of each connected subgraphs are maintained in lists. Initially, make distinct lists for all vertices. When two vertices have a common edge, merge two lists, labeling vertices. Repeat the step for all edges Appending another list to a list of length k has cost of O(k), as the end of the list has to be found The total cost is O(n+m ) 8 An Improvement Using Tree Structures Appending a short list to the end of a long list causes inefficiency Keeping the list length information, the longer list can always be appended to the end of the shorter one The cost of appending is, at worst, proportional to half the length of the resultant list The total cost becomes log Each group is represented as a tree Children nodes have a pointer to its parent For a new edge, make a pointer from the root of one tree to an element of the other Labeling can be done after all edges are processed 9 60
Computational Complexity of Union Find Using Trees With no control of the tree depths, the worst case is the same as when using lists Controlling the tree sizes and making the root of smaller tree to the root of the larger, amortized complexity of a single merger becomes O(log n) While finding the root, each nodes on the way can be made to point to its grandparent Using this, the amortized complexity of O( (n)) is obtained, where α(n) is the inverse Ackermann function; as the Ackermann function grows very rapidly, this can be regarded as a constant Making Paths Shorter While Finding the Root 6 6 Summary Next Week Graphs can model a variety of problems Graphs can be represented in several ways in memory, each with different characteristics Priority queues can be represented in heap data structures Several representative graph algorithms Minimum spanning tree Shortest path Union find More graph algorithms Topological Sort Maximum Clique Hamiltonian Path and Cycle Graph Coloring Maximum Flow/Minimum Cut Matching 6 6 6