Graph Algorithms Chapter 22 CPTR 430 Algorithms Graph Algorithms
Why Study Graph Algorithms? Mathematical graphs seem to be relatively specialized and abstract Why spend so much time and effort on algorithms for such an obscure area? As it turns out, many practical problems can be solved by applying some basic graph algorithms CPTR 430 Algorithms Graph Algorithms 2
Notation A graph is a mathematical structure that consists of a set of vertices and a set of edges G G E V is the number of vertices E is the number of edges In the context of asymptotic notation, V means V and E means E For example, O V 2 really means O V 2 CPTR 430 Algorithms Graph Algorithms 3
Graph Representation Adjacency matrix Adjacency list CPTR 430 Algorithms Graph Algorithms 4
Undirected Graph Representations 0 4 3 2 0 2 3 4 0 0 0 0 0 2 0 0 0 3 0 0 4 0 0 0 2 3 4 4 0 4 2 3 3 4 2 3 0 CPTR 430 Algorithms Graph Algorithms 5
Directed Graph Representations 0 2 3 4 5 0 2 3 4 5 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 3 0 0 0 0 0 4 0 0 0 0 0 5 0 0 0 0 0 0 2 3 4 5 4 5 3 5 3 4 CPTR 430 Algorithms Graph Algorithms 6
Which Representation is Better? Adjacency lists are good for sparse graphs Sparse graph: E V 2 The list nodes could instead contain pointers to vertices For digraphs: sum of lengths of the lists = For undirected graphs: sum of lengths of the lists = 2 Space required: Θ V E Adjacency matrix is better for dense graphs Dense graph: E is not mush less than Can quickly tell if two vertices share an edge Space required: Θ V 2, independent of the number of edges Undirected graphs can use a triangular matrix Unweighted graphs require only one bit per edge! V E 2 E CPTR 430 Algorithms Graph Algorithms 7
Weighted Graphs For each u v E, w : E w u v is the weight of edge u v Weighted graphs represent edge costs, lengths, time, etc. Adjacency list: the edge weight is stored in the node for the edge in the list Adjacency matrix: the edge weight is stored in matrix 0 0 4 2 8 8 9 3 0 2 CPTR 430 Algorithms Graph Algorithms 8
Breadth-first Search Used in a number of classic graph algorithms: Prim s minimal spanning tree Dijkstra s single source shortest path Finds all the vertices reachable from a source vertex, s Produces a breadth-first tree whose root is s Computes the length of the path to each reachable vertex Path length = number of edges in the path Breadth-first all the vertices at distance k are located before any reachable vertices farther than distance k are found CPTR 430 Algorithms Graph Algorithms 9
Coloring the Vertices BFS marks vertices to manage its progress Vertices are: Colored white initially Colored gray when first discovered Colored black when all their neighbors have been discovered u v E and u is black v is either gray or black Gray vertices represent the frontier between discovered and undiscovered vertices 0 4 3 2 CPTR 430 Algorithms Graph Algorithms 0
Breadth-first Tree BFS essentially builds a tree The tree s root is the source vertex Vertex v discovered while scanning vertex u neighbors has an edge from parent u to child v the tree 0 4 3 2 CPTR 430 Algorithms Graph Algorithms
Implementation Particulars Augment vertices with additional information: color (black, gray, white) (color) predecessor (parent vertex in the BF tree), π v (predecessor) distance (number of edges from root [source vertex] to this vertex), d v (distance) Use a queue to manage the gray vertices CPTR 430 Algorithms Graph Algorithms 2
How It Works First, for each vertex: set its color to white (undiscovered) set its predecessor to null (not in the BF tree) set its distance to (may not even be a path to it from the source vertex) For the start vertex s: color s gray set d s to 0 place s in the queue CPTR 430 Algorithms Graph Algorithms 3
How It Works (cont.) As long as the queue is not empty, extract a vertex u from the queue and: for each vertex v adjacent to u: color v gray set d v set π v to d u to u Place v into the queue Color u black CPTR 430 Algorithms Graph Algorithms 4
Analysis of Running Time Use aggregate analysis Every vertex must be colored and properly initialized the running time for initialization is O V After initialization, no vertex is ever made white again each vertex is enqueued at most once Unreachable vertices will not be enqueued Recall that the time to enqueue and dequeue elements is O all queue operations take O V time CPTR 430 Algorithms Graph Algorithms 5
Analysis of Running Time (cont.) Each time a vertex is dequeued: its adjacent vertices (via its adjacency list) are each are checked to see if they need to be enqueued (i.e., are white) a vertex s adjacency list is checked once only the sum of the vertices in all adjacency lists is Θ And entry in an adjacency list represents an edge The time spent scanning the adjacency lists is O the total time for breadth-first search is E E O V O V O E O V E This means that BFS runs in time linear in the size of the (complete) adjacency list representation of the graph CPTR 430 Algorithms Graph Algorithms 6
Shortest Path Define δ s v to be the shortest distance from vertex s to vertex v The minimum number of edges in any path from s to v δ s v no path exists between s and v u v E δ s v δ s u (Lemma 22.) u is reachable from s v is reachable from s u u u The shortest path from s to v cannot be longer than the shortest path from s to u followed by edge u v δ s v δ s u is unreachable from s δ s δ s v δ s CPTR 430 Algorithms Graph Algorithms 7
Shortest Path Property of BFS For any directed or undirected graph G distance attribute of each vertex v V V E, BFS assigns δ s v the Proof: Lemma 22.2 First, show something weaker: v V d v δ s v Proceed by induction on the number of enqueue operations Basis case: Is it true after the first enqueue operation? Inductive step: True after k enqueue operations true after k enqueue operations? CPTR 430 Algorithms Graph Algorithms 8
Basis Step This is the case where s, the source vertex, is enqueued Due to the initialization part of the algorithm: d s 0 δ s s and v V s d v δ s v In either situation, v V d v δ s v CPTR 430 Algorithms Graph Algorithms 9
Inductive Step A vertex is enqueued only if it is white Let vertex v be a white vertex discovered while considering vertex u s neighbors I.e., u v E, and v is white Vertex v must be enqueued The inductive step is then d v d u δ s u δ s v Assignment in BFS I.H. Lemma 22. Once enqueued, v s color is set to gray, so it can never be enqueued again d v is never changed again, and the I.H. is maintained CPTR 430 Algorithms Graph Algorithms 20
Lemma 22.3 Let v v r v2 be the vertices, in order, in the queue during some part of the execution of BFS on graph v is the head of the queue v r is the tail of the queue V E d v r d v i r 2 d v i d v i CPTR 430 Algorithms Graph Algorithms 2
Proof of Lemma 22.3 Proceed by induction of the number of queue operations (not just the number of enqueue operations) Basis case: Does the lemma hold after the first queue operation? Inductive step: True after k queue operations true after k queue operations? CPTR 430 Algorithms Graph Algorithms 22
Basis Case The first queue operation is always enqueuing s, the source vertex Here, r d v r d v i : d s d v i d v d v holds vacuously CPTR 430 Algorithms Graph Algorithms 23
Inductive Step We must show it holds for both enqueue and dequeue operations Queue head v dequeued v 2 becomes the new head If the queue becomes empty then Lemma 22.3 holds vacuously d v d v 2, by the I.H. d v r d v d v 2, and the other inequalities at unaffected CPTR 430 Algorithms Graph Algorithms 24
Inductive Step (cont.) When vertex v is enqueued, it becomes v r Vertex v is enqueued while scanning vertex u s neighbors u v E Vertex u was the queue head, but it was dequeued before its neighbors are scanned By the I.H., the new head v has d v d v r d v d u d v d u By the I.H., d v r d v r d u d u d v d v r and the other inequalities are unaffected, CPTR 430 Algorithms Graph Algorithms 25
Corollary 22.4 Lemma 22.3 leads to Corollary 22.4: For any two vertices v i and v j that are enqueued during the execution of BFS where v i is enqueued before v j, d v i d v j when v j is enqueued. Thus vertices are enqueued with montonically increasing d values over time CPTR 430 Algorithms Graph Algorithms 26
BFS Correctness For a graph G BFS: V E (directed or undirected), and source vertex s V, discovers every vertex v V that is reachable from s properly assigns the d attribute of vertices, such that from s, d v δ s v v V reachable finds a shortest path from s to any reachable vertex v: s to π v by edge π v v (where v s) followed CPTR 430 Algorithms Graph Algorithms 27
BFS Correctness Proof Proceed by contradiction Assume some vertex is assigned an incorrect d value (i.e., its d attribute length of shortest path to it from s) Let v be the vertex with a minimum δ (d v δ s v ) s v with an incorrect d value v s (Why?) d v δ d v s v (Lemma 22.2) and d v δ s v δ s v v is reachable from s; otherwise, δ s v δ s v d v CPTR 430 Algorithms Graph Algorithms 28
BFS Correctness Proof (cont.) Let u be the vertex immediately preceding v on a shortest path from s to v δ s v δ s u δ s u d u δ s v δ s u and s has a minimum δ s v with incorrect d It follows from all of the above that d v δ s v δ s u d u We will use these relationships to demonstrate a contradiction CPTR 430 Algorithms Graph Algorithms 29
BFS Correctness Proof (cont.) When vertex u is dequeued, v is either white (undiscovered), gray (discovered) or black (closed): Suppose v is white The BFS algorithm sets d v d v d u (previous slide) d u, contradicting the fact that Suppose v is black v has already been removed from the queue, (Corollary 22.4) Again, this contradicts d v d u so d v (previous slide) d u CPTR 430 Algorithms Graph Algorithms 30
BFS Correctness Proof (cont.) Suppose v is gray v was colored gray because its is adjacent to some other vertex, w, that was dequeued at some time before u is dequeued Since it is adjacent to w, d v d w d u Rightarrow d v d u (Corollary 22.4) d w, again the same contradiction Therefore, v V d v δ s v π v δ u s v d v d u δ s π v plus the edge π v v CPTR 430 Algorithms Graph Algorithms 3
Depth-first Search Search as deep as possible instead of visiting all neighboring vertices first (as BFS does) The neighbors of the most recently discovered vertex are considered first BFS is queue based; DFS is stack based The stack in maintained implicitly via recursion CPTR 430 Algorithms Graph Algorithms 32
DFS Algorithm Particulars The DFS algorithm in your book uses the three colors (white, gray, black) as in BFS Two timestamps are used (d and f ): d is the time the vertex is discovered (grayed) f is the time the vertex is finished (backened) The two different timestamps are used for reasoning about the behavior of DFS CPTR 430 Algorithms Graph Algorithms 33
dfsvisit(): DFS Running Time As in BFS, use aggregate analysis DFS calls dfsvisit() exactly once for every vertex in the graph it is O V not counting the time for the calls of dfsvisit() dfsvisit() is invoked only on unmarked vertices and it immediately marks any vertex it processes dfsvisit() is invoked only on unmarked vertices (or white vertices) CPTR 430 Algorithms Graph Algorithms 34
DFS Running Time (cont.) The loop in dfsvisit(v) scans the vertices adjacent to v; if adj v the set of vertices adjacent to v, then the loop takes Θ adj v vertex v The total time for all executions of dfsvisit() is thus is time for v V adj v Θ E DFS therefore takes Θ V E time The simpler DFS takes just Θ E time CPTR 430 Algorithms Graph Algorithms 35
Parenthesis structure E If a DFS is performed on G V, then for any two vertices u one the following three conditions hold: v V exactly The intervals d u f u and d v f v is a descendent of the other in a depth-first forest are disjoint, and neither vertex The interval d u f u d v f v, and u is a descendent of v in a depth-first tree is contained entirely within the interval The interval d v f v d u f u, and v is a descendent of u in a depth-first tree is contained entirely within the interval CPTR 430 Algorithms Graph Algorithms 36
White Path Theorem In a depth-first forest of graph G V, vertex v is a descendent of vertex u iff at time d u (the time DFS discovers u) v can be reached from u following a path consisting entirely of white vertices E See proof on Page 545 CPTR 430 Algorithms Graph Algorithms 37
DFS Edge Classification Tree edges edges in the depth-first forest G π Back edges edges u v depth-first tree (self-loops are back edges) connecting a vertex u to an ancestor v in a Forward edges non-tree edges descendent v in a depth-first tree u v connecting a vertex u to a Cross edges all other edges, e.g.: connect two vertices that where neither is an ancestor of the other connect vertices in different depth-first trees CPTR 430 Algorithms Graph Algorithms 38
Using DFS to Classify Edges DFS can be modified to classify edge types as it proceeds Color an edge first explored white edge gray edge black edge u v based on the color of vertex v when the edge is tree edge back edge cross edge or forward edge CPTR 430 Algorithms Graph Algorithms 39
Edges in Undirected Graphs Edges u v and v u are the same edge in undirected graphs The classification chosen is the one appropriate for the first edge encountered If If u v color v u color is encountered first, then both is encountered first, then both u v u v and and v u v u are given v s are given u s Undirected graphs contain no forward or cross edges CPTR 430 Algorithms Graph Algorithms 40
Theorem 22.0 In a DFS of an undirected graph G, every edge of G is either a tree edge or a back edge Let u v be an arbitrary edge in G Suppose w.l.g. that d u d v v must be discovered and finished before we finish u (i.e., u is still gray) since v is on u s adjacency list Edge u v explored first in the direction from u to v v is undiscovered (white) until then, since otherwise the edge would already been explored in the direction from v to u u v becomes a tree edge Edge u v explored first in the direction from v to u u is still gray when the edge is explored u v is a back edge, CPTR 430 Algorithms Graph Algorithms 4
Topological Sort A linear ordering of all the vertices of a directed acyclic graph (dag) G such that if u v is an edge in G, then u appears before v in the ordering In a topological sort you can arrange all the vertices in a line with all edges pointing pointing from left to the right Dags are useful for scheduling and prioritizing events A topological sort allows proper ordering for single thread execution CPTR 430 Algorithms Graph Algorithms 42
Dags Have No Cycles A directed graph G is acyclic A depth-first search of G yields no back edges Suppose there is a back edge u v. It follows that vertex v is an ancestor of vertex u in the depth-first forest. Thus, there is a path from v to u in G, and the back edge u v completes a cycle. Suppose that G contains a cycle c. Let v be the first vertex discovered in c, and let u v be the edge in c before v. At time d v the vertices of c form a path of white vertices from v to u. According to the whitepath theorem, u will be a descendent of v in the depth-first forest. Thus, u v is a back edge. CPTR 430 Algorithms Graph Algorithms 43
Topological Sort Algorithm public Vector toposort(graph G) { Vector topolist = new Vector(); Vertex.makeMark(); for ( Vertex u in G.vertices ) if (!u.ismarked() ) dfsvisit(u, topolist); return topolist; } private void dfsvisit(vertex u, Vector topolist) { u.mark(); for ( Vertex v in u.adjacents ) if (!v.ismarked() ) dfsvisit(v); topolist.insertelementat(u, 0); } CPTR 430 Algorithms Graph Algorithms 44
Topological Sort Correctness To prove the correctness of topological sort we must show that when DFS is performed on a dag G V E, if u v E, then f u f v Consider edge u v explored by DFS v cannot be gray because v gray v is white v is black v is an ancestor of u u v is a back edge violating Lemma 22. v is black or white v is a descendent of u f u f v v is already finished We are exploring from u u is not finished u is not yet timestamped for f f v f u CPTR 430 Algorithms Graph Algorithms 45
Topological Sort Running Time DFS takes time O E Insertion onto the front of a linked list takes O V vertices time for each of the The total time for the topological sort is O V E This is time linear in the number of edges and vertices CPTR 430 Algorithms Graph Algorithms 46
Strongly Connected Components V E V such that for every pair of vertices u A strongly connected component of a digraph G set of vertices C have u v and v u G v is reachable from u and v is reachable from u is the transpose of graph G E is a maximal v C, we G V, where E u v E E is the set of edges in G with the directions reversed Given an adjacency list representation of G, the time to create G O V E v To compute the strongly connected components of digraph G, perform two depth-first searches: one on G, the other on G Running time: Θ V E u is CPTR 430 Algorithms Graph Algorithms 47