Graph Algorithms Parallel and Distributed Computing Department of Computer Science and Engineering (DEI) Instituto Superior Técnico May, 0 CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 /
Outline Problems Described on Graphs Graph Representations Parallel Graph Algorithms Connected Components Maximal Independent Set Breath First Search CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 /
Graph Definition Graph G = (V, E) V is a finite set of vertices (or nodes) E is a finite set of edges (or lines) Directed graph: (u, v) E, incident from vertex u to vertex v Weighted graph: value (cost) w assigned to each (u, v) E Path from a vertex u to v: sequence <v 0, v, v,..., v k > of vertices, v 0 = u, v k = v, and (v i, v i+ ) E path length: number of edges in a path, or sum of weights of the edges CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 /
Sample Application Areas of Graphs AREA VERTICES EDGES Communication telephones, computers fiber optic cable Circuits gates, registers, processors wires Mechanical joints rods, beams, springs Hydraulic reservoirs, pumping stations pipelines Financial stocks, currency transactions Transportation street intersections, airports highways, air routes Scheduling tasks precedence constraints Software systems functions function calls Internet web pages hyperlinks Games board positions legal moves Social networks people, actors, terrorists friendships, movie casts, etc Protein interaction networks proteins protein-protein interactions Genetic regulatory networks genes regulatory interactions Neural networks neurons synapses Infectious disease people infections Electrical power grid transmission stations cable Chemical compounds molecules chemical bonds CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 /
Graph Representation 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Adjacency Matrix ( E V ) Adjacency List ( E V ) CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 /
Distributed Graph Representation each processor stores the entire graph CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 /
Distributed Graph Representation each processor stores the entire graph each processor stores n p vertices (and all edges out of these vertices) CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 /
Distributed Graph Representation each processor stores the entire graph each processor stores n p vertices (and all edges out of these vertices) How to create these p vertex partitions? CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 /
Distributed Graph Representation each processor stores the entire graph each processor stores n p vertices (and all edges out of these vertices) How to create these p vertex partitions? graph partitioning algorithms: recursively optimize while minimizing edges crossing partitions balance vertices or edges inside partitions? CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 /
D Graph Partioning consider a logical D processor grid (p = p r p c ) and the matrix representation of the graph assign each processor a sub-matrix (edges within the sub-matrix) Example: consider the previous graph, and p = : 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 /
Sample Graph Algorithms Searches (breadth-first, depth-first, A*) Single-source shortest paths (Dijkstra, Bellman-Ford, DAG) All-pairs shortest paths (Johnson, Floyd-Warshall) Minimum spanning tree (Kruskal, Prim) Components (connected, strongly connected, biconnected) Maximum cardinality matching Max-flow (Edmonds-Karp, push-relabel) Sparse matrix ordering (Cuthill-McKee, King, Sloan, minimum degree) Layout (Kamada-Kawai, Fruchterman-Reingold, Gursoy-Atun) Betweenness centrality PageRank Isomorphism Vertex coloring Transitive closure Dominator tree CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 8 /
All-pairs Shortest Path Floyd-Warshall Algorithm:. for k 0 to V. for i 0 to V. for j 0 to V. d[i, j] min(d[i, j], d[i, k] + d[k, j]) Complexity: Θ( V ) CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 9 /
Graph Algorithms Irregular and unbalanced Non-local Data driven High data to computation ratio CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 0 /
Connected Components Connected Components Equivalence classes of vertices under the is reachable from relation for undirected graphs. Serial algorithm: perform depth-first search (DFS) on a graph. Number of trees in the forest = separate connected component. CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 /
Parallel Connected Components 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 /
Parallel Connected Components 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Processor Processor CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 /
Parallel Connected Components Each process computes DFS to obtain a forest of trees for its subgraph. Processor Processor Merge forests of trees to form the final solution. CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 /
Parallel Connected Components Merge pairs of spanning forests using disjoint sets of vertices. Consider the following operations on the disjoint sets: find(x) returns pointer to representative element of the set containing x each set has its own unique representative union(x, y) merges the sets containing the elements x and y the sets are assumed disjoint prior to the operation A simple linked list, with clever heuristics (ranking and path compression) provides a very efficient implementation for these operations. CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 /
Parallel Connected Components To merge forest A into forest B: for each edge (u,v) of A, perform find operations on u and v to determine if u and v are in same tree of B: if not, then union the two trees (sets) of B containing u and v else, no union operation is necessary Merging forest A and forest B requires at most: ( V ) find operations ( V ) union operations These operations can be performed in time Θ( V ). CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 /
Analysis of Parallel Connected Components Partition the V V adjacency matrix into p blocks V Each processor computes local spanning forest: Θ( p ) Merging approach: use a tree of processes. log p merging stages each merge stage takes time Θ( V ) total cost due to merging is Θ( V log p) During each merging stage, spanning forests are sent between nearest neighbors: Θ( V ) edges of the spanning forest are transmitted Parallel execution time: Θ( V p ) + Θ( V log p) (the corresponding scalability function is p log p...) CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 /
Maximal Independent Sets Independent Vertices A set of vertices I V is called independent if no pair of vertices in I is connected via an edge in G. Maximal Independent Sets (MIS) An independent set is called maximal if by including any other vertex not in I, the independence property is violated. CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 /
Serial Algorithms for MIS Practical MIS algorithms are incremental in nature. Start with an empty set. Add the vertex with the smallest degree. Remove node and adjacent vertices from graph. While graph not empty, Goto. CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 8 /
Serial Algorithms for MIS Practical MIS algorithms are incremental in nature. Start with an empty set. Add the vertex with the smallest degree. Remove node and adjacent vertices from graph. While graph not empty, Goto. CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 8 /
Serial Algorithms for MIS Practical MIS algorithms are incremental in nature. Start with an empty set. Add the vertex with the smallest degree. Remove node and adjacent vertices from graph. While graph not empty, Goto. CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 8 /
Serial Algorithms for MIS Practical MIS algorithms are incremental in nature. Start with an empty set. Add the vertex with the smallest degree. Remove node and adjacent vertices from graph. While graph not empty, Goto. CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 8 /
Serial Algorithms for MIS Practical MIS algorithms are incremental in nature. Start with an empty set. Add the vertex with the smallest degree. Remove node and adjacent vertices from graph. While graph not empty, Goto. CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 8 /
Serial Algorithms for MIS Practical MIS algorithms are incremental in nature. Start with an empty set. Add the vertex with the smallest degree. Remove node and adjacent vertices from graph. While graph not empty, Goto. These algorithms are intrinsically serial! CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 8 /
Luby s MIS Algorithm Randomized algorithm. Start with an empty set. Assign a random number to each vertex. Vertices whose random number are smaller than all of the numbers assigned to their adjacent vertices are included in the MIS. Vertices adjacent to the newly inserted vertices are removed. While graph not empty, Goto. CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 9 /
Luby s MIS Algorithm Randomized algorithm. Start with an empty set. Assign a random number to each vertex. Vertices whose random number are smaller than all of the numbers assigned to their adjacent vertices are included in the MIS. Vertices adjacent to the newly inserted vertices are removed. While graph not empty, Goto. CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 9 /
Luby s MIS Algorithm Randomized algorithm. Start with an empty set. Assign a random number to each vertex. Vertices whose random number are smaller than all of the numbers assigned to their adjacent vertices are included in the MIS. Vertices adjacent to the newly inserted vertices are removed. While graph not empty, Goto. CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 9 /
Luby s MIS Algorithm Randomized algorithm. Start with an empty set. Assign a random number to each vertex. Vertices whose random number are smaller than all of the numbers assigned to their adjacent vertices are included in the MIS. Vertices adjacent to the newly inserted vertices are removed. While graph not empty, Goto. CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 9 /
Luby s MIS Algorithm Randomized algorithm. Start with an empty set. Assign a random number to each vertex. Vertices whose random number are smaller than all of the numbers assigned to their adjacent vertices are included in the MIS. Vertices adjacent to the newly inserted vertices are removed. While graph not empty, Goto. This algorithm will terminate in O(log V ) iterations. Why is this a good algorithm to parallelize? CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 9 /
Luby s MIS Algorithm Randomized algorithm. Start with an empty set. Assign a random number to each vertex. Vertices whose random number are smaller than all of the numbers assigned to their adjacent vertices are included in the MIS. Vertices adjacent to the newly inserted vertices are removed. While graph not empty, Goto. This algorithm will terminate in O(log V ) iterations. Why is this a good algorithm to parallelize? How will the parallel formulation proceed? shared memory distributed memory CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 9 /
Breath First Search Breath-first Search Graph traversal algorithm that is performed by starting at the source node and visit nodes in order of distance from the source. for each vertex v in V-{s} color[v]=white; d[v]=oo; p[v]=nil; color[s]=gray; d[s]=0; p[s]=nil; enqueue(q, s); while Q not empty u = gettopqueue(q); for each v adjacent to u if(color[v] == WHITE) color[v] = GREY; d[v] = d[u] + ; p[v] = u; enqueue(q, v); color[u] = BLACK; CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 0 /
Breath First Search Breath-first Search Graph traversal algorithm that is performed by starting at the source node and visit nodes in order of distance from the source. for each vertex v in V-{s} color[v]=white; d[v]=oo; p[v]=nil; color[s]=gray; d[s]=0; p[s]=nil; enqueue(q, s); while Q not empty u = gettopqueue(q); for each v adjacent to u if(color[v] == WHITE) color[v] = GREY; d[v] = d[u] + ; p[v] = u; enqueue(q, v); color[u] = BLACK; d=0 p=nil d= p= d= p= d= p= d= p= d= p= d= p= CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 0 /
Breath First Search Parallel BFS? for each vertex v in V-{s} color[v]=white; d[v]=oo; p[v]=nil; color[s]=gray; d[s]=0; p[s]=nil; enqueue(q, s); while Q not empty u = gettopqueue(q); for each v adjacent to u if(color[v] == WHITE) color[v] = GREY; d[v] = d[u] + ; p[v] = u; enqueue(q, v); color[u] = BLACK; CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 /
Breath First Search Parallel BFS? for each vertex v in V-{s} color[v]=white; d[v]=oo; p[v]=nil; color[s]=gray; d[s]=0; p[s]=nil; enqueue(q, s); while Q not empty u = gettopqueue(q); for each v adjacent to u if(color[v] == WHITE) color[v] = GREY; d[v] = d[u] + ; p[v] = u; enqueue(q, v); color[u] = BLACK; Primite task: enumerate out-edges of a vertex. Distributed adjacency list Problems: shared global queue vertex coloring CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 /
Distributed Queue Fundamental operations: gettopqueue: retrieves from top of queue enqueue: add to end of queue empty: signals termination CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 /
Distributed Queue Fundamental operations: gettopqueue: retrieves from top of queue enqueue: add to end of queue empty: signals termination Distributed queue: each node stores a local queue. gettopqueue: always from local queue enqueue: local if owner, otherwise sends to owner empty: wait for remote sends CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 /
Vertex Coloring Store data on the node that owns the vertex (or edge). Two alternatives: send message request color of a remote node CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 /
Vertex Coloring Store data on the node that owns the vertex (or edge). Two alternatives: send message request color of a remote node send message if color of remote node is locally changed CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 /
Vertex Coloring Store data on the node that owns the vertex (or edge). Two alternatives: send message request color of a remote node send message if color of remote node is locally changed Or: keep local copies of adjacent nodes CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 /
Vertex Coloring Store data on the node that owns the vertex (or edge). Two alternatives: send message request color of a remote node send message if color of remote node is locally changed Or: keep local copies of adjacent nodes broadcast message if color of a node (any node) is changed CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 /
Vertex Coloring Store data on the node that owns the vertex (or edge). Two alternatives: send message request color of a remote node send message if color of remote node is locally changed Or: keep local copies of adjacent nodes broadcast message if color of a node (any node) is changed Some problems may require arbitration when the data of a node is changed. CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 /
Review Problems Described on Graphs Graph Representations Parallel Graph Algorithms Connected Components Maximal Independent Set Breath First Search CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 /
Next Class parallel sorting CPD (DEI / IST) Parallel and Distributed Computing 0-0-0 /