Parallel Graph Algorithms

Transcription

1 Parallel Graph Algorithms Design and Analysis of Parallel Algorithms 5DV050/VT3

2 Part I Introduction

3 Overview Graphs definitions & representations Minimal Spanning Tree (MST) Prim s algorithm Single Source Shortest Paths (SSSP) Dijkstra s algorithm All-Pairs Shortest Paths (APSP) Dijkstra s algorithm Source partitioned Source parallel Floyd s algorithm Transitive closure Connected components

4 Graphs: Definitions Graph Graph G represented by the tuple G = (V, E), where V is the set of vertices and and E is the set of edges Undirected graph An edge e E is an unordered pair {u, v} where u, v V Edges drawn without arrows Directed graphs An edge or arc e E is an ordered pair (u, v) directed from u to v where u, v V Edge (u, v) drawn as an arrow from u to v Paths A path from v to v n consists of a sequence v, v 2,..., v n of vertices such that (v k, v k+ ) E for all k. Simple path: all vertices are distinct (v i v j if i j) Cycle: v = v n Acyclic graph: Graph containing no cycles

5 Examples of graphs Undirected Directed

6 Graphs: Properties A graph is connected if there exists a path between every pair of vertices A graph is complete if there exists an edge between every pair of vertices A graph G = (V, E ) is a sub-graph of graph G = (V, E) if V V and E E A tree is a connected acyclic graph A forest is a collection of trees A graph G = (V, E) is sparse if E is much smaller than V 2 Weighted graphs: A weighted graph G is represented by a 3-tuple G = (V, E, w), where w is a real-valued function that maps edges to weights (typically non-negative). The length of a path is the sum of the weights of all edges along the path.

7 Adjacency matrix Non-weighted graphs: { if (vi, v j ) E, a i,j = 0 otherwise Weighted graphs: w(v i, w j ) if (v i, v j ) E, a i,j = 0 if i = j, otherwise A = Suitable for dense graphs

8 Adjacency lists G = (V, E) is represented by the list Adj[... V ] of lists For each v V, Adj[v] is a list of all vertices that has an edge in common with v 2 3 Adj[] 2 Adj[2] Adj[3] Adj[4] Adj[5] Suitable for sparse graphs

9 Part II Minimum Spanning Tree (MST)

10 Minimum Spanning Tree (MST) A spanning tree of a graph G = (V, E) is a sub-graph of G that is also a tree and contains all vertices of G The weight of a (spanning) tree is the sum of the weights of all edges in the tree An MST is a spanning tree with minimum weight (If G = (V, E) is not connected, then it cannot have an MST but instead has a minimum spanning forest) We assume that G is connected, otherwise we first find the connected components and then find an MST of each component

11 Prim s algorithm Greedy algorithm: Start from a single vertex and add one edge at a time without creating cycles The constructed tree will be an MST (not obvious)

12 Prim s algorithm: Example (/6) a b c d d[] 0 5 e f a 2 3 f b 5 c e d

13 Prim s algorithm: Example (2/6) a b c d d[] 0 2 e 4 f a 2 3 f b 5 c e d

14 Prim s algorithm: Example (3/6) a b c d d[] 0 2 e 4 f 2 a 2 3 f b 5 c e d

15 Prim s algorithm: Example (4/6) a b c d d[] 0 2 e f 2 a 2 3 f b 5 c e d

16 Prim s algorithm: Example (5/6) a b c d d[] 0 2 e f 2 a 2 3 f b 5 c e d

17 Prim s algorithm: Example (6/6) a b c d d[] 0 2 e f 2 a 2 3 f b 5 c e d

18 Prim s algorithm : PRIM(V, E, w, r) 2: // Initialize 3: V T := {r} 4: d[r] := 0 5: for all v (V V T ) do 6: d[v] := w(r, v) if (r, v) E else d[v] := 7: end for 8: // Add edges one by one 9: while V T V do 0: Find vertex u (V V T ) such that d[u] = min{d[v] v (V V T )} : V T := V T {u} 2: for all v (V V T ) do 3: d[v] := min{d[v], w(u, v)} 4: end for 5: end while 6: // Note: Only the weight of the MST has been computed

19 Parallelizing Prim s algorithm d[v] is updated for all v Cannot parallelize outer while loop (unless sparse...) Instead we parallelize the inner for loop: min-map! Every process holds a block column of adjacency matrix A: A = [A A 2 ] A p and corresponding part of vector d Process P i responsible for updating its part of d Find global minimum (line 0) with all-reduce Update d in parallel (line 3)

20 Analysis of the parallel Prim s algorithm Per iteration: Computation (line 3): Θ(n/p) All-reduce (line 0): Θ(log 2 p) Total for n iterations: T P (n, p) = Θ(n 2 /p) + Θ(n log 2 p) T S (n) = Θ(n 2 ) Iso-efficiency condition: n = Ω(p log 2 p) Iso-efficiency function: W = n 2 = Ω(p 2 log 2 2 p)

21 Part III Single-Source Shortest Paths (SSSP)

22 Dijkstra s algorithm Dijkstra s algorithm: Essentially identical to Prim s algorithm, except Instead of d[u] store l[u]: the length from r to u Parallel Dijkstra s algorithm: Analogous to parallel Prim s algorithm

23 Parallel Dijkstra s algorithm with lower-bounded weights Parallel Dijkstra limited by n synchronization points Can reduce the number of sync points by expanding multiple edges at once Assume all edge weights w(u, v) After selecting vertex u with l[u] = l min we can also select vertices u where l[u ] l min +.

24 Part IV All-Pairs Shortest Paths (APSP)

25 All-Pairs Shortest Paths (APSP) APSP: Find the length of the shortest path between all pairs of vertices The lengths can be summarized in a square matrix D = (d i,j ) where d i,j is the length of the shortest path from vertex i to vertex j.

26 Dijkstra s algorithm applied to all-to-all shortest paths Source-partitioned approach: Distribute the sources Run sequential -vertex Dijkstra on all processors in parallel Complexity: Θ(n 2 ) Θ(n/p) = Θ(n 3 /p) Perfectly parallel Degree of concurrency limited to p = O(n) Each processor must have access to the entire graph Source-parallel approach: Partition processors into groups (e.g., of size p) Distribute sources over the groups Run parallel -vertex Dijkstra on all processor groups in parallel Complexity: [ Θ(n 2 / p) + Θ(n log 2 p) ] n/ p = Θ(n 3 /p) + Θ((n 2 / p) log 2 p) Degree of concurrency p = O(n 2 ) Each processor needs access only to a sub-graph

27 Floyd s algorithm Given G = (V, E, w) Let V k := {v,..., v k } (first k vertices of G) For any pair v i, v j V, consider all paths whose intermediate vertices belong to the subset V k Let p (k) i,j be the shortest such path and let d (k) i,j If the vertex v k is not in the path, then p (k) i,j be its length = p (k ) i,j If the vertex v k is in the path, then it can be split into two paths: One from v i to v k and one from v k to v j where both paths use vertices only from V k In that case, the weight of the path is d (k) i,j = d (k ) i,k + d (k ) k,j

28 Floyd s algorithm : FLOYD(A) 2: D (0) := A 3: for k = to n do 4: for i = to n do 5: for j = to n do 6: d (k) i,j 7: end for 8: end for 9: end for := min{d (k ) i,j, d (k ) i,k + d (k ) k,j } Similar in structure to matrix multiplication Parallelized using SUMMA-style algorithm

29 Part V Transitive closure

30 Transitive closure If G = (V, E) is a graph, then its transitive closure is the graph G = (V, E ), where E := {(v i, v j ) exists path from v i to v j in G} Gives the connectivity matrix A such that a i,j = if i = j or a path from v i to v j exists in G and a i,j = otherwise Method : Set the weights in G to and solve an all-pairs shortest paths problem followed by a reinterpretation of the output Method 2: Modify Floyd s algorithm by replacing min with logical or and + with logical and: d (k) i,j := d (k ) i,j or (d (k ) i,k and d (k ) k,j )

31 Part VI Connected components

32 Connected components Partition V into maximal disjoint subsets C, C 2,..., C r such that V = C C 2 C r and u, v C i if and only if u is reachable from v and vice versa The graph below has three connected components

33 Depth-first search based algorithm Perform depth-first traversal of the graph to generate a spanning forest Each tree in the forest defines a connected component Parallel formulation: Give sub-graph G i = (V, E i ) to process P i Perform sequential algorithm on each sub-graph in parallel Merge forests pair-wise using log 2 p steps

34 Forest merging find(x): Tree to which x belongs union(u, v): Merge trees to which u and v belong Merge forest A into forest B: Send A to processor holding B For each edge (u, v) (there are at most n ) in A: find(u) in B find(v) in B Same tree? Do nothing Different trees? union(u, v) in B Discard A, continue with B Using appropriate set data structure and algorithms, find(x) and union(u, v) have expected constant time complexity Complexity (D block partitioning): T P (n, p) = Θ(n 2 /p) + Θ(n log 2 p)

35 Part VII Johnson s algorithm

36 Johnson s algorithm (SSSP) Recall Dijkstra s algorithm: Find unprocessed u such that d[u] = min{d[v] v unprocessed} Update for all unprocessed vertices v: d[v] := min{d[v], d[u] + w(u, v)} Dijkstra s algorithm assumes dense graph For sparse graph, we instead use Johnson s algorithm which stores unprocessed vertices in a priority queue based on d[v] (smallest value first) Take minimum distance vertex from the priority queue Update adjacent vertices

37 Johnson s algorithm : JOHNSON(V, E, r) 2: Q := V 3: for all v Q do 4: d[v] := 5: end for 6: d[r] := 0 7: while Q do 8: u := ExtractMin(Q) 9: for each v adjacent to u do 0: if v Q and d[u] + w(u, v) < d[v] then : d[v] := d[u] + w(u, v) 2: end if 3: end for 4: end while

38 Parallel Johnson s (centralized queue) Maintain Q at a centralized location Processors compute new values and request updates of Q Major bottleneck No asymptotic speedup, since O( E ) updates that are serialized and each take time O(log 2 n) leading to the same complexity as the sequential formulation Moreover, only E / V vertices can be updated in parallel in each iteration, and this number is small since the graph is assumed sparse

39 Parallel Johnson s (distributed queue) Distributed queue: Manage Q using distributed algorithm Requires a machine with very low latency to be practical Even if the update complexity is reduced from O(log 2 n) to O(), we can expect no more than a O(log 2 n) speedup since the updates are applied sequentially Safe vertices: Let u be a vertex with minimal d[u] All vertices v with d[v] = d[u] can be processed in parallel If we know that the minimum edge weight is, then we can relax this to all vertices v with d[v] d[u] +

40 Parallel Johnson s (unsafe vertices) Process also unsafe vertices in parallel Leads to an algorithm that is no longer equivalent to the sequential algorithm Process p vertices at the top of Q in parallel Each process maintains its own priority queue The distances might no longer correspond to the shortest paths Detect instances of wrongly computed distances and re-process the corresponding vertices

41 Part VIII Weighted matchings

42 Weighted matchings A matching M(G) of a graph G = (V, E) is any sub-graph of G where each vertex is incident to at most one edge Let w(e) be the weight of an edge e We define the weight w(g) of a graph G as the sum of all edge weights A maximum weighted matching M (G) of G is a matching whose weight w(m (G)) is maximum among all matchings of G

43 Matchings and parallel graph partitioning

44 Sequential greedy weighted matching algorithm : MATCHING(V, E) 2: M(G) := 3: while E do 4: Pick locally heaviest edge e from E 5: Add e to M(G) 6: Remove e = {u, v} and all edges incident to u and v from E 7: end while Approximates a maximal matching within a factor 2

45 Parallel greedy weighted matching algorithm For each vertex v in parallel: : PMATCHING(V, E) 2: R := 3: Initialize N to the neighborhood of v (all vertices adjacent to v) 4: Let the candidate c be the vertex connected to v by the locally heaviest edge in N 5: if c then 6: Send req to c 7: end if 8: while N do 9: Receive message m from vertex u 0: If m = req, then R := R {u} : If m = drop, then N := N {u} and update c if u = c and if c send req to c 2: If c and c R, send drop to all vertices in N except c 3: end while