PREGEL: A SYSTEM FOR LARGE-SCALE GRAPH PROCESSING

Transcription

1 PREGEL: A SYSTEM FOR LARGE-SCALE GRAPH PROCESSING Grzegorz Malewicz, Matthew Austern, Aart Bik, James Dehnert, Ilan Horn, Naty Leiser, Grzegorz Czajkowski (Google, Inc.) SIGMOD 2010 Presented by : Xiu Zhang

2 Motivations Computation Models System Architecture False Toleration Applications Experiments

3 MOTIVATION 3

4 Motivation Large Graphs Computation is needed: Social Media Transportation

5 Motivation Documents=vertices Links=edges Web graph

6 Graph Algorithms Pattern Matching Search through the entire graph Identify similar components Traversals Define a specific start point Iteratively explore the graph Global measurements Compute one value for graph, based on all its vertices or edges

7 Challenges for Graph Algorithms Poor Locality of memory access Very little computation work required per vertex, however iterate many times Shortest Path Changing degree of parallelism over course of execution Connect Component Analysis

8 Possible Solutions Custom distributed frame work for each alg. Existing distributed computing platforms MapReduce unnecessarily slow, hard to implement Single-computer graph algorithm libraries Scale limitation Existing parallel graph systems Fault tolerance Parallel BGL and CGMgraph

9 Inspired by Valiant s Bulk Synchronous Parallel (BSP) mode Vertex centric computation

10 COMPUTATION MODEL 10

11 Computation Model(BSP) asynchronization Source: 11

12 : Message Passing Model Vertex: A unique identifier A modifiable, user defined value Edge: Source vertex and Target vertex identifiers A modifiable, user defined value

13 Basic Organization Supersteps: Iterations Invoke user defined function for each vertex Read messages sent to V in superstep S-1 Send messages that will be received in S+1 Modify the state of V and the outgoing edges Make topology changes Introduce/Delete/Modify edges(vertices) Votes to halt if no further work to do

14 State machine for a vertex Termination Condition All vertices are simultaneously inactive There are no messages in transit

15 Example Single Source Shortest Path Find shortest path from a source node to all target nodes Example taken from talk by Taewhi Lee,

16 Example: SSSP Parallel BFS in Inactive Vertex Active Vertex x x Edge weight Message 16

17 Example: SSSP Parallel BFS in Inactive Vertex Active Vertex 5 7 x x Edge weight Message

18 Example: SSSP Parallel BFS in Inactive Vertex Active Vertex x Edge weight 5 7 x Message

19 Example: SSSP Parallel BFS in Inactive Vertex Active Vertex x Edge weight x Message

20 Example: SSSP Parallel BFS in Inactive Vertex Active Vertex x Edge weight 5 7 x Message

21 Example: SSSP Parallel BFS in Inactive Vertex Active Vertex x Edge weight 5 7 x Message

22 Example: SSSP Parallel BFS in Inactive Vertex Active Vertex x Edge weight x Message

23 Example: SSSP Parallel BFS in Inactive Vertex Active Vertex x Edge weight 5 7 x Message

24 SYSTEM ARCHITECTURE 24

25 System Architecture system uses the master/worker model Master Coordinates workers Recovers faults of workers Worker Processes its task Communicates with the other workers Persistent data is in distributed storage system Temporary data is stored on local disk 25

26 Execution 26

27 Execution 27

28 Execution 28

29 Execution 29

30 Execution 30

31 FALSE TOLERANCE 31

32 Fault Tolerance Checkpointing The master periodically instructs the workers to save the state of their partitions to persistent storage e.g., Vertex values, edge values, incoming messages Failure detection Master uses regular ping messages to detect worker failures 32

33 Fault Tolerance Recovery The master reassigns graph partitions to the currently available workers The workers all reload their partition states from most recent available checkpoint 33

34 APPLICATIONS 34

35 PageRank the importance of a document the number of references to it the importance of the source documents themselves A = A given page T 1. T n = Pages that point to page A (citations) d = Damping factor between 0 and 1 (usually kept as 0.85) C(T) = number of links going out of T PR(A) = the PageRank of page A PR( A) (1 d) d PR( T1 ) ( C( T ) 1 PR( T2 ) C( T ) 2... PR( Tn ) ) C( T ) n 35

36 PageRank Courtesy: Wikipedia 36

37 PageRank Iterative loop till convergence Initial value of PageRank of all pages = 1.0; While ( sum of PageRank of all pages numpages > epsilon) { for each Page Pi in list { PageRank(Pi) = (1-d); for each page Pj linking to page Pi { PageRank(Pi) += d (PageRank(Pj)/numOutLinks(Pj)); } } } 37

38 Page Rank In

39 EXPERIMENTS 39

40 Experiments: (Shortest Paths) 1 billion vertex binary tree: varying number of worker tasks 40

41 Experiments: binary trees: varying graph sizes on 800 worker tasks 41

42 Experiments Log-normal random graphs, mean out-degree (thus over 127 billion edges in the largest case): varying graph sizes on 800 worker tasks 42

43 Conclusion Distributed system for large scale graph processing Think like a vertex computation model (intuitive API) 43

44 Limitations Inefficient if different regions of the graph converge at different speed Slowest machine Dense Graphs

45 THANK YOU ANY QUESTIONS?