King Abdullah University of Science and Technology. CS348: Cloud Computing. Large-Scale Graph Processing

Transcription

1 King Abdullah University of Science and Technology CS348: Cloud Computing Large-Scale Graph Processing Zuhair Khayyat 10/March/2013

2 The Importance of Graphs A graph is a mathematical structure that represents pairwise relations between entities or objects. Such as: Physical communication networks Web pages links Social interaction graphs Protein-to-protein interactions Graphs are used to abstract application-specific features into a generic problem, which makes Graph Algorithms *

3 Graph algorithm characteristics* Data-Drivin Computations: Computations in graph algorithms depends on the structure of the graph. It is hard to predict the algorithm behavior Unstructured Problems: Different graph distributions requires distinct load balancing techniques. Poor Data Locality. High Data Access to Computation Ratio: Runtime can be dominated by waiting memory fetches. *Lumsdaine et. al, Challenges in Parallel Graph Processing

4 Challenges in Graph processing Graphs grows fast; a single computer either cannot fit a large graph into memory or it fits the large graph with huge cost. Custom implementations for a single graph algorithm requires time and effort and cannot be used on other algorithms Scientific parallel applications (i.e. parallel PDE solvers) cannot fully adapt to the computational requirements of graph algorithms*. Fault tolerance is required to support large scale processing. *Lumsdaine et. al, Challenges in Parallel Graph Processing

5 Why Cloud in Graph Processing Easy to scale up and down; provision machines depending on your graph size. Cheaper than buying a physical large cluster. Can be used in the cloud as Software as a services to support online social networks.

6 Large Scale Graph Processing Systems that tries to solve the problem of processing large graphs in parallel: MapReduce auto task scheduling, distributed disk based computations: Pegasus X-Rime Pregel - Bulk Synchronous Parallel Graph Processing: Giraph GPS Mizan GraphLab Asynchronous Parallel Graph

7 Pregel* Graph Processing Consists of a series of synchronized iterations (supersteps); based on Bulk Synchronous Parallel computing model. Each superstep consists of: Concurrent computations Communication Synchronization barrier Vertex centric computation, the user's compute() function is applied individually on each vertex, which is able to: Send message to vertices in the next superstep Receive messages from the previous superstep *Malewicz et. al., Pregel: A System for Large-Scale Graph Processing

8 Pregel messaging Example 1 Superstep 0 A B D C

9 Pregel messaging Example 1 Superstep 0 Superstep 1 A B A 22 B 9 15 D C D C 47

10 Pregel messaging Example 1 Superstep 0 Superstep 1 A B A 22 B 9 15 D C D C 47 Superstep , 9 A B D 14 C 15

11 Pregel messaging Example 1 Superstep 0 Superstep 1 A B A 22 B 9 15 D C D C 47 Superstep , 9 Superstep 3 5-2, 7 A B A B D 14 C D 9 C 55

12 Vertex's State All vertices are active at superstep 1 All active vertices runs user function compute() at any superstep A vertex deactivates itself by voting to halt, but returns to active if it received messages. Pregel terminates of all vertices are inactive

13 Pregel Example 2 Data Distribution (Hash-based partitioning) Worker 1 Worker 2 Worker 3 Computation Communication Synchronization Barrier Terminate Yes Done? No

14 Pregel Example 3 Max

15 Pregel Example 3 Max

16 Pregel Example 3 Max

17 Pregel Example 3 Max

18 Pregel Example 4 Max code Vertex value class Class MaxFindVertex:public Vertex<double, void, double> { public: virtual void Compute(MessageIterator* msgs) { int currmax = GetValue(); SendMessageToAllNeighbors(currMax); for ( ;!msgs->done(); msgs- >Next()) { if (msgs->value() > currmax) currmax = msgs->value(); Edge value class Message class Send current Max Check messages and store max Store new max

19 Pregel Message Optimizations Message Combiners: A special function that combines the incoming messages for a vertex before running compute() Can run on the message sending or receiving worker Global Aggregators : A shared object accessible to all vertices. that is synchronized at the end of each superstep, i.e., max and min aggregators.

20 Pregel Guarantees Scalability: process vertices in parallel, overlap computation and communication. Messages will be received without duplication in any order. Fault tolerance through check points

21 Pregel's Limitations Pregel's superstep waits for all workers to finish at the synchronization barrier. That is, it waits for the slowest worker to finish. Smart partitioning can solve the load balancing problem for static algorithms. However not all algorithms are static, algorithms can have a variable execution behaviors which leads to an unbalanced supersteps.

22 *Khayyat et. al., Mizan: A System for Dynamic Load Balancing in Large-scale Graph Processing Mizan* Graph Processing Mizan is an open source graph processing system, similar to Pregel, developed locally at KAUST. Mizan employs dynamic graph repartitioning without affecting the correctness of graph processing to rebalanced the execution of the supersteps for all types of workloads.

23 Source of Imbalance in BSP

24 Source of Imbalance in BSP

25 Types of Graph Algorithms Stationary Graph Algorithms: Algorithms with fixed message distribution across superstep All vertices are either active or inactive at same time i.e. PageRank, Diameter Estimation and weakly connected components. Non-stationary Graph Algorithms Algorithms with variable message distribution across supersteps Vertices can be active and inactive independent to others

26 Mizan architecture Each Mizan worker contains three distinct main components: BSP Processor, communicator and storage manager. The distributed hash table (DHT) is used to maintain the location of each vertex The migration planner interacts with other components during the BSP barrier

27 Mizan's Barriers

28 Dynamic migration: Statistics Mizan monitors the following for every vertex: Response time Remote outgoing messages Incoming messages

29 Dynamic migration: planning Mizan's migration planner runs after the BSP barrier and creates a new barrier. The planning includes the following steps: Identifying unbalanced workers. Identifying migration objective: Response time Incoming messages Outgoing messages

30 Mizan's Migration Work-flow

31 Mizan PageRank Compute() Example void compute(messageiterator<mdouble> * messages, uservertexobject<mlong, mdouble, mdouble, mlong> * data,messagemanager<mlong, mdouble, mdouble, mlong> * comm) { double currval = data >getvertexvalue().getvalue(); double newval = 0; double c = 0.85; while (messages >hasnext()) { double tmp = messages >getnext().getvalue(); newval = newval + tmp; } Processing Messages } newval = newval * c + (1.0 c) / ((double) vertextotal); mdouble outval(newval / ((double) data >getoutedgecount())); if (data >getcurrentss() <= maxsuperstep) { for (int i = 0; i < data >getoutedgecount(); i++) { comm >sendmessage(data >getoutedgeid(i), outval); data >getoutedgeid(i); } } else { data >votetohalt(); } data >setvertexvalue(mdouble(newval)); Termination Condition Sending to Neighbors

32 Mizan PageRank Combiner Example void combinemessages(mlong dst, messageiterator<mdouble> * messages,messagemanager<mlong, mdouble, mdouble, mlong> * mmanager) { double newval = 0; while (messages >hasnext()) { double tmp = messages >getnext().getvalue(); newval = newval + tmp; } } mdouble messageout(newval); mmanager >sendmessage(dst,messageout);

33 Mizan Max Aggregator Example class maxaggregator: public IAggregator<mLong> { Public: mlong aggvalue; maxaggregator() { aggvalue.setvalue(0); } void aggregate(mlong value) { if (value > aggvalue) { aggvalue = value; } } mlong getvalue() { return aggvalue; } void setvalue(mlong value) { this >aggvalue = value; } }; virtual ~maxaggregator() {}

34 Class Assignment Your assignment is to configure, install and run Mizan on a single Linux machine throw following this tutorial: By the end of the tutorial, you should be able to execute the command on your machine: mpirun np 2./Mizan 0.1b u ubuntu g web Google.txt w 2 Deliverables: you store the output of of the above command and submit it by Wednesday's class. Any questions regarding the tutorial or to get an account for a Ubuntu machine, contact me on: zuhair.khayyat@kaust.edu.sa