DNA Interaction Network

Transcription

1

2 Social Network Web Network Social Network DNA Interaction Network Follow Network User-Product Network

3

4

5 Nonuniform network comm costs Contentiousness of the memory subsystems Nonuniform comp requirement Nonuniform comm requirement Time-varying skewness

6 Architecture- and Workload-Aware Graph (Re)Partitioning Aragon [BigGraphs 14] Planar+ [To submit 17] (small dynamic graphs) (large dynamic graphs) Paragon [EDBT 16] Argo [BigData 16] (median-size dynamic graphs) (static graphs) Planar [ICDE 16] Sargon [ICDE 17] (large dynamic graphs) (skew-resistant)

7 Architecture- and Workload-Aware Graph (Re)Partitioning Aragon [BigGraphs 14] Planar+ [To submit 17] (small dynamic graphs) (large dynamic graphs) Paragon [EDBT 16] Argo [BigData 16] (median-size dynamic graphs) (static graphs) Planar [ICDE 16] Sargon [ICDE 17] (large dynamic graphs) (skew-resistant)

8

9 Planar Sk+4 Sk+5 Planar Sk+2 Planar Sk+1 Planar Planar Sk Migration Planning What vertices to move? Where to move? Phase-2: Physical Vertex Migration Perform the Migration Plan Phase-3: Convergence Check Still beneficial? Phase-1: Logical Vertex Migration Phase-1a: Minimizing Comm Cost Phase-1b: Ensuring Balanced Partitions

10 Planar Sk+4 Sk+5 Planar Sk+2 Planar Sk+1 Planar Planar Sk Migration Planning What vertices to move? Where to move? Phase-2: Phase-2:Physical Vertex Location Vertex Migration Update Each vertex has up-to-date Perform the Migration Plan locations of their neighbors Phase-3: Convergence Check Still beneficial? Phase-1: Logical Vertex Migration Phase-1a: Minimizing Comm Cost Phase-1b: Ensuring Balanced Partitions

11 Physical Vertex Migration Starts Repartitioning Sk+2 Planar Planar Sk+4 Sk+5 Planar Sk+1 Planar Sk Planar Converge

12

13 Socket 0 core L1 L2 Socket 1 core core L1 L1 L2 L2 QPI/ HT L3 Inter-socket Link Controller Memory Controller Socket 0 core core L1 L1 L2 L2 L3 Inter-socket Link Controller Memory Machine 0 Socket 1 core core L1 L1 L2 L2 QPI/ HT L3 Memory Controller Memory Inter-socket Link Controller Memory Controller core L1 L2 L3 Inter-socket Link Controller Memory Machine 1 Memory Controller Memory

14

15

16

17 Socket 0 core L1 L2 Socket 1 core core L1 L1 L2 L3 Inter-socket Link Controller L2 QPI/ HT Memory Controller Socket 0 core core L1 L1 L2 L2 L3 Inter-socket Link Controller Memory Machine 0 Socket 1 core core L1 L1 L2 L3 Memory Controller Memory Inter-socket Link Controller L2 QPI/ HT Memory Controller core L1 L2 L3 Inter-socket Link Controller Memory Machine 1 Memory Controller Memory

18 λ 1x 1.18x Hours CPU Time Saving 1.5x 1.7x PARAGON 25h PLANAR 27h PLANAR+ 43h uniplanar+ 10h 2.8x

19 λ 1x 1.18x Hours CPU Time Saving 1.5x 1.7x PARAGON 25h PLANAR 27h PLANAR+ 43h uniplanar+ 10h 2.8x

20

21 Architecture- and Workload-Aware Graph (Re)Partitioning Aragon [BigGraphs 14] Planar+ [To submit 17] (small dynamic graphs) (large dynamic graphs) Paragon [EDBT 16] Argo [BigData 16] (median-size dynamic graphs) (static graphs) Planar [ICDE 16] Sargon [ICDE 17] (large dynamic graphs) (skew-resistant)

22

23

24 Vertex Stream Partitioner......

25

26

27 Bottleneck Network Memory

28

29 50x 38x 4x 9x 6x 3x 1x 12x 9x 1x 1.2x 1x

30

31

32 SSSP Execution Time (s) m:s:c METIS LDG 1:2: ,632 2:2: ,565 4:2: :2: x

33 SSSP Execution Time (s) SSSP LLC Misses (in Millions) m:s:c m:s:c METIS LDG 1:2: ,632 2:2: ,565 4:2: :2:1 222 METIS LDG 1:2:8 10,292 44,117 2:2:4 10,626 44, :2:2 2,541 1, :2: x 235x

34 SSSP Execution Time (s) SSSP LLC Misses (in Millions) m:s:c m:s:c METIS LDG 1:2: ,632 2:2: ,565 4:2: :2:1 222 METIS LDG 1:2:8 10,292 44,117 2:2:4 10,626 44, :2:2 2,541 1, :2: x 235x

35 SSSP Execution Time (s) SSSP LLC Misses (in Millions) m:s:c m:s:c METIS LDG METIS LDG 1:2: ,632 1:2:8 10,292 44,117 2:2: ,565 2:2:4 10,626 44,689 4:2: :2:2 2,541 1,061 8:2: :2: x METIS had lower execution time and LLC misses than LDG. Edge-cut matters. Higher edge-cut-->higher comm-->higher contention 235x

36

37 Architecture- and Workload-Aware Graph (Re)Partitioning Aragon [BigGraphs 14] Planar+ [To submit 17] (small dynamic graphs) (large dynamic graphs) Paragon [EDBT 16] Argo [BigData 16] (median-size dynamic graphs) (static graphs) Planar [ICDE 16] Sargon [ICDE 17] (large dynamic graphs) (skew-resistant)

38

39

40

41

42

43 Assign a label vector to each vertex to indicate: the time periods the vertex is active in whether it is a high- or low-degree vertex the hotness of the vertex

44 Vertex Stream Partitioner......

45

46 Workloads BFS and SSSP (one randomly selected source vertex) Dataset Orkut ( V =3M, E =234M) # of Traces Collected 5 Similarity Percentage of the vertices overlapped in the peak superstep Workloads Avg. Similarity Std. Deviation BFS 60.80% 8.43% SSSP 64.73% 10.63%

47 2x 1.68x 1.57x 1x Up to 2x speedups (hours CPU time saving).

48

49 Thanks! Architecture- and Workload-Aware Graph (Re)Partitioning Aragon [BigGraphs 14] Planar+ [To submit 17] (small dynamic graphs) (large dynamic graphs) Paragon [EDBT 16] Argo [BigData 16] (median-size dynamic graphs) (static graphs) Planar [ICDE 16] Sargon [ICDE 17] (large dynamic graphs) (skew-resistant)

50