TCEP: Traffic Consolidation for Energy-Proportional High-Radix Networks

Transcription

1 TCEP: Traffic Consolidation for Energy-Proportional High-Radix Networks Gwangsun Kim Arm Research Hayoung Choi, John Kim KAIST

2 High-radix Networks Dragonfly network in Cray XC30 system 1D Flattened butterfly (fully connected) Image source: Cray A large number of narrow links low network diameter, high path diversity Energy-proportionality can be challenging Links use high-speed signaling High energy consumption regardless of load ( Idle packets transmitted) 2D Flattened butterfly (fully connected within each dimension)

3 Motivation Data center networks can be significantly underutilized Resources provisioned to meet peak demand Low link utilization measured by Facebook Network energy waste can be high at low system utilization Exploit link power-gating opportunity in high-radix routers [Rot et al., SIGCOMM 15] [Abts et al., ISCA 10]

4 Motivation Data center networks can be significantly underutilized Resources provisioned to meet peak demand Low link utilization measured by Facebook Network energy waste can be high at low system utilization Exploit link power-gating opportunity in high-radix routers Link power-gating challenges: - How to maximize power reduction? - How to keep network connected? - How to achieve scalability? - How to minimize performance impact? - How to load-balance network? [Rot et al., SIGCOMM 15] [Abts et al., ISCA 10]

5 Contents Background / Motivation Traffic consolidation Maintaining connectivity Criteria for selecting links to power-gate Power-aware load-balanced routing Evaluation Conclusion

6 Traffic Consolidation Energy-proportionality requires aggressive power-gating Consolidate flows onto fewer links thru non-minimal routing Flow 0: 50% link util. Flow 1: 25% link util. Router Router Flow 0: 50% link util. Traffic consolidation Flow 1: 50% link util. Flow 2: 50% link util. Flow 2: 50% link util. Flow 1: 25% link util.

7 Subnetwork-based Distributed Approach Subnetwork: routers that are fully connected in a dimension Independently manage power with local information R4 R5 R6 R7 R8 R Fully connected within each dimension 2D Flattened butterfly

8 Subnetwork-based Distributed Approach Subnetwork: routers that are fully connected in a dimension Independently manage power with local information R4 R5 R6 R7 Row subnetworks R8 R D Flattened butterfly

9 Subnetwork-based Distributed Approach Subnetwork: routers that are fully connected in a dimension Independently manage power with local information Column subnetworks R4 R5 R6 R7 R8 R D Flattened butterfly

10 Subnetwork-based Distributed Approach Subnetwork: routers that are fully connected in a dimension Independently manage power with local information Consists of a single subnetwork R4 R5 R6 R7 R8 R D Flattened butterfly 1D Flattened butterfly

11 Root Network Maintaining Connectivity Constantly checking connectivity can incur high overhead Subset of links that are always ON to keep all nodes connected Star topology minimal # of links and low network diameter Root network links Other links Rearrange the root network R7 R4 R6 R5 Root network for 1D Flattened butterfly Max. hop count = 2

12 Root Network for Higher Dimensions Star topology is formed within each subnetwork Root network links Other links R4 R5 R6 R7 R8 R Root network for 2D Flattened butterfly

13 Root Network for Higher Dimensions Star topology is formed within each subnetwork Further reducing ON links? too complex, little added benefit (4.3% for radix-64 routers) Root network links Other links R4 R5 R6 R7 R8 R Root network for 2D Flattened butterfly

14 Root Network for Higher Dimensions Star topology is formed within each subnetwork Further reducing ON links? too complex, little added benefit (4.3% for radix-64 routers) Other links can be power-gated without affecting connectivity Root network links Other links R4 R5 R6 R7 R8 R Root network for 2D Flattened butterfly

15 Observation on Maximizing Path Diversity Which links should be ON for high path diversity? OFF ON (root network) ON (additional links) Provides better path diversity Approach 1: distribute ON links Approach 2: concentrate ON links

16 Hub Routers for High Path Diversity Hub routers created small-world network Similarly, airlines create hub airports to reduce cost Quantitative results: 1D Flattened butterfly (32 routers, 1024 nodes) No non-minimal paths with more than 2 hops Random distribution: average from 10,000 samples Normalized number of total paths Hub airport Concentrate to "hub" routers Total # of paths improved by 1.9x Randomly distribute Fraction of active links (%) Edge: Direct flights by United Airlines

17 Hub Routers for High Path Diversity Hub routers created small-world network Similarly, airlines create hub airports to reduce cost Quantitative results: 1D Flattened butterfly (32 routers, 1024 nodes) No non-minimal paths with more than 2 hops Normalized number of total paths TCEP concentrates ON links to a small number of hub routers. Random distribution: average from 10,000 samples Hub airport Concentrate to "hub" routers Total # of paths improved by 1.9x Randomly distribute Fraction of active links (%) Edge: Direct flights by United Airlines

18 Hub Routers for High Path Diversity Hub routers created small-world network Similarly, airlines create hub airports to reduce cost Quantitative results: 1D Flattened butterfly (32 routers, 1024 nodes) No non-minimal paths with more than 2 hops Random distribution: average from 10,000 samples Normalized number of total paths Concentrate to "hub" routers Total # of paths improved by 1.9x Randomly distribute Fraction of active links (%)

19 Observation on Minimizing Impact on Network Differentiate the type of traffic (minimally vs. non-minimally routed) Making power-gating decision at OFF Minimally routed ON Power-gate Candidate 1 Increased hop count & BW usage Non-minimally routed Power-gate Candidate 2 The same hop count & BW usage

20 Observation on Minimizing Impact on Network Differentiate the type of traffic (minimally vs. non-minimally routed) OFF Making power-gating decision at Minimally routed ON Power-gate Candidate 1 Increased hop count & BW usage TCEP prioritizes power-gating links with the least amount of minimally routed traffic. Non-minimally routed Power-gate Candidate 2 The same hop count & BW usage

21 Problem with Load-balanced Routing No global link state information non-minimal path can become significantly longer Baseline non-minimal routing: Source Intermediate (INTM) router Destination SRC R4 R5 R6 Congestion R7 R8 R9 0 INTM DEST Baseline (no power-gating) 4 hops

22 Problem with Load-balanced Routing No global link state information non-minimal path can become significantly longer Baseline non-minimal routing: Source Intermediate (INTM) router Destination Some links are OFF! SRC R4 R5 R6 Congestion R7 SRC R4 R5 R6 Congestion R7 INTM INTM R8 R9 0 1 R8 R DEST Baseline (no power-gating) 4 hops With power-gating 5 DEST 8 hops

23 Problem with Load-balanced Routing No global link state information non-minimal path can become significantly longer Baseline non-minimal routing: Source Intermediate (INTM) router Destination SRC R4 R5 R6 Congestion R7 SRC R4 R5 R6 Congestion R7 INTM INTM R8 R9 0 1 R8 R DEST Baseline (no power-gating) 4 hops With power-gating 5 DEST 8 hops

24 Proposed PAL Routing PAL: Power-Aware progressive Load-balanced (Routing) Instead of global randomization, locally randomize Compared to the baseline load-balanced routing: The same maximum hop count (2 hops in each dimension) No additional virtual channels (dimension-order routing) INTM_Y INTM_X R4 SRC R5 R6 Congestion R7 DEST_X R8 R hops DEST

25 Other Issues Addressed in the Paper Challenge: the observations can lead to different decisions We propose a low-complexity algorithm to reconcile them Only one link turned on/off from a router per epoch avoid supply voltage shift Routers keep track of non-minimal paths within a subnetwork More details in the paper..

26 Methodology Booksim: cycle-accurate interconnection network simulator SST/Macro + Booksim for real workload evaluation Previous work compared: SLaC (Staged Laser Control) [HPCA 16] A stage corresponds to a row of routers in 2D Flattened butterfly Turn on/off a stage based in a coarse-grained manner Network configuration: Parameter Topology Virtual channel Link wake-up delay Value 512-node 2D Flattened butterfly 6 VCs for baseline, 7 VCs for TCEP and SLaC 1 µs (=epoch length. Sensitivity study in paper)

27 Synthetic Traffic Results Performance Average packet latency (cycles) Energy Normalized energy per flit Uniform random traffic pattern Baseline SLaC TCEP Injection rate (flits/cycle/node) Uniform random traffic pattern Baseline SLaC TCEP Injection rate (flits/cycle/node) Average packet latency (cycles) Normalized energy per flit Bit reverse traffic pattern Baseline SLaC TCEP 7x throughput difference Injection rate (flits/cycle/node) Bit reverse traffic pattern Baseline SLaC TCEP Up to 73% lower energy Injection rate (flits/cycle/node)

28 Multi-workload Scenario Results Two different batch workloads running simultaneously High and low injection rates 100 random node mappings Energy Ratio (SLaC/TCEP) Uniform random traffic pattern % lower energy by TCEP Mapping number Runtime was similar (within 0.3%) Random permutation traffic pattern Energy Ratio (SLaC/TCEP) % lower energy by TCEP Mapping number TCEP was x faster than SLaC

29 Real Workload Results Packet latency SLaC significantly increased latency for some workloads Normalized average packet latency Network energy Baseline SLaC TCEP 4.51 Significant energy savings by both SLaC and TCEP HILO BoxMG MG NB FB BigFFT GMEAN Overall, 46% lower compared to SLaC Normalized energy Baseline SLaC TCEP HILO BoxMG MG NB FB BigFFT GMEAN

30 Conclusion TCEP consolidates traffic to proactively power-gate links Key observations Concentrating ON links to hub routers for high path diversity Differentiate minimally and non-minimally routed traffic PAL (Power-Aware progressive Load-balanced) routing Load-balanced routing without global link state information Keep the max. hop count the same as baseline without power-gating Results (compared to SLaC) For adversarial traffic patterns, up to 7x higher throughput For multi-workload scenarios, up to ~70% lower energy and runtime