Fast-Response Multipath Routing Policy for High-Speed Interconnection Networks

Transcription

1 HPI-DC 09 Fast-Response Multipath Routing Policy for High-Speed Interconnection Networks Diego Lugones, Daniel Franco, and Emilio Luque Leonardo Fialho Cluster 09 August 31 New Orleans, USA

2 Outline Scope Congestion Problem Congestion Control Approaches FR-DRB Performance Evaluation Conclusions

3 Scope Clusters Computing Systems: Alternative to massive parallel computers Current use: High Performance Computing Systems (HPC) Internet Servers Storage Area Network (SANs) Usually based on high-speed interconnection networks, which provide faster communication between processing nodes. System Architecture Charts extracted from the top500 list of supercomputers. ( nov-2008)

4 Top500 Scope High-speed interconnection networks: Myrinet, InfiniBand, Quadrics Main features: High bandwidth, Low latencies Additional features: Lossless networks, Flexible topologies Network performance may be affected by congestion Interconnect family Top100 Charts extracted from the top500 list of supercomputers. ( nov-2008)

5 Topologies of interest : Direct Networks - Meshes 2D/3D - Torus 2D/3D Indirect Networks (MINs) - Fat-trees - Butterflies Tradeoff between cost and performance Are able to exploit locality Provide alternative paths Scope

6 Outline Scope Congestion Problem Congestion Control Approaches FR-DRB Performance Evaluation Conclusions

7 Congestion Problem Network contention: Several packets request the same output port One makes progress, the others wait Network congestion: Persistent network contention Network performance degradation (High latency) Local congestion is produced even under low traffic loads (Low throughput) It is quickly propagated by flow control (in lossless nets) Caused by: No match between communication load and network topology Inefficient resource utilization

8 Outline Scope Congestion Problem Congestion Control Approaches FR-DRB Performance Evaluation Conclusions

9 Congestion Control Approaches Static solutions: - Based on increasing network resources more than strictly necessary (over-provisioning) Reactive solutions: - Message Throttling - Buffers Management - Adaptive Load Balancing Based on 3 phases: - Traffic Monitoring - Congestion Notification - Congestion Control

10 Outline Scope Congestion Problem Congestion Control Approaches FR-DRB Performance Evaluation Conclusions

11 Fast-Response Dynamic Routing Balancing FR-DRB is a new routing policy for high-speed interconnection networks Design goals: Provide communication load balancing to improve network throughput using alternative paths between source and destination nodes Minimize reaction speed to congestion Congestion Control: Sources are notified about congestion in order to distribute traffic load over multiple paths simultaneously to avoid the congested area. Implementation requirements: Performance counters in switches (IBA like) Deterministic routing (as base routing) Multipath feature (IBA like)

12 FR-DRB FR-DRB basic procedure: A watchdog timer is started when packet is injected Packet latency is monitored at switches ports and transported by packets themselves. Congestion is detected when timer exceeds a limit value Source nodes perform a path expansion to increase the effective bandwidth between source and destination Packet latency is sent back to the source node using an acknowledge message In congestion absence an Ack message is sent back to stop the watchdog timer

13 FR-DRB Interconnection network Node level Lat<Thres Channel level Free output port? yes yes no Link level Communication tasks Monitoring-Notification Fast Decision Control

14 Latency monitoring Packet latency is monitored at switches ports and transported by packets themselves. Notification is performed by Ack s Timer S D Generate Ack

15 Path expansion Path expansion is performed by aggregating alternative paths to the original one Alternative paths are created selecting two intermediate nodes which are neighbours of source and destination nodes. Packets are routed from Src IN1 IN2 Dst Metapath Original path Alternative path IN1 IN2

16 Intermediate nodes selection According to the central node in direct networks According to the common ancestors in indirect networks SRC DST Common Ancestors

17 Metapath configuration Executed when the Ack is received, or the watchdog timer reaches the time limit Metapath latency L(Mp) is calculated Metapath AP 1 Src Node AP 2 AP 3 L(Mp) is compared to a congestion index (threshold) in order to dinamically expand or constrict the metapath

18 Alternative path selection Alternative paths are selected according to their latencies Paths with lower latencies are more frequently used L (AP 1) Src A L (AP 2) AP i ~ 1/L (AP i) L (AP 3) L (AP 1 ) Shared path Src B L (AP 2 ) L (AP 3 ) - Selection is based on the information of src-dst path - However, a collective effect is achieved

19 Outline Scope Congestion Problem Congestion Control Approaches FR-DRB Performance Evaluation Conclusions

20 Performance Evaluation Workloads - Specific pattern (Hot-spot) - Benchmark traffic Matrix transpose Shuffle permutation Bit reversal Routing policies Torus network - Deterministic routing (Dimension Order Routing, DOR) - Minimal Adaptive routing (Turn Model) - Oblivious routing (Valiant) - Dynamic Routing Balancing (DRB) Fat tree network - Cycle priority (CP) - First Free (FF)

21 Performance Evaluation Metrics - Load balancing (Latency surfaces) - Average latency - Average Throughput - Transient response Simulation assumptions: Fat-tree topology (64 switches, 4 nodes/sw) Torus topology (1024 switches, 4 nodes/sw) 64 KB memories at input/output ports Serial full-duplex pipelined links (BW=1 Gbps) 4KB packets Max. Metapath size: 6

22 Communication Load Balancing Specific pattern: (Quadruple Hot-spot) - Pattern designed to achieve a heavy communication unbalance - Four different sets of destinations are selected with higher probability by senders - Deterministic routing performs the worst congestion case

23 Communication Load Balancing Specific pattern: (Quadruple Hot-spot) - Performance comparison (higher bounds) Lower worst-case latencies

24 Average metrics Specific pattern: (Quadruple Hot-spot) - Accepted load [bits/us] vs. Offered load [bits/us] - Average End-to-End latency [ms] vs. Accepted load [bits/us]

25 Transient response Specific pattern: (Quadruple Hot-spot) - Latency vs. time

26 Torus network Benchmark Traffic Average metrics (Benchmarks)

27 Fat-tree network Benchmark Traffic Average metrics (Benchmarks)

28 Outline Scope Congestion Problem Congestion Control Approaches FR-DRB Performance Evaluation Conclusions

29 Conclusions FR-DRB controls the performance degradation produced by packet congestion in network resources. A global load balancing is achieved because source nodes involved in congestion are notified about latency state as fast as it starts. Congestion is controlled configuring a metapath with new and less loaded alternatives paths which tries to maintain the throughput high. Thanks to FR-DRB the efficient management of network resources allows a heavy communication load without increasing the cost.

30 HPI-DC 09 Fast-Response Multipath Routing Policy for High-Speed Interconnection Networks Diego Lugones, Daniel Franco, and Emilio Luque Leonardo Fialho Cluster 09 August 31 New Orleans, USA