TDT Appendix E Interconnection Networks

Transcription

1 TDT 4260 Appendix E Interconnection Networks

2 Review Advantages of a snooping coherency protocol? Disadvantages of a snooping coherency protocol? Advantages of a directory coherency protocol? Disadvantages of a directory coherency protocol? Why must a given machine architecture expose a memory consistency model architecturally? What is a strict memory model? What is a weak memory model?

3 Conceptual overview

4 Motivation Basic network technology assumed known Motivation Increased importance System-to-system connections Intra system connections Increased demands Bandwidth, latency, reliability,... Vital part of system design

5 Types of networks Number of devices and distance OCN On-chip network Functional units, register files, caches, Also known as: Network on Chip (NoC) SAN System/storage area network Multiprocessor and multicomputer, storage LAN Local area network WAN Wide area network Trend: Switches replace buses

6 E.2: Connecting two devices Destination implicit

7 Software to Send and Receive SW Send steps 1: Application copies data to OS buffer 2: OS calculates checksum, starts timer 3: OS sends data to network interface HW and says start SW Receive steps 3: OS copies data from network interface HW to OS buffer 2: OS calculates checksum, if matches send ACK; if not, deletes message (sender resends when timer expires) 1: If OK, OS copies data to user address space and signals application to continue Sequence of steps for SW: protocol

8 Basic Network Structure and Functions Media and Form Factor Metal layers InfiniBand connectors Ethernet Media type Cat5E twisted pair Fiber Optics Coaxial cables Printed circuit boards Myrinet OCNs SANs connectors LANs WANs >1,000 Distance (meters) 8

9 Packet latency Sender Sender Overhead Transmission time (size/bandwidth) Receiver (processor busy) Time of Flight Transmission time (size/bandwidth) Transport Latency Receiver Overhead (processor busy) Total Latency Total Latency = Sender Overhead + Time of Flight + Message Size / bandwidth + Receiver Overhead

10 E.3: Connecting multiple devices (1/3) New issues Topology What paths are possible for packets? Routing Which of the possible paths are allowable (valid) for packets? Arbitration When are paths available for packets? Switching How are paths allocated to packets?

11 E.3: Connecting multiple devices (2/3) Two types of topology Shared media Switched media Shared media (bus) Arbitration Carrier Sensing Collision Detection Routing is simple Only one possible path

12 Connecting multiple devices (3/3) Switched media Point-to-point connections Routing for each packet Arbitration for each connection Comparison Much higher aggregate BW in switched network than shared media network Shared media is cheaper Distributed arbitration simpler for switched

13 E.4: Interconnection Topologies One switch or bus can connect a limited number of devices Complexity, cost, technology, Interconnected switches needed for larger networks Topology: connection structure What paths are possible for packets? All pairs of devices must have path(s) available A network is partitioned by a set of links if their removal disconnects the graph Bisection bandwidth Important for performance

14 Crossbar Common topology for connecting CPUs and I/O units Also used for interconnecting CPUs Fast and expensive (O(N 2 )) Non-blocking I/O I/O P P C C M M M M

15 Omega network Source Destination x2 switches Straigh t Crossover Upper broadcast 111 Lower broadcast 111 Example of multistage network. Combine multiple 2x2 crossbars Usually log 2 n stages for n inputs - O(N log N) Can block

16 Linear Arrays and Rings Distributed switched networks Node = switch + 1-n end s Linear array= 1D grid 2D grid Torus has wrap-around connections CRAY with 3D torus External I/O P $ Mem ctrl and NI Switch Mem

17 Trees Diameter and average distance are logarithmic k-ary tree, height d = log k N address = d-vector of radix k coordinates describing path down from root Fixed number of connections per (i.e. fixed degree) Bisection bandwidth = 1 near the root

18 E.5: Routing, Arbitration, Switching Routing Which of the possible paths are allowable for packets? Set of operations needed to compute a valid path Executed at source, intermediate, or even at destination s Arbitration When are paths available for packets? Resolves packets requesting the same resources at the same time For every arbitration, there is a winner and possibly many losers Losers are buffered (lossless) or dropped on overflow (lossy) Switching How are paths allocated to packets? The winning packet (from arbitration) proceeds towards destination Paths can be established one fragment at a time or in their entirety

19 Routing Shared Media Broadcast to everyone Switched Media needs real routing. Options: Source-based routing: message specifies path to the destination (changes of direction) Virtual Circuit: circuit established from source to destination, message picks the circuit to follow Destination-based routing: message specifies destination, switch must pick the path Deterministic: always follow same path Adaptive: pick different paths to avoid congestion, failures Randomized routing: pick between several good paths to balance network load

20 Routing mechanism Need to select output port for each input packet And fast Simple arithmetic in regular topologies Ex: x, y routing in a grid (first x then y) west (-x) x < 0 east (+x) x > 0 south (-y) x = 0, y < 0 north (+y) x = 0, y > 0 Unidirectional links sufficient for torus (+x, +y) Dimension-order routing Reduce relative address of each dimension in order to avoid deadlock

21 Deadlock How can it arise? necessary conditions: shared resources incrementally allocated non-preemptible How do you handle it? constrain how channel resources are allocated (deadlock avoidance) Add a mechanism that detects likely deadlocks and fixes them (deadlock recovery) TRC (0,0) TRC (0,1) TRC (0,2) TRC (0,3) TRC (1,0) TRC (1,1) TRC (1,2) TRC (1,3) TRC (2,0) TRC (2,1) TRC (2,2) TRC (2,3) X X TRC (3,0) TRC (3,1) TRC (3,2) TRC (3,3)

22 Arbitration (1/2) Several simultaneous requests to shared resource Ideal: Maximize usage of network resources Problem: Starvation Fairness needed Figure: Two phase arb. Request, Grant Poor usage

23 Arbitration (2/2) Three phases Multiple requests Better usage But: Increased latency

24 Switching Allocating paths for packets Two techniques: Circuit switching (connection oriented) Communication channel Allocated before first packet Packet headers don t need routing info Wastes bandwidth Packet switching (connection less) Each packet handled independently Can t guarantee response time Two types next slide

25 Store & Forward vs Cut-Through Routing Time Store & Forward Routing Cut-Through Routing Source Dest Dest Cut-through (on blocking) Virtual cut-through (spools rest of packet into buffer) Wormhole (buffers only a few flits, leaves tail along route)

26 Putting it all together (Ring Network)

27 Putting it all together (Ring Basics) Two different rings (one clockwise, one counterclockwise). Carries all communication back and forth between cores and Level Three caches. Also carries information between Level Three caches and memory controllers. Also carries information between L3s/Memory and coherent IO devices such as PCIe.

28 Putting it all together (Ring Basics) Routing: Put message on ring which has the shortest distance to destination. CPU L3 L3 CPU CPU L3 L3 CPU CPU L3 L3 CPU CPU L3 L3 CPU

29 Putting it all together (Ring Switching) Circuit Switched or Packet Switched? Circuit Switching ties up resources for too long, and hurts performance. Packet switching enables better utilization of available resources. CPU L3 L3 CPU CPU L3 L3 CPU CPU L3 L3 CPU CPU L3 L3 CPU

30 Blocking vs Non-blocking Switch0 Switch1 Block sending until we know for sure we have a free spot at Switch1 Switch0 Switch1 Go ahead and send to switch1, because you know switch1 will also send this same cycle (nobody is blocking)

31 Putting it all together (Ring Deadlock) Assume the ring is blocking (if something present at the next switch point, don t send a packet until the spot is free). What if each switch point is full of data? If the ring is non-blocking, then we don t have a deadlock, things just keep going in circles. It is possible to build blocking rings that are deadlock free, using virtual channels Now that the ring is nonblocking, what does that mean for arbitration? CPU L3 L3 CPU CPU L3 L3 CPU CPU L3 L3 CPU CPU L3 L3 CPU Deadlock!

32 Putting it all together (Ring Arbitration) The ring is non-blocking to avoid deadlock. How do we get something onto the ring? Packets on the ring ALWAYS get priority over packets trying to get on the ring. What does this remind you of? Hint, think of traffic. CPU L3 L3 CPU CPU L3 L3 CPU CPU L3 L3 CPU CPU L3 L3 CPU Any problems with this approach?

33 Putting it all together (Ring Livelock) Remember there are two rings. What if the packets on both rings are targeting the same end location? One gets off the network, and the other goes around the ring again! Bounce! What happens if the packet keeps getting bounced? Guarantee that a bounced packet always gets priority the next time it arbitrates to get off the ring. Any problems? Need to ensure that two bounced packets don t try to get off the ring at the same time CPU L3 L3 CPU CPU L3 L3 CPU CPU L3 L3 CPU CPU L3 L3 CPU

34 Putting it all together (Ring Performance) Because a ring is non-blocking, once on the ring, the latency between two s is deterministic! Simple Analytical models can prove that under uniform traffic, peak aggregate BW of a ring is 8*W, where W is the width of the bi directional ring. Note the result is independent of the number of s, meaning that the BW does not scale up with the number of s. Good rule of thumb is to design your system so that you are always operating at below 50% of the peak utilization Need to account for all traffic on the ring (cache requests, responses, acks, coherency messages,..) Latency Unloaded latency Delivered BW Peak

35 Putting it all together (Mesh Network) Intel Single Chip Cloud Computer research project

36 Putting it all together (Ring Basics) Multiple mesh networks for different types of traffic Carries communication between different s. Think of memory as being distributed evenly across the whole chip. Also carries information IOs and memory controllers

37 Putting it all together (Mesh Arbitration) At each switch point, multiple packets could be trying to go the same direction. Must block one of the packets. Meshes are blocking networks. Each local switch point is locally fair, but the network is globally unfair when going across the network. Standard policy is round robin at each switch point In non-blocking networks, we worry about livelock, in blocking networks, we need to worry about deadlock.

38 Putting it all together (Mesh Routing) Which path to take between two different s? Least loaded?

39 Putting it all together (Mesh Routing) Which path to take between two different s? Least loaded? Routing policy must avoid deadlocks. Two approaches: Prevent deadlocks from happening (easy) Detect deadlocks and recover (hard)

40 Putting it all together (Mesh Routing) The turn model may be used to prevent deadlocks. Eight possible turns in a 2d network. Choose any subset of turns that cannot be used to form a cycle

41 Putting it all together (Mesh Switching) Not circuit switched. wormhole routing is used for large packets

42 Putting it all together (Mesh Performance) A blocking network has non-deterministic latency. Simple Analytical models can prove that under uniform traffic, peak aggregate BW of a mesh is 4*W*sqrt(N), where N is the number of s. Note the result contains the number of s, so adding s increases aggregate BW (but decreases BW/). Good rule of thumb is to design your system so that you are always operating at below 50% of the peak utilization Need to account for all traffic on the mesh (cache requests, responses, acks, coherency messages,..) Latency Unloaded latency Delivered BW Peak

43 Putting it all together (Mesh vs Ring) 50 BW/Node for 32B network BW/Node ring mesh Nodes