Interconnection Networks

Transcription

1 C/ECE 757: Advanced Computer Architecture II (arallel Computer Architecture) Interconnection Network Design (Chapter 10) Copyright 2001 ark D. Hill University of Wisconsin-adison lides are derived from work by arita Adve (Illinois), Babak Falsafi (CU), Alvy Lebeck (Duke), teve Reinhardt (ichigan), and J.. ingh (rinceton). Thanks! Interconnection Networks Goal: Communication between computers Warning: Terminology-rich environment Focus on Networks for arallel Computing today s ystem Area Networks exhibit many of the same properties Traffic atterns Arbitrary (bisection bandwidth matters) Near-neighbor ermutations (e.g., FFT) ulticasts or Broadcasts? Latency or Bandwidth Critical? (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE age 1

2 Terms Network characterized by Topology physical structure of the graph Routing Algorithm through which paths can message flow witching trategy How data in message traverses its route Circuit witched vs acket witched Flow Control When does a packet (or portions of it) move along its route (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE ore Terms Given topology constructed by linking switches and network interfaces, must deliver packet from node A to node B Link: cable with connectors on each end connect switches to other switches or network interfaces witch: N inputs N outputs (degree N) hit: inimum # of bits physically moved across link in one cycle Flit: inimum # of bits move across link as a single unit acket: unit that requires routing information, some number of flits (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE age 2

3 Outline Topology witching, Routing, & Deadlock witch Design Flow Control Case tudies (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE Topology tructure of the interconnect Determines witch Degree: number of links from a node Diameter: number of links crossed between nodes on maximum shortest path Average distance: number of hops to random destination Bisection: minimum number of links that separate the network into two halves Don t forget Network Interface (NI) On- and off-ramp to the interconnect freeway NI latency can dominate interconnect latency (reducing the importance of topology) (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE age 3

4 Topology Node Node Overhe ad Application W Interface HW Interface link link bandwidth Application W Interface HW Interface link link bandwidth Interconnect Bisection Bandwidth Latency (C) 2003 J. E. mith C/ECE Direct interconnect erformance/cost: witch cost: N Wire cost: const Avg. latency: const Bisection B/W: const Not neighbor optimized ay be local optimized Buses emory Capable of broadcast (good for coherence) Bandwidth not scalable => major problem Hierarchical buses?» Bisection B/W remains constant» Becomes neighbor optimized cache roc emory cache roc emory cache roc emory cache roc (C) 2003 J. E. mith C/ECE age 4

5 Rings Direct interconnect erformance/cost: witch cost: N Wire cost: N Avg. latency: N / 2 Bisection B/W: const neighbor optimized, if bi-directional probably local optimized RING Not easily scalable Hierarchical rings?» Bisection B/W remains constant» Becomes neighbor optimized (C) 2003 J. E. mith C/ECE Hypercubes n-dimensional unit cube Direct interconnect erformance/cost: witch cost: N log 2 N Wire cost: (N log 2 N) /2 Avg. latency: (log 2 N) /2 Bisection B/W: N /2 neighbor optimized probably local optimized latency and bandwidth scale well, BUT individual switch complexity grows => max size is built in (C) 2003 J. E. mith C/ECE age 5

6 2D Torus Direct interconnect erformance/cost: witch cost: N Wire cost: 2 N Avg. latency: N 1/2 Bisection B/W: 2 N 1/2 neighbor optimized probably local optimized (C) 2003 J. E. mith C/ECE D Torus, contd. Cost scales well Latency and bandwidth do not scale as well as hypercube, BUT difference is relatively small for practical-sized systems Weave nodes to make inter-node latencies const. Routing can be tricky (deadlocks) 2D esh similar, but without wraparound (C) 2003 J. E. mith C/ECE age 6

7 Direct interconnect erformance/cost: witch cost: N Wire cost: 3 N Avg. latency: 3(N 1/3 / 2) Bisection B/W: 2 N 2/3 neighbor optimized probably local optimized 3D Torus (C) 2003 J. E. mith C/ECE Cost scales well 3D Torus, contd. Latency and bandwidth do not scale as well as hypercube, BUT difference is relatively small for practical-sized systems eems to have become an interconnect of choice: Cray, Intel, Tera, DAH, etc. Routing can be tricky (deadlocks) 3D esh similar, but without wraparound (C) 2003 J. E. mith C/ECE age 7

8 Crossbars Indirect interconnect erformance/cost: witch cost: N 2 Wire cost: 2 N Avg. latency: const Bisection B/W: N Not neighbor optimized Not local optimized emory emory emory emory emory emory emory emory corssbar rocessor rocessor rocessor rocessor (C) 2003 J. E. mith C/ECE Crossbars, contd. Capable of broadcast No network conflicts Cost does not scale well Often implemented with muxes mux mux mux mux (C) 2003 J. E. mith C/ECE age 8

9 ultistage Interconnection Networks (INs) AKA: Banyan, Baseline, Omega networks, etc. Indirect interconnect Crossbars interconnected with shuffles Can be viewed as overlapped UX trees Destination address 0101 specifies the path The shuffle interconnect routes addresses in a binary fashion This can most easily be seen with INs in the form at right 1011 (C) 2003 J. E. mith C/ECE INs, contd. f switch outputs => decode log 2 f bits in switch erformance/cost: witch cost: (N log f N) / f Wire cost: N log f N Avg. latency: log f N Bisection B/W: N Not neighbor optimized roblem (in combination with latency growth) Not local optimized Capable of broadcast Commonly used in large UA systems (C) 2003 J. E. mith C/ECE age 9

10 ultistage Nets, Equivalence By rearranging switches, multistage nets can be shown to be equivalent A B C D E F G H A B C D E G F H A I A I B J B J C K C D L D N E E K F N F L G O G O H H (C) 2003 J. E. mith C/ECE Fat Trees Tree-like, with constant bandwidth at all levels Closely related to INs Indirect interconnect erformance/cost: witch cost: N log f N Wire cost: f N log f N Avg. latency: approx 2 log f N Bisection B/W: f N neighbor optimized may be local optimized Capable of broadcast (C) 2003 J. E. mith C/ECE age 10

11 Fat Trees, Contd. The IN-derived Fat Tree, is, in fact, a Fat Tree: However, the switching "nodes" in effect do not have full crossbar connectivity Q R T U V W X Q R T U V W X I J K L N O I J K L N O A B C D E F G H A B C D E F G H (C) 2003 J. E. mith C/ECE Important Topologies N = 1024 Type Degree Diameter Ave Dist Bisection Diam Ave D 1D mesh 2 N-1 2N/3 1 2D mesh 4 2(N 1/2-1) 2N 1/2 / 3 N 1/ D mesh 6 3(N 1/3-1) 3N 1/3 / 3 N 2/3 ~30 ~10 nd mesh 2n n(n 1/n - 1) nn 1/n / 3 N (n-1) / n (N = k n ) Ring 2 N / 2 N/4 2 2D torus 4 N 1/2 N 1/2 / 2 2N 1/ k-ary n-cube 2n n(n 1/n ) nn 1/n / (3D) (N = k n ) nk/2 nk/4 2k n-1 Hypercube n n = LogN n/2 N/ (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE age 11

12 Topologies (cont) N = 1024 Type Degree Diameter Ave Dist Bisection Diam Ave D 2D Tree 3 2Log 2 N ~2Log 2 N 1 20 ~20 4D Tree 5 2Log 4 N 2Log 4 N - 2/ kd k+1 Log k N 2D fat tree 4 Log 2 N N 2D butterfly 4 Log 2 N Log 2 N N/ C-5 Thinned Fat Tree (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE Butterfly ultistage: nodes at ends, switches in middle N/2 Butterfly N/2 Butterfly All paths equal length Unique path from any input to any output Conflicts cause tree saturation Benes Network N Butterfly Reversed N Butterfly Routes all permutations w/o conflict Notice similarity to Fat Tree (Fold in half) Randomization is major breakthrough (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE age 12

13 Outline Topology witching, Routing, & Deadlock witch Design Flow Control Case tudies (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE ABCs of Networks tarting oint: end bits between 2 computers Queue on each end Can send both ways ( Bi-directional, Full Duplex ) Rules for communication? protocol ynchronous send» Need Request & Response signaling Name for standard group of bits sent: acket (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE age 13

14 A imple Example What is the packet format? Fixed? (for HW Interpretation) Number bytes? Request/ Response Address/Data 1 bit 32 bits 0: lease send data from Address 1: acket contains data corresponding to request (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE Questions About imple Example What if more than 2 computers want to communicate? Need node identifier field (destination) in packet Routing and topology What if packet is garbled in transit? Add error detection field in packet (e.g., CRC) What if packet is lost? ore elaborate protocols to detect loss (e.g., NAK, time outs) What if multiple processes/machine? Dispatch Queue per process Questions such as these lead to more complex protocols and packet formats (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE age 14

15 General acket Format Header routing and control information ayload carries data (non HW specific information) can be further divided (framing, protocol stacks ) Error Code generally at tail of packet so it can be generated on the way out Header ayload Error Code (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE essage vs. acket A essage may be composed of several packets Applications reason about messages Network transfers packets mall fixed size packets. roblems? Fragmentation and reassembly (W overhead) Variable ize packets. roblems? Congestion (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE age 15

16 acket witched vs Circuit witched Circuit witched Establish Route then end Data Telephone system acket witched Route each packet individually Delivery Guarantees Reliable In order, what if not? (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE tore-and-forward Cut-through Virtual cut-through Wormhole Routing (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE age 16

17 tore and Forward tore-and-forward policy: each switch waits for the full packet to arrive in the switch before it is sent on to the next switch (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE Cut Through Cut-through routing: switch examines the header, decides where to send the message, and then starts forwarding it immediately (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE age 17

18 Virtual Cut-Through What to do if output port is blocked? Lets the tail continue when the head is blocked, absorbing the whole message into a single switch. Requires a buffer large enough to hold the largest packet. Degenerates to store-and-forward with high contention (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE Wormhole When the head of the message is blocked the message stays strung out over the network otentially blocks other messages (needs only buffer the piece of the packet that is sent between switches). C-5 used it, with each switch buffer being 4 bits per port. yrinet uses it Interaction with ack ize Can cause tree saturation (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE age 18

19 tore and Forward vs. Cut-Through Advantage Latency reduces from function of: number of intermediate switches times the size of the packet to time for 1st part of the packet to negotiate the switches + the packet size interconnect BW (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE Routing Algorithm How do I know where a packet should go? Arithmetic ource-based Table Lookup Adaptive route based on network state (e.g., contention) (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE age 19

20 Arithmetic Routing For regular topology, simple arithmetic to determine route 2D esh (Also called NEW network) packet header contains signed offset to destination switch ++ or -- one field of header (x or y dimension) when x == 0 and y == 0, then at correct processor Requires ALU in switch ust recompute CRC (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE ource Based and Table Lookup Routing ource Based ource specifies output port for each switch in route Very imple witches no control state strip output port off header yrinet uses this Table Lookup Very mall Header, index into table for output port Big tables, must be kept up to date... (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE age 20

21 Deterministic v.s. Adaptive Routing Deterministic follows a prespecified route mesh: dimension-order routing» (x1, y1) -> (x2, y2)» first Dx = x2 - x1,» then Dy = y2 - y1, hypercube: edge-cube routing» X = x0x1x2...xn -> Y = y0y1y2...yn» R = X xor Y» Traverse dimensions of differing address in order tree: common ancestor Adaptive route determined by contention for output port (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE Adaptive Routing Essential for fault tolerance at least multipath Can improve utilization of the network imple deterministic algorithms easily run into bad permutations Fully/partially adaptive, minimal/non-minimal Can introduce complexity or anomalies Little adaptation goes a long way! (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE age 21

22 Deadlock Break a Necessary Condition Use more than one resource Not willing to release resource in use Cycle in order of recourse use Guri ohi (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE Deadlock Free Routing Virtual Channels Not virtual cut-through Add buffers so flits of wormhole packets can be interleaved Up*-Down* Number switches: higher = farther away from processors route up, make one turn, route down Turn odel Routing Restrict order of turns» West First» North Last» Negative First Can increase number of hops (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE age 22

23 inimal turn restrictions in 2D +y -x +x West-first north-last -y negative first (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE Outline Topology witching, Routing, & Deadlock witch Design Flow Control Case tudies (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE age 23

24 A Generic witch At minimum, must route inputs to outputs Receiver Input Buffer Output Buffer Transmitter Input orts Cross-bar Output orts Control Routing, cheduling VLI makes it easier to create larger fully connected switches (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE witch Design Issues ports, pin limited data path (cross bar designs) non-blocking crossbar routing logic per input ALU table Finite tate achine for cut-through (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE age 24

25 witch Buffering need to absorb some transient peaks shared buffer pool need high bandwidth one output could hog all buffer space input buffers (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE Input Buffering buffer per port routing logic head of line (HOL) blocking subsequent packet may be routed to unused output port (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE age 25

26 Output Buffering Buffers logically associated with output split on either side of cross bar Arbitration for physical link (output scheduling) static priority random round-robin oldest-first Effects of adaptive routing? elect output based on availability requires feedback from output port (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE tacked Dimension witch Uses only 2x2 switch to build higher dimension switch Host out z in 2x2 z out y in 2x2 y out x in 2x2 x out Host in (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE age 26

27 Outline Topology witching, Routing, & Deadlock witch Design Flow Control Case tudies (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE Congestion Control acket switched networks do not reserve bandwidth; this leads to contention olution: prevent packets from entering until contention is reduced (e.g., metering lights) Options: End-to-end Flow Control Link-level Flow Control (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE age 27

28 Flow Control acket discarding: If a packet arrives at a switch and there is no room in the buffer, the packet is discarded no communication between switches, requires higher level protocol Flow control: between pairs of receivers and senders; use feedback to tell the sender when it is allowed to send the next packet (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE Link-Level Flow Control Ready Data Transfer single flit when receiver is ready Could have long links with many flits in flight (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE age 28

29 Credit-based (Window) Flow Control Receiver gives N credits to sender sender decrements count stops sending if zero receiver sends back credit as it drains its buffer bundle credits to reduce overhead ust account for link latency (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE Water Level high water, low water stop & go back to source switch (yrinet) can send redundant stop/go Incoming phits top Go Outgoing phits (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE age 29

30 Outline Topology witching, Routing, & Deadlock witch Design Flow Control Case tudies (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE Case tudy Cray T3D 1024 switch nodes each connected to 2 processors 3D Torus, bidirectional, 300 B/s Link: 16 bits, 8 control bits Variable size packet (multiple of 16 bits) Logical request & response networks 2 virtual channels each for deadlock tacked dimension routing Wormhole for large packets, virtual cut-through for small packets (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE age 30

31 IB -2 (Vulcan) witch has eight bidirectional 40 B/s links Link: 8 data bits, 1 tag, 1 reverse flow-control Flit is 16 bits, phit is 8 input FIFO + output FIFO + central buffer byte segments (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE Real achines Wide links, smaller routing delay Tremendous variation (C) 2001 ark D. Hill from Adve,Falsafi, Lebeck, Reinhardt, & ingh C/ECE age 31