Parallelism in Network Systems

Transcription

1 High Performance Switching Telecom Center Workshop: and outing Sept 4, 997. Parallelism in Network Systems Joint work with Sundar Iyer HP Labs, 0 th September, 00 Nick McKeown Professor of Electrical Engineering and Computer Science, Stanford University nickm@stanford.edu

2 Background Parallelism is an obvious way to improve performance aw cycles per second Enable multiple threads of execution Parallelism in computer systems Optimize for the common case, and accept the occasional snafu. Emphasis on correctness: cache coherency, data dependency, data bypass in a pipeline. Parallelism in network systems Need performance guarantees. Does my 00Gb/s router really operate at 00Gb/s? Is my pipeline screwed up downstream? Mis-sequencing packets is a no-no.

3 Outline I ll be describing some examples of parallelism and load-balancing to scale packet switches:. Simple example Parallel Packet Switch (PPS): multiple packet switches in parallel.. General problem Constraint Sets. More interesting example Distributed Shared Memory (DSM) router. 4. If there s still time Parallel packet buffers. Load-balanced -stage routers. Problems we ll encounter:. Mis-sequencing packets,. esource conflicts.

4 Basic idea of parallelism Serial system Some Some packet packet processing processing function; function; e.g. e.g. header header processor, processor, address address lookup, lookup, packet packet buffer buffer or or complete complete packet packet switch. switch. Parallel system /k /k k /k /k Q: If packets are load-balanced randomly, what can say about the performance of the resulting system? 4

5 Problem Example of Parallelism causing mis-sequencing / / / / If packet takes longer to process/store than packet, then their departure order could be reversed. 5

6 Problem Example of Parallelism causing resource conflicts ead ead packet packet,, then then,, then then / / / / If packet and packet in the same memory, they can t be read contiguously. 6

7 Outline I ll be describing some examples of parallelism and load-balancing to scale packet switches:. Simple example Parallel Packet Switch (PPS): multiple packet switches in parallel.. General problem Constraint Sets. More interesting example Distributed Shared Memory (DSM) router. 4. If there s still time Parallel packet buffers. Load-balanced -stage routers. Problems we ll encounter:. Mis-sequencing packets,. esource conflicts. 7

8 Parallel Packet Switch (PPS) Goals. Build a big packet switch out of lots of little packet switches,. Each memory to run slower than line rate,. No packet mis-sequencing, 4. Emulation of a FCFS OQ switch. 8

9 Architecture of a PPS Definition: A PPS consists of multiple identical lowerspeed packet-switches operating independently and in parallel. An incoming stream of packets is spread, packet-by-packet, by a demultiplexor across the slower packet-switches, then recombined by a multiplexor at the output. 9

10 Architecture of a PPS Demultiplexor (/k) OQ Switch (/k) Multiplexor Demultiplexor Multiplexor OQ Switch Demultiplexor Demultiplexor N=4 OQ Switch k= Multiplexor Multiplexor N=4 (/k) (/k) 0

11 Emulation of an ideal OQ Switch OQ Switch PPS =? Yes No

12 Emulation Scenario Layer Layer / Layer / N=4 N=4 /

13 Layer Layer Layer N=4 N=4 Why is there no Choice at the Input? 4 j 4 j 4 j j 5 j j 4 4 j j 5 4 j 4 j j j 5

14 esult of no Choice Layer 4 5 j j 54 Layer Layer 4 N=4 N=4 4

15 Increasing choice using speedup 4 5 j j (/) 5 j Layer 4 (/) j j Layer 5 j Layer N=4 N=4 (/) (/) 5

16 Effect of Speedup on Choice /k Layer Layer k/ S A speedup of S =, with k = 0 links Layer 9 Layer 0 6

17 Definition Available Input Link Set (AIL) AIL(i,n) is the set of layers to which external input port i can start writing a packet to, at time slot n. 7

18 Definition Departure Time of a Packet (n ) The departure time of a packet, n, is the time it would have departed from an equivalent FIFO OQ switch. 8

19 Definition Available Output Link Set (AOL) AOL(j,n ) is the set of layers that output j can start reading a packet from, at time slot n. 9

20 Observation Inputs can only send to the AIL set. Outputs can only read from the AOL set. 5 j j (/) Layer 4 (/) j j Layer j Layer N=4 N=4 (/) (/) 0

21 Lower Bounds on Choice Sets Minimum size of AIL, AOL: AIL, AOL (Number of links) - (Max number of busy links) k = k S

22 Overcoming resource conflict A packet must be sent to a link which belongs to both the AIL and the AOL set: AIL AOL φ AIL + AOL > k ( k k/ S + ) + ( k k/ S + ) > k S > k/( k+ )

23 Parallel Packet Switch esults If S > k/(k+) then each packet is guaranteed to find a layer that belongs to both the AIL and AOL sets. If S > k/(k+) then a PPS can precisely emulate a FIFO output queued switch for all traffic patterns.

24 Outline I ll be describing some examples of parallelism and load-balancing to scale packet switches:. Simple example Parallel Packet Switch (PPS): multiple packet switches in parallel.. General problem Constraint Sets. More interesting example Distributed Shared Memory (DSM) router. 4. If there s still time Parallel packet buffers. Load-balanced -stage routers. Problems we ll encounter:. Mis-sequencing packets,. esource conflicts. 4

25 Output-Queued outer one output λ µ = Gives 00% throughput if and output work conserving λ < G/D/ Work-conservation: 00% throughput, minimizes delay, delay guarantees possible Problem : Memory bandwidth not scalable Previous attempts : Very complicated or ad hoc Our approach : Deterministic parallelism 5

26 Parallel Output-Queued outer one output many outputs λ k k λ µ = λ µ = k λ N k N µ = λ Gives 00% throughput if < k and output work conserving k Is this work conserving? 6

27 Parallel Output-Queued outer (May not be work-conserving) C5? B5 B6 A5 A Time slot = C6 B6 B5 A6 B k= N= A8 A7 C At most two memory operations per time slot: write and read 7

28 Problem : General Problem Can we design a parallel outputqueued work-conserving router from slower parallel memories? Theorem : (sufficiency) A parallel output-queued router is work-conserving with N memories that can perform at most one memory operation per time slot 8

29 e-stating the Problem There are K cages able to hold an infinite number of pigeons. Assume that time is slotted, and in any one time slot At most N pigeons can arrive and at most N can depart. At most pigeon can enter or leave a cage via a pigeon hole. We know when a pigeon will depart. For any router: How many cages, K, do we need so that all N pigeons can be immediately placed in a cage when they arrive, and can depart at the right time? 9

30 Time = t Intuition DT=t DT=t+X Memory Only one packet can enter a memory at time t Only one packet can enter or leave a memory at time t Only one packet can enter or leave a memory at any time DT=t+X 0

31 Constraint Sets When a packet arrives in a time slot it must choose a memory not chosen by. The N other packets that arrive at that timeslot.. The N other packets that depart at that timeslot.. The N - other packets that can depart at the same time as this packet departs (in future). Proof: By the pigeon-hole principle, N memories (one memory operation per time slot) are sufficient for the router to be work-conserving.

32 The Parallel Shared Memory outer At most one operation a write or a read per time slot Memory A4 Memory A4 A5 C? Arriving Packets A A C B B Memory Memory Memory Memory Memory Departing Packets A B C K=8 C Memory k = 7 memories don t suffice... but 8 do.

33 Outline I ll be describing some examples of parallelism and load-balancing to scale packet switches:. Simple example Parallel Packet Switch (PPS): multiple packet switches in parallel.. General problem Constraint Sets. More interesting example Distributed Shared Memory (DSM) router. 4. If there s still time Parallel packet buffers. Load-balanced -stage routers. Problems we ll encounter:. Mis-sequencing packets,. esource conflicts.

34 Distributed Shared Memory outer Switch Fabric Memories Memories Memories Line Card Line Card Line Card N. Central memories are distributed to line cards and shared,. Memory and line cards can be added incrementally,. N memories N suffice for the router to be workconserving. 4

35 Problem: Corollary What is the switch bandwidth for a workconserving DSM router? Corollary: (sufficiency) A switch bandwidth of 4N is sufficient for a distributed shared memory router to be work-conserving Intuition: There are a maximum of memory accesses and port access. 5

36 Problem Corollary What is the switching algorithm for a workconserving DSM router? Bus : No algorithm needed, but impractical Corollary Crossbar : Algorithm needed because only permutations are allowed An edge coloring algorithm can switch packets for a work-conserving distributed shared memory route Intuition Follows from König s theorem - Any bipartite graph with maximum degree has an edge coloring with colors 6

37 Summary - outers which give 00% throughput Fabric # Mem. Mem. BW Total Memory BW Switch BW Switch Algorithm Output-Queued Bus N (N+) N(N+) N None Shared Mem. Bus N N N None Input Queued Crossbar N N N MWM N 6N N Maximal CIOQ (Cisco) Crossbar N 6N N Time eserve * PSM Bus k N/k N N C. Sets DSM (Juniper) Xbar N N N 4 N N 4N 4N 6N 4N Edge Color C. Sets C. Sets PPS - OQ Clos Nk (N+)/k N(N+) 4N C. Sets PPS Shared Memory Clos Nk Nk 4N/k N/k 4N N 4N N C. Sets None 7

38 Summary - outers which give delay guarantees Output-Queued Fabric Bus # Mem. N Mem. BW (N+) Total Memory BW N(N+) Switch BW N Switch Algorithm None Shared Mem. Bus N N N None Input Queued Crossbar N N N - CIOQ (Cisco) Crossbar N N 6N 6N N N Marriage Time eserve PSM Bus k 4N/k 4N 4N C. Sets DSM (Juniper) Xbar N N N N 4N 6N 5N 8N 6N Edge Color C. Sets C. Sets PPS - OQ Clos Nk (N+)/k N(N+) 6N C. Sets PPS Shared Memory Clos Nk Nk 6N/k N/k 6N N 6N N C. Sets - 8

39 Outline I ll be describing some examples of parallelism and load-balancing to scale packet switches:. Simple example Parallel Packet Switch (PPS): multiple packet switches in parallel.. General problem Constraint Sets. More interesting example Distributed Shared Memory (DSM) router. 4. If there s still time Parallel packet buffers. Load-balanced -stage routers. Problems we ll encounter:. Mis-sequencing packets,. esource conflicts. 9

40 Packet Buffering Line rate, Memory Scheduler Line rate, Memory Line rate, N Line rate, N Scheduler Scheduler Input or Output Line Card Shared Memory Buffer Big: For TCP to work well, the buffers need to hold one TT (about 0.5s) of data. Fast: Clearly, the buffer needs to store (retrieve) packets as fast as they arrive (depart). 40

41 An Example Packet buffers for a 40Gb/s line card 0Gbits Write ate, One 40B packet every 8ns Buffer Memory Buffer Manager ead ate, One 40B packet every 8ns Scheduler requests causes random access Problem is solved if a memory can be (random) accessed every 4 ns and store 0Gb of data 4

42 Available Memory Technology Use SAM? + Fast enough random access time, but - Too low density to store 0Gbits of data. Use DAM/LDAM? + High density means we can store data, but - Can t meet random access time. 4

43 Problem Problem: How can we design high speed packet buffers from commodity available memories? 4

44 Can t we just use lots of DAMs in parallel? Buffer Memory Buffer Memory Buffer Memory Buffer Memory Buffer Memory Buffer Memory Buffer Memory Buffer Memory 40B 0B Write ate, One 40B packet every 8ns 0B 0B Buffer Manager ead ate, One 40B packet every 8ns Scheduler equests 44

45 Works fine if there is only one FIFO queue 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 40B 0B 40B Write ate, One 40B packet every 8ns 0B 40B40B0B40B40B 40B40B0B40B40B Buffer Manager (on chip SAM) 0B ead ate, 40B One 40B packet every 8ns Scheduler equests Aggregate 0B for the queue in fast SAM and read and write to all DAMs in parallel 45

46 In practice, buffer holds many FIFOs e.g. In an IP outer, Q might be 00. In an ATM switch, Q might be B 0B 0B 0B 0B 0B 0B 0B We don t know which head of line packet the scheduler will request next? 0B 0B 0B 0B Q 40B 0B?B Write ate, One 40B packet every 8ns 0B 0B 0B Buffer Manager (on chip SAM) 0B ead ate,?b One 40B packet every 8ns Scheduler equests 46

47 Parallel Packet Buffer Hybrid Memory Hierarchy Large DAM memory holds the body of FIFOs Q Writing b bytes DAM eading b bytes b = degree of parallelism Arriving Packets Q Buffer Manager (ASIC with on chip SAM) Q Departing Packets Small tail SAM cache for FIFO tails Small head SAM cache for FIFO heads Scheduler equests 47

48 Problem e-stating the Problem What is the minimum size of the SAM needed for the parallel buffer so that every packet is available immediately in the SAM when requested? Theorem (Necessity) An SAM size of size Qw = Q(b )( + lnq) bytes is necessary. w is the size of each queue 48

49 Theorem (sufficiency) Theorem An SAM cache of size Qw = Qb( +ln Q) bytes is sufficient so that every packet is available immediately in the SAM when requested Examples:. 40Gb/s line card, b=640, Q=8: SAM = 560kBytes. 60Gb/s line card, b=560, Q=5: SAM = 0MBytes 49

50 Problem Theorem What is the minimum size of the SAM so that every packet is available in the SAM within a bounded pipeline latency when requested? Theorem (necessity and sufficiency): An SAM cache of size Qw = Q(b ) bytes is both necessary and sufficient if the pipeline latency is Q(b ) + time slots. 50

51 Intuition The algorithm replenishes the queue which is going to be in the most urgent need of replenishment If we use a lookahead buffer to know the requests in advance, we can identify the queue which will empty earliest This increases the pipeline latency from when a request is made until the byte is available. Example:. 60Gb/s line card, b=560, Q=8: SAM = 60kBytes, latency is 8µs. 5

52 Discussion Q=000, b = 0 Queue Length for Zero Latency Queue Length, w dw dx x Queue Length for Maximum Latency Pipeline Latency, x

53 Outline I ll be describing some examples of parallelism and load-balancing to scale packet switches:. Simple example Parallel Packet Switch (PPS): multiple packet switches in parallel.. General problem Constraint Sets. More interesting example Distributed Shared Memory (DSM) router. 4. If there s still time Parallel packet buffers. Load-balanced -stage routers. Problems we ll encounter:. Mis-sequencing packets,. esource conflicts. 5

54 eferences All available from: Parallel Packet Switch (PPS) "Analysis of the Parallel Packet Switch Architecture" Sundar Iyer and Nick McKeown IEEE/ACM Transactions on Networking, April 00.. Constraint Sets and DSM outers "outers with a Single Stage of Buffering" Sundar Iyer, ui Zhang, and Nick McKeown ACM SIGCOMM Aug Fast Packet Buffers "Designing Packet Buffers for outer Line Cards" Sundar Iyer,.. Kompella, and Nick McKeown Paper in Submission, October Load-Balanced outers "Scaling Internet outers Using Optics" Isaac Keslassy, ShangTse Chuang, Kyoungsik Yu, David Miller, Mark Horowitz, Olav Solgaard, Nick McKeown ACM SIGCOMM Aug