Inside Internet Routers

Transcription

1 Inside Internet Routers 3035/GZ0 Networked Systems Kyle Jamieson Lecture 9 Department of Computer Science University College London

2 Today Inside internet routers Longest- prefix lookup The Luleå algorithm Router architecture Crossbar scheduling and the islip algorithm Cisco Gigabit Switch Router 86

3 IP lookup problem: Context Given incoming packet with IP address x, choose output port number outport(x) to deliver packet Then will configure switching fabric to connect inport(x) à outport(x) What kind of data structure will support this?

4 Compu?ng the longest prefix match Test: (DesYnaYon & Prefix mask) == Prefix Des?na?on: : : " 7: 00000" 7: " Forwarding Table: Prefix Outgoing link / / / /3 9: "

5 Compu?ng the longest prefix match Test: (DesYnaYon & Prefix mask) == Prefix Des?na?on: : : " 7: 00000" 7: " Forwarding Table: Prefix Outgoing link / / / /3 4: : 0000" 8: "

6 Compu?ng the longest prefix match Test: (DesYnaYon & Prefix mask) == Prefix Des?na?on: : : " 7: 00000" 7: " Forwarding Table: Prefix Outgoing link / / / /3 0: : " 0: "

7 Compu?ng the longest prefix match Test: (DesYnaYon & Prefix mask) == Prefix Des?na?on: : : " 7: 00000" 7: " Forwarding Table: Prefix Outgoing link / / / /3 0: : " 6: " Algorithmic problem: How do we do this fast?

8 Simple algorithms are too slow Scan the forwarding table one entry at a Yme See if the desynayon matches the prefix Keep track of the entry with longest- matching prefix If no match, use default route Overhead is linear in size of the forwarding table Today, that means 00,000-50,000 entries! And, the router may have just a few nanoseconds before the next packet arrives Need greater efficiency to keep up with line speed Beeer algorithms Hardware implementayons

9 The forwarding problem SONET opycal fiber links OC- Gbits/s: backbones of secondary ISPs OC- 0 Gbits/s: widespread in the core OC- 40 Gbits/s: deployed in a few core links Have to handle minimum- sized packets (40 64 bytes) at line speed At 0 Gbits/s have 3 5 ns to decide for each packet DRAM latency 50 ns; SRAM latency 5 ns

10 First apempt: Binary tree Each bit in prefix corresponds to one level of the tree, staryng from bit 0 at the root Some nodes correspond to valid prefixes in the IP forwarding table ConvenYon: Bit 0 is the most significant bit of IP address; bit 3 is the least significant bit; e.g.: 0 0* 0 * 0* Bit 0 Bit 3 000*

11 Binary tree When a packet arrives: Search the tree based on the desynayon address, remembering all matching nodes Deepest matching node corresponds to the LPM Running Yme? Scales directly with # bits in prefix; each of which involves a slow memory lookup 0 0* * 0 0* *

12 Patricia tree Binary tree, but internal nodes indicate which bit posiyons to test Allows router to skip bit tests, speed matching Leaves contain: Key (IP address), and Mask (# of significant bits) Match (query IP & mask) with key If match fails, invoke the following backtracking step Backtracking step: Move up tree one level at a Yme, logical- AND search key with mask of node and restart the search at that node If no mask exists, move up the tree one level

13 bit 0 bit 0 0x bit default x xff bit 3 bit 4 bit 3 0xff bit bit 30 bit xff xff end bit 3 bit xffffffe

14 Patricia tree: Examples. Search IP = = hex 7F Bit 0 clear à branch ler Bit set à branch right Bit 3 set à branch right Search key ?= address : match à return (host match). Search IP = = hex 7F Bit 0 clear, bit set à branch ler, right Bit 3 clear à branch ler Search key & network mask 0xFF ?= : match à return (network match) 3. Search IP = = hex 7F Bit 0 clear, bit set, bit 3 set à branch ler, right, right Search key ?= address : no match Backtrack up one level to bit 3 node, new search key = search key & mask 0xFF = Bit 3 now clear à branch ler Search key & network mask 0xFF ?= : match à return (network match) 4. Search IP =.0.0. = hex 0x Bit 0 clear, bit set, bit 3 set à branch ler, right, right Search key.0.0.?= address : no match Backtrack to bit 3 node, new search key = Bit 3 now clear à branch ler Search key & 0xFF ?= : no match Backtrack to bit node: no mask so conynue upward to bit 0 node (root) New search key & 0x = Bit 0 clear, bit clear à branch ler, ler to default route

15 Patricia tree: Summary Large rouyng tables Patricia trees (4.4 BSD) use 4 bytes for leaves Size: Mbytes à Mbytes Worst case, have to backtrack and traverse enyre tree Memory accesses are the boeleneck of lookup Recall: Superlinear growth trend in number of rouyng table entries Result: IP lookup becomes the boeleneck in fast routers; can we improve its speed?

16 Luleå algorithm: Mo?va?on Degermark et al., Small forwarding tables for fast rou;ng lookups in Proc. of SIGCOMM 97 Naïve binary tree is huge Won t fit in fast CPU cache memory in a sorware router Memory bandwidth becomes limiyng factor in a hardware router Therefore, goal becomes: How can we minimize memory accesses and the size of data structure? Method for compressing the binary tree using bit- vectors Design for 4 6K different next- hops

17 This just in

18 Luleå: Basic idea Back to the binary tree Imagine tree is full down to some level, e.g. three Now construct a bit vector, one entry per each level- three leaf Put informayon that was in leaves in a pointer array One entry per set bit, to save space Discard the binary tree! Now just need to find the right index into the pointer array given an IP address and the bit vector? 0 0* 0 0* * 0 bit vector next hop next hop pointer array

19 Luleå algorithm CIDR longest prefix match rule: e supersedes e Divide a complete binary tree into three levels Level : one big node represenyng enyre tree depth 6 bits Levels and 3: chunks describe poryons of the tree The binary tree is sparse, and most accesses fall into levels and/or Level e e IP address space: 3 possible addresses Level Level 3

20 Luleå algorithm: Level Covers all prefixes of length 6 Cut across tree at depth 6 bit vector of length 6 Root head =, genuine head =, member of genuine head = 0 Divide bit vector into bit masks, each 6 bits long Genuine head Root head One bit mask: depth 6

21 Luleå algorithm: Level One bit mask: Head informa?on stored in pointer array: type () pointer (4) pix" Next- hop table: L chunk One 6- bit pointer per bit set equal to in the bit- mask Pointer composed of bits of type info; 4 bits of indexing info Genuine heads: index into next- hop table Root heads: index into array of Level (L) chunks Problem: given an IP address, find the index pix into the pointer array

22 Luleå: Finding pointer group Group pointers by 6- bit bit masks; how to find pointer group? Bit vector is 6 total length, so there are bit masks Code word array code ( entries) One entry per bit mask, so indexed by top bits of IP address 6- bit offset six: num/ptrs to skip to find st ptr for that bit mask in ptr array Four bit masks, max 4 6 = 64 bits set, 0 six 63, so value may be too big Base index array base ( 0 entries) One base index per four code words: num/ptrs to skip for those four bit masks Indexed by top 0 bits of IP address e.g. bit vector: code: ten" six" 0! base: 0 6 0! 3

23 Luleå: Finding pointer group () 3 IP address bix ix code: bit codeword six ten base: pix := + Extract top 0 bits from IP address: bix! Extract top bits from IP address: ix! Skip code[ix].six + base[bix] pointers in the pointer table; then we are at the group of pointers

24 Luleå: Finding pointer index within pointer group a(n) number of possible bit masks of length n a(0) = ; a(n) = + a(n ) à a(4) + = 678 So maptable rows indexed with 0 bits ten field of code indexes maptable maptable entries are 4- bit offsets maptable entries: pre- computed and constant ten IP value varies depending address 3 0 on tree depth Code word array: r 0 0 IP address bix ix code: 4 bit ten" code: r r r r5 4 maptable: maptable:" Base index array: six" codeword six ten maptable entry base: pix := ;

25 Luleå: Summary of finding pointer 3 IP address bix bit ix codeword code: six ten 0 maptable: base: pix := + + ; 0 ten = code[ix].ten! six = code[ix].six! pix = base[bix] + six + maptable[ten][bit]! pointer = level_pointers[pix]!

26 Luleå algorithm: Levels and 3 Consist of chunks, pointed to by root heads Chunk covers subtree of height 8, so 56 heads Three types of chunk: Sparse: - 8 heads, array of 8- bit indices of the heads Dense: 9-64 heads, like Level but only one base index Very dense: heads, same format as Level pix" 4 L chunk

27 Luleå: Summary Tradeoff mutability and table construcyon Yme for speed Adding a rouyng entry requires rebuilding enyre table RouYng tables don t oren change Boeom line Lookup: Eight memory references touching 4 bytes Table: 50 Kbytes for 40,000 entries; 4 5 bytes/entry Current state of the art in router IP lookup Implemented in hardware as well as sorware routers

28 Today Inside internet routers Longest- prefix lookup The Luleå algorithm Router architecture Crossbar scheduling and the islip algorithm Cisco Gigabit Switch Router 86

29 Router architecture. Data path: funcyons performed on each datagram Forwarding decision Switching fabric (backplane) Output link scheduling. Control plane: funcyons performed relayvely infrequently RouYng table informayon exchange with others ConfiguraYon and management

30 Input port func?onality Input port Layer Line termination Layer Data link processing Layer 3 Lookup, forwarding R Switch fabric IP address lookup CIDR longest- prefix match Copy of forwarding table from control processor Check IP header, decrement TTL, recalculate checksum, prepend next- hop link- layer address Possible queuing, depending on design

31 Switching fabric Job of switching fabric is to get a packet from an input port to the right output port How can this be done?. Copy it into some memory locayon and out again. Send it over a shared hardware bus 3. Crossbar interconnect 4. Self- rouyng fabric

32 Switching via memory First generayon routers: tradiyonal computers with switching under direct control of CPU. Packet copied from input port across shared bus to RAM. Packet copied from RAM across shared bus to output port ü Simple design ü All ports share queue memory in RAM Speed limited by CPU: must process every packet [Image: N. McKeown]

33 Switching via shared bus Datagram moves from input port memory to output port memory via a shared bus e.g. Cisco 5600: 3 Gbit/s bus; sufficient speed for access routers ü Eliminate CPU boeleneck Bus contenyon: switching speed limited by bus bandwidth CPU speed syll a factor [Image: N. McKeown]

34 Crossbar interconnect Why do we need switched backplanes? Shared buses divide bandwidth among contenders Electrical reason: speed of bus limited by # connectors Replaces shared bus Up to n connects join n inputs to n outputs MulYple input ports communicate simultaneously [Image: N. McKeown]

35 Switching via crossbar Datagram moves from input port memory to output port memory via the crossbar e.g. Cisco 000 family: 60 Gbit/s; sufficient speed for core routers ü Eliminates bus boeleneck ü Custom ASIC forwarding engines replace general purpose CPUs Requires algorithm to determine crossbar configurayon Crossbar [Image: N. McKeown]

36 Switching via self- rou?ng fabric Input port appends self- rou6ng header to packet Self- rouyng header contains output port Output port removes self- rouyng header Example: Banyan- Batcher architecture

37 Self- rou?ng fabric example: Banyan network ComposiYon: comparator elements Self- rouyng header: use i th bit for i th stage Block if two arriving packets have same value Banyan is collision free if packets are presented in sorted ascending order First layer moves packets to correct upper or lower half based on st bit (0, ) Banyan with four arriving packets

38 Sor?ng networks x y x y Comparator notayon y i = x i if x x y = x, y = x otherwise InserYon sort by recursive definiyon Batcher network: an efficient sorter (not shown here) Batcher- Banyan architecture for collision- free switching x x x 3 x n x n x n+ x x x 3 x 4 x 5 x 6 SorYng network for n elements

39 Output port func?onality Output port Switch fabric Layer Data link processing Layer Line termination Output queuing required when datagrams arrive from fabric faster than line transmission rate

40 Where does queuing occur? Central issue in switch design: three choices At input ports (input queuing) At output ports (output queuing) Some combinayon of the above n = max(# input ports, # output ports)

41 Output queuing No buffering at input ports, therefore: Mul?ple packets may arrive to an output port in one cycle; requires switch fabric speedup of n Output port buffers all packets Drawback: Output port speedup required: n Switch fabric

42 Input queuing Input ports buffer packets Send at most one packet per cycle to an output Switch fabric forwarding speedup required: n Switch fabric

43 Input queuing: Head- of- line blocking One packet per cycle sent to any output Blue packet blocked despite available capacity at output ports and in switch fabric Reduces throughput of the switch Switch fabric

44 Virtual output queuing On each input port, one input queue per output port Input port places packet in virtual output queue (VOQ) corresponding to output port of forwarding decision ü No head- of- line blocking ü All ports (input and output) operate at same rate Need to schedule fabric, choosing which VOQs when Input port Q(,) Output ports (3) Lookup, forwarding Q(,) Q(,3) Switch fabric

45 Virtual output queuing [Image: N. McKeown]

46 Today Inside internet routers Longest- prefix lookup The Luleå algorithm Router architecture Crossbar scheduling and the islip algorithm Cisco Gigabit Switch Router 86

47 Crossbar scheduling algorithm: goals. High throughput Low queue occupancy in VOQs Sustain 00% of rate R on all n inputs, n outputs. StarvaYon- free Don t allow any one virtual output queue to be unserved indefinitely 3. Speed of execuyon Should not be the performance boeleneck in the router 4. Simplicity of implementayon Will likely be implemented on a special purpose chip

48 islip algorithm: Introduc?on McKeown, 999 Model problem as a biparyte graph Input port = graph node on ler Output port = graph node on right Edge (i, j) indicates packets in VOQ Q(i, j) at input port i Scheduling = a biparyte matching (no two edges connected to the same node) Inputs Request graph Outputs Inputs Bipar?te matching Outputs

49 islip: High- level overview For simplicity, we will look at single- iterayon islip One iterayon per packet Each itera?on consists of three phases:. Request phase: all inputs send requests to outputs. Grant phase: all outputs grant requests to some input 3. Accept phase: input chooses an output s grant to accept

50 islip: Accept and grant counters Each input port i has a round- robin accept counter a i Each output port j has a round- robin grant counter g j Round robin counter:,, 3,, n,,, Inputs Outputs 4 3 a a a g g g 3 a g 4

51 islip: One itera?on in detail. Request phase Input sends a request to all backlogged outputs. Grant phase Output j grants the next request grant pointer g j points to 3. Accept phase Input i accepts the next grant its accept pointer a i points to For all inputs k that have accepted, increment then a k g ak Inputs Outputs 4 3 a a a g g g 3 a g 4

52 islip example a a g g Two inputs, two outputs Input always has traffic for outputs and Input always has traffic for outputs and All accept and grant counters iniyalized to

53 islip example: Packet?me Request phase a a g g Grant phase a a g g Accept phase a a g g

54 islip example: Packet?me Request phase a a g g Grant phase a a g g Accept phase a a g g

55 islip example: Packet?me 3 Request phase a a g g Grant phase a a g g Accept phase a a g g

56 Implemen?ng islip Request vector: Grant arbiters Accept arbiters Decision vector: r = r = r = r = Request phase Grant phase Accept phase

57 Implemen?ng islip: General circuit

58 Implemen?ng islip: Inside an arbiter Highest priority Incrementer

59 Internetworking II: Virtual Networks and Traffic Engineering The Domain Name System Pre- Reading: P & D, SecYons 4.3 and 9..3; S & K DNS Case Study NEXT TIME