DiffServ over Network Processors: Implementation and Evaluation

Transcription

1 DiffServ over Network Processors: Implementation and Evaluation Authors:Ying-Dar Lin, Yi-Neng, Shun-Chin Yang, and Yu-Shen Lin Speaker: Yi-Neng Lin Department of Computer and Information Science National Chiao Tung University Hsinchu, Taiwan Outline Y Motivation Y Introduction Y Hardware architecture of IXP12 Y Design and implementation of DiffServ over IXP12 Y External benchmark Y Internal benchmark Y Conclusions and future works 1

2 Motivation Y Scalable data-plane processing in Firewall, DiffServ, and WebSwitch Y Solutions Y General processor Y General processor + ASIC Y General processor + co-processors (NP) Y Offload the data-plane processing to co-processors Introduction(1/2) Y Why Network Processor? Y Scalability and programmability Y Hardware-based threads in co-processors, zero context swap overhead Y Specifically designed instruction set for networking purpose Instructions of Instructiondescription Instructionsof x86processor IXP12 ALU PerformALUwithshift inoneinstruction ALU(ADDor SUB) +shift IMMED Load an immediate value with shift Load + shift FIND_BSET LOAD_BSET 1 Determinethepositionof thefirst set bit ina16-bit fieldof aregister At least 5instructionstotest one single bit 2 Shift optionprovided BR_BSET Branchif thespecifiedbit inaregister isset Shift +bit test +JUMP HASH1_64 Performone64-bit hashoperation Manyinstructions needed 2

3 Introduction(2/2) Y Related work--ip forwarding over IXP12 [Spalink, SOSP 18] Y Identify that SDRAM is the bottleneck in IP forwarding Y May not be generalized for other complex applications Y Investigated issues Y Map DiffServ modules to IXP12 Y Flow scalability Y Aggregate throughput Y Internal simulations Y Bottlenecks of IXP12 in DiffServ þýüûþýúùøý öõôú ôóýú 3

4 Components of IXP12 Y Y Processors Y Integrated StrongArm Core for control-plane Y Six integrated programmable microengines for data-plane Each has four threads Interfaces and Storage Y 64-bit IX Bus, 42Gbps peak bandwidth Y 32MB, 64-bit SDRAM interface (up to 256MB) Y 2MB, 32-bit SRAM interface (up to 8MB) Y 2K instruction headroom (named control store ) for each microengine IXP12 Block Diagram IXP12 Network Processor 32 PCI Bus Unit StrongARM Core SDRAM (up to 256MB) SRAM (upto8mb) SDRAM Memory Unit SRAM Memory Unit Microengine1 Microengine2 Microengine3 Microengine4 Microengine5 Microengine6 Boot ROM (up ot 8MB) IX Bus InterfaceUnit Ready Bus Sequencer 64 FIFO Bus (IXBus) 66/85MHz 1Mb/1Mb/1Gb Ethernet MAC Another IXP 12 ü 4

5 Design and Implementation of DiffServ over IXP12 Y Packet flow in a DiffServ edge router Y Data-Plane Architecture Y Detail packet flow chart Y Algorithm description Packet Flow in a DiffServ Edge Router DiffServ (Differentiated Services) edge router Service Configuration Traffic MF classifier Policer Marker Queue Mngt Scheduler þ 5

6 Data-Plane Architecture Thread allocation: consider the size of the control store (2K) receiver thread Rx 8 threads for 8 1/1 ports 8 threads for 1 Giga port StrongARM token bucket timer nextpac ipverify classification policing not EOP packet store enqueue lmatch marking scheduling scheduler thread 1 threads for 1/1 ports 1 threads for 1 Giga port Tx transmitter thread Tx_ReadAssignment 3 threads for 1/1 ports 3 threads for 1 Giga port Tx fill DiffServ Packet Flow in IXP12 The basic unit in IXP12 is the 64 bytes MAC Packet (MP) 6

7 Multi-Dimensional Range Matching [Lakshman, 1998] A rule is a pair of IP addresses which specifies a range Use Bit Vector table to record the overlapped rules in an interval FIND_BSET can be applied for finding the first matched fule Source IP dimension Src_ip of the packet #3 #7 #2 #5 #6 #1 Time complexity: O(log n) Space complexity: O(n^2) # BVn bit vector for this interval Intervals B E SRAM BV table 512bit BV1 BVn Deficit Round Robin [Shreedhar, 1996] Step1 Round Robin Pointer Packet Queues Deficit Counter Step2 Round Robin Pointer Deficit Counter # # #2 5 5 # # # # # Quantum Size 5 Quantum Size 5 7

8 Packet in the Queue SRAM_QUEUE_DESCRIPTOR_BASE Per Queue, Per Port SRAM_BUFF_DESCRIPTOR_BASE PACKET_FREELIST SDRAM_PKT_BUFF_BASE Actual Packet Storage Head Addr Tail Addr Packet Count 32 Next buf_handle buf_des_addr* bytes MP 64 bytes MP 32 Next buf_handle buf_des_addr*64 32 Next buf_handle NULL buf_des_addr*64 þýüûúùø öüúõôóùûò YScalability test YAggregate throughput test IXP12 8

9 Scalability Test 5 EF Flows Maximum load that results no packet loss = 58% Flow fairness test (Len=64bytes, input port x1, 5 flows, BW=744/5=148pps) Throughput (pps) Input load 1% 5% Flow index Aggregate Throughput Throughput (Mbps) Aggregate throughput (Len=64bytes, worst case) 1FE on 1ME 2FE on 2 ME 4FE on 1 ME 8FE on 2 ME 1Gigaon 2ME 8FE + 1Giga Number of policy rules Wire speed (18Gbps) in IP forwarding while 29Mbps in DiffServ? -- The complex computation and the delay of SRAM access ü 9

10 úýüûúùø öüúõôóùûò Host PC StrongARM App HAL Transactor IXP12 Transactor IO 1 Input Port: -6 2 Wire-speed input traffic virtual devices SRAM SDRAM MEs IX bus Simulation Results Algorithm for classifier SRAM Util ME Util SDRAM Util Bottleneck Linear Search 55% 8% 9% SRAM Range Matching 353% 1% 13% ME YSRAM bottleneck YA bottleneck unit needs not to be 1% utilized YBursty SRAM access YMay also cause an idle microengine YME bottleneck YComplex data calculations in Range Matching þ 1

11 Performance Statistics History(RM) Performance statistics History(LN) 11

12 Possible Solutions for Bottlenecks (1/2) Y SRAM bottleneck Y Divide one large SRAM into smaller banks YEach has an interface for accessing a portion of the address space YAn arbitrator decides which bank to go Y Redundant memory modules Y Another memory architecture YQDR SRAM (16Gbps, 2~3 times faster than ordinary SRAM) Y Additional cache memory for exploiting locality Possible Solutions for Bottlenecks (2/2) Y ME bottleneck Y Fair, dynamic workload assignment Y Hardware upgrade Y Programming optimization 12

13 Conclusion and Future Works (1/2) Y ó øüóüúý ùú üúõôóùûò üû üû úüý ûò û õü û Y ó ýùý ú ùú ýýøüúüõò úüý ûò û õü û Y óùøø õ úýû ø ý ûü ó øü ù ø õùý ú úø Y Y ö ýýøüúüõò ý ø ùý ú Y þýýüûúûùø öõôóýö ý ûü ûò Y úùó õ ù úóüúý ûòø ù ý Y õôü ø ú ùó ú ý ü ýôûüù ú ù Conclusions and Future Works (2/2) Application specific bottleneck: bottleneck may shift! 13

14 References [1] IXP12 Data Sheet, Intel document number , May 2 [2] T Spalink, S Karlin, L Peterson, and Y Gottlieb, Building a Robust Software-Based Router Using Network Processors, Proceedings of the 18th ACM Symposium on Operating Systems Principles (SOSP) [3]SBlake,DBlack,MCarlson,EDavies,ZWang,andWWeiss, AnArchitecturefor Differentiated Services, RFC 2475, Dec 1998 [4] P Gupta, and N McKeown, Packet Classification on Multiple Fields, ACM SIGCOMM 99 [5]VSrinivasan,GVarghese,SSuri,andMWaldvogel, FastandScalableLayerFour Switching, ACM SIGCOMM 98 [6] A Demers, S Keshav, and S Shenker, Analysis and Simulation of a Fair Queuing Algorithm, ACM SIGCOMM 89 [7] TV Lakshman, and D Stiliadis, High-Speed Policy-based Packet Forwarding Using Efficient Multi-dimensional Range Matching, ACM SIGCOMM 98 [8] M Shreedhar, and G Varghese, Efficient Fair Queuing Using Deficit Round-Robin, IEEE/ACM Transactions on Networking, June 1996, vol 4, no 3, pp [9] LV Nguyen, T Eyers, and JF Chicharo, Differentiated Service Performance Analysis, Fifth IEEE Symposium on Computers and Communications, 2, pp [1] JK Muppala, T Bancherdvanich, and A Tyagi, VoIP Performance on Differentiated Services Enabled Network, IEEE International Conference on Network, 2, pp [11] J Harju, and P Kivimaki, Co-operation and Comparison of Diffserv and Intserv: performance measurements, 25th Annual IEEE Conference on Local Computer Networks, 2, pp [12] Z Di, and HT Mouftah, Performance Evaluation of Per-Hop Forwarding Behaviors in the DiffServ Internet, Fifth IEEE Symposium on Computers and Communications, 2, pp