Field-programmable Port Extender (FPX) August 2001 Workshop. John Lockwood, Assistant Professor

Transcription

1 Field-programmable Port Extender (FPX) August 2001 Workshop John Lockwood, Assistant Professor Washington University Department of Computer Science Applied Research Lab 1 Brookings Drive Saint Louis, MO Supported by: NSF ANI and Xilinx Corp. FPX Network Platform 1 Workshop Objectives Learn to accelerate network processing with reprogrammable hardware Understand System-On-Chip design Perform Hardware/Software Co-design Obtain hands-on experience with the Field Programmable Port Extender (FPX) FPX Network Platform 2

2 Outline Motivation Hardware enables packet processing at link speed FPGAs provide flexibility for dynamically reconfiguration Networks can be extended with FPGAs to provide enhanced functionality and performance. Research Results Field programmable Port Extender (FPX) allows modules to be dynamically installed in a network. FPX serves as an open platform for rapid prototype of firewall and router plug-in modules Courses and Workshops held at Washington University to develop new System-On-Chip networking modules FPX Network Platform 3 Workshop Activities Integrate FPX Infrastructure Components SDRAM Memory Controller Internet Protocol Wrappers Operate FPX Software Tools NCHARGE Control Software PARBIT FPGA Tools Implement Networking s on the FPX Cell processing module in VHDL Packet Processing KCPSM Active Networking FPX Network Platform 4

3 FPX Workshop Agenda: Times and Location Wednesday, Aug 15, am: Breakfast 5th floor Jolley Atrium 9am-Noon: Session I Sever 201 Lab Lunch WashU Campus 1pm-5pm: Session II Sever 201 Lab Thursday, Aug 16, am: Breakfast 5 th floor Jolley Atrium 9am-Noon: Session III Sever 201 Lab Lunch 5th floor Jolley Atrium 1pm-5pm: Session IV Sever 201 Lab On-line Agenda: FPX Network Platform 5 Building Networks with Reprogrammable hardware FPX Network Platform 6

4 S R S R Technology Options for Evolvable Internet Hardware Reprogrammable Hardware Fully Reprogrammable Flexibility Microprocessor Network Processors High Performance ASIC Performance FPX Network Platform 7 Reprogrammable Device Configuration G1 G2 G3 G4 H1 F1 F2 F3 F4 CLB 4 LUT G 4 LUT F 3 LUT H Din M M M M Clk D D Q Q YQ Y XQ X CLB : Primitive element of FPGA GRM Local Routing Routing : Interconnection of Blocks CLB PIP 3rd Generation LUT-based FPGA FPGA : Matrix of CLBs and Routing s Pad Routing CLB Matrix I/O... Macro Block (up, Mem) FPX Network Platform 8

5 The FPX Network Platform FPX Network Platform 9 Configuration of Network Packet Processor Network Packets Line Card OC3/ OC12/ OC48 FPX Fieldprogrammable Port Extender Line Card OC3/ OC12/ OC48 Network Packets Packet processing hardware performs: Packet classification Packet forwarding Address Translation modification Packet buffering Active Networking (Application-level data processing) FPX Network Platform 10

6 Field Programmable Port Extender FPX Network Platform 11 Configuration of Internet Router Network Packets Network Packets Line Card OC3/ OC12/ OC48 Line Card OC3/ OC12/ OC48 FPX FPX Fieldprogrammable Port Extender Fieldprogrammable Port Extender Additionally, Router interface performs: Internet route lookup Traffic policing and shaping IPP IPP IPP IPP Gigabit Switch Fabric OPP OPP OPP OPP FPX Network Platform 12

7 Port Processing at edge of Gigabit Switch Original Network Switch Line card connects to Gigabit switch backplane Line Card Optics Fiber Gigabit switch backplane VRM SDRAMs FPX-Enhanced Router Line card connects to Gigabit switch backplane Line Card FPX Fiber Optics NID SRAMs Gigabit switch backplane VRM SDRAMs FPX Network Platform 13 Complete Router Platform FPX Network Platform 14

8 Combination Router Hardware and Software Line Card OC3/ OC12/ OC48 SPC Smart Port Card FPX Fieldprogrammable Port Extender IPP IPP WUGS OPP OPP Line Card OC3/ OC12/ OC48 SPC Smart Port Card FPX Fieldprogrammable Port Extender IPP IPP Gigabit Switch Fabric OPP OPP Implement link speed opertions on hardware Implement higher-level functions in software Migrate functionality on the critical path FPX Network Platform 15 Port Configuration WUGS FPX SPC Intel Embedded To/From other network ports SDRAM SRAM SelectMap Programming Interface Control Cell Processor Program SRAM Error Check Circuit temp ccp Four Port Switch NID SDRAM SRAM Synch, Buffered Interface Virtual Circuit Lookup Table Asynchronous Interface Pentium North Bridge PCI Bus PCI Interface Three Port Switch APIC SRAM South Bridge Line Card Switching Hardware-based Packet Processing Software-based Packet Processing FPX Network Platform 16

9 The FPX Architecture FPX Network Platform 17 Architecture of the FPX Large Xilinx FPGA Attaches to SRAM and SDRAM Reprogrammable over network Provides two user-defined Interfaces SDRAM SRAM SDRAM SRAM NID Provides Utopia Interfaces between switch & line card Forwards cells to Programs Program SRAM Switch LineCard NID FPX Network Platform 18

10 Field Programmable Port Extender NID : Network Interface Device : Reprogrammable Application Device FPX Network Platform 19 FPX SRAM Provide low latency for fast table-lookups Zero Bus Turnaround (ZBT) allows back-to-back read / write operations every 10ns Dual, Independent Memories 36-bit wide bus FPX Network Platform 20

11 FPX SDRAM Dual, independent SDRAM memories 64-bit wide, 100 MHz 64MByte / : 128 Mbyte total [expandable] Burst-based transactions [1-8 word transfers] Latency of 14 cycles to Read/Write 8-word burst FPX Network Platform 21 Hardware Device FPX Network Platform 22

12 FPX Interface FPX Network Platform 23 FPX Interfaces Provides Well defined Interface Utopia-like 32-bit fast data interface Flow control allows back-pressure Flow Routing Arbitrary permutations of packet flows through ports Dynamically Reprogrammable Other modules continue to operate even while new module is being reprogrammed Memory Access Shared access to SRAM and SDRAM Request/Grant protocol FPX Network Platform 24

13 Reprogrammable Application Device () SDRAM SRAM SDRAM SRAM Network Interfaces to NID Spatial Re-use of FPGA Resources s implemented using FPGA logic logic can be individually reprogrammed Shared Access to off-chip resources Memory Interfaces to SRAM and SDRAM Common path to send and receive data FPX Network Platform 25 Network Hardware Interface D_MOD_IN[31:0] SOC_MOD_IN TCA_MOD_OUT fpx_module.vhd SRAM_GR SRAM_D_IN[35:0] SDRAM_GR SDRAM_DATA[63:0] CLK RESET_L ENABLE_L Interface FPX Network SRAM Interface SDRAM Interface Interface D_MOD_OUT[31:0] SOC_MOD_OUT TCA_MOD_IN READY_L SRAM_REQ SRAM_D_OUT[35:0] SRAM_ADDR[17:0] SRAM_WR_RD SDRAM_RQ SDRAM_EN SDRAM_BL[4:0] SDRAM_ADDR[26:0] SDRAM_WR_RD SDRAM_OP_FIN FPX Network Platform 26

14 Interface Cell I/O and Flow Control 32-bit wide UTOPIA-style interface w/ unique timing Off-chip Memory Access Arbitrated access to SRAM and SDRAM via standard interface Control (clock, reset, and reconfiguration control) FPX Network Platform 27 Infrastructure Services FPX Network Platform 28

15 Routing Traffic Flows Between s Traffic flows routed among Switch Line Card.Switch.Linecard NID Functions Check packets for errors ccp Process commands Control, status, & reprogramming Implement per-flow forwarding ccp Switch LineCard FPX Network Platform 29 Example NID Routing Configurations Hardware Bypass (Default) No s Installed One Installed One Installed Egress Processing Ingress Processing ccp NID Switch LineCard Default Flow Action (Bypass) ccp NID Switch LineCard Egress (SW) Processing (Per-flow Output Queueing) ccp NID Switch LineCard Ingress (LC) Processing (IP Routing) Ingress+Egress Loopback Chained Egress Processing ccp 0 NID Switch LineCard Full Processing (Packet Routing and Reassembly) ccp NID Switch LineCard Full Loopback Testing (System Test) ccp NID Switch LineCard Dual Egress Processing (Chained s) FPX Network Platform 30

16 NID Flow Entry Example I 45 d1 d2 d3 d4 One Installed 2 3 NID 0 1 ccp Switch LineCard Ingress (LC) Processing (IP Routing) Problem: Route flow on I 45 from Switch to Mod_sw Route Flow on I 45 from Mod_sw to LC Route Flow on I 45 from Linecard to Switch Solution I [45] = 1,3,X,0 FPX Network Platform 31 FPX Control : NCHARGE FPX Network Platform 32

17 HDR H OpCode PL1 PL2 PL3 PL4 PL5 PL6 PL7 PL8 PL9 PL10 PL11 Control Cell Format for 32/36 bit SRAM Memory Operations V V V GFC / VPI D D D H OpCode R R R F F F Sequence # I PAD ID V - Valid Command: 1 = Valid command, 0 = Invalid, EOC D - Device: 1 = Device 1, 0 = Device 0 R - Read or Write: 1 = Read, 0 = Write F - 32 or 36 bit: 1 = 36 bit, 0 = 32 bit ADDR(18:0) WORD 0 (31:0) WORD 1 (31:0) ADDR(18:0) WORD 0 (31:0) ADDR(18:0) WORD 0 WORD 1 WORD N CM DATA N/A 1 0 W0(35:32) W0(35:32) X C(7:5) PTI W1(35:32) N/A N/A CRC VPI = 0x000, I = 0x0023 (35) Control Cell OpCode = 0x14 SRAM Memory Operation OpCode = 0x15 SRAM Memory Operation Response ID = 0x00 Control Cell Processor 36 Bit format 2 Address 36 Bit format 1 Address 32 Bit format N Address Pictorial view of fpx_control interfaced with hardware Switch Controller Fpx_control {0-7}.{0/1} FPX Network Platform 33 Controlling the FPX Fip Memory Remote Manager Applications Basic Read Telnet WEB Send Fip Access CGI Basic Send Software Controller NCHARGE NCHARGE I 76 (NID), I 100 () I 115 (NID), I 123 () OC-3 Link (up to 32 Is) Washington University Gigabit Switch NID NID Methods of Communication - NCHARGE -Telnet - Web Interface / CGI - Basic_send - User Applications Software Plug-ins - Concepts - Functionality Emulation Nid_listener Rad_listener FPX Network Platform 34

18 Web Access to NCHARGE Web Access Provides: - Radio Button Interface - Allows user to submit commands using CGI scripts - Provides for Switch Reset - FPX Network Platform 35 FPX Control and Reconfiguration FPX Network Platform 36

19 SDRAM (backside) PCB Trace Density (backside) SDRAM 100MHz 62.5 MHz ccp Example: Write Configuration Memory Switch Controller generates command to write 32-byte data element to Configuration Memory Control Cell sent containing Memory Address and Switch Element routes control cell to FPX NID Element on FPX writes from Payload into Rad Configuration Memory Configuration Memory Switch Controller Control Cell VPI I = 0x34 H OPCODE Reserved CM PTI = 0 0 WUGS Switch Element WUGS Switch Backplane Connector OPPLC OPPLC OPP LC OPPLC OPPLC OPP LC OPPLC OPPLC SRAM 8Mbit ZBT SRAM 8Mbit ZBT Reprogrammable Application Device Virtex1000E fg680 Program FIFO OSC OSC VRM 2.5V (backside) NID EPROM NID Network Interface Device VRM 1.8V (backside) FPX OC3 / OC12 / OC48 Linecard Connector Sequence # CRC FPX Network Platform 37 Reprogramming Logic NID programs at boot from EPROM Switch Controller writes configuration memory to NID Bitfile for arrives transmitted over network via control cells Switch Controller issues {Full/Partial} reconfigure command NID reads config memory to program Performs complete or partial reprogramming of Switch Element OPP LC OPP LC OPP LC OPP LC OPP LC OPP LC OPP LC OPP LC WUGS Switch Backplane Connector SRAM 8Mbit ZBT SRAM 8Mbit ZBT SDRAM (backside) Reprogrammable Application Device Virtex1000E fg680 (backside) SDRAM PCB Trace Density VRM 2.5V (backside) NID Program FIFO EPROM OSC 100MHz OSC 62.5 MHz NID Network Interface Device VRM 1.8V (backside) OC3 / OC12 / OC48 Linecard Connector FPX Network Platform 38

20 SRAM SDRAM SRAM SDRAM System-On-Chip Design using Dynamic Hardware Plugins (DHP) FPX Network Platform 39 Combining s within the Chip s fit together at static I/O interfaces Intrachip Switching Partial reprogramming of FPGA used to install/remove modules s added and removed while other modules process packts SDRAM SRAM FPX FPX FPX... Statically-configured Long Lines provide chip-wide routing FPGA s Long Lines Loading / Unloading FPX Network Platform 40

21 Implementing modules in Virtex1000E Virtex 1000E logic resources Ingress Path Globally accessible IOBs 64 x 96 CLB array 4 flops/luts per CLB 96 Block SelectRAMs 4096 bits per block 6 columns of 16 blocks 6 columns of dedicated interconnect DHP s 64 x 12 CLB array (768 CLBs, 3072 flops) Double DHP s 64 x 24 CLB array (1536 CLBs, 6144 flops) 16 BRAMs (8KB) per 3 DHP s per path 1 SRAM interface per path 1 SDRAM interface per path IOB Ring CLB columns VersaRing DHP Egress Path FPX Network Platform 41 DHP Double DHP Double DHP DHP DHP BRAMs BRAM Interconnect Dynamic Hardware Plugins [On FPGA] TARGET 1 TARGET 2 TARGET 3 TARGET 4 T1 (7,8) T2 (7,20) T3 (7,68) T4 (7,80) Left IOBs Top IOBs I N F R A S T R U C U T U R E Bottom IOBs Right IOBs Synthesis Constraints Infrastructure with target regions reserved for DHP modules insertions PARBIT parameters Target Locations (Row, Col) RAM RAM RAM RAM RAM RAM FPX Network Platform 42

22 Example Application : String Processing FPX Network Platform 43 Hello, World Function Compare Header [5 bytes in 2 words] Payload [48 bytes in 12 words] JWL:ARL 07/00 VPI I=5 VPI=X I=5 PAD PAD H E L L H E L L Match O P5 P6 P7 P8 P9 P10 P11.. Payload.... Payload.. P44 P45 P46 P47 P44 P45 P46 P47 32 bits Match O W O Match+Write R L D. Write... Copy... Copy FPX Network Platform 44

23 Evolvable Internet Hardware Compile Design (Vcom) Tweak Design Synthesize To Logic (Synplicity) Observe Network Behaviour Place and Route FPGA Gates (Xilinx) Route Traffic Through (Ncharge) Download to (Ncharge) FPX Network Platform 45 Modular Interface to SDRAM FPX Network Platform 46

24 SDRAM Controller Overview SDRAM Controller 0 1 SDRAM 2 FPX Network Platform 47 Example : Leaky Bucket bursty data leaking data - Buffers the incoming cells in a FIFO - Generates tokens at a regular interval - Gives out a cell from the FIFO when the number of tokens > 0 - Destroys a token when a cell is given out FPX Network Platform 48

25 Internet Protocol Wrappers FPX Network Platform 49 Payload Processing Environment Higher-Level data processing on the FPX Wrapper Framework Net App Wrapper Wrapper FPX Network Platform 50

26 Frame / IP Packet / UDP / Application Layers Interfaces to Off-Chip Memories Application-level Hardware Input Output UDP Processor IP Processor Frame Processor Cell Processor FPX Network Platform 51 Soft-core Active Network Processor : The KCPSM Network FPX Network Platform 52

27 The FPX KCPSM KCPSM ATM CELLS UDP PACKETS PORT_ID I/O BUS DATA MEMORY ADDR INST PROGRAM MEMORY UDP PACKETS ATM CELLS D_MOD_IN SOC_MOD_IN TCA_MOD_IN ADDR D_MOD_IN CONTROL SIGNALS DATA BUS ADDR INTERFACE INST D_OUT_MOD CONTROL SIGNALS PROTOCOL WRAPPERS D_OUT_MOD SOC_OUT_MOD TCA_OUT_MOD CLK RESET_L ENABLE_L READY_L FPX Network Platform 53 Simulating a KCPSM Program The input data cell The The st rd th st rd th word word nd th nd th th th specifies word specifies word is is is is the the it it Internet data Internet data ATM is ATM string is string UDP a UDP Protocol data Cell Protocol data Cell Header Hello Header Hello packet packet Header World Header World The output data cell The The st rd th st rd th word word nd th nd th th th specifies word specifies word is is is is the the it it data Internet data Internet ATM is string ATM is string UDP a UDP Protocol data Cell Protocol data Cell Uryyb Header Uryyb Header packet packet Header Jbeyq Header Jbeyq FPX Network Platform 54

28 Applications for the FPX: Fast IP Lookup (FIPL) FPX Network Platform 55 Fast IP Lookup Algorithm Function: Search for best matching prefix using Trie algorithm Prefix Next Hop * 01* * 2 110* * * * * Contributors Will Eatherton, Zubin Dittia, Jon Turner, David Taylor, David Wilke, FPX Network Platform 56

29 Hardware Implementation in the FPX SRAM 1 SRAM 2 Extract IP Headers Packet Reassembler SRAM 1 Interface 0 Remap Is 1 Request Grant for IP packets IP Lookup 0 Engine counter On-Chip Cell Store 1 FPGA Control Cell Processor LC NID FPGA SW FPX Network Platform 57 Fast IP Lookup (FIPL) Application Lookup (X.Y.Z.W) Route add /24 8 Route delete /16 Commands FIPL Memory Manager RAM FIPL Control cells Fast IP Lookup External SRAM Hardware Lookup Software FPGA Nexthop FPX Network Platform 58

30 Conclusion Field programmable Port Extender (FPX) Platform Open Platform for hardware development Modular Interfaces Allows integration components to build System-on-Chip (SoC) Library of Internet packet processing functions Simplifies design of new functionality Interoperable w/existing software systems Unifies Active Networking Hardware with Software FPX Homepage FPX Network Platform 59 Acknowledgements Several Individuals have contributed to this work: Washington University Jon Turner Sarang Dharmapurikar Todd Sproull David Taylor Florian Braun Henry Fu Dave Lim Edson Horta Xilinx Dave Parlour FPX Network Platform 60