A Deep Dive on the QorIQ T1040 L2 Switch

Transcription

1 A Deep Dive on the QorIQ T1040 L2 Switch FTF-NET-F0007 Suchit Lepcha Application Engineering Manager F e b TM External Use

2 Agenda Overview Switch Functions Software Conclusion External Use 1

3 PCIe PCIe PCIe PCIe SATA 2.0 SATA 2.0 TDM/HDLC TDM/HDLC T1040 Architecture Processor 4x e5500, 64b, up to 1.4GHz Each with 256KB backside L2 cache 256KB Shared Platform Cache w/ecc Supports up to 64GB addressability (36 bit physical addressing) Memory SubSystem 32/64b DDR3L/4 Controller up to 1600MT/s Cygnus Switch Fabric High Speed Serial IO 4x PCIe Gen2 Controllers 2x SATA 2.0, 3Gb/s 2x USB 2.0 with PHY Network IO FMan packet Parse/Classify/Distribute Lossless Flow Control, IEEE 1588 Up to 4x 10/100/1000 Ethernet Controllers 8-Port Gigabit Ethernet Switch QUICC Engine HDLC, 2x TDM Green Energy Operation Fanless operation quad-core 1.2GHz Packet lossless deepsleep Programmable wake-on-packet Wake-on-timer/GPIO/USB/IRQ Security Fuse Processor Security Monitor 16b IFC Power Management SD/MMC+ 2x DUART 2x I2C SPI, GPIO 2x USB 2.0 w/phy DIU Power Architecture 256 KB e5500 Backside L2 Cache 32 KB 32 KB PAMU Security 5.x (XoR, CRC) Pattern Match Engine 2.0 D-Cache Queue Mgr. Buffer Mgr. PAMU I-Cache Parse, Classify, Distribute 8 Port Switch Device 28HPM Process 780-pin C4 FC package 23x23mm, 0.8mm pitch Power targets Enable Convection cooled system design CoreNet Coherency Fabric PAMU 2xDMA 8-Lane 5GHz SERDES 256KB Platform Cache QUICC Engine 32/64-bit DDR3L/4 DDR2/3 Memory Controller PAMU Perf Monitor Datapath Acceleration SEC- crypto acceleration PME- Reg-ex Pattern Matcher Peripheral Access Mgmt Unit Real Time Debug Watchpoint Cross Trigger CoreNet Trace External Use 2

4 L2 Switch Summary Fully non-blocking wire speed Ethernet switch with WRED 8x user facing ports 2Mbit packet memory 8k addresses 4k VLAN support Jumbo frame support (10kB) 8x QoS, 8x Queues/Port External Use 3

5 T1040: Gigabit Ethernet Switch Advanced Features Priority flow control - lossless Lower latency and shared buffer management Advanced classification, shaping and policing Power savings With support for latest standards including IEEE 802.3az Energy Efficient Ethernet (EEE) Cost savings Through switch integration, low-pin count QSGMII connectivity and port count / cost optimization Increased ROI - Lower TTM and high re-use Integrated solution kit with software reuse potential Support for Full featured L2 software stacks Management I/F SGMII SGMII SGMII QMan I/F SGMII 2.5G 2.5G BMan I/F TCAM 1K QSGMII 2.5G 2.5G Fabric I/F IEEE 1588v2 IEEE 1588v2 SGMII Sec SGMII Parse, Classify, Distribute Quad 4 x SGMII or 2 x SGMII or PHY 8K s 4K VLANs RMON Counters QSGMII FMan L2- Switch Quad PHY 5GHz SERDES External Use 4

6 Packet Flow E5500 QMan I/F 2.5G BMan I/F 2.5G Fabric I/F Sec IEEE 1588v2 Parse, Classify, Distribute FMan Management I/F 2.5G 2.5G IEEE 1588v2 L2- Switch TCAM 1K 8K s 4K VLANs Control Packets Packet forwarding RMON Counters WAN traffic SGMII SGMII SGMII SGMII QSGMII SGMII SGMII QSGMII 5GHz SERDES Quad 4 x SGMII or 2 x SGMII or PHY Quad PHY External Use 5

7 Bandwidth, cost and power Generic Enterprise Router Features Higher QoS benefit lossless behavior 8 queues/port PFC (Priority based Flow Control) Sophisticated classification Enterprise Complex classification requirements - benefit treat user traffic differently and offload the processor Higher ACL requirements - benefit redirect/drop/deny access Delivering all of these in low power SME Gateway Features Enterprise Router External Use 6

8 Agenda Overview Switch Functions Block Diagram Forward Frames Learning Avoid loops System Interface Software Conclusion External Use 7

9 Block Diagram CPU Interface Shared Queue System Buffer Memory Shared Memory Pool 2Mbit v Memory Controller Shapers and Schedulers MIIM Controller MIIM TCAM Packet Processing Ingress and Egress Ingress Processing L2 Forwarding DLB Policers Frame Classification (QoS VLAN) Translation/Remarking IS1 TCAM Ingress Statistics Security Enforcement /IP Binding IS2 TCAM v Egress Processing Rewriter VLAN Translation Push/pop tags DSCP remapping ES0 TCAM Egress Statistics Register Access CPU Port Module Port #10 CPU Frame Extraction and Rejection v Control Signals System Bus System Clock 156MHz System Reset v Port Module Interface Switch Core 10 switch ports: 8x be + 2x be/ 2.5GbE Port Module #0 Port Module #7 2.5G Port Module #8 2.5G Port Module #9 External Use 8

10 Agenda Overview Switch Functions Block Diagram Forward Frames Interface Ingress Processing Shared Queue System Egress Processing Learning Avoid loops System Interface Software Conclusion External Use 9

11 Block Shared Queue System Shared Memory Pool 2Mbit Memory Controller Shapers and Schedulers MIIM Controller MIIM Ingress Processing L2 Forwarding DLB Policers Frame Classification (QoS VLAN) Translation/Remarking IS1 TCAM Ingress Statistics Security Enforcement /IP Binding IS2 TCAM Egress Processing Rewriter VLAN Translation Push/pop tags DSCP remapping ES0 TCAM Egress Statistics Register Access CPU Port Module Port #10 CPU Frame Extraction and Rejection Control Signals System Bus System Clock 156MHz System Reset ATPG Enable Port Module Interface 10 switch ports: 8x be + 2x be/ 2.5GbE Port Module #0 Port Module #7 2.5G Port Module #8 2.5G Port Module #9 External Use 10

12 Functions VLAN Tag aware frame size check Frame Check Sequence (FCS) check Pause frame identification Energy Efficient Ethernet (EEE) IEEE 802.3az External Use 11

13 IEEE 802.3az Saves power during low data utilization periods Works in 100BASE-TX & 1000BASE-T speeds Additionally, new 10BASE-Te mode reduces 10Mbit transmit from 5Vpp to 3.3V When both link partners support 802.3az: during auto-negotiation, PHYs advertise their EEE idle capabilities ~0%-60% per port power is saved on both systems depending upon link utilization in the PHY; 0%-35% typical at the up/switch/phy level Actual measurements will need to be made for T F104 (QSGMII PHY) Backward compatible to support non-802.3az PHYs However, for 802.3az to save energy, both link partners must support 802.3az External Use 12

14 Agenda Overview Switch Functions Block Diagram Forward Frames Interface Ingress Processing Basic Classification Advanced Classification Policing L2 Forwarding Shared Queue System Egress Processing Learning Avoid loops System Interface Software Conclusion External Use 13

15 Ingress Processing Block Shared Memory Pool 2Mbit Shared Queue System Memory Controller Shapers and Schedulers MIIM Controller MIIM TCAM Packet Processing Ingress and Egress Ingress Processing L2 Forwarding DLB Policers Frame Classification (QoS VLAN) Translation/Remarking IS1 TCAM Ingress Statistics Security Enforcement /IP Binding IS2 TCAM v Egress Processing Rewriter VLAN Translation Push/pop tags DSCP remapping ES0 TCAM Egress Statistics Register Access CPU Port Module Port #10 CPU Frame Extraction and Rejection Control Signals System Bus System Clock 156MHz System Reset ATPG Enable Port Module Interface Port Module #0 Port Module #7 2.5G Port Module #8 2.5G Port Module #9 External Use 14

16 Basic and Advanced Frame Classification Frame Data Basic Classification Frame Acceptance Untagged, S-tagged, C-tagged Special frames Frame Data Discard Advanced Classification IS1 First Lookup QoS Class DP Level Classified DSCP VLAN VLAN tag from frame Port VLAN VLAN Tag Header VLAN pop count IS1 Second Lookup Classified VLAN VLAP pop cnt QoS, DP, and DSCP PCP from tag (inner or outer) DSCP from frame, trusted values only Remap/rewrite of DSCP Port default QoS Class DP Level DSCP Value IS1 Third Lookup Custom lookup Aggregation Code L2-L4 frame data Aggregation Code IS1 Key: port mask inner and outer VLAN tags S, D SIP, DIP TCP/UDP ports frame type, DSCP, range checkers External Use 15

17 Basic Classification Frame acceptance Valid VLAN tags Valid addresses VLAN classification Untagged port are part of default VLAN Tagged ports classified based on TCI (PCP, DEI, and VID) and TPID (C-tag or S-tag) QoS, DP, and DSCP frames colored green/yellow based on QoS and DP Aggregation Code Based on information from /IP address, TCP/IP port numbers Preamble Ether SFD Destination Source Type/S ize Payload CRS/FCS Inter frame Gap n Preamble Ether SFD Destination Source 802.1Q Header Type/S ize Payload CRS/FCS Inter frame Gap n TPID TCI PCP DEI VID External Use 16

18 Advance Multi-stage Classification Three TCAMs with different purposes: IS1: L3-aware Ethernet classification IS2: Security handling (ACLs), other control protocols ES0: Egress handling (QoS, VLAN) TCAM sizes: IS1, IS2, ES0: entries depend on complexity of rules Classification Results QoS handling VLAN handling ACL actions External Use 17

19 IS1 Action Each IS1 lookup results in an action vector Following fields can be overwritten as action: DSCP value QoS value DP value PAG value VID (VLAN ID) FID (Filter identifier) PCP/DEI Custom ACE Type: Custom lookup in IS2 External Use 18

20 Comprehensive Classification and Statistics Ingress classification with the following parameters: Mapping to policers Ingress statistics (bytes and frames) Green/Yellow/Red Arrivals Green/Yellow discards related to L2 forward and congestion avoidance (WRED) Egress TCAM lookup for per-port encapsulation and statistics Egress statistics (bytes and frames) Green/Yellow Departures External Use 19

21 IS2 Actions Permit Deny Police Redirect CPU copy Mirror External Use 20

22 Policing & Shaping Supports policing ingress and egress traffic Supports shaping of egress traffic Time Policing Time Shaping Time Time External Use 21

23 Policers 3-levels of hierarchical policing Up to 3 policers per frame: Queue, Port, and VCAP IS2 policers MEF-compliant DLB policers A total of 163 DLB policers 88 queue policers Eight policers per port 11 port policers One policer for each port 64 VCAP IS2 policers Four global storm policers for all the ingress traffic Erroneous frames, pause frames or control frames are not presented to policers External Use 22

24 Layer 2 Forwarding The switch has 8K Table and 4K VLAN Table L2 forwarding done on the basis of : VLAN classification Security enforcement (result of IS2) addresses Learning (disabled/unsecure/secure) Link aggregation Mirroring External Use 23

25 Agenda Overview Switch Functions Block Diagram Forward Frames Learning Addresses VLAN Multicast Avoid loops System Interface Software Conclusion External Use 24

26 Learning: Addresses 8,192 addresses 4,096 VLANs (IEEE 802.1Q) Wire speed hardware based learning Per-port CPU-based learning with option for secure CPU-based learning Learning can also be disabled CPU can add entries in the table External Use 25

27 Learning: VLAN Independent VLAN learning addresses are learnt separately on each VLAN in independent VLAN Shared VLAN learning A table entry is shared among multiple VLANs Provider Bridging (VLAN Q-in-Q) support (IEEE 802.1ad) Choice between using inner or outer VLAN tags External Use 26

28 Learning: Multicast Upto 8,192 multicast groups Internet Group Management Protocol (IGMPv2/v3) support Multicast Listener Discovery (MLD) support Multicast Learning IGMP and MLD frames are copied to CPU CPU can create entries of multicast addresses in table Multicast addresses in table do no age Multicast frames with unknown addresses are forwarded to all the ports External Use 27

29 Agenda Overview Switch Functions Block Diagram Forward Frames Learning Avoid loops Loop Problems Spanning Tree Protocol System Interface Software Conclusion External Use 28

30 Spanning Tree Protocol IEEE802.1D standardized Spanning Tree Protocol Cisco introduced Per-VLAN Spanning Tree (PVST) and Per-VLAN Spanning Tree Plus (PVST+) The IEEE defined Rapid Spanning Tree Protocol (RSTP) as 802.1w and Multiple Spanning Tree Protocol (MSTP) in IEEE 802.1s (later merged in IEEE 802.1Q-2005) External Use 29

31 STP Evolution Rapid Spanning Tree Protocol (RSTP) While STP takes seconds to respond a topology change, RSTP does it in few seconds (typically 2-6 seconds) RSTP added couple of new port classification Alternate port: An alternate path to the root bridge Back port: A backup/redundant path to a segment where another bridge port already connects Multiple Spanning Tree Protocol (MSTP) MSTP configures a separate Spanning Tree for each VLAN group Balances port utilization External Use 30

32 STP Support BPDUs are terminated by the switch core The switch stack running on e5500 core responsible for implementing the protocol The switch supports Redirecting BPDU frames to CPU Configuring ports as State per VLAN BPDU Reception BPDU Generation Frame Forwarding S learning Discarding Yes Yes No No Learning (not supported per VLAN) Yes Yes No Yes Forwarding Yes Yes Yes Yes External Use 31

33 Agenda Overview Switch Functions Block Diagram Forward Frames Learning Avoid loops System Interface Software Conclusion External Use 32

34 CPU Interface Block Shared Queue System CPU Interface Shared Memory Pool 2Mbit Memory Controller Shapers and Schedulers MIIM Controller MIIM Ingress Processing L2 Forwarding DLB Policers Frame Classification (QoS VLAN) Translation/Remarking IS1 TCAM Ingress Statistics Security Enforcement /IP Binding IS2 TCAM Egress Processing Rewriter VLAN Translation Push/pop tags DSCP remapping ES0 TCAM Egress Statistics Register Access CPU Port Module Port #10 CPU Frame Extraction and Rejection v Control Signals System Bus System Clock 156MHz System Reset Port Module Interface Port Module #0 Port Module #7 2.5G Port Module #8 2.5G Port Module #9 External Use 33

35 System Interface System bus interface (32b) Switch register access MIIM/MDIO master ctrl Connects to TBI SerDes Phy For external Phys, MIIM interface of FMAN should be used Three control signals per port Link status Next page Autoneg status MIIM Controller Register Access CPU Port Module Port #10 CPU Frame Extraction and Rejection v CPU Interface MIIM Control Signals System Bus System Clock 156MHz System Reset ATPG Enable External Use 34

36 Interfaces Port 0-7 : Eight ports 10/100/1000 Mbps in fullduplex mode and 10/100 Mbps in half-duplex mode SerDes supports 6x ports or 2x QSGMII ports Port 8-9: Two 2.5G ports These ports are connected to FMAN Port 10: One internal CPU Port This is a logical port to be used as management interface CPU port is through the CPU extraction queue Management I/F SGMII SGMII SGMII QMan I/F SGMII / 2.5G 2.5G BMan I/F TCAM 1K QSGMII / 2.5G 2.5G Fabric I/F IEEE 1588v2 IEEE 1588v2 SGMII Sec SGMII Parse, Classify, Distribute 8K s 4K VLANs RMON Counters QSGMII FMan L2- Switch 5GHz SERDES External Use 35

37 Agenda Overview Switch Functions Software Conclusion External Use 36

38 MIPS Core PPC Core1 PPC Core PPC Core2 SW Background 2 different stacks/applications L2 control stack (Switch) L3/L4 network stack (Router) Legacy operation: Separate SoC dedicated cores. Dedicated devices, drivers, even operating systems. T1040 operation: Share cores using affinity or partitions (AMP) Dedicated devices/portals for L2 and L3/L4 traffic. Clean separation of control and data-path traffic. Clean separation of configuration of L2 (switch driver) and L3/L4 traffic (network stack). L2 Control traffic L2 data traffic L2 Control traffic L2 data traffic External Switch Eth Eth Eth Eth T1040: Switch + Router SoC (Option 1) Eth Eth Eth Eth L2 Control Stack Switch Driver Registers L2 Switch L2 Control Stack Switch Driver Registers L2 Switch Eth Eth Legacy Router SoC Eth Eth L3/L4 NWStack Ethernet Driver Portals DPAA L3/L4 NWStack Ethernet Driver Portals DPAA Eth Eth Eth Eth L3/L4 traffic L3/L4 traffic External Use 37

39 L2-switch SW - T1040 What we offer Linux Linux Non-Linux Linux L3,L4, SEC Control Apps GPL L2 Stack User-Space GPL Customer Mgmt Customer L2 Stack VTSS Mgmt API (GPL) JSON/RPC VTSS SMBStaX Linux User-Space Customer Mgmt Customer L2 Stack Linux Network Stack (L3/L4) ASF Eth / SEC Driver L2 Switch Driver Kernel GPL L2 Switch API Switch Configuration Non-Linux FM- Lib QM-Lib SEC-Lib BM-Lib PME-Lib QM/BM FM L2 Switch FLib HW LAN-WAN LAN-LAN L2 Control External Use 38

40 Summary The T1040 include an integrated gigabit Ethernet switch that supports wire-speed switching for all packet sizes The enterprise class switch supports features like VLAN, QoS, STP, IGMP etc Variety of Switch SW solutions to suit different customers Switch API Vitesse Stack External Use 39

41 Introducing The QorIQ LS2 Family Breakthrough, software-defined approach to advance the world s new virtualized networks New, high-performance architecture built with ease-of-use in mind Groundbreaking, flexible architecture that abstracts hardware complexity and enables customers to focus their resources on innovation at the application level Optimized for software-defined networking applications Balanced integration of CPU performance with network I/O and C-programmable datapath acceleration that is right-sized (power/performance/cost) to deliver advanced SoC technology for the SDN era Extending the industry s broadest portfolio of 64-bit multicore SoCs Built on the ARM Cortex -A57 architecture with integrated L2 switch enabling interconnect and peripherals to provide a complete system-on-chip solution External Use 40

42 QorIQ LS2 Family Key Features SDN/NFV Switching Data Center Wireless Access Unprecedented performance and ease of use for smarter, more capable networks High performance cores with leading interconnect and memory bandwidth 8x ARM Cortex-A57 cores, 2.0GHz, 4MB L2 cache, w Neon SIMD 1MB L3 platform cache w/ecc 2x 64b DDR4 up to 2.4GT/s A high performance datapath designed with software developers in mind New datapath hardware and abstracted acceleration that is called via standard Linux objects 40 Gbps Packet processing performance with 20Gbps acceleration (crypto, Pattern Match/RegEx, Data Compression) Management complex provides all init/setup/teardown tasks Leading network I/O integration 8x1/10GbE + 8x, Sec on up to 4x 1/10GbE Integrated L2 switching capability for cost savings 4 PCIe Gen3 controllers, 1 with SR-IOV support 2 x SATA 3.0, 2 x USB 3.0 with PHY External Use 41

43 See the LS2 Family First in the Tech Lab! 4 new demos built on QorIQ LS2 processors: Performance Analysis Made Easy Leave the Packet Processing To Us Combining Ease of Use with Performance Tools for Every Step of Your Design External Use 42

44 Freescale Semiconductor, Inc. External Use