Gigabit Ethernet Switch Application Guide

Transcription

1 Gigabit Ethernet Switch Application Guide C-WARE SOFTWARE TOOLSET, VERSION 2.4 CSTAGBE-UG/D Rev 02

2 Copyright 2004 Motorola, Inc. All rights reserved. No part of this documentation may be reproduced in any form or by any means or used to make any derivative work (such as translation, transformation, or adaptation) without written permission from Motorola. Motorola reserves the right to revise this documentation and to make changes in content from time to time without obligation on the part of Motorola to provide notification of such revision or change. Motorola provides this documentation without warranty, term, or condition of any kind, either implied or expressed, including, but not limited to, the implied warranties, terms or conditions of merchantability, satisfactory quality, and fitness for a particular purpose. Motorola may make improvements or changes in the product(s) and/or the program(s) described in this documentation at any time. C-3e, C-5, C-5e, C-Port, and C-Ware are all trademarks of C-Port, a Motorola Company. Motorola and the stylized Motorola logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners.

3 CSTAGBE-UG/D Rev 02 CONTENTS About This Guide Guide Overview Using PDF Documents Guide Conventions Revision History References to CST Pathnames Related Product Documentation CHAPTER 1 Gigabit Ethernet Switch Application Guide Overview Prerequisite Reading System Configuration Application Feature Overview Feature Overview and Standards Support IEEE 802.3x PAUSE Layer 2 MAC Bridging/Bridge Flooding Q VLAN Switching Layer 3 IP Routing ICMP Support Gigabit Ethernet Autonegotiation IP Multicast Routing Fabric Port Support RMON Statistics Application Components Used Application Control and Data Flow Resource Utilization XPRC CP CPRC CPRC/SDP Interface MOTOROLA GENERAL BUSINESS INFORMATION CSTAGBE-UG/D REV 02

4 4 CONTENTS RxSDP Initial RxSDP Processing RxByte Processor TxSDP TxByte Processor Additional TxSDP Processing Parallel to Serial Conversion TLU BMU QMU FP Fabric Port Activation Restriction on This Application s Fabric Port Functionality on NP Hardware Host Processor Interaction Offline Table Building Supplied Application Files XPRC CPRC SDP FDP Host Binaries Simulation Files Design Implementation and Optimization Gigabit Ethernet CSTAGBE-UG/D REV 02 MOTOROLA GENERAL BUSINESS INFORMATION

5 CSTAGBE-UG/D Rev 02 ABOUT THIS GUIDE Guide Overview This document describes the design and features of the C-Ware Gigabit Ethernet Switch application (application identifier gbeswitch). This guide is intended for users of the C-Ware Software Toolset (CST) who want to build any application provided in the CST or who want to develop new C-Ware-based applications targeted to a C-Port network processor device. This guide contains one chapter that covers the following major topics: Overview System Configuration Application Feature Overview Application Control and Data Flow Resource Utilization Supplied Application Files Design Implementation and Optimization MOTOROLA GENERAL BUSINESS INFORMATION CSTAGBE-UG/D REV 02

6 6 ABOUT THIS GUIDE Using PDF Documents Electronic documents are provided as PDF files. Open and view them using the Adobe Acrobat Reader application, version 3.0 or later. If necessary, download the Acrobat Reader from the Adobe Systems, Inc. web site: PDF files offer several ways for moving among the document s pages, as follows: To move quickly from section to section within the document, use the Acrobat bookmarks that appear on the left side of the Acrobat Reader window. The bookmarks provide an expandable outline view of the document s contents. To display the document s Acrobat bookmarks, press the Display both bookmarks and page button on the Acrobat Reader tool bar. To move to the referenced page of an entry in the document s Contents or Index, click on the entry itself, each of which is hot linked. To follow a cross-reference to a heading, figure, or table, click the blue text. To move to the beginning or end of the document, to move page by page within the document, or to navigate among the pages you displayed by clicking on hyperlinks, use the Acrobat Reader navigation buttons shown in this figure: Beginning of document End of document Previous or next hyperlink Previous page Next page CSTAGBE-UG/D REV 02 MOTOROLA GENERAL BUSINESS INFORMATION

7 Guide Conventions 7 Table 1 summarizes how to navigate within an electronic document. Table 1 Navigating Within a PDF Document TO NAVIGATE THIS WAY Move from section to section within the document. Move to an entry in the document s Contents or Index. Follow a cross-reference (highlighted in blue text). Move page by page. Move to the beginning or end of the document. Move backward or forward among a series of hyperlinks you have selected. CLICK THIS A bookmark on the left side of the Acrobat Reader window The entry itself The cross-reference text The appropriate Acrobat Reader navigation buttons The appropriate Acrobat Reader navigation buttons The appropriate Acrobat Reader navigation buttons Guide Conventions The following visual elements are used throughout this guide, where applicable: This icon and text designates information of special note. Warning: This icon and text indicate a potentially dangerous procedure. Instructions contained in the warnings must be followed. Warning: This icon and text indicate a procedure where the reader must take precautions regarding laser light. This icon and text indicate the possibility of electrostatic discharge (ESD) in a procedure that requires the reader to take the proper ESD precautions. MOTOROLA GENERAL BUSINESS INFORMATION CSTAGBE-UG/D REV 02

8 8 ABOUT THIS GUIDE Revision History Table 2 provides details about changes made for each revision of this guide. Table 2 Build System Conventions Guide Revision History REVISION DATE DATE CHANGES 02 7/2002 In Chapter 1 under the section Fabric Port Activation corrected the description of how to enable the application s use of the Fabric Port. 01 9/2001 New document. CSTAGBE-UG/D REV 02 MOTOROLA GENERAL BUSINESS INFORMATION

9 References to CST Pathnames 9 References to CST Pathnames You typically install the C-Ware Software Toolset (CST) on your development workstation in a directory path suggested by the installation procedure, such as: C:\C-Port\Cstx.y\ (on Windows 2000/XP) /usr/yourlogin/c-port/cstx.y/ (on Sun SPARC Solaris and Linux) or: /usr/cport/c-port/cstx.y/ or: /opt/c-port/cstx.y/ where x is a major version number and y is a minor (or intermediate) version number. You typically install each CST version under some directory path...\c-port\cstx.y\. However, the user can install the CST in any directory on the development workstation. The user can also install more than one CST version on the same workstation. Therefore, to refer to installed CST directories, we use pathnames that are relative to the...\c-port\cstx.y\ path, which is the root of a given CST installation. For example, the apps\gbeswitch\ directory path refers to the location of the Gigabit Ethernet Switch application that is installed as part of the CST. The full path of this directory on a Windows 2000/XP system might be C:\C-Port\Cst2.1\apps\gbeSwitch\, so this convention is convenience for shortening the pathname. Other top-level directories that are installed as part of the CST include bin\, diags\, Documentation\, services\, and so on. These directories are described in the C-Ware Software Toolset Getting Started Guide document, which is part of the CST documentation set. MOTOROLA GENERAL BUSINESS INFORMATION CSTAGBE-UG/D REV 02

10 10 ABOUT THIS GUIDE Related Product Documentation Table 3 C-Ware Application Library Documentation Set Table 3 lists the documentation for the C-Ware library of reference applications. DOCUMENT NAME PURPOSE DOCUMENT ID AAL-5 Fabric Port SAR to Gigabit Ethernet Describes the key characteristics of the gbeoc12sarfp CSTAA5F2G-UG Switch Application Guide applications. AAL-5 SAR to Gigabit Ethernet Switch Application Guide Describes the key characteristics of the gbeoc12sar application. CSTAA52G-UG FibreChannel to Gigabit Ethernet IP Gateway Application Guide Frame Relay to ATM to 10/100 Ethernet Switch Router Application Guide Describes the key characteristics of the gbefc application. Describes the key characteristics of the switchrouter application. CSTAFC2G-UG CSTAFRAE-UG Gigabit Ethernet Switch Application Guide Describes the key characteristics of the gbeswitch application. CSTAGBE-UG Multi-PHY Switch Application Guide Describes the key characteristics of the mphyswitch application. CSTAMPHYS-UG Packet Over SONET Switch Application Guide Describes the key characteristics of the posoc48sc application. CSTAPOS-UG Packet Over SONET to Ethernet Switch Describes the key characteristics of the enetoc3switch CSTAPOS2E-UG Application Guide application. Packet Over SONET to Gigabit Ethernet Switch Application Guide Voice Over IP to Voice Over ATM Media Gateway Application Guide Fabric Processor Configuration Component Guide GMII Gigabit Ethernet Autonegotiation Component Guide ICMP Support Component Guide MPC750 SBC Host Stack Support Component Guide PHY Configuration Component Guide QMU Configuration and RC Support Component Guide SONET Monitoring Component Guide Describes the key characteristics of the posgbeswitch CSTAPOS2G-UG application. Describes the key characteristics of the voiptovoatmswitch CSTAVOIP-UG application. Describes the key characteristics of the fabrics application CSTCFPC-UG component. Describes the key characteristics of the gmiiautoneg application CSTCGEAN-UG component. Describes the key characteristics of the ip application component. CSTCICMP-UG Describes the key characteristics of the stacksupport application CSTCMHSS-UG component. Describes the key characteristics of the phy application CSTCPHYC-UG component. Describes the key characteristics of the queueutils application CSTCQRCS-UG component. Describes the key characteristics of the sonet application CSTCSMC-UG component. CSTAGBE-UG/D REV 02 MOTOROLA GENERAL BUSINESS INFORMATION

11 CSTAGBE-UG/D Chapter 1 Rev 02 GIGABIT ETHERNET SWITCH APPLICATION GUIDE Overview This document is a functional and design specification for the gbeswitch application in the CST. This document goes into detail about the following topics: System Configuration Application Feature Overview Application Control and Data Flow Resource Utilization Supplied Application Files Design Implementation and Optimization Prerequisite Reading Readers of this document are assumed to have read or be familiar with the topics in the following documents in the CST: C-Ware Software Toolset Getting Started Guide How to get started with the CST. Build System Conventions Description of how the build system works. System Configuration This application runs on the following modules as a part of the C-Ware Development System (CDS): C-5 Switch Module This application runs with the following Physical Interface Module (PIM): 2 x Gigabit Ethernet (TBI) MOTOROLA GENERAL BUSINESS INFORMATION CSTAGBE-UG/D REV 02

12 12 CHAPTER 1: GIGABIT ETHERNET SWITCH APPLICATION GUIDE Application Feature Overview The gbeswitch application in the CST is a 2-port Gigabit Ethernet layer 2/3 switch. The Gigabit Ethernet interfaces support TBI (ten-bit interface) encoding. The gbeswitch application supports what is referred to as dual-cluster Gigabit Ethernet. That is, 2 clusters on the C-5 NP are used to support a single Gigabit Ethernet interface. This configuration is useful for achieving high performance switching output and allows each cluster to concentrate on processing data in a single direction, allowing more instruction memory and cycles for other tasks. One of the drawbacks of this solution is that the port density of the application is one-half that if a single-cluster solution was used. Feature Overview and Standards Support This application supports the following features: IEEE 802.3x MAC Control PAUSE support Layer 2 Bridge Forwarding Layer 2 Bridge Flooding 802.1p priority assignment of frames 802.1Q VLAN switching Layer 3 forwarding (IP routing) ICMP Support for TTL expired, destination unreachable and redirect IP multicast routing Fabric Port support RMON statistics IEEE 802.3x PAUSE IEEE 802.3x PAUSE is a standard for Ethernet MAC devices that provides link-level flow control. That is, it provides a mechanism for an Ethernet interface (1000BaseT Gigabit in this application s case) to stop transmission of frames for a predetermined amount of time. This feature is invoked by the C-5 NP to it s link partner when it is running low on packet buffers. When this happens, the C-5 NP sends the MAC PAUSE frame to it s link partner on the given interface. CSTAGBE-UG/D REV 02 MOTOROLA GENERAL BUSINESS INFORMATION

13 Application Feature Overview 13 When one of these frames is received by the link partner, the C-5 NP stops sending data over the interface for the requested amount of time. Layer 2 MAC Bridging/Bridge Flooding 802.1D Bridging is the process of forwarding MAC frames at layer 2 of the Open Systems Interconnect (OSI) model. This forwarding process involves: 1 Receiving MAC frames from a physical interface (for this implementation, 10BASE-T, 100BASE-T (or Fast Ethernet), and 1000BASE-T (or Gigabit Ethernet). 2 Resolving the MAC Destination Address (DA) by looking it up in the Bridge Address Table (also known as the Bridge Forwarding Database) to obtain the egress port(s). 3 Resolving the MAC Source Address (SA) by looking it up in the Bridge Address Table to detect a new SA that needs to be learned, or a SA move on a bridge port. 4 Transmitting the same MAC frame out one or more egress ports based on the MAC DA lookup result. Each MAC frame that is to be bridged through the system must have its address resolved by looking it up in the Bridge Address Table. The Bridge Address Table maps a MAC address to a bridge port. A successful lookup provides the forwarding application with the egress port to use for forwarding the MAC frame. An unsuccessful lookup typically results in the transmission of the MAC frame out every port in the system, other than the one on which it arrived. (This process is sometimes referred to as bridge flooding.) Some more complex bridging algorithms call for forwarding the MAC frame out a subset of all the ports in the system. Every MAC frame that enters the system has associated with it a MAC Source Address (SA). When a packet arrives, its MAC SA is looked up in the Bridge Address Table and, if not found or if the associated port has changed, the MAC address is added to or replaced in the Bridge Address Table. This process is referred to as Bridge Address learning or an address move. If an address is not heard from for five minutes, it is removed from the Bridge Address Table. The 802.1D topology protocol is called the Spanning Tree Protocol (STP). STP is used to detect loops in a layer 2 topology, based on receiving update messages with the same identifier on two different ports. If this condition occurs, the algorithm selects one of the bridge ports to disable so that no loop in the network exists and there is exactly one path from every source to every destination at layer 2. MOTOROLA GENERAL BUSINESS INFORMATION CSTAGBE-UG/D REV 02

14 14 CHAPTER 1: GIGABIT ETHERNET SWITCH APPLICATION GUIDE Note that STP is not a part of this application. For a complete description of the 802.1D protocol and its features, see IEEE 802.1D/D17 (ISO/IEC Final DIS ) Information technology Telecommunications and information exchange between systems Local and metropolitan area networks Common specifications Part 3: Media Access Control (MAC) Bridges Q VLAN Switching 802.1Q VLAN tag based switching is the process of forwarding MAC frames based upon the VLAN associated with an incoming frame determined from either an 802.1Q VLAN header or the VLAN assigned to the port on which the frame was received. This forwarding process involves: 1 Receiving a MAC frame on a physical interface. 2 Associating a VLAN identifier (VID) with the incoming frame. 3 Determining the forwarding ID (FID) of the VLAN associated with the incoming frame. 4 Determining whether or not the frame should be tagged at the egress port with its associated VID. 5 Resolving the MAC Destination Address (DA) by looking it up in the Bridge Address Table using the determined FID value to obtain the egress ports. 6 Resolving the MAC Source Address (SA) by looking it up in the Bridge Address Table using the determined FID value to detect whether or not the SA needs to be learned or has moved to a new bridge port. 7 Transmitting the frame out the egress ports in the appropriate format (tagged or untagged) based upon the FID database entry and the MAC DA lookup result. Each MAC frame received by the Ethernet Switch application is assigned a VLAN identifier (VID). This VID is determined by the value specified in an 802.1Q header when present or by inheriting the value associated with the receiving interface. An interface is configured with a port VLAN identifier (PVID) used for mapping a VLAN to a frame without a VLAN header. Once a VLAN is associated with an incoming frame, the corresponding Forwarding Information Database (FID) entry associated with that VLAN is looked up. A VLAN can belong to only one FID, but a FID can map to several VLANs. Any subsequent lookup on the MAC DA or SA in the Bridge Address Table includes the FID as part of the key uniquely identifying a MAC Address in the table. Once a forwarding decision is made, the egress ports will transmit the frame either tagged with the appropriate 802.1Q header CSTAGBE-UG/D REV 02 MOTOROLA GENERAL BUSINESS INFORMATION

15 Application Feature Overview 15 or untagged according to the specifications in the FID entry originally retrieved by the ingress port. For a more complete description of the 802.1Q forwarding specification see IEEE P802.1Q/D11 Draft Standard for Local and Metropolitan Area Networks: Virtual Bridged Local Area Networks. Layer 3 IP Routing IP routing is the process of forwarding IP frames at layer 3 based upon the IP Destination Address (IP DA). An advantage of IP routing is that it can be used between dissimilar network media types (for example, between Ethernet and OC-3 SONET). This application only covers Ethernet networks with a single datalink encapsulation (Ethernet DIX/Type II). ICMP Support The gbeswitch application supports three types of ICMP messages: ICMP Time Exceeded ICMP No Route ICMP Destination Unreachable The reason that these three message types are supported in this application is that they are based on events that happen in the data path, as opposed to the control path. The ICMP support in this application is implemented by the C-Ware ip application component, which is documented separately. See the document on this topic by looking in the apps/components/ip/doc/ directory in the CST. Gigabit Ethernet Autonegotiation The Gigabit Ethernet application implements TBI physical layer encoding. Gigabit Ethernet TBI PHYs do not implement Gigabit Ethernet autonegotiation the same way that GMII PHYs do. The result is that systems that use TBI PHYs must implement the Gigabit Ethernet autonegotiation protocol outside of the PHY, because all of the data is in-band. (When GMII is available in PHYs, these PHYs will implement this state machine themselves, much like RMII in 10/100 PHYs). The Gigabit Ethernet application implements this state machine in the RxSDP and CPRC software programs. The state machine is driven from 16bit Gigabit Ethernet pages that are data segments on the wire. By virtue of the contents of these pages, the states of the machine can be determined. MOTOROLA GENERAL BUSINESS INFORMATION CSTAGBE-UG/D REV 02

16 16 CHAPTER 1: GIGABIT ETHERNET SWITCH APPLICATION GUIDE The autonegotiation process configures the PHY s duplex mode, flow control mode, and line speed. IP Multicast Routing The gbeswitch application also supports IPv4 Multicast Routing. This feature is supported by doing lookups on the IP SA + IP DA if the frame is a MAC layer muilticast and an IPv4 datagram. The entries in the multicast table are keyed off of the IP SA + IP DA and they return a mask of ports that the IP multicast frame should be sent out. Fabric Port Support The gbeswitch application supports a back to back Fabric Port implementation. This functionality is provided by the C-Ware fabrics application component. See the apps/components/fabrics/doc/ directory for documentation on this component. RMON Statistics The gbeswitch application supports the etherstats group of statistics for RMON-1. These statistics are accumulated by the RxByte and CPRC for a given CP and stored in CP DMEM where they can accessed from either the XP or host processor Application Components Used The CST provides a number of application components that are used across applications. This application uses the following application components provided in the CST: queueutils fabrics (back to back) ip phy (TBI bit micro-code) stacksupport (with host-enabled version) See the documentation in the apps/components/<componentname>/doc/ directory for the documentation on the software components that this application uses. CSTAGBE-UG/D REV 02 MOTOROLA GENERAL BUSINESS INFORMATION

17 Application Control and Data Flow 17 Application Control and Data Flow This is the control and data flow for this application. Figure 1 Control and data flow Queue Storage (SRAM) Fabric Table Storage and Statistics (SRAM) External Host CPU SDRAM PCI Processor Boundary Cluster QMU ➍ ➍ ➍ ➌ ➌ FP ➎ ➍ TLU Ring Bus Global Bus ➎ Payload Bus ➊ ➌➌ ➎ XP ➋ ➎ ➋ BMU ➊ CP0 CP1 CP2 CP3 CP4 CP5 CP6 CP7 CP8 CP9 CP10 CP11 CP12 CP13 CP14 CP15 Gigabit RX Gigabit TX Gigabit RX Gigabit TX C-5 NP ➊ This is the port 1 -> 2 data path. PHY ➋ This is the port 2 -> 1 data path. ➌ This is the MAC, VLAN, IP (unicast or multicast ) address lookup and response for Gigabit Ethernet RX. ➍ This is the enqueue and dequeue to/from the QMU. ➎ This is the host packet receive and transmit control path. MOTOROLA GENERAL BUSINESS INFORMATION CSTAGBE-UG/D REV 02

18 18 CHAPTER 1: GIGABIT ETHERNET SWITCH APPLICATION GUIDE Resource Utilization To support this application s features, the processors within the C-5 NP are performing a variety of tasks. The sections below enumerate what functionality is taking place and how the different processors are being used. XPRC The Executive Processor RISC core (XPRC) is a general-purpose processor that provides the management, control, and exception processing functions. The XP controls C-5 NP boot up, configuration, and initialization of all of the system components. The XP application is split into two sections. The first section performs service initialization, configures system resources, and loads the Channel Processors. The second section completes any necessary initialization and starts the Channel Processors before entering the main loop. This partitioning scheme uses the available IMEM resource to its fullest. The main application code is overlaid on the initialization section, reducing the amount of IMEM used during run-time. For more information see the C-Ware Application Building Guide. In the case of the Ethernet application, the XP is responsible for the addition of table entries to the TLU via the Ring Bus, passing of control and data information to and from the CPRC, and running of latency-sensitive control protocols that are not in the data path (such as Ethernet autonegotiation). This application is designed to work with an external TCP/IP stack, so no bridge aging or learning functions run on the XPRC. CP The C-5 NP s 16 Channel Processors (CPs) are the components most closely associated with processing data from a physical interface. Each CP performs several functions that aid in the processing of Ethernet frames. CPRC The Channel Processor RISC Core (CPRC) is a general-purpose processor core that supports a MIPS TM 1-like instruction subset. The CPRC program uses the C-Ware Communication Programming Interface (CPI) that provides a high-level abstraction of the C-5 NP s function calls for ease-of-programming the C-5 NP. The CPRC program is used for the following functions in the Ethernet Switch application: Processing the lookup results from the TLU. Constructing a descriptor for forwarding to the QMU. CSTAGBE-UG/D REV 02 MOTOROLA GENERAL BUSINESS INFORMATION

19 Resource Utilization 19 Characterizing the frame and making forwarding decisions based on the TLU lookup results and frame parsing. Draining the eight priority queues per CPRC. Initiating receive program on the RxSDP Initiating transmit program on TxSDP CPRC/SDP Interface The CPRC interfaces to the TxByte processor via a set of special purpose registers called Merge Space registers. The Merge Space registers are used to communicate information such as whether outgoing frames should be routed, bridged, or tagged with a VLAN identifier. The CPRC and the RxByte processor share a set of registers to do table lookup processing over the Ring Bus. Lookup requests are referenced using a request tag, that refers to one of four Ring Bus registers assigned for initiating requests. Slots 0 and 1 are accessible to the SDP, while slots 0 through 3 are accessible to the CPRC. Each slot contains a 4Byte control register and two 4Byte data registers. These data registers contain information to be delivered to the TLU such as the type of lookup being requested and the key information to use in determining a match. Depending on the type of lookup being issued and the length of the key, the number of slots needed for any single lookup can be 1, 2, or 4. The following request tags are used by the SDP in the Ethernet application: Tag 0: MAC DA lookup Uses both slots 0 and 1 to support a 96 bit key (MAC + FID). Tag 0: MAC SA lookup Uses both slots 0 and 1 to support a 96 bit key (MAC + FID). Tag 0: VLAN to FID lookup Uses only slot 0. Tag 1: IP DA lookup Uses only slot 1. Tag 0: IP Multicast Uses both slots 0 and 1 to support a 96 bit key (IP SA +IP DA). The following request tags are used by the CPRC in the Ethernet application: Tag 2: MAC DA lookup Uses both slots 2 and 3 to launch a lookup on the MAC DA plus the FID result obtained from a VLAN to FID lookup. Tag 2: MAC SA lookup Uses both slots 2 and 3 to launch a lookup on the MAC SA plus the FID result obtained from a VLAN to FID lookup. MOTOROLA GENERAL BUSINESS INFORMATION CSTAGBE-UG/D REV 02

20 20 CHAPTER 1: GIGABIT ETHERNET SWITCH APPLICATION GUIDE Note that because the same request tag is used for multiple lookups, the application needs to check the AVAIL bit in the Ring Bus transmit control register to make sure the register is available for use by the program. The CPRC also has access to a set of Ring Bus receive control registers for checking lookup responses referred to as response tags. These response tags map to one of eight response slots available to CPRC, each of which contains a 4Byte control and 8Byte data portion. One response slot is therefore necessary for every eight bytes of data being returned from a lookup. When a lookup is initiated by either the SDP or the CPRC, the slot on which the response should land is indicated in the request. The status of the response is then checked by the CPRC using the appropriate response tag and the data returned is read upon receipt of a successful lookup indication. The following response slots are assigned in the Ethernet application: Response Tag 0: MAC DA response Uses slots 0 and 1 to support the return of 16 bytes of data. Response Tag 2: MAC SA response Uses slots 2 and 3 to support the return of 16 bytes of data. Response Tag 4: IP DA response Uses slots 4 and 5 to support the return of 16 bytes of data. Response Tag 4: IP Multicast response Uses slots 4 and 5 to support the return of 16 bytes of data. Response Tag 6: VLAN VID to FID response Uses only slot 6 to support the return of eight bytes of data. Note that response Tag 4 is used for two different types of lookups. (This works because an IP DA response and IP Multicast response are never launched for the same frame.) Also, the response tags are spread out to accommodate the amount of data being returned and avoid corrupting data waiting to be read from the other lookups. CSTAGBE-UG/D REV 02 MOTOROLA GENERAL BUSINESS INFORMATION

21 Resource Utilization 21 RxSDP The Receive Serial Data Processor (RxSDP) performs bit- and byte-level interpretation and parsing of the incoming Ethernet stream. In addition to providing general receive processing, the RxSDP supports the following Ethernet-specific Switching functions: Frame delineation Preamble detection Control character detection Header parsing Lookup initiation Header validation CRC validation CPRC forwarding path determination Initial RxSDP Processing The initial processing of the incoming data stream involves: Serial to Parallel Conversion The 8bit data is converted from serial to parallel format for processing by the RxSDP. In the case of Gigabit Ethernet the incoming data stream is first converted from 10bit to 8bit format before being converted from serial to parallel format. Preamble Detection The RxSDP is programmed to detect the Ethernet preamble and start of frame (SOF) delimiter. After the bit pattern is detected, these bytes are stripped off of the frame before additional RxSDP functional parsing of the Ethernet frame. PCS Receive State Machine The RxSDP implements the Physical Coding Sub-layer (PCS) receive state machine according to 802.3z, section 36. That is, the RxSDP detects the start of frame, end of frame, and idle Gigabit Ethernet control characters. It also validates that the control characters are received at the appropriate time (such that no idle characters are received between the start and end of frame). Frame Length Validation The RxSDP validates that incoming frames are in the range of 64 to 1518 bytes (to 1522 bytes if 802.1Q). MOTOROLA GENERAL BUSINESS INFORMATION CSTAGBE-UG/D REV 02

22 22 CHAPTER 1: GIGABIT ETHERNET SWITCH APPLICATION GUIDE RxByte Processor The SDP Receive Byte (RxByte) Processor is a serial processor within the RxSDP. It is responsible for the byte-level processing of Ethernet data from the physical interface. Specific physical layer and MAC functions include: Header Detection The RxByte processor detects the start of the layer 2 MAC and layer 3 IP header of incoming Ethernet frames. Header Validation The RxByte processor validates the layer 3 IP headers of the incoming Ethernet frame. All of the validation that is required by an IP Router is done by the RxByte processor with the exception of validating the IP DA or IP SA. Validation on the IP DA or IP SA is done by virtue of not installing illegal IP Addresses into the Table Lookup Unit (TLU), which results in failed lookup results from the TLU. Initiation of MAC DA, MAC SA, IP DA Lookups, IP Multicast, and VLAN to FID The RxByte processor launches lookups for the MAC DA, MAC SA, IP DA, IP Multicast, and VLAN to FID to the TLU for evaluation by the CPRC while the packet is being received, so that the CPRC does not require these lookups. Detection of 802.1p Priority The RxByte processor program detects if there are any 802.1p priority bits present and communicates them to the CPRC via extract space so that they can be used to map incoming Ethernet frames to a priority for transmit. Detection of Control Frames The RxByte processor detects control frames such as ARP, 802.3x flow control frames, or 802.1z autonegotiation pages so that the CPRC can process them accordingly. Validation of the Ethernet CRC The RxByte processor s CRC-32 engine calculates an ongoing Ethernet CRC for the received frame and verifies that the CRC on the frame is correct. Determination of CPRC Forwarding Path The RxByte processor provides information to the CPRC informing it of what forwarding path (such as unicast, flooded, routed, or IP multicast) to use for the frame. Error and Status Reporting The RxByte processor program reports errors and frame status to the CPRC via extraction space. This includes any physical layer error, CRC, frame length, header validation errors, and so on. CSTAGBE-UG/D REV 02 MOTOROLA GENERAL BUSINESS INFORMATION

23 Resource Utilization 23 The RxByte processor passes information about incoming packets through extract space. Table 4 describes how extract space is used between RxByte and the CPRC. Table 4 Extract Space Field Descriptions FIELD NAME SIZE OFFSET DESCRIPTION hdrstatus 1 0 Status code of MAC and IP header processing (valid when header parsing is complete) framestatus 1 1 Status code of MAC frame processing (valid when entire frame has been parsed) rxpath 1 2 Indicates which RX processing CPRC should apply protocol 1 3 Derived from MAC protocol (IP, Pause, ARP, etc.) mactype 2 4 Not used in this application framelen 2 6 Length of frame in bytes unused2 4 8 Reserved priority p priority field badframecount 1 13 Count of drops while SDP waits for scope vlanid 2 14 VLAN ID macdahi 4 16 Four MSBs of MAC destination address macdalo 2 20 Two LSBs of MAC destination address unused Reserved macsahi 4 24 Four MSBs of MAC source address macsalo 2 28 Two LSBs of MAC source address unused Reserved ipcontrol 4 32 Not used in this application version 1 36 IP version (should be 4) IHL 1 37 IP header length TOS 1 38 IP Type of Service field pad 1 39 Reserved length 2 40 IP total length indentifier 2 42 IP identifier field flags 1 44 IP flags pad Reserved MOTOROLA GENERAL BUSINESS INFORMATION CSTAGBE-UG/D REV 02

24 24 CHAPTER 1: GIGABIT ETHERNET SWITCH APPLICATION GUIDE FIELD NAME SIZE OFFSET DESCRIPTION fragoffset 2 46 IP fragmentation offset TTL 1 48 IP time to live protocol 1 49 IP protocol field checksum 2 50 IP checksum sourceaddr 4 52 IP source address destaddr 4 56 IP destination address pad Reserved TxSDP The Transmit Serial Data Processor (TxSDP) performs bit- and byte-level transmission of the outgoing Ethernet stream. In addition to providing general transmit processing, the TxSDP specifically supports the following Ethernet Switching functions: Modification of the IP header Decrementing of the IP Time-to-Live (TTL) Recalculation of the IP Header Checksum Regeneration of the Ethernet CRC Regeneration of 802.1Q VLAN headers for tagged frames Ethernet MAC functions. TxByte Processor The SDP Transmit Byte (TxByte) Processor is a serial processor within the TxSDP. It is responsible for the following functionality in the Ethernet application: Transmits bytes The TxByte processor reads bytes from CP DMEM and then loads the bytes into the large FIFO in order to stage transmission to the TxBit processor. Frame modification In the case of IP routing, the TxByte processor modifies the datalink header according to the MAC DA and MAC SA in Merge Space (populated by the CPRC), decrements the TTL, and updates the IP Checksum. In the case of 802.1Q VLAN switching, the TxByte processor adds the 802.1Q header and VLAN tag when appropriate. CSTAGBE-UG/D REV 02 MOTOROLA GENERAL BUSINESS INFORMATION

25 Resource Utilization 25 CRC generation The Ethernet CRC is recalculated and appended to the end of the Ethernet frame. Ethernet padding If the length of the Ethernet frame to be transmitted is less than 60 bytes (without the CRC), the TxByte processor pads the frame out to 60 bytes to meet minimum Ethernet packet size requirements. CPRC coordination For transmit error handling (such as collision detection when operating in half-duplex mode), the RxByte processor coordinates and communicates status information with the CPRC via both Merge Space and Control Space. The TxByte processor receives information about outgoing packets through merge space. Table 5 describes how merge space is used between TxByte and the CPRC. Table 5 GbE Merge Space Field Descriptions FIELD NAME SIZE OFFSET DESCRIPTION pausetime 2 0 Amount of time to silence port txalgorithm 1 2 Indicates how SDP should treat outgoing packet datalinkheadersize 1 3 Not used in this application macdahi 4 4 Four MSBs of MAC destination address vlantag 2 8 Outgoing VLAN tag to apply to frame macdalo 2 10 Two LSBs of MAC destination address macsahi 4 12 Four MSBs of MAC source address unused Reserved macsalo 2 18 Two LSBs of MAC source address Additional TxSDP Processing In addition to the processing performed by the TxByte Processor, the TxSDP performs the following Ethernet functions to support bit-level transmit processing of an Ethernet frame: IFG generation The TxBit processor generates the correct Inter-Frame Gap (IFG) according to the speed of the physical interface. Preamble generation The TxBit processor generates both the Ethernet preamble and start-of-frame delimiter for the Ethernet frame. MOTOROLA GENERAL BUSINESS INFORMATION CSTAGBE-UG/D REV 02

26 26 CHAPTER 1: GIGABIT ETHERNET SWITCH APPLICATION GUIDE PCS transmit state machine The TxBit processor implements the PCS transmit state machine according to 802.3z, section 36 (that is, insert the start-of-frame, end-of-frame, and idle characters into the Gigabit Ethernet data stream). Parallel to Serial Conversion The 8-bit data is converted from parallel to serial format for transmission by the TxSDP. In the case of Gigabit Ethernet the outgoing data stream is also converted from 8bit to 10bit format before being transmitted to the PHY. TLU The Table Lookup Unit (TLU) provides lookup table management. The TLU stores its table information in external SRAM that is managed by the TLU s SRAM controller. The Ethernet Switch application creates four different tables in the TLU for data path forwarding support: A Bridge Address Table for layer 2 forwarding An IP Routing Table for layer 3 IP forwarding An IP Multicast Forwarding Cache Table for forwarding IP DA Multicast addresses A VLAN To FID table for associating VLANs to a FID These tables are implemented by the following types of TLU tables: Bridge Address Table: Hash/Trie/Key IP Routing Table: VpTrie/Data (C-5 NP) or LPM (C-5e NP) IP Multicast Table: Hash/Trie/Key VID table: Data table The Bridge Address Table is used for resolving layer 2 MAC addresses into egress bridge ports for layer 2 forwarding. This is implemented in the TLU as a 96bit keyed hash table that uses a Trie table for collision resolution and a key table to store the key and associated data. The key is a combination of the 2Byte FID value associated with the incoming frame and the 48bit MAC address, leaving 32 bits in the key unused and padded to zero. CSTAGBE-UG/D REV 02 MOTOROLA GENERAL BUSINESS INFORMATION

27 Resource Utilization 27 If the incoming frame is received without an 802.1Q VLAN tag, then the lookup is performed using a default FID value programmed into the SDP control registers. Otherwise, the SDP launches a lookup on the VID in the 802.1Q header to determine the FID, which the CPRC will use to launch the Bridge Address Table lookups on the MAC DA and SA. The data returned in each lookup includes the port on which the address is located, aging information, and other control information as shown in Table 6. Table 6 Bridge Table Field Descriptions FIELD NAME SIZE OFFSET DESCRIPTION queueid 1 0 Queue ID of egress interface fabricid 1 1 Fabric ID of remote line card flags 1 2 Describes the type of bridge entry agecount 1 3 Age of the entry tableindex 4 4 Index into key table portion of the bridge table. This is used for faster updates. pad 8 8 Reserved The forwarding path uses this information to determine the egress port based on the MAC DA lookup, and whether or not to update aging information or perform an add or move operation for the source address based on the MAC SA lookup. The IP Routing Table is used for resolving layer 3 IP addresses into egress Ethernet interfaces. It also provides a new MAC DA for re-encapsulation on the egress Ethernet interface. The IP Routing table is implemented in the TLU as a 16bit Indexed Pointers table that resolves the first 16 bits of the IP address, and a VP Trie table that resolves the last 16 bits of the IP Address. The associated data of the entries in the IP routing table is kept in a data table in the TLU. The data stored in each entry is shown in Table 7. Table 7 IP Forwarding Table Field Descriptions FIELD NAME SIZE OFFSET DESCRIPTION maskbits 1 0 Number of significant bits in the key queueid 1 1 Queue ID of the egress interface type 1 2 Indicates type of IP route (local, remote, etc.) ifindex 1 3 Egress interface number appdata1 4 4 Application specific data. In this application the data would be the MAC destination address. MOTOROLA GENERAL BUSINESS INFORMATION CSTAGBE-UG/D REV 02

28 28 CHAPTER 1: GIGABIT ETHERNET SWITCH APPLICATION GUIDE FIELD NAME SIZE OFFSET DESCRIPTION appdata2 4 8 Same as above fabricid 1 12 Fabric ID of the remote line card pad Reserved pad Reserved The IP Multicast Forwarding Cache Table is used for resolving IP Multicast destinations into a port map of Ethernet egress interfaces. It is implemented using a 96bit keyed hash table that uses a Trie table for collision resolution and a Key table to store the key and associated data. The key is a combination of the IP SA and IP DA (32 bits each), leaving 32 bits in the key unused and padded to zero. The associated data contains a forwarding vector representing the egress interfaces as shown in Table 8. Table 8 IP Multicast Forwarding Table Field Descriptions FIELD NAME SIZE OFFSET DESCRIPTION fwdvector 4 0 Bitmap of queues to which multicasts are sent vectorcount 4 4 Number of queues to which mulicasts are sent fabricvector 2 8 Bitmap of fabric IDs to which multicasts are sent egressvnid 2 10 Egress queue pad Reserved The VLAN to FID table is used for determining the FID associated with a VLAN. It is a simple Data table with 4096 entries indexed by the VLAN identifier associated with a frame. The data portion contains the FID value for that VLAN and control information indicating whether or not the frame should be tagged with its associated VLAN at the egress ports. The entry format is described in Table 9. Table 9 VLAN to FID Table Field Descriptions FIELD NAME SIZE OFFSET DESCRIPTION fidvalue 2 0 FID value mapped to the looked up VLAN ID untaggedset 2 2 Bitmask indicating which members of the VLAN do not send a VLAN tag memberset 2 4 Bitmask of CPs which are members of the VLAN membercount 2 6 Number of CPs in the VLAN CSTAGBE-UG/D REV 02 MOTOROLA GENERAL BUSINESS INFORMATION

29 Resource Utilization 29 BMU The Buffer Management Unit (BMU) manages the C-5 NP s data buffers for payload management. The Ethernet application allocates one buffer pool per CP. The size of the buffers in each buffer pool is 2048 bytes (the buffer sizes must be a power of 2). The CPRC sets up packet transactions to SDRAM for packet payload that fire under hardware control. The BMU ensures that packet data that is copied into CP DMEM via the RxByte processor is correctly moved via a DMA transaction to the correct SDRAM buffer and offset for each port. Under certain circumstances, the RxByte processor writes the MAC header to the SDRAM buffer along with the frame payload. Specifically, the RxByte stores the MAC header in SDRAM if the frame has a destination address of one of the following forms: Unicast MAC destination address not matching the local MAC address Multicast MAC destination address that is not an IP multicast MAC address (which are of the form E-xx-xx-xx) Broadcast MAC destination address The RxByte processor will not write the MAC header to the SDRAM buffer if the MAC destination address is of one of the following forms: Unicast MAC destination address matching the local MAC address Multicast MAC destination address matching the IP multicast group (which are of the form E-xx-xx-xx) QMU The Queue Management Unit (QMU) manages off-chip descriptor queues that are used to provide QoS support to the gbeswitch application. The descriptors are stored in external SRAM. The application allocates eight queues per Ethernet interface. Each of these queues is populated by the 802.1p priority in the 802.1Q VLAN header. Each queue is drained on each port in strict priority order according to the CPRC transmit program. MOTOROLA GENERAL BUSINESS INFORMATION CSTAGBE-UG/D REV 02

30 30 CHAPTER 1: GIGABIT ETHERNET SWITCH APPLICATION GUIDE The initialization descriptor which is sent from the XP to the CPRC is described in Table 10. Table 11 shows the format of the descriptor used to forward PDUs from one port to another. Table 10 Initialization Descriptor FIELD NAME SIZE OFFSET DESCRIPTION fabricenabled 1 0 Boolean indicates whether or not the fabric port is being used. fabricid 1 1 ID of this line card on the switching fabric bridgekeytblid 1 2 Table ID of the key portion of the bridge table macbcastcount 1 3 Number of CPs to which broadcasts are sent macbcastvector 2 4 Bitmap of CPs to which broadcasts are sent bridgecontrolqueue 2 6 Queue number to which control messages are sent macinternaladdresshi4 4 8 Four MSBs of port s MAC address macinternaladdresslo Two LSBs of port s MAC address bridgetblid 1 14 Table ID of bridge table mplstableid 1 15 Not used in this application Table 11 Forwarding Descriptor FIELD NAME SIZE OFFSET DESCRIPTION bufhandle 4 0 Buffer handle of PDU being forwarded length 2 4 Length of PDU VNID_client 2 6 Upper 12 MSBs contain VNID indicating egress queue. Lower 4 MSBs contain the transmit client: IP, MAC Pause, ICMP, etc. appdata1 4 8 Application specific data. In this application the data would be the MAC destination address. appdata Same as above appdata Same as above CSTAGBE-UG/D REV 02 MOTOROLA GENERAL BUSINESS INFORMATION

31 Resource Utilization 31 FP This application configures the FP to operate in back to back mode for connection to a another C-5 NPthrough a switch fabric. The Fabric Processor (FP) is used for off-chip data forwarding. The gbeswitch application uses the FP to forward descriptors and data to another C-5 NP. When frames are forwarded over the fabric, the descriptor sent to the Fabric Port must have certain information specified in bit locations expected by the Fabric Control Engine (FCE). This information allows the Fabric Port to know where in buffer memory the frame is stored, the length of the frame, and the target queue to which the frame should be sent at the destination C-5 NP. The descriptor s data format is: Bit Position 31 0 Field Name Buffer Handle Bit Position Field Name Frame Length Virtual Network ID (VNID) App. Data Bit Position Field Name Application Data Bit Position Field Name Application Data FIELD NAME BIT POSITION DESCRIPTION Buffer Handle 31:0 This is the buffer handle assigned to the PDU. The quantity includes such information as the pool ID, BTAG, and multicast flag. Application Data 32:35 These bits are available to the application for use in its own internal forwarding logic. Frame Length 63:48 Indicates the length of the frame. Virtual Network ID 47:36 This field is used to inform the Fabric Port on the target C-5 NP to which queue(s) this descriptor ought to be sent. If the frame is a unicast (multicast bit in the Buffer Handle is not set), then the VNID is interpreted as the egress queue and the descriptor is sent directly to it. If the frame is a multicast, the Fabric Port does a lookup on the VNID, which returns the target queue(s) to which the descriptor is subsequently sent. Application Data 127:48 These bits are available to the application for use in its own internal forwarding logic. MOTOROLA GENERAL BUSINESS INFORMATION CSTAGBE-UG/D REV 02