Freescale Semiconductor, Inc...

Transcription

1 Packet Over SONET Switch Application Guide C-WARE SOFTWARE TOOLSET, VERSION 2.4 CSTAPOS-UG/D Rev 00

2 Copyright 2004 Motorola, Inc. All rights reserved. No part of this documentation may be reproduced in any form or by any means or used to make any derivative work (such as translation, transformation, or adaptation) without written permission from Motorola. Motorola reserves the right to revise this documentation and to make changes in content from time to time without obligation on the part of Motorola to provide notification of such revision or change. Motorola provides this documentation without warranty, term, or condition of any kind, either implied or expressed, including, but not limited to, the implied warranties, terms or conditions of merchantability, satisfactory quality, and fitness for a particular purpose. Motorola may make improvements or changes in the product(s) and/or the program(s) described in this documentation at any time. C-3e, C-5, C-5e, C-Port, and C-Ware are all trademarks of C-Port, a Motorola Company. Motorola and the stylized Motorola logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners.

3 CONTENTS CSTAPOS-UG/D Rev 00 CHAPTER 1 About This Guide Guide Overview Using PDF Documents Guide Conventions Revision History References to CST Pathnames Related Product Documentation Packet Over SONET Switch Application Guide Overview Prerequisite Reading System Configuration Application Feature Overview Feature Overview and Standards Support Layer 3 IP Routing Way Aggregation ICMP Support Instrumentation for Performance Analysis Application Components Used Application Control and Data Flow Resource Utilization XPRC CP CPRC CPRC/SDP Interface SDP RxBit RxSync RxByte TxByte TxBit MOTOROLA GENERAL BUSINESS INFORMATION CSTAPOS-UG/D REV 00

4 4 CONTENTS TLU IP Unicast Routing Table BMU QMU FP Fabric Port Activation Host Processor Interaction Supplied Application Files Application Wide XPRC CPRC SDP FDP Host Binaries Simulation files Design Issues M-5 Configuration and Usage Speculative Enqueues PHY Support Alternate Application Configurations Table Building CSTAPOS-UG/D REV 00 MOTOROLA GENERAL BUSINESS INFORMATION

5 CSTAPOS-UG/D Rev 00 ABOUT THIS GUIDE Guide Overview This document describes the design and features of the C-Ware Packet Over SONET Switch application (application identifier posoc48c). This guide is intended for users of the C-Ware Software Toolset (CST) who want to build any application provided in the CST or who want to develop new C-Ware-based applications targeted to a C-Port network processor device. This guide contains one chapter that covers the following major topics: Overview System Configuration Application Feature Overview Application Control and Data Flow Resource Utilization Supplied Application Files Design Issues Alternate Application Configurations MOTOROLA GENERAL BUSINESS INFORMATION CSTAPOS-UG/D REV 00

6 6 ABOUT THIS GUIDE Using PDF Documents Electronic documents are provided as PDF files. Open and view them using the Adobe Acrobat Reader application, version 3.0 or later. If necessary, download the Acrobat Reader from the Adobe Systems, Inc. web site: PDF files offer several ways for moving among the document s pages, as follows: To move quickly from section to section within the document, use the Acrobat bookmarks that appear on the left side of the Acrobat Reader window. The bookmarks provide an expandable outline view of the document s contents. To display the document s Acrobat bookmarks, press the Display both bookmarks and page button on the Acrobat Reader tool bar. To move to the referenced page of an entry in the document s Contents or Index, click on the entry itself, each of which is hot linked. To follow a cross-reference to a heading, figure, or table, click the blue text. To move to the beginning or end of the document, to move page by page within the document, or to navigate among the pages you displayed by clicking on hyperlinks, use the Acrobat Reader navigation buttons shown in this figure: Beginning of document Previous page End of document Next page Previous or next hyperlink CSTAPOS-UG/D REV 00 MOTOROLA GENERAL BUSINESS INFORMATION

7 Guide Conventions 7 Guide Conventions Table 1 summarizes how to navigate within an electronic document. Table 1 Navigating Within a PDF Document TO NAVIGATE THIS WAY Move from section to section within the document. Move to an entry in the document s Contents or Index. Follow a cross-reference (highlighted in blue text). Move page by page. Move to the beginning or end of the document. Move backward or forward among a series of hyperlinks you have selected. The following visual elements are used throughout this guide, where applicable: This icon and text designates information of special note. CLICK THIS A bookmark on the left side of the Acrobat Reader window The entry itself The cross-reference text The appropriate Acrobat Reader navigation buttons The appropriate Acrobat Reader navigation buttons The appropriate Acrobat Reader navigation buttons Warning: This icon and text indicate a potentially dangerous procedure. Instructions contained in the warnings must be followed. Warning: This icon and text indicate a procedure where the reader must take precautions regarding laser light. This icon and text indicate the possibility of electrostatic discharge (ESD) in a procedure that requires the reader to take the proper ESD precautions. MOTOROLA GENERAL BUSINESS INFORMATION CSTAPOS-UG/D REV 00

8 8 ABOUT THIS GUIDE Revision History Table 2 provides details about changes made for each revision of this guide. Table 2 Build System Conventions Guide Revision History REVISION DATE CST REVISION CHANGES 00 5/2003 New document. CSTAPOS-UG/D REV 00 MOTOROLA GENERAL BUSINESS INFORMATION

9 References to CST Pathnames 9 References to CST Pathnames You typically install the C-Ware Software Toolset (CST) on your development workstation in a directory path suggested by the installation procedure, such as: C:\C-Port\Cstx.y\ (on Windows 2000/XP) /usr/yourlogin/c-port/cstx.y/ (on Sun SPARC Solaris and Linux) or: /usr/cport/c-port/cstx.y/ or: /opt/c-port/cstx.y/ where x is a major version number and y is a minor (or intermediate) version number. You typically install each CST version under some directory path...\c-port\cstx.y\. However, the user can install the CST in any directory on the development workstation. The user can also install more than one CST version on the same workstation. Therefore, to refer to installed CST directories, we use pathnames that are relative to the...\c-port\cstx.y\ path, which is the root of a given CST installation. For example, the apps\gbeswitch\ directory path refers to the location of the Gigabit Ethernet Switch application that is installed as part of the CST. The full path of this directory on a Windows 2000/XP system might be C:\C-Port\Cst2.1\apps\gbeSwitch\, so this convention is convenience for shortening the pathname. Other top-level directories that are installed as part of the CST include bin\, diags\, Documentation\, services\, and so on. These directories are described in the C-Ware Software Toolset Getting Started Guide document, which is part of the CST documentation set. MOTOROLA GENERAL BUSINESS INFORMATION CSTAPOS-UG/D REV 00

10 10 ABOUT THIS GUIDE Related Product Documentation Table 3 C-Ware Application Library Documentation Set Table 3 lists the documentation for the C-Ware library of reference applications. DOCUMENT NAME PURPOSE DOCUMENT ID AAL-5 Fabric Port SAR to Gigabit Ethernet Describes the key characteristics of the gbeoc12sarfp CSTAA5F2G-UG Switch Application Guide applications. AAL-5 SAR to Gigabit Ethernet Switch Application Guide Describes the key characteristics of the gbeoc12sar application. CSTAA52G-UG FibreChannel to Gigabit Ethernet IP Gateway Application Guide Frame Relay to ATM to 10/100 Ethernet Switch Router Application Guide Describes the key characteristics of the gbefc application. Describes the key characteristics of the switchrouter application. CSTAFC2G-UG CSTAFRAE-UG Gigabit Ethernet Switch Application Guide Describes the key characteristics of the gbeswitch application. CSTAGBE-UG Multi-PHY Switch Application Guide Describes the key characteristics of the mphyswitch application. CSTAMPHYS-UG Packet Over SONET Switch Application Guide Describes the key characteristics of the posoc48sc application. CSTAPOS-UG Packet Over SONET to Ethernet Switch Describes the key characteristics of the enetoc3switch CSTAPOS2E-UG Application Guide application. Packet Over SONET to Gigabit Ethernet Switch Application Guide Voice Over IP to Voice Over ATM Media Gateway Application Guide Fabric Processor Configuration Component Guide GMII Gigabit Ethernet Autonegotiation Component Guide ICMP Support Component Guide MPC750 SBC Host Stack Support Component Guide PHY Configuration Component Guide QMU Configuration and RC Support Component Guide SONET Monitoring Component Guide Describes the key characteristics of the posgbeswitch CSTAPOS2G-UG application. Describes the key characteristics of the voiptovoatmswitch CSTAVOIP-UG application. Describes the key characteristics of the fabrics application CSTCFPC-UG component. Describes the key characteristics of the gmiiautoneg application CSTCGEAN-UG component. Describes the key characteristics of the ip application component. CSTCICMP-UG Describes the key characteristics of the stacksupport application CSTCMHSS-UG component. Describes the key characteristics of the phy application CSTCPHYC-UG component. Describes the key characteristics of the queueutils application CSTCQRCS-UG component. Describes the key characteristics of the sonet application CSTCSMC-UG component. CSTAPOS-UG/D REV 00 MOTOROLA GENERAL BUSINESS INFORMATION

11 Chapter 1 Freescale Semiconductor, Inc. CSTAPOS-UG/D Rev 00 PACKET OVER SONET SWITCH APPLICATION GUIDE Overview Prerequisite Reading This document is a functional and design specification for the C-Ware posoc48c application in the CST. This document goes into detail about the following topics: System Configuration Application Feature Overview Application Control and Data Flow Resource Utilization Supplied Application Files Design Issues Alternate Application Configurations Readers of this document are assumed to have read or be familiar with the topics in the following documents in the CST: C-Ware Software Toolset Getting Started Guide How to get started with the CST. Build System Conventions Description of how the build system works. C-5e Network Processor Version A0 Architecture Guide Description of the C-5e NP and its capabilities. Particularly sequence number processing by the QMU. M-5 Channel Adapter Version A0 Architecture Guide Description of the M-5 CA, its interfaces, and its capabilities. Specifically, the single PHY OC-48c front ports configuration. MOTOROLA GENERAL BUSINESS INFORMATION CSTAPOS-UG/D REV 00

12 12 CHAPTER 1: PACKET OVER SONET SWITCH APPLICATION GUIDE Freescale Semiconductor, Inc. System Configuration This application will run on the following modules as a part of the C-Ware Development System (CDS): Application Feature Overview Feature Overview and Standards Support C-5e Switch Module This application runs with the following Physical Interface Modules (PIMs): Front port M-5 CA The posoc48c application in the CST is a single PHY OC-48c Port to Fabric switching application. The Fabric mode currently supported is C-5e NP back-to-back. Packet forwarding is performed at layer 3 (IP routing). This application supports the following features: Layer 3 forwarding (IP routing) 16 way aggregation via the M-5 CA utilizing sequence numbers with the QMU to maintain correct sequencing of traffic with cross-cluster aggregation. ICMP Support for TTL expired, destination unreachable, and redirect Fabric Port support for C-5 NP back-to-back mode PPP statistics Instrumentation for performance Analysis, using the C-Ware Integrated Performance Analyzer (CWIPA) Even though this application is a Packet Over SONET application, all SONET framing and overhead processing is performed by a SONET framer on the other side of the M-5 CA The SONET blocks within the SDP are not utilized. As such none of the SONET overhead information is accessible via the C-5e NP chip and the Host would need to interface with the SONET framer to access that information. Layer 3 IP Routing IP routing is the process of forwarding IP frames at layer 3 based upon the IP Destination Address (IP DA). CSTAPOS-UG/D REV 00 MOTOROLA GENERAL BUSINESS INFORMATION

13 Application Feature Overview 13 Application Components Used 16 Way Aggregation 16 way aggregation is a new capability in the C-5e NP enabled through the use of the M-5 CA. Previously only four-way aggregation was supported and relied on tokens in the software to maintain correct traffic sequencing. 16 way aggregation relies on sequence numbers between the M-5 CA and the QMU to maintain traffic sequencing allowing aggregation across a larger group of CPRCs. ICMP Support The posoc48c application supports three types of ICMP messages: ICMP Time Exceeded ICMP No Route ICMP Destination Unreachable The reason that these three message types are supported in this application is that they are based on events that happen in the data path, as opposed to the control path. Instrumentation for Performance Analysis The posoc48c application includes probe points throughout the CPRC code that are used in combination with the C-Ware Performance Analyzer to collect detailed performance data during simulation of the application. The CST provides a number of application components that are used across applications. This application uses the following application components provided in the CST: queueutils fabrics (Back-To-Back) ip See the documentation in the apps/components/<componentname>/doc/ directory for the documentation on the software components that this application uses. MOTOROLA GENERAL BUSINESS INFORMATION CSTAPOS-UG/D REV 00

14 14 CHAPTER 1: PACKET OVER SONET SWITCH APPLICATION GUIDE Freescale Semiconductor, Inc. Application Control and Data Flow Figure 1 depicts the control and data flow for this application. Figure 1 Control and Data Flow Queue Storage (SRAM) Cluster QMU C-5 NP Enqueues/ Dequeues Enqueues/ Dequeues with Sequence Numbers Fabric FP IP DA slookups and Lookup Results PHY Table Storage and Statistics (SRAM) Ring Bus Global Bus Payload Bus All CPs within a cluster and all clusters for this application perform the same functions. To keep the diagram simple, only one cluster is shown but the information can be extended to all clusters. Traffic flow is only from the clusters to the fabric and vice versa. There is no traffic flow from one cluster to another since all clusters are connected to a single OC-48c port. There is no interaction with XP or host as part of traffic handling for this application. TLU External Host CPU PCI XP Processor Boundary SDRAM CP0 CP1 CP2 CP3 CP4 CP5 CP6 CP7 CP8 CP9 CP10 CP11 CP12 CP13 CP14 CP15 M-5 Frames (Sequence numbers in header) M-5 BMU IP Pkts CSTAPOS-UG/D REV 00 MOTOROLA GENERAL BUSINESS INFORMATION

15 Resource Utilization 15 Resource Utilization To support this application s features, the processors within the C-5e NP are performing a variety of tasks. The sections below enumerate what functionality is taking place and how the different processors are being used. XPRC CP Like all applications, the XPRC is used for boot and initialization of the chip. This application uses a two-phase XPRC initialization. The first phase allocates queues and buffer pools, and configures the mode registers for each CP for single PHY OC-48c on the front ports. It then initiates loading of the second phase XPRC program. The second phase XPRC initialization initializes the services functions, configures the QMU, initializes the host processor interface, initializes the TLU with the static table data, configures the Fabric Port for back-to-back operation and enables it, loads the CPs, enables them, and sends them their initialization messages. Configuration of the Fabric Port is controlled by an optional parameter during either the packload or the invocation of the C-5 Simulator. This parameter is used to override the fabric ID during the application s run. Traffic received on the OC-48c port whose corresponding lookup returns a route with the same fabric ID will be handled as an ICMP redirect. After initialization, the application only uses the XPRC for print servicing to support textual output for the CPs out a serial port connection. This application uses the CPs for the following: Initiating receive and transmit programs on the RxSDP and TxSDP Support 16-way aggregation utilizing an M-5 on the front ports Processing lookup results from the TLU to characterize a frame and make forwarding decisions Constructing descriptors for forwarding frames via the QMU to the Fabric Port Processing descriptors from the QMU for forwarding frames from the Fabric Port to the M-5 CA. MOTOROLA GENERAL BUSINESS INFORMATION CSTAPOS-UG/D REV 00

16 16 CHAPTER 1: PACKET OVER SONET SWITCH APPLICATION GUIDE Freescale Semiconductor, Inc. CPRC In this application, after initialization, the CPRC is only used for data forwarding. Initialization for the CPRC consists of: 1 Zeroing out statistics 2 Creating task threads for the ingress and egress processing 3 Processing the initialization message from the XPRC 4 Setting the aggregation mode for queue services to 16 way aggregation 5 Initializing the buffer pool to be used, preloading the two ingress data scopes with buffers 6 Preloading the transmit pending scopes with buffers for the egress processing to deallocate 7 Initializing descriptors to be used by the ingress and egress processing 8 Registering QMU errors to be handled by an interrupt handler 9 Setting up launch pads for the SDP to use in launching lookups 10 Configuring and enabling the SDP blocks. After the SDP is enabled, control is transferred to the ingress processing thread. The initialization message from the XPRC currently contains only two pieces of information, whether the fabric is enabled and, if enabled, the fabric id. The information configured for the SDP blocks, if any, will be discussed in the section on the appropriate SDP block. CPRC Ingress Processing As soon as the extract space is made available by the SDP, the CPRC checks that no errors were detected during header parsing. ICMP TTL expired is handled as a header error. If errors were detected, the packet is discarded and statistics are updated based on the frame status reported by the SDP. If no errors were detected, it checks that the routing protocol identified by the SDP is IP, if it isn t, the packet is dropped as an unsupported protocol. If the protocol is valid, the lookup results for the lookup launched by the SDP are retrieved. CSTAPOS-UG/D REV 00 MOTOROLA GENERAL BUSINESS INFORMATION

17 Resource Utilization 17 If a route was not found, the packet is handled as an ICMP destination unreachable. If the route is valid, the length reported in the header is checked. If length is too small, the packet is discarded. If the length is sufficiently large, a speculative enqueue is performed to provide the QMU with knowledge that the sequence number has started reception. Processing then waits for the payload reception to complete. A final check of frame status from the SDP is then done to detect CRC errors or oversized frames. If a frame error is detected at this point, and a speculative enqueue was performed, a commit with error is performed. The frame is then discarded and statistics updated based on the frame status. If there are no errors detected, either a normal enqueue, or a commit with valid status is performed to forward the frame to the fabric. Conditions are then initialized to support the next reception and the scope switched releasing it back to the SDP. Context swaps to the egress processing thread are performed whenever the processing is stalled waiting for an event in another component of the system to occur. Most notable among these are; waiting for an extract scope from the SDP, waiting for the lookup results from the TLU, waiting for the payload reception to complete, and waiting to allocate a buffer either for initializing conditions for the next reception or in preparing an ICMP response. Coordination of ingress processing within a cluster is managed by the SDP via token passing. This will be discussed in further detail in the section on the appropriate SDP block. Coordination of ingress processing across clusters is managed by the M-5 CA and QMU through the utilization of sequence numbers and distributing the traffic for the OC-48c port across all clusters. Please see the M-5 CA documentation for a full description of how this is accomplished. CPRC Egress Processing Egress processing first waits for the dequeue token for the cluster. After the token is available, the processing begins waiting for a non-empty transition on the output queue. When traffic is present in the queue for transmission, a dequeue action with sequence number is initiated. The processing then waits for the dequeue action to complete. If the dequeue was not successful and was not a dry dequeue, the associated buffer is deallocated, the processing goes back to waiting for another transition. If the dequeue was successful, and the frame is sufficiently small, the dequeue token is passed to the next CPRC in the cluster, otherwise token passing is delayed until later. The buffer from the last MOTOROLA GENERAL BUSINESS INFORMATION CSTAPOS-UG/D REV 00

18 18 CHAPTER 1: PACKET OVER SONET SWITCH APPLICATION GUIDE Freescale Semiconductor, Inc. transmission for this scope is then deallocated and the processing waits for merge space to become available. Once merge space is available, the CPRC fills out the necessary information in the merge space and switches scope to initiate transmission of the frame. If token passing was delayed, the CPRC now begins monitoring the number of bytes remaining to be transmitted to the M-5 CA. Once the bytes drop below the required threshold, the dequeue token is passed to the next CPRC in the cluster. This delay is necessary to prevent overflowing of the M-5 CA FIFOs in the case of two large frames being transmitted back-to-back by CPRCs within the same cluster. Once the token is passed, or immediately when no delay is required, the buffer associated with this transmission is saved off so it may be deallocated later and the CPRC returns to waiting for the dequeue token. CPRC/SDP Interface The CPRC interfaces to the TxByte processor via a set of special purpose registers called Merge Space registers. The Merge Space registers are used to communicate information such as length of the outgoing frame, sequence number for the frame, the value for the PPP protocol field and the transmit algorithm to be used for transmission. The CPRC interfaces to the RxByte processor via a set of special purpose registers call Extract Space registers. The contents of extract space contains the routing path to be used and a frame status followed by information from the IP header. The Merge and Extract space usage is presented in a tabular format later in the appropriate SDP section. The CPRC and the RxByte processor share a set of registers to do table lookup processing over the Ring Bus. Lookup requests are referenced using a request tag, that refers to one of four Ring Bus registers assigned for initiating requests. Slots 0 and 1 are accessible to the SDP, while slots 0 through 3 are accessible to the CPRC. Each slot contains a 4Byte control register and two 4Byte data registers. These data registers contain information to be delivered to the TLU such as the type of lookup being requested and the key information to use in determining a match. Depending on the type of lookup being issued and the length of the key, the number of slots needed for any single lookup can be 1, 2, or 4. The following request tag is used by the SDP in the posoc48c application: Tag 1: IP DA lookup Uses only slot 1. CSTAPOS-UG/D REV 00 MOTOROLA GENERAL BUSINESS INFORMATION

19 Resource Utilization 19 The CPRC in this application, initiates no lookups. The CPRC also has access to a set of Ring Bus receive control registers for checking lookup responses referred to as response tags. These response tags map to one of eight response slots available to CPRC, each of which contains a 4Byte control and 8Byte data portion. One response slot is therefore necessary for every eight bytes of data being returned from a lookup. When a lookup is initiated by either the SDP or the CPRC, the slot on which the response should land is indicated in the request. The status of the response is then checked by the CPRC using the appropriate response tag and the data returned is read upon receipt of a successful lookup indication. The following response slot is assigned in the posoc48c application: Response Tag 4: IP DA response Uses slots 4 and 5 to support the return of 16 bytes of data. MOTOROLA GENERAL BUSINESS INFORMATION CSTAPOS-UG/D REV 00

20 20 CHAPTER 1: PACKET OVER SONET SWITCH APPLICATION GUIDE Freescale Semiconductor, Inc. SDP The SDP in this application is responsible for decapsulation/encapsulation of both an unstuffed PPP header and the M-5 CA header and trailer. The receive processing is also responsible for detecting errors with the frame and launching the lookups required by the CPRC for routing. The transmit processing is also responsible for decrementing the IP header TTL field and adjusting the IP header checksum accordingly, monitoring a flow control signal from the M-5 CA and only transmitting when it is valid to do so, and generating the CRC value to be used in the M-5 CA trailer. RxBit RxBit waits for a transition of the PhyStatus0 signal to indicate valid data and the start of a frame. The processor owning the token streams the first byte of data upstream, passes the token, and then streams the data stream to the RxSync processor until the PhyStatus0 signal indicates that data is no longer valid. The processor then sends an EOF marker upstream with Merge9 set and a frame status value. Currently there are no errors detected by the RxBit processor and it always sends a frame status value indicating success. The processing then returns to waiting for PhyStatus0 to indicate valid data again. The RxBit processor does not currently use any control space information from the CPRC. RxSync RxSync currently serves no function in this application, it simply streams all data received to the RxByte processor. RxByte The RxByte processing can be broken down into the following sequential steps: Wait for Token - RxByte utilizes a token to determine which byte processor should forward its data stream upstream. The processors without the token simply sink the data stream until they have the token. The token is passed before the IP header is pased. Report Dropped Frames to CPRC - At the beginning of processing for every frame to be forwarded to the CPRC the RxByte reports a count of any frames dropped by RxByte due to CPRC unavailability (that is, CPRC hadn t released a scope yet) Wait For Valid Data - Simply monitoring the FIFO from RxSync for valid data bytes Process M-5 CA Header - Strip off the sequence number and frame length from the M-5 CA header and write them to extract space. CSTAPOS-UG/D REV 00 MOTOROLA GENERAL BUSINESS INFORMATION

21 Resource Utilization 21 Process PPP Header - Strip off the address and control bytes of the PPP header. Strip off the protocol field of the PPP header and write it to extract space. The protocol field is checked to verify that it specifies IP. If IP is not specified, a frame status of Unknown Protocol is written to extract space and the rest of the frame is simply streamed to the buffer and not parsed. Process IP Header - Parse and validate the Version and Header Length fields from the IP header. Currently this application only supports Version 4 with a Header Length of 5 (no options). If either field is invalid the frame status in extract space is updated to indicate the unsupported feature and the rest of the frame is simply streamed and not parsed. It should be noted that even when valid, the IP header is streamed to the buffer and not stripped like the M-5 CA and PPP headers. The rest of the IP header is streamed and the header checksum validated, errors being reflected in the frame status in extract space. Assuming the header is valid, RxByte will then launch a lookup on the IP destination address. Set L1 Done - Once the IP header processing is complete RxByte sets the L1 Done flag signalling the CPRC that header extraction and processing is complete and the CPRC can begin processing for the frame. Stream the Payload - The remainder of the frame is streamed using a 4 byte deep pipeline. The pipeline is used to avoid streaming the3 bytes of M-5 trailer and the 1 byte frame status from RxBit. Check Status Bytes - The status bytes from RxBit and the M-5 CA are processed and the frame status in extract space is updated with any errors indicated. If there are any errors indicated, the CRC check will not be performed. Check CRC - RxByte feeds the incoming stream through a CRC accumulator as it is processing the frame. After it has fed the CRC field from the M-5 CA trailer through the accumulator, the value of the accumulator is checked to determine is there has been any data corruption. Note, the status byte from RxBit should not be fed through the CRC accumulator. If any CRC errors are detected, the frame status in extract space will be updated to reflect the error. Delay - In order to ensure that the final byte streamed to the buffer has been written, RxByte must delay 10 clocks after the last byte was streamed before switching scope to give the DMA time to complete the action. MOTOROLA GENERAL BUSINESS INFORMATION CSTAPOS-UG/D REV 00

22 22 CHAPTER 1: PACKET OVER SONET SWITCH APPLICATION GUIDE Freescale Semiconductor, Inc. Switch Scope and Check Ownership - RxByte now switches to the other scope and checks whether the CPRC has released it yet. If the CPRC has already released the new scope RxByte starts over with the Wait for Token step. Otherwise it continues with the next step. Wait For Scope Ownership - While RxByte is waiting for the CPRC to release a scope, it continues to read any incoming data on the FIFO and discards the bytes read. It keeps track of how many frames are encountered in the discarded data and reported the count in the Report Dropped Frames step when the CPRC releases the scope. Table 4 defines the use of extract space by this application. Many of these fields come from the IP header. Please see the standards for a complete description of the fields. The TCP portion of extract space is not used in this application. Table 4 Extract Space Field Descriptions FIELD NAME SIZE OFFSET DESCRIPTION protocol 1 0 Indicates which RX processing CPRC should apply internalstatus 1 1 Not used in this application framestatus 1 2 Status code of frame processing (valid when entire frame has been parsed) badframecount 1 3 Count of drops while SDP waits for scope length 2 4 Length of frame from M-5 header protocoltype 2 6 Protocol from PPP header ipcontrol 4 8 Not used in this application version 1 12 IP version (should be 4) IHL 1 13 IP header length TOS 1 14 IP Type of Service field pad 1 15 Reserved length 2 16 IP total length indentifier 2 18 IP identifier field flags 1 20 IP flags pad Reserved fragoffset 2 22 IP fragmentation offset TTL 1 24 IP time to live protocol 1 25 IP protocol field CSTAPOS-UG/D REV 00 MOTOROLA GENERAL BUSINESS INFORMATION

23 Resource Utilization 23 checksum 2 26 IP checksum sourceaddr 4 28 IP source address destaddr 4 32 IP destination address pad Reserved sourceport 1 40 Source Port from TCP header destport 1 41 Destination Port from TCP header seqnumber 4 42 Sequence number from TCP header acknumber 4 46 Acknowledgement Number from TCP header headerlen 1 50 Number of 32 bit words in the TCP header codebits 1 51 Control bits from the TCP header window 2 52 Number of data octets sender is willing to accept checksum 2 54 Checksum on TCP header urgentptr 2 56 Urgent pointer offset from sequence number from TCP header m5seqnumber 2 58 Sequence number from the M-5 header reserved Unused in this application TxByte The TxByte processing can be broken down into the following sequential steps: Wait For Valid Data - TxByte waits for valid data to appear in the FIFO from the CPRC. Once valid data is present, TxByte initializes the CRC accumulator and begins processing the data. All data bytes output by the processing will also be run through the CRC accumulator. Encapsulate with M-5 CA Header - TxByte generates and outputs the sequence number and frame length fields of the M-5 Header based on data from merge space. Encapsulate with PPP Header - TxByte generates and outputs a constant PPP header using 0xFF for the address field, 0x03 for the control field, and 0x0021 representing the IP protocol for the protocol field. Stream Initial Portion of IP Header - TxByte streams from the FIFO the first 8 bytes of the IP header. Decrement TTL - TxByte modifies the byte for the TTL field of the IP header to decrement it by 1 in support of the IP protocol. MOTOROLA GENERAL BUSINESS INFORMATION CSTAPOS-UG/D REV 00

24 24 CHAPTER 1: PACKET OVER SONET SWITCH APPLICATION GUIDE Freescale Semiconductor, Inc. Stream the Rest of the IP Header - TxByte streams the rest of the IP header from the FIFO, modifying the IP header checksum before sending to accommodate the decrement of the TTL field. Stream the Payload - TxByte now streams bytes from the FIFO until it receives an indication that there is no more valid data in the FIFO (that is, Data9). Encapsulate with M-5 Trailer - TxByte generates and outputs a status field indicating a good frame and outputs the CRC field based on the value read from the CRC accumulator. Switch Scope -TxByte now switches to the other scope and returns to the Wait For Valid Data step. Table 5 defines the use of merge space by this application. Table 5 Merge Space Field Descriptions FIELD NAME SIZE OFFSET DESCRIPTION unused 4 0 Unused in this application length 2 4 Length to be inserted in M-5 header seqno 2 6 Sequence number to be inserted in M-5 CA header TxBit TxBit waits for an indication of valid data in its FIFO. While there is no valid data, it will transmit idle characters to the M-5 CA. It does not matter what character is used for idle as long as the transmit enable signal is not asserted when the character is transmitted. Once valid data is present in the FIFO, TxBit asserts the transmit enable signal and begins streaming the data from the FIFO to the M-5 CA. Once the FIFO is emptied of valid data, the TxBit processor de-asserts the transmit enable signal. TxBit then checks the PhyStatus0 signal to see if flow control is being asserted by the M-5 CA. If flow control is asserted, the TxBit processor will transmit idle characters until the flow control is de-asserted. When flow control is not asserted, TxBit will return to monitoring the FIFO for valid data. CSTAPOS-UG/D REV 00 MOTOROLA GENERAL BUSINESS INFORMATION

25 Resource Utilization 25 TLU This application has the following table: BMU IP Routing table These tables are implemented by the following types of TLU tables: IP Routing Table: Hash/Trie/Key No route learning is currently supported for any of these tables in this application. IP Unicast Routing Table The IP Routing Table provides a longest prefix match lookup on the IP destination address which will provide an egress queue to be used for forwarding the frame. This table only provides support for IP Unicast and the application contains no support for IP multicast. Table 6 IP Unicast Routing Table Field Descriptions FIELD NAME SIZE OFFSET DESCRIPTION fabricid 1 0 Fabric Id for the fabric port pad1 1 1 Unused in this application pad2 2 2 Unused in this application pad3 4 4 Unused in this application queueid 4 8 Queue ID for the egress interface fabric vnid 4 12 VNID for fabric routes The application allocates one buffer pool per Channel Processor. It also allocates 4 pools for the Fabric Processor. T Table 7 Buffer Usage NUMBER OF LARGE INTERFACE POOLS BUFFERS BUFFER SIZE CPRCs Fabric Currently the only portion of the frame written to buffer is the IP packet, all M-5 CA and PPP related headers and trailers as stripped from the data stream and not written to the buffer space. MOTOROLA GENERAL BUSINESS INFORMATION CSTAPOS-UG/D REV 00

26 26 CHAPTER 1: PACKET OVER SONET SWITCH APPLICATION GUIDE Freescale Semiconductor, Inc. QMU This application allocates 4 queues for the CPs which are shared by all 16 CPs, 8 queues for the fabric, and 8 queues for the XPRC. Currently the application uses a control queue for each CP and a single queue for traffic for all CPs. During the initialization phase of the application s XP program, each CP receives an initialization descriptor from the XP. Table 8 documents the descriptor format used for the initialization of the CPs. Table 8 Initialization Descriptor Field Descriptions FIELD NAME SIZE OFFSET DESCRIPTION ctlcommand 1 0 Unused in this application ctldata1 1 1 Unused in this application ctldata2 1 2 Unused in this application fabricenabled 1 3 Indicates whether the fabric is a valid forwarding port. Always true in this application. fabricid 4 4 Local fabric id CSTAPOS-UG/D REV 00 MOTOROLA GENERAL BUSINESS INFORMATION

27 Resource Utilization 27 PDUs are forwarded from ingress port to egress port or from one processing block to another. Data describing the PDU is contained in a descriptor. Table 9 documents the descriptor used in this application. Table 9 PDU Forwarding Descriptor FIELD NAME SIZE OFFSET DESCRIPTION bufhandle 4 0 Buffer handle of PDU being forwarded length 2 4 Length of PDU VNID_client 2 6 Upper 12 MSBs contain VNID which becomes the egress queue id on the other C-5 NP when going across the fabric. Lower 4 MSBs contain the transmit client protocol which in this application is always IP. appdata1 4 8 Unused in this application appdata Unused in this application sequencenumber 2 14 Used to hold the sequence number to be inserted into the M-5 CA header. MOTOROLA GENERAL BUSINESS INFORMATION CSTAPOS-UG/D REV 00

28 28 CHAPTER 1: PACKET OVER SONET SWITCH APPLICATION GUIDE Freescale Semiconductor, Inc. FP This application configures the FP to operate in back to back mode for connection to a another C-5 through a switch fabric. The Fabric Processor (FP) is used for off-chip data forwarding. The posoc48c application uses the FP to forward descriptors and data to another C-5 NP. When frames are forwarded over the fabric, the descriptor sent to the Fabric Port must have certain information specified in bit locations expected by the Fabric Control Engine (FCE). This information allows the Fabric Port to know where in buffer memory the frame is stored, the length of the frame, and the target queue to which the frame should be sent at the destination C-5 NP. The descriptor s data format is: Bit Position 31 0 Field Name Buffer Handle Bit Position Field Name Frame Length Virtual Network ID (VNID) App. Data Bit Position Field Name Application Data Bit Position Field Name FIELD NAME BIT POSITION DESCRIPTION Application Data Buffer Handle 31:0 This is the buffer handle assigned to the PDU. The quantity includes such information as the pool ID, BTAG, and multicast flag. Application Data 32:35 These bits are available to the application for use in its own internal forwarding logic. Frame Length 63:48 Indicates the length of the frame. Virtual Network ID 47:36 This field is used to inform the Fabric Port on the target C-5 NP to which queue this descriptor ought to be sent. The VNID is interpreted as the egress queue by the fabric on the target C-5 NP and the descriptor is sent directly to it. Application Data 127:48 These bits are available to the application for use in its own internal forwarding logic. When each C-5 NP is initialized, it is assigned an ID within the fabric. Tables such as the IP Routing Table and IP Flow Table also associate each of their entries with such a C-5 NP instance in the fabric. If the target address to which a frame is being forwarded resides on another C-5 NP, a descriptor is prepared and sent to the appropriate traffic queue representing the target C-5 NP. CSTAPOS-UG/D REV 00 MOTOROLA GENERAL BUSINESS INFORMATION

29 Supplied Application Files 29 Fabric Port Activation Since there is only one port on the front ports, the posoc48c application can only perform forwarding when the fabric is active. A Fabric ID is assigned to the application at run-time activating the fabric. A default fabric id of 12 has been built into the application and will be used if one is not specified at initialization. The Fabric Processor will not transmit frames on the hardware unless there is another C-5 switch module in the system or a special loopback connection on the C-Ware Development System (CDS). Please see the README file for the application for details on using runtime arguments to assign a fabric id. Host Processor Interaction In this application, there is no host processor support - besides what is included in the C-5 Device Driver. Supplied Application Files Application Wide XPRC Below is a list of the files that are a part of this application and a brief description of their contents. appconfig.h: Provides a set of defines to configure queues, buffers and FP cell size. ipif.h: Provides IP header and error codes used by ipfilterparse.h processing for communicating with the CPRC processing. posif.h: Provides types and defines used for communication between XPRC, CPRC, and SDP. Also provides PPP statistics structure. m5sphybitif.h: Provides control space definition for RxBit and TxBit and codes for frame status from RxBit. iptableif.h: Provides IP Routing Table data structures and types. xpmaininit.c: Provides the first phase of XPRC initialization. xpmain.c: Provides the second phase of XPRC initialization and continuous print service in support of CPRC textual output. tle_restore.h: Produced by the offline table building software, the file provides routines and data used to initialize the TLU with the static routing tables. tle_writes.h: Produced by the offline table building software, the file provides routines and data used to initialize the TLU with the static routing tables. MOTOROLA GENERAL BUSINESS INFORMATION CSTAPOS-UG/D REV 00

30 30 CHAPTER 1: PACKET OVER SONET SWITCH APPLICATION GUIDE Freescale Semiconductor, Inc. CPRC ipcp.h: Provides types used for IP routing on the CPRC and inline processing for IP table lookup results and determination of ICMP destination unreachable and redirect conditions. SDP FDP poscp.h: Provides types used by packet processing code on the CPRC and function prototypes for poscp.c. poscpinline.h: Provides inline processing for error statistics and packet transmission. posqueueutilsinline.h: Provides useful inline debug routing for tracking packet forwarding queue usage. Disabled unless QUEUE_DEBUG flag is compiled in. oc48cpmain.c: Provides CPRC initialization code. ipinitcp.c: Provides code to initialize structures used by IP processing. poscp.c: Provides initialization code to initialize structures used by packet forwarding, register interrupts, and enable SDPs. Also provides QMU error interrupt handling and packet drop processing. posoc48cp.c: Provides ingress and egress processing for the CPRC. ipfilterparse.h: Provides IP header processing for RxByte. oc48rxbyte.c: Provides M-5 CA and PPP decapsulation and payload processing for RxByte. oc48rxsync.c: Provides pass through processing for RxSync. m5sphyrxbit.c: Provides RxBit processing. m5sphytxbit.c: Provides TxBit processing. oc48txbyte.c: Provides TxByte processing. Fabric processing for this application is provided by the Fabrics component. Please see the documentation for that component for a list of files involved in the Back-to-Back Fabric processing. Host There are no host files provided for this application since this application has no special host interaction beyond the standard host driver. CSTAPOS-UG/D REV 00 MOTOROLA GENERAL BUSINESS INFORMATION

31 Design Issues 31 Binaries posoc48c.pkg Package file for this application posoc48c.dsc Package descriptor file for this application posoc48cxpinit.map Memory map for the XPRC initialization program. posoc48cxpinit.dcp Program load for the XPRC initialization program. Simulation files posoc48cxp.map Memory map for the XPRC program posoc48cxp.dcp Program load for the XPRC program oc48cp.map Memory map for the CPRC program oc48cp.dcp Program load for the CPRC program m5sphyucode.sdp Program load for the SDP program config Configuration file for software simulator MC.config Used to efficiently configure buffer pools in simulation config.fdploopback Configuration file for software simulator where FDP is put inexternal loopback sim.in Input file for software simulator m5simoc48ccommands.in Input file for software M5 simulator Tlu.State TLU SRAM state used for this application for C-5e NP simulation runs Design Issues This application will support OC-48 data rates. To achieve this it must utilize both the M-5 CA and speculative enqueues. M-5 Configuration and Usage The M-5 CA can be configured into a number of different configurations but there is only one configuration for single PHY operation and the rest of the configurations are for multi-phy operations which do not apply to this application. In this configuration the M-5 CA will distribute the traffic for the OC-48c flow across all 16 CPs/SDPs. Since the C-5e NP does not contain efficient mechanisms for coordinating across clusters, to avoid out-of-sequence traffic problems the M-5 CA will generate sequence numbers and will embed them in an M-5 CA header attached to each frame. These sequence numbers are extracted by the SDP and forwarded by the CPRC when enqueuing the frame MOTOROLA GENERAL BUSINESS INFORMATION CSTAPOS-UG/D REV 00

32 32 CHAPTER 1: PACKET OVER SONET SWITCH APPLICATION GUIDE Freescale Semiconductor, Inc. to the QMU. The QMU utilizes the sequence numbers to control the sequence in which the enqueues are actually handed to the Q-5 TMC ensuring that once the frames are enqueued on the Q-5 TMC they are in the proper sequence. Speculative Enqueues When transmitting to the M-5 CA there are limitations on back-to-back transmission of large frames from the same cluster. Since flow control from the M-5 CA is only checked between frames, the C-5 NP must prevent another CP within the same cluster from starting processing of a frame until the remaining portion of the current frame to be transmitted is below a given threshold. This is accomplished by delaying the passing of the dequeue token. For more detail on this, please see the section on Egress Processing within the CPRC section. The use of the M-5 CA and sequence numbered enqueuing via the QMU introduces a problem when using variable sized traffic, multi-cluster aggregation, and the old approach of only enqueuing once the entire payload has been received. In the case of a large PDU followed by many small PDUs, hundreds of PDUs might back up waiting to be enqueued which is more than can be supported. This problem is solved through the use of speculative enqueues. Speculative enqueuing allows the CPRC to reduce latency between the start of a PDU and the enqueue to the QMU. Whenever the PDU length is greater than 64 bytes, the CPRC will perform a speculative enqueue once the forwarding route is determined. This allows the QMU to allow later enqueues to proceed while the payload of the earlier PDU continues to be received and written to buffer space. Once the entire PDU has been received, the CPRC performs a commit action, with or without an error status based on the final frame status. Since the enqueues are sequence numbered, even though the later enqueues are allowed to proceed, they can not be dequeued until the commit has occurred. Dequeues are guaranteed to occur in sequence based on the sequence numbers. When a speculative enqueue which has not been committed is the next PDU to be dequeued the queue will appear empty to the dequeuing processor. In addition, once a CPRC has performed a speculative enqueue only a commit action will be allowed from the CPRC. An attempt by the same CPRC to perform an enqueue when there is a speculative enqueue outstanding will result in an error return from the QMU. An additional impact is incurred by the dequeuing processor when speculative enqueues are used. Speculative enqueues which are committed with an error status remain in the queue and the dequeuing processor must dequeue and discard these PDUs. CSTAPOS-UG/D REV 00 MOTOROLA GENERAL BUSINESS INFORMATION