Frame Manager (FMan) Internals

Transcription

1 Frame Manager (FMan) Internals AN130 David Lapp Senior System Architect

2 This session is an introduction to Frame Manager Internals Introduction It is intended to be stand alone but it is helpful to have viewed An Introduction to the QorIQ Data Path Acceleration Architecture

3 What is the Datapath Acceleration Architecture (DPAA)? The QorIQ DPAA is a comprehensive architecture which integrates all aspects of packet processing in the SoC, addressing issues and requirements resulting from the multicore nature of QorIQ SoCs. The DPAA includes: Cores Network and packet I/O Hardware offload accelerators The infrastructure required to facilitate the flow of packets between the above The DPAA also addresses various performance related requirements especially those created by the high speed network I/O found on multicore SoCs such as the P4080

4 Frame Manager (FMan) in the DPAA Infrastructure components Queue Manager (QMan) Buffer Manager (BMan) Hardware accelerators SEC Cryptographic accelerator PME Pattern matching engine Cores and Network I/O Frame Manager (FMan) FMan is a key component in the DPAA and makes use of the DPAA infrastructure (QMan and BMan) Queue Manager Buffer Manager Ethernet Interfaces Frame Manager SEC PME Frame Manager CoreNet Ethernet Interfaces Memory e500mc e500mc Core e500mc Core D$ Core I$ e500mc L2$ D$ Core I$ e500mc L2$ D$ Core I$ e500mc L2$ D$ Core I$ e500mc L2$ D$ Core I$ e500mc L2$ D$ Core I$ L2$ L2$ D$ I$ L2$ D$ I$

5 Frame Managers in P4080 P4080 Power Architecture 128KB e500-mc Core Backside L2 Cache 32KB 32KB D-Cache I-Cache 1024KB Frontside L3 Cache 1024KB Frontside L3 Cache 64-bit DDR-2 / 3 Memory Controller 64-bit DDR-2 / 3 Memory Controller eopenpic PreBoot Loader Security Monitor Internal BootROM CoreNet Coherency Fabric PAMU PAMU PAMU PAMU PAMU Peripheral Access Mgmt Unit Power Mgmt SD/MMC SPI elbc Security 4.0 Queue Mgr. Frame Manager Parse, Classify, Distribute Frame Manager Parse, Classify, Distribute RapidIO Message Unit (RMU) 2x DMA Real Time Debug Watchpoint Cross Trigger 2x DUART 4x I 2C 2x USB 2.0/ULPI Test Port/ SAP Pattern Match Engine 2.0 Buffer Mgr. 10GE Buffer 1GE 1GE 1GE 1GE 10GE Buffer 1GE 1GE 1GE 1GE PCIe PCIe SRIO PCIe SRIO Perf CoreNet Monitor Trace Aurora Clocks/Reset GPIO CCSR P4080 has two Frame Managers 18-Lane 5GHz SERDES

6 FMan features and functions Frame Manager (FMan) supports: L2/L3/L4 protocol parsing and header validation Standard and user defined protocols supported Queue selection Exact match classification for picking control packets out of ingress traffic Hash based for load distribution IEEE 1588 time stamp support RMON/ifMIB stats Color aware dual rate policing right size buffer acquisition from BMan buffer pools Picks buffer based on R ed frame size Per port egress rate limiting TCP/UDP T checksum calculation One 10GE MAC and 4 GE MACs Max 12 packets of Parse/Classify/Distribute

7 FMan internal architecture FMan supports a flexible pipeline of packet processing elements Frame data and per frame context is stored in internal memory while frame is processed Main packet body is stored in buffer(s) in external memory after full packet is received. Only (up to 256B) header held in internal buffers for majority of processing This memory is accessible to all processing elements which may update context after they process a frame DMA transfers data between external and internal memory on behalf of other processing elements Frame Processor Manager (FPM) schedules frames for processing by different elements to create appropriate pipeline When finished processing a frame each element in the pipeline determines a Next Invoked Action (NIA) which FPM uses to direct the frame to the next element Default pipeline configured for each port Processing elements can alter pipeline from the default flow FPM 10GE CoreNet Policer Classifier Frame Manager (FMan) DMA GE Memory BMI To QMan Keygen (Distribution) Parser GE GE GE QMI To BMan

8 IC Internal Context Each frame has an Internal Context (IC) stored in FMan memory The IC holds information about the frame which is passed to later processing elements for their use Can be pre-pended to frame buffer on R to pass this information to software Parser Result Identifies protocol stack, and offsets to all headers Key Hash key Software only needs to detect collisions to finalize classification Time stamp (IEEE 1588)

9 IC Internal Context IC FD Frame Descriptor AD Action Descriptor CCBASE KS Key Size HPNIA Hash PR Parse Result Time Stamp Key - (?56 bytes ) DD debug Partition ID BPID Buffer Pool ID Address Format Offset Length/ Weight OOB status/ command FQ ID Policer Profile Port ID[0,0:5] Parse result data -inc. LCV, Class. Plan ID DME DMA Error PHE Physical Error FSE Frame Size Error DIS -Discarded EOF Extract Out of Frame NSS No Scheme Selected FCL Frame Colour IPP Illegal Policer Profile PTE Parser Time Out ISP Invalid Soft Parser PHE Header Error FPT Processing TO

10 Example R flow MAC (R) Parser Keygen Generate CCBASE BMan (empty buffer) BMI (Allocate internal buffers) Classifier BMI (Get External buffer) Keygen Generate hash Calculate FQID DMA (write context + header) QMan (enqueue) DMA (write data) Policer QMI Queue to CPU Frame queue Packet in Parse Classify and Distribute Distribute

11 BMI R processing Receive frame from MAC Store in FMan memory until R complete Calculate raw L4 checksum and pass to parser Allocate Internal Context (IC) in FMan memory Set initial values on per-port basis NIA HPNIA Parser Result Allocate external buffer for frame store Initiate transfer to memory from FMan memory Various options for empty space prepend, postpend, head/tail cut, IC prepend Request QMI enqueue De-allocate buffers on enqueue fail

12 BMI T processing Request QMI dequeue Based on per port rate limiting, internal memory fill Initiate transfer from external memory to FMan memory Calculate raw checksum Transfer frame to MAC Pass L4 checksum (offset/value) to MAC De-allocate internal buffers after transfer to MAC complete

13 Parser Performs parsing of common L2/L3/L4 headers, including tunneled protocols Can be augmented by the user to parse other standard protocols Can also parse proprietary, userdefined headers at any layer: Self-describing, using standard fields such as proprietary Ethertype, Protocol ID, Next Header, etc. Non-Self-Describing through configuration. Parse results, including proprietary fields, can be used by the classifier, and/or software. Soft parse can modify any field in parse results Other L2 Other L3 Other L4 PPPoE PPP Min Encap GRE Ethernet VLAN VLAN IPv4 Tunneled IPv4 TCP/UDP IPv6 Tunneled IPv6 s=1* s=1* MPLS s=0 s=1* MPLS MPLS to Other L4 s=0 IPSec

14 Soft Parsing of proprietary headers Example Frame DA SA EtTyp=8100 VLAN EtTyp= UDF EtTyp=... TOS... PID... SIP DIP SPort DPort... Ethernet HS VLAN HS PortID Eth Offset L2 Result VLAN Offset L3 Result IP Offset L4 Result L4 Offset Shim Result Shim1 Offset Shim2 Offset Shim3 Offset Other L3 Parser Result (PR) IPv4 TCP/UDP

15 Start Parse Parser Classification Plan ID ID α LCV Parse Ethernet LECM ID β Multi/ Broadcast Detected! Valid! Parse IPv4 ID β No Multicast Detected! Valid! To KeyGen through Parse Result Parser results consist of: Classification Plan ID Lineup Confirmation Vector (LCV) bit vector which represents the protocol stack found by parser Offsets to protocol headers Lineup Enable Configuration Mask (LECM) configured per protocol Logical OR of LECMs determines LCV Each frame has an initial Classification Plan ID set based on the ingress port configuration Classification plan offset added to Plan ID for e.g. Multicast/Broadcast Classification plan allows for differentiation of classification/distribution for multicast or broadcast frames

16 Classification Classification supports: 1. Direction of specific types of traffic to specific queues (instead of has based load distribution) 2. Fine grained control over hash based load distribution 3. Frame drop And selection of a policer profile Or Next table Classification tables can be from 10 s to 100 s entries big Up to 3 (cascaded) tables of up to 256 entries each Total key size for all entries in all tables cannot exceed 512B Limit is driven by design goal to support 12 Key can be up to 56B Key may be different for each of the 3 tables Tables are searched as an ordered list first match stops lookup Each entry s key is maskable Total size of all tables in FMan cannot exceed 32KB Key Key Key Action FQ x FQ y Table x Table y Action FQ Action x FQ yfq x Table FQ x y Table Table y q Table r Key Action Key FQ Action x Key FQ yfq Action x Table FQ x yfq x Table Table y FQ x y Table Drop y FQ z KeyGen Policing Profile Packet Handling Drop FQ ID Keygen scheme

17 Keygen Keygen can perform two distinct functions: Calculates initial Classification Base (CCBASE) for Classifier using Parser derived LCV Determines a Frame Queue ID and Policer profile based on frame header values and Parser derived LCV and Plan ID If both functions are required for frame processing on a given port then Keygen is found twice in the pipeline: once before Classifier, once after Classifier Keygen is configured by defining schemes Keygen has up to 256 schemes held in memory internal to Keygen Up to 32 consecutive schemes can be grouped together with the same Plan ID LCV is used to identify which scheme in a Plan to use Scheme defines how to generate key, FQID range, Policer profile range Classifier may also direct Keygen to a specific scheme

18 Keygen: hash based distribution Logical AND of LCV and Classification Plan ID ( 256 pcs) defines input to scheme matcher First match between AND result and 32 schemes is executed Parser Classification Plan ID ID α ID β ID β LCV Multi/ Broadcast Detected! No Multicast Detected! Valid! Valid! Start Parse Parse Ethernet Parse IPv4 LECM α β γ δ Class. Plan ID0 LEV Class. Plan ID1 LEV Class. Plan ID2 LEV Class. Plan ID3 LEV Get first match Scheme 0 Scheme 1 Scheme 2 Scheme 3 Scheme definition: 256 x Match Vector 0 Match Vector 1 Match Vector 2 Match Vector 3 32 x 32 x

19 Keygen Key generation details Keygen schemes contain a set of tuples which describe how to generate the key V: Valid DV: Default Value (point to other registers) Size: Type 1: Rotate right (1 byte) Type 0: Number of bytes extracted Mask : Bit mask Type: Type 1: Extract command (extract to FQID) Type 0: Generic extract Command (extract to key) HT: Point to header offset in Parse Result, with or without validation EO: Extract offset Index mode: FQID = Key[0:23] & Hash Mask Data0[0:23].. Data7[0:23] FQID base Hash mode FQID = Shifted Hash Key[0:23] & Hash Mask Data0[0:23].. Data7[0:23] FQID base

20 KeyGen Generic Extract Command Example 1 Frame header IPv4 Queue Offset HT = IPv4 EO = 0 Offset to be extracted Key Hash && FQ offset to be filtered out, size is by default 1 byte Queue Offset 64 bit hashed key Mask = 0x0F && Hash mask = 0x0000F Result Queue Offset 24 bit FQID FQ Base Queue Offset

21 KeyGen Generic Extract Command Example 2 Frame header IPv4 Extra header field HT = IPv4 EO = 0 Field to be filtered out. Size = 4 bits Key && Extra header field Mask = 0x0F Result Ex h fld && Ex h fld Hash 64 bit hashed key

22 Implements RFC2698, RFC4115 Three Color output "GREEN", "YELLOW", & "RED" Four parameter policing CIR - Committed Information Rate PIR - Peak Information Rate CBS - Committed Burst Size PBS - Peak Burst Size Color aware or color blind policing Policing can be based on packet count or byte count Quick drop mode on RED colored packets Memory internal to policer holds 256 profiles Statistics per profile Timestamp Policer Rate Processor Profile RAM

23 -Match: -MAC UCast DA -MAC MCast/BCast -Verify CRC Frame: 1. SCTPoIPoEnet: Control 2. UDPoIPoEnet: Dataplane (priority on IP ToS field) -Transfer frame to memory -HPNIA for this port = Coarse classify -Calculate 1s complement checksum over frame MAC BMI Parser -IPv4 detected on Etype -Check IPv4: -Checksum -Version -Multiple length checks -Check UDP: -Checksum -Length -Generate Parser Result: -Offsets (inc SCTP/UDP) -L2/L3/L4 SCTP -Filter SCTP traffic -AD: FQID/Policer Profile -All other traffic to KeyGen -Optionally exact match against UDP/IP addresses/ports first Coarse Classify!=SCTP Coarse classification table 1: AD0: FQID = Control, NIA = policer AD1: NIA = KeyGen Example IP Protocol = SCTP AD Parser Code = IPv4 Protocol QMI BMI Policer KeyGen -Enqueue to Qman Enqueue -Call QMI -Deallocate Bman buffers on enqueue fail -Police 256 profiles -SCTP if needed -Data plane traffic if needed -FQID = FQID_base + IP ToS field (+hash of UDP/IP/GTP/etc) SCTP 8 (per ToS field) Queues 7x 24 bit FQID FQ ID Base FQ ID Base IP ToS IP ToS Hash of e.g. IP/UDP/ GTP/proprietary fields To High priority Control WQ/ Channel To high priority Data Plane WQ/shared channel To low priority Data Plane WQ/ shared channel. Equal allocation among FQs

24