Leveraging Data Plane Acceleration Techniques on the QorIQ P4080 Processor

Transcription

1 June 2010 Leveraging Data Plane Acceleration Techniques on the QorIQ P4080 Processor For High-Performance Network Security Applications (v1.0) John Rekesh Software Architect, Software Products Division

2 Agenda Introduce Data Plane Control Plane model Performance pointers for network security packet processing Ingress processing On the cores and look-aside Egress processing VortiQa software implementation as a reference 2

3 QorIQ P4080E Processor Block Diagram 128-Kbyte Backside L2 Cache Power Architecture e500-mc Core 1024-Kbyte Frontside L3 Cache 32-Kbyte 32-Kbyte 1024-Kbyte D-Cache I-Cache Frontside L3 Cache 64-bit DDR-2 / 3 Memory Controller 64-bit DDR-2 / 3 Memory Controller eopenpic PreBoot Loader Security Monitor Internal BootROM PAMU PAMU PAMU CoreNet Coherency Fabric PAMU PAMU Peripheral Access Mgmt Unit Power Mgmt SD/MMC SPI elbiu M2SB Security 4.0 Queue Mgr. Frame Manager Parse, Classify, Distribute Frame Manager Parse, Classify, Distribute SRIO Message Unit DMA Real Time Debug Watchpoint Cross Trigger DUART 2x I 2C 2x USB 2.0/ULPI Test Port/ SAP Pattern Match Engine 2.0 Buffer Mgr. 10GE Buffer 1GE 1GE 1GE 1GE 10GE Buffer 1GE 1GE 1GE 1GE PCIe PCIe SRIO PCIe SRIO Perf Monitor CoreNet Trace Aurora Clocks/Reset GPIO CCSR 18-Lane 5 GHz SerDes 3

4 Control Plane / Data Plane Model Network Security Applications Session based holds state for various traffic flows Firewall, IPsec VPN, Intrusion Detection/Prevention, Anti-virus/Anti-spam Data Plane (DP) Session establishment Sometimes done in control plane Security processing of established flows Many cores SMP or AMP model Control Plane (CP) Management functions A few cores 2 X 10Gig ports CPDP packet path CPDP message path CP SMP Linux DP- LWE 4

5 Data Plane: Normal and Fast Path Example DP Normal Path May require locking ACL/Policy lookup Session Management dequeue Lookup Normal Path enqueue Egress Create/Modify/Delete Timers Fast Path Push to fast path if qualified DP Firewall Fast Path Lockless, cache-optimized Sessions pushed by FW if: Traffic not meant for SELF No ALGs required, No frag/re-assembly IPS and Application detection are not required FMan handles many integrity checks and checksums Do remaining packet integrity checks and forward Optimized forwarding DP IPsec Fast Path Lockless, cache optimized Inbound processing Lookup SA based on SPI Send packet to SEC engine Outbound processing on returned packet Outbound processing Lookup session Send packet to SEC engine Forward return packet from SEC engine 5

6 Data Plane (DP) Packet Flow - Acceleration Overview F/B/QMan Ingress Offload Buffer allocation Packet integrity checks Classification, schemas and annotations Traffic policing Congestion avoidance and pause frames Work/traffic prioritization and distribution Flow-order preservation Stashing Soft parsing SEC, PME Look-Aside Offload IPsec/IKE cipher, hash, crypto algorithms Intelligent IPsec protocol processing Intelligent SSL protocol processing Regular expression search Stateful rule based matching F/B/QMan Egress Offload Congestion avoidance Traffic shaping/scheduling Checksums VortiQa Networking Software in Multicore Environment QoS, Firewall F/B/QMan Ingress Offload Ingress Packets PDCP, IPSec SEC, PME Look-Aside Offload IPS, GTP-U, etc. F/B/QMan Egress Offload Egress Packets 6

7 FMan primed with buffer pools of select frame sizes Multiple buffer pools to best accommodate for varying packet sizes Allocation provides for head room (annotation area) in the packet/frame Headroom holds VortiQa packet descriptor FMan parse results populated into VortiQa packet descriptor Source IP, Destination IP, type of protocols detected (bitfields), offsets of IP, TCP, UDP headers etc. FMan CRC-64 hash result also available Avoids subsequent parsing in downstream modules CRC-64 hash result may be used to locate session in software hash table Classification of frame based on multiple schemas IPsec, TCP/UDP, Default Soft parsing for IP Fragments Ingress Packet Parsing Example Application metadata space Per-packet state sharing, flags, temporary allocations, packet chains Annotation Area Parse results bit vector Offsets to headers Layer 2 Frame Buffer from BMan Crc-64 hash result Extracted N-tuple ESP/AH SPI Not available by default Via Soft Parser 7

8 Multiple Schemas Example ESP/AH (IPsec), TCP/UDP, IP Fragments, Default FMan computes an FQID based on N-tuples specified in schema, enqueues to FQ FQID computed against a BASE ID + Hash/Shift/Mask on selected fields in frame FQs are pre-created and placed into Channel/Work Queues during initialization Possible Optimizations Use FMan crc64 result (N bits) to lookup hash table Or use 1-to-1 mapping between FQ and Hash bucket FQ Context B field points to hash bucket Enable context stashing for FQ Hash bucket entry stashed to core Ingress Packet Classification Example IP Frag IPSec (AH/ESP) TCP/UDP Default Frame Queues (FQID Range) IPsec Hash Lookup TCP/UDP Hash Lookup 8

9 Some ingress QoS features may be offloaded to hardware FMan can also compute a policer profile ID 256 policer profiles supported (hardware) Multiple flows may share a single policer profile Policer supports a token-bucket algorithm to limit or color packets Two-rate three color marking Option to drop RED packets FMan enqueues frame to FQ (in QMan) if policer allows Congestion avoidance 256 Congestion groups in QMan Defined over FQs. A group of FQs may be part of one congestion group Tail Drop, RED, WRED algorithms Congestion state notification Policing and Congestion Avoidance FMan can generates pause frames on Ethernet i/f, on congestion notification from QMan 9

10 VortiQa Pool Channel Distribution Example All ingress FQs on one pool channel All DP cores dequeue from the same channel Benefits Even load balancing 100% core utilization at max load Problems Flow order preservation Cache-thrashing Need for locks Problem mitigation Hold FQ on portal till critical section over Only one core sees frame from a given flow at any time Avoids locking requirement Enable/Control stashing to cores Frame annotations Packet headers + relevant data Flow-related structure (FQ context A field) Heavy use of pre-fetching in cores Pre-fetch required state into caches before use Multiple pre-fetches throughout code execution 10

11 VortiQa Ingress Packet Distribution Example Pool Channels CP #1 FQ FQ FQ FQ CP # CP Pool Channel CP-DP DP #1 FQ FQ FMAN FQ FQ DP #2 DP Pool Channel DP #N One pool channel for all CP cores For CP->DP communication WQ 0 CP/DP messages WQ 6,7 CP data packets One pool channel for all DP cores WQ 0 DP/CP messages WQ 1 Configuration Traffic WQ 3 SEC / PME Traffic WQ 5 Timer Messages WQ 6,7 Data Traffic Additional Pool channel if DSCP based classification and priority is required. 11

12 Processing on Data plane Cores Prefetch of state information before use Use of Decorated Storage Instructions Statistics, other counters Fast fire-and-forget atomic operations Use of RCUs in-lieu of locks Use of FQs in lieu of shared software queues Avoids locking and minimizes contention Benefits from stashing Use of BMan buffer pools for software 64 pools available Separation of read and r/w variables into different cache-lines Code locality of reference Better instruction cache utilization Locking specific code, data in caches (L1, L2, L3) 12

13 Crypto and Look-aside Processing Asynchronous Protocol offload IPsec ESP, SSL, TLS, DTLS, SRTP, MACSec/LinkSec Other protocols i WiFi, WiMAX, 3G RLC PDU (*), LTE PCP U-Plane (*), LTE PDCP C-Plane (*) Asynchronous Crypto offload Public Key, DES, AES, RC4, Message Digests, RNG, PRF, CRC, Snow 3G, Kasumi Asynchronous PME engine Pattern matching (Regular expressions) on flows (10 Gbps) Stateful rule engine Decorated Storage Unit Atomic add/increment/decrement/read of 32-bit, 64-bit values Higher weightage for look-aside FQs FMan offline parsing ports Mac-less virtual FMan ports Offload Parse-classify-police-distribute operations For De-tunneled (e.g. IPsec) packets For Packets from other interfaces FMan also performs some basic protocol header integrity checks 13

14 QorIQ P4080 Processor Egress Channel per Tx port Scheduler Deficit Round Robin Strict Priority, Weighted Queues FMan Shaper per TX port Token bucket CIR, CBS Bits/sec, Frames/sec FMan Checksum Offload IP, UDP, TCP Offload basic QoS ToS/DSCP-based flow priorities E.g. Strict priority for voice traffic Smooth egress traffic Congestion State Notification Lockless egress queuing 14

15 Accelerators Capacity/Performance FMan Performance 18Mpps x 2 Units - Parse/Classify/Distribute QMan Performance and Capacity 21.5M enqueues/sec, 21.5 M dequeues/sec from software portals Total: 100.5M enqueue/dequeue / sec 51 channels (15 pool channels, 10 dedicated channels) 8 work queues per channel 408 work queues, 16M frame queues 256 policer profiles, 256 congestion groups Tail drop on an FQ does not require a group SEC Performance 10 Gbps symmetric encryption/integrity algorithms 10, b RSA public key operations/sec PME Performance and Capacity 10 Gbps scanning performance 32,000 regular expression patterns, 32,000 stateful rules 15

16