QorIQ Based Multicore LTE Layer 2 Software

Transcription

1 July 2009 QorIQ Based Multicore LTE Layer 2 Software Keith Shields

2 AC CELE R ATIO N CORENET FABRIC Freescale LTE System Enablement Overview: Software; Devices; AMC boards CONTROL RF PA RF RF PA PA RF Small RF Sign. Small RF Sign. Small Sign. LTE NODE B U/D Conv U/D. Conv U/D. Conv Conv. FPGA PHYSICAL/DSP L2/MAC NETWORK IF LTE PHY Software 3GPP Rel 8 -LTE LTE L2 Software 3GPP Rel 8 -LTE Physical Processing X1 -S2 Software Reference Software Reference Transport Processing MAC/RLC e e e e e e e500 e500 e500 e500 CORENET FABRIC FABRIC FABRIC SYSTEM Acceleration FUNCTIONS Pattern LTE Support Matching MEMORY CONTROL Decompression / WIMAX Compression Support L3 M3 Cache Viterbi Crypto Decode Security Table Turbo CONNECTIVITY Table Lookups Decode 10/100/1000 PCI Express QUICC Data Data Path Resource Ethernet RapidIO FFT/IFFT Engine Management etsec etc. ACCELERATION Pattern Matching Decompression / Compression Crypto Security Table Lookups Data Path Resource Management CORENET FABRIC SYSTEM FUNCTIONS MEMORY CONTROL L3 Cache CONNECTIVITY 10/100/1000 PCI Express QUICC Ethernet RapidIO Engine etsec etc. MSC8156 P20/30/40x Enablement code developed to date is deployed in customer systems

3 P4080 Multicore Architecture P Kbyte Backside e500-mc 1024-Kbyte Frontside L3 Cache 32-Kbyte 32-Kbyte 1024-Kbyte Frontside L3 Cache 64-bit DDR-2 / 3 Memory Controller 64-bit DDR-2 / 3 Memory Controller eopenpic PreBoot Loader Security Monitor Internal BootROM PAMU PAMU PAMU Net Coherency Fabric PAMU PAMU Peripheral Access Mgmt Unit Power Mgmt SD/MMC SPI elbiu M2SB Security 4.0 Queue Mgr. Frame Manager Parse, Classify, Distribute Frame Manager Parse, Classify, Distribute SRIO Message Unit DMA Real Time Debug Watchpoint Cross Trigger DUART 2x I 2C 2x USB 2.0/ULPI Test Port/ SAP Pattern Match Engine 2.0 Buffer Mgr. 10GE Buffer 1GE 1GE 1GE 1GE 10GE Buffer 1GE 1GE 1GE 1GE PCIe PCIe SRIO PCIe SRIO Perf Monitor Net Trace Aurora Clocks/Reset GPIO CCSR 18-Lane 5GHz SERDES

4 Accelerating Customer Wireless QorIQ Development It s All About the Software Millions of lines of legacy LTE code need to be written in a parallel fashion to best utilize multicore devices How to efficiently partition and run complex enodeb functionality across a multi-core system SMP vs AMP vs LWE Scheduler, MAC, RLC, PDCP, IPSEC, GTP, SCTP.. Demonstrate Efficient and LTE SPECIFIC DPAA benefits Prove FSL QorIQ performance for LTE use-cases Understand and remove system bottlenecks Component benchmarks not sufficient, need full system functionality Single-threaded Legacy Software MPC8548 Multicore Software P4080 x P2x

5 LTE Overview

6 3G Evolution and Architecture Radio Side (LTE Long Term Evolution) Improvements in spectral efficiency, user throughput, latency Simplification of the radio network Efficient support of packet based services Network Side (SAE System Architecture Evolution) Improvement in latency, capacity, throughput Simplification of the core network Optimization for IP traffic and services Simplified support and handover to non-3gpp access technologies

7 LTE Protocol Stacks Application IP PDCP RLC Relay PDCP RLC GTP-U UDP GTP-U UDP Relay GTP-U UDP IP GTP-U UDP User Plane IP/IPSec IP/IPSec IP/IPSec IP/IPSec MAC MAC L2 L2 L2 L2 L1 L1 L1 L1 L1 L1 UE LTE-Uu S1-U S5/S8 SGi enb Serving GW PDN GW NAS RRC PDCP RRC PDCP Relay S1-AP SCTP NAS S1-AP SCTP Control Plane RLC RLC IP/IPSec IP/IPSec MAC MAC L2 L2 L1 L1 L1 L1 LTE-Uu enb S1-MME MME

8 The main services and functions of PDCP for the user plane include: Header compression and decompression: ROHC Transfer of user data between RRC and RLC layers Ciphering The main services and functions of PDCP for the control plane include: Ciphering and Integrity Protection Transfer of control plane data between RRC and RLC layers. PDCP Overview IP Header Data ROHC Data Cipher Data PDCP Header Data PDCP Header + Checksum CK

9 Protocol Stack Flow #1 Data from Network ( ) ROHC PDCP Cipher (X%X$x) R R R Data PDCP SN Oct 1 Oct 2 RLC... MAC-I MAC-I (cont.) Oct N-3 Oct N-2 RLC Header Data MAC-I (cont.) Oct N-1 MAC-I (cont.) Oct N To MAC

10 Protocol Stack Flow #2 RLC MAC E LI 2 D/C RF P FI E SN SN E LI 1 LI 1 LI 2 E LI K-1 LI K-1 E LI K LI K Data Oct 1 Oct 2 Oct 3 Oct 4 Oct 5 Oct [2+1.5*K-2] Oct [2+1.5*K-1] Oct [2+1.5*K] Oct [2+1.5*K+1] Oct N MAC Header Data To Layer 1 Phy

11 Packet Network LTE Data Flow Layer 3 PS NAS User Traffic RRC RRC PDUs Layer 2 PDCP Ctrl PDCP RLC Ctrl PDCP PDUs RLC Radio Bearers Layer 1 Physical Channels RLC PDUs MAC Ctrl MAC L1 Ctrl MAC PDUs PHY (DL-OFDM, UL-SC-FDMA) Logical Channels Transport Channels :20

12 Packet Network LTE Data Flow Layer 3 PS NAS User Traffic RRC RRC PDUs Layer 2 PDCP Ctrl PDCP RLC Ctrl PDCP PDUs RLC Radio Bearers Downlink Flow Layer 1 Physical Channels RLC PDUs MAC Ctrl MAC L1 Ctrl MAC PDUs PHY (DL-OFDM, UL-SC-FDMA) Logical Channels Transport Channels :20

13 Freescale LTE Layer 2 Solution Overview Software Support

14 Category RTOS Support API Validation/Test 1.MAC - Medium Access Control Layer 2. RLC - Radio Link Control Freescale LTE Layer 2 Software Deliverables Specification / Features RTOS agnostic implementation Example includes software ported to Linux user mode Full software abstraction between data plane & control plane and data plane & scheduler through well-defined and documented APIs. Software tested on unit, integration and system levels Software test environment is part of the software delivery package. Compatible to standard: 3GPP (MAC) (V.8.3.0) Includes sample downlink / uplink scheduler Compatible to standard: 3GPP (RLC) (V.8.3.0) 3. L2/L1 interface Implements an efficient L2/L1 interface designed for seamless integration with Freescale L1 solution (available today) Easy L2/L1 interface, out-of-the-box experience through validated test cases.(over srio) 4. Framework Example integrated processing chain running under Linux (available today): o Demonstrates integration of L2 modules o Provides known development/test environment 5. PDCP - Packet Data Convergen ce Protocol 6.IPSEC 7.RRC Header ROHC & encryption Full header implementation (including HO) available now RoHC - Available through third party (available today) Provided with specific optimisations for Freescale architectures Air interface encryption (available today) Algorithm implemented on SEC Engine Freescale Software (FastpathU), formerly Intoto 3 rd party or customer development

15 LTE Layer 2 Software Components PDCP Control Scheduler RLC MAC IF1 Common Harness / Operating System - Harness & Operating System: System level test harness utilizing operating system timers and ethernet stack - Scheduler: Priority based round-robin scheduler - Control: API to facilitate configuration and execution of core modules - PDCP: Packet Data Convergence Protocol as specified by 3GPP utilises 3 rd party ROHC implementation - RLC: Radio Link Control as specified by 3GPP MAC: Medium Access Control as specified by 3GPP Common: Generic functionality utilised by multiple modules e.g. linked list implementation, TTI event Timers etc. - IF1 interface: Covers the protocol and LTE specific aspects of the DSP L1 interface

16 Development Tools KDBG Insight DDD GDB LTE L2 Development Environment Target Platforms Controller AMCs 8548/8572/P2x/P4xAMCs Metrowerks Standard C Codebase GCC Binutils Run-Time Environment 85xx x86 x86 Linux Linux Cygwin DSP AMCs 8144/8156AMCs Industry Standard Carriers Pico/Micro TCA Proprietary Systems UBOOT BIOS BIOS

17 Freescale LTE Layer 2 QorIQ Solution Overview Software Support

18 LTE QorIQ OS Implementation Multicore CPU's can permit a number of processing scenarios Services Forwarding/ Data Plane Control Plane Linux AMP Linux Light Weight Linux Executive AMP SMP We are benchmarking our LTE Layer 2 implementation to determine the optimal mix and fit for QorIQ architectures

19 Multicore Simulation Environment Applications SMP/AMP Operating System Optimized High-Speed Drivers Hypervisor Freescale API Functional Cycle-Accurate IDE (compiler / debugger / build tools) Applications SMP/AMP Operating System Optimized High Speed Drivers Hypervisor Freescale Multicore Silicon Simulation to Hardware Same Software Freescale-supplied SDK items

20 P4080 Datapath infrastructure blocks: Queue and Buffer managers P Kbyte Backside e500-mc 1024-Kbyte Frontside L3 Cache 32-Kbyte 32-Kbyte 1024-Kbyte Frontside L3 Cache 64-bit DDR-2 / 3 Memory Controller 64-bit DDR-2 / 3 Memory Controller eopenpic PreBoot Loader Security Monitor Internal BootROM PAMU PAMU PAMU Net Coherency Fabric PAMU PAMU Peripheral Access Mgmt Unit Power Mgmt SD/MMC SPI elbiu M2SB Security 4.0 Queue Mgr. Frame Manager Parse, Classify, Distribute Frame Manager Parse, Classify, Distribute SRIO Message Unit DMA Real Time Debug Watchpoint Cross Trigger DUART 2x I 2C 2x USB 2.0/ULPI Test Port/ SAP Pattern Match Engine 2.0 Buffer Mgr. 10GE Buffer 1GE 1GE 1GE 1GE 10GE Buffer 1GE 1GE 1GE 1GE PCIe PCIe SRIO PCIe SRIO Perf Monitor Net Trace Aurora Clocks/Reset GPIO CCSR 18-Lane 5GHz SERDES

21 Linux Control Partition Control Path DPAA Pool Channel Implementation Lightweight Executive PDCP/RLC/MAC/Sch Doorbells Shared Memory INBAND Via FQ s QMAN Dedicated Channel portal portal QMAN Pool Channel portal channel channel channel WQ7 WQ6 WQ5 WQ4 WQ3 WQ2 WQ1 WQ0 WQ7 WQ6 WQ5 WQ4 WQ3 WQ2 WQ1 WQ0 WQ7 WQ6 WQ5 WQ4 WQ3 WQ2 WQ1 WQ0 Frame Queues Classify

22 Fixed Partitioning IPSEC PDCP MAC/RLC Scheduler Scheduler Scheduler SMP Linux SMP Linux #8 #1 Small fixed function LWE kernel with high icache hit ratio Linux LWE RRC PDCP RLC MAC PHY GTP UDP Multicore Software Partitioning Sch IPSEC IP L2/L1 MAC RLC PDCP IPSEC SCTP TCP SMP Linux SMP Linux enodeb Dynamic Load Balancing Single LWE App SMP Linux SMP Linux Large Multifunction kernel with lower icache hit ratio but with dynamic load balancing 22

23 Traditional LTE L2 Downlink vs. DPAA solution IPSEC DE/COMPRESSION Reduces Payload & increases throughput ROHC GTP Traditional Implementation (Linux SMP) Potential Buffer Copy Synchronization/lock Point DE/CIPHERING Provides security INTEGRITY Ensures data is relevant & non-corrupt DE/CIPHER INTEGRITY SCHEDULER PDCP RLC MAC DPAA LWE run to completion implementation (objective: offload e500mc cores) Buffer copies are replaced by DPAA enqueue / dequeue operations Locking/synchronization is handled in hardware. IF1 L1 UE

24 System Partitioning / Initialization DPAA partitioning and initialization is driven by the Linux device tree which is passed to both the hypervisor and guest OS s on startup. The device tree is Power.org epapr compliant with extensions to support the new DPAA features. The device tree details Partitioning of cores i.e., Linux/LWE. Physical Memory Areas. Allocation of all Phyiscal resources eg. Network ports, serial ports etc Portals BMan Pools FQ Allocation etc The Hypervisor parses the device tree and allocates resources as required.

25 In order arrival of 3 packets P1 ODP/ORP PDCP Example Packet order is preserved ODP ORP P1 P2 P1 P2 P2 P2 processing time is greater than P1 or Parallel Processing Elements s is held until P2 is processed through the ORP

26 In order arrival of 3 packets P1 ODP/ORP PDCP Example Packet order is preserved ODP ORP P1 P2 P1 P2 P2 Parallel Processing Elements s

27 P1 ODP/ORP PDCP Example In order arrival of 3 packets ODP P2 P1 P2 ORP Packet order is preserved P1 P2 Parallel Processing Elements s

28 In order arrival of 3 packets P1 ODP/ORP PDCP Example Packet order is preserved ODP ORP P1 P1 P2 P2 P2 Parallel Processing Elements s

29 In order arrival of 3 packets P1 ODP/ORP PDCP Example Packet order is preserved ODP ORP P1 P1 P2 P2 P2 P2 processing time is greater than P1 or Parallel Processing Elements s is held until P2 is processed through the ORP

30 In order arrival of 3 packets P1 ODP/ORP PDCP Example Packet order is preserved ODP ORP P1 P1 P2 P2 P2 Parallel Processing Elements s

31 QMan Software Portal components Dequeue Interface Enqueue Interface N s CI CI QMan Interrupts PI Dequeue Response Ring (DQRR) Dequeue Commands Enqueue Command Ring (EQCR) PI CI PI Message Ring (MR) Management Command/Response Registers Software portals have 4 components Dequeue: Command registers + DQRR Enqueue: EQCR Messages: MR Asynchronous error messages (e.g. enqueue rejections) Management commands: command/response registers Interrupts can be used to signal availability of data or space (in EQCR) Rings provide finite size FIFOs Up to 16 entries for DQRR, 8 entries for EQCR and MR Portal components are implemented inside QMan to reduce access latency Unlike traditional BD rings which are in memory and registers QMan can push (stash) DQRR entries across net into the appropriate core s cache PI and CI are the basic mechanisms used with rings but other forms of notification of data availability and data consumption are supported When these other mechanisms are used QMan maintains PI/CI

32 PDCP QMan Stashing Example Dequeue Interface QMan can stash DQRR entries across net into the appropriate core s cache. The stash size 0 -> 3 Cache lines ( 64 bytes) can be set for the following components on FQ creation. Frame Data Actual Packet Data CI PI Interrupts Dequeue Response Ring (DQRR) Dequeue Commands QMan FQ Context Per Queue Context Data ie PDCP user context, Sequence Numbers, ROHC context, Cipher keys etc. Frame Annotation Per Packet Context Data ie Mapping of PDCP Bearer ID to internal structures.

33 Example DPAA main() for FQ Creation and Dequeue Main() {.... Init.. fq = qm_new_fq(g_qm_portal, fq, channel, priority, pdcp_dl_context, 0, 0, 0, MT_SHARED, 0);.. Frame Queue Creation sets the system connectivity. FQ s have a number of attributes which determine run time behaviour ie HELD ACTIVE, ORP/ODP while (1) { if (entry = qm_dq_dqrr_entry(g_qm_portal)) { context = (struct lte_context *)(entry->contextb); context->handler(context, entry); qm_dqrr_cci_consume(g_qm_portal->p, 1); } else { idle_loop(); } } } Processing is data driven - By the time data arrives at the core we do not need to parse channel, priority, FQ as data processing is driven by context

34 Comparison of example LTE Data handler() LWE Run to Completion approach void pdcp_dl(struct pdcp_dl_context_t *context, struct qm_dqrr_entry *entry) { SBL2_BUFFER_T buffer; struct qm_fd fd; buffer.length = fd.length20 - ETH_HLEN; buffer.data = (uint8_t *) ptov_dpa(fd.addr_lo); buffer.offset = fd.offset + ETH_HLEN; if (SBL2_PDCP_DL_DataTransfer(&(context->bearer), &buffer)) { fd.length20 = buffer.length; fd.offset = buffer.offset; qman_enqueue_performance(g_qm_portal, RLC_CHANNEL, &fd, 0, 0); } else { printf("pdcp DL Drop Packet.\n"); } P4080 DPAA based PDCP Handler Abstraction of the bman buffer to SBL2_BUFFER_T type allows the same SBL2_PDCP_DL_DataTransfer code to be reused for Linux/SMPLinux/LWE/RTOS void pdcp_dl(uint16 bearer_id, UINT8 *sdu, UINT16 sdu_length) { SBL2_PDCP_RADIO_BEARER_T *bearer = &SBL2_PDCP->raidio_bearer[bearer_id]); Linux SMP approach pdu = SBL2_GetBuffer(); LOCK(&(bearer->mutex); MEMCPY(sdu,pdu, sdu_length); SBL2_PDCP_DL_DataTransfer(bearer, &pdu, sdu_length); UNLOCK(&(bearer->mutex); } Linux SMP based PDCP Handler Although simpler the original code has a buffer allocation/copy and mutex lock around the data processing

35 LTE Layer 2 Summary

36 LTE Layer 2 Code Summary Evolving multicore code set derived from mature single core base. Code developed to date has the benefit of deployment feedback. Robust software development and management process established. Simulation environment facilitating early code development. Multicore code generation is underway.

37 Q&A Thank you for attending this presentation. We ll now take a few moments for the audience s questions and then we ll begin the question and answer session.

38