QorIQ multicore communications platform software introduction Boot loader bring-up Linux bring-up Q&A

Transcription

1 Aug Freescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, t he Energy Efficient Solutions logo, mobilegt, PowerQUICC, QorIQ, StarCore and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, ColdFire+, CoreNet, Flexis, Kinetis, MXC, Platform in a Package, Processor Expert, QorIQ Qonverge, Qorivva, QUICC Engine, SMAROS, TurboLink, VortiQa and Xtrinsic are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners Freescale Semiconductor, Inc.

2 QorIQ multicore communications platform software introduction Boot loader bring-up Linux bring-up Q&A Freescale on Kaixin Tag yourself in photos and upload your own! Weibo? Please use hashtag #FTF Package, Processor Expert, QorIQ Qonverge, Qorivva, QUICC Engine, SMAROS, TurboLink, VortiQa and Xtrinsic are trademarks of Freescale Semiconductor, Inc.

3 P Kbyte Backside L2 Cache Power Architecture e500-mc Core 1024-Kbyte Frontside L3 Cache 32-Kbyte 32-Kbyte 1024-Kbyte D-Cache I-Cache Frontside L3 Cache 64-bit DDR-2 / 3 Memory Controller 64-bit DDR-2 / 3 Memory Controller eopenpic PreBoot Loader Security Monitor Internal BootROM PAMU PAMU PAMU CoreNet Coherency Fabric PAMU PAMU Peripheral Access Mgmt Unit Power Mgmt SD/MMC SPI elbiu M2SB Security 4.0 Queue Mgr. Frame Manager Parse, Classify, Distribute Frame Manager Parse, Classify, Distribute SRIO Message Unit DMA Real Time Debug Watchpoint Cross Trigger DUART 2x I 2C 2x USB 2.0/ULPI Test Port/ SAP Pattern Match Engine 2.0 Buffer Mgr. 10GE Buffer 1GE 1GE 1GE 1GE 10GE Buffer 1GE 1GE 1GE 1GE PCIe PCIe SRIO PCIe SRIO Perf Monitor CoreNet Trace Aurora Clocks/Reset GPIO CCSR 18-Lane 5 GHz SERDES 3 Package, Processor Expert, QorIQ Qonverge, Qorivva, QUICC Engine, SMAROS, TurboLink, VortiQa and Xtrinsic are trademarks of Freescale Semiconductor, Inc.

4 Multicore Applications Communications Stacks And APIs Run Time Options Linux 3 rd Party Proprietary Light Weight Exec (Data Path Acceleration) Optimized Board Support Package Virtualization/Hypervisor Cores Accelerators Peripherals Multicore Applications: SMP and AMP programming models Component model for scalability Communication Stacks/APIs: Silicon optimized Open and scalable Linux: Control plane processing SMP support User space DPAA Light Weight Executive: Data path acceleration library Run to completion BSP: Silicon optimized Full featured Open source Hypervisor: Security among cores Messaging among cores System-level event handling Debug support 4 Package, Processor Expert, QorIQ Qonverge, Qorivva, QUICC Engine, SMAROS, TurboLink, VortiQa and Xtrinsic are trademarks of Freescale Semiconductor, Inc.

5 Expect to see more Linux if only because it is hard to make money selling and supporting a proprietary RTOS Linux is not perfect, but has a huge and flexible feature set Steady improvement in real time Scheduler gets more flexible and supports affinity Linux commercial support trends are interesting Maybe the open source model wins for the OS foundation. Give a little, take a lot cooperation on commodity aspects Expect to see more use of affinity because of hardware trends and to make performance more predictable This is already an interesting model USDPAA 5 Package, Processor Expert, QorIQ Qonverge, Qorivva, QUICC Engine, SMAROS, TurboLink, VortiQa and Xtrinsic are trademarks of Freescale Semiconductor, Inc.

6 Basic Features SMP and AMP Virtualization: HyperVisor and KVM 36-bit and 64-bit Ethernet: etsec, VeTSEC, DPAA, USDPAA Connection: PCIe, srio, USB Storage: DDR3, NOR, NAND, esdhc/mmc, SATA, RAID5 Power management: Sleep, Deep Sleep, Jog Specific: elbc, GPIO User Space: USDPAA Performance IPForward IPSec NAS 6 Package, Processor Expert, QorIQ Qonverge, Qorivva, QUICC Engine, SMAROS, TurboLink, VortiQa and Xtrinsic are trademarks of Freescale Semiconductor, Inc.

7 Board Support Package (BSP) GNU Toolchain U-Boot Boot Loader Linux 2.6 kernel Embedded Hypervisor Software Light Weight Executive (LWE) LWE Example Applications (LWE Apps) Good starting point for Linux porting 7 Package, Processor Expert, QorIQ Qonverge, Qorivva, QUICC Engine, SMAROS, TurboLink, VortiQa and Xtrinsic are trademarks of Freescale Semiconductor, Inc.

8 Master u-boot repository Master Linux repository SDK release code=cw_bsp Follow instructions in the BSP documentation to extract u-boot package P4080 as example 8 Package, Processor Expert, QorIQ Qonverge, Qorivva, QUICC Engine, SMAROS, TurboLink, VortiQa and Xtrinsic are trademarks of Freescale Semiconductor, Inc.

9 What is U-Boot? Open source firmware for hardware platforms Widely used in embedded systems PowerPC, ARM, DSP What does U-Boot do? Initializes bare board Loads OS Formal U-boot website DENX Software Engineering More at Mail List 9 Package, Processor Expert, QorIQ Qonverge, Qorivva, QUICC Engine, SMAROS, TurboLink, VortiQa and Xtrinsic are trademarks of Freescale Semiconductor, Inc.

10 ./u-boot board common arch disk drivers include lib net doc tpl examples fs nand_spl tools post Relevant directories for modifications u-boot/. u-boot/board u-boot/common u-boot/include u-boot/arch/powerpc u-boot/nand_spl 10 Package, Processor Expert, QorIQ Qonverge, Qorivva, QUICC Engine, SMAROS, TurboLink, VortiQa and Xtrinsic are trademarks of Freescale Semiconductor, Inc.

11 Reset vector points to FLASH (0xfffffffc) Begins execution in FLASH Initializes DUARTs Checks for information on version numbers, clocks Initializes DDR Relocates itself to RAM Continues execution in RAM Releases other cores Initializes higher level devices Ethernet, PCIe, SATA, USB, etc. Begins command interpreter 11 Package, Processor Expert, QorIQ Qonverge, Qorivva, QUICC Engine, SMAROS, TurboLink, VortiQa and Xtrinsic are trademarks of Freescale Semiconductor, Inc.

12 Range IP Size 0x0000_0000 0x7fff_ffff DDR 2G Cacheable 0x8000_0000 0x9fff_ffff PEX 1 Mem 512M non-cacheable 0xa000_0000 0xbfff_ffff PEX 2/SRIO Mem 512M non-cacheable 0xc000_0000 0xdfff_ffff PEX 3 Mem 512M non-cacheable 0xf800_0000 0xf802_ffff PCI IO range 256K non-cacheable 0xf400_0000 0xf41f_ffff Bman 2M 0xf420_0000 0xf43f_ffff Qman 2M Localbus non-cacheable 0xe000_0000 0xe80f_ffff Promjet/free 128M non-cacheable 0xe800_0000 0xefff_ffff FLASH 128M non-cacheable 0xffdf_0000 0xffdf_7fff PIXIS 32K non-cacheable TLB0 0xffd0_0000 0xffd0_3fff L1 for stack 16K Cacheable TLB0 0xfe00_0000 0xfeff_ffff CCSR 16M non-cacheable 12 Package, Processor Expert, QorIQ Qonverge, Qorivva, QUICC Engine, SMAROS, TurboLink, VortiQa and Xtrinsic are trademarks of Freescale Semiconductor, Inc.

13 start arch/powerpc/cpu/mpc85xx/start.s MMU, LAW setup CPU, Clocks Serial DDR Misc CPU init Relocate to DDR Relocate to RAM space Flash PCIe Misc init/mp init Network init 13 Package, Processor Expert, QorIQ Qonverge, Qorivva, QUICC Engine, SMAROS, TurboLink, VortiQa and Xtrinsic are trademarks of Freescale Semiconductor, Inc.

14 Drivers written in C code Need stack space to call function Lock the cache for stack Unlock before linux boot * 0xe800_0000 0xefff_ffff FLASH 128M non-cacheable * 0xffdf_0000 0xffdf_7fff PIXIS 32K non-cacheable TLB0 * * 0xffd0_0000 0xffd0_3fff L1 for stack 16K Cacheable TLB0 * 0xffe0_0000 0xffef_ffff CCSR 1M non-cacheable Files: arch/powerpc/cpu/mpc85xx/start.s 14 Package, Processor Expert, QorIQ Qonverge, Qorivva, QUICC Engine, SMAROS, TurboLink, VortiQa and Xtrinsic are trademarks of Freescale Semiconductor, Inc.

15 Configuring memory map Files include/configs/p4080ds.h, corenet_ds.h board/freescale/corenet_ds/tlb.c, law.c cpu/mpc85xx/tlb.c drivers/misc/fsl_law.c include/asm-ppc/immap_85xx.h Code snippets: /* *I*G* - Flash, localbus */ /* This will be changed to *I*G* after relocation to RAM. */ SET_TLB_ENTRY(1, CONFIG_SYS_FLASH_BASE, CONFIG_SYS_FLASH_BASE_PHYS, MAS3_SX MAS3_SR, MAS2_W MAS2_G, 0, 2, BOOKE_PAGESZ_256M, 1), SET_LAW(CONFIG_SYS_FLASH_BASE_PHYS, LAW_SIZE_256M, LAW_TRGT_IF_LBC), 15 Package, Processor Expert, QorIQ Qonverge, Qorivva, QUICC Engine, SMAROS, TurboLink, VortiQa and Xtrinsic are trademarks of Freescale Semiconductor, Inc.

16 SYSCLK value needed to get started Platform frequency FPGA registers SCLK0, 1, 2 Input to ICS307 clock generator code Obtain the board and ddr clocks Two DUARTs (four UARTs) on P4080 Two serial outputs on DS u-boot provides driver for NS16550 compatible DUARTs Needs platform frequency Serial port very useful for early debug Printf for debugging 16 Package, Processor Expert, QorIQ Qonverge, Qorivva, QUICC Engine, SMAROS, TurboLink, VortiQa and Xtrinsic are trademarks of Freescale Semiconductor, Inc.

17 Relocating code Supports multiple controllers and SPD-based (I 2 C EEPROM) initialization Supports interleaving Files: arch/powerpc/cpu/mpc8xxx/ddr-gen3.c include/configs/corenet_ds.h board/freescale/corenet_ds/ddr.c board specific parameters Code snippets: #define CONFIG_NUM_DDR_CONTROLLERS 2 #define CONFIG_DIMM_SLOTS_PER_CTLR 1 #define CONFIG_CHIP_SELECTS_PER_CTRL (2 * CONFIG_DIMM_SLOTS_PER_CTLR) #define CONFIG_DDR_SPD #define CONFIG_FSL_DDR3 #define CONFIG_SYS_SPD_BUS_NUM 1 #define SPD_EEPROM_ADDRESS1 0x51 #define SPD_EEPROM_ADDRESS2 0x52 Memory Controller 1 Memory Controller 2 CS0 CS1 CS0 CS1 17 Package, Processor Expert, QorIQ Qonverge, Qorivva, QUICC Engine, SMAROS, TurboLink, VortiQa and Xtrinsic are trademarks of Freescale Semiconductor, Inc.

18 Set up local bus memory map BRn, ORn Enable timebase register Files: arch/powerpc/cpu/mpc85xx/cpu_init.c Code snippets: void cpu_init_f (void) { volatile ccsr_lbc_t *memctl = (void *)(CONFIG_SYS_MPC85xx_LBC_ADDR); : : #ifdef CONFIG_FSL_CORENET #endif corenet_tb_init(); init_used_tlb_cams(); } /* Invalidate the CPC before DDR gets enabled */ invalidate_cpc(); 18 Package, Processor Expert, QorIQ Qonverge, Qorivva, QUICC Engine, SMAROS, TurboLink, VortiQa and Xtrinsic are trademarks of Freescale Semiconductor, Inc.

19 Relocates u-boot code to top of memory For 2 GB on P4080DS, in the last 1 MB Set up of stack space, etc. Continues to run from DDR after relocation Files: arch/powerpc/cpu/mpc85xx/start.s arch/powerpc/lib/board.c Code snippets: /* * void relocate_code (addr_sp, gd, addr_moni) * * This "function" does not return, instead it continues in RAM * after relocating the monitor code. * * r3 = dest * r4 = src * r5 = length in bytes * r6 = cachelinesize */.globl relocate_code relocate_code: mr r1,r3 /* Set new stack pointer */ mr r9,r4 /* Save copy of Init Data pointer */ mr r10,r5 /* Save copy of Destination Address */ 19 Package, Processor Expert, QorIQ Qonverge, Qorivva, QUICC Engine, SMAROS, TurboLink, VortiQa and Xtrinsic are trademarks of Freescale Semiconductor, Inc.

20 CFI flash support Reads on POR Sets up structures much later in the driver * Localbus non-cacheable * * 0xe000_0000 0xe80f_ffff Promjet/free 128M non-cacheable * 0xe800_0000 0xefff_ffff FLASH 128M non-cacheable * 0xffdf_0000 0xffdf_7fff PIXIS 32K non-cacheable TLB0 * Files: include/configs/corenet_ds.h drivers/mtd/cfi_flash.c Code Snippets: This information is used by the driver to detect the flash using CFI query at both possible addresses and populate its info table #define CONFIG_FLASH_CFI_DRIVER #define CONFIG_SYS_FLASH_CFI #define CONFIG_SYS_FLASH_AMD_CHECK_DQ7 #define CONFIG_SYS_FLASH_BANKS_LIST {CONFIG_SYS_FLASH_BASE_PHYS + 0x , CONFIG_SYS_FLASH_BASE_PHYS} /* Protect RCW and FMAN microcode sectors for the NOR flash boot bank */ #define CONFIG_SYS_FLASH_AUTOPROTECT_LIST { {CONFIG_SYS_FLASH_BASE + 0x , 1}, \ {CONFIG_SYS_FMAN_FW_ADDR, 1} } 20 Package, Processor Expert, QorIQ Qonverge, Qorivva, QUICC Engine, SMAROS, TurboLink, VortiQa and Xtrinsic are trademarks of Freescale Semiconductor, Inc.

21 Check for PCIe interfaces on the SOC Basic PCIe enumeration Useful when pcie devices are needed at boot time e.g. RTC/NVRAM on nvidia 1575 /* * 0x8000_0000 0x9fff_ffff PEX 1 Mem 512M non-cacheable * 0xa000_0000 0xbfff_ffff PEX 2 Mem 512M non-cacheable * 0xc000_0000 0xdfff_ffff PEX 3 Mem 512M non-cacheable Files: include/configs/corenet_ds.h drivers/pci/fsl_pci_init.c board/freescale/corenet_ds/corenet_ds.c Code Snippets: void fsl_pci_init(struct pci_controller *hose) { : : } #define CONFIG_SYS_PCIE1_ADDR (CONFIG_SYS_CCSRBAR+0x200000) #define CONFIG_SYS_PCIE2_ADDR (CONFIG_SYS_CCSRBAR+0x201000) #define CONFIG_SYS_PCIE3_ADDR (CONFIG_SYS_CCSRBAR+0x202000) : : #define CONFIG_SYS_PCIE3_MEM_PHYS 0xc #define CONFIG_SYS_PCIE3_IO_PHYS 0xf Package, Processor Expert, QorIQ Qonverge, Qorivva, QUICC Engine, SMAROS, TurboLink, VortiQa and Xtrinsic are trademarks of Freescale Semiconductor, Inc.

22 Power-on Reset Core0 comes out of reset at the reset vector 0xFFFF_FFFC and all other cores are in boot hold off mode Core0 runs u-boot After setting BSTRL/BSTAR to secondary core s start page, core0 kicks off each additional secondary core Each secondary core comes out of reset at secondary_start_page (boot page), initialize resources specific that that core (caches, MMU, etc.) and enters a spin loop Core0 loads Linux image and device tree and boots Linux 22 Package, Processor Expert, QorIQ Qonverge, Qorivva, QUICC Engine, SMAROS, TurboLink, VortiQa and Xtrinsic are trademarks of Freescale Semiconductor, Inc.

23 epapr describes specifics on how secondary CPUs are booted for a system with multiple CPUs Default boot architecture The boot program releases all CPUs from hardware reset One CPU is designated to be the client program s boot CPU All other CPUs are secondary and are placed into loop where the CPUs spin, waiting for a spin table field to change that directs them where to go Control is transferred to the client program on the boot CPU When the client program is ready for secondary cores to start, it releases them by writing the spin table field with the desired address The architecture allows for other custom-defined secondary CPU release mechanisms as well Reference: epapr (Power.org Standard for Embedded Power Architecture Platform Requirements (epapr). power.org, 2008.) 23 Package, Processor Expert, QorIQ Qonverge, Qorivva, QUICC Engine, SMAROS, TurboLink, VortiQa and Xtrinsic are trademarks of Freescale Semiconductor, Inc.

24 In the u-boot source, the entry point is the reset vector 0x0_FFFF_FFFC containing a simple branch to _start_e500, which is at the base of default boot page at 0x0_FFFF_F000; both are found in cpu/mpc85xx/start.s Reset vector _start_e500() will: _start_e500() Boot other cores call _start_e500() Start command interpreter Prepare device trees 1. Enable L1 caches 2. Config interrupt vectors 3. Config MMU and LAWs 4. Config L1D RAM 5. Config local bus 6. Config DDR 7. Relocate to DDR 1. Setup BSTRL and BSTAR 2. Release all other cores from boot holdoff using BRR and wait for them to boot. cpu/mpc85xx/mp.c 1. Copy HW device tree to DDR 2. Copy HV config tree to DDR 3. Copy hv.uimage to DDR 4. Launch hypervisor 24 Package, Processor Expert, QorIQ Qonverge, Qorivva, QUICC Engine, SMAROS, TurboLink, VortiQa and Xtrinsic are trademarks of Freescale Semiconductor, Inc.

25 PA10 At this point you can do source-level debugging and set breakpoints in start.s until the address-space switch at the rfi before switch_as: (start.s, line 305) 25 Package, Processor Expert, QorIQ Qonverge, Qorivva, QUICC Engine, SMAROS, TurboLink, VortiQa and Xtrinsic are trademarks of Freescale Semiconductor, Inc.

26 #define DEBUG printf 26 Package, Processor Expert, QorIQ Qonverge, Qorivva, QUICC Engine, SMAROS, TurboLink, VortiQa and Xtrinsic are trademarks of Freescale Semiconductor, Inc.

27 Linux on QorIQ P4080 Processor: Overview P4080 BSP Linux: Bare metal and guest OS Linux kernel configuration Device tree SMP boot process DPAA and the Linux Ethernet Driver DPAA components Life of a packet Ingress Egress Linux driver structure 27 Package, Processor Expert, QorIQ Qonverge, Qorivva, QUICC Engine, SMAROS, TurboLink, VortiQa and Xtrinsic are trademarks of Freescale Semiconductor, Inc.

28 Formal Linux Kernel website Where to download kernel git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git Linux kernel mail list Package, Processor Expert, QorIQ Qonverge, Qorivva, QUICC Engine, SMAROS, TurboLink, VortiQa and Xtrinsic are trademarks of Freescale Semiconductor, Inc.

29 The easiest way to create a.config file is to simply copy the appropriate config file in./arch/powerpc/configs to.config cp arch/powerpc/configs/p4080_ds_defconfig.config Typing make menuconfig results in a console-based, menu-driven utility that can be used to specify the desired configuration 29 Package, Processor Expert, QorIQ Qonverge, Qorivva, QUICC Engine, SMAROS, TurboLink, VortiQa and Xtrinsic are trademarks of Freescale Semiconductor, Inc.

30 ./linux-2.6.xx Doc. arch block crypto drivers fs include init kernel lib mm scripts security sound ipc net usr arch/powerpc/boot/dts/p4080ds.dts arch/powerpc/configs/p4080_ds_defconfig arch/powerpc/boot/ arch/powerpc/platforms/85xx/ 30 Package, Processor Expert, QorIQ Qonverge, Qorivva, QUICC Engine, SMAROS, TurboLink, VortiQa and Xtrinsic are trademarks of Freescale Semiconductor, Inc.

31 Bare metal Linux Use the device tree compiler to build the appropriate dts file in the./arch/powerpc/boot/dts directory OR dtc -I dts -O dtb -R 8 -b 0 -S 0x3000 p4080ds.dts > p4080ds.dtb make p4080ds.dtb (from linux-2.6/) Guest OS Linux The device tree for each partition is generated by hypervisor from a system configuration file that maps the system partitioning and hardware 31 Package, Processor Expert, QorIQ Qonverge, Qorivva, QUICC Engine, SMAROS, TurboLink, VortiQa and Xtrinsic are trademarks of Freescale Semiconductor, Inc.

32 In the Linux kernel, the entry point is _start, found in arch/powerpc/kernel/head_fsl_booke.s start _kernel() is in init/main.c, it will: _start_kernel()... _start setup_arch() call _start_kernel() mpc85xx_smp_init() { smp_ops = &smp_85xx_ops; } Kernel_init() smp_prepare_cpu() smp_ops->probe() in main.c smp_init() cpu_up() smp_cpus_done() smp_ops->kick_cpu() smp_ops->setup_cpu() 32 Package, Processor Expert, QorIQ Qonverge, Qorivva, QUICC Engine, SMAROS, TurboLink, VortiQa and Xtrinsic are trademarks of Freescale Semiconductor, Inc.

33 Let s look at smp_mpc85xx_kick_cpu() found in: arch/powerpc/platforms/85xx/mpc85xx_smp.c smp_mpc85xx_kick_cpu() {. /* Get the BPTR */ bptr_vaddr = ioremap(get_immrbase() + MPC85xx_BPTR_OFFSET, 4); /* Set the BPTR to the secondary boot page */ oldbptr = in_be32(bptr_vaddr); bptr = (BPTR_EN ( pa((unsigned) secondary_start_page) >> 12)); out_be32(bptr_vaddr, bptr); /* Kick that CPU */ smp_85xx_release_core(nr); /* Wait a bit for the CPU to take the exception. */ while (( secondary_hold_acknowledge!= nr) && (++n < 1000)) mdelay(1); } /* Restore the BPTR */ out_be32(bptr_vaddr, oldbptr); 33 Package, Processor Expert, QorIQ Qonverge, Qorivva, QUICC Engine, SMAROS, TurboLink, VortiQa and Xtrinsic are trademarks of Freescale Semiconductor, Inc.

34 Core 0 kicks off core 1 by setting EEBPCR[CPU1_EN] Let s look at smp_85xx_release_core() in arch/powerpc/mpc85xx/mpc85xx_smp.c smp_85xx_release_core(int nr) { /* * Startup Core #nr. */ ecm_vaddr = ioremap(get_immrbase() + MPC85xx_ECM_OFFSET, MPC85xx_ECM_SIZE); pcr = in_be32(ecm_vaddr + (ECM_PORT_CONFIG_OFFSET >> 2)); pcr = EEBPCR_CPU1_EN; out_be32(ecm_vaddr + (ECM_PORT_CONFIG_OFFSET >> 2), pcr); } EEBPCR 34 Package, Processor Expert, QorIQ Qonverge, Qorivva, QUICC Engine, SMAROS, TurboLink, VortiQa and Xtrinsic are trademarks of Freescale Semiconductor, Inc.

35 Core1 starts up from secondary_start_page defined in arch/powerpc/kernel/head_fsl_booke.s secondary_start_page: 4K The code in this page must not exceed 1023 instructions. The 1024th is a branch instruction It will initialize I-cache and D-cache Use TLB1[1] to map 16M memory Jump to early_start early_start If it is core1, it will not call start_kernel(), but jump to secondary_start secondary_start jumps to start_secondary() start_secondary() defined in arch/powerpc/kernel/smp.c Call smp_ops->setup_cpu() Call cpu_idle() 35 Package, Processor Expert, QorIQ Qonverge, Qorivva, QUICC Engine, SMAROS, TurboLink, VortiQa and Xtrinsic are trademarks of Freescale Semiconductor, Inc.

36