Multicore Boot Process

Transcription

1 November 5, 2008 Multicore Boot Process PN110 Jeff Zhu NMG Applications of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. Freescale Semiconductor, Inc

2 Agenda Multicore operating system architecture: SMP vs. AMP Power-on reset sequence and default boot configuration e500 core boot options and features Configuration of resources: SMP vs AMP MPC8572DS SMP u-boot/linux boot process AMP u-boot/linux boot process Questions and answers of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. Freescale Semiconductor, Inc

3 Multicore Operating System Architecture: SMP vs. AMP of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. Freescale Semiconductor, Inc

4 SMP Concepts Symmetrical MultiProcessing An architecture that provides fast performance by making multiple CPUs available to complete individual processes simultaneously (multiprocessing) Any idle CPU can be assigned any task, and additional CPUs can be added to improve performance and handle increased loads SMP uses a single operating system and shares common memory, all the IO and interrupt resources Processes and threads are distributed among CPUs CPU I/O Single OS CPU Memory controller Two cores one OS of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. Freescale Semiconductor, Inc

5 AMP Concepts Asymmetrical MultiProcessing Each CPU runs its own OS which may be same or different from each other. Each CPU can be assigned with specific application to run If Linux, two copies of uimage are needed, but located at different physical address spaces! OS 1 CPU OS 2 CPU Both CPUs must cooperate to share the resources Neither OS can own the whole system I/O and interrupts are separated Static configuration for resources If Linux, resources allocation can be done by two device trees (dts) I/O Memory controller Two cores two OS s of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. Freescale Semiconductor, Inc

6 Power-on Reset Sequence And Default Boot Configuration of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. Freescale Semiconductor, Inc

7 Power-On Reset Sequence HRESET HRESET_REQ (high impedance) SYSCLK TRESET SRESET PLL Configs POR Configs ASLEEP (High Impedance) READY 1 Multiplexed with TRIG_OUT. (High Impedance) of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. Freescale Semiconductor, Inc

8 Power-On Reset Sequence (cont.) HRESET HRESET_REQ (high impedance) SYSCLK Minimum HRESET assertion time = 1us TRESET SRESET PLL Configs POR Configs ASLEEP (High Impedance) READY 1 Multiplexed with TRIG_OUT. (High Impedance) of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. Freescale Semiconductor, Inc

9 Power-On Reset Sequence (cont.) HRESET HRESET_REQ (high impedance) SYSCLK TRESET SRESET PLL Configs POR Configs Minimum POR (High Impedance) ASLEEPconfigs input setup time = 4 sysclks before negation of HRESET. READY 1 (High Impedance) Multiplexed with TRIG_OUT. of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. Freescale Semiconductor, Inc

10 Power-On Reset Sequence (cont.) HRESET HRESET_REQ (high impedance) SYSCLK TRESET SRESET PLL Configs POR Configs Minimum POR (High Impedance) ASLEEPconfigs input hold time = 2 sysclks after negation of HRESET. READY 1 (High Impedance) Multiplexed with TRIG_OUT. of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. Freescale Semiconductor, Inc

11 Power-On Reset Sequence (cont.) HRESET HRESET_REQ (high impedance) SYSCLK TRESET SRESET PLL Configs POR Configs Maximum (High Impedance) ASLEEP POR configs valid-to-high impedance = 5 sysclks after negation of HRESET. READY 1 (High Impedance) Multiplexed with TRIG_OUT. of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. Freescale Semiconductor, Inc

12 Power-On Reset Sequence (cont.) HRESET HRESET_REQ (high impedance) SYSCLK TRESET SRESET PLL Configs POR Configs The POR config (High Impedance) ASLEEP states are latched in POR configuration value registers in the Global Utilities Register Block of the CCSR space. READY 1 (High Impedance) Multiplexed with TRIG_OUT. of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. Freescale Semiconductor, Inc

13 POR Configurations Signals may be pulled high or low by external resistors Some signals may have internal pull-ups Signals may be pulled low with 4.7K ohm resistors PLL configuration signals do not have internal pull-up resistors of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. Freescale Semiconductor, Inc

14 Default Boot Configuration Physical boot ROM 8 MB address range 0x0_FF80_0000 to 0x_FFFF_FFFF located on the Local Bus GPCM 32-bit ROM Both cores are allowed to boot Boot sequencer is disabled Default core TLB maps 4 KB boot page Default LAWs map 8 MB boot space, 1 MB CCSRBAR and L2 SRAM of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. Freescale Semiconductor, Inc

15 CCB clock PLL ratio e500 core PLL ratio, separate signals for each core DDR clock PLL ratio Boot sequencer CPU boot configuration Boot ROM location DDR SDRAM type RapidIO size and device ID etsecn mode, width Host/agent configuration I/O port selection SerDes configuration POR Config Considerations of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. Freescale Semiconductor, Inc

16 e500 Core Boot Options and Features of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. Freescale Semiconductor, Inc

17 Boot Page Translation Provides flexibility in where in system memory each e500 core can fetch its 4 KB boot page from Each core default boots from a 4 KB page at 0x0_FFFF_Fnnn Boot page translation allows the each core to be independently configured so its 4 KB boot page points to anywhere in the 36-bit memory map Can be configured by the boot sequencer prior to any core booting Boot Page Translation Register (BPTR) of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. Freescale Semiconductor, Inc

18 Boot Page Translation (cont.) Core Core Fetch from default 4 KB boot page: 0x0_FFFF_Fnnn Platform YES Replace 0x0_FFFF_Fnnn With BPTR[BOOT_PAGE] = 0xAB_CDEF Is BPTR[EN]=1 NO Fetch 0x0_FFFF_Fnnn Fetch 0xA_BCDE_Fnnn of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. Freescale Semiconductor, Inc

19 CPU Boot Configuration Provides the ability to control when and which cores are allowed to boot Boot holdoff mode will prevent a core from booting, e.g., the core will not be permitted to fetch the reset vector This feature can be used to allow external masters or a single core to configure the SOC to some desired state, then release the remaining cores from boot holdoff to boot in the already configured environment. CPU Boot Configuration of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. Freescale Semiconductor, Inc

20 Boot ROM Location The default boot ROM is located off of the local bus 32bit GPCM. HW will configure any boot ROM location on the local bus. All others must be configured by the boot sequencer. Boot ROM Location of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. Freescale Semiconductor, Inc

21 Boot Sequencer Boot sequencer is essentially a DMA engine that reads from one or more I 2 C serial EEPROM device to write data to CCSRBAR or ALTCBAR space. If enabled by POR config, all cores are held in reset until boot sequencer completes. Allows configuration of SOC and memory prior to booting the cores. The boot sequencer program stored in EEPROM must follow a defined data format. A boot sequencer failure will cause assertion of HRESET_REQ_ Boot Sequencer Configuration of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. Freescale Semiconductor, Inc

22 Boot Sequencer Data Frame Format ACS = 0 Writes to CCSRBAR 1 MB window ACS = 1 Writes to ALTCBAR 1 MB window ADDR[0:17] specifies the address of the word size access into the 1 MB window BYTE_EN[0:3] specifies which byte(s) within a word to write DATA[0:31] contain the 32-bits of write data of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. Freescale Semiconductor, Inc

23 CRC covers all bits stored in EEPROM prior to the CRC Preamble = 0xAA55AA CONT=0 indicates last frame. If CONT is cleared, the entire first 3 bytes must be cleared Boot Sequencer EEPROM Format of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. Freescale Semiconductor, Inc

24 Alternate Configuration Space Allows boot sequencer access to the entire 36-bit memory map through a configurable 1 MB window. Boot sequencer can access the 1 MB CCSRBAR space to configure the ALTCAR registers, thus allowing the boot sequencer access to the entire memory map, 1 MB block at a time. Memory accessed via ALTBAR do not need LAWs. Alternate Configuration Base Address Register (ALTCBAR) Alternate Configuration Attribute Register (ALTCAR) of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. Freescale Semiconductor, Inc

25 Configuration of Resources: SMP vs. AMP of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. Freescale Semiconductor, Inc

26 Configuration of Resources: SMP In an SMP environment: All cores execute from one Linux image Care must be taken to not reinitialize shared resources that have already been configured LAWs, DDR, MPIC, L2, etsec, etc. MPC8572 SMP u-boot and Linux U-boot on core0 is responsible for: Initializing the memory map and DDR Linux on core0 is responsible for initializing all shared resources not already configured by U-boot and kicking off core1. MPIC, etsec, PCI/PEX drivers When core1 boots the same Linux image, it will enter Linux at a specific location initializing resources specific only to core1 such as caches and MMU, then enters an idle loop. of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. Freescale Semiconductor, Inc

27 Configuration of Resources: AMP In an AMP environment: All cores execute their own Linux image Care must be taken to not reinitialize shared resources that have already been configured LAWs, DDR, MPIC, L2, etsec, etc. AMP u-boot and Linux U-boot on core0 is responsible for: Initializing the memory map and DDR, then kicks off core1 Linux on core0 is responsible for initializing all assigned resources specified in core0 dts. MPIC interrupts, etsec, PCI/PEX drivers U-boot on core1 is responsible for initializing only core1 MMU, caches, etc. Linux on core1 is responsible for initializing all assigned resources specified in core1 dts. of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. Freescale Semiconductor, Inc

28 MPC8572DS SMP u-boot/linux Boot Process of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. Freescale Semiconductor, Inc

29 Power-on Reset SMP Boot Process: MPC8572 Alpha BSP Core0 comes out of reset at the reset vector 0xFFFF_FFFC and core1 is in hold off mode Core0 runs u-boot Core0 starts up the kernel After setting BPTR to secondary core s start page, core0 kicks off core1 Core1 is out of reset at secondary_start_page(boot page) NOTE: Only Linux kernel runs on core1 (U-boot is not aware of core1!) core0 0xFFFF_FFFC kick_cpu core1 Secondary_start_page U-BOOT KERNEL KERNEL of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. Freescale Semiconductor, Inc

30 SMP Boot Process: U-boot Boot Process In the u-boot source, the entry point is the reset vector 0x0_FFFF_FFFC containing a simple branch to _start_e500, which is at the base of default boot page at 0x0_FFFF_F000. Both are found in cpu/mpc85xx/start.s. Reset vector call _start_e500() _start_e500() will: _start_e500() Jump to DDR Start command interpreter Prepare Linux device tree 1. Enable L1 caches 2. Config interrupt vectors 3. Config MMU and LAWs 4. Config L1D RAM 5. Config local bus 6. Config DDR 1. Copy device tree to DDR 2. Copy uimage to DDR 3. Copy Ramdisk to DDR (optionally) 4. Launch kernel of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. Freescale Semiconductor, Inc

31 SMP Boot Process: Linux SMP Boot Process In the Linux kernel, the entry point is _start found in arch/powerpc/kernel/head_fsl_booke.s _start start _kernel() is in init/main.c, it will: _start_kernel()... setup_arch() call _start_kernel() mpc85xx_smp_init() { smp_ops = &smp_85xx_ops; } Kernel_init() smp_prepare_cpu() smp_ops->probe() in main.c smp_init() cpu_up() smp_cpus_done() smp_ops->kick_cpu() smp_ops->setup_cpu() of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. Freescale Semiconductor, Inc

32 SMP Boot Process: Code Starting Point for Core 1 Core1 will start executing from boot page setting in BPTR When each e500 core comes out of reset, its MMU has one 4-KB boot page defined at 0x0_FFFF_Fnnn MPC8572 provides the boot page translation capability for the core that has different boot page address Boot Page Translation Register (BPTR) is enabled and set to the page of secondary_start_page by core0 before kicking off core1 Boot Page Translation Register (BPTR) of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. Freescale Semiconductor, Inc

33 SMP Boot Process: Core1 Start Up (cont.) Let s look at smp_mpc85xx_kick_cpu() found in: arch/powerpc/platforms/85xx/mpc85xx_smp.c smp_mpc85xx_kick_cpu() {. } /* Get the BPTR */ bptr_vaddr = ioremap(get_immrbase() + MPC85xx_BPTR_OFFSET, 4); /* Set the BPTR to the secondary boot page */ oldbptr = in_be32(bptr_vaddr); bptr = (BPTR_EN ( pa((unsigned) secondary_start_page) >> 12)); out_be32(bptr_vaddr, bptr); /* Kick that CPU */ smp_85xx_release_core(nr); /* Wait a bit for the CPU to take the exception. */ while (( secondary_hold_acknowledge!= nr) && (++n < 1000)) mdelay(1); /* Restore the BPTR */ out_be32(bptr_vaddr, oldbptr); of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. Freescale Semiconductor, Inc

34 SMP Boot Process: Core1 Kick-Off Process Core 0 kicks off core 1 by setting EEBPCR[CPU1_EN] Let s look at smp_85xx_release_core() in arch/powerpc/mpc85xx/mpc85xx_smp.c smp_85xx_release_core(int nr) { /* * Startup Core #nr. */ ecm_vaddr = ioremap(get_immrbase() + MPC85xx_ECM_OFFSET, MPC85xx_ECM_SIZE); pcr = in_be32(ecm_vaddr + (ECM_PORT_CONFIG_OFFSET >> 2)); pcr = EEBPCR_CPU1_EN; out_be32(ecm_vaddr + (ECM_PORT_CONFIG_OFFSET >> 2), pcr); } EEBPCR of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. Freescale Semiconductor, Inc

35 SMP Boot Process: Core1 Kernel Code Flow Core1 starts up from secondary_start_page defined in arch/powerpc/kernel/head_fsl_booke.s secondary_start_page: 4K The code in this page must not exceed 1023 instruction. The1024th is a branch instruction. It will initialize I-Cache and D-Cache Use TLB1[1] to map 16M memory Jump to early_start early_start If it is core1, it will not call start_kernel(), but jump to secondary_start secondary_start jumps to start_secondary() start_secondary() defined in arch/powerpc/kernel/smp.c Call smp_ops->setup_cpu() Call cpu_idle() of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. Freescale Semiconductor, Inc

36 MPC8572DS Alpha BSP Location Available at search keyword MPC8572DS Linux. Freescale Linux BSP for MPC8572DS U-Boot Linux Kernel: Linux RFS for Freescale MPC8572DS of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. Freescale Semiconductor, Inc

37 AMP u-boot/linux Boot Process of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. Freescale Semiconductor, Inc

38 AMP Linux Boot Process Core0 comes out of reset and core1 in hold off mode Core0 executes u-boot kick off core1 in u-boot Core0 starts up kernel with assigned resources specified in core0 device tree Core1 comes out of reset and executes u-boot Core1 starts up kernel with assigned resources specified in core1 device tree of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. Freescale Semiconductor, Inc

39 MPC8572 AMP Implementation Development is on-going. Common parts with MPC8641D Core0 releases core1 in u-boot Core0 and core1 run its own kernel image Resources are not shared except for MPIC Only core0 will initialize MPIC Interrupts should be assigned to each core separately based on device tree Difference MPC8641D has low memory offset mode, MPC8572 doesn t MPC8572 can relocate exception vectors MPC8572 will use PHYSICAL_START to redirect core1 s kernel base MPC8572 can release core with new u-boot command cpu of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. Freescale Semiconductor, Inc

40 Questions and Answers of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. Freescale Semiconductor, Inc

41 Related Session Resources Session Location Online Literature Library Sessions Session ID Title Demos Pedestal ID Demo Title

42