Achieve Fastest System Startup Sequences. How to tune an Embedded System. Embedded Systems Design Conference ARM vs. x86 July 3, 2014 Kei Thomsen MicroSys Electronics GmbH
Agenda Target: reduce startup time of embedded systems. Meaning of Fast-Boot. Comparison of Operating Systems, function, benefit and startup times. Embedded Systems, boot medium and behavior. Identifying time consuming artifacts. Summary. 2
Meaning of Fast-Boot? Sense of time depends on the Application Is a Windows Start below 60 seconds fast? Do I want to wait 2 seconds for my camera to start? An endoscope must be available after a power fault within 0.8 seconds, including graphic on 2 screens: Not possible? It is possible! Fast-Boot = System start time from reset to running application in shortest time. 3
Comparison of Operating Systems, function, benefit and startup times. Standard Operating System Linux, Windows (CE) All Inclusive Plus. extensive and complex. Many functions already included and build in. Simple to set up, as everything is included. Many components available, were it is difficult to understand what it is for, or needed by other components. Long startup times Real Time Operating System (RTOS) OS-9, QNX, VxWorks Bed & Breakfast. simple and fast. Additional functions are added and configured. Simple to create and extend the functionality volume. Only functions, which are really used / needed. No nice to have features. Very fast startup times 4
Function und benefit in Linux (Embedded) The Kernel-Start of a Linux System initializes the build-in Kernel driver. Which are they and which are not used or maybe used later? System services are started. Which one in which order? And what are they for? Which one can I remove? Hugh functionality. But which functions and services are really used for the correct function of the final system? Are late optimizations and reductions of the base system still fulfilling the warrant system features? 5
Suggestions for Linux optimizations Best way to start with is a minimal system and add / activate the needed function on demand. Reduction of the overall system size and number of services / processes. Every Linux System is different and is based on a ready to use Linux-Distribution (LTIB, Debian, ELDK ). For a start time optimization a consolidated Linux knowledge for the used distribution is needed. First to understand the concept of the system start function: SysV-Init, BSD-Init, busybox, launchd, runit, systemd, upstart or others? Less System Services, less CPU time needed at startup. Try different mass storage medias with different file system types. Faster loading of OS, Services and Applications. 6
Function and benefit in Real Time Operating Systems (RTOS) RTOSe based on different architecture Microware OS-9 WindRiver VxWorks QNX Kernel is build of single components, which are exactly adapted and selected for the system. MMU, Cache, Interrupt & Exception Handling Driver Middleware Includes only the required functionality. No unused functions wasting space and time. 7
Function and benefit in Real Time Operating Systems (RTOS) Small, clear, fast Function of the components simple to detect and identifiable by name Kernel, PCF, epicirq, SPPro1000e One single Developer can oversee and understand the complete functional volume and its components System understanding without magic Additional functions and features are added and configured. Clearly arranged, reproducible, simple to validate 8
Comparison of boot medium SD/µSD CF-Card NOR Flash NAND Flash Connection SD Controller IDE Controller CPU Bus NAND Controller Bus width (Bit) 1 or 4 8 8 / 16 / 32 / 64 8 Flash Type MLC (SLC very rare) MLC (SLC very rare) NOR MLC and SLC Speed (MB/sec.) >20 5 8 (>20 DMA) 10-80 5-8 Size (typical) 1-32 GB 32 MB - 32 GB 2-64 MB ¼ 4 GB Price/MB Extrem low cost Very low cost Expensive Low cost OS Startup (Sec.) 1,5 2,4 0,3 2,4 Pro Simple to replace Simple to replace Extrem fast, direct execution Contra Most time only consumer quality (MLC) Most time only consumer quality (MLC) Less capacity Driver effort Medium Medium to high Only for writing High Direct soldered chips with high capacity Defects by writing, difficult to replace Recommendation Periodic replacement Periodic replacement No file system Main use as read only device Frustration factor Medium Medium Low High (Values of an OS-9 System with graphical application) MLC = Multi Level Cell NAND Flash SLC = Single Level Cell NAND Flash 9
Embedded System boot time optimization Compressed or uncompressed OS-Image Is the load time longer than the decompression time? Small OS-Image Faster load of the OS Driver in the OS might be faster than U-Boot driver due to IRQ and DMA Late loading of components therefore faster Large OS-Image As we are already loading, load everything in one chunk No loading of components needed As the boot mediums, boards, operating systems and components are too different, there is no generic statement possible and the optimization has to be determined individual. 10
Embedded System boot time optimization Example system: FreeScale i.mx53 1GHz Linux (ELDK 5.4 from Denx) Application QT Graphic Regular Start (before): 24 seconds Uncompressed OS Image: less 3 seconds SD instead of NAND (OS Image and file system): less 4 seconds Now 17 Seconds 11
Embedded System time consuming actions at startup Boot medium Ethernet Link Auto Negotiation 1-3 seconds for link detection 5+ seconds timeout (if no cable attached) If the kernel is waiting for completion, these are 1-5+ seconds, doing nothing, just waiting. USB - Bus Explore Each Hub 2-3 seconds to detect and start Each Device ~1 second to detect and start Better: start Ethernet & USB after the scheduling is available in the kernel so that the detection can run as background task during the remaining system startup. In our example: another 6 seconds saved. 12
Identify time consuming artifacts Almost every Embedded Hardware has a serial connection for the system start (U-Boot, Kernel messages, Shell console). Most Terminal Programs like TeraTerm can do data logging with time stamps. Hint: Use the real serial (COM1) of the PC if possible. The USB-Serial-Adapter are using a large buffer, so that multiple lines are transferred at once and getting the same time stamp. Recommended baud rate >= 115200 13
Time stamp analysis [Wed Sep 05 13:53:06.961 2012] NAND read: device 0 offset 0x1fb00000, size 0x300000 [Wed Sep 05 13:53:07.471 2012] 3145728 bytes read: OK [Wed Sep 05 13:53:07.491 2012] NAND read: device 0 offset 0x200000, size 0x400000 [Wed Sep 05 13:53:08.171 2012] 4194304 bytes read: OK [Wed Sep 05 13:53:08.211 2012] ## Booting kernel from Legacy Image at 01000000... [Wed Sep 05 13:53:08.211 2012] Image Name: Linux-3.3.1 [Wed Sep 05 13:53:08.211 2012] Created: 2012-06-06 6:21:20 UTC [Wed Sep 05 13:53:08.211 2012] Image Type: PowerPC Linux Kernel Image (gzip compressed) [Wed Sep 05 13:53:08.211 2012] Data Size: 3486786 Bytes = 3.3 MiB [Wed Sep 05 13:53:08.751 2012] ## Flattened Device Tree blob at 00c00000 [Wed Sep 05 13:53:08.751 2012] Booting using the fdt blob at 0xc00000 [Wed Sep 05 13:53:08.751 2012] [Wed Sep 05 13:53:11.811 2012] Uncompressing Kernel Image... OK Loading Device Tree to 00ffa000, end 00ffffff... OK [Wed Sep 05 13:53:15.201 2012] ADDRCONF(NETDEV_UP): eth0: link is not ready [Wed Sep 05 13:53:16.221 2012] PHY: mdio@ffe24000:01 - Link is Up - 1000/Half [Wed Sep 05 13:53:16.221 2012] ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Δt 0.5s Δt 0.7s Δt 3.1s Δt 1.0s 14
Summary Not quick nor simple to tune a system afterwards. Many hardware understanding and operating system knowledge needed. Appropriate Operating System selection: Not only looking for fancy features Commercial Real Time Operating Systems are not more expensive than a free Linux system. Add components and function on demand, instead of trying to remove then afterwards. Selection of the boot medium. Logging with time stamp to analyze the system startup. 15
Question? More information in the exhibition area at MicroSys Kei Thomsen OS-9 Development, embedded System Design & Engineering Email: thomsen@microsys.de Internet: www.microsys.de MicroSys Electronics GmbH Mühlweg 1 D-82054 Sauerlach 16