Speeding up the Booting Time of a Toro Appliance

Size: px

Start display at page:

Download "Speeding up the Booting Time of a Toro Appliance"

Dale Stone
5 years ago
Views:

1 Speeding up the Booting Time of a Toro Appliance Matias E. Vara Larsen matiasevara@gmail.com

2 Application-oriented Kernel Toro Kernel Process Memory Devices Filesystem Networking Toro is an embedded kernel including five units: - Process - Memory - Filesystem - Networking - Devices, e.g., Block Device, Network Device Each unit provides minimalist APIs accessible from the embedded application

3 Application-oriented Kernel BeginThread() ThreadSleep() RegisterBlockDriver() RegisterNetworkInterface() Toro Kernel Process Memory Devices GetMem() FreeMem() Filesystem Networking FileOpen() FileClose() Toro is an embedded kernel including five units: - Process - Memory - Filesystem - Networking - Devices, e.g., Block Device, Network Device Each unit provides minimalist APIs accessible from the embedded application

4 Application-oriented Kernel Toro Kernel Process Memory Devices Filesystem Networking Uses Microservice User application and kernel units are compiled in a single binary The application includes only the component required

5 Application-oriented Kernel Toro Kernel Process Memory Devices Filesystem Networking User application and kernel units are compiled in a single binary program HelloWorld; The application includes only the component required uses Memory, Filesystem, Ext2, E1000; Uses Microservice begin // // Your Code // end.

6 Application-oriented Kernel VM Toro Kernel Process Memory Devices Launches Filesystem CloudIt.sh Networking Uses Toro.elf Uses Microservice Builder Image

7 Application-oriented Kernel VM Toro Kernel Process Memory Devices Launches Filesystem Networking CloudIt.sh It s all talk until the code runs. - Ward Cunningham Uses Toro.elf Uses Microservice Builder Image

8 Application-oriented Kernel Toro Kernel Process Memory Devices Filesystem VM Launches Booting time is 1,5s CloudIt.sh Networking Uses Toro.elf Builder Uses Microservice Time to build a new image is about 1s Image (~ 4MB)

9 Application-oriented Kernel Toro Kernel Process Memory Devices Filesystem VM Launches Booting time is 1,5s CloudIt.sh Networking Uses Toro.elf Uses Microservice Builder Image (~ 4MB) These timings can be improved Time to build a new to enhance continuous deployment of image is about 1s microservices

10 Booting in Toro VMM Initialization Bootloader Kernel Initialization

11 Booting in Toro VMM Initialization Bootloader Kernel Initialization - Initialization of the device model - BIOS - Other stuff

12 Booting in Toro VMM Initialization Bootloader Kernel Initialization - Initialization of the device model - BIOS - Other stuff - Initialize hardware - Initialize processors, e.g., setup and enable paging, enable long mode, etc - Load the kernel into memory. In this case the image s size is very important

13 Booting in Toro VMM Initialization Bootloader Kernel Initialization - Initialization of the device model - BIOS - Other stuff - Initialize hardware - Initialize processors, e.g., setup and enable paging, enable long mode This presentation - Load the kerneldeals into with memory. In this case the different approaches image s size is very important to improve these times

14 Outline Speeding Up the Bootloader Speeding Up the Virtual Machine Monitor (VMM) Evaluation Conclusion QA

15 Speeding Up the Bootloader Context: The generated image is a copy of the kernel in memory The bootloader just read from the disk the image and then it writes it to memory Problem: The resulting image is huge The bootloader is still complex Proposal: Load Toro by using the -kernel option in QEMU/KVM (see Issue #223 at Github)

16 Multiboot Header Re ad s MultiBootloader Kernel Binary (elf32) QEMU/KVM.text $ kvm -kernel Toro.elf.data Memory

17 Multiboot Header MultiBootloader Kernel Binary (elf32) ds a e R QEMU/KVM.text $ kvm -kernel Toro.elf.data Memory

18 Multiboot Header MultiBootloader Kernel Binary (elf32) QEMU/KVM.text $ kvm -kernel Toro.elf ds a Lo.data Memory MultiBootloader.text.data

19 Multiboot Header MultiBootloader Kernel Binary (elf32) QEMU/KVM.text $ kvm -kernel Toro.elf.data Processor is already in protected mode m Ju ps Memory MultiBootloader (MutibootMain()).text (KernelMain()).data

20 Speeding Up the Bootloader Benefits: Reduce image size since it is only an elf32 binary from 4MB to 130kb Reduce bootloader complexity since QEMU loads the kernel into memory and yield the CPU to protected mode Reduce booting time from 1.5s to 0.5s

21 Speeding Up the Bootloader Benefits: Reduce image size since it is only an elf32 binary from 4MB to 130kb Reduce bootloader complexity since QEMU loads the kernel into memory and yield the CPU to protected mode Reduce booting time from 1.5s to 0.5s Drawbacks: VMM has to support the loading of a multiboot kernel Supports only elf32, so some magic is needed to make it work with elf64 We still have to jump to long mode

22 Speeding Up the Bootloader Benefits: Reduce image size since it is only an elf32 binary from 4MB to 130kb Reduce bootloader complexity since QEMU loads the kernel into memory and yield the CPU to protected mode Reduce booting time from 1.5s to 0.5s Drawbacks: VMM has to support the loading of a multiboot kernel Supports only elf32, so some magic is needed to make it work with elf64 We still have to jump to long mode Qemu-lite works around these but project seems discontinued (Port of Toro at Issue #192)

23 Outline Speeding Up the Bootloader Speeding Up the VMM Evaluation Conclusion QA

24 Speeding Up the VMM We study three approaches to improve the time spent in VMM initialization We focus on KVM/QEMU-based VMM These approaches are: QBoot, NEMU and Firecraker These approaches simplifies some aspect of the VMM, e.g., loading the of the kernel, hardware initialization or device model

25 Guest (Mode) Device Emulation BIOS VMM KVM Driver In-kernel device emulation Linux Kernel Bare-metal host

26 Guest (Mode) Device Emulation BIOS (QBoot) QEMU QBoot: - Minimal x86 firmware for QEMU to boot Linux A couple hardware initialization routines written mostly from scratch but with good help from SeaBIOS source code - Limit of 8 MB for vmlinuz+initrd+cmdline $ kvm -bios bios.bin -kernel Toro.elf KVM Driver In-kernel device emulation Linux Kernel Bare-metal host

27 Guest (Mode) Device Emulation (Minimal) BIOS NEMU (VMM) NEMU[1]: - Based on QEMU only for x86-64 and aarch64 - Reduced device model by focusing on non-emulated devices to reduce the VMM s footprint and the attack surface - Proposes a new machine type named virt which is thinner and only boots from UEFI [1] Honey-I-Shrunk-the-Hypervisor, Building a Legacy Free Platform for QEMU, Robert Bradford, Intel KVM Driver In-kernel device emulation Linux Kernel Bare-metal host

28 Guest (Mode) Linux-based Device Emulation BIOS Firecracker (VMM) KVM Driver In-kernel device emulation Firecracker: - Simple VMM implemented in Rust developed by Amazon Web Services to accelerate the speed and efficiency of services like AWS Lambda and AWS Fargate - Sets vcpu to long mode, sets pages tables the Linux way and expects kernel to be in vmlinux format (64-bit ELF uncompressed) Linux Kernel Bare-metal host

29 Evaluation We measured the time that takes the kernel to start to execute, i.e., the time since the VM is launched until the KernelMain() is executed We compared these times by using the presented solutions See Issue #276 at Github for more information

30 Results 4 cores Intel(R) Atom(TM) CPU 2.40GHz 8 GB of physical memory Approach Image Binary Binary with QBoot QEMU/KVM (2.5.0) 1457 ms 452 ms 132 ms NEMU (#39af42) 309 ms 95 ms Firecracker (0.14.0) 17ms $ echo Hello World! avg: ms

31 Conclusion Booting time improved by a factor x11 when using multiboot and QBoot Booting time improved by a factor x85 when using Firecracker Trade-off between the needed work to adapt the kernel and minimizing booting time

32 QA Torokernel wiki at github My first Three examples with Toro Test Toro in 5 minutes (or less...) torokernel-docker-qemu-webservices at Github

QA http://www.torokernel.io torokernel@gmail.

33 QA Torokernel wiki at github s! k l My ll fo first Three examples with Toro a at s h T Test Toro in 5 minutes (or less...) torokernel-docker-qemu-webservices at Github

64-bit ARM Unikernels on ukvm

64-bit ARM Unikernels on ukvm Wei Chen Senior Software Engineer Tokyo / Open Source Summit Japan 2017 2017-05-31 Thanks to Dan Williams, Martin Lucina, Anil Madhavapeddy and other Solo5