RT- Xen: Real- Time Virtualiza2on. Chenyang Lu Cyber- Physical Systems Laboratory Department of Computer Science and Engineering

Transcription

1 RT- Xen: Real- Time Virtualiza2on Chenyang Lu Cyber- Physical Systems Laboratory Department of Computer Science and Engineering

2 Embedded Systems Ø Consolidate 100 ECUs à ~10 multicore processors. Ø Integrate multiple systems on a common platform. q q q Infotainment on Linux or Android Safety-critical control on AUTOSAR Virtualization: COQOS, Integrity Multivisor, Xen automotive Ø Must preserve real-time performance on a virtualized platform! Source: hbp:// and- virtualizaeon- in- automoeve- environments 4/13/15 1

3 Virtualiza2on is not real- 2me today Ø Existing hypervisors provide no guarantee on latency q Xen: credit scheduler, [credit, cap] q VMware ESXi: [reservation, share, limitation] q Microsoft Hyper-V: [reserve, weight, limit] Ø Public clouds lack service level agreement on latency q EC2, Compute Engine, Azure: #VCPUs Current platforms provision resources, not latency! 4/13/15 2

4 Challenges Ø Support real-time applications in a virtualized environment. q Latency guarantees to tasks running in virtual machines (VMs). q Real-time performance isolation between VMs. Ø Multi-level real-time performance provisioning q Virtualization within a host q Communication and I/O q Cloud resource management 4/13/15 3

5 RT- Xen Ø Real-time schedulers in the Xen hypervisor q Real-time performance for tasks running in virtual machines (VMs). q Real-time performance isolation between VMs. q Experimentation of real-time scheduling at the hypervisor level. Ø Build on compositional scheduling theory q VMs specify resource interfaces. q Real-time guarantees to tasks in VMs. Ø Incorporated in Xen 4.5 as the rtds scheduler. q rtds: Real-Time Deferrable Server 4/13/15 4

6 Xen Virtualiza2on Architecture Ø Xen: type-1, baremetal hypervisor q Domain-0: drivers, tool stack to control VMs. q Guest Domain: para-virtualized or fully virtualized OS. Ø Xen scheduler q Guest OS runs on VCPUs. q Xen schedules VCPUs on PCPUs. q Credit scheduler: round-robin with proportional share. Real- Time Task VCPU OS Sched OS Sched OS Sched Xen Scheduler PCPUs 4/13/15 5

7 Composi2onal Scheduling Ø Analytical real-time guarantees to tasks running in VMs. Ø VM resource interfaces q Hides task-specific information q Multicore: a set of VCPUs each with an interface <period, budget > q Computed based on compositional scheduling analysis Hypervisor Resource Interface Scheduler Resource Interface Scheduler Workload Resource Interface Scheduler Workload Virtual Machines 4/13/15 6

8 Global vs. Par22oned Scheduling Ø Global scheduling q Shared global run queue q Allow VCPU migration across cores q Work conserving utilize any available cores q Migration overhead and cache penalty Ø Partitioned scheduling q Assign and bind VCPUs to cores q Schedule VCPUs on each core independently q Cores may idle when others have work pending q No migration overhead or cache penalty 4/13/15 7

9 Scheduling VCPU as Servers T2 (10, 3) T1 (10, 3) Eme Periodic Server (5,3) Budget not work- conserving Eme Deferrable Server (5,3) Budget back- to- back execueon Eme 8

10 Run Queues Ø rt-global: all cores share one run queue with a spinlock Ø rt-partition: one run queue per core Ø A run queue q holds VCPUs that are runnable (not idle) q divided into two parts: with budget; without budget q sorted by priority (DM or EDF) within each part 4/13/15 9

11 RT- Xen: Real- Time Scheduling in Xen Ø Single-core RT-Xen 1.0 [EMSOFT 11] Ø Single-core enhanced RT-Xen 1.1 [RTAS 12] Ø Multi-core scheduling RT-Xen 2.0 [EMSOFT 14] Ø Real-time deferrable server (rtds) [Xen 4.5] Fixed Priority (DM) Capacity Reclaiming Periodic Work Conserving Periodic Polling Deferrable Periodic Periodic Deferrable Sporadic Global Scheduling Deferrable Periodic Periodic Deferrable ParEEoned Scheduling Dynamic Priority (EDF) 4/13/15 10

12 Experimental Setup Ø Hardware: Intel i7 processor, 6 cores, 3.33 GHz q Allocate 1 VCPU for Domain-0, pinned to PCPU 0 q All guest VMs use the remaining cores q Software q Xen 4.3 patched with RT-Xen q Guest OS: Linux patched with LITMUS Ø Workload q Period tasks q Allocate tasks à VMs 4/13/15 11

13 RT- Xen 2.0: Credit Scheduler Credit misses deadlines at 22% of CPU capacity. RT-Xen delivers real-time performance at 78% of CPU capacity. 4/13/15 12

14 RT- Xen 2.0: Scheduling Overhead rt-global has extra overhead due to global lock. Credit has poor max overhead due to load balancing. 4/13/15 13

15 RT- Xen 2.0: Theory vs. Experiments Fraction of schedulable tasksets gedf (RT Xen) pedf (RT Xen) pedf (CSA Theory) gedf (CSA Theory) Task Set Utilization gedf < pedf theoretically due to pessimistic analysis. gedf > pedf empirically, thanks to work-conserving global scheduling. 4/13/15 14

16 RT- Xen 2.0: Deferrable vs. Periodic Work-conserving wins empirically! gedf+ds à best real-time performance. This is the rtds scheduler in Xen /13/15 15

17 RT- Xen 2.0: How about Cache? pedf in guest OS gedf > pedf for cache intensive workload. Benefit of global scheduling dominates migration cost. Shared cache mitigates cache penalty due to migration. 4/13/15 16

18 Conclusion Ø Embedded system integration demands real-time virtualization. Ø RT-Xen: real-time VM scheduling on multicore processors. Ø Insights from experimental study. q Tradeoff between theoretical guarantees vs. real performance. q gedf+ds à work conserving wins empirically (Xen 4.5). Ø More from RT-Xen q Inter-domain communication and network I/O. [IWQoS'13][RTAS'15] q Cache-aware compositional scheduling. [RTSS'13] 4/13/15 17

19 Conclusion Ø Embedded system integration demands real-time virtualization. Ø RT-Xen: real-time VM scheduling on multicore processors. How to turn RT-Xen into a Ø Insights from experimental study. certifiable, hard real-time q Tradeoff between theoretical guarantees vs. real performance. q gedf+ds à work conserving wins empirically (Xen 4.5). Ø More from RT-Xen platform? q Inter-domain communication and network I/O. [IWQoS'13][RTAS'15] q Cache-aware compositional scheduling. [RTSS'13] 4/13/15 18

20 Check out RT- Xen RT- Xen hbps://sites.google.com/site/realemexen/ Incorporated in Xen 4.5 as the rtds scheduler Contributors Washington University in St. Louis: Sisu Xi, Chris Gill University of Pennsylvania: Meng Xu, Insup Lee, Linh Phan, Oleg Sokolsky 4/13/15 19