RT- Xen: Real- Time Virtualiza2on from embedded to cloud compu2ng

Size: px

Start display at page:

Download "RT- Xen: Real- Time Virtualiza2on from embedded to cloud compu2ng"

Alexia White
6 years ago
Views:

1 RT- Xen: Real- Time Virtualiza2on from embedded to cloud compu2ng Chenyang Lu Cyber- Physical Systems Laboratory Department of Computer Science and Engineering

q q q Infotainment on Linux or Android Safety-critical control on AUTOSAR Example: COQOS, Integrity Multivisor, Xen automotive

2 Real- Time Virtualiza2on for Cars Ø Consolidate 100 ECUs à 10 multicore processors. Ø Integrate multiple systems on a common platform à virtualization. q q q Infotainment on Linux or Android Safety-critical control on AUTOSAR Example: COQOS, Integrity Multivisor, Xen automotive branch Ø Must preserve real-time guarantees on a virtualized platform! LeB: hdp:// and- virtualizagon- in- automogve- environments Right: hdp://community.arm.com/docs/doc /27/14 1

3 Real- Time Cloud Ø Internet of Things à Cyber-Physical Systems q Smart manufacturing, smart transportation, smart grid. q Internet-scale sensing and control à real-time cloud computing. Ø Example: Intelligent Transportation Systems q Data center collects data from cameras and roadside detectors. q Control the traffic signals and message signs in real-time. q Transportation information feed to drivers. q Sydney: controlling 3,400 signals at 1s round-trip latency. Ø Latency-sensitive applications, e.g., cloud gaming q Xbox One: cloud offloading computation of environmental elements q Sony acquired Gaikai, an open cloud gaming platform. 5/27/14 2

Virtualiza2on is not real- 2me today Ø Existing hypervisors provide no guarantee on latency q Xen: credit scheduler, [credit, cap] q VMware ESXi: [reservation, share, limitation] q

4 Virtualiza2on is not real- 2me today Ø Existing hypervisors provide no guarantee on latency q Xen: credit scheduler, [credit, cap] q VMware ESXi: [reservation, share, limitation] q Microsoft Hyper-V: [reserve, weight, limit] Ø Public clouds lack service level agreement on latency q EC2, Compute Engine, Azure: #VCPUs Current platforms provision resources, not latency! 5/27/14 3

5 Challenges Ø Support real-time applications in a virtualized environment. q Latency guarantees to tasks running in virtual machines (VMs). q Real-time performance isolation between VMs. Ø Multi-level real-time performance provisioning q Virtualization within a host q Communication and I/O q Cloud resource management 5/27/14 4

6 RT- Xen Ø Real-time hypervisor based on Xen q Real-time VM scheduling q Real-time communication Ø Build on compositional scheduling theory q VMs specify resource interfaces q Real-time guarantees to tasks in VMs Ø Open source q Xen patch in progress Ø RT-OpenStack: cloud management based on RT-Xen 5/27/14 5

7 Xen Virtualiza2on Architecture Ø Xen: type-1, baremetal hypervisor q Domain-0: drivers, tool stack to control VMs. q Guest Domain: para-virtualized or fully virtualized OS. Ø Xen scheduler q Guest OS runs on VCPUs. q Xen schedules VCPUs on PCPUs. q Credit scheduler: round-robin with proportional share. Real- Time Task VCPU OS Sched OS Sched OS Sched Xen Scheduler PCPUs 5/27/14 6

8 Composi2onal Scheduling Ø Analytical real-time guarantees to tasks running in VMs. Ø VM resource interfaces q Hides task-specific information q Multicore: <period, budget, #VCPU> q Computed based on compositional scheduling analysis Hypervisor Resource Interface Scheduler Resource Interface Scheduler Workload Resource Interface Scheduler Workload Virtual Machines 5/27/14 7

9 Real- Time Scheduling Policies Ø Global scheduling q Shared global run queue q Allow VCPU migration across cores q Work conserving q Migration overhead and cache penalty Ø Partitioned scheduling q Assign and bind VCPUs to cores q Schedule VCPUs on each core independently q Not work conserving: cores may idle when others have work pending q No migration overhead or cache penalty Ø Priority scheme q Static priority: Rate Monotonic q Dynamic priority: Earliest Deadline First (EDF) 5/27/14 8

10 Scheduling VCPU as Server T2 (5, 3) T1 (5, 3) Gme Periodic Server (5,3) Budget Not work conserving Gme Deferrable Server (5,3) Budget back- to- back execugon Gme 5/27/14 9

11 Run Queues Ø rt-global: all cores share one run queue with a spinlock Ø rt-partition: one run queue per core Ø A run queue q Holds VCPUs that are runnable (not idle) q Divided into two parts: with budget; without budget q Sorted by priority (DM or EDF) within each part 5/27/14 10

12 Scheduling Func2ons Ø do_schedule q triggered every 1 ms q consume current VCPU s budget q replenish a VCPU s budget if it is the end of its period q return runnable VCPU with the highest priority Ø context_switch Ø context_saved q global scheduler only q insert VCPU back to run queue after context switch Ø wake_up q triggered by timer or I/O q call do_schedule if necessary 5/27/14 11

13 RT- Xen: Real- Time Scheduling in Xen Ø Single-core RT-Xen 1.0 Ø Single-core enhanced RT-Xen 1.1 Ø Multi-core scheduling RT-Xen 2.0 q RT-global q RT-partition Fixed Priority (RM) Capacity Reclaiming Periodic Work Conserving Periodic Polling Deferrable Periodic Periodic Deferrable Sporadic Global Scheduling Deferrable Periodic Periodic Deferrable ParGGoned Scheduling Dynamic Priority (EDF) 5/27/14 12

14 Experimental Setup Ø Hardware: Intel i7 processor, 6 cores, 3.33 GHz q Allocate 1 VCPU for Domain-0, pinned to PCPU 0 q All guest VMs use the remaining cores q Software q Xen 4.3 patched with RT-Xen q Guest OS: Linux patched with LITMUS Ø Workload q Period tasks: synthetic, ARINC 653 avionics workload (RT-Xen 1.1) q Allocate tasks à VMs 5/27/14 13

15 RT- Xen 2.0: Credit Scheduler Credit misses deadlines at 22% of CPU capacity. RT-Xen delivers real-time performance at 78% of CPU capacity. 5/27/14 14

16 RT- Xen 2.0: Scheduling Overhead rt-global has extra overhead due to global lock. Credit has poor max overhead due to load balancing. 5/27/14 15

17 RT- Xen 2.0: Theory vs. Experiments Fraction of schedulable tasksets gedf (RT Xen) pedf (RT Xen) pedf (CSA Theory) gedf (CSA Theory) Task Set Utilization gedf > pedf empirically, thanks to work-conserving global scheduling. gedf < pedf theoretically due to pessimistic analysis. 5/27/14 16

18 RT- Xen 2.0: Deferrable vs. Periodic Work-conserving wins empirically! Deferable Server (DS) > Periodic Server. gedf+ds à best real-time performance. 5/27/14 17

19 RT- Xen 2.0: How about Cache? pedf in guest OS gedf > pedf for cache intensive workload. Benefit of global scheduling dominates migration cost. Shared cache mitigates cache penalty due to migration. 5/27/14 18

20 Real- Time Inter- VM Communica2on Ø Real-time communication between high priority VMs under contention from other VMs Ø Local communication between VMs on a same host q ~20 VMs on a single host is not rare. q Co-locate VMs to reduce communication cost. Ø Network communication (on-going) 5/27/14 19

21 Is real- 2me VM scheduling enough? 1 CDF Plot 0.5 RT Xen, Original Dom Micro Seconds 5K data points Dom 0 Dom 1 Dom 2 Dom 5 Dom 4 Dom 3 VMM Scheduler C 0 C 1 C 2 C 3 C 4 C 5 5/27/14 20

$Xen Communica2on Architecture A Domain 1 Domain 0 negf netback[0] { rx_ac2on();$ Domain n C nearont negf sobnet_data negf D nearont Significant priority inversion

Domain n C nearont negf sobnet_data negf D nearont Significant priority inversion

22 Xen Communica2on Architecture A Domain 1 Domain 0 negf netback[0] { rx_ac2on(); tx_ac2on(); } negf B Domain 2 nearont Domain m negf TX netback RX negf nearont Domain n C nearont negf sobnet_data negf D nearont Significant priority inversion in Domain 0 Packets are fetched in a round-robin order. Sharing one queue in softnet_data. 21

$RTCA: Real- Time Communica2on Architecture A Domain 1 Domain 0 negf netback[0] { rx_ac2on(); tx_ac2on(); } negf A Domain 2 nearont Domain m negf TX netback RX negf nearont Domain$

23 RTCA: Real- Time Communica2on Architecture A Domain 1 Domain 0 negf netback[0] { rx_ac2on(); tx_ac2on(); } negf A Domain 2 nearont Domain m negf TX netback RX negf nearont Domain n B nearont negf sobnet_data negf B nearont Preserve prioritization in Domain 0 Packets are fetched by priority, up to batch size Queues are separated by priority in softnet_data 22

24 Real- Time Inter- VM Communica2on Median 75 th percen2le 95 th percen2le Domain- 0 Original RTCA Original RTCA Original RTCA Base Light Medium Heavy RTCA maintains low latency for high-priority VMs despite contention from low-priority VMs 23

25 Conclusion Ø Diverse applications demand real-time virtualization. q Embedded real-time systems. q Internet-scale cyber-physical systems. q Latency-sensitive cloud applications. Ø RT-Xen provides real-time performance q Efficient implementation of diverse real-time scheduling policies. q Leverage compositional scheduling theory à analytical guarantee. q Resource interface à resource allocation for latency bounds. q RTCA supports real-time inter-vm communication. Ø On-going q RT-Xen patch for Xen core distribution. q RT-OpenStack: integration with OpenStack on the way. 5/27/14 24

26 Check out RT- Xen RT- Xen hdps://sites.google.com/site/realgmexen/ Team Ø Sisu Xi, Justin Wilson, Chong Li, Chris Gill, Washington University Ø Meng Xu, Jaewoo Lee, Insup Lee, Lihn Phan, Oleg Solosky, University of Pennsylvania 5/27/14 25

RT- Xen: Real- Time Virtualiza2on. Chenyang Lu Cyber- Physical Systems Laboratory Department of Computer Science and Engineering

RT- Xen: Real- Time Virtualiza2on Chenyang Lu Cyber- Physical Systems Laboratory Department of Computer Science and Engineering Embedded Systems Ø Consolidate 100 ECUs à ~10 multicore processors. Ø Integrate