Overview of System Virtualization: The most powerful platform for program analysis and system security. Zhiqiang Lin

Size: px

Start display at page:

Download "Overview of System Virtualization: The most powerful platform for program analysis and system security. Zhiqiang Lin"

Myles Pierce
5 years ago
Views:

1 CS 6V81-05: System Security and Malicious Code Analysis Overview of System Virtualization: The most powerful platform for program analysis and system security Zhiqiang Lin Department of Computer Science University of Texas at Dallas April 4 th, 2012

2 Outline 1 Background 2 Virtualization Types 3 Key Techniques 4 Summary

3 Outline 1 Background 2 Virtualization Types 3 Key Techniques 4 Summary

4 Why virtualization? Why not virtualize? Today s data center Cloud computing Virtual resource pool Fault resist Cost Effect Easy to maintain

5 What is Virtualization What is Virtualization Linux Linux (devel) XP Vista MacOS Virtual Machine Monitor Hardware

6 Virtualization Properties Isolation Encapsulation Interposition

7 Isolation Fault Isolation Fundamental property of virtualization Performance Isolation Accomplished through scheduling and resource allocation

8 Encapsulation All VM state can be captured into a file Operate on VM by operating on file mv, cp, rm Complexity Proportional to virtual HW model Independent of guest software configuration

9 Interposition All guest actions go through monitor Monitor can inspect, modify, deny operations Example: Compression Encryption Profiling Translation

10 Why Not the OS? It about interfaces VMMs operate at the hardware interface Hardware interface are typically smaller, better defined than software interfaces Disadvantages of being in the monitor Low visibility into what the guest is doing

11 Virtualization benefits Increased resource utilization: Mobility Server consolidation Multiplexing Enhanced Security Test and Deployment...

12 Virtualization: server consolidation Virtualization: server consolidation Physical Servers Virtual Server Host Virtual Machines Guests

13 Mobility: load balance (Migration) Mobility: load balance Server 1 CPU Utilization = Server 2 CPU Utilization = 90% 50% VM Guest 1 VM Guest 2 VM Guest 3 VM Guest 4 VM Guest 5 None VM Guest 6 VM Guest 7

14 Mobility: load balance (Migration) Mobility: load balance Server 1 CPU Utilization = Server 2 70% CPU Utilization = 70% VM Guest 1 VM None Guest 2 VM Guest 3 VM Guest 4 VM Guest 5 None VM Guest 2 VM Guest 6 VM Guest 7

15 Enhanced Security Enhanced Security Conventional VM Secure VM Firewall VMM

16 Testing and Deployment Production VM Production VM Development VM QA VM Production VM Production VM Develop Test Deploy

17 Outline 1 Background 2 Virtualization Types 3 Key Techniques 4 Summary

18 Types of Virtualization Process Virtualization Language construction Java,.NET Cross-ISA emulation Apple s PowerPC to Intel X86 Transition Application virtualization Sandboxing, mobility

19 Types of Virtualization Process Virtualization Language construction Java,.NET Cross-ISA emulation Apple s PowerPC to Intel X86 Transition Application virtualization Sandboxing, mobility System Virtualization VMware Xen Microsoft s Viridian

20 Taxonomy Taxonomy Process VMs System VMs Same ISA Different ISA Same ISA Different ISA Multiprogrammed Systems Dynamic Translators Classic-System VMs Whole-System VMs Dynamic Binary Optimizers HLL VMs Hosted VMs Co-designed VMs

21 System Virtual Machine Monitor Architectures Traditional Hosted Hybrid VMware Workstation VMware ESX Xen Hypervisor

22 Traditional What is Virtualization Linux Linux (devel) XP Vista MacOS Virtual Machine Monitor Hardware Examples: IBM VM/370, Stanford DISCO

23 Hosted Virtual Machines Goal: Why Run Virtual Machines as an application on an existing Operating System (QEMU, VirtualPC, VirtualBox, VMware-workstation) Application continuity Reuse existing device drivers Leverage OS support File system CPU Scheduler VM management platform

24 Hosted Monitor Architecture (VirtualBox) Hosted Monitor Architecture User App Guest OS (Linux) World Switch Kernel Module Host OS (Window XP) Virtual Machine Monitor Hardware

25 Hosted Monitor Architecture (VirtualBox) Hosted Monitor Architecture User App Guest OS (Linux) Kernel Module Host OS (Window XP) Virtual Machine Monitor CPU / Memory Virtualization Hardware

26 Hosted Monitor Architecture (VirtualBox) Hosted Monitor Architecture User App Guest OS (Linux) Kernel Module Host OS (Window XP) Device I/O Network, Disk, Display, Keyboard, Timer, USB Virtual Machine Monitor Hardware

27 Hosted Monitor Architecture (VirtualBox) Hosted Monitor Architecture User App Guest OS (Linux) Kernel Module Host OS (Window XP) Interrupts Virtual Machine Monitor Hardware

28 Hosted Monitor Scheduling (VirtualBox) Hosted Monitor Scheduling User App User App Guest OS (Linux) Guest OS (Vista) Kernel Module Host OS (Window XP) Virtual Machine Monitor Virtual Machine Monitor Hardware

29 Hosted Monitor Scheduling (VirtualBox) Hosted Monitor Scheduling 3 User App User App Guest OS (Linux) Guest OS (Vista) 1 CPU Scheduler 2 Kernel Module Host OS (Window XP) Virtual Machine Monitor Virtual Machine Monitor Hardware

30 Hosted Monitor Scheduling (VirtualBox) Hosted Monitor Scheduling User App User App Guest OS (Linux) Guest OS (Vista) 5 CPU Scheduler Kernel Module Host OS (Window XP) 4 Virtual Machine Monitor Virtual Machine Monitor Hardware

31 Hosted Monitor Scheduling (VirtualBox) Hosted Monitor Scheduling 8 User App User App Guest OS (Linux) Guest OS (Vista) 6 CPU Scheduler 7 Kernel Module Host OS (Window XP) Virtual Machine Monitor Virtual Machine Monitor Hardware

32 Hosted Architecture Tradeoffs Positives Installs like an application No disk partitioning needed Virtual disk is a file on host file system No host reboot needed Runs like an application Uses host schedulers

33 Hosted Architecture Tradeoffs Positives Installs like an application No disk partitioning needed Virtual disk is a file on host file system No host reboot needed Runs like an application Negatives Uses host schedulers I/O path is slow Requires world switch Relies on host scheduling May not be suitable for intensive VM workloads

VMware ESX 2.0 Source: http://www.vmware.

34 VMware ESX 2.0 Source:

35 Hybrid Ex 2 - Xen 3.0 Para -virtualization Linux Guest Hardware-supported virtualization Unmodified Windows Isolated Device Drivers Source: Ottawa Linux Symposium 2006 presentation.

36 Hypervisor Hardware-supported Hardware-supported single-use single-use monitor monitor Characteristics Characteristics Small size Small size Runs in a special Runs in a special hardware mode hardware mode Guest OS runs in Guest OSnormal runs in privileged level normal Uses privileged level Uses Security System management Security Fault tolerance System management Fault tolerance Hypervisor Operating System Hypervisor Hardware User Mode Kernel Mode Monitor Mode

37 Outline 1 Background 2 Virtualization Types 3 Key Techniques 4 Summary

38 Key Techniques to implement Virtualization Instruction Interpretation Binary Translation Trap-and-emulate Para-virtualization (hardware virtualization)

39 I Instruction Interpretation Emulate Fetch/Decode/Execute pipeline in software Positives Easy to implement Minimal complexity Negatives Slow! Bochs:

40 Example: CPUState static struct { uint32 GPR[16]; uint32 LR; uint32 PC; int IE; int IRQ; } CPUState; void CPU_CLI(void) { CPUState.IE = 0; } void CPU_STI(void) { CPUState.IE = 1; }

41 Example: Virtualizing the Interrupt Flag w/ Instruction Interpreter void CPU_Run(void) { while (1) { inst = Fetch(CPUState.PC); CPUState.PC += 4; switch (inst) { case ADD: CPUState.GPR[rd] = GPR[rn] + GPR[rm]; break;... case CLI: CPU_CLI(); break; case STI: CPU_STI(); break; } if (CPUState.IRQ && CPUState.IE) { CPUState.IE = 0; CPU_Vector(EXC_INT); } } } void CPU_CLI(void) { CPUState.IE = 0; } void CPU_STI(void) { CPUState.IE = 1; } void CPU_Vector(int exc) { CPUState.LR = CPUState.PC; CPUState.PC = distab[exc]; }

42 II. Binary Translator Binary Translator Guest Code Translator TC Index Translation Cache Callouts CPU Emulation Routines

43 Basic Blocks Guest Code Basic Blocks vpc mov ebx, eax cli and ebx, ~0xfff mov ebx, cr3 sti ret Straight-line code Control flow Basic Block

44 Binary Translator Binary Translation Guest Code Translation Cache vpc mov ebx, eax cli and ebx, ~0xfff mov ebx, cr3 sti ret mov ebx, eax call HANDLE_CLI and ebx, ~0xfff mov [CO_ARG], ebx call HANDLE_CR3 call HANDLE_STI jmp HANDLE_RET start

45 Unprivileged Privileged III. Trap and Emulate Trap and Emulate Guest OS + Applications Page Fault Undef Instr virq MMU Emulation CPU Emulation I/O Emulation Virtual Machine Monitor

46 Software VMM Software VMM Direct Exec (user) Faults, syscalls, interrupts Guest Kernel Execution VMM Translated Code (guest kernel) IRET, sysret Traces, faults, interrupts, I/O

47 IV. Hardware virtualization, Para-virtualization (e.g., Xen) Modify operating systems to let OS to cooperate with VMM Let OS runs in Ring1 and user apps run in Ring3 Change sensitive instructions to explicit calls to the VMM No need to trap and emulate

48 Traditional Address Spaces Traditional Address Spaces 0 4GB Process Virtual Address Space Background Process Operating System Background Process Operating System 0 4GB Current Process Operating System Virtual Address Space 0 4GB RAM Frame Buffer Devices ROM Physical Address Space

49 Shadow Page Tables Sol: Shadow Page Tables Virtual CR3 Guest Page Table Guest Page Table Guest Page Table Real CR3 Shadow Page Table Shadow Page Table Shadow Page Table

50 Guest Write to CR3 Guest Write to CR3 Virtual CR3 Guest Page Table Guest Page Table Guest Page Table Real CR3 Shadow Page Table Shadow Page Table Shadow Page Table

51 Guest Write to CR3 Guest Write to CR3 Virtual CR3 Guest Page Table Guest Page Table Guest Page Table Real CR3 Shadow Page Table Shadow Page Table Shadow Page Table

52 Outline 1 Background 2 Virtualization Types 3 Key Techniques 4 Summary

53 Summary Virtualization (or virtualisation), in computing, is the creation of a virtual (rather than actual) version of something, such as a hardware platform, operating system, storage device, or network resources Impacts Cloud computing Data centers Provides greater opportunities to security

54 References Xen and the art of virtualization Memory resource management for Vmware Esx Server (great virtualization resources) CPU Virtualization- Basic Virtualization Memory Virtualization

Introduction to Virtual Machines. Carl Waldspurger (SB SM 89 PhD 95) VMware R&D

Introduction to Virtual Machines Carl Waldspurger (SB SM 89 PhD 95) VMware R&D Overview Virtualization and VMs Processor Virtualization Memory Virtualization I/O Virtualization Typesof Virtualization Process