LIA. Large Installation Administration. Virtualization

Transcription

1 LIA Large Installation Administration Virtualization

2 2 Virtualization What is Virtualization "a technique for hiding the physical characteristics of computing resources from the way in which other systems, applications, or end users interact with those resources." Virtualization is the process of making things more abstract in order to make them easier to use.

3 3 Examples?

4 4 Storage virtualization Files Linear sequence of bytes Instead of blocks on a disk (or magnetic particles) Disk partitioning LBA RAID - redundant array of independent disks Logical Volume management Combines disks and partitions into logical disks.

5 5 Network virtualization VLAN Channel bonding multiple links combined offered a single, higherbandwidth link Computer clusters Multiple logical networks on same physical wires multiple discrete computers into larger metacomputers e.g. Hadoop Virtual NICs and bridges for VM communication

6 6 Resource virtualization Multiprogramming Each process thinks it has CPU to itself Virtual memory Present linear address space composed of nonconsecutive blocks of: Physical memory Disk space

7 7 Virtual Machines

8 8 Credits Slides largely based on: Virtual Machines: Versatile Platforms for Systems and Processes James E. Smith Ravi Nair Morgan Kaufmann Publishers 2005

9 9 Why Virtual Machines? Isolate applications in separate VMs Sandbox applications for security Support different OSes concurrently Legacy applications on legacy OSes Application testing using VMs with known state Testing OS upgrades, training OS development

10 10 Why Virtual Machines? LIA context: Resource utilization Server consolidation Facilitate maintenance Basis for cloud computing

11 11 Computer Architecture Recap

12 12 Instruction Set Architecture Software ISA Hardware E.g. x86_64

13 13 User + System ISA Application Programs System ISA Operating System Hardware User ISA

14 14 User ISA = For doing computations Simple Memory Instructions Move data from memory to registers and v.v. Integer Instructions Floating-Point Instructions Branch instructions Jump to address Jump to address if... Part of SHA1 in assembly: addl movl xorl andl xorl %esi, %e; %c, %esi; %d, %esi; %b, %esi; %d, %esi;

15 15 System ISA (1/2) = Management of system resources System Resources: Management: Main memory Storage Other I/O devices Fair allocation between user programs Prevent concurrent/unauthorized access Role of the Operating System

16 16 System ISA (2/2) OS requires special privileges over user programs OS runs in CPU Kernel mode Apps run in CPU User Mode x86: Implemented via 2 privilege levels / rings User Mode Kernel Mode Ring

17 17 System ISA Instructions Processor Management Memory Management Manage page table, TLB: virtual memory I/O Management Change to user mode + run application Timer interrupt gives control back to OS load and store to/from device Traps Change to kernel mode from application On purpose (system call) or on exception

18 18 System Call Instruction Application Programs Operating System System call Handler I/O instructions Hardware Disk System call: read(file) E.g. syscall on x86 or svc on ARM.

19 19 System Call = Mechanism for User Mode code to request services from Kernel Mode What services? Read/write to files and devices Create processes i.e. use Operating System abstractions: File abstraction for storing blocks on disk Process abstraction for running different code in parallel

20 20 System Call Interface (1/2) Application Programs System calls Operating System ISA Hardware

21 21 System Call Interface (2/2) Read/write files or devices: CreateFile(...) ReadFile(...) WriteFile(...) SetConsoleMode(...) Manipulate processes open(filename, ) read(fd,data, ) write(fd,data, ) ioctl() fork() exit() More... CreateProcess( ) ExitProcess( )

22 22 System calls via libraries Application Programs Libraries System calls Operating System Hardware E.g. libc

23 23 Architecture Model Application Programs Run Libraries System calls Kernel Mode Operating System Interrupts, Traps, faults Privileged instructions Hardware User Mode

24 24 Virtual Machines

25 25 Virtualize the machine? What is the machine? Machine is defined by an interface 3 interfaces that can be virtualized: 1. Instruction Set Architecture (ISA) 2. Application Binary Interface (ABI) 3. Application Programming Interface (API)

26 26 Interface 1: ISA Application Programs Libraries System calls Operating System ISA Hardware Virtualize a complete machine, running an OS supporting multiple processes = System VM

27 27 Interface 2: ABI Application Programs Libraries System calls Operating System ABI = Hardware Virtualize the environment of a single process = Process VM System calls + user ISA

28 28 Interface 3: API Application Programs Libraries System calls Operating System API = Hardware Libraries + user ISA

29 29 Example Virtualizing ISA Support a machine's complete ISA VM/370 Xen* KVM*

30 30 Example Virtualizing ABI Run binaries unmodified on different platform Sun WABI Run Win32-x86 binaries on Solaris-SPARC Digital FX!32 Run Win32-x86 binaries on Win32-Alpha

31 31 Example Virtualizing API Recompile applications from source Runs on any platform with same API E.g. Linux-x86 and Linux-ARM (Assuming platform-independent code)

32 32 VM Implementations Application Programs Application Programs Libraries Guest OS Virtual Machine Monitor Virtual Machine Monitor Hardware (a) System VM Host OS Hardware (b) Process VM

33 33 What ISA? Same or different Same: Run Win32-x86 on Linux-x86 Diff: Run Linux-ARM on Win32-x86 Application Programs Libraries OS Source ISA Virtual Machine Monitor Target ISA Hardware

34 34 Taxonomy Process VMs System VMs Same ISA Different ISA Same ISA Different ISA Multiprogrammed Systems Emulators/ Translators Classic-System VMs Whole-System VMs Hosted VMs Codesigned VMs High-level Language VMs

35 35 Example: Windows Multiprogramming Win32 native Win32 native Win32 API Windows OS One CPU, illusion of processes running in parallel X86 Hardware Process

36 36 Example: Android Java High-level Language VM (HLL-VM) Java VM + Libs Linux OS ARM Hardware Different ISA: Java vs. ARM

37 37 Example: Android Emulation Java Java VM Win32 native Linux OS Win32 API ARM VM Runtime Windows OS X86 Hardware Process

38 38 Example: Android Emulation on Xen Java Java VM Linux OS Win32 Native Linux Native ARM VM Runtime Windows OS Xen domain Linux OS Xen Hypervisor X86 Hardware Linux Native

39 39 Example: VMWare Applications VMApp Guest OS VM Driver VMMonitor Host OS X86 Hardware

40 40 Example: AS/400 Application Programs Higher level ISA OS Source ISA Virtual Machine Monitor Target ISA Allow evolution of hardware ISA Hardware

41 41 Taxonomy Examples Process VMs System VMs Same ISA Different ISA Same ISA Different ISA Multiprogrammed Systems UNIX Emulators/ Translators FX!32 Classic-System VMs VM/370 Whole-System VMs ARM VM runtime High-level Language VMs Java VM, MS CLR Hosted VMs VMware, Xen, Docker* Codesigned VMs AS/400

42 42 Implementing Virtual Machines with Different ISAs

43 43 VM implementation: Emulation Emulation = implement interface of one system on another system with a different interface Example: x86 instruction addl %edx,4(%eax) Emulated via PowerPC instructions: lwz lddi lwzx lwz add stw r4,0(r1) r5,r4,4 r5,r2,r5 r4,12(r1) r5,r4,r5 r5,12(r1)

44 44 Emulation Model e Sa Sb Source Executing instruction e changes state of Source machine from Sa to Sb

45 45 Emulation Model e Sa Sb Source S'b Target e' S'a Real machine has corresponding state S'a Performs e by means of instruction(s) e'

46 46 Recall: Registers + Memory r0 r1 r2 rn PC CPU MEM

47 47 Example: Change registers e Source CPU MEM CPU MEM e' Target CPU MEM CPU MEM

48 48 Example: Change registers e Source CPU MEM CPU MEM e' Target CPU MEM CPU MEM CPU state of Source kept in Target memory, not registers!

49 49 Emulation Performance + Methods Can be slow because of mapping Source to Target! Range of emulation methods: Interpretation: Interpretation Binary translation Decode a single source instruction and execute using target instructions Binary translation: Translate a block of source instructions once and reuse

50 50 Interpretation Source instruction is a series of bytes Different formats RISC: clean and simple CISC: complex with legacy Non-hardware: Java bytecodes Complexity of format influences interpretation performance!

51 51 Example Formats x86: Prefixes Opcode 0-4 bytes Opcode ModR/M SIB optional optional optional Displace- Immediate ment 0,1,2,4 0,1,2,4 bytes Software developer's manual: 3796 pages! Java: Opcode Index Opcode Index1 Index2 Opcode Data1 Data2 Java VM Specification: 604 pages

52 52 x86 Format Prefixes 0-4 bytes Opcode Opcode ModR/M SIB optional optional optional Displace- Immediate ment 0,1,2,4 0,1,2,4 bytes Prefixes: Repetition for strings, overrides for address and operand sizes ModR/M: addressing mode and which register SIB: base register, index register, index scale Displacement: offset to be added to address Immediate: variable length operand

53 53 Binary Translation Per-instruction interpretation slow Alternative: Especially when complex Translate blocks of source instructions once Reuse cf. Just-in-Time compilers Hard

54 54 Performance Tradeoff E(n) = time needed to execute an instruction n times Formula: E(n) = S + n*t S = startup time T = time required per emulation of the instruction Interpretation: S low, T high Binary translation S high, T low

55 55 Performance Tradeoff Interpretation E Binary Translation n In practise: automatic profiling, often used code is binary translated

56 56 OS Emulation For Process VMs have to emulate whole ABI User ISA System call interface System call instructions (e.g. sysenter) emulated Translate from Source OS to Target OS Same OS: straightforward to hard Diff OS: straightfoward to impossible No guarantees that Target OS has same features as Source! E.g. fsync()