Logical Domains (LDoms)

Size: px

Start display at page:

Download "Logical Domains (LDoms)"

Ophelia McGee
6 years ago
Views:

1 Logical Domains (LDoms) Liam Merwick LDoms Developer Sun Microsystems SPARC Platform S/W Group

2 Contents Background Features Architecture Components Future Roadmap Configuration Examples Q & A Page: 2

3 Background Page: 3

4 Consolidation USE IMPROVE EVANGELIZE Conventional Unix server utilisation is 7 to 15% Data center compression > Goal: Increase utilisation of equipment 10% Utilisation 20% Time 10% Page: 4

Solutions from Sun Hard Partitions Virtual Machines OS

Server Database Identity Server File Server Web Server Mail

Server OS Server Multiple OSes Single OS Trend to flexibility

Containers (Zones + SRM) Trend to isolation Solaris Resource

5 Solutions from Sun Hard Partitions Virtual Machines OS Virtualisation Resource Mgmt. Server Database Identity Server File Server Web Server Mail Server Calendar Server Database Web Server SunRay Server Database Server OS Server Multiple OSes Single OS Trend to flexibility Dynamic System Domains SPARC Logical Domains Solaris xvm Solaris Containers (Zones + SRM) Trend to isolation Solaris Resource Manager (SRM) VMware ESX Microsoft Virtual Server Solaris Containers for Linux lications Solaris Trusted Extensions Page: 5

6 Virtualisation for SPARC platforms New sun4v architecture Operating System Solaris X update (genunix) sun4u code Solaris X (genunix) Solaris X (sun4v) US-Z CPU code sun4v interface CPU Z SPARC hypervisor SPARC CPU Page: 6 Platform

7 Virtual Machine for SPARC USE IMPROVE EVANGELIZE Thin software layer between OS and platform hardware Hypervisor + sun4v interface Virtualises machine HW and isolates OS from register-level Delivered with platform not OS Not itself an OS sun4v virtual machine User OpenBoot User Solaris Hypervisor User SPARC hardware stable interface sun4v Page: 7

Logical Domains Technology USE IMPROVE EVANGELIZE

domain is a full virtual machine, with a dynamically

independent OS > Protection & isolation via SPARC hardware

Environment of choice Linux FreeBSD LDoms Hypervisor

8 Logical Domains Technology USE IMPROVE EVANGELIZE Virtualisation and partitioning of machine resources > Each domain is a full virtual machine, with a dynamically reconfigurable subset of machine resources, and its own independent OS > Protection & isolation via SPARC hardware and LDoms Hypervisor firmware LDom A LDom B LDom C LDom D OS Environment of choice Linux FreeBSD LDoms Hypervisor Platform Hardware CPU CPU CPU CPU CPU CPU CPU CPU Memory Memory Memory Memory I/O I/O Page: 8

9 Features Page: 9

10 Fundamentals Each virtual machine should appear as an entirely independent machine > own kernel, patches, tuning parameters > own user accounts, administrators > own disks > own network interfaces, MAC & IP addresses > Start, stop and reboot independently of each other Paravirtualised OS Page: 10

11 Features Add and remove cpus while OS is running Protection and isolation via SPARC hardware and LDoms Hypervisor firmware Service domains for I/O > direct I/O access Page: 11

12 Features UltraSPARC-T1 specific Up to 32 LDoms per box > CPU threads individually assignable to different LDoms Memory assignable to with an 8KB granularity Two PCI busses independently assignable to the same or different LDoms. Guest Domains can be configured, started and stopped independently > Without requiring a power-cycle of machine Page: 12

13 Features UltraSPARC-T2 specific 64 threads => up to 64 domains > more domains per box than with UltraSPARC-T1 On-chip NIU assignable to any domain. Page: 13

14 Architecture Page: 14

15 Architecture Logical Domains overview Hypervisor LDoms Manager Virtualised I/O Dynamic Reconfiguration (DR) Page: 15

16 Hypervisor Support Hypervisor software is responsible for maintaining separation between domains > Using extensions built into a sun4v CPU Also provides Logical Domain Channels (LDCs) so that domains can communicate with each other > Mechanism by which domains can be virtually networked with each other, or provide services to each other Page: 16

17 LDoms Manager One Manager per host HV > Controls Hypervisor and all its LDoms Exposes control interface via CLI Maps Logical Domains to physical resources > Constraint engine > Heuristic binding of LDoms to resources > Assists with performance optimisation > Assists in event of failures / blacklisting Page: 17

18 Direct I/O Logical Domain Traditional model > Existing drivers and devices continue to work Device Driver Logical Domain owns PCI root and tree Nexus Driver Privileged Hyper Privileged Hypervisor Virtual Nexus I/F Hardware I/O Bridge I/O MMU PCI Root PCI-Express Page: 18

19 Virtualised I/O Privileged Logical Domain A Virtual Device Driver Service Domain Virtual Device Service Device Driver /pci@b/qlc@6 Nexus Driver /pci@b Hyper Privileged Hypervisor Domain Channel Virtual Nexus I/F Hardware I/O MMU PCI R oot I/O Bridge PCI B Page: 19

20 Virtual (Block) Disk device USE IMPROVE EVANGELIZE Logical Domain 1 Logical Domain 2 Logical Domain 3 Service Domain vdisk Server vdisk Server vdisk client Driver vdisk client driver vdisk client driver Device- Driver Hypervisor Virtual SAN 1 Virtual SAN 2 I/O Bridge FC-AL I/F Page: 20

21 Virtual Ethernet device Logical Domain 1 Logical Domain 2 Logical Domain 3 Service Domain V-Ether Switch V-Ether Switch V-Ether Driver V-Ether Driver V-Ether Driver Device- Driver Hypervisor Virtual LAN 1: /24 Virtual LAN 2: 63.24/16 I/O Bridge Gb Ether I/F Page: 21

22 Dynamic Reconfiguration (DR) USE IMPROVE EVANGELIZE Ability to dynamically grow or shrink compute capacity of an LDom on demand No need to re-boot Solaris Simply add / remove: > CPUs > Memory (future) > I/O (future) Improve utilisation by balancing resources between LDoms Page: 22

23 LDoms Manager Logical Domain 1 Logical Domain 2 Logical Domain 3 Service Domain LDoms Manager OS DR Stack OS DR Stack OS DR Stack Hypervisor HV DR Stack Page: 23

24 Example: vcpu reconfiguration Logical Domain 1 Logical Domain 1 Logical Domain 2 V V V V V P P Hypervisor Hypervisor P P P Example command line operations: % ldm remove-vcpu 1 Logical_Domain_1 % ldm add-vcpu 1 Logical_Domain_2 Page: 24

25 Components Page: 25

26 Components Hardware & Software Hardware > UltraSPARC-T1 (Niagara) family of CMT processors > T1000 Netra CP3060 T6300 > T2000 Netra T2000 > UltraSPARC-T2 (Niagara2) family of CMT processors > T5120 T5220 T6320 Operating System > OpenSolaris (build 70+ recommended) > Solaris 10 11/06 (U3) or later > Linux Page: 26

27 Components Hardware & Software Firmware > or later > 6.5.x and 7.0.x now available LDoms Manager Software > (Solaris package - available as Free download) > 1.0 (For use with 6.4.x firmware) OS can and will be released/updated independently of Firmware/LDoms Manager Page: 27

28 Future Roadmap Page: 28

29 Latest Release: LDoms USE IMPROVE EVANGELIZE Rebooting of Control, Service and IO domains > Without bringing down guest domains Support for new platforms Updates to Firmware and LDoms Manager package Upgrade to Solaris 10 8/07 (U4) recommended > Independent of LDoms Page: 29

30 Key Features on Roadmap Warm Migration > Suspend and resume Live Migration Hybrid I/O Dynamic Reconfiguration > Virtual and Physical I/O > Memory Booting/Installing from DVD/ISO Using ZFS/SVM volumes Page: 30

31 Configuration Examples Page: 31

32 Configuring a machine to run LDoms Make resources available (remove from control domain) primary#> ldm set-mau 2 primary primary#> ldm set-vcpu 8 primary primary#> ldm set-mem 8g primary Configure services primary#> ldm add-vcc port-range= primary-vcc0 primary primary#> ldm add-vds primary-vds0 primary primary#> ldm add-vsw net-dev=e1000g0 primary-vsw0 primary Page: 32

33 Adding another domain (I) Add disk device primary#> ldm add-vdsdev /dev/dsk/c1t0d0s2 Adding resources primary#> ldm create domain1 primary#> ldm add-mem 2g domain1 primary#> ldm add-vcpu 4 domain1 primary#> ldm add-vdisk vdisk1 vol1@primary-vds0 domain1 primary#> ldm add-vnet vnet1 primary-vsw0 domain1 Show domain constraints primary#> ldm list -l domain1 Page: 33

34 Adding another domain (II) Booting the newly created domain primary#> ldm bind domain1 primary#> ldm start domain1 Connect to the console primary#> telnet localhost 5000 Page: 34

35 LDoms Community Newly Created community Website > Alias > Page: 35

36 More information BigAdmin Portal > Sun Blueprint > Beginners Guide to LDoms: Understanding and Deploying Logical Domains > Training Course > Solaris 10 Logical Domains Administration (SA-345-S10) > Page: 36

37 Liam Merwick Sun Microsystems

an introduction to logical domains

O c ta v e O r g e r o n an introduction to logical domains I n t h e F e b r u a r y 2 0 0 8 i s s u e o f ;login:, I discussed advanced topics concerning networking and storage for logical domains (LDoms).