Best Practices for designing Farms and Clusters

Transcription

1 Best Practices for designing Farms and Clusters Daniel McLean Systems Engineer Strategic Accounts Thank you to: Nathan Wheat, Andy Meakin and Michael Francis

2 Objectives Speak and be understood! What is a cluster? How can designing a cluster help with scaling out a Virtual Infrastructure? An approach to system sizing and chargeback. An approach to sizing the cluster.

3 Overview VI Cluster

4 Overview VI Cluster Compute Units

5 Building the Virtual Enterprise Compute Units Compute Units Compute Units VI Cluster VI Cluster VI Cluster Compute Units Compute Units Compute Units VI Cluster VI Cluster VI Cluster

6 Why Many Clusters? VI Cluster Compute Units Tier 1 Service Offering N+2, dual path, highly redundant servers, HA, DRS, VCB. Tier 1 storage with data mirroring. Resource guarantees. Tier 4 service offering No resource guarantees, HA if possible, oversubscription, DRS, Tier 3 storage Technological Performance Software/Hardware Limits Services offered (DRS, HA) Tiered Storage or servers Redundancy Geographic Social/Political Business Unit Environmental Dev/Test, QA, Production DMZ

7 Common Questions One Answer VMs on a server? LUNs to a server? It How Depends! Many NICs per server? VMs on a LUN? LUNs per server?

8 How Will This Help? The It Depends is based on each customer environment. Sometimes we don t know how many VMs we need to have Sometimes we don t know the load of a system before we test it The information discussed is to gain an appreciation of what needs to be considered. Move the focus of your designs away from servers and onto clusters. This will likely resolve performance, vmotion, scalability issues. Experience and VMware recommendations will put you in the ball park. A Professional services engagement will hit the home run.

9 Clusters What is a VI Cluster VI Cluster

10 Clusters Elements of a Cluster VI Cluster Compatible CPU between all hosts to support vmotion. Speed and quantity can differ. A common Gb network between all hosts for vmotion, host management and VMs Enough memory in hosts to support VMs. Consistency amongst hosts. Shared storage accessible to all hosts for Virtual Machines

11 The Virtual Infrastructure Unit A compute unit of infrastructure Desiging for the normal, not the exception. PROs Helps with planning new Virtual Machines and with sizing the cluster Is a great way to charge back to business units CONs Is static and can result in some underutilisation (Small) VI Unit

12 VI Unit a Working Example Think about the most basic server in your environment your SOE (Standard Operating Environment) We are mainly concerned with RAM and CPU Decide on the ratio of vcpu to physical CPUs We will use 8:1 Create 3 VM sizes Light, Standard & Heavy This only alters the memory and the CPU. Disk and network come later VM Element Base Unit Notes vcpu 1 1/8 of a physical CPU Memory 512Mb Minimum memory to run corporate server or desktop VM Network 1 1/8 of a physical Gb NIC Disk 10Gb + Data Server Element Light 1 Unit vcpu 1 Memory 512Mb To hold standard image OS and base apps Standard 2 Units Heavy 4 Units 1 1 or 2 1Gb 2Gb

13 VI Unit Physical Server Sizing We use the ratio to determine the physical server size. (we used 8:1) Multiple the VI Unit memory by this ratio to work out how much RAM is needed per CPU in the machine. 512Mb x 8 = 4Gb 2* Gb NICs for the server (Mgmt, vmotion) + 1Gb for every CPU HBA, Min of 2 For my example VI Unit (1 vcpu, 512Mb RAM) *Add separate NICs if using iscsi software initiator Physical Server Size CPU Memory 2-way 2 8Gb 4 4-way 4 16Gb 8 NIC 8-way 8 32Gb 10 Number of Units Light Standard 2-way way Heavy 8-way

14 Cluster Sizing We have our VI Unit, now we need somewhere to run it. The Cluster. What do we need to consider when sizing the cluster? How big is the storage, how big is the server? What affects performance for better or worse? How do we charge for it? Goal To reach our desired utilisation across all elements Memory, CPU, Disk etc. We don t want to overload the servers and have lots of free space on the disk volumes. To recover the cost of the infrastructure

15 Considerations for Sizing a Cluster VIUs per DataStore The number of VMs or VI Units per datastore Determined by amount of IO going to and from the storage Will depend on type of storage and load from VM Affected by Storage performance VM Load Infrastructure changes Redo Logs Good Rule of thumb for FC SAN is Less then 100 Lightly loaded VMs (80) Less then 40 Standard VMs Less then 20 Heavy VMs VIUs/DS x DS = Total VIUs per cluster E.g 80 per DS x 5DS = 480 VIUs per cluster

16 Considerations for Sizing a Cluster Hosts per DataStore The number of ESX hosts connected to one Datastore Affected by Number of Infrastructure Changes leading to VMFS locks Power operations, vmotions, creation and deletion of VMs, Redo Logs Better in ESX3 then ESX2 still must be considered The more Infrastructure Changes the less hosts per cluster The number of hosts per DS will determine the number of Hosts in the Cluster. E.g. 12 hosts per datastore = 12 hosts per Cluster

17 Considerations for Sizing a Cluster DataStores per Host The number of Datastores connected to a single host. Performance affected by Type of array Enterprise Active/Active High performance Mid-range Active/Passive Mid performance NAS/iSCSI/FC SAN Speed of HBA and number of paths Type of VM Light or Heavy IO Often a recommendation by the storage vendor The number of DS per Host will determine the maximum number of DS for the Cluster. E.g. 6 DS per Host = 6 DS per Cluster

18 Considerations for Sizing a Cluster - Summary VIUs per DataStore Hosts per DataStore DataStores per Host Light 80:1 Standard 40:1 Heavy 20:1 ESX2 10:1 ESX3 16:1 Enterprise 12:1 Mid-range 6:1 VIUs/DS X DS = total number of VMs for cluster. Hosts/DS = total number of hosts for the cluster DS/Host = Max number of DS for Cluster

19 Templates and ISO Datastore Cluster 1 Staging Server Template/ISO LUN Cluster 2

20 Putting It Together We need to consider all aspects when sizing the cluster. If we only take one or two items into account when we design we get huge potential for problems. Example: Limit VMs per Datastore to 30. Valid for high performing VMs. The hosts are 4 way servers expected to run 20 VMs per server. After some rapid growth there are now 600 VMs running in the environment. To run this load there are: 28 Physical hosts and 20 datastores. Serious performance issues start to occur. (Storage/SP contention) Still only running 20 VMs per server and 30 per datastore

21 Putting It Together Start with any item that you know about VMs a known quantity required Servers a given type of server (Dual or Quad core) Storage a known array Consider all aspects of the cluster and you won t go wrong Ensure that the assumptions you make do not go outside any of the boundaries unless you test, test and test. Got it Wrong? Easy to change, move the boundaries of the cluster around. Set in stone? No, as hardware and software changes, so will these limits Storage, servers, software gets faster and better with each new release. Be flexible

22 Storage Sizing - VMs Virtual Machines Storage System Volume for OS (all VMs) Data Volume (Some VMs) Data Volume is sized per VM based on the application requirements. System Volume is standard for all VMs e.g. 10Gb System Volume Be frugal with what you include. Do you need the i386 dir locally Don t be too Frugal, you still need the apps installed, some free space and a swap file VM Application Data storage placement: Light Standard Medium Heavy Data volumes in VMDK Data volumes in VMDK Data volumes in VMDK or RDM (Raw Disk Mapping), sized according to application best practices (no of disks and RAID level) Data volumes on RDM (Raw Disk Mapping), sized according to application best practices (no of disks and RAID level)

23 Storage Sizing - Datastore DataStore Sizing Need to consider VM disk files VM Swap Configuration files Redo/snapshots files Metadata Example Total number of VM disks (40VMs) 40 X 10Gb = 400Gb Add VM config files and swap (80VIUs) Swap = ½ of Memory VIU Memory = 512Mb 80 x 256 = 21Gb Logs and config 10Mb per VM x 40 = 400Mb ~422Gb of space for 40VMs Still need to add space for Redo logs VCB will need this

24 Chargeback Costing a Virtual Infrastructure Complex question Finance Function Must adhere to finance rules Some conditions Generally must be fixed. Otherwise hard to budget for next year Utilisation charging is difficult to budget. Focus on cost allocation/recovery independent of utilisation Different approaches Focus of today is on fixed cost recovery Suits environments that need fixed, predictable costs for an accounting period or budget cycle Simple to administer and report Pay for performance and availability Defer metering or utilisation based chargeback until later

25 Cost Recovery Base Costs Base Costs Virtual Infrastructure Server Hardware, Storage, Network, Datacenter space (Power, cooling) Infrastructure Software VMware etc. Provisioning costs Design, installation, maintenance = Total Cost to Build Cluster Base Cost to VI Unit cost to VM cost Divide the total Base cost by number of VI Units in cluster. A per unit cost Multiply the number cost of the VIU by the number of VIU in a VM. E.g. A heavy VM is 4 x VIU cost = Cost of 1 VI Unit Goal When all VI Units are sold in a cluster, the cost of the infrastructure should be fully covered. These are defined before any VM is deployed. Effectively replaces what used to be a server cost. Consider Hardware depreciation, costed over the life of the hardware so you have money to replace it when required. A tax to cover growth Different values for different SLA and availability.

26 Cost Recovery Service Costs Service Costs Virtual Machines Software for VM OS, Apps, agents (backup and monitoring) Hardware for VM Additional storage, installation, maintenance Include Needs to cover items unique to that VM special support requirements, different SLA s etc. = Cost to deploy a VM = Cost to maintain a VM Goal The cost is defined at time of design. Depends on what service is being offered. i.e. a database server has addition disk requirements and a platinum SLA. Consider Software Capex and Opex per VM. What infrastructure services are standard (back up, DR etc), what cost do these attract. A service Catalogue for common VMs A cost template for DBMS, Webservers, Infrastructure Servers)

27 Putting the Cost Together Total Cost of VM VM Service Cost + VI Unit Cost x VI Units Create different chargeback tiers to match the cluster tiers Price for Performance Keep the price attractive If your model comes out a lot higher then for a physical system, customers will shy away Consider Promotional pricing early on to get people on the system Don t compare 1 physical to 1 VM when looking at costs. To get HA for a physical server means at least 2 servers for 1 application VMs are much better then physical servers More services and flexibility with Virtual

28 Summary Were you able to understand me? Clusters are the building block of Virtual Infrastructure Get the cluster design correct and the environment will sing. Consider all aspects of the cluster when building it VM, storage, network, Server Use VI Units to help in sizing estimates and to assist in charge back Chargeback should map to the finance process. An approach to sizing the cluster.

29 Presentation Download Please remember to complete your session evaluation form and return it to the room monitors as you exit the session The presentation for this session can be downloaded at Enter the following to download (case-sensitive): Username: cbv_rep Password: cbvfor9v9r

30 Some or all of the features in this document may be representative of feature areas under development. Feature commitments must not be included in contracts, purchase orders, or sales agreements of any kind. Technical feasibility and market demand will affect final delivery.