Performance of VSA in VMware vsphere 5

Transcription

1 Perfrmance f VSA in VMware vsphere 5 Perfrmance Study TECHNICAL WHITE PAPER

2 Table f Cntents Intrductin... 3 Executive Summary... 3 Test Envirnment... 3 Key Factrs f VSA Perfrmance... 4 Cmmn Strage Perfrmance Cnsideratins... 5 Lcal RAID Adapter... 5 VSA Data Replicatin... 5 Mix f Reads and Writes... 6 Perfrmance Metrics... 6 Applicatin / VM... 6 NFS Datastre... 6 Physical SCSI Adapter... 6 Mixed Wrklad Test... 7 IOBlazer Test... 8 Best Practices Cnclusin Abut the Authr Acknwledgements TECHNICAL WHITE PAPER /2

3 Intrductin vsphere 5 cmbined with vsphere Strage Appliance (VSA) allws fr the creatin f shared strage frm lcal strage. This enables the use f vsphere features like vmtin, distributed resurce scheduling (DRS), and high availability (HA) withut a SAN. VSA uses either tw r three vsphere 5 hsts and the lcal strage that resides n them t create a VSA strage cluster. This cluster prvides shared strage in the frm f NFS shares that can be used t hst virtual machines (VMs). Additinally, VSA prvides redundancy with hardware RAID and data replicatin acrss the cluster s that even if ne f the server ndes becmes unavailable r a hard disk fails, the strage will cntinue t be available and useable via the remaining hsts. This results in a highly available, shared strage array created ut f lcal strage with n requirement fr additinal dedicated strage hardware. The perfrmance f VSA is directly related t the hardware cnfiguratin f the systems used fr its cluster. Big differences in capacity, perfrmance, and price exist depending n the exact cnfiguratin used. The data replicatin perfrmed by VSA acrss its cluster ndes prvides high availability, but als has an effect n perfrmance. Testing was dne with a mixed wrklad t examine applicatin perfrmance and infrastructure peratins. A set f tests with an I/O generatin tl were als run t examine what happens acrss the hsts in a VSA cluster and t illustrate hw t mnitr and manage perfrmance. Executive Summary VSA prvides the basic capabilities f shared strage t envirnments where it was nt pssible befre. This enables advanced features f a virtual infrastructure fr envirnments that are as simple as just tw hsts and an Ethernet switch. VSA is able t run a mix f applicatin wrklads while als supprting a range f dynamic virtual infrastructure peratins. As with any strage system, there are tradeffs f capacity, price, and perfrmance that can be made t achieve an ptimal slutin based n requirements. Test Envirnment A VSA cluster can be built n tw r three vsphere 5 hsts. The lcal strage f the hsts are cnfigured as hardware RAID 1/0 LUNs and used t create shared strage that is presented as an NFS share n each hst. The NFS shares reside n each vsphere 5 hst and can be used t hst VMs with vsphere 5 hsts using NFS t access VMs that are stred n the NFS datastres. VSA installatin and management was designed t be very simple and easy t use. The VSA installer des a thrugh check f requirements and will prmpt fr anything that is either missing r incrrect t be fixed befre prceeding. Once installatin is cmplete, there is nthing further that needs t be cnfigured r adjusted. All f the lcal strage n the hsts used in the VSA cluster is taken and used fr the VSA cluster. The data replicatin between ndes will be enabled and wrking at all times after setup and cannt be disabled. In the event f a nde failure, the NFS share that was hsted n the failed nde will be autmatically brught up n the hst with the replica cpy f the data in the VSA cluster. The server resurces used by the VSA cluster are small in terms f CPU and memry, leaving the majrity f the server available fr running ther VMs. Each VSA appliance VM uses ne vcpu with a 2GHz reservatin and 1GB f RAM. A test envirnment fr VSA was cnfigured with three vsphere 5 hsts. Each hst was a tw-scket Intel Xen x GHz-based server with 96GB f RAM and eight 300GB 10,000 RPM SAS disks attached t an internal RAID cntrller with 512MB f cache and set up in a RAID 1/0 LUN (the RAID level VSA supprts). TECHNICAL WHITE PAPER /3

4 Figure 1. VSA Test Cnfiguratin with Three Hsts The VSA cluster is layered n tp f this simple hardware. A 1-vCPU VSA appliance VM n each hst used fur 1Gb Ethernet netwrk cnnectins and the lcal strage t create an NFS share n each hst. Each NFS share was backed by a lcal cpy and a replicated cpy n anther hst in the cluster. Tw f the netwrk cnnectins per hst were dedicated fr a back-end cluster cmmunicatin and replicatin netwrk where the VSA VMs replicated the data between the hsts. The ther tw prts were used t prvide access t the NFS shares. Figure 1 prvides a diagram shwing the test cnfiguratin and key aspects f the VSA cluster. Key Factrs f VSA Perfrmance In mst aspects, the perfrmance f VSA is determined by the same things that determine the perfrmance f any strage array r LUN. There are, hwever, sme aspects that are different r have a bigger impact fr VSA than in ther envirnments. Understanding these aspects f VSA perfrmance will aid in planning a deplyment, and mnitring r managing the perfrmance f a VSA cluster. TECHNICAL WHITE PAPER /4

5 Key factrs in the perfrmance f VSA: Cmmn Strage Perfrmance Factrs Number f disks Rtatinal speeds f disks Size f disks RAID type I/O request size Randmness f I/O wrklads Lcal RAID Cntrller Replicatin f Data Between VSA hsts Mix f Reads and Writes Cmmn Strage Perfrmance Cnsideratins Each disk is capable f a certain perfrmance given a certain type f wrklad. Depending n the type f disk, this perfrmance varies. Fr many I/O-intensive enterprise applicatins, the key factr is hw many read and write peratins the disk can cmplete quickly. This is usually called Input / Output Operatins Per Secnd r IOPS. The speed f the disk has a direct effect n the number f IOPS. Estimates vary quite a bit n the IOPS capabilities f different speeds f disks. A 10,000 RPM disk is ften estimated t get frm 120 t 150 IOPS, and 15,000 RPM disks frm 170 t 200 IOPS. In a strage array, these disks are cmbined tgether int a SCSI lgical unit (smetimes referred t as a LUN r virtual disk) where the data is spread acrss all f the disks. This means that the number f IOPS that the SCSI lgical unit is capable f is nw the sum f all the individual disks, with cnsideratin fr the RAID type used t create the lgical unit. With VSA clusters, RAID 1/0 is the nly supprted RAID type. Lcal RAID Adapter The perfrmance f the lcal SCSI RAID adapter is a big factr fr VSA. It is imprtant t get a gd RAID adapter with ample nbard nn-vlatile write cache and ensure that the cache settings are enabled. If nnvlatile write cache is disabled n the RAID adapter, it will lead t lwer-than-expected perfrmance. VSA Data Replicatin The replicatin f data in a VSA cluster happens autmatically and there is nthing that needs t be dne t manage it. When planning r examining the perfrmance f a VSA cluster, it is imprtant t understand the impact that the replicatin has n available strage and the number f disk write peratins. VSA presents an NFS datastre t each hst in its cluster, which is hw ESXi servers cnnect t and use the shared strage that VSA creates. Additinally, all data is synchrnusly replicated frm its primary nde t a RAID 1 mirrr, lcated n a secnd hst in the VSA cluster. This allws fr the data in a VSA cluster t remain available in the event that ne nde ges ffline r is lst fr any reasn. In rder t d this replicatin, VSA divides the available lcal strage n each hst and uses half fr a primary and half fr a RAID 1 replica. Additinally, in rder t keep the replica in sync, all writes that ccur n the exprted primary datastre must als ccur n the replica. Each write peratin will result in a write t the primary and a write t the replica. Wrklads using exprted datastre will use resurces n tw hsts the hst fr the primary and the hst fr the replica. Replicatin is synchrnus in the VSA cluster, meaning that a write wn t be acknwledged and cmpleted until it is cmmitted t bth the primary datastre and its replica. TECHNICAL WHITE PAPER /5

6 Mix f Reads and Writes The mix f reads and writes is always an imprtant aspect f strage perfrmance. Depending n the RAID type used, the number f physical reads r writes that ccur fr each lgical read r write is different. RAID 1/0 is used fr the physical strage cnfiguratin f the hsts participating in a VSA cluster. In a RAID 1/0, each lgical read results in ne physical read, but each lgical write results in tw physical writes s that the data can be kept in sync n bth sides f the RAID 1/0 mirrr. In additin t this, VSA replicates all data t a secnd hst via a sftware-based netwrk RAID 1 mirrr. This means that each lgical write frm a VM will result in fur physical writes. Tw fr the replicatin and then tw at the SCSI lgical unit (LUN) level due t RAID 1/0. When planning a VSA deplyment, expect each write by a VM t result in fur writes t disk. Tw physical writes will ccur n the primary hst and tw will ccur n the hst with the replica cpy. Perfrmance Metrics Strage perfrmance fr VMs using a VSA cluster can be measured with a variety f metrics. T btain a cmplete analysis f VSA strage perfrmance, evaluate three key aspects: applicatin / VM, NFS datastre, and physical SCSI adapter. Applicatin / VM The perfrmance f the actual applicatin running inside the VM is measured in a variety f ways. An applicatin that reads and writes its data t VSA strage will reprt its perfrmance in sme type f respnse time r thrughput metric. Inside the VM there are als OS-level tls such as perfmn (Windws) r istat (Linux). These OS-level perfrmance mnitring tls can reprt strage-specific perfrmance in terms f IOPS and latency. At the VM level, there are als cunters in vcenter and esxtp that can prvide the same type f infrmatin as perfmn and istat prvide fr strage. In vcenter, infrmatin frm these cunters is fund by ging t the perfrmance tab f a VM and lking at Strage. In esxtp interactive mde, press v t get t the Virtual Disks utput screen. NFS Datastre VSA prvides an NFS datastre n each hst in its cluster which can be used t hst many VMs. These NFS datastres are visible t the vsphere hsts and are used t hst the VMs virtual disk files. Perfrmance metrics at this level reprt a summary fr all the VMs n the datastre. In vcenter, lk at the datastres labeled VSADs t access the perfrmance infrmatin fr the VSA NFS datastres. In esxtp interactive mde, press u t get t the disk device screen. Physical SCSI Adapter Viewing perfrmance fr the physical SCSI adapter fr the lcal strage used by VSA will shw all f the I/O data fr VSA n that hst. This includes bth the primary and replica. The physical SCSI adapter is belw the VSA appliance VMs in the stack, which means that all I/O will be measured here regardless f whether it is fr the VSA primary r replica. This is different frm the NFS datastre view, where nly the primary peratins are measured. This additinal data that includes the replica peratins is needed t create the full picture f VSA perfrmance. In vcenter, this data is accessed under the Strage Adapters and in esxtp it is reached by pressing d t lk at the disk screen. It will mst likely be the adapter labeled vmhba1, but it culd be different if there are multiple lcal SCSI adapters present. TECHNICAL WHITE PAPER /6

7 Mixed Wrklad Test In rder t test the perfrmance f VSA in such a way as t simulate a real-wrld mix f applicatins, VMware VMmark 2.0 was selected as a test wrklad. VMmark 2.0 includes a Micrsft Exchange mailserver, DVD Stre database server, three DVD Stre applicatin servers, an OLIO database server, an OLIO applicatin server, and a standby VM. Additinally, it includes infrastructure peratins vmtin, Strage vmtin, and VM deplyment. The VMs were spread acrss the three hsts in the VSA cluster with the mail server and standby n hst 1, the DVD Stre VMs n hst 2, and the Oli VMs n hst 3. vmtin and VM deply infrastructure peratins were run in additin t the standard VMmark2 applicatin wrklads. Strage vmtin was disabled fr these tests because there was nt anther VSA cluster available t be used as a target. VSA was able t handle this mixed wrklad simulating thusands f users with the 10,000 RPM SAS disk cnfiguratin. The number f sustained IOPS fr the test was apprximately 2,500 acrss all three hsts. This IOPS number was measured by adding the IOPS f the physical adapters fr all three hsts. It includes the primary, replica, and infrastructure peratins. A key measure f perfrmance is hw the applicatin perfrms and a key factr in VSA perfrmance is the lcal RAID cntrller. In rder t shw the effect f the lcal RAID cntrller n applicatin perfrmance, VMmark 2.0 tests were run with the cache enabled and disabled. In rder fr the test t run successfully with the cache disabled, it was necessary t disable the infrastructure peratins. This meant that it was necessary t als disable them fr the cache-enabled test t get cmparable test results. Figure 2 shws the results frm these tests. 4,500 Applicatin Respnse Time 4,000 Respnse Time (ms) 3,500 3,000 2,500 2,000 1,500 1, Exchange Mailserver Oli DVD Stre Cntrller Cache Disabled Cntrller Cache Enabled Figure 2. VMmark 2.0 Applicatin Respnse Time with RAID Cntrller Cache Disabled and Enabled The results shw that the cache f the RAID cntrller has a huge effect n perfrmance. The tw I/O-intensive wrklads shwed dramatic changes in respnse time. By default, the write cache is usually enabled n RAID cntrllers and this test was dne nly t shw its imprtance. The mst likely scenari fr the write cntrller TECHNICAL WHITE PAPER /7

8 cache becming disabled is when its nbard battery lses its charge and the cntrller disables the cache t prtect data integrity. It is als interesting that OLIO, which is nt very I/O-intensive, had almst n change in respnse, shwing the strage perfrmance des nt affect all wrklads. IOBlazer Test IOBlazer was used t illustrate hw wrklad affects the VSA cluster. IOBlazer prduces as many disk I/Os as it can, based n the utstanding I/Os parameter, and reprts the number f IOPS and average respnse time r latency fr thse peratins. Because each VSA nde is a hst fr a primary datastre and a replica datastre, a wrklad n ne nde will cause IOPS t ccur n tw ndes. A simple test scenari with the VSA-based cluster was run with three phases t shw what happens. Three Windws Server bit VMs were set up, ne n each f the three VSA NFS datastres, with IOBlazer installed t generate an I/O wrklad. The I/O prfile used was 8k blck size, randm, 50% reads, 50% writes, and 32 utstanding I/Os. Five IOBlazer tests were run in successin with ne immediately fllwing the next, in three phases. In the first phase, each f the VMs ran the IOBlazer wrklad ne at a time s that nly ne VM was active at a time. In the secnd phase, tw VMs ran IOBlazer at the same time. In the third phase, all three VMs ran IOBlazer at the same time. Perfrmance was recrded using esxtp n all three hsts. The key strage perfrmance metrics frm these tests are shwn in the fllwing graphs. The placement f the three test VMs is imprtant t understand the results f the tests. Each hst in the VSA cluster hsted ne f the NFS datastres. One f the VMs was placed n each datastre,. Fr the purpses f measuring strage perfrmance, this designeffectively simulates the placement f ne VM n each f the hsts in the VSA cluster.. Frm IOBlazer s perspective, the IOPS and latency numbers fr the tests shwed apprximately a 2x increase in IOPS frm ne VM t all three VMs, while latency increased by apprximately 3ms. Figure 3 shws the IOBlazer reprted results acrss all three test phases. IOBlazer: IOPS IOBlazer: Latency 3, , IOPS 2,000 1,500 1, Latency (ms) VM1 VM2 VM3 VM1 and VM2 All 3 VMs 0 VM1 VM2 VM3 VM1 and VM2 All 3 VMs Figure 3. IOBlazer IOPS and Latency Results as Reprted by IOBlazer During Testing with ne VM per VSA Datastre TECHNICAL WHITE PAPER /8

9 Figure 4 shws the IOPS frm the same IOBlazer test as measured at the NFS datastre. The esxtp IOPS data fr the NFS vlumes shws the same thing as the IOBlazer data that was measured frm within the VM. This is because nly ne VM was running n each datastre. The graph als shws the ttal IOPS fr the VSA cluster acrss all three NFS datastres. The ttal line is the same as the individual hst lines in phase ne f the test when nly ne VM is under lad. In phases tw and three, as multiple datastres becme busy, the ttal fr the cluster rises, reflecting the increased lad. The amunt f IOPS acrss the cluster des nt triple, but dubles in the final phase when all three datastres are active. Figure 4. Perfrmance f VSA NFS Datastre Level During IOBlazer Testing The esxtp data fr the strage adapter allws us t see what the IOPS were acrss each hst, including the replicatin activity as shwn in Figure 5. This explains why effective IOPS at the VM and datastre levels d nt triple as the test prgresses frm ne t three hsts. Even thugh nly ne VM is active n ne NFS datastre, the I/Os are ccurring n tw hsts because data is being written t bth the primary and replica. Additinally, reads can be dne n either the primary r replica depending n lad acrss the VSA ndes. Because ne active wrklad actually affects tw hsts, nce tw hsts are active, all f the hsts are nw active. In Figure 5, the ttal number f IOPS des nt increase linearly between phases tw and three f the test because all f the disks were already busy nce tw hsts were active. Using the strage adapter perfrmance metrics shws the impact f the replicas and hw many IOPS are actually ccurring at the physical adapter level. TECHNICAL WHITE PAPER /9

10 Figure 5. Perfrmance Measured fr the Physical Adapter During IOBlazer Tests In Figure 3 and Figure 4, the number f IOPS is apprximately 1,200 when a single VM is running the IOBlazer wrklad. In Figure 3, this is reprted frm within the VM frm the test applicatin s perspective. In Figure 4, this is reprted frm the NFS datastre level perspective. Because there is nly ne VM per datastre, the number is the same. The mix f read and writes used in this test was 50/50, meaning that there are 600 reads and 600 writes ccurring. In Figure 5, the number f IOPS acrss the cluster is apprximately 1800 in each f the three cases where nly ne VM is actively running IOBlazer. This is as expected because the 600 reads are nly ccurring nce, while the 600 writes are ccurring twice. Because the reads can ccur n either the primary r replica, the number f IOPS n either VSA nde is similar. TECHNICAL WHITE PAPER /10

11 Best Practices There is a tradeff between capacity, perfrmance, and price when deciding n the disk cnfiguratin fr the VSA hsts. SATA disks prvide higher capacities at lwer prices and lwer reliability levels, but d nt perfrm as well as SAS disks. SAS disks are mre reliable and have higher perfrmance, but cst mre than SATA disks. Increasing the number f disks will increase perfrmance and s will increasing the speed f disks, but ding either f these will result in higher csts. The hardware used fr VSA is a big factr in perfrmance. Ensure that the RAID adapter in the VSA hsts has sizeable cache and the cache is enabled. VSA requires a RAID 1/0 cnfiguratin fr the lcal strage n the hsts and als replicates data acrss the cluster t prvide highly available shared strage. When planning a deplyment f VSA, the cmbinatin f RAID 1/0 and data replicatin mean that useable capacity will be ¼ f raw capacity and each write by a VM will result in tw physical writes n the primary hst and tw physical writes n the replica hst. VSA perfrmance can be mnitred at three levels t get a cmplete picture f the envirnment. The applicatinr VM-level perfrmance prvides the view f the end user. The NFS datastre perfrmance shws hw much each f the VSA datastres is being laded by all f the VMs that they serve. The physical SCSI adapter view shws the ttal impact f bth VSA primary and replica data cpies n the hst. Cnclusin VSA allws features like vmtin, DRS, Strage vmtin, and HA t be pssible using nly lcal strage. This enables advanced vsphere capabilities nw t be pssible fr envirnments as small as just tw servers. vsphere prvides the VSA envirnment with tls t mnitr and manage perfrmance and understanding the key factrs f VSA perfrmance helps drive a successful deplyment.. Abut the Authr Tdd Muirhead is a perfrmance engineer at VMware fcusing n database, mail server, and strage perfrmance with vsphere. Acknwledgements Thanks t the VSA, VMmark, and Perfrmance Engineering teams fr their assistance and feedback thrughut. VMware, Inc Hillview Avenue Pal Alt CA USA Tel Fax Cpyright 2011 VMware, Inc. All rights reserved. This prduct is prtected by U.S. and internatinal cpyright and intellectual prperty laws. VMware prducts are cvered by ne r mre patents listed at VMware is a registered trademark r trademark f VMware, Inc. in the United States and/r ther jurisdictins. All ther marks and names mentined herein may be trademarks f their respective cmpanies. Item: PS-VSA