Consulting Solutions WHITE PAPER XenDesktop XenDesktop Planning Guide Storage Best Practices

Transcription

1 Consulting Solutions WHITE PAPER XenDesktop XenDesktop Planning Guide Storage Best Practices

2 Contents Overview... 3 Guidelines... 4 Organizational Requirements... 4 Technical Requirements... 5 Performance Requirements... 5 Functional Requirements... 8 Avoid Bottlenecks... 9 High Availability File System Alignment Validate and Monitor Planning Choosing the appropriate RAID Level Number of Disks (IOPS) Caching Connectivity Provisioning Services XenDesktop Machine Creation Services XenServer Appendix Page 2

3 Overview The Citrix Storage planning guide provides a list of best practices, recommendations and performance related tips that cover the most critical areas of storage integration with Citrix XenDesktop. It is not intended as a comprehensive guide for planning and configuring storage infrastructures, nor as a storage training handbook. Due to scope, this guide provides some device-specific information. For additional device- specific configuration, Citrix suggests reviewing the storage vendor s documentation, the storage vendor s hardware compatibility list, and contacting the vendor s technical support if necessary. The following provides a non-comprehensive list of available vendor specific reference architectures and best practice guides, which are highly recommended: Citrix XenServer and NetApp Storage Best Practices DR Solution for XenDesktop on NetApp EMC Infrastructure for Virtual Desktops HP Converged Infrastructure enterprise reference architecture for client virtualization Dell Virtual Remote Desktop Solution Reference Architecture Cisco Validated Design Page 3

4 Guidelines The initial step in order to plan a storage infrastructure is as with any other planning process to assess and to understand the specific storage needs of an organization. Those needs consist of organizational as well as technical requirements, which are discussed within this section. While this section does discuss how to plan for and work with various storage architectures and technologies, it is not possible to provide architectural blueprints because of the wide variety of storage systems and technologies available on the market. Therefore it is strongly recommended to involve storage vendors or specialized partners in the planning process. Organizational Requirements Organizational storage requirements go beyond having sufficient capacity for the data stored on each server and workstation. The following provides a non-exclusive list of items for consideration: Does it align with IT strategy? In general, storage planning should fully align with a company s IT strategy, in terms of preferred storage vendor / partner, budget, anticipated growth, and expected new workloads or applications. Can data be consolidated or existing storage infrastructures be leveraged? Consolidating storage or leveraging existing systems reduces the cost of managing storage on multiple systems and increases the efficiency of storage allocation and backup tasks. However, consolidation also introduces risk, because more applications, workloads and users depend on fewer systems; so an outage has a greater impact than in a more distributed installation. If it is planned to consolidate storage, it might be necessary to investigate ways to increase storage availability. It is important to understand that besides disk space storage performance (latency, IOPS) and the respective impact when adding additional systems is a vital criteria when consolidating storage. Especially in VDI environments this is underestimated in many cases. Is high availability required and how long can data recovery take? Some critical systems must be continuously available. For example, it might require that the systems experience zero downtime. Other workloads might be able to tolerate a one-hour window of downtime. Planning needs to take into account the organization s as well as workload or application specific tolerance for downtime during data recovery. Will other systems / infrastructures be influenced? Depending on the size and setup of a storage system, it can influence other systems or infrastructures. For example, when planning for data replication, the utilization of the LAN or the WAN in between the data centers will increase and it could therefore introduce latency for other systems that rely on the same communication paths. Page 4

5 Which level of storage expertise exists? Planning as well as subsequent operations may require special knowledge, especially when storage solutions with a high degree of complexity or with complex management interfaces will be implemented. It is important to assess existing knowledge, possible synergies and to understand training needs when introducing a new storage infrastructure. Technical Requirements Determining the technical requirements can be very a complex procedure, mainly depending on the amount and complexity of the applications and workloads which will be connected to the storage infrastructure. In order to keep this guide focused, only two aspects (performance and functionality) for Citrix related workloads will be considered. Performance Requirements Understanding the I/O characteristics and performance requirements of an application or workload is vital in order to be successful in designing and deploying a scalable storage infrastructure which meets the performance expectations of the business. Important data points that should be measured for all storage dependent applications and workloads include: Typical I/O rates (I/O s per second (IOPS), IO size (request size), MB/s (throughput)) Read and Write ratio Sequential vs. Random I/O ratio For XenApp and XenDesktop systems, typically all I/O will be random. Sequential IO is generated by databases or other high data volume applications only Note: The following resources provide information about typical I/O characteristics for virtual desktop environments (for reference purposes only): Page 5

6 Using Windows, these data points can be gathered by means of the built-in Performance Monitor, which offers the ability to track the following counters (object Physical Disk): Information Read counter Write counter Combined counter about IOPS Disk Reads/sec Disk Writes/sec Disk Transfers/sec I/O size Avg. Disk Bytes/Read Avg. Disk Bytes/Write Avg. Disk Bytes/Transfer By using the results of these counters the following equations help to determine the missing pieces of information: Throughput: Disk Transfers/sec * Avg. Disk Bytes/Transfer % of reads: (Disk Reads/sec * 100) / Disk Transfers/sec % of writes: (Disk Writes/sec * 100) / Disk Transfers/sec Performance Monitor also exposes similar counters per process in order to gather performance related information at an application level rather than at a machine level. Alternatively the Windows built-in Resource Monitor as well as the SysInternals Process Monitor or other 3 rd party tools (i.e. hiomon) provide similar data. Page 6

7 Using XenServer or other Linux based operating systems, these data points can be gathered by means of the iostat command. In order to run at a regular interval, over a certain period of time and to capture the output in a format that can be imported to Excel, it is necessary to run iostat as follows: iostat -xk awk 'BEGIN {print "Time Device rrqm/s wrqm/s r/s w/s rkb/s wkb/s avgrq-sz avgqu-sz await svctm %util"} /sda1/ {print strftime("%h:%m:%s"),$0}' >iostat.csv & The command is comprised of the following parameters, which may need to be modified for every environment: Parameter iostat xk /sda1/ >iostat.csv & Meaning Runs iostat with extended statistics (in KB format) at a 15 second interval 5760 times (= 24 hours) Filters the output for block device sda1 Writes the iostat output into the file iostat.csv Runs in the background to free up the shell console for other commands By using iostat the following counters can be tracked: Information Read counter Write counter about IOPS r/s w/s I/O size rkb/s wkb/s By using the results of these counters, the following equations help to determine the missing pieces of information: Throughput: rkb/s + wkb/s % of reads: % of writes: By using XenTop on a XenServer host, the aforementioned counters can be obtained on a per virtual machine basis. Please refer to the Appendix of this document for further information. Page 7

8 Functional Requirements Determining the functional requirements is basically a balancing act between the prerequisites of a workload or application and the technical capabilities of any given storage architecture or storage protocol. The following table outlines this procedure for the current versions of Provisioning Services (PVS. Further information can be found within the planning section of this document. Protocol / Functionality PVS central vdisk management PVS caching of vdisks CIFS NFS iscsi FC X X (X) 1 (X) 1 (X) 2 X X X 1: Additional software (clustered file system) required 2: Opportunistic Locking configuration of Windows needs to be changed. Please refer to the Appendix of this document for further information. Page 8

9 Avoid Bottlenecks Most applications and workloads are very sensitive with regards to the latency of storage I/O operations. For example within virtual desktop environments, high I/O latency negatively impacts the user experience. It is vital to avoid bottlenecks within the storage infrastructure. Networking: In scenarios with congested network interfaces or links, data packets may be dropped. This can severely impact storage traffic based on standard Ethernet, such as CIFS, NFS, iscsi or FCoE as dropped packets have to be retransmitted, which adds latency to the storage I/O operation. For shared interfaces (e.g. Blade Chassis uplinks or Host NICs) or central network links (e.g. inter-switch, inter-data center) oversubscription scenarios are likely. While VLANs allow logical separation of network traffic at layer 2 of the OSI model, they provide no mechanism to allow oversubscription of physical links; a 10GBit/s link has a maximum of 10Gbit/s no matter how many VLANs are configured. It is therefore a best practice to dedicate network interfaces and links for applications and workloads with very high I/O requirements. For maximum performance and reliability, a fully dedicated, nonshared network infrastructure is recommended for all storage protocols which leverage standard networking technologies. SAN Fabric. While a dedicated SAN Fabric, which relies on Fibre Channel rather than Ethernet based communication, is typically a continuously monitored, optimized and lossless network for storage traffic, it potentially can imply the same risks as described within the networking section. Storage Hardware: In multi-tenant (or other shared) scenarios with unpredictable data access patterns, Storage Processors, NAS heads or other shared hardware such as switches or HBAs can become a bottleneck. Proactive monitoring as well as trend and capacity planning are imperative. Furthermore, it is a best practice to leverage all available paths and storage controllers to distribute the load across all available resources. For large environments with unpredictable high volume data traffic, dedicating the storage infrastructure for virtual desktops can become the preferred option, in order to keep data latency consistently on the lowest level possible. Spindles: Avoid hosting multiple workloads with high I/O characteristics on the same set of disk spindles concurrently. Furthermore, it is a good practice to have an adequate number of spindles to support your I/O requirements with an acceptable latency Page 9

10 RAID levels. The RAID level of a storage sub-system can have a direct impact on the performance of the applications and workloads attached to this particular storage. While some RAID levels are optimized for writes (i.e. RAID 0, 1, 10) others such as RAID 5 or 6 are not (which can multiply the storage utilization performance wise by a factor 4 or 6 (write penalty)). Therefore, it is best practice to configure the RAID level (whenever possible) based on the I/O pattern of the related applications and workloads. High Availability It is a best practice to implement all storage related components in a highly available manner. This includes HBAs or NICs, network paths, storage controllers and disk shelves. The following diagram outlines a sample HA configuration for an iscsi based storage attached to a XenServer: In the environment shown above, XenServer MultiPathing allows storage I/O to leverage multiple redundant network paths to the storage infrastructure, which can increase performance and provide high availability. Once again, the storage array must support this configuration. As previously discussed, this diagram shows a dedicated, isolated switch and network infrastructure for storage traffic. Page 10

11 File System Alignment In order to minimize the utilization of the storage sub-systems, it is best practice to fully align the file systems at all layers (i.e. VM, Hypervisor, Storage). For optimal performance, the starting offset of a file system should align with the start of a block in the next lower layer of storage. For example, an NTFS file system that resides on a LUN should have an offset that is divisible by the block size or stripe size of the storage array presenting the LUN. Misalignment of block boundaries at any one of these storage layers can result in performance degradation as multiple storage I/O operations can be required to access a single block of data. 2 Please note that while Windows 7 and Windows Server 2008 or above should automatically align disk partitions correctly, it is recommended to verify this for every new workload. The following figure outlines a misalignment scenario for a Windows XP NTFS partition. Validate and Monitor Validating the storage infrastructure prior to deployment in production is essential in order to isolate and address potential I/O bottlenecks, optimize throughput and to tune all components which sit in the data path including HBA s, disk controllers, device cache, storage processors, disks, RAID levels, and where applicable, network and SAN fabric components. Subsequent monitoring is also essential to ensure that performance levels are being maintained in line with agreed service levels and to provide data which his useful for scalability analysis and capacity planning. Testing and monitoring processes should include at the very least: Optimization and rationalization of firmware versions for all storage components Storage I/O path utilization Utilization of the storage controller / NAS Head Read versus Write I/O distribution Cache hit rates and utilization Utilization of the disk spindles (performance / avail. space) Throughput, latency and queue depth 2 Page 11

12 Planning Planning a storage infrastructure is a highly individual task, which, as outlined earlier, depends on technical as well as unique organizational criteria. It is recommended to involve a storage vendor or a partner in the early phases of a storage design initiative in order to achieve the best aligned storage design. Nevertheless, this section will provide guidance for planning a storage infrastructure. Choosing the appropriate RAID Level To choose the optimal RAID level, it is necessary to consider the IOPS and read/write ratio generated by a given application or workload in combination with the individual capabilities of a RAID level. For hosting read intense workloads, RAID levels that are optimized for read operations, such as RAID 0, 1, 5 10, are optimal. This is because these RAID levels allow read operations to be spread across all disks within the RAID set simultaneously. For hosting write intense workloads, such as Provisioning Services Write Cache and Machine Creation Services differencing disks, RAID levels such as RAID 0, 10 are optimal, as these are optimized for writes or have a low write penalty. The following table outlines the key quantitative attributes of the most commonly used RAID levels: RAID Level Capacity Fault Tolerance Read Performance (random) Write Performance (random) 0 100% None Very Good Very Good (Write Penalty 0) 1 50% Good Very Good Good (Write Penalty 2) 5 Disk size * (# of disks -1) Good Very Good Bad (Write Penalty 4) 10 50% Very Good Very Good Good (Write Penalty 2) Note: The write penalty is inherent in RAID data protection techniques, which require multiple disk I/O requests for each application write request, and ranges from minimal (mirrored arrays) to substantial (RAID Levels 5 and 6 3 as outlined within the example below (calculating RAID5 parity during a write / without optimization technologies): 1) Read the old data 2) Read the old parity and calculate the difference between old data and new data, and old parity and new parity 3) Write the new data 4) Write the new parity 3 Page 12

13 Number of Disks (IOPS) To determine the number of disks required, from a performance point of view, it is important to understand the performance characteristics of each disk, the characteristics of the RAID level and the performance requirements of the given workload. The following examples outline the necessary calculations. The basic calculation is: The following table provides examples of how many disks are required for the given scenarios. RAID Level RAW IOPS (per disk) Workload IOPS Read % Write % Disks Count RAID % 80% % 20% 12 RAID 1/ % 80% % 20% 14 RAID % 80% % 20% 19 Caching In order to achieve the IOPS required with the least number of disks and to mitigate peak load scenarios, caching techniques are provided by the vendors at various levels within the storage stack. The read/write ratio of a workload determines the required caching configuration, as outlined below: Workload Read Cache Write Cache MCS LUN 50% 50% PVS Write Cache - 100% XenDesktop - 100% (installed) XenApp (installed) - 100% Page 13

14 Connectivity Storage subsystems can be attached to one or many computer systems by means of various protocols, such as NFS, iscsi or Fibre Channel. Determining the functional requirements of an application or workload and matching this with the individual capabilities of a storage protocol are of the same importance during the planning phase as considering high availability and throughput. As discussed earlier, it is general best practice to use MultiPathing or automatic failover techniques, for storage paths. In order to plan for the required throughput, it is necessary to sum up the throughputs for every individual system that uses a shared component or network path. Following an example for an environment with 100 similar virtual machines (hosted on 10 virtualization hosts and connected to one NAS head) is provided: Average Peak Throughput per VM 10 Mbps 30 Mbps Throughput per Host Throughput for Storage 100 Mbps (10 VMs*10Mbps) 1 Gbps (10 Hosts*100 Mbps) 300 Mbps (10 VMs*30Mbps) 3 Gbps (10 Hosts*300 Mbps) The NIC used for storage communication needs to be a 1 Gbit/s adapter in order to cope with the peak load. The NAS head as well as its network connection need to support 3 GBit/s worth of data traffic, in order to support the peak load of all systems. Provisioning Services While a Citrix Provisioning Server itself is a standard Windows Server, it still has some specific requirements when it comes to storage integration. The following items outline the common best practices. vdisk Storage In most cases, vdisks are used in read-only mode (standard) where multiple target devices simultaneously access a single vdisk. Therefore the following best practices focus on standard mode scenarios only. Storage Protocol and Caching: In order to enable the Windows Server 2008 R2 (recommended OS) to cache parts of the actively streamed vdisks in memory, it is important to either Page 14

15 o Locate the vdisk Store on a block level storage device. Block level storage devices are local disks, or iscsi / Fibre Channel attached LUNs. o Locate the vdisk on a NFS or CIFS share and modify the Opportunistic Locking configuration of Windows. Please refer to the Appendix of this document for further information. Note: In case the CIFS protocol is used to access the vdisks, it is strongly recommended to leverage SMB 2.x whenever possible. Please refer to the Appendix of this document for further information. Note: Citrix recommends Windows Server 2008 R2 as platform for Provisioning Services due to the improved caching mechanisms and the ability to address large amounts of memory. To allow Windows to cache vdisks effectively, a minimum of 2GB RAM per active vdisk (XenDesktop) or 4GB RAM per active vdisk (XenApp) should be configured in addition to 2GB for the OS itself. I/O Pattern: Standard mode vdisks are accessed by the Provisioning Servers with an almost 100% random read I/O pattern (except during maintenance). Because of the caching mechanisms of Windows, which were described earlier, vdisks will be read into free memory and the storage utilization will drop significantly. Availability: Although certain parts of active vdisks can be cached in memory, the vdisk files must be available at all times. Therefore, vdisk stores need to be hosted on highly available storage sub-systems (local or shared). This is especially true in scenarios with a shared vdisk store (i.e. via CIFS / NFS), where high availability is vital. Write Cache Storage I/O Pattern: The write cache holds all writes from a PVS target device and has a mixed I/O pattern depending on the status and the uptime of an individual target. Over time, the probability that items will be read from the write cache will increase. In general a read/write ratio of 5/95 or 20/80 for the write cache is expected. All I/Os are random. The storage sub-system used for Write Cache storage should be optimized accordingly. Target Device RAM: All read and write operations on a Provisioning Services target device will also get cached in the system cache memory of the target device. This includes files read from PVS and file data written to the write cache. It is important to ensure that your Windows targets have enough memory for adequate Windows file caching so that rereads are reduced. Page 15

16 XenDesktop Machine Creation Services While the Machine Creation Services feature of XenDesktop 5 has no direct integration with storage sub-systems, it still has certain specific requirements, which are listed below: Thin Provisioning: The functionality of Machine Creation Services is comparable with a linked-clone approach, although it allows the clones to be non-persistent in addition to the common persistent only option. In order to minimize the storage space required for the clones, Thin Provisioning and automatic disk space reclamation is required. With regard to a XenServer based environment, it is best practice to deploy Machine Creation Services using NFS based storage repositories, which includes these features by default. Please refer to the Appendix of this document for further information. The following diagram illustrates the Thin Provisioning disk space savings: I/O Pattern. In general MCS based workloads have a random read/write I/O pattern. In scenarios without IntelliCache the read/write ratio is expected to be 50/50 and the overall IOPS are about 1.6 times higher than for Provisioning Services based workloads, as the Provisioning Services vdisk caching mechanisms do not apply. For IntelliCache scenarios with non-persistent desktops the read / write ratio observed at the shared storage will be 100% read, as all writes will be cached on the local hard disks of the XenServers. For persistent desktops the read / write ratio will vary based on the fill level of the cache per XenServer. Page 16

17 XenServer The following items outline the common storage best practices for XenServer environments. Number of VMs per LUN: Storage LUNs that are connected to a XenServer using FC or iscsi may be represented as a single iscsi queue on the storage side (storage dependent). In the case where a LUN is shared among multiple virtual machines disks, all I/O has to serialize through the iscsi queue and only one virtual disk s traffic can traverse the queue at any point in time. This leaves all other virtual disks traffic waiting in line. Although it is possible to parallelize iscsi commands by implementing active/active MultiPathing or optimizing the command traversal by techniques such as command batching, LUNs and their respective iscsi queue may become congested and the performance of the VMs may decrease. Therefore it is a best practice to implement no more than virtual machines per LUN on average. The actual VM per LUN ratio is environment and workload specific and needs to be determined by means of performance monitoring (i.e. VM specific read/write latency) and with reference to the best practices of the storage vendor. Note: This does not apply to NFS based shared storage, where tests have shown that even with hundreds of VMs performance does not decline. Software Initiators: When using the XenServer s built-in software iscsi Initiator or the NFS client, the network protocol processing takes place within the Control Domain of the host system, which increases its CPU usage and memory footprint. In scenarios with a high utilization of the host system, it is a best practice to implement hardware based iscsi Initiators or Network Interface Cards with TCP offloading capabilities. Note: With the release of XenServer 5.6 FP1 the control domain (Dom0) is now equipped with 4 vcpus by default, which mitigates the risk of Dom0 congestion. For environments with very high volume network traffic, it is best practice to configure IRQ Balancing. Please refer to the Appendix of this document for further information. Bonding and MultiPathing. In general MultiPathing can leverage all available paths, whereas bonding can use one active data path only. This is related to the SLB Load Balancing algorithm, which is used for balancing data traffic for network bonds in XenServer. The algorithm will assign one static data path (NIC) per VM and the XenServer Control Domain (Dom0), which handles all I/O traffic, is seen as a single VM. Therefore it is recommended to use MultiPathing wherever possible. Please refer to the Appendix of this document for further information. Page 17

18 Local Disk VM storage / IntelliCache: The functionality of IntelliCache, supported in XenServer 5.6 FP1 and above, leverages the local XenServer disk subsystem to cache read operations which originate from your NAS. Using the local disks of a server to cache read and partially write requests of virtual machines can decrease the load seen on the shared storage massively, but it dramatically increases the load on the local disks. To ensure the local disks will not become a bottleneck with IntelliCache or other scenarios that make use of local storage for VM disks, it is a best practice to closely monitor the disk utilization using iostat or other monitoring tools. Furthermore, high performance disks (e.g. SAS 15k / SSD) should be configured in a RAID level that is optimal for random reads and writes. In addition it is highly recommended to equip storage array controllers with a minimum of 512MB read cache as well as (battery backed) write cache. Page 18

19 Appendix The following items provide further insights into multiple topics discussed within this document: How to Use the XenServer Xentop Utility Why / How to modify Windows Opportunistic Locking: Tuning CIFS / SMB for XenApp, Provisioning Services and File Servers Advanced Memory and Storage Considerations for Provisioning Services IRQ Balancing: Distributing Guest Traffic Over Physical CPUs in XenServer Configuring iscsi Multipathing Support for XenServer Using NFS for Machine Creation Services How to Use IntelliCache with XenDesktop ( Standard RAID levels explained ( Page 19

20 Product Versions Product Version XenDesktop 5.0 XenServer 5.6 Provisioning Server 5.6 Revision History Revision Change Description Updated By Date 1.0 Document created Thomas Berger Architect Gaby Hoffmann Senior Consultant Olivier Withoff Architect Peter David Architect Padraig Sweeney Escalation Engineer Christian Ferber Systems Engineer Martin Rowan Director Nicholas Rintalan Sr. Architect Dan Allen Sr. Architect Mark Nijmeijer Director Daniel Feller Lead Architect Tarkan Kocoglu Director August 12, 2011 About Citrix Citrix Systems, Inc. (NASDAQ:CTXS) is the leading provider of virtualization, networking and software as a service technologies for more than 230,000 organizations worldwide. Its Citrix Delivery Center, Citrix Cloud Center (C3) and Citrix Online Services product families radically simplify computing for millions of users, delivering applications as an on-demand service to any user, in any location on any device. Citrix customers include the world s largest Internet companies, 99 percent of Fortune Global 500 enterprises, and hundreds of thousands of small businesses and prosumers worldwide. Citrix partners with over 10,000 companies worldwide in more than 100 countries. Founded in 1989, annual revenue in 2010 was $1.9 billion Citrix Systems, Inc. All rights reserved. Citrix, Access Gateway, Branch Repeater, Citrix Repeater, HDX, XenServer, XenApp, XenDesktop and Citrix Delivery Center are trademarks of Citrix Systems, Inc. and/or one or more of its subsidiaries, and may be registered in the United States Patent and Trademark Office and in other countries. All other trademarks and registered trademarks are property of their respective owners. Page 20