WHITE PAPER Resolving Common Challenges in Virtualized Environments Sponsored by: Symantec Carla Arend January 2011 Nick Sundby IDC OPINION IDC EMEA, 389 Chiswick High Road, London, W4 4AE, U.K. Tel.+44.20.8987.7100 www.idc.com The adoption of virtualization in the enterprise is transforming the way IT is deployed, managed and delivered to the business units. Virtualization has now become mainstream, and many organisations have virtualized more than 50% of their server infrastructure. However, broad adoption of server virtualization can lead to new challenges regarding storage management, data protection, security and availability that need to be considered and properly addressed in order to reap the maximum benefit of server virtualization. While virtualization delivers many advantages, it is not perfect. IDC research shows that taking the time to virtualize IT will bring about a large number of benefits. It will save time and money, and there are many management and automation tools that can be purchased via numerous vendors. Despite this, as with the introduction of any new technology, there are challenges that need to be faced and overcome not least of which may be the realisation that the link between workloads and physical platforms has been broken. Virtualization involves a new way of thinking, both for the IT managers and the users that they serve. IDC believes that a structured and careful deployment will yield benefits in moving to a more flexible, efficient and resilient IT infrastructure. IN THIS WHITE PAPER The operation of physical and virtual environments poses challenges on IT management. This IDC White Paper identifies some of the key challenges that occur in virtualized environments and highlights some of the key IT services that complement and enhance a virtualized environment. The paper is aimed at IT managers of enterprises. SITUATION OVERVIEW There are many benefits when moving from a physical to a virtual infrastructure, and many companies have made this transition already in parts of their datacentre. Realising these server virtualization benefits does require some effort. Skills will be needed that may be new to some organisations, and thus planning should be taken seriously and with forethought. Virtualization, in fact, may not be a good fit for your organisation and so, where possible, a consultant or VAR with a strong virtualization experience and methodology should be used.
IDC estimates that for the first time in 2010 the majority of new server deployments will be on a virtual infrastructure as opposed to a physical infrastructure. Businesses are deploying virtualization for the following reasons: Technical reasons for moving from a physical to a virtual infrastructure: Operational efficiency and utilisation. The primary reason to adopt a virtual infrastructure is to increase utilisation of servers and storage and move away from an architecture where each application has its dedicated hardware, which most often runs at low utilisation rates. Virtualization enables organisations to significantly increase their operational efficiency through consolidation of their server estate, and increase utilisation through shared virtualized pools of server resources. Central management. Another reason for moving towards a virtualized infrastructure is the possibility to provide virtual desktops to end users, which increases the operational efficiency of managing a company's desktop infrastructure by providing central management of the desktop infrastructure for example, patching, software installs and updates, endpoint security management, data protection, etc. Speed and agility. One of the key objectives of IT managers today is to increase the responsiveness of IT to changing business needs, particularly through much faster provisioning of new business applications and services. This objective is mainly achieved through the deployment of virtualization technologies, which enable IT to provision new services in hours rather than weeks. Business reasons for moving from a physical to a virtual infrastructure: Cost reduction. Virtualization can bring huge cost savings to organisations, because they usually consolidate many smaller servers onto fewer bigger servers, as well as remote office IT into the datacenter, and the overall hardware requirement will go down, due to higher utilisation rates. Organisations will typically achieve a reduction in cost of acquiring, implementing and operating IT through the deployment of virtualization. Governance. Virtualization is also a foundation for automated policy-based management to ensure governance and compliance and appropriate service levels to the business, because virtualization enables IT managers to manage across their virtual estate and thus automate a larger proportion of their tasks. Cloud. Once an organisation has deployed virtualization, it is much easier to move workloads to a third-party cloud-based IT provider for bursty workloads. Reasons why virtualization may not be appropriate: Legacy applications. Homegrown legacy applications are usually not easy to virtualize, if at all, and many organisations choose to leave them untouched in their physical server environment until they have found a way to extract their data and retire the application. 2 #IDCWP36S 2011 IDC
The Move Towards Virtualization: Implications and Challenges When moving from a physical to a virtual infrastructure, organisations typically encounter a new set of challenges. The implementation of virtualization is often driven by the desire to increase operational efficiency and utilisation of the datacentre. However, introducing a new technology into the datacentre also generates a degree of change and, when implementing virtualization, organisations also need to think about the impact of virtualization on their risk posture because they might expose themselves to new risks that stem from the different architecture of virtualization. New risks that can arise are related to disaster recovery, security, data protection, storage management and IT and information governance. The overarching challenge is how to implement virtualization and increase operational efficiency while at the same time maintaining an appropriate risk profile. The specific challenges are described in the following points. Storage Challenges Primary storage capacity growth. Storage capacity requirements are increasing significantly as virtual machines sprawl and lead to inefficient storage usage due to virtual machine image duplication. Some users also find that VMDK and VDI files can grow to be significantly larger than the space allocated to the encapsulated application, and that shrinking the files is not as easy as enlargement. This is an area where space-optimised copies of virtual machine and desktop images work particular well by eliminating data duplication for OS machine images, as there is usually a high level of duplication when comparing two VM or virtual desktop image files. VMDK and VDI efficient backup. Data growth has almost returned to the preeconomic crisis levels of 50% 60% average per year, and one of the driving factors is virtual machine sprawl and the need to back up the associated VM and virtual desktop image files. There are two challenges related to virtual machine and virtual desktop backup: either there is no backup, because virtual machines or virtual desktop images have been copied and deployed without enabling backup, or large amounts of data are being backed up that are not necessary, like unused virtual machines or multiple copies of the same operating system. IDC finds that deduplication ratios of 50:1 or better can be achieved when backing up VM image files. Performance impact. The I/O pattern from a server hosting multiple virtual machines is typically more random due to the mix of traffic from multiple sources. This can compromise storage caching algorithms, so an analysis of the I/O requirements for important applications is a worthwhile exercise. Virtual desktop deployments can also introduce I/O "bootstorm" issues when multiple clients request boot images from the storage at the same time. Speed of provisioning. Virtualization is typically deployed to increase the responsiveness of IT to business needs and to reduce the time to provision new services. However, when provisioning large numbers of VMDKs or VDIs in a manual process, it will still take hours or days to complete. In order to deliver on the agility promise of virtualization, provisioning of virtual machines and desktops needs to happen within minutes rather than days, even at large scale. 2011 IDC #IDCWP36S 3
Security Challenges Lost control. Virtual machines are sprawling, as it is very easy to copy them and create new virtual machines as needed. Thus, IT managers are losing control over the provisioning of IT resources and have a hard time enforcing IT policies in virtual environments. Data movement. Virtual machines and their data are moving around the infrastructure more easily. Consequently, protection of the perimeter is not enough anymore, and the data itself needs to be protected instead. Securing the host. Many organisations focus on securing the guest (virtual machine) but forget to secure the host (physical server). It is important to secure the host, because the host is running an operating system and is thus vulnerable to malware and viruses. It is important that this protection is as light and efficient as possible, so that performance of the host and guest is not impacted and the security process does not consume valuable CPU cycles and I/O. It is also important that the security scans of the VMs running on a physical host are done randomly to avoid all VMs being scanned at the same time, which would also degrade performance. Scalability and securing the guest. When protecting virtual machines, it is important to manage the security challenges that arise from large-scale deployments, like for example the boot scan problem, when a majority of virtual desktops get spun up at 8:30 in the morning, or all apps are invoked at the same time. In a setup where each guest virtual machine has its own security installed, the problem is that on-boot scans are using CPU and degrade virtual machine performance, particularly in large environments. Application Availability Challenges Business continuity and high availability. How to ensure business continuity and high availability of applications that are run on multiple physical servers? Organisations are moving through different stages of maturity, where they start by implementing traditional clustering solutions for high availability, which increase the hardware requirement, then moving on to deploying virtualizationspecific technologies that enable them to move workloads around the virtual infrastructure in case of hardware failure. However, this technology falls short of understanding the health of the application as it does not monitor the application state inside the virtual machine. VMware has high availability capabilities built in, but they do not monitor the applications running in the guest. Consequently, only the physical host and the guest virtual machine are protected, but not the application inside the virtual machine. IT managers have tried to solve this issue by deploying traditional clustering solutions within the virtual machines. This approach does not solve the problem, as it disables the capabilities to move the virtual machines around as it disables vmotion and DRS. Protection against unplanned downtime is another aspect. In a virtualized environment, virtual machines can be moved intentionally, for example in the case of planned downtime. However, in the event of unplanned downtime, for example when a physical host crashes, virtual machines are not automatically moved to another physical server. 4 #IDCWP36S 2011 IDC
Management complexity. There are many management products out there, both for physical and virtual environments. Each of these has its own management interface/console, making it difficult to manage the infrastructure as a whole (unified monitoring, reporting, problem identification etc.) from the application layer into the virtual layer. When the use of many different tools is required, IT managers incur the cost of training their staff on many different tools and increase the risk of failure due to bad integration and more manual processes. Data and Virtual Machine Protection Challenges Inconsistent VM backup. Server virtualization adds a number of complexities to data and server protection, not the least of which is that a VM may or may not be running when a backup is scheduled. This has resulted in a number of approaches to the protection of virtualized environments and some choices to be considered by the administrator. If the VM is expected to be running, then it can run a backup agent just as it would in a physical environment. If the VM is not running, it may be preferable to back up the host system that includes the image file for the VM. Backing up a dormant VM also avoids the possibility of inconsistency in the backup, but the complete backup set includes the image file, configuration files and also some system state information that is held in memory. More complexity is added by the fact that the VM may be restored to a different system altogether. Overwhelming CPU capacity. A natural (and welcome) consequence of server virtualization is that the physical server will usually be running at far higher levels of utilisation than a non-virtualized server. The downside of this is that there is less spare CPU capacity to handle data protection tasks such as backup. It will not be acceptable for production systems to slow to a crawl while backups or restores are running, so care must be taken to provision sufficient server resource to maintain performance. Complexity and cost of traditional disaster recovery. Traditional disaster recovery solutions are often costly and complex, and may not meet recovery objectives even after duplicating hardware for protected applications. Testing can be difficult because the documentation can be hard to keep up to date. Virtualization allows recovery of a virtual machine to any other machine, either specific duplicate hardware or platform provided by a third-party cloud host. This can significantly reduce hardware costs and lower the complexity of maintaining a backup site. Duplicating backup data. As mentioned earlier, deduplication is particularly effective when backing up virtual servers and virtual desktops, because of the high levels of duplication. Two virtual machines or desktops may be 90% 95% identical, and provides huge potential for data deduplication in the backup process. There are two options for deduplication in virtual environments: When an agent is installed in all virtual machines, data can be deduplicated across all virtual machines at the source. The whole virtual machine is sent to a depuplication server and gets deduplicated. 2011 IDC #IDCWP36S 5
How to Manage Virtualized Infrastructures Efficiently Best Practices In this section we will describe what features and functions a customer should be looking for when buying products to complement their virtual environments. Manage from a single console. As management is still mostly done by the technology group, it is important for the security and storage, data protection and availability groups to manage from one console across physical and virtual environments, whereas it is important to the VMware administrator to manage the whole virtual infrastructure from vsphere. Understand your whole environment, from the application over the data into the physical infrastructure. When introducing another layer of abstraction to the IT infrastructure (like virtualization), it is important to maintain a view of the whole environment in order to understand and resolve problems quickly. Resolve performance bottlenecks. When suddenly increasing the utilisation of an under-utilised server, it becomes more important to manage that server's performance. Depending on the application being run on the server, this increased utilisation could cause problems, so care must be taken with capacity planning. Manage VMs through an NFS file system. Companies are increasingly treating virtual machines as files and thus support them with network attached storage (NAS) instead of storage area networks (SANs) for block storage. Managing virtual machines through an NFS file system is more efficient than within a SAN. Automate movement between tiers according to usage. Automated movement between tiers according to usage is important as it is hard to predict which virtual machines will be "hot" and which will only show little activity. Thin provisioning in virtual environments is also important, as it is hard to predict the storage capacity growth rates for each virtual machine. Scale independently of raw storage capacity. By deploying thin provisioning technologies, companies can acquire storage as their need for storage capacity is actually growing, and not based on an over-provisioning strategy, where each virtual machine gets a set capacity from the start. Being able to scale the storage processing layer independently of the raw storage capacity layer provides significant cost savings without impacting the service levels delivered. Save primary storage space for storing VMDK and VDI files. When creating VMDK and VDI images, companies can save space by creating a "master" or "gold" image, comparing the gold image and the new image and storing only the difference between the two images. Typically around 80% of VMDK or VDI files are associated with the same operating system or applications. The latter is particular true for VDIs, where all employees use for instance Windows 7 as OS, Microsoft Office, Adobe, etc. and where the applications running within a VDI are the same as well. This is also true for virtual servers, albeit at smaller scale. The OS and applications are now only stored once, thus significantly reducing required primary storage space for the virtual infrastructure. 6 #IDCWP36S 2011 IDC
FUTURE OUTLOOK Server virtualization will continue to be one of the most powerful transformative forces in IT and will be the foundation for several future developments. These include: Cloud computing. Arguably as important a development as the transition from the water wheels and windmills of the pre-industrial age to power utility and the national grid. Virtualization delivers the flexibility and resilience that make cloud computing possible, in both public and private form. Dynamic datacentre. The future highly-virtualized datacentre will bring reduced scale-out complexity, integrated autonomic management, dynamic energy optimisation and greater business resilience. Resources can be pooled into "ensembles" that can be managed as a single node. New innovative products that are applicable to both the physical and virtual world. These can be managed from one interface and ensure the same security, protection and availability characteristics as in the physical world. Significant growth is also expected in the fields of storage and client virtualization. Storage is taking an increasing share of overall IT investment and virtualization allows a far more flexible approach to tiering, provisioning, migration and data protection. Client virtualization is delivering significant benefit for some companies through management cost savings, greater security and the ability to tailor applications to users. CONCLUSION The continued success of virtualization is reflected in the fact that business decision makers have built it into their thinking and become advocates of the technology. In many cases, it is a top IT priority due to its ability to save time, reduce operational expenses and improve overall datacentre efficiency. However, challenges lie ahead as IT works to take the initial success of virtualization and transition it to a fullyvirtualized environment. The reality is that the transition will have little to do with technology and will instead be heavily biased towards IT and business process. Knowledgeable VARs, ISVs and consultants are a key enabler in the move to a virtualized world, and should be leveraged at all stages of the migration to a virtualized world. Many have high levels of experience and expertise, and are well placed to advise on a successful deployment. A key element of this is in managing and setting user expectations. Investing in server virtualization should be a no-brainer for a certain class of predictable workloads. IT often has in-depth knowledge regarding these applications and their performance characteristics. Constant analysis is essential to maintaining momentum as organisations target the next wave of applications that are suitable for virtualizing. In addition, management tools that provide end-to-end visibility, performance metrics and a holistic view will help optimise investments in virtualization and drive increased confidence for future applications. 2011 IDC #IDCWP36S 7
Copyright Notice External Publication of IDC Information and Data Any IDC information that is to be used in advertising, press releases, or promotional materials requires prior written approval from the appropriate IDC Vice President or Country Manager. A draft of the proposed document should accompany any such request. IDC reserves the right to deny approval of external usage for any reason. Copyright 2011 IDC. Reproduction without written permission is completely forbidden. 8 #IDCWP36S 2011 IDC