W H I T E P A P E R S u c c e s s f u l D a t a c e n t e r V i r t u a l i z a t i o n D e p e n d s o n C o m p r e h e n s i v e M a n a g e m e n t S o l u t i o n s Sponsored by: VMware Mary Johnston Turner May 2009 I D C O P I N I O N Global Headquarters: 5 Speen Street Framingham, MA 01701 USA P.508.872.8200 F.508.935.4015 www.idc.com Virtualization is increasingly being deployed in large-scale production datacenters to enable more agile business processes and more efficient use of IT resources. As the total number of virtual machines (VMs) and the density of VMs per physical server increase, IT managers are quickly finding that virtual machine management needs to extend beyond configuration automation to encompass application performance monitoring, dynamic capacity planning, and automated change and compliance management. Vendors such as VMware are rapidly expanding their virtual machine management portfolio to help IT managers keep up with these escalating requirements. I N T H I S W H I T E P A P E R This white paper describes the impact that virtual machine implementations have on datacenter management requirements and outlines the major types of functions IT managers need to support in order to run virtual environments efficiently while maintaining required business service levels. The paper concludes by discussing how VMware is extending and enhancing its management capabilities to address these important emerging requirements. S I T U A T I O N O V E R V I E W Increasing business expectations and the demand for IT organizations to deliver highquality services with limited headcounts dictate that senior IT leaders improve productivity from existing staff and "do more with less." Business managers now expect IT to meet or exceed their demands quickly in areas such as improving time to market, ensuring compliance, providing disaster recovery, and increasing competitive advantage. IT organizations are making the choice to deploy virtualization in order to constrain capital costs associated with the growth of physical servers and to support more flexible business environments where shared, pooled datacenter resources can be assigned as needed to a variety of applications and workloads. IDC expects the pace of virtual machine implementation to accelerate over the next several years as many organizations increase the number of VMs deployed per machine and scale up the use of virtualization to support a wider range of mission-critical application workloads.
To the surprise of many IT managers, however, traditional change management, configuration management, performance management, and capacity planning tools and processes are not sufficient to address the system management needs of largescale virtual environments for a number of reasons, including: The sheer number of rapid changes to configuration items, crumb trails, and dependency maps exceeds the capabilities of tools and processes designed for more static environments. The growth and frequency of requests to set up and tear down VMs overwhelm change boards and processes used to coordinate provisioning across server, storage, and network teams. Service desk and problem/incident management processes need visibility into VMs' performance and resource allocation in real time in order to address service-level problems reported by end users. IDC's research finds that virtual machine management is becoming an increasing priority for datacenter managers. Effective, scaled-up virtual server management requires tools and processes designed to accommodate frequent change. These tools must be able to replicate and automate standardized processes and integrate management workflows across multivendor physical and virtual environments. The larger the organization's virtualization deployment, the more important comprehensive virtualization management becomes to achieving the organization's business goals. In a recent IDC survey, over half (56%) of organizations that have implemented more than 50 VMs reported that the management of their virtual environment is very important to achieving their business goals, compared with just 24% of organizations with smaller implementations that feel the same way (see Figure 1). As the scope and scale of virtual environments increase, the need for effective management strategies becomes more evident. 2 #218361 2009 IDC
F I G U R E 1 I m po r t an c e of V i r t u a l E n v i r o n m en t M an a g e m e n t t o R eachi n g O r g an i z at i o n ' s B u si n e ss Goal s Very important Somewhat important Not important/undecided 0 10 20 30 40 50 60 (% of respondents) 51 or more VMs (n = 54) 50 or fewer VMs (n = 46) Source: IDC's Virtual Infrastructure Management Survey, 2009 C o m p r e h e n s i v e M a n a g e m e n t E n v i r o n m e n t C r i t i c a l t o E f f e c t i v e L a r g e - S c a l e V i r t u a l D a t a c e n t e r O p e r a t i o n s To keep up with the pace of change required by larger-scale virtual machine implementations, IT decision makers can take a lesson from the development of physical datacenter management solutions over two decades. In the mainframe era, systems management was largely focused on job scheduling and the allocation of expensive computing resources within the mainframe. With the advent of distributed computing and the proliferation of the number of machines needing to be managed, datacenter management strategies expanded to address a much wider array of tasks, including discovery, performance monitoring, root cause analysis, and capacity planning. More recently, IT management solutions have expanded to encompass system provisioning, workflow automation, and patch management. With the emergence of production-scale virtualized infrastructure environments, IT teams are identifying a new set of management requirements that are necessary to keep business services up and running across rapidly changing virtual environments (see Figure 2). 2009 IDC #218361 3
F I G U R E 2 E v o l u t i o n o f D a t ac en t e r A r c h i t ec t u r e s and M an a g e m en t R e qu i r e m en t s Mainframe Distributed Virtualization 1960s 1970s 1980s 1990s 2000s 2008+ Mainframe virtualization Run-book automation Configuration Discovery Root cause Service analysis mapping ITIL Job Patch scheduling Provisioning CMDBs Event VM provisioning Compliance Performance management Capacity planning Disaster recovery Automation Physical/virtual management integration Source: IDC, 2009 Effective management of larger-scale virtualized infrastructure environments requires tools that have been designed to accommodate the unique nature of virtualized architectures and can recognize the rapid movement of workloads on the fly. With respect to production-scale virtual machine management, many IT organizations are quickly recognizing the need for virtualization-aware capabilities in the following areas: Automated provisioning Disaster recovery Dynamic configuration and change control Compliance Capacity planning Application performance monitoring Root cause analysis Chargeback Integration with physical system management systems and processes 4 #218361 2009 IDC
In the case of provisioning and disaster recovery, this means that tools and processes must be able to rapidly discover resources, implement approved configurations, and dynamically maintain approved software and hardware profiles. IT teams need to be able to quickly evaluate utilization and conduct capacity planning in an environment where resource allocations are adjusted frequently, and they need to be able to evaluate how changes in VM configurations and resource allocations impact the end-toend performance of mission-critical applications. They also need to be able to document compliance with change control guidelines and approved configuration profiles. Root cause and dependency analysis becomes critical in heavily virtualized environments. The interaction of hardware, VMs, databases, and applications makes diagnosing end-user performance problems critical to meeting quality of service SLAs. If the root cause of a problem is associated with lack of capacity, virtualization can enable immediate remediation by adding more resources to affected VMs or even deploying new application instances. Tools that can look across physical and virtual systems are critical to isolating and resolving root cause quickly. IDC's research shows that more CIOs are demanding tighter coordination and integration of virtual system and physical system management processes and tools. These decision makers understand that integration and automation of workflows, CMDBs, and policies are necessary in order to realize the cost-saving promises of virtualization on a broad scale. By deploying a set of coordinated, comprehensive operations management, monitoring, and planning tools across the virtualized datacenter, IT teams are able to increase staff productivity, reduce the amount of time spent on routine tasks, and ensure that end-to-end service levels are maintained. F U T U R E O U T L O O K The number and density of virtual servers deployed across development and production environments will continue to increase dramatically over the next several years. In order to hold down IT staffing and support costs while keeping up with the expected growth of these operationally complex virtualized datacenters, IT managers need to implement a comprehensive virtualization management environment. To ensure a successful role out of these tools, IT teams will need to recognize how virtualization alters system management process requirements with regard to the design of operational workflows and the speed at which decisions must be implemented. IT organizations will find that virtualization changes many aspects of their day-to-day environment. In particular, it significantly increases the importance of using well-defined management best practice processes and business-driven policies to control the environment. IDC recommends that IT organizations take the following steps when they begin to design and implement comprehensive virtual machine management strategies: Take an incremental approach. IT staff should consider existing manual tasks that can be standardized and simplified using the full range of available tools and then map out an implementation plan that proves the value of the tools and provides a model for understanding organizational and workflow impacts. 2009 IDC #218361 5
Look to leverage automation. For most IT organizations, virtualization is being rolled out across server, storage, desktop, and application teams in various stages. Automation of many management functions at each layer offers each team opportunities to standardize processes and realize cost savings from more efficient operations. Plan and budget for comprehensive management from the start. IT should plan for the use of comprehensive management tools from the onset of any virtualization project and should build in a solid budget for both tools and training. Integrate with existing management solutions and processes when possible. Virtual machine management tools and processes should integrate with existing management solutions whenever possible in order to streamline operations and ensure tight coordination across all hardware and software tiers. Measure and monitor success. IT should look at tactical outcomes and measure cost savings and ROI incrementally while also creating a longer-term strategy that recognizes the collective benefit from better integrating and automating a wide range of physical and virtual management activities. Managing virtualized infrastructure on a broad scale requires IT decision makers to apply the same disciplines and best practices to the design, implementation, and operation of their virtual environment that they apply to their physical environment. IDC believes these types of mature virtualization management environments are required to fully move virtual machine technologies into broad-based production datacenters. V M W A R E ' S A P P R O A C H During the past several years, VMware's core virtual machine hypervisor capabilities have matured to support a range of automatic migration and configuration management capabilities. The launch of the VMware vsphere 4 environment adds more built-in visibility across application, network, storage, and server resources. For customers looking to scale up use of virtual machines across their production datacenters, VMware is rapidly extending its portfolio of management tools to provide a full, comprehensive suite. Collectively, these tools are branded with the VMware vcenter logo. Currently available products include: vcenter Server provides a centralized interface to the full range of vcenter management tools using wizard-driven templates and to support task scheduling and alerting. vcenter Server Heartbeat protects the vcenter Server and its database against hardware, operating system, application, and network downtime. vcenter Site Recovery Manager maintains and applies predefined templates and profiles as needed to restore VMs and required configurations. 6 #218361 2009 IDC
vcenter Lab Manager provides self-service provisioning of multi-vm environments for use in application development and test labs as well as release management scenarios. vcenter Lifecycle Manager automates the management of the VM lifecycle workflows, from request to approval to decommissioning. vcenter Orchestrator provides out-of-the-box workflows that can help administrators automate over 800 manual virtual machine management tasks. A workflow library provides blueprints for creating and customizing additional workflows. Throughout 2009 VMware plans to continue to extend the vcenter portfolio to address requirements around application performance monitoring and capacity planning. Products that will be available in 2009 include: vcenter AppSpeed will provide performance management and service-level reporting for applications running within virtual machines. It will supply administrators with visibility into multitier application performance, usage, and dependencies across virtual and physical infrastructure. vcenter ConfigControl will add additional configuration, dependency, and compliance discovery capabilities and allow administrators to search, model, visualize, and compare historic and real-time data. vcenter CapacityIQ will enable real-time capacity planning and what-if impact analysis to help administrators better utilize resources and avoid overprovisioning or underprovisioning of VMs. vcenter Chargeback will help administrators better understand the cost of VMs and allocate those costs appropriately across the organization. When VMware completes the rollout of its full management suite, the firm will have completed an extremely important transition from upstart technology provider to core datacenter infrastructure vendor. By providing customers with a comprehensive management suite, VMware will enable large-scale virtualized datacenters to continue to expand cost-effectively while maintaining mission-critical application workload SLAs. C H A L L E N G E S A N D O P P O R T U N I T I E S However, to fully realize the benefits of these virtual machine technologies, IT managers need to invest in comprehensive management tools, policies, and processes that will enable them to operate these complex environments costeffectively at high levels of service. IDC's research indicates that many organizations are reluctant to invest in sophisticated virtualization management tools because they are concerned about possible disruptions to existing management processes. They also worry that their IT staff members do not have sufficient skills or time to take advantage of the tools' capabilities. 2009 IDC #218361 7
In smaller-scale virtualized environments, where change is relatively infrequent, many organizations can get by with limited, mostly manual management capabilities that are isolated from the broader datacenter operations environment. However, as the scale of virtual machine deployments increases, many IT administrators struggle to keep up with operational requirements. CIOs find they need to assign more staff to manage the constant change inherent in a large-scale virtual environment, yet the organization still experiences lengthy delays when it comes to provisioning new resources or troubleshooting performance problems. Poorly managed virtual machine environments are also likely to suffer from VM sprawl where VMs go unused and consume resources that could otherwise be assigned to new workloads. Without sufficiently mature management tools and processes, the cost of added staff, inefficient hardware utilization, and unexpected downtime can substantially decrease the expected ROI. To drive virtualization deep into the core of enterprise-class datacenters, VMware will be challenged to educate customers about how critical these state-of-the-art management capabilities are to the successful operation of large-scale virtualized datacenters. VMware needs to help educate VM administrators about the value of these tools and help motivate investments in training so that the tools can be used effectively. VMware will also be challenged to help customers do a better job of integrating its virtual machine management tools into broader datacenter operations environments and to accommodate what are expected to be increasingly heterogeneous virtual server environments. As virtualization becomes more mainstream, CIOs will expect the same staff members to manage the full range of physical and virtual servers that are deployed across the datacenter. C O N C L U S I O N Virtualization enables IT organizations to implement more agile infrastructure environments by decoupling the application stack from the underlying hardware and operating system. However, in large-scale deployments, virtualization can also create many new operational complexities that require IT teams to rethink workflows and deploy new, virtual machine aware management tools. The successful ramp-up of large-scale virtual infrastructure environments is highly dependent on the use of comprehensive management tools and processes that are fully aware of the unique capabilities and challenges created by virtualization technologies. C o p y r i g h t N o t i c e External Publication of IDC Information and Data Any IDC information that is to be used in advertising, press releases, or promotional materials requires prior written approval from the appropriate IDC Vice President or Country Manager. A draft of the proposed document should accompany any such request. IDC reserves the right to deny approval of external usage for any reason. Copyright 2009 IDC. Reproduction without written permission is completely forbidden. 8 #218361 2009 IDC