NEXT-GENERATION DATACENTER MANAGEMENT

Transcription

1 NEXT-GENERATION DATACENTER MANAGEMENT From DCIM to DCSO Sometimes described as the operating or ERP system for the datacenter, datacenter infrastructure management (DCIM) technology helps operators manage their datacenters more effectively. But in order to help customers fully optimize and leverage their investments, deliver better services, and manage their resources in real time, suppliers are introducing new classes of software tools that 451 Research is calling datacenter service optimization (DCSO). KEY FINDINGS A considerable number of innovations in datacenter management are discussed and categorized in this report. Key trends are greater automation, greater control, and integration with associated systems to achieve business and service-level control. Most datacenter operators will increasingly be forced into comparisons with the most competitive public cloud service providers, which can operate at very high levels of efficiency and benefit from scale, high utilization, and orchestration and automation tools. The value of DCIM is partly, if not largely, as an enabler for datacenters to use such approaches and technologies. Suppliers of DCIM and other related systems that want to sell large deployments with advanced capabilities need to more effectively demonstrate the value of DCIM not just for today s datacenter, but for tomorrow s. This is not something most suppliers have done well. Current DCIM products are powerful, but their capabilities are increasingly pushing beyond the definitional scope of DCIM, including further up the IT stack and beyond the datacenter walls. In this report, we describe a new overarching category of software known as DCSO that is inclusive of DCIM yet extends beyond it. Independent functions include datacenter business planning, costing, energy resource management, and converged physical/virtual datacenter management. DCSO systems will provide a more business-, cost- and service-oriented view of the datacenter, drawing on external resources, integrating with multiple systems, and enabling the optimization of datacenter services in real time and throughout the lifecycle of a facility. DCIM systems managing the underlying facilities infrastructure are a critical and subsidiary function. JANUARY RESEARCH: DATACENTER TECHNOLOGIES

2 ABOUT 451 RESEARCH 451 Research is a leading global analyst and data company focused on the business of enterprise IT innovation. Clients of the company at end-user, service-provider, vendor and investor organizations rely on 451 Research s insight through a range of syndicated research and advisory services to support both strategic and tactical decision-making Research, LLC and/or its Affiliates. All Rights Reserved. Reproduction and distribution of this publication, in whole or in part, in any form without prior written permission is forbidden. The terms of use regarding distribution, both internally and externally, shall be governed by the terms laid out in your Service Agreement with 451 Research and/or its Affiliates. The information contained herein has been obtained from sources believed to be reliable. 451 Research disclaims all warranties as to the accuracy, completeness or adequacy of such information. Although 451 Research may discuss legal issues related to the information technology business, 451 Research does not provide legal advice or services and their research should not be construed or used as such. 451 Research shall have no liability for errors, omissions or inadequacies in the information contained herein or for interpretations thereof. The reader assumes sole responsibility for the selection of these materials to achieve its intended results. The opinions expressed herein are subject to change without notice. New York 20 West 37th Street, 6th Floor New York, NY Phone: Fax: San Francisco 140 Geary Street, 9th Floor San Francisco, CA Phone: Fax: London Gower Street London, UK WC1E 6HH Phone: +44 (0) Fax: +44 (0) Boston 125 Broad Street, 4th Floor Boston, MA Phone: Fax: RESEARCH: DATACENTER TECHNOLOGIES i

3 TABLE OF CONTENTS SECTION 1: EXECUTIVE OVERVIEW INTRODUCTION KEY FINDINGS METHODOLOGY SECTION 2: DATACENTER ECONOMICS AND THE ROLE OF DCIM DCIM S GROWING PAINS DATACENTERS AND THE COMPETITIVE IMPERATIVE Cloud Providers Are Changing the Competitive Environment Datacenter Services Will Be Compared for Cost and Value Leading Operators Will Employ Every Tactic, Technology and Strategy to Be Competitive The Costs of New-Build Datacenters Will Fall, Not Rise Energy Costs Will Be Driven Down All New-Build Datacenters Will Have Efficient Cooling and Low PUE Ratios More Datacenters Will Be Built With Reduced Resilience and Redundancy Greater Use of Commodity and Open-Source Hardware Datacenter Automation Will Become a Competitive Weapon THE CRITICAL ROLE OF MANAGEMENT SOFTWARE Figure 1: Datacenter Management Maturity Model SECTION 3: DCIM TODAY THE DCIM MARKET DCIM: DEFINITION AND COMPONENTS Figure 2: DCIM, Classic View DCIM COMPONENTS Datacenter Monitoring Asset Management Cooling Optimization and Environmental Management ii NEXT-GENERATION DATACENTER MANAGEMENT

4 3.4 DCIM CURRENT VIEW Figure 3: DCIM, Current View DCIM IS NOT AN ISLAND Converging Technology End-to-End Reporting and Planning End-to-End Service-Level Assurance Management of Availability and Risk Reduction Application Efficiency and Optimization Common Databases and Processes SECTION 4: DCIM EVOLUTION ON FOUR FRONTS ENHANCE AND RE-ARCHITECT: SMART, FAST AND FRIENDLY Real Time Open and Layered Architectures Open Interfaces, Mobile Access Security and Availability Cloud and SaaS Big Data and Analytics Expert Systems and Smart Algorithms Scalability Data Management Service and Business Orientation EXTEND AND DEVELOP THE WHOLE PRODUCT Assimilation of Established Capabilities Into DCIM Specialist Systems for Specialist Markets More Process Automation More Control Greater Support for Multiple Datacenters CONVERGE AND INTEGRATE Figure 4: Software That Integrates With DCIM IT Service Management Server and Converged Systems Management RESEARCH: DATACENTER TECHNOLOGIES iii

5 4.3.3 Energy Procurement and Management Financial Management Building Management Systems Building Information Modeling (BIM) OPTIMIZE AND PLAN FOR END-TO-END SERVICE MANAGEMENT SECTION 5: FROM DCIM TO DCSO DCSO DEFINED Figure 5: The Capabilities of DCIM and DCSO Systems COMPONENTS OF DCSO Figure 6: The Components of DCSO Are Built on the Foundation of DCIM DCSO AS AN INTEGRATING FRAMEWORK Figure 7: DCSO Incorporates DCIM Components Figure 8: DSCO as an Integrating Framework DESCRIPTION OF DCSO COMPONENTS Datacenter Service Management Datacenter Energy Optimization Datacenter Planning and Costing Software SECTION 6: INTEGRATION: THE NEXT FRONTIER 43 SECTION 7: SUPPLIERS PRIMARILY DCIM OR FACILITY-FOCUSED VENDORS ABB Concurrent Thinking Emerson Network Power Huawei itracs (CommScope) Nlyte Software Power Assure Romonet Schneider Electric iv NEXT-GENERATION DATACENTER MANAGEMENT

6 Sentilla Siemens TSO Logic PRIMARILY IT SYSTEMS VENDORS CA Technologies HP IBM Intel VMware INDEX OF COMPANIES RESEARCH: DATACENTER TECHNOLOGIES v

7 The following is an excerpt from an independently published 451 Research report, Next-Generation Datacenter Management released in January To purchase the full report or to learn about additional 451 Research services, please visit or SECTION 1 Executive Overview 1.1 INTRODUCTION In the years ahead, most datacenters, along with the entire IT infrastructure that is housed within them, will become much more efficient, more agile and capable of achieving higher levels of availability at a lower cost. Many strategies and technologies will be deployed to help businesses achieve this, and one of these will be the committed use of comprehensive management systems across the enterprise, and up and down the IT stack. More and more IT and datacenter monitoring, control, analysis and reporting software will be deployed, along with improved cloud orchestration, virtual machine management, and software-driven (software-defined) datacenter technology. These technologies have become the focal point for datacenter innovation. While improvements in processors, networking and storage continue along their impressive paths and have kept pace with dramatic increases in IT service demand, utilization of these technologies remains doggedly low and inefficient, and imbalances between datacenter infrastructure supply and datacenter service demand continue. The challenge for datacenter managers, and for product suppliers, is how to use management tools to improve the efficiency, agility and availability of datacenters, and indeed of all IT services. Price pressure is likely to be intense as large cloud providers drive efficiencies at an industrial scale. The focus of this report is on the datacenter, and in particular, how datacenter management software is changing and will continue to change. Specifically, the goal of this report is threefold: 1. To clarify the positioning and capabilities of current datacenter management technology 2. To clarify why DCIM and related investments are likely to be required in order to compete in the years ahead 3. To clarify how DCIM is evolving, and will likely evolve and converge with associated or adjacent technology categories, such as IT service management, cloud orchestration and other systems. 451 RESEARCH: DATACENTER TECHNOLOGIES 1

8 An overriding theme of this report is that DCIM technology as it is understood today is not only evolving rapidly, but that the next generation of products is extending and converging with associated business, IT and financial management systems. This will prove challenging for some. In this report, 451 Research examines some of the current terminology for datacenter management systems and sets out new or different terminology, notably the terms DCSO datacenter service optimization and DCSM datacenter service management to reflect the need for and evolution toward a more service-oriented and business-oriented view. 1.3 METHODOLOGY This report is primarily qualitative. From 2008 to the present, 451 Research has been closely tracking, defining and influencing the emerging category of DCIM. Over this period, the 451 Datacenter Technologies (DCT) team has not only tracked the evolution of DCIM technology but also worked closely with many vendors, giving opinion, feedback and direction on their product roadmaps and market engagement strategies. The same team regularly advised datacenter operators and corporate buyers on the procurement and use of datacenter software. Additionally, the DCT team liaised with the Uptime Institute, an independent division of The 451 Group, and with its membership, all of whom own or operate datacenters. Limited quantitative research is also cited in this report, based on 451 Research studies and surveys on DCIM adoption, pricing, customer challenges and preferences. Reports such as this one represent a holistic perspective on key emerging markets in the enterprise IT space. These markets evolve quickly, though, so 451 Research offers additional services that provide critical marketplace updates. These updated reports and perspectives are presented on a daily basis via the company s core intelligence service 451 Market Insight. Forward-looking M&A analysis and perspectives on strategic acquisitions and the liquidity environment for technology companies are also updated regularly via 451 Market Insight, which is backed by the industry-leading 451 M&A KnowledgeBase. Emerging technologies and markets are also covered in additional 451 practices, including our CloudScape, Enterprise Security, Information Management, Infrastructure Computing for the Enterprise (ICE) and 451 Market Monitor services. All of these 451 services, which are accessible via the Web, provide critical and timely analysis specifically focused on the business of enterprise IT innovation. For more in-depth information on DCIM, 451 Research publishes regular vendor, technology and market updates as part of our regular Market Insight research service. For more information, contact client.services@451research.com. 2 NEXT-GENERATION DATACENTER MANAGEMENT

9 This report was prepared over several months by 451 Research s DCT analyst team. This report was written by Andy Lawrence, Research Vice President, Datacenter Technologies, 451 Research; and Rhonda Ascierto, Research Manager, Datacenter Technologies, 451 Research. This report is one of a series published by 451 Research in the field of DCIM over late 2012 and It complements our DCIM Market Monitor forecasts and reports; and our Beyond the Basics series: A Guide to Procuring DCIM Software and A Guide to DCIM Suppliers. Further inquiries should be addressed to andy.lawrence@451research.com. Andy Lawrence Research Vice President, Datacenter Technologies & Eco-Efficient IT Andy Lawrence is the VP of Research for Datacenter Technologies and Eco-Efficient IT at 451 Research. He also works closely with the Uptime Institute, part of The 451 Group, and was the program director for Uptime Symposium, a global datacenter management and strategy event, in 2010 and Andy is the author of several major reports covering eco-efficient IT; power management; policy, legislation and compliance; and datacenter management and energy-efficiency software. He has represented 451 Research at the Green Grid, and contributed the eco-efficiency research component in the European Union s Optimis project on open cloud services. He has also contributed to a collaborative international program to develop a carbon footprinting method for IT, and is actively researching the business impact of prefabricated datacenters. Rhonda Ascierto Research Manager, Datacenter Technologies & Eco-Efficient IT Rhonda Ascierto is a Research Manager in the Datacenter Technologies and Eco-Efficient IT practices at 451 Research. Rhonda has been analyzing the crossroads of IT and business for more than 12 years. She focuses on datacenter innovation and energy management in datacenters and across the enterprise. She also covers technologies that enable the efficient use of all resources and help to minimize the environmental impact of business activity. Before joining 451 Research, Rhonda was a Senior Analyst in Energy and Sustainability IT at Ovum. Previously, she had a successful career as an IT reporter and editor, mostly in Silicon Valley, at a number of wellknown publications. 451 RESEARCH: DATACENTER TECHNOLOGIES 3

10 SECTION 2 Datacenter Economics and the Role of DCIM 2.3 THE CRITICAL ROLE OF MANAGEMENT SOFTWARE In many if not all of the examples discussed above, monitoring, management and control software plays a key supporting or central role in improving efficiency, reliability or agility it lies at the heart of efficient datacenter management. In most cases, a number of software systems are required, and these are integrated together to provide a more coordinated, efficient, business-oriented service from the datacenter. This is a recurring theme of this report: For those organizations for which efficient, agile IT and datacenter services are core, the effective and extensive use of management software (not just DCIM, but integrated, related systems) is becoming a strategic necessity. Increasingly, a range of software tools and subsystems are being combined to offer greater infrastructure and IT monitoring, management and control, and they are increasingly being used for a clear, accurate, real-time, end-to-end view of IT service delivery. In some cases, as noted, the effective use of software can be used to manage down risks, move workloads away from failure points, and reduce the need for peak provisioning; in this way, hardware and infrastructure as well as human capital costs may be reduced. The use of software in this way has led to some suppliers using the term software-defined datacenter (SDDC) to describe highly automated, smart datacenters. Because of its fairly specific history, and its association with IT rather than datacenter management, 451 Research is not using the term SDDC in this way. 1 However, we do use the phrase software-driven datacenter to describe how the most advanced datacenters will make extensive use of automated tools and management to reduce costs and risk at every point. In this report, we also introduce the term DCSO (datacenter service optimization) to describe a number of optimization-, service- and business-oriented management products and functions that are not generally part of core DCIM, but that will be increasingly important if a datacenter operation is to remain competitive. This is discussed in detail in Section 5. New terminology is necessary to describe the new technologies and approaches that are being used by the most adaptable, economically sustainable, and best-managed datacenters those where managers have accurate and meaningful information about their datacenter s assets, resource use and status, from the lowest level of infrastructure up into the middle or higher echelons of the IT stack. They can use this information for planning, forecasting and management, for real-time decision making, and if practical, to inform and drive auto- 1. The software-defined datacenter is an evolution from the software-defined network, and later, softwaredefined storage. The terminology relates to the opportunity and technology for moving management functions away from the specific physical devices and centralizing them in management systems run on standard servers. SDDC is a term used by VMWare, among others, and is not generally applied to the datacenter infrastructure, where this approach is problematic (although some opportunities do exist). 4 NEXT-GENERATION DATACENTER MANAGEMENT

11 mated systems. It is no coincidence that the competitive threat in datacenters over the next several years will come from other datacenter operators who are using integrated management tools often aggressively. In the course of our recent research, 451 Research has found that few operators, suppliers or observers question the need for advanced management tools, including sophisticated DCIM tools but some certainly do. These skeptics are not against the use of software where appropriate, but they argue that DCIM systems have become too expensive, they do too many things, and that they attempt to solve problems or supply information and functions that most operators do not need. This, some feel, is reflected in the market data, which shows that while DCIM sales are rising rapidly, they are still very small in the context of the total addressable market. 2 There is probably no anomaly here. DCIM is still a nascent technology, and it is following a maturity curve (see Figure 1). Some suppliers are ahead of the curve and offer more expensive, comprehensive and ambitious solutions they are inevitably addressing a smaller, early-adopter market and have to engage in educative sales processes; others offer lower cost systems with limited function and are able to sell these products more easily to a wider group. As the industry moves up the maturity curve, as we believe it will, then the more advanced systems will be required. These will be more integrated with IT and other management systems; they will frequently act as control systems, with decisions taken based on their built-in logic or analytics. In Figure 1 we have updated our longstanding datacenter management maturity model to show a combination of datacenter infrastructure and management tools likely to be required to reach the highest levels of efficiency. 2. The DCIM market was worth $595m in 2013, projected to rise to just over $1.8bn in 2016, according to the 451 Research Market Monitor service. However, patchy demand may (or may not) lead to a downgrading of these figures see forthcoming reports in RESEARCH: DATACENTER TECHNOLOGIES 5

12 FIGURE 1: DATACENTER MANAGEMENT MATURITY MODEL 3 The maturity curve shown above, which was first introduced by 451 Research in 2011, demonstrates this point. 4 This maturity model has been adopted by some DCIM vendors and has been slightly updated here to emphasize the importance of integration with other management systems and of supporting business (including financial) objectives in other words, it includes the use of DCSO products (see Section 5). As organizations travel up through the datacenter maturity curve, they move from the use of isolated, semi-proprietary and fairly basic software tools toward the use of comprehensive systems that provide a holistic, real-time picture of the exact status of their datacenter. Eventually, we (and others) believe the software will be used to optimize the datacenter, to carry out many of the tasks without direct operator intervention, and, critically, to integrate much more with a range of other software tools to provide business-oriented controls and information (i.e., DCSO). 3. Previously, this table was titled DCIM Maturity Model ; however, due to the broader nature of the some of the components, it has been relabeled Datacenter Management Maturity model. It is not related to, but does complement, the datacenter maturity model proposed by The Green Grid. 4. Datacenter Infrastructure Management Software: Monitoring, Managing and Optimizing the Datacenter, by Andy Lawrence and John Stanley, 451 Research NEXT-GENERATION DATACENTER MANAGEMENT

13 SECTION 3 DCIM Today 3.1 THE DCIM MARKET DCIM consists of a set of mostly software technologies intended to help manage the operational status and assets of one or more datacenters. These include monitoring tools, databases, policy engines and rules, and analytics and reporting systems. In 2013, there were more than 55 companies offering these products, most of which have been on the market less than five years. 5 Although the abbreviation DCIM was first widely used sometime around , many of the underlying components were already deployed as standalone products in larger datacenters for example, to monitor power and environmental conditions, and, less commonly, to construct asset registers, maps and processes for managing changes. Total DCIM sales are rising at more than 40% a year. In 2013, 451 Research estimates the DCIM software market was worth around $595m and in 2016, it will be worth $1.8bn. The leading group of suppliers by DCIM revenue includes Emerson Network Power (Trellis, Aperture and other DCIM component products), Schneider Electric (StruxureWare for Data Centers), Nlyte Software (Nlyte DCIM suite) and CA Technologies (CA DCIM and CA ecometer). A sizeable list of other companies is growing fast and others could yet emerge in a leading position, including new market entrants. DCIM is evolving rapidly, and newer and forward-looking systems have much greater embedded intelligence, integrate with building management and IT systems and business processes, and can provide sophisticated management planning and reporting tools. In its most advanced deployments, DCIM is used as a controlling suite of systems, akin to an operating system, that is the foundation for optimization and automation. As DCIM becomes more integrated with a range of other IT management systems, the lines between different systems will become blurred. Indeed, this is already happening: DCIM products are increasingly carrying out functions that have traditionally been seen as being part of IT systems or service management (such as IT asset management or IT server control). Similarly, other systems, such as cloud billing and orchestration software, are requiring data (such as power use at the device level) that is traditionally held within a DCIM function. There are many examples and scenarios where this integrated view will prove highly valuable see Section 5.3. Although still nascent, it is clear that customers recognize the value in integrating DCIM and IT management systems. A small but growing number of DCIM customers are now using flexible, bi-directional APIs, mostly supplied by their DCIM vendor(s) or partners, to connect DCIM with their ITSM products. 5. See Beyond The Basics: A Guide to DCIM Suppliers, by Rhonda Ascierto, 451 Research RESEARCH: DATACENTER TECHNOLOGIES 7

14 In addition, there is a flowering of innovation as suppliers of all types offer more features and attempt to address some of the opportunities and requirements in modern datacenter management. These include the need to buy and manage energy more efficiently; to coordinate IT and energy demand; to plan investments more strategically using clear blueprints and predictive operating models; and to fully allocate costs and risks, so that services may be executed at the best locations, and so that they may be effectively and flexibly priced (or costed). 8 NEXT-GENERATION DATACENTER MANAGEMENT

15 SECTION 4 DCIM Evolution on Four Fronts 451 Research has identified four distinct ways in which DCIM software is evolving: 1. Enhance and re-architect to maintain competitive, high-performance open systems 2. Extend and develop features and versions to create the whole product 3. Converge and integrate with adjacent and related software tools 4. Optimize and plan for end-to-end service management. One of the effects of these innovations is that the IT department will take more control of the purchasing and management of datacenter software or will at least become more active and influential in the buying process. This might be good news for datacenter software mangers on the IT side are more sophisticated and less hesitant buyers of software than facilities-side managers, the typical/traditional buyer of DCIM. The IT-side buyers will have greater familiarity with data management issues, security, scalability, and will be more familiar with APIs, development tools, and the capabilities of databases, application servers and the middleware required. 4.1 ENHANCE AND RE-ARCHITECT: SMART, FAST AND FRIENDLY Because of its close links to some proprietary and low-level infrastructure and programming devices, expectations around DCIM software have not always been high: some products of just a few years ago more closely resembled those of two decades ago; they had non-intuitive interfaces, limited programmability, point-to-point connectivity, used proprietary protocols and scaled badly. While this is still true of some BMSs, DCIM systems have evolved rapidly. In 2014, datacenter software is in the mainstream of IT, and suppliers of DCIM and related software have, in most cases, reengineered their software to conform to modern requirements and have used modern tools to do so. But they are under pressure to continue innovating. The key generic strands of DCIM development are discussed below REAL TIME Not all DCIM systems are or were designed to support real-time data such as power consumption or location. Capacity/asset management systems, in particular, often use modeled or estimated data, which can cause a variety of problems (from tripping circuit breakers to inaccurate financial audits, and persistent and time consuming problems with database integrity). There is, however, a strong trend toward the use of real-time data which means that it will be necessary to invest more in data collection and tracking technologies (e.g., RFID tags from companies 451 RESEARCH: DATACENTER TECHNOLOGIES 9

16 such as RF Code and others), and in integrating monitoring systems with asset management systems. Systems will also have to be scalable (see below) and have robust data management architectures OPEN AND LAYERED ARCHITECTURES DCIM systems are mostly written using a combination of SQL databases, application servers, and web services interfaces. Some have open-source components. In most cases, it is or should be possible for other software programs and data to interface with DCIM systems without too much specialist product knowledge. But there are some exceptions especially for heavily graphical packages OPEN INTERFACES, MOBILE ACCESS Most DCIM systems but again not all are or will be written with web interfaces, including HTML5. Proprietary interfaces and the use of agents cause some problems and, while they remain widely used, are no longer always the primary or only interface SECURITY AND AVAILABILITY Security concerns around DCIM are growing open architectures, the use of IP addressable devices, and mobile devices make datacenter management systems more vulnerable to hacking and malware (the Stuxnet virus attack proved that even systems with no obvious ports onto the Internet and that use non-standard but programmable equipment can be vulnerable). We expect DCIM vendors to invest more in security and to advertise their conformance and support for standards and certifications, such as ISO 27001, much more in future. Suppliers will need to be more mindful of separating control systems from the IT they support and design their systems to fully utilize distributed architectures without single points of failure CLOUD AND SAAS In spite of some early resistance, customer interest in DCIM systems delivered via cloud or SaaS is building rapidly. 451Research expects all major DCIM suppliers to offer SaaS options by the end of 2014; some smaller companies will offer only SaaS versions. (In addition, of course, datacenter management systems will need to support cloud applications and services generally.) BIG DATA AND ANALYTICS With low-level monitoring set to commoditize over time, greater investment and focus will be placed on the ability of DCIM systems not only to collect and aggregate data (as they all do now) but to draw insights from ever larger amounts of fine-grained data. For example, an advanced DCIM system will draw metrics, key performance indicators and trends by collecting many small as well as unstructured data points. This data can be used not only to identify problems, trends and opportunities for optimization but also to predict issues. 10 NEXT-GENERATION DATACENTER MANAGEMENT

17 4.1.7 EXPERT SYSTEMS AND SMART ALGORITHMS The higher levels of advanced DCIM require that more decision making is made by the system itself even where there are still humans (rather than machines) that actually act on advice and apply the controls. This means greater use of embedded learning technologies, expert system and algorithms to enable the system to identify and act on issues rapidly. DCIM cooling optimization systems from companies such as Vigilent, SynapSense and senergy Thermal apply learned rules about cooling before fine-tuning controls SCALABILITY Ever growing amounts of data from a wider range of correlated sources and ever more volatile IT and environmental conditions means that DCIM systems need to be architected to ensure they can handle vast amounts of data and will be able to correlate it in real time. Larger and more sophisticated DCIM deployments will require the use of technologies such as three-tiered architectures (a rack or locally sited device, such as Emerson s Universal Management Gateway, collects and processes data locally before communicating with the central management system, for example); in-memory databases; and databases that have been optimized for scale (possibly non-sql databases). DCIM monitoring systems will need to be able to quickly correlate data from disparate sources DATA MANAGEMENT Collecting large amounts of data doesn t just create scalability issues. Datacenter management systems will need to be deployed with data management strategies in mind: which data is to be collected at what levels; which data is to be aggregated; who will have access to what data; how and what data will be kept, and for how long; and which data should be treated as key and most reliable (power data, for example, could be collected in multiple ways from different places). In many datacenters, this has been addressed in an ad hoc fashion, and many of these capabilities will not be built into DCIM software SERVICE AND BUSINESS ORIENTATION While IT management has increasingly moved toward user-friendly, business-oriented interfaces and processes (often using consumer-influenced technology), DCIM systems have only slowly moved in this direction. Just in the past two years have some of the leading suppliers offered the open, intuitive designs expected of software in more general enterprise IT markets. 451 RESEARCH: DATACENTER TECHNOLOGIES 11

18 4.3 CONVERGE AND INTEGRATE However datacenter management software develops (and whatever labels are given to the various functions), the end result (at least for those datacenter operators committed to using DCIM software to manage effectively) will eventually be much the same: managers will have access to a rich framework of data sources, applications, services and middleware software components that can be plugged together to support all the technical and commercial operations of the datacenter. Many of the software components or components that DCIM integrates with are shown in Figure 4. The data sources/integrations toward the bottom are facilities oriented, while the ones nearer the top are more IT and business service oriented. FIGURE 4: SOFTWARE THAT INTEGRATES WITH DCIM Integration with many of the facilities-oriented functions is already a necessary and core function of DCIM systems, while in other areas particularly the more IT and business-service oriented areas interfaces are currently rare. Over time, integration of DCIM with a wider range of datacenter systems will become more common. A key question in datacenter management is: How will all this integration be achieved and how will it be supplied? At one extreme, some suppliers and buyers will favor a more integrated whole product approach, while others will prefer a gradual approach, plugging together the parts to give more flexibility but at the cost of some greater complexity and, possibly, expense. In Section 5, we describe a new, overarching category of software datacenter service optimization that includes the independent functions of datacenter business planning, costing, energy optimization, and converged IT and infrastructure management, or datacenter service management. When used or converged with DCIM functions, DCSO software can be used to support end-to-end service management and optimization. We envisage that the DCIM/DCSO combination will be become a key integrating platform for datacenter operations in the nearto-medium term. 12 NEXT-GENERATION DATACENTER MANAGEMENT

19 But other key products (such as building management, IT asset management, financial management and ERP) will overlap with DCIM and DCSO while remaining outside. Given that no one supplier will be able to provide all functions, it will continue to be important that components can be plugged together into an open framework. It is already clear that some fragmentation of function is occurring in the datacenter market, with suppliers deciding to focus on certain core or specialist functions and leaving the backbone DCIM functions to bigger companies. Open standards and interfaces, not just for exchanging data but processes, with a hierarchy of control, will therefore be critical. Tight integration within product families, or between the products of alliance partners, is unlikely to stop; these present an opportunity for product differentiation and maintaining competitive edge. But DCIM products are opening up and becoming less proprietary. Even large, market-leading suites from suppliers such as Schneider and Emerson which can be used in conjunction with hardware or subsystems from these (and other) companies are now being designed on common components, with published interfaces, so that additional systems can be plugged in. The overall architecture will need to conform to open standards such as web services as well as supporting a wide range of protocols, and some already do this. Below we discuss the integrations we consider particularly important IT SERVICE MANAGEMENT Convergence and/or tight integration with ITSM systems (such as IBM Tivoli, HP BTO, BMC Remedy, and a long list of service tools) is clearly becoming an important strategic direction for DCIM and is driving considerable innovation. This is dealt with in Section SERVER AND CONVERGED SYSTEMS MANAGEMENT IT and facilities infrastructure have traditionally been managed almost entirely separately. Servers, for example, can report back key indicators such as power use, fan speeds and temperatures while most DCIM uses sensors just a few feet away, collecting much of the same data (and usually much more) for facilities managers. The ability to create closed loops between power use, IT demand, and cooling introduce the opportunity for very significant advances in efficiency and the opportunity to control and manage peak loads an important advance that could save significant capital. IT management systems such as HP OneView (formerly Insight Manager), IBM Director, Intel DCM and VCE s Vision Intelligent Operations are all increasingly being linked to DCIM systems while some DCIM systems, notably Emerson s Trellis, are being used to directly monitor and manager IT. This is discussed in Section ENERGY PROCUREMENT AND MANAGEMENT One of the central functions of DCIM is to manage power use and distribution. At present, however, the ability to not only track but also manage energy demand by IT and datacenter energy generation and sourcing is limited. This will be an important area in the future; this is dealt with in Section 5, below. 451 RESEARCH: DATACENTER TECHNOLOGIES 13

20 4.3.4 FINANCIAL MANAGEMENT DCIM systems are not primarily designed to manage datacenters from a financial point of view. While these functions will be improved, we expect better integrations to be developed between DCIM and financial systems so that resource use can be monitoring and allocated, billing and chargeback facilitated, and assets tracked. This is also dealt with in Section BUILDING MANAGEMENT SYSTEMS Although links to BMSs are commonly available, they do not always work well. BMSs and DCIM systems will, in some cases, converge altogether, especially for prefabricated modular datacenters, or where the datacenter is the building BUILDING INFORMATION MODELING (BIM) There is growing interest in managing the design of datacenters throughout the lifecycle, from the drawing board onwards, and in building models that show how the datacenters fit into their environment (cabling, power, access, public buildings and services, etc.). BIM models are primarily used by architects and engineers to construct detailed, accurate models of buildings; such models are also able to link the models of nearby buildings, even creating modeled smart cities. The integration between BIM models and DCIM systems present an opportunity to model how datacenters can be designed and then operated efficiently in their environment. Some suppliers already integrate BIM models into their DCIM systems, and we expect more will do so in the future. Operators will find that their experiences of putting this framework together will vary widely. Some of the components will be highly packaged or productized on delivery (even baked into prefab datacenters, for example, or simply delivered as a plug-and-play service); some will be delivered as a collection of tools that can be highly customized; and some will installed and developed in a pick-and-mix fashion by datacenter and IT management from a variety of suppliers over time. While sophisticated DCIM suites are being developed, especially by a small group of leading suppliers, it is clear that no one supplier is yet able to provide all the components to support the entire datacenter and all the systems it must interact with. 4.4 OPTIMIZE AND PLAN FOR END-TO-END SERVICE MANAGEMENT As we have already discussed, the datacenter environment is changing rapidly: all operators are seeking to deliver much more agile, efficient and cost-effective services. And while availability remains a critical concern, it is clear that not every service, application or customer needs a platinum-class service with all the associated costs. As an analogy, many of today s high-end datacenters are like full-service airlines: reliable and expensive, and vital for a certain class of customer. But they are being supplemented by the lower cost services, which will be tiered in terms of what has been paid for, or what can be reasonably expected for the price. And whatever class of service is being delivered, competition means that management of costs is now far more important. 14 NEXT-GENERATION DATACENTER MANAGEMENT

21 This leads a new way of thinking about datacenter management: Not all classes of work, customers or jobs (whether compute, storage or network) can be treated as of equal weight by the datacenter management, and it will be important to understand service level requirements and the costs associated with delivering these services. Furthermore, it will be necessary, rather like an airline, to be able to vary prices according to demand, to costs and to availability. This planning, furthermore, will have to extend far into the future, as well as over the next minutes or hours. It will involve management, costing and purchasing of available resources, as well as management of demand by IT. Today s DCIM tools will be critical in supporting this, but they do not cover all requirements. Best-execution venue, IT service showback and chargeback, service-based costing, datacenter business planning, forward energy planning and purchasing all supported by real-time data and optimizing analytics will increasingly be required, and these involve the integration of multiple management systems. Some of these capabilities will be supported within existing DCIM systems. But 451 Research also believes that many of these functions can be grouped as a separate but closely related class of technologies, which we describe in detail in the next section. 451 RESEARCH: DATACENTER TECHNOLOGIES 15

22 SECTION 5 From DCIM to DCSO DCIM products, features and capabilities continue to evolve and expand. Yet the scope of DCIM largely remains at the physical level addressing the management of critical systems, such as cooling and power distribution, as well as IT assets and other supporting infrastructure within the datacenter. For some time, it has been clear that DCIM as a category does not generally include related yet distinct datacenter management products and features that are emerging as DCIM integrates further up the IT stack with virtualization and IT service management as well as with other systems such as building management and enterprise energy management. We have introduced the term DCSO (pronounced DEE-soh ) to categorize these new types of products that are beyond the scope of DCIM but that will clearly be core to next generation datacenter management. DCSO products are closely related and largely dependent on DCIM functions but may be classed as a standalone category. Most of these functions are being talked about already, and in a few cases delivered, but they have not hitherto been clearly mapped out or defined. We believe the category DCSO as defined below captures new and emerging datacenter management functions, capabilities and products that will not only extend, but amplify and enhance the value of stand-alone DCIM products. We also use the term datacenter service management (DCSM), since DCSM is a core building-block component of the broader DCSO category. 6 DCIM systems manage the physical critical assets within the datacenter to optimize dayto-day operations, while DCSO systems manage both physical and virtual assets, within the datacenter and beyond, to enable a more business-, costand service-oriented view of datacenter services. 6. The term datacenter service management (DCSM) has been occasionally, but not consistently, used by a few companies and individuals in recent years. As far as we can see, it was first used by Emerson Network Power, one of the leaders in DCIM software, to describe its early software capabilities in the area of convergence between DCIM and ITSM. This is similar to, but not identical to, the way that 451 Research is applying the term (which we arrived at independently). Emerson and 451 Research agree that the term is useful and descriptive, and is now in the public domain. 16 NEXT-GENERATION DATACENTER MANAGEMENT

23 5.1 DCSO DEFINED DCSO extends the capabilities and scope of DCIM; it is not a redefinition of DCIM or a replacement. DCIM is primarily focused on the monitoring and management of the physical critical systems, such as cooling and power distribution equipment, the physical IT assets, and the supporting infrastructure within the datacenter. The term DCSO captures a category of activities that are still in their infancy: the construction of cross-functional processes that span the IT-Facilities divide; the integration of ITIL-like disciplines and the management of services and change; and the growing interest in methodically allocating the full costs of applications and services as part of an operational process, with real-time optimization as an end goal. Similarly, it is likely that more progressive large consumers of energy will invest to optimize energy efficiency and costs through analysis and active intervention in energy supply markets and by managing the energy demand from their IT workloads. The term DSCO is intended to capture that transition to a more business-, cost- and service-oriented view of the datacenter, drawing on external resources, integrating with multiple systems, and ultimately enabling the automated optimization of datacenter services in real time. In time, DCIM may become part of DCSO, but today DCIM is a foundational subcomponent of DCSO. Datacenter Service Optimization is a category of integrated software tools and systems that are used to proactively manage the supply and demand of physical and virtual resources within a datacenter, and beyond it into the digital infrastructure and power chain, both dynamically and throughout the lifecycle. DCSO systems are used to plan and optimize datacenter resources and services for availability, agility, and financial, operational and energy efficiency. Physical and virtual resources include critical systems; assets; power; compute and IT services; and applications. DCSO software tools will usually integrate or interact with multiple systems or services, such as IT service management, virtual machine management, or energy pricing and sourcing. DCSO systems will typically integrate with or include components of datacenter infrastructure management. 451 RESEARCH: DATACENTER TECHNOLOGIES 17

24 FIGURE 5: THE CAPABILITIES OF DCIM AND DCSO SYSTEMS In short, DCIM systems manage the physical critical assets within the datacenter to optimize day-to-day operations, while DCSO systems manage both physical and virtual assets, within the datacenter and beyond, to enable a more business-, cost-, and services-oriented view of datacenter services. 18 NEXT-GENERATION DATACENTER MANAGEMENT

25 SECTION 6 Integration: The Next Frontier As we have alluded to, there are many different ways to implement both DCIM and DCSO: some argue in favor of a loose framework of best-of-breed components, others for tighter suites from a single supplier. Similarly, there are many approaches to integrating DCIM and DCSO tools into the wider digital infrastructure, and the degrees of integration can vary widely. In this section, we discuss the broad issues around integration this will be covered in greater depth in forthcoming reports. At present, most DCIM products are integrated with other applications or data sources either by specialist connectors or low-level open protocols (e.g., from a monitoring system to a cooling system using a proprietary connector, or using Modbus, BACnet or other protocols). Web services (XML, SOAP, etc.) are also commonly used for data exchange. For visibility and discovery of devices, DCIM products are mostly self-contained, although some now collect data on devices either through direct integrations (for example, with Intel s Data Center Manager) or by auto-discovery, using IP and SNMP. Emerson is able to converse with many devices using its own out-of-band interfaces. Once data such as this is available to a DCIM package, it becomes a more powerful system, capable of displaying data on common dashboards or integrating data into single databases for metrics and business intelligence. The more advanced DCIM packages are particularly strong in this area. Some can also create secondary alarms and produce derived metrics that can only be created by having access to multiple data sources. The kinds of integration described as part of DCSO go beyond this and involve using both data and workflows from external systems (bi-directionally) to enable the automation of end-to-end business and operational processes. In the datacenter, this becomes especially challenging as it involves correlating the interrelationships between the physical (an asset), the logical (its location and network status) and the virtual (the hypervisor and applications of a virtualized server). This has led some suppliers/analysts to propose the idea of a DC-CMDB (datacenter configuration management database), which involves extending the disciplines of ITIL into the physical database. This means that all the databases (DCIM assets; ITSM assets; and monitoring, maintenance, and software application assets, for example) are using common descriptions and all the interrelationships, technical data, ownerships and dependencies are managed and, to a degree, synchronized (note: this would not be one database, but a federation of interlinked systems). While the idea of a DC-CMDB has not met with much interest from suppliers or customers, there is a general move in this direction, even if the same terminology is not used. Schneider, for example, has been working to standardize on architecture for 451 RESEARCH: DATACENTER TECHNOLOGIES 19