Abstract Adaptive Internet Data Centers Jerome Rolia, Sharad Singhal, Richard Friedrich Hewlett Packard Labs, Palo Alto, CA, USA {jar sharad richf}@hpl.hp.com Trends in Internet infrastructure indicate that organizations with mission-critical needs are outsourcing applications to a small number of Internet data center providers. These providers operate networks and services at a global scale, and offer customers (and other service providers) a variety of services ranging from high bandwidth networking to fullfledged application hosting and IT management services. These global service providers offer their services through large data centers. Each will eventually contain 10,000-50,000 servers with high-speed network connections to other data centers and the Internet. The large scale and rapid evolution of these data centers offers significant challenges for the data center operators. This paper describes issues that face data centers as they scale in size and our current research to support such scalability. It explains our approach to: introduce greater numbers of applications into these environments, support automated management, and make it attractive for organizations to outsource the hosting of greater numbers of applications. Keywords : Internet data centers, resource management, virtual application environments 1 Introduction s are increasingly relying on their computing applications to support customer interactions and global inter and intra-organizational workflows. al effectiveness and customer loyalty requires secure, continuous, and responsive application behavior. Rapidly evolving application features, workloads, and infrastructure technology make the management of applications and their supporting infrastructure a significant challenge. One approach for dealing with such problems is to outsource the hosting of critical applications to data centers. This paper gives an overview of data centers and application hosting. It describes issues that face data centers as they increase in size and motivates our recent research on Internet data center infrastructure. Data centers can offer customers different levels of services. At a minimum, the first generation data centers offer a reliable physical infrastructure along with some value added management services. Physical infrastructure includes redundant air conditioning, hazard management (such as sprinkler systems), power systems, as well as redundant network connectivity. This gives an ideal physical environment for critical systems infrastructure. We define an application as a collection of operating system processes that reside on one or more compute nodes to provide some service. There are many models for co-locating applications in these data centers. We distinguish models for co-location based on how applications and data center resources are shared.
Data center resources Data center resources Data center resources Data center resources Case A Case B Case C Figure 1 hosting examples Figure 1 shows several application hosting scenarios. s may support individual users, many users from a single organization, or many users from multiple organizations. Similarly data center computing resources can be dedicated to specific instance of an application or shared by multiple instances of an application or even multiple applications. Thus, for example case A in Figure 1, a web based customer resource management (CRM) application may support users from several organizations with its application components running on dedicated computing resources. The users are only exposed to their own organization s data. s are isolated by the application itself so they are not aware of other organizations. In case B, a web page hosting service may serve several organizations from a single compute node with one web server per customer organization. Alternatively, as in case C, each organization could receive its own compute node. s with dedicated compute resources typically place their resources in a locked cage within the data center. Co-location alternatives have implications regarding isolation, security, reliability, cost, predictability, and data center scalability. Case C offers the path of least resistance for introducing greater numbers of applications into Internet data centers but has operational scalability issues that must be addressed. We consider these issues in this paper. Data center service providers also offer value added management services. Typical services include health and performance monitoring for networks, servers, and application processes. On-site staff may have the capability to restart applications or servers. Other services may help to manage security, limit the impact of denial of service attacks, backup data, distribute applications and/or content to several global locations, conduct capacity planning exercises, and deploy more server resources for applications when required. Value added management services help organizations that outsource to realize the advantages of centralized management. They also increase profitability for the data center provider and offer a means to differentiate its service from that of its competition. Our research is focused on the run-time and operational scalability of Internet data centers. Section 2 of this paper presents goals for large Internet data centers, i.e. those with 10,000 or more servers, and introduces the notion of virtual application environments (VAE). VAEs provide a model for encapsulating system resources in a way that supports the management of these very large systems. Section 3 describes modern application and operating system platforms and how they relate to the notion of VAEs. Section 4 explains how we exploit VAEs to reduce application provisioning and lifecycle
costs and to provide continuous operation and managed performance. Summary remarks are provided in Section 5. 2 Large Internet Data Centers Economies of scale for physical infrastructure encourage the growth of Internet data centers to the tens of thousands of compute nodes. However it is unclear that current centers can scale to this magnitude. Large data centers need a very large number of applications to feed their potential capacity. Therefore it must be attractive for organizations to outsource the hosting of more applications than is the case today. Also it is not clear that current Internet data center management practices will scale for such large systems. Consider a 50,000 server data center with an average of 20 servers per organization. This implies approximately 2500 cages, system specific agreements, and management processes for managing each cage's 20 or so servers. A tremendous physical effort will be needed to intervene within these cages simply to add, update, and replace computing resources. Data center operators will have to react to common problems in different ways depending on each customer's unique agreements. These issues lead to the following goals for large Internet data centers: It should be straightforward to introduce both new and legacy applications to the data center this is essential to exploit potential data center capacity. Data center management services must be automated so that very large numbers of applications can be managed in a cost effective manner. s that outsource application hosting should be able to delegate responsibility for more comprehensive management services to the data center -- to realize the benefits of centralized infrastructure management and encourage outsourcing. Now we present our assumptions about the nature of large scale Internet data centers and an approach for achieving the above goals. We assume that the majority of servers in large data centers will be commodity servers running common operating systems such as Linux, Windows, and FreeBSD. We refer to these as front-end servers. Specific server products will be chosen to minimize price and maximize performance. Price is a function of purchase price, power consumption, and computing capacity per cubic meter. presentation (web servers), logic, and data object servers will execute on such platforms. This is the case today for many of the Internet's most popular sites. Furthermore, most application platform vendors currently support one or more of these operating systems or plan to support them. Back-end servers will typically host databases and other persistent storage servers such as file servers. These rely on more scalable and robust multi-processor operating systems such as HP-UX, Sun Solaris, IBM AIX and MVS, and Windows 2000 Data Center servers. Commodity servers will be connected to backend servers via network switching fabrics. The use of cages in this next generation data center will be limited to the proportionally small number of backend servers that require infrequent physical intervention.
Typical management actions for hosted applications include upgrading application content, application lifecycle functions such as installation and changing software versions, and adding processing resources to better meet performance objectives. To achieve our goals we must minimize physical intervention for these kinds of actions. Upgrading content is application specific and likely to remain in the hands of an application's organization. However application lifecycle support and performance management offer many opportunities for automation. We focus on automating the management of the more rapidly changing and numerous front-end servers. To support automation and ease of introducing applications into the data center we introduce the notion of virtual application environments (VAE). A VAE presents an environment to an application that is consistent with the application s configuration requirements. For example, every application has certain requirements regarding network connectivity, file systems, middleware, and storage services. It can have explicit layers of servers and many local area networks. Network and storage fabrics are configured to make a portion of the data center's resources and back end servers appear to an application as a dedicated environment. Data center management services create these VAEs in a manner that isolates them from one another. Each VAE is allocated appropriate network, compute, and storage resources. In the next section we describe control and monitoring interfaces for hosted applications. The control interface enables us to add and remove resources from an application and thus supports application lifecycle management. Furthermore monitoring information provides the data center management systems with real-time feedback on application behavior. This information is used to make decisions about when to add/remove resources from an application. A goal of our research is to develop a service level directed resource management system that accepts monitoring information, compares the information with service level objectives, decides changes to resource allocation, and then dynamically implements the changes. We view the system as dynamically allocating resources, such as nodes, to VAEs from a pool of shared data center resources. Note that each VAE has a rigid multitier software and networking structure with rich state information that is determined by the nature of its application. For this reason it is not helpful to think of allocating applications to individual data center resources. Instead resources must be introduced within an application environment so that they can join the application's context. This is the dual of the standard view of load balancing where application components are allocated dynamically to a pool of shared system resources. This latter form of load balancing may occur within applications but in general not, due to security concerns, between hosted applications (except case B of Figure 1). We believe this change in paradigm is important for the multi-tier applications typical of large Internet data center environments. 3 and Operating System Platforms This section describes application and operating system platforms and their synergy with the management paradigm we propose. s can face significantly different workloads over time. Time of day, time of month, growth in the number of users, and special events can cause rapid and sometimes
unpredictable changes to workload intensity and mix. It is common for workload intensity to change by a factor of 3 to 7 depending on time of day alone. Special events (e.g., web-based promotions) can cause even more significant changes to resource requirements. These continuous changes combined with the complexity of applications makes it difficult to over-provision resources for all possible workload conditions [1]. Instead our goal is to allocate resources to applications as needed. This section describes common patterns for controlling the allocation of resources to applications. We discuss relevant features of load balancers, application specific middleware, and operating system platforms. An application can be as simple as a single operating system process that provides a service. Alternatively it can have multiple tiers with tens or hundreds of operating system processes cooperating on common goals. In this paper we focus on the more complex applications. processes are typically organized in clusters. For example, a cluster of web server processes, each running on its own compute node, may serve a common set of web pages. When web page request volume increases more web servers (one per node) can be added to the cluster. If request volume is low, a node can be removed from the cluster and reused for some other purpose. The most common way of supporting clusters of web servers is to use a load balancer. A load balancer receives all requests for a virtual IP address. The load balancer then uses a lookup table to choose a web server from the cluster that serves that IP address. It forwards the request to that server. Various policies are used to divide the work across the cluster of servers. By managing the load balancer lookup table we can add and/or remove web server nodes from the application. Clusters and this load-balancing pattern enable scalability and continuous operation for such applications. The level of indirection offered by the load balancer permits us to dynamically change the capacity of the application without making the application unavailable. If one of n application servers in a cluster fails, the system retains n-1/n of its capacity. For all but small values of n, such clusters can typically maintain their capacity with minimal impact on responsiveness. Load balancers are typically redundant to ensure they are not single points of failure. Various technologies are used to implement load balancers. They offer different features, price-points, and capacity. Variants of this load-balancing pattern also appear in subsequent application tiers. tiers are typically partitioned into clusters of application server objects, which encapsulate the code that implements the application. To enable scalability and fault tolerance, some applications may also partition individual tiers into many distinct clusters. Server objects can be added and/or removed from these clusters to manage the compute capacity of the cluster. Each application server object can reside on its own node. Requests (such as remote procedure calls) to server objects are typically made within the context of sessions. A calling process s first request goes to the load balancer which then forwards it to a specific server object. Subsequent requests within the caller s session go directly to the server object. When the session completes the calling process s relationship with the server object is dissolved. When a server object is removed from the cluster, it must warm down and complete any currently active sessions. Variants of this
pattern are common in platforms that aim to support scalability and continuous operation. These include platforms based on the Java 2 Enterprise Edition (J2EE) [2] and Microsoft Dynamic Networking Architecture (DNA) COM+ [3], and legacy platforms such as SAP [4]. Clustering and load balancing are problematic once an application reaches the database tier. Loosely coupled distributed databases have performance and management challenges that preclude common usage. In general a database must execute on a robust platform with adequate capacity. Trends in this area are to use multi-processor or tightly coupled multi-node platforms. Typically a configuration is chosen with more processors than are needed. Processors are often included but only activated when they are first needed. This is often referred to as capacity on demand. More advanced operating system platforms can support 64 or more processors. Some of these platforms (for example Sun s ES series [SUN00] and Hewlett-Packard s forthcoming Superdome series [SUP00]) offer the ability to create virtual nodes. These platforms enable the dynamic allocation of server resources to virtual nodes. This provides for the sharing of the resources (namely processors and memory) across different organizations applications even for the database tier. To summarize, many application and operating system technologies provide interfaces that permit dynamic changes in resources to applications. These changes can be made for application lifecycle support and/or for performance management. The next section describes management services that interact with these interfaces to automate the management of large Internet data centers. 4 Management for Large Internet Data Centers This section describes the relationship between data center management systems, application VAEs, and application lifecycle support and performance management. We focus on the management of front-end server resources. We assume two management interfaces. A control interface manages the allocation of commodity nodes to application clusters, and a monitoring interface reports application and node metrics to the data center management system. The data center management system uses these interfaces to allocate and de-allocate commodity resources for the VAEs. The implementation of these interfaces is application, cluster, and operating system platform specific. The control interface supports two operations. The first causes a new node to join an application cluster with an appropriately configured application server object. The second causes a node to remove its application server object from the application cluster. This allows the management system to dynamically change the resources allocated to the application. The same control mechanisms can also be used to add resources to the application in case some resource fails. specific mechanisms are needed to maintain application state in case of failure. The control interface offers direct application lifecycle support. Data center management systems can cause resources to be allocated to application server objects which can start within their VAE and join their applications. These mechanisms can be used to install, start, stop, and uninstall applications. Furthermore if an application is designed to allow
components from multiple versions to co-exist, and inter-operate, then the control interface can be used to manage application upgrades within live environments. This is normally considered a difficult challenge for data centers, because continuous operation is required even though applications can change frequently (daily to weekly). Using the control interface, the management system could slowly and systematically cause nodes with older version server processes to shut down and introduce nodes with the new versions of server processes. The monitoring interface provides information on service levels so that the data center management system can make decisions regarding resource allocation. The monitoring system tracks node resource consumption and possibly application responsiveness. This information is compared with application service level objectives. If thresholds are violated then the management system is notified through the monitoring interface. Policies can be implemented by the management system to cause the addition of new resources through the control interface. In addition, if too many resources are allocated to an application, then resources can be removed. In this way a data center s commodity resources can be shared across multiple applications according to service level agreements. s need not attempt to continuously over-provision the numerous front-end server resources for their application clusters. Instead they can focus on hosting agreements that specify required responsiveness and throughput. We believe that this combined with application lifecycle support offers compelling value to organizations and encourages outsourcing for greater numbers of applications. The data center management services we describe must also scale to support the large numbers of servers and applications that are envisioned. It must be reliable and secure, offer continuous operation, and provide support for software and hardware license accounting. There are many research challenges to be addressed. The synergy between application and host monitoring data abstraction (node only vs. application specific metrics, level of detail required) and resource allocation algorithms must be evaluated. The utility of control and queuing theory for decision making must be assessed. 5 Summary This paper describes issues and goals for the management of large Internet data center environments. We argue that comprehensive application lifecycle and performance management services are needed to compel organizations to host large numbers of applications in these environments. This provides a motivation for our research on data center infrastructure. We introduce the notion of virtual application environments (VAE). These environments encapsulate portions of data center resource and are customized to appear to hosted applications in a manner that satisfies application configuration requirements. This makes it easier to introduce greater numbers of applications into large data center environments. Many application and operating system platforms are designed to support continuous operation and scalability. These environments provide subsystem specific methods for dynamically adding and removing resources and for performance and availability monitoring. We rely on monitoring and control interfaces to interact with these methods to provide the data center management system with the ability to learn about current application behavior, cause appropriate changes in resource allocation, and to offer
application lifecycle support. This approach presents a dual to the classic load balancing paradigm of allocating applications to resources; with our approach system resources are allocated to applications via VAEs. By integrating monitoring, control, and the notions of VAEs, comprehensive management services can be offered to organizations that outsource the hosting of their applications. These services let organizations focus on requirements and service level agreements for hosting instead of the details regarding how hosting is accomplished. We believe this division of responsibilities provides the long-term value that will further encourage outsourcing. Automating the management of the numerous front-end servers better enables data centers to grow in size. Trademarks HP-UX is a trademark of Hewlett-Packard, Solaris is a trademark of Sun Microsystems, AIX is a trademark of IBM Corporation, and Windows 2000 is a trademark of Microsoft. Bibliography [1] M. Litoiu and J. Rolia, Object Allocation for Distributed s with Complex Workloads, B. Haverkort, H. Bohnenkamp, C. Smith (Eds.), Tools 2000, Lecture Notes in Computer Science volume 1786, pages 25-39, Springer, March 2000. [2] Anne Thomas, Java 2 Enterprise Edition, Ensuring Consistency, Interoperability, and Portability, White paper, June 1999. http://java.sun.com/j2ee/j2eeannethomas.pdf. [3] Microsoft Distributed InterNet Architecture White Paper, April 1999, http://www.microsoft.com/dna/about/whitepapers/dnawp.asp [4] SAP Computing Center Management System (CCMS), Release 4, 1998, http://www.sap.com/solutions/technology/pdf/moniarch.pdf. [5] http://www.sun.com/servers/highend/10000/tour/domains.html. [6] http://news.cnet.com/news/0-1003-200-1475663.html.