IBM Clustering Solutions for Unix Servers

Transcription

1 Jane Wright, Ann Katan Product Report 3 December 2003 IBM Clustering Solutions for Unix Servers Summary IBM offers two families of clustering products for the eserver pseries: HACMP for high-availability and pseries Cluster 1600 for simplified management and parallel-processing applications. Table of Contents Overview Analysis Pricing Competitors Strengths Limitations Recommended Gartner Research Insight List Of Tables Table 1: eserver Cluster 1600 Size Table 2: Price List: Unix Clustering Products Table 3: Comparison: Unix Server Clustering Solutions Gartner Reproduction of this publication in any form without prior written permission is forbidden. The information contained herein has been obtained from sources believed to be reliable. Gartner disclaims all warranties as to the accuracy, completeness or adequacy of such information. Gartner shall have no liability for errors, omissions or inadequacies in the information contained herein or for interpretations thereof. The reader assumes sole responsibility for the selection of these materials to achieve its intended results. The opinions expressed herein are subject to change without notice.

2 Corporate Headquarters IBM New Orchard Road Armonk, NY 10504, U.S.A. Tel: Internet: Overview In this report we focus on the latest versions of the individual clustering products for IBM s Unix-based servers. None of the current versions of the products covered in this report run on Advanced Interactive Executive (AIX) v.4.x, although previous versions of many of these products are still offered for sale for usewithaixv.4.xsystems. The eserver pseries Cluster 1600 is made up of two or more servers or partitions within servers that are aggregated together and managed as a single computing resource. Clusters such as the pseries Cluster 1600 can be used for different purposes. Clusters can be used as a single computing resource for high performance computing (HPC). HPC applications include weather forecasting, drug design and engineering analysis. As a single large computing resource, the Cluster 1600 can be used in commercial enterprises for very large-scale business intelligence applications. For large data centers, a pseries Cluster 1600 can be used as a way to manage many different model servers (both pseries and xseries running Linux) from a central point of control. Additionally independently of what applications the computers are being used for, a cluster can be used to provide a high-availability environment for those applications. A cluster member or node can be a complete server or a partition within a server. The nodes in the Cluster 1600 are connected to each other using off-the-shelf, high-speed interconnects or proprietary switches and switch adapters. Each member of the cluster (or node) independently runs its own copy of the operating system. (Thus, a node is simply an operating system instance.) Each node also runs the clustering software that makes the cluster look like a single system. With the pseries Cluster 1600, IBM offers two different cluster programs to accomplish this mission: Parallel Systems Support Program (PSSP) and Cluster Systems Management (CSM). To make a cluster a high-availability cluster, a pseries Cluster 1600 installation also has to run High Availability Clustered Multiprocessing (HACMP) on the nodes that will become the high-availability cluster. HACMP provides a mechanism for using the monitoring functions that are embedded within the operating system to report state and status changes. When an event is reported, HACMP will execute site-specific and application-specific scripts for each application that is affected by the event. In the worst case, if a node changes state from running to halted, HACMP could restart all or just some of the applications that were running on that node on the other nodes in the cluster. (While most high-availability cluster software vendors refer to this as a failover, IBM now calls this a fallover.) In addition to running on a Cluster 1600, HACMP can be run on many existing RS/6000 and pseries models connected to each other using off-the-shelf network hardware. When HACMP is run in a group of privately networked nodes the servers or partitions within a server that all run HACMP are called a cluster. 3 December

3 It is possible for a node to belong to more than one cluster: for example, a PSSP cluster and an HACMP cluster. Furthermore, the two clusters do not necessarily have to have the same members. Thus we can have a cluster that consists of a Cluster 1600 running CSM or PSSP; a Cluster 1600 running either CSM or PSSP and HACMP; or a collection of networked servers running HACMP. With HACMP v.5.1, IBM has combined most of its previous HACMP options into a single product offering. HACMP v.5.1 can only run on AIX v.5.1 or AIX v.5.2. HACMP v.5.1 provides the same functions as were previously found in v.4.5 HACMP Enhanced Scalability with Concurrent Resource Manager and includes a number of new features around performance and security. IBM also offers HACMP/XD (Extended Distance) that uses IBM Enterprise Storage System (ESS) Peerto-Peer Remote Copy (PPRC) or IP-based mirroring for data replication. HACMP/XD automates the management of PPRC. For IP-based mirroring, HACMP/XD has a fully integrated copy of HAGEO 2.4. HAGEO provides data synchronization during production, failover (fallover in IBM parlance), recovery and restoration. Both of the technologies included with HACMP/XD provide the basis for a business continuance solution. pseries Cluster 1600 has continued to add new pseries processors into its building block framework. In addition, IBM introduced a new switch adapter, the first switch and switch adapter that are supported under the pseries Cluster 1600 CSM control program. Prior to the introduction of the pseries High Performance Switch (pseries HPS), the SP and SP2 switches were only supported in a Cluster 1600 running PSSP. Note that the pseries HPS is only available on pseries 655 and pseries 690 models. HPS has a peak bandwidth of 2 GB/sec, unidirectional. The bandwidth for the SP Switch is 150 MB/sec unidirectional, and the bandwidth of the SP Switch2 is 500 MB/sec unidirectional. Analysis HACMP 5.1 Description An HACMP cluster can have up to 32 nodes. Within the cluster, HACMP detects failures and then initiates recovery actions for system, process, network and adapter failures. HACMP manages the failover of applications to another node to provide application availability. Resource Groups are the key to how HACMP manages application availability. Every application (or set of applications) that is required to be highly available is assigned to a resource group. The resource group includes all the components (for example, disk and network adapters) that are required for the application to execute properly. When a resource that an application requires fails, the resource group automatically moves over to another node in the cluster that has access to all the required resources. Each node in the cluster is assigned a priority with respect to a resource group. In general, the node with the next highest priority in the resource chain assumes responsibility for an application in response to a component failure. HACMP offers extremely fine-grained fallback mechanisms that control where the application will execute when a failed resource rejoins the cluster. HACMP v.5.1 supports enhanced concurrent mode that allows concurrent shared access to disk subsystems through a locking application programming interface (API) that applications can use. The concurrent access mode allows a single database image to be accessed by multiple instances of a single application or multiple applications that are distributed across the cluster. The concurrent access works at the raw disk level. Included with HACMP for the first time, is the General Parallel File System (GPFS) that implements concurrent shared-disk access. GPFS is a shared file system optimized for high-performance parallel file transfers. Note that the standard AIX Journaling File System (JFS or JFS2) doesn t support concurrent access from multiple nodes. 3 December

4 Failover recovery time improves with each release of HACMP. HACMP v.5.1 provides fast disk fallover that uses the HACMP enhanced concurrent mode to allow more than one node access to a disk subsystem. With enhanced concurrent mode, HACMP doesn t need to break disk reserves or update and synchronize metadata. As a result disk fallover time is reduced to less than 10 seconds. HACMP offers a comprehensive set of tools to simplify cluster installation and subsequent management, including the System Management Interface Toolkit (SMIT) that gives administrators access to the individual management tools. The Cluster Node Snapshot utility can capture and save the configuration of an established cluster for diagnosing system or cluster configuration problems. The Cluster Single Point of Control (C-SPOC) automatically replicates information to all cluster nodes. In this way, users and groups and their IDs are consistent across the cluster. In addition, storage configuration changes and changes in concurrent volumes (create, import, remove, mirror) are performed once and replicated to the other nodes in the cluster. These utilities have been substantially rewritten for HACMP v.5.1 to provide greater efficiency and a more consistent user interface. HACMP/XD HACMP/XD includes a copy of HAGEO 2.4. It supports cluster configurations where the nodes that comprise the cluster can be located at two widely separated sites. A HACMP/XD cluster is limited to eight nodes. HAGEO 2.4 includes GeoRM v.2.4, which provides real-time data mirroring between geographically distributed systems, either synchronously or asynchronously. With asynchronous mirroring, both local and remote writes begin simultaneously, but control is returned to the application when the local write completes. The data that has not yet been written to the remote disk can only lag the local disk data by a (system defined) number of bytes. When using synchronous mirroring, data is first written to the remote site and then is written to the local site. Control is returned to the application only after both writes are completed. Mirror Write Consistency (MWC) is a variation of synchronous mirroring that starts both local and remote writes simultaneously. Synchronous mirroring can introduce severe impact on application performance depending on the quantity of data that must be mirrored and the speed of the network connections. While asynchronous mirroring reduces some of the performance impact of synchronous mirroring caused by waiting for the remote I/O to complete, it will not necessarily prevent data loss when an outage occurs, since the remote writes may not have completed prior to the failure. The choice of using synchronous or asynchronous mirroring is a trade-off between data integrity vs. enhanced application performance. The choice of which mirroring technique to use can be made on an application-by-application basis. eserver Cluster 1600 The eserver Cluster 1600 is derived from the RS/6000 SP, a system based on a massively parallel processing (MPP) architecture. The PSSP, first developed for the RS/6000 SP, allows the nodes in the SP to be controlled from a single administrative workstation. For use as a technical computing engine, the RS/6000 SP included an extensive software stack that includes: General Parallel File System (GPFS) Parallel Environment for AIX Parallel Engineering Scientific Subroutine Language for AIX 3 December

5 LoadLeveler for AIX 5L These products are now available on AIX v.5.2. A Cluster 1600 running IBM s DB2 Universal Database (UDB), relational database management system provides an excellent platform for deploying large-scale business intelligence (BI) applications. IBM with a five-node cluster running DB2 was one of only two vendors to benchmark its system on the TPC-H decision support benchmark using a 10-GB database. A Cluster 1600 can consist of up to 128 nodes. The overall performance of the cluster is based on server performance as well as the interconnect technology in use. Table 1: eserver Cluster 1600 Size Node Type pseries 670/690 SP Switch SP Switch2 Switchless pseries pseries pseries Partitions per pseries 690 Server Partitions per pseries 670 Server Partitions per pseries 655 Server Partitions per pseries 650 Server Partitions per pseries 630 Server Total Partitions per Cluster The Cluster 1600 requires a separate control workstation when using either of the management options: PSSP or CSM. It is possible to run HACMP within a Cluster 1600 to provide a high-availability cluster solution; however, there may be some functional limitations when running HACMP in a Cluster December

6 Pricing Table 2: Price List: Unix Clustering Products HACMP v. 5.1 HACMP/XD v.5.1 Parallel System Support Programs v.3.5 Cluster System Management Each node must be separately licensed, and the license fee is based on the number of processors in the node. List Price per Processor (US$) (1) 2,855 per processor 5,595 per processor 1,000 to 28,125 per node 350 per processor (1) Varies depending on type of node on which the software is installed. GSA Pricing. Competitors IBM offers a mix-and-match set of cluster offerings on its pseries servers to address many of the issues facing data center managers. Clustering solutions are available that provide the ability to manage a large number of servers from a single management station. A cluster can also be a platform for running highperformance computing applications including business intelligence. Additionally there are cluster offerings that can provide a platform for achieving high-application availability. All of the major Unix vendors have cluster products that run exclusively on their server hardware and operating system that address many of these issues. With some of the vendor offerings, these capabilities that is, clustering for management, clustering for high-performance computing, and clustering for high availability may be available within a single product. This is not he case with IBM s clustering solutions for pseries servers, where different products are required to meet different requirements. IBM s AIX operating system has, for the most part, not been a target for independent software vendors (ISVs) who offer failover clustering products. These heterogeneous high-availability clustering products offer failover and other services identically on multiple platforms including Windows and Linux in addition to a proprietary Unix. Thus for data centers that have servers from many different vendors, running different operating systems, and that also need to improve application availability, a third-party solution that provides similar capabilities with an identical management interface across these heterogeneous systems could prove to be an excellent choice. The only ISV that provides a clustering solution that runs on AIX is VERITAS with the VERITAS Cluster Server (VCS). The clustering product to choose to run on pseries servers depends on the problems to be solved and the data center environment. In terms of a high-availability clustering solution, there are basic features and functions that each solution should be measured against. These basic features and functions include: Whether there are agents or scripts available for the applications that will be run on the cluster to take advantage of the cluster s availability and scalability attributes. Whether or not the cluster presents itself as a single system image. Speed of the cluster interconnect (the faster, the better especially for scenarios such as parallel database clusters or clusters offering concurrent file access). 3 December

7 The ability to reconfigure the hardware and upgrade the software online. Types of load balancing supported by the clustering solution (the more extensive, the better). Number of nodes in the cluster. The number of products that need to be integrated for a solution. The table Comparison: Unix Server Clustering Solutions compares IBM s HACMP and HACMP/XD against the major competitor for a clustering solution on pseries servers, VCS. Table 3: Comparison: Unix Server Clustering Solutions Server Configuration HACMP 5.1 VCS 3.5 Max. No. of Nodes Supported for Failover Supports Failover Between Partitions Node Interconnect Technology (cluster interconnect; private network) Gigabit Ethernet (125 MB/sec), asynchronous 10/100 Ethernet, Gigabit Ethernet transfer mode (ATM), Fiber Distributed Data Interface (FDDI), Fibre Channel, Token Ring, SP Switch, HPS Max. No. of Nodes Supported for Shared Database Access 32 0 Operating System Supported AIX 5L v.5.1; AIX 5L v.5.2 AIX 4.3.3, AIX 5L 5.1, 5.2; HP-UX 11/11i; Solaris 2.6, 7, 8, 9; Windows, Linux Storage Configuration Server/Storage Interconnects Fibre Channel, Small Computer System Interface (SCSI), Serial Storage Architecture (SSA) Storage Vendors Supported IBM, HDS, EMC Many Failover to Nodes With Mirrored Data Concurrent Disk Access Supported (no lock manager) Concurrent Disk Access Supported With Lock Manager Single System Image Fibre Channel, SCSI No Clusterwide File System (same file naming from all nodes) Cluster Alias/Clusterwide Network (connections to individual nodes not required) Clusterwide Ownership of Input/Output (I/O) Devices (device naming the same from all nodes) Cluster Management Server Administration Performed Separately on Each Node (1) (1) No No 3 December

8 Table 3: Comparison: Unix Server Clustering Solutions Server Configuration HACMP 5.1 VCS 3.5 Server Administrative Commands Can Be Replicated to All Cluster Nodes for Execution (2) Cluster Cloning Supported No Single Event Manager Single-Error Log Users Are Authorized Once for All Cluster Nodes Consolidated Management Station Web-Based Management Change Management Add/Remove Nodes Without Bringing Down the Cluster Configuration Changes Applied Once for the Cluster No Perform Rolling Upgrades of Operating System Across the Cluster (cluster remains available during process) Perform Rolling Upgrades of Cluster Software Across the Cluster Perform Rolling Upgrades of Applications Across the Cluster (applications remain available to users during process) Application Failover Application Restart on Failover Application Restart From Checkpoint No No Application Can Be Locked to Specific Nodes (for performance) Applications Automatically Load Balanced Across Cluster (1) General Parallel File System (GPFS) 2.1. Not supported with AIX JFS. (2) Requires PSSP or CSM. Strengths Disaster Tolerance No HACMP/XD offers a fully automated, high-availability, geographically distributed cluster without any a priori distance limitations. Practical distance limitations are dependent on the speed of the network connection and the amount of data that must be replicated. Improved System Management HACMP offers tools that allow certain types of administrative actions to be replicated across the nodes in a cluster. HACMP also offers simplified installations with the ability to save cluster configurations and clone them. The Cluster 1600 provides a management view and interface that makes the multiple independent nodes appear as a single system. Rapid Error Detection and Notification 3 December

9 HACMP uses the Reliable Scalable Cluster Technology (RSCT) technology layer embedded within AIX 5L for rapid event detection and notification. Limitations Product Overlaps There are significant overlaps in management capabilities when using HACMP on a pseries Cluster 1600 that uses PSSP or CSM. HACMP provides tools to simplify cluster management for high-availability or parallel database clusters. PSSP and CSM offer a management control workstation that allows a group of nodes connected through a private LAN to be managed as a single unit. Of course, customers who are installing a cluster only for high availability or customers who are turning to clusters solely for improved manageability may prefer to buy one product to meet their requirements, rather than buying more than they need. But we believe that most customers want their clusters to provide high availability and single system image manageability all at once. Relatively High Cost Having to use and integrate more than one product, such as HACMP and a Cluster 1600, can prove costly in both initial expenditures as well as ongoing support. Limited Backward Compatibility The latest versions of the various products support only those servers running AIX 5L. Many of the older servers, particularly Cluster 1600 eservers that were formerly RS/6000 SP systems, ran AIX 4.X. The upgrade to AIX 5L requires 64-bit applications to be recompiled. Users with older systems may not want to make this major software upgrade. Recommended Gartner Research Veritas Cluster Server for Unix Environments, DPRO Insight IBM has two complementary clustering systems in its product line: HACMP and pseries Cluster HACMP was developed to reduce the number and duration of outages for commercial applications where high availability in the form of a failover solution is often required. HACMP provides clustering in the traditional Unix sense, where it has come to be synonymous with high availability and failover. HACMP is a mature product that has undergone several product updates and has achieved wide commercial acceptance. With the fine grain control available for resource groups, HACMP is capable of providing near continuous availability for applications including disaster recovery. Cluster 1600, on the other hand, is an outgrowth of the RS/6000 SP, which was a system designed to exploit parallelism in large scale, computationally intense applications, including business intelligence. In an RS/6000 SP, PSSP was the basic management facility that coordinated the execution of the decomposed parts of the application. PSSP has been unbundled to support both switch-based and nonswitch-based Cluster 1600 systems. CSM originally provided a common centralized management interface for Linux clusters running on IBM s xseries servers as well as non-switch-based pseries running AIX. Furthermore, CSM running on a pseries server running AIX can now be used to manage a cluster that includes both pseries and xseries servers. With the introduction of the new pseries High Performance Switch, CSM can now be used to 3 December

10 manage a high-performance business application, such as business intelligence. With this evolution, IBM has taken the opportunity to bring massively parallel processing (mpp) technology to a wider audience. While it is possible to use Cluster 1600 configurations with HACMP, the interactions of these two products need to be considered prior to deploying this type of solution. 3 December