Introduction to IBM System Storage SVC 2145-DH8 and IBM Storwize V7000 model 524 Guide v1.0 Bhushan Gavankar, Sarvesh S. Patel IBM Systems and Technology Group June 2014 Copyright IBM Corporation, 2014
Table of contents Abstract...2 Key highlights common to both platforms...2 Fast, scalable, and flexible... 2 Initial setup... 2 New quad port converged network adapter... 3 Quick assist cards... 3 Additional new features in SVC 2145-DH8...4 A scalable 2U DH8 node... 4 Scalable write cache and higher node reliability... 5 Increase in MBps performance... 5 SSD support... 6 Additional new features in Storwize V7000 Gen 2...6 Scalable write cache...7 Increase in performance... 7 Support for 20 expansion enclosures... 7 Summary...7 Resources...8 About the authors...8 Trademarks and special notices...9 1
Abstract This paper provides a brief overview of the new features and technologies used in the IBM System Storage SAN Volume Controller (SVC) and IBM Storwize V7000 platforms, which make them more scalable and flexible, and provide better performance. The IBM System Storage group is the refreshing hardware for the SVC and Storwize V7000 platforms. At the time of publishing this paper, for SVC, a new SVC model 2145-DH8 and for Storwize V7000, a new model 524 have been released. The machine type for Storwize V7000 refresh remains the same, that is, 2076; however, the model gets changed to 524. For simplicity, SVC 2145-DH8 is referred as DH8 and the new Storwize V7000 model 524 is referred as Gen 2 in this paper. Key highlights common to both platforms While IBM System Storage SAN Volume Controller (SVC) and IBM Storwize platforms cater to different requirement space, there are some commonalities found in DH8 and Gen 2 platforms, as the key design objective for both these platforms is to increase scalability, flexibility, and performance. Fast, scalable, and flexible To meet the objectives, base hardware configuration of both the platforms got beefed-up to support more advanced processors, more memory and faster interconnects. This is also first time when a SVC platform and a Storwize platform will share the same processors. Below points provide details, which should help understand the changes made across both the platforms in order to help us meet the goals. Processors: Both the platforms make use of the Ivy Bridge processors from Intel, which has 8 cores. In case of DH8, users have an option to insert a second 8-core Ivy Bridge processor; which can be dedicated for running IBM Real-time Compression workloads. Memory: 32 GB of memory for cache with an option to have another 32 GB is added for Real-time Compression workloads. PCIe technology: Both the platforms have multiple PCIe Gen3 slots, as compared to dual PCIe Gen 2 slots in previous versions. This shift to PCIe Gen3 allows each PCIe lane to get a maximum speed of 1000 MBps. Optional adapters: In previous models, the only option that customers had was to add an additional dual port 10Gbps converged network adapter. The base model has dual ports 1Gbps adapter, plus quad port 8Gbps Fibre Channel (FC) adapter. In both of the new platforms, the base models come with three 1Gbps Ethernet ports onboard. However, customers have an option to select multiple add-on adapters for driving host I/O and offloading compression workloads. Note 1: There is a plan to support 16Gbps FC ports on both these platforms in future releases. Initial setup SVC and Storwize platforms have been used by IBM customers in various environments and hence there was a need to have wide range for the initial setup of the equipment. 2
SVC has a front panel with options for customers to configure the management IP address and tell the cluster to get started. Then do EZSetup. Storwize platforms use a combination of InitTool (USB key) and EZSetup to configure a storage box for first use. Customer feedback was generally good, however, a need was felt to make further improvements to address few challenges seen in data centers. IT policies at a few companies place restriction to carry a USB key inside the data centers. Lack of visual interface to guide during initial setup Inconsistency between products for performing the initial setup The solution provided in these platforms is to use a technician port. The fourth 1Gbps Ethernet port is labeled as the T port, (that is, the technician port) and will not be available for normal host data traffic. Instead, it will run a DHCP and DNS service, so that when users connect their notebook to this T port through an RJ45 cable, an IP address will be issued. And, irrespective of the web query made by the user, the user would be directed to the initial setup or service log page. Users would be in position to perform basic node service management tasks through the web page provided by the technician port. New quad port converged network adapter In addition to the onboard three 1Gbps Ethernet ports and optional quad port 8Gbps FC adapters, both the platforms optionally support a new converged network adapter that have four ports running at 10Gbps Ethernet. Customers have an option of running FCoE and 10Gb iscsi protocols on the new adapter. The new converged network adapter doubles the available bandwidth as compared to the previous models such as SVC 2145-CG8 and Storwize V7000. Quick assist cards Adding to the list of first, both these platforms support Intel-based quick assist cards that can be used for offloading Real-time Compression workloads. While the DH8 platform can optionally support adding two quick assist cards, Storwize V7000 Gen 2 can support adding only one quick assist card. Note: Storwize V7000 Gen 2 has one quick assist card built on the system board itself. 3
Additional new features in SVC 2145-DH8 This section describes about the new hardware features introduced in the SVC 2145-DH8 model. This section also provides information regarding new batteries, SSDs, and other features. A scalable 2U DH8 node Figure 1: Front view of SVC 2145-DH8 Figure 2: Rear view of SVC 2145-DH8 4
All previous SVC models, for example: 2145-8G4, 2145-CF8, and 2145-CG8 are 1U platforms. They require additional 1U of rack space for mounting uninterruptible power supply (UPS). Needless to mention, this increases the cabling requirements to connect the UPS with the nodes. In many data centers, IT managers preferred to provide additional 1U of space to ensure that the cabling is rightly done. With DH8, the battery module has been integrated in the 2U node. Thus, the total size used up in the data center is still the same. However, there are a couple of benefits by doubling the height of each node. Integrating the battery inside the node eliminates the need for external cabling for connecting UPS. In addition, the preferences of racking might save on rack space. Increasing the size of an SVC node gives us more space inside the node. In earlier models, there was space only to have two cards inside each node. However, a 2U DH8 node allows for up to six PCIe Gen3 slots, giving users the flexibility to support more adapters. (Note: Optional second processor is required in order to populate cards in the second PCIe riser card.) Scalable write cache and higher node reliability While 2145-CG8 allowed only 24 GB of cache in each node, DH8 allows for 32 GB memory for cache and an optional 32 GB upgrade for use with Real-time Compression. Previous SVC nodes had one internal boot drive, while DH8 has two boot drives with full installation of SVC code. To support a larger write cache, DH8 writes data from volatile memory across both the drives, in event of AC power loss, effectively doubling the rate at which data is written to disk. Dual boot drives can also help increase reliability while doing a code upgrade on any DH8 node. During upgrade when a node shuts down, the hardened data will be written to both the internal drives, so that the node can survive even if one of the internal drives fails. Increase in MBps performance SVC 2145-CG8 model came with standard dual port 1Gbps Ethernet onboard adapter and quad port 8Gbps FC adapter. It allowed customers to add optional dual port 10Gbps converged network adapter for running FCoE and 10G iscsi traffic. (The FCoE protocol is supported for SVC clustering and virtualizing backend storage). So, effectively with the optional card in place, customers would get 2 Gbps pipe from 1G Ethernet ports + 32 Gbps of pipe from FC adapter and additional 20 Gbps of pipe from converged network adapter. DH8 comes with standard, three 1 Gbps Ethernet onboard ports (fourth 1 Gbps Ethernet port is dedicated as technician port). However, due to the higher form factor (2U), it allows inserting six cards per node. The system board has six PCIe Gen3 slots, out of which three slots can be used to install either quad port 8 Gbps FC adapters or quad port 10 Gbps converged network adapters. This effectively increases the available pipe to 96 Gbps when using the 8 Gbps FC adapter; while in case of 10 Gbps converged network adapter, the bandwidth increases to 40 Gbps when using one quad port CNA. The capability to support multiple cards allows DH8 to dramatically increase the I/O bandwidth and virtualize IBM Flash System more efficiently. It also gives the flexibility to storage administrators for dedicating few I/O ports for SVC clustering plus backend storage to SVC traffic, while reserving other I/O 5
ports for host I/O. If done appropriately, this increases the write MBps performance, as seen by applications. Note: Non-disruptive node upgrade from CG8 to DH8 in an iscsi SAN (only), requires storage administrator to take extra precaution. This is due to the difference in the number of 1Gbps Ethernet ports available on both platforms. CG8 has two 1Gbps and two 10Gbps Ethernet interfaces per node. DH8 has three 1Gbps and four 10Gbps Ethernet interfaces per node (if optional quad port converged network adapter is inserted in DH8). During upgrade from CG8 to DH8 node, the IP address assigned for the first and second 10Gbps Ethernet interfaces on CG8 gets assigned to the third 1Gbps Ethernet interface and the first 10Gbps Ethernet interface of the DH8 node respectively. This might cause performance drops and temporary disruptions. Refer to IBM information center documentation for detailed steps to upgrade from a running CG8 cluster to DH8 nodes in an iscsi environment. SSD support In 2145-CG8 model, there was an option to add four drives (SSDs) in to each CG8 node, allowing a total of eight SSDs. These SSDs were accessible only to the node in which they were inserted. With DH8, user can add one 12 Gbps serial-attached SCSI (SAS) interface card to directly attach up to 48 SSDs per node pair (I/O group). These SSDs are provided in external enclosures and dual ported to allow connections to both the nodes. Additional new features in Storwize V7000 Gen 2 Storwize V7000 Gen 2 is a 2U control enclosure, the same form factor as existing Storwize V7000 control enclosure. However, each canister in Gen 2 now occupies a full 2U inside the 2U chassis. This increase in height allows users to have three PCIe Gen 3 slots along with having the same Intel Ivy Bridge processor found in SVC 2145-DH8 nodes. Figure 3: Back view of Storwize V7000 Gen 2 6
Scalable write cache In Storwize V7000 Gen 1, write cache was limited to 8GB. However, Gen 2 comes with a 32GB cache and with an optional upgrade of up to 64GB in which case, the additional 32GB is available for Real-time Compression workloads. Increase in performance Storwize V7000 Gen 1 came with standard quad port 1Gbps Ethernet onboard adapter and quad port 8Gbps FC adapter. It allowed customers to add optional dual port 10Gbps converged network adapter for running FCoE and 10G iscsi traffic. So, with the optional card in place, customers would get a 4 Gbps pipe from 1Gbps Ethernet ports + 32Gbps of pipe from FC adapter and additional 20 Gbps of pipe from the converged network adapter. Gen 2 comes with standard, three 1 Gbps Ethernet onboard ports (the fourth 1 Gbps Ethernet port is dedicated as a technician port). However, due to the higher form factor (2U), it allows inserting three cards per canister. The system board has three PCIe Gen3 slots, out of which two slots can be used to install either quad port 8 Gbps FC adapters or one quad port 10 Gbps converged network adapter. This effectively increases the available pipe to 64 Gbps when using the same 8 Gbps FC adapter; while in case of 10Gbps converged network adapter, the pipe increases to 40 Gbps. The increase in the performance of input/output operations per second (IOPS) is due to the two times increase in the disks that can be attached behind a Gen 2 control enclosure. Support for 20 expansion enclosures Storwize V7000 Gen 2 control enclosure can attach more than two times of the disk possible with Gen 1. Gen 1 supported having a maximum of nine expansion enclosures, each with 24 Small Form Factor (SFF) drives. This allows for a maximum of 240 drives per controller. Gen 2 supports having a maximum of 20 expansion enclosures allowing a maximum of 504 drives per controller. The system can cluster with up to four control enclosures, which enables a big storage space provided by around 84 enclosures (80 expansion enclosures plus 4 control enclosures). Summary The design of SVC 2145-DH8 and Storwize V7000 Gen 2 is geared towards making the platforms more scalable, flexible, and give higher performance without consuming more space in customer s data centers. As seen by the details provided in this paper, these platforms can allow storage administrators to provide more bandwidth for applications by adding more I/O adapters, and benefit high compression workloads by adding an optional second processor (in DH8), memory, and quick assist card. Enhancements made in these platforms provide an important foundation to add support for 16Gbps FC ports and virtualize IBM Flash Systems in the upcoming SVC releases. 7
Resources The following websites provide useful references to supplement the information contained in this paper: IBM Systems on IBM PartnerWorld ibm.com/partnerworld/systems/ IBM Publications Center http://www.elink.ibmlink.ibm.com/public/applications/publications/cgibin/pbi.cgi?cty=us IBM Redbooks ibm.com/redbooks IBM developerworks ibm.com/developerworks About the authors Sarvesh S Patel is a staff software engineer in IBM Systems and Technology Group SVC and Storwize family. He has 6 years of experience in storage test. He was a functional test lead for qualifying 10G iscsi and FCoE on IBM quad port adapter on the new platforms. You can reach Sarvesh at sarvpate@in.ibm.com. Bhushan Gavankar is a test architect in IBM Systems and Technology Group SVC and Storwize family with primary focus around performance testing across iscsi and FCoE protocols. He has around 15 years of experience in networking, systems, high availability, and storage domains. You can reach Bhushan at bgavanka@in.ibm.com. 8
Trademarks and special notices Copyright IBM Corporation 2014. References in this document to IBM products or services do not imply that IBM intends to make them available in every country. IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol ( or ), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at www.ibm.com/legal/copytrade.shtml. Intel, Intel Inside (logos), MMX, and Pentium are trademarks of Intel Corporation in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others. Information is provided "AS IS" without warranty of any kind. All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer. Information concerning non-ibm products was obtained from a supplier of these products, published announcement material, or other publicly available sources and does not constitute an endorsement of such products by IBM. Sources for non-ibm list prices and performance numbers are taken from publicly available information, including vendor announcements and vendor worldwide homepages. IBM has not tested these products and cannot confirm the accuracy of performance, capability, or any other claims related to non-ibm products. Questions on the capability of non-ibm products should be addressed to the supplier of those products. All statements regarding IBM future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. Contact your local IBM office or IBM authorized reseller for the full text of the specific Statement of Direction. Some information addresses anticipated future capabilities. Such information is not intended as a definitive statement of a commitment to specific levels of performance, function or delivery schedules with respect to any future products. Such commitments are only made in IBM product announcements. The information is presented here to communicate IBM's current investment and development activities as a good faith effort to help with our customers' future planning. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput or performance improvements equivalent to the ratios stated here. 9
Photographs shown are of engineering prototypes. Changes may be incorporated in production models. Any references in this information to non-ibm websites are provided for convenience only and do not in any manner serve as an endorsement of those websites. The materials at those websites are not part of the materials for this IBM product and use of those websites is at your own risk. 10