IBM Storage Tier Advisor Tool with IBM Easy Tier

Transcription

1 IBM Storage Tier Advisor Tool with IBM Easy Tier Samrat Dutta, Shrikant V Karve IBM Systems and Technology Group ISV Enablement July 2014 Copyright IBM Corporation, 2014

2 Table of contents Abstract...1 Getting started...1 About IBM Storage Tier Advisor Tool... 1 About this paper... 1 Assumptions... 2 What is new in Storage Tier Advisor Tool Download and setup...4 Prediction and output recommendation result...5 Products and browsers supported by Storage Tier Advisor Tool The HTML report...7 Pool recommendations...9 Resources...10 About the authors...10 Trademarks and special notices...11

3 Abstract In this white paper, the procedure to use the code release in IBM System Storage SAN Volume Controller (SVC) version is described. The white paper gives information regarding the usage of Storage Tier Advisor Tool and how it predicts and provides various levels of recommendation configuration. The white paper also gives a brief introduction to the Storage Tier Advisor Tool and provides guidance on how to install, upgrade, and navigate through the recommendations provided by it. It also gives a brief description about what is new in this release of Storage Tier Advisor Tool and describes a complete page-wise walk-through of the report generated by Storage Tier Advisor Tool. Getting started This section gives a brief idea about the IBM Storage Tier Advisor Tool and its integration with the IBM Easy Tier function. About IBM Storage Tier Advisor Tool IBM Storage Tier Advisor Tool is a stand-alone tool that predicts the addition of solid-state drive (SSD), enterprise drive, and nearline drive capacity in conjunction with the Easy Tier function. Storage Tier Advisor Tool is a Microsoft Windows console application that analyzes the heat data files produced by Easy Tier and produces a graphical display of the amount of hot data per volume and predictions of how SSD, enterprise drive, and nearline drive capacity can benefit performance of the system and each storage pool. The main advantage of the Storage Tier Advisor Tool is that it generates recommendation configurations according to the heat data file so that customers can have a clear view of their system configuration and current system status and benefit from the recommendations provided. The heat data files are found under the /dumps folder on the configuration node, named dpa_heat.node_name.time_stamp.data, and are provided as input files to the Storage Tier Advisor Tool executable, which then generates the recommendations in the form of an easily readable HTML report. The heat data files are produced approximately once per day when Easy Tier is active on one or more storage pools and summarize the activity per volume since the prior heat data file was produced. Any existing heat data file is erased if available for more than 7 days. The heat file generated by SVC version contains all the necessary information that is read by the Storage Tier Advisor Tool for prediction and generates the recommendation according to these intermediate prediction data. The Storage Tier Advisor Tool has wide support of all the new SVC version Easy Tier migration features such as auto-rebalance, performance improvement, IOPS improvement, bandwidth improvement, capacity improvement, and so on. About this paper The purpose of this paper is to provide a thorough understanding of how the Storage Tier Advisor Tool works and how it provides recommendation configurations. This paper provides a self-guided, hands-on evaluation of the Storage Tier Advisor Tool for the storage administrators and IT professionals and guides them through the different kinds of recommendations provided by the tool and how it can be used to 1

4 improve the performance of their storage systems. The paper also attempts to demonstrate how to get started with the tool and show installation and upgrade procedures. This paper is intended to provide an overview of the steps required to successfully use and benefit from the recommendations provided by the Storage Tier Advisor Tool. This is not meant to be a substitute for product documentation and encourages users to refer to the product documentation, information center, and the command-line interface (CLI) guide of the Storage Tier Advisor Tool for more details. Assumptions The following assumptions are considered while writing this white paper. The SVC clusters are successfully installed with the Easy Tier function enabled (at the time of this publication) The SVC nodes should have IBM SVC code levels (or above) for generating the heat data files of the Easy Tier 3 function. The storage area network (SAN) is configured as per the product documentation and Easy Tier function is enabled and the infrastructure is ready (Windows platform) to support the Storage Tier Advisor Tool installation and viewing the results. The user has the basic understanding and awareness of Storage Tier Advisor Tool SVC terminology STAT SAN NAS SSD ENT IOPS BW NL SVC or IBM SAN Volume Controller Table 1: Terminology and abbreviation Brief description IBM Storage Tier Advisor Tool Storage area network Network-attached storage Solid-state drive Enterprise drive Input/Output operations per second Bandwidth Nearline drive Unless explicitly specified, a general term that is used to describe all applicable SVC platforms IBM SVC (CG8, CF8, 8G4), IBM Storwize V7000, Storwize V5000, Storwize V3700 and IBM PureFlex System storage nodes. What is new in Storage Tier Advisor Tool IBM Storage Tier Advisor Tool version was previously released and supported for IBM System Storage DS8000 and after its successful usage by its customers, Storage Tier Advisor Tool is now released with support for SVC Easy Tier and also for the Storwize family of products. The feature enhancements in this release include: Better readability and result display Storage Tier Advisor Tool provides an illustrated blueprint of the customer s configuration and allies it with an eloquent and comprehending picture of the recommendations, thus supplying the benefits brought by it that can be availed by the customer. In the previous version of Storage Tier Advisor Tool, the report was segregated into various links, each of which specifically addressed a part of the report, that is, the system summary, volume 2

5 heat distribution and all similar features were disjointed and could be navigated through the main menu link. In version of Storage Tier Advisor Tool, there is function oriented nexus between the links. The details are clustered, based on system summary and system-wide recommendations related to each storage pool has a dedicated page that contains the workload on each tier of the storage pool, skew curve of the workload distribution, tier-wise recommendation, and volume heat distribution. This gives a better visibility to the customer where each specific page has a particular significance, for example, the user can find everything related to storage pool performance and recommendation improvements at one place, thus reducing the customer's pain to move around and search for information throughout the report. At several places, colored notations are used to give the user a better visibility of percentage completion of capacity for any tier, pool, or volume. The details of each page are provided later in the paper. Any three-tier combination support In versions before SVC 7.3.0, Easy Tier supported only two tiers in the form of generic_ssd and generic_hdd. In SVC release, the Easy Tier function can support any three-tier combination thus dividing the workload efficiently between high-speed and cost-effectiveness in the form of any three-tier combination: SSD, enterprise, and nearline. Because of the Easy Tier support of three tiers in this release, the Storage Tier Advisor Tool can also support any three-tier combination. Because Storage Tier Advisor Tool is an advisory tool for storage pool performance and providing recommendations, it articulates the storage pool details in the storage pool performance and improvement recommendation page of the report, where it provides the average utilization of the managed disk (MDisk) IOPS for each tier of the storage pool. For example, if a storage pool is created by tiers: SSD, enterprise, and nearline, then the storage pool performance and improvement recommendation page displays the average utilization of the MDisk IOPS for each tier in a tabular form, also showing the projected utilization of MDisk IOPS for each MDisk that together constitute that particular tier of that storage pool. It also maintains compatibility with earlier versions of Storage Tier Advisor Tool. Multi-granularity recommendation improvement Storage Tier Advisor Tool provides multi-granularity improvement recommendation in different sectors including auto-rebalance, performance, IOPS, bandwidth, and the capacity to support Easy Tier multiple data migration policies. The recommendations provided by STAT give customers a better perception of their current configuration so that customers can use these recommendations to improve their systems. Multi-dimensional useful performance data export This STAT release comes with extensive features for multi-dimensional useful performance data export. It provides a complete workload distribution across tiers. It displays the skew of the workload and projected workload distribution on the top-tier secondary storage that displays the projected skew of the workload on the secondary storage device. It also provides a complete dump of the workload categorization that divides extents into five categories according to their data activity. And finally it displays the data movement that summarizes all the Easy Tier movements' statistics. This data 3

6 gives customers a deep insight of their workload and data movement through Easy Tier and can be used for better understanding of their configuration. Download and setup You can download Storage Tier Advisor Tool directly from: ibm.com/support/docview.wss?uid=ssg1s and install it on a Microsoft Windows 7 computer. The tool comes packaged as an ISO file, which needs to be extracted to a temporary location as shown in Figure 1. It contains the installer, located at temporary_location\images\stat\disk1\instdata\novm\. Figure 1: Storage Tier Advisor Tool ISO file When the installer starts, it asks for few basic questions about the installation directory of the Storage Tier Advisor Tool, but by default, it is installed in the IBM/STAT folder within the Program Files folder of the system s directory, likely C: drive, that is, at C:\Program Files\IBM\STAT\. The user can also specify the output directories. If a previous version of the Storage Tier Advisor Tool is already installed, then this installer automatically detects it and prompts for uninstalling the previous version first. After uninstalling, it continues with the rest of the installation and will be placed in the same location. The installer must be invoked from a Windows command prompt console with the file specified as a parameter. The user can also specify the output directory. The typical usage of the Storage Tier Advisor Tool is as follows: STAT.exe [-h] [-o output_path] input_filename The Storage Tier Advisor Tool creates a set of HTML files and the user can then open the resulting index.html file in the browser to view the results. A typical generation of the output files is as follows: Figure 2: Output from Storage Tier Advisor Tool The index.html file is the entrance point of the complete report generated by the Storage Tier Advisor Tool. The Data_files folder contains a list of files for various purposes which together integrate to form the HTML report. The following figure shows the list of files stored in the Data_files folder. 4

7 Figure 3: List of files in the Data_files folder The user can directly use index.html for accessing the complete report generated by the Storage Tier Advisor Tool. However, if the user wants to specifically navigate to individual pages of the report, then System Summary.html can be directly double-clicked to view the system summary report and Systemwide recommendation.html can be directly double-clicked to view the system-wide recommendation. The data movement, skew curve, and workload category data that can be viewed in the report are also accessible to the customer using the three CSV files in the Data_files folder. The data is already exported in these files for offline use by the customer. Prediction and output recommendation result The prediction generated by the Storage Tier Advisor Tool is primarily pool-based and is run based on several factors. Auto-rebalance recommendation: Given that one tier is IOPS/bandwidth overloaded, the Storage Tier Advisor Tool suggests adding new MDisks to expand this tier so that it is immune to be overloaded. Because IOPS/bandwidth overload can create a bottleneck at the customers' site for performance, auto-rebalance recommendations give a prior penetration into the system state when it is detected and thus can be improved by adding new MDisks so that after the tier is dilated, it will be easily resistant of any added overload, thus reducing coercion created on the tier when the IOPS/bandwidth is overloaded. Performance improvement recommendation: Mainly focusing on response time reduction by full utilization of the available MDisks space or by adding new MDisks, the Storage Tier Advisor Tool calculates how much data must be migrated and how many MDisks must be added to gain such performance improvement. New SSD is expected to be added for this kind of recommendation. Based on the current system configuration, there is always a tradeoff between high-speed and cost-efficiency and when it is detected that response time is taking a hit, it aims at 5

8 abridging the response time by trying to employ the uncommitted MDisks' space or by adroitly augmenting new MDisks. IOPS improvement recommendation: mainly focuses on IOPS improvement by full utilization of the available MDisks space or adding new MDisks. The Storage Tier Advisor Tool will calculate how much data should be migrated and how many MDisks should be added to gain such IOPS improvement. New enterprise drive is expected to be added for this kind of recommendation. As enterprise drives are best utilized because of its capability of both good speed and higher availability, when IOPS are taking a toll, it is expected to wax enterprise drives for such scenarios for gaining IOPS melioration. Bandwidth improvement recommendation: mainly focuses on bandwidth improvement by full utilization of the available MDisks space or adding new MDisks. The Storage Tier Advisor Tool will calculate how much data should be migrated and how many MDisks should be added to gain such bandwidth improvement. New enterprise or nearline drive is expected to be added for this kind of recommendation. Nearline drives can be best used for archival data that is seldom used. The Storage Tier Advisor Tool is intelligent enough to formulate the need for the current system configuration and when it detects that a bandwidth amendment is required, it calculates the amount of data migration needed and how much MDisk enhancement can suffice the bandwidth improvement. Based on the computation, it furnishes the drive types that can gratify the bandwidth requirement and mostly recommends for addition of new enterprise or nearline drives for this kind of recommendation. Capacity improvement recommendation: mainly focuses on capacity expansion by full utilization of the available MDisks space or adding new MDisks. The Storage Tier Advisor Tool will calculate how much data should be migrated and how many MDisks should be added to gain such capacity improvement. New nearline drive is expected to be added for this kind of recommendation. Based on the needs of the current system configuration, capacity improvement requirements can often occur and it is mostly a tradeoff between what kind of capacity improvement is required for what kind of scenarios. As a capacity bottleneck mostly occurs at customers' site due to storage of high volumes of data and because SSD or enterprise is mostly used for high-speed data accessibility, when humongous mass of data is considered, nearline drives are the best options for customers since it gives customers high value for storage with lower costs. Hence Storage Tier Advisor Tool mostly recommends the addition of nearline drives for capacity improvement recommendations. The Storage Tier Advisor Tool recommends only arrays and for a given type of array, it specifies how many arrays can be added to gain the expected performance improvement. Drives and external MDisks are filtered out from recommendation. The output recommendation result is divided into pool level and system level. For pool-level display, each pool's recommendation information is provided in the output and for system-level recommendation, Storage Tier Advisor Tool summarizes all the pool's recommendation information and divides the summary recommendation into at most three sets according to the recommendation configuration: SSD, enterprise, nearline. 6

9 Products and browsers supported by Storage Tier Advisor Tool Storage Tier Advisor Tool supports heat data files produced by Easy Tier on SAN Volume Controller 6.x, 7.1, 7.2, 7.3, Storwize V x, 7.1, 7.2, 7.3, Storwize V x, 7.1, 7.2, 7.3 and Storwize V3x00 6.x, 7.1, 7.2, 7.3. The process for using Storage Tier Advisor Tool is same on all the products, with the heat data file generated under the /dumps folder in the configuration node and then using it to generate an HTML report of the complete I/O distribution and provide recommendation configurations. Because Storage Tier Advisor Tool generates a HTML report, it is supported by Internet Explorer 10, Mozilla Firefox 27 and Mozilla Firefox ESR 24 and Google Chrome 33. The HTML report When Storage Tier Advisor Tool is supplied with the dpa_heat file, it generates an HTML report with the entrance point as index.html. The first page of the report is the summary of the system that is being monitored showing the total number of pools monitored, total volumes monitored, total capacity monitored, hot data capacity, data validity, and system state. It is followed by a complete list of storage pools that were monitored showing capacity, configuration, tier status, and data management status for each pool. The data management status displayed on the Summary Report page shows how data is managed in the particular extent pool and is denoted by various color nodes depicting specific significance. The total capacity of the storage pool is divided into two broad parts: data that is managed by Easy Tier and data that is unallocated. A typical data management status will be displayed as shown in the following figure. Figure 4: Data management status The different color combinations denote the following data. The green portion of the bar represents data that is managed by Easy Tier. The black portion of the bar represents unallocated data, that is, data that is not managed by Easy Tier. Each portion of the bar displays both capacity and I/O percentage of the extent pool (except that the black portion of the bar only displays the capacity of the unallocated data) by following the "Capacity/IO Percentage" format. (*) The dark and light purple portion of the data management status is specific for DS8000. When any of the storage pools are clicked, it navigates to the performance statistics and improvement recommendation of that specific pool. This page portrays a vivid distribution of each tier that constructed the pool displaying the MDisks, number of IOPS threshold, utilization of MDisk IOPS, and projected utilization of MDisk IOPS for each MDisk of each tier. There is also a threshold set for maximum allowed IOPS. The utilization of the MDisk IOPS is the current MDisk IOPS, calculated as a moving average, as a percentage of the maximum allowed IOPS threshold for the device type (such as SATA and SSD) and the 7

10 projected utilization of MDisk IOPS is the expected MDisk IOPS, calculated as a moving average, after rebalancing operations have been completed, as a percentage of the maximum allowed IOPS threshold for the device type (such as SATA and SSD). Both the utilization of the MDisks IOPS and the projected utilization of MDisk IOPS are shown using a color bar denoting the percentage of MDisk IOPS utilization in comparison to the average utilization of MDisks IOPS. A typical bar depicting the utilization of MDisks IOPS can be like this: Figure 5: Bar displaying utilization of MDisk IOPS as compared to the average utilization of MDisk IOPS The color codes provide the following representation: The blue portion of the bar represents the percentage range which is below the tier average utilization of MDisk IOPS. The orange portion of the bar represents the percentage range which is above the tier average utilization of MDisk IOPS but below the maximum allowed IOPS threshold. The red portion of the bar represents percentage range which is above the maximum allowed IOPS threshold. This is followed by the workload distribution curve across tiers, followed by the recommended configuration for each tier combination based on the storage pool. These results contain five kinds of recommendations: Recommended SSD Configuration, Recommended ENT Configuration, Recommended SATA Configuration, Recommended NL Configuration and Recommended SSD + ENT Configuration. For each kind of recommendation, the recommendation result will be listed in a table format, which contains recommendation title, selection list, table head, and table content. Finally, the page concludes with the volume heat distribution of that storage pool which shows the VDisk ID, configuration size, I/O percentage of extent pool, tier, capacity on tier, and heat distribution for each VDisk of that storage pool. The heat distribution for each VDisk is displayed using a color bar which represents the type of data on that volume. A typical heat distribution bar looks as shown in the following figure. Figure 6: Heat distribution bar showing type of data in a volume The color codes provide the following representation: The blue portion of the bar represents the capacity of cold data in the volume. Data is considered cold when it is either not used heavily or the I/O per second on that data has been very less. The orange portion of the bar represents the capacity of warm data in the volume. Data is considered warm when it is relatively heavily used than data that is cold or the IOPS on that data is relatively more than that on the data that is cold. The red portion of the bar represents the capacity of hot data in the volume. Data is considered hot when it is used most heavily or the IOPS on that data has been highest. The complete system-wide recommendation can be viewed on another page which shows the recommended configuration for the tiers as applicable for the configuration of the system. At most, it shows three recommended configurations: SSD, enterprise, and nearline. Each recommendation displays 8

11 the storage pools, recommendation, and total improvement. At most, six storage pools can be displayed for each tier recommendation. Pool recommendations The pool recommendation is mainly decided by the workload status for specific tiers. There can be three specific scenarios for each tier. IOPS overload: This denotes that there is at least one MDisk in the tier for which the IOPS exceeds the IOPS threshold set for that tier. Bandwidth overload: This denotes that there is at least one MDisk in the tier for which the bandwidth exceeds the bandwidth threshold set for that tier. IOPS skewed: This scenario occurs when the top 20% extents take more than 80% of IOPS, and in such a case STAT suggests to move these 20% extents to tier 0 and the remaining hot data to tier 1 to get a combined tiers configuration recommendation. As a result, for example, when it is displayed that the enterprise tier is IOPS/bandwidth overloaded, it means that there is at least one MDisk in the tier for which the IOPS or the bandwidth exceeds the IOPS or bandwidth threshold respectively. When it shows that the enterprise tier is IOPS skewed, it means that the standard deviation of the current IOPS utilization of the MDisks in the tier exceeds a pre-defined threshold (10% of the average IOPS utilization across MDisks in the tier). With Easy Tier supporting 3-tier combination in this release, there can be several combinations of tiers for a storage pool such as SSD, ENT, NL, SSD + ENT, SSD + NL, ENT + NL, SSD + ENT + NL. For all the combinations of tiers, there can be several scenarios where the IOPS is overloaded or the bandwidth is overloaded or the IOPS is skewed and the recommendation is provided based on the configuration of the system at the time when the heat data file was analyzed by Storage Tier Advisor Tool. 9

12 Resources The following websites provide useful references to supplement the information contained in this paper: IBM Storage Tier Advisor Tool ibm.com/support/docview.wss?uid=ssg1s IBM Publications Center IBM Redbooks ibm.com/redbooks IBM developerworks ibm.com/developerworks About the authors Shrikant Karve is a test architect for SAN Volume Controller in IBM Systems and Technology Group SVC and Storwize family. He is a veteran in storage with extensive knowledge in both SAN and NAS. As part of the IBM Storage Tier Advisor Tool, he led the complete effort. You can reach Shrikant at shrkarve@in.ibm.com. Samrat Dutta is a software engineer in IBM Systems and Technology Group SVC and Storwize family. He has 2.5 years of experience in storage (SAN and NAS) test. As part of the IBM Storage Tier Advisor Tool, he was one of the primary testers of the application. You can reach Samrat at samrat_dutta@in.ibm.com. 10

13 Trademarks and special notices Copyright IBM Corporation References in this document to IBM products or services do not imply that IBM intends to make them available in every country. IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol ( or ), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others. Information is provided "AS IS" without warranty of any kind. All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer. Information concerning non-ibm products was obtained from a supplier of these products, published announcement material, or other publicly available sources and does not constitute an endorsement of such products by IBM. Sources for non-ibm list prices and performance numbers are taken from publicly available information, including vendor announcements and vendor worldwide homepages. IBM has not tested these products and cannot confirm the accuracy of performance, capability, or any other claims related to non-ibm products. Questions on the capability of non-ibm products should be addressed to the supplier of those products. All statements regarding IBM future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. Contact your local IBM office or IBM authorized reseller for the full text of the specific Statement of Direction. Some information addresses anticipated future capabilities. Such information is not intended as a definitive statement of a commitment to specific levels of performance, function or delivery schedules with respect to any future products. Such commitments are only made in IBM product announcements. The information is presented here to communicate IBM's current investment and development activities as a good faith effort to help with our customers' future planning. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput or performance improvements equivalent to the ratios stated here. 11

14 Photographs shown are of engineering prototypes. Changes may be incorporated in production models. Any references in this information to non-ibm websites are provided for convenience only and do not in any manner serve as an endorsement of those websites. The materials at those websites are not part of the materials for this IBM product and use of those websites is at your own risk. 12