HITACHI DYNAMIC TIERING WEBTECH SERIES

Transcription

1 HITACHI DYNAMIC TIERING WEBTECH SERIES SESSION 2 OF 3 STEVE BURR, SOLUTION ARCHITECT, GSS SERVICES ENGINEERING JOHN HARKER, SENIOR PRODUCT MARKETING MANAGER JULY 13, 20 AND 27, 2011

2 WEBTECH EDUCATIONAL SERIES Hitachi Dynamic Tiering WebTech Series Learn about fine-grain automated storage tiering, which used to be an exotic technology available from only a few sources. Now it is rapidly becoming a standard capability of advanced storage systems. To use it properly, you need to understand new factors in system design, such as an application s storage locality of reference pattern. In this webinar series, you will hear from the experts about how to take best advantage of this new technology on Hitachi Virtual Storage Platform. The 3 sessions in this webinar series will take you through useful topics with prepared remarks and interactive discussion: A deep look at the theory and design of Hitachi Dynamic Tiering How to size and configure a system Installation and operation You ll learn how to: Determine if an application s data is a good fit for automated tiered storage movements. Select and size tiers within a Dynamic Tiering pool. Use replication and migration with Dynamic Tiering virtual volumes Properly operate and monitor a Dynamic Tiering system

3 HITACHI VIRTUAL STORAGE PLATFORM HITACHI DYNAMIC TIERING WORKSHOP PART 1 PRINCIPLES PART 2 CONFIGURATION PART 3 OPERATION STEVE BURR SOLUTIONS ARCHITECT, GLOBAL SOLUTIONS SERVICES JOHN HARKER SENIOR PRODUCT MARKETING MANAGER JULY 2011

4 WORKSHOP SERIES PURPOSE A workshop and master class series on Hitachi Dynamic Tiering Summarize and contrast with Hitachi Dynamic Provisioning, if appropriate Technical in nature (where it aids understanding) Content-heavy To include many recommended practices Some new material Aims to help your evolution with the technology and therefore maximize the benefits 3 parts Part 1 Principles Part 2 Configuration and Design Part 3 Operation

5 PART 2 CONFIGURATION AGENDA Introduction Pool Design, Specifications, Design Rules External Tiers Tier Design and Sizing DP-VOL Design Overprovisioning

6 INTRODUCTION

7 HITACHI DYNAMIC TIERING

8 HITACHI DYNAMIC TIERING IMPROVE SAS PERFORMANCE AT SUB-SSD COST Problem or pain HDT Pool TIER 0 Need better than SAS Performance Won t pay for SSD for an entire set of volumes Capability Couple SSD and SAS storage to deliver near Tier 0 performance at a lower cost Hitachi Dynamic Provisioning Virtual Volume SSD TIER 1 SAS Benefits: Meet high performance application requirements

9 HITACHI DYNAMIC TIERING REDUCED CAPEX AND OPEX FOR LESS CRITICAL APPLICATIONS HDT Pool Problem or pain Frequent Accesses TIER 1 Need price economics SAS of SATA for scaling storage Poor SATA performance Hitachi Dynamic Provisioning Virtual Volume Infrequent References TIER 2 Capability SATA Use SATA to scale, but mix in a little SAS to boost performance Improve response times Better economics and bigger gains for large systems Benefits: High performance archival storage

10 POOL DESIGN

11 PERFORMANCE OF HDT AND HDP Normal disk design rules apply to pooled storage. Read/write, sequential/random, small/large block All imply specific disk architectures Hitachi Dynamic Tiering and Hitachi Dynamic Provisioning do not make slow disks faster, they just spread the capacity and the I/O to minimize hot spots. Most performance problems are related to hot spots. For heavy, sustained workloads, you will still need to consider: Smaller, faster drives With Dynamic Provisioning: RAID-1+0 Solid State Drive (SSD) The capacity and composition of a pool should be the larger of: The total capacity of the applications The quantity and type of disks suitable to accommodate the total I/O of the applications No need to factor in any overhead for back end. I/O latency is hardly affected. Is there any overhead? Yes, but rarely seen. There is a small amount of work to do to find the data. That is done by the virtual storage director (VSD).

12 POOL DESIGN FOR PRODUCTION HDT POOLS At least 4 disk groups per tier (except SSD) Use a disk group for one purpose only: 1 tier. Create a single pool volume occupying the whole disk group, or if disk group > 2.99TB, use all logical devices (LDEVs) created (equal: optional) Do not divide a disk group across 2 pools or pool and non-hdt Keep all the same: Same RAID level RAID-6 is now our general recommendation Same size HDT will enforce these HDP rules: Same rotation Same technology FC, SATA, SAS All external or internal The penalty for mixing is far heavier than you might expect Ideally distribute over available back end directors, SAS paths, trays, etc.

13 FEW LARGER POOLS OR MORE SMALLER POOLS Use larger pools unless you have a reason not to. More distribution of hot and cold Fewer management points Strategic reasons Customer separation and SLA Storage tiers, classes, chargeback (but risk losing the benefit of pooling) Overprovisioning: Separation of risk (development and production) ShadowImage primary volumes and secondary volumes are best separate Some good evidence that random data and sequential logs mix well in the same pool. Consider combining sequential and random loads so extra spindles benefit all. (But in all cases?) Keep data + log DP-VOLs separate so cache algorithms optimize them. Workload class separation: Steady time-critical online transaction processing (OLTP) and bursty decision support, or the pool must be sized for the peak load (Care in choice of monitor scheme is required). Reliability: On enterprise, I don t consider it a factor. On Hitachi AMS or external storage, I accept it is a factor to consider.

14 POOL SPECIFICATIONS Specification HDP HDT Page size Pools: number, capacity Pool-vols: number, capacity V-VOLs: number, capacity 42MB 128 (HDP+HDT+COW), Max 1.1PB/system 1024/pool each 8GB-4TB Max 63,232/system each 46MB-60TB* Storage type Internal and external Internal and external* RAID levels R5, R6, R1 R5, R6 Disk type SAS15K, SAS10K, SAS7.2K, SATA-W/V, SATA-E, SSD, EXT Tiers Reduce and shrink pool * caveats SAS15K, SAS10K, SATA-E, SATA-W/V, SSD, EXT YES, but cannot remove first pool-vol

15 HDT TIERS Maximum 3 tiers in 1 pool (not restricted to: SSD, SAS, SATA) Tier is defined or allocated by I/O throughput of the media (at design time). Media with shortest response time are positioned as higher tiers, and media with longer response time are positioned as lower tiers. Tier order is based on media type and rotational speed (rpm) only. Differences in performance according to RAID levels are excluded when determining order of tiers. Media supported by VSP 2.5" SSD (200GB) 3.5" SSD (400GB) 2.5" SAS15Krpm (146GB) 2 Order of tiers 1 Add Tier Tier1 Tier2 Tier3 SAS SATA SSD Add Tier1 Tier2 Tier3 SSD SAS SATA Insert based on order of tiers Move other media to lower tiers 2.5" SAS10Krpm (300GB) 2.5" SAS10Krpm (600GB) 2.5" SAS 7.2Krpm (500GB) 4 3.5" SATA (2TB) 5 3 Delete Tier Tier1 Tier2 Tier3 SSD SAS SATA Delete Tier1 Tier2 Tier3 SAS SATA Move other media to upper tiers External 6

16 TIER DESIGN RULES Mixing within a tier is never good. Each tier should have one RAID type. RAID-5: 3+1 or RAID-5: 7+1 or RAID-6: 6+2 For example: Tier 1 SSD-7+1 Tier 2 SAS 10K-3+1 Tier 3 SATA-6+2 Avoid concatenation Reversing tier performance not recommended (e.g., R6 above R5) First pool volume added cannot be removed from the pool About 4GB reserved for management Don t start with SSD as pool volume #1 A tier of external storage is supported but must be designed carefully Media tier groupings supported by Hitachi VSP 2.5" SSD (200GB) 3.5" SSD (400GB) Order of tiers 2.5" SAS15Krpm (146GB) 2 2.5" SAS10Krpm (300GB) 2.5" SAS10Krpm (600GB) 2.5" SAS 7.2Krpm (500GB) 4 3.5" SATA (2TB) 5 External 6 1 3

17 EXTERNAL POOL AND TIER DESIGN Balance and dedication apply equally to internal and external. External pools see higher and more sustained workloads than server LUNs. Dedicate disks and ports. Balance inside external storage: Use owned paths, trays, ports as appropriate Additional rules Ensure there are enough ports. Evaluate queue depth carefully. Too low or too high will cause problems Reliability of an external pool is controlled by external factors. Use the best storage: model, appropriate RAID level, redundant paths. I recommend simple but well monitored connectivity. Direct connect is simple but impossible to monitor. Switch or director, but: If possible separate from server SAN. Duplex switches for highest availability. Actively monitor for errors and imbalance. Use VSAN or physically separate A and B switches. Upgrade one at a time. Do not combine Hitachi Virtual Storage Platform, Hitachi Dynamic Provisioning or Hitachi Dynamic Tiering and external pools (Hitachi Adaptable Modular Storage, HDP)

18 HDT DESIGN WITH EXTERNAL TIERS Media tier groupings supported by Hitachi VSP Order of tiers External storage is always classed as the lowest tier. 2.5" SSD (200GB) 3.5" SSD (400GB) 2.5" SAS15Krpm (146GB) 2 1 This cannot be changed. 2.5" SAS10Krpm (300GB) 2.5" SAS10Krpm (600GB) 3 External storage cannot be recommended in all scenarios. External tiers, such as include for unforeseen capacity requirements, are a bad idea. External HDT design topics Storage reliability Connectivity design Planning tiers for capacity and performance 2.5" SAS 7.2Krpm (500GB) 4 3.5" SATA (2TB) 5 External 6

19 STORAGE RELIABILITY WITH EXTERNAL STORAGE Applies equally to HDT, HDP and static provisioning A pool is as reliable as its weakest component. VSP cannot make unreliable storage more reliable. In fact, putting it in a pool does the opposite. Consider where external storage should be applied. In reliability design, consider the storage and its connectivity. Use Raid-6 where available. Use all possible reliability measures available to you. Monitor for and action to prevent latent failure: The probability of 2 simultaneous links failing is low. The probability of 2 separate failures is higher. If you do not actively monitor and repair, then failures may remain hidden. One failure is then enough for an outage. We do this in enterprise storage architecture: component redundancy, predictive fault monitoring, hot swap drives, Hi-Track, etc. With external storage, it is external.

20 CONNECTIVITY DESIGN WITH EXTERNAL STORAGE Applies to HDT, HDP and static provisioning But especially HDT and HDP Pool I/O is intense and sustained, more than server Need to avoid delay, queue or blocks VSP EXT The connection bottleneck Internal storage: up to 16 x 6GB SAS channels, internal queuing optimized External connectivity Likely to be less ports used, less in parallel. Likely to be more queuing, storage optimized for server access Need to design the correct queue depth between the 2 boxes compatible with external back end IOP, and tier IOP requests Queue depth: Too small problems, too large problems Maintenance of external resources must be managed. Avoid over connectivity: distribute devices across paired ports.

21 HDT, EXTERNAL STORAGE AND CACHE Compared to internal volumes, the performance of external volumes may be low. Therefore, the external volume may become a bottleneck, giving more load to the cache. In particular, if external and internal volumes coexist, the external volumes give load to the cache and prevent the internal volumes from achieving the expected performance. For this reason, special attention is required when external volumes are used.

22 HDP, EXTERNAL STORAGE AND CLPR It was a general best practice to have external storage in a separate cache logical partition (CLPR). Valid for HDP pools of external devices and LUNs from external devices Why? Relatively poor performance of external storage If the external storage received more IOPS than it could accept, a queue of write I/O would build up and be seen as high cache write pending (CWP). High CWP, if not contained in a CLPR, impacts other users. Poor external performance is mainly a write issue. External storage is often associated with lower cost storage. Write is the painful workload on disk systems, especially RAID-5 or RAID-6, where every write involves multiple back end transactions. SATA, especially, performs much more poorly on write-heavy workloads. Reads are self-constraining. If an external pool has a poor read time, the longer delay will naturally slow down subsequent read. Hence the recommendation for Hitachi Dynamic Provisioning. This does not apply to Hitachi Dynamic Tiering.

23 HDP, EXTERNAL STORAGE AND CLPR Note: The recommendation for HDP+EXTERNAL+CLPR is because of the overload to the external storage. If the storage were operated within its design parameters, then CLPR would not be needed. Many designers used the CLPR, just in case. In fact, we can divide external design scenarios into 2 use classes: Use class #1: External storage is designed and operated within its design parameters. In this case, a CLPR is not actually required (but may be employed by a conservative designer). Use class #2: External storage is run intentionally overloaded. The idea is that low-cost storage is used in non-critical applications where the service level matters far less than reducing the storage costs. This second scenario is where the CLPR prevents CWP from affecting other more critical applications.

24 HDT AND EXTERNAL TIER DESIGN As always, plan for use, capacity, performance and I/O profile. Design the pool as a whole: an HDT pool operates as a single object. It is the CLPR of the DP-VOL that affects things anyway. Although pool volumes can be placed in different CLPR, it does result in what you might want. Any cache write pending would impact the whole pool. HDT has 2 strategies for external storage use with HDT. Pool with an external tier designed and operated within all design parameters Where external tier is operated overloaded NOT OK This second scenario cannot be recommended with Dynamic Tiering, unless pool use is moderate and non-critical. [Development? Backup? Batch?] For this scenario, have: Pool, V-VOL and Pool-VOL in a separate CLPR. OK

25 POOL USE STRATEGY REVISITED You can design a pool to optimize for maximum performance (at reasonable cost) Or you can design a pool to optimize for minimum cost (with reasonable performance). Trying to optimize for both at the same time is less likely to be fruitful. External storage design is an illustration.

26 TIER DESIGN AND SIZING

27 INFORMATION TO SIZE DYNAMIC TIERING The more information you have the better a design you can create - Historical IOPS/LU - Utilization for each part (HDP/HDT pool, PG, CHA, DKA, etc) Being realistic: Make best effort initial design Then tune design as it grows - Point in time IOPS/LU - Throughput(MB/s)/LU - LU size (GB) - Application type - I/O characteristic (random/sequential ratio, read/write ratio, block size, cache hit, etc.) plus - IOPS/page in LU Scenario#1 Scenario#2 Scenario#3 Scenario#4 New purchase Migration from existing Hitachi storage (USP-V, USP) Migration from existing non- Hitachi storage (EMC DMX) Upgrade of RAID700 with configuration change or addition Target Scenario

28 INFORMATION FOR A DESIGN The information detail will vary Capacity alone Capacity and IOPS Capacity and IOPS per tier, application or owner Capacity and IOPS per LUN Capacity and IOPS per page Growth plans: capacity and performance, certainty Growth similar or different? Infrastructure and other requirements Alerts, SNMP, toolset (Hitachi Device Manager, scripting, etc.) Storage strategy, overprovisioning, plans

29 WHY THE SHAPE MATTERS These 4 graphs have identical capacity and total IOPS SSD+SAS+SATA is not the only answer The far right graph is probably best on HDP The far left graph is classic for HDT Middle graphs may imply different pool compositions (15K+10K, SAS+SATA) Flat graphs contraindicate wide tier differences. Or use HDP

30 BLIND DESIGN Greenfield implementations could have little more than capacity. Even capacity and IOPS aren t much help. Rule of thumb 2%/18%/80% (SSD/SAS/SATA) There is little solid data behind this: 3/22/75? or 0/30/70? or 0/100/0? We have profiles for some of the common workloads. We have tools to perform cost/fit analysis. The results can be surprising. Several quite different options can be quite close in cost. A single tier is not unusual. Need to consider future too. Should plan for continuous monitoring

31 HDT PERFORMANCE SIZING Best to consider the I/O load by tier. Maximum performance (throughput) of HDT Pool is the sum of performance of each tier. Pool performance = [Tier 1 Performance] + [Tier 2 Performance] + [Tier 3 Performance] Tier 1 Performance = [Tier 1 (RAID X) Performance ] Number of Parity Groups Tier 2 Performance = [Tier 2 (RAID X) Performance ] Number of Parity Groups Tier 3 Performance = [Tier 3 (RAID X) Performance ] Number of Parity Groups Maximum performance is achieved where I/O pattern is optimum load for each tier. Maximum performance may not be achieved depending on I/O pattern when load of each page is balanced, etc. Performance Max performance of HDT Pool Tier 2 Tier 3 Tier 1 All Tier tiers 13 2 overloaded within range Number of PG in Tier 1 Number of PG in Tier 2 Number of PG in Tier 3 Number of PG

32 SIZING STRATEGIES Capacity alone Capacity and IOPS You can only adopt a rule of thumb approach like 2/18/80 Capacity and IOPS per tier, application or owner Capacity and IOPS per LUN We have tools for analyzing many well known applications and making performance recommendations with cost benefit Capacity and IOPS per page Can fully reverse-engineer the pool for a custom analysis and optimal result

33 DP-VOL DESIGN HDT AND HDP DESIGN RULES THE SAME

34 LDEV DESIGN WITH VSP Maximum logical device (LDEV) sizes on Hitachi Virtual Storage Platform: HDP or HDT LDEV 46MB-60TB (was 4TB) But you cannot replicate or migrate > 4TB Internal static volumes 2.99TB (virtual device limit) Static external volume 60TB (was 4TB) LUSE 2.99 x 36TB (please don t use they are inflexible and obsolete). Because we can make a LUN of 60TB, it doesn t mean the OS can handle it. May limit at 2TB or other numbers. Broken host bus adapter (HBA) drivers may limit, broken utilities. For information about the maximum LU capacity supported, contact the vendor of your operating system.

35 DP-VOL SIZE, NUMBER AND USE 1 A size catalog: For your own sanity To maximize flexibility with replication and migration Can resize online. ENTERPRISE: Multiple of 42MB most capacity efficient MODULAR: Multiple of 1GB most efficient DP-VOLs: separate (random vs. sequential) so can optimize cache algorithms Best practice (for HDT, HDP and non-hdp) for performance-critical applications is to have one data object (edb, mdb, ldb, dbf) alone in a LUN. This improves locality (especially Windows c.f.). This costs too much with static volumes. But with HDT or HDP, this best practice now costs nothing!

36 DP-VOL SIZE, NUMBER AND USE 2 How many? How large? Stripe over wide stripe? Oracle ASM and/or HDP? Especially, consolidated virtualized workloads No simple rule From a storage-only point of view, fewer-bigger is better. But the actual use of the data and OS internal design changes things. Storage works better doing multiple things in parallel, not one at a time. More devices increase internal queuing in the OS. More devices permit more queue depth available to the application. If you over-consolidate, the ideal queue depth exceeds the sweet spot (<=8). It is also much harder to manage high queue depths. Occasionally stripe over stripe gives problems: ruins locality and write coalescing, often alignment mismatch, use large OS level stripe units for random data Sustained sequential data, however, benefits from small striping (ASM default) Avoid mixing large and small devices on the same server (and ideally on any port) the small LUNs grab the Q slots needed for the large heavy I/O LUNs VMware best on its own port (for this and reserve management)

37 DISK ALIGNMENT 1 Relevant for HDT, HDP and non-hdp If the operating system logical disk alignment is different from the physical disk/cache slot alignment, unnecessary work is required, especially Raid-5 Disk/cache slot (256/16KB) not HDP slot (42/32MB) It s free, why not do it? It could gain 10% Most important to look at: VMware Linux Header MBR Partition start Partition alignment I/O I/O I/O I/O Application or file system 4K, 8K, 16K Logical Physical I/O I/O I/O I/O Modular: 16KB Enterprise: 512 or 256KB

38 THE OVERPROVISIONING QUESTION Thin provisioning = overprovisioning Dynamic Provisioning and Dynamic Tiering support thin provisioning, but to use it is your choice. Many will be planning to evolve into overprovisioning. Many wish to avoid over-provisioning completely. Many will control it carefully. You are in good company. Caution is a good trait in an administrator. This doesn t apply if you are overprovisioning. But I believe you will eventually overprovision. Don t fool yourself: The moment a disk purchase is delayed because it s all allocated but only half used, you are overprovisioning

39 DP-VOLS AND OVERPROVISIONING It is your choice Most factors are at the server level: application, file system and the user base The keys to successful overprovisioning: Planning Education Cooperation

40 QUESTIONS AND DISCUSSION

41 UPCOMING WEBTECH SESSIONS July Hitachi Dynamic Tiering WebTech Series (3 sessions) July 13, 20 and 27 at 9am PT, 12pm ET (1 hour each Wednesday for 3 weeks) August Ensure Data Relevance with Hitachi Data Discovery, August 3, 9am PT, 12pm ET Save Power. Save Space. Save Money, August 17, 9am PT, 12pm ET Please check for: Link to the recording, the presentation and Q&A (available next week) Schedule and registration for upcoming WebTech sessions

42 THANK YOU