Storage Virtualization with IBM SAN Volume Controller 1G04

Transcription

1 Storage Virtualization with IBM SAN Volume Controller Torsten Rothenwaldt IBM European Storage Center DECUS IT-Symposium G04 Agenda Review different architectures of disk block virtualization. Explain implementation details of the IBM SAN Volume Controller as example of in-band appliance. Discuss current enterprise usage of disk/file virtualization

2 Table of contents Disk block virtualization IBM SAN Volume Controller Storage virtualization in enterprises today 3 Disk (Drive) Virtualization nnnn Physical data layout Cylinder/Head/Sector addresses Media defects Virtual data layout Logical Block Addresses (LBA) Defect-free 4 2

3 Block-Level Virtualization Bound to a physical machine Fixed size and (pseudo-) geometry Limited performance Do break (occasionally) Anywhere (can be moved or replaced) Size as needed (can grow, shrink, or morph) Performance scaling Reliable as needed Where does block-level virtualization reside? 5 Virtualization in the Storage Subsystem Provisioning Per-host Virtualization RAID subsystem +Heterogeneous hosts +Mature industry & products Performance Stable & reliable Perceived security - Homogeneous storage -Management Device-specific per-box 6 3

4 Virtualization in the Host Provisioning Per-host still Virtualization Host RAID subsystem +Flexibility (any storage) +File system coupling (online growth, re-layout, movement, snapshots, ) -Server-centric management of RAID subsystems of volumes -Complexity shared data clusters 7 Out-of-Band Virtualization Provisioning Data center-wide Virtualization Host Infrastructure RAID subsystem SAN appliance separate box In-host +Data center-wide management +Shorter I/O path +Light-weight compared to full volume manager -May be host-invasive Host-specific - Appliance availability - Secure protocols 8 4

5 Virtualization in the Network: In-Band Provisioning Data center-wide Virtualization Host SAN Host access Device access Infrastructure RAID subsystem +Data center-wide management Heterogeneous storage Heterogeneous hosts -Complexity User (un)familiarity Integration needs Needs clustering -Performance perceptions 9 Virtualization Devices for In-Band Server-based device (appliance) Virtualizes a variety of physical storage using different HBAs. One pool. Implements complex storage solutions inexpensively Adds another layer (managed separately) Less interoperability issues (no fabric integration) Appears as standard device. Switch-based device (fabric application) Network optimized High port counts Expensive Elaborated functions not available Fabric integration critical Standardization just beginning (ANSI T11 Fabric Application Interface Standard)

6 Table of contents Disk block virtualization IBM SAN Volume Controller Storage virtualization in enterprises today 11 Example of a Block-Level In-Band Appliance: IBM SAN Volume Controller J. S. Glider, C. F. Fuente, W. J. Scales: The software architecture of a SAN storage control system. IBM Systems Journal, Vol. 42 (2003) Nr. 2, pp ( ) IBM Redbooks: SG IBM System Storage SAN Volume Controller SG Implementing the SVC in an OEM Environment SG Using the SVC for Business Continuity ( )

7 IBM SVC: Architecture Virtual LUNs Storage Netw ork Virtual LUNs Virtual LUNs Virtual LUNs Node Node Node Node Node Node NodeN Node Managed Disks 13 IBM SVC: Zoning Host Zone: -Hosts zoned only to SVC -See only Virtual Disks that SVC allows to see Device Zone: -Devices zoned only to SV -See only SVC nodes as connected hosts

8 IBM SVC: Managed Disks SVC does not perform RAID functions Utilizes RAID capability of backend storage server RAID-5, RAID-10, or RAID-1 recommended Normally LUNs "surfaced" from storage systems are what the hosts on the SAN see as physical disks Disks surfaced by the storage systems are discovered by SVC and referred to as Managed Disks (mdisks) Spare capacity on mdisks can be reallocated transparently and dynamically R5 MD R10 MD R5 MD R1 MD R5 MD R1 MD SANVC R5 VLUN R10 VLUN R5 VLUN R1 VLUN R5 VLUN R1 VLUN RAID Controller RAID Controller 15 IBM SVC: Managed Disk Groups Once the mdisks are available to SVC, the user assigns them to one or more pools called Managed Disk Groups These MDGs are addressed by the SVC in terms of extents: Extent size is determined at MDG creation time, default 16 MB, max 512 MB Extent size determines maximum amount of storage SVC can manage 16-MB extents = 64 TB, 512-MB extents = 2 PB Cannot migrate vdisks between MDGs with different extent sizes Can migrate extents from mdisk to mdisk R1 Pool Extent 1 Extent 2 Extent 3 R5 Pool R1 R1 R1 Extent 4 Extent 5... R5 R5 R5 R10 Pool R10 R10 R10 Extent n

9 IBM SVC: Virtual Disks From these extents, the user can build "virtual disks" Various policies can be used to build them: Striped taking an extent in turn from each disk in the pool or a subset of the disks in the pool (Virtual Disk striped across multiple disks) Sequential using a single disk in the pool (Virtual Disk mapped sequentially to managed disk) Image Mode Virtual Disk = Physical LUN SVC Cache Enabled or Disabled Real physical capacity must be available to create a vdisk Virtual disks can be expanded, reduced, or deleted IO governing can be enabled to limit IO/s or MB/s Extent 1a Extent 1a Extent 1b Extent 1c Extent 1d Extent 1e Extent 1f Extent 1g Extent 2a Extent 2b Extent 2c Extent 2d Extent 2e Extent 2f Extent 2g Extent 3a Extent 3b Extent 3c Extent 3d Extent 3e Extent 3f Extent 3g Create a striped virtual disk Extent 2a Extent 3a Extent 1b Extent 2b Extent 3b Extent 1c Extent 2c Extent 3c 17 A host vdisk is a collection of Extents each 16 MB MB Cache Disabled Virtual Disks Can be used with Image Mode VDisks only to enable use of copy services of underlying disk subsystem Can be used with striped mode VDisks May be useful to preserve cache resources Example: Use for TSM/HSM disk pools Example: Use for test/dev VDisks in mixed production environment Cache Disabled Virtual Disks are basically normal VDisks and can utilize SVC capabilities like migration and copy services Can use SVC copy services on the source VDisk Cannot use SVC copy services on the target VDisk Provide a way for parallel direct access to the underlying disk subsystem

10 IBM SVC: IO Groups and Hosts Each virtual disk is assigned to a particular IO Group (node pair). Every node in the cluster is aware of the virtual disk, but only owning IO Group services requests. All IO is targeted at either of the nodes in the IO Group for purposes of caching and load balancing. It is these virtual disks that SVC presents to hosts on the SAN as targets of IO. The virtual disks are mapped to hosts (SDD for mutli-path operation). Can be mapped to multiple hosts for use with clustering software. The hosts see these as physical disks (in terms of the OS). SVC knows hosts as groups of HBA WWPNs. IO Group VDisk WWPN1 WWPN2 WWPN3 Host WWPN4 19 SVC Combined Physical & Logical View Virtual Disks Mapped to Hosts Virtual Disk vdisk0 vdisk1 vdisk2 vdisk3 vdisk4 Virtual Disks are associated with particular I/O Groups SVC Cluster I/O Group 1 I/O Group 2 Managed Disk Groups are accessible by all I/O Groups in the Cluster. Managed Disk Groups Managed Disk mdiskgrp0 [FAStT Group] - 40GB mdisk0 10GB mdisk1 10GB mdisk2 10GB mdisk3 10GB mdisk4 mdiskgrp1 [ESS Group] - 60GB mdisk5 mdisk6 LUN FAStT 10GB FAStT 10GB FAStT 10GB FAStT 10GB ESS ESS ESS

11 IBM SVC: Clustering Cluster comprised of 2 8 nodes but administered as single image No Linux clustering software, failover/failback is function of SVC code One node automatically designated config/boss node for cluster Assigned cluster IP address Responsible for coordination of node transitions Automatic failover of config node and IP address Auto-restart of a node on failure and re-admission to cluster Cluster requires majority of nodes remain operating to ensure quorum Two nodes is a special case Quorum disk is elected as a tie-breaker Quorum disks are existing mdisks under SVC and use 1 extent A node stores a write in its own write cache and the write cache of its partner node before acknowledging to the host application On node failure, surviving node empties write cache and proceeds in write-through mode 21 IBM SVC: Copy Services VD 1 MD VD 2 MD VD 2 3 IO Group MD VD 3 4 MD VD 4 5 MD 5 VD 6 MD 6 MD 7 VD 7 MD 8 VD 1 MD VD 2 MD VD 2 3 MD VD 3 4 IO Group MD VD 4 5 MD 5 VD 6 MD 6 MD 7 VD 7 MD 8 LUN 1 LUN 2 LUN 3 LUN 4 RAID controller 1 LUN 1 LUN 2 LUN 3 LUN 4 RAID controller 2 LUN 1 LUN 2 LUN 3 LUN 4 RAID controller 3 LUN 1 LUN 2 LUN 3 LUN 4 RAID controller 4 SAN Data SAN Migration Peer-to-Peer Remote SAN FlashCopy outside the (PPRC) box outside the outside box the box (Metro / Global Mirror)

12 IBM SVC Global Mirror Long distance asynchronous remote mirroring function Practically unlimited distances for business continuity Does not wait for secondary I/O before completing host I/O Minimizes performance impact to applications Designed to maintain consistent secondary copy at all times Once initial copy has completed Built on Metro Mirror code base Metro and Global Mirror delivered as single feature Offers great implementation flexibility No additional license charge for existing MM users 23 Global Mirror: Overview Single remote copy partnership per cluster Supports consistency groups that span I/O groups within cluster Consistency group consists of MM or GM relationships not both Cluster 1 Partnership Cluster 2 Vdisk A Relationship Vdisk C Vdisk B Consistency Group Relationship Vdisk D

13 Global Mirror: Algorithm I/O complete acknowledged to host at primary once following occurs: Data is committed to cache in both nodes in I/O group Log of I/O and its sequence number committed on both nodes of I/O group Remote I/O is transmitted concurrently with processing of local I/O Local host I/O normally completes before remote I/O completes Extra messages passed between nodes in cluster Used to detect concurrent host I/O among VDisks in same consistency group Concurrent I/O is assigned a shared sequence number During high loads many I/Os will share sequence number to minimize overhead Remote I/Os are applied at secondary in sequence number order Writes within a sequence number are non-dependant and written in any order once received at secondary Many sequence number batches can be outstanding on the long distance link At secondary site a single sequence number is applied at a time 25 Global Mirror: Link Tolerance Existing redundancy used to continue operation if node or link fails: Recovery begins by re-driving writes that were in flight. Log of I/O established during initial host write used to support recovery. Recovery uses read to primary VDisk to capture data for replay of write Autonomic Link Tolerance feature to prevent performance problems leading to application outage: Monitors write latency impact of GM and considers stopping a GM relationship or consistency group if impact exceeds a threshold Link tolerance threshold allows bursts of work to be tolerated Defines period of time poor performance can be tolerated before GM stops Settable from 1 minute to 1 day in 10 second increments (default 5 minutes) Disabled by setting to

14 Table of contents Disk block virtualization IBM SAN Volume Controller Storage virtualization in enterprises today 27 Why To Use Storage Virtualization The management nightmare Too many servers, operating systems, storage systems, management consoles, Too complex policies Too large migration projects Availability requirements Storage resource utilization Storage performance

15 SVC Version 4.1 Supported Environments Novell NetWare Clustering VMWare Win / NW guests Microsoft MSCS MPIO, VSS, GDS x64 IBM AIX HACMP/XD GPFS / VIO Sun Solaris 10 VCS Clustering SUN Cluster HP/UX 11iV2 TRU64 OpenVMS ServiceGuard Clustering Linux (Intel/Power/zLinux) RHEL/SUSE W / LVM RHEL AS 4 Ia32, x64, System p IBM BladeCenter Win/Linux/VMWare/AIX OPM/FCS/IBS Hosts Cisco McData iscsi to hosts Via Cisco IPS Point-in-time Copy Full volume Copy on write SAN with 4Gbps fabric Continuous Copy Metro Mirror Global Mirror SAN SAN Volume Controller SAN Volume Controller IBM ESS, FAStT IBM DS DS4K / 6K DS8000 DS4800 IBM N series 5200, 5500 Hitachi Lightning 9980V 9970V 9910/9960 Hitachi Thunder xxV 9520V Hitachi Tagma Store USP, NSC55 HP EVA 3000/ / HP MA/EMA HP XP 48, , 1024 EMC Symm 8000 DMX EMC/Dell CLARiiON FC4700 CX2/3/4/5/6/700 Sun 9910/ /9980 NetApp STK FAS D173, 178, 220, 3020, , 280 FLX210, 240, 280, Performance: SVC SPC-1 Benchmark _IBM_SPC1_executive-summary.pdf

16 Performance: SPC-2 Benchmark Newest addition to the Storage Performance Council Benchmarks Composite of three workloads to measure sequential performance Large file processing scientific and large-scale financial processing Large database queries data mining and business intelligence Video on demand streaming movies to end users SVC posted the highest results in this industry-recognized storage performance test SVC (3.517 GB/s) + DS8300 (3.217 GB/s) lead the industry in this new benchmark 31 Enterprise Usage Today Widely accepted by customers: Tape drive and library virtualization Disk block level virtualization in LVM and storage subsystem In-band file system virtualization (NAS) Aggregated networks (iscsi, long distance connections) Beginning of the lifecycle: Disk block fabric-layer virtualization (in-band and out-of-band) Niche markets (this may change): Virtual SANs File and record virtualization

17 Why Do Customers Hesitate? Reliability SAN and storage failures may affect the whole datacenter. SAN and storage failures may destroy data. Security implications not clear. Complexity Lack of standards. Products difficult to understand and evaluate. Multi-vendor support. Market situation Religious wars about architectures. Too many different solutions. 33 Disclaimer Information Copyright 2007 by International Business Machines Corporation. No part of this document may be reproduced or transmitted in any form without written permission from IBM Corporation. All statements regarding IBM 's future directio n and intent are subject to cha nge or withdra wa l without notice, and represent goals and objectives only. Refere nce s in this document to curre nt or future IBM produc ts, progra ms, or services does not imp ly tha t IBM intend s to make such such products, programs or services available in all countries in which IBM operates or does business. THE INFORMATION PROVIDED IN THIS DOCUMENT IS DISTRIBUTED "AS IS" WITHOUT ANY WARRANTY, EITHER EXPRESS OR IMPLIED. IBM EXP RESSLY DISCLAIMS ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOS E OR INFRINGEMENT. IBM shall have no responsibility to update this information. IBM products are warranted according to the terms and conditions of the agreements (e.g., IBM Customer Agreement, Statement of Limited Warranty, International Program License Agreement, etc.) under which they are provided. The provision of the information contained herein is not intended to, and does not, grant any right or license under any IBM patents or copyrights. Inquiries regarding patent or copyright licenses sho uld be made, in writing, to : IBM Director of Licensing IBM Corporation North Castle Drive Armonk, NY U.S.A

18 Closing slide Thank you DECUS IT-Symposium G