Jai Menon and the rest of the team IBM Research Autonomic Storage Systems April 11, 2002

Transcription

1 Jai Menon and the rest of the team IBM Research Autonomic Storage Systems April 11, 2002 Copyright IBM Corporation All rights reserved. Presentation File Name

2 Why do we need Autonomic Storage? Storage cost increasing as fraction of total server HW (>50%) Compute intensive I/O intensive Data intensive 100 Percent of server cost Storage management is a major concern of customers Management personnel cost conservatively 2-3x purchase Not getting simpler: More options, more tools, more acronyms Requirements are increasing (e.g., 24x7 operation)

3 Storage Management & TCO Gartner TCO estimates Purchase 20.0% Environmental 14.0% Management 66% ITCentrix survey 2001: 0.83 storage PY / TB-year... which is >$100/GB-year Source: Gartner Research, 2000

4 Storage Administrators per Terabyte Required Headcount (low is better) (not Autonomic) (some Autonomic aspects) Source: IDC, 2001

5 Today -- diversity of storage infrastructures and management enterprise clients branch office servers Intranet home clients enterprise servers (web, DB, SAP, app) NAS NAS VPN Internet storage networks enterpise storage controllers Disk array

6 Our Vision Single storage infrastructure and common management for the enterprise Storage System manages autonomically to application requirements - goal driven, policy-based placement, replication and migration of data to meet perf goals backup/restore/replication to meet availability goals capacity growth and performance growth predictions automated provisioning of storage networks and storage servers Minimize complexity of storage hardware minimize number of components eliminate cables reduce environmentals fail-in-place Move from 1 TB/storage admin to 1 PB/storage admin

7 Today - No common file system; no central mgmt. Unix Unix File System... NT NT File System Storage Network Virtualization free space in one OS not usable by another OS no enterprise-wide flexibility in file placement no enterprise-wide name space backup/restore/hsm is not centralized or server-free Moving apps between OSes requires moving data cannot share files

8 Storage Tank - Common Mgmt. & File System for the Enterprise enterprise clients home clients enterprise servers (web, DB, SAP, app) Win2K AIX Solaris Linux ST agent ST agent ST agent ST agent data SN Intranet Storage Server Storage Server Storage Server NAS Internet/VPN Metadata Data Data Backup Capabilities: performance and semantics like SAN sharing like NAS policy-based, centralized storage management (better than both)

9 Geographically distributed Storage Tank Geographically distributed server clusters forming single namespace - Replication of files/containers for good local performance - Extended protocols for consistency across replicas - Migration of primary copy of data to point of use Win2K ST Agent Control Network (IP) AIX Solaris ST Agent ST Agent Linus ST Agent ST Server Cluster data SAN Control Network (IP) Win2K AIX Solaris Linus ST Agent ST Agent ST Agent ST Agent data SAN ST Server Cluster Win2K Control Network (IP) AIX Solaris Linus SFO Tank NYC Fargo ST Server Cluster ST Agent data ST Agent SAN ST Agent ST Agent

10 Autonomic Provisioning of Storage Application A Enterprise Data Center Application B Application C Application has - capacity requirements - performance requirements - recovery time requirements Storage Tank Servers SN Perf Guaranteed LUNs Manage to Application Objectives: - StorageTank server management - LUN management - Backup/replication management LUN LUN LUN LUN LUN LUN LUN LUN LUN LUN LUN LUN LUN = virtual disk Copyright Copyright IBM IBM Corporation 2001

11 Technology for Data Placement Run Discovery Run 1 Original Layout Hot Cold Run 2 ALIS3 - Heat + Run Clustering Run 1 Run 2 ALIS2 - Run Clustering

12 Technology for short term forecasting of system demands - procedures similar to electrical utility procedures - 6 months of historical data until April 29 was used for above prediction of April 30

13 Collective Intelligent Bricks (CIB) A different storage architecture Monolithic Scaling is extremely coarse High Management costs High entry cost Very robust components Failure disruptions can be major Failed components repaired Modular Scaling is fine grain Low Management costs Low entry cost Moderately robust components Failure disruptions are small Failed components not repaired

14 IceCube System - "a Pile of Bricks" Bricks are stacked on vertical columns (not shown) for power insertion and heat removal Bricks communicate with neighbors in a 3D mesh 10 Gb/s link = good bad Cube Logic View = 3D Mesh Almaden prototype - 3x3x3 bricks, 324 disks, 32 TB, 16 cm on a side => could store all the books of the library of congress, 2002

15 An IceCube Intelligent Brick TB 600 SpecInt 200 W 12 disks 7.5" / side 6 ports

16 1PetaByte: Conventional vs IceCube (w/ 250-GB, 18-W, 3.5" disks in either system) 32 racks / 800 sq-ft Today... Glycol/Water Chiller C R A C C R A C 800 ft kw 75+ db Noise Waterpipes required 31 kw Water Future... Glycol/Water Chiller 25 kw IceCube C D U C D U 14 kw 150 kw 1 Cube / 120 sq-ft 190 kw quiet Waterpipes required 14% Floorspace

17 IceCube: Storage and/or Compute Server Colorcode: Blue=Storage Bricks Yellow=Compute Bricks Green=Mixed Bricks A: Host Storage Server B: Compute Server Storage Server C: Functions mixed in one Cube D: Functions mixed within bricks

18 Our Vision Single storage infrastructure and common management (Storage Tank) Storage System manages autonomically to application requirements - goal driven, policy-based (Storage Tank, ALOMS, SledRunner, ALIS)) placement, replication and migration of data to meet perf goals backup/restore/replication to meet availability goals capacity growth and performance growth predictions automated provisioning of storage networks and storage servers Minimize complexity of storage hardware (CIBs) minimize number of components eliminate cables reduce environmentals fail-in-place Move from 1 TB/storage admin to 1 PB/storage admin Kill Complexity

19 Our Vision - Single infrastructure for the enterprise enterprise clients branch office servers Intranet home clients enterprise servers (web, DB, SAP, app) NAS NAS VPN Internet storage networks enterpise storage controllers Disk array cache data permanent data

20 Operate analogous to electrical utilities Select a measure of system demand KWH and KW ops/sec Forecast demand autoregressive time series (short run) autoregressive time series (advanced) Establish system objective system reliability goal; by service class response time goal; by service class Evaluate how controllable parameters affect objective track spin-up times, reliability, capacity and variances of sources track request handling ability of nodes and LUNs and their variance Adjust controllable parameters to meet system demand bring generation sources on and off line assign LUNs and nodes to applications and change data placement

21 Collective Intelligent Bricks (CIB) Scale seamlessly from a few TB to several PB while providing enterprise class performance, fault tolerance and data integrity building block is a small brick holding 8 3.5" or " disk drives each brick integrates necessary storage, controller, virtualization and switch logic Total Cost of Ownership significantly less than today's storage systems it is completely self-managing (less admin costs) it uses a fail-in-place service strategy (less service costs, fewer chances for human error) overprovision system by 10-20%, software swaps in spares cheaper hardware (lower hardware costs) it consumes less power and floor space (lower environmental costs) Cost per gigabyte is significantly below that of traditional storage controllers use desktop drives lower network costs

22 Impact of CIB on Storage Networks Today Current Direction (more complex) Application Application CIB (simpler) Application Host Host SN Host Host Aggregation appliance Host... Aggregation appliance Host Host Brick Host Brick Host Intelligent Intelligent Brick Brick Disk array Disk array Servers Switches Storage Bricks Servers Virtualization appliances Switches IBM Large TotalStorage Controllers Servers Storage Bricks Copyright IBM IBM Corporation 2001