Nový IBM Storwize V7000 Unified block-file storage system Simon Podepřel Storage Sales simon_podeprel@cz.ibm.com
Agenda V7000 Unified Overview IBM Active Cloud Engine for V7kU 2
Overview V7000 Unified
Just about one year after first GA, we will be delivering the next major step forward in the evolution of IBM Storage 4
Technology Integration Continues IBM brings you the IBM Storwize V7000 Unified Storage System Snapshots Innovative GUI SAN Easy Tier Virtualization RAID V7000 Unified V7000 Unified IBM Storwize V7000 Unified 5 IBM Active Cloud Engine Clustered Systems File Support
Introducing IBM Storwize V7000 Unified A unified storage system with tightly integrated management for both block and file capabilities Support for NFS/CIFS/FTP/HTTPS/SCP file protocols in addition to existing block functions (FCP and iscsi) File replication and file level snapshots for business continuity and disaster recovery in addition to existing block functions V7000 Unified V7000 Unified 6 6
Storwize V7000 Unified System Components What is the Storwize V7000 Unified (V7kU) Traditional Storwize V7000 (V7k) Up to 9 expansion enclosures 240 disk drives maximum All V7k supported back-end storage subsystems File Modules Quantity of 2 Minimum V7kU configuration requires 6U Storwize V7000 Unified General Availability November 30, 2011 7
Storwize V7000 Unified System Components Cont. 8 8 The V7kU Includes all V7k v6.3 updates Greater flexibility for remote mirror Replicate between V7k and almost any other storage subsystem via SVC Balance remote data currency with network bandwidth cost to better meet application requirements Additional drive options 200GB and 400GB SFF SSDs More SSD options at lower cost 3TB 7.2K RPM LFF HDD 50% greater maximum system capacity Enhanced Tivoli Storage FlashCopy Manager support VMware vsphere 4 and vsphere 5 DB2, Oracle, and SAP on HP-UX Storwize V7000 General Availability November 18, 2011
Benefits to You - IBM Storwize V7000 Unified Modular, midrange disk storage that grows as your needs grow Unified model scales to 240 HDDs (ten 24 bay control enclosures) Expand Easily Base model scales to 480 HDDs (twenty 24 bay control enclosures) Support for 10Gbps iscsi server attach Up to 7x improvement in iscsi throughput vs. 1Gbps iscsi Upgradable from existing Storwize V7000 systems Enterprise class capabilities in a midrange offering Virtualization, thin provisioning, FlashCopy, Easy Tier all built in Sophisticated local and remote mirroring High Performance in a Midrange Disk System Up to almost 1M IOPS Up to 82,000 IOPS for database-like 70/30 R/W workload Easy to set up and manage Innovative, intuitive GUI eliminates complexity Non-disruptive data migration included, unique in a product of this class Ushering in a new era in storage efficiency Save up to 75% of space using space efficient FlashCopy Increase disk utilization up to 30% Investment protection Virtualize external storage arrays and extend asset life IBM Active Cloud Engine TM with V7000 Unified Cloud Storage is an attractive means of delivering storage services while saving both time and cost 9 9 Start Small
IBM Storwize V7000 Clustered Systems Clients can Scale UP and/or Scale OUT as business needs demand SCALE UP to add capacity SCALE OUT to add performance Cluster two systems to provide smarter scalability Buy only as much capability as you need, when you need it Increase performance, connectivity and cache Clustered system operates and is managed as a single system Non-disruptive scalability vs. disruptive controller swaps 10 10 Note: IBM Storwize V7000 Unified does not currently support clustered systems. A maximum of 240 drives are currently supported. Note: IBM Storwize V7000 base model does support clustered systems with up to 480 drives.
IBM Storwize V7000 Unified Disk Drive Choices for Clients High Performance 240 HDDs per single system *480 HDDs per dual clustered system 2.5 Solid State Drives 200 GB, 300 GB, 400 GB 2.5 15K SAS Disk Drives 146 GB, 300 GB Medium Performance 240 HDDs per single system *480 HDDs per dual clustered system 2.5 10K SAS Disk Drives 300 GB, 450 GB, 600 GB Lowest Cost 240 HDDs per single system *480 HDDs per dual clustered system 2.5 7.2K Near-Line SAS Disk Drive 1 TB Lowest Cost 120 HDDs per single system *240 HDDs per dual clustered system 3.5 7.2K Near-Line SAS Disk Drives 2 TB, 3 TB * IBM Storwize V7000 Unified does not currently support clustered systems. IBM Storwize V7000 base model does support clustered systems. 11
IBM Storwize V7000 Unified - Virtualized Storage All IBM Storwize V7000 Unified storage is virtualized Enables rapid, flexible provisioning and simple configuration changes Enables non disruptive movement of data among tiers of storage, including Easy Tier Enables data placement optimization to improve performance Storwize V7000 Unified also virtualizes external Fibre Channel storage Capacity from existing storage systems becomes part of Storwize V7000/U system Single user interface to manage all storage, regardless of vendor: Designed to significantly improve productivity Virtualized storage inherits all Storwize V7000 Unified rich function including FlashCopy, Easy Tier, and thin provisioning Helps improve existing storage utilization External Virtualization Improve availability by practically eliminating storage-related causes of application downtime Migrate data from external storage to Storwize V7000 Unified storage or keep external storage as part of Storwize V7000 Unified system long-term Extends life and enhances value of existing storage assets Supports storage systems from IBM, EMC, HP, 3PAR, Sun/Oracle, Hitachi, NetApp, Fujitsu, Bull, Compellent, NEC, Pillar, and more 12 12
IBM Storwize V7000 Unified Integrates Storage Administration Integrates file and block applications in a single storage system Adds supports for file access protocols - NFS, CIFS, FTP, HTTPS, SCP Supports file replication and snapshots for business continuity 13 13
The V7kU File Modules Additional hardware and software on top of the V7k File Modules (FMs) provide file services CIFS, FTP, HTTP(S), NFS, and SCP Software stack originates from the Scale Out NAS (SONAS) product Large system made of many racks of nodes and storage Highly scalable clustered solution Roots in the Scale Out File Services (SoFS) project 14 14
A Glance at SONAS to Better Understand the V7kU The SONAS system is comprised of 3 types of nodes or roles Node Management Node Storage Node The interface node provides the connections to the customer s IP network for file serving capabilities (CIFS, NFS, HTTPS, FTP). The management node provides the user interface for configuring, administering and monitoring the SONAS system. The storage node provides the connection to the backend storage SONAS R1.3 combines the management and interface roles into 1 node pair, yet allows for many more interface nodes Highly scalable The V7kU shrinks the SONAS system further by combining all 3 roles into a single node pair 2 File Modules 15 15
V7kU Connections 10 GbE NAS IO Customer s Switch GbE GPFS Sync Direct Cable Connections GbE Management Customer s switch 8Gb Fibre Channel Direct Attach (No Switch) SAS 4 lanes at 6 Gbps per strand (24 Gbps), 2 strands per SAS Chain (48 Gbps), 2 SAS chains (96 Gbps), Totaling 600 MBps full duplex 16 16
Requirements for customer infrastructure Authentication server (choose one) Windows Active Directory (with optional SFU or NIS) LDAP (Lightweight Directory Access Protocol) Samba PDC (optional with NIS) Load Balancing/Name Resolution DNS round-robin, each incoming request allocated to next available IP adress on a file module Time server for synchronization NTP service required to synchronize file modules with authentication services Network Connectivity Ports: 4 x 1 GbE for File Modules, min. 2 x 1 GbE for V7000 4 x IP adresses for File Modules and V7000 control canisters Management: 1 x IP adress for Unified cluster and min. 1x IP adress for V7000 cluster Optional 10GbE for file service (3x) or iscsi attachment V7000 (2x) Min 1x public IP adress per interface for file services (recommended: 2x) Optional: Backup server: choice of NDMP or TSM TSM HSM 17 Antivirus scan engine
V7kU Environment Customer Provided Services Authentication Server Domain Name Server Network Time Server (Optional) Remote V7kU File Modules V7kU Software delivered as a single IBM program product All needed V7kU software components are packaged, integrated, tested, & pre-installed at the factory on V7kU hardware Network Shared Disk Via Fibre Channel * * TSM/HSM or NDMP over Ethernet only 18 18
V7kU Installation V7k USB key process has been extended to cover entire V7kU Process flow: On Windows workstation enter IP addresses for V7000 and File Modules Plug USB key into V7k Wait for blinking light to stop Remove USB key Then plug USB key into one of File Modules Wait for blue blinking lights to go off Remove USB key Return USB key to Windows workstation to confirm results USB key configures network addressing between V7k & V7kU Allows them to refer to and address each other USB key also carries credential (ssh key) from V7k to File Module Enables secure communications between the two Utilizes the NasUser role 19 19
File System Volume Management Commands provided to configure volumes for file system use mkdisk, mkfs (for GPFS) These commands will perform multiple steps automatically Configure volume on V7k (mkvdisk) Create NSD in GPFS Add NSD to GPFS file system File system volumes are marked with the file system to which they belong Known to V7k layer and reported in Volumes panels of the GUI Volumes that are in a file system have a restricted command set According to those functions that are supported and work as needed No FlashCopy, Metro Mirror or Global Mirror Cannot be thin provisioned Cannot use volume mirroring No migration between storage pools or to image mode Cannot be resized 20 20
Management Integration V7kU presents a single management interface to cover all aspects of the system Graphical User (GUI) hosted on File Module provides access to all features Command Line (CLI) hosted on File Module forwards requests to File or Block layers as needed 21 21
Management Access All management access is via the V7kU V7k commands generated in V7kU then forwarded to the V7k V7K GUI elements integrated into V7kU GUI 22 22
The Graphical User (GUI) 23 23
IBM Active Cloud engine for V7kU
IBM Active Cloud Engine for Storwize V7000 Unified Policy-based File Administration Optimize performance by placing files onto the most appropriate storage pool Lower storage costs over time by moving inactive files to archive disk or tape, and deleting unwanted, expired files Improve administrator productivity by automating file management Improve data protection with policy-based file backup Files Storage Pools Migrate inactive data to tape, tape lib, or de-duplication device via TSM Server 25
IBM General Parallel File System IBM General Parallel File System (GPFS) Clustered file system across multiple hardware nodes Basis of the SONAS technology which the V7kU leverages In continuous development since 1995 Developed for High Performance Computing (HPC) Has run on 3900 nodes with 4 PBs of storage GPFS provides Concurrent high-speed file access to applications executing on multiple nodes of clusters Single flat file system V7kU in the simplest sense is a two node SONAS cluster with a V7k providing disks 26 26
Network Shared Disks GPFS file systems span across multiple physical disks These disks are called Network Shared Disks (NSDs) Storage pools are logical collections of disks Each storage pool contains disks with similar performance characteristics V7kU has a V7k providing disks Both V7k internal and virtualized external storage The NSDs that are visible to GPFS are V7k Vdisks Usually made up of multiple disks 27 27
Fastest disk pool in the file system is Pool 1. In a file system with tiered pools, this is SAS pool. Pool 2 is a customer named user pool If HSM is used, Pool 3 is the external pool, which is managed by the external TSM Server 28 SAS NSD SAS NSD TSM Server FILE SYSTEM NL SAS NSD NL SAS NSD NL SAS NSD POOL1 SAS NSD SAS NSD POOL2 POOL3 NL SAS NSD NL SAS NSD NL SAS NSD EXTERNAL POOL Physical or Virtual Tape Storage V7kU Storage Pools Perform complex operations such as moving, mirroring, or deleting files; providing storage virtualization and a single management context. Partition file system storage for considerations such as: Improved price-performance by matching the cost of storage to the value of the data Improved performance by: Reducing the contention for premium storage Reducing the impact of slower devices Allowing you to retrieve archived data when needed Improved reliability by providing for: Replication based on need Better failure containment Creation of new storage pools as needed Two types of storage pools: Internal Storage Pools Max of 8 per file system 1 required as the system storage pool 7 for user pools External Storage Pools Intended for use as near-line storage 28
Create and write files /home /appl IBM Scale Out NAS/V7kU Policy Engine /data /web Workload autobalanced across all interface nodes Storage nodes nodes /home/appl/data/web/important_big_spreadsheet.xls /home/appl/data/web/big_architecture_drawing.ppt /home/appl/data/web/unstructured_big_video.mpg Global Namespace nodes nodes >.. Storage.. Storage... nodes nodes > scale out scale out Note: all three files, in same directory, but each allocated to different physical storage pool Data striped across all disks in storage pool. High performance, auto-tuning, autoload balancing Logical Physical Tier 1: SAS drives Tier 2: 1TB SATA drives Tier 3: 2TB SATA drives 29
Write file one user scale out more disk drives /home /appl IBM Scale Out NAS/V7kU Policy Engine /data /web node Global Namespace /home/appl/data/web/important_big_spreadsheet.xls > scale node node Storage.. Storage.. Storage... nodes nodes nodes > scale Write performance is aggregate of all disk drives in storage pool More disks and storage pods provides more performance out out Logical Physical Tier 1 Tier 3 30
Read files one user /home /appl IBM Scale Out NAS /V7kU Policy Engine /data Storage nodes /web Global Namespace nodes /home/appl/data/web/important_big_spreadsheet.xls nodes nodes >.. Storage.. Storage... nodes nodes > Parallel read of file, performance is aggregate all disk drives in that storage pool. scale out scale out Logical Physical Tier 1 Tier 2 Tier 3 31
Read files multiple users in parallel /home /appl IBM Scale Out NAS/V7kU Policy Engine /data Storage nodes /web nodes Global Namespace /home/appl/data/web/important_big_spreadsheet.xls /home/appl/data/web/big_architecture_design.ppt /home/appl/data/web/unstructured_big_video.mpg nodes nodes >.. Storage.. Storage... nodes nodes > scale out scale out Parallel streaming reads of multiple files to multiple users Logical Physical Tier 1 Tier 2 Tier 3 32
Read files one user scale out more disk drives /home /appl IBM Scale Out NAS/V7kU Policy Engine /data Storage nodes /web nodes Global Namespace /home/appl/data/web/important_big_spreadsheet.xls nodes nodes >.. Storage.. Storage... nodes nodes > Read performance is aggregate of all disk drives in storage pool. More disks and storage pods provides more performance scale out scale out Logical Physical Tier 1 Tier 3 33
Push asynchronous updates to remote site /home /appl /home/appl/data/web/important_big_spreadsheet.xls /home/appl/data/web/big_architecture_drawing.ppt /data /home/appl/data/web/unstructured_big_video.mpg /web IBM Scale Out NAS/V7kU 3. Parallel transmit to remote site(s) 1. Read policies Global Namespace hash Remote Scale Out NAS Hash determines incremental changes to send Policy Engine 2. Scan, hash compare node Storage node node Storage node hash Remote Scale Out NAS Tier 1: SAS drive Tier 2: SATA drives 34
Scan engine reads Storage Policies Automated Tiered Storage policy statement examples: Migration policies, evaluated periodically rule 'cleangold' migrate from pool TIER1' threshold (90,70) to pool TIER2 rule 'hsm' migrate from pool TIER3' threshold(90,85) weight(current_timestamp access_time) to pool HSM' where file_size > 1024kb rule 'cleansilver' when day_of_week()=monday migrate from pool 'silver' to pool 'bronze' where access_age > 30 days Deletion policies, evaluated periodically rule 'purgebronze' when day_of_month()=1 delete from pool 'bronze' where access_age>365 days There are also Policies for: File based Backup/Archive Restore/Retrieve Many more options.. 35
Scan Engine /home Scan Engine reads internal V7kU file system metadata Does not need to read the file or directory tree All nodes can participate in scan of file system /appl /data IBM Scale Out NAS/V7kU 1. Start scan /web Metadata scan Scan > 10 million files/minute per node Some or all nodes (both storage and interface) can participate in parallel scan engine Global Namespace Policy Engine 2. Read policies node nodes node node 3. Parallel Scan 4. Return results of scan Scale Out Storage nodes Storage nodes Storage nodes Storage nodes Storage nodes Storage nodes Tier 1 Tier 2 Tier 3 36 36
Data Movement /home /appl /home/appl/data/web/important_big_spreadsheet.xls /home/appl/data/web/big_architecture_drawing.ppt /data /home/appl/data/web/unstructured_big_video.mpg IBM Scale Out NAS/V7kU /web Note: all three files, no change to logical directory 6. All nodes (both storage and interface) can participate in parallel data movement 5. Carry out policy based on results of scan Policy Engine node Global Namespace nodes node node Scale Out Storage nodes Storage nodes Storage nodes Storage nodes Storage nodes Storage nodes Tier 1 Tier 2 Tier 3 37 37
Backup acceleration using Tivoli Storage Manager Scan engine results Data Tivoli Storage Manager backup server Any TSM-supported device including: ProtectTier de-dup Virtual Tape Library Tape IBM Scale Out NAS IBM V7kU has special exploitation for high-speed backup by Tivoli Storage Manager (TSM) V7kU scan engine quickly identify incremental changes in the file system, then passes the list of changed files directly to TSM Avoids need to walk the directory trees to identify changed files Reduces backup window to the time needed to copy changes to external TSM storage Parallel data movement using multiple interface nodes => to TSM Server 38
Backup / Restore using Tivoli Storage Manager /home /appl /home/appl/data/web/important_big_spreadsheet.xls /home/appl/data/web/big_architecture_drawing.ppt /data /home/appl/data/web/unstructured_big_video.mpg /web IBM Scale Out NAS/V7kU Global Namespace Tivoli Storage Manager Policy Engine 2. Pass list of changed files to TSM node node 1. Scan to identify changed files Storage node Storage node Tier 1: SAS drive Tier 2: SATA drives 3. Parallel data movement using multiple interface nodes to TSM server - backup to Tape, Virtual Tape Lib, or De-duplication 39
Hierarchical Storage Management to tape using TSM Scan engine results TSM/HSM Server Data Migrate inactive data to tape, tape lib, or de-duplication device IBM Scale Out NAS IBM V7kU provides policy-driven Hierarchical Storage Management to tape Scan engine used to identify files needing to be migrated to tape Passes list of required data blocks to be migrated directly to Tivoli Storage Manager / Hierarchical Storage Manager (TSM/HSM) Stub file is left on disk auto-recall if accessed Store petabytes of data, on terabytes of disk 40
The Power of One IBM Storwize V7000 Unified One system One management framework One GUI One unified block and file solution Storwize V7000 Unified supports a 6U block and file solution managed from one GUI IBM Active Cloud Engine TM Further enhanced GUI Enterprise-class capability High performance in a midrange disk system Integrated non disruptive migration function Integrated no-charge Easy Tier function Integrated no-charge thin provisioning function Similar benefits to SVC when virtualizing external storage Storwize V7000 Unified General Availability November 30, 2011 41 41
Thank You! Thank You For more information, please visit ibm.com/storage/storwizev7000 42 42
43