Intelligent Storage for HPC: Sun StorageTek QFS and Sun StorageTek Storage Archive Manager Harriet Coverston

Transcription

1 Intelligent Storage for HPC: Sun StorageTek QFS and Sun StorageTek Storage Archive Manager Harriet Coverston Distinguished Engineer Sun Microsystems, Inc. May, 2008

2 HPC Why It Matters to Sun What happens in HPC today often becomes mainstream beyond HPC in the years ahead > HPC drives leading-edge technologies HPC applications all have similar requirements > Consolidated Storage > Performance & Scalability > Parallel processing Expect this same phenomenon in data management > HPC is tackling HUGE data requirements/issues HPC is a growth target at SUN (one of four) Page 2

3 Sun's Advanced HPC Data Management Products Sun StorageTek QFS SAN File System > High performance parallel file system > Transparent user interface > Production ready > Sun StorageTek Storage Archive Manager (SAM) > Policy based automatic data migration and protection > Full device streaming > Tiered storage > Page 3

4 Open Source Sun opened the source for SAM and Shared QFS, metadata server and clients > Plans are to post SAM and Shared QFS 5.0 development code periodically > We are developing in the open > We are expanding our community Sun opened Libsam, the APIs which allow the user to manage data in a SAM-QFS file system from within an application program > Page 4

5 Shared QFS (SQFS) Large, existing, and loyal customer base > Stable base, shipping since Aug 2002 Targets large enterprises, grid, and HPC > Clients run on Solaris (SPARC x64 & x86) & Linux > Metadata server runs on Solaris (SPARC & x64) > HA option with SunCluster Built in HSM with SAM with optional WORM functionality for business compliance SQFS currently supports 256 nodes Next release, SQFS will support thousands of nodes > Targets HPC clusters Page 5

6 Shared QFS with SAM Integrated Data Management Client Client LAN Network Meta Data Server handles name services and space management Meta data path Data path Meta Data Server Block Storage Block Storage Client... Client n SAN Block Storage Block Storage Data is accessed directly from devices Block Storage Block Storage SL500 (Tape Library) Page 6

7 Shared QFS Customer Benefits Data consolidation with SAN file sharing > HBO 5000 hours of programming to manage > Provided the scalability to store and manage large files created by program-length video with the performance necessary to meet HBO's demanding throughput goals Performance and scalability > Tune file system to the application > Near raw I/O performance > File system I/O performance scales linearly with the hardware Parallel processing w/ multi-node read/write access SAM provides automatic & continuous data protection Page 7

8 Shared QFS Certified w/suncluster SunCluster HA failover support > Standalone QFS > HA-NFS over QFS > Shared QFS Metadata Server failover > Supports clients outside the cluster Oracle RAC runs on Shared QFS with SunCluster for high availability > Oracle certified on 9i and 10g > Shared QFS license is free for this configuration Shared QFS transactional performance matches raw Page 8

9 SAM Customer Benefits Policy based automatic data migration > Media can be disk, tape, or optical > Local or remote copies > Classification is path, owner, group, size, wildcard & access Media format is tar > Small files are put into a tar container so data is streamed at device speeds out to the tape Keeps all data available, but not on high cost storage > Moves data across the tiers according to access patterns On-demand, transparent file retrieval Continuous backup no waiting until midnight Page 9

10 Policy Driven Sam Processes Transparently Archive from disk cache to removable media without operator intervention based on policies Time based archiving Manage disk space and Release archived files from disk cache based on policies Automatically Stage released files back to disk cache when accessed Option to pre-stage and option to bypass disk cache Recycle removable media by repacking media Page 10

11 SAM Migration Facility Move from foreign HSM to SAM > Import metadata into a SAM-QFS file system > Copy foreign HSM data to SAM in the background > Production up and active during the migration process Migrated German Weather from Amass/DataMgr to SAM Sun partnered with reseller HMK > Moved 10 million files into a SAM-QFS in 8 hours > Successfully migrated 700+ TB of Amass/DataMgr to SAM in 161 days > Production operational during migration Migrations for DMF, UniTree, Amass, and Veritas Page 11

12 Support for Monitoring SAM and QFS The monitoring console(shown here) lets admins quickly understand their SAM environment Potential trouble spots are indicated by severity icons in the left hand panel. notifications can be configured to alert admins of problems with file systems, archiving and archive media System metrics provide archive media reports and file data distribution charts Faults provide a record of adverse conditions that have occurred in the system (including tape alerts) Page 12

13 SAM's Archives are OPEN! Media format is open, not proprietary tar format > Files can be recovered with or without SAM our media format is open, NOT proprietary Metadata about the data is on the archives > If file system metadata is lost, the archives can be recovered with a procedure we call the Ultimate Disaster Recovery Page 13

14 Store Data Forever! Future-proof Data Storage for Data Preservation > Archive files are self-describing, standard > No lock-in, open TAR format > Move data to newer media overtime, transparently Page 14

15 QFS Scalability for HPC 15

16 Coping with HPC Storage Complexity Increasing bandwidth requirements > 2GB/sec per TF = 2TB/sec peak for a petaflop machine > 1TB/sec sustained I/O bandwidth for a petaflop machine Increasing demand for metadata ops > Finding any one file among trillions of files > Finding anything in the petabytes of data (data mining) Extreme concurrency > HPC compute scaling means more processors which means more concurrent threads which means more concurrent I/O requests Seek Efficiency > Disk drive latency is the about the same as 1990 drives and this is the bottleneck Page 16

17 Intelligent Storage for HPC Intelligent, secure storage (T10/1729-D OSD-2) helps solve storage complexities > Move higher-level storage functions out to the devices > Execute these functions in parallel in order to scale > Support secure client access because credentials are checked on every access Data aware intelligent storage can support > Object caching and pre-fetching > Object seach > Object repair AND most important, it is a standard (ANSI T10)! Page 17

18 Object Storage Distributes space allocation > Allows bandwidth to scale up with capacity increases Knowledge of the data pushed to object storage device (OSD) > Better resource utilization in multi-host configurations > Key to multi-host quality-of-service policies > Guaranteed rate I/O, guaranteed latency, etc. Security at the OSD Standards based (ANSI T10/1355) Page 18

19 Scale SQFS with Intelligent Storage T10/OSD provides standards-based intelligent object-based storage > Version 1 ratified 2004; version 2 expected soon Intelligent storage increases horizontal scale > Space allocation moves to the storage nodes > Space allocation is done in parallel by the storage nodes > Bandwidth scales up as capacity increases Current roadmap plans include releasing Shared QFS with object-based storage end of this year in an OpenSolaris HPC distro > Plans are to support 1000s of nodes Page 19

20 History of T10 OSD at Sun May, Speaker at DTC, Intelligent Storage in Commercial HPC May, Speaker at DTC, Object Storage at Sun > DARPA Phase 2 filesystem Projects included T10 OSD petascale distributed file system, Storage Pools, and Archive Metadata Database for semantic access. May, Speaker at DTC, OpenSolaris T10 OSD Reference Implementation May, Speaker at DTC, Intelligent Storage for HPC Page 20

21 RDMA for InfiniBand Storage access bandwidth, overhead, and latency are all limited by iscsi over TCP/IP InfiniBand RDMA increases the bandwidth and reduces overhead Current roadmap plans include releasing InfiniBand RDMA support for Shared QFS end of this year, 2008 > iser initiator and target supported in Solaris Page 21

22 Scale Shared QFS Page 22

23 Scale Shared QFS Page 23

24 Intelligent Storage Components INITIATOR Shared QFS enhanced to support T10 OSD, new file system type mb Solaris initiator device driver, sosd Sun Command SCSI Architecture iscsi enhanced for iser RMDA TARGET ISCSI target enhanced for iser RDMA Common Multiprotocol SCSI Target (COMSTAR) T10 LUN provider Object QFS file system, allocators only, no namespace, new file system type mat Page 24

25 OpenSolaris.org Projects All of the object-based storage projects are being developed in the open: SAM-QFS project > OSD Project > iser Project > Common Multiprotocol SCSI target > Page 25

26 Scale Shared QFS Page 26

27 Intelligent Storage Is The Future... Further de-couple the physical storage from applications > A paradigm shift for file systems > Storage nodes become peers of compute nodes > Storage nodes can be hybrids of disks, tapes, optical, DRAM, flash memory, etc. Intelligent object-based storage enables a platform for new innovations in storage > Underlying storage technologies can evolve independently of the data that they store and the protocols that access them! Page 27

28 Harriet Coverston

29 HPC Storage Solutions What We Hear from Our Customers Scalable Compute Engine Data Server (Scratch) Long-term Retention and Archive (Archive) Maximum aggregate B/W at lowest possible costs Scalable to hundreds of Petabytes Scalable to GB/sec B/W Low latency, temporary storage, accessible by all compute nodes Ease of deployment and administration Reasonable availability and reliability Deep repository for massive amounts of data User access to data regardless of location High performance movement in and out of the Compute Engine Data Cache Ease of deployment and policy based administration Self protecting no backups Seamless Transfer Page 29

30 HPC Storage Solutions High Bandwidth Scalable Storage Cluster with Lustre Compute Cluster Long-Term Data Retention with SAM-QFS Near Line Archive Metadata Servers IB Network Load SAN Archive Object Storage Farm Data Movers Sun Microsystems, confidential, internal use only Home Directories Tier 1 Archive

31 Scalable Storage Cluster High Bandwidth Data Engine Compute Cluster Metadata Servers IB Network Load Achieves extreme scale and aggregate bandwidth required by Fusion for a low price/performance Leverages both the SunFire x4500 storage ervers & Lustre - the defacto standard HPC parallel file system Built with industry standard open technologies Infiniband, open source software High speed, low-latency interconnects with simplified cabling via Magnum Ideal for HPC cache & temp. storage Data Movers Object Storage Farm Pre-integrate systems arrive at your site ready to run Page 31

32 Long-Term Retention & Archive Staging, Storing & Maintaining HPC Data Near Line Archive Provides a massive on-line/near-line repository to compliment the Scalable Storage Cluster Leverages Sun StorageTek Tape Libraries, Modular Arrays and SAM-QFS SAN Archive Data Movers Policy driven engine to automate moving data sets in to and results out of the Object Storage Farm Enables Tape Libraries as a large near-line repository Stores data in open formats (TAR) allowing technology refresh and avoiding vendor lockin Home Directories Pre-integrate systems arrive at your site Tier 1 Archive ready to run Page 32

33 HPC Solution: Lustre + QFS + SAM > The combination of Lustre, QFS and SAM, and the Sun Fire X4500 create a truly unique and innovative storage solution that addresses the broadest range of HPC storage requirements in the industry. Page 33

34 TACC Supercomputer Storage HPC Storage Solutions Compute Engine Data Cache Long-term Retention and Archive Will scale to over Will scale to over > 72 GB/sec. sustained bandwidth > Petabytes of raw capacity > 200 Petabytes of near-line > 3.1 Petabytes of on-line Configuration includes > Configuration includes > 5 StorageTek SL8500s > 48 StorageTek T10000Bs > 10 StorageTek 6540s > 6 SunFire Metadata servers with SAM-QFS > 72 SunFire x4500s > Over 3, GB drives > 8 racks Seamless Transfer Page 34

35 Customer Snapshot: Media, Entertainment & Internet Services HBO Digital repository for its SD and HD programming Sun StorageTek QFS high-performance file-sharing software provided the scalability needed to store the large program-length video files and the powerful performance required to meet HBO's demanding throughput goals. Sun Java technology ensured cross-platform compatibility and aligned with HBO's strategy to develop its Video Network systems in the Java programming language. Page 35

36 Customer Success Story Challeng e Solutio n Benefit s Deliver secure, highly cost-effective access to user image files Consolidate management of numerous Line-of-Business storage silos Simplify storage administration while maintaining flexibility Fix the backup problem Partnered with ACS, designed a new SAM-QFS policy managed archive with integrated protection. 30 PB build-out Used extensive PS-led POC to prove performance, availability and scale Tied to Sun exclusive offerings: Link Aggregation, NIC's with TOE engines (Atlas and Neptune), X4600 movers Reduced backup time of main Photo archive from impossible to Zero 100X scale with same architecture Capital savings estimated at $400k over 2 years Media Migration no longer an issue Page 36

37 Customer Success Story Challeng e Solutio n Benefit s Ensure secure, highly available access to medical image files Consolidate management of numerous departmental application and storage silos Simplify storage administration while maintaining flexibility Partnering with Phillips, designed a new PACS system and storage infrastructure Used the SAN to drive consolidation and promote shared storage across the company Deployed a comprehensive ILM solution to take advantage of disk and tape storage tiers with SAM-FS as central mgmt. Reduced backup time of image database by 80% Improved compliance with HIPPA Capital savings of $400k over 5 years Improved image access time by 300% Page 37

38 Customer Success Story Digital Content Archiving Industry leader in video post-production Locations in US and EAME Digital Media Environment > > Implemented tiered storage solution from Sun > Managing massive amounts of shared digital content View, edit, store uncompressed data between global facilities SAM-QFS, 6540, X4500, SL8500, T10000 Streamlined digital file-based workflows > > Archiving content cost effectively Generating new revenue streams Page 38

39 Customer Success Story Digital Asset Management w/open Text Artesia > Retrieve, reuse, repurpose content realtime > Implemented in 4 months > Improved production productivity 10%-40% Page 39

40 Customer Success Story Industry: Healthcare Revenues: N/A Sun Products & Services: Storage & Services Challenge Meet HIPAA data protection requirements Centralize data storage Reduce network expense and complexity Solution NYU Medical Center implemented the Sun StorageTek llm storage infrastructure and created a tier 3 storage system to provide disaster recovery capabilities. Results Cost-effectively Running the Patient records retrievable in seconds Extended Improved collaboration among healthcare providers Enterprise ILM strategy provides automated data retention Improved data integrity and accessibility Sun StorageTek created a tailored information lifecycle management solution to meet our regulatory, business and budgetary objectives. Our PACS application performs better and is a key element to delivering better patient care and enhanced practitioner productivity. Chris Petillo, Director of PACS, NYU Medical Center