SFA12KX and Lustre Update
|
|
- Aldous Campbell
- 6 years ago
- Views:
Transcription
1 Sep 2014 SFA12KX and Lustre Update Maria Perez Gutierrez HPC Specialist HPC Advisory Council
2 Agenda SFA12KX Features update Partial Rebuilds QoS on reads Lustre metadata performance update 2
3 SFA12KX Features Update 3
4 Big Data & Cloud Infrastructure DDN FY14 Focus Areas Big Data Platform Management DirectMon * GA = Future Release Analytics Reference Architectures SATA FY14 SFA12KX/E 40GB/s/1.7M IOPS 1,680 Drives supported Embedded Computing Petascale Lustre Storage EXAScaler 10Ks of Clients 1TB/s+, HSM Linux HPC Clients NFS SFA GB/s, 600K IOPS 7700x 7700 E* Enterprise Scale-Out File Storage GRIDScaler ~10K Clients 1TB/s+, HSM Linux/Windows HPC Clients NFS & CIFS Storage Fusion Architecture Core Storage Platforms Flexible Drive Configuration SAS SSD Infinite Memory Engine Distributed File System Buffer Cache* [Demo] SFX Automated Flash Caching Read Context Commit Instant Commit WolfCreek FY14 FY14 Cloud Foundation S3/Swift Cloud Tiering WOS Trillion Unique Objects Geo-Replicated Cloud Storage 256 Million Objects/Second Self-Healing Cloud Parallel Boolean Search WOS Drives in 4U Self-Contained Servers Highlights Platform SFA Hardening Higher speed Embedded WolfCreek 7700E* 7700x* Full speed on IME PFS acceleration More use cases under review WOS S3/Swift WOS Access Cost reduction Performance improvements DDN Confidential NDA Required, Roadmap Subject to Change
5 SFA 12K Family addition: SFA12KX High-Scale Active/Active Block Storage Appliance, Available May 2014 Specifications Appliances CPU Memory & Battery-Backed Cache SFX Flash cache Active/Active Block Controllers Dual Socket Intel 8Core Ivy-Bridge 128GB DDR GB Mirrored Up to 12.8 TB Write Intensive SSDs SFA12KX with 20x SS8460 Enclosures RAID Levels RAID 1/5/6 IB Host Connectivity 16 x 56Gb (FDR) FC Host Connectivity 32 x 16Gb/s Drive Support 2.5 and 3.5 SAS, SATA, SSD Max JBOD Expansion Up to 20 Supported JBODs SS7000, SS8460 SFA12KX GA: Q Bandwidth (GB/s) Write* Read* * Large Block, Full Stripe ** Read I/O Includes Parity Verification 30 Infiniband SRP * Estimated Performance DDN Confidential NDA Required, Roadmap Subject to Change
6 Introducing 12KX (Q2 2014) Rate (MB/s) 25,000 20,000 15,000 10,000 5,000 0 Random Write Throughput Pool Count Sequential Write Throughput 512K I/O Size 1M I/O Size 2M I/O Size 4M I/O Size 8M I/O Size ü Over 40 GB/s Reads AND Writes ü Over 20 GB/s MIRRORED Writes ü Linear Scalability ü SFX Ready ü Latest Intel Processor ü 32 Ports FC-16 Rate (MB/s) 45,000 40,000 35,000 30,000 25,000 20,000 15,000 10,000 5, Pool Count 64K I/O Size 256K I/O Size 512K I/O Size 1M I/O Size 2M I/O Size 4M I/O Size 8M I/O Size
7 SFA Feature Baseline Priority, Queuing, Real-Time QoS for Read Operations; Critical for Striped File Performance Consistency ReACT IO Fairness Lowest Latency for Small IO Highest Bandwidth for Large & Streaming I/O Dialed-in Host I/O priority during rebuilds Storage Fusion Fabric Highly-Over-Provisioned Backend - RAIDed Fabric Withstands Failures of Drives, Enclosures, Cabling, etc Performance, Flash Real-Time, Multi-CPU RAID Engine Interrupt-Free; Massively Parallel I/O Delivery System RAID Rebuild Acceleration SFX Cache, Automated, data-driven caching system enabling hybrid SSD & HDD systems Reliabiltiy, Availability & Monitoring - Mgmt Raid 10 (8+2) Fast Pool Rebuilds partial rebuilds Rebuild priority adjustable SSD Life-counter High Density Array Up to 840 HDDs in a single rack. 84 HDDs / SSDs in 4U Data Integrity & Security DirectProtect RAID1/5/6 Real-Time Data Integrity Verification for every I/O SED management Data at-rest encryption of all user data. Instant Crypto Erase DDN Confidential NDA Required, Roadmap Subject to Change
8 SFA Partial Rebuild Example persistent partial rebuild Complete enclosure removed for F/W upgrade. Controllers send I/O destined for failed enclosure to available drives. Holds corresponding metadata in synchronous mirrored cache. Controller 1 fails. I/O continues. Complete system outage due to power failure. controller 2 writes cache to onboard drives. Power restored. Controller cache restored. Upgraded Enclosure undergoes partial rebuild of cached data in minutes. I/O I/O I/O I/O 84 * 4TB Disks (336TB) removed for hours, rebuilt in minutes
9 SFA Quality of Service Read retry timeouts coupled with DirectProtect DIF Raid6 (8+2) of NL-SAS Disks, one of the member has ~100% higher avg latencies than others. Production not impacted thanks to DDN s QoS
10 Lustre Metadata Benchmark and Performance / How to scale on Lustre metadata performance 10
11 Lustre Metadata Performance Lustre metadata is a crucial performance metric for many Lustre user LU-56 SMP Scaling (Lustre-2.3) DNE (Lustre-2.4) Metadata performance is related to small file performance on Lustre But, metadata performance is still a little mysterious J Performance differentiation by metadata type and access patterns? What is the impact of hardware resources for metadata performance? This presentation: use standard metadata benchmark tools to analyze metadata performance on Lustre today
12 Lustre Metadata Benchmark Tools mds-survey Build into Lustre code Similar to obdfilter-suvey Generates loads on MDS to simulate Lustre metadata performance mdtest Major metadata benchmark tool used by many large HPC sites Runs on clients using MPI Several metadata operation and access patterns are supported
13 Single Client Metadata Performance Limitation Single client Metadata performance does not scale with threads. ops/sec Single Client Metadata Performance (Unique) File creation File stat File removal Number of Threads lustre/include/lustre_mdc.h /** * Serializes in-flight MDT-modifying RPC requests to preserve idempotency. * * This mutex is used to implement execute-once semantics on the MDT. * The MDT stores the last transaction ID and result for every client in * its last_rcvd file. If the client doesn't get a reply, it can safely * resend the request and the MDT will reconstruct the reply being aware * that the request has already been executed. Without this lock, * execution status of concurrent in-flight requests would be * overwritten. * * This design limits the extent to which we can keep a full pipeline of * in-flight requests from a single client. This limitation could be * overcome by allowing multiple slots per client in the last_rcvd file. */ struct mdc_rpc_lock { /** Lock protecting in-flight RPC concurrency. */ struct mutex rpcl_mutex; /** Intent associated with currently executing request. */ struct lookup_intent *rpcl_it; /** Used for MDS/RPC load testing purposes. */ int rpcl_fakes; }; Client can send many metadata requests to MDS simultaneously, but MDS needs to store each client's last transaction ID and it's serialized. LU-5319 supports multiple slots per client in last_rcvd file (Under development by Intel and Bull).
14 Modified mdtest for Lustre Basic Function Supports multiple mount points on a single client Helps generating heavy metadata load from single client Background Originally developed by Liang Zhen for LU-56 work We rebased and cleaned up codes and made few enhancements Enables metadata benchmarks on a small number of clients Regression testing MDS server sizing Performance optimization
15 Performance Comparison Single Lustre client mounts /lustre_0, /lustre_1,... /lustre_31 for single filesystem # mdtest n u d /lustre_{0-15} Single Client Metadata Performance (Unique, single mountpoint) Single Client Metadata Performance (Unique, multimountpoints) File creation File stat File removal File creation File stat File removal ops/sec ops/sec Number of Threads Number of Threads
16 Benchmark Configuration Client MDS OSS 2 x MDS(2 x E5-2676v2, 128GB memory) 4 x OSS(2 x E5-2680v2, 128GB memory) 32 x Client(2 x E5-2680, 128GB memory) SFA12K x NL-SAS for OST 8 x SSD for MDT Lustre-2.5 for Servers Lustre , Lustre for Client RP0 RP1 RP0 RP1
17 Metadata Benchmark Method Tested Metadata Operations Directory/File Creation Directory/File Stat Directory/File Removal Access patterns To Unique Directory and shared directory o P0 -> /lustre/dir0/file.0.0, P1 -> /lustre/dir1/file.0.1 (Unique) o P0 -> /lustre/dir/file.0.0, P1 -> /lustre/dir/file.1.0 (Shared)
18 Lustre Metadata Performance Impact MDS's CPU speed Metadata Performance comparison (Unique Directory) 32 clients(1024 mount points), 1024 processes, 1.28M Files Tested on 16 CPU cores with 2.1, 2.5, 2.8, 3.3 and 3.6GHz CPU Speed (MDS) Directory Operation(Unique) File Operation(Unique) Dir Creation Dir Stats Dir Removal File Creation Fie Stats File Removal 180% 180% 160% 140% 120% 100% 20% 38% 160% 140% 120% 100% 60% 38% 80% 80% 60% 60% 40% 40% 20% 20% 0% 2.1GHz 2.5GHz 2.8GHz 3.3GHz 3.6GHz 70% 0% 2.1GHz 2.5GHz 2.8GHz 3.3GHz 3.6GHz
19 Lustre Metadata Performance Impact MDS's CPU speed Metadata Performance comparison (Shared Directory) 32 clients(1024 mount points), 1024 processes, 1.28M Files Tested on 16 CPU cores with 2.1, 2.5, 2.8, 3.3 and 3.6GHz CPU Speed (MDS) Directory Operation(Shared) Dir Creation Dir Stats Dir Removal File Operation(Shared) File Creation Fie Stats File Removal 180% 180% 160% 140% 22% 120% 50% 160% 140% 120% 58% 30% 100% 100% 80% 80% 60% 60% 40% 40% 20% 20% 0% 2.1GHz 2.5GHz 2.8GHz 3.3GHz 3.6GHz 0% 2.1GHz 2.5GHz 2.8GHz 3.3GHz 3.6GHz
20 Lustre Metadata Performance Impact MDS's CPU Cores Metadata Performance comparison (Unique Directory) 32 clients(1024 mount points), 1024 processes, 1.28M Files Tested on 3.3GHz CPU speed with 8, 12 and 16 CPU cores w/wo logical processors Directory Operation(Unique) Dir Creation Dir Stats Dir Removal File Operation(Unique) File Creation Fie Stats File Removal 250% 250% 200% 200% % 150% 150% 100% 25% 100% 50% 50% 0% 0% 100%
21 Lustre Metadata Performance Impact MDS's CPU Cores Metadata Performance comparison (Shared Directory) 32 clients(1024 mount points), 1024 processes, 1.28M Files Tested on 3.3GHz CPU speed with 8, 12 and 16 CPU cores w/wo logical processors Directory Operation(Shared) File Operation(Shared) 160% 140% Dir Creation Dir Stats Dir Removal 180% 160% File Creation Fie Stats File Removal Creation(Sshare) and Stat do not scale No scale on 12->16CPU 120% 100% 80% 60% 140% 120% 100% 80% 60% 60% 40% 40% 20% 20% 0% 0%
22 Lustre Metadata Performance MDSs Scalability (Unique Directory) Lustre Metadata Scalability (Unique) (32 clients, 1024 mount points) multiple mount points ops/sec Directory Creation 100% by DNE File Creation File Removal Dir Creation Dir Removal File Creation(DNE) File Removal(DNE) Number of Threads 50% of File creation
23 Lustre Metadata Performance MDSs Scalability (Shared Directory) Lustre Metadata Scalability (Shared) (32 clients, 1024 mount points) multiple mount points 80% of Performance compared to UniqueDir patterns ops/sec Directory Creation File Creation File Removal Dir Creation Dir Removal Number of Threads
24 Lustre Metadata Performance File creation and removal for small files Creating files with actual file size (4K, 8K, 16K, 32K, 64K and 128K) (Stripe Count=1) Small File Performance (Unique Directory) (32 clients, 1024 mount points) File Creation File Read File Removal Small File Performance (Shared Directory) (32 clients, 1024 mount points) File Creation File Read File Removal ops/sec ops/sec Small file performance bounds on metadata performance, but no performance impacts with file size File Size(byte) File Size(byte)
25 Summary Observations MDS Server resources significantly affect Lustre Metadata performance Performance scales well by number of CPU core and CPU Speed in unique directory access, but not CPU bound for shared directory access pattern Collected baseline results with 16 CPU cores, but need more tests on CPU cores Performance is highly dependent on metadata access pattern Example: Directory Creation vs. File Creation With actual file size (instead of zero byte), less impact in the case of a small number of OST(e.g. up to 40 OST), but testing on large number of OSTs is needed
26 Metadata Performance: Future Work Known Issues and Optimizations Client-side metadata optimization and especially single-client metadata performance Various performance regressions in Lustre 2.5/2.6 that need to be addressed (e.g. LU5608) Areas of Future Investigation Real-world metadata use scenarios and metadata problems Real-world small-file performance (e.g. life sciences) Impact of OST data structures on real world metadata performance DNE scalability on very large systems with many MDSs/MDTs and many OSSs/OSTs
Lustre Metadata Fundamental Benchmark and Performance
09/22/2014 Lustre Metadata Fundamental Benchmark and Performance DataDirect Networks Japan, Inc. Shuichi Ihara 2014 DataDirect Networks. All Rights Reserved. 1 Lustre Metadata Performance Lustre metadata
More informationDDN. DDN Updates. Data DirectNeworks Japan, Inc Shuichi Ihara. DDN Storage 2017 DDN Storage
DDN DDN Updates Data DirectNeworks Japan, Inc Shuichi Ihara DDN A Broad Range of Technologies to Best Address Your Needs Protection Security Data Distribution and Lifecycle Management Open Monitoring Your
More information朱义普. Resolving High Performance Computing and Big Data Application Bottlenecks with Application-Defined Flash Acceleration. Director, North Asia, HPC
October 28, 2013 Resolving High Performance Computing and Big Data Application Bottlenecks with Application-Defined Flash Acceleration 朱义普 Director, North Asia, HPC DDN Storage Vendor for HPC & Big Data
More informationStore Process Analyze Collaborate Archive Cloud The HPC Storage Leader Invent Discover Compete
Store Process Analyze Collaborate Archive Cloud The HPC Storage Leader Invent Discover Compete 1 DDN Who We Are 2 We Design, Deploy and Optimize Storage Systems Which Solve HPC, Big Data and Cloud Business
More informationIntel Enterprise Edition Lustre (IEEL-2.3) [DNE-1 enabled] on Dell MD Storage
Intel Enterprise Edition Lustre (IEEL-2.3) [DNE-1 enabled] on Dell MD Storage Evaluation of Lustre File System software enhancements for improved Metadata performance Wojciech Turek, Paul Calleja,John
More information2012 HPC Advisory Council
Q1 2012 2012 HPC Advisory Council DDN Big Data & InfiniBand Storage Solutions Overview Toine Beckers Director of HPC Sales, EMEA The Global Big & Fast Data Leader DDN delivers highly scalable & highly-efficient
More informationDDN s Vision for the Future of Lustre LUG2015 Robert Triendl
DDN s Vision for the Future of Lustre LUG2015 Robert Triendl 3 Topics 1. The Changing Markets for Lustre 2. A Vision for Lustre that isn t Exascale 3. Building Lustre for the Future 4. Peak vs. Operational
More informationDDN. DDN Updates. DataDirect Neworks Japan, Inc Nobu Hashizume. DDN Storage 2018 DDN Storage 1
1 DDN DDN Updates DataDirect Neworks Japan, Inc Nobu Hashizume DDN Storage 2018 DDN Storage 1 2 DDN A Broad Range of Technologies to Best Address Your Needs Your Use Cases Research Big Data Enterprise
More informationHPC Storage Use Cases & Future Trends
Oct, 2014 HPC Storage Use Cases & Future Trends Massively-Scalable Platforms and Solutions Engineered for the Big Data and Cloud Era Atul Vidwansa Email: atul@ DDN About Us DDN is a Leader in Massively
More informationLustre2.5 Performance Evaluation: Performance Improvements with Large I/O Patches, Metadata Improvements, and Metadata Scaling with DNE
Lustre2.5 Performance Evaluation: Performance Improvements with Large I/O Patches, Metadata Improvements, and Metadata Scaling with DNE Hitoshi Sato *1, Shuichi Ihara *2, Satoshi Matsuoka *1 *1 Tokyo Institute
More informationLustreFS and its ongoing Evolution for High Performance Computing and Data Analysis Solutions
LustreFS and its ongoing Evolution for High Performance Computing and Data Analysis Solutions Roger Goff Senior Product Manager DataDirect Networks, Inc. What is Lustre? Parallel/shared file system for
More informationThe current status of the adoption of ZFS* as backend file system for Lustre*: an early evaluation
The current status of the adoption of ZFS as backend file system for Lustre: an early evaluation Gabriele Paciucci EMEA Solution Architect Outline The goal of this presentation is to update the current
More informationEnhancing Lustre Performance and Usability
October 17th 2013 LUG2013 Enhancing Lustre Performance and Usability Shuichi Ihara Li Xi DataDirect Networks, Japan Agenda Today's Lustre trends Recent DDN Japan activities for adapting to Lustre trends
More informationDDN and Flash GRIDScaler, Flashscale Infinite Memory Engine
1! DDN and Flash GRIDScaler, Flashscale Infinite Memory Engine T. Cecchi - September 21 st 2016 HPC Advisory Council 2! DDN END-TO-END DATA LIFECYCLE MANAGEMENT BURST & COMPUTE SSD, DISK & FILE SYSTEM
More informationLustre overview and roadmap to Exascale computing
HPC Advisory Council China Workshop Jinan China, October 26th 2011 Lustre overview and roadmap to Exascale computing Liang Zhen Whamcloud, Inc liang@whamcloud.com Agenda Lustre technology overview Lustre
More informationIME (Infinite Memory Engine) Extreme Application Acceleration & Highly Efficient I/O Provisioning
IME (Infinite Memory Engine) Extreme Application Acceleration & Highly Efficient I/O Provisioning September 22 nd 2015 Tommaso Cecchi 2 What is IME? This breakthrough, software defined storage application
More informationHigh-Performance Lustre with Maximum Data Assurance
High-Performance Lustre with Maximum Data Assurance Silicon Graphics International Corp. 900 North McCarthy Blvd. Milpitas, CA 95035 Disclaimer and Copyright Notice The information presented here is meant
More informationXyratex ClusterStor6000 & OneStor
Xyratex ClusterStor6000 & OneStor Proseminar Ein-/Ausgabe Stand der Wissenschaft von Tim Reimer Structure OneStor OneStorSP OneStorAP ''Green'' Advancements ClusterStor6000 About Scale-Out Storage Architecture
More informationEmerging Technologies for HPC Storage
Emerging Technologies for HPC Storage Dr. Wolfgang Mertz CTO EMEA Unstructured Data Solutions June 2018 The very definition of HPC is expanding Blazing Fast Speed Accessibility and flexibility 2 Traditional
More informationUsing DDN IME for Harmonie
Irish Centre for High-End Computing Using DDN IME for Harmonie Gilles Civario, Marco Grossi, Alastair McKinstry, Ruairi Short, Nix McDonnell April 2016 DDN IME: Infinite Memory Engine IME: Major Features
More informationA ClusterStor update. Torben Kling Petersen, PhD. Principal Architect, HPC
A ClusterStor update Torben Kling Petersen, PhD Principal Architect, HPC Sonexion (ClusterStor) STILL the fastest file system on the planet!!!! Total system throughput in excess on 1.1 TB/s!! 2 Software
More informationDDN About Us Solving Large Enterprise and Web Scale Challenges
1 DDN About Us Solving Large Enterprise and Web Scale Challenges History Founded in 98 World s Largest Private Storage Company Growing, Profitable, Self Funded Headquarters: Santa Clara and Chatsworth,
More informationHigh Capacity network storage solutions
High Performance, High Capacity network storage solutions 7 th TERENA Storage Meeting Poznan, Sept. 9, 2010 Toine Beckers: tbeckers@ddn.comcom Auke Kuiper: akuiper@ddn.com 2009 DataDirect Networks, Inc.
More informationDemonstration Milestone for Parallel Directory Operations
Demonstration Milestone for Parallel Directory Operations This milestone was submitted to the PAC for review on 2012-03-23. This document was signed off on 2012-04-06. Overview This document describes
More informationSGI Overview. HPC User Forum Dearborn, Michigan September 17 th, 2012
SGI Overview HPC User Forum Dearborn, Michigan September 17 th, 2012 SGI Market Strategy HPC Commercial Scientific Modeling & Simulation Big Data Hadoop In-memory Analytics Archive Cloud Public Private
More informationIntroducing Panasas ActiveStor 14
Introducing Panasas ActiveStor 14 SUPERIOR PERFORMANCE FOR MIXED FILE SIZE ENVIRONMENTS DEREK BURKE, PANASAS EUROPE INTRODUCTION TO PANASAS Storage that accelerates the world s highest performance and
More informationIME Infinite Memory Engine Technical Overview
1 1 IME Infinite Memory Engine Technical Overview 2 Bandwidth, IOPs single NVMe drive 3 What does Flash mean for Storage? It's a new fundamental device for storing bits. We must treat it different from
More informationLCE: Lustre at CEA. Stéphane Thiell CEA/DAM
LCE: Lustre at CEA Stéphane Thiell CEA/DAM (stephane.thiell@cea.fr) 1 Lustre at CEA: Outline Lustre at CEA updates (2009) Open Computing Center (CCRT) updates CARRIOCAS (Lustre over WAN) project 2009-2010
More informationIsilon Performance. Name
1 Isilon Performance Name 2 Agenda Architecture Overview Next Generation Hardware Performance Caching Performance Streaming Reads Performance Tuning OneFS Architecture Overview Copyright 2014 EMC Corporation.
More informationIsilon Scale Out NAS. Morten Petersen, Senior Systems Engineer, Isilon Division
Isilon Scale Out NAS Morten Petersen, Senior Systems Engineer, Isilon Division 1 Agenda Architecture Overview Next Generation Hardware Performance Caching Performance SMB 3 - MultiChannel 2 OneFS Architecture
More informationWhite Paper. EonStor GS Family Best Practices Guide. Version: 1.1 Updated: Apr., 2018
EonStor GS Family Best Practices Guide White Paper Version: 1.1 Updated: Apr., 2018 Abstract: This guide provides recommendations of best practices for installation and configuration to meet customer performance
More informationLeveraging Software-Defined Storage to Meet Today and Tomorrow s Infrastructure Demands
Leveraging Software-Defined Storage to Meet Today and Tomorrow s Infrastructure Demands Unleash Your Data Center s Hidden Power September 16, 2014 Molly Rector CMO, EVP Product Management & WW Marketing
More informationApplication Performance on IME
Application Performance on IME Toine Beckers, DDN Marco Grossi, ICHEC Burst Buffer Designs Introduce fast buffer layer Layer between memory and persistent storage Pre-stage application data Buffer writes
More informationCSCS HPC storage. Hussein N. Harake
CSCS HPC storage Hussein N. Harake Points to Cover - XE6 External Storage (DDN SFA10K, SRP, QDR) - PCI-E SSD Technology - RamSan 620 Technology XE6 External Storage - Installed Q4 2010 - In Production
More informationFeedback on BeeGFS. A Parallel File System for High Performance Computing
Feedback on BeeGFS A Parallel File System for High Performance Computing Philippe Dos Santos et Georges Raseev FR 2764 Fédération de Recherche LUmière MATière December 13 2016 LOGO CNRS LOGO IO December
More informationAndreas Dilger. Principal Lustre Engineer. High Performance Data Division
Andreas Dilger Principal Lustre Engineer High Performance Data Division Focus on Performance and Ease of Use Beyond just looking at individual features... Incremental but continuous improvements Performance
More informationLustre 2.8 feature : Multiple metadata modify RPCs in parallel
Lustre 2.8 feature : Multiple metadata modify RPCs in parallel Grégoire Pichon 23-09-2015 Atos Agenda Client metadata performance issue Solution description Client metadata performance results Configuration
More informationScalability Testing of DNE2 in Lustre 2.7 and Metadata Performance using Virtual Machines Tom Crowe, Nathan Lavender, Stephen Simms
Scalability Testing of DNE2 in Lustre 2.7 and Metadata Performance using Virtual Machines Tom Crowe, Nathan Lavender, Stephen Simms Research Technologies High Performance File Systems hpfs-admin@iu.edu
More informationTHE SUMMARY. CLUSTER SERIES - pg. 3. ULTRA SERIES - pg. 5. EXTREME SERIES - pg. 9
PRODUCT CATALOG THE SUMMARY CLUSTER SERIES - pg. 3 ULTRA SERIES - pg. 5 EXTREME SERIES - pg. 9 CLUSTER SERIES THE HIGH DENSITY STORAGE FOR ARCHIVE AND BACKUP When downtime is not an option Downtime is
More informationHitachi Virtual Storage Platform Family
Hitachi Virtual Storage Platform Family Advanced Storage Capabilities for All Organizations Andre Lahrmann 23. November 2017 Hitachi Vantara Vorweg: Aus Hitachi Data Systems wird Hitachi Vantara The efficiency
More informationEvaluation Report: Improving SQL Server Database Performance with Dot Hill AssuredSAN 4824 Flash Upgrades
Evaluation Report: Improving SQL Server Database Performance with Dot Hill AssuredSAN 4824 Flash Upgrades Evaluation report prepared under contract with Dot Hill August 2015 Executive Summary Solid state
More informationSSD Architecture Considerations for a Spectrum of Enterprise Applications. Alan Fitzgerald, VP and CTO SMART Modular Technologies
SSD Architecture Considerations for a Spectrum of Enterprise Applications Alan Fitzgerald, VP and CTO SMART Modular Technologies Introduction Today s SSD delivers form-fit-function compatible solid-state
More informationWhat is QES 2.1? Agenda. Supported Model. Live demo
What is QES 2.1? Agenda Supported Model Live demo QES-Based Unified Storage Windows Server Block File iscsi CIFS NFS QES 2.1 One Architecture & Three Configurations SSD SSD Spinning Disk Hybrid All Flash
More informationArchitecting a High Performance Storage System
WHITE PAPER Intel Enterprise Edition for Lustre* Software High Performance Data Division Architecting a High Performance Storage System January 2014 Contents Introduction... 1 A Systematic Approach to
More informationEntry-level Intel RAID RS3 Controller Family
PRODUCT Brief Entry-Level Intel RAID RS3 Controller Portfolio Entry-level Intel RAID RS3 Controller Family 12Gb/s connectivity and basic data protection RAID matters. Rely on Intel RAID. Cost-effective
More informationDELL Terascala HPC Storage Solution (DT-HSS2)
DELL Terascala HPC Storage Solution (DT-HSS2) A Dell Technical White Paper Dell Li Ou, Scott Collier Terascala Rick Friedman Dell HPC Solutions Engineering THIS WHITE PAPER IS FOR INFORMATIONAL PURPOSES
More informationMicrosoft Exchange Server 2010 workload optimization on the new IBM PureFlex System
Microsoft Exchange Server 2010 workload optimization on the new IBM PureFlex System Best practices Roland Mueller IBM Systems and Technology Group ISV Enablement April 2012 Copyright IBM Corporation, 2012
More informationSun Lustre Storage System Simplifying and Accelerating Lustre Deployments
Sun Lustre Storage System Simplifying and Accelerating Lustre Deployments Torben Kling-Petersen, PhD Presenter s Name Principle Field Title andengineer Division HPC &Cloud LoB SunComputing Microsystems
More informationExtraordinary HPC file system solutions at KIT
Extraordinary HPC file system solutions at KIT Roland Laifer STEINBUCH CENTRE FOR COMPUTING - SCC KIT University of the State Roland of Baden-Württemberg Laifer Lustre and tools for ldiskfs investigation
More informationParallel File Systems for HPC
Introduction to Scuola Internazionale Superiore di Studi Avanzati Trieste November 2008 Advanced School in High Performance and Grid Computing Outline 1 The Need for 2 The File System 3 Cluster & A typical
More informationLustre on ZFS. At The University of Wisconsin Space Science and Engineering Center. Scott Nolin September 17, 2013
Lustre on ZFS At The University of Wisconsin Space Science and Engineering Center Scott Nolin September 17, 2013 Why use ZFS for Lustre? The University of Wisconsin Space Science and Engineering Center
More informationDell Fluid Data solutions. Powerful self-optimized enterprise storage. Dell Compellent Storage Center: Designed for business results
Dell Fluid Data solutions Powerful self-optimized enterprise storage Dell Compellent Storage Center: Designed for business results The Dell difference: Efficiency designed to drive down your total cost
More informationIBM System Storage DCS3700
IBM System Storage DCS3700 Maximize performance, scalability and storage density at an affordable price Highlights Gain fast, highly dense storage capabilities at an affordable price Deliver simplified
More informationInfinite Memory Engine Freedom from Filesystem Foibles
1 Infinite Memory Engine Freedom from Filesystem Foibles James Coomer 25 th Sept 2017 2 Bad stuff can happen to filesystems Malaligned High Concurrency Random Shared File COMPUTE NODES FILESYSTEM 3 And
More informationAll-Flash High-Performance SAN/NAS Solutions for Virtualization & OLTP
All-Flash High-Performance SAN/NAS Solutions for Virtualization & OLTP All-flash configurations are designed to deliver maximum IOPS and throughput numbers for mission critical workloads and applicati
More informationRAIDIX Data Storage Solution. Clustered Data Storage Based on the RAIDIX Software and GPFS File System
RAIDIX Data Storage Solution Clustered Data Storage Based on the RAIDIX Software and GPFS File System 2017 Contents Synopsis... 2 Introduction... 3 Challenges and the Solution... 4 Solution Architecture...
More informationLustre* is designed to achieve the maximum performance and scalability for POSIX applications that need outstanding streamed I/O.
Reference Architecture Designing High-Performance Storage Tiers Designing High-Performance Storage Tiers Intel Enterprise Edition for Lustre* software and Intel Non-Volatile Memory Express (NVMe) Storage
More informationAll-Flash High-Performance SAN/NAS Solutions for Virtualization & OLTP
All-Flash High-Performance SAN/NAS Solutions for Virtualization & OLTP All-flash configurations are designed to deliver maximum IOPS and throughput numbers for mission critical workloads and applicati
More informationFhGFS - Performance at the maximum
FhGFS - Performance at the maximum http://www.fhgfs.com January 22, 2013 Contents 1. Introduction 2 2. Environment 2 3. Benchmark specifications and results 3 3.1. Multi-stream throughput................................
More informationData Management. Parallel Filesystems. Dr David Henty HPC Training and Support
Data Management Dr David Henty HPC Training and Support d.henty@epcc.ed.ac.uk +44 131 650 5960 Overview Lecture will cover Why is IO difficult Why is parallel IO even worse Lustre GPFS Performance on ARCHER
More informationDataON and Intel Select Hyper-Converged Infrastructure (HCI) Maximizes IOPS Performance for Windows Server Software-Defined Storage
Solution Brief DataON and Intel Select Hyper-Converged Infrastructure (HCI) Maximizes IOPS Performance for Windows Server Software-Defined Storage DataON Next-Generation All NVMe SSD Flash-Based Hyper-Converged
More informationDeep Learning Performance and Cost Evaluation
Micron 5210 ION Quad-Level Cell (QLC) SSDs vs 7200 RPM HDDs in Centralized NAS Storage Repositories A Technical White Paper Don Wang, Rene Meyer, Ph.D. info@ AMAX Corporation Publish date: October 25,
More informationNetApp High-Performance Storage Solution for Lustre
Technical Report NetApp High-Performance Storage Solution for Lustre Solution Design Narjit Chadha, NetApp October 2014 TR-4345-DESIGN Abstract The NetApp High-Performance Storage Solution (HPSS) for Lustre,
More informationNVMFS: A New File System Designed Specifically to Take Advantage of Nonvolatile Memory
NVMFS: A New File System Designed Specifically to Take Advantage of Nonvolatile Memory Dhananjoy Das, Sr. Systems Architect SanDisk Corp. 1 Agenda: Applications are KING! Storage landscape (Flash / NVM)
More informationHewlett Packard Enterprise HPE GEN10 PERSISTENT MEMORY PERFORMANCE THROUGH PERSISTENCE
Hewlett Packard Enterprise HPE GEN10 PERSISTENT MEMORY PERFORMANCE THROUGH PERSISTENCE Digital transformation is taking place in businesses of all sizes Big Data and Analytics Mobility Internet of Things
More informationScaling to Petaflop. Ola Torudbakken Distinguished Engineer. Sun Microsystems, Inc
Scaling to Petaflop Ola Torudbakken Distinguished Engineer Sun Microsystems, Inc HPC Market growth is strong CAGR increased from 9.2% (2006) to 15.5% (2007) Market in 2007 doubled from 2003 (Source: IDC
More informationDeep Learning Performance and Cost Evaluation
Micron 5210 ION Quad-Level Cell (QLC) SSDs vs 7200 RPM HDDs in Centralized NAS Storage Repositories A Technical White Paper Rene Meyer, Ph.D. AMAX Corporation Publish date: October 25, 2018 Abstract Introduction
More informationAn Exploration of New Hardware Features for Lustre. Nathan Rutman
An Exploration of New Hardware Features for Lustre Nathan Rutman Motivation Open-source Hardware-agnostic Linux Least-common-denominator hardware 2 Contents Hardware CRC MDRAID T10 DIF End-to-end data
More informationNew HPE 3PAR StoreServ 8000 and series Optimized for Flash
New HPE 3PAR StoreServ 8000 and 20000 series Optimized for Flash AGENDA HPE 3PAR StoreServ architecture fundamentals HPE 3PAR Flash optimizations HPE 3PAR portfolio overview HPE 3PAR Flash example from
More informationCopyright 2012 EMC Corporation. All rights reserved.
1 FLASH 1 ST THE STORAGE STRATEGY FOR THE NEXT DECADE Richard Gordon EMEA FLASH Business Development 2 Information Tipping Point Ahead The Future Will Be Nothing Like The Past 140,000 120,000 100,000 80,000
More information2008 International ANSYS Conference
2008 International ANSYS Conference Maximizing Productivity With InfiniBand-Based Clusters Gilad Shainer Director of Technical Marketing Mellanox Technologies 2008 ANSYS, Inc. All rights reserved. 1 ANSYS,
More informationMAHA. - Supercomputing System for Bioinformatics
MAHA - Supercomputing System for Bioinformatics - 2013.01.29 Outline 1. MAHA HW 2. MAHA SW 3. MAHA Storage System 2 ETRI HPC R&D Area - Overview Research area Computing HW MAHA System HW - Rpeak : 0.3
More informationHPE Scalable Storage with Intel Enterprise Edition for Lustre*
HPE Scalable Storage with Intel Enterprise Edition for Lustre* HPE Scalable Storage with Intel Enterprise Edition For Lustre* High Performance Storage Solution Meets Demanding I/O requirements Performance
More informationData Movement & Tiering with DMF 7
Data Movement & Tiering with DMF 7 Kirill Malkin Director of Engineering April 2019 Why Move or Tier Data? We wish we could keep everything in DRAM, but It s volatile It s expensive Data in Memory 2 Why
More informationAn Overview of Fujitsu s Lustre Based File System
An Overview of Fujitsu s Lustre Based File System Shinji Sumimoto Fujitsu Limited Apr.12 2011 For Maximizing CPU Utilization by Minimizing File IO Overhead Outline Target System Overview Goals of Fujitsu
More informationAtrato SOLVE - Scalable Offload Logical Volume Engine
Atrato SOLVE - Scalable Offload Logical Volume Engine Dr. Sam Siewert 1 Atrato Velocity Series Atrato Virtualization Software (AVS) New Levels of Storage Virtualization Application intelligence and autonomics
More informationLUG 2012 From Lustre 2.1 to Lustre HSM IFERC (Rokkasho, Japan)
LUG 2012 From Lustre 2.1 to Lustre HSM Lustre @ IFERC (Rokkasho, Japan) Diego.Moreno@bull.net From Lustre-2.1 to Lustre-HSM - Outline About Bull HELIOS @ IFERC (Rokkasho, Japan) Lustre-HSM - Basis of Lustre-HSM
More informationNetwork Request Scheduler Scale Testing Results. Nikitas Angelinas
Network Request Scheduler Scale Testing Results Nikitas Angelinas nikitas_angelinas@xyratex.com Agenda NRS background Aim of test runs Tools used Test results Future tasks 2 NRS motivation Increased read
More informationAccelerating Spectrum Scale with a Intelligent IO Manager
Accelerating Spectrum Scale with a Intelligent IO Manager Ray Coetzee Pre-Sales Architect Seagate Systems Group, HPC 2017 Seagate, Inc. All Rights Reserved. 1 ClusterStor: Lustre, Spectrum Scale and Object
More informationStorage Update and Storage Best Practices for Microsoft Server Applications. Dennis Martin President, Demartek January 2009 Copyright 2009 Demartek
Storage Update and Storage Best Practices for Microsoft Server Applications Dennis Martin President, Demartek January 2009 Copyright 2009 Demartek Agenda Introduction Storage Technologies Storage Devices
More informationDeploy a High-Performance Database Solution: Cisco UCS B420 M4 Blade Server with Fusion iomemory PX600 Using Oracle Database 12c
White Paper Deploy a High-Performance Database Solution: Cisco UCS B420 M4 Blade Server with Fusion iomemory PX600 Using Oracle Database 12c What You Will Learn This document demonstrates the benefits
More informationOracle Performance on M5000 with F20 Flash Cache. Benchmark Report September 2011
Oracle Performance on M5000 with F20 Flash Cache Benchmark Report September 2011 Contents 1 About Benchware 2 Flash Cache Technology 3 Storage Performance Tests 4 Conclusion copyright 2011 by benchware.ch
More informationLustre * Features In Development Fan Yong High Performance Data Division, Intel CLUG
Lustre * Features In Development Fan Yong High Performance Data Division, Intel CLUG 2017 @Beijing Outline LNet reliability DNE improvements Small file performance File Level Redundancy Miscellaneous improvements
More informationArchitecting Storage for Semiconductor Design: Manufacturing Preparation
White Paper Architecting Storage for Semiconductor Design: Manufacturing Preparation March 2012 WP-7157 EXECUTIVE SUMMARY The manufacturing preparation phase of semiconductor design especially mask data
More informationNST6000 UNIFIED HYBRID STORAGE. Performance, Availability and Scale for Any SAN and NAS Workload in Your Environment
DATASHEET TM NST6000 UNIFIED HYBRID STORAGE Performance, Availability and Scale for Any SAN and NAS Workload in Your Environment UNIFIED The Nexsan NST6000 unified hybrid storage system is ideal for organizations
More informationFast and Easy Persistent Storage for Docker* Containers with Storidge and Intel
Solution brief Intel Storage Builders Storidge ContainerIO TM Intel Xeon Processor Scalable Family Intel SSD DC Family for PCIe*/NVMe Fast and Easy Persistent Storage for Docker* Containers with Storidge
More informationNext Generation Computing Architectures for Cloud Scale Applications
Next Generation Computing Architectures for Cloud Scale Applications Steve McQuerry, CCIE #6108, Manager Technical Marketing #clmel Agenda Introduction Cloud Scale Architectures System Link Technology
More informationHigh Performance Computing. NEC LxFS Storage Appliance
High Performance Computing NEC LxFS Storage Appliance NEC LxFS-z Storage Appliance In scientific computing the efficient delivery of data to and from the compute is critical and often challenging to execute.
More informationDemystifying Storage Area Networks. Michael Wells Microsoft Application Solutions Specialist EMC Corporation
Demystifying Storage Area Networks Michael Wells Microsoft Application Solutions Specialist EMC Corporation About Me DBA for 7+ years Developer for 10+ years MCSE: Data Platform MCSE: SQL Server 2012 MCITP:
More informationApplying DDN to Machine Learning
Applying DDN to Machine Learning Jean-Thomas Acquaviva jacquaviva@ddn.com Learning from What? Multivariate data Image data Facial recognition Action recognition Object detection and recognition Handwriting
More informationHCI: Hyper-Converged Infrastructure
Key Benefits: Innovative IT solution for high performance, simplicity and low cost Complete solution for IT workloads: compute, storage and networking in a single appliance High performance enabled by
More informationIBM System Storage DS8870 Release R7.3 Performance Update
IBM System Storage DS8870 Release R7.3 Performance Update Enterprise Storage Performance Yan Xu Agenda Summary of DS8870 Hardware Changes I/O Performance of High Performance Flash Enclosure (HPFE) Easy
More informationCold Storage: The Road to Enterprise Ilya Kuznetsov YADRO
Cold Storage: The Road to Enterprise Ilya Kuznetsov YADRO Agenda Technical challenge Custom product Growth of aspirations Enterprise requirements Making an enterprise cold storage product 2 Technical Challenge
More informationEfficient Object Storage Journaling in a Distributed Parallel File System
Efficient Object Storage Journaling in a Distributed Parallel File System Presented by Sarp Oral Sarp Oral, Feiyi Wang, David Dillow, Galen Shipman, Ross Miller, and Oleg Drokin FAST 10, Feb 25, 2010 A
More informationArchitected for Performance. NVMe over Fabrics. September 20 th, Brandon Hoff, Broadcom.
Architected for Performance NVMe over Fabrics September 20 th, 2017 Brandon Hoff, Broadcom Brandon.Hoff@Broadcom.com Agenda NVMe over Fabrics Update Market Roadmap NVMe-TCP The benefits of NVMe over Fabrics
More information<Insert Picture Here> Oracle Storage
Oracle Storage Jennifer Feng Principal Product Manager IT Challenges Have Not Slowed Increasing Demand for Storage Capacity and Performance 2010 New Digital Data ( Replicated (¼ Created,
More informationIT Certification Exams Provider! Weofferfreeupdateserviceforoneyear! h ps://
IT Certification Exams Provider! Weofferfreeupdateserviceforoneyear! h ps://www.certqueen.com Exam : 000-115 Title : Storage Sales V2 Version : Demo 1 / 5 1.The IBM TS7680 ProtecTIER Deduplication Gateway
More informationThe Future of Interconnect Technology
The Future of Interconnect Technology Michael Kagan, CTO HPC Advisory Council Stanford, 2014 Exponential Data Growth Best Interconnect Required 44X 0.8 Zetabyte 2009 35 Zetabyte 2020 2014 Mellanox Technologies
More informationIBM Storwize V7000 Unified
IBM Storwize V7000 Unified Pavel Müller IBM Systems and Technology Group Storwize V7000 Position Enterprise Block DS8000 For clients requiring: Advanced disaster recovery with 3-way mirroring and System
More informationImproved Solutions for I/O Provisioning and Application Acceleration
1 Improved Solutions for I/O Provisioning and Application Acceleration August 11, 2015 Jeff Sisilli Sr. Director Product Marketing jsisilli@ddn.com 2 Why Burst Buffer? The Supercomputing Tug-of-War A supercomputer
More information