A ClusterStor update. Torben Kling Petersen, PhD. Principal Architect, HPC
|
|
- Johnathan Caldwell
- 5 years ago
- Views:
Transcription
1 A ClusterStor update Torben Kling Petersen, PhD Principal Architect, HPC
2 Sonexion (ClusterStor) STILL the fastest file system on the planet!!!! Total system throughput in excess on 1.1 TB/s!! 2
3 Software Releases - Overview New in 1.4 GridRAID (formerly called PDRAID) exclusively for Cray Sonexion 1600 (until May2014) New and Improved Monitoring Dashboard High level view into the entire storage Node Status File System Throughput Inventory Top System Statistics SSU+n Systems Containing one SSU Enclosure and Up to Three ESU Enclosures The SSU+n feature, where the maximum value for n=3, whereby up to 3 Expansion Storage Units (ESUs) can be added to each SSU GUI Guest Account A built-in "guest" account for read-only access to ClusterStor Manager NIS GUI Support Added GUI support for configuring NIS as an option for Lustre users. 3
4 Declustered Parity RAID Geometry Data 0 Data 0 Parity 0 Data 1 Data 30 Spare 0 Data 4 Data 2 Parity 1 Data 3 Data 10 2 Spare 1 Data 1 Parity 0 Parity 81 Data 3 Data 6 Parity 0 Data 0 Data 6 Parity Data 47 Parity 5 Data 7 Data 3 Data 13 2 Parity 0 Data 1 Parity 1 Data 5 Spare 2 Disk 0 Disk 1 Disk 2 Disk 3 Disk 4 Disk 5 Dis k6 Disk 7 Disk 8 Disk 9 PD-RAID array PD-RAID geometry for an array is defined as: P drive (N+K+A) example: 41 (8+2+2) P is the total number of disks in the array N is the number of data blocks per stripe K is the number of Parity blocks per stripe A is the number of distributed spare disk drives 4
5 ClusterStor Grid RAID Declustered Parity - Geometry PD RAID geometry for an array is defined as: - P drive (N+K+A) - example: 41 (8+2+2) P is the total number of disks in the array N is the number of data blocks per stripe K is the number of Parity blocks per stripe A is the number of distributed spare disk drives 5
6 ClusterStor OS Mandatory to effectively implement high capacity drives and solutions Feature ClusterStor Grid RAID De-clustered RAID 6: Up to 400% faster time to repair Rebuild of 6TB drive MD RAID ~ 33.3 hours Grid RAID ~ 9.5 hours Repeal Amdahl s Law: speed of a parallel system is gated by the performance of the slowest component Minimize file system fragmentation Traditional RAID Benefit Recover from a disk failure and return to full data protection faster Minimizes application impact to widely striped file performance Improved allocation and layout maximizes sequential data placement 4 to1 Reduction in Object Storage Targets Simplifies scalability challenges ClusterStor Integrated Management End to end CLI and GUI config, monitoring and management reduces Opex Grid RAID Parity Rebuild Disk Pool #1 Parity Rebuild Disk Pool #2 Parity Rebuild Disk Pool #3 Object Storage Server Parity Rebuild Disk Pool #1 Object Storage Server Parity Rebuild Disk Pool #4 6
7 Performance of MD-RAID (IOR) CS MD-RAID IOR with 32x client MB/s Read (MB/s) Write(MB/s) * No of Client threads (32 clients) * read Direct I/O -t=32 write Buffered I/O -t=8 7
8 Effects of Grid-RAID (IOR Direct I/O) CS Grid-RAID - IOR with 8x clients MB/s Read - 32/128 Write - 32/128 * No of Client threads (8 clients) * max_rpcs_in_flight = 32 max_dirty_mb = 128 8
9 CIFS/NFS Gateway 10/40 GbE CIFS/NFS Gateway Cluster Infiniband/40 GbE Lustre LNet Lustre Clients ClusterStor 1500/6000 9
10 Announcing ClusterStor Engineered Solution Platform ClusterStor Management Unit (CMU) HA System Management Servers HA Lustre Management Servers HA Metadata Management Servers 2 x 24 port Management Network switch 2 x 36 port FDR IB or 40GbE Data Network CS Up to 50% faster MB/s Read: 12,344 Write: 12,075 MB/sec Read: 8,553 Write: 8,514 MB/sec Scalable Storage Units (SSU) 1 Storage Unit (SSU) 1 Storage Unit (SSU) + 1 Expansion Unit (ESU) 2 x HA Storage Servers 82 x SAS drives (2x OSTs) 2 x High Capacity SSDs Grid-RAID 10
11 Initial CS9000 Single SSU Performance Results MB/s Write Read client nodes -FDR IB 4 or 8 threads per node IOR parameters: Direct I/O mode, File per process, Transfer size 64MB 11
12 ClusterStor 6000 vs 9000 Specification ClusterStor 6000 ClusterStor 9000 Object Storage Servers Sandy Bridge 8C, 1.8GHz w/ 32GB 1600MHz Ivy Bridge 10C, 1.9GHz w/ 32GB 1866MHz Enclosure 5U 84 Titan RAS Enhanced 5U 84 Titan (Side Card FRU) RAID MDRAID Only Grid RAID Only Disk 4/3/2TB 4/3/2TB (6/5TB TBD Pending Avail.) SAS Lane Config. x8 SAS per 42 HDDs x12 SAS per 42 HDDs Flash Accelerator & Journals 2x 100GB SSDs (1+1) 2 x 800GB SSDs (1+1) ESU / EBOD Exp. Yes up to 3 in the field Yes up to 1 in the field IOR Performance 6GB/sec per SSU - 4TB drives 8.5GB/sec per SSU - 4TB drives. > 9GB/sec per SSU with 5+TB drives ClusterStor Mgmt Unit (CMU) MDS/MGS Nodes 2x Sandy Bridge 8C, 2.7GHz w/ 64GB 1600MHz MGMT Nodes Single Sandy Bridge 8C, 2.7GHz w/ 32GB 1600MHz MDS/MGS Nodes 2x Ivy Bridge 10C, 3.3GHz w/ 64GB 1866MHz MGMT Nodes Single Ivy Bridge 8C, 2.6GHz w/ 32GB 1600MHz 12
13 Management tools etc.
14 ClusterStor dashboard 14
15 The company that put ENTERPRISE into Lustre Often imitated, never beaten!!
Xyratex ClusterStor6000 & OneStor
Xyratex ClusterStor6000 & OneStor Proseminar Ein-/Ausgabe Stand der Wissenschaft von Tim Reimer Structure OneStor OneStorSP OneStorAP ''Green'' Advancements ClusterStor6000 About Scale-Out Storage Architecture
More informationSeagate ExaScale HPC Storage
Seagate ExaScale HPC Storage Miro Lehocky System Engineer, Seagate Systems Group, HPC1 100+ PB Lustre File System 130+ GB/s Lustre File System 140+ GB/s Lustre File System 55 PB Lustre File System 1.6
More informationHIGH-PERFORMANCE STORAGE FOR DISCOVERY THAT SOARS
HIGH-PERFORMANCE STORAGE FOR DISCOVERY THAT SOARS OVERVIEW When storage demands and budget constraints collide, discovery suffers. And it s a growing problem. Driven by ever-increasing performance and
More informationThe current status of the adoption of ZFS* as backend file system for Lustre*: an early evaluation
The current status of the adoption of ZFS as backend file system for Lustre: an early evaluation Gabriele Paciucci EMEA Solution Architect Outline The goal of this presentation is to update the current
More informationRefining and redefining HPC storage
Refining and redefining HPC storage High-Performance Computing Demands a New Approach to HPC Storage Stick with the storage status quo and your story has only one ending more and more dollars funneling
More informationEnterprise HPC storage systems
Enterprise HPC storage systems Advanced tuning of Lustre clients Torben Kling Petersen, PhD HPC Storage Solutions Seagate Technology Langstone Road, Havant, Hampshire PO9 1SA, UK Torben_Kling_Petersen@seagate.com
More informationToward Understanding Life-Long Performance of a Sonexion File System
Toward Understanding Life-Long Performance of a Sonexion File System CUG 2015 Mark Swan, Doug Petesch, Cray Inc. dpetesch@cray.com Safe Harbor Statement This presentation may contain forward-looking statements
More informationThe ClusterStor Engineered Solution for HPC. Dr. Torben Kling Petersen, PhD Principal Solutions Architect - HPC
The ClusterStor Engineered Solution for HPC Dr. Torben Kling Petersen, PhD Principal Solutions Architect - HPC Forward Looking Statement The following information contains, or may be deemed to contain,
More informationZEST Snapshot Service. A Highly Parallel Production File System by the PSC Advanced Systems Group Pittsburgh Supercomputing Center 1
ZEST Snapshot Service A Highly Parallel Production File System by the PSC Advanced Systems Group Pittsburgh Supercomputing Center 1 Design Motivation To optimize science utilization of the machine Maximize
More informationHigh-Performance Lustre with Maximum Data Assurance
High-Performance Lustre with Maximum Data Assurance Silicon Graphics International Corp. 900 North McCarthy Blvd. Milpitas, CA 95035 Disclaimer and Copyright Notice The information presented here is meant
More informationSun Lustre Storage System Simplifying and Accelerating Lustre Deployments
Sun Lustre Storage System Simplifying and Accelerating Lustre Deployments Torben Kling-Petersen, PhD Presenter s Name Principle Field Title andengineer Division HPC &Cloud LoB SunComputing Microsystems
More informationSonexion /900 Release Notes
Sonexion 1.5 1600/900 Release Notes Contents Contents 1 About Sonexion 1.5 1600/900 Release Notes...3 2 Software Versions and Requirements...4 3 What s New for Sonexion 1.5.0 1600/900...6 4 Known Issues
More informationLustre2.5 Performance Evaluation: Performance Improvements with Large I/O Patches, Metadata Improvements, and Metadata Scaling with DNE
Lustre2.5 Performance Evaluation: Performance Improvements with Large I/O Patches, Metadata Improvements, and Metadata Scaling with DNE Hitoshi Sato *1, Shuichi Ihara *2, Satoshi Matsuoka *1 *1 Tokyo Institute
More informationSonexion GridRAID Characteristics
Sonexion GridRAID Characteristics CUG 2014 Mark Swan, Cray Inc. 1 Safe Harbor Statement This presentation may contain forward-looking statements that are based on our current expectations. Forward looking
More informationBlue Waters I/O Performance
Blue Waters I/O Performance Mark Swan Performance Group Cray Inc. Saint Paul, Minnesota, USA mswan@cray.com Doug Petesch Performance Group Cray Inc. Saint Paul, Minnesota, USA dpetesch@cray.com Abstract
More informationIntel Enterprise Edition Lustre (IEEL-2.3) [DNE-1 enabled] on Dell MD Storage
Intel Enterprise Edition Lustre (IEEL-2.3) [DNE-1 enabled] on Dell MD Storage Evaluation of Lustre File System software enhancements for improved Metadata performance Wojciech Turek, Paul Calleja,John
More informationDDN. DDN Updates. Data DirectNeworks Japan, Inc Shuichi Ihara. DDN Storage 2017 DDN Storage
DDN DDN Updates Data DirectNeworks Japan, Inc Shuichi Ihara DDN A Broad Range of Technologies to Best Address Your Needs Protection Security Data Distribution and Lifecycle Management Open Monitoring Your
More informationSFA12KX and Lustre Update
Sep 2014 SFA12KX and Lustre Update Maria Perez Gutierrez HPC Specialist HPC Advisory Council Agenda SFA12KX Features update Partial Rebuilds QoS on reads Lustre metadata performance update 2 SFA12KX Features
More informationLustreFS and its ongoing Evolution for High Performance Computing and Data Analysis Solutions
LustreFS and its ongoing Evolution for High Performance Computing and Data Analysis Solutions Roger Goff Senior Product Manager DataDirect Networks, Inc. What is Lustre? Parallel/shared file system for
More informationData Management. Parallel Filesystems. Dr David Henty HPC Training and Support
Data Management Dr David Henty HPC Training and Support d.henty@epcc.ed.ac.uk +44 131 650 5960 Overview Lecture will cover Why is IO difficult Why is parallel IO even worse Lustre GPFS Performance on ARCHER
More informationLBRN - HPC systems : CCT, LSU
LBRN - HPC systems : CCT, LSU HPC systems @ CCT & LSU LSU HPC Philip SuperMike-II SuperMIC LONI HPC Eric Qeenbee2 CCT HPC Delta LSU HPC Philip 3 Compute 32 Compute Two 2.93 GHz Quad Core Nehalem Xeon 64-bit
More informationDDN. DDN Updates. DataDirect Neworks Japan, Inc Nobu Hashizume. DDN Storage 2018 DDN Storage 1
1 DDN DDN Updates DataDirect Neworks Japan, Inc Nobu Hashizume DDN Storage 2018 DDN Storage 1 2 DDN A Broad Range of Technologies to Best Address Your Needs Your Use Cases Research Big Data Enterprise
More informationHPE Scalable Storage with Intel Enterprise Edition for Lustre*
HPE Scalable Storage with Intel Enterprise Edition for Lustre* HPE Scalable Storage with Intel Enterprise Edition For Lustre* High Performance Storage Solution Meets Demanding I/O requirements Performance
More informationIME Infinite Memory Engine Technical Overview
1 1 IME Infinite Memory Engine Technical Overview 2 Bandwidth, IOPs single NVMe drive 3 What does Flash mean for Storage? It's a new fundamental device for storing bits. We must treat it different from
More informationWelcome! Virtual tutorial starts at 15:00 BST
Welcome! Virtual tutorial starts at 15:00 BST Parallel IO and the ARCHER Filesystem ARCHER Virtual Tutorial, Wed 8 th Oct 2014 David Henty Reusing this material This work is licensed
More informationAccelerating Spectrum Scale with a Intelligent IO Manager
Accelerating Spectrum Scale with a Intelligent IO Manager Ray Coetzee Pre-Sales Architect Seagate Systems Group, HPC 2017 Seagate, Inc. All Rights Reserved. 1 ClusterStor: Lustre, Spectrum Scale and Object
More informationXyratex ClusterStor TM The Future of HPC Storage. Ken Claffey, Alan Poston & Torben Kling-Petersen. Networked Storage Solutions
Xyratex ClusterStor TM 3000 The Future of HPC Storage Ken Claffey, Alan Poston & Torben Kling-Petersen Networked Storage Solutions Notices The information in this document is subject to change without
More informationArchitecting a High Performance Storage System
WHITE PAPER Intel Enterprise Edition for Lustre* Software High Performance Data Division Architecting a High Performance Storage System January 2014 Contents Introduction... 1 A Systematic Approach to
More informationCSCS HPC storage. Hussein N. Harake
CSCS HPC storage Hussein N. Harake Points to Cover - XE6 External Storage (DDN SFA10K, SRP, QDR) - PCI-E SSD Technology - RamSan 620 Technology XE6 External Storage - Installed Q4 2010 - In Production
More informationStore Process Analyze Collaborate Archive Cloud The HPC Storage Leader Invent Discover Compete
Store Process Analyze Collaborate Archive Cloud The HPC Storage Leader Invent Discover Compete 1 DDN Who We Are 2 We Design, Deploy and Optimize Storage Systems Which Solve HPC, Big Data and Cloud Business
More informationNetApp: Solving I/O Challenges. Jeff Baxter February 2013
NetApp: Solving I/O Challenges Jeff Baxter February 2013 1 High Performance Computing Challenges Computing Centers Challenge of New Science Performance Efficiency directly impacts achievable science Power
More informationDDN and Flash GRIDScaler, Flashscale Infinite Memory Engine
1! DDN and Flash GRIDScaler, Flashscale Infinite Memory Engine T. Cecchi - September 21 st 2016 HPC Advisory Council 2! DDN END-TO-END DATA LIFECYCLE MANAGEMENT BURST & COMPUTE SSD, DISK & FILE SYSTEM
More information2012 HPC Advisory Council
Q1 2012 2012 HPC Advisory Council DDN Big Data & InfiniBand Storage Solutions Overview Toine Beckers Director of HPC Sales, EMEA The Global Big & Fast Data Leader DDN delivers highly scalable & highly-efficient
More informationAn Exploration of New Hardware Features for Lustre. Nathan Rutman
An Exploration of New Hardware Features for Lustre Nathan Rutman Motivation Open-source Hardware-agnostic Linux Least-common-denominator hardware 2 Contents Hardware CRC MDRAID T10 DIF End-to-end data
More informationDesign Concepts & Capacity Expansions of QNAP RAID 50/60
Design Concepts & Capacity Expansions of QNAP RAID 50/60 Your challenges, our solutions: Insufficient protection of RAID 5? Will performance be degraded when using RAID 50/60? Why You Should Use RAID 50/60
More informationDDN s Vision for the Future of Lustre LUG2015 Robert Triendl
DDN s Vision for the Future of Lustre LUG2015 Robert Triendl 3 Topics 1. The Changing Markets for Lustre 2. A Vision for Lustre that isn t Exascale 3. Building Lustre for the Future 4. Peak vs. Operational
More informationComputer Science Section. Computational and Information Systems Laboratory National Center for Atmospheric Research
Computer Science Section Computational and Information Systems Laboratory National Center for Atmospheric Research My work in the context of TDD/CSS/ReSET Polynya new research computing environment Polynya
More informationScalability Testing of DNE2 in Lustre 2.7 and Metadata Performance using Virtual Machines Tom Crowe, Nathan Lavender, Stephen Simms
Scalability Testing of DNE2 in Lustre 2.7 and Metadata Performance using Virtual Machines Tom Crowe, Nathan Lavender, Stephen Simms Research Technologies High Performance File Systems hpfs-admin@iu.edu
More informationInfiniBand Networked Flash Storage
InfiniBand Networked Flash Storage Superior Performance, Efficiency and Scalability Motti Beck Director Enterprise Market Development, Mellanox Technologies Flash Memory Summit 2016 Santa Clara, CA 1 17PB
More informationIsilon Performance. Name
1 Isilon Performance Name 2 Agenda Architecture Overview Next Generation Hardware Performance Caching Performance Streaming Reads Performance Tuning OneFS Architecture Overview Copyright 2014 EMC Corporation.
More informationCS143: Disks and Files
CS143: Disks and Files 1 System Architecture CPU Word (1B 64B) ~ x GB/sec Main Memory System Bus Disk Controller... Block (512B 50KB) ~ x MB/sec Disk 2 Magnetic disk vs SSD Magnetic Disk Stores data on
More informationIsilon Scale Out NAS. Morten Petersen, Senior Systems Engineer, Isilon Division
Isilon Scale Out NAS Morten Petersen, Senior Systems Engineer, Isilon Division 1 Agenda Architecture Overview Next Generation Hardware Performance Caching Performance SMB 3 - MultiChannel 2 OneFS Architecture
More informationAll-Flash High-Performance SAN/NAS Solutions for Virtualization & OLTP
All-Flash High-Performance SAN/NAS Solutions for Virtualization & OLTP All-flash configurations are designed to deliver maximum IOPS and throughput numbers for mission critical workloads and applicati
More informationNetwork Storage Appliance
Enterprise File System HyperFS integrates high-performance SAN, NAS and data protection under one namespace. No Per Seat Client Cost Based on concurrent RAID performance, connectivity, bandwidth, drive
More informationThe Software Defined Online Storage System at the GridKa WLCG Tier-1 Center
The Software Defined Online Storage System at the GridKa WLCG Tier-1 Center CHEP 2018, Sofia Jan Erik Sundermann, Jolanta Bubeliene, Ludmilla Obholz, Andreas Petzold STEINBUCH CENTRE FOR COMPUTING (SCC)
More informationNetApp High-Performance Storage Solution for Lustre
Technical Report NetApp High-Performance Storage Solution for Lustre Solution Design Narjit Chadha, NetApp October 2014 TR-4345-DESIGN Abstract The NetApp High-Performance Storage Solution (HPSS) for Lustre,
More informationAndreas Dilger, Intel High Performance Data Division Lustre User Group 2017
Andreas Dilger, Intel High Performance Data Division Lustre User Group 2017 Statements regarding future functionality are estimates only and are subject to change without notice Performance and Feature
More informationThe term "physical drive" refers to a single hard disk module. Figure 1. Physical Drive
HP NetRAID Tutorial RAID Overview HP NetRAID Series adapters let you link multiple hard disk drives together and write data across them as if they were one large drive. With the HP NetRAID Series adapter,
More informationBrent Welch. Director, Architecture. Panasas Technology. HPC Advisory Council Lugano, March 2011
Brent Welch Director, Architecture Panasas Technology HPC Advisory Council Lugano, March 2011 Panasas Background Technology based on Object Storage concepts invented by company founder, Garth Gibson, a
More informationThe Spider Center-Wide File System
The Spider Center-Wide File System Presented by Feiyi Wang (Ph.D.) Technology Integration Group National Center of Computational Sciences Galen Shipman (Group Lead) Dave Dillow, Sarp Oral, James Simmons,
More informationAn Overview of Fujitsu s Lustre Based File System
An Overview of Fujitsu s Lustre Based File System Shinji Sumimoto Fujitsu Limited Apr.12 2011 For Maximizing CPU Utilization by Minimizing File IO Overhead Outline Target System Overview Goals of Fujitsu
More informationStorage for HPC, HPDA and Machine Learning (ML)
for HPC, HPDA and Machine Learning (ML) Frank Kraemer, IBM Systems Architect mailto:kraemerf@de.ibm.com IBM Data Management for Autonomous Driving (AD) significantly increase development efficiency by
More informationAndreas Dilger. Principal Lustre Engineer. High Performance Data Division
Andreas Dilger Principal Lustre Engineer High Performance Data Division Focus on Performance and Ease of Use Beyond just looking at individual features... Incremental but continuous improvements Performance
More informationBreakthrough Cloud Performance NeoSapphire All-Flash Arrays. AccelStor, Inc.
Breakthrough Cloud Performance NeoSapphire All-Flash Arrays AccelStor, Inc. Company Profile About AccelStor, Inc. Established in November 2014 Management team President: Charles Tsai, Ph.D. Vice president:
More informationLessons from Post-processing Climate Data on Modern Flash-based HPC Systems
Lessons from Post-processing Climate Data on Modern Flash-based HPC Systems Adnan Haider 1, Sheri Mickelson 2, John Dennis 2 1 Illinois Institute of Technology, USA; 2 National Center of Atmospheric Research,
More informationINFINIDAT Storage Architecture. White Paper
INFINIDAT Storage Architecture White Paper Abstract The INFINIDAT enterprise storage solution is based upon the unique and patented INFINIDAT Storage Architecture (ISA). The INFINIDAT Storage Architecture
More informationNew HPE 3PAR StoreServ 8000 and series Optimized for Flash
New HPE 3PAR StoreServ 8000 and 20000 series Optimized for Flash AGENDA HPE 3PAR StoreServ architecture fundamentals HPE 3PAR Flash optimizations HPE 3PAR portfolio overview HPE 3PAR Flash example from
More informationAn ESS implementation in a Tier 1 HPC Centre
An ESS implementation in a Tier 1 HPC Centre Maximising Performance - the NeSI Experience José Higino (NeSI Platforms and NIWA, HPC Systems Engineer) Outline What is NeSI? The National Platforms Framework
More informationFeedback on BeeGFS. A Parallel File System for High Performance Computing
Feedback on BeeGFS A Parallel File System for High Performance Computing Philippe Dos Santos et Georges Raseev FR 2764 Fédération de Recherche LUmière MATière December 13 2016 LOGO CNRS LOGO IO December
More informationExperiences with HP SFS / Lustre in HPC Production
Experiences with HP SFS / Lustre in HPC Production Computing Centre (SSCK) University of Karlsruhe Laifer@rz.uni-karlsruhe.de page 1 Outline» What is HP StorageWorks Scalable File Share (HP SFS)? A Lustre
More informationNEC Express5800 A2040b 22TB Data Warehouse Fast Track. Reference Architecture with SW mirrored HGST FlashMAX III
NEC Express5800 A2040b 22TB Data Warehouse Fast Track Reference Architecture with SW mirrored HGST FlashMAX III Based on Microsoft SQL Server 2014 Data Warehouse Fast Track (DWFT) Reference Architecture
More informationBuilding Self-Healing Mass Storage Arrays. for Large Cluster Systems
Building Self-Healing Mass Storage Arrays for Large Cluster Systems NSC08, Linköping, 14. October 2008 Toine Beckers tbeckers@datadirectnet.com Agenda Company Overview Balanced I/O Systems MTBF and Availability
More information朱义普. Resolving High Performance Computing and Big Data Application Bottlenecks with Application-Defined Flash Acceleration. Director, North Asia, HPC
October 28, 2013 Resolving High Performance Computing and Big Data Application Bottlenecks with Application-Defined Flash Acceleration 朱义普 Director, North Asia, HPC DDN Storage Vendor for HPC & Big Data
More informationImproved Solutions for I/O Provisioning and Application Acceleration
1 Improved Solutions for I/O Provisioning and Application Acceleration August 11, 2015 Jeff Sisilli Sr. Director Product Marketing jsisilli@ddn.com 2 Why Burst Buffer? The Supercomputing Tug-of-War A supercomputer
More informationLustre TM. Scalability
Lustre TM Scalability An Oak Ridge National Laboratory/ Lustre Center of Excellence White Paper February 2009 2 Sun Microsystems, Inc Table of Contents Executive Summary...3 HPC Trends...3 Lustre File
More informationVirtual SQL Servers. Actual Performance. 2016
@kleegeek davidklee.net heraflux.com linkedin.com/in/davidaklee Specialties / Focus Areas / Passions: Performance Tuning & Troubleshooting Virtualization Cloud Enablement Infrastructure Architecture Health
More informationArchitecting Storage for Semiconductor Design: Manufacturing Preparation
White Paper Architecting Storage for Semiconductor Design: Manufacturing Preparation March 2012 WP-7157 EXECUTIVE SUMMARY The manufacturing preparation phase of semiconductor design especially mask data
More informationASKAP Central Processor: Design and Implementa8on
ASKAP Central Processor: Design and Implementa8on Calibra8on and Imaging Workshop 2014 Ben Humphreys ASKAP So(ware and Compu3ng Project Engineer 3rd - 7th March 2014 ASTRONOMY AND SPACE SCIENCE Australian
More informationAll-Flash High-Performance SAN/NAS Solutions for Virtualization & OLTP
All-Flash High-Performance SAN/NAS Solutions for Virtualization & OLTP All-flash configurations are designed to deliver maximum IOPS and throughput numbers for mission critical workloads and applicati
More informationHigh Performance Storage Solutions
November 2006 High Performance Storage Solutions Toine Beckers tbeckers@datadirectnet.com www.datadirectnet.com DataDirect Leadership Established 1988 Technology Company (ASICs, FPGA, Firmware, Software)
More informationIBM System Storage DS8870 Release R7.3 Performance Update
IBM System Storage DS8870 Release R7.3 Performance Update Enterprise Storage Performance Yan Xu Agenda Summary of DS8870 Hardware Changes I/O Performance of High Performance Flash Enclosure (HPFE) Easy
More information1 Title. 2 Team Information. 3 Abstract. 4 Problem Statement and Hypothesis. Zest: The Maximum Reliable TBytes/sec/$ for Petascale Systems
1 Title Zest: The Maximum Reliable TBytes/sec/$ for Petascale Systems 2 Team Information The Pittsburgh Supercomputing Center Advanced Systems Group: Nathan Stone, Doug Balog, Paul Nowoczynski, Jason Sommerfield,
More informationLustre / ZFS at Indiana University
Lustre / ZFS at Indiana University HPC-IODC Workshop, Frankfurt June 28, 2018 Stephen Simms Manager, High Performance File Systems ssimms@iu.edu Tom Crowe Team Lead, High Performance File Systems thcrowe@iu.edu
More informationThe Oracle Database Appliance I/O and Performance Architecture
Simple Reliable Affordable The Oracle Database Appliance I/O and Performance Architecture Tammy Bednar, Sr. Principal Product Manager, ODA 1 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
More informationUsing DDN IME for Harmonie
Irish Centre for High-End Computing Using DDN IME for Harmonie Gilles Civario, Marco Grossi, Alastair McKinstry, Ruairi Short, Nix McDonnell April 2016 DDN IME: Infinite Memory Engine IME: Major Features
More informationChapter 10: Mass-Storage Systems
Chapter 10: Mass-Storage Systems Silberschatz, Galvin and Gagne 2013 Chapter 10: Mass-Storage Systems Overview of Mass Storage Structure Disk Structure Disk Attachment Disk Scheduling Disk Management Swap-Space
More informationChapter 10: Mass-Storage Systems. Operating System Concepts 9 th Edition
Chapter 10: Mass-Storage Systems Silberschatz, Galvin and Gagne 2013 Chapter 10: Mass-Storage Systems Overview of Mass Storage Structure Disk Structure Disk Attachment Disk Scheduling Disk Management Swap-Space
More informationBeeGFS. Parallel Cluster File System. Container Workshop ISC July Marco Merkel VP ww Sales, Consulting
BeeGFS The Parallel Cluster File System Container Workshop ISC 28.7.18 www.beegfs.io July 2018 Marco Merkel VP ww Sales, Consulting HPC & Cognitive Workloads Demand Today Flash Storage HDD Storage Shingled
More informationData Analytics and Storage System (DASS) Mixing POSIX and Hadoop Architectures. 13 November 2016
National Aeronautics and Space Administration Data Analytics and Storage System (DASS) Mixing POSIX and Hadoop Architectures 13 November 2016 Carrie Spear (carrie.e.spear@nasa.gov) HPC Architect/Contractor
More informationHitachi Virtual Storage Platform Family
Hitachi Virtual Storage Platform Family Advanced Storage Capabilities for All Organizations Andre Lahrmann 23. November 2017 Hitachi Vantara Vorweg: Aus Hitachi Data Systems wird Hitachi Vantara The efficiency
More informationFhGFS - Performance at the maximum
FhGFS - Performance at the maximum http://www.fhgfs.com January 22, 2013 Contents 1. Introduction 2 2. Environment 2 3. Benchmark specifications and results 3 3.1. Multi-stream throughput................................
More informationPART-I (B) (TECHNICAL SPECIFICATIONS & COMPLIANCE SHEET) Supply and installation of High Performance Computing System
INSTITUTE FOR PLASMA RESEARCH (An Autonomous Institute of Department of Atomic Energy, Government of India) Near Indira Bridge; Bhat; Gandhinagar-382428; India PART-I (B) (TECHNICAL SPECIFICATIONS & COMPLIANCE
More informationJohn Kaitschuck, Senior Staff Engineer/Technologist CSSG, June, 2017 Seagate LUG 2017 Presentation
The Effects of Fragmentation and Capacity on Lustre File System Performance John Kaitschuck, Senior Staff Engineer/Technologist CSSG, June, 2017 Seagate LUG 2017 Presentation AGENDA History/Background
More informationI/O at the German Climate Computing Center (DKRZ)
I/O at the German Climate Computing Center (DKRZ) Julian M. Kunkel, Carsten Beyer kunkel@dkrz.de German Climate Computing Center (DKRZ) 16-07-2015 Outline 1 Introduction 2 Workload 3 System View 4 Obstacles
More informationLustre on ZFS. At The University of Wisconsin Space Science and Engineering Center. Scott Nolin September 17, 2013
Lustre on ZFS At The University of Wisconsin Space Science and Engineering Center Scott Nolin September 17, 2013 Why use ZFS for Lustre? The University of Wisconsin Space Science and Engineering Center
More informationRepair Policy: Specifies how the operating system proceeds with repairing virtual disks in the specified storage
What is Parallel Rebuild / Fast Rebuild? Windows Server 2012 R2 Storage Spaces includes the ability to automatically rebuild storage spaces using free space in a storage pool instead of using hot spares.
More informationLock Ahead: Shared File Performance Improvements
Lock Ahead: Shared File Performance Improvements Patrick Farrell Cray Lustre Developer Steve Woods Senior Storage Architect woods@cray.com September 2016 9/12/2016 Copyright 2015 Cray Inc 1 Agenda Shared
More informationNext-Generation NVMe-Native Parallel Filesystem for Accelerating HPC Workloads
Next-Generation NVMe-Native Parallel Filesystem for Accelerating HPC Workloads Liran Zvibel CEO, Co-founder WekaIO @liranzvibel 1 WekaIO Matrix: Full-featured and Flexible Public or Private S3 Compatible
More informationAll-Flash High-Performance SAN/NAS Solutions for Virtualization & OLTP
All-Flash High-Performance SAN/NAS Solutions for Virtualization & OLTP All-flash configurations are designed to deliver maximum IOPS and throughput numbers for mission critical workloads and applicati
More informationDesign and Architecture of Dell Acceleration Appliances for Database (DAAD): A Practical Approach with High Availability Guaranteed
Design and Architecture of Dell Acceleration Appliances for Database (DAAD): A Practical Approach with High Availability Guaranteed Kai Yu, Yuxiang Gao Dell Global Solutions Engineering Group Peng Zhang,
More informationEfficient Object Storage Journaling in a Distributed Parallel File System
Efficient Object Storage Journaling in a Distributed Parallel File System Presented by Sarp Oral Sarp Oral, Feiyi Wang, David Dillow, Galen Shipman, Ross Miller, and Oleg Drokin FAST 10, Feb 25, 2010 A
More informationJMR ELECTRONICS INC. WHITE PAPER
THE NEED FOR SPEED: USING PCI EXPRESS ATTACHED STORAGE FOREWORD The highest performance, expandable, directly attached storage can be achieved at low cost by moving the server or work station s PCI bus
More informationDemonstration Milestone for Parallel Directory Operations
Demonstration Milestone for Parallel Directory Operations This milestone was submitted to the PAC for review on 2012-03-23. This document was signed off on 2012-04-06. Overview This document describes
More informationIntroducing Panasas ActiveStor 14
Introducing Panasas ActiveStor 14 SUPERIOR PERFORMANCE FOR MIXED FILE SIZE ENVIRONMENTS DEREK BURKE, PANASAS EUROPE INTRODUCTION TO PANASAS Storage that accelerates the world s highest performance and
More informationRAID Levels Table RAID LEVEL DESCRIPTION EXAMPLE
Product: RAID Enabled NVRs and Hybrids Page: 1 of 6 Summary RAID (Redundant Array of Independent Disks) is a data storage virtualization technology that combines multiple disk drive components into a logical
More informationRobin Hood 2.5 on Lustre 2.5 with DNE
Robin Hood 2.5 on Lustre 2.5 with DNE Site status and experience at German Climate Computing Centre in Hamburg Carsten Beyer High Performance Computing Center Exclusively for the German Climate Research
More informationMission-Critical Lustre at Santos. Adam Fox, Lustre User Group 2016
Mission-Critical Lustre at Santos Adam Fox, Lustre User Group 2016 About Santos One of the leading oil and gas producers in APAC Founded in 1954 South Australia Northern Territory Oil Search Cooper Basin
More informationDeep Learning Performance and Cost Evaluation
Micron 5210 ION Quad-Level Cell (QLC) SSDs vs 7200 RPM HDDs in Centralized NAS Storage Repositories A Technical White Paper Don Wang, Rene Meyer, Ph.D. info@ AMAX Corporation Publish date: October 25,
More informationCold Storage: The Road to Enterprise Ilya Kuznetsov YADRO
Cold Storage: The Road to Enterprise Ilya Kuznetsov YADRO Agenda Technical challenge Custom product Growth of aspirations Enterprise requirements Making an enterprise cold storage product 2 Technical Challenge
More informationUnder the Hood of Oracle Database Appliance. Alex Gorbachev
Under the Hood of Oracle Database Appliance Alex Gorbachev Mountain View, CA 9-Nov-2011 http://bit.ly/pythianasmwebinar 2 Alex Gorbachev CTO, The Pythian Group Blogger OakTable Network member Oracle ACE
More information