Data Movement & Storage Using the Data Capacitor Filesystem

Size: px
Start display at page:

Download "Data Movement & Storage Using the Data Capacitor Filesystem"

Transcription

1 Data Movement & Storage Using the Data Capacitor Filesystem Justin Miller Big Data for Science Workshop July 2010

2 Challenges for DISC Keynote by Alex Szalay identified the challenges that researchers face Scientific data doubles every year Amount of data is a barrier to extracting knowledge Problem of today: data access How can we minimize data movement?

3 Workflow Example Single Compute 0$/$)1+.%&" *+,-./")!"#+.%&"!"#"$%&'"%(#)*+,-./"%

4 Workflow Example Multiple Compute 3$/$)4+.%&" *+,-./"!"#+.%&")01 *+,-./"!"#+.%&")02!"#"$%&'"%(#)*+,-./"%

5 Workflow Example Visualization 3$/$)4+.%&" *+,-./")!"#+.%&")01!"#"$%&'"%(#) *+,-./"% *+,-./")!"#+.%&")02 56#.$768$/6+9!"#+.%&"

6 Workflow Example Archive 3$/$)4+.%&" *+,-./")!"#+.%&")01!"#"$%&'"%(#) *+,-./"% *+,-./")!"#+.%&")02 56#.$768$/6+9!"#+.%&" :$-" ;%&'6<"

7 Data Movement & Storage This is an unsustainable workflow Works for GB, maybe single TB, but not more Every resource is another series of transfers Data movement is in the way of doing work Good reasons to add resources to workflow And we haven t addressed other drawbacks

8 IU Central Filesystem Workflow 3$/$)4+.%&" *+,-./")!"#+.%&")01!"#"$%&'"%(#) *+,-./"%!"#" $"%"&'#() *+,-./")!"#+.%&")02 56#.$768$/6+9!"#+.%&" :$-" ;%&'6<"

9 IU s Data Capacitor Filesystem National Science Foundation funded in 2005 Funds purchased 535TB of Lustre storage 339TB available as production service Data Capacitor name comes from electronics capacitors provides transient storage of electrons absorbs and evens out peaks in flow provides consistent output

10 Idea of Data Capacitor Centralized short-term storage for IU resources Store your data to compute against, and use for scratch space during your run Possibility exists for mid-term storage

11 Data Capacitor Centralized Storage Compute using IU s supercomputer Big Red Compute using IU s Quarry cluster Archive to IU s massive HPSS tape archive hierarchical storage archive your data to tape

12 Central to IU Cyberinfrastructure

13 Physics Research Dr. Chuck Horowitz, IU physicist Interested in the behavior of neutron stars Studying the behavior of nuclear matter near saturation density can form interesting phase "nuclear pasta Using MDGRAPE-2 hardware for increased performance

14 Physics Research Particle interaction is simulated via molecular dynamics using specialized MDGRAPE-2 hardware configurations are saved Post processing creates VTK frames Visualization system ingests frame data displays as movie

15 Physics Research!"#$%&'( )'*"%+,' 7/&/!/$/,.&"+ -.*%/0.1/&."2 )'*"%+,' 3/$' 4+,5.6'

16 Earth Science Research Linked Environments for Atmospheric Discovery (LEAD) WxChallenge The WxChallenge is a meteorological forecast competition. Compete to forecaste maximum and minimum temperatures, precipitation, and maximum wind speeds for select U.S. cities over a tenweek period each semester

17 LEAD Workflow 0'.&1'+ -.&.!"#$%&'( )'*"%+,' -.&.!.$.,/&"+!"#$%&'+ 2,/'3,'!4%*&'+ 5+.3*6'+ )'*"%+,'*

18 Extend the Centralized FS Model The natural progression is to be central to more resources Make data available to more resources IU did this by extending the filesystem across the wide-area network (WAN) Data Capacitor WAN (DC-WAN) New FS separate from the original DC

19 Data Capacitor WAN

20 Data Capacitor WAN Tradeoffs The benefit of a centralized WAN filesystem is the illusion of locality Your data is transferred behind the scenes across the network At worst your data will be transferred slower than you like At best it is as fast, or faster, than local storage; typically comparable across research networks

21 DC-WAN Namespace Mapping WAN FS challenge is heterogeneous user identification across sites The numeric user identification (UID) for a particular user not the same across sites You don t have to worry about this because DC-WAN does the conversion Indiana TACC PSC NCSA SDSC jupmille tg jupmille jupmille jupmille uid= uid= uid=43415 uid=40436 uid=502639

22 Physics Research with DC-WAN ;'+-$#.')"()" *)+,-./ 0/&)-12/!-&.'"<(5= >>?(+'$/& 3'&-#$'4#.')" 0/&)-12/ 8#.# *#,#2'.)1 9!:!"#$%&'&()" *)+,-./ 0/&)-12/ 5#,/!126'7/

23 Astronomy with DC-WAN ABCD(+'$/&!"#$%&'&()" *)+,-./ 0/&)-12/ 79:8(3/$/&2),/ ;"/(6/<1//(9+#</1 =;69> 6#.# *#,#2'.)1 7!8 3#,/!124'5/ Image NOAO/AURA/NSF

24 Center for the Remote Sensing of Ice Sheets (CReSIS) Workflow 5*''67-68!"#$%&' (')"%*+' 2-&-!-$-+0&"* 3.4,-$'.*+/01'.6&-*&0+-

25 Gas Giant Planet Research G9- G),,2B3&H(<=G% FE?=A)4$2 1)23"4)5",).6 7$2.3&'$ 0-9% :&B"6"<=CD +"," -"#"'),.& /%0!"#$ %&'()*$ 89: 9,"&;*)44$<=89

26 Demo Small sample of Gas Giant Planet Research Data is on DC-WAN, which is mounted on two different resources Compute on PSC s Pople (SGI Altix 4700) Post-process and visualize results on IU machine that has proprietary software (IDL v7.0); view over network

27 IU s Data Capacitor WAN Filesystem Funded by Indiana University in TB of storage available as production service Centralized short-term storage for nationwide resources, including TeraGrid Use your data on the best resource for your needs Short-term storage like DC, possibility exists for mid-term storage

28 Based on Lustre Filesystem Lustre is a parallel distributed file system Available under the GNU GPL Used by U.S. government, movie studios, financial institutions, oil and gas industry 7 of the top 10 HPC systems on the June 2009 "Top 500" list 52 of the top 100 run Lustre in 2010

29 Based on Lustre Filesystem Lustre filesystems can support up to tens of thousands of client systems, petabytes (PBs) of storage and hundreds of gigabytes per second (GB/s) of I/O throughput. Scalable filesystem uses separate servers to aggregate for performance storage backend is hidden from the client

30 Lustre Filesystem Architecture Lustre presents all clients with standard POSIX filesystem interface Filesystem mount My scratch directory for example: IU: /N/dcwan/scratch/jupmille/ PSC: /N/dcwan/scratch/jupmille/ TACC: /N/dcwan/scratch/jupmille/ NCSA: /N/dcwan/scratch/jupmille/ Standard commands ls, cp, cat, etc. from the command line

31 Lustre Filesystem Architecture Metadata Server (MDS) stores the filesystem metadata such as filenames, directories, and permissions. file operations such as open/close Object Storage Server (OSS) bulk I/O servers Object Storage Targets (OST) back-end storage devices

32 Lustre Filesystem Architecture '() *)) *)) *)) *))!"#$%& *)+ *)+ *)+ *)+ *)+ *)+ *)+ *)+ *)+ *)+ *)+

33 Data Capacitor Hardware 8 pairs Dell PowerEdge x 3.0 GHz Dual Core Xeon Myrinet 10G Ethernet Dual port Qlogic 2432 HBA (4 x FC) 2.6 Kernel (RHEL 5), Lustre DDN S2A9550 Controllers Over 2.4 GB/sec measured throughput each 339Tb of spinning SATA disk

34 Data Capacitor WAN Hardware 2 pairs Dell PowerEdge x 3.0 GHz Dual Core Xeon Myrinet 10G Ethernet Dual port Qlogic 2432 HBA (4 x FC) 2.6 Kernel (RHEL 5), Lustre DDN S2A9550 Controllers Over 2.4 GB/sec measured throughput 339Tb of spinning SATA disk

35 Getting the Most out of Lustre Lustre is optimized for large files (where large is >1Mb), not so good for small files Lustre has aggressive client side caching if you plan reading the same files more than once, big win Lustre allows you to control how your data is striped across the OSTs, so optimization based on your I/O patterns can reap benefits in throughput

36 Lustre WAN Future DC-WAN will be mounted on the India and Sierra FutureGrid clusters In the testing phase right now IU s Lustre UID mapping code will be used in a new TeraGrid Lustre-WAN project in development now

37 Thank you for listening. Questions are welcome. Please use moderators for Q&A Justin Miller Data Capacitor Team

Indiana University s Lustre WAN: The TeraGrid and Beyond

Indiana University s Lustre WAN: The TeraGrid and Beyond Indiana University s Lustre WAN: The TeraGrid and Beyond Stephen C. Simms Manager, Data Capacitor Project TeraGrid Site Lead, Indiana University ssimms@indiana.edu Lustre User Group Meeting April 17, 2009

More information

Indiana University's Lustre WAN: Empowering Production Workflows on the TeraGrid

Indiana University's Lustre WAN: Empowering Production Workflows on the TeraGrid Indiana University's Lustre WAN: Empowering Production Workflows on the TeraGrid Stephen C. Simms Manager, Data Capacitor Project TeraGrid Site Lead, Indiana University ssimms@indiana.edu The Data Capacitor

More information

Regional & National HPC resources available to UCSB

Regional & National HPC resources available to UCSB Regional & National HPC resources available to UCSB Triton Affiliates and Partners Program (TAPP) Extreme Science and Engineering Discovery Environment (XSEDE) UCSB clusters https://it.ucsb.edu/services/supercomputing

More information

Emerging Technologies for HPC Storage

Emerging Technologies for HPC Storage Emerging Technologies for HPC Storage Dr. Wolfgang Mertz CTO EMEA Unstructured Data Solutions June 2018 The very definition of HPC is expanding Blazing Fast Speed Accessibility and flexibility 2 Traditional

More information

LCE: Lustre at CEA. Stéphane Thiell CEA/DAM

LCE: Lustre at CEA. Stéphane Thiell CEA/DAM LCE: Lustre at CEA Stéphane Thiell CEA/DAM (stephane.thiell@cea.fr) 1 Lustre at CEA: Outline Lustre at CEA updates (2009) Open Computing Center (CCRT) updates CARRIOCAS (Lustre over WAN) project 2009-2010

More information

Scaling a Global File System to the Greatest Possible Extent, Performance, Capacity, and Number of Users

Scaling a Global File System to the Greatest Possible Extent, Performance, Capacity, and Number of Users Scaling a Global File System to the Greatest Possible Extent, Performance, Capacity, and Number of Users Phil Andrews, Bryan Banister, Patricia Kovatch, Chris Jordan San Diego Supercomputer Center University

More information

Sun Lustre Storage System Simplifying and Accelerating Lustre Deployments

Sun Lustre Storage System Simplifying and Accelerating Lustre Deployments Sun Lustre Storage System Simplifying and Accelerating Lustre Deployments Torben Kling-Petersen, PhD Presenter s Name Principle Field Title andengineer Division HPC &Cloud LoB SunComputing Microsystems

More information

Computer Science Section. Computational and Information Systems Laboratory National Center for Atmospheric Research

Computer Science Section. Computational and Information Systems Laboratory National Center for Atmospheric Research Computer Science Section Computational and Information Systems Laboratory National Center for Atmospheric Research My work in the context of TDD/CSS/ReSET Polynya new research computing environment Polynya

More information

DDN s Vision for the Future of Lustre LUG2015 Robert Triendl

DDN s Vision for the Future of Lustre LUG2015 Robert Triendl DDN s Vision for the Future of Lustre LUG2015 Robert Triendl 3 Topics 1. The Changing Markets for Lustre 2. A Vision for Lustre that isn t Exascale 3. Building Lustre for the Future 4. Peak vs. Operational

More information

High-Performance Lustre with Maximum Data Assurance

High-Performance Lustre with Maximum Data Assurance High-Performance Lustre with Maximum Data Assurance Silicon Graphics International Corp. 900 North McCarthy Blvd. Milpitas, CA 95035 Disclaimer and Copyright Notice The information presented here is meant

More information

Lustre / ZFS at Indiana University

Lustre / ZFS at Indiana University Lustre / ZFS at Indiana University HPC-IODC Workshop, Frankfurt June 28, 2018 Stephen Simms Manager, High Performance File Systems ssimms@iu.edu Tom Crowe Team Lead, High Performance File Systems thcrowe@iu.edu

More information

Parallel File Systems. John White Lawrence Berkeley National Lab

Parallel File Systems. John White Lawrence Berkeley National Lab Parallel File Systems John White Lawrence Berkeley National Lab Topics Defining a File System Our Specific Case for File Systems Parallel File Systems A Survey of Current Parallel File Systems Implementation

More information

Parallel File Systems Compared

Parallel File Systems Compared Parallel File Systems Compared Computing Centre (SSCK) University of Karlsruhe, Germany Laifer@rz.uni-karlsruhe.de page 1 Outline» Parallel file systems (PFS) Design and typical usage Important features

More information

HPC Storage Use Cases & Future Trends

HPC Storage Use Cases & Future Trends Oct, 2014 HPC Storage Use Cases & Future Trends Massively-Scalable Platforms and Solutions Engineered for the Big Data and Cloud Era Atul Vidwansa Email: atul@ DDN About Us DDN is a Leader in Massively

More information

Architecting Storage for Semiconductor Design: Manufacturing Preparation

Architecting Storage for Semiconductor Design: Manufacturing Preparation White Paper Architecting Storage for Semiconductor Design: Manufacturing Preparation March 2012 WP-7157 EXECUTIVE SUMMARY The manufacturing preparation phase of semiconductor design especially mask data

More information

DVS, GPFS and External Lustre at NERSC How It s Working on Hopper. Tina Butler, Rei Chi Lee, Gregory Butler 05/25/11 CUG 2011

DVS, GPFS and External Lustre at NERSC How It s Working on Hopper. Tina Butler, Rei Chi Lee, Gregory Butler 05/25/11 CUG 2011 DVS, GPFS and External Lustre at NERSC How It s Working on Hopper Tina Butler, Rei Chi Lee, Gregory Butler 05/25/11 CUG 2011 1 NERSC is the Primary Computing Center for DOE Office of Science NERSC serves

More information

Parallel File Systems for HPC

Parallel File Systems for HPC Introduction to Scuola Internazionale Superiore di Studi Avanzati Trieste November 2008 Advanced School in High Performance and Grid Computing Outline 1 The Need for 2 The File System 3 Cluster & A typical

More information

Feedback on BeeGFS. A Parallel File System for High Performance Computing

Feedback on BeeGFS. A Parallel File System for High Performance Computing Feedback on BeeGFS A Parallel File System for High Performance Computing Philippe Dos Santos et Georges Raseev FR 2764 Fédération de Recherche LUmière MATière December 13 2016 LOGO CNRS LOGO IO December

More information

Data Movement and Storage. 04/07/09 1

Data Movement and Storage. 04/07/09  1 Data Movement and Storage 04/07/09 www.cac.cornell.edu 1 Data Location, Storage, Sharing and Movement Four of the seven main challenges of Data Intensive Computing, according to SC06. (Other three: viewing,

More information

Store Process Analyze Collaborate Archive Cloud The HPC Storage Leader Invent Discover Compete

Store Process Analyze Collaborate Archive Cloud The HPC Storage Leader Invent Discover Compete Store Process Analyze Collaborate Archive Cloud The HPC Storage Leader Invent Discover Compete 1 DDN Who We Are 2 We Design, Deploy and Optimize Storage Systems Which Solve HPC, Big Data and Cloud Business

More information

Lustre overview and roadmap to Exascale computing

Lustre overview and roadmap to Exascale computing HPC Advisory Council China Workshop Jinan China, October 26th 2011 Lustre overview and roadmap to Exascale computing Liang Zhen Whamcloud, Inc liang@whamcloud.com Agenda Lustre technology overview Lustre

More information

TGCC OVERVIEW. 13 février 2014 CEA 10 AVRIL 2012 PAGE 1

TGCC OVERVIEW. 13 février 2014 CEA 10 AVRIL 2012 PAGE 1 STORAGE @ TGCC OVERVIEW CEA 10 AVRIL 2012 PAGE 1 CONTEXT Data-Centric Architecture Centralized storage, accessible from every TGCC s compute machines Make cross-platform data sharing possible Mutualized

More information

NetApp High-Performance Storage Solution for Lustre

NetApp High-Performance Storage Solution for Lustre Technical Report NetApp High-Performance Storage Solution for Lustre Solution Design Narjit Chadha, NetApp October 2014 TR-4345-DESIGN Abstract The NetApp High-Performance Storage Solution (HPSS) for Lustre,

More information

LUG 2012 From Lustre 2.1 to Lustre HSM IFERC (Rokkasho, Japan)

LUG 2012 From Lustre 2.1 to Lustre HSM IFERC (Rokkasho, Japan) LUG 2012 From Lustre 2.1 to Lustre HSM Lustre @ IFERC (Rokkasho, Japan) Diego.Moreno@bull.net From Lustre-2.1 to Lustre-HSM - Outline About Bull HELIOS @ IFERC (Rokkasho, Japan) Lustre-HSM - Basis of Lustre-HSM

More information

Storage on the Lunatic Fringe. Thomas M. Ruwart University of Minnesota Digital Technology Center Intelligent Storage Consortium

Storage on the Lunatic Fringe. Thomas M. Ruwart University of Minnesota Digital Technology Center Intelligent Storage Consortium Storage on the Lunatic Fringe Thomas M. Ruwart University of Minnesota Digital Technology Center Intelligent Storage Consortium tmruwart@dtc.umn.edu Orientation Who are the lunatics? What are their requirements?

More information

Beyond Petascale. Roger Haskin Manager, Parallel File Systems IBM Almaden Research Center

Beyond Petascale. Roger Haskin Manager, Parallel File Systems IBM Almaden Research Center Beyond Petascale Roger Haskin Manager, Parallel File Systems IBM Almaden Research Center GPFS Research and Development! GPFS product originated at IBM Almaden Research Laboratory! Research continues to

More information

SAN, HPSS, Sam-QFS, and GPFS technology in use at SDSC

SAN, HPSS, Sam-QFS, and GPFS technology in use at SDSC SAN, HPSS, Sam-QFS, and GPFS technology in use at SDSC Bryan Banister, San Diego Supercomputing Center bryan@sdsc.edu Manager, Storage Systems and Production Servers Production Services Department Big

More information

Lustre HSM at Cambridge. Early user experience using Intel Lemur HSM agent

Lustre HSM at Cambridge. Early user experience using Intel Lemur HSM agent Lustre HSM at Cambridge Early user experience using Intel Lemur HSM agent Matt Rásó-Barnett Wojciech Turek Research Computing Services @ Cambridge University-wide service with broad remit to provide research

More information

SGI Overview. HPC User Forum Dearborn, Michigan September 17 th, 2012

SGI Overview. HPC User Forum Dearborn, Michigan September 17 th, 2012 SGI Overview HPC User Forum Dearborn, Michigan September 17 th, 2012 SGI Market Strategy HPC Commercial Scientific Modeling & Simulation Big Data Hadoop In-memory Analytics Archive Cloud Public Private

More information

1. ALMA Pipeline Cluster specification. 2. Compute processing node specification: $26K

1. ALMA Pipeline Cluster specification. 2. Compute processing node specification: $26K 1. ALMA Pipeline Cluster specification The following document describes the recommended hardware for the Chilean based cluster for the ALMA pipeline and local post processing to support early science and

More information

GPFS Experiences from the Argonne Leadership Computing Facility (ALCF) William (Bill) E. Allcock ALCF Director of Operations

GPFS Experiences from the Argonne Leadership Computing Facility (ALCF) William (Bill) E. Allcock ALCF Director of Operations GPFS Experiences from the Argonne Leadership Computing Facility (ALCF) William (Bill) E. Allcock ALCF Director of Operations Argonne National Laboratory Argonne National Laboratory is located on 1,500

More information

LustreFS and its ongoing Evolution for High Performance Computing and Data Analysis Solutions

LustreFS and its ongoing Evolution for High Performance Computing and Data Analysis Solutions LustreFS and its ongoing Evolution for High Performance Computing and Data Analysis Solutions Roger Goff Senior Product Manager DataDirect Networks, Inc. What is Lustre? Parallel/shared file system for

More information

HPC Capabilities at Research Intensive Universities

HPC Capabilities at Research Intensive Universities HPC Capabilities at Research Intensive Universities Purushotham (Puri) V. Bangalore Department of Computer and Information Sciences and UAB IT Research Computing UAB HPC Resources 24 nodes (192 cores)

More information

Coordinating Parallel HSM in Object-based Cluster Filesystems

Coordinating Parallel HSM in Object-based Cluster Filesystems Coordinating Parallel HSM in Object-based Cluster Filesystems Dingshan He, Xianbo Zhang, David Du University of Minnesota Gary Grider Los Alamos National Lab Agenda Motivations Parallel archiving/retrieving

More information

Implementing a Hierarchical Storage Management system in a large-scale Lustre and HPSS environment

Implementing a Hierarchical Storage Management system in a large-scale Lustre and HPSS environment Implementing a Hierarchical Storage Management system in a large-scale Lustre and HPSS environment Brett Bode, Michelle Butler, Sean Stevens, Jim Glasgow National Center for Supercomputing Applications/University

More information

Intel Enterprise Edition Lustre (IEEL-2.3) [DNE-1 enabled] on Dell MD Storage

Intel Enterprise Edition Lustre (IEEL-2.3) [DNE-1 enabled] on Dell MD Storage Intel Enterprise Edition Lustre (IEEL-2.3) [DNE-1 enabled] on Dell MD Storage Evaluation of Lustre File System software enhancements for improved Metadata performance Wojciech Turek, Paul Calleja,John

More information

Using file systems at HC3

Using file systems at HC3 Using file systems at HC3 Roland Laifer STEINBUCH CENTRE FOR COMPUTING - SCC KIT University of the State of Baden-Württemberg and National Laboratory of the Helmholtz Association www.kit.edu Basic Lustre

More information

Blue Waters I/O Performance

Blue Waters I/O Performance Blue Waters I/O Performance Mark Swan Performance Group Cray Inc. Saint Paul, Minnesota, USA mswan@cray.com Doug Petesch Performance Group Cray Inc. Saint Paul, Minnesota, USA dpetesch@cray.com Abstract

More information

An Overview of Fujitsu s Lustre Based File System

An Overview of Fujitsu s Lustre Based File System An Overview of Fujitsu s Lustre Based File System Shinji Sumimoto Fujitsu Limited Apr.12 2011 For Maximizing CPU Utilization by Minimizing File IO Overhead Outline Target System Overview Goals of Fujitsu

More information

Scalability Testing of DNE2 in Lustre 2.7 and Metadata Performance using Virtual Machines Tom Crowe, Nathan Lavender, Stephen Simms

Scalability Testing of DNE2 in Lustre 2.7 and Metadata Performance using Virtual Machines Tom Crowe, Nathan Lavender, Stephen Simms Scalability Testing of DNE2 in Lustre 2.7 and Metadata Performance using Virtual Machines Tom Crowe, Nathan Lavender, Stephen Simms Research Technologies High Performance File Systems hpfs-admin@iu.edu

More information

Isilon: Raising The Bar On Performance & Archive Use Cases. John Har Solutions Product Manager Unstructured Data Storage Team

Isilon: Raising The Bar On Performance & Archive Use Cases. John Har Solutions Product Manager Unstructured Data Storage Team Isilon: Raising The Bar On Performance & Archive Use Cases John Har Solutions Product Manager Unstructured Data Storage Team What we ll cover in this session Isilon Overview Streaming workflows High ops/s

More information

Open SFS Roadmap. Presented by David Dillow TWG Co-Chair

Open SFS Roadmap. Presented by David Dillow TWG Co-Chair Open SFS Roadmap Presented by David Dillow TWG Co-Chair TWG Mission Work with the Lustre community to ensure that Lustre continues to support the stability, performance, and management requirements of

More information

WVU RESEARCH COMPUTING INTRODUCTION. Introduction to WVU s Research Computing Services

WVU RESEARCH COMPUTING INTRODUCTION. Introduction to WVU s Research Computing Services WVU RESEARCH COMPUTING INTRODUCTION Introduction to WVU s Research Computing Services WHO ARE WE? Division of Information Technology Services Funded through WVU Research Corporation Provide centralized

More information

Data Movement & Tiering with DMF 7

Data Movement & Tiering with DMF 7 Data Movement & Tiering with DMF 7 Kirill Malkin Director of Engineering April 2019 Why Move or Tier Data? We wish we could keep everything in DRAM, but It s volatile It s expensive Data in Memory 2 Why

More information

Leveraging Software-Defined Storage to Meet Today and Tomorrow s Infrastructure Demands

Leveraging Software-Defined Storage to Meet Today and Tomorrow s Infrastructure Demands Leveraging Software-Defined Storage to Meet Today and Tomorrow s Infrastructure Demands Unleash Your Data Center s Hidden Power September 16, 2014 Molly Rector CMO, EVP Product Management & WW Marketing

More information

IBM Storwize V7000 Unified

IBM Storwize V7000 Unified IBM Storwize V7000 Unified Pavel Müller IBM Systems and Technology Group Storwize V7000 Position Enterprise Block DS8000 For clients requiring: Advanced disaster recovery with 3-way mirroring and System

More information

Building Self-Healing Mass Storage Arrays. for Large Cluster Systems

Building Self-Healing Mass Storage Arrays. for Large Cluster Systems Building Self-Healing Mass Storage Arrays for Large Cluster Systems NSC08, Linköping, 14. October 2008 Toine Beckers tbeckers@datadirectnet.com Agenda Company Overview Balanced I/O Systems MTBF and Availability

More information

INFOBrief. Dell-IBRIX Cluster File System Solution. Key Points

INFOBrief. Dell-IBRIX Cluster File System Solution. Key Points INFOBrief Dell-IBRIX Cluster File System Solution High-performance parallel, segmented file system for scale-out clusters, grid computing, and enterprise applications Capable of delivering linear scalability

More information

Using DDN IME for Harmonie

Using DDN IME for Harmonie Irish Centre for High-End Computing Using DDN IME for Harmonie Gilles Civario, Marco Grossi, Alastair McKinstry, Ruairi Short, Nix McDonnell April 2016 DDN IME: Infinite Memory Engine IME: Major Features

More information

The Blue Water s File/Archive System. Data Management Challenges Michelle Butler

The Blue Water s File/Archive System. Data Management Challenges Michelle Butler The Blue Water s File/Archive System Data Management Challenges Michelle Butler (mbutler@ncsa.illinois.edu) NCSA is a World leader in deploying supercomputers and providing scientists with the software

More information

Experiences with HP SFS / Lustre in HPC Production

Experiences with HP SFS / Lustre in HPC Production Experiences with HP SFS / Lustre in HPC Production Computing Centre (SSCK) University of Karlsruhe Laifer@rz.uni-karlsruhe.de page 1 Outline» What is HP StorageWorks Scalable File Share (HP SFS)? A Lustre

More information

Data Management. Parallel Filesystems. Dr David Henty HPC Training and Support

Data Management. Parallel Filesystems. Dr David Henty HPC Training and Support Data Management Dr David Henty HPC Training and Support d.henty@epcc.ed.ac.uk +44 131 650 5960 Overview Lecture will cover Why is IO difficult Why is parallel IO even worse Lustre GPFS Performance on ARCHER

More information

Overview of HPC at LONI

Overview of HPC at LONI Overview of HPC at LONI Le Yan HPC Consultant User Services @ LONI What Is HPC High performance computing is to use supercomputers to solve problems computationally The most powerful supercomputer today

More information

The Stampede is Coming Welcome to Stampede Introductory Training. Dan Stanzione Texas Advanced Computing Center

The Stampede is Coming Welcome to Stampede Introductory Training. Dan Stanzione Texas Advanced Computing Center The Stampede is Coming Welcome to Stampede Introductory Training Dan Stanzione Texas Advanced Computing Center dan@tacc.utexas.edu Thanks for Coming! Stampede is an exciting new system of incredible power.

More information

Overview of the Texas Advanced Computing Center. Bill Barth TACC September 12, 2011

Overview of the Texas Advanced Computing Center. Bill Barth TACC September 12, 2011 Overview of the Texas Advanced Computing Center Bill Barth TACC September 12, 2011 TACC Mission & Strategic Approach To enable discoveries that advance science and society through the application of advanced

More information

Shared Parallel Filesystems in Heterogeneous Linux Multi-Cluster Environments

Shared Parallel Filesystems in Heterogeneous Linux Multi-Cluster Environments LCI HPC Revolution 2005 26 April 2005 Shared Parallel Filesystems in Heterogeneous Linux Multi-Cluster Environments Matthew Woitaszek matthew.woitaszek@colorado.edu Collaborators Organizations National

More information

Lustre* is designed to achieve the maximum performance and scalability for POSIX applications that need outstanding streamed I/O.

Lustre* is designed to achieve the maximum performance and scalability for POSIX applications that need outstanding streamed I/O. Reference Architecture Designing High-Performance Storage Tiers Designing High-Performance Storage Tiers Intel Enterprise Edition for Lustre* software and Intel Non-Volatile Memory Express (NVMe) Storage

More information

Lustre at the OLCF: Experiences and Path Forward. Galen M. Shipman Group Leader Technology Integration

Lustre at the OLCF: Experiences and Path Forward. Galen M. Shipman Group Leader Technology Integration Lustre at the OLCF: Experiences and Path Forward Galen M. Shipman Group Leader Technology Integration A Demanding Computational Environment Jaguar XT5 18,688 Nodes Jaguar XT4 7,832 Nodes Frost (SGI Ice)

More information

Mission-Critical Lustre at Santos. Adam Fox, Lustre User Group 2016

Mission-Critical Lustre at Santos. Adam Fox, Lustre User Group 2016 Mission-Critical Lustre at Santos Adam Fox, Lustre User Group 2016 About Santos One of the leading oil and gas producers in APAC Founded in 1954 South Australia Northern Territory Oil Search Cooper Basin

More information

irods at TACC: Secure Infrastructure for Open Science Chris Jordan

irods at TACC: Secure Infrastructure for Open Science Chris Jordan irods at TACC: Secure Infrastructure for Open Science Chris Jordan What is TACC? Texas Advanced Computing Center Cyberinfrastructure Resources for Open Science University of Texas System 9 Academic, 6

More information

Managing HPC Active Archive Storage with HPSS RAIT at Oak Ridge National Laboratory

Managing HPC Active Archive Storage with HPSS RAIT at Oak Ridge National Laboratory Managing HPC Active Archive Storage with HPSS RAIT at Oak Ridge National Laboratory Quinn Mitchell HPC UNIX/LINUX Storage Systems ORNL is managed by UT-Battelle for the US Department of Energy U.S. Department

More information

Academic Workflow for Research Repositories Using irods and Object Storage

Academic Workflow for Research Repositories Using irods and Object Storage 1 Academic Workflow for Research Repositories Using irods and Object Storage 2016 irods User s Group Meeting 9 June 2016 Randall Splinter, Ph.D. HPC Research Computing Solutions Architect RSplinter@ 770.633.2994

More information

Three Generations of Linux Dr. Alexander Dunaevskiy

Three Generations of Linux Dr. Alexander Dunaevskiy Three Generations of Linux Server Systems as TSM-Platform at 17.08.2007 Dr. Alexander Dunaevskiy dunaevskiy@lrz.de d LRZ in Words LRZ is the biggest supercomputing centre in Germany LRZ provides a wide

More information

Xyratex ClusterStor6000 & OneStor

Xyratex ClusterStor6000 & OneStor Xyratex ClusterStor6000 & OneStor Proseminar Ein-/Ausgabe Stand der Wissenschaft von Tim Reimer Structure OneStor OneStorSP OneStorAP ''Green'' Advancements ClusterStor6000 About Scale-Out Storage Architecture

More information

Filesystems on SSCK's HP XC6000

Filesystems on SSCK's HP XC6000 Filesystems on SSCK's HP XC6000 Computing Centre (SSCK) University of Karlsruhe Laifer@rz.uni-karlsruhe.de page 1 Overview» Overview of HP SFS at SSCK HP StorageWorks Scalable File Share (SFS) based on

More information

DDN About Us Solving Large Enterprise and Web Scale Challenges

DDN About Us Solving Large Enterprise and Web Scale Challenges 1 DDN About Us Solving Large Enterprise and Web Scale Challenges History Founded in 98 World s Largest Private Storage Company Growing, Profitable, Self Funded Headquarters: Santa Clara and Chatsworth,

More information

LS-DYNA Best-Practices: Networking, MPI and Parallel File System Effect on LS-DYNA Performance

LS-DYNA Best-Practices: Networking, MPI and Parallel File System Effect on LS-DYNA Performance 11 th International LS-DYNA Users Conference Computing Technology LS-DYNA Best-Practices: Networking, MPI and Parallel File System Effect on LS-DYNA Performance Gilad Shainer 1, Tong Liu 2, Jeff Layton

More information

BlueGene/L. Computer Science, University of Warwick. Source: IBM

BlueGene/L. Computer Science, University of Warwick. Source: IBM BlueGene/L Source: IBM 1 BlueGene/L networking BlueGene system employs various network types. Central is the torus interconnection network: 3D torus with wrap-around. Each node connects to six neighbours

More information

DELL Terascala HPC Storage Solution (DT-HSS2)

DELL Terascala HPC Storage Solution (DT-HSS2) DELL Terascala HPC Storage Solution (DT-HSS2) A Dell Technical White Paper Dell Li Ou, Scott Collier Terascala Rick Friedman Dell HPC Solutions Engineering THIS WHITE PAPER IS FOR INFORMATIONAL PURPOSES

More information

I/O Challenges: Todays I/O Challenges for Big Data Analysis. Henry Newman CEO/CTO Instrumental, Inc. April 30, 2013

I/O Challenges: Todays I/O Challenges for Big Data Analysis. Henry Newman CEO/CTO Instrumental, Inc. April 30, 2013 I/O Challenges: Todays I/O Challenges for Big Data Analysis Henry Newman CEO/CTO Instrumental, Inc. April 30, 2013 The Challenge is Archives Big data in HPC means archive and archive translates to a tape

More information

Datura The new HPC-Plant at Albert Einstein Institute

Datura The new HPC-Plant at Albert Einstein Institute Datura The new HPC-Plant at Albert Einstein Institute Nico Budewitz Max Planck Institue for Gravitational Physics, Germany Cluster Day, 2011 Outline 1 History HPC-Plants at AEI -2009 Peyote, Lagavulin,

More information

NetApp: Solving I/O Challenges. Jeff Baxter February 2013

NetApp: Solving I/O Challenges. Jeff Baxter February 2013 NetApp: Solving I/O Challenges Jeff Baxter February 2013 1 High Performance Computing Challenges Computing Centers Challenge of New Science Performance Efficiency directly impacts achievable science Power

More information

A High-Performance Storage and Ultra- High-Speed File Transfer Solution for Collaborative Life Sciences Research

A High-Performance Storage and Ultra- High-Speed File Transfer Solution for Collaborative Life Sciences Research A High-Performance Storage and Ultra- High-Speed File Transfer Solution for Collaborative Life Sciences Research Storage Platforms with Aspera Overview A growing number of organizations with data-intensive

More information

CERN Lustre Evaluation

CERN Lustre Evaluation CERN Lustre Evaluation Arne Wiebalck Sun HPC Workshop, Open Storage Track Regensburg, Germany 8 th Sep 2009 www.cern.ch/it Agenda A Quick Guide to CERN Storage Use Cases Methodology & Initial Findings

More information

Efficient Object Storage Journaling in a Distributed Parallel File System

Efficient Object Storage Journaling in a Distributed Parallel File System Efficient Object Storage Journaling in a Distributed Parallel File System Presented by Sarp Oral Sarp Oral, Feiyi Wang, David Dillow, Galen Shipman, Ross Miller, and Oleg Drokin FAST 10, Feb 25, 2010 A

More information

Dell EMC Ready Bundle for HPC Digital Manufacturing Dassault Systѐmes Simulia Abaqus Performance

Dell EMC Ready Bundle for HPC Digital Manufacturing Dassault Systѐmes Simulia Abaqus Performance Dell EMC Ready Bundle for HPC Digital Manufacturing Dassault Systѐmes Simulia Abaqus Performance This Dell EMC technical white paper discusses performance benchmarking results and analysis for Simulia

More information

High Performance Storage Solutions

High Performance Storage Solutions November 2006 High Performance Storage Solutions Toine Beckers tbeckers@datadirectnet.com www.datadirectnet.com DataDirect Leadership Established 1988 Technology Company (ASICs, FPGA, Firmware, Software)

More information

I/O at the Center for Information Services and High Performance Computing

I/O at the Center for Information Services and High Performance Computing Mich ael Kluge, ZIH I/O at the Center for Information Services and High Performance Computing HPC-I/O in the Data Center Workshop @ ISC 2015 Zellescher Weg 12 Willers-Bau A 208 Tel. +49 351-463 34217 Michael

More information

Application Performance on IME

Application Performance on IME Application Performance on IME Toine Beckers, DDN Marco Grossi, ICHEC Burst Buffer Designs Introduce fast buffer layer Layer between memory and persistent storage Pre-stage application data Buffer writes

More information

Lustre on ZFS. At The University of Wisconsin Space Science and Engineering Center. Scott Nolin September 17, 2013

Lustre on ZFS. At The University of Wisconsin Space Science and Engineering Center. Scott Nolin September 17, 2013 Lustre on ZFS At The University of Wisconsin Space Science and Engineering Center Scott Nolin September 17, 2013 Why use ZFS for Lustre? The University of Wisconsin Space Science and Engineering Center

More information

HPC Hardware Overview

HPC Hardware Overview HPC Hardware Overview John Lockman III April 19, 2013 Texas Advanced Computing Center The University of Texas at Austin Outline Lonestar Dell blade-based system InfiniBand ( QDR) Intel Processors Longhorn

More information

IME (Infinite Memory Engine) Extreme Application Acceleration & Highly Efficient I/O Provisioning

IME (Infinite Memory Engine) Extreme Application Acceleration & Highly Efficient I/O Provisioning IME (Infinite Memory Engine) Extreme Application Acceleration & Highly Efficient I/O Provisioning September 22 nd 2015 Tommaso Cecchi 2 What is IME? This breakthrough, software defined storage application

More information

NFS, GPFS, PVFS, Lustre Batch-scheduled systems: Clusters, Grids, and Supercomputers Programming paradigm: HPC, MTC, and HTC

NFS, GPFS, PVFS, Lustre Batch-scheduled systems: Clusters, Grids, and Supercomputers Programming paradigm: HPC, MTC, and HTC Segregated storage and compute NFS, GPFS, PVFS, Lustre Batch-scheduled systems: Clusters, Grids, and Supercomputers Programming paradigm: HPC, MTC, and HTC Co-located storage and compute HDFS, GFS Data

More information

Shared Services Canada Environment and Climate Change Canada HPC Renewal Project

Shared Services Canada Environment and Climate Change Canada HPC Renewal Project Shared Services Canada Environment and Climate Change Canada HPC Renewal Project CUG 2017 Redmond, WA, USA Deric Sullivan Alain St-Denis & Luc Corbeil May 2017 Background: SSC's HPC Renewal for ECCC Environment

More information

Co-existence: Can Big Data and Big Computation Co-exist on the Same Systems?

Co-existence: Can Big Data and Big Computation Co-exist on the Same Systems? Co-existence: Can Big Data and Big Computation Co-exist on the Same Systems? Dr. William Kramer National Center for Supercomputing Applications, University of Illinois Where these views come from Large

More information

Building Effective CyberGIS: FutureGrid. Marlon Pierce, Geoffrey Fox Indiana University

Building Effective CyberGIS: FutureGrid. Marlon Pierce, Geoffrey Fox Indiana University Building Effective CyberGIS: FutureGrid Marlon Pierce, Geoffrey Fox Indiana University Some Worthy Characteristics of CyberGIS Open Services, algorithms, data, standards, infrastructure Reproducible Can

More information

Dell TM Terascala HPC Storage Solution

Dell TM Terascala HPC Storage Solution Dell TM Terascala HPC Storage Solution A Dell Technical White Paper Li Ou, Scott Collier Dell Massively Scale-Out Systems Team Rick Friedman Terascala THIS WHITE PAPER IS FOR INFORMATIONAL PURPOSES ONLY,

More information

Data storage services at KEK/CRC -- status and plan

Data storage services at KEK/CRC -- status and plan Data storage services at KEK/CRC -- status and plan KEK/CRC Hiroyuki Matsunaga Most of the slides are prepared by Koichi Murakami and Go Iwai KEKCC System Overview KEKCC (Central Computing System) The

More information

The Stampede is Coming: A New Petascale Resource for the Open Science Community

The Stampede is Coming: A New Petascale Resource for the Open Science Community The Stampede is Coming: A New Petascale Resource for the Open Science Community Jay Boisseau Texas Advanced Computing Center boisseau@tacc.utexas.edu Stampede: Solicitation US National Science Foundation

More information

Dell EMC Ready Bundle for HPC Digital Manufacturing ANSYS Performance

Dell EMC Ready Bundle for HPC Digital Manufacturing ANSYS Performance Dell EMC Ready Bundle for HPC Digital Manufacturing ANSYS Performance This Dell EMC technical white paper discusses performance benchmarking results and analysis for ANSYS Mechanical, ANSYS Fluent, and

More information

BeoLink.org. Design and build an inexpensive DFS. Fabrizio Manfredi Furuholmen. FrOSCon August 2008

BeoLink.org. Design and build an inexpensive DFS. Fabrizio Manfredi Furuholmen. FrOSCon August 2008 Design and build an inexpensive DFS Fabrizio Manfredi Furuholmen FrOSCon August 2008 Agenda Overview Introduction Old way openafs New way Hadoop CEPH Conclusion Overview Why Distributed File system? Handle

More information

XtreemStore A SCALABLE STORAGE MANAGEMENT SOFTWARE WITHOUT LIMITS YOUR DATA. YOUR CONTROL

XtreemStore A SCALABLE STORAGE MANAGEMENT SOFTWARE WITHOUT LIMITS YOUR DATA. YOUR CONTROL XtreemStore A SCALABLE STORAGE MANAGEMENT SOFTWARE WITHOUT LIMITS YOUR DATA. YOUR CONTROL Software Produkt Portfolio New Products Product Family Scalable sync & share solution for secure data exchange

More information

Isilon Performance. Name

Isilon Performance. Name 1 Isilon Performance Name 2 Agenda Architecture Overview Next Generation Hardware Performance Caching Performance Streaming Reads Performance Tuning OneFS Architecture Overview Copyright 2014 EMC Corporation.

More information

UK LUG 10 th July Lustre at Exascale. Eric Barton. CTO Whamcloud, Inc Whamcloud, Inc.

UK LUG 10 th July Lustre at Exascale. Eric Barton. CTO Whamcloud, Inc Whamcloud, Inc. UK LUG 10 th July 2012 Lustre at Exascale Eric Barton CTO Whamcloud, Inc. eeb@whamcloud.com Agenda Exascale I/O requirements Exascale I/O model 3 Lustre at Exascale - UK LUG 10th July 2012 Exascale I/O

More information

Crossing the Chasm: Sneaking a parallel file system into Hadoop

Crossing the Chasm: Sneaking a parallel file system into Hadoop Crossing the Chasm: Sneaking a parallel file system into Hadoop Wittawat Tantisiriroj Swapnil Patil, Garth Gibson PARALLEL DATA LABORATORY Carnegie Mellon University In this work Compare and contrast large

More information

Implementing a Hierarchical Storage Management system in a large-scale Lustre and HPSS environment

Implementing a Hierarchical Storage Management system in a large-scale Lustre and HPSS environment Implementing a Hierarchical Storage Management system in a large-scale Lustre and HPSS environment Brett Bode, Michelle Butler, Jim Glasgow, Sean Stevens National Center for Supercomputing Applications

More information

Cluster Setup and Distributed File System

Cluster Setup and Distributed File System Cluster Setup and Distributed File System R&D Storage for the R&D Storage Group People Involved Gaetano Capasso - INFN-Naples Domenico Del Prete INFN-Naples Diacono Domenico INFN-Bari Donvito Giacinto

More information

Lustre TM. Scalability

Lustre TM. Scalability Lustre TM Scalability An Oak Ridge National Laboratory/ Lustre Center of Excellence White Paper February 2009 2 Sun Microsystems, Inc Table of Contents Executive Summary...3 HPC Trends...3 Lustre File

More information

Introduction The Project Lustre Architecture Performance Conclusion References. Lustre. Paul Bienkowski

Introduction The Project Lustre Architecture Performance Conclusion References. Lustre. Paul Bienkowski Lustre Paul Bienkowski 2bienkow@informatik.uni-hamburg.de Proseminar Ein-/Ausgabe - Stand der Wissenschaft 2013-06-03 1 / 34 Outline 1 Introduction 2 The Project Goals and Priorities History Who is involved?

More information

Next-Generation NVMe-Native Parallel Filesystem for Accelerating HPC Workloads

Next-Generation NVMe-Native Parallel Filesystem for Accelerating HPC Workloads Next-Generation NVMe-Native Parallel Filesystem for Accelerating HPC Workloads Liran Zvibel CEO, Co-founder WekaIO @liranzvibel 1 WekaIO Matrix: Full-featured and Flexible Public or Private S3 Compatible

More information