Blue Waters System Overview. Greg Bauer

Size: px
Start display at page:

Download "Blue Waters System Overview. Greg Bauer"

Transcription

1 Blue Waters System Overview Greg Bauer

2 The Blue Waters EcoSystem Petascale EducaIon, Industry and Outreach Petascale ApplicaIons (CompuIng Resource AllocaIons) Petascale ApplicaIon CollaboraIon Team Support Outstanding User and ProducIon Support WAN ConnecIons, ConsulIng, System Management, Security, OperaIons, Great Lakes ConsorIum for Petascale CompuIng Value added hardware (external networking, IDS, nearline storage, import/export, etc) and so9ware (Eclipse, HPSS, VisualizaIon, computaional libraries, etc) Blue Waters Base System Processors, Memory, Interconnect, Online Storage, System So9ware, Programming Environment NaIonal Petascale CompuIng Facility 2

3 Blue Waters Computing System Aggregate Memory 1.5 PB Scuba Subsystem Storage Configuration for User Best Access 120+ Gb/sec Gbps WAN 10/40/100 Gb Ethernet Switch IB Switch External Servers >1 TB/sec 100 GB/sec Spectra Logic: 300 usable PB Sonexion: 26 usable PB 3

4 National Petascale Computing Facility Modern Data Center 90,000+ ft 2 total 30,000 ft 2 6 foot raised floor 20,000 ft 2 machine room gallery with no obstructions or structural support elements Energy Efficiency LEED certified Gold Power Utilization Efficiency, PUE = MW current capacity expandable Highly instrumented 4

5 PRAC PI Institutions June 2012 Note UIUC is only institution leading more than one PRAC (4) UIUC has Co-PIs on several others 5

6 The Movement of Data I2 NPCF ICCN DWDM ICCN L3 MREN GLCPC/ Science teams Gb/s XSEDE NLR framenet XSEDE RP sites NLR OmniPop ESnet 6

7 Gemini Fabric (HSN) DSL 48 Nodes MOM 64 Nodes BOOT Nodes SDB Nodes XE6 Compute Nodes 5,688 Blades 22,640 Nodes 362,240 Cores RSIP Nodes Network GW Nodes Unassigned Nodes Cray XE6/XK7-276 Cabinets XK7 GPU Nodes 768 Blades 3,072 Nodes 24,576 Cores 3,072 GPUs LNET Routers 582 Nodes Boot RAID SMW Boot Cabinet eslogin 4 Nodes Import/Export 28 Nodes InfiniBand fabric 10/40/100 Gb Ethernet Switch Cyber ProtecIon IDPS HPSS Data Mover 50 Nodes Management Node Sonexion 25+PB online storage OSTs NCSAnet esservers Cabinets Near- Line Storage 300+PB NPCF 7

8 QDR IB 1200 GB/s Online disk >25PB /home, /project /scratch 300 GB/s LNET(s) Protocols LNET Core IB Switches TCP/IP (10 GbE)) GridFTP (TCP/IP)) SCSI (FCP) Cray Gemini High Speed Network Network GW 100 GB/s eslogin I/E(s) 100 GB/s 10 GbE 40GbE Switch 440 Gb/s Ethernet from site network LAN/WAN rsip GW 55 GB/s 100 GB/s 100 GB/s 1.2PB Disk IB 40 GbE Near Line 300+PB Tape FC8 100 GB/s All storage sizes given as the amount usable 8

9 Blue Waters Nearline/Archive System Spectra Logic T-Finity Dual-arm robotic tape libraries High availability and reliability, with built-in redundancy Blue Waters Archive Capacity: 380 PBs (raw), 300 PBs (usable) Bandwidth: 100 GB/sec (sustained) Redundant arrays of independent tapes RAIT for increased reliability 9

10 Largest Nearline environment LAN 10GigE HPSS core (DB2) 1.2PB Disk IB HPSS(S) 100 GB/s 100 GB/s 300+PB Tape FC8 HPSS Data mover QDR IB switch 366 IBM TS1140 Jag tape drives Six 15,930-slot dual-arm libraries 300+PB data storage HPSS disk cache 1.2 PB 10

11 Online Storage home : 144 OSTs : 2.2 PB useable : 1 TB quota projects: 144 OSTs : 2.2 PB useable : 3 TB group quota scratch: 1440 OSTs : 22 PB useable : 500 TB group quota Cray Sonexion with Lustre for all filesystems. All visible from compute nodes. 11

12 GO with Globus Online GridFTP client development for IE and HPSS nodes. Enabled data striping with GridFTP. Managed file transfers. Command line interface. Globus connect for sites without GridFTP endpoints. 12

13 Blue Waters High Speed Network InfiniBand Login Server(s) Network(s) Y GigE Fibre Channel SMW X Z Interconnect Network Infiniband Boot Raid Lustre Blue Waters 3D Torus Size 23 x 24 x 24 Compute Nodes Cray XE6 Compute Cray XK7 Accelerator Operating System Boot System Database Service Nodes Lustre File System Login/Network LNET Routers Login Gateways Network 13

14 Blue Waters XE6 Node Blue Waters contains 22,640 XE6 compute nodes Node Characteris_cs Number of Core Modules* Peak Performance Gflops/sec HT3 HT3 Memory Size 64 GB per node Memory Bandwidth (Peak) Interconnect InjecIon Bandwidth (Peak) 102 GB/sec 9.6 GB/sec per direcion *Each core module includes bit wide FP unit and 2 integer units. This is often advertised as 2 cores, leading to a 32 core node. Z X Y 14

15 Cray XK7 and a Path to the Future Blue Waters contains 3,072 NVIDIA Kepler (GK110) GPUs XK7 Compute Node Characteris_cs Host Processor AMD Series 6200 (Interlagos) PCIe Gen2 Host Processor Performance Gflops HT3 HT3 Kepler Peak (DP floaing point) Host Memory > 1.3 Tflops 32GB 51 GB/sec Z Y Kepler Memory 6GB GDDR5 capacity > 180 GB/sec X 15

16 Blue Waters Software Environment Languages Compilers Programming Models IO Libraries Tools Optimized Scientific Libraries Fortran C C++ Python UPC Performance Analysis Cray Compiling Environment (CCE) GNU Debuggers Distributed Memory (Cray MPT) MPI SHMEM Shared Memory OpenMP 3.0 NetCDF HDF5 ADIOS Resource Manager Environment setup Modules Debugging Support Tools Fast Track Debugger (CCE w/ DDT) Abnormal Termination Processing LAPACK ScaLAPACK BLAS (libgoto) Iterative Refinement Toolkit Cray Performance Monitoring and Analysis Tool PAPI PerfSuite Tau Allinea DDT lgdb Prog. Env. Eclipse Traditional PGAS & Global View UPC (CCE) CAF (CCE) Visualization VisIt Paraview YT STAT Cray Comparative Debugger # Data Transfer GO HPSS Cray Adaptive FFTs (CRAFFT) FFTW Cray PETSc (with CASK) Cray Trilinos (with CASK) Cray Linux Environment (CLE)/SUSE Linux Cray developed Under development Licensed ISV SW 3 rd party packaging NCSA supported Cray added value to 3 rd party 16

17 Summary Outstanding Computing System The largest installation of Cray s most advanced technology Extreme-scale Lustre file system with advances in reliability/ maintainability Extreme-scale archive with advanced RAIT capability Most balanced system in the open community Blue Waters is capable of addressing science problems that are memory, storage, compute, or network intensive or any combination. Use of innovative technologies provides a path to future systems Illinois/NCSA is a leader in developing and deploying these technologies as well as contributing to community efforts. 17

Blue Waters System Overview

Blue Waters System Overview Blue Waters System Overview Blue Waters Computing System Aggregate Memory 1.5 PB Scuba Subsystem Storage Configuration for User Best Access 120+ Gb/sec 100-300 Gbps WAN 10/40/100 Gb Ethernet Switch IB

More information

Blue Waters Super System

Blue Waters Super System Blue Waters Super System Michelle Butler 4/12/12 NCSA Has Completed a Grand Challenge In August, IBM terminated their contract to deliver the base Blue Waters system NSF asked NCSA to propose a change

More information

Implementing a Hierarchical Storage Management system in a large-scale Lustre and HPSS environment

Implementing a Hierarchical Storage Management system in a large-scale Lustre and HPSS environment Implementing a Hierarchical Storage Management system in a large-scale Lustre and HPSS environment Brett Bode, Michelle Butler, Sean Stevens, Jim Glasgow National Center for Supercomputing Applications/University

More information

User Training Cray XC40 IITM, Pune

User Training Cray XC40 IITM, Pune User Training Cray XC40 IITM, Pune Sudhakar Yerneni, Raviteja K, Nachiket Manapragada, etc. 1 Cray XC40 Architecture & Packaging 3 Cray XC Series Building Blocks XC40 System Compute Blade 4 Compute Nodes

More information

Is Petascale Complete? What Do We Do Now?

Is Petascale Complete? What Do We Do Now? Is Petascale Complete? What Do We Do Now? Dr. William Kramer National Center for Supercomputing Applications, University of Illinois Blue Waters Computing System Aggregate Memory 1.6 PB 10/40/100 Gb Ethernet

More information

Adventures in Parallelization On One of the Largest Supercomputers in The Country. Nathan Sloat

Adventures in Parallelization On One of the Largest Supercomputers in The Country. Nathan Sloat Adventures in Parallelization On One of the Largest Supercomputers in The Country Nathan Sloat RNA Secondary Structure RNA Base units: nucleotides (nt) Four nucleotides: adenosine, cytosine, guanosine,

More information

The Cray Programming Environment. An Introduction

The Cray Programming Environment. An Introduction The Cray Programming Environment An Introduction Vision Cray systems are designed to be High Productivity as well as High Performance Computers The Cray Programming Environment (PE) provides a simple consistent

More information

Lecture 20: Distributed Memory Parallelism. William Gropp

Lecture 20: Distributed Memory Parallelism. William Gropp Lecture 20: Distributed Parallelism William Gropp www.cs.illinois.edu/~wgropp A Very Short, Very Introductory Introduction We start with a short introduction to parallel computing from scratch in order

More information

The Arm Technology Ecosystem: Current Products and Future Outlook

The Arm Technology Ecosystem: Current Products and Future Outlook The Arm Technology Ecosystem: Current Products and Future Outlook Dan Ernst, PhD Advanced Technology Cray, Inc. Why is an Ecosystem Important? An Ecosystem is a collection of common material Developed

More information

8/19/13. Blue Waters User Monthly Teleconference

8/19/13. Blue Waters User Monthly Teleconference 8/19/13 Blue Waters User Monthly Teleconference Extreme Scaling Workshop 2013 Successful workshop in Boulder. Presentations from 4 groups with allocations on Blue Waters. Industry representatives were

More information

Programming Environment 4/11/2015

Programming Environment 4/11/2015 Programming Environment 4/11/2015 1 Vision Cray systems are designed to be High Productivity as well as High Performance Computers The Cray Programming Environment (PE) provides a simple consistent interface

More information

DVS, GPFS and External Lustre at NERSC How It s Working on Hopper. Tina Butler, Rei Chi Lee, Gregory Butler 05/25/11 CUG 2011

DVS, GPFS and External Lustre at NERSC How It s Working on Hopper. Tina Butler, Rei Chi Lee, Gregory Butler 05/25/11 CUG 2011 DVS, GPFS and External Lustre at NERSC How It s Working on Hopper Tina Butler, Rei Chi Lee, Gregory Butler 05/25/11 CUG 2011 1 NERSC is the Primary Computing Center for DOE Office of Science NERSC serves

More information

The Cray Programming Environment. An Introduction

The Cray Programming Environment. An Introduction The Cray Programming Environment An Introduction Vision Cray systems are designed to be High Productivity as well as High Performance Computers The Cray Programming Environment (PE) provides a simple consistent

More information

Preparing GPU-Accelerated Applications for the Summit Supercomputer

Preparing GPU-Accelerated Applications for the Summit Supercomputer Preparing GPU-Accelerated Applications for the Summit Supercomputer Fernanda Foertter HPC User Assistance Group Training Lead foertterfs@ornl.gov This research used resources of the Oak Ridge Leadership

More information

Titan - Early Experience with the Titan System at Oak Ridge National Laboratory

Titan - Early Experience with the Titan System at Oak Ridge National Laboratory Office of Science Titan - Early Experience with the Titan System at Oak Ridge National Laboratory Buddy Bland Project Director Oak Ridge Leadership Computing Facility November 13, 2012 ORNL s Titan Hybrid

More information

Cori (2016) and Beyond Ensuring NERSC Users Stay Productive

Cori (2016) and Beyond Ensuring NERSC Users Stay Productive Cori (2016) and Beyond Ensuring NERSC Users Stay Productive Nicholas J. Wright! Advanced Technologies Group Lead! Heterogeneous Mul-- Core 4 Workshop 17 September 2014-1 - NERSC Systems Today Edison: 2.39PF,

More information

Introducing the next generation of affordable and productive massively parallel processing (MPP) computing the Cray XE6m supercomputer.

Introducing the next generation of affordable and productive massively parallel processing (MPP) computing the Cray XE6m supercomputer. Introducing the next generation of affordable and productive massively parallel processing (MPP) computing the Cray XE6m supercomputer. Building on the reliability and scalability of the Cray XE6 supercomputer

More information

Illinois Proposal Considerations Greg Bauer

Illinois Proposal Considerations Greg Bauer - 2016 Greg Bauer Support model Blue Waters provides traditional Partner Consulting as part of its User Services. Standard service requests for assistance with porting, debugging, allocation issues, and

More information

Present and Future Leadership Computers at OLCF

Present and Future Leadership Computers at OLCF Present and Future Leadership Computers at OLCF Al Geist ORNL Corporate Fellow DOE Data/Viz PI Meeting January 13-15, 2015 Walnut Creek, CA ORNL is managed by UT-Battelle for the US Department of Energy

More information

Wednesday : Basic Overview. Thursday : Optimization

Wednesday : Basic Overview. Thursday : Optimization Cray Inc. Wednesday : Basic Overview XT Architecture XT Programming Environment XT MPT : CRAY MPI Cray Scientific Libraries CRAYPAT : Basic HOWTO Handons Thursday : Optimization Where and How to Optimize

More information

The Blue Water s File/Archive System. Data Management Challenges Michelle Butler

The Blue Water s File/Archive System. Data Management Challenges Michelle Butler The Blue Water s File/Archive System Data Management Challenges Michelle Butler (mbutler@ncsa.illinois.edu) NCSA is a World leader in deploying supercomputers and providing scientists with the software

More information

Portable and Productive Performance with OpenACC Compilers and Tools. Luiz DeRose Sr. Principal Engineer Programming Environments Director Cray Inc.

Portable and Productive Performance with OpenACC Compilers and Tools. Luiz DeRose Sr. Principal Engineer Programming Environments Director Cray Inc. Portable and Productive Performance with OpenACC Compilers and Tools Luiz DeRose Sr. Principal Engineer Programming Environments Director Cray Inc. 1 Cray: Leadership in Computational Research Earth Sciences

More information

High Performance Computing and Data Resources at SDSC

High Performance Computing and Data Resources at SDSC High Performance Computing and Data Resources at SDSC "! Mahidhar Tatineni (mahidhar@sdsc.edu)! SDSC Summer Institute! August 05, 2013! HPC Resources at SDSC Hardware Overview HPC Systems : Gordon, Trestles

More information

Blue Waters I/O Performance

Blue Waters I/O Performance Blue Waters I/O Performance Mark Swan Performance Group Cray Inc. Saint Paul, Minnesota, USA mswan@cray.com Doug Petesch Performance Group Cray Inc. Saint Paul, Minnesota, USA dpetesch@cray.com Abstract

More information

HPC Technology Update Challenges or Chances?

HPC Technology Update Challenges or Chances? HPC Technology Update Challenges or Chances? Swiss Distributed Computing Day Thomas Schoenemeyer, Technology Integration, CSCS 1 Move in Feb-April 2012 1500m2 16 MW Lake-water cooling PUE 1.2 New Datacenter

More information

CRAY XK6 REDEFINING SUPERCOMPUTING. - Sanjana Rakhecha - Nishad Nerurkar

CRAY XK6 REDEFINING SUPERCOMPUTING. - Sanjana Rakhecha - Nishad Nerurkar CRAY XK6 REDEFINING SUPERCOMPUTING - Sanjana Rakhecha - Nishad Nerurkar CONTENTS Introduction History Specifications Cray XK6 Architecture Performance Industry acceptance and applications Summary INTRODUCTION

More information

CS500 SMARTER CLUSTER SUPERCOMPUTERS

CS500 SMARTER CLUSTER SUPERCOMPUTERS CS500 SMARTER CLUSTER SUPERCOMPUTERS OVERVIEW Extending the boundaries of what you can achieve takes reliable computing tools matched to your workloads. That s why we tailor the Cray CS500 cluster supercomputer

More information

Productive Performance on the Cray XK System Using OpenACC Compilers and Tools

Productive Performance on the Cray XK System Using OpenACC Compilers and Tools Productive Performance on the Cray XK System Using OpenACC Compilers and Tools Luiz DeRose Sr. Principal Engineer Programming Environments Director Cray Inc. 1 The New Generation of Supercomputers Hybrid

More information

Basic Specification of Oakforest-PACS

Basic Specification of Oakforest-PACS Basic Specification of Oakforest-PACS Joint Center for Advanced HPC (JCAHPC) by Information Technology Center, the University of Tokyo and Center for Computational Sciences, University of Tsukuba Oakforest-PACS

More information

Sun Lustre Storage System Simplifying and Accelerating Lustre Deployments

Sun Lustre Storage System Simplifying and Accelerating Lustre Deployments Sun Lustre Storage System Simplifying and Accelerating Lustre Deployments Torben Kling-Petersen, PhD Presenter s Name Principle Field Title andengineer Division HPC &Cloud LoB SunComputing Microsystems

More information

Bridging the Gap Between High Quality and High Performance for HPC Visualization

Bridging the Gap Between High Quality and High Performance for HPC Visualization Bridging the Gap Between High Quality and High Performance for HPC Visualization Rob Sisneros National Center for Supercomputing Applications University of Illinois at Urbana Champaign Outline Why am I

More information

Cray Operating System Plans and Status. Charlie Carroll May 2012

Cray Operating System Plans and Status. Charlie Carroll May 2012 Cray Operating System Plans and Status Charlie Carroll May 2012 Cray Operating Systems and I/O Compute Node Linux NVIDIA GPU driver Compute node Service node OS File systems: Lustre Networking HSN: Gemini

More information

Managing HPC Active Archive Storage with HPSS RAIT at Oak Ridge National Laboratory

Managing HPC Active Archive Storage with HPSS RAIT at Oak Ridge National Laboratory Managing HPC Active Archive Storage with HPSS RAIT at Oak Ridge National Laboratory Quinn Mitchell HPC UNIX/LINUX Storage Systems ORNL is managed by UT-Battelle for the US Department of Energy U.S. Department

More information

Designed for Maximum Accelerator Performance

Designed for Maximum Accelerator Performance Designed for Maximum Accelerator Performance A dense, GPU-accelerated cluster supercomputer that delivers up to 329 double-precision GPU teraflops in one rack. This power- and spaceefficient system can

More information

HPC Saudi Jeffrey A. Nichols Associate Laboratory Director Computing and Computational Sciences. Presented to: March 14, 2017

HPC Saudi Jeffrey A. Nichols Associate Laboratory Director Computing and Computational Sciences. Presented to: March 14, 2017 Creating an Exascale Ecosystem for Science Presented to: HPC Saudi 2017 Jeffrey A. Nichols Associate Laboratory Director Computing and Computational Sciences March 14, 2017 ORNL is managed by UT-Battelle

More information

Pedraforca: a First ARM + GPU Cluster for HPC

Pedraforca: a First ARM + GPU Cluster for HPC www.bsc.es Pedraforca: a First ARM + GPU Cluster for HPC Nikola Puzovic, Alex Ramirez We ve hit the power wall ALL computers are limited by power consumption Energy-efficient approaches Multi-core Fujitsu

More information

Introduction to SahasraT. RAVITEJA K Applications Analyst, Cray inc E Mail :

Introduction to SahasraT. RAVITEJA K Applications Analyst, Cray inc E Mail : Introduction to SahasraT RAVITEJA K Applications Analyst, Cray inc E Mail : raviteja@cray.com 1 1. Introduction to SahasraT 2. Cray Software stack 3. Compile applications on XC 4. Run applications on XC

More information

Our Workshop Environment

Our Workshop Environment Our Workshop Environment John Urbanic Parallel Computing Scientist Pittsburgh Supercomputing Center Copyright 2015 Our Environment Today Your laptops or workstations: only used for portal access Blue Waters

More information

The Red Storm System: Architecture, System Update and Performance Analysis

The Red Storm System: Architecture, System Update and Performance Analysis The Red Storm System: Architecture, System Update and Performance Analysis Douglas Doerfler, Jim Tomkins Sandia National Laboratories Center for Computation, Computers, Information and Mathematics LACSI

More information

Breakthrough Science via Extreme Scalability. Greg Clifford Segment Manager, Cray Inc.

Breakthrough Science via Extreme Scalability. Greg Clifford Segment Manager, Cray Inc. Breakthrough Science via Extreme Scalability Greg Clifford Segment Manager, Cray Inc. clifford@cray.com Cray s focus The requirement for highly scalable systems Cray XE6 technology The path to Exascale

More information

Smarter Clusters from the Supercomputer Experts

Smarter Clusters from the Supercomputer Experts Smarter Clusters from the Supercomputer Experts Maximize Your Results with Flexible, High-Performance Cray CS500 Cluster Supercomputers In science and business, as soon as one question is answered another

More information

GPFS on a Cray XT. Shane Canon Data Systems Group Leader Lawrence Berkeley National Laboratory CUG 2009 Atlanta, GA May 4, 2009

GPFS on a Cray XT. Shane Canon Data Systems Group Leader Lawrence Berkeley National Laboratory CUG 2009 Atlanta, GA May 4, 2009 GPFS on a Cray XT Shane Canon Data Systems Group Leader Lawrence Berkeley National Laboratory CUG 2009 Atlanta, GA May 4, 2009 Outline NERSC Global File System GPFS Overview Comparison of Lustre and GPFS

More information

HPC Hardware Overview

HPC Hardware Overview HPC Hardware Overview John Lockman III April 19, 2013 Texas Advanced Computing Center The University of Texas at Austin Outline Lonestar Dell blade-based system InfiniBand ( QDR) Intel Processors Longhorn

More information

NCAR Globally Accessible Data Environment (GLADE) Updated: 15 Feb 2017

NCAR Globally Accessible Data Environment (GLADE) Updated: 15 Feb 2017 NCAR Globally Accessible Data Environment (GLADE) Updated: 15 Feb 2017 Overview The Globally Accessible Data Environment (GLADE) provides centralized file storage for HPC computational, data-analysis,

More information

Scalable Computing at Work

Scalable Computing at Work CRAY XT4 DATASHEET Scalable Computing at Work Cray XT4 Supercomputer Introducing the latest generation massively parallel processor (MPP) system from Cray the Cray XT4 supercomputer. Building on the success

More information

Cray XD1 Supercomputer Release 1.3 CRAY XD1 DATASHEET

Cray XD1 Supercomputer Release 1.3 CRAY XD1 DATASHEET CRAY XD1 DATASHEET Cray XD1 Supercomputer Release 1.3 Purpose-built for HPC delivers exceptional application performance Affordable power designed for a broad range of HPC workloads and budgets Linux,

More information

LCE: Lustre at CEA. Stéphane Thiell CEA/DAM

LCE: Lustre at CEA. Stéphane Thiell CEA/DAM LCE: Lustre at CEA Stéphane Thiell CEA/DAM (stephane.thiell@cea.fr) 1 Lustre at CEA: Outline Lustre at CEA updates (2009) Open Computing Center (CCRT) updates CARRIOCAS (Lustre over WAN) project 2009-2010

More information

An Introduction to OpenACC

An Introduction to OpenACC An Introduction to OpenACC Alistair Hart Cray Exascale Research Initiative Europe 3 Timetable Day 1: Wednesday 29th August 2012 13:00 Welcome and overview 13:15 Session 1: An Introduction to OpenACC 13:15

More information

Short Talk: System abstractions to facilitate data movement in supercomputers with deep memory and interconnect hierarchy

Short Talk: System abstractions to facilitate data movement in supercomputers with deep memory and interconnect hierarchy Short Talk: System abstractions to facilitate data movement in supercomputers with deep memory and interconnect hierarchy François Tessier, Venkatram Vishwanath Argonne National Laboratory, USA July 19,

More information

The Stampede is Coming Welcome to Stampede Introductory Training. Dan Stanzione Texas Advanced Computing Center

The Stampede is Coming Welcome to Stampede Introductory Training. Dan Stanzione Texas Advanced Computing Center The Stampede is Coming Welcome to Stampede Introductory Training Dan Stanzione Texas Advanced Computing Center dan@tacc.utexas.edu Thanks for Coming! Stampede is an exciting new system of incredible power.

More information

Using SDSC Systems (part 2)

Using SDSC Systems (part 2) Using SDSC Systems (part 2) Running vsmp jobs, Data Transfer, I/O SDSC Summer Institute August 6-10 2012 Mahidhar Tatineni San Diego Supercomputer Center " 1 vsmp Runtime Guidelines: Overview" Identify

More information

The Eclipse Parallel Tools Platform

The Eclipse Parallel Tools Platform May 1, 2012 Toward an Integrated Development Environment for Improved Software Engineering on Crays Agenda 1. What is the Eclipse Parallel Tools Platform (PTP) 2. Tour of features available in Eclipse/PTP

More information

Use of Common Technologies between XT and Black Widow

Use of Common Technologies between XT and Black Widow Use of Common Technologies between XT and Black Widow CUG 2006 This Presentation May Contain Some Preliminary Information, Subject To Change Agenda System Architecture Directions Software Development and

More information

It s a Multicore World. John Urbanic Pittsburgh Supercomputing Center Parallel Computing Scientist

It s a Multicore World. John Urbanic Pittsburgh Supercomputing Center Parallel Computing Scientist It s a Multicore World John Urbanic Pittsburgh Supercomputing Center Parallel Computing Scientist Waiting for Moore s Law to save your serial code started getting bleak in 2004 Source: published SPECInt

More information

Overview of the Texas Advanced Computing Center. Bill Barth TACC September 12, 2011

Overview of the Texas Advanced Computing Center. Bill Barth TACC September 12, 2011 Overview of the Texas Advanced Computing Center Bill Barth TACC September 12, 2011 TACC Mission & Strategic Approach To enable discoveries that advance science and society through the application of advanced

More information

Introduction of Oakforest-PACS

Introduction of Oakforest-PACS Introduction of Oakforest-PACS Hiroshi Nakamura Director of Information Technology Center The Univ. of Tokyo (Director of JCAHPC) Outline Supercomputer deployment plan in Japan What is JCAHPC? Oakforest-PACS

More information

Xyratex ClusterStor6000 & OneStor

Xyratex ClusterStor6000 & OneStor Xyratex ClusterStor6000 & OneStor Proseminar Ein-/Ausgabe Stand der Wissenschaft von Tim Reimer Structure OneStor OneStorSP OneStorAP ''Green'' Advancements ClusterStor6000 About Scale-Out Storage Architecture

More information

ZEST Snapshot Service. A Highly Parallel Production File System by the PSC Advanced Systems Group Pittsburgh Supercomputing Center 1

ZEST Snapshot Service. A Highly Parallel Production File System by the PSC Advanced Systems Group Pittsburgh Supercomputing Center 1 ZEST Snapshot Service A Highly Parallel Production File System by the PSC Advanced Systems Group Pittsburgh Supercomputing Center 1 Design Motivation To optimize science utilization of the machine Maximize

More information

John Fragalla TACC 'RANGER' INFINIBAND ARCHITECTURE WITH SUN TECHNOLOGY. Presenter s Name Title and Division Sun Microsystems

John Fragalla TACC 'RANGER' INFINIBAND ARCHITECTURE WITH SUN TECHNOLOGY. Presenter s Name Title and Division Sun Microsystems TACC 'RANGER' INFINIBAND ARCHITECTURE WITH SUN TECHNOLOGY SUBTITLE WITH TWO LINES OF TEXT IF NECESSARY John Fragalla Presenter s Name Title and Division Sun Microsystems Principle Engineer High Performance

More information

Storage Supporting DOE Science

Storage Supporting DOE Science Storage Supporting DOE Science Jason Hick jhick@lbl.gov NERSC LBNL http://www.nersc.gov/nusers/systems/hpss/ http://www.nersc.gov/nusers/systems/ngf/ May 12, 2011 The Production Facility for DOE Office

More information

Computer Science Section. Computational and Information Systems Laboratory National Center for Atmospheric Research

Computer Science Section. Computational and Information Systems Laboratory National Center for Atmospheric Research Computer Science Section Computational and Information Systems Laboratory National Center for Atmospheric Research My work in the context of TDD/CSS/ReSET Polynya new research computing environment Polynya

More information

Scaling Across the Supercomputer Performance Spectrum

Scaling Across the Supercomputer Performance Spectrum Scaling Across the Supercomputer Performance Spectrum Cray s XC40 system leverages the combined advantages of next-generation Aries interconnect and Dragonfly network topology, Intel Xeon processors, integrated

More information

Current Status of the Next- Generation Supercomputer in Japan. YOKOKAWA, Mitsuo Next-Generation Supercomputer R&D Center RIKEN

Current Status of the Next- Generation Supercomputer in Japan. YOKOKAWA, Mitsuo Next-Generation Supercomputer R&D Center RIKEN Current Status of the Next- Generation Supercomputer in Japan YOKOKAWA, Mitsuo Next-Generation Supercomputer R&D Center RIKEN International Workshop on Peta-Scale Computing Programming Environment, Languages

More information

Resources Current and Future Systems. Timothy H. Kaiser, Ph.D.

Resources Current and Future Systems. Timothy H. Kaiser, Ph.D. Resources Current and Future Systems Timothy H. Kaiser, Ph.D. tkaiser@mines.edu 1 Most likely talk to be out of date History of Top 500 Issues with building bigger machines Current and near future academic

More information

Cray Scientific Libraries: Overview and Performance. Cray XE6 Performance Workshop University of Reading Nov 2012

Cray Scientific Libraries: Overview and Performance. Cray XE6 Performance Workshop University of Reading Nov 2012 Cray Scientific Libraries: Overview and Performance Cray XE6 Performance Workshop University of Reading 20-22 Nov 2012 Contents LibSci overview and usage BFRAME / CrayBLAS LAPACK ScaLAPACK FFTW / CRAFFT

More information

ASKAP Central Processor: Design and Implementa8on

ASKAP Central Processor: Design and Implementa8on ASKAP Central Processor: Design and Implementa8on Calibra8on and Imaging Workshop 2014 Ben Humphreys ASKAP So(ware and Compu3ng Project Engineer 3rd - 7th March 2014 ASTRONOMY AND SPACE SCIENCE Australian

More information

Lessons learned from Lustre file system operation

Lessons learned from Lustre file system operation Lessons learned from Lustre file system operation Roland Laifer STEINBUCH CENTRE FOR COMPUTING - SCC KIT University of the State of Baden-Württemberg and National Laboratory of the Helmholtz Association

More information

The NCAR Yellowstone Data Centric Computing Environment. Rory Kelly ScicomP Workshop May 2013

The NCAR Yellowstone Data Centric Computing Environment. Rory Kelly ScicomP Workshop May 2013 The NCAR Yellowstone Data Centric Computing Environment Rory Kelly ScicomP Workshop 27 31 May 2013 Computers to Data Center EVERYTHING IS NEW 2 NWSC Procurement New facility: the NWSC NCAR Wyoming Supercomputing

More information

User Orientation on Cray XC40 SERC, IISc

User Orientation on Cray XC40 SERC, IISc User Orientation on Cray XC40 SERC, IISc Sudhakar Yerneni & Patricia Balle C O M P U T E S T O R E A N A L Y Z E Copyright 2014 Cray Inc. 1 Agenda Introduction to Cray XC40 architecture. IISc's Cray system

More information

Interconnect Your Future

Interconnect Your Future Interconnect Your Future Gilad Shainer 2nd Annual MVAPICH User Group (MUG) Meeting, August 2014 Complete High-Performance Scalable Interconnect Infrastructure Comprehensive End-to-End Software Accelerators

More information

Comet Virtualization Code & Design Sprint

Comet Virtualization Code & Design Sprint Comet Virtualization Code & Design Sprint SDSC September 23-24 Rick Wagner San Diego Supercomputer Center Meeting Goals Build personal connections between the IU and SDSC members of the Comet team working

More information

MegaGauss (MGs) Cluster Design Overview

MegaGauss (MGs) Cluster Design Overview MegaGauss (MGs) Cluster Design Overview NVIDIA Tesla (Fermi) S2070 Modules Based Solution Version 6 (Apr 27, 2010) Alexander S. Zaytsev p. 1 of 15: "Title" Front view: planar

More information

Outline. March 5, 2012 CIRMMT - McGill University 2

Outline. March 5, 2012 CIRMMT - McGill University 2 Outline CLUMEQ, Calcul Quebec and Compute Canada Research Support Objectives and Focal Points CLUMEQ Site at McGill ETS Key Specifications and Status CLUMEQ HPC Support Staff at McGill Getting Started

More information

SuperMike-II Launch Workshop. System Overview and Allocations

SuperMike-II Launch Workshop. System Overview and Allocations : System Overview and Allocations Dr Jim Lupo CCT Computational Enablement jalupo@cct.lsu.edu SuperMike-II: Serious Heterogeneous Computing Power System Hardware SuperMike provides 442 nodes, 221TB of

More information

Data Management. Parallel Filesystems. Dr David Henty HPC Training and Support

Data Management. Parallel Filesystems. Dr David Henty HPC Training and Support Data Management Dr David Henty HPC Training and Support d.henty@epcc.ed.ac.uk +44 131 650 5960 Overview Lecture will cover Why is IO difficult Why is parallel IO even worse Lustre GPFS Performance on ARCHER

More information

Debugging CUDA Applications with Allinea DDT. Ian Lumb Sr. Systems Engineer, Allinea Software Inc.

Debugging CUDA Applications with Allinea DDT. Ian Lumb Sr. Systems Engineer, Allinea Software Inc. Debugging CUDA Applications with Allinea DDT Ian Lumb Sr. Systems Engineer, Allinea Software Inc. ilumb@allinea.com GTC 2013, San Jose, March 20, 2013 Embracing GPUs GPUs a rival to traditional processors

More information

Design and Evaluation of a 2048 Core Cluster System

Design and Evaluation of a 2048 Core Cluster System Design and Evaluation of a 2048 Core Cluster System, Torsten Höfler, Torsten Mehlan and Wolfgang Rehm Computer Architecture Group Department of Computer Science Chemnitz University of Technology December

More information

Cray RS Programming Environment

Cray RS Programming Environment Cray RS Programming Environment Gail Alverson Cray Inc. Cray Proprietary Red Storm Red Storm is a supercomputer system leveraging over 10,000 AMD Opteron processors connected by an innovative high speed,

More information

The Spider Center-Wide File System

The Spider Center-Wide File System The Spider Center-Wide File System Presented by Feiyi Wang (Ph.D.) Technology Integration Group National Center of Computational Sciences Galen Shipman (Group Lead) Dave Dillow, Sarp Oral, James Simmons,

More information

DATARMOR: Comment s'y préparer? Tina Odaka

DATARMOR: Comment s'y préparer? Tina Odaka DATARMOR: Comment s'y préparer? Tina Odaka 30.09.2016 PLAN DATARMOR: Detailed explanation on hard ware What can you do today to be ready for DATARMOR DATARMOR : convention de nommage ClusterHPC REF SCRATCH

More information

HOKUSAI System. Figure 0-1 System diagram

HOKUSAI System. Figure 0-1 System diagram HOKUSAI System October 11, 2017 Information Systems Division, RIKEN 1.1 System Overview The HOKUSAI system consists of the following key components: - Massively Parallel Computer(GWMPC,BWMPC) - Application

More information

Transport Simulations beyond Petascale. Jing Fu (ANL)

Transport Simulations beyond Petascale. Jing Fu (ANL) Transport Simulations beyond Petascale Jing Fu (ANL) A) Project Overview The project: Peta- and exascale algorithms and software development (petascalable codes: Nek5000, NekCEM, NekLBM) Science goals:

More information

NCAR s Data-Centric Supercomputing Environment Yellowstone. November 28, 2011 David L. Hart, CISL

NCAR s Data-Centric Supercomputing Environment Yellowstone. November 28, 2011 David L. Hart, CISL NCAR s Data-Centric Supercomputing Environment Yellowstone November 28, 2011 David L. Hart, CISL dhart@ucar.edu Welcome to the Petascale Yellowstone hardware and software Deployment schedule Allocations

More information

GPUs and Emerging Architectures

GPUs and Emerging Architectures GPUs and Emerging Architectures Mike Giles mike.giles@maths.ox.ac.uk Mathematical Institute, Oxford University e-infrastructure South Consortium Oxford e-research Centre Emerging Architectures p. 1 CPUs

More information

Barcelona Supercomputing Center

Barcelona Supercomputing Center www.bsc.es Barcelona Supercomputing Center Centro Nacional de Supercomputación EMIT 2016. Barcelona June 2 nd, 2016 Barcelona Supercomputing Center Centro Nacional de Supercomputación BSC-CNS objectives:

More information

Scaling a Global File System to the Greatest Possible Extent, Performance, Capacity, and Number of Users

Scaling a Global File System to the Greatest Possible Extent, Performance, Capacity, and Number of Users Scaling a Global File System to the Greatest Possible Extent, Performance, Capacity, and Number of Users Phil Andrews, Bryan Banister, Patricia Kovatch, Chris Jordan San Diego Supercomputer Center University

More information

Architecting Storage for Semiconductor Design: Manufacturing Preparation

Architecting Storage for Semiconductor Design: Manufacturing Preparation White Paper Architecting Storage for Semiconductor Design: Manufacturing Preparation March 2012 WP-7157 EXECUTIVE SUMMARY The manufacturing preparation phase of semiconductor design especially mask data

More information

HPE Scalable Storage with Intel Enterprise Edition for Lustre*

HPE Scalable Storage with Intel Enterprise Edition for Lustre* HPE Scalable Storage with Intel Enterprise Edition for Lustre* HPE Scalable Storage with Intel Enterprise Edition For Lustre* High Performance Storage Solution Meets Demanding I/O requirements Performance

More information

ABySS Performance Benchmark and Profiling. May 2010

ABySS Performance Benchmark and Profiling. May 2010 ABySS Performance Benchmark and Profiling May 2010 Note The following research was performed under the HPC Advisory Council activities Participating vendors: AMD, Dell, Mellanox Compute resource - HPC

More information

Toward portable I/O performance by leveraging system abstractions of deep memory and interconnect hierarchies

Toward portable I/O performance by leveraging system abstractions of deep memory and interconnect hierarchies Toward portable I/O performance by leveraging system abstractions of deep memory and interconnect hierarchies François Tessier, Venkatram Vishwanath, Paul Gressier Argonne National Laboratory, USA Wednesday

More information

Experiences with ENZO on the Intel Many Integrated Core Architecture

Experiences with ENZO on the Intel Many Integrated Core Architecture Experiences with ENZO on the Intel Many Integrated Core Architecture Dr. Robert Harkness National Institute for Computational Sciences April 10th, 2012 Overview ENZO applications at petascale ENZO and

More information

Resources Current and Future Systems. Timothy H. Kaiser, Ph.D.

Resources Current and Future Systems. Timothy H. Kaiser, Ph.D. Resources Current and Future Systems Timothy H. Kaiser, Ph.D. tkaiser@mines.edu 1 Most likely talk to be out of date History of Top 500 Issues with building bigger machines Current and near future academic

More information

Eldorado. Outline. John Feo. Cray Inc. Why multithreaded architectures. The Cray Eldorado. Programming environment.

Eldorado. Outline. John Feo. Cray Inc. Why multithreaded architectures. The Cray Eldorado. Programming environment. Eldorado John Feo Cray Inc Outline Why multithreaded architectures The Cray Eldorado Programming environment Program examples 2 1 Overview Eldorado is a peak in the North Cascades. Internal Cray project

More information

Sami Saarinen Peter Towers. 11th ECMWF Workshop on the Use of HPC in Meteorology Slide 1

Sami Saarinen Peter Towers. 11th ECMWF Workshop on the Use of HPC in Meteorology Slide 1 Acknowledgements: Petra Kogel Sami Saarinen Peter Towers 11th ECMWF Workshop on the Use of HPC in Meteorology Slide 1 Motivation Opteron and P690+ clusters MPI communications IFS Forecast Model IFS 4D-Var

More information

The Titan Tools Experience

The Titan Tools Experience The Titan Tools Experience Michael J. Brim, Ph.D. Computer Science Research, CSMD/NCCS Petascale Tools Workshop 213 Madison, WI July 15, 213 Overview of Titan Cray XK7 18,688+ compute nodes 16-core AMD

More information

The Cielo Capability Supercomputer

The Cielo Capability Supercomputer The Cielo Capability Supercomputer Manuel Vigil (LANL), Douglas Doerfler (SNL), Sudip Dosanjh (SNL) and John Morrison (LANL) SAND 2011-3421C Unlimited Release Printed February, 2011 Sandia is a multiprogram

More information

Rechenzentrum HIGH PERFORMANCE SCIENTIFIC COMPUTING

Rechenzentrum HIGH PERFORMANCE SCIENTIFIC COMPUTING Rechenzentrum HIGH PERFORMANCE SCIENTIFIC COMPUTING Contents Scientifi c Supercomputing Center Karlsruhe (SSCK)... 4 Consultation and Support... 5 HP XC 6000 Cluster at the SSC Karlsruhe... 6 Architecture

More information

Oak Ridge National Laboratory

Oak Ridge National Laboratory Oak Ridge National Laboratory Lustre Scalability Workshop Presented by: Galen M. Shipman Collaborators: David Dillow Sarp Oral Feiyi Wang February 10, 2009 We have increased system performance 300 times

More information

IBM Power Systems HPC Cluster

IBM Power Systems HPC Cluster IBM Power Systems HPC Cluster Highlights Complete and fully Integrated HPC cluster for demanding workloads Modular and Extensible: match components & configurations to meet demands Integrated: racked &

More information

Piz Daint: Application driven co-design of a supercomputer based on Cray s adaptive system design

Piz Daint: Application driven co-design of a supercomputer based on Cray s adaptive system design Piz Daint: Application driven co-design of a supercomputer based on Cray s adaptive system design Sadaf Alam & Thomas Schulthess CSCS & ETHzürich CUG 2014 * Timelines & releases are not precise Top 500

More information