Automated Verifica/on of I/O Performance. F. Delalondre, M. Baerstchi. EPFL/Blue Brain Project - confiden6al
|
|
|
- Blaze Parrish
- 6 years ago
- Views:
Transcription
1 Automated Verifica/on of I/O Performance F. Delalondre, M. Baerstchi
2 Requirements Support Scien6sts Crea6vity Minimize Development 6me Maximize applica6on performance
3 Performance Analysis System Performance Applica6on Performance Applica6on on System Performance (Real 6me)
4 Performance Analysis System Performance Applica6on Performance Applica6on on System Performance (Real 6me)
5 Performance Analysis System Performance Applica6on Performance Applica6on on System Performance (Real 6me)
6 Performance Analysis System Performance Applica6on Performance Applica6on on System Performance (Real 6me)
7 Performance Analysis System Performance Applica6on Performance Applica6on on System Performance (Real 6me) Goal: Regression tes6ng & fast input to Developer/System Engineer
8 Performance Analysis System Performance Applica6on Performance Applica6on on System Performance (Real 6me) Goal: Regression tes6ng & fast input to Developer/System Engineer
9 Scien/fic Use Cases Interac6ve Supercompu6ng Tradi6onal Applica6on Use Case using GPFS File System
10 Interac/ve Supercompu/ng Machine u6liza6on does not maoer Time to scien6fic delivery maoers
11 Interac/ve Supercompu/ng Machine u6liza6on does not maoer Time to scien6fic delivery maoers
12 Interac/ve Supercompu/ng Machine u6liza6on does not maoer Time to scien6fic delivery maoers Monitoring
13 Interac/ve Supercompu/ng Machine u6liza6on does not maoer Time to scien6fic delivery maoers Steering Monitoring
14 Interac/ve Supercompu/ng Data Path 4096 Blue Gene/Q Compute Nodes 40 Nodes IdataPlex 64x2x2 GB/s 256 GB/s IB 64 Blue Gene/Q I/O Nodes 3 40x56 Gb/s 280 GB/s IB 64x40 Gb/s 320 GB/s Infiniband Switch
15 Regular Use Case Data Path 4096 Blue Gene/Q Compute Nodes 40 Nodes IdataPlex 64x2x2 GB/s 256 GB/s Infiniband Switch 8 40x56 Gb/s 280 GB/s 7 10x2x56Gb/s 135 GB/s GSS Servers 10x12x6Gb/s 72 GB/s 9 GSS Disk Drives 177 SAS Disk/Server 50Mb/s per disk => 88 GB/s
16 Regression Tes/ng & Performance Benchmark System I/O Regression Tes/ng Regression of System a]er Maintenance? Is the system delivering maximum performance?
17 Regression Tes/ng & Performance Benchmark System I/O Regression Tes/ng Regression of System a]er Maintenance? Is the system delivering maximum performance? Input to Developers & System Engineers System performance (bandwidth, latency, ) Scaling numbers: I/O fabric satura6on point Best I/O configura6on (block size, )
18 Tes/ng Framework For each path, test performance & scaling & I/O parameters All tests must be fully scripted (no manual interven6on) Tests include IOR, NsdPerf, Qperf, gpfsperf, Ib_read_*, ib_write_* Tests executed using Jenkins Con6nuous Integra6on Framework
19 IOR to I/O Node Memory 4096 Blue Gene/Q Compute Nodes 40 Nodes IdataPlex 64x2x2 GB/s 256 GB/s 1 IB 64 Blue Gene/Q I/O Nodes 40x56 Gb/s 280 GB/s IB 64x40 Gb/s 320 GB/s Infiniband Switch
20 IOR to I/O Node Memory IBM Blue Gene/Q I/O scaling cnk - IO node memory Memory Bandwith (MB/s) Posix w Posix r MPI IO w MPI IO r Number of Nodes
21 IOR to I/O Node Memory IBM Blue Gene/Q I/O scaling cnk - IO node memory GB/s Memory Bandwith (MB/s) Posix w Posix r MPI IO w MPI IO r Number of Nodes Write performance scaling loss >2 racks (~74% peak [1]), almost linear scaling every 5-10 runs (~94% peak) Read opera6on twice slower but linear scaling (~56% peak) To be tested at larger scale Why is it important? Interac6ve Supercompu6ng (ISC) [1] D. Chen, N.A. Eisley, P. Heidelberger, R.M. Senger, Y. Sugawara, S. Kumar, V. Salapura, D.L. SaOerfield, B. Steinmacher- Burow, J.J. Parker, The IBM Blue Gene/Q Interconnec6on Network and Message Unit, SC11 Proceedings, Networking, Storage and Analysis, 2011
22 IB Test - I/O Nodes to Viz Nodes 4096 Blue Gene/Q Compute Nodes 64x2x2 GB/s 256 GB/s 2 IB 40 Nodes IdataPlex 64 Blue Gene/Q I/O Nodes 3 40x56 Gb/s 280 GB/s IB 64x40 Gb/s 320 GB/s Infiniband Switch
23 IB Test - I/O Nodes to Viz Nodes Test Setup Pair (I/O nodes, Cluster Node) Increase Number of nodes up to 40 Observed Performance per Node & Outliers Detec6on of misconfigura6on/faulty card
24 IB Test - I/O Nodes to Viz Nodes 4096 Blue Gene/Q Compute Nodes 64x2x2 GB/s 256 GB/s 2 IB 40 Nodes IdataPlex 64 Blue Gene/Q I/O Nodes 3 40x56 Gb/s 280 GB/s IB 64x40 Gb/s 320 GB/s Infiniband Switch
25 IOR to Disk 64 Blue Gene/Q I/O Nodes 40 Nodes IdataPlex 64x40 Gb/s 320 GB/s 40x56 Gb/s 280 GB/s Infiniband Switch 10x2x56Gb/s 135 GB/s GSS Servers 10x12x6Gb/s 72 GB/s GSS Disk Drives 177 SAS Disk/Server 50Mb/s per disk => 88 GB/s
26 IOR to Disk Test Setup Read/write, MPI/Posix Various transfer sizes & access paoerns Observed Satura6on at 41 GB/s in op6mal configura6on Crashing research system when performing IOPS (4k) & large GPFS block size
27 I/O Nodes to GSS Servers 64 Blue Gene/Q I/O Nodes 40 Nodes IdataPlex 64x40 Gb/s 320 GB/s Infiniband Switch 8 40x56 Gb/s 280 GB/s 7 10x2x56Gb/s 135 GB/s 10 GSS Servers 10x12x6Gb/s 72 GB/s 9 GSS Disk Drives 177 SAS Disk/Server 50Mb/s per disk => 88 GB/s
28 I/O Nodes to GSS Servers 64 Blue Gene/Q I/O Nodes 64x40 Gb/s 320 GB/s 8: NSDperf Qperf/ib 40 Nodes IdataPlex 40x56 Gb/s 280 GB/s 7: Nsdperf /qperf/ib Infiniband Switch 10x2x56Gb/s 135 GB/s 10 GSS Servers What service can we run/install On GSS servers? 10x12x6Gb/s 72 GB/s GSS Disk Drives 177 SAS Disk/Server 50Mb/s per disk => 88 GB/s 9: What is the best test?
29 Performance Analysis System Performance Applica/on Performance Applica/on on System Performance (Real /me)
30 Can we go one step further? Reduce HPC development cycle by fast trouble shoo6ng Monitoring HPC/Simula6on plarorm real 6me & provide input to BBP Portal
31 Building an HPC Development Tool Building/simula6on Hardware monitoring Console Ok/not ok HW So]ware/Hardware mapping So]ware monitoring Whole Infrastructure Ok/not ok SW BG/Q Cluster EPFL Cluster Lugano EPFL/Blue Brain Project - confiden6al BG/Q
![Building an HPC Development Tool Building/simula6on Hardware monitoring Console Ok/not ok HW So]ware/Hardware mapping So]ware](/docs-images/71/66117449/images/32-6.jpg "monitoring Against DB Ok/not ok SW Whole Infrastructure BG/Q Cluster EPFL Cluster Lugano EPFL/Blue Brain Project - confiden6al")
32 Building an HPC Development Tool Building/simula6on Hardware monitoring Console Ok/not ok HW So]ware/Hardware mapping So]ware monitoring Against DB Ok/not ok SW Whole Infrastructure BG/Q Cluster EPFL Cluster Lugano EPFL/Blue Brain Project - confiden6al BG/Q
33 Building an HPC Development Tool Git Graphical Interface Responsible Patch set Console Performance DB & Graph So]ware monitoring Perf Numbers Ok/not ok SW EPFL/Blue Brain Project - confiden6al Graph
34 Building an HPC Development Tool Git Graphical Interface Responsible Patch set Console Profiling Performance DB & Graph So]ware monitoring Perf Numbers Ok/not ok SW EPFL/Blue Brain Project - confiden6al Graph
35 Building an HPC Development Tool Vtune intel cluster(x86) HPM - Extrae Scalasca (BG/Q) Profiling So]ware/Hardware mapping Recording BG/Q Whole Infrastructure So]ware monitoring Recording Ok/not ok SW BG/Q Cluster EPFL Cluster Lugano EPFL/Blue Brain Project - confiden6al
![Building an HPC Development Tool Vtune intel cluster(x86) HPM - Extrae Scalasca (BG/Q) Debugging So]ware/Hardware mapping Recording](/docs-images/71/66117449/images/36-8.jpg "BG/Q Whole Infrastructure So]ware monitoring Recording Ok/not ok SW BG/Q Cluster EPFL Cluster Lugano EPFL/Blue Brain Project -")
36 Building an HPC Development Tool Vtune intel cluster(x86) HPM - Extrae Scalasca (BG/Q) Debugging So]ware/Hardware mapping Recording BG/Q Whole Infrastructure So]ware monitoring Recording Ok/not ok SW BG/Q Cluster EPFL Cluster Lugano EPFL/Blue Brain Project - confiden6al
![Building an HPC Development Tool Debugger Interface Responsible Patch set Console Debugging So]ware](/docs-images/71/66117449/images/37-6.jpg "monitoring So]ware/Hardware mapping Whole Infrastructure BG/Q Ok/not ok SW BG/Q Cluster Lugano Cluster")
37 Building an HPC Development Tool Debugger Interface Responsible Patch set Console Debugging So]ware monitoring So]ware/Hardware mapping Whole Infrastructure BG/Q Ok/not ok SW BG/Q Cluster Lugano Cluster EPFL
38 Thank you
Automated Configuration and Administration of a Storage-class Memory System to Support Supercomputer-based Scientific Workflows
Automated Configuration and Administration of a Storage-class Memory System to Support Supercomputer-based Scientific Workflows J. Bernard 1, P. Morjan 2, B. Hagley 3, F. Delalondre 1, F. Schürmann 1,
I/O at JSC. I/O Infrastructure Workloads, Use Case I/O System Usage and Performance SIONlib: Task-Local I/O. Wolfgang Frings
Mitglied der Helmholtz-Gemeinschaft I/O at JSC I/O Infrastructure Workloads, Use Case I/O System Usage and Performance SIONlib: Task-Local I/O Wolfgang Frings W.Frings@fz-juelich.de Jülich Supercomputing
Economic Viability of Hardware Overprovisioning in Power- Constrained High Performance Compu>ng
Economic Viability of Hardware Overprovisioning in Power- Constrained High Performance Compu>ng Energy Efficient Supercompu1ng, SC 16 November 14, 2016 This work was performed under the auspices of the U.S.
Operational Robustness of Accelerator Aware MPI
Operational Robustness of Accelerator Aware MPI Sadaf Alam Swiss National Supercomputing Centre (CSSC) Switzerland 2nd Annual MVAPICH User Group (MUG) Meeting, 2014 Computing Systems @ CSCS http://www.cscs.ch/computers
Analyzing the High Performance Parallel I/O on LRZ HPC systems. Sandra Méndez. HPC Group, LRZ. June 23, 2016
Analyzing the High Performance Parallel I/O on LRZ HPC systems Sandra Méndez. HPC Group, LRZ. June 23, 2016 Outline SuperMUC supercomputer User Projects Monitoring Tool I/O Software Stack I/O Analysis
Analytics in the cloud
Analytics in the cloud Dow we really need to reinvent the storage stack? R. Ananthanarayanan, Karan Gupta, Prashant Pandey, Himabindu Pucha, Prasenjit Sarkar, Mansi Shah, Renu Tewari Image courtesy NASA
An Overview of Fujitsu s Lustre Based File System
An Overview of Fujitsu s Lustre Based File System Shinji Sumimoto Fujitsu Limited Apr.12 2011 For Maximizing CPU Utilization by Minimizing File IO Overhead Outline Target System Overview Goals of Fujitsu
Blue Gene/Q. Hardware Overview Michael Stephan. Mitglied der Helmholtz-Gemeinschaft
Blue Gene/Q Hardware Overview 02.02.2015 Michael Stephan Blue Gene/Q: Design goals System-on-Chip (SoC) design Processor comprises both processing cores and network Optimal performance / watt ratio Small
Parallel I/O on JUQUEEN
Parallel I/O on JUQUEEN 4. Februar 2014, JUQUEEN Porting and Tuning Workshop Mitglied der Helmholtz-Gemeinschaft Wolfgang Frings w.frings@fz-juelich.de Jülich Supercomputing Centre Overview Parallel I/O
From Rack Scale to Network Scale: NVMe Over Fabrics Enables Exabyte Applica>ons. Zivan Ori, CEO & Co-founder, E8 Storage
From Rack Scale to Network Scale: NVMe Over Fabrics Enables Exabyte Applica>ons Zivan Ori, CEO & Co-founder, E8 Storage NVMe Over Fabrics Who? Why? How? Who is using NVMe over fabrics? Why do they need
IBM Blue Gene/Q solution
IBM Blue Gene/Q solution Pascal Vezolle vezolle@fr.ibm.com Broad IBM Technical Computing portfolio Hardware Blue Gene/Q Power Systems 86 Systems idataplex and Intelligent Cluster GPGPU / Intel MIC PureFlexSystems
I/O Monitoring at JSC, SIONlib & Resiliency
Mitglied der Helmholtz-Gemeinschaft I/O Monitoring at JSC, SIONlib & Resiliency Update: I/O Infrastructure @ JSC Update: Monitoring with LLview (I/O, Memory, Load) I/O Workloads on Jureca SIONlib: Task-Local
Store Process Analyze Collaborate Archive Cloud The HPC Storage Leader Invent Discover Compete
Store Process Analyze Collaborate Archive Cloud The HPC Storage Leader Invent Discover Compete 1 DDN Who We Are 2 We Design, Deploy and Optimize Storage Systems Which Solve HPC, Big Data and Cloud Business
OPENFABRICS INTERFACES: PAST, PRESENT, AND FUTURE
12th ANNUAL WORKSHOP 2016 OPENFABRICS INTERFACES: PAST, PRESENT, AND FUTURE Sean Hefty OFIWG Co-Chair [ April 5th, 2016 ] OFIWG: develop interfaces aligned with application needs Open Source Expand open
CONTAINERIZING JOBS ON THE ACCRE CLUSTER WITH SINGULARITY
CONTAINERIZING JOBS ON THE ACCRE CLUSTER WITH SINGULARITY VIRTUAL MACHINE (VM) Uses so&ware to emulate an en/re computer, including both hardware and so&ware. Host Computer Virtual Machine Host Resources:
Outline. March 5, 2012 CIRMMT - McGill University 2
Outline CLUMEQ, Calcul Quebec and Compute Canada Research Support Objectives and Focal Points CLUMEQ Site at McGill ETS Key Specifications and Status CLUMEQ HPC Support Staff at McGill Getting Started
RAIDIX Data Storage Solution. Clustered Data Storage Based on the RAIDIX Software and GPFS File System
RAIDIX Data Storage Solution Clustered Data Storage Based on the RAIDIX Software and GPFS File System 2017 Contents Synopsis... 2 Introduction... 3 Challenges and the Solution... 4 Solution Architecture...
Outline. In Situ Data Triage and Visualiza8on
In Situ Data Triage and Visualiza8on Kwan- Liu Ma University of California at Davis Outline In situ data triage and visualiza8on: Issues and strategies Case study: An earthquake simula8on Case study: A
I/O and Scheduling aspects in DEEP-EST
I/O and Scheduling aspects in DEEP-EST Norbert Eicker Jülich Supercomputing Centre & University of Wuppertal The research leading to these results has received funding from the European Community's Seventh
NCAR Globally Accessible Data Environment (GLADE) Updated: 15 Feb 2017
NCAR Globally Accessible Data Environment (GLADE) Updated: 15 Feb 2017 Overview The Globally Accessible Data Environment (GLADE) provides centralized file storage for HPC computational, data-analysis,
TECHNICAL GUIDELINES FOR APPLICANTS TO PRACE 11th CALL (T ier-0)
TECHNICAL GUIDELINES FOR APPLICANTS TO PRACE 11th CALL (T ier-0) Contributing sites and the corresponding computer systems for this call are: BSC, Spain IBM System X idataplex CINECA, Italy The site selection
Stockholm Brain Institute Blue Gene/L
Stockholm Brain Institute Blue Gene/L 1 Stockholm Brain Institute Blue Gene/L 2 IBM Systems & Technology Group and IBM Research IBM Blue Gene /P - An Overview of a Petaflop Capable System Carl G. Tengwall
Sami Saarinen Peter Towers. 11th ECMWF Workshop on the Use of HPC in Meteorology Slide 1
Acknowledgements: Petra Kogel Sami Saarinen Peter Towers 11th ECMWF Workshop on the Use of HPC in Meteorology Slide 1 Motivation Opteron and P690+ clusters MPI communications IFS Forecast Model IFS 4D-Var
PART-I (B) (TECHNICAL SPECIFICATIONS & COMPLIANCE SHEET) Supply and installation of High Performance Computing System
INSTITUTE FOR PLASMA RESEARCH (An Autonomous Institute of Department of Atomic Energy, Government of India) Near Indira Bridge; Bhat; Gandhinagar-382428; India PART-I (B) (TECHNICAL SPECIFICATIONS & COMPLIANCE
The IBM Blue Gene/Q: Application performance, scalability and optimisation
The IBM Blue Gene/Q: Application performance, scalability and optimisation Mike Ashworth, Andrew Porter Scientific Computing Department & STFC Hartree Centre Manish Modani IBM STFC Daresbury Laboratory,
Feedback on BeeGFS. A Parallel File System for High Performance Computing
Feedback on BeeGFS A Parallel File System for High Performance Computing Philippe Dos Santos et Georges Raseev FR 2764 Fédération de Recherche LUmière MATière December 13 2016 LOGO CNRS LOGO IO December
Blue Waters I/O Performance
Blue Waters I/O Performance Mark Swan Performance Group Cray Inc. Saint Paul, Minnesota, USA mswan@cray.com Doug Petesch Performance Group Cray Inc. Saint Paul, Minnesota, USA dpetesch@cray.com Abstract
IME (Infinite Memory Engine) Extreme Application Acceleration & Highly Efficient I/O Provisioning
IME (Infinite Memory Engine) Extreme Application Acceleration & Highly Efficient I/O Provisioning September 22 nd 2015 Tommaso Cecchi 2 What is IME? This breakthrough, software defined storage application
CLOUD SERVICES. Cloud Value Assessment.
CLOUD SERVICES Cloud Value Assessment www.cloudcomrade.com Comrade a companion who shares one's ac8vi8es or is a fellow member of an organiza8on 2 Today s Agenda! Why Companies Should Consider Moving Business
DELL EMC ISILON F800 AND H600 I/O PERFORMANCE
DELL EMC ISILON F800 AND H600 I/O PERFORMANCE ABSTRACT This white paper provides F800 and H600 performance data. It is intended for performance-minded administrators of large compute clusters that access
Interconnect Your Future
Interconnect Your Future Smart Interconnect for Next Generation HPC Platforms Gilad Shainer, August 2016, 4th Annual MVAPICH User Group (MUG) Meeting Mellanox Connects the World s Fastest Supercomputer
HPC Storage Use Cases & Future Trends
Oct, 2014 HPC Storage Use Cases & Future Trends Massively-Scalable Platforms and Solutions Engineered for the Big Data and Cloud Era Atul Vidwansa Email: atul@ DDN About Us DDN is a Leader in Massively
Habanero Operating Committee. January
Habanero Operating Committee January 25 2017 Habanero Overview 1. Execute Nodes 2. Head Nodes 3. Storage 4. Network Execute Nodes Type Quantity Standard 176 High Memory 32 GPU* 14 Total 222 Execute Nodes
How to sleep *ght and keep your applica*ons running on IPv6 transi*on. The importance of IPv6 Applica*on Tes*ng
How to sleep *ght and keep your applica*ons running on IPv6 transi*on The importance of IPv6 Applica*on Tes*ng About this presenta*on It presents a generic methodology to test the IPv6 func*onality of
TECHNICAL GUIDELINES FOR APPLICANTS TO PRACE 6 th CALL (Tier-0)
TECHNICAL GUIDELINES FOR APPLICANTS TO PRACE 6 th CALL (Tier-0) Contributing sites and the corresponding computer systems for this call are: GCS@Jülich, Germany IBM Blue Gene/Q GENCI@CEA, France Bull Bullx
Implica(ons of Non Vola(le Memory on So5ware Architectures. Nisha Talagala Lead Architect, Fusion- io
Implica(ons of Non Vola(le Memory on So5ware Architectures Nisha Talagala Lead Architect, Fusion- io Overview Non Vola;le Memory Technology NVM in the Datacenter Op;mizing sobware for the iomemory Tier
Visualiza(on So-ware and Hardware for In- Silico Brain Research. Stefan Eilemann Visualiza0on Team Lead Blue Brain Project, EPFL
Visualiza(on So-ware and Hardware for In- Silico Brain Research Stefan Eilemann Visualiza0on Team Lead Blue Brain Project, EPFL Blue Brain Project / Human Brain Project BBP: Swiss na0onal research project
A Generic Methodology of Analyzing Performance Bottlenecks of HPC Storage Systems. Zhiqi Tao, Sr. System Engineer Lugano, March
A Generic Methodology of Analyzing Performance Bottlenecks of HPC Storage Systems Zhiqi Tao, Sr. System Engineer Lugano, March 15 2013 1 Outline Introduction o Anatomy of a storage system o Performance
SUN CUSTOMER READY HPC CLUSTER: REFERENCE CONFIGURATIONS WITH SUN FIRE X4100, X4200, AND X4600 SERVERS Jeff Lu, Systems Group Sun BluePrints OnLine
SUN CUSTOMER READY HPC CLUSTER: REFERENCE CONFIGURATIONS WITH SUN FIRE X4100, X4200, AND X4600 SERVERS Jeff Lu, Systems Group Sun BluePrints OnLine April 2007 Part No 820-1270-11 Revision 1.1, 4/18/07
JÜLICH SUPERCOMPUTING CENTRE Site Introduction Michael Stephan Forschungszentrum Jülich
JÜLICH SUPERCOMPUTING CENTRE Site Introduction 09.04.2018 Michael Stephan JSC @ Forschungszentrum Jülich FORSCHUNGSZENTRUM JÜLICH Research Centre Jülich One of the 15 Helmholtz Research Centers in Germany
UCX: An Open Source Framework for HPC Network APIs and Beyond
UCX: An Open Source Framework for HPC Network APIs and Beyond Presented by: Pavel Shamis / Pasha ORNL is managed by UT-Battelle for the US Department of Energy Co-Design Collaboration The Next Generation
Can Parallel Replication Benefit Hadoop Distributed File System for High Performance Interconnects?
Can Parallel Replication Benefit Hadoop Distributed File System for High Performance Interconnects? N. S. Islam, X. Lu, M. W. Rahman, and D. K. Panda Network- Based Compu2ng Laboratory Department of Computer
MPI-IO Performance Optimization IOR Benchmark on IBM ESS GL4 Systems
MPI-IO Performance Optimization IOR Benchmark on IBM ESS GL4 Systems Xinghong He HPC Application Support IBM Systems WW Client Centers May 24 2016 Agenda System configurations Storage system, compute cluster
AWS: Basic Architecture Session SUNEY SHARMA Solutions Architect: AWS
AWS: Basic Architecture Session SUNEY SHARMA Solutions Architect: AWS suneys@amazon.com AWS Core Infrastructure and Services Traditional Infrastructure Amazon Web Services Security Security Firewalls ACLs
A Case for High Performance Computing with Virtual Machines
A Case for High Performance Computing with Virtual Machines Wei Huang*, Jiuxing Liu +, Bulent Abali +, and Dhabaleswar K. Panda* *The Ohio State University +IBM T. J. Waston Research Center Presentation
Lustre2.5 Performance Evaluation: Performance Improvements with Large I/O Patches, Metadata Improvements, and Metadata Scaling with DNE
Lustre2.5 Performance Evaluation: Performance Improvements with Large I/O Patches, Metadata Improvements, and Metadata Scaling with DNE Hitoshi Sato *1, Shuichi Ihara *2, Satoshi Matsuoka *1 *1 Tokyo Institute
Oncilla - a Managed GAS Runtime for Accelerating Data Warehousing Queries
Oncilla - a Managed GAS Runtime for Accelerating Data Warehousing Queries Jeffrey Young, Alex Merritt, Se Hoon Shon Advisor: Sudhakar Yalamanchili 4/16/13 Sponsors: Intel, NVIDIA, NSF 2 The Problem Big
Beyond Petascale. Roger Haskin Manager, Parallel File Systems IBM Almaden Research Center
Beyond Petascale Roger Haskin Manager, Parallel File Systems IBM Almaden Research Center GPFS Research and Development! GPFS product originated at IBM Almaden Research Laboratory! Research continues to
PREDICTING COMMUNICATION PERFORMANCE
PREDICTING COMMUNICATION PERFORMANCE Nikhil Jain CASC Seminar, LLNL This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under Contract
ZEST Snapshot Service. A Highly Parallel Production File System by the PSC Advanced Systems Group Pittsburgh Supercomputing Center 1
ZEST Snapshot Service A Highly Parallel Production File System by the PSC Advanced Systems Group Pittsburgh Supercomputing Center 1 Design Motivation To optimize science utilization of the machine Maximize
Latest Trends in Database Technology NoSQL and Beyond
Latest Trends in Database Technology NoSQL and Beyond Sebas>an Marsching www.aquenos.com Why we want more than SQL Performance / Data Size Opera>onal Costs Availability 2 NoSQL NoSQL Not Only SQL 3 NoSQL
DELIVERABLE D5.5 Report on ICARUS visualization cluster installation. John BIDDISCOMBE (CSCS) Jerome SOUMAGNE (CSCS)
DELIVERABLE D5.5 Report on ICARUS visualization cluster installation John BIDDISCOMBE (CSCS) Jerome SOUMAGNE (CSCS) 02 May 2011 NextMuSE 2 Next generation Multi-mechanics Simulation Environment Cluster
Scalasca support for Intel Xeon Phi. Brian Wylie & Wolfgang Frings Jülich Supercomputing Centre Forschungszentrum Jülich, Germany
Scalasca support for Intel Xeon Phi Brian Wylie & Wolfgang Frings Jülich Supercomputing Centre Forschungszentrum Jülich, Germany Overview Scalasca performance analysis toolset support for MPI & OpenMP
IBM Information Technology Guide For ANSYS Fluent Customers
IBM ISV & Developer Relations Manufacturing IBM Information Technology Guide For ANSYS Fluent Customers A collaborative effort between ANSYS and IBM 2 IBM Information Technology Guide For ANSYS Fluent
Blue Gene/Q A system overview
Mitglied der Helmholtz-Gemeinschaft Blue Gene/Q A system overview M. Stephan Outline Blue Gene/Q hardware design Processor Network I/O node Jülich Blue Gene/Q configurations (JUQUEEN) Blue Gene/Q software
libhio: Optimizing IO on Cray XC Systems With DataWarp
libhio: Optimizing IO on Cray XC Systems With DataWarp May 9, 2017 Nathan Hjelm Cray Users Group May 9, 2017 Los Alamos National Laboratory LA-UR-17-23841 5/8/2017 1 Outline Background HIO Design Functionality
Improved Solutions for I/O Provisioning and Application Acceleration
1 Improved Solutions for I/O Provisioning and Application Acceleration August 11, 2015 Jeff Sisilli Sr. Director Product Marketing jsisilli@ddn.com 2 Why Burst Buffer? The Supercomputing Tug-of-War A supercomputer
Next-Generation NVMe-Native Parallel Filesystem for Accelerating HPC Workloads
Next-Generation NVMe-Native Parallel Filesystem for Accelerating HPC Workloads Liran Zvibel CEO, Co-founder WekaIO @liranzvibel 1 WekaIO Matrix: Full-featured and Flexible Public or Private S3 Compatible
Efficient Object Storage Journaling in a Distributed Parallel File System
Efficient Object Storage Journaling in a Distributed Parallel File System Presented by Sarp Oral Sarp Oral, Feiyi Wang, David Dillow, Galen Shipman, Ross Miller, and Oleg Drokin FAST 10, Feb 25, 2010 A
Con$nuous Integra$on Development Environment. Kovács Gábor
Con$nuous Integra$on Development Environment Kovács Gábor kovacsg@tmit.bme.hu Before we start anything Select a language Set up conven$ons Select development tools Set up development environment Set up
OPEN MPI WITH RDMA SUPPORT AND CUDA. Rolf vandevaart, NVIDIA
OPEN MPI WITH RDMA SUPPORT AND CUDA Rolf vandevaart, NVIDIA OVERVIEW What is CUDA-aware History of CUDA-aware support in Open MPI GPU Direct RDMA support Tuning parameters Application example Future work
Enterprise2014. GPFS with Flash840 on PureFlex and Power8 (AIX & Linux)
Chris Churchey Principal ATS Group, LLC churchey@theatsgroup.com (610-574-0207) October 2014 GPFS with Flash840 on PureFlex and Power8 (AIX & Linux) Why Monitor? (Clusters, Servers, Storage, Net, etc.)
Data Management. Parallel Filesystems. Dr David Henty HPC Training and Support
Data Management Dr David Henty HPC Training and Support d.henty@epcc.ed.ac.uk +44 131 650 5960 Overview Lecture will cover Why is IO difficult Why is parallel IO even worse Lustre GPFS Performance on ARCHER
CSCS HPC storage. Hussein N. Harake
CSCS HPC storage Hussein N. Harake Points to Cover - XE6 External Storage (DDN SFA10K, SRP, QDR) - PCI-E SSD Technology - RamSan 620 Technology XE6 External Storage - Installed Q4 2010 - In Production
5.4 - DAOS Demonstration and Benchmark Report
5.4 - DAOS Demonstration and Benchmark Report Johann LOMBARDI on behalf of the DAOS team September 25 th, 2013 Livermore (CA) NOTICE: THIS MANUSCRIPT HAS BEEN AUTHORED BY INTEL UNDER ITS SUBCONTRACT WITH
Application Acceleration Beyond Flash Storage
Application Acceleration Beyond Flash Storage Session 303C Mellanox Technologies Flash Memory Summit July 2014 Accelerating Applications, Step-by-Step First Steps Make compute fast Moore s Law Make storage
The RAMDISK Storage Accelerator
The RAMDISK Storage Accelerator A Method of Accelerating I/O Performance on HPC Systems Using RAMDISKs Tim Wickberg, Christopher D. Carothers wickbt@rpi.edu, chrisc@cs.rpi.edu Rensselaer Polytechnic Institute
Implementing MPI on Windows: Comparison with Common Approaches on Unix
Implementing MPI on Windows: Comparison with Common Approaches on Unix Jayesh Krishna, 1 Pavan Balaji, 1 Ewing Lusk, 1 Rajeev Thakur, 1 Fabian Tillier 2 1 Argonne Na+onal Laboratory, Argonne, IL, USA 2
Real Parallel Computers
Real Parallel Computers Modular data centers Overview Short history of parallel machines Cluster computing Blue Gene supercomputer Performance development, top-500 DAS: Distributed supercomputing Short
Deep Learning Performance and Cost Evaluation
Micron 5210 ION Quad-Level Cell (QLC) SSDs vs 7200 RPM HDDs in Centralized NAS Storage Repositories A Technical White Paper Don Wang, Rene Meyer, Ph.D. info@ AMAX Corporation Publish date: October 25,
Scaling to Petaflop. Ola Torudbakken Distinguished Engineer. Sun Microsystems, Inc
Scaling to Petaflop Ola Torudbakken Distinguished Engineer Sun Microsystems, Inc HPC Market growth is strong CAGR increased from 9.2% (2006) to 15.5% (2007) Market in 2007 doubled from 2003 (Source: IDC
Introduc)on to the RCE September 21, 2010 Len Wisniewski
Introduc)on to the RCE September 21, 2010 Len Wisniewski IQSS technical services Resource support Research Compu)ng Environment (RCE): cluster compu)ng for sta)s)cal research Desktop support Computer lab
Parallel File Systems. John White Lawrence Berkeley National Lab
Parallel File Systems John White Lawrence Berkeley National Lab Topics Defining a File System Our Specific Case for File Systems Parallel File Systems A Survey of Current Parallel File Systems Implementation
Improving Data Movement Performance for Sparse Data Patterns on the Blue Gene/Q Supercomputer
Improving Data Movement Performance for Sparse Data Patterns on the Blue Gene/Q Supercomputer Huy Bui, Jason Leigh Electronic Visualization Laboratory (EVL) University of Illinois at Chicago Chicago, IL,
Readme for Platform Open Cluster Stack (OCS)
Readme for Platform Open Cluster Stack (OCS) Version 4.1.1-2.0 October 25 2006 Platform Computing Contents What is Platform OCS? What's New in Platform OCS 4.1.1-2.0? Supported Architecture Distribution
Data storage services at KEK/CRC -- status and plan
Data storage services at KEK/CRC -- status and plan KEK/CRC Hiroyuki Matsunaga Most of the slides are prepared by Koichi Murakami and Go Iwai KEKCC System Overview KEKCC (Central Computing System) The
PrepAwayExam. High-efficient Exam Materials are the best high pass-rate Exam Dumps
PrepAwayExam http://www.prepawayexam.com/ High-efficient Exam Materials are the best high pass-rate Exam Dumps Exam : C4060-155 Title : System x Server Family Sales V1 Vendors : IBM Version : DEMO Get
High Performance MPI on IBM 12x InfiniBand Architecture
High Performance MPI on IBM 12x InfiniBand Architecture Abhinav Vishnu, Brad Benton 1 and Dhabaleswar K. Panda {vishnu, panda} @ cse.ohio-state.edu {brad.benton}@us.ibm.com 1 1 Presentation Road-Map Introduction
Redefining x86 A New Era of Solutions
Redefining x86 A New Era of Solutions Piotr Nowik IBM Systems and Technology 1 1 What is it? MY NOTEBOOK 2 2 Smarter Computing The IT infrastructure that enables a Smarter Planet 3 3 IBM PureFlex and System
Future Routing Schemes in Petascale clusters
Future Routing Schemes in Petascale clusters Gilad Shainer, Mellanox, USA Ola Torudbakken, Sun Microsystems, Norway Richard Graham, Oak Ridge National Laboratory, USA Birds of a Feather Presentation Abstract
Hyper-converged storage for Oracle RAC based on NVMe SSDs and standard x86 servers
Hyper-converged storage for Oracle RAC based on NVMe SSDs and standard x86 servers White Paper rev. 2016-05-18 2015-2016 FlashGrid Inc. 1 www.flashgrid.io Abstract Oracle Real Application Clusters (RAC)
Oracle Exadata X7. Uwe Kirchhoff Oracle ACS - Delivery Senior Principal Service Delivery Engineer
Oracle Exadata X7 Uwe Kirchhoff Oracle ACS - Delivery Senior Principal Service Delivery Engineer 05.12.2017 Oracle Engineered Systems ZFS Backup Appliance Zero Data Loss Recovery Appliance Exadata Database
Single-Points of Performance
Single-Points of Performance Mellanox Technologies Inc. 29 Stender Way, Santa Clara, CA 9554 Tel: 48-97-34 Fax: 48-97-343 http://www.mellanox.com High-performance computations are rapidly becoming a critical
Multifunction Networking Adapters
Ethernet s Extreme Makeover: Multifunction Networking Adapters Chuck Hudson Manager, ProLiant Networking Technology Hewlett-Packard 2004 Hewlett-Packard Development Company, L.P. The information contained
Low-Overhead Flash Disaggregation via NVMe-over-Fabrics Vijay Balakrishnan Memory Solutions Lab. Samsung Semiconductor, Inc.
Low-Overhead Flash Disaggregation via NVMe-over-Fabrics Vijay Balakrishnan Memory Solutions Lab. Samsung Semiconductor, Inc. 1 DISCLAIMER This presentation and/or accompanying oral statements by Samsung
Application Performance on IME
Application Performance on IME Toine Beckers, DDN Marco Grossi, ICHEC Burst Buffer Designs Introduce fast buffer layer Layer between memory and persistent storage Pre-stage application data Buffer writes
NAMD Performance Benchmark and Profiling. November 2010
NAMD Performance Benchmark and Profiling November 2010 Note The following research was performed under the HPC Advisory Council activities Participating vendors: HP, Mellanox Compute resource - HPC Advisory
Con$nuous Deployment with Docker Andrew Aslinger. Oct
Con$nuous Deployment with Docker Andrew Aslinger Oct 9. 2014 Who is Andrew #1 So#ware / Systems Architect for OpenWhere Passion for UX, Big Data, and Cloud/DevOps Previously Designed and Implemented automated
TECHNICAL GUIDELINES FOR APPLICANTS TO PRACE 13 th CALL (T ier-0)
TECHNICAL GUIDELINES FOR APPLICANTS TO PRACE 13 th CALL (T ier-0) Contributing sites and the corresponding computer systems for this call are: BSC, Spain IBM System x idataplex CINECA, Italy Lenovo System
Making Supercomputing More Available and Accessible Windows HPC Server 2008 R2 Beta 2 Microsoft High Performance Computing April, 2010
Making Supercomputing More Available and Accessible Windows HPC Server 2008 R2 Beta 2 Microsoft High Performance Computing April, 2010 Windows HPC Server 2008 R2 Windows HPC Server 2008 R2 makes supercomputing
believe in more SDN for Datacenter A Simple Approach
believe in more SDN for Datacenter A Simple Approach 1 Agenda ACI Overview Fabric Policy Constructs Hypervisor Support A migra>on scenario One management umbrella: UCS Director Q&A 2 Applica,on Language
Enabling web-based interactive notebooks on geographically distributed HPC resources. Alexandre Beche
Enabling web-based interactive notebooks on geographically distributed HPC resources Alexandre Beche Outlines 1. Context 2. Interactive notebook running on cluster(s) 3. Advanced
HPC Performance in the Cloud: Status and Future Prospects
HPC Performance in the Cloud: Status and Future Prospects ISC Cloud 2012 Josh Simons, Office of the CTO, VMware 2009 VMware Inc. All rights reserved Cloud Cloud computing is a model for enabling ubiquitous,
George Markomanolis IO500 Committee: John Bent, Julian M. Kunkel, Jay Lofstead 2017-11-12 http://www.io500.org IBM Spectrum Scale User Group, Denver, Colorado, USA Why? The increase of the studied domains,
Benchmarking computers for seismic processing and imaging
Benchmarking computers for seismic processing and imaging Evgeny Kurin ekurin@geo-lab.ru Outline O&G HPC status and trends Benchmarking: goals and tools GeoBenchmark: modules vs. subsystems Basic tests
Interconnect Your Future
Interconnect Your Future Gilad Shainer 2nd Annual MVAPICH User Group (MUG) Meeting, August 2014 Complete High-Performance Scalable Interconnect Infrastructure Comprehensive End-to-End Software Accelerators
Birds of a Feather Presentation
Mellanox InfiniBand QDR 4Gb/s The Fabric of Choice for High Performance Computing Gilad Shainer, shainer@mellanox.com June 28 Birds of a Feather Presentation InfiniBand Technology Leadership Industry Standard
Design and Evaluation of a 2048 Core Cluster System
Design and Evaluation of a 2048 Core Cluster System, Torsten Höfler, Torsten Mehlan and Wolfgang Rehm Computer Architecture Group Department of Computer Science Chemnitz University of Technology December
Progress Report on Transparent Checkpointing for Supercomputing
Progress Report on Transparent Checkpointing for Supercomputing Jiajun Cao, Rohan Garg College of Computer and Information Science, Northeastern University {jiajun,rohgarg}@ccs.neu.edu August 21, 2015
Tuning I/O Performance for Data Intensive Computing. Nicholas J. Wright. lbl.gov
Tuning I/O Performance for Data Intensive Computing. Nicholas J. Wright njwright @ lbl.gov NERSC- National Energy Research Scientific Computing Center Mission: Accelerate the pace of scientific discovery