A Long-distance InfiniBand Interconnection between two Clusters in Production Use
|
|
|
- Naomi Morton
- 5 years ago
- Views:
Transcription
1 A Long-distance InfiniBand Interconnection between two Clusters in Production Use Sabine Richling, Steffen Hau, Heinz Kredel, Hans-Günther Kruse IT-Center, University of Heidelberg, Germany IT-Center, University of Mannheim, Germany SC 11, State of the Practice, 16. November 2011 Richling, Hau, Kredel, Kruse (URZ/RUM) Long-distance InfiniBand Connection Seattle, November / 25
2 Outline 1 Background D-Grid and bwgrid bwgrid MA/HD 2 Interconnection of two bwgrid clusters 3 Cluster Operation Node Management User Management Job Management 4 Performance MPI Performance Storage Access Performance 5 Summary and Conclusions Richling, Hau, Kredel, Kruse (URZ/RUM) Long-distance InfiniBand Connection Seattle, November / 25
3 D-Grid and bwgrid bwgrid Virtual Organization (VO) Community project of the German Grid Initiative D-Grid Project partners are the Universities in Baden-Württemberg bwgrid Resources Compute clusters at 8 locations Central storage unit in Karlsruhe bwgrid Objectives Verifying the functionality and the benefit of Grid concepts for the HPC community in Baden-Württemberg Managing organizational, security, and license issues Development of new cluster and Grid applications Richling, Hau, Kredel, Kruse (URZ/RUM) Long-distance InfiniBand Connection Seattle, November / 25
4 bwgrid Resources Compute Cluster Site Nodes Mannheim 140 Heidelberg 140 Karlsruhe 140 Stuttgart 420 Tübingen 140 Ulm/Konstanz 280 Freiburg 140 Esslingen 180 Total 1580 Mannheim Frankfurt (interconnected to a single cluster) Karlsruhe Heidelberg Stuttgart Esslingen Central Storage with backup 128 TB without backup 256 TB Total 384 TB Freiburg Tübingen Ulm (joint cluster with Konstanz) München Richling, Hau, Kredel, Kruse (URZ/RUM) Long-distance InfiniBand Connection Seattle, November / 25
5 bwgrid Situation in MA/HD before interconnection Diversity of applications (1 128 nodes per job) Many first time HPC users! Access with local University Accounts (Authentication via LDAP/AD) User MA User HD LDAP AD Cluster Mannheim InfiniBand InfiniBand Cluster Heidelberg Richling, Hau, Kredel, Kruse (URZ/RUM) Long-distance InfiniBand Connection Seattle, November / 25
6 bwgrid Situation in MA/HD before interconnection Grid certificate allows access to all bwgrid clusters Feasible only for more experienced users other bwgrid resources VORM Grid User MA Grid Certificate VO Registration Grid User HD VORM LDAP AD Cluster Mannheim InfiniBand InfiniBand Cluster Heidelberg Richling, Hau, Kredel, Kruse (URZ/RUM) Long-distance InfiniBand Connection Seattle, November / 25
7 Interconnection of bwgrid clusters MA/HD Proposal in 2008 Acquisition and Assembly until May 2009 Running since July 2009 InfiniBand over Ethernet over fibre optics: Obsidian Longbow adaptor InfiniBand connector (black cable), fibre optic connector (yellow cable) Richling, Hau, Kredel, Kruse (URZ/RUM) Long-distance InfiniBand Connection Seattle, November / 25
8 MPI Performance Prospects Measurements for different distances (HLRS, Stuttgart, Germany) Bandwidth MB/sec for up to km Richling, Hau, Kredel, Kruse (URZ/RUM) Long-distance InfiniBand Connection Seattle, November / 25
9 MPI Performance Interconnection MA/HD Cluster Mannheim Cluster Heidelberg InfiniBand Obsidian 28 km Obsidian InfiniBand Latency is high 145 µsec = 143 µsec light transit time + 2 µsec local latency Bandwidth is as expected about 930 MB/sec (local bandwidth MB/sec) Obsidian needs a license for 40 km Obsidian has buffers for larger distances Activation of buffers with license License for 10 km is not sufficient Richling, Hau, Kredel, Kruse (URZ/RUM) Long-distance InfiniBand Connection Seattle, November / 25
10 MPI Bandwidth Influence of the Obsidian License IMB PingPong - buffer size 1 GB bandwidth [Mbytes/sec] Sep 00:00 23 Sep 00:00 30 Sep 00:00 start time [date hour] 07 Oct 00:00 Richling, Hau, Kredel, Kruse (URZ/RUM) Long-distance InfiniBand Connection Seattle, November / 25
11 bwgrid Cluster Mannheim/Heidelberg Overview VORM LDAP Cluster Mannheim Grid User MA InfiniBand Lustre bwfs MA PBS Admin passwd Obsidian 28 km Obsidian Grid User HD InfiniBand Lustre bwfs HD VORM AD Cluster Heidelberg 140 nodes cluster InfiniBand Network Login/Admin Server Directory service Storage Richling, Hau, Kredel, Kruse (URZ/RUM) Long-distance InfiniBand Connection Seattle, November / 25
12 Node Management Administration server provides DHCP service for the nodes (MAC-to-IP address configuration file) NFS export for root file system NFS directory for software packages accessible via module utilities queuing and scheduling system Node administration adjusted shell scripts originally developed by HLRS IBM management module (command line interface and Web-GUI) Richling, Hau, Kredel, Kruse (URZ/RUM) Long-distance InfiniBand Connection Seattle, November / 25
13 User Management Users should have exclusive access to compute nodes user names and user-ids must be unique direct connection to PBS for user authorization via PAM module Authentication at the access nodes directly against directory services: LDAP (MA) and AD (HD) or with D-Grid certificate Combining information from directory services from both universities Prefix for group names Adding offsets to user-ids and group-ids Activated user names from MA and HD must be different Activation process Adding a special attribute for the user in the directory service (for authentication) Updating the user database of the cluster (for authorization) Richling, Hau, Kredel, Kruse (URZ/RUM) Long-distance InfiniBand Connection Seattle, November / 25
14 User Management Generation of configuration files Directory service MA user unique! Directory service HD user group LDAP +prefix ma +prefix hd user id group id group AD user id group id Adminserver passwd Richling, Hau, Kredel, Kruse (URZ/RUM) Long-distance InfiniBand Connection Seattle, November / 25
15 Job Management Interconnection (high latency, limited bandwidth) provides enough bandwidth for I/O operations not sufficient for all kinds of MPI jobs Jobs run only on nodes located either in HD or in MA (realized with attributes provides by the queuing system) Before interconnection In Mannheim: mostly single node jobs free nodes In Heidelberg: many MPI jobs long waiting times With interconnection better resource utilization (see Ganglia report) Richling, Hau, Kredel, Kruse (URZ/RUM) Long-distance InfiniBand Connection Seattle, November / 25
16 Ganglia Report during activation of the interconnection Richling, Hau, Kredel, Kruse (URZ/RUM) Long-distance InfiniBand Connection Seattle, November / 25
17 MPI Performance Measurements Numerical model High-Performance Linpack (HPL) benchmark OpenMPI Intel MKL Model variants Calculations on a single cluster with up to 1024 CPU cores Calculations on the interconnected cluster with up to 2048 CPU cores symmetrically distributed Richling, Hau, Kredel, Kruse (URZ/RUM) Long-distance InfiniBand Connection Seattle, November / 25
18 Results for a single cluster HPL 1.0a local load parameter (matrix size) n p =40000 n p =30000 n p =20000 n p =10000 speed-up p/ln(p) simple model (Kruse 2009) all CPU configurations have equal probability number of processors p Richling, Hau, Kredel, Kruse (URZ/RUM) Long-distance InfiniBand Connection Seattle, November / 25
19 Results for interconnected cluster speed-up HPL 1.0a for p > 256: reduced speed-up by a factor of 4 for p > 500: constant (decreasing) speed-up MA-HD load parameter (matrix size) n p =40000 n p =30000 n p =20000 n p = number of processors p Richling, Hau, Kredel, Kruse (URZ/RUM) Long-distance InfiniBand Connection Seattle, November / 25
20 Performance model Improvement of simple analytical model (Kruse 2009) to analyze the characteristics of the interconnection high latency of 145 µsec limited bandwidth of 930 MB/sec (modelled as shared medium) Result for Speed-up: p S(p) ( ) 3 ln p n p (1 + 4p)c(p) p number of processors n p load parameter (matrix size) c(p) dimensionless function representing the communication topology Richling, Hau, Kredel, Kruse (URZ/RUM) Long-distance InfiniBand Connection Seattle, November / 25
21 Speed-up of the model Results: Limited bandwidth is the performance bottleneck for shared connection between the clusters Double bandwidth: 25 % improvement for n p = % improvement with a ten-fold bandwidth Jobs run on nodes located either in MA or in HD Richling, Hau, Kredel, Kruse (URZ/RUM) Long-distance InfiniBand Connection Seattle, November / 25
22 Long-term MPI performance Latency between two random nodes in HD or in MA IMB 3.2 PingPong 14 buffer size 0 GB 12 latency [microsec] Jan 12 Feb 26 Feb 11 Mar start time [date] 25 Mar 08 Apr 22 Apr Richling, Hau, Kredel, Kruse (URZ/RUM) Long-distance InfiniBand Connection Seattle, November / 25
23 Long-term MPI performance Bandwidth between two random nodes in HD or in MA IMB 3.2 PingPong 2000 buffer size 1 GB bandwidth [Mbytes/sec] Jan 12 Feb 26 Feb 11 Mar start time [date] 25 Mar 08 Apr 22 Apr Richling, Hau, Kredel, Kruse (URZ/RUM) Long-distance InfiniBand Connection Seattle, November / 25
24 Storage Access Performance IOzone benchmark for 32 GB file with records size 4 MB (node storage) write read Mbytes/sec MA-HD MA-MA HD-HD HD-MA Richling, Hau, Kredel, Kruse (URZ/RUM) Long-distance InfiniBand Connection Seattle, November / 25
25 Summary and Conclusions Interconnection network (Obsidian and InfiniBand switches) is stable and works reliable Bandwidth of 930 MB/sec is sufficient for Lustre file system access single system administration lower administration costs better load balance Setting up a federated authorization is challenging but worthwhile Further reduction of administration costs Lower access barrier for potential users Characteristics of the interconnection is not sufficient for all kinds of MPI jobs Jobs remain on one side of the combined cluster Possible improvements: Adding more parallel fibre lines (very expensive) Investigation of different job scheduler configurations Richling, Hau, Kredel, Kruse (URZ/RUM) Long-distance InfiniBand Connection Seattle, November / 25
Operating two InfiniBand grid clusters over 28 km distance
Operating two InfiniBand grid clusters over 28 km distance Sabine Richling, Steffen Hau, Heinz Kredel, Hans-Günther Kruse IT-Center University of Heidelberg, Germany IT-Center University of Mannheim, Germany
Performance Analysis and Prediction for distributed homogeneous Clusters
Performance Analysis and Prediction for distributed homogeneous Clusters Heinz Kredel, Hans-Günther Kruse, Sabine Richling, Erich Strohmaier IT-Center, University of Mannheim, Germany IT-Center, University
A Long-distance InfiniBand Interconnection between two Clusters in Production Use
A Long-distance InfiniBand Interconnection between two Clusters in Production Use Sabine Richling IT-Center University of Heidelberg 6912 Heidelberg, Germany richling@urz.uniheidelberg.de Heinz Kredel
bwgrid Treff am URZ Sabine Richling, Heinz Kredel Universitätsrechenzentrum Heidelberg Rechenzentrum Universität Mannheim 29.
bwgrid Treff am URZ Sabine Richling, Heinz Kredel Universitätsrechenzentrum Heidelberg Rechenzentrum Universität Mannheim 29. April 2010 Richling/Kredel (URZ/RUM) bwgrid Treff SS 2010 1 / 49 Course Organization
Lessons learned from Lustre file system operation
Lessons learned from Lustre file system operation Roland Laifer STEINBUCH CENTRE FOR COMPUTING - SCC KIT University of the State of Baden-Württemberg and National Laboratory of the Helmholtz Association
Hands-On Workshop bwunicluster June 29th 2015
Hands-On Workshop bwunicluster June 29th 2015 Agenda Welcome Introduction to bwhpc and the bwunicluster Modules - Software Environment Management Job Submission and Monitoring Interactive Work and Remote
bwfortreff bwhpc user meeting
bwfortreff bwhpc user meeting bwhpc Competence Center MLS&WISO Universitätsrechenzentrum Heidelberg Rechenzentrum der Universität Mannheim Steinbuch Centre for Computing (SCC) Funding: www.bwhpc-c5.de
Now SAML takes it all:
Now SAML takes it all: Federation of non Web-based Services in the State of Baden-Württemberg Sebastian Labitzke Karlsruhe Institute of Technology (KIT) Steinbuch Centre for Computing (SCC) labitzke@kit.edu
Assistance in Lustre administration
Assistance in Lustre administration Roland Laifer STEINBUCH CENTRE FOR COMPUTING - SCC KIT University of the State of Baden-Württemberg and National Laboratory of the Helmholtz Association www.kit.edu
Habanero Operating Committee. January
Habanero Operating Committee January 25 2017 Habanero Overview 1. Execute Nodes 2. Head Nodes 3. Storage 4. Network Execute Nodes Type Quantity Standard 176 High Memory 32 GPU* 14 Total 222 Execute Nodes
A Case for High Performance Computing with Virtual Machines
A Case for High Performance Computing with Virtual Machines Wei Huang*, Jiuxing Liu +, Bulent Abali +, and Dhabaleswar K. Panda* *The Ohio State University +IBM T. J. Waston Research Center Presentation
Scalable Ethernet Clos-Switches. Norbert Eicker John von Neumann-Institute for Computing Ferdinand Geier ParTec Cluster Competence Center GmbH
Scalable Ethernet Clos-Switches Norbert Eicker John von Neumann-Institute for Computing Ferdinand Geier ParTec Cluster Competence Center GmbH Outline Motivation Clos-Switches Ethernet Crossbar Switches
Datura The new HPC-Plant at Albert Einstein Institute
Datura The new HPC-Plant at Albert Einstein Institute Nico Budewitz Max Planck Institue for Gravitational Physics, Germany Cluster Day, 2011 Outline 1 History HPC-Plants at AEI -2009 Peyote, Lagavulin,
Communication has significant impact on application performance. Interconnection networks therefore have a vital role in cluster systems.
Cluster Networks Introduction Communication has significant impact on application performance. Interconnection networks therefore have a vital role in cluster systems. As usual, the driver is performance
An Overview of Fujitsu s Lustre Based File System
An Overview of Fujitsu s Lustre Based File System Shinji Sumimoto Fujitsu Limited Apr.12 2011 For Maximizing CPU Utilization by Minimizing File IO Overhead Outline Target System Overview Goals of Fujitsu
Reduces latency and buffer overhead. Messaging occurs at a speed close to the processors being directly connected. Less error detection
Switching Operational modes: Store-and-forward: Each switch receives an entire packet before it forwards it onto the next switch - useful in a general purpose network (I.e. a LAN). usually, there is a
Cluster Network Products
Cluster Network Products Cluster interconnects include, among others: Gigabit Ethernet Myrinet Quadrics InfiniBand 1 Interconnects in Top500 list 11/2009 2 Interconnects in Top500 list 11/2008 3 Cluster
DELL EMC ISILON F800 AND H600 I/O PERFORMANCE
DELL EMC ISILON F800 AND H600 I/O PERFORMANCE ABSTRACT This white paper provides F800 and H600 performance data. It is intended for performance-minded administrators of large compute clusters that access
Computing Model Tier-2 Plans for Germany Relations to GridKa/Tier-1
ATLAS Tier-2 Computing in D GridKa-TAB, Karlsruhe, 30.9.2005 München Computing Model Tier-2 Plans for Germany Relations to GridKa/Tier-1 GridKa-TAB, 30.9.05 1 ATLAS Offline Computing ~Pb/sec PC (2004)
BlueGene/L. Computer Science, University of Warwick. Source: IBM
BlueGene/L Source: IBM 1 BlueGene/L networking BlueGene system employs various network types. Central is the torus interconnection network: 3D torus with wrap-around. Each node connects to six neighbours
ARCHER/RDF Overview. How do they fit together? Andy Turner, EPCC
ARCHER/RDF Overview How do they fit together? Andy Turner, EPCC a.turner@epcc.ed.ac.uk www.epcc.ed.ac.uk www.archer.ac.uk Outline ARCHER/RDF Layout Available file systems Compute resources ARCHER Compute
Sami Saarinen Peter Towers. 11th ECMWF Workshop on the Use of HPC in Meteorology Slide 1
Acknowledgements: Petra Kogel Sami Saarinen Peter Towers 11th ECMWF Workshop on the Use of HPC in Meteorology Slide 1 Motivation Opteron and P690+ clusters MPI communications IFS Forecast Model IFS 4D-Var
HPC Architectures. Types of resource currently in use
HPC Architectures Types of resource currently in use Reusing this material This work is licensed under a Creative Commons Attribution- NonCommercial-ShareAlike 4.0 International License. http://creativecommons.org/licenses/by-nc-sa/4.0/deed.en_us
ABySS Performance Benchmark and Profiling. May 2010
ABySS Performance Benchmark and Profiling May 2010 Note The following research was performed under the HPC Advisory Council activities Participating vendors: AMD, Dell, Mellanox Compute resource - HPC
Our new HPC-Cluster An overview
Our new HPC-Cluster An overview Christian Hagen Universität Regensburg Regensburg, 15.05.2009 Outline 1 Layout 2 Hardware 3 Software 4 Getting an account 5 Compiling 6 Queueing system 7 Parallelization
Choosing Resources Wisely Plamen Krastev Office: 38 Oxford, Room 117 FAS Research Computing
Choosing Resources Wisely Plamen Krastev Office: 38 Oxford, Room 117 Email:plamenkrastev@fas.harvard.edu Objectives Inform you of available computational resources Help you choose appropriate computational
Non-Uniform Memory Access (NUMA) Architecture and Multicomputers
Non-Uniform Memory Access (NUMA) Architecture and Multicomputers Parallel and Distributed Computing Department of Computer Science and Engineering (DEI) Instituto Superior Técnico February 29, 2016 CPD
Data Management. Parallel Filesystems. Dr David Henty HPC Training and Support
Data Management Dr David Henty HPC Training and Support d.henty@epcc.ed.ac.uk +44 131 650 5960 Overview Lecture will cover Why is IO difficult Why is parallel IO even worse Lustre GPFS Performance on ARCHER
Rechenzentrum HIGH PERFORMANCE SCIENTIFIC COMPUTING
Rechenzentrum HIGH PERFORMANCE SCIENTIFIC COMPUTING Contents Scientifi c Supercomputing Center Karlsruhe (SSCK)... 4 Consultation and Support... 5 HP XC 6000 Cluster at the SSC Karlsruhe... 6 Architecture
Non-Uniform Memory Access (NUMA) Architecture and Multicomputers
Non-Uniform Memory Access (NUMA) Architecture and Multicomputers Parallel and Distributed Computing Department of Computer Science and Engineering (DEI) Instituto Superior Técnico September 26, 2011 CPD
Composite Metrics for System Throughput in HPC
Composite Metrics for System Throughput in HPC John D. McCalpin, Ph.D. IBM Corporation Austin, TX SuperComputing 2003 Phoenix, AZ November 18, 2003 Overview The HPC Challenge Benchmark was announced last
MVAPICH MPI and Open MPI
CHAPTER 6 The following sections appear in this chapter: Introduction, page 6-1 Initial Setup, page 6-2 Configure SSH, page 6-2 Edit Environment Variables, page 6-5 Perform MPI Bandwidth Test, page 6-8
Gateways to Discovery: Cyberinfrastructure for the Long Tail of Science
Gateways to Discovery: Cyberinfrastructure for the Long Tail of Science ECSS Symposium, 12/16/14 M. L. Norman, R. L. Moore, D. Baxter, G. Fox (Indiana U), A Majumdar, P Papadopoulos, W Pfeiffer, R. S.
Real Parallel Computers
Real Parallel Computers Modular data centers Overview Short history of parallel machines Cluster computing Blue Gene supercomputer Performance development, top-500 DAS: Distributed supercomputing Short
Non-Uniform Memory Access (NUMA) Architecture and Multicomputers
Non-Uniform Memory Access (NUMA) Architecture and Multicomputers Parallel and Distributed Computing MSc in Information Systems and Computer Engineering DEA in Computational Engineering Department of Computer
LS-DYNA Best-Practices: Networking, MPI and Parallel File System Effect on LS-DYNA Performance
11 th International LS-DYNA Users Conference Computing Technology LS-DYNA Best-Practices: Networking, MPI and Parallel File System Effect on LS-DYNA Performance Gilad Shainer 1, Tong Liu 2, Jeff Layton
2008 International ANSYS Conference
2008 International ANSYS Conference Maximizing Productivity With InfiniBand-Based Clusters Gilad Shainer Director of Technical Marketing Mellanox Technologies 2008 ANSYS, Inc. All rights reserved. 1 ANSYS,
Session Case study. Planning of different broadband solutions in the last mile for urban and suburban areas. suburban areas
ITU/ITC Regional Seminar on Network Evolution to NGN and Fixed Mobile Convergence Session 2.1.4 Case study Planning of different broadband solutions in the last mile for urban and suburban areas Ignat
Shared Parallel Filesystems in Heterogeneous Linux Multi-Cluster Environments
LCI HPC Revolution 2005 26 April 2005 Shared Parallel Filesystems in Heterogeneous Linux Multi-Cluster Environments Matthew Woitaszek matthew.woitaszek@colorado.edu Collaborators Organizations National
Sun Lustre Storage System Simplifying and Accelerating Lustre Deployments
Sun Lustre Storage System Simplifying and Accelerating Lustre Deployments Torben Kling-Petersen, PhD Presenter s Name Principle Field Title andengineer Division HPC &Cloud LoB SunComputing Microsystems
Parallel File Systems Compared
Parallel File Systems Compared Computing Centre (SSCK) University of Karlsruhe, Germany Laifer@rz.uni-karlsruhe.de page 1 Outline» Parallel file systems (PFS) Design and typical usage Important features
Cray XD1 Supercomputer Release 1.3 CRAY XD1 DATASHEET
CRAY XD1 DATASHEET Cray XD1 Supercomputer Release 1.3 Purpose-built for HPC delivers exceptional application performance Affordable power designed for a broad range of HPC workloads and budgets Linux,
QLogic TrueScale InfiniBand and Teraflop Simulations
WHITE Paper QLogic TrueScale InfiniBand and Teraflop Simulations For ANSYS Mechanical v12 High Performance Interconnect for ANSYS Computer Aided Engineering Solutions Executive Summary Today s challenging
A More Realistic Way of Stressing the End-to-end I/O System
A More Realistic Way of Stressing the End-to-end I/O System Verónica G. Vergara Larrea Sarp Oral Dustin Leverman Hai Ah Nam Feiyi Wang James Simmons CUG 2015 April 29, 2015 Chicago, IL ORNL is managed
Experiences with HP SFS / Lustre in HPC Production
Experiences with HP SFS / Lustre in HPC Production Computing Centre (SSCK) University of Karlsruhe Laifer@rz.uni-karlsruhe.de page 1 Outline» What is HP StorageWorks Scalable File Share (HP SFS)? A Lustre
Interconnect Your Future
Interconnect Your Future Gilad Shainer 2nd Annual MVAPICH User Group (MUG) Meeting, August 2014 Complete High-Performance Scalable Interconnect Infrastructure Comprehensive End-to-End Software Accelerators
My operating system is old but I don't care : I'm using NIX! B.Bzeznik BUX meeting, Vilnius 22/03/2016
My operating system is old but I don't care : I'm using NIX! B.Bzeznik BUX meeting, Vilnius 22/03/2016 CIMENT is the computing center of the University of Grenoble CIMENT computing platforms 132Tflops
VREDPro HPC Raytracing Cluster
1 HPC Raytracing Cluster... 1 1.1 Introduction... 1 1.2 Configuration... 2 1.2.1 Cluster Options... 4 1.2.2 Network Options... 5 1.2.3 Render Node Options... 6 1.2.4 Preferences... 6 1.2.5 Starting the
Determining Optimal MPI Process Placement for Large- Scale Meteorology Simulations with SGI MPIplace
Determining Optimal MPI Process Placement for Large- Scale Meteorology Simulations with SGI MPIplace James Southern, Jim Tuccillo SGI 25 October 2016 0 Motivation Trend in HPC continues to be towards more
InfiniBand and Next Generation Enterprise Networks
InfiniBand and Next Generation Enterprise Networks Paul Morkel and Brian Savory ADVA Optical Networking The pace of technical innovation is accelerating Adoption of new technologies are taking hold at
Beyond Petascale. Roger Haskin Manager, Parallel File Systems IBM Almaden Research Center
Beyond Petascale Roger Haskin Manager, Parallel File Systems IBM Almaden Research Center GPFS Research and Development! GPFS product originated at IBM Almaden Research Laboratory! Research continues to
Altix Usage and Application Programming
Center for Information Services and High Performance Computing (ZIH) Altix Usage and Application Programming Discussion And Important Information For Users Zellescher Weg 12 Willers-Bau A113 Tel. +49 351-463
Design and Evaluation of a 2048 Core Cluster System
Design and Evaluation of a 2048 Core Cluster System, Torsten Höfler, Torsten Mehlan and Wolfgang Rehm Computer Architecture Group Department of Computer Science Chemnitz University of Technology December
The Red Storm System: Architecture, System Update and Performance Analysis
The Red Storm System: Architecture, System Update and Performance Analysis Douglas Doerfler, Jim Tomkins Sandia National Laboratories Center for Computation, Computers, Information and Mathematics LACSI
Parallel Computer Architecture II
Parallel Computer Architecture II Stefan Lang Interdisciplinary Center for Scientific Computing (IWR) University of Heidelberg INF 368, Room 532 D-692 Heidelberg phone: 622/54-8264 email: Stefan.Lang@iwr.uni-heidelberg.de
ANALYSIS OF CLUSTER INTERCONNECTION NETWORK TOPOLOGIES
ANALYSIS OF CLUSTER INTERCONNECTION NETWORK TOPOLOGIES Sergio N. Zapata, David H. Williams and Patricia A. Nava Department of Electrical and Computer Engineering The University of Texas at El Paso El Paso,
Scalability and Classifications
Scalability and Classifications 1 Types of Parallel Computers MIMD and SIMD classifications shared and distributed memory multicomputers distributed shared memory computers 2 Network Topologies static
Experiences with the Parallel Virtual File System (PVFS) in Linux Clusters
Experiences with the Parallel Virtual File System (PVFS) in Linux Clusters Kent Milfeld, Avijit Purkayastha, Chona Guiang Texas Advanced Computing Center The University of Texas Austin, Texas USA Abstract
Birds of a Feather Presentation
Mellanox InfiniBand QDR 4Gb/s The Fabric of Choice for High Performance Computing Gilad Shainer, shainer@mellanox.com June 28 Birds of a Feather Presentation InfiniBand Technology Leadership Industry Standard
NCAR Globally Accessible Data Environment (GLADE) Updated: 15 Feb 2017
NCAR Globally Accessible Data Environment (GLADE) Updated: 15 Feb 2017 Overview The Globally Accessible Data Environment (GLADE) provides centralized file storage for HPC computational, data-analysis,
Feedback on BeeGFS. A Parallel File System for High Performance Computing
Feedback on BeeGFS A Parallel File System for High Performance Computing Philippe Dos Santos et Georges Raseev FR 2764 Fédération de Recherche LUmière MATière December 13 2016 LOGO CNRS LOGO IO December
Optimizing LS-DYNA Productivity in Cluster Environments
10 th International LS-DYNA Users Conference Computing Technology Optimizing LS-DYNA Productivity in Cluster Environments Gilad Shainer and Swati Kher Mellanox Technologies Abstract Increasing demand for
COSC 6385 Computer Architecture - Multi Processor Systems
COSC 6385 Computer Architecture - Multi Processor Systems Fall 2006 Classification of Parallel Architectures Flynn s Taxonomy SISD: Single instruction single data Classical von Neumann architecture SIMD:
Technical guide. Windows HPC server 2016 for LS-DYNA How to setup. Reference system setup - v1.0
Technical guide Windows HPC server 2016 for LS-DYNA How to setup Reference system setup - v1.0 2018-02-17 2018 DYNAmore Nordic AB LS-DYNA / LS-PrePost 1 Introduction - Running LS-DYNA on Windows HPC cluster
MOHA: Many-Task Computing Framework on Hadoop
Apache: Big Data North America 2017 @ Miami MOHA: Many-Task Computing Framework on Hadoop Soonwook Hwang Korea Institute of Science and Technology Information May 18, 2017 Table of Contents Introduction
Determining the MPP LS-DYNA Communication and Computation Costs with the 3-Vehicle Collision Model and the Infiniband Interconnect
8 th International LS-DYNA Users Conference Computing / Code Tech (1) Determining the MPP LS-DYNA Communication and Computation Costs with the 3-Vehicle Collision Model and the Infiniband Interconnect
Extraordinary HPC file system solutions at KIT
Extraordinary HPC file system solutions at KIT Roland Laifer STEINBUCH CENTRE FOR COMPUTING - SCC KIT University of the State Roland of Baden-Württemberg Laifer Lustre and tools for ldiskfs investigation
The National Analysis DESY
The National Analysis Facility @ DESY Yves Kemp for the NAF team DESY IT Hamburg & DV Zeuthen 10.9.2008 GridKA School NAF: National Analysis Facility Why a talk about an Analysis Facility at a Grid School?
Managing CAE Simulation Workloads in Cluster Environments
Managing CAE Simulation Workloads in Cluster Environments Michael Humphrey V.P. Enterprise Computing Altair Engineering humphrey@altair.com June 2003 Copyright 2003 Altair Engineering, Inc. All rights
Single-Points of Performance
Single-Points of Performance Mellanox Technologies Inc. 29 Stender Way, Santa Clara, CA 9554 Tel: 48-97-34 Fax: 48-97-343 http://www.mellanox.com High-performance computations are rapidly becoming a critical
MERCED CLUSTER BASICS Multi-Environment Research Computer for Exploration and Discovery A Centerpiece for Computational Science at UC Merced
MERCED CLUSTER BASICS Multi-Environment Research Computer for Exploration and Discovery A Centerpiece for Computational Science at UC Merced Sarvani Chadalapaka HPC Administrator University of California
UAntwerpen, 24 June 2016
Tier-1b Info Session UAntwerpen, 24 June 2016 VSC HPC environment Tier - 0 47 PF Tier -1 623 TF Tier -2 510 Tf 16,240 CPU cores 128/256 GB memory/node IB EDR interconnect Tier -3 HOPPER/TURING STEVIN THINKING/CEREBRO
DELIVERABLE D5.5 Report on ICARUS visualization cluster installation. John BIDDISCOMBE (CSCS) Jerome SOUMAGNE (CSCS)
DELIVERABLE D5.5 Report on ICARUS visualization cluster installation John BIDDISCOMBE (CSCS) Jerome SOUMAGNE (CSCS) 02 May 2011 NextMuSE 2 Next generation Multi-mechanics Simulation Environment Cluster
Virtualization of the ATLAS Tier-2/3 environment on the HPC cluster NEMO
Virtualization of the ATLAS Tier-2/3 environment on the HPC cluster NEMO Ulrike Schnoor (CERN) Anton Gamel, Felix Bührer, Benjamin Rottler, Markus Schumacher (University of Freiburg) February 02, 2018
Scaling to Petaflop. Ola Torudbakken Distinguished Engineer. Sun Microsystems, Inc
Scaling to Petaflop Ola Torudbakken Distinguished Engineer Sun Microsystems, Inc HPC Market growth is strong CAGR increased from 9.2% (2006) to 15.5% (2007) Market in 2007 doubled from 2003 (Source: IDC
Advanced Data Placement via Ad-hoc File Systems at Extreme Scales (ADA-FS)
Advanced Data Placement via Ad-hoc File Systems at Extreme Scales (ADA-FS) Understanding I/O Performance Behavior (UIOP) 2017 Sebastian Oeste, Mehmet Soysal, Marc-André Vef, Michael Kluge, Wolfgang E.
Next-Generation NVMe-Native Parallel Filesystem for Accelerating HPC Workloads
Next-Generation NVMe-Native Parallel Filesystem for Accelerating HPC Workloads Liran Zvibel CEO, Co-founder WekaIO @liranzvibel 1 WekaIO Matrix: Full-featured and Flexible Public or Private S3 Compatible
Enabling Performance-per-Watt Gains in High-Performance Cluster Computing
WHITE PAPER Appro Xtreme-X Supercomputer with the Intel Xeon Processor E5-2600 Product Family Enabling Performance-per-Watt Gains in High-Performance Cluster Computing Appro Xtreme-X Supercomputer with
bwfdm Communities - a Research Data Management Initiative in the State of Baden-Wuerttemberg
bwfdm Communities - a Research Data Management Initiative in the State of Baden-Wuerttemberg Karlheinz Pappenberger Tromsø, 9th Munin Conference on Scholarly Publishing, 27/11/2014 Overview 1) Federalism
Pushing the Limits. ADSM Symposium Sheelagh Treweek September 1999 Oxford University Computing Services 1
Pushing the Limits ADSM Symposium Sheelagh Treweek sheelagh.treweek@oucs.ox.ac.uk September 1999 Oxford University Computing Services 1 Overview History of ADSM services at Oxford October 1995 - started
RDMA Read Based Rendezvous Protocol for MPI over InfiniBand: Design Alternatives and Benefits
RDMA Read Based Rendezvous Protocol for MPI over InfiniBand: Design Alternatives and Benefits Sayantan Sur Hyun-Wook Jin Lei Chai D. K. Panda Network Based Computing Lab, The Ohio State University Presentation
INTEGRATING HPFS IN A CLOUD COMPUTING ENVIRONMENT
INTEGRATING HPFS IN A CLOUD COMPUTING ENVIRONMENT Abhisek Pan 2, J.P. Walters 1, Vijay S. Pai 1,2, David Kang 1, Stephen P. Crago 1 1 University of Southern California/Information Sciences Institute 2
Cray events. ! Cray User Group (CUG): ! Cray Technical Workshop Europe:
Cray events! Cray User Group (CUG):! When: May 16-19, 2005! Where: Albuquerque, New Mexico - USA! Registration: reserved to CUG members! Web site: http://www.cug.org! Cray Technical Workshop Europe:! When:
UL HPC Monitoring in practice: why, what, how, where to look
C. Parisot UL HPC Monitoring in practice: why, what, how, where to look 1 / 22 What is HPC? Best Practices Getting Fast & Efficient UL HPC Monitoring in practice: why, what, how, where to look Clément
Introduction to High Performance Parallel I/O
Introduction to High Performance Parallel I/O Richard Gerber Deputy Group Lead NERSC User Services August 30, 2013-1- Some slides from Katie Antypas I/O Needs Getting Bigger All the Time I/O needs growing
Real Application Performance and Beyond
Real Application Performance and Beyond Mellanox Technologies Inc. 2900 Stender Way, Santa Clara, CA 95054 Tel: 408-970-3400 Fax: 408-970-3403 http://www.mellanox.com Scientists, engineers and analysts
MM5 Modeling System Performance Research and Profiling. March 2009
MM5 Modeling System Performance Research and Profiling March 2009 Note The following research was performed under the HPC Advisory Council activities AMD, Dell, Mellanox HPC Advisory Council Cluster Center
S THE MAKING OF DGX SATURNV: BREAKING THE BARRIERS TO AI SCALE. Presenter: Louis Capps, Solution Architect, NVIDIA,
S7750 - THE MAKING OF DGX SATURNV: BREAKING THE BARRIERS TO AI SCALE Presenter: Louis Capps, Solution Architect, NVIDIA, lcapps@nvidia.com A TALE OF ENLIGHTENMENT Basic OK List 10 for x = 1 to 3 20 print
Advances of parallel computing. Kirill Bogachev May 2016
Advances of parallel computing Kirill Bogachev May 2016 Demands in Simulations Field development relies more and more on static and dynamic modeling of the reservoirs that has come a long way from being
OBTAINING AN ACCOUNT:
HPC Usage Policies The IIA High Performance Computing (HPC) System is managed by the Computer Management Committee. The User Policies here were developed by the Committee. The user policies below aim to
Readme for Platform Open Cluster Stack (OCS)
Readme for Platform Open Cluster Stack (OCS) Version 4.1.1-2.0 October 25 2006 Platform Computing Contents What is Platform OCS? What's New in Platform OCS 4.1.1-2.0? Supported Architecture Distribution
APENet: LQCD clusters a la APE
Overview Hardware/Software Benchmarks Conclusions APENet: LQCD clusters a la APE Concept, Development and Use Roberto Ammendola Istituto Nazionale di Fisica Nucleare, Sezione Roma Tor Vergata Centro Ricerce
n N c CIni.o ewsrg.au
@NCInews NCI and Raijin National Computational Infrastructure 2 Our Partners General purpose, highly parallel processors High FLOPs/watt and FLOPs/$ Unit of execution Kernel Separate memory subsystem GPGPU
Parallel File Systems for HPC
Introduction to Scuola Internazionale Superiore di Studi Avanzati Trieste November 2008 Advanced School in High Performance and Grid Computing Outline 1 The Need for 2 The File System 3 Cluster & A typical
Apace Systems. Avid Unity Media Offload Solution KIT
Apace Systems Networked Storage for Video Backup 6TB Unity in 8 Hours! Instant restore! WOW!!!! Apace Systems Avid Unity Media Offload Solution KIT Backup / restore / shared storage / expanded access from
MPI Optimizations via MXM and FCA for Maximum Performance on LS-DYNA
MPI Optimizations via MXM and FCA for Maximum Performance on LS-DYNA Gilad Shainer 1, Tong Liu 1, Pak Lui 1, Todd Wilde 1 1 Mellanox Technologies Abstract From concept to engineering, and from design to
The Why and How of HPC-Cloud Hybrids with OpenStack
The Why and How of HPC-Cloud Hybrids with OpenStack OpenStack Australia Day Melbourne June, 2017 Lev Lafayette, HPC Support and Training Officer, University of Melbourne lev.lafayette@unimelb.edu.au 1.0
Real Parallel Computers
Real Parallel Computers Modular data centers Background Information Recent trends in the marketplace of high performance computing Strohmaier, Dongarra, Meuer, Simon Parallel Computing 2005 Short history
Intel Cluster Toolkit Compiler Edition 3.2 for Linux* or Windows HPC Server 2008*
Intel Cluster Toolkit Compiler Edition. for Linux* or Windows HPC Server 8* Product Overview High-performance scaling to thousands of processors. Performance leadership Intel software development products
Sharing High-Performance Devices Across Multiple Virtual Machines
Sharing High-Performance Devices Across Multiple Virtual Machines Preamble What does sharing devices across multiple virtual machines in our title mean? How is it different from virtual networking / NSX,