SUSE. High Performance Computing. Kai Dupke. Meike Chabowski. Senior Product Manager SUSE Linux Enterprise

Transcription

1 SUSE High Performance Computing Kai Dupke Meike Chabowski Senior Product Manager SUSE Linux Enterprise Senior Product Marketing Manager SUSE Linux Enterprise Distribution: pdf any Date: Not a public document.

2 3

3 HPC Impacts Our Lives Space Research 4

4 5

5 HPC Impacts Our Lives Weather & Climate 6

6 7

7 HPC Impacts Our Lives Oil & Gas 8

8 9

9 HPC Impacts Our Lifes - Entertainment 10

10 From Cinema... to Games 11

11 High Productivity Computing Big Data or HPC?? 12

12 Overview

13 HPC Overview SUSE High Performance Computing 14 Solving computational, data-intensive, or numerically-intensive tasks Reducing the time and effort required to set-up and maintain HPC clusters Ensuring that all components of the HPC stack work together

14 HPC Development SUSE High Performance Computing Yesterday Today Academia and Research Financial Services Oil and Gas Semiconductor Life Sciences Manufacturing Tomorrow 15 Academia and Research Departmental and workgroup clusters High Productivity Computing

15 Market Segmentation SUSE High Performance Computing Super Computer 'top 500' System Ready Key drivers Budget >500K$ Special build HW +++ only performance count Self-supported Partner supported Divisional <500K$ +++ Customized HW Partner driven Partner supported SUSE supported IHV ISV SI Departmental <250K$ ++ Commodity HW Business driven SUSE supported Channel SUSE Work Group <100K$ + Customer driven Home brewed Channel Shop HPC class 16 GTM HPC-IHV

16 Split Market SUSE HPC Commercial High Productivity Computing Scientific Top 500-class Lighthouse projects Government sponsored Generic workloads Often self-supported by Academic staff 17 Specialized hardware Highly specialized application ROI and reliability are key Data Center support Commodity hardware

17 SUSE Linux Enterprise HPC

18 SUSE Strong in HPC Market! SUSE HPC MULTI- and MANY-CORE PROCESSOR SUPPORT Intel, AMD, POWER.. COOPERATION TECHNOLOGY Kernel 3.x Lustre enablement Ceph storage platform up to 8192 cores.. SUSE Since 1992 Strong in Top500 HIGH PRODUCTIVITY COMPUTING Total Baker Hughes Texas Instruments.. 19 IBM SGI HP Dell.. Bull NEC Cray Cisco.. ACADEMIC AND RESEARCH LRZ / SuperMUC BSC / MareNostrum Tokyo Institute of Technology Beijing Computing Center NASA..

19 Why Linux? SUSE High Performance Computing Open Source benefits Innovation 20 Easy to customize, maintain and improve Beowulf Clusters born on Linux Modularity GUI overhead not required appliance form factors Linux Standards Large base of tools, including remote management Hardware availability Large vendor ecosystem surrounding Linux HPC clusters

20 Linux Preferred for HPC SUSE High Performance Computing Linux runs on more than 97% of the world's top 500 supercomputers* is used by nearly 90% of general clusters Linux is used in the majority of HPC systems, from smaller departmental implementations to larger, integrated cluster solutions *top500.org July

21 Why SUSE Linux Enterprise Server For High Performance Computing 22 Early player in HPC, pushing innovation and new technologies Highly reliable, interoperable and manageable server operating system Built to power mission-critical workloads in physical, virtual and cloud environments The natural successor to UNIX, backed by proven services for UNIX migration Special features to improve performance Backed by established ecosystem support and certificates The only Linux recommended by Microsoft

22 SUSE Additional Features SUSE High Performance Computing Up-to-date Linux Kernel for optimal performance CPU Management and System Activity CPUset System, CPUset command line tool Sysstat package IRQbalance OpenFabrics Enterprise Distribution (OFED) 23 Remote Direct Memory Access (RDMA) switched fabric technologies, high-speed data transport technologies for server and storage connectivity SystemTap, LTTng 2.0 Lustre enabled Kernel

23 SUSE Advanced I/O Processing SUSE High Performance Computing Asynchronous I/O (AIO) Modular I/O Scheduler Algorithm most suitable for workload can be chosen dynamically Multi-core/hyper-threading processor support Execute threads in parallel within each individual processor Supports up to 4096 cores per system Intel I/O Acceleration 24 Input/output processing that permits other processing to continue before the transmission has finished Offloads the CPU towards the network card, thus allowing the system to continue processing data while I/O is taking place

24 Update

25 Simplify projects! SUSE HPC 26 Simplified model Number of socket pairs matter Socket pairs are accumulated per system Head nodes and compute nodes are threaten equal

26 Example regular HPC setup SUSE HPC 2 (8 sockets) head nodes for redundancy / scalability 100 (4 sockets) compute nodes 416 sockets total (order: 208x 1-2 sockets) Client 27 Head Node 2x 8 sockets + Compute Nodes 100x 4 sockets = total 416 sockets

27 Keep it running! SUSE HPC 29 SUSE Vendor Support Maintenance Standard & Priority support for the whole system

28 Recent Developments SUSE High Performance Computing Storage Release of SUSE Storage with Infiniband support ARM64 SUSE Linux Enterprise 11 SP4 with latest HW enablement (Intel, Power8) Network Partnering with Cavium Higher network throughput SUSE Linux Enterprise for ARM64 Added tunables in the IP stack (for lower latency) Cloud 30 MS Azure with SLES 12: RDMA & Infiniband

29 Partner, Customers

30 Customers and Partners SUSE High Performance Computing Customers Partners 32

31 PANGEA Total Exploration SUSE High Performance Computing 33 Oil & Gas exploration Process seismic data Simulation of deposit fluids Superior Performance 2.3 petaflops 10x performance increase Equivalent to 27,000 PCs Future is coming 6.7 petaflops Equivalent to 80,000 PCs... from our point of view, SGI plus SUSE Linux Enterprise Server was a complete, integrated solution.... SUSE Linux Enterprise Server gives us the ability to keep scaling on ever larger machines. Diego Klahr HPC Engineer Total

32 Intel Cluster Ready Program SUSE High Performance Computing Designed to simplify purchasing, deployment and management of HPC clusters SUSE Linux Enterprise Server is Intel Cluster Ready and powers many certified Intel Cluster Ready systems intel Cluster Ready recipes are available with SUSE Linux Enterprise Server 36 Reference designs to help hardware vendors, platform integrators, and system integrators design and build certified Intel Cluster Ready systems

33 Outlook

34 Challenge SUSE High Performance Computing HPC market fast developing Stack components provided by various vendors Some stack components run in parallel Mix of small and big vendors Segmented into commercial and scientific 38

35 Outlook SUSE High Performance Computing SUSE Linux Enterprise SP1 beta program running SUSE HPC Evaluating optimized SUSE Linux Enterprise for HPC Your input is needed! Forward looking statement, might change without notice.

36 Learn more Thank you. Kai Dupke Senior Product Manager SUSE Linux Enterprise 40 Meike Chabowski Senior Product Marketing Manager SUSE Linux Enterprise

37 Backup

38 HPC Stack

39 HPC Stack SUSE High Performance Computing Application Queuing / Management Software & Tools PBS Pro Moab IBM LSF Bright CM Storage EXT3 XFS BTRFS OCFS2 NFS pnfs GPFS pnfs IBRIX Message Passing Interface MPI Network 10G OFED Parastation TCP offload MPICH Lustre cephfs SUSE Linux Enterprise Server Hardware = SUSE supported 43 = SUSE Partner = SUSE future Intel HP openmpi SGI

40 SuperMUC

41 SuperMUC Facts SUSE High Performance Computing 45 60x faster, one of the fastest HPC systems in Europe 20x better performance per Watt, provide green HPC > 155,000 Intel Xeon Processor, migration from Itanium2 to x86

42 LRZ - Leibniz Rechenzentrum Europe s supercomputer run SUSE Linux Enterprise Server Business challenge: LRZ is part of the Gauss Centre for Supercomputing (GCS), which operates the most powerful HPC infrastructure in Europe, and needs to provide researchers across Europe with a reliable and powerful HPC platform, which enables users to make faster progress in their complex research projects. To reduce the environmental impact of HPC, the institution aimed at improving the energy efficiency leverage established automation solutions to maximise the efficiency and manageability of the new supercomputing platform. Solution: Working with SUSE and IBM, LRZ implemented SuperMUC with approx. 9,400 general purpose computing nodes, a peak performance of three Petaflop/s, comprised of 155,000 Intel Xeon processor cores and more than 300 TB main memory. LRZ chose to run SuperMUC on SUSE Linux Enterprise Server, leveraging SUSE s proven HPC expertise and leading automation tools such as AutoYaST, which allows systems to be installed without manual intervention. Benefits: Completed easy and smooth migration from previous Itanium 2 infrastructure to new x86 processor architecture Considerably simplified configuration and automation of the new system, using the automation capabilities of AutoYaST(integrated with SLES) Improved the energy efficiency: SuperMUC delivers appro. 20 times more performance per watt than its predecessor Boosted overall performance by a factor of We have relied on SUSE Linux Enterprise Server for 15 years, and have always been very satisfied. The SUSE team is close at hand, should we require support or guidance. We have received highly competent support over the years, and look forward to collaborating with them. Dr. Herbert Huber Division Head of Supercomputing Leibniz Rechenzentrum

43 SuperMUC Facts and Business Aspects SUSE High Performance Computing 47

44 SuperMUC System Overview 48

45 SuperMUC System Overview 49

46 SuperMUC Business Aspects SUSE High Performance Computing Hot Water Cooling reduce cooling cost Use free air cooling Use of system heat for heating and technical processes RAS driven high system availability Full maintained SUSE Linux Enterprise Server Full support via IBM and SUSE Automated deployment less management cost 50 Full use of SUSE's autoyast feature

47 SuperMUC SUSE benefits SUSE High Performance Computing 51 Support for Itanium2 and x86 Smooth migration of old to new system No additional staff training needed Great support experience Cooperation for more than 15 years Backed by SUSE's winning support Easy deployment methods SUSE's autoyast used today Other SUSE offerings SUSE Cloud, SUSE Manager considered We have relied on SUSE Linux Enterprise Server for 15 years, and have always been very satisfied. The SUSE team is close at hand, should we require support or guidance. We have received highly competent support over the years, and look forward to collaborating with them. Dr. Herbert Huber Division Head of Supercomputing Leibniz Rechenzentrum

48 52 Corporate Headquarters (Worldwide) Join us on: Maxfeldstrasse Nuremberg Germany

49 Unpublished Work of SUSE. All Rights Reserved. This work is an unpublished work and contains confidential, proprietary and trade secret information of SUSE. Access to this work is restricted to SUSE employees who have a need to know to perform tasks within the scope of their assignments. No part of this work may be practiced, performed, copied, distributed, revised, modified, translated, abridged, condensed, expanded, collected, or adapted without the prior written consent of SUSE. Any use or exploitation of this work without authorization could subject the perpetrator to criminal and civil liability. General Disclaimer This document is not to be construed as a promise by any participating company to develop, deliver, or market a product. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. SUSE makes no representations or warranties with respect to the contents of this document, and specifically disclaims any express or implied warranties of merchantability or fitness for any particular purpose. The development, release, and timing of features or functionality described for SUSE products remains at the sole discretion of SUSE. Further, SUSE reserves the right to revise this document and to make changes to its content, at any time, without obligation to notify any person or entity of such revisions or changes. All SUSE marks referenced in this presentation are trademarks or registered trademarks of Novell, Inc. in the United States and other countries. All third-party trademarks are the property of their respective owners.