Interconnect Your Future Paving the Road to Exascale

Size: px

Start display at page:

Download "Interconnect Your Future Paving the Road to Exascale"

Arthur Montgomery
5 years ago
Views:

1 Interconnect Your Future Paving the Road to Exascale CHPC, December 2017

2 90% of the World Data was Created in the Last 2 Years 2017 Mellanox Technologies 2

3 The Future Depends on Fastest Interconnects 1Gb/s 10Gb/s 40/56/100/200Gb/s 2017 Mellanox Technologies 3

Leading Interconnect, Leading Performance Bandwidth 100Gb/s 200Gb/s 20Gb/s 40Gb/s 56Gb/s 10Gb/s 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 Same

4 Leading Interconnect, Leading Performance Bandwidth 100Gb/s 200Gb/s 20Gb/s 40Gb/s 56Gb/s 10Gb/s Same Software Interface usec 2.5usec 1.3usec 0.7usec Latency 0.6usec <0.5usec 2017 Mellanox Technologies 4

Company Overview Ticker: MLNX Leading provider of high-throughput, low-latency Server and Storage Interconnect EDR 100Gb/s (5 th generation) InfiniBand and 10/25/40/50/100GbE HDR 200G coming 1Q18

5 Company Overview Ticker: MLNX Leading provider of high-throughput, low-latency Server and Storage Interconnect EDR 100Gb/s (5 th generation) InfiniBand and 10/25/40/50/100GbE HDR 200G coming 1Q18 Reduces application wait-time for data Dramatically increases ROI on data center infrastructure Company headquarters: Yokneam, Israel; Sunnyvale, California ~ 3,000 employees worldwide Solid financial position Record revenue last 2 years >$1B On a run rate close to $1B annual Sales First Country Manager in South Africa this month 2017 Mellanox Technologies 5

6 Leading Supplier of End-to-End Interconnect Solutions Server / Compute Switch / Gateway Virtual Protocol Interconnect Virtual Protocol Interconnect 100/200G InfiniBand 100/200G InfiniBand 10/40/56/100GbE 10/40/56/100GbE Storage Front / Back-End Comprehensive End-to-End InfiniBand and Ethernet Portfolio (VPI) ICs Adapter Cards NPU & Multicore Switches/Gateways Software Metro / WAN Cables/Modules TILE NPS 2017 Mellanox Technologies 6

7 HPC and AI Needs the Most Intelligent Interconnect SmartNIC System on a Chip Higher Data Speeds Faster Data Processing Better Data Security Adapters Switches Cables & Transceivers 2017 Mellanox Technologies 7

8 Legacy approach: Men flee lions and CPU does it all 2017 Mellanox Technologies 8

9 Innovative approach: Lions flee Men and Network offload CPU 2017 Mellanox Technologies 9

10 Innovative approach: Lions flee Men and Network offload CPU 2017 Mellanox Technologies 10

11 The Need for Intelligent and Faster Interconnect CPU-Centric (Onload) Data-Centric (Offload) Must Wait for the Data Creates Performance Bottlenecks Analyze Data as it Moves! Faster Data Speeds and In-Network Computing Enable Higher Performance and Scale 2017 Mellanox Technologies 11

In-Network Computing Delivers Highest Performance

for HPC and Machine Learning Applications In-Network

Application Performance Self-Healing Technology 5000X

RDMA GPU Acceleration Technology 10X Performance

12 In-Network Computing Delivers Highest Performance In-Network Computing 10X Performance Acceleration Critical for HPC and Machine Learning Applications In-Network Computing 35X Performance Acceleration Delivers Highest Application Performance Self-Healing Technology 5000X Faster Network Recovery Unbreakable Data Centers GPUDirect RDMA GPU Acceleration Technology 10X Performance Acceleration Critical for HPC and Machine Learning Applications 2017 Mellanox Technologies 12

Mellanox Connects the World s Fastest Supercomputer National Supercomputing Center in Wuxi, China #1 on the TOP500 List 93 Petaflop performance, 3X higher versus #2 on the TOP500 41K nodes, 10

13 Mellanox Connects the World s Fastest Supercomputer National Supercomputing Center in Wuxi, China #1 on the TOP500 List 93 Petaflop performance, 3X higher versus #2 on the TOP500 41K nodes, 10 million cores, 256 cores per CPU Mellanox adapter and switch solutions * Source: Report on the Sunway TaihuLight System, Jack Dongarra (University of Tennessee), June 20, 2016 (Tech Report UT-EECS ) 2017 Mellanox Technologies 13

Mellanox Accelerates Next Generation HPC

14 Mellanox Accelerates Next Generation HPC and AI Systems Summit CORAL System Sierra CORAL System Fastest Supercomputer in Japan Fastest Supercomputer in Canada InfiniBand Dragonfly+ Topology Highest Performance with InfiniBand 2017 Mellanox Technologies 14

15 Maximum Efficiency and Return on Investment Mellanox smart interconnect solutions enable In-Network Computing and CPU-Offloading Critical with CPU accelerators and higher scale deployments Ensures highest system efficiency and overall return on investment InfiniBand System: NASA, System Efficiency: 84% System: NCAR, System Efficiency: 90% System: US Army, System Efficiency: 94% Omni-Path System: CINECA, System Efficiency: 57% System: Barcelona, System Efficiency: 62% System: TACC, System Efficiency: 53% 1.7X Higher System Efficiency! 43% System Resources not Utilized! 2017 Mellanox Technologies 15

HW Tag-Matching Accelerates Application Performance DiRAC is the integrated supercomputing facility for theoretical modelling and HPC-based research in particle

16 HW Tag-Matching Accelerates Application Performance DiRAC is the integrated supercomputing facility for theoretical modelling and HPC-based research in particle physics, nuclear physics, astronomy and cosmology Hardware Tag-Matching Accelerates Performance by 90% and Deliver Superior Scaling 2017 Mellanox Technologies 16

17 InfiniBand Enables Highest Applications Efficiency Relative applications efficiency per cluster size OmniPath efficiency relative to InfiniBand, various applications (GROMCS, NAMD, Fluent, LS-DYNA, VASP) 2017 Mellanox Technologies 17

18 InfiniBand Computational Fluid Dynamic Fluent Application Example InfiniBand Always Wins: $441,690 Best Performance. Best R.O.I. System Cost OmniPath $982,560 With Free Network InfiniBand $540,870 $441,690 Cost 45% Savings Lower TCO Based on 64-node Scalable Unit Performance 2017 Mellanox Technologies 18

19 Application Performance Comparison Fluent (Oil_Rig_7M) Intel Claims OPA 17% Better Reality EDR 71% Better Intel Claims OPA 50% Better Reality EDR 93% Better Higher is better Higher is better Mellanox InfiniBand performance versus Intel published results for OmniPath and InfiniBand InfiniBand with 32-cores Broadwell CPU outperforms OmniPath with 36-cores CPUs InfiniBand Delivers 93% Higher Performance and Superior Scaling 32-Nodes InfiniBand Cluster Outperforms 64-Nodes Omni-Path Cluster EDR (Mellanox) Mellanox test results, including the usage of Mellanox In-Network Computing and other acceleration engines. OmniPath (Intel) Intel published test results for OmniPath. EDR (Intel) Intel published test results for EDR InfiniBand. Mellanox testing done with 32-cores Broadwell CPUs, Intel testing were done with 36-cores Broadwell CPUs. A normalized chart estimates performance comparison if all would have been done on 36-cores Broadwell CPUs Mellanox Technologies 19

20 InfiniBand Chemistry Simulation VASP Application Example InfiniBand Always Wins: $98,718 Best Performance. Best R.O.I. System Cost OmniPath $245,640 With Free Network InfiniBand $146,922 $98,718 Cost 40% Savings Lower TCO Based on 16-node Scalable Unit Performance 2017 Mellanox Technologies 20

21 Application Performance Comparison VASP (HEZB), Skylake CPUs Higher is better 17% Higher performance 51% Higher performance 58% Higher performance 71% Higher performance 8 nodes InfiniBand cluster delivers 24% Higher Performance versus 16 nodes Omni-Path cluster InfiniBand Delivers 48% Higher Performance and Superior Scaling 8-Nodes InfiniBand Skylake Cluster Outperforms 16-Nodes Omni-Path Skylake Cluster 2017 Mellanox Technologies 21

22 An Intelligent Network Unlocks the Power of AI RDMA Supercharges Leading AI Frameworks Cognitive Toolkit 60% Higher ROI Mellanox Supercharges Leading AI Companies 50% Lower CapEx & OpEx 2017 Mellanox Technologies 22

23 The Advantages of HDR InfiniBand Solutions Maximum Performance and Highest Return on Investment World s First 200G Switch World s First 200G Adapter 2017 Mellanox Technologies 23

Highest-Performance 200Gb/s Interconnect Solutions 200Gb/s Adapter, 0.

200Gb/s) 40 HDR (200Gb/s) InfiniBand Ports 80 HDR100 InfiniBand Ports

Ports (10 / 25 / 40 / 50 / 100 / 200 GbE) Throughput of 6.

/ 100 / 200Gb/s) VCSELs, Silicon Photonics and Copper MPI, SHMEM/PGAS, UPC

24 Highest-Performance 200Gb/s Interconnect Solutions 200Gb/s Adapter, 0.6us latency 200 million messages per second (10 / 25 / 40 / 50 / 56 / 100 / 200Gb/s) 40 HDR (200Gb/s) InfiniBand Ports 80 HDR100 InfiniBand Ports Throughput of 16Tb/s, <90ns Latency GbE, GbE, /50GbE Ports (10 / 25 / 40 / 50 / 100 / 200 GbE) Throughput of 6.4Tb/s Transceivers Active Optical and Copper Cables (10 / 25 / 40 / 50 / 56 / 100 / 200Gb/s) VCSELs, Silicon Photonics and Copper MPI, SHMEM/PGAS, UPC For Commercial and Open Source Applications Leverages Hardware Accelerations 2017 Mellanox Technologies 24

25 Upcoming HDR InfiniBand Increases Mellanox Leadership HDR InfiniBand the Most Scalable Switch 1.6X Better 2.8X Better 4.6X Better 80-Ports Top of Rack Switch 3200 Nodes with Only 2-Teir Switch Network 128K Nodes with Only 3-Tier Switch Network 400-Nodes 1600-Nodes 1.6X Switch Saving, 2X Cable Saving InfiniBand: 15 ToR Switches; OmniPath: 24 ToR Switches 4X Cable Saving, 2X Power Saving InfiniBand: 1 Modular Switch; OmniPath: 64 ToR + 2 Modular Switches 2017 Mellanox Technologies 25

Mellanox Dragonfly+ Accelerates Fastest Supercomputer in Canada Large Scale Innovative Dragonfly+ Topology InfiniBand Leadership Performance Advanced

26 Mellanox Dragonfly+ Accelerates Fastest Supercomputer in Canada Large Scale Innovative Dragonfly+ Topology InfiniBand Leadership Performance Advanced Adaptive Routing Highest Performance at Lower TCO In-Network Computing with SHARP and Tag Matching offloads Highest Network Utilization 2017 Mellanox Technologies 26

Scalability for X86, Power, GPU, ARM and

Interconnect to Unleash The Power of All

27 End-to-End Interconnect Solutions for All Platforms Highest Performance and Scalability for X86, Power, GPU, ARM and FPGA-based Compute and Storage Platforms 10, 20, 25, 40, 50, 56, 100 and 200Gb/s Speeds X86 POWER GPU ARM FPGA Smart Interconnect to Unleash The Power of All Compute Architectures 2017 Mellanox Technologies 27

SKA Project: 100+ Mellanox Ethernet Switches World s Largest Radio Telescope 64 x SX1012 in 64 Antenna stations connected to the data center via LR4 Optical Transceiver 40 x SX1710 forming 384 ports

sec, said Francois Kapp, sub-system manager at SKA Africa. This enormous amount of data requires high-bandwidth, low jitter networks.

28 SKA Project: 100+ Mellanox Ethernet Switches World s Largest Radio Telescope 64 x SX1012 in 64 Antenna stations connected to the data center via LR4 Optical Transceiver 40 x SX1710 forming 384 ports VMS in the data center PIM-BiDir with load sharing and OSPF The amount of data that will be generated by the MeerKAT antennas is equivalent to more than 5 million DVDs every day or about 64 DVDs per sec, said Francois Kapp, sub-system manager at SKA Africa. This enormous amount of data requires high-bandwidth, low jitter networks. After a comprehensive review of the various products in the market, we concluded that Mellanox s SwitchX-2 Ethernet switches and optical cables will provide us with the best value enterprise-class Ethernet infrastructure that meets all the requirements for our systems Mellanox Technologies 28

29 Microsoft: RDMA highly recommended Presentation given to Microsoft s partners by: Erin Chapple General Manager / Windows Server and Jeffry Soveler / Technical Fellow & Lead Architect for the Enterprise Cloud Group delivered clear message: Don't waste your money on buying servers that don t have RDMA NICs, and followed with a short clarification since all of Microsoft s new technologies will heavily rely on RDMA capable networking Mellanox Technologies 29

30 InfiniBand Delivers Best Return on Investment 30%-250% Higher Return on Investment Up to 50% Saving on Capital and Operation Expenses Highest Applications Performance, Scalability and Productivity Weather Automotive Chemistry Molecular Dynamics Genomics 1.3X Better 2X Better 1.4X Better 2.5X Better 1.3X Better 2017 Mellanox Technologies 30

31 TOP500 Mellanox Accelerated Supercomputers Nov 2017 Wuxi Supercomputing Center 93 Petaflop World s Fastest Supercomputer 41K nodes Japan Agency for Marine-Earth Science and Technology 19 Petaflop Japan s Fastest Supercomputer NASA Ames Research Center 6 Petaflop Enabling Space Exploration 20K nodes 2017 Mellanox Technologies 31

Production Petaflop National Center for Atmospheric

8 Petaflop Enabling Weather Research Exploration &

32 TOP500 Mellanox Accelerated Supercomputers Nov Enabling Energy Explorations Total Exploration Production Petaflop National Center for Atmospheric Research 4.8 Petaflop Enabling Weather Research Exploration & Production ENI S.p.A. 3.2 Petaflop Enabling Energy Explorations 2017 Mellanox Technologies 32

InfiniBand The Smart Choice for HPC Platforms and Applications We chose a co-design

The only interconnect that really could deliver that was Mellanox s InfiniBand.

One of the big reasons we use InfiniBand and not an alternative is that we ve got

Watch Video Watch Video Watch Video InfiniBand is the most advanced interconnect

33 InfiniBand The Smart Choice for HPC Platforms and Applications We chose a co-design approach supporting in the best possible manner our key applications. The only interconnect that really could deliver that was Mellanox s InfiniBand. In HPC, the processor should be going 100% of the time on a science question, not on a communications question. This is why the offload capability of Mellanox s network is critical. One of the big reasons we use InfiniBand and not an alternative is that we ve got backwards compatibility with our existing solutions. Watch Video Watch Video Watch Video InfiniBand is the most advanced interconnect technology in the world, with dramatic communication overhead reduction that fully unleashes cluster performance. We have users that move tens of terabytes of data and this needs to happen very, very rapidly. InfiniBand is the way to do it. InfiniBand is the best that is required for our applications. It enhances and unlocks the potential of the system. Watch Video Watch Video Watch Video 2017 Mellanox Technologies 33

34 Thank You

Interconnect Your Future Enabling the Best Datacenter Return on Investment. TOP500 Supercomputers, November 2017

Interconnect Your Future Enabling the Best Datacenter Return on Investment TOP500 Supercomputers, November 2017 InfiniBand Accelerates Majority of New Systems on TOP500 InfiniBand connects 77% of new HPC