IBM Technology and Solutions for Artificial Intelligence and HPC

Transcription

1 IBM Technology and Solutions for Artificial Intelligence and HPC AI System Architecture IBM Power9 and Beyond Ulrich Walter Supercomputers Built for AI

2 The race is on

3 Use cases for AI Automation Prediction Security Discovery Process Automation Facial Recognition Fraud detection Social Engeneering and profiling Autonomous Driving Pattern Recognition Robotics Compliance Check

4 AI DEMANDS A DIFFERENT TYPE OF SYSTEM IBM Power Systems provides the cutting-edge advances in AI that data scientists demand, and the critical reliability that IT needs. IBM has reimagined infrastructure for the journey to AI

5 And so it began Mov SI,NumBytes Mov CX,[SI Les DI,SegAdr1 Lds SI,SegAdr2 Mov AX,0 Jne Dec AX RAM A1 A2 B1 B2 A3 A4 B3 B4 A5 A6 B5 B6 Von Neumann Bottleneck

6 New workloads require different architectures TPU2 = Tensor Processing Unit Accelerates neural network computations IBM TrueNorth Neuromorphic CMOS integrated circuit Quantum Computing

7 New Kids on the Block GPU powered AI, Analytics, Databases and Storage Languages Storage Analytics Tool, Frameworks and Libraries GPU optimized databases AI Frameworks

8 The OpenPower Ecosystem of Innovators Creates True Differentiation in Performance and Cost S822LC For High Performance Computing S822LC For Big Data S821LC S812LC S822LC IBM Power Systems LC Portfolio With IBM warranty and enterprise support structure Member-contributed Innovation Ecosystem-Driven Customer Choice IBM Systems 8

9 From data to information The five phases of AI/DL - From data collection to action - From information to knowledge From knowledge to action/reaction Image&Video Voice&Sound Text Sensor Time to Collect and filter Time to Analyse and prepare Time to learn Time to reason & conclude Big Data, PB Large Data, TB Large Data, TB Small Data, GB, MB

10 Challenges on storage for analytics and AI/DL data Attributes and Considerations Data Phases Collect Volume Velocity xb/s IO/s TB PB EB Inference Prepare Veracity TRUST Learn Analyse Variability Value Intervall IBM Systems

11 The five phases of DL/AI in the data Supply Chain technical view Detect and Collect Image&Video Voice&Sound ComInt, ELInt, SigInt Data Creation Device type Latency Volume Velocity Intervall Interface Text Sensor Storage intensive Data source Data model Data Type API Originator Veracity Data preparation Storage Bandwidth Latency Capacity I/O Blocksize Filesystem Storage Type Data preprocessing Database Toolchain Analytics Frameworks Languages Data selection Analyse Compress/Map Reduce Tag/Aggregate Transformation and validation Verified data Volume Storage Type Filesystem IBM Power 9 AC922 Database Toolchain Languages Value Learn Distributed Deep Learning Comparison and intrepretation Combine Conclude/Reason Control, loopback and update Knowledge Base Continuos Learning Compute intensive Bandwidth Storage-RAM-CPU-GPU Compute Performance Latency I/O throughput Data Locality Memory size Compute (Training & Inference) Inference Applied Knowledge Toolchain CNN Languages Database Platforms FPGA Applications Appliances IBM Spectrum Family and other storage options Flash + NVMe Disk Object Store Storage Rich Servers Tape & LTFS Shared PCIe-4 For IB P9 P9 NVME Flash High I/O Storage Data locality and low latency CAPI Coherent Accellerator Processor Interface for Data CPU-GPU NVLINK GB IBM Systems

12 IBM AI Reference Architecture from Experimentation to Expansion Experimentation Single Tenant Stabilization & Production Secure Multitenant Expansion Enterprise Scale / Multiple Lines of Business Data Scientist s workstations Existing Organization Infrastructure High-Speed Network Subsystem Master & Failover Master Nodes IBM Power Systems LC921 & LC922 Existing Organization Infrastructure High-Speed Network Subsystem Login Nodes IBM Power Systems LC921 & LC922 IBM Power Systems AC922 Internal SAS drives & NVM s Training Cluster IBM Power Systems AC922 IBM Elastic Storage Server (ESS) Training & Inference Cluster IBM Power Systems AC922, LC921 & LC922 IBM Elastic Storage Server (ESS) Services & Support IBM PowerAI Enterprise Red Hat Enterprise Linux (RHEL) IBM Power System & x86 Servers IBM Spectrum Scale / IBM Elastic Storage Server (ESS) One software stack from experimentation to expansion Copyright IBM Corporation

13 Performance = important, but bandwidth matters IBM Systems

14 IBM AC922 An Acceleration Superhighway Designed for the AI Era Delivering Enterprise-Class AI The Best Server for Enterprise AI IBM Systems

15 15 POWER9 Acceleration Server (AC922) 2U server - 19 Rack enclosure 2 POWER9 Scale Out SMT4 Processors (Up to 22-cores per socket) 4 or 6 NVidia Tesla GPU processors (NVLink 2.0 attached) SXM2 Slot Form Factor 300W V100 GPU 16/32 GB NVRAM Up to 1TB Total DDR4 Industry Standard memory RDIMMs 16 IS RDIMM slots 341 GB/s total system memory peak bandwidth 4 PCIe Gen4 Low Profile slots 2 SFF (2.5 ) SATA bays (HDD or SSD) 2 Ethernet 1Gb ports Air and Water cooled versions Linux (RHEL) only platform with BMC service processor 2016 IBM Corporation 15

16 Resolve the PCI-E bottleneck for your code with POWER9 and NVLink 2.0: Transfer data 5.6X faster than the CUDA Host-Device Bandwidth of tested x86 platforms THE Platform for developers requiring CPU-GPU Bandwidth POWER9 is the only processor with NVLink 2.0 from CPU to GPU Delivering 5.6X Host-Device bandwidth vs Xeon E v4 based systems with CUDA H2D Bandwidth Test No code changes are required to leverage NVLink capability Application performance could be further increased with application code optimization Results are based on IBM Internal Measurements running the CUDA H2D/D2H Bandwidth Test Hardware: Power AC922; 3240 cores (2 x 20c chips), POWER9 with NVLink 2.0; 2.25 GHz, 1024 GB memory, 4xTesla V100 GPU; RHEL 7.4 for Power LE (POWER9).; S822LC for HPC; 20 cores (2 x 10c chips), POWER8 with NVLink 1.0; 2.86 GHz, 1024 GB memory, Tesla P100 GPU, RHEL 7.3 Competitive HW: 2x Xeon E v4; 20 cores (2 x 10c chips) / 40 threads; Intel Xeon E v4; 2.4 GHz; 512 GB memory, 4xTesla P100 GPU, Ubuntu 16.04

17 NVLink Evolution in POWER HPC P100 GPU with NVLink GB/s NVLink between CPUs and GPUs enables fast memory access to large data sets in system memory Two NVLink connections between each GPU and CPU-GPU leads to faster data exchange Time saving +300% vs x86 Add. CAPI feature for fast IO to storage and network Graphics Memory IBM POWER9 bandwidth optimized V100GPU with NVLink GB/s Graphics Memory +87% Bandwidth Vs NVLINK % vs x86 PCIe3 + Tensor Cores + V100 Technology System Memory Graphics Memory Graphics Memory System Memory Power8 Chip with NVLink x86 NVIDIA GPU IBM POWER GB/s NVLink only between GPUs Long lasting ramp-up times due to PCIe Bottleneck Reduced efficiency PCIe x16 Graphics Memory Graphics Memory System Memory System Memory GB/s GB/s GB/s Power9 Chip with NVLink 2.0 IBM POWER9 scalability optimized GB/s POWER9 +20% Bandwidth + Tensor Cores + V100 Graphics Memory Technology + 3 GPUs/CPU + 33% more GPU RAM + watercooled IBM Systems

18 Time (secs) The NVLINK 2.0 effect Caffe with LMS (Large Model Support) Runtime of 1000 Iterations Train Large AI Models Hours 3.8x Faster ~4 Times Faster POWER9 Servers with NVLink to GPUs vs x86 Servers with PCIe to GPUs Mins 0 Xeon x v4 w/ 4x V100 GPUs Power AC922 w/ 4x V100 GPUs GoogleNet model on Enlarged ImageNet Dataset (2240x2240)

19 2016 IBM Corporation 19

20 Industry Usage of Storage Class Memory Memory CPU Switch Memory Industry initiatives put SCM where it s easy Easier from hardware development perspective Very limiting for end users Allows interfaces to be used as control point SCM accessed via NVMe: Latency advantage reduced by software stack Bandwidth limited by PCIe infrastructure IOPs limited by CPUs consumed to run NVMe stack NVMe SCM accessed over DDR memory buses: Load/Store access model improves over NVMe Eliminates CPU cycles spent on NVMe stack Reduces latency by removing SW pathlength Creates DRAM vs SCM configuration tradeoffs Capacity and bandwidth spread across 2 tiers Both tiers are sub-optimized 2016 IBM Corporation 20

21 Auto Hyper-Parameter Tuning Hyper-parameters Learning rate Decay rate Batch size Optimizer Gradient Decedent, Adadelta, Momentum, RMSProp Momentum (for some optimizers) LSTM hidden unit size Random Tree-based Parzen Estimator (TPE) Spark search jobs are generated dynamically and executed in parallel Bayesian Multi-tenant Spark Cluster IBM Spectrum Conductor with Spark

22 Snap ML Distributed GPU-Accelerated Machine Learning Library Snap Machine Learning (ML) Library Logistic Regression Linear Regression APIs for Popular ML Frameworks Support Vector Machines (SVM) More Coming Soon libglm (C++ / CUDA Optimized Primitive Lib) Distributed Hyper- Parameter Optimization (coming soon) Distributed Training 2

23 Resources Power AI Developer Portal OpenPower on Youtube TSRG Web form Link For IBMers: TSRG Web form Link for Business Partners Partners: Power AI Vision IBM Systems

24 Legal Notices Copyright 2016 by International Business Machines Corporation. All rights reserved. No part of this document may be reproduced or transmitted in any form without written permission from IBM Corporation. Product data has been reviewed for accuracy as of the date of initial publication. Product data is subject to change without notice. This document could include technical inaccuracies or typographical errors. IBM may make improvements and/or changes in the product(s) and/or program(s) described herein at any time without notice. Any statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. References in this document to IBM products, programs, or services does not imply that IBM intends to make such products, programs or services available in all countries in which IBM operates or does business. Any reference to an IBM Program Product in this document is not intended to state or imply that only that program product may be used. Any functionally equivalent program, that does not infringe IBM's intellectually property rights, may be used instead. THE INFORMATION PROVIDED IN THIS DOCUMENT IS DISTRIBUTED "AS IS" WITHOUT ANY WARRANTY, EITHER OR IMPLIED. IBM LY DISCLAIMS ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NONINFRINGEMENT. IBM shall have no responsibility to update this information. IBM products are warranted, if at all, according to the terms and conditions of the agreements (e.g., IBM Customer Agreement, Statement of Limited Warranty, International Program License Agreement, etc.) under which they are provided. Information concerning non-ibm products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products in connection with this publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-ibm products. IBM makes no representations or warranties, ed or implied, regarding non-ibm products and services. The provision of the information contained herein is not intended to, and does not, grant any right or license under any IBM patents or copyrights. Inquiries regarding patent or copyright licenses should be made, in writing, to: IBM Director of Licensing IBM Corporation North Castle Drive Armonk, NY U.S.A. IBM Systems 2

25 25 Legal Notices IBM, the IBM logo, ibm.com, IBM System Storage, IBM Spectrum Storage, IBM Spectrum Control, IBM Spectrum Protect, IBM Spectrum Archive, IBM Spectrum Virtualize, IBM Spectrum Scale, IBM Spectrum Accelerate, Softlayer, and XIV are trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at The following are trademarks or registered trademarks of other companies. Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries. IT Infrastructure Library is a Registered Trade Mark of AXELOS Limited. Linear Tape-Open, LTO, the LTO Logo, Ultrium, and the Ultrium logo are trademarks of HP, IBM Corp. and Quantum in the U.S. and other countries. Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. Java and all Java-based trademarks and logos are trademarks or registered trademarks of Oracle and/or its affiliates. Cell Broadband Engine is a trademark of Sony Computer Entertainment, Inc. in the United States, other countries, or both and is used under license therefrom. ITIL is a Registered Trade Mark of AXELOS Limited. UNIX is a registered trademark of The Open Group in the United States and other countries. * All other products may be trademarks or registered trademarks of their respective companies. Notes: Performance is in Internal Throughput Rate (ITR) ratio based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput improvements equivalent to the performance ratios stated here. All customer examples cited or described in this presentation are presented as illustrations of the manner in which some customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics will vary depending on individual customer configurations and conditions. This publication was produced in the United States. IBM may not offer the products, services or features discussed in this document in other countries, and the information may be subject to change without notice. Consult your local IBM business contact for information on the product or services available in your area. All statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. Information about non-ibm products is obtained from the manufacturers of those products or their published announcements. IBM has not tested those products and cannot confirm the performance, compatibility, or any other claims related to non-ibm products. Questions on the capabilities of non-ibm products should be addressed to the suppliers of those products. Prices subject to change without notice. Contact your IBM representative or Business Partner for the most current pricing in your geography IBM Corporation 25 This presentation and the claims outlined in it were reviewed for compliance with US law. Adaptations of these claims for use in other geographies must be reviewed by the local country counsel for compliance with local laws.