IBM Power Systems. Artificial Intelligence mit IBM Power 9 und Power AI / AI Vision. Ulrich Walter

Transcription

1 IBM Power Systems Artificial Intelligence mit IBM Power 9 und Power AI / AI Vision Ulrich Walter

2 A New Era of Computing has Emerged Centralized Computing Personal Computer Data Distributed Computing Client/ Server Reporting E-Business Office Productivity Smarter Planet Data Warehousing Context Cognitive Era Big Data & Predictive Analytics Insight Cognitive Transactional Database Business Intelligence Big Data & Advanced Analytics Actionable Insight in context

3 Why Deep Learning Is Happening Now Big Data Deep Neural Networks High performance POWER CPUs, Increased bandwidth and storage capacities Powerful Accelerators 3

4 Sic Transit Gloria Mundi 2017 Google Brain NVIDIA Volta GPU ~ 0,3kW/h ~ 120 TFLOPS Servers ~ 8 MW/h ~ 50 TFLOPS 3 NVIDIA PASCAL GPUs ~ 0,9kW/h ~ 62 TFLOPS

5 Obama: My Successor Will Govern a Country Being Transformed by AI

6 From data to information Image&Video Voice&Sound Text Sensor ComInt, ELInt, SigInt From information to knowledge Decision From knowledge to action/reaction Time to analyse Time to action Time to reason & conclude

7 The four phases of DL/AI in the data Supply Chain technical view Detect and Collect Image&Video Voice&Sound Text Sensor ComInt, ELInt, SigInt Store/Analyze Compress/Map Reduce Tag/Aggregate Knowledge Base Learn Distributed Deep Learning Comparison and intrepretation Combine Conclude/Reason Inference Applied Knowledge Platforms FPGA Applications Appliances Data Data Compute Storage Bandwidth Latency Intervall Network Interface Storage Bandwidth Latency I/O Blocksize Network Interface Storage Type Infiniband Network Storage Bandwidth Latency I/O Blocksize Locality IBM Spectrum Family and other storage options Flash Disk Storage Rich Servers NVME Flash High I/O Data locality and low latency CAPI Coherent Accellerator Processor Interface for Data

8 IBM Platform for Deep Learning / Artificial Intelligence The four phases of DL/AI in the data Supply Chain Detect and Collect Image&Video Voice&Sound Text Sensor ComInt, ELInt, SigInt Store/Analyze Compress/Map Reduce Tag/Aggregate Knowledge Base Learn Distributed Deep Learning Comparison and intrepretation Combine Conclude/Reason Inference Applied Knowledge Platforms FPGA Applications Appliances APIs IBM Storage for Analytics & Deep Learning Complementing IBM AI Vision for automation and scaleout DDL and PowerAI Framework Analytic Frameworks and solutions : Filesystems Hadoop IBM Spectrum Scale BeeGFS Supporting libraries: IBM Storage For Big Data and Analytics CEPH/XFS Deep Learning Frameworks Supporting Libraries OpenBLAS Distributed Frameworks IBM Elastic Storage Server (ESS) Extreme Scalability Breakthrough performance Integrated solution IB and Etn Support IBM Power System 822LC Scalable technology Open Power design Linux only Flash, SAS SSD IB and Etn Support IBM POWER AC922 Breakthrough performance for DL/AI and HPC with native NVLINK Complementing Cloud Services

9 POWER9 The only processor specifically designed for the AI era. 4x 5x+ >2x OpenCAPI More threads for high performance cores vs x86 more I/O bandwidth than x86 more memory bandwidth P9 NVLink 2.0 PCIe Gen4

10 AC922 2 Socket, 4 GPU (SXM2) Air Cooled Node Processor Two POWER9 SCMs TDP: 190W, RDP: 250W Cores: 16, 20c Memory Maximum of 16 DDR4 IS RDIMM slots Direct attach to POWER9 module 128 to 2048GB configurations 8, 16, 32, 64GB, 128GB DIMMs Internal Storage RAID 0, 1, 0 to 2 SFF SATA HDD or SSD, default is 0 disks PCIe adapter form factor NVMe flash adapter PCIe Gen4 Slots 2 PCIe x16 Gen4 LP slot 1 PCIe x4 Gen4 LP Slot 1, Dual port IB EDR NIC, shared by two POWER9 sockets. GPU NVIDIA Volta GPU s (300W max) SXM2 form factor, NVLink GPU s per POWER9 socket Optimized GPU bandwidth, all POWER9 NVlinks to GPUs 2 GPUs base (1 per P9), feature upgrade to 4 GPUs Native I/O Host 2-port USB 3.0 (1F, 1R) Management port Enclosure Form Factor 2U 19 rack enclosure, ~710mm (28 ) deep O/S: Linux Only Ubuntu (with updates) RHEL For POWER9/CORAL (limited availability in 2017) Sapphire, Opal KVM Not required RAS 5 year MTBF Concurrent maintenance HDD (when installed) N+1 redundant hot swap cooling 2, 2200W bulk power supplies N+1 redundant, 4 GPU configuration 200VAC, 277VAC, 400VDC input voltage support Air Cooled Version Shown Energy Efficiency On-chip power management, power gating Advanced thermal management 80+ Platinum Power Supply Compliant Service Interface Industry BMC service controller w/ OpenBMC firmware Operator interface & FRU LEDs Certifications FCC: Class A for Servers Acoustics: Rack configuration not to exceed 9.5B Environment: ASHRAE A3 (5-40C, 8-85% RH, 3050m max) Offering Air cooled only

11 The NVLink difference CPU-GPU P9 with 2 nd Gen NVLink enables 5.6x faster data movement from CPU-GPU in 4 GPU system In 6 GPU system bandwidth is minimally reduced but balanced by higher compute capability Results are based on IBM Internal Measurements running the CUDA H2D Bandwidth Test Hardware: Power AC922; 32 cores (2 x 16c chips), POWER9 with NVLink 2.0; 2.25 GHz, 1024 GB memory, 4xTesla V100 GPU; Ubuntu S822LC for HPC; 20 cores (2 x 10c chips), POWER8 with NVLink; 2.86 GHz, 512 GB memory, Tesla P100 GPU Competitive HW: 2x Xeon E v4; 20 cores (2 x 10c chips) / 40 threads; Intel Xeon E v4; 2.4 GHz; 1024 GB memory, 4xTesla V100 GPU, Ubuntu 16.04

12 POWER9 Data Capacity and Throughput Big caches for massively parallel compute and heterogeneous interaction L3 Cache 120 MB shared Capacity NUCA Cache 10 MB Capacity + 512k L2 per SMT8 Core Enhanced replacement with reuse & data type awareness 12 * 20-way associativity Extreme switching bandwidth for the most demanding compute and accellerated workloads High Throughput on-chip fabric Over 7 TB/s On-chip switch Move data in/out at 256 GB/ per SMT 8 Core 12

13 POWER9 Premier Accelleration Platform State of the art I/O accelleration attachement signaling PCIe Gen 4 * 48 lanes 192 GB duplex bandwidth 25G Link * 48 lanes 300 GB/s duplex bandwidth POWER 9 Robust accelarated compute option with open standards On-Chip Accellearation Gzip x1,842 Compression x2,aes/sha x2 CAPI 2.0 4x bandwidth of POWER 8 using PCIe Gen 4 high bandwidth, low latency using 25G link NVLINK 2.0 Next generation of CPU/GPU bandwidth 13

14 NVLink Evolution in IBM Power AC922 5x Faster Data Communication x86 IBM POWER9 bandwidth optimized NVIDIA GPU V100GPU with NVLink Graphics Memory Graphics Memory 150GB/s Graphics Memory Fast Transfer via NVLink System Memory GB/s PCIe x16 1 TB System Memory 170 GB/s Power9 Chip Store Large Models with NVLink 2.0 in System Memory IBM POWER9 scalability optimized GB/s Graphics Memory GB/s GB/s Graphics Memory System Memory GB/s POWER9

15 Train larger, more complex models Traditional Model Support Limited memory on GPU forces trade-off in model size / data resolution Large Model Support Use system memory and GPU to support more complex and higher resolution data DDR4 CPU DDR4 POWER CPU NVLink PCIe Graphics Memory GPU Graphics Memory GPU Leveraging NVLink and coherence enables larger and more complex models Improves model accuracy with more images and higher resolution images

16 Time (secs) Large AI Models Train ~4 Times Faster Caffe with LMS (Large Model Support) Runtime of 1000 Iterations POWER9 Servers with NVLink to GPUs vs x86 Servers with PCIe to GPUs Hours 3.8x Faster Mins Xeon x v4 w/ 4x V100 GPUs Power AC922 w/ 4x V100 GPUs GoogleNet model on Enlarged ImageNet Dataset (2240x2240)

17 Speedup Deep learning training takes days to weeks Distributed Deep Learning (DDL) 16 Days Down to 7 Hours 58x Faster Near Ideal Scaling to 256 GPUs Limited scaling to multiple x86 servers 16 Days Ideal Scaling DDL Actual Scaling 95%Scaling with 256 GPUS 16 PowerAI with DDL enables scaling to 100s of servers 1 System 7 Hours 64 Systems Number of GPUs ResNet-101, ImageNet-22K ResNet-50, ImageNet-1K 17 Caffe with PowerAI DDL, Running on Minsky (S822Lc) Power System

18 IBM PowerAI Vision Productivity End-to-end capability PowerAI Vision Accuracy Cost efficiency Enterprise scale 18

19 AI Infrastructure Stack Applications Segment Specific: Finance, Retail, Healthcare Cognitive APIs (Eg: Watson) In-House APIs Speech, Vision, NLP, Sentiment Machine & Deep Learning Libraries & Frameworks Distributed Computing TensorFlow, Caffe, SparkML Spark, MPI PowerAI Transform & Prep Data (ETL) Data Lake & Data Stores Hadoop HDFS, NoSQL DBs Accelerated Servers Storage Accelerated Infrastructure

20 Auto-ML for Images & Video PowerAI Vision Label Train Deploy PowerAI PowerAI Enterprise PowerAI: Open Source ML Frameworks Large Model Support (LMS) Distributed Deep Learning (DDL) Auto ML IBM Spectrum Conductor with Spark Cluster Virtualization, Auto Hyper-Parameter Optimization Deep Learning Impact (DLI) Module Data & Model Management, ETL, Visualize, Advise Accelerated Infrastructure Accelerated Servers Storage 20

21 Auto Hyper-Parameter Tuning Hyper-parameters Learning rate Decay rate Batch size Optimizer o GradientDecedent, Adadelta, Momentum, RMSProp Momentum (for some optimizers) LSTM hidden unit size Random Tree-based Parzen Estimator (TPE) Spark search jobs are generated dynamically and executed in parallel Bayesian Multi-tenant Spark Cluster IBM Spectrum Conductor with Spark 21

22 Snap ML Distributed GPU-Accelerated Machine Learning Library Snap Machine Learning (ML) Library Logistic Regression Linear Regression APIs for Popular ML Frameworks Support Vector Machines (SVM) More Coming Soon libglm (C++ / CUDA Optimized Primitive Lib) Distributed Hyper- Parameter Optimization Distributed Training 22

23 DL-Insight: Monitoring and Optimization tool for Deep Learning PowerAI Vision : The Deep Learning Development Platform for image/video analysis Vision Recognition Service Layer Image Labeling and Preprocessing Video Labeling Service Custom Learning for Image Classification Custom Learning for Object Detection Self-defined Training with graphic/visual monitoring Inference API deployment Service Management Layer Image preprocessing management Data label management Data set management Training task management Model management Inference API management Data preprocessing component ML/DL Computation Layer Model Training Component (Caffe, and others) Inference Component (Caffe, and others) Resource management layer (CPU/GPU/FPGA) (Docker, Kubernetes) Docker, KVM (POWER) Accelerator (GPU/FPGA) Data Store (distributed FS and object store) Network

24 PowerAI Vision enables enterprise level DNN easier PowerAI Vision automates the deep learning development cycles for developers. Deep knowledges of ML/DL and computer vision have been embedded into PowerAI Vision. On-line training by looping the inference and training capability together User could use the deployed API for visual recognition Steps automatically done by PowerAI Vision Start Inference Define training task Prepare training Data DNN Model selection DL training framework preparation Data Preprocessing Configure the training hyperparameter DNN Model Training Package the new DNN model together with preprocessing into inference proc., and deploy API User defined categories Data set management Format transformation Support both training and evaluation sets Support different preprocessing plugin Provide base models for different scenarios Predict training time Training process visualization Training with GPU Scalability and HA deployment are supported

25 Optimizing the development of AI with IBM AI Vision Define Training Task Prepare Data Data Processing DNN Model Selection DL Framework Preparation Configure training parameter DNN model training Typical Challenges in AI projects Time consuming, expensive and questionable outcome No experience on DNN design and development No experience on computer vision No experience on how to build a platform to support enterprise scale deep learning, including data preparation, training, and inference Package the new DNN model together with preprocessing into inference proc. Application API Define Training Task Prepare Data Data Processing Automation done by IBM AI Vision DNN Model Selection DL Framework Preparation Configure training parameter DNN model training Package the new DNN model together with preprocessing into inference proc. Application API AI Vision automates the deep learning development cycles for developers. Deep knowledges of ML/DL and computer vision have been embedded into AI Vision. Reduces time, cost and complexity for AI integration

26 Features : Data Labeling Labeling for classification : Allow user to upload image data and label the categories Labeling for object detection: Allow user to upload image data and define bounding box Support the data upload for different purposes and different format All the labelled data could be exported as zip file and download 26

27 Feature: Data Labeling / Video Data Platform Support multiple level management (Figure 1) Labeling task Video stream Label data Figure 1 Figure 2 Figure 3 27 Support manual labeling and auto-labeling (figure 2) Manual capture and periodic capture Add user defined new attributes Provide tag management and statistics (figure 3)

28 Feature : Data preprocessing Data preprocessing is very important for some domain specific learning. Supported multiple types of data preprocessing for image data. Faces: Cars: Pedestrians: Data preprocessing can be extended by customized algorithms. 28

29 Feature: Training Training task management Custom learning for classification and object detection Transfer learning is used to get high accuracy and efficiency Support different training strategy, precise first, accuracy first, and customized configuration. Visualization for training progress Provide the time estimation for the new training task

30 Feature : Inference for different deployment one click to package the model and preprocessing into a new docker instance and launch it in container cloud Fig.1 REST API will be automatically exposed for each model and managed. Fig.2 Provide easy test for the new API Heat-map is provided. Support negative for classification Fig.3 Classification Fig.4 object detection Support the inference deployment on different architecture and location CPU, GPU, and FPGA Data center, branch, and edge 30

31 PowerAI Inference Engine (AccDNN): Automatically generate deep learning accelerator Automatically enable deep learning from cloud to edge Enhance productivity PowerAI Inference Engine tool Trained Caffe CNN model in data center FPGA Accelerator bit-file for edge translation synthesis download Net Model File Verilog File FPGA Bit File FPGA Execution name: "dummy-net" layers { name: "data" } layers { name: "conv" } layers { name: "pool" } more layers layers { name: "loss" } --input module--- conv conv_instance( ) pool pool_instance( ) more layers loss loss_instance( ) --output module--- Net.bit FPGA chip range from $20 to $1K

32 The value of AI in connected data islands for a hyperconnected and cognitive universe Wearables & mobility Infotainment, industrial & military health and fitness Security, defence, protection of cyber crime Health & research Weather, climate research & Agriculture car2x, autonomous vehicles and intelligent traffic systems Connected Home Industry 4.0 Retail and Marketing Banking, finance & insurance Energy, utilities and Smart cities

33 33

34 Legal Notices Copyright 2016 by International Business Machines Corporation. All rights reserved. No part of this document may be reproduced or transmitted in any form without written permission from IBM Corporation. Product data has been reviewed for accuracy as of the date of initial publication. Product data is subject to change without notice. This document could include technical inaccuracies or typographical errors. IBM may make improvements and/or changes in the product(s) and/or program(s) described herein at any time without notice. Any statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. References in this document to IBM products, programs, or services does not imply that IBM intends to make such products, programs or services available in all countries in which IBM operates or does business. Any reference to an IBM Program Product in this document is not intended to state or imply that only that program product may be used. Any functionally equivalent program, that does not infringe IBM's intellectually property rights, may be used instead. THE INFORMATION PROVIDED IN THIS DOCUMENT IS DISTRIBUTED "AS IS" WITHOUT ANY WARRANTY, EITHER OR IMPLIED. IBM LY DISCLAIMS ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NONINFRINGEMENT. IBM shall have no responsibility to update this information. IBM products are warranted, if at all, according to the terms and conditions of the agreements (e.g., IBM Customer Agreement, Statement of Limited Warranty, International Program License Agreement, etc.) under which they are provided. Information concerning non-ibm products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products in connection with this publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-ibm products. IBM makes no representations or warranties, ed or implied, regarding non-ibm products and services. The provision of the information contained herein is not intended to, and does not, grant any right or license under any IBM patents or copyrights. Inquiries regarding patent or copyright licenses should be made, in writing, to: IBM Director of Licensing IBM Corporation North Castle Drive Armonk, NY U.S.A. 34

35 Legal Notices IBM, the IBM logo, ibm.com, IBM System Storage, IBM Spectrum Storage, IBM Spectrum Control, IBM Spectrum Protect, IBM Spectrum Archive, IBM Spectrum Virtualize, IBM Spectrum Scale, IBM Spectrum Accelerate, Softlayer, and XIV are trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at The following are trademarks or registered trademarks of other companies. Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries. IT Infrastructure Library is a Registered Trade Mark of AXELOS Limited. Linear Tape-Open, LTO, the LTO Logo, Ultrium, and the Ultrium logo are trademarks of HP, IBM Corp. and Quantum in the U.S. and other countries. Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. Java and all Java-based trademarks and logos are trademarks or registered trademarks of Oracle and/or its affiliates. Cell Broadband Engine is a trademark of Sony Computer Entertainment, Inc. in the United States, other countries, or both and is used under license therefrom. ITIL is a Registered Trade Mark of AXELOS Limited. UNIX is a registered trademark of The Open Group in the United States and other countries. * All other products may be trademarks or registered trademarks of their respective companies. Notes: Performance is in Internal Throughput Rate (ITR) ratio based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput improvements equivalent to the performance ratios stated here. All customer examples cited or described in this presentation are presented as illustrations of the manner in which some customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics will vary depending on individual customer configurations and conditions. This publication was produced in the United States. IBM may not offer the products, services or features discussed in this document in other countries, and the information may be subject to change without notice. Consult your local IBM business contact for information on the product or services available in your area. All statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. Information about non-ibm products is obtained from the manufacturers of those products or their published announcements. IBM has not tested those products and cannot confirm the performance, compatibility, or any other claims related to non-ibm products. Questions on the capabilities of non-ibm products should be addressed to the suppliers of those products. Prices subject to change without notice. Contact your IBM representative or Business Partner for the most current pricing in your geography. 35 This presentation and the claims outlined in it were reviewed for compliance with US law. Adaptations of these claims for use in other geographies must be reviewed by the local country counsel for compliance with local laws.