TVM Stack Overview. Tianqi Chen
|
|
|
- Silvia Owens
- 5 years ago
- Views:
Transcription
1 TVM Stack Overview Tianqi Chen
2 Beginning of TVM Story
3 Acclerator Beginning of TVM Story
4 Acclerator Beginning of TVM Story
5 Beginning of TVM Story Acclerator // Pseudo-code for convolution program for the VIA accelerator // Virtual Thread 0 0x00: LOAD(PARAM[ 0-71]) // LD@TID0 0x01: LOAD(ACTIV[ 0-24]) // LD@TID0 0x02: LOAD(LDBUF[ 0-31]) // LD@TID0 0x03: PUSH(LD->EX) // LD@TID0 0x04: POP (LD->EX) // EX@TID0 0x05: EXE (ACTIV[ 0-24],PARAM[ 0-71],LDBUF[ 0-31],STBUF[ 0-7]) // EX@TID0 0x06: PUSH(EX->LD) // EX@TID0 0x07: PUSH(EX->ST) // EX@TID0 0x08: POP (EX->ST) // ST@TID0 0x09: STOR(STBUF[ 0-7]) // ST@TID0 0x0A: PUSH(ST->EX) // ST@TID0 // Virtual Thread 1 0x0B: LOAD(ACTIV[25-50]) // LD@TID1 0x0C: LOAD(LDBUF[32-63]) // LD@TID1 0x0D: PUSH(LD->EX) // LD@TID1 0x0E: POP (LD->EX) // EX@TID1 0x0F: EXE (ACTIV[25-50],PARAM[ 0-71],LDBUF[32-63],STBUF[32-39]) // EX@TID1 0x10: PUSH(EX->LD) // EX@TID1 0x11: PUSH(EX->ST) // EX@TID1 0x12: POP (EX->ST) // ST@TID1 0x13: STOR(STBUF[32-39]) // ST@TID1 0x14: PUSH(ST->EX) // ST@TID1 // Virtual Thread 2 0x15: POP (EX->LD) // LD@TID2 0x16: LOAD(PARAM[ 0-71]) // LD@TID2 0x17: LOAD(ACTIV[ 0-24]) // LD@TID2 0x18: LOAD(LDBUF[ 0-31]) // LD@TID2 0x19: PUSH(LD->EX) // LD@TID2 0x1A: POP (LD->EX) // EX@TID2 0x1B: POP (ST->EX) // EX@TID2 0x1C: EXE (ACTIV[ 0-24],PARAM[ 0-71],LDBUF[ 0-31],STBUF[ 0-7]) // EX@TID2 0x1D: PUSH(EX->ST) // EX@TID2 0x1E: POP (EX->ST) // ST@TID2 0x1F: STOR(STBUF[ 0-7]) // ST@TID2 // Virtual Thread 3 0x20: POP (EX->LD) // LD@TID3 0x21: LOAD(ACTIV[25-50]) // LD@TID3 0x22: LOAD(LDBUF[32-63]) // LD@TID3 0x23: PUSH(LD->EX) // LD@TID3 0x24: POP (LD->EX) // EX@TID3 0x25: POP (ST->EX) // EX@TID2 0x26: EXE (ACTIV[25-50],PARAM[ 0-71],LDBUF[32-63],STBUF[32-39]) // EX@TID3 0x27: PUSH(EX->ST) // EX@TID3 0x28: POP (EX->ST) // ST@TID3 0x29: STOR(STBUF[32-39]) // ST@TID3 (a) Blocked convolution program with multiple thread contexts // Convolution access pattern dictated by micro-coded program. // Each register index is derived as a 2-D affine function. // e.g. idx rf = a rf y+b rf x+c rf 0, where c rf 0 is specified by // micro op 0 fields. for y in [0 i) for x in [0 j) rf[idx rf 0 ] += GEVM(act[idx act 0 ], par[idx par 0 ]) rf[idx rf 1 ] += GEVM(act[idx act 1 ], par[idx par 1 ]) rf[idx rf n ] += GEVM(act[idx act n ], par[idx par n ]) (b) Convolution micro-coded program // Max-pool, batch normalization and activation function // access pattern dictated by micro-coded program. // Each register index is derived as a 2D affine function. // e.g. idx dst = a dst y+b dst x+c dst 0, where c dst 0 is specified by // micro op 0 fields. for y in [0 i) for x in [0 j) // max pooling rf[idx dst 0 ] = MAX(rf[idx dst 0 ], rf[idx src 0 ]) rf[idx dst 1 ] = MAX(rf[idx dst 1 ], rf[idx src 1 ]) // batch norm rf[idx dst m ] = MUL(rf[idx dst m ], rf[idx src m ]) rf[idx dst m+1 ] = ADD(rf[idx dst m+1 ], rf[idx src m+1 ]) rf[idx dst m+2 ] = MUL(rf[idx dst m+2 ], rf[idx src m+2 ]) rf[idx dst m+3 ] = ADD(rf[idx dst m+3 ], rf[idx src m+3 ]) // activation rf[idx dst n-1 ] = RELU(rf[idx dst n-1 ], rf[idx src n-1 ]) rf[idx dst n ] = RELU(rf[idx dst n ], rf[idx src n ]) (c) Max pool, batch norm and activation micro-coded program
6 Existing Approach Frameworks Hardware
7 Existing Approach Frameworks High-level data flow graph Hardware
8 Existing Approach Frameworks High-level data flow graph Primitive Tensor operators such as Conv2D Hardware
9 Existing Approach Frameworks High-level data flow graph Primitive Tensor operators such as Conv2D eg. cudnn Offload to heavily optimized DNN operator library Hardware
10 Existing Approach: Engineer Optimized Tensor Operators!4
11 Existing Approach: Engineer Optimized Tensor Operators!4
12 Existing Approach: Engineer Optimized Tensor Operators Matmul: Operator Specification C = tvm.compute((m, n), lambda y, x: tvm.sum(a[k, y] * B[k, x], axis=k))!4
13 Existing Approach: Engineer Optimized Tensor Operators Matmul: Operator Specification C = tvm.compute((m, n), lambda y, x: tvm.sum(a[k, y] * B[k, x], axis=k))!4
14 Existing Approach: Engineer Optimized Tensor Operators Matmul: Operator Specification C = tvm.compute((m, n), lambda y, x: tvm.sum(a[k, y] * B[k, x], axis=k)) Vanilla Code for y in range(1024): for x in range(1024): C[y][x] = 0 for k in range(1024): C[y][x] += A[k][y] * B[k][x]!4
15 Existing Approach: Engineer Optimized Tensor Operators Matmul: Operator Specification C = tvm.compute((m, n), lambda y, x: tvm.sum(a[k, y] * B[k, x], axis=k)) Loop Tiling for Locality for yo in range(128): for xo in range(128): C[yo*8:yo*8+8][xo*8:xo*8+8] = 0 for ko in range(128): for yi in range(8): for xi in range(8): for ki in range(8): C[yo*8+yi][xo*8+xi] += A[ko*8+ki][yo*8+yi] * B[ko*8+ki][xo*8+xi]!5
16 Existing Approach: Engineer Optimized Tensor Operators Matmul: Operator Specification C = tvm.compute((m, n), lambda y, x: tvm.sum(a[k, y] * B[k, x], axis=k)) Map to Accelerators inp_buffer AL[8][8], BL[8][8] acc_buffer CL[8][8] for yo in range(128): for xo in range(128): vdla.fill_zero(cl) for ko in range(128): vdla.dma_copy2d(al, A[ko*8:ko*8+8][yo*8:yo*8+8]) vdla.dma_copy2d(bl, B[ko*8:ko*8+8][xo*8:xo*8+8]) vdla.fused_gemm8x8_add(cl, AL, BL) vdla.dma_copy2d(c[yo*8:yo*8+8,xo*8:xo*8+8], CL) Human exploration of optimized code!6
17 Limitations of Existing Approach Frameworks cudnn
18 Limitations of Existing Approach Frameworks cudnn
19 Limitations of Existing Approach Frameworks cudnn
20 Limitations of Existing Approach Frameworks cudnn
21 Limitations of Existing Approach Frameworks cudnn
22 Limitations of Existing Approach Frameworks New operator introduced by operator fusion optimization potentially benefit: 1.5x speedup cudnn
23 Limitations of Existing Approach Frameworks New operator introduced by operator fusion optimization potentially benefit: 1.5x speedup cudnn
24 Limitations of Existing Approach Frameworks New operator introduced by operator fusion optimization potentially benefit: 1.5x speedup cudnn
25 Limitations of Existing Approach Frameworks New operator introduced by operator fusion optimization potentially benefit: 1.5x speedup cudnn Engineering intensive
26 Learning-based Learning System Frameworks High-level data flow graph and optimizations Hardware
27 Learning-based Learning System Frameworks High-level data flow graph and optimizations Hardware
28 Learning-based Learning System Frameworks High-level data flow graph and optimizations Hardware aware Search Space of Optimized Tensor Programs Hardware
29 Learning-based Learning System Frameworks High-level data flow graph and optimizations Hardware aware Search Space of Optimized Tensor Programs Machine Learning based Program Optimizer Hardware
30 Learning-based Learning System Frameworks High-level data flow graph and optimizations Hardware aware Search Space of Optimized Tensor Programs Machine Learning based Program Optimizer directly generate optimized program for new operator workloads and hardware Hardware
31 Learning-based Learning System Frameworks High-level data flow graph and optimizations Hardware aware Search Space of Optimized Tensor Programs Machine Learning based Program Optimizer Hardware
32 Learning-based Learning System Frameworks High-level data flow graph and optimizations Hardware aware Search Space of Optimized Tensor Programs Machine Learning based Program Optimizer Hardware
33 Hardware-aware Search Space Tensor Expression Language (Specification) C = tvm.compute((m, n), lambda y, x: tvm.sum(a[k, y] * B[k, x], axis=k)) Hardware
34 Hardware-aware Search Space Tensor Expression Language (Specification) C = tvm.compute((m, n), lambda y, x: tvm.sum(a[k, y] * B[k, x], axis=k)) Define search space of hardware aware mappings from expression to hardware program Based on Halide s compute/schedule separation Hardware
35 Hardware-aware Search Space CPUs Compute Primitives Memory Subsystem L3 scalar vector L2 L2 L1D L1I L1D L1I implicitly managed Loop Transformations Cache Locality Vectorization Reuse primitives from prior work: Halide, Loopy
36 Challenge to Support Diverse Hardware Backends CPUs GPUs TPU-like specialized Accelerators
37 Hardware-aware Search Space GPUs Compute Primitives Memory Subsystem scalar vector L2 SM SM TX/L1 TX/L1 RF RF RF RF mixed
38 Hardware-aware Search Space GPUs Compute Primitives Memory Subsystem scalar vector L2 SM SM TX/L1 TX/L1 RF RF RF RF mixed Shared memory among compute cores
39 Hardware-aware Search Space GPUs Compute Primitives Memory Subsystem scalar vector L2 SM SM TX/L1 TX/L1 RF RF RF RF mixed Shared memory among compute cores Use of Shared Thread Memory Cooperation
40 Hardware-aware Search Space TPU-like Specialized Accelerators Compute Primitives tensor Memory Subsystem Unified Buffer FIFO Acc explicitly managed
41 Hardware-aware Search Space TPU-like Specialized Accelerators Compute Primitives tensor Memory Subsystem Unified Buffer FIFO Acc explicitly managed
42 Compute primitives Tensorization Challenge
43 Tensorization Challenge Compute primitives scalar
44 Tensorization Challenge Compute primitives scalar vector
45 Tensorization Challenge Compute primitives scalar vector tensor
46 Tensorization Challenge Compute primitives scalar vector tensor Hardware designer: declare tensor instruction interface with Tensor Expression w, x = t.placeholder((8, 8)), t.placeholder((8, 8)) k = t.reduce_axis((0, 8)) y = t.compute((8, 8), lambda i, j: t.sum(w[i, k] * x[j, k], axis=k)) def gemm_intrin_lower(inputs, outputs): ww_ptr = inputs[0].access_ptr( r") xx_ptr = inputs[1].access_ptr("r") zz_ptr = outputs[0].access_ptr("w") compute = t.hardware_intrin("gemm8x8", ww_ptr, xx_ptr, zz_ptr) reset = t.hardware_intrin("fill_zero", zz_ptr) update = t.hardware_intrin("fuse_gemm8x8_add", ww_ptr, xx_ptr, zz_ptr) return compute, reset, update gemm8x8 = t.decl_tensor_intrin(y.op, gemm_intrin_lower) declare behavior lowering rule to generate hardware intrinsics to carry out the computation
47 Tensorization Challenge Compute primitives scalar vector tensor Hardware designer: declare tensor instruction interface with Tensor Expression Tensorize: transform program to use tensor instructions w, x = t.placeholder((8, 8)), t.placeholder((8, 8)) k = t.reduce_axis((0, 8)) y = t.compute((8, 8), lambda i, j: t.sum(w[i, k] * x[j, k], axis=k)) def gemm_intrin_lower(inputs, outputs): ww_ptr = inputs[0].access_ptr( r") xx_ptr = inputs[1].access_ptr("r") zz_ptr = outputs[0].access_ptr("w") compute = t.hardware_intrin("gemm8x8", ww_ptr, xx_ptr, zz_ptr) reset = t.hardware_intrin("fill_zero", zz_ptr) update = t.hardware_intrin("fuse_gemm8x8_add", ww_ptr, xx_ptr, zz_ptr) return compute, reset, update gemm8x8 = t.decl_tensor_intrin(y.op, gemm_intrin_lower) declare behavior lowering rule to generate hardware intrinsics to carry out the computation scalar tensor
48 Hardware-aware Search Space TPU-like Specialized Accelerators Compute Primitives tensor Memory Subsystem Unified Buffer FIFO Acc explicitly managed
49 Hardware-aware Search Space TPU-like Specialized Accelerators Compute Primitives tensor Memory Subsystem Unified Buffer FIFO Acc explicitly managed
50 Hardware-aware Search Space Tensor Expression Language C = tvm.compute((m, n), lambda y, x: tvm.sum(a[k, y] * B[k, x], axis=k)) Hardware
51 Hardware-aware Search Space Tensor Expression Language C = tvm.compute((m, n), lambda y, x: tvm.sum(a[k, y] * B[k, x], axis=k)) Primitives in prior work: Loop Thread Cache Halide, Loopy Transformations Bindings Locality Hardware
52 Hardware-aware Search Space Tensor Expression Language C = tvm.compute((m, n), lambda y, x: tvm.sum(a[k, y] * B[k, x], axis=k)) Primitives in prior work: Loop Thread Cache Halide, Loopy Transformations Bindings Locality New primitives for GPUs, Thread Latency and enable TPU-like Cooperation Tensorization Hiding Accelerators Hardware
53 Hardware-aware Search Space Tensor Expression Language C = tvm.compute((m, n), lambda y, x: tvm.sum(a[k, y] * B[k, x], axis=k)) Loop Transformations Thread Bindings Cache Locality Thread Cooperation Tensorization Latency Hiding Hardware
54 Hardware-aware Search Space Tensor Expression Language C = tvm.compute((m, n), lambda y, x: tvm.sum(a[k, y] * B[k, x], axis=k)) Loop Transformations Thread Bindings Cache Locality Thread Cooperation Tensorization Latency Hiding Hardware
55 Hardware-aware Search Space Tensor Expression Language C = tvm.compute((m, n), lambda y, x: tvm.sum(a[k, y] * B[k, x], axis=k)) Billions Loop Thread Cache of possible Transformations Bindings Locality optimization Thread Latency choices Cooperation Tensorization Hiding Hardware
56 Learning-based Learning System Frameworks High-level data flow graph and optimizations Hardware aware Search Space of Optimized Tensor Programs Machine Learning based Program Optimizer Hardware
57 Learning-based Learning System Frameworks High-level data flow graph and optimizations Hardware aware Search Space of Optimized Tensor Programs Machine Learning based Program Optimizer Hardware
58 Learning-based Learning System Frameworks High-level data flow graph and optimizations Hardware aware Search Space of Optimized Tensor Programs AutoTVM: Machine Learning based Program Optimizer Hardware
59 Some Quick Results
60 End to End Inference Performance (Nvidia Titan X) Tensorflow Apache MXNet Tensorflow-XLA Time(ms) ResNet-18 MobileNet 0.0 LSTM LM DQN DCGAN
61 End to End Inference Performance (Nvidia Titan X) Backed by cudnn Tensorflow Apache MXNet Tensorflow-XLA Time(ms) ResNet-18 MobileNet 0.0 LSTM LM DQN DCGAN
62 End to End Inference Performance (Nvidia Titan X) Tensorflow Apache MXNet TVM: without graph optimizations Tensorflow-XLA Time(ms) ResNet-18 MobileNet 0.0 LSTM LM DQN DCGAN
63 End to End Inference Performance (Nvidia Titan X) Tensorflow Tensorflow-XLA Apache MXNet TVM: without graph optimizations TVM: all optimizations Time(ms) ResNet-18 MobileNet 0.0 LSTM LM DQN DCGAN
64 End to End Inference Performance (Nvidia Titan X) Tensorflow Tensorflow-XLA Apache MXNet TVM: without graph optimizations TVM: all optimizations Competitive on standard models Time(ms) ResNet-18 MobileNet 0.0 LSTM LM DQN DCGAN
65 End to End Inference Performance (Nvidia Titan X) Tensorflow Tensorflow-XLA Apache MXNet TVM: without graph optimizations TVM: all optimizations Competitive on standard models Time(ms) Bigger gap on less conventional models 0.0 ResNet-18 MobileNet 0.0 LSTM LM DQN DCGAN
66 Across Hardware Platforms ARM CPU(A53) ARM GPU(MALI) Tensorflow Lite TVM w/o graph opt TVM ARMComputeLib TVM w/o graph opt TVM Time(ms) ResNet-18 MobileNet DQN Time (ms) float32 float16 ResNet-18 float32 float16 MobileNet float32 float16 DQN
67 Supporting New Specialized Accelerators Frameworks High-level data flow graph and optimizations Hardware aware Search Space of Optimized Tensor Programs Machine Learning based Program Optimizer LLVM CUDA
68 Supporting New Specialized Accelerators Frameworks High-level data flow graph and optimizations Hardware aware Search Space of Optimized Tensor Programs Machine Learning based Program Optimizer LLVM CUDA VTA: Open, Customizable Deep Learning Accelerator Edge FPGA Data Center FPGA ASIC
69 More on the High-Level Optimizations Frameworks High-level data flow graph and optimizations Hardware aware Search Space of Optimized Tensor Programs Machine Learning based Program Optimizer LLVM CUDA VTA: Open, Customizable Deep Learning Accelerator Edge FPGA Data Center FPGA ASIC
70 More on the High-Level Optimizations Frameworks Hardware aware Search Space of Optimized Tensor Programs Machine Learning based Program Optimizer LLVM CUDA VTA: Open, Customizable Deep Learning Accelerator Edge FPGA Data Center FPGA ASIC
71 More on the High-Level Optimizations Frameworks Relay: High-Level Differentiable IR Hardware aware Search Space of Optimized Tensor Programs Machine Learning based Program Optimizer LLVM CUDA VTA: Open, Customizable Deep Learning Accelerator Edge FPGA Data Center FPGA ASIC
72 TVM: Learning-based Deep Learning Compiler
73 TVM: Learning-based Deep Learning Compiler High-Level Differentiable IR
74 TVM: Learning-based Deep Learning Compiler High-Level Differentiable IR Tensor Expression IR
75 TVM: Learning-based Deep Learning Compiler High-Level Differentiable IR Tensor Expression IR LLVM, CUDA, Metal
76 TVM: Learning-based Deep Learning Compiler High-Level Differentiable IR Tensor Expression IR LLVM, CUDA, Metal VTA Edge FPGA Cloud FPGA ASIC
77 TVM: Learning-based Deep Learning Compiler Optimization High-Level Differentiable IR Tensor Expression IR AutoTVM LLVM, CUDA, Metal VTA AutoVTA Edge FPGA Cloud FPGA ASIC Hardware Fleet
78 TVM: Learning-based Deep Learning Compiler Optimization High-Level Differentiable IR Tensor Expression IR AutoTVM LLVM, CUDA, Metal VTA AutoVTA Edge FPGA Cloud FPGA ASIC Hardware Fleet
79 TVM: Learning-based Deep Learning Compiler Optimization High-Level Differentiable IR Tensor Expression IR AutoTVM LLVM, CUDA, Metal VTA AutoVTA Edge FPGA Cloud FPGA ASIC Hardware Fleet
80 TVM Open Source Community Prefer public archivable discussion Open RFC discussion Bring in new members by merit
81 TVM: Learning-based Deep Learning Compiler
82 TVM: Learning-based Deep Learning Compiler High-Level Differentiable IR
83 TVM: Learning-based Deep Learning Compiler High-Level Differentiable IR Tensor Expression IR
84 TVM: Learning-based Deep Learning Compiler High-Level Differentiable IR Tensor Expression IR LLVM, CUDA, Metal
85 TVM: Learning-based Deep Learning Compiler High-Level Differentiable IR Tensor Expression IR LLVM, CUDA, Metal VTA Edge FPGA Cloud FPGA ASIC
86 TVM: Learning-based Deep Learning Compiler Optimization High-Level Differentiable IR Tensor Expression IR AutoTVM LLVM, CUDA, Metal VTA AutoVTA Edge FPGA Cloud FPGA ASIC Hardware Fleet
87 TVM: Learning-based Deep Learning Compiler Optimization High-Level Differentiable IR Tensor Expression IR AutoTVM LLVM, CUDA, Metal VTA AutoVTA Edge FPGA Cloud FPGA ASIC Hardware Fleet
88 TVM: Learning-based Deep Learning Compiler Optimization High-Level Differentiable IR Tensor Expression IR AutoTVM LLVM, CUDA, Metal VTA AutoVTA Edge FPGA Cloud FPGA ASIC Hardware Fleet
TVM: An Automated End-to-End Optimizing Compiler for Deep Learning
TVM: An Automated End-to-End Optimizing Compiler for Deep Learning Tianqi Chen, Thierry Moreau, Ziheng Jiang, Lianmin Zheng, Eddie Yan, Meghan Cowan, Haichen Shen, Leyuan Wang, Yuwei Hu, Luis Ceze, Carlos
VTA: Open & Flexible DL Acceleration. Thierry Moreau TVM Conference, Dec 12th 2018
VTA: Open & Flexible DL Acceleration Thierry Moreau TVM Conference, Dec 12th 2018 TVM Stack High-Level Differentiable IR Tensor Expression IR LLVM CUDA Metal TVM Stack High-Level Differentiable IR Tensor
Automatic Code Generation TVM Stack
Automatic Code Generation TVM Stack CSE 599W Spring TVM stack is an active project by saml.cs.washington.edu and many partners in the open source community The Gap between Framework and Hardware Frameworks
End to End Optimization Stack for Deep Learning
End to End Optimization Stack for Deep Learning Presenter: Tianqi Chen Paul G. Allen School of Computer Science & Engineering University of Washington Collaborators University of Washington AWS AI Team
arxiv: v2 [cs.lg] 20 May 2018
![arxiv: v2 [cs.lg] 20 May 2018](/thumbs/81/82954790.jpg "arxiv: v2 [cs.lg] 20 May 2018")
TVM: An Automated End-to-End Optimizing Compiler for Deep Learning Tianqi Chen 1, Thierry Moreau 1, Ziheng Jiang 2, Lianmin Zheng 3, Eddie Yan 1 Meghan Cowan 1, Haichen Shen 1, Leyuan Wang 4, Yuwei Hu
AutoTVM & Device Fleet
AutoTVM & Device Fleet ` Learning to Optimize Tensor Programs Frameworks High-level data flow graph and optimizations Hardware Learning to Optimize Tensor Programs Frameworks High-level data flow graph
TVM: An Automated End-to-End Optimizing Compiler for Deep Learning
TVM: An Automated End-to-End Optimizing Compiler for Deep Learning Tianqi Chen and Thierry Moreau, University of Washington; Ziheng Jiang, University of Washington, AWS; Lianmin Zheng, Shanghai Jiao Tong
Relay: a high level differentiable IR. Jared Roesch TVMConf December 12th, 2018
Relay: a high level differentiable IR Jared Roesch TVMConf December 12th, 2018!1 This represents months of joint work with lots of great folks:!2 TVM Stack Optimization Relay High-Level Differentiable
arxiv: v1 [cs.lg] 11 Jul 2018
![arxiv: v1 [cs.lg] 11 Jul 2018](/thumbs/90/102842714.jpg "arxiv: v1 [cs.lg] 11 Jul 2018")
VTA: An Open Hardware-Software Stack for Deep Learning Thierry Moreau, Tianqi Chen, Ziheng Jiang, Luis Ceze, Carlos Guestrin, Arvind Krishnamurthy Paul G. Allen School of Computer Science & Engineering,
Optimizing CNN Inference on CPUs
Optimizing CNN Inference on CPUs Yizhi Liu, Yao Wang, Yida Wang With others in AWS AI Agenda Deep learning inference optimization Optimization on Intel CPUs Evaluation Make DL inference easier and faster
Scalable Distributed Training with Parameter Hub: a whirlwind tour
Scalable Distributed Training with Parameter Hub: a whirlwind tour TVM Stack Optimization High-Level Differentiable IR Tensor Expression IR AutoTVM LLVM, CUDA, Metal VTA AutoVTA Edge FPGA Cloud FPGA ASIC
Research Faculty Summit Systems Fueling future disruptions
Research Faculty Summit 2018 Systems Fueling future disruptions Wolong: A Back-end Optimizer for Deep Learning Computation Jilong Xue Researcher, Microsoft Research Asia System Challenge in Deep Learning
Deep Learning Accelerators
Deep Learning Accelerators Abhishek Srivastava (as29) Samarth Kulshreshtha (samarth5) University of Illinois, Urbana-Champaign Submitted as a requirement for CS 433 graduate student project Outline Introduction
Deep Learning Compiler
Deep Learning Compiler AWS AI Acknowledgement Amazon Sagemaker Neo Enables developers to train machine learning models once and run them anywhere in the cloud and at the edge Hardware targets Intel CPU,
Recurrent Neural Networks. Deep neural networks have enabled major advances in machine learning and AI. Convolutional Neural Networks
Deep neural networks have enabled major advances in machine learning and AI Computer vision Language translation Speech recognition Question answering And more Problem: DNNs are challenging to serve and
Lecture 6: Optimize for Hardware Backends. CSE599G1: Spring 2017
Lecture 6: Optimize for Hardware Backends CSE599G1: Spring 2017 Where are we High level Packages User API Programming API Gradient Calculation (Differentiation API) System Components Computational Graph
Can FPGAs beat GPUs in accelerating next-generation Deep Neural Networks? Discussion of the FPGA 17 paper by Intel Corp. (Nurvitadhi et al.
Can FPGAs beat GPUs in accelerating next-generation Deep Neural Networks? Discussion of the FPGA 17 paper by Intel Corp. (Nurvitadhi et al.) Andreas Kurth 2017-12-05 1 In short: The situation Image credit:
Speculations about Computer Architecture in Next Three Years. Jan. 20, 2018
Speculations about Computer Architecture in Next Three Years shuchang.zhou@gmail.com Jan. 20, 2018 About me https://zsc.github.io/ Source-to-source transformation Cache simulation Compiler Optimization
HKG OpenCL Support by NNVM & TVM. Jammy Zhou - Linaro
HKG18-417 OpenCL Support by NNVM & TVM Jammy Zhou - Linaro Agenda OpenCL Overview OpenCL in NNVM & TVM Current Status OpenCL Introduction Open Computing Language Open standard maintained by Khronos with
NVIDIA GPU CLOUD DEEP LEARNING FRAMEWORKS
TECHNICAL OVERVIEW NVIDIA GPU CLOUD DEEP LEARNING FRAMEWORKS A Guide to the Optimized Framework Containers on NVIDIA GPU Cloud Introduction Artificial intelligence is helping to solve some of the most
Xilinx ML Suite Overview
Xilinx ML Suite Overview Yao Fu System Architect Data Center Acceleration Xilinx Accelerated Computing Workloads Machine Learning Inference Image classification and object detection Video Streaming Frame
Two FPGA-DNN Projects: 1. Low Latency Multi-Layer Perceptrons using FPGAs 2. Acceleration of CNN Training on FPGA-based Clusters
Two FPGA-DNN Projects: 1. Low Latency Multi-Layer Perceptrons using FPGAs 2. Acceleration of CNN Training on FPGA-based Clusters *Argonne National Lab +BU & USTC Presented by Martin Herbordt Work by Ahmed
Unified Deep Learning with CPU, GPU, and FPGA Technologies
Unified Deep Learning with CPU, GPU, and FPGA Technologies Allen Rush 1, Ashish Sirasao 2, Mike Ignatowski 1 1: Advanced Micro Devices, Inc., 2: Xilinx, Inc. Abstract Deep learning and complex machine
The HammerBlade: An ML-Optimized Supercomputer for ML and Graphs
The HammerBlade: An ML-Optimized Supercomputer for ML and Graphs Prof. Michael B. Taylor (PI) University of Washington Prof. Adrian Sampson Cornell University Prof. Luis Ceze University of Washington Prof.
Neural Network Exchange Format
Copyright Khronos Group 2017 - Page 1 Neural Network Exchange Format Deploying Trained Networks to Inference Engines Viktor Gyenes, specification editor Copyright Khronos Group 2017 - Page 2 Outlook The
Embarquez votre Intelligence Artificielle (IA) sur CPU, GPU et FPGA
Embarquez votre Intelligence Artificielle (IA) sur CPU, GPU et FPGA Pierre Nowodzienski Engineer pierre.nowodzienski@mathworks.fr 2018 The MathWorks, Inc. 1 From Data to Business value Make decisions Get
GPU-Accelerated Deep Learning
GPU-Accelerated Deep Learning July 6 th, 2016. Greg Heinrich. Credits: Alison B. Lowndes, Julie Bernauer, Leo K. Tam. PRACTICAL DEEP LEARNING EXAMPLES Image Classification, Object Detection, Localization,
Deep learning in MATLAB From Concept to CUDA Code
Deep learning in MATLAB From Concept to CUDA Code Roy Fahn Applications Engineer Systematics royf@systematics.co.il 03-7660111 Ram Kokku Principal Engineer MathWorks ram.kokku@mathworks.com 2017 The MathWorks,
GPU Coder: Automatic CUDA and TensorRT code generation from MATLAB
GPU Coder: Automatic CUDA and TensorRT code generation from MATLAB Ram Kokku 2018 The MathWorks, Inc. 1 GPUs and CUDA programming faster Performance CUDA OpenCL C/C++ GPU Coder MATLAB Python Ease of programming
Processing-in-Memory for Energy-efficient Neural Network Training: A Heterogeneous Approach
2018 51st Annual IEEE/ACM International Symposium on Microarchitecture Processing-in-Memory for Energy-efficient Neural Network Training: A Heterogeneous Approach Jiawen Liu*, Hengyu Zhao*, Matheus Almeida
Master Informatics Eng.
Advanced Architectures Master Informatics Eng. 2018/19 A.J.Proença Data Parallelism 3 (GPU/CUDA, Neural Nets,...) (most slides are borrowed) AJProença, Advanced Architectures, MiEI, UMinho, 2018/19 1 The
High Performance Computing
High Performance Computing 9th Lecture 2016/10/28 YUKI ITO 1 Selected Paper: vdnn: Virtualized Deep Neural Networks for Scalable, MemoryEfficient Neural Network Design Minsoo Rhu, Natalia Gimelshein, Jason
Open Standards for Vision and AI Peter McGuinness NNEF WG Chair CEO, Highwai, Inc May 2018
Copyright Khronos Group 2018 - Page 1 Open Standards for Vision and AI Peter McGuinness NNEF WG Chair CEO, Highwai, Inc peter.mcguinness@gobrach.com May 2018 Khronos Mission E.g. OpenGL ES provides 3D
Nvidia Jetson TX2 and its Software Toolset. João Fernandes 2017/2018
Nvidia Jetson TX2 and its Software Toolset João Fernandes 2017/2018 In this presentation Nvidia Jetson TX2: Hardware Nvidia Jetson TX2: Software Machine Learning: Neural Networks Convolutional Neural Networks
Inference Optimization Using TensorRT with Use Cases. Jack Han / 한재근 Solutions Architect NVIDIA
Inference Optimization Using TensorRT with Use Cases Jack Han / 한재근 Solutions Architect NVIDIA Search Image NLP Maps TensorRT 4 Adoption Use Cases Speech Video AI Inference is exploding 1 Billion Videos
Wu Zhiwen.
Wu Zhiwen zhiwen.wu@intel.com Agenda Background information OpenCV DNN module OpenCL acceleration Vulkan backend Sample 2 What is OpenCV? Open Source Compute Vision (OpenCV) library 2500+ Optimized algorithms
A GPU-Inspired Soft Processor for High- Throughput Acceleration
A GPU-Inspired Soft Processor for High- Throughput Acceleration Jeffrey Kingyens and J. Gregory Steffan Electrical and Computer Engineering University of Toronto 1 FGPA-Based Acceleration In-socket acceleration
Lecture 1 Introduction. I. Why Study Compilers? II.Example: Machine Learning III. Course Syllabus
Lecture 1 Introduction I. Why Study Compilers? II.Example: Machine Learning III. Course Syllabus Chapters 1.1-1.5, 8.4, 8.5, 9.1 CS243: Introduction 1 Why Study Compilers? CS243: Introduction 2 1 Reasons
Improving Performance of Machine Learning Workloads
Improving Performance of Machine Learning Workloads Dong Li Parallel Architecture, System, and Algorithm Lab Electrical Engineering and Computer Science School of Engineering University of California,
Lecture 9: Scheduling. CSE599G1: Spring 2017
Lecture 9: Scheduling CSE599G1: Spring 2017 Next Week Two Joint Sessions with Computer Architecture Class Different date, time and location, detail to be announced Wed: ASICs for deep learning Friday:
Deep Learning on Modern Architectures. Keren Zhou 4/17/2017
Deep Learning on Modern Architectures Keren Zhou 4/17/2017 HPC Software Stack Application Algorithm Data Layout CPU GPU MIC Others HPC Software Stack Deep Learning Algorithm Data Layout CPU GPU MIC Others
Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware
Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware Florian Tramèr (joint work with Dan Boneh) Intel, Santa Clara August 30 th 2018 Trusted execution of ML: 3 motivating
Deep Learning Frameworks with Spark and GPUs
Deep Learning Frameworks with Spark and GPUs Abstract Spark is a powerful, scalable, real-time data analytics engine that is fast becoming the de facto hub for data science and big data. However, in parallel,
Deploying Deep Learning Networks to Embedded GPUs and CPUs
Deploying Deep Learning Networks to Embedded GPUs and CPUs Rishu Gupta, PhD Senior Application Engineer, Computer Vision 2015 The MathWorks, Inc. 1 MATLAB Deep Learning Framework Access Data Design + Train
Graph Streaming Processor
Graph Streaming Processor A Next-Generation Computing Architecture Val G. Cook Chief Software Architect Satyaki Koneru Chief Technology Officer Ke Yin Chief Scientist Dinakar Munagala Chief Executive Officer
Machine Learning on VMware vsphere with NVIDIA GPUs
Machine Learning on VMware vsphere with NVIDIA GPUs Uday Kurkure, Hari Sivaraman, Lan Vu GPU Technology Conference 2017 2016 VMware Inc. All rights reserved. Gartner Hype Cycle for Emerging Technology
When MPPDB Meets GPU:
When MPPDB Meets GPU: An Extendible Framework for Acceleration Laura Chen, Le Cai, Yongyan Wang Background: Heterogeneous Computing Hardware Trend stops growing with Moore s Law Fast development of GPU
Deep Learning: Transforming Engineering and Science The MathWorks, Inc.
Deep Learning: Transforming Engineering and Science 1 2015 The MathWorks, Inc. DEEP LEARNING: TRANSFORMING ENGINEERING AND SCIENCE A THE NEW RISE ERA OF OF GPU COMPUTING 3 NVIDIA A IS NEW THE WORLD S ERA
A NEW COMPUTING ERA JENSEN HUANG, FOUNDER & CEO GTC CHINA 2017
A NEW COMPUTING ERA JENSEN HUANG, FOUNDER & CEO GTC CHINA 2017 TWO FORCES DRIVING THE FUTURE OF COMPUTING 10 7 Transistors (thousands) 10 6 10 5 1.1X per year 10 4 10 3 10 2 1.5X per year Single-threaded
Shrinath Shanbhag Senior Software Engineer Microsoft Corporation
Accelerating GPU inferencing with DirectML and DirectX 12 Shrinath Shanbhag Senior Software Engineer Microsoft Corporation Machine Learning Machine learning has become immensely popular over the last decade
Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware
Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware Florian Tramèr (joint work with Dan Boneh) Stanford security lunch June 13 th Trusted execution of ML: 3 motivating
POWERING THE AI REVOLUTION JENSEN HUANG, FOUNDER & CEO GTC 2017
POWERING THE AI REVOLUTION JENSEN HUANG, FOUNDER & CEO GTC 2017 LIFE AFTER MOORE S LAW 10 7 40 Years of Microprocessor Trend Data 10 6 10 5 Transistors (thousands) 1.1X per year 10 4 10 3 1.5X per year
Matchbox. automatic batching for imperative deep learning NVIDIA GTC, 2018/3/28. James Bradbury
Matchbox automatic batching for imperative deep learning James Bradbury NVIDIA GTC, 2018/3/28 Roadmap Imperative deep learning How manual batching works Other tools for automatic batching The Matchbox
Enabling Technology for the Cloud and AI One Size Fits All?
Enabling Technology for the Cloud and AI One Size Fits All? Tim Horel Collaborate. Differentiate. Win. DIRECTOR, FIELD APPLICATIONS The Growing Cloud Global IP Traffic Growth 40B+ devices with intelligence
NVIDIA DATA LOADING LIBRARY (DALI)
NVIDIA DATA LOADING LIBRARY (DALI) RN-09096-001 _v01 September 2018 Release Notes TABLE OF CONTENTS Chapter Chapter Chapter Chapter Chapter 1. 2. 3. 4. 5. DALI DALI DALI DALI DALI Overview...1 Release
Characterizing and Benchmarking Deep Learning Systems on Modern Data Center Architectures
Characterizing and Benchmarking Deep Learning Systems on Modern Data Center Architectures Talk at Bench 2018 by Xiaoyi Lu The Ohio State University E-mail: luxi@cse.ohio-state.edu http://www.cse.ohio-state.edu/~luxi
CUDA OPTIMIZATION WITH NVIDIA NSIGHT ECLIPSE EDITION
CUDA OPTIMIZATION WITH NVIDIA NSIGHT ECLIPSE EDITION WHAT YOU WILL LEARN An iterative method to optimize your GPU code Some common bottlenecks to look out for Performance diagnostics with NVIDIA Nsight
GPU FOR DEEP LEARNING. 周国峰 Wuhan University 2017/10/13
GPU FOR DEEP LEARNING chandlerz@nvidia.com 周国峰 Wuhan University 2017/10/13 Why Deep Learning Boost Today? Nvidia SDK for Deep Learning? Agenda CUDA 8.0 cudnn TensorRT (GIE) NCCL DIGITS 2 Why Deep Learning
High-Performance Data Loading and Augmentation for Deep Neural Network Training
High-Performance Data Loading and Augmentation for Deep Neural Network Training Trevor Gale tgale@ece.neu.edu Steven Eliuk steven.eliuk@gmail.com Cameron Upright c.upright@samsung.com Roadmap 1. The General-Purpose
The OpenVX Computer Vision and Neural Network Inference
The OpenVX Computer and Neural Network Inference Standard for Portable, Efficient Code Radhakrishna Giduthuri Editor, OpenVX Khronos Group radha.giduthuri@amd.com @RadhaGiduthuri Copyright 2018 Khronos
Operator Vectorization Library A TensorFlow Plugin
Operator Vectorization Library A TensorFlow Plugin Matthew Pickett, Karen Brems, Florian Raudies Hewlett Packard Labs HPE-2016-94 Keyword(s): Machine learning; GPU acceleration; TensorFlow Abstract: TensorFlow
Accelerate Machine Learning on macos with Intel Integrated Graphics. Hisham Chowdhury May 23, 2018
Accelerate Machine Learning on macos with Intel Integrated Graphics Hisham Chowdhury May 23, 2018 Apple Machine Learning Stack Machine Learning Application 1 Machine Learning Application 2 Vision Natural
GPUfs: Integrating a file system with GPUs
GPUfs: Integrating a file system with GPUs Mark Silberstein (UT Austin/Technion) Bryan Ford (Yale), Idit Keidar (Technion) Emmett Witchel (UT Austin) 1 Traditional System Architecture Applications OS CPU
Toward Scalable Deep Learning
한국정보과학회 인공지능소사이어티 머신러닝연구회 두번째딥러닝워크샵 2015.10.16 Toward Scalable Deep Learning 윤성로 Electrical and Computer Engineering Seoul National University http://data.snu.ac.kr Breakthrough: Big Data + Machine Learning
Red Fox: An Execution Environment for Relational Query Processing on GPUs
Red Fox: An Execution Environment for Relational Query Processing on GPUs Haicheng Wu 1, Gregory Diamos 2, Tim Sheard 3, Molham Aref 4, Sean Baxter 2, Michael Garland 2, Sudhakar Yalamanchili 1 1. Georgia
GPU ACCELERATED DATABASE MANAGEMENT SYSTEMS
CIS 601 - Graduate Seminar Presentation 1 GPU ACCELERATED DATABASE MANAGEMENT SYSTEMS PRESENTED BY HARINATH AMASA CSU ID: 2697292 What we will talk about.. Current problems GPU What are GPU Databases GPU
Accelerating Binarized Convolutional Neural Networks with Software-Programmable FPGAs
Accelerating Binarized Convolutional Neural Networks with Software-Programmable FPGAs Ritchie Zhao 1, Weinan Song 2, Wentao Zhang 2, Tianwei Xing 3, Jeng-Hau Lin 4, Mani Srivastava 3, Rajesh Gupta 4, Zhiru
Deep Learning Inference in Facebook Data Centers: Characterization, Performance Optimizations, and Hardware Implications
Deep Learning Inference in Facebook Data Centers: Characterization, Performance Optimizations, and Hardware Implications Jongsoo Park Facebook AI System SW/HW Co-design Team Sep-21 2018 Team Introduction
NVIDIA TESLA V100 GPU ARCHITECTURE THE WORLD S MOST ADVANCED DATA CENTER GPU
NVIDIA TESLA V100 GPU ARCHITECTURE THE WORLD S MOST ADVANCED DATA CENTER GPU WP-08608-001_v1.1 August 2017 WP-08608-001_v1.1 TABLE OF CONTENTS Introduction to the NVIDIA Tesla V100 GPU Architecture...
NVIDIA Think about Computing as Heterogeneous One Leo Liao, 1/29/2106, NTU
NVIDIA Think about Computing as Heterogeneous One Leo Liao, 1/29/2106, NTU GPGPU opens the door for co-design HPC, moreover middleware-support embedded system designs to harness the power of GPUaccelerated
CSCI 402: Computer Architectures. Parallel Processors (2) Fengguang Song Department of Computer & Information Science IUPUI.
CSCI 402: Computer Architectures Parallel Processors (2) Fengguang Song Department of Computer & Information Science IUPUI 6.6 - End Today s Contents GPU Cluster and its network topology The Roofline performance
Scaling Distributed Machine Learning
Scaling Distributed Machine Learning with System and Algorithm Co-design Mu Li Thesis Defense CSD, CMU Feb 2nd, 2017 nx min w f i (w) Distributed systems i=1 Large scale optimization methods Large-scale
NVIDIA AI BRAIN OF SELF DRIVING AND HD MAPPING. September 13, 2016
NVIDIA AI BRAIN OF SELF DRIVING AND HD MAPPING September 13, 2016 AI FOR AUTONOMOUS DRIVING MAPPING KALDI LOCALIZATION DRIVENET Training on DGX-1 NVIDIA DGX-1 NVIDIA DRIVE PX 2 Driving with DriveWorks
S WHAT THE PROFILER IS TELLING YOU: OPTIMIZING GPU KERNELS. Jakob Progsch, Mathias Wagner GTC 2018
S8630 - WHAT THE PROFILER IS TELLING YOU: OPTIMIZING GPU KERNELS Jakob Progsch, Mathias Wagner GTC 2018 1. Know your hardware BEFORE YOU START What are the target machines, how many nodes? Machine-specific
Integrating NVIDIA Deep Learning Accelerator (NVDLA) with RISC-V SoC on FireSim
Integrating NVIDIA Deep Learning Accelerator (NVDLA) with RISC-V SoC on FireSim Farzad Farshchi, Qijing Huang, Heechul Yun University of Kansas, University of California, Berkeley SiFive Internship Rocket
Xilinx ML Suite Overview
Xilinx ML Suite Overview Jim Heaton Sr. FAE Deep Learning explores the study of algorithms that can learn from and make predictions on data Deep Learning is Re-defining Many Applications Cloud Acceleration
CUDA 5 and Beyond. Mark Ebersole. Original Slides: Mark Harris 2012 NVIDIA
CUDA 5 and Beyond Mark Ebersole Original Slides: Mark Harris The Soul of CUDA The Platform for High Performance Parallel Computing Accessible High Performance Enable Computing Ecosystem Introducing CUDA
Scheduling Image Processing Pipelines
Lecture 15: Scheduling Image Processing Pipelines Visual Computing Systems Simple image processing kernel int WIDTH = 1024; int HEIGHT = 1024; float input[width * HEIGHT]; float output[width * HEIGHT];
Onto Petaflops with Kubernetes
Onto Petaflops with Kubernetes Vishnu Kannan Google Inc. vishh@google.com Key Takeaways Kubernetes can manage hardware accelerators at Scale Kubernetes provides a playground for ML ML journey with Kubernetes
COSC 6339 Accelerators in Big Data
COSC 6339 Accelerators in Big Data Edgar Gabriel Fall 2018 Motivation Programming models such as MapReduce and Spark provide a high-level view of parallelism not easy for all problems, e.g. recursive algorithms,
Multi-dimensional Parallel Training of Winograd Layer on Memory-Centric Architecture
The 51st Annual IEEE/ACM International Symposium on Microarchitecture Multi-dimensional Parallel Training of Winograd Layer on Memory-Centric Architecture Byungchul Hong Yeonju Ro John Kim FuriosaAI Samsung
PRIME: A Novel Processing-in-memory Architecture for Neural Network Computation in ReRAM-based Main Memory
Scalable and Energy-Efficient Architecture Lab (SEAL) PRIME: A Novel Processing-in-memory Architecture for Neural Network Computation in -based Main Memory Ping Chi *, Shuangchen Li *, Tao Zhang, Cong
Copperhead: Data Parallel Python Bryan Catanzaro, NVIDIA Research 1/19
Copperhead: Data Parallel Python Bryan Catanzaro, NVIDIA Research 1/19 Motivation Not everyone wants to write C++ or Fortran How can we broaden the use of parallel computing? All of us want higher programmer
Versal: AI Engine & Programming Environment
Engineering Director, Xilinx Silicon Architecture Group Versal: Engine & Programming Environment Presented By Ambrose Finnerty Xilinx DSP Technical Marketing Manager October 16, 2018 MEMORY MEMORY MEMORY
Maximizing Server Efficiency from μarch to ML accelerators. Michael Ferdman
Maximizing Server Efficiency from μarch to ML accelerators Michael Ferdman Maximizing Server Efficiency from μarch to ML accelerators Michael Ferdman Maximizing Server Efficiency with ML accelerators Michael
Adaptable Computing The Future of FPGA Acceleration. Dan Gibbons, VP Software Development June 6, 2018
Adaptable Computing The Future of FPGA Acceleration Dan Gibbons, VP Software Development June 6, 2018 Adaptable Accelerated Computing Page 2 Three Big Trends The Evolution of Computing Trend to Heterogeneous
Why AI Frameworks Need (not only) RDMA?
Why AI Frameworks Need (not only) RDMA? With Design and Implementation Experience of Networking Support on TensorFlow GDR, Apache MXNet, WeChat Amber, and Tencent Angel Bairen Yi (byi@connect.ust.hk) Jingrong
Implementing Long-term Recurrent Convolutional Network Using HLS on POWER System
Implementing Long-term Recurrent Convolutional Network Using HLS on POWER System Xiaofan Zhang1, Mohamed El Hadedy1, Wen-mei Hwu1, Nam Sung Kim1, Jinjun Xiong2, Deming Chen1 1 University of Illinois Urbana-Champaign
MIXED PRECISION TRAINING: THEORY AND PRACTICE Paulius Micikevicius
MIXED PRECISION TRAINING: THEORY AND PRACTICE Paulius Micikevicius What is Mixed Precision Training? Reduced precision tensor math with FP32 accumulation, FP16 storage Successfully used to train a variety
Introduction to Deep Learning in Signal Processing & Communications with MATLAB
Introduction to Deep Learning in Signal Processing & Communications with MATLAB Dr. Amod Anandkumar Pallavi Kar Application Engineering Group, Mathworks India 2019 The MathWorks, Inc. 1 Different Types
High-Performance Training for Deep Learning and Computer Vision HPC
High-Performance Training for Deep Learning and Computer Vision HPC Panel at CVPR-ECV 18 by Dhabaleswar K. (DK) Panda The Ohio State University E-mail: panda@cse.ohio-state.edu http://www.cse.ohio-state.edu/~panda
Fast Hardware For AI
Fast Hardware For AI Karl Freund karl@moorinsightsstrategy.com Sr. Analyst, AI and HPC Moor Insights & Strategy Follow my blogs covering Machine Learning Hardware on Forbes: http://www.forbes.com/sites/moorinsights
EXTENDING THE REACH OF PARALLEL COMPUTING WITH CUDA
EXTENDING THE REACH OF PARALLEL COMPUTING WITH CUDA Mark Harris, NVIDIA @harrism #NVSC14 EXTENDING THE REACH OF CUDA 1 Machine Learning 2 Higher Performance 3 New Platforms 4 New Languages 2 GPUS: THE
Scaling Throughput Processors for Machine Intelligence
Scaling Throughput Processors for Machine Intelligence ScaledML Stanford 24-Mar-18 simon@graphcore.ai 1 MI The impact on humanity of harnessing machine intelligence will be greater than the impact of harnessing
Chapter 2 Parallel Hardware
Chapter 2 Parallel Hardware Part I. Preliminaries Chapter 1. What Is Parallel Computing? Chapter 2. Parallel Hardware Chapter 3. Parallel Software Chapter 4. Parallel Applications Chapter 5. Supercomputers
Neural Cache: Bit-Serial In-Cache Acceleration of Deep Neural Networks
Neural Cache: Bit-Serial In-Cache Acceleration of Deep Neural Networks Charles Eckert Xiaowei Wang Jingcheng Wang Arun Subramaniyan Ravi Iyer Dennis Sylvester David Blaauw Reetuparna Das M-Bits Research
Generating Fast Data Planes for Data-Intensive Systems
Generating Fast Data Planes for Data-Intensive Systems Shoumik Palkar and Matei Zaharia Stanford DAWN Many New Data Systems We are building many specialized high-performance data systems. Graph Engines
NVIDIA GTX200: TeraFLOPS Visual Computing. August 26, 2008 John Tynefield
NVIDIA GTX200: TeraFLOPS Visual Computing August 26, 2008 John Tynefield 2 Outline Execution Model Architecture Demo 3 Execution Model 4 Software Architecture Applications DX10 OpenGL OpenCL CUDA C Host
Lift: a Functional Approach to Generating High Performance GPU Code using Rewrite Rules
Lift: a Functional Approach to Generating High Performance GPU Code using Rewrite Rules Toomas Remmelg Michel Steuwer Christophe Dubach The 4th South of England Regional Programming Language Seminar 27th
Introduction to CUDA CME343 / ME May James Balfour [ NVIDIA Research
Introduction to CUDA CME343 / ME339 18 May 2011 James Balfour [ jbalfour@nvidia.com] NVIDIA Research CUDA Programing system for machines with GPUs Programming Language Compilers Runtime Environments Drivers