Bridging Analog Neuromorphic and Digital von Neumann Computing

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Bridging Analog Neuromorphic and Digital von Neumann Computing"

Transcription

1 Bridging Analog Neuromorphic and Digital von Neumann Computing Amir Yazdanbakhsh, Bradley Thwaites Advisors: Hadi Esmaeilzadeh and Doug Burger Qualcomm Mentors: Manu Rastogiand Girish Varatkar Alternative Computing Technologies (ACT) Lab Georgia Institute of Technology Qualcomm Innovation Fellowship

2 Energy is a primary constraint Data Center Mobile Internet of Things

3 Data growth vs performance Data growth trends: IDC's Digital Universe Study, December 2012 Performance growth trends: Esmaeilzadeh et al, Dark Silicon and the End of Multicore Scaling, ISCA

4 Approximate computing Embracing error Relax the abstraction of near perfect accuracy in Processing Storage Communication Allows errors to happen to improve performance resource utilization efficiency 6

5 Avoiding overkill design Approximate Computing Application Programming Language Compiler Architecutre Microarchitecture Cost Precision Reliability Cost Circuit Physical Device

6 Adding a third dimension Embracing Error Processor Pareto.Fron0er Data center Energy Desktop Mobile IoT Performance

7 Navigating a three dimensional space Processor Pareto.Fron0er Data center Energy Desktop IoT Mobile Performance

8 Finding the Pareto surface Energy IoT Processor Pareto.Fron0er Mobile Data center Desktop Truffle [ALOS 12] FLEXJAVA [FSE 15] RFVP [PACT 14, IEEE D&T 15] Axilog [DATE 15, IEEE Micro 15] D- NPUs [MICRO 12] A- NPUs [ISCA 14] SNNAP [HPCA 15] GNPU [Micro 15] MITHRA [TechCon 15] Performance (13.5, 11.1, 10%)

9 Accelerating GPU Accelerators Bridging Neuromophic and von Neumann Computing Unleashing the Beast Amir Yazdanbakhsh et al., Neural Acceleration for GPU Throughput Processors, MICRO 2015.

10 Neural Transformation Analog Neural Network Analog Neural Network

11 Analog NPU Integration CPU x 0 x i x n DAC DAC DAC I(x 0 ) I(x i ) I(x n ) R 0 X (I(xi )R(w i )) ADC R(w i ) R(w n ) A-NPU V to I V to I V to I SM A-NPU SM A-NPU SM A-NPU SM SM SM A-NPU A-NPU A-NPU SM SM SM A-NPU A-NPU A-NPU SM SM SM A-NPU A-NPU A-NPU y sigmoid( X (I(x i )R(w i ))) SM SM SM SM A-NPU A-NPU A-NPU A-NPU General- Purpose Code Acceleration with Limited- Precision Analog Computation ISCA 2014 Neural Acceleration for GPU Throughput Processors Micro 2015

12 s w0 w 0 s x0 x 0 s wn w n s xn x n I( x 0 ) Current' Steering' DAC I( x n ) Current' Steering' DAC Resistor' Ladder Resistor' Ladder R( w 0 ) R( w n ) I + (w 0 x 0 ) Diff' Pair I (w 0 x 0 ) V + X wi x i V ( w 0 x 0 ) V X wi x i + -" I + (w n x n ) Diff' Amp Diff' Pair y sigmoid V V ( w n x n ) I (w n x n ) Flash ADC s y y X wi x i

13 Analog Compilation Workflow Limited Bit-Width Topology Restriction Circuit Non-idealities Annotated CUDA Code uchar4'p'='tex2d(img,'x,'y); #pragma(begin_approx) a=min(r,'min(g,b)); b=max(r,'max(g,b)); z=((a+b)'>'254)'?'255:'0; #pragma(end_approx) dst[img.width'*'y'+'x]'='z; Compiler + Customized Training Algorthim Application uchar4'p'='tex2d(img,'x,'y); send.n_data5%r0; send.n_data5%r1; send.n_data5%r2; recv.n_data5%r4; dst[img.width'*'y'+'x]'='z; Accelerator Config w 0 = 0.03,, w 8=0.10 SM SM SM SM A-NPU A-NPU A-NPU A-NPU SM SM SM SM A-NPU A-NPU A-NPU A-NPU SM SM SM SM A-NPU A-NPU A-NPU A-NPU SM SM SM SM A-NPU A-NPU A-NPU A-NPU Programming Compilation (Profiling, Training, Code Generation) Execution

14 Benchmarks Image Processing binarization 27 PTX instructions Finance blackscholes 96 PTX instructions Machine Learning convolution 886 PTX instructions Robotics inversek2j 132 PTX instructions 3D Gaming jmeint 2,250PTX instructions Error: 11.43% Error: 8.23% Error: 9.29% Error: 10.25% Error: 19.70% Image Processing laplacian 51 PTX instructions Machine Vision meanfilter 35 PTX instructions Numerical Analysis newton- raph 44 PTX instructions Image Processing sobel 86 PTX instructions Medical Imaging srad 110 PTX instructions Error: 9.87% Error: 9.21% Error: 11.23% Error: 8.03% Error: 9.87%

15 Analog Neuromorphic versus Conventional Computing

16 I 1 I 0 I 2 I out = I 0 + I 1 + I 2 Kirchhoff's Law + V o I(x n ) R(w n ) V o = I(x n ).R(w n ) Ohm s Law Saturation Property of Transistors

17 Speedup Energy Reduction Energy Delay Quality Reduction 10 % Publications [1] Amir Yazdanbakhsh et al., Neural Acceleration for GPU Throughput Processors Micro [2] Renée St. Amant et al., General- Purpose Code Acceleration with Limited- Precision Analog Computation ISCA 2014.

18 Application Programming Language Compiler Architecutre Microarchitecture Circuit Physical Device Software Architecture Memory Hardware Design FLEXJAVA: Language Support for Safe and Modular Approximate Programming [FSE 2015] ExpAX: A Framework for Automating Approximate Programming [Tech Report 2014] Neural Acceleration for GPU Throughput Processors [Micro 2015] MITHRA: Controlling Quality Tradeoffs in Approximate Acceleration [TechCon 2015] General- Purpose Code Acceleration with Limited- Precision Analog Computation [ISCA 2014] Mitigating the Bandwidth Bottleneck with Approximate Load Value Prediction [IEEE Design and Test 2015] Rollback- Free Value Prediction with Approximate Loads [PACT 2014] Axilog: Abstractions for Approximate Hardware Design and Reuse [IEEE Micro 2015] Axilog: Language Support for Approximate Hardware Support [DATE 2015]

19 Rollback Free Value Prediction Front End Pipelines Load / Store Unit Pipeline L1 Cache Write back Interconnection Network Memory Partition L2 Cache Off-chip DRAM Streaming Multiprocessor (SM)

20 Rollback Free Value Prediction Front End Pipelines Load / Store Unit Pipeline L1 Cache Write back Interconnection Network Memory Partition L2 Cache Off-chip DRAM Streaming Multiprocessor (SM)

21 Rollback Free Value Prediction Front End Pipelines Load / Store Unit Pipeline L1 Cache Write back Full Interconnection Network Memory Partition L2 Cache Off-chip DRAM Streaming Multiprocessor (SM)

22 Rollback Free Value Prediction Front End Pipelines RFVP Predictor Load / Store Unit Pipeline L1 Cache Write back Interconnection Network Memory Partition L2 Cache Off-chip DRAM Streaming Multiprocessor (SM) RFVP Predictor quickly predicts values for approximate load misses RFVP technique mitigates the memory bandwidth bottleneck

23 Speedup 1.4 Energy Reduction 1.3 Bandwidth Consumption Reduction 1.5 Quality Reduction 10 % Publications [1] Amir Yazdanbakhsh et al., Mitigating the Bandwidth Bottleneck with Approximate Load Value Prediction IEEE Design and Test [2] Amir Yazdanbakhsh et al., RFVP: Rollback- Free Value Prediction with Safe- to- Approximate Loads Architecture and Code Optimization (TACO) [submitted]. [3] Bradley Thwaites et al., Rollback- Free Value Prediction with Approximate Loads International Conference on Parallel Architectures and Compilation Techniques (PACT) 2014.

24 module fir (clk, rst, x, y) clk rst input clk, rst; d0 d1 d2 d3 x input [15:0] x; b0 b1 b2 b3 output [31:0] y; m0 m1 m2 m3 * w0 w1 w2 w3 multiplier m1 (b1, d1, w1); multiplier m2 (b2, d2, w2); a1 a2 a3 w4 w5 adder a1 (w0, w1, w4); adder a2 (w2, w4, w5); register r1 (clk, rst, x, d0); register r2 (clk, rst, d0, d1); endmodule y

25 module fir (clk, rst, x, y) clk input clk, rst; rst d0 d1 d2 d3 x input [15:0] x; output [31:0] y; b0 b1 b2 b3 m0 m1 m2 m3 * w0 w1 w2 w3 multiplier m1 (b1, d1, w1); multiplier m2 (b2, d2, w2); + a1 + a2 w4 + a3 w5 adder a1 (w0, w1, w4); adder a2 (w2, w4, w5); register r1 (clk, rst, x, d0); register r2 (clk, rst, d0, d1); relax(y); endmodule relax(y) y

26 module fir (clk, rst, x, y) clk input clk, rst; rst d0 d1 d2 d3 x input [15:0] x; output [31:0] y; b0 b1 b2 b3 m0 m1 m2 m3 * w0 w1 w2 w3 multiplier m1 (b1, d1, w1); multiplier m2 (b2, d2, w2); + a1 + a2 w4 + a3 w5 adder a1 (w0, w1, w4); adder a2 (w2, w4, w5); register r1 (clk, rst, x, d0); register r2 (clk, rst, d0, d1); relax(y); endmodule relax(y) y

27 module fir (clk, rst, x, y) clk input clk, rst; rst d0 d1 d2 d3 input [15:0] x; x output [31:0] y; b0 b1 b2 b3 m0 m1 m2 m3 * multiplier m1 (b1, d1, w1); w0 w1 w2 w3 restrict(w1) restrict(w2) restrict(w3) multiplier m2 (b2, d2, w2); a1 a2 a3 w4 w5 adder a1 (w0, w1, w4); adder a2 (w2, w4, w5); register r1 (clk, rst, x, d0); register r2 (clk, rst, d0, d1); relax(y); restrict(w1); restrict(w2); endmodule relax(y) y

28 module fir (clk, rst, x, y) clk input clk, rst; rst d0 d1 d2 d3 input [15:0] x; x output [31:0] y; b0 b1 b2 b3 m0 m1 m2 m3 * multiplier m1 (b1, d1, w1); w0 w1 w2 w3 restrict(w1) restrict(w2) restrict(w3) multiplier m2 (b2, d2, w2); a1 a2 a3 w4 w5 adder a1 (w0, w1, w4); adder a2 (w2, w4, w5); register r1 (clk, rst, x, d0); register r2 (clk, rst, d0, d1); relax(y); restrict(w1); restrict(w2); endmodule relax(y) y

29 Energy Reduction Area Reduction Code Annotations Quality Reduction 10 % Publications [1] Divya Mahajan et al., Axilog: Abstractions for Approximate Hardware Design and Reuse IEEE Micro [2] Amir Yazdanbakhsh et al., Axilog: Language Support for Approximate Hardware Design Design Automation and Test in Europe (DATE) 2015.

30 Finding the Pareto surface Energy IoT Processor Pareto.Fron0er Mobile Data center Desktop Truffle [ALOS 12] FLEXJAVA [FSE 15] RFVP [PACT 14, IEEE D&T 15] Axilog [DATE 15, IEEE Micro 15] D- NPUs [MICRO 12] A- NPUs [ISCA 14] SNNAP [HPCA 15] GNPU [Micro 15] MITHRA [TechCon 15] Performance (13.5, 11.1, 10%)

Axilog: Language Support for Approximate Hardware Design

Axilog: Language Support for Approximate Hardware Design Axilog: Language Support for Approximate Hardware Design Amir Yazdanbakhsh Divya Mahajan Bradley Thwaites Jongse Park Anandhavel Nagendrakumar Sindhuja Sethuraman Kar,k Ramkrishnan Nishanthi Ravindran

More information

AxBench: A Multiplatform Benchmark Suite for Approximate Computing

AxBench: A Multiplatform Benchmark Suite for Approximate Computing AxBench: A Multiplatform Benchmark Suite for Approximate Computing Amir Yazdanbakhsh, Divya Mahajan, and Hadi Esmaeilzadeh Georgia Institute of Technology Pejman Lotfi-Kamran Institute for Research in

More information

AxBench: A Benchmark Suite for Approximate Computing Across the System Stack

AxBench: A Benchmark Suite for Approximate Computing Across the System Stack AxBench: A Benchmark Suite for Approximate Computing Across the System Stack Amir Yazdanbakhsh Divya Mahajan Pejman Lotfi-Kamran Hadi Esmaeilzadeh Alternative Computing Technologies (ACT) Lab School of

More information

RFVP: Rollback-Free Value Prediction with Safe-to-Approximate Loads

RFVP: Rollback-Free Value Prediction with Safe-to-Approximate Loads RFVP: Rollback-Free Value Prediction with Safe-to-Approximate Loads Amir Yazdanbakhsh Gennady Pekhimenko Bradley Thwaites Hadi Esmaeilzadeh Taesoo Kim Onur Mutlu Todd C. Mowry Georgia Institute of Technology

More information

Microprocessor Trends and Implications for the Future

Microprocessor Trends and Implications for the Future Microprocessor Trends and Implications for the Future John Mellor-Crummey Department of Computer Science Rice University johnmc@rice.edu COMP 522 Lecture 4 1 September 2016 Context Last two classes: from

More information

Chapter 1: Fundamentals of Quantitative Design and Analysis

Chapter 1: Fundamentals of Quantitative Design and Analysis 1 / 12 Chapter 1: Fundamentals of Quantitative Design and Analysis Be careful in this chapter. It contains a tremendous amount of information and data about the changes in computer architecture since the

More information

Neural Acceleration for GPU Throughput Processors

Neural Acceleration for GPU Throughput Processors Neural Acceleration for Throughput Processors Amir Yazdanbakhsh Jongse Park Hardik Sharma Pejman Lotfi-Kamran Hadi Esmaeilzadeh Alternative Computing Technologies (ACT) Lab School of Computer Science,

More information

PRIME: A Novel Processing-in-memory Architecture for Neural Network Computation in ReRAM-based Main Memory

PRIME: A Novel Processing-in-memory Architecture for Neural Network Computation in ReRAM-based Main Memory Scalable and Energy-Efficient Architecture Lab (SEAL) PRIME: A Novel Processing-in-memory Architecture for Neural Network Computation in -based Main Memory Ping Chi *, Shuangchen Li *, Tao Zhang, Cong

More information

FLEXJAVA:)Language'Support' for'safe'and'modular' Approximate'Programming

FLEXJAVA:)Language'Support' for'safe'and'modular' Approximate'Programming FLEXJAVA:)Language'Support' for'safe'and'modular' Approximate'Programming Jongse Park,'Hadi'Esmaeilzadeh,'Xin'Zhang,' Mayur Naik,'William'Harris Alternative'Computing'Technologies'(ACT)'Lab Georgia'Institute'of'Technology

More information

Exploiting Hidden Layer Modular Redundancy for Fault-Tolerance in Neural Network Accelerators

Exploiting Hidden Layer Modular Redundancy for Fault-Tolerance in Neural Network Accelerators Exploiting Hidden Layer Modular Redundancy for Fault-Tolerance in Neural Network Accelerators Schuyler Eldridge Ajay Joshi Department of Electrical and Computer Engineering, Boston University schuye@bu.edu

More information

Neural Acceleration for General-Purpose Approximate Programs

Neural Acceleration for General-Purpose Approximate Programs 2012 IEEE/ACM 45th Annual International Symposium on Microarchitecture Neural Acceleration for General-Purpose Approximate Programs Hadi Esmaeilzadeh Adrian Sampson Luis Ceze Doug Burger University of

More information

The Effect of Temperature on Amdahl Law in 3D Multicore Era

The Effect of Temperature on Amdahl Law in 3D Multicore Era The Effect of Temperature on Amdahl Law in 3D Multicore Era L Yavits, A Morad, R Ginosar Abstract This work studies the influence of temperature on performance and scalability of 3D Chip Multiprocessors

More information

Copyright 2012, Elsevier Inc. All rights reserved.

Copyright 2012, Elsevier Inc. All rights reserved. Computer Architecture A Quantitative Approach, Fifth Edition Chapter 1 Fundamentals of Quantitative Design and Analysis 1 Computer Technology Performance improvements: Improvements in semiconductor technology

More information

A Case for Core-Assisted Bottleneck Acceleration in GPUs Enabling Flexible Data Compression with Assist Warps

A Case for Core-Assisted Bottleneck Acceleration in GPUs Enabling Flexible Data Compression with Assist Warps A Case for Core-Assisted Bottleneck Acceleration in GPUs Enabling Flexible Data Compression with Assist Warps Nandita Vijaykumar Gennady Pekhimenko, Adwait Jog, Abhishek Bhowmick, Rachata Ausavarangnirun,

More information

CS 475: Parallel Programming Introduction

CS 475: Parallel Programming Introduction CS 475: Parallel Programming Introduction Wim Bohm, Sanjay Rajopadhye Colorado State University Fall 2014 Course Organization n Let s make a tour of the course website. n Main pages Home, front page. Syllabus.

More information

Lecture 1: Introduction and Basics

Lecture 1: Introduction and Basics CS 515 Programming Language and Compilers I Lecture 1: Introduction and Basics Zheng (Eddy) Zhang Rutgers University Fall 2017, 9/5/2017 Class Information Instructor: Zheng (Eddy) Zhang Email: eddyzhengzhang@gmailcom

More information

DNN ENGINE: A 16nm Sub-uJ DNN Inference Accelerator for the Embedded Masses

DNN ENGINE: A 16nm Sub-uJ DNN Inference Accelerator for the Embedded Masses DNN ENGINE: A 16nm Sub-uJ DNN Inference Accelerator for the Embedded Masses Paul N. Whatmough 1,2 S. K. Lee 2, N. Mulholland 2, P. Hansen 2, S. Kodali 3, D. Brooks 2, G.-Y. Wei 2 1 ARM Research, Boston,

More information

System-on-Chip Architecture for Mobile Applications. Sabyasachi Dey

System-on-Chip Architecture for Mobile Applications. Sabyasachi Dey System-on-Chip Architecture for Mobile Applications Sabyasachi Dey Email: sabyasachi.dey@gmail.com Agenda What is Mobile Application Platform Challenges Key Architecture Focus Areas Conclusion Mobile Revolution

More information

ELCT 912: Advanced Embedded Systems

ELCT 912: Advanced Embedded Systems ELCT 912: Advanced Embedded Systems Lecture 2-3: Embedded System Hardware Dr. Mohamed Abd El Ghany, Department of Electronics and Electrical Engineering Embedded System Hardware Used for processing of

More information

Computer Architecture: Multi-Core Processors: Why? Onur Mutlu & Seth Copen Goldstein Carnegie Mellon University 9/11/13

Computer Architecture: Multi-Core Processors: Why? Onur Mutlu & Seth Copen Goldstein Carnegie Mellon University 9/11/13 Computer Architecture: Multi-Core Processors: Why? Onur Mutlu & Seth Copen Goldstein Carnegie Mellon University 9/11/13 Moore s Law Moore, Cramming more components onto integrated circuits, Electronics,

More information

Advanced Computer Architecture (CS620)

Advanced Computer Architecture (CS620) Advanced Computer Architecture (CS620) Background: Good understanding of computer organization (eg.cs220), basic computer architecture (eg.cs221) and knowledge of probability, statistics and modeling (eg.cs433).

More information

Computing architectures Part 2 TMA4280 Introduction to Supercomputing

Computing architectures Part 2 TMA4280 Introduction to Supercomputing Computing architectures Part 2 TMA4280 Introduction to Supercomputing NTNU, IMF January 16. 2017 1 Supercomputing What is the motivation for Supercomputing? Solve complex problems fast and accurately:

More information

ECE 8823: GPU Architectures. Objectives

ECE 8823: GPU Architectures. Objectives ECE 8823: GPU Architectures Introduction 1 Objectives Distinguishing features of GPUs vs. CPUs Major drivers in the evolution of general purpose GPUs (GPGPUs) 2 1 Chapter 1 Chapter 2: 2.2, 2.3 Reading

More information

Embedded Systems: Hardware Components (part I) Todor Stefanov

Embedded Systems: Hardware Components (part I) Todor Stefanov Embedded Systems: Hardware Components (part I) Todor Stefanov Leiden Embedded Research Center Leiden Institute of Advanced Computer Science Leiden University, The Netherlands Outline Generic Embedded System

More information

CUDA. Matthew Joyner, Jeremy Williams

CUDA. Matthew Joyner, Jeremy Williams CUDA Matthew Joyner, Jeremy Williams Agenda What is CUDA? CUDA GPU Architecture CPU/GPU Communication Coding in CUDA Use cases of CUDA Comparison to OpenCL What is CUDA? What is CUDA? CUDA is a parallel

More information

Computer system energy management. Charles Lefurgy

Computer system energy management. Charles Lefurgy 28 July 2011 Computer system energy management Charles Lefurgy Outline A short history of server power management POWER7 EnergyScale AMESTER power measurement tool Challenges ahead 2 A brief history of

More information

ECE 486/586. Computer Architecture. Lecture # 2

ECE 486/586. Computer Architecture. Lecture # 2 ECE 486/586 Computer Architecture Lecture # 2 Spring 2015 Portland State University Recap of Last Lecture Old view of computer architecture: Instruction Set Architecture (ISA) design Real computer architecture:

More information

Computer and Information Sciences College / Computer Science Department CS 207 D. Computer Architecture

Computer and Information Sciences College / Computer Science Department CS 207 D. Computer Architecture Computer and Information Sciences College / Computer Science Department CS 207 D Computer Architecture The Computer Revolution Progress in computer technology Underpinned by Moore s Law Makes novel applications

More information

Efficiency and Programmability: Enablers for ExaScale. Bill Dally Chief Scientist and SVP, Research NVIDIA Professor (Research), EE&CS, Stanford

Efficiency and Programmability: Enablers for ExaScale. Bill Dally Chief Scientist and SVP, Research NVIDIA Professor (Research), EE&CS, Stanford Efficiency and Programmability: Enablers for ExaScale Bill Dally Chief Scientist and SVP, Research NVIDIA Professor (Research), EE&CS, Stanford Scientific Discovery and Business Analytics Driving an Insatiable

More information

ADVANCES IN PROCESSOR DESIGN AND THE EFFECTS OF MOORES LAW AND AMDAHLS LAW IN RELATION TO THROUGHPUT MEMORY CAPACITY AND PARALLEL PROCESSING

ADVANCES IN PROCESSOR DESIGN AND THE EFFECTS OF MOORES LAW AND AMDAHLS LAW IN RELATION TO THROUGHPUT MEMORY CAPACITY AND PARALLEL PROCESSING ADVANCES IN PROCESSOR DESIGN AND THE EFFECTS OF MOORES LAW AND AMDAHLS LAW IN RELATION TO THROUGHPUT MEMORY CAPACITY AND PARALLEL PROCESSING Evan Baytan Department of Electrical Engineering and Computer

More information

Profiling-Based L1 Data Cache Bypassing to Improve GPU Performance and Energy Efficiency

Profiling-Based L1 Data Cache Bypassing to Improve GPU Performance and Energy Efficiency Profiling-Based L1 Data Cache Bypassing to Improve GPU Performance and Energy Efficiency Yijie Huangfu and Wei Zhang Department of Electrical and Computer Engineering Virginia Commonwealth University {huangfuy2,wzhang4}@vcu.edu

More information

Thomas Moscibroda Microsoft Research. Onur Mutlu CMU

Thomas Moscibroda Microsoft Research. Onur Mutlu CMU Thomas Moscibroda Microsoft Research Onur Mutlu CMU CPU+L1 CPU+L1 CPU+L1 CPU+L1 Multi-core Chip Cache -Bank Cache -Bank Cache -Bank Cache -Bank CPU+L1 CPU+L1 CPU+L1 CPU+L1 Accelerator, etc Cache -Bank

More information

TDT 4260 lecture 3 spring semester 2015

TDT 4260 lecture 3 spring semester 2015 1 TDT 4260 lecture 3 spring semester 2015 Lasse Natvig, The CARD group Dept. of computer & information science NTNU http://research.idi.ntnu.no/multicore 2 Lecture overview Repetition Chap.1: Performance,

More information

VOLTA: PROGRAMMABILITY AND PERFORMANCE. Jack Choquette NVIDIA Hot Chips 2017

VOLTA: PROGRAMMABILITY AND PERFORMANCE. Jack Choquette NVIDIA Hot Chips 2017 VOLTA: PROGRAMMABILITY AND PERFORMANCE Jack Choquette NVIDIA Hot Chips 2017 1 TESLA V100 21B transistors 815 mm 2 80 SM 5120 CUDA Cores 640 Tensor Cores 16 GB HBM2 900 GB/s HBM2 300 GB/s NVLink *full GV100

More information

SOLVING THE DRAM SCALING CHALLENGE: RETHINKING THE INTERFACE BETWEEN CIRCUITS, ARCHITECTURE, AND SYSTEMS

SOLVING THE DRAM SCALING CHALLENGE: RETHINKING THE INTERFACE BETWEEN CIRCUITS, ARCHITECTURE, AND SYSTEMS SOLVING THE DRAM SCALING CHALLENGE: RETHINKING THE INTERFACE BETWEEN CIRCUITS, ARCHITECTURE, AND SYSTEMS Samira Khan MEMORY IN TODAY S SYSTEM Processor DRAM Memory Storage DRAM is critical for performance

More information

Power Limitations and Dark Silicon Challenge the Future of Multicore

Power Limitations and Dark Silicon Challenge the Future of Multicore See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/254463832 Power Limitations and Dark Silicon Challenge the Future of Multicore Article in ACM

More information

Chapter 6. Parallel Processors from Client to Cloud. Copyright 2014 Elsevier Inc. All rights reserved.

Chapter 6. Parallel Processors from Client to Cloud. Copyright 2014 Elsevier Inc. All rights reserved. Chapter 6 Parallel Processors from Client to Cloud FIGURE 6.1 Hardware/software categorization and examples of application perspective on concurrency versus hardware perspective on parallelism. 2 FIGURE

More information

Toward a Memory-centric Architecture

Toward a Memory-centric Architecture Toward a Memory-centric Architecture Martin Fink EVP & Chief Technology Officer Western Digital Corporation August 8, 2017 1 SAFE HARBOR DISCLAIMERS Forward-Looking Statements This presentation contains

More information

TETRIS: Scalable and Efficient Neural Network Acceleration with 3D Memory

TETRIS: Scalable and Efficient Neural Network Acceleration with 3D Memory TETRIS: Scalable and Efficient Neural Network Acceleration with 3D Memory Mingyu Gao, Jing Pu, Xuan Yang, Mark Horowitz, Christos Kozyrakis Stanford University Platform Lab Review Feb 2017 Deep Neural

More information

Multi2sim Kepler: A Detailed Architectural GPU Simulator

Multi2sim Kepler: A Detailed Architectural GPU Simulator Multi2sim Kepler: A Detailed Architectural GPU Simulator Xun Gong, Rafael Ubal, David Kaeli Northeastern University Computer Architecture Research Lab Department of Electrical and Computer Engineering

More information

Efficient Hardware Acceleration on SoC- FPGA using OpenCL

Efficient Hardware Acceleration on SoC- FPGA using OpenCL Efficient Hardware Acceleration on SoC- FPGA using OpenCL Advisor : Dr. Benjamin Carrion Schafer Susmitha Gogineni 30 th August 17 Presentation Overview 1.Objective & Motivation 2.Configurable SoC -FPGA

More information

Computer Architecture: Dataflow/Systolic Arrays

Computer Architecture: Dataflow/Systolic Arrays Data Flow Computer Architecture: Dataflow/Systolic Arrays he models we have examined all assumed Instructions are fetched and retired in sequential, control flow order his is part of the Von-Neumann model

More information

ENIAC - background. ENIAC - details. Structure of von Nuemann machine. von Neumann/Turing Computer Architecture

ENIAC - background. ENIAC - details. Structure of von Nuemann machine. von Neumann/Turing Computer Architecture 168 420 Computer Architecture Chapter 2 Computer Evolution and Performance ENIAC - background Electronic Numerical Integrator And Computer Eckert and Mauchly University of Pennsylvania Trajectory tables

More information

Computer Architecture

Computer Architecture Computer Architecture Lecture 2: Fundamental Concepts and ISA Dr. Ahmed Sallam Based on original slides by Prof. Onur Mutlu What Do I Expect From You? Chance favors the prepared mind. (Louis Pasteur) كل

More information

Reconfigurable Multicore Server Processors for Low Power Operation

Reconfigurable Multicore Server Processors for Low Power Operation Reconfigurable Multicore Server Processors for Low Power Operation Ronald G. Dreslinski, David Fick, David Blaauw, Dennis Sylvester, Trevor Mudge University of Michigan, Advanced Computer Architecture

More information

GPU Architecture Overview

GPU Architecture Overview Course Overview Follow-Up GPU Architecture Overview Patrick Cozzi University of Pennsylvania CIS 565 - Fall 2012 Read before or after lecture? Project vs. final project Closed vs. open source Feedback

More information

SP-CNN: A SCALABLE AND PROGRAMMABLE CNN-BASED ACCELERATOR

SP-CNN: A SCALABLE AND PROGRAMMABLE CNN-BASED ACCELERATOR .. SP-CNN: A SCALABLE AND PROGRAMMABLE CNN-BASED ACCELERATOR.. IN THIS ARTICLE, THE AUTHORS USE A CELLULAR NEURAL NETWORK (CNN) A NEURAL Dilan Manatunga Hyesoon Kim Saibal Mukhopadhyay Georgia Institute

More information

Algorithms and Parallel Computing

Algorithms and Parallel Computing Algorithms and Parallel Computing Algorithms and Parallel Computing Fayez Gebali University of Victoria, Victoria, BC A John Wiley & Sons, Inc., Publication Copyright 2011 by John Wiley & Sons, Inc. All

More information

Computer Architecture Lecture 2: Fundamental Concepts and ISA. Prof. Onur Mutlu Carnegie Mellon University Spring 2015, 1/14/2014

Computer Architecture Lecture 2: Fundamental Concepts and ISA. Prof. Onur Mutlu Carnegie Mellon University Spring 2015, 1/14/2014 18-447 Computer Architecture Lecture 2: Fundamental Concepts and ISA Prof. Onur Mutlu Carnegie Mellon University Spring 2015, 1/14/2014 Agenda for Today Finish up logistics from last lecture Why study

More information

Register File Organization

Register File Organization Register File Organization Sudhakar Yalamanchili unless otherwise noted (1) To understand the organization of large register files used in GPUs Objective Identify the performance bottlenecks and opportunities

More information

CSE 591: GPU Programming. Introduction. Entertainment Graphics: Virtual Realism for the Masses. Computer games need to have: Klaus Mueller

CSE 591: GPU Programming. Introduction. Entertainment Graphics: Virtual Realism for the Masses. Computer games need to have: Klaus Mueller Entertainment Graphics: Virtual Realism for the Masses CSE 591: GPU Programming Introduction Computer games need to have: realistic appearance of characters and objects believable and creative shading,

More information

Master Program (Laurea Magistrale) in Computer Science and Networking. High Performance Computing Systems and Enabling Platforms.

Master Program (Laurea Magistrale) in Computer Science and Networking. High Performance Computing Systems and Enabling Platforms. Master Program (Laurea Magistrale) in Computer Science and Networking High Performance Computing Systems and Enabling Platforms Marco Vanneschi Multithreading Contents Main features of explicit multithreading

More information

Computer Organization & Assembly Language Programming. CSE 2312 Lecture 2 Introduction to Computers

Computer Organization & Assembly Language Programming. CSE 2312 Lecture 2 Introduction to Computers Computer Organization & Assembly Language Programming CSE 2312 Lecture 2 Introduction to Computers 1 Languages, Levels, Virtual Machines A multilevel machine 2 Contemporary Multilevel Machines A six-level

More information

AC-DIMM: Associative Computing with STT-MRAM

AC-DIMM: Associative Computing with STT-MRAM AC-DIMM: Associative Computing with STT-MRAM Qing Guo, Xiaochen Guo, Ravi Patel Engin Ipek, Eby G. Friedman University of Rochester Published In: ISCA-2013 Motivation Prevalent Trends in Modern Computing:

More information

Software Defined Hardware

Software Defined Hardware Software Defined Hardware For data intensive computation Wade Shen DARPA I2O September 19, 2017 1 Goal Statement Build runtime reconfigurable hardware and software that enables near ASIC performance (within

More information

ANALOG IP WITH INTELLIGENT IP FROM SYSTEM TO SILICON

ANALOG IP WITH INTELLIGENT IP FROM SYSTEM TO SILICON ANALOG IP WITH INTELLIGENT IP FROM SYSTEM TO SILICON Torsten Reich, Fraunhofer IIS/EAS IIP Analog IP with Intelligent IP from system to silicon What is Intelligent IP (on silicon level)? Intelligent IP

More information

Tutorial 11. Final Exam Review

Tutorial 11. Final Exam Review Tutorial 11 Final Exam Review Introduction Instruction Set Architecture: contract between programmer and designers (e.g.: IA-32, IA-64, X86-64) Computer organization: describe the functional units, cache

More information

CPU Architecture Overview. Varun Sampath CIS 565 Spring 2012

CPU Architecture Overview. Varun Sampath CIS 565 Spring 2012 CPU Architecture Overview Varun Sampath CIS 565 Spring 2012 Objectives Performance tricks of a modern CPU Pipelining Branch Prediction Superscalar Out-of-Order (OoO) Execution Memory Hierarchy Vector Operations

More information

Concurrent/Parallel Processing

Concurrent/Parallel Processing Concurrent/Parallel Processing David May: April 9, 2014 Introduction The idea of using a collection of interconnected processing devices is not new. Before the emergence of the modern stored program computer,

More information

Detecting Memory-Boundedness with Hardware Performance Counters

Detecting Memory-Boundedness with Hardware Performance Counters Center for Information Services and High Performance Computing (ZIH) Detecting ory-boundedness with Hardware Performance Counters ICPE, Apr 24th 2017 (daniel.molka@tu-dresden.de) Robert Schöne (robert.schoene@tu-dresden.de)

More information

CMPE 665:Multiple Processor Systems CUDA-AWARE MPI VIGNESH GOVINDARAJULU KOTHANDAPANI RANJITH MURUGESAN

CMPE 665:Multiple Processor Systems CUDA-AWARE MPI VIGNESH GOVINDARAJULU KOTHANDAPANI RANJITH MURUGESAN CMPE 665:Multiple Processor Systems CUDA-AWARE MPI VIGNESH GOVINDARAJULU KOTHANDAPANI RANJITH MURUGESAN Graphics Processing Unit Accelerate the creation of images in a frame buffer intended for the output

More information

Introduction to Parallel Computing

Introduction to Parallel Computing Portland State University ECE 588/688 Introduction to Parallel Computing Reference: Lawrence Livermore National Lab Tutorial https://computing.llnl.gov/tutorials/parallel_comp/ Copyright by Alaa Alameldeen

More information

Slides compliment of Yong Chen and Xian-He Sun From paper Reevaluating Amdahl's Law in the Multicore Era. 11/16/2011 Many-Core Computing 2

Slides compliment of Yong Chen and Xian-He Sun From paper Reevaluating Amdahl's Law in the Multicore Era. 11/16/2011 Many-Core Computing 2 Slides compliment of Yong Chen and Xian-He Sun From paper Reevaluating Amdahl's Law in the Multicore Era 11/16/2011 Many-Core Computing 2 Gene M. Amdahl, Validity of the Single-Processor Approach to Achieving

More information

Network-on-chip (NOC) Topologies

Network-on-chip (NOC) Topologies Network-on-chip (NOC) Topologies 1 Network Topology Static arrangement of channels and nodes in an interconnection network The roads over which packets travel Topology chosen based on cost and performance

More information

Understanding Outstanding Memory Request Handling Resources in GPGPUs

Understanding Outstanding Memory Request Handling Resources in GPGPUs Understanding Outstanding Memory Request Handling Resources in GPGPUs Ahmad Lashgar ECE Department University of Victoria lashgar@uvic.ca Ebad Salehi ECE Department University of Victoria ebads67@uvic.ca

More information

The Future of Interconnect Technology

The Future of Interconnect Technology The Future of Interconnect Technology Michael Kagan, CTO HPC Advisory Council Stanford, 2014 Exponential Data Growth Best Interconnect Required 44X 0.8 Zetabyte 2009 35 Zetabyte 2020 2014 Mellanox Technologies

More information

DesignWare IP for IoT SoC Designs

DesignWare IP for IoT SoC Designs DesignWare IP for IoT SoC Designs The Internet of Things (IoT) is connecting billions of intelligent things at our fingertips. The ability to sense countless amounts of information that communicates to

More information

TDT 4260 lecture 2 spring semester 2015

TDT 4260 lecture 2 spring semester 2015 1 TDT 4260 lecture 2 spring semester 2015 Lasse Natvig, The CARD group Dept. of computer & information science NTNU 2 Lecture overview Chapter 1: Fundamentals of Quantitative Design and Analysis, continued

More information

Simplify System Complexity

Simplify System Complexity 1 2 Simplify System Complexity With the new high-performance CompactRIO controller Arun Veeramani Senior Program Manager National Instruments NI CompactRIO The Worlds Only Software Designed Controller

More information

Comparing Memory Systems for Chip Multiprocessors

Comparing Memory Systems for Chip Multiprocessors Comparing Memory Systems for Chip Multiprocessors Jacob Leverich Hideho Arakida, Alex Solomatnikov, Amin Firoozshahian, Mark Horowitz, Christos Kozyrakis Computer Systems Laboratory Stanford University

More information

Advanced Computer Architecture

Advanced Computer Architecture Advanced Computer Architecture 1 L E C T U R E 0 J A N L E M E I R E Course Objectives 2 Intel 4004 1971 2.3K trans. Intel Core 2 Duo 2006 291M trans. Where have all the transistors gone? Turing Machine

More information

Computer Architecture Lecture 2: Fundamental Concepts and ISA. Prof. Onur Mutlu Carnegie Mellon University Spring 2014, 1/15/2014

Computer Architecture Lecture 2: Fundamental Concepts and ISA. Prof. Onur Mutlu Carnegie Mellon University Spring 2014, 1/15/2014 18-447 Computer Architecture Lecture 2: Fundamental Concepts and ISA Prof. Onur Mutlu Carnegie Mellon University Spring 2014, 1/15/2014 Announcement Rachata s Office Hours this week Today 4:30-5:30pm Sunday

More information

XMEGA Series Of AVR Processor. Presented by: Manisha Biyani ( ) Shashank Bolia (

XMEGA Series Of AVR Processor. Presented by: Manisha Biyani ( ) Shashank Bolia ( XMEGA Series Of AVR Processor Presented by: Manisha Biyani (200601217) Shashank Bolia (200601200 Existing Microcontrollers Problems with 8/16 bit microcontrollers: Old and inefficient architecture. Most

More information

Using Industry Standards to Exploit the Advantages and Resolve the Challenges of Multicore Technology

Using Industry Standards to Exploit the Advantages and Resolve the Challenges of Multicore Technology Using Industry Standards to Exploit the Advantages and Resolve the Challenges of Multicore Technology September 19, 2007 Markus Levy, EEMBC and Multicore Association Enabling the Multicore Ecosystem Multicore

More information

CSE 291: Mobile Application Processor Design

CSE 291: Mobile Application Processor Design CSE 291: Mobile Application Processor Design Mobile Application Processors are where the action are The evolution of mobile application processors mirrors that of microprocessors mirrors that of mainframes..

More information

This Unit: Putting It All Together. CIS 371 Computer Organization and Design. Sources. What is Computer Architecture?

This Unit: Putting It All Together. CIS 371 Computer Organization and Design. Sources. What is Computer Architecture? This Unit: Putting It All Together CIS 371 Computer Organization and Design Unit 15: Putting It All Together: Anatomy of the XBox 360 Game Console Application OS Compiler Firmware CPU I/O Memory Digital

More information

CSE 160 Lecture 24. Graphical Processing Units

CSE 160 Lecture 24. Graphical Processing Units CSE 160 Lecture 24 Graphical Processing Units Announcements Next week we meet in 1202 on Monday 3/11 only On Weds 3/13 we have a 2 hour session Usual class time at the Rady school final exam review SDSC

More information

Ultra Low Power (ULP) Challenge in System Architecture Level

Ultra Low Power (ULP) Challenge in System Architecture Level Ultra Low Power (ULP) Challenge in System Architecture Level - New architectures for 45-nm, 32-nm era ASP-DAC 2007 Designers' Forum 9D: Panel Discussion: Top 10 Design Issues Toshinori Sato (Kyushu U)

More information

Challenges to Embedding Computer Vision J. Scott Gardner General Manager and Editor-in-Chief Embedded Vision Alliance (www.embedded-vision.

Challenges to Embedding Computer Vision J. Scott Gardner General Manager and Editor-in-Chief Embedded Vision Alliance (www.embedded-vision. Challenges to Embedding Computer Vision J. Scott Gardner General Manager and Editor-in-Chief Embedded Vision Alliance (www.embedded-vision.com) May 16, 2011 Figure 1 HAL 9000 a machine that sees. Source:

More information

Interconnection Networks: Topology. Prof. Natalie Enright Jerger

Interconnection Networks: Topology. Prof. Natalie Enright Jerger Interconnection Networks: Topology Prof. Natalie Enright Jerger Topology Overview Definition: determines arrangement of channels and nodes in network Analogous to road map Often first step in network design

More information

04 - DSP Architecture and Microarchitecture

04 - DSP Architecture and Microarchitecture September 11, 2015 Memory indirect addressing (continued from last lecture) ; Reality check: Data hazards! ; Assembler code v3: repeat 256,endloop load r0,dm1[dm0[ptr0++]] store DM0[ptr1++],r0 endloop:

More information

Massively Parallel Computing on Silicon: SIMD Implementations. V.M.. Brea Univ. of Santiago de Compostela Spain

Massively Parallel Computing on Silicon: SIMD Implementations. V.M.. Brea Univ. of Santiago de Compostela Spain Massively Parallel Computing on Silicon: SIMD Implementations V.M.. Brea Univ. of Santiago de Compostela Spain GOAL Give an overview on the state-of of-the- art of Digital on-chip CMOS SIMD Solutions,

More information

Bottleneck Identification and Scheduling in Multithreaded Applications. José A. Joao M. Aater Suleman Onur Mutlu Yale N. Patt

Bottleneck Identification and Scheduling in Multithreaded Applications. José A. Joao M. Aater Suleman Onur Mutlu Yale N. Patt Bottleneck Identification and Scheduling in Multithreaded Applications José A. Joao M. Aater Suleman Onur Mutlu Yale N. Patt Executive Summary Problem: Performance and scalability of multithreaded applications

More information

Computer Organization and Design, 5th Edition: The Hardware/Software Interface

Computer Organization and Design, 5th Edition: The Hardware/Software Interface Computer Organization and Design, 5th Edition: The Hardware/Software Interface 1 Computer Abstractions and Technology 1.1 Introduction 1.2 Eight Great Ideas in Computer Architecture 1.3 Below Your Program

More information

Wenisch Final Review. Fall 2007 Prof. Thomas Wenisch EECS 470. Slide 1

Wenisch Final Review. Fall 2007 Prof. Thomas Wenisch  EECS 470. Slide 1 Final Review Fall 2007 Prof. Thomas Wenisch http://www.eecs.umich.edu/courses/eecs470 Slide 1 Announcements Wenisch 2007 Exam is Monday, 12/17 4 6 in this room I recommend bringing a scientific calculator

More information

CME 213 S PRING Eric Darve

CME 213 S PRING Eric Darve CME 213 S PRING 2017 Eric Darve Summary of previous lectures Pthreads: low-level multi-threaded programming OpenMP: simplified interface based on #pragma, adapted to scientific computing OpenMP for and

More information

Configurable Multiprocessing: An FIR

Configurable Multiprocessing: An FIR Configurable Multiprocessing: An FIR Filter Example Cmpware, Inc. Introduction Multiple processors on a device common Thousands of 32-bit RISC CPUs possible Advantages in: Performance Power consumption

More information

Name card info (inside)

Name card info (inside) CS 475: Parallel Programming Introduction Sanjay Rajopadhye (with updates by Wim Bohm, Cathie Olschanowski) Colorado State University Fall 2016 Name card info (inside) n Name: Sanjay Rajopadhye n Pronunciation

More information

CUDA Optimization: Memory Bandwidth Limited Kernels CUDA Webinar Tim C. Schroeder, HPC Developer Technology Engineer

CUDA Optimization: Memory Bandwidth Limited Kernels CUDA Webinar Tim C. Schroeder, HPC Developer Technology Engineer CUDA Optimization: Memory Bandwidth Limited Kernels CUDA Webinar Tim C. Schroeder, HPC Developer Technology Engineer Outline We ll be focussing on optimizing global memory throughput on Fermi-class GPUs

More information

TEACHING PLAN GUIDE. Course Content/Activity Sem /17 Chapters from 10 th Edition textbook. Week. Week Feb. 17.

TEACHING PLAN GUIDE. Course Content/Activity Sem /17 Chapters from 10 th Edition textbook. Week. Week Feb. 17. TEACHING PLAN GUIDE Week Course Content/Activity Sem 2 2016/17 Chapters from 10 th Edition textbook Lab Activity Date/Notes 1 Chapter 1: Basic Concepts and Computer Evolution Organization and architecture

More information

Recurrent Neural Networks. Deep neural networks have enabled major advances in machine learning and AI. Convolutional Neural Networks

Recurrent Neural Networks. Deep neural networks have enabled major advances in machine learning and AI. Convolutional Neural Networks Deep neural networks have enabled major advances in machine learning and AI Computer vision Language translation Speech recognition Question answering And more Problem: DNNs are challenging to serve and

More information

CSE502 Graduate Computer Architecture. Lec 22 Goodbye to Computer Architecture and Review

CSE502 Graduate Computer Architecture. Lec 22 Goodbye to Computer Architecture and Review CSE502 Graduate Computer Architecture Lec 22 Goodbye to Computer Architecture and Review Larry Wittie Computer Science, StonyBrook University http://www.cs.sunysb.edu/~cse502 and ~lw Slides adapted from

More information

Trends in the Infrastructure of Computing

Trends in the Infrastructure of Computing Trends in the Infrastructure of Computing CSCE 9: Computing in the Modern World Dr. Jason D. Bakos My Questions How do computer processors work? Why do computer processors get faster over time? How much

More information

15-740/ Computer Architecture, Fall 2011 Midterm Exam II

15-740/ Computer Architecture, Fall 2011 Midterm Exam II 15-740/18-740 Computer Architecture, Fall 2011 Midterm Exam II Instructor: Onur Mutlu Teaching Assistants: Justin Meza, Yoongu Kim Date: December 2, 2011 Name: Instructions: Problem I (69 points) : Problem

More information

Neural Networks as Function Primitives

Neural Networks as Function Primitives Neural Networks as Function Primitives Software/Hardware Support with X-FILES/DANA Schuyler Eldridge 1 Tommy Unger 2 Marcia Sahaya Louis 1 Amos Waterland 3 Margo Seltzer 3 Jonathan Appavoo 2 Ajay Joshi

More information

High Performance Computing with Accelerators

High Performance Computing with Accelerators High Performance Computing with Accelerators Volodymyr Kindratenko Innovative Systems Laboratory @ NCSA Institute for Advanced Computing Applications and Technologies (IACAT) National Center for Supercomputing

More information

Lecture 2: Performance

Lecture 2: Performance Lecture 2: Performance Today s topics: Technology wrap-up Performance trends and equations Reminders: YouTube videos, canvas, and class webpage: http://www.cs.utah.edu/~rajeev/cs3810/ 1 Important Trends

More information

Instruction Set And Architectural Features Of A Modern Risc Processor

Instruction Set And Architectural Features Of A Modern Risc Processor Instruction Set And Architectural Features Of A Modern Risc Processor PowerPC, as an evolving instruction set, has since 2006 been named Power 1 History, 2 Design features The result was the POWER instruction

More information

Statement of Research

Statement of Research On Exploring Algorithm Performance Between Von-Neumann and VLSI Custom-Logic Computing Architectures Tiffany M. Mintz James P. Davis, Ph.D. South Carolina Alliance for Minority Participation University

More information

Thermal-Aware 3D IC Physical Design and Architecture Exploration

Thermal-Aware 3D IC Physical Design and Architecture Exploration Thermal-Aware 3D IC Physical Design and Architecture Exploration Jason Cong & Guojie Luo UCLA Computer Science Department cong@cs.ucla.edu http://cadlab.cs.ucla.edu/~cong Supported by DARPA Outline Thermal-Aware

More information