THE PROGRAMMER S GUIDE TO THE APU GALAXY. Phil Rogers, Corporate Fellow AMD

Size: px

Start display at page:

Download "THE PROGRAMMER S GUIDE TO THE APU GALAXY. Phil Rogers, Corporate Fellow AMD"

Malcolm Nichols
5 years ago
Views:

1 THE PROGRAMMER S GUIDE TO THE APU GALAXY Phil Rogers, Corporate Fellow AMD

2 THE OPPORTUNITY WE ARE SEIZING Make the unprecedented processing capability of the APU as accessible to programmers as the CPU is today. 2 The Programmer s Guide to the APU Galaxy June 2011

3 OUTLINE The APU today and its programming environment The future of the heterogeneous platform AMD Fusion System Architecture Roadmap Software evolution A visual view of the new command and data flow 3 The Programmer s Guide to the APU Galaxy June 2011

APU: ACCELERATED PROCESSING UNIT The APU has arrived and it is a great advance over previous platforms Combines scalar processing on CPU with parallel processing on the GPU and high bandwidth access

4 APU: ACCELERATED PROCESSING UNIT The APU has arrived and it is a great advance over previous platforms Combines scalar processing on CPU with parallel processing on the GPU and high bandwidth access to memory How do we make it even better going forward? Easier to program Easier to optimize Easier to load balance Higher performance Lower power 4 The Programmer s Guide to the APU Galaxy June 2011

LOW POWER E-SERIES AMD FUSION APU: ZACATE E-Series APU 2 x86 Bobcat CPU cores Array of Radeon Cores Discrete-class DirectX 11 performance 80 Stream Processors 3rd Generation Unified Video

5 LOW POWER E-SERIES AMD FUSION APU: ZACATE E-Series APU 2 x86 Bobcat CPU cores Array of Radeon Cores Discrete-class DirectX 11 performance 80 Stream Processors 3rd Generation Unified Video Decoder PCIe Gen2 Single-channel W TDP Performance: Up to 8.5GB/s System Memory Bandwidth Up to 90 Gflop of Single Precision Compute 5 The Programmer s Guide to the APU Galaxy June 2011

6 TABLET Z-SERIES AMD FUSION APU: DESNA Z-Series APU 2 x86 Bobcat CPU cores Array of Radeon Cores Discrete-class DirectX 11 performance 80 Stream Processors 3rd Generation Unified Video Decoder PCIe Gen2 Single-channel W TDP w/ Local Hardware Thermal Control Performance: Up to 8.5GB/s System Memory Bandwidth Suitable for sealed, passively cooled designs 6 The Programmer s Guide to the APU Galaxy June 2011

MAINSTREAM A-SERIES AMD FUSION APU: LLANO A-Series APU Up to four x86 CPU cores AMD Turbo CORE frequency acceleration Array of AMD Radeon Cores Discrete-class DirectX 11 performance 3rd Generation

7 MAINSTREAM A-SERIES AMD FUSION APU: LLANO A-Series APU Up to four x86 CPU cores AMD Turbo CORE frequency acceleration Array of AMD Radeon Cores Discrete-class DirectX 11 performance 3rd Generation Unified Video Decoder Blu-ray 3D stereoscopic display PCIe Gen2 Dual-channel DDR3 45W TDP Performance: Up to 29GB/s System Memory Bandwidth Up to 500 Gflops of Single Precision Compute 7 The Programmer s Guide to the APU Galaxy June 2011

win over time DirectX SW developers want their applications to run on multiple

8 COMMITTED TO OPEN STANDARDS AMD drives open and de-facto standards Compete on the best implementation Open standards are the basis for large ecosystems Open standards always win over time DirectX SW developers want their applications to run on multiple platforms from multiple hardware vendors 8 The Programmer s Guide to the APU Galaxy June 2011

9 Single-thread Performance Throughput Performance rn Application Performance A NEW ERA OF PROCESSOR PERFORMANCE Single-Core Era Multi-Core Era Heterogeneous Systems Era Enabled by: Moore s Law Voltage Scaling Constrained by: Power Complexity Enabled by: Moore s Law SMP architecture Constrained by: Power Parallel SW Scalability Enabled by: Abundant data parallelism Power efficient GPUs Temporarily Constrained by: Programming models Comm.overhead Assembly C/C++ Java pthreads OpenMP / TBB Shader CUDA OpenCL!!!? we are here we are here we are here Time Time (# of processors) Time (Data-parallel exploitation) 9 The Programmer s Guide to the APU Galaxy June 2011

Architecture Maturity & Programmer Accessibility Poor Excellent EVOLUTION OF HETEROGENEOUS COMPUTING Standards s Era

Adventurous programmers Exploit early programmable shader cores in the GPU Make your program look like graphics to the

types Multiple address spaces with explicit data movement Specialized work queue based structures Kernel mode dispatch

Parallel programs User mode dispatch Pre-emption and context switching See Herb Sutter s Keynote tomorrow for a cool

10 Architecture Maturity & Programmer Accessibility Poor Excellent EVOLUTION OF HETEROGENEOUS COMPUTING Standards s Era Architected Era AMD Fusion System Architecture GPU Peer Processor Proprietary s Era Graphics & Proprietary -based APIs Adventurous programmers Exploit early programmable shader cores in the GPU Make your program look like graphics to the GPU CUDA, Brook+, etc OpenCL, DirectCompute -based APIs Expert programmers C and C++ subsets Compute centric APIs, data types Multiple address spaces with explicit data movement Specialized work queue based structures Kernel mode dispatch Mainstream programmers Full C++ GPU as a co-processor Unified coherent address space Task parallel runtimes Nested Data Parallel programs User mode dispatch Pre-emption and context switching See Herb Sutter s Keynote tomorrow for a cool example of plans for the architected era! The Programmer s Guide to the APU Galaxy June 2011

FSA FEATURE ROADMAP Physical Integration Optimized Platforms Architectural Integration System

CPU and GPU GPU compute context switch Unified Memory Controller User mode schedulng GPU uses

Manufacturing Technology Bi-Directional Power Mgmt between CPU and GPU Fully coherent memory

11 FSA FEATURE ROADMAP Physical Integration Optimized Platforms Architectural Integration System Integration Integrate CPU & GPU in silicon GPU Compute C++ support Unified Address Space for CPU and GPU GPU compute context switch Unified Memory Controller User mode schedulng GPU uses pageable system memory via CPU pointers GPU graphics pre-emption Quality of Service Common Manufacturing Technology Bi-Directional Power Mgmt between CPU and GPU Fully coherent memory between CPU & GPU Extend to Discrete GPU 11 The Programmer s Guide to the APU Galaxy June 2011

partners to join us, in all areas Hardware companies Operating Systems Tools and Middleware

12 AMD FUSION SYSTEM ARCHITECTURE AN OPEN PLATFORM Open Architecture, published specifications FSAIL virtual ISA FSA memory model FSA dispatch ISA agnostic for both CPU and GPU Inviting partners to join us, in all areas Hardware companies Operating Systems Tools and Middleware Applications FSA review committee planned 12 The Programmer s Guide to the APU Galaxy June 2011

FSA INTERMEDIATE LAYER - FSAIL FSAIL is a virtual ISA for parallel programs Finalized to ISA by a JIT compiler or Finalizer Explicitly parallel Designed for data parallel programming Support for

13 FSA INTERMEDIATE LAYER - FSAIL FSAIL is a virtual ISA for parallel programs Finalized to ISA by a JIT compiler or Finalizer Explicitly parallel Designed for data parallel programming Support for exceptions, virtual functions, and other high level language features Syscall methods GPU code can call directly to system services, IO, printf etc Debugging support 13 The Programmer s Guide to the APU Galaxy June 2011

14 FSA MEMORY MODEL Designed to be compatible with C++0x, Java and.net Memory ls Relaxed consistency memory model for parallel compute performance Loads and stores can be re-ordered by the finalizer Visibility controlled by: Load.Acquire, Store.Release Fences Barriers 14 The Programmer s Guide to the APU Galaxy June 2011

15 Stack FSA Software Stack Apps AppsApps Apps AppsApps Apps Apps Apps Apps Apps Apps Domain Libraries FSA Domain Libraries OpenCL 1.x, DX Runtimes, User s Graphics Kernel FSA JIT Task Queuing Libraries FSA Runtime FSA Kernel Hardware - APUs, CPUs, GPUs AMD user mode component AMD kernel mode component All others contributed by third parties or AMD 15 The Programmer s Guide to the APU Galaxy June 2011

OPENCL AND FSA FSA is an optimized platform architecture for OpenCL Not an alternative to OpenCL OpenCL on FSA will benefit from Avoidance of wasteful copies Low latency dispatch Improved memory

16 OPENCL AND FSA FSA is an optimized platform architecture for OpenCL Not an alternative to OpenCL OpenCL on FSA will benefit from Avoidance of wasteful copies Low latency dispatch Improved memory model Pointers shared between CPU and GPU FSA also exposes a lower level programming interface, for those that want the ultimate in control and performance Optimized libraries may choose the lower level interface 16 The Programmer s Guide to the APU Galaxy June 2011

TASK QUEUING RUNTIMES Popular pattern for task and data parallel programming on SMP systems today Characterized by: A work queue per core Runtime library that divides large loops into tasks and

17 TASK QUEUING RUNTIMES Popular pattern for task and data parallel programming on SMP systems today Characterized by: A work queue per core Runtime library that divides large loops into tasks and distributes to queues A work stealing runtime that keeps the system balanced FSA is designed to extend this pattern to run on heterogeneous systems 17 The Programmer s Guide to the APU Galaxy June 2011

18 TASK QUEUING RUNTIME ON CPUS Work Stealing Runtime Q Q Q Q CPU Worker CPU Worker CPU Worker CPU Worker X86 CPU X86 CPU X86 CPU X86 CPU CPU Threads GPU Threads Memory 18 The Programmer s Guide to the APU Galaxy June 2011

19 TASK QUEUING RUNTIME ON THE FSA PLATFORM Work Stealing Runtime Q Q Q Q Q CPU Worker CPU Worker CPU Worker CPU Worker GPU Manager X86 CPU X86 CPU X86 CPU X86 CPU Radeon GPU CPU Threads GPU Threads Memory 19 The Programmer s Guide to the APU Galaxy June 2011

X86 CPU X86 CPU Fetch and Dispatch CPU Threads GPU Threads Memory S I M D S I

20 TASK QUEUING RUNTIME ON THE FSA PLATFORM Work Stealing Runtime Q Q Q Q Q CPU Worker CPU Worker CPU Worker CPU Worker GPU Manager Memory X86 CPU X86 CPU X86 CPU X86 CPU Fetch and Dispatch CPU Threads GPU Threads Memory S I M D S I M D S I M D S I M D S I M D 20 The Programmer s Guide to the APU Galaxy June 2011

21 FSA SOFTWARE EXAMPLE - REDUCTION float foo(float); float myarray[ ]; Task<float, ReductionBin> sum([myarray](indexrange<1> index) [[device]] { float sum = 0.; for (size_t I = index.begin(); I!= index.end(); i++) { sum += foo(myarray[i]); } return float; }); float sum = task.enqueuewithreduce(partition<1, Auto>(1920), [] (int x, int y) [[device]] { return x+y; }, 0.); 21 The Programmer s Guide to the APU Galaxy June 2011

compute dispatch improves tomorrow under FSA 22

22 HETEROGENEOUS COMPUTE DISPATCH How compute dispatch operates today in the driver model How compute dispatch improves tomorrow under FSA 22 The Programmer s Guide to the APU Galaxy June 2011

23 TODAY S COMMAND AND DISPATCH FLOW Command Flow Data Flow Application A Direct3D User Soft Queue Kernel Command Buffer DMA Buffer A GPU HARDWARE Hardware Queue 23 The Programmer s Guide to the APU Galaxy June 2011

24 TODAY S COMMAND AND DISPATCH FLOW Command Flow Data Flow Application A Direct3D User Soft Queue Kernel Command Buffer DMA Buffer Command Flow Data Flow Application B Direct3D User Soft Queue Kernel A GPU HARDWARE Command Buffer DMA Buffer Command Flow Data Flow Application C Direct3D User Soft Queue Kernel Hardware Queue Command Buffer DMA Buffer 24 The Programmer s Guide to the APU Galaxy June 2011

25 TODAY S COMMAND AND DISPATCH FLOW Command Flow Data Flow Application A Direct3D User Soft Queue Kernel Command Buffer DMA Buffer Application B Command Flow Direct3D User Data Flow Soft Queue Kernel A C B A B GPU HARDWARE Command Buffer DMA Buffer Command Flow Data Flow Application C Direct3D User Soft Queue Kernel Hardware Queue Command Buffer DMA Buffer 25 The Programmer s Guide to the APU Galaxy June 2011

26 TODAY S COMMAND AND DISPATCH FLOW Command Flow Data Flow Application A Direct3D User Soft Queue Kernel Command Buffer DMA Buffer Application B Command Flow Direct3D User Data Flow Soft Queue Kernel A C B A B GPU HARDWARE Command Buffer DMA Buffer Command Flow Data Flow Application C Direct3D User Soft Queue Kernel Hardware Queue Command Buffer DMA Buffer 26 The Programmer s Guide to the APU Galaxy June 2011

FUTURE COMMAND AND DISPATCH FLOW Application C C C C C C Application codes to the hardware User mode queuing Application B Optional Dispatch Buffer Hardware Queue B B B GPU HARDWARE Hardware

27 FUTURE COMMAND AND DISPATCH FLOW Application C C C C C C Application codes to the hardware User mode queuing Application B Optional Dispatch Buffer Hardware Queue B B B GPU HARDWARE Hardware scheduling Low dispatch times No APIs Application A Hardware Queue A A A No Soft Queues No User s No Kernel Transitions No Overhead! Hardware Queue 27 The Programmer s Guide to the APU Galaxy June 2011

28 FUTURE COMMAND AND DISPATCH CPU <-> GPU Application / Runtime CPU1 CPU2 GPU 28 The Programmer s Guide to the APU Galaxy June 2011

29 FUTURE COMMAND AND DISPATCH CPU <-> GPU Application / Runtime CPU1 CPU2 GPU 29 The Programmer s Guide to the APU Galaxy June 2011

30 FUTURE COMMAND AND DISPATCH CPU <-> GPU Application / Runtime CPU1 CPU2 GPU 30 The Programmer s Guide to the APU Galaxy June 2011

31 FUTURE COMMAND AND DISPATCH CPU <-> GPU Application / Runtime CPU1 CPU2 GPU 31 The Programmer s Guide to the APU Galaxy June 2011

WHERE ARE WE TAKING YOU? Switch the compute, don t move the data!

efficient program model All cores capable, with performance

sets Support for task based programming models Solutions for all

32 WHERE ARE WE TAKING YOU? Switch the compute, don t move the data! Every processor now has serial and parallel cores Simple and efficient program model All cores capable, with performance differences Platform Design Goals Easy support of massive data sets Support for task based programming models Solutions for all platforms Open to all 32 The Programmer s Guide to the APU Galaxy June 2011

33 THE FUTURE OF HETEROGENEOUS COMPUTING IS HERE The architectural path for the future is clear Programming patterns established on Symmetric Multi-Processor (SMP) systems migrate to the heterogeneous world An open architecture, with published specifications and an open source execution software stack Heterogeneous cores working together seamlessly in coherent memory Low latency dispatch No software fault lines 33 The Programmer s Guide to the APU Galaxy June 2011

35 Disclaimer & Attribution The information presented in this document is for informational purposes only and may contain technical inaccuracies, omissions and typographical errors. The information contained herein is subject to change and may be rendered inaccurate for many reasons, including but not limited to product and roadmap changes, component and motherboard version changes, new model and/or product releases, product differences between differing manufacturers, software changes, BIOS flashes, firmware upgrades, or the like. There is no obligation to update or otherwise correct or revise this information. However, we reserve the right to revise this information and to make changes from time to time to the content hereof without obligation to notify any person of such revisions or changes. NO REPRESENTATIONS OR WARRANTIES ARE MADE WITH RESPECT TO THE CONTENTS HEREOF AND NO RESPONSIBILITY IS ASSUMED FOR ANY INACCURACIES, ERRORS OR OMISSIONS THAT MAY APPEAR IN THIS INFORMATION. ALL IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE ARE EXPRESSLY DISCLAIMED. IN NO EVENT WILL ANY LIABILITY TO ANY PERSON BE INCURRED FOR ANY DIRECT, INDIRECT, SPECIAL OR OTHER CONSEQUENTIAL DAMAGES ARISING FROM THE USE OF ANY INFORMATION CONTAINED HEREIN, EVEN IF EXPRESSLY ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. AMD, the AMD arrow logo, and combinations thereof are trademarks of Advanced Micro Devices, Inc. All other names used in this presentation are for informational purposes only and may be trademarks of their respective owners. OpenCL and the OpenCL logo are trademarks of Apple, Inc. and are used by permission by Khronos Advanced Micro Devices, Inc. All Rights Reserved. 35 The Programmer s Guide to the APU Galaxy June 2011

HETEROGENEOUS SYSTEM ARCHITECTURE: PLATFORM FOR THE FUTURE

HETEROGENEOUS SYSTEM ARCHITECTURE: PLATFORM FOR THE FUTURE Haibo Xie, Ph.D. Chief HSA Evangelist AMD China OUTLINE: The Challenges with Computing Today Introducing Heterogeneous System Architecture (HSA)