HW/SW Programmable Engine:

Transcription

1 HW/SW Programmable Engine: Domain Specific Architecture for Project Everest Juanjo Noguera, Goran Bilski, Jan Langer, Baris Ozgul, Tim Tuan, David Clarke, Peter McColgan, Sneha Date, Zachary Dickman, Pedro Duarte, Stephan Munz, Jose Marques, Sridhar Subramanian, Saurabh Mathur, Malav P. Shah, Chris Dick, Paul Newson, Kaushik Barman, Ramon Uribe, Helen Tarn, Engin Tunali, Richard Walke, Vinod Kathail, Shail Aditya Gupta, Akella Sastry, Mukund Sivaraman, Rishi Surendran, Abraham Lee, Chia-Jui Hsu, Kumud Bhandari, Jack Lo, Kristof Denolf, Phil James-Roxby, Samuel Bayliss, Kees Vissers, Ralph Wittig, Gaurav Singh 21 st August 2018

2 Heterogenous Computing Architecture Everest Heterogeneous Architecture Application-Level Performance Enabled by SW Programmable Engine Processing System SW Programmable Engine Programmable Logic Compute Efficiency Reduced Power Programmable I/O (GT, AMS) 20x Xilinx 7nm Everest 4x Xilinx 16nm UltraScale+ 40% Less Power ML Inference 5G Wireless Power >> 2 7nm Everest is the First Product to Integrate this New Architecture Tape Out in 2018

3 Everest Architecture Block Diagram SW Programmable Engine Domain Specific Architecture Hardened 7nm technology Throughput-oriented, low-latency Processing System Application Processor SW Programmable Engine SW PE SW PE SW PE SW PE SW PE SW PE I/O (GT, AMS) Transceivers PCIe Programmable Logic Soft logic Real-Time Processor OCM Programmable Logic DDR HBM Flexibility LUT BRAM Custom memory hierarchy SMMU DSP URAM AMS Chips with Different Implementations of the Architecture will be Announced Shortly >> 3

4 Market Requirements and Trends: Data Center Increasing Compute Lower Latency Requirements Heterogeneous Workloads ML Becoming an Essential Part of Data Processing Video + ML Genomics + ML Risk Modelling + ML Database + ML Network IPS + ML Storage + ML Source: Canziani et. al, An Analysis of Deep Neural Network Models for Practical Applications >> 4

5 ML Inference on Heterogenous Architecture Convolutions Fully Connected Layers Pooling Activations single depth slice X y i = 0 y y i = x i x y i = a i x i y y i = x i Y ReLU ReLU/PReLU Video Genomics Storage Database Network IPS Risk modeling Processing System SW Programmable Engine Programmable Logic I/O (GT, AMS) Feature Map Data Volume* Custom Hierarchy *Figure credit: >> 5

6 Packet Processing and Wired Backhaul Higher Layer Processing Baseband Processing Switching Beam Forming & MMIO + Some Baseband Transforms Digital Radio ADC / DAC Analogue Radio Antenna Array Market Requirements and Trends: Wireless 5G 5G Complexity is 100X that of 4G Still Evolving Standard New Technologies in 5G Massive MIMO Multiple antenna, frequency bands Changing functional partitioning ETRI RWS , 5G Vision and Enabling Technologies: ETRI Perspective 3GPP RAN Workshop Phoenix, Dec Transport & CTRL L2 L7 Modulation & FEC IQ Switch Linear Algebra ifft/ FFT DUC, CFR, DPD, DDC PA, LNA, Diplexer >> 6

7 Packet Processing and Wired Backhaul Higher Layer Processing Baseband Processing Switching Beam Forming & MMIO + Some Baseband Transforms Digital Radio ADC / DAC Analogue Radio Antenna Array 5G Wireless on Heterogenous Architecture 5G Wireless Infrastructure (i.e., base-station) Digital Radio with ADC/DAC Compute Maps to PE Mapping Example CPRI DPD Update DUC DPD AMS Control Maps to PS Processing System DPD Update SW Programmable Engine DUC DPD Programmable Logic I/O AMS CPRI 1: DUC: Digital Up Converter 2: DPD: Digital Pre-Distortion 3: AMS: Analog Mix-Signal (ADC/DAC) 4: CPRI: Common Public Radio Interface >> 7 I/O Maps to PL

8 Architecture Overview

9 Tile-Based Architecture Non-Blocking Interconnect Up to 200+ GB/s bandwidth per tile PS I/O PL Interconnect Local Multi-bank implementation Shared across neighbor cores Local ISA-based Processor ML Extensions ISA-based Processor Programmable (e.g., C/C++) Data Mover 5G Wireless Extensions >> 9 Cascade Interface Partial results to next core Data Mover Non-neighbor data communication Integrated synchronization primitives

10 Tile-Based Architecture (Continued) PS I/O Massive multi-core engines Increase in compute, memory and communication bandwidth PL Modular and scalable architecture Instantiate multiple tiles for more compute 10 s-100 s Instances* *Device Specific Distributed memory hierarchy Maximize memory bandwidth Architectural Focus: Deterministic Performance & Low Latency >> 10

11 Data Movement Architecture: Examples (1/2) 1 Dataflow Pipeline Overlap Compute and Communication PE 0 B0 B1 PE 1 B2 B3 PE 2 Comm Compute Comm Compute Comm Compute 2 Dataflow Graph 3 Streaming Multicast Mem PE Mem PE PE 0 PE 1 PE Mem PE Mem PE PE 2 Mem PE Mem PE PE 3 >> 11

12 Data Movement Architecture: Examples (2/2) 4 Non-neighbor communication PE 0 PE 1 Physical Mapping 5 Cascade streaming PE 1 PE 2 PE 3 PE 0 PE 2 PE 3 >> 12

13 Device Integration PS PL I/O Up to TB/s of bandwidth between PE and PL * *Device Specific CDC CDC CDC Programmable Logic Clock-domain crossing (CDC) between PL & PE clocks DSP LUT BRAM URAM >> 13

14 Data Movement Architecture: Examples PL Integration PL Accelerator PL Accelerator Pre/Post Processing Co-processing Mem PE 0 PE 1 Mem Custom On-chip Hierarchy Custom On-Chip Hierarchy Leverages on-chip memory resources in PL (e.g., BRAM, URAM) High on-chip memory bandwidth and lower memory access latency >> 14

15 Programming Environment

16 Programming Environment Full Device Programming Solution Fully Integrated with Tools for Other Everest Fabrics High Level Programming Languages (e.g., C/C++) C/C++ C/C++ DPD Update CPRI DUC DPD AMS Device-Level Programming SW Programmable Engine PS Programmable Logic I/O CPU Compiler SW PE Compiler PL Compiler >> 16

17 Complete Development Stack C/C++ C/C++ Device-Level Programming CPU Compiler SW PE Compiler PL Compiler 1 Full SW Programming Tool Chain (Single- and Multi-) 2 Performance-Optimized Libraries (Examples) 3 Run-Time (Examples) IDE Wireless 4G/5G Library Error Management Compiler Debugger ML (BLAS) Library Management Boot + Configuration Performance Analysis Vision Library Power/Thermal Management >> 17

18 High-level Compiler for ML Inference Users Working at ML Framework Level Users don t directly program PE Deep Learning Frameworks Seamless Migration Same experience as 16nm devices Current models continue to work Xilinx ML Compiler Xilinx 16nm UltraScale+ Xilinx Everest w/ PE >> 18

19 Application Results

20 Kernel-Level Performance: ML and Wireless 5G Processor Efficiency Peak Theoretical Performance 95% 80% 98% ML Convolutions FFT DPD Block-based Matrix Multiplication (32 64) (64 32) 1024-pt FFT/iFFT Volterra-based forward-path DPD High Compute Efficiency for Key Functions in ML and Wireless 5G >> 20

21 Programmable Engine: Summary Image Classification (GoogleNet CNN) Sub 1ms Latency 20x Application-level Performance Enabled by SW Programmable Engine Massive MIMO Radio (DUC, DDC, CFR, DPD) ML Inference 5G Wireless Power Reduction Xilinx Everest 7nm TX Sample Rate Up to 2Gsps 4x Xilinx UltraScale+ 16nm 40% Less Power Compute Efficiency Domain specific architecture Increase in compute density Xilinx 7nm Everest Heterogenous Architecture High-throughput, low-latency PL flexibility Custom memory hierarchy Multiple Applications ML Inference for Cloud DC Wireless 5G: Radio, Baseband ADAS/AD embedded vision Wired: DOCSIS cable access SW Programmable SW programmable (e.g., C/C++) Compile, execute, debug Optimized software libraries >> 23

22 KEYNOTE Adaptable Intelligence: The Next Computing Era Victor Peng, Xilinx CEO AUGUST 21 A.M. Connect Learn Share XDF connects software developers and system designers to the deep expertise of Xilinx engineers, partners, and industry leaders. Silicon Valley October 1-2 Learn More >> 22

23 Adaptable. Intelligent.