Emergence of the Memory Centric Architectures

Transcription

1 Emergence of the Memory Centric Architectures Balint Fleischer Chief Scientist

2 AI is Everywhere Business Consumer Advising the CEO External Sensing: Market trends, Competitive environment, Customer sentiment, Demand Internal Sensing: Production Systems, Supply chain, Asset utilization, Employee Moral Product Recommendation Personalization Personal Assistants New customer experiences Understanding intentions Anticipating needs Health Assistant Sentiment Analytics Fraud Detection Medical Diagnosis Object recognition Chatbots Threat Detection Preventative Maintenance ediscovery Language Translation Smart City Management Smart Manufacturing Customer Service Etc., Direction guide Communication Assistant Shopping Assistant Entertainment Education Transportation Etc.,

3 Real Time AI is Putting a Pressure on Platform Scaling <Seconds Minutes Hours/Days Operation Personalization Quality Control Alerting, Routing Network Mgmt Etc. Efficiency Failure Prediction Yield Mgmt Service Pricing Traffic Congestion Etc. Business Optimization Asset utilization Product mix Customer sentiment Competitive trends Etc. Accuracy & Response time Impacts Business

4 AI is Impacting Enterprise IT Architecture Next generation Enterprise IT User s view Sensing Layer Enterprise Goals To identify connections between events, people and trends Discover new insights Uncovering breakthroughs and predicting trends Enabling new customer experience via service personalization Reinventing business models and operations AI Enhanced Applications Enterprise Data Base

5 AI Computing is Challenging Optimized for Estimation Probabilistic calculations DNN, ML algorithms etc., Data Intensive New Algorithms, New Architecture AI Processing Perf Demand GAP Classic CPU Perf Roadmap Optimized for Accuracy Logical operations Arithmetic operations Data Store and Retrieve * Applied or narrow AI

6 Processor Research to Improve AI Performance Processing & Circuits Reducing Power and Cost New process technologies Near Threshold Switching DL Optimized Architectures Bio Inspired systems Reducing Resource Requirement Domain specific Architectures SW optimizations Compilers Energy efficient ConvNet** Binary Weight Networks* XNOR Networks* Compression, Pruning Algorithms & Architectures Platform Architecture Reducing Energy of Data Movement Memory Hierarchies Computing in Memory Low Latency Networks XNOR-Net: ImageNet Classification using Binary Convolutional networks, Mohammed Rastegari, Et al 2016 ** Energy-Efficient ConvNets Through Approximate Computing, Bert Moons, Et al. 2016

7 Roadmap for a Faster, more efficient AI processor ~100MMAC/S/mW ~1MMAC/S/pW Today Best General Purpose Systems Digital Analytics Systems Advanced AI engines NeuroMemristive Systems Neuromorphic Systems Bio Inspired Computing CPU based X86 ARM Power GPU based NVIDIA Xeon PHI AMD FPGA based Teradeep Altera Xilinx DeePhi ASIC based TPU Wave Graphcore Movidus EyeRiss, NeuFlow Neurostream Neurocube Others Note: Company and project names for reference only. No implied endorsement by Huawei Research examples: Vector-Matrix Multipliers MultiCore Systems Etc. Prime, Isaac Research examples: Minitaur, SpiNNaker, TrueNorth. NeuroGrid, Neurocluster, BrainScales, ROLLS, others

8 Platform Research to Improve AI Performance Processing & Circuits Reducing Power and Cost New process technologies Near Threshold Switching DL Optimized Architectures Bio Inspired systems Reducing Resource Requirement Domain specific Architectures SW optimizations Compilers Energy efficient ConvNet** Binary Weight Networks* XNOR Networks* Compression, Pruning Algorithms & Architectures Platform Architecture Reducing Energy of Data Movement Memory Hierarchies Computing in Memory Low Latency Networks XNOR-Net: ImageNet Classification using Binary Convolutional networks, Mohammed Rastegari, Et al 2016 ** Energy-Efficient ConvNets Through Approximate Computing, Bert Moons, Et al. 2016

9 Reducing Latency to improve Performance Use case: Large Platform Memory Memory Hierarchy Technology choices CPU L1-L3 cache On Die/On Package Memory DRAM, SCM NVDIMM-P, NVDIMM-N NVMe drives, NVDIMM-F Value Very Low Latency <30nsec Extreme High BW Low Latency <1 usec, Very High BW Byte access, Maybe Non Volatile Medium Latency <100usec, Medium BW, Block access Use case: Enterprise Data Base NVMe drives, SSD Moderate latency across network>1msec, Moderate BW across network Data Management tools and architecture features are key Balint Fleischer

10 Optimizing Processor Data Movements Data Proximity Co Locating Data and Processing Server Memory Application In Memory Compute Integrating Processing into memory array Processing Element In Memory Data Store CPU Low Latency <1usec Medium/High BW Large capacity Improved energy efficiency (local data access) General Purpose CPU Very Low Latency <<1 usec Extreme High BW Limited capacity Very good energy efficiency Specialized, Embedded processing Hybrid Memory Cube based concept

11 Supporting Ubiquitous AI processing Unifying CPU+AI into One Memory Centric Design for Scalability Large On Package Memory In Memory Compute For max BW to engine Data Center Fabric Memory Hub based architecture Distributed Memory to Memory Protocols In Memory Distributed Data Store support Local Data Store support

12 Emerging Memory Technologies to Support Scalability Note: Product names for reference only. No implied endorsement by Huawei DRAM based HBM2, HMC, DDR5 NAND based 3DNAND, Z-NAND DIMM based NVDIMM-F; NVDIMM-N; NVDIMM-P New Memory types 3DXpoint, NRAM, MRAM, ReRAM

13 Data Center for AI based Workloads NVM based architecture Improved scaling performance Enhanced response time AI workload Optimized Server Platform to lower processing cost, increase performance, improve prediction accuracy and lower energy consumption Persistent Enterprise Data Base Large Platform Memory for Data Proximity and Distributed In Memory Data Store High Bisectional BW Non Blocking Fabric to support large data streams predictable performance provisioning flexibility Special pool (BMP) for Low latency Compute Latency optimized protocols for improving scalability predictable performance provisioning flexibility

14 Thank you 11/26/2017 Balint Fleischer Huawei 14