Computing with Memory for Energy-Efficient Robust Systems

Transcription

1 Computing with Memory for Energy-Efficient Robust Systems

2

3 Somnath Paul Swarup Bhunia Computing with Memory for Energy-Efficient Robust Systems 123

4 Somnath Paul Intel Labs Hillsboro, OR, USA Swarup Bhunia Department of EECS Case Western Reserve University Cleveland, OH, USA ISBN ISBN (ebook) DOI / Springer New York Heidelberg Dordrecht London Library of Congress Control Number: Springer Science+Business Media New York 2014 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection with reviews or scholarly analysis or material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of this publication or parts thereof is permitted only under the provisions of the Copyright Law of the Publisher s location, in its current version, and permission for use must always be obtained from Springer. Permissions for use may be obtained through RightsLink at the Copyright Clearance Center. Violations are liable to prosecution under the respective Copyright Law. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein. Printed on acid-free paper Springer is part of Springer Science+Business Media (

5 Preface Energy-efficiency and reliability have emerged as major barriers to performance scalability in nanometer regime. The situation is particularly worse in the case of data-intensive workloads that require handling large volume of data from the domain of multimedia, informatics, data mining, and security applications. In the post-dennard scaling era, with technology scaling only providing linear reduction in energy compared to cubic reduction in previous technology nodes, there is a renewed thrust to investigate novel computing frameworks which can continue the roadmap for energy-efficient computing. Conventional computing solutions, namely, Von-Neumann machines like general-purpose processors are incapable of satisfying the energy-efficiency requirements for data-intensive applications. The attention has therefore shifted to non-von-neumann architectures such as graphics processing units (GPUs), field-programmable gate arrays (FPGAs), and other fixed-function and reconfigurable accelerators. A close investigation of the systemlevel performance and energy bottlenecks for data-intensive applications, however, reveals that managing off-chip data movement and memory access is as important as improving the efficiency of computing solutions on-chip. For these data-heavy applications, energy required in on-chip computation constitutes only a small fraction of the total energy consumption. It is primarily contributed by transportation of the data from off-chip memory to on-chip computing elements a limitation popularly referred to as the Von-Neumann bottleneck. In such a scenario, improving the compute energy through parallel processing or on-chip hardware acceleration brings minor improvements to the total energy required by the system. Technology trends suggest that while on-chip integration density increases exponentially, offchip bandwidth improves only at a linear rate over the same period of time. Hence, there is a critical need to develop novel hardware architecture that enables energyefficient computing for diverse data-intensive applications. To address this need, in this book, we propose a novel distributed and scalable memory-centric reconfigurable hardware architecture with associated software framework for application mapping. The computing model that is realized by the hardware architecture is referred to as memory-based computing or MBC. Hardware reconfigurable systems have been demonstrated to be more energy-efficient v

6 vi Preface compared to software programmable systems. However, they often incur large area, delay, and power overhead due to the presence of a programmable interconnect. The memory-centric reconfigurable computing model proposed in this work mitigates this overhead by optimally mapping the input application in a spatiotemporal fashion to the underlying computing elements of this framework. The framework therefore retains the flexibility and energy-efficiency of reconfigurable hardware, while making them more scalable at advanced technology nodes. The computing elements for the proposed hardware are generic enough to collectively map applications of arbitrary size and granularity. The book provides comprehensive description on the hardware and software architecture for memory-based computing and presents effective circuit, architecture, and software co-optimizations, which can collectively improve the energy-efficiency of the proposed memory-centric reconfigurable architecture. We also describe the functional verifications of the applications mapped to the MBC framework using both simulation and hardware emulation-based approaches. The proposed hardware solution can be used as a stand-alone reconfigurable framework or utilized as a co-processor for data-intensive applications. This coprocessor, referred to as Memory Array centric Hardware Accelerator (MAHA), offers the following distinctive advantages compared to alternative reconfigurable accelerator platforms, e.g., FPGA, Coarse-Grained Reconfigurable Architecture (CGRA), and GPU-based acceleration: (1) it exploits high-density and low accesstime/energy of nanoscale memory for storing both data and lookup table-based function responses; (2) the block-based memory architecture is used to create large number of parallel memory-centric configurable computing resources (memorycentric nano-processors, otherwise referred to as memory logic blocks or MLBs), which can be effective for massive data-parallel applications; (3) each MLB implements a distinct instruction-set architecture optimized for data-intensive applications including support for lookup operations of varying bitwidth, support for complex fused operations, and support for mixed-granular operations; and (4) an optimal hierarchical bus-based architecture that leverages on the nature of data movements across MLBs at different levels of hierarchy. Information about data movement is localized into MLBs by including it inside the instructions, which enables effective error protection. The project codevelops a software framework for application mapping into an array of MLBs from the control and data flow description of an application kernel. The memory inside each processing element can be malleably used to either perform computation (in the form of a LUT) or store data. The framework therefore benefits a wide range of compute and data-intensive workloads. Our investigations, included in this work, demonstrate that MBC achieves significant improvement in energy-efficiency over FPGA at the same technology node. The improvement is more pronounced at scaled nodes and enhances with low-power design techniques such as voltage scaling. We establish that drastic improvements in area and energyefficiency can be achieved when the input application is expressed as a PLA and the LUT-based implementation is replaced with a content-addressable memory (CAM)- based architecture. In the nanometer regime, memories are prone to parametric and

7 Preface vii runtime failures. In this work, we demonstrate that the effect of these parametric failures can be mitigated through intelligent resource allocation step in the MBC software flow. Runtime failures were shown to be tolerated using a novel reliability aware reconfigurable error-correction code (ECC) assignment approach. In this work, we have therefore critically addressed the challenges of improving the energyefficiency and reliability of the MBC framework itself. The benefits of memory-based computing becomes even more apparent for computing with off-chip nonvolatile memory arrays. This holds for conventional NAND Flash memory architecture as well as emerging nonvolatile memory technologies such as Spin Torque Transfer Memory (STTM), which are amenable to dense nonvolatile memory design. For data-intensive applications, this architecture exploits high-density nanoscale memory to process the data in situ and enables massively data-parallel processing for multimedia, signal processing, security, and informatics applications with ultra-low power dissipation. We further improve the energy-efficiency for in-memory computing by exploring the large design space for MAHA for a variety of data-intensive applications. This involves optimizing the nonvolatile memory organization (both NAND Flash and STTM), array of nano-processors and the interconnect hierarchy. The hardware architecture for each of the nano-processors has been optimized considering the commonly used operations for these data-intensive applications. For example, the support for fused operations as well as lookup operations of varying bitwidth enables efficient application mapping. Each MLB supports mixed-granular datapath operations including random logic functions (e.g., bit manipulation functions), which can help efficient implementation of many functions in informatics and multimedia domain. Furthermore, the routing requirements are embedded into the instructions, thus making reconfiguration process more efficient. Unlike the distributed and finegrained configuration of FPGA, it enables us to efficiently protect the configuration bits using error-correction code (ECC). As we show in our work, migrating the data-intensive workload from on-chip compute solutions to an off-chip NVM-based computing framework can significantly improve system-level energy-efficiency. The improvement is, however, not constant across the board, but varies with the nature of the application. We therefore propose a formulation based on simple application behavior to determine a priori if an application can benefit from off-chip in-memory computing. Compared to general-purpose processors, data-intensive workloads therefore enjoy X improvement when migrated to the off-chip MAHA framework. Existing accelerators, including conventional FPGA, CGRA, and GPU-based accelerators can deliver the desired performance benefit, but suffer from the energy barrier owing to off-chip memory access. The proposed NVMbased MAHA architecture can alleviate this bottleneck and deliver low-power system operation for data-intensive workloads. Memory-based computing blends the benefits of spatial and temporal computing and also the goodness of software and hardware reconfigurable frameworks in a single fabric. The fact that it is memory-centric makes it particularly useful for emerging data-heavy applications like informatics and amenable to implementation using emerging dense nonvolatile memories. We also believe the new research

8 viii Preface direction put forward in this work will excite the target readership consisting of students, researchers, and practitioners and they will benefit from it. We believe that the content will also remain highly relevant in future since current technology trends and emerging technology solutions favor the case of in-memory computing frameworks. We are very grateful to all students of Nanoscale Research Lab at Case Western Reserve University for their help and support in the course of the MBC project. We also acknowledge the contribution of Prof. Saibal Mukhopadhyay and his students from the GREEN Lab at the School of Electrical and Computer Engineering, Georgia Institute of Technology. We also remain sincerely thankful to Springer, USA and all their publication staff. Hillsboro, OR, USA Cleveland, OH, USA Somnath Paul Swarup Bhunia

9 Contents Part I Introduction 1 Challenges in Computing for Nanoscale Technologies End of Dennard Scaling Sustaining Reliability of Operation Addressing Energy-Efficiency for Data-Intensive Workloads... 6 References A Survey of Computing Architectures Von-Neumann and Non-Von-Neumann Architectures Reconfigurable Computing: State of the Art Challenges in Reconfigurable Hardware Design References Motivation for a Memory-Based Computing Hardware Motivation for a New Computing Model Proposed Solution and Benefits References Part II Memory Based Computing: Overview 4 Key Features of Memory-Based Computing Computing in Memory Versus Computing with Memory Highlights of Memory-Based Computing Framework References Overview of Hardware and Software Architectures Overview of Hardware Organization and Software Flow Distinction with Existing Hardware Frameworks References ix

10 x Contents 6 Application of Memory-Based Computing Motifs Contexts Domains References Part III Hardware Framework 7 A Memory Based Generic Reconfigurable Framework A Generic Reconfigurable Framework A Generic Reconfigurable Framework with MBC References MAHA Hardware Architecture A Typical Memory Organization Instrumenting Memory for Hardware Acceleration References Part IV Software Framework 9 Application Analysis Application Description Using a CDFG Decomposition Fusion References Application Mapping to MBC Hardware Resource-Aware Scheduling Packing of Partitions to Multi-MLB Framework Placement Report Generation Bitfile Generation and Functional Validation Design Space Exploration References Part V MBC Design Space Exploration 11 Design Space Exploration for MBC Based Generic Reconfigurable Computing Framework Benchmarks and Experimental Setup MLB Architecture Exploration Comparison with a Fully-Spatial Reconfigurable Architecture References

11 Contents xi 12 Design Space Exploration for MAHA Framework Benchmarks and Simulation Setup MLB Architecture Exploration References Preferential Memory Design for MBC Frameworks Case Study I: SRAM Array Case Study II: STTRAM Array Optimization in Application Mapping References PLA Based Application Mapping in MBC Function Representation as PLA Function Representation Using CAM Based MBC Framework Power and Performance of a CAM Based MBC Framework References Part VI Off-Chip Hardware Acceleration Using MBC 15 Background and Motivation Von-Neumann Bottleneck Mitigating Von Neumann Bottleneck Through In-Memory Computing References Off-Chip MAHA Using NAND Flash Technology Overview of Current Flash Organization Modifications to Flash Array Organization ECC Computation and Modes of Operation References Improvement in Energy-Efficiency with Off-Chip MAHA Design Space Exploration for Off-Chip MAHA Energy and Performance for Mapped Applications Hardware Emulation Based Validation References Part VII Improving Reliability of Operations in MBC 18 Mitigating the Effect of Parametric Failures in MBC Frameworks Effect of Parameter Variations on QoS A Variation-Aware Preferential Mapping Approach Preferential Memory Design References

12 xii Contents 19 Mitigating the Effect of Runtime-Failures in MBC Frameworks Impact of Parameter Variation on Runtime Failures Reliability-Driven ECC Allocation for Multiple Bit Error Resilience Experimental Results References Summary

13 Acronyms CAM CPU CDFG CGRA CMOS DRAM EDP FPGA FeRAM GPP GPU LUT MBC MLB MAHA MRAM NVM PE PI PCM PLA PTM RRAM SRAM STTM VLSI Content-addressable memory Central processing unit Control data flow graph Coarse-grained reconfigurable array Complementary metal-oxide semiconductor Dynamic random-access memory Energy-delay product Field-programmable gate array Ferro-electric random access memory General purpose processor Graphics processing unit Lookup table Memory-based computing Memory logic block Malleable hardware accelerator Magnetoresistive random access memory Non-volatile memory Processing element Programmable interconnect Phase change memory Programmable logic array Predictive technology model Resistive random access memory Static random-access memory Spin-transfer torque memory Very-large-scale integration xiii