Embedded Systems. Series Editors

Transcription

1 Embedded Systems Series Editors Nikil D. Dutt, Department of Computer Science, Zot Code 3435, Donald Bren School of Information and Computer Sciences, University of California, Irvine, CA , USA Peter Marwedel, TU Dortmund, Informatik 12, Otto-Hahn-Str. 16, Dortmund, Germany Grant Martin, Tensilica Inc., Scott Blvd., Santa Clara, CA 95054, USA For further volumes:

2 Benny Akesson Kees Goossens Memory Controllers for Real-Time Embedded Systems Predictable and Composable Real-Time Systems 123

3 Benny Akesson Faculty of Electrical Engineering Eindhoven University of Technology Potentiaal/PT 9.11, Den Dolech MB Eindhoven Netherlands k.b.akesson@tue.nl Kees Goossens Faculty of Electrical Engineering Eindhoven University of Technology Potentiaal/PT 9.34, Den Dolech MB Eindhoven Netherlands k.g.w.goossens@tue.nl ISBN e-isbn DOI / Springer New York Dordrecht Heidelberg London Library of Congress Control Number: Springer Science+Business Media, LLC 2011 All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+Business Media, LLC, 233 Spring Street, New York, NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. Printed on acid-free paper Springer is part of Springer Science+Business Media (

4 Preface The authors of this book first met in the context of a master project in Philips Research in Eindhoven, the Netherlands, back in the summer of One was a student in Computer Science and Engineering from Lund University of Technology in Sweden, and the other a principal research scientist, leading the team of researchers designing a predictable network-on-chip called Æthereal. The work on the network-on-chip had been going on for several years, and the team had realized that predictable system-level guarantees also required a predictable solution for the memory system, which was the topic of the master project. Inspiration for the memory controller designed in this project was found in two different groups within Philips. One group made statically scheduled SDRAM controllers for high-performance video pipelines with firm real-time requirements, which inspired the idea of precomputed memory patterns. The other group made dynamically scheduled memory controllers for digital TV and set-top boxes, requiring dynamic scheduling to reduce latencies of latency-sensitive memory clients. The memory controller proposed in this book is a hybrid design that combines the approaches of these groups by dynamically scheduling statically computed memory patterns. From these two groups, the authors would particularly like to thank Frits Steenhof and Ad Siereveld for the inspiration and creative discussions. At the end of the 9 month master project, it was clear that only the surface of a large research topic had been scratched and that much interesting work remained to be done. An opportunity to continue the work in the Electronic Systems group at Eindhoven University of Technology presented itself through a Philips-funded PhD position in the PreMaDoNa project. The bulk of the research presented in this book was carried out in the scope of this project during the following 4 years and resulted in a thesis, several publications, and both a simulation model and a hardware implementation of the memory controller integrated in a design flow. This would not have been possible without the contributions from several master students. Thank you Markus Ringhofer, Eelke Strooisma, Getachew Teshome, Williston Hayes, and Winston Siauw for your hard work and for all the good times. The authors also v

5 vi Preface express their gratitude to Prof. Lambert Spaanenburg and Prof. Jef van Meerbergen for being the key enablers of the master project and the PhD project, respectively. During the PhD project, parts of Philips Research turned into NXP Semiconductors, and the research on the Æthereal network-on-chip finished. The fruits of this research were combined with processor tiles featuring Silicon Hive VLIW cores and MicroBlaze cores, resulting in the CoMPSoC platform and design flow. This effort demonstrated that it was possible to design a predictable and composable platform capable of concurrently executing a mix of real-time and non-real-time applications. Since then, this platform has been extensively used both as a research vehicle and for embedded system education at Eindhoven University of Technology, Delft University of Technology, and NXP Semiconductors. For every door you close in research, another one opens. Both authors of this book are currently employed at Eindhoven University of Technology extending the work on the CoMPSoC platform and the predictable and composable memory controller. The authors express their gratitude to all the members of the CoMPSoC team, and in particular to those contributing to the memory controller research. Thank you Karthik Chandrasekar, Firew Siyoum, Anand Khot, Sven Goossens, Tim Kouters, and Manil Dev Gomony, for continuing to push the boundaries of real-time memory controllers. Hard and passionate work takes its toll and sacrifices evenings and weekends when required. The authors are grateful for the support from family and friends, in particular for the friendship and help from Andreas Hansson. Finally, Benny acknowledges the support of his wife María Eugenia Martelli, whose love and understanding made this journey easier and more enjoyable. Eindhoven, the Netherlands Benny Akesson and Kees Goossens

6 Intended Audience This book is generally intended for readers interested in Systems-on-Chips with real-time applications. It targets senior architects and engineers, as well as academics (both teachers and students), looking for a perspective on the design and use of memory controllers in these systems. It is especially well-suited for readers looking to use SDRAM memories in systems with hard or firm real-time requirements. The book provides enough background on memories and memory controllers to be self-contained and provides extensive background references for the interested reader. There is a strong focus on real-time concepts, such as predictability and composability, and only a brief discussion about memory controller architectures for high-performance computing. The reader learns step-by-step how to go from an unpredictable SDRAM memory offering highly variable bandwidth and latency to a predictable and composable shared memory providing guaranteed bandwidth and latency to isolated applications. This journey covers concepts for making memories and arbiters behave in a predictable and composable manner, as well as architecture descriptions of hardware blocks that implement the concepts. The book will appeal to readers with different levels of prior knowledge and different learning goals: Novice/student & teachers: learns/teaches about system verification trends and challenges, the general architecture of SDRAM and memory controllers, and the problem of using these in Systems-on-Chips with real-time applications. Practitioner: learns about concrete concepts, architectures, and implementations for predictable and composable memory controllers. Expert: learns about complete design philosophy and concepts for predictable and composable memory controllers that are directly applicable to different system approaches, such as time-triggered architectures, precision-timed architectures, and different platforms, such as MERASA and CoMPSoC. vii

7 Contents 1 Introduction Trends in Embedded System Design Applications Platform-Based Design Platform Architecture Mapping Verification SDRAM and Real-Time Requirements Problem Statement Requirements Predictability Abstraction Composability Automation System Context Contributions Outline Summary Proposed Solution Predictability Overview of Approach Predictable SDRAM Back-End Predictable Arbitration Abstraction Composability Automation Summary SDRAM Memories and Controllers Introduction to SDRAM ix

8 x Contents SDRAM Architecture The SDRAM Protocol Timing Constraints Formal Model Memory Efficiency Refresh Efficiency Read/Write Efficiency Bank Efficiency Command Efficiency Data Efficiency Gross and Net Efficiencies Memory Efficiency Trend Memory Controllers Bus and Arbiter Command Generator Memory Map Summary Predictable SDRAM Back-End Overview of Predictable SDRAM Controller Arbitration Command Generator Memory Map Memory Patterns Scheduling Rules Pattern Descriptions Pattern Set Dominance Memory Efficiency Bound Refresh Efficiency Read/Write Efficiency Bank and Command Efficiency Data Efficiency Latency Bound Memory Pattern Generation Design Decisions Access Pattern Termination Branch and Bound As-Soon-As-Possible Scheduling Bank Scheduling Computing Auxiliary Patterns Architecture and Synthesis Experimental Results Experimental Setup Algorithm Evaluation Bounding Net Bandwidth... 99

9 Contents xi Tightness of Net Bandwidth Bound Summary Resource Arbitration Arbiter Requirements Formal Model Requested Service Model Provided Service Model Latency-Rate Servers Time-Division Multiplexing Overview Analysis Frame-Based Static-Priority Arbitration Overview Analysis Credit-Controlled Static-Priority Arbitration Overview Active Period Rate Regulation Analysis Experimental Results Experimental Setup Evaluation of Service Guarantee Latency Distributions Tightness of Service Latency Bound Allocation Properties Summary Composable Resource Front-End Overview of Approach Formal Model Architecture Architecture Overview Atomizer Delay Block Data Bus Synthesis Results Experiments SRAM Experiments SDRAM Experiments Summary Configuration Formal Model Memory Pattern Generation Normalization of Requirements Arbiter Configuration

10 xii Contents Bandwidth Allocation Priority Assignment Denormalization of Allocation Requirement Verification Experimental Results Summary Related Work Resource Arbitration SDRAM Controllers Composable Service Conclusions and Future Work Conclusions Predictability Abstraction Composability Automation Future Work Reducing Power Consumption Opportunities with 3D Integration Improved Arbiter Configuration Reconfiguration Data-Flow Model of Memory Controller A System XML Specification A.1 Architecture Specification A.2 Use-Case Specification B Glossary B.1 List of Abbreviations B.2 List of Symbols References Index

11 List of Figures Fig. 1.1 Example design flow comprised of a partitioning, platform exploration, mapping, and a verification step... 2 Fig. 1.2 A JPEG decoder application consisting of three tasks... 4 Fig. 1.3 Starting and stopping applications triggers use-case transitions... 5 Fig. 1.4 The design productivity gap... 6 Fig. 1.5 The platform template considered in this book... 7 Fig. 1.6 Processing element and resource communicating via a standard protocol... 9 Fig. 1.7 Multiple processing elements sharing a resource Fig. 1.8 Tasks are mapped to processing elements, data structures to memories, and communication channels to the interconnect as a part of the mapping process Fig. 1.9 The SDRAM architecture consists of banks, rows, and columns Fig Four systems demonstrating all combinations of the predictability and composability properties. (a) Predictable and composable system. (b) Predictable system. (c) Composable system. (d) Neither predictable nor composable system Fig Simplified architecture of a CoMPSoC instance with two processor tiles and both centralized SRAM and SDRAM memories Fig The proposed predictable and composable memory controller Fig. 2.1 Overview of predictable memory controller Fig. 2.2 The behaviors of some important SDRAM commands Fig. 2.3 Read pattern and write patterns with burst length 8 for a DDR (a) Read pattern. (b) Write pattern Fig. 2.4 Mapping from requests to patterns to SDRAM bursts Fig. 2.5 Overview of the predictable SDRAM back-end xiii

12 xiv List of Figures Fig. 2.6 A predictable SDRAM controller supporting two requestors Fig. 2.7 The LR server abstraction Fig. 2.8 LR arbiters are a subset of predictable arbiters Fig. 2.9 An instance of a predictable and composable SDRAM controller, supporting two requestors Fig Simplified overview of the automated configuration flow Fig. 3.1 The SDRAM architecture Fig. 3.2 Example of SDRAM timing constraints Fig. 3.3 Two bursts of 8 words are required to read or write 8 words that are misaligned Fig. 3.4 The most important building blocks of a general SDRAM controller Fig. 3.5 Illustration of a continuous memory map Fig. 3.6 Best case for a requestor reading four independent bursts with BL =4from a DDR2-400 using a continuous memory map Fig. 3.7 Worst-case for a requestor reading four independent bursts with BL =4from a DDR2-400 using a continuous memory map Fig. 3.8 Worst-case command sequence for a request consisting of four bursts to a DDR2-400 using a continuous memory map Fig. 3.9 Illustration of an interleaved memory map Fig Worst-case command sequence for a request consisting of four bursts to a DDR2-400 using an interleaved memory map Fig. 4.1 Fig. 4.2 Example pattern sets illustrating the four different dominance classes. (a) A read-dominant pattern set. (b) A write-dominant pattern set. (c) A mix-read-dominant pattern set. (d) A mix-write-dominant pattern set Illustration of how the dominance class of a pattern set changes as t read is incremented or decremented Fig. 4.3 A sequence of patterns and corresponding bursts Fig. 4.4 Refresh efficiency accounts for refresh patterns Fig. 4.5 Read/write efficiency accounts for switching patterns Fig. 4.6 Bank and conflict efficiencies remove overhead within read and write patterns, leaving only data bursts Fig. 4.7 Data efficiency accounts for data that is not useful to requestors, leaving only requested data bursts Fig. 4.8 The minimum distance between two refresh patterns... 76

13 List of Figures xv Fig. 4.9 Adding NOPs to the beginning of an access pattern may reduce the length of a switching pattern. (a) Original patterns. (b) Two NOP commands added to beginning of read pattern Fig Issuing all bursts to a bank before moving on to the next gives more time between activate and reads/writes, and more time to precharge before reactivating. (a)one burst per bank before moving on. (b) All bursts to one bank before moving on Fig The branch and bound algorithm creates pattern by exploring a tree of SDRAM commands Fig Number of valid patterns fitting our design decisions at BC =2for a DDR2-400 SDRAM device Fig Conceptual illustration of the ASAP scheduling algorithm Fig Prematurely scheduled activate commands result in longer access patterns. (a) The ASAP algorithm results in increasingly large distances between activate commands and their corresponding write commands. (b) A pattern with balanced distances between activate commands and write commands Fig Conceptual illustration of the bank scheduling algorithm for BC = Fig Memory efficiency results for DDR (a) Bounds on gross efficiency and gross bandwidth for the different algorithms. (b) Bank scheduling gross efficiency breakdown Fig Memory efficiency results for DDR (a) Bounds on gross efficiency and gross bandwidth for the different algorithms. (b) Bank scheduling gross efficiency breakdown Fig Bank scheduling gross efficiency breakdown for DDR Fig Bank scheduling gross efficiency breakdown for DDR Fig Gross efficiency and gross bandwidth comparisons between different DDR2 and DDR3 memories. (a) Gross efficiency comparison. (b) Gross bandwidth comparison Fig Bound on net bandwidth for different memories and request sizes. (a) Net bandwidthwith a DDR (b) Net bandwidth with a DDR (c) Net bandwidth with a DDR (d) Net bandwidth with a DDR Fig Net bandwidth plotted over time for a DDR2-400 memory with and without worst-case switches. (a) The first 16 μs of the simulation. (b) The first 160 μs of the simulation

14 xvi List of Figures Fig. 5.1 An arbiter comprising a rate regulator and a scheduler Fig. 5.2 A requested service curve, w, a provided service curve, w, and representations of the related concepts Fig. 5.3 Example service curves in a LR server Fig. 5.4 Illustration of worst-case starting time and finishing time in a LR server Fig. 5.5 Centralized and distributed slot assignment strategies for TDM. (a) Centralized assignment strategy. (b) Distributed assignment strategy Fig. 5.6 Example of coupling between allocation granularity, latency, and allocated bandwidth. (a) Initial case. (b) Increasing allocated rate reduces service latency. (c) Increasing frame size reduces over-allocation, but increases service latency Fig. 5.7 Worst-case situation under FBSP arbitration Fig. 5.8 The upper bound on provided service, ŵ, in a CCSP arbiter is determined by the allocated rate and the allocated burstiness Fig. 5.9 Illustration of the relation between being live, backlogged, and active Fig TDM latency distribution for two assignment strategies. (a) Use-case with equal allocated rates. (b) Use-case with diverse allocated rates Fig FBSP latency distribution for use-case with equal allocated rates Fig FBSP latency distribution for use-case with diverse allocated rates. (a) Descending priorities. (b) Ascending priorities Fig CCSP latency distribution for use-case with equal allocated rates Fig CCSP latency distribution for use-case with diverse allocated rates. (a) Descending priorities. (b) Ascending priorities Fig Arbiter latency distribution for use-case with equal allocated rates Fig Arbiter latency distribution for use-case with diverse allocated rates. (a) Descending priorities. (b) Ascending priorities Fig Maximum measured latency and bound, expressed in service cycles, for the requestors in the use-case. (a) Maximum measured service latency and bound for r 0. (b) Maximum measured service latency and bound for r 1.(c) Maximum measured service latency and bound for r 2.(d) Maximum measured service latency and bound for r

15 List of Figures xvii Fig Maximum measured latency and bound, expressed in clock cycles at 200 MHz, for the requestors in the use-case. (a) Maximum measured service latency and bound for r 0.(b) Maximum measured service latency and bound for r 1.(c) Maximum measured service latency and bound for r 2.(d) Maximum measured service latency and bound for r Fig Over-allocated rate for CCSP and FBSP Fig Successful allocations and priority assignments for CCSP and FBSP Fig Success rate when increasing precision with CCSP Fig Success rate when increasing precision with FBSP Fig. 6.1 Temporally independent interfaces are created by delaying responses and flow control Fig. 6.2 The trade-off between service latency and net bandwidth Fig. 6.3 An instance of the proposed architecture supporting two requestors Fig. 6.4 Delay Block architecture Fig. 6.5 Diverging finishing times prevented by discrete approximation of the completion latency Fig. 6.6 Synthesis results for the Atomizer. (a) Cellareafor different buffer sizes. (b) Maximum frequency and corresponding cell area for different buffer sizes Fig. 6.7 Synthesis results for the Delay Block. (a)cellareafor different buffer sizes and precisions. (b) Maximum frequency and corresponding cell area for different buffer sizes with 10 bits of precision Fig. 6.8 Synthesis results for the Data Bus with a CCSP arbiter. (a) Cell area for different number of requestors and precisions. (b) Maximum frequency and corresponding cell area for different number of requestors with 10 bits of precision Fig. 6.9 The first 200 requests of r 2 in the SRAM use-case Fig Atoms finish before the computed bound, since they are served non-preemptively Fig SRAM controller behaving in a composable manner. (a) Request releases are unaffected by other requestors. (b) Worst-case Response Buffer space is unaffected by other requestors Fig Using a work-conserving arbiter to distribute unallocated bandwidth may significantly reduce finishing times Fig The first 200 requests of r 2 in the SDRAM use-case

16 xviii Fig List of Figures SDRAM controller behaving in a composable manner. (a) Request releases are unaffected by other requestors. (b) Worst-case Response Buffer space is unaffected by other requestors Fig. 7.1 Overview of the automated configuration flow Fig. 7.2 Configuration of CCSP and FBSP consists of a bandwidth allocation step and a priority assignment step Fig. 7.3 LR servers cannot capture service provided with multiple rates to a requestor Fig. 7.4 The percentage of use-cases with bandwidth and latency requirements satisfied using pattern generators with fixed and iterating burst counts Fig. 8.1 Two arbiters regulating requested service and provided service, respectively. (a) Requested service regulation. (b) Provided service regulation

17 List of Tables Table 3.1 Table 3.2 List of relevant timing parameters for a 64 MB x16 (512 Mb) DDR2-400 memory device Comparison of timing constraints in nanoseconds and clock cycles for a DDR2-400 and a DDR Table 4.1 Worst-case patterns for mix-dominant pattern sets Table 4.2 List of relevant timing parameters for some different 64 MB x16 (512 Mb) memory devices with page sizes of 2 KB Table 4.3 Pattern generation results for the DDR2-400 memory Table 4.4 Pattern generation results for the DDR2-800 memory Table 4.5 Pattern generation results for the DDR3-800 memory Table 4.6 Pattern generation results for the DDR memory Table 5.1 Requestor configuration and service latency bounds Table 5.2 Bandwidth and service latency results Table 5.3 Bandwidth and service latency results with malfunctioning requestor using a regular staticpriority arbiter Table 5.4 Requestor specification Table 6.1 SRAM use-case specification and configuration Table 6.2 SDRAM use-case specification and configuration Table 7.1 Use-case specification Table 7.2 Output from pattern generation stage Table 7.3 Output from normalization stage Table 7.4 Results from the bandwidth allocation stage Table 7.5 Results from priority assignment stage xix

18 xx List of Tables Table 7.6 Output from denormalization stage Table 7.7 Allocated bandwidths and service latencies together with their corresponding bounds Table 7.8 Output from normalization stage with BC = Table B.1 List of symbols

19 List of Algorithms 4.1 Pseudo-code of the ASAP scheduling algorithm Pseudo-code of the bank scheduling algorithm Mechanism for discrete approximation of completion latency Optimal priority assignment algorithm xxi