Computer-Hardware Evaluation of Mathematical Functions

Transcription

1 Computer-Hardware Evaluation of Mathematical Functions

2 This page intentionally left blank

3 Computer-Hardware Evaluation of Mathematical Functions Amos Omondi Zed Consultants, Canada ICP Imperial College Press

4 Published by Imperial College Press 57 Shelton Street Covent Garden London WC2H 9HE Distributed by Computer-Hardware Evaluation of Mathematical Functions Downloaded from World Scientific Publishing Co. Pte. Ltd. 5 Toh Tuck Link, Singapore USA office: 27 Warren Street, Suite , Hackensack, NJ UK office: 57 Shelton Street, Covent Garden, London WC2H 9HE Library of Congress Cataloging-in-Publication Data Omondi, Amos R., author. Computer-hardware evaluation of mathematical functions / Amos Omondi, Zed Consultants, Canada. pages cm Includes bibliographical references and index. ISBN (hardcover : alk. paper) 1. Computer arithmetic. 2. Computer arithmetic and logic units. 3. Computer algorithms. 4. Integrated circuits--verification. 5. Computer architecture. I. Title. QA76.9.C62O dc British Library Cataloguing-in-Publication Data A catalogue record for this book is available from the British Library. Copyright 2016 by Imperial College Press All rights reserved. This book, or parts thereof, may not be reproduced in any form or by any means, electronic or mechanical, including photocopying, recording or any information storage and retrieval system now known or to be invented, without written permission from the publisher. For photocopying of material in this volume, please pay a copying fee through the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA. In this case permission to photocopy is not required from the publisher. Printed in Singapore

5 To Anne, Miles, and Micah

6 This page intentionally left blank

7 Preface Lo, the enraptured arithmetician. Easily satisfied, he asks for no Brussels lace, or a coach and six. To calculate contents his liveliest desires, and obedient numbers are within his reach. Elie de Joncourt, 1762 This book deals primarily with the methods used in the computerhardware evaluation of some elementary transcendental functions and some algebraic functions: trigonometric functions and their inverses, hyperbolic functions and their inverses, exponentials, logarithms, reciprocals, and square roots and their inverses. Nevertheless, a great deal of what is discussed is equally applicable to numerous other functions. Of necessity, some material on multiplication and division is also included. Titles of technical books are always quite straightforward and rarely require explanation. But, on the basis of some remarks from two people who looked at parts of the manuscript, I believe a few words on the hardware are in order; the few words are also a good starting point for a more detailed explanation on what this book is about and what background is required of the reader. The main point is this: contrary to what the title might suggest to some, this book does not contain numerous diagrams of logic circuits, gate-diagrams, diagrams that depict hardware blocks, etc., and discussions of such. On the other hand, leaving out the hardware could, perhaps, have led the reader to expect a much wider coverage than is the case. This book is not about hardware in the aforementioned sense, but it is also not about software algorithms, even though the algorithms given may be implemented in software. Perhaps the following remarks will be clarifying. vii

8 viii Computer-Hardware Evaluation of Mathematical Functions Computer-Hardware Evaluation of Mathematical Functions Downloaded from At the level of architecture, hardware for the evaluation of the functions listed above will consist of relatively simple functional units adders, multipliers, shifters, multiplexors, and so forth and the challenge is how best to put these together. At its highest level, that putting together is largely about the choice of methods, and the considerations are different from those in software implementation. Three examples will give some idea of the differences: The evaluation of a polynomial approximation for a given function: The basic hardware required for the evaluation of any polynomial is an adder, a multiplier, and some memory to hold the polynomialcoefficients; and these, perhaps with some replication, can be put together in fairly straightforward ways. The basic issue is then that of which (or what type of) polynomial, and the considerations for hardware implementation are different from those for software implementation. For example, in the former, the degree of the polynomial is very important, and even a single term going from degree n to degree n + 1 can make a substantial difference, especially in high-performance implementations. Sharing hardware: For hardware implementation, cost-effectiveness is critical; so it is helpful if the same hardware can be used for different functions. One therefore strives for methods that are very similar. On the other hand, the software-writer is unlikely to be particularly concerned with, say, whether one subroutine is used for both division and square-root evaluation, or whether there is one subroutine per function. Basic operations: Consider rational functions, which are quotients of polynomials. These can provide very good approximations for many functions and have been used routinely in software implementations, but their implementation requires division. In hardware, however, division is about as hard as the evaluation of the functions of interest and is therefore undesirable as a basic operation. This limits the range of algorithms that may usefully be considered for hardware evaluation. The emphasis of this book is therefore on those methods that are suitable for hardware evaluation, rather than on nitty-gritty hardware details that will mostly be straightforward for the well-prepared reader: the primary issues are how the methods have been derived, how they work, and how well they work. The reader who is familiar with the design of basic

9 Preface ix Computer-Hardware Evaluation of Mathematical Functions Downloaded from arithmetic units will be able to readily see how the methods can be implemented; we do not dwell on such matters but do make particular points as necessary. Therefore, a background in computer arithmetic, while helpful, is not strictly necessary. (On the other hand, the reader who requires details of implementations will readily find a good supply of descriptions in the published literature; however, many of the best are also particular to specific technologies.) A brief explanation of the origins of the book might perhaps help in giving the reader a better idea of what to expect as well as what background is expected: Some years ago, when I was with the School of Electrical and Electronic Engineering at Yonsei University (Seoul), I taught a course on computer arithmetic; this book is made up of the lecture notes updated and given some polish for the function evaluation half of that course. The readership I have in mind is largely similar to the students who took that course: postgraduate students in computer science, computer engineering, and electrical and electronic engineering. Nevertherless, none of the material is beyond an advanced undergraduate student. This book might also be useful, as an introduction to the subject matter, to professionals engaged in designing and building arithmetic units and to prospective researchers in the field. Indeed, the subject matter is so fascinating that even the merely-curious should find this book quite enjoyable. The aforementioned course was a complete one on computer arithmetic, and by the time function-evaluation came along, basic arithmetic-unit design had all been done. The ideal background for this book therefore consists of a reasonable understanding of digital logic, some knowledge of arithmetic-unit design, and a level of mathematics that one may reasonably expect of a well-equipped undergraduate student in science or engineering. The first two are not strict requirements. Some aspects of the basic arithmetic-unit algorithms are indeed essential and have been included in Chapter 2. Beyond that, what is covered in this book does not require much knowledge of arithmetic units, provided one is prepared to accept that adders, multipliers, dividers, and so forth can be realized in digital logic, as indeed is the case; it might even be that the reader already has in mind such building blocks and simply wishes to know how best to put them together for the evaluation of some function. There are a few places where statements are made on the comparative cost and performance of hardware; the reader who does not have the requisite background may simply take those on faith and proceed.

10 x Computer-Hardware Evaluation of Mathematical Functions Computer-Hardware Evaluation of Mathematical Functions Downloaded from The level of detail and choice of material (beyond the obvious) were largely determined by feedback from the original course, but I believe they are appropriate for similar readership elsewhere. On detail, there are places where, in response to students, I covered much more than I initially intended to; for example, what is Chapter 6 was originally just a few pages summarizing some basic results. And there are other places where faith seemed sufficient as the main ingredient. What all this means is that the atypical reader might consider the coverage somewhat uneven. I beg the forbearance of the reader who is unhappy with that, but the primary purpose and target-readership should be kept in mind. That applies as well to another possible source of minor irratations: the repetition, in a few places, of one or two short paragraphs. This is a left-over habit from the original lecture notes that shows up wherever it is necessary to draw the reader s attention to a significant explanation that has already been given, but I have considered that it would be helpful to not have him or her search back over a hundred pages or so to find the first instance. For the material, I did not cover only state-of-the-art methods. There are two main reasons for that. First, I considered that a (very) slightly historical view of how the state of the art came about would be helpful to the students not just in understanding the current current algorithms and their implementations but also in preparing them to understand and appreciate future advances. Second, it seemed to me unwise to make any assumptions about what one might be interested in putting to use and in what ways. For example, the assumption of existing building blocks (such as are to be found in FPGAs) will limit choices in what to implement, and it might well be that old methods are more suitable than the very best ; on the other hand, the situation is very different if the assumption is that everything can be done from scratch (as in custom ASIC). At the other extreme, from both the historical and the current (in practical terms), I have included a very small number of methods whose practicality is probably shaky and which best belong in the to-be-continued category. These methods embody novel ideas, and it is not unreasonable to expect that the situation will change with advances in technology and refinements. I considered it important that, say, students who might be interested in undertaking research in the field should be aware of such methods. Chapter 1 is on errors, range reduction, and rounding. Error-control is obviously important in any numerical computation, and the general concerns are how to keep errors low, how to determine error bounds for a given algorithm, and what the bounds are in particular cases. Range reduction

11 Preface xi Computer-Hardware Evaluation of Mathematical Functions Downloaded from the conversion of an evaluation on one argument range into one on a smaller range is necessary because most of the algorithms of interest work only for arguments in limited ranges, and others work better (e.g. converge faster) when the argument ranges are sufficiently constrained. And proper rounding is generally problematic for transcendental functions; in contrast with the case of basic arithmetic operations and algebraic functions in general, a great deal of effort and care are necessary in rounding for the transcendental functions. Redundant number-representation systems, in which a single number can have several representations, play a significant role in the design and implementations of algorithms for high-speed arithmetic. Chapter 2 consists of an introduction to such systems and their use in high-speed multiplication and division. Algorithms for division, whether or not they use such number systems, are very significant in relation to several algorithms that are discussed in this book: many of the latter algorithms have their origins in division algorithms and are very similar. In implementation, this means that essentially the same hardware can be used for all, and it should be noted that division will usually be implemented as a standard and required operation. The reader who has a good background in basic computer arithmetic may skip this chapter and return to particular sections according to references elsewhere. Chapter 3 covers the CORDIC algorithms, which are based on the idea of rotation in a geometric space. These algorithms are among the moststudied for the evaluation of elementary functions. Their chief merit is that they facilitate the use of essentially the same hardware for a large number of functions, and so implementations will have excellent cost:performance ratios if several functions are to be evaluated. Chapter 4 extends the material in Chapter 3, by including, for high-performance implementations, the use of redundant number representations and radices larger than two. Chapter 5 is on normalization, in which a function-evaluation consists of the computation of two sequences, such that as the values of one are driven to some constant, those of the other are driven to the value to be computed of the function at hand. The technique is a uniform approach to the direct evaluation of the functions covered in the CORDIC approach and of other functions that are evaluated only indirectly through CORDIC. Also included in the chapter are discussions on the use of redundant number representations and large radices for high performance. Chapter 6 covers polynomial evaluation in some generality. Although Taylor and Maclaurin series will be familiar to all students in science and

12 xii Computer-Hardware Evaluation of Mathematical Functions Computer-Hardware Evaluation of Mathematical Functions Downloaded from engineering, their practical limitations in numerical computation, especially for hardware evaluation, sometimes constitute a gap that must be filled; and familiarity with, say, Chebyshev polynomials and their applications might not be standard. For hardware evaluation, the polynomials are generally low-order ones, combined with some form of table lookup. Chapter 7 is on such table lookup and polynomial approximations. The last chapter is on the evaluation of reciprocals, square roots, and inverse square roots. These functions are especially important given the frequency with which they are used, and this is especially so for the square root, which is included in the IEEE standard on floating-point arithmetic. The first appearance of these functions is halfway through this book, but this chapter comes at the end because many of the methods described are based on those in the preceding chapters: normalization, polynomials and table lookup, and the use of redundant number representations. Comments, notification of errors, and so forth: I shall be grateful and indebted if any reader can spare the time to communicate these, to amos.omondi.ca@ieee.org Sir, In your otherwise beautiful poem ( The Vision of Sin ) there is a verse that reads Every moment a man dies, Every moment one is born. It must be manifest that if this were true, the population of the world would be at a standstill. In truth the rate of birth is slightly in excess of that of death. I would suggest that in the next edition of your poem you have it read Every moment a man dies, Every moment 1 1 is born. 16 Strictly speaking, this is not correct, the actual figure is so long that I cannot get it into a line, but I believe the figure 1 1 will be sufficiently accurate for poetry. 16 I am, Sir, yours, etc. Charles Babbage to Tennyson Letter, 1842

13 Acknowledgments All Thanks and Praise to The Almighty I was very fortunate in the staff at Imperial College Press that I worked with, and I am grateful for their high levels of professionalism. Alice Oven, commissioning editor, made sure that the initial phase went smoothly and quickly. Catharina Weijman, production editor, similarly made sure that the subsequent phases were as pain-free possible and also carried out thorough and helpful final checks. And Maia Vaswani is an author s dream of a copy editor: she was fast, completely thorough, and managed to catch several non-trivial technical errors. My friend Clare Davidson was kind enough to proof-read the first draft of the manuscript. I am indebted to her for that. xiii

14 This page intentionally left blank

15 Preface Acknowledgments Contents 1. Errors, Range Reduction, and Rounding Errors Computation errors Propagated errors Iterative algorithms Range reduction Rounding Redundant Representations and High-Speed Arithmetic Redundant number representations Carry save representation Redundant signed-digit representation High-speed multiplication Multiplier recoding Squaring High-speed division Multiplicative normalization Additive normalization SRT Very high radix vii xiii 3. CORDIC Trigonometric functions xv

16 xvi Computer-Hardware Evaluation of Mathematical Functions 3.2 Inverse trigonometric functions Hyperbolic functions and inverses Linear functions Errors and datapath precision Implementation Computer-Hardware Evaluation of Mathematical Functions Downloaded from 4. High-Performance CORDIC De Lugish CORDIC Correcting-Rotations CORDIC Branching CORDIC Differential CORDIC Double-Rotations CORDIC High-radix CORDIC Very-high-radix CORDIC Normalization Algorithms Normalization constants Reciprocals Exponential and logarithm functions Exponential Logarithm Trigonometric functions and inverses Square root and inverse High-performance exponential and logarithm Early termination and zero skipping Redundant representation and high radix Very-high-radix computation Polynomial and Rational-Function Approximations Quality of approximation Taylor series Chebyshev polynomials Legendre polynomials Interpolation Rational functions Table Lookup and Segmented Polynomial Approximations Polynomial-based table lookup

17 Contents xvii Computer-Hardware Evaluation of Mathematical Functions Downloaded from Bipartite tables Multipartite tables Addition Table lookup Addition Table-driven polynomial approximation Segmented polynomial approximation Uniform segmentation Segment boundaries and numbers Hierarchical segmentation LUT cascades Address remapping Errors Reciprocals, Square Roots, and Inverse Square Roots Polynomial approximations Reciprocals Newton Raphson method Goldschmidt normalization Square root and inverse Newton Raphson method and variations Goldschmidt normalization Multiplicative normalization: non-redundant digit set Multiplicative normalization: redundant digit set Additive normalization: redundant digit set SRT algorithms High-radix computation References 397 Index 411