Computer Arithmetic. 1. Floating-point representation of numbers (scientific notation) has four components, for example, 3.
|
|
- Whitney Mosley
- 5 years ago
- Views:
Transcription
1 ECS231 Handout Computer Arithmetic I: Floating-point numbers and representations 1. Floating-point representation of numbers (scientific notation) has four components, for example, sign significandbase exponent 2. Computers use binary representation of numbers. The floating-point representation of a nonzero binary number x is of the form (a) It is normalized, i.e., b 0 = 1 (the hidden bit) x = ±b 0.b 1 b 2 b p 1 2 E. (1) (b) Precision (= p) is the number of bits in the significand (mantissa) (including the hidden bit). (c) Machine epsilon ǫ = 2 (p 1), the gap between the number 1 and the smallest floatingpoint number that is greater than 1. (d) The unit in the last place, ulp(x) = 2 (p 1) 2 E = ǫ 2 E. If x > 0, then ulp(x) is the gap between x and the next larger floating-point number. If x < 0, then ulp(x) is the gap between x and the smaller floating-point number (larger in absolute value). 3. Special numbers: 0, 0,,, NaN(= Not a Number ). 4. All computers designed since 1985 use the IEEE Standard for Binary Floating-Point Arithmetic (ANSI/IEEE Std ) represent each number as a binary number and use binary arithmetic. Essentials of the IEEE standard: consistent representation of floating-point numbers by all machines adopting the standard; correctly rounded floating-point operations, using various rounding modes; consistent treatment of exceptional situation such as division by zero. 5. IEEE single format takes 32 bits (=4 bytes) long: 8 23 s E f sign exponent binary point fraction 1
2 It represents the number ( 1) s (1.f) 2 E 127 Note that (1) the leading 1 in the fraction need not be stored explicitly, because it is always 1. This hidden bit accounts for the 1. here. (2) The E 127 in the exponent is to avoid the need for storage of a sign bit. E is a normalized number, and E min = ( ) 2 = (1) 10, E max = ( ) 2 = (254) 10. The range of positive normalized numbers is from N min = E min 127 = to N max = Emax 127 = ( ) Special repsentations for 0, ± and NaN: zero = ± ± = ± NaN = ± otherwise 6. IEEE double format takes 64 bits (= 8 bytes) long: s E f sign exponent binary point fraction It represents the numer ( 1) s (1.f) 2 E 1023 The range of positive normalized numbers is from N min = to N max = Special repsentations for 0, ± and NaN. 7. IEEE extended format, with at least 15 bits available for the exponent and at least 63 bits for the fractional part of the significant. (Pentium has 80-bit extended format) 8. Precision and machine epsilon of the IEEE formats Format Precision p Machine epsilon ǫ = 2 p 1 single 24 ǫ = double 53 ǫ = extended 64 ǫ =
3 9. Rounding Let a positive real number x be in the normalized range, i.e., N min x N max, and be written in the normalized form x = (1.b 1 b 2 b p 1 b p b p+1...) 2 E, Then the closest floating-point number less than or equal to x is x = 1.b 1 b 2 b p 1 2 E, i.e., x is obtained by truncating. The next floating-point number bigger than x is x + = ((1.b 1 b 2 b p 1 )+( )) 2 E, therefore, also the next one that bigger than x. If x is negative, the situtation is reversed. Correctly rounding modes: round down: round(x) = x ; round up: round(x) = x + ; round towards zero: round(x) = x of x 0; round(x) = x + of x 0; round to nearest: round(x) = x orx +, whicheverisnearertox, exceptthatifx > N max, round(x) =, and if x < N max, round(x) =. In the case of tie, i.e., x and x + are the same distance from x, the one with its least significant bit equal to zero is chosen When the round to nearest (IEEE default rounding mode) is in effect, and Therefore, we have abserr(x) = round(x) x 1 2 ulp(x) relerr(x) = round(x) x x the max. rel. representation error = 1 2 ǫ = II: Floating point arithmetic 1. IEEE rules for correctly rounded floating-point operations: if x and y are correctly rounded floating-point numbers, then fl(x+y) = round(x+y) = (x+y)(1+δ) fl(x y) = round(x y) = (x y)(1+δ) fl(x y) = round(x y) = (x y)(1+δ) fl(x/y) = round(x/y) = (x/y)(1+δ) where δ 1 2ǫ for the round to nearest, IEEE standard also requires that correctly rounded remainder and square root operations be provided. 3
4 2. IEEE standard response to exceptions Event Example Set result to Invalid operation 0/0, 0 NaN Division by zero Finite nonzero/0 ± Overflow x > N max ± or ±N max underflow x 0, x < N min ±0, ±N min or subnormal Inexact whenever fl(x y) x y correctly rounded value 3. Let ˆx and ŷ be the floating-point numbers and that ˆx = x(1+τ 1 ) and ŷ = y(1+τ 2 ), for τ i τ 1 where τ i could be the relative errors in the process of collecting/getting the data from the original source or the previous operations. Question: how do the four basic arithmetic operations behave? 4. Addition and subtraction fl(ˆx+ŷ) = (ˆx+ŷ)(1+δ), δ 1 2 ǫ = x(1+τ 1 )(1+δ)+y(1+τ 2 )(1+δ) = x+y +x(τ 1 +δ +O(τǫ))+y(τ 2 +δ +O(τǫ)) ( = (x+y) 1+ x x+y (τ 1 +δ +O(τǫ))+ y ) x+y (τ 2 +δ +O(τǫ)) (x+y)(1+ ˆδ), where ˆδ can be bounded as follows: Three possible cases: ˆδ x + y x+y (τ ǫ+o(τǫ) ). (a) If x and y have the same sign, i.e., xy > 0, then x+y = x + y ; this implies Thus fl(ˆx+ŷ) approximates x+y well. ˆδ τ + 1 ǫ+o(τǫ) 1. 2 (b) If x y x+y 0, then ( x + y )/ x+y 1; this implies that ˆδ could be nearly or much bigger than 1. Thus fl(ˆx+ŷ) may turn out to have nothing to do with the true x + y. This is so called catastrophic cancellation which happens when a floating-point number is subtracted from another nearly equal floating-point number. Cancellation causes relative errors or uncertainties already presented in ˆx and ŷ to be magnified. (c) In general, if ( x + y )/ x+y is not too big, fl(ˆx+ŷ) provides a good approximation to x+y. 5. Examples of catastrophic cancellation Example 1. Computing n+1 n straightforward causes substantial loss of significant digits for large n 4
5 n fl( n+1) fl( n) fl(fl( n+1) fl( n) 1.00e e e e e e e e e e e e e e e e e e e e e e e e e e e e+00 Catastrophic cancellation can sometimes be avoided if a formula is properly reformulated. In the present case, one can compute n+1 n almost to full precision by using the equality n+1 n = 1 n+1+ n. Consequently, the computed results are n fl(1/( n+1+ n)) 1.00e e e e e e e e e e e e e e-09 In fact, one can show that fl(1/( n+1+ n)) = ( n+1 n)(1+δ), where δ 5ǫ+O(ǫ 2 ) (try it!) Example 2. Consider the function Note that f(x) = 1 cosx x 2 = 1 2 ( ) sin(x/2) 2. x/2 0 f(x) < 1/2 for all x 0. Compare the computed values for x = using the above two expressions (assume that the value of cosx rounded to 10 significant figures). 6. Multiplication and Division: fl(ˆx ŷ) = (ˆx ŷ)(1+δ) = xy(1+τ 1 )(1+τ 2 )(1+δ) xy(1+ ˆδ ), fl(ˆx/ŷ) = (ˆx/ŷ)(1+δ) = (x/y)(1+τ 1 )(1+τ 2 ) 1 (1+δ) xy(1+ ˆδ ), where ˆδ = τ 1 +τ 2 +δ +O(τǫ), ˆδ = τ 1 τ 2 +δ +O(τǫ). Thus ˆδ 2τ ǫ+o(τǫ) ˆδ 2τ ǫ+o(τǫ). Therefore, multiplication and Division are very well-behaved! 5
6 III: Floating point error analysis 1. Forward and backward error analysis We illustrate the basic idea through a simple example. Consider the computation of an inner product of two vector x, y R 3 x T y def = x 1 y 1 +x 2 y 2 +x 3 y 3, assuming already x i s and y j s are floating-point numbers. It is likely that fl(x y) is computed in the following order. fl(x T y) = fl(fl(fl(x 1 y 1 )+fl(x 2 y 2 ))+fl(x 3 y 3 )). Adopting the floating-point arithmetic model, we have fl(x T y) = fl(fl(x 1 y 1 (1+ǫ 1 )+x 2 y 2 (1+ǫ 2 ))+x 3 y 3 (1+ǫ 3 )) where ǫ i 1 2 ǫ and δ j 1 2 ǫ. = fl((x 1 y 1 (1+ǫ 1 )+x 2 y 2 (1+ǫ 2 ))(1+δ 1 )+x 3 y 3 (1+ǫ 3 )) = ((x 1 y 1 (1+ǫ 1 )+x 2 y 2 (1+ǫ 2 ))(1+δ 1 )+x 3 y 3 (1+ǫ 3 ))(1+δ 2 ) = x 1 y 1 (1+ǫ 1 )(1+δ 1 )(1+δ 2 )+x 2 y 2 (1+ǫ 2 )(1+δ 1 )(1+δ 2 ) +x 3 y 3 (1+ǫ 3 )(1+δ 2 ), Now there are two ways to interpret the errors in the computed fl(x T y): (a) We have fl(x T y) = x T y +E, where E = x 1 y 1 (ǫ 1 +δ 1 +δ 2 )+x 2 y 2 (ǫ 2 +δ 1 +δ 2 )+x 3 y 3 (ǫ 3 +δ 2 )+O(ǫ 2 ). It implies that E 1 2 ǫ(3 x 1y 1 +3 x 2 y 2 +2 x 3 y 3 )+O(ǫ 2 ) 3 2 ǫ x T y +O(ǫ 2 ). This bound on E tells the worst case difference between the exact x T y and its computed value. Such an error analysis is so-called Forward Error Analysis. (b) We can also write where 1 fl(x T y) = ˆx T ŷ = (x+ x) T (y + y), ˆx 1 = x 1 (1+ǫ 1 ), ŷ 1 = y 1 (1+δ 1 )(1+δ 2 ) y 1 (1+ ˆδ 1 ), ˆx 2 = x 2 (1+ǫ 2 ), ŷ 2 = y 2 (1+δ 1 )(1+δ 2 ) y 2 (1+ ˆδ 2 ), ˆx 3 = x 3 (1+ǫ 3 ), ŷ 3 = y 3 (1+δ 2 ) y 3 (1+ ˆδ 3 ). It can be seen that ˆδ 1 = ˆδ 2 ǫ+o(ǫ 2 ) and ˆδ 3 1 2ǫ. This says the computed value fl(x T y) is the exact inner product of a slightly perturbed ˆx and ŷ. Such an error analysis is so-called Backward Error Analysis. 1 There are many ways to distribute factors (1+ǫ i) and (1+δ j) to x i and y j. In this case it is even possible to make either ˆx x or ŷ y. 6
7 IV: Exercises 1. The polynomial p 1 (x) = (x 1) 6 has the value zero at x = 1 and is positive elsewhere. The expanded form of the polynomial p 2 (x) = x 6 6x 5 +15x 4 20x 3 +15x 2 6x+1, is mathematically equivalent. Plot p 1 (x) and p 2 (x) for 101 equally spaced points in the interval [0.995,1.005]. Explain the plots. (You should evaluate the polynomial p 2 (x) by Horner s rule). 2. The power series for sinx is sin(x) = x x3 3! + x5 5! x7 7! + Here is a Matlab function that uses this series to compute sinx. function s = powersin(x) % POWERSIN(x) tries to comptue sin(x) from a power series s = 0; t = x; n = 1; while s + t ~= s; s = s + t; t = -x.^2/((n+1)*(n+2)).*t; n = n + 2; end (a) What cases the while loop to terminate? (b) Answer each of the following questions for x = π/2,11π/2,21π/2 and 31π/2. How accurate is the computed results? How many terms are required? What is the largest term (i.e, the last t) in the series? (c) What do you conclude about the use of floating point arithmetic and power series to evaluate functions? 3. Show how to rewrite the following expressions to avoid catastrophic cancellation for the indicated arguments. (a) x+1 1, x 0. (b) sinx siny, x y. (c) x 2 y 2, x y. (d) (1 cosx)/sinx, x 0. (e) c = (a 2 +b 2 2ab cosθ) 1/2, a b, θ 1. 7
8 4. Consider the function f(x) = ex 1, x which arises in various applications. By L Hopital s rule, we know that lim f(x) = 1. x 0 (a) Compute the values of f(x) for x = 10 n, n = 1,2,...,16. Do your results agree with theoretical expectations? Explain why. (b) Perform the computation in part (a) again, this time using the mathematically equivalent formulation f(x) = ex 1 log(e x ), (evaluated as indicated with no simplification). If this works any better, can you explain why? V: Further reading 1. The following article based on lecture notes of Prof. W. Kahan of the University of California at Berkeley provides an excellent review of IEEE float point arithmetics. D. Goldberg. What every computer scientist should know about floating-point arithmetic. ACM Computing Surveys, 18(1):5 48, The following book gives a broad overview of numerical computing, with special focus on the IEEE standard for binary floating-point arithmetic. M. Overton. Numerical computing with IEEE floating-point arithemetic. SIAM, Philadelphia, ISBN Student price $20.00 directly from 3. A classical book on error analysis, where the notion of backward error analysis is invented, is J.H. Wilkinson. Rounding Errors in Algebraic Process. Prentice-Hall, Englewood, NJ, Reprinted by Dover, New York, A contemporary treatment of error analysis and its applications to numerical analysis is N.J. Higham, Accuracy and stability of Numerical Algorithms. second edition, SIAM, Philadelphia, Websites for discussion of numerical disasters: T. Huckle, Collection of software bugs huckle/bugse.html K. Vuik, Some disasters caused by numerical errors D. Arnold, Some disasters attributable to bad numerical computing arnold/disasters/disasters.html 8
Floating-Point Arithmetic
Floating-Point Arithmetic ECS30 Winter 207 January 27, 207 Floating point numbers Floating-point representation of numbers (scientific notation) has four components, for example, 3.46 0 sign significand
More informationClasses of Real Numbers 1/2. The Real Line
Classes of Real Numbers All real numbers can be represented by a line: 1/2 π 1 0 1 2 3 4 real numbers The Real Line { integers rational numbers non-integral fractions irrational numbers Rational numbers
More informationFloating-point representation
Lecture 3-4: Floating-point representation and arithmetic Floating-point representation The notion of real numbers in mathematics is convenient for hand computations and formula manipulations. However,
More information2 Computation with Floating-Point Numbers
2 Computation with Floating-Point Numbers 2.1 Floating-Point Representation The notion of real numbers in mathematics is convenient for hand computations and formula manipulations. However, real numbers
More information2 Computation with Floating-Point Numbers
2 Computation with Floating-Point Numbers 2.1 Floating-Point Representation The notion of real numbers in mathematics is convenient for hand computations and formula manipulations. However, real numbers
More informationBinary floating point encodings
Week 1: Wednesday, Jan 25 Binary floating point encodings Binary floating point arithmetic is essentially scientific notation. Where in decimal scientific notation we write in floating point, we write
More informationIntroduction to floating point arithmetic
Introduction to floating point arithmetic Matthias Petschow and Paolo Bientinesi AICES, RWTH Aachen petschow@aices.rwth-aachen.de October 24th, 2013 Aachen, Germany Matthias Petschow (AICES, RWTH Aachen)
More informationMAT128A: Numerical Analysis Lecture Two: Finite Precision Arithmetic
MAT128A: Numerical Analysis Lecture Two: Finite Precision Arithmetic September 28, 2018 Lecture 1 September 28, 2018 1 / 25 Floating point arithmetic Computers use finite strings of binary digits to represent
More informationFloating-Point Arithmetic
Floating-Point Arithmetic Raymond J. Spiteri Lecture Notes for CMPT 898: Numerical Software University of Saskatchewan January 9, 2013 Objectives Floating-point numbers Floating-point arithmetic Analysis
More informationTable : IEEE Single Format ± a a 2 a 3 :::a 8 b b 2 b 3 :::b 23 If exponent bitstring a :::a 8 is Then numerical value represented is ( ) 2 = (
Floating Point Numbers in Java by Michael L. Overton Virtually all modern computers follow the IEEE 2 floating point standard in their representation of floating point numbers. The Java programming language
More informationAM205: lecture 2. 1 These have been shifted to MD 323 for the rest of the semester.
AM205: lecture 2 Luna and Gary will hold a Python tutorial on Wednesday in 60 Oxford Street, Room 330 Assignment 1 will be posted this week Chris will hold office hours on Thursday (1:30pm 3:30pm, Pierce
More informationComputational Methods. Sources of Errors
Computational Methods Sources of Errors Manfred Huber 2011 1 Numerical Analysis / Scientific Computing Many problems in Science and Engineering can not be solved analytically on a computer Numeric solutions
More informationScientific Computing: An Introductory Survey
Scientific Computing: An Introductory Survey Chapter 1 Scientific Computing Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002. Reproduction
More informationComputational Mathematics: Models, Methods and Analysis. Zhilin Li
Computational Mathematics: Models, Methods and Analysis Zhilin Li Chapter 1 Introduction Why is this course important (motivations)? What is the role of this class in the problem solving process using
More informationMathematical preliminaries and error analysis
Mathematical preliminaries and error analysis Tsung-Ming Huang Department of Mathematics National Taiwan Normal University, Taiwan August 28, 2011 Outline 1 Round-off errors and computer arithmetic IEEE
More informationCS321 Introduction To Numerical Methods
CS3 Introduction To Numerical Methods Fuhua (Frank) Cheng Department of Computer Science University of Kentucky Lexington KY 456-46 - - Table of Contents Errors and Number Representations 3 Error Types
More informationNumerical Computing: An Introduction
Numerical Computing: An Introduction Gyula Horváth Horvath@inf.u-szeged.hu Tom Verhoeff T.Verhoeff@TUE.NL University of Szeged Hungary Eindhoven University of Technology The Netherlands Numerical Computing
More informationBindel, Fall 2016 Matrix Computations (CS 6210) Notes for
1 Logistics Notes for 2016-09-07 1. We are still at 50. If you are still waiting and are not interested in knowing if a slot frees up, let me know. 2. There is a correction to HW 1, problem 4; the condition
More information2.1.1 Fixed-Point (or Integer) Arithmetic
x = approximation to true value x error = x x, relative error = x x. x 2.1.1 Fixed-Point (or Integer) Arithmetic A base 2 (base 10) fixed-point number has a fixed number of binary (decimal) places. 1.
More informationReview Questions 26 CHAPTER 1. SCIENTIFIC COMPUTING
26 CHAPTER 1. SCIENTIFIC COMPUTING amples. The IEEE floating-point standard can be found in [131]. A useful tutorial on floating-point arithmetic and the IEEE standard is [97]. Although it is no substitute
More informationRoundoff Errors and Computer Arithmetic
Jim Lambers Math 105A Summer Session I 2003-04 Lecture 2 Notes These notes correspond to Section 1.2 in the text. Roundoff Errors and Computer Arithmetic In computing the solution to any mathematical problem,
More informationCS321. Introduction to Numerical Methods
CS31 Introduction to Numerical Methods Lecture 1 Number Representations and Errors Professor Jun Zhang Department of Computer Science University of Kentucky Lexington, KY 40506 0633 August 5, 017 Number
More informationFloating-Point Numbers in Digital Computers
POLYTECHNIC UNIVERSITY Department of Computer and Information Science Floating-Point Numbers in Digital Computers K. Ming Leung Abstract: We explain how floating-point numbers are represented and stored
More informationFloating-Point Numbers in Digital Computers
POLYTECHNIC UNIVERSITY Department of Computer and Information Science Floating-Point Numbers in Digital Computers K. Ming Leung Abstract: We explain how floating-point numbers are represented and stored
More informationReview of Calculus, cont d
Jim Lambers MAT 460/560 Fall Semester 2009-10 Lecture 4 Notes These notes correspond to Sections 1.1 1.2 in the text. Review of Calculus, cont d Taylor s Theorem, cont d We conclude our discussion of Taylor
More informationFinite arithmetic and error analysis
Finite arithmetic and error analysis Escuela de Ingeniería Informática de Oviedo (Dpto de Matemáticas-UniOvi) Numerical Computation Finite arithmetic and error analysis 1 / 45 Outline 1 Number representation:
More informationChapter 3. Errors and numerical stability
Chapter 3 Errors and numerical stability 1 Representation of numbers Binary system : micro-transistor in state off 0 on 1 Smallest amount of stored data bit Object in memory chain of 1 and 0 10011000110101001111010010100010
More informationCSCI 402: Computer Architectures. Arithmetic for Computers (3) Fengguang Song Department of Computer & Information Science IUPUI.
CSCI 402: Computer Architectures Arithmetic for Computers (3) Fengguang Song Department of Computer & Information Science IUPUI 3.5 Today s Contents Floating point numbers: 2.5, 10.1, 100.2, etc.. How
More informationMost nonzero floating-point numbers are normalized. This means they can be expressed as. x = ±(1 + f) 2 e. 0 f < 1
Floating-Point Arithmetic Numerical Analysis uses floating-point arithmetic, but it is just one tool in numerical computation. There is an impression that floating point arithmetic is unpredictable and
More informationUNIVERSITY OF MASSACHUSETTS Dept. of Electrical & Computer Engineering. Digital Computer Arithmetic ECE 666
UNIVERSITY OF MASSACHUSETTS Dept. of Electrical & Computer Engineering Digital Computer Arithmetic ECE 666 Part 4-C Floating-Point Arithmetic - III Israel Koren ECE666/Koren Part.4c.1 Floating-Point Adders
More informationMath 340 Fall 2014, Victor Matveev. Binary system, round-off errors, loss of significance, and double precision accuracy.
Math 340 Fall 2014, Victor Matveev Binary system, round-off errors, loss of significance, and double precision accuracy. 1. Bits and the binary number system A bit is one digit in a binary representation
More informationScientific Computing: An Introductory Survey
Scientific Computing: An Introductory Survey Chapter 1 Scientific Computing Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002. Reproduction
More informationfractional quantities are typically represented in computers using floating point format this approach is very much similar to scientific notation
Floating Point Arithmetic fractional quantities are typically represented in computers using floating point format this approach is very much similar to scientific notation for example, fixed point number
More informationLecture Notes to Accompany. Scientific Computing An Introductory Survey. What is scientific computing?
Lecture Notes to Accompany Scientific Computing An Introductory Survey Second Edition by Michael T. Heath Scientific Computing What is scientific computing? Design and analysis of algorithms for solving
More informationFloating Point Representation in Computers
Floating Point Representation in Computers Floating Point Numbers - What are they? Floating Point Representation Floating Point Operations Where Things can go wrong What are Floating Point Numbers? Any
More informationFloating-Point Arithmetic
Floating-Point Arithmetic 1 Numerical Analysis a definition sources of error 2 Floating-Point Numbers floating-point representation of a real number machine precision 3 Floating-Point Arithmetic adding
More informationAMTH142 Lecture 10. Scilab Graphs Floating Point Arithmetic
AMTH142 Lecture 1 Scilab Graphs Floating Point Arithmetic April 2, 27 Contents 1.1 Graphs in Scilab......................... 2 1.1.1 Simple Graphs...................... 2 1.1.2 Line Styles........................
More informationNumerical Methods 5633
Numerical Methods 5633 Lecture 2 Marina Krstic Marinkovic mmarina@maths.tcd.ie School of Mathematics Trinity College Dublin Marina Krstic Marinkovic 1 / 15 5633-Numerical Methods Organisational Assignment
More informationFloating-point Arithmetic. where you sum up the integer to the left of the decimal point and the fraction to the right.
Floating-point Arithmetic Reading: pp. 312-328 Floating-Point Representation Non-scientific floating point numbers: A non-integer can be represented as: 2 4 2 3 2 2 2 1 2 0.2-1 2-2 2-3 2-4 where you sum
More informationLecture Objectives. Structured Programming & an Introduction to Error. Review the basic good habits of programming
Structured Programming & an Introduction to Error Lecture Objectives Review the basic good habits of programming To understand basic concepts of error and error estimation as it applies to Numerical Methods
More informationComputational Economics and Finance
Computational Economics and Finance Part I: Elementary Concepts of Numerical Analysis Spring 2015 Outline Computer arithmetic Error analysis: Sources of error Error propagation Controlling the error Rates
More informationComputing Basics. 1 Sources of Error LECTURE NOTES ECO 613/614 FALL 2007 KAREN A. KOPECKY
LECTURE NOTES ECO 613/614 FALL 2007 KAREN A. KOPECKY Computing Basics 1 Sources of Error Numerical solutions to problems differ from their analytical counterparts. Why? The reason for the difference is
More informationUNIVERSITY OF MASSACHUSETTS Dept. of Electrical & Computer Engineering. Digital Computer Arithmetic ECE 666
UNIVERSITY OF MASSACHUSETTS Dept. of Electrical & Computer Engineering Digital Computer Arithmetic ECE 666 Part 4-B Floating-Point Arithmetic - II Israel Koren ECE666/Koren Part.4b.1 The IEEE Floating-Point
More informationFloating-point numbers. Phys 420/580 Lecture 6
Floating-point numbers Phys 420/580 Lecture 6 Random walk CA Activate a single cell at site i = 0 For all subsequent times steps, let the active site wander to i := i ± 1 with equal probability Random
More informationEE878 Special Topics in VLSI. Computer Arithmetic for Digital Signal Processing
EE878 Special Topics in VLSI Computer Arithmetic for Digital Signal Processing Part 4-B Floating-Point Arithmetic - II Spring 2017 Koren Part.4b.1 The IEEE Floating-Point Standard Four formats for floating-point
More informationThe Perils of Floating Point
The Perils of Floating Point by Bruce M. Bush Copyright (c) 1996 Lahey Computer Systems, Inc. Permission to copy is granted with acknowledgement of the source. Many great engineering and scientific advances
More informationFloating-point representations
Lecture 10 Floating-point representations Methods of representing real numbers (1) 1. Fixed-point number system limited range and/or limited precision results must be scaled 100101010 1111010 100101010.1111010
More informationFloating-point representations
Lecture 10 Floating-point representations Methods of representing real numbers (1) 1. Fixed-point number system limited range and/or limited precision results must be scaled 100101010 1111010 100101010.1111010
More informationUp next. Midterm. Today s lecture. To follow
Up next Midterm Next Friday in class Exams page on web site has info + practice problems Excited for you to rock the exams like you have been the assignments! Today s lecture Back to numbers, bits, data
More informationComputational Economics and Finance
Computational Economics and Finance Part I: Elementary Concepts of Numerical Analysis Spring 2016 Outline Computer arithmetic Error analysis: Sources of error Error propagation Controlling the error Rates
More informationHani Mehrpouyan, California State University, Bakersfield. Signals and Systems
Hani Mehrpouyan, Department of Electrical and Computer Engineering, California State University, Bakersfield Lecture 3 (Error and Computer Arithmetic) April 8 th, 2013 The material in these lectures is
More informationME 261: Numerical Analysis. ME 261: Numerical Analysis
ME 261: Numerical Analysis 3. credit hours Prereq.: ME 163/ME 171 Course content Approximations and error types Roots of polynomials and transcendental equations Determinants and matrices Solution of linear
More informationSection 1.4 Mathematics on the Computer: Floating Point Arithmetic
Section 1.4 Mathematics on the Computer: Floating Point Arithmetic Key terms Floating point arithmetic IEE Standard Mantissa Exponent Roundoff error Pitfalls of floating point arithmetic Structuring computations
More information1.2 Round-off Errors and Computer Arithmetic
1.2 Round-off Errors and Computer Arithmetic 1 In a computer model, a memory storage unit word is used to store a number. A word has only a finite number of bits. These facts imply: 1. Only a small set
More informationFloating Point Numbers
Floating Point Numbers Summer 8 Fractional numbers Fractional numbers fixed point Floating point numbers the IEEE 7 floating point standard Floating point operations Rounding modes CMPE Summer 8 Slides
More informationClass 5. Data Representation and Introduction to Visualization
Class 5. Data Representation and Introduction to Visualization Visualization Visualization is useful for: 1. Data entry (initial conditions). 2. Code debugging and performance analysis. 3. Interpretation
More informationObjectives. look at floating point representation in its basic form expose errors of a different form: rounding error highlight IEEE-754 standard
Floating Point Objectives look at floating point representation in its basic form expose errors of a different form: rounding error highlight IEEE-754 standard 1 Why this is important: Errors come in two
More informationECE232: Hardware Organization and Design
ECE232: Hardware Organization and Design Lecture 11: Floating Point & Floating Point Addition Adapted from Computer Organization and Design, Patterson & Hennessy, UCB Last time: Single Precision Format
More informationBasics of Computation. PHY 604:Computational Methods in Physics and Astrophysics II
Basics of Computation Basics of Computation Computers store information and allow us to operate on it. That's basically it. Computers have finite memory, so it is not possible to store the infinite range
More informationScientific Computing. Error Analysis
ECE257 Numerical Methods and Scientific Computing Error Analysis Today s s class: Introduction to error analysis Approximations Round-Off Errors Introduction Error is the difference between the exact solution
More informationData Representation and Introduction to Visualization
Data Representation and Introduction to Visualization Massimo Ricotti ricotti@astro.umd.edu University of Maryland Data Representation and Introduction to Visualization p.1/18 VISUALIZATION Visualization
More informationFloating point numbers in Scilab
Floating point numbers in Scilab Michaël Baudin May 2011 Abstract This document is a small introduction to floating point numbers in Scilab. In the first part, we describe the theory of floating point
More informationOutline. 1 Scientific Computing. 2 Approximations. 3 Computer Arithmetic. Scientific Computing Approximations Computer Arithmetic
Outline 1 2 3 Michael T. Heath 2 / 46 Introduction Computational Problems General Strategy What is scientific computing? Design and analysis of algorithms for numerically solving mathematical problems
More informationReal Numbers finite subset real numbers floating point numbers Scientific Notation fixed point numbers
Real Numbers We have been studying integer arithmetic up to this point. We have discovered that a standard computer can represent a finite subset of the infinite set of integers. The range is determined
More informationFloating Point Representation. CS Summer 2008 Jonathan Kaldor
Floating Point Representation CS3220 - Summer 2008 Jonathan Kaldor Floating Point Numbers Infinite supply of real numbers Requires infinite space to represent certain numbers We need to be able to represent
More informationFloating-point Error
I am HAL 9000 computer production Number 3. I became operational at the Hal Plant in Urbana, Illinios, on January 12, 1997. The quick brown fox jumps over the lazy dog. The rain in Spain is mainly in the
More informationCHAPTER 2 SENSITIVITY OF LINEAR SYSTEMS; EFFECTS OF ROUNDOFF ERRORS
CHAPTER SENSITIVITY OF LINEAR SYSTEMS; EFFECTS OF ROUNDOFF ERRORS The two main concepts involved here are the condition (of a problem) and the stability (of an algorithm). Both of these concepts deal with
More informationComputational Methods CMSC/AMSC/MAPL 460. Representing numbers in floating point Error Analysis. Ramani Duraiswami, Dept. of Computer Science
Computational Methods CMSC/AMSC/MAPL 460 Representing numbers in floating point Error Analysis Ramani Duraiswami, Dept. of Computer Science Class Outline Recap of floating point representation Matlab demos
More informationWhat Every Programmer Should Know About Floating-Point Arithmetic
What Every Programmer Should Know About Floating-Point Arithmetic Last updated: October 15, 2015 Contents 1 Why don t my numbers add up? 3 2 Basic Answers 3 2.1 Why don t my numbers, like 0.1 + 0.2 add
More informationIntroduction to Computer Programming with MATLAB Calculation and Programming Errors. Selis Önel, PhD
Introduction to Computer Programming with MATLAB Calculation and Programming Errors Selis Önel, PhD Today you will learn Numbers, Significant figures Error analysis Absolute error Relative error Chopping
More informationDivide: Paper & Pencil
Divide: Paper & Pencil 1001 Quotient Divisor 1000 1001010 Dividend -1000 10 101 1010 1000 10 Remainder See how big a number can be subtracted, creating quotient bit on each step Binary => 1 * divisor or
More informationNumerical computing. How computers store real numbers and the problems that result
Numerical computing How computers store real numbers and the problems that result The scientific method Theory: Mathematical equations provide a description or model Experiment Inference from data Test
More informationUNIVERSITY OF MASSACHUSETTS Dept. of Electrical & Computer Engineering. Digital Computer Arithmetic ECE 666
UNIVERSITY OF MASSACHUSETTS Dept. of Electrical & Computer Engineering Digital Computer Arithmetic ECE 666 Part 4-A Floating-Point Arithmetic Israel Koren ECE666/Koren Part.4a.1 Preliminaries - Representation
More informationErrors in Computation
Theory of Errors Content Errors in computation Absolute Error Relative Error Roundoff Errors Truncation Errors Floating Point Numbers Normalized Floating Point Numbers Roundoff Error in Floating Point
More informationFloating Point (with contributions from Dr. Bin Ren, William & Mary Computer Science)
Floating Point (with contributions from Dr. Bin Ren, William & Mary Computer Science) Floating Point Background: Fractional binary numbers IEEE floating point standard: Definition Example and properties
More informationFloating Point Arithmetic
Floating Point Arithmetic CS 365 Floating-Point What can be represented in N bits? Unsigned 0 to 2 N 2s Complement -2 N-1 to 2 N-1-1 But, what about? very large numbers? 9,349,398,989,787,762,244,859,087,678
More informationCOMP2611: Computer Organization. Data Representation
COMP2611: Computer Organization Comp2611 Fall 2015 2 1. Binary numbers and 2 s Complement Numbers 3 Bits: are the basis for binary number representation in digital computers What you will learn here: How
More informationFloating Point Numbers. Lecture 9 CAP
Floating Point Numbers Lecture 9 CAP 3103 06-16-2014 Review of Numbers Computers are made to deal with numbers What can we represent in N bits? 2 N things, and no more! They could be Unsigned integers:
More informationVHDL IMPLEMENTATION OF IEEE 754 FLOATING POINT UNIT
VHDL IMPLEMENTATION OF IEEE 754 FLOATING POINT UNIT Ms. Anjana Sasidharan Student, Vivekanandha College of Engineering for Women, Namakkal, Tamilnadu, India. Abstract IEEE-754 specifies interchange and
More informationCS 6210 Fall 2016 Bei Wang. Lecture 4 Floating Point Systems Continued
CS 6210 Fall 2016 Bei Wang Lecture 4 Floating Point Systems Continued Take home message 1. Floating point rounding 2. Rounding unit 3. 64 bit word: double precision (IEEE standard word) 4. Exact rounding
More informationIntroduction to Numerical Computing
Statistics 580 Introduction to Numerical Computing Number Systems In the decimal system we use the 10 numeric symbols 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 to represent numbers. The relative position of each symbol
More informationData Representation Floating Point
Data Representation Floating Point CSCI 2400 / ECE 3217: Computer Architecture Instructor: David Ferry Slides adapted from Bryant & O Hallaron s slides via Jason Fritts Today: Floating Point Background:
More informationReals 1. Floating-point numbers and their properties. Pitfalls of numeric computation. Horner's method. Bisection. Newton's method.
Reals 1 13 Reals Floating-point numbers and their properties. Pitfalls of numeric computation. Horner's method. Bisection. Newton's method. 13.1 Floating-point numbers Real numbers, those declared to be
More informationFloating Point Arithmetic
Floating Point Arithmetic Computer Systems, Section 2.4 Abstraction Anything that is not an integer can be thought of as . e.g. 391.1356 Or can be thought of as + /
More informationFloating-point operations I
Floating-point operations I The science of floating-point arithmetics IEEE standard Reference What every computer scientist should know about floating-point arithmetic, ACM computing survey, 1991 Chih-Jen
More informationLecture 17 Floating point
Lecture 17 Floating point What is floatingpoint? A representation ±2.5732 10 22 NaN Single, double, extended precision (80 bits) A set of operations + = * / rem Comparison < = > Conversions between different
More informationFloating-Point Arithmetic
Floating-Point Arithmetic if ((A + A) - A == A) { SelfDestruct() } Reading: Study Chapter 3. L12 Multiplication 1 Approximating Real Numbers on Computers Thus far, we ve entirely ignored one of the most
More informationMATH 353 Engineering mathematics III
MATH 353 Engineering mathematics III Instructor: Francisco-Javier Pancho Sayas Spring 2014 University of Delaware Instructor: Francisco-Javier Pancho Sayas MATH 353 1 / 20 MEET YOUR COMPUTER Instructor:
More informationData Representation Floating Point
Data Representation Floating Point CSCI 2400 / ECE 3217: Computer Architecture Instructor: David Ferry Slides adapted from Bryant & O Hallaron s slides via Jason Fritts Today: Floating Point Background:
More informationComputer arithmetics: integers, binary floating-point, and decimal floating-point
n!= 0 && -n == n z+1 == z Computer arithmetics: integers, binary floating-point, and decimal floating-point v+w-w!= v x+1 < x Peter Sestoft 2010-02-16 y!= y p == n && 1/p!= 1/n 1 Computer arithmetics Computer
More informationComputing Integer Powers in Floating-Point Arithmetic
Computing Integer Powers in Floating-Point Arithmetic Peter Kornerup Vincent Lefèvre Jean-Michel Muller May 2007 This is LIP Research Report number RR2007-23 Ceci est le Rapport de Recherches numéro RR2007-23
More informationComputer Representation of Numbers and Computer Arithmetic
Computer Representation of Numbers and Computer Arithmetic January 17, 2017 Contents 1 Binary numbers 6 2 Memory 9 3 Characters 12 1 c A. Sandu, 1998-2015. Distribution outside classroom not permitted.
More informationIntroduction to numerical algorithms
Introduction to numerical algorithms Given an algebraic equation or formula, we may want to approximate the value, and while in calculus, we deal with equations or formulas that are well defined at each
More informationFloating Point Numbers
Floating Point Numbers Computer Systems Organization (Spring 2016) CSCI-UA 201, Section 2 Instructor: Joanna Klukowska Slides adapted from Randal E. Bryant and David R. O Hallaron (CMU) Mohamed Zahran
More informationFloating Point Numbers
Floating Point Numbers Computer Systems Organization (Spring 2016) CSCI-UA 201, Section 2 Fractions in Binary Instructor: Joanna Klukowska Slides adapted from Randal E. Bryant and David R. O Hallaron (CMU)
More informationOn the definition of ulp (x)
On the definition of ulp (x) Jean-Michel Muller CNRS Laboratoire LIP, projet Arenaire (CNRS, ENS Lyon, INRIA, Univ. Lyon ), 46 Allée d Italie, 69364 Lyon Cedex 07, France Function ulp (acronym for unit
More informationTruncation Errors. Applied Numerical Methods with MATLAB for Engineers and Scientists, 2nd ed., Steven C. Chapra, McGraw Hill, 2008, Ch. 4.
Chapter 4: Roundoff and Truncation Errors Applied Numerical Methods with MATLAB for Engineers and Scientists, 2nd ed., Steven C. Chapra, McGraw Hill, 2008, Ch. 4. 1 Outline Errors Accuracy and Precision
More informationIEEE Standard 754 Floating Point Numbers
IEEE Standard 754 Floating Point Numbers Steve Hollasch / Last update 2005-Feb-24 IEEE Standard 754 floating point is the most common representation today for real numbers on computers, including Intel-based
More informationThe Sign consists of a single bit. If this bit is '1', then the number is negative. If this bit is '0', then the number is positive.
IEEE 754 Standard - Overview Frozen Content Modified by on 13-Sep-2017 Before discussing the actual WB_FPU - Wishbone Floating Point Unit peripheral in detail, it is worth spending some time to look at
More informationLecture 10. Floating point arithmetic GPUs in perspective
Lecture 10 Floating point arithmetic GPUs in perspective Announcements Interactive use on Forge Trestles accounts? A4 2012 Scott B. Baden /CSE 260/ Winter 2012 2 Today s lecture Floating point arithmetic
More information