# Floating Point Considerations

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Chapter 6 Floating Point Considerations In the early days of computing, floating point arithmetic capability was found only in mainframes and supercomputers. Although many microprocessors designed in the 1980's started to have floating point coprocessors, their floating-point arithmetic speed was about three orders of magnitude slower than that of mainframes and supercomputers. With advances in microprocessor technology, many microprocessors designed in the 1990's, such as Intel Pentium III and AMD Athlon, started to have high performance floating point capabilities that rival supercomputers. High speed floating point arithmetic has become a standard feature for microprocessors today. As a result, it has also become important for application programmers to understand and take advantage of floating point arithmetic in developing their applications. In particular, we will focus on the accuracy of floating point arithmetic, the precision of floating point number representation, and how they should be taken into consideration in parallel computation Floating Point Format The IEEE Floating Point Standard is an effort for the computer manufacturers to conform to a common representation and arithmetic convention for floating point data. Most, if not all, of the computer manufacturers in the world have accepted this standard. In particular, virtually all microprocessors designed in the future will either fully conform to or almost fully conform to the IEEE Floating Point Standard. Therefore, it is important for application developers to understand the concept and practical considerations of this standard. A floating point number system starts with the representation of a numerical value as bit patterns. In the IEEE Floating Point Standard, a numerical value is represented in three groups of bits: sign (S), exponent (E), and mantissa (M). Each (S, E, M) pattern uniquely identifies a numeric value according to the following formula: value = (-1) S * M * {2 E }, where 1.0 M < 2.0 (1) The interpretation of S is simple: S=0 means a positive number and S=1 a negative number. Mathematically, any number, including -1, when raised to the power of 0, results David Kirk/NVIDIA and Wen-mei Hwu,

2 in 1. Thus the value is positive. On the other hand, when -1 is raised to the power of 1, it is -1 itself. With a multiplication by -1, the value becomes negative. The interpretation of M and E bits are, however, much more complex. We will use the following example to help explain the interpretation of M and E bits. Assume for the sake of simplicity that each floating point number consists of a 1-bit sign, 3-bit exponent, and 2-bit mantissa. We will use this hypothetical 6-bit format to illustrate the challenges involved in encoding E and M. As we discuss these values, we will sometime need to express number as either in decimal place value or in binary place value. Numbers expressed in decimal place value will have subscript D and those as binary place value will have subscript B. For example, 0.5 D (5* 10-1 since the place to the right of the decimal point carries a weight of 10-1 ) is the same as 0.1 B (1*2-1 since the place to the right of the decimal point carries a weight of 2-1 ). Normalized representation of M Formula (1) requires that 1.0 B M < 10.0 B, which makes the mantissa value for each floating point number unique. For example, the only one mantissa value allowed for 0.5 D is M =1.0: 0.5 D = 1.0 B * 2-1 Another potential candidate would be 0.1 B * 2 0, but the value of mantissa would be too small according to the rule. Similarly, 10.0 B * 2-2 is not legal because the value of the mantissa is too large. In general, with the restriction that 1.0 B M < 10.0 B, every floating point number has exactly one legal mantissa value. The numbers that satisfy this restriction will be referred to as normalized numbers. Because all mantissa values that satisfy the restriction are of the form 1.XX, we can omit the 1. part from the representation. Therefore, the mantissa value of 0.5 in a 2-bit mantissa representation is 00, which is derived by omitting 1. from This makes a 2-bit mantissa effectively a 3-bit mantissa. In general, with IEEE format, an n-bit mantissa is effectively an (n+1)-bit mantissa. Excess encoding of E If n bits are used to represent the exponent E, the value 2 n-1-1 is added to the two's complement representation for the exponent to form its excess representation. A two s complement representation is a system where the negative value of a number can be derived by first complementing every bit of the value and add one to the result. In our 3- bit exponent representation, there are three bits in the exponent. Therefore, the value = 011 will be added to the 2 s complement representation of the exponent value. The following table shows the 2 s complement representation and the excess presentation of each decimal exponent value. In our example, the exponent for 0.5 D is -1. The two s David Kirk/NVIDIA and Wen-mei Hwu,

3 complement representation of -1 can be derived by first complementing 001, the representation of 1, into 110 and then adding 001 to get 111. The excess presentation adds another 011 to the 2 s complement representation, as shown in the table, which results in s complement Decimal value Excess-representation (reserved pattern) Figure 1 Excess-3 encoding, sorted by 2 s complement ordering The advantage of excess representation is that an unsigned comparator can be used to compare signed numbers. As shown in Figure 2, the excess-3 code increases monotonically from 3 to 3. The code from 3 is 000 and that for 3 is 110. Thus, if one uses an unsigned number comparator to compare excess-3 code for any number from 1 to 3, the comparator gives the correct comparison result in terms of which number is larger, smaller, etc. For example, if one compares excess-3 codes 001 and 100 with an unsigned comparator, 001 is smaller than 100. This is the right conclusion since the values that they represent, -2 and 1, have exactly the same relation. This is a desirable property for hardware implementation since unsigned comparators are smaller and faster than signed comparators. 2 s complement Decimal value Excess (reserved pattern) Figure 2 Excess-3 encoding, sorted by excess-3 ordering Now we are ready to represent 0.5 D with our 6-bit format: 0.5 D = , where S = 0, E = 010, and M = (1.)00 David Kirk/NVIDIA and Wen-mei Hwu,

4 That is, the 6-bit representation for 0.5 D is With normalized mantissa and excess-coded exponent, the value of a number with an n-bit exponent is (-1) S * 1.M * 2 (E- (2 ^( n-1))+ 1) 6.2. Representable Numbers The representable numbers of a number format are the numbers that can be exactly represented in the format. For example, if one uses a 3-bit unsigned integer format, the representable numbers would be: Figure 3 Representable numbers of a 3-bit unsigned format Neither -1 nor 9 can be represented in the format given above. We can draw a number line to identify all the representable numbers, as shown in Figure 4 where all representatble numbers of the 3-bit unsigned integer format are marked with stars Figure 4 representable numbers of a 3-bit unsigned integer format The representable numbers of a floating point format can be visualize in a similar manner. In Table 1, we show all the representable numbers of what we have so far and two variations. We use a 5-bit format to keep the size of the table manageable. The format consists of 1-bit S, 2-bit E (excess-1 coded), and 2-bit M (with 1. part omitted). The nozero column gives the representable numbers of the format we discussed thus far. Note that with this format, 0 is not one of the representatble numbers. David Kirk/NVIDIA and Wen-mei Hwu,

5 No-zero Abrupt underflow Denorm E M S=0 S=1 S=0 S=1 S=0 S= (2-1 ) *2-3 -(2-1 +1*2-3 ) 0 0 1*2-2 -1* *2-3 -(2-1 +2*2-3 ) 0 0 2*2-2 -2* *2-3 -(2-1 +3*2-3 ) 0 0 3*2-2 -3* (2 0 ) 2 0 -(2 0 ) 2 0 -(2 0 ) *2-2 -(2 0 +1*2-2 ) *2-2 -(2 0 +1*2-2 ) *2-2 -(2 0 +1*2-2 ) *2-2 -(2 0 +2*2-2 ) *2-2 -(2 0 +2*2-2 ) *2-2 -(2 0 +2*2-2 ) *2-2 -(2 0 +3*2-2 ) *2-2 -(2 0 +3*2-2 ) *2-2 -(2 0 +3*2-2 ) (2 1 ) 2 1 -(2 1 ) 2 1 -(2 1 ) *2-1 -(2 1 +1*2-1 ) *2-1 -(2 1 +1*2-1 ) *2-1 -(2 1 +1*2-1 ) *2-1 -(2 1 +2*2-1 ) *2-1 -(2 1 +2*2-1 ) *2-1 -(2 1 +2*2-1 ) *2-1 -(2 1 +3*2-1 ) *2-1 -(2 1 +3*2-1 ) *2-1 -(2 1 +3*2-1 ) 11 Reserved pattern Table 1 Representable numbers of no-zero, abrupt underflow, and denorm formats A look at how these representable numbers populate the number line, as shown in Figure 5, provides further insights about these representable numbers. In Figure 5, we show only the positive representable numbers. The negative numbers are symmetric to their positive counterparts on the other side of Figure 5. Representable numbers of the no-zero representation Three things stand out. First, 0 is not representable in this format. Because 0 is one of the most important numbers, not being able to represent 0 in a number representation system is a serious deficiency. Second, the representable numbers become closer to each other towards the neighborhood of 0. This is a desirable behavior because as the absolute value of these numbers become smaller, it is more important to represent them more accurately. Having the representable numbers closer to each other makes it possible to represent numbers more accurately. Unfortunately this trend does not hold for the very vicinity of 0, which leads to the third point: there is a gap of representable numbers in the vicinity of 0. This is because the range of normalized mantissa precludes 0. This is another serious deficiency. The representation introduces significantly larger errors when representing numbers between 0 and 0.5 compared to the errors for the larger numbers between 0.5 and 1.0. David Kirk/NVIDIA and Wen-mei Hwu,

6 One method that has been used to accommodate 0 into a normalized number system is the abrupt underflow convention, which is illustrated by the second column of Table 1. Whenever E is 0, the number is interpreted as 0. In our 5-bit format, this method takes away eight representable numbers (four positive and four negative) in the vicinity of 0 and makes them all 0. Although this method makes 0 representable, it does create an even larger gap between representable numbers in 0's vicinity, as shown in Figure 6. It is obvious, when compared with Figure 5, the gap of representable numbers has been enlarged significantly with the vicinity of 0. This is very problematic since many numeric algorithms rely on the fact that the accuracy of number representation is higher for the very small numbers near zero. These algorithms generate small numbers and eventually use them as denominators. The errors for these small numbers can be greatly magnified in the division process Figure 6 Representable numbers of the abrupt underflow format The actual method adopted by the IEEE standard is called denormalization. The method relaxes the normalization requirement for numbers very close to 0. That is, whenever E=0, the mantissa is no longer assumed to be of the form 1.XX. Rather, it is assumed to be 0.XX. In general, if the n-bit exponent is 0, the value is 0.M * 2-2 ^(n-1) + 2 For example, in Table 1, the denormalized representation has exponent value 00 and mantissa value 01. Using the denromalized formula, the value it represents is 0.01*2 0 = 2 2. Figure 7 shows the representable numbers for the denormalization format. The representation now has a uniformly spaced representable numbers in the close vicinity of 0. This eliminates the undesirable gap in the previous two methods Figure 7. Representable numbers of a denormalization format. We are now ready to discuss the concept of precision. The precision of a floating point representation is measured by the maximal error that we can introduce to a floating point number by reprensenting that number as one of the representable numbers. The smaller the error is, the higher the precision. In Figure 7, the error instrduced by a number representation is always smaller than 0.25 D, which occurs if we choose the largest representative number that is smaller than the number we would like to represent. Obviously, the precision is a floating point representation can be improved by adding more David Kirk/NVIDIA and Wen-mei Hwu,

7 bits to mantissa. Adding one bit to the representation in Figure 7 would improve the precision by reducing the maximal error by half. Thus, we say that a number system has higher precision when it uses more bits for mantissa Special Bit Patterns and Precision The actual IEEE format has one more special bit pattern. When all exponent bits are 1s, the number represented is an infinity value if the mantissa is 0 or a Not a Number (NaN) if the mantissa is not 0. All special bit patterns of the IEEE floating point format are shown in the following table. exponent mantissa meaning NaN 11 1 =0 (-1) S * denormalized 00 0 =0 0 All other numbers are normalized floating-point numbers. Single precision numbers have 1-bit S, 8-bit E, and 23-bit M. Double precision numbers have 1-bit S, 11-bit E, and 52-bit M. Since a double precision number has 29 more bits for mantissa, the largest error for representating a number is reduced to 1/2 29 of that of the single precision format! All representable numbers fall between - (negative infinity) and + (positive infinite). An can be created by overflow, e.g., divided by zero. Any representable number divided by + or - results in 0. NaN (Not a Number) is generated by operations whose input values do not make sense, for example, 0/0, 0*, /, -. They are also used to for data that have not be properly initialized in a program. There are two types on NaN s in the IEEE standard: signaling and quiet. Signaling NaN causes an exception when used as input to arithmetic operations. For example the operation signaling NaN raises an exception. Signaling NaN s are used in situations where the programmer would like to make sure that the program execution be interrupted whenever any NaN values are used in floating point computations. These situations usually mean that there is something wrong with the execution of the program. In mission critical applications, the execution cannot continue until the validity of the execution can be verified with a separate means. For example, software engineers often mark all the uninitialized data as signaling NaN. This practice ensures the detection of using uninitialized data during program execution. David Kirk/NVIDIA and Wen-mei Hwu,

8 Quiet NaN generates a quiet NaN when used as input to arithmetic operations. For example, the operation (1.0 + quiet NaN) generates a quiet NaN. Quiet NaN s are typically used in applications where the user can review the output and decide if the application should be re-run with a different input for more valid results. When the results are printed, Quite NaN s are printed as NaN so that the user can spot them in the output file easily Arithmetic Accuracy and Rounding of Mantissa Now that we have a good understanding of the IEEE floating point format, we now move to the concept of arithmetic accuracy. The accuracy of a floating point arithmetic operation is measured by the maximal error introduced by the operation. The smaller the error is, the higher the accuracy. The most common source of error in floating point arithmetic is when the operation generates a result that cannot be exactly represented and thus requires rounding. Rounding occurs if the mantissa of the result value needs too many bits to be represented exactly. The cause of rounding is typically pre-shifting in floating point arithmetic. When two input operands to a floating-point addition or subtraction have different exponents, the mantissa of the one with the smaller exponent is typically rightshifted until the exponents are equal. As a result, the final result can have more bits than the format can accommodate. This can be illustrated with a simple example based on the 5-bit representation in Table 1. Assume that we need to add 1.00*2-2 (0, 00, 01) to 1.00*2 1 (0, 10, 00), that is, we need to perform 1.00* *2-2. The ideal result would be * 2 1. However, we can easily see that this ideal result is not a representable number in a 5-bit representation. Thus, the best one can do is to generate the closest representable number, which is 1.01 * 2 1. By doing so, we introduce an error, *2 1, which is half the place value of the least significant place. We refer to this as 0.5 ULP (Units in the Last Place). If the hardware is designed to perform arithmetic and rounding operations perfectly, the most error that one should introduce should be no more than 0.5 ULP. This is the accuracy achieved by the addition and subtraction operations in G80/280. In practice, some of the arithmetic hardware units, such as division and transcendental functions, are typically implemented with iterative approximation algorithms. If the hardware does not perform sufficient number of iterations, the result may have an error larger than 0.5 ULP. For example, the division in G80/280 can introduce an error that is twice the place value of the least place of the mantissa Algorithm Considerations Numerical algorithms often need to sum up a large number of values. For example, the dot product in matrix multiplication needs to sum up pair-wise products of input matrix elements. Ideally, the order of summing these values should not affect the final total since addition is an associative operation. However, with finite precision, the order of summing these values can affect the accuracy of the final result. For example, if we need to add four numbers in our 5-bit representation: 1.00* * * *2-2. David Kirk/NVIDIA and Wen-mei Hwu,

9 If we add up the numbers in strict sequential order, we have the following sequence of operations: 1.00* * * *2-2 = 1.00* * *2-2 = 1.00* *2-2 = 1.00*2 1 Note that in the second step and third step, the smaller operand simply disappears because they are too small compared to the larger operand. The pre-shifting of mantissa produces a zero value for the smaller operand. Now, let s consider a parallel algorithm where the first two values are added and the second two operands are added in parallel. The algorithm than add up the pair-wise sum: (1.00* *2 0 ) + (1.00* *2-2 ) = 1.00* *2-1 = 1.01*2 1 Note that the results are different from the sequential result! This is because the sum of the third and fourth values is large enough that it remains non-zero after pre-shifting. This discrepancy between sequential algorithms and parallel algorithms often surprises application developers who are not familiar with floating point precision and accuracy considerations. Although we showed a scenario where a parallel algorithm produced a more accurate result than a sequential algorithm, the reader should be able to come up with a slightly different scenario where the parallel algorithm produces a less accurate result than a sequential algorithm. Experienced application developers either make sure that the variation in the final result can be tolerated or to ensure that the data is grouped in a way that the parallel algorithm results in the most accurate results. David Kirk/NVIDIA and Wen-mei Hwu,

### GPU Floating Point Features

CSE 591: GPU Programming Floating Point Considerations Klaus Mueller Computer Science Department Stony Brook University Objective To understand the fundamentals of floating-point representation To know

### Floating-Point Numbers in Digital Computers

POLYTECHNIC UNIVERSITY Department of Computer and Information Science Floating-Point Numbers in Digital Computers K. Ming Leung Abstract: We explain how floating-point numbers are represented and stored

### Floating Point Arithmetic

Floating Point Arithmetic CS 365 Floating-Point What can be represented in N bits? Unsigned 0 to 2 N 2s Complement -2 N-1 to 2 N-1-1 But, what about? very large numbers? 9,349,398,989,787,762,244,859,087,678

### 2 Computation with Floating-Point Numbers

2 Computation with Floating-Point Numbers 2.1 Floating-Point Representation The notion of real numbers in mathematics is convenient for hand computations and formula manipulations. However, real numbers

### Floating Point Puzzles The course that gives CMU its Zip! Floating Point Jan 22, IEEE Floating Point. Fractional Binary Numbers.

class04.ppt 15-213 The course that gives CMU its Zip! Topics Floating Point Jan 22, 2004 IEEE Floating Point Standard Rounding Floating Point Operations Mathematical properties Floating Point Puzzles For

### Floating point. Today! IEEE Floating Point Standard! Rounding! Floating Point Operations! Mathematical properties. Next time. !

Floating point Today! IEEE Floating Point Standard! Rounding! Floating Point Operations! Mathematical properties Next time! The machine model Chris Riesbeck, Fall 2011 Checkpoint IEEE Floating point Floating

### Systems I. Floating Point. Topics IEEE Floating Point Standard Rounding Floating Point Operations Mathematical properties

Systems I Floating Point Topics IEEE Floating Point Standard Rounding Floating Point Operations Mathematical properties IEEE Floating Point IEEE Standard 754 Established in 1985 as uniform standard for

### Floating point. Today. IEEE Floating Point Standard Rounding Floating Point Operations Mathematical properties Next time.

Floating point Today IEEE Floating Point Standard Rounding Floating Point Operations Mathematical properties Next time The machine model Fabián E. Bustamante, Spring 2010 IEEE Floating point Floating point

### CS101 Lecture 04: Binary Arithmetic

CS101 Lecture 04: Binary Arithmetic Binary Number Addition Two s complement encoding Briefly: real number representation Aaron Stevens (azs@bu.edu) 25 January 2013 What You ll Learn Today Counting in binary

### System Programming CISC 360. Floating Point September 16, 2008

System Programming CISC 360 Floating Point September 16, 2008 Topics IEEE Floating Point Standard Rounding Floating Point Operations Mathematical properties Powerpoint Lecture Notes for Computer Systems:

### Chapter 2 Float Point Arithmetic. Real Numbers in Decimal Notation. Real Numbers in Decimal Notation

Chapter 2 Float Point Arithmetic Topics IEEE Floating Point Standard Fractional Binary Numbers Rounding Floating Point Operations Mathematical properties Real Numbers in Decimal Notation Representation

### Data Representation Floating Point

Data Representation Floating Point CSCI 2400 / ECE 3217: Computer Architecture Instructor: David Ferry Slides adapted from Bryant & O Hallaron s slides via Jason Fritts Today: Floating Point Background:

### What Every Programmer Should Know About Floating-Point Arithmetic

What Every Programmer Should Know About Floating-Point Arithmetic Last updated: October 15, 2015 Contents 1 Why don t my numbers add up? 3 2 Basic Answers 3 2.1 Why don t my numbers, like 0.1 + 0.2 add

### C NUMERIC FORMATS. Overview. IEEE Single-Precision Floating-point Data Format. Figure C-0. Table C-0. Listing C-0.

C NUMERIC FORMATS Figure C-. Table C-. Listing C-. Overview The DSP supports the 32-bit single-precision floating-point data format defined in the IEEE Standard 754/854. In addition, the DSP supports an

### 1.2 Round-off Errors and Computer Arithmetic

1.2 Round-off Errors and Computer Arithmetic 1 In a computer model, a memory storage unit word is used to store a number. A word has only a finite number of bits. These facts imply: 1. Only a small set

### Floating-point Arithmetic. where you sum up the integer to the left of the decimal point and the fraction to the right.

Floating-point Arithmetic Reading: pp. 312-328 Floating-Point Representation Non-scientific floating point numbers: A non-integer can be represented as: 2 4 2 3 2 2 2 1 2 0.2-1 2-2 2-3 2-4 where you sum

### 1. NUMBER SYSTEMS USED IN COMPUTING: THE BINARY NUMBER SYSTEM

1. NUMBER SYSTEMS USED IN COMPUTING: THE BINARY NUMBER SYSTEM 1.1 Introduction Given that digital logic and memory devices are based on two electrical states (on and off), it is natural to use a number

### Binary Addition & Subtraction. Unsigned and Sign & Magnitude numbers

Binary Addition & Subtraction Unsigned and Sign & Magnitude numbers Addition and subtraction of unsigned or sign & magnitude binary numbers by hand proceeds exactly as with decimal numbers. (In fact this

### CS429: Computer Organization and Architecture

CS429: Computer Organization and Architecture Dr. Bill Young Department of Computer Sciences University of Texas at Austin Last updated: September 18, 2017 at 12:48 CS429 Slideset 4: 1 Topics of this Slideset

### 4 Operations On Data 4.1. Foundations of Computer Science Cengage Learning

4 Operations On Data 4.1 Foundations of Computer Science Cengage Learning Objectives After studying this chapter, the student should be able to: List the three categories of operations performed on data.

### Chapter 3 Arithmetic for Computers (Part 2)

Department of Electr rical Eng ineering, Chapter 3 Arithmetic for Computers (Part 2) 王振傑 (Chen-Chieh Wang) ccwang@mail.ee.ncku.edu.tw ncku edu Depar rtment of Electr rical Eng ineering, Feng-Chia Unive

### Characters, Strings, and Floats

Characters, Strings, and Floats CS 350: Computer Organization & Assembler Language Programming 9/6: pp.8,9; 9/28: Activity Q.6 A. Why? We need to represent textual characters in addition to numbers. Floating-point

### Scientific Computing. Error Analysis

ECE257 Numerical Methods and Scientific Computing Error Analysis Today s s class: Introduction to error analysis Approximations Round-Off Errors Introduction Error is the difference between the exact solution

### Inf2C - Computer Systems Lecture 2 Data Representation

Inf2C - Computer Systems Lecture 2 Data Representation Boris Grot School of Informatics University of Edinburgh Last lecture Moore s law Types of computer systems Computer components Computer system stack

### Logic, Words, and Integers

Computer Science 52 Logic, Words, and Integers 1 Words and Data The basic unit of information in a computer is the bit; it is simply a quantity that takes one of two values, 0 or 1. A sequence of k bits

### Chapter 2: Number Systems

Chapter 2: Number Systems Logic circuits are used to generate and transmit 1s and 0s to compute and convey information. This two-valued number system is called binary. As presented earlier, there are many

### Administrivia. CMSC 216 Introduction to Computer Systems Lecture 24 Data Representation and Libraries. Representing characters DATA REPRESENTATION

Administrivia CMSC 216 Introduction to Computer Systems Lecture 24 Data Representation and Libraries Jan Plane & Alan Sussman {jplane, als}@cs.umd.edu Project 6 due next Friday, 12/10 public tests posted

### 3.5 Floating Point: Overview

3.5 Floating Point: Overview Floating point (FP) numbers Scientific notation Decimal scientific notation Binary scientific notation IEEE 754 FP Standard Floating point representation inside a computer

### IEEE-754 floating-point

IEEE-754 floating-point Real and floating-point numbers Real numbers R form a continuum - Rational numbers are a subset of the reals - Some numbers are irrational, e.g. π Floating-point numbers are an

### Computer Systems C S Cynthia Lee

Computer Systems C S 1 0 7 Cynthia Lee 2 Today s Topics LECTURE: Floating point! Real Numbers and Approximation MATH TIME! Some preliminary observations on approximation We know that some non-integer numbers

### IEEE Standard 754 Floating Point Numbers

IEEE Standard 754 Floating Point Numbers Steve Hollasch / Last update 2005-Feb-24 IEEE Standard 754 floating point is the most common representation today for real numbers on computers, including Intel-based

### Module 2: Computer Arithmetic

Module 2: Computer Arithmetic 1 B O O K : C O M P U T E R O R G A N I Z A T I O N A N D D E S I G N, 3 E D, D A V I D L. P A T T E R S O N A N D J O H N L. H A N N E S S Y, M O R G A N K A U F M A N N

### Classes of Real Numbers 1/2. The Real Line

Classes of Real Numbers All real numbers can be represented by a line: 1/2 π 1 0 1 2 3 4 real numbers The Real Line { integers rational numbers non-integral fractions irrational numbers Rational numbers

### 15213 Recitation 2: Floating Point

15213 Recitation 2: Floating Point 1 Introduction This handout will introduce and test your knowledge of the floating point representation of real numbers, as defined by the IEEE standard. This information

### FP_IEEE_DENORM_GET_ Procedure

FP_IEEE_DENORM_GET_ Procedure FP_IEEE_DENORM_GET_ Procedure The FP_IEEE_DENORM_GET_ procedure reads the IEEE floating-point denormalization mode. fp_ieee_denorm FP_IEEE_DENORM_GET_ (void); DeNorm The denormalization

### Number Systems. Binary Numbers. Appendix. Decimal notation represents numbers as powers of 10, for example

Appendix F Number Systems Binary Numbers Decimal notation represents numbers as powers of 10, for example 1729 1 103 7 102 2 101 9 100 decimal = + + + There is no particular reason for the choice of 10,

### The Perils of Floating Point

The Perils of Floating Point by Bruce M. Bush Copyright (c) 1996 Lahey Computer Systems, Inc. Permission to copy is granted with acknowledgement of the source. Many great engineering and scientific advances

### ECE 2020B Fundamentals of Digital Design Spring problems, 6 pages Exam Two 26 February 2014

Instructions: This is a closed book, closed note exam. Calculators are not permitted. If you have a question, raise your hand and I will come to you. Please work the exam in pencil and do not separate

### Computer Arithmetic Ch 8

Computer Arithmetic Ch 8 ALU Integer Representation Integer Arithmetic Floating-Point Representation Floating-Point Arithmetic 1 Arithmetic Logical Unit (ALU) (2) Does all work in CPU (aritmeettis-looginen

### Signed umbers. Sign/Magnitude otation

Signed umbers So far we have discussed unsigned number representations. In particular, we have looked at the binary number system and shorthand methods in representing binary codes. With m binary digits,

### Name: CMSC 313 Fall 2001 Computer Organization & Assembly Language Programming Exam 1. Question Points I. /34 II. /30 III.

CMSC 313 Fall 2001 Computer Organization & Assembly Language Programming Exam 1 Name: Question Points I. /34 II. /30 III. /36 TOTAL: /100 Instructions: 1. This is a closed-book, closed-notes exam. 2. You

### 4 Operations On Data 4.1. Foundations of Computer Science Cengage Learning

4 Operations On Data 4.1 Foundations of Computer Science Cengage Learning Objectives After studying this chapter, the student should be able to: List the three categories of operations performed on data.

### Floating Point Numbers

Floating Point Numbers Summer 8 Fractional numbers Fractional numbers fixed point Floating point numbers the IEEE 7 floating point standard Floating point operations Rounding modes CMPE Summer 8 Slides

### Section 1.4 Mathematics on the Computer: Floating Point Arithmetic

Section 1.4 Mathematics on the Computer: Floating Point Arithmetic Key terms Floating point arithmetic IEE Standard Mantissa Exponent Roundoff error Pitfalls of floating point arithmetic Structuring computations

### Number Systems and Computer Arithmetic

Number Systems and Computer Arithmetic Counting to four billion two fingers at a time What do all those bits mean now? bits (011011011100010...01) instruction R-format I-format... integer data number text

### EC121 Mathematical Techniques A Revision Notes

EC Mathematical Techniques A Revision Notes EC Mathematical Techniques A Revision Notes Mathematical Techniques A begins with two weeks of intensive revision of basic arithmetic and algebra, to the level

### Floating Point Numbers. Lecture 9 CAP

Floating Point Numbers Lecture 9 CAP 3103 06-16-2014 Review of Numbers Computers are made to deal with numbers What can we represent in N bits? 2 N things, and no more! They could be Unsigned integers:

### 8/30/2016. In Binary, We Have A Binary Point. ECE 120: Introduction to Computing. Fixed-Point Representations Support Fractions

University of Illinois at Urbana-Champaign Dept. of Electrical and Computer Engineering ECE 120: Introduction to Computing Fixed- and Floating-Point Representations In Binary, We Have A Binary Point Let

### CHAPTER 2 Data Representation in Computer Systems

CHAPTER 2 Data Representation in Computer Systems 2.1 Introduction 37 2.2 Positional Numbering Systems 38 2.3 Decimal to Binary Conversions 38 2.3.1 Converting Unsigned Whole Numbers 39 2.3.2 Converting

### CHAPTER 2 Data Representation in Computer Systems

CHAPTER 2 Data Representation in Computer Systems 2.1 Introduction 37 2.2 Positional Numbering Systems 38 2.3 Decimal to Binary Conversions 38 2.3.1 Converting Unsigned Whole Numbers 39 2.3.2 Converting

### Signed Binary Numbers

Signed Binary Numbers Unsigned Binary Numbers We write numbers with as many digits as we need: 0, 99, 65536, 15000, 1979, However, memory locations and CPU registers always hold a constant, fixed number

### Computer arithmetics: integers, binary floating-point, and decimal floating-point

n!= 0 && -n == n z+1 == z Computer arithmetics: integers, binary floating-point, and decimal floating-point v+w-w!= v x+1 < x Peter Sestoft 2010-02-16 y!= y p == n && 1/p!= 1/n 1 Computer arithmetics Computer

### Computer Architecture and Organization

3-1 Chapter 3 - Arithmetic Computer Architecture and Organization Miles Murdocca and Vincent Heuring Chapter 3 Arithmetic 3-2 Chapter 3 - Arithmetic Chapter Contents 3.1 Fixed Point Addition and Subtraction

### Principles of Computer Architecture. Chapter 3: Arithmetic

3-1 Chapter 3 - Arithmetic Principles of Computer Architecture Miles Murdocca and Vincent Heuring Chapter 3: Arithmetic 3-2 Chapter 3 - Arithmetic 3.1 Overview Chapter Contents 3.2 Fixed Point Addition

### Independent Representation

Independent Representation 1 Integer magnitude + 1 2 3 4 5 6 7 8 9 0 Float sign sign of exponent mantissa On standard 32 bit machines: INT_MAX = 2147483647 which gives 10 digits of precision, (i.e. the

### The Sign consists of a single bit. If this bit is '1', then the number is negative. If this bit is '0', then the number is positive.

IEEE 754 Standard - Overview Frozen Content Modified by on 13-Sep-2017 Before discussing the actual WB_FPU - Wishbone Floating Point Unit peripheral in detail, it is worth spending some time to look at

### LogiCORE IP Floating-Point Operator v6.2

LogiCORE IP Floating-Point Operator v6.2 Product Guide Table of Contents SECTION I: SUMMARY IP Facts Chapter 1: Overview Unsupported Features..............................................................

### FPSim: A Floating- Point Arithmetic Demonstration Applet

FPSim: A Floating- Point Arithmetic Demonstration Applet Jeffrey Ward Departmen t of Computer Science St. Cloud State University waje9901@stclouds ta te.edu Abstract An understan ding of IEEE 754 standar

### Imelda C. Go, South Carolina Department of Education, Columbia, SC

PO 082 Rounding in SAS : Preventing Numeric Representation Problems Imelda C. Go, South Carolina Department of Education, Columbia, SC ABSTRACT As SAS programmers, we come from a variety of backgrounds.

### Lecture Notes: Floating-Point Numbers

Lecture Notes: Floating-Point Numbers CS227-Scientific Computing September 8, 2010 What this Lecture is About How computers represent numbers How this affects the accuracy of computation Positional Number

### Double Precision Floating Point Core VHDL

Double Precision Floating Point Core VHDL Introduction This document describes the VHDL double precision floating point core, posted at www.opencores.org. The Verilog version of the code is in folder fpu_double,

### IEEE Floating Point Numbers Overview

COMP 40: Machine Structure and Assembly Language Programming (Fall 2015) IEEE Floating Point Numbers Overview Noah Mendelsohn Tufts University Email: noah@cs.tufts.edu Web: http://www.cs.tufts.edu/~noah

### Co-processor Math Processor. Richa Upadhyay Prabhu. NMIMS s MPSTME February 9, 2016

8087 Math Processor Richa Upadhyay Prabhu NMIMS s MPSTME richa.upadhyay@nmims.edu February 9, 2016 Introduction Need of Math Processor: In application where fast calculation is required Also where there

### COMP 122/L Lecture 2. Kyle Dewey

COMP 122/L Lecture 2 Kyle Dewey Outline Operations on binary values AND, OR, XOR, NOT Bit shifting (left, two forms of right) Addition Subtraction Twos complement Bitwise Operations Bitwise AND Similar

### What Every Computer Scientist Should Know About Floating Point Arithmetic

What Every Computer Scientist Should Know About Floating Point Arithmetic E Note This document is an edited reprint of the paper What Every Computer Scientist Should Know About Floating-Point Arithmetic,

### Data Representation and Introduction to Visualization

Data Representation and Introduction to Visualization Massimo Ricotti ricotti@astro.umd.edu University of Maryland Data Representation and Introduction to Visualization p.1/18 VISUALIZATION Visualization

### 4/8/17. Admin. Assignment 5 BINARY. David Kauchak CS 52 Spring 2017

4/8/17 Admin! Assignment 5 BINARY David Kauchak CS 52 Spring 2017 Diving into your computer Normal computer user 1 After intro CS After 5 weeks of cs52 What now One last note on CS52 memory address binary

### Computer Architecture Review. Jo, Heeseung

Computer Architecture Review Jo, Heeseung Computer Abstractions and Technology Jo, Heeseung Below Your Program Application software Written in high-level language System software Compiler: translates HLL

### A complement number system is used to represent positive and negative integers. A complement number system is based on a fixed length representation

Complement Number Systems A complement number system is used to represent positive and negative integers A complement number system is based on a fixed length representation of numbers Pretend that integers

### CS101 Introduction to computing Floating Point Numbers

CS101 Introduction to computing Floating Point Numbers A. Sahu and S. V.Rao Dept of Comp. Sc. & Engg. Indian Institute of Technology Guwahati 1 Outline Need to floating point number Number representation

### 10.1. Unit 10. Signed Representation Systems Binary Arithmetic

0. Unit 0 Signed Representation Systems Binary Arithmetic 0.2 BINARY REPRESENTATION SYSTEMS REVIEW 0.3 Interpreting Binary Strings Given a string of s and 0 s, you need to know the representation system

### UNIT - I: COMPUTER ARITHMETIC, REGISTER TRANSFER LANGUAGE & MICROOPERATIONS

UNIT - I: COMPUTER ARITHMETIC, REGISTER TRANSFER LANGUAGE & MICROOPERATIONS (09 periods) Computer Arithmetic: Data Representation, Fixed Point Representation, Floating Point Representation, Addition and

### EC2303-COMPUTER ARCHITECTURE AND ORGANIZATION

EC2303-COMPUTER ARCHITECTURE AND ORGANIZATION QUESTION BANK UNIT-II 1. What are the disadvantages in using a ripple carry adder? (NOV/DEC 2006) The main disadvantage using ripple carry adder is time delay.

### Arithmetic Logic Unit

Arithmetic Logic Unit A.R. Hurson Department of Computer Science Missouri University of Science & Technology A.R. Hurson 1 Arithmetic Logic Unit It is a functional bo designed to perform "basic" arithmetic,

### Description Hex M E V smallest value > largest denormalized negative infinity number with hex representation 3BB0 ---

CSE2421 HOMEWORK #2 DUE DATE: MONDAY 11/5 11:59pm PROBLEM 2.84 Given a floating-point format with a k-bit exponent and an n-bit fraction, write formulas for the exponent E, significand M, the fraction

### More Programming Constructs -- Introduction

More Programming Constructs -- Introduction We can now examine some additional programming concepts and constructs Chapter 5 focuses on: internal data representation conversions between one data type and

### Lecture 10. Floating point arithmetic GPUs in perspective

Lecture 10 Floating point arithmetic GPUs in perspective Announcements Interactive use on Forge Trestles accounts? A4 2012 Scott B. Baden /CSE 260/ Winter 2012 2 Today s lecture Floating point arithmetic

### Divisibility Rules and Their Explanations

Divisibility Rules and Their Explanations Increase Your Number Sense These divisibility rules apply to determining the divisibility of a positive integer (1, 2, 3, ) by another positive integer or 0 (although

### 18-642: Floating Point Blues

18-642: Floating Point Blues 10/23/2017 1 Floating Point Math Anti-Patterns: Not accounting for roundoff errors Tests for floating point equality Not handling special values Float used if integer does

### Computer Representation of Numbers and Computer Arithmetic

Computer Representation of Numbers and Computer Arithmetic January 17, 2017 Contents 1 Binary numbers 6 2 Memory 9 3 Characters 12 1 c A. Sandu, 1998-2015. Distribution outside classroom not permitted.

### Arithmetic and Bitwise Operations on Binary Data

Arithmetic and Bitwise Operations on Binary Data CSCI 224 / ECE 317: Computer Architecture Instructor: Prof. Jason Fritts Slides adapted from Bryant & O Hallaron s slides 1 Boolean Algebra Developed by

### Midterm Exam Answers Instructor: Randy Shepherd CSCI-UA.0201 Spring 2017

Section 1: Multiple choice (select any that apply) - 20 points 01. Representing 10 using the 4 byte unsigned integer encoding and using 4 byte two s complements encoding yields the same bit pattern. (a)

### DRAM uses a single capacitor to store and a transistor to select. SRAM typically uses 6 transistors.

Data Representation Data Representation Goal: Store numbers, characters, sets, database records in the computer. What we got: Circuit that stores 2 voltages, one for logic 0 (0 volts) and one for logic

### 1010 2?= ?= CS 64 Lecture 2 Data Representation. Decimal Numbers: Base 10. Reading: FLD Digits: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9

CS 64 Lecture 2 Data Representation Reading: FLD 1.2-1.4 Decimal Numbers: Base 10 Digits: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 Example: 3271 = (3x10 3 ) + (2x10 2 ) + (7x10 1 ) + (1x10 0 ) 1010 10?= 1010 2?= 1

### Data Representation. DRAM uses a single capacitor to store and a transistor to select. SRAM typically uses 6 transistors.

Data Representation Data Representation Goal: Store numbers, characters, sets, database records in the computer. What we got: Circuit that stores 2 voltages, one for logic ( volts) and one for logic (3.3

### 17. Instruction Sets: Characteristics and Functions

17. Instruction Sets: Characteristics and Functions Chapter 12 Spring 2016 CS430 - Computer Architecture 1 Introduction Section 12.1, 12.2, and 12.3 pp. 406-418 Computer Designer: Machine instruction set

### Lecture 5. Computer Arithmetic. Ch 9 [Stal10] Integer arithmetic Floating-point arithmetic

Lecture 5 Computer Arithmetic Ch 9 [Stal10] Integer arithmetic Floating-point arithmetic ALU ALU = Arithmetic Logic Unit (aritmeettis-looginen yksikkö) Actually performs operations on data Integer and

### CSC201, SECTION 002, Fall 2000: Homework Assignment #2

1 of 7 11/8/2003 7:34 PM CSC201, SECTION 002, Fall 2000: Homework Assignment #2 DUE DATE Monday, October 2, at the start of class. INSTRUCTIONS FOR PREPARATION Neat, in order, answers easy to find. Staple

### Representation of Numbers and Arithmetic in Signal Processors

Representation of Numbers and Arithmetic in Signal Processors 1. General facts Without having any information regarding the used consensus for representing binary numbers in a computer, no exact value

### Digital Signal Processing Introduction to Finite-Precision Numerical Effects

Digital Signal Processing Introduction to Finite-Precision Numerical Effects D. Richard Brown III D. Richard Brown III 1 / 9 Floating-Point vs. Fixed-Point DSP chips are generally divided into fixed-point

### CS311 Lecture: Representing Information in Binary Last revised 8/2707

CS311 Lecture: Representing Information in Binary Last revised 8/2707 Objectives: 1. To review binary representation for unsigned integers 2. To introduce octal and hexadecimal shorthands 3. To review

### Digital Computers and Machine Representation of Data

Digital Computers and Machine Representation of Data K. Cooper 1 1 Department of Mathematics Washington State University 2013 Computers Machine computation requires a few ingredients: 1 A means of representing

### Learning Log Title: CHAPTER 3: ARITHMETIC PROPERTIES. Date: Lesson: Chapter 3: Arithmetic Properties

Chapter 3: Arithmetic Properties CHAPTER 3: ARITHMETIC PROPERTIES Date: Lesson: Learning Log Title: Date: Lesson: Learning Log Title: Chapter 3: Arithmetic Properties Date: Lesson: Learning Log Title:

### Arithmetic and Logical Operations

Arithmetic and Logical Operations 2 CMPE2c x +y + sum Or in tabular form Binary Addition Carry Out Sum B A Carry In Binary Addition And as a full adder a b co ci sum 4-bit Ripple-Carry adder: Carry values

### Arithmetic Operations

Arithmetic Operations Arithmetic Operations addition subtraction multiplication division Each of these operations on the integer representations: unsigned two's complement 1 Addition One bit of binary

### Truncation Errors. Applied Numerical Methods with MATLAB for Engineers and Scientists, 2nd ed., Steven C. Chapra, McGraw Hill, 2008, Ch. 4.

Chapter 4: Roundoff and Truncation Errors Applied Numerical Methods with MATLAB for Engineers and Scientists, 2nd ed., Steven C. Chapra, McGraw Hill, 2008, Ch. 4. 1 Outline Errors Accuracy and Precision

### The Design and Implementation of a Rigorous. A Rigorous High Precision Floating Point Arithmetic. for Taylor Models

The and of a Rigorous High Precision Floating Point Arithmetic for Taylor Models Department of Physics, Michigan State University East Lansing, MI, 48824 4th International Workshop on Taylor Methods Boca

### Chapter 2 Bits, Data Types, and Operations

Chapter 2 Bits, Data Types, and Operations How do we represent data in a computer? At the lowest level, a computer is an electronic machine. works by controlling the flow of electrons Easy to recognize