Written Homework 3. Floating-Point Example (1/2)

Transcription

1 Written Homework 3 Assigned on Tuesday, Feb 19 Due Time: 11:59pm, Feb 26 on Tuesday Problems: 3.22, 3.23, 3.24, 3.41, 3.43 Note: You have 1 week to work on homework 3. 3 Floating-Point Example (1/2) Q: How to represent (= ) 0.75 = ( ) //normalized number S = 1 Fraction = Stored exponent = 1 + Bias Single: = 126 = Double: = 1022 = Single: Double:

2 Floating-Point Example (2/2) Q: What decimal value is represented by the following single-precision floating-point number? S = 1 Stored exponent = = 129 Fraction = x = ( 1) 1 ( ) 2 ( ) = ( 1) = Denormalized Numbers Exponent = 0 Þ hidden bit becomes 0, i.e. S -Bias x = (-1) (0+ Fraction) 2 n Smaller than normalized 1.xxxx numbers n Allows for gradual underflow n Denormalized# whose fraction = 0? x = (-1) S (0+ 0) 2 -Bias = ±0.0 Two representations of 0.0! 6 2

3 Representation of Infinity and NaN Exponent = , Fraction = ±Infinity e.g., 1.0/0.0 Can be used in subsequent calculations, avoiding the need for overflow check Exponent = , Fraction Not-a-Number (NaN) Indicates illegal or undefined result e.g., 0.0 / 0.0, Inf - Inf Can still be used in subsequent calculations 7 IEEE 754 Encoding of Floating Point Numbers 0 is a special case in the IEEE 754 Std. Now, it seems we have no way to represent 1.0, which would be 1.0x2 0 (i.e., exponent zero, multiplies the hidden one)! 8 3

4 What is Floating-Point# s Precision? S Exponent Precision: Minimum difference between two floating point numbers. For single (23-bit fraction): approximately 2 23 Equivalent to 23 log decimal digits of precision Not sufficient for scientific computing! For double (52-bit fraction): approximately 2 52 Equivalent to 52 log decimal digits of precision also called machine epsilon Minimum epsilon s.t. 1 + epsilon > 1. Fraction 9 Floating-Point Addition (decimal) First, consider a decimal example (suppose 4 digits of significand and 2 digits of exponent) Align decimal points (equal exponent) 4 Steps Shift the number with smaller exponent Now, exponents are equal 2. Add significands = Normalize result & check for over/underflow Round and renormalize the output if necessary

5 Floating-Point Addition (binary) Similarly, consider a 4-digit binary example (i.e., ) Step 1. Align binary points (s.t. equal exponent) Shift the number with smaller exponent Step 2. Add significands = Step 3. Normalize result & check for over/underflow , //-4 is between -127 and 128. Step 4. Round and renormalize if necessary (no change) = The FP Adder Hardware Step 1 Larger exp Smaller frac Larger frac Step 2 New faction Step 3 Step

6 FP Adder Hardware It is much more complex than Integer Adder Doing it in one clock cycle would make the clock cycle too long! but using a slower clock would penalize all instructions. So, FP Adder takes several cycles 14 Next: Floating Point Multiplication Given two Operands (i.e., 2 normalized inputs) as follows: (-1) S1 m1 2 E1 // m1 is the significand: 1.xxxxx (-1) S2 m2 2 E2 // m2 is the significand Exact Result? It is: (-1) S m 2 E Sign s: s = 1 if s1 ¹ s2; s = 0 otherwise Significand m: m1 * m2 //i.e., multiply two significands Exponent E: E1 + E2 Fixing result: Round the output m to fit the significand precision Check for overflow if E out of range Implementation: The major part is multiplying 2 significands 15 6

7 Floating-Point Multiplication Again, first consider a decimal example (suppose 4 digits of significand and 2 digits of exponent) Add exponents New exponent = = 5 2. Multiply the two significands = Þ Normalize result & check for over/underflow Round and renormalize if necessary Determine sign of result from signs of operands Floating-Point Multiplication Now, let s consider a 4-digit binary example ( ) 1. Add exponents = 3 (not biased) 2. Multiply significands (see next slide) = Þ Normalize result & check for over/underflow // no over/underflow 4. Round and renormalize if necessary // (no change) 5. Determine sign: +ve ve Þ ve =

8 1.000 x FP MIPS Instructions (1/2) Floating Point Hardware is an adjunct processor that extends the existing MIPS ISA It is called Coprocessor 1 (c1) There are 32 separate FP registers 32 single-precision: $f0, $f1, $f31 Can be paired for storing double-precision: $f0/$f1, $f2/$f3, Hence, 16 double-precision registers FP MIPS Instructions can operate only on FP registers Also, they have special Load and Store instructions lwc1, swc1 //single precision ldc1, sdc1 //double precision e.g., lwc1 $f8, 32($sp) 19 8

9 CPU (central processing unit) FPU (floating point unit) "coprocessor 1" mfc1 register $0,..,$31 integer arithmetic division multiplication logical ops mtc1 register $f0,.. $f31 floating point arithmetic divison multiplication int float convert sw lwc1 lw swc1 Memory (2^32 bytes) 20 FP MIPS Instructions (2/2) Single-precision arithmetic: add.s, sub.s, mul.s, div.s e.g., add.s $f0, $f1, $f6 //F0=F1+F6 Double-precision arithmetic: add.d, sub.d, mul.d, div.d e.g., mul.d $f4, $f4, $f6 //F4=F4*F6 Comparison: c.xx.s, c.xx.d (xx could be eq, lt, le, ) This instruction will set a special FP condition-code bit e.g. c.lt.s $f3, $f4 Branch on FP condition code true or false bc1t ( branch C1 true ), bc1f ( branch C1 false ) e.g., bc1t TargetLabel 21 9

10 FP MIPS Example: F to C C code: float f2c (float fahr) { return ((5.0/9.0)*(fahr )); } fahr in $f12, result in $f0, literals stored in Global memory Compiled MIPS code: f2c: lwc1 $f16, const5($gp) lwc1 $f18, const9($gp) div.s $f16, $f16, $f18 lwc1 $f18, const32($gp) sub.s $f18, $f12, $f18 mul.s $f0, $f16, $f18 jr $ra //F16 = 5.0/9.0 //product result 22 Accuracy of Floating Point Numbers Only a subset of real numbers can be represented by computer! 26 10

11 Accurate Arithmetic Fact: Floating-point numbers are approximations of real numbers! 53 bits vs infinite number of real numbers (consider [0.0, 1.0]) IEEE Std 754 offers a rounding control. Allow programmer to fine-tune numerical behavior of a computation HW always keeps 2 extra bits of precision (guard, round) They are used during intermediate computations Not all hardware implement all Rounding Control options Most programming languages and libraries just use the default. 27 Accurate Arithmetic Guard & Round bits IEEE 754 standard specifies the use of 2 extra bits on the right during intermediate calculations Guard bit and Round bit Example: Add and assuming 3 significant digits and without guard and round bits = With guard and round bits ROUND

12 IEEE Std 754 has 4 different rounding modes 1st is the default; Others are called directed rounding. Round to Nearest round to the nearest value And Ties to Even : If the number falls midway, it is rounded to the nearest even number Round toward 0 directed rounding towards zero (or truncation) Round toward + directed rounding towards positive infinity (ceiling) Round toward directed rounding towards negative infinity (floor) $1.40 $1.60 $1.50 $2.50 -$1.50 Round2Nearest $1.00 $2.00 $2.00 $2.00 -$ A conceptual view: First compute exact result Then make it fit into the desired precision Possibly round to fit into significand Examples Rounding modes (illustrate with $ rounding) $1.40 $1.60 $1.50 $2.50 -$1.50 Zero $1.00 $1.00 $1.00 $2.00 -$ $1.00 $1.00 $1.00 $2.00 -$ $2.00 $2.00 $2.00 $3.00 -$1.00 Nearest even $1.00 $2.00 $2.00 $2.00 -$2.00 Rounding methods in case of tie cases (fraction = 0.5) No problems in case of fraction 0.5 However, IRS always round up 0.5! Fused Multiply Add: a = a + (b x c). //round only once at the end! 30 12

13 A Deep Learning Example Is it like alchemy? 31 Interpretation of Data The BIG Picture Bits have no inherent meaning! Could be anything E.g., 32 bits, what does it mean? Interpretation depends on the instruction applied. Computer representations of real numbers have limited range and limited precision You must know they are only approximations (with rounding errors)