Computer Architecture, Lecture 14: Does your computer know how to add?

Transcription

1 Computer Architecture, Lecture 14: Does your computer know how to add? Hossam A. H. Fahmy Cairo University Electronics and Communications Engineering 1 / 25

2 A strange behavior What do you expect from this code? 1 #include<s t d i o. h> 2 3 int main ( void ) 4 { 5 f l o a t x=+0.0, y= 0.0; 6 7 p r i n t f ( \n\ t \ t The amazing r e s u l t s : \n ) ; 8 p r i n t f ( \n 10ˆ30 = %f, E30 ) ; 9 p r i n t f ( \n 10ˆ30 10ˆ30 = 0= %f, E E30 ) ; 10 p r i n t f ( \n 10ˆ60 = 10ˆ30 10ˆ30 = %f, E E30 ) ; 11 p r i n t f ( \n 10ˆ60 = 10ˆ30 10ˆ30 = %f, ( f l o a t ) ( E E30 ) ) ; 12 p r i n t f ( \n 1. 0 / ( ) = %f, 1. 0 / x ) ; 13 p r i n t f ( \n 1. 0 / ( 0. 0 ) = %f, 1. 0 / y ) ; 14 p r i n t f ( \n ( 0.0)= %f, x+y ) ; 15 p r i n t f ( \n 0.0 ( 0.0)= %f, y y ) ; 16 p r i n t f ( \n 1. 0 / ( ) + 1/( 0.0)= %f, ( 1. 0 / x ) +(1.0/ y ) ) ; 17 p r i n t f ( \n\n\ t+ /+ /+ /+ /+ /+ /+ /+ /+ /+ /\n\n ) ; 18 } 2 / 25

3 Simple results! Here is the output. 1 The amazing r e s u l t s : ˆ30 = ˆ30 10ˆ30 = 0= ˆ60 = 10ˆ30 10ˆ30 = ˆ60 = 10ˆ30 10ˆ30 = i n f / ( ) = i n f / ( 0. 0 ) = i n f ( 0.0)= ( 0.0)= / ( ) + 1/( 0.0)= nan /+ /+ /+ /+ /+ /+ /+ /+ /+ / 3 / 25

4 Scientific notation The value of a number in scientific notation has six attributes: ± d 0 d 1 d 2 d t β ±exp The computer representation of floating point numbers is similar. 4 / 25

5 Simple tests Consider the following C code 1 #include <s t d i o. h> 2 3 int main ( void ) 4 { 5 double x, y ; 6 double twox, threex ; 7 double z1, z2 ; 8 9 x = 0. 1 ; y = 0. 3 ; twox = 2. 0 x ; t h r e e x = 3. 0 x ; z1= t h r e e x y ; 14 z2= twox y + x ; p r i n t f ( \n 3x y = %e, z1 ) ; 17 p r i n t f ( \n 3x y = %f, z1 ) ; 18 p r i n t f ( \n 3x y = %40.40 f, z1 ) ; 19 p r i n t f ( \n 3x y = %e \n, 3. 0 x y ) ; p r i n t f ( \n 2x y+x= %e \n, z2 ) ; p r i n t f ( \n ( 3 x y ) /(2 x y+x )= %e or %e \n, z1 / z2, ( 3. 0 x y ) / ( 2. 0 x y+ x ) ) ; p r i n t f ( \n (4/3 1) 3 1 = %e \n\n, ( 4. 0 / ) ) ; 26 } 5 / 25

6 Results of simple tests Once we compile and run this code the result is: 1 2 3x y = e x y = x y = x y = e x y+x= e (3x y ) /(2x y+x )= e+00 or e (4/3 1) 3 1 = e 16 6 / 25

7 Do we need decimal? (1/10) β=10 = (0.1) β=10 but in binary it is ( ) β=2 which the computer rounds into a finite representation. For a computer using binary64, if y = 0.30 and x = 0.10 then 3x y = Furthermore, 2x y + x = Leading to the wonderful surprise that 3x y 2x y + x = 2! (x=0.1,y=0.3) For a human, is kg = 0.05 kg? 7 / 25

8 Humans and decimal numbers If both measurements are normalized to and stored in a format with 16 digits as ( ) they are indistinguishable and give the incorrect impression of a much higher accuracy ( kg). To maintain the distinction, we should store first measurement second measurement with all those leading zeros. Both are members of the same cohort. 8 / 25

9 IEEE standard The old IEEE 754 standard of 1985 had only binary floating point formats. The revised IEEE standard includes also decimal64 and decimal128 formats. Sign Combination Trailing Significand ± exponent and MSD t = 10J bits 64 bits: 1 bit 13 bits, bias = bits, digits 128 bits: 1 bit 17 bits, bias = bits, digits The IEEE standard specifies five exceptional conditions that may occur during an arithmetic operation: invalid, division by zero, overflow, underflow, and inexact result. It also specifies five rounding directions for inexact results. 9 / 25

10 Real numbers to floating numbers RA RZ x x x x + odd even 0 even odd RNE RNA RNZ Note the difference between RNE, RNA, and RNZ in tie cases. 10 / 25

11 Back to addition Decimal examples: ? = / 25

12 Multiplication 1 No alignment is necessary. 2 Multiply the significands. 3 Add the exponents. 4 The sign bit of the result is the XOR of the two operand signs. Is it really that simple? 12 / 25

13 Division 1 No alignment is necessary. 2 Divide the significands. 3 Subtract the exponents. 4 The sign bit of the result is the XOR of the two operand signs. You know it is not that simple! 13 / 25

14 What is different in DFP? Decimal formats may have leading zeros, and may be in either binary integer decimal (BID) or densely packed decimal (DPD) encoding. 14 / 25

15 Energy The use of hardware for decimal operations instead of software leads to a much shorter time to finish the arithmetic operation (factor may be 100 to 1 or more) and no energy consumed in overheads such as fetching and decoding of software instructions. However, the additional circuits consume static power when idle and burn energy. In general, the use of decimal HW is much more energy efficient than SW. 15 / 25

16 Verification Produce high quality test vectors for the various arithmetic operations Our test vectors discovered bugs in several designs: in all the decimal64 and decimal128 designs of SilMinds floating point arithmetic operations. In fact this project started in order to verify those specific designs which were subsequently corrected. in the FMA and Sqrt operations of the decimal floating point library decnumber developed by Dr. Mike Cowlishaw while at IBM. We reported the FMA errors in July and August of 2010 and the SQRT error in December 2010 all against version in the FMA for decimal128 in the Intel decimal floating-point math library developed by Dr. Marius Cornea from Intel. We reported the errors in July 2011 against version 2.0 which was subsequently fixed in version 2.0 update 1 of August (See 16 / 25

17 World published research Universities (representative list) Wisconsin, USA: DPD add, mul, div, sqrt. SDC, Spain: DPD mul, elementary functions. Malaca, Spain: rounding for BID, mul, add. Sh-B, Iran: redundant decimal. Cairo, Egypt: DPD add, mul, div, sqrt, fma, energy, verification. Sask., Canada: DPD log and antilog. Utah, USA: Complete SW decimal library and adaptation of gcc. Companies (complete list) IBM, UK: SW implementation of DPD. IBM, USA: DPD HW add, mul, div, sqrt, and SW DFPAL. Intel, USA: SW implementation of BID. SAP, Germany: All internal representations are now DPD. SilMinds, Egypt: DPD HW for add, mul, div, sqrt, fma, pow,.... Fujitsu, Japan: DPD HW add, mul, and div. 17 / 25

18 Patents in USA 1 R. Samy, H. A. H. Fahmy, T. Eldeeb, R. Raafat, Y. Farouk, M. Elkhouly, and A. Mohamed, Decimal floating-point fused multiply-add unit, Apr US Patent number 8,694, A. Mohamed, H. A. H. Fahmy, R. Raafat, Y. Farouk, M. Elkhouly, R. Samy, and T. Eldeeb, Rounding unit for decimal floating-point division, June US Patent number 8,751, T. ElDeeb, H. A. H. Fahmy, and M. Y. Hassan, Decimal elementary functions computation, July US Patent number 8,788, A. Mohamed, R. Raafat, H. A. H. Fahmy, T. Eldeeb, Y. Farouk, R. Samy, and M. Elkhouly, Parallel redundant decimal fused-multiply-add circuit, Aug US Patent number 8,805, R. Raafat, A. Mohamed, H. A. H. Fahmy, Y. Farouk, M. Elkhouly, T. Eldeeb, and R. Samy, Decimal floating-point square-root unit using Newton-Raphson iterations, Aug US Patent number 8,812, A. A. Ayoub and H. A. H. Fahmy, BID to BCD/DPD converters, Sept US Patent number 9,134, A. A. Ayoub, H. A. H. Fahmy, and T. Eldeeb, DPD/BCD to BID converters, Sept US Patent number 9,143, T. Eldeeb, H. A. H. Fahmy, A. Elhosiny, M. Y. Hassan, Y. Aly, and R. Raafat, Decimal floating-point processor, Apr US Patent number 9,323, / 25

19 Decimal future Inclusion in language standards (already in C, Fortran, Python, and COBOL). Wider adoption in industry. Use of decimal in new applications. HW implementation of decimal elementary functions. Units combining binary and decimal. Verification engines for compliance. 19 / 25

20 Real world problems Check for some disasters caused by numerical errors. 1 Patriot Missile Failure 2 Explosion of the Ariane 5 3 The Vancouver Stock Exchange 4 Rounding error changes Parliament makeup / 25

21 Optimization! Check this code: (adapted from The pitfalls of verifying floating-point computations by David Monniaux) 1 #include<s t d i o. h> 2 3 #include<math. h> 4 5 double modulo ( double x, double mini, double maxi ) { 6 double d e l t a = maxi mini ; 7 double d e c l = x mini ; 8 double q = decl / delta ; 9 return x f l o o r ( q ) d e l t a ; 10 } int main ( ) { 13 double r = modulo ( n e x t a f t e r ( , 0. 0 ), 180.0, ) ; 14 p r i n t f ( \n The v a l u e i s %40.40 f \n, r ) ; 15 } 21 / 25

22 Optimization! Check this code: (adapted from The pitfalls of verifying floating-point computations by David Monniaux) 1 #include<s t d i o. h> 2 3 #include<math. h> 4 5 double modulo ( double x, double mini, double maxi ) { 6 double d e l t a = maxi mini ; 7 double d e c l = x mini ; 8 double q = decl / delta ; 9 return x f l o o r ( q ) d e l t a ; 10 } int main ( ) { 13 double r = modulo ( n e x t a f t e r ( , 0. 0 ), 180.0, ) ; 14 p r i n t f ( \n The v a l u e i s %40.40 f \n, r ) ; 15 } With gcc -O0 we get: With gcc -O1 we get: / 25

23 You cannot always optimize! 1 #include<s t d i o. h> 2 3 #include<math. h> 4 5 double modulo ( double x, double mini, double maxi ) { 6 double d e l t a = maxi mini ; 7 double d e c l = x mini ; 8 p r i n t f ( \n d e c l=%f \n, d e c l ) ; 9 double q = decl / delta ; 10 return x f l o o r ( q ) d e l t a ; 11 } int main ( ) { 14 double r = modulo ( n e x t a f t e r ( , 0. 0 ), 180.0, ) ; 15 p r i n t f ( \n The v a l u e i s %40.40 f \n, r ) ; 16 } With gcc -O0 we get: With gcc -O1 we get: / 25

24 Solution? The problem disappears in 1 double modulo ( double x, double mini, double maxi ) { 2 double d e l t a = maxi mini ; 3 return x f l o o r ( ( x mini ) / d e l t a ) d e l t a ; 4 } Quote: If hundreds of thousands of systems featuring a defective component are manufactured every year, there will be real failures happening when the systems are deployed and they will be very difficult to recreate and diagnose. If the system is implemented in single precision, the odds are considerably higher with the same probabilistic assumptions: a single 100 Hz system would break down twice a day. 24 / 25

25 Summary Arithmetic blocks are everywhere in digital circuits. It is possible to change the representation in order to ease the implementation of certain tasks. Floating point programs and circuits are tricky : hard to design and hard to verify. 25 / 25