A Dynamic Program Analysis to find Floating-Point Accuracy Problems

Transcription

1 1 A Dynamic Program Analysis to find Floating-Point Accuracy Problems Florian Benz fbenz@stud.uni-saarland.de Andreas Hildebrandt andreas.hildebrandt@uni-mainz.de Sebastian Hack hack@cs.uni-saarland.de PLDI 2012, Beijing June 13, 2012

2 2 Introduction Floating-point arithmetic is ubiquitous Almost every language has a floating-point data type Most PCs and supercomputers have floating-point accelerators Not well understood by most developers

3 2 Introduction Floating-point arithmetic is ubiquitous Almost every language has a floating-point data type Most PCs and supercomputers have floating-point accelerators Not well understood by most developers Our Contribution A dynamic program analysis that assists developers in understanding and tracking down floating-point arithmetic issues in real-world programs.

4 3 Insufficient Precision Floating-Point Arithmetic Problems Finite Precision Cancellation Finding the Cause

5 4 Insufficient Precision Floating-Point Arithmetic Problems Finite Precision Cancellation Finding the Cause

6 5 Problem: Insufficient Precision float e = f; float sum = 1.0f; int i; for (i = 0; i < 5; i++) { sum += e; } Finally sum = 1.0

7 5 Problem: Insufficient Precision float e = f; float sum = 1.0f; int i; for (i = 0; i < 5; i++) { sum += e; } Finally sum = 1.0 Higher precision yields sum = Single precision machine epsilon: f

8 6 Solution: Side by Side in Higher Precision Original float e = f; float sum = 1.0f; int i; for (i = 0; i < 5; i++) { sum += e; }

9 6 Solution: Side by Side in Higher Precision Original float e = f; float sum = 1.0f; int i; for (i = 0; i < 5; i++) { sum += e; } Higher precision sum += e; Side-by-side computation in higher precision

10 6 Solution: Side by Side in Higher Precision Original float e = f; float sum = 1.0f; int i; for (i = 0; i < 5; i++) { sum += e; } Higher precision e = ; sum = 1.0; sum += e; Side-by-side computation in higher precision Shadowing every floating-point value

11 7 Solution: Side by Side in Higher Precision Original float e = f; float sum = 1.0f; int i; for (i = 0; i < 5; i++) { sum += e; } Higher precision e = ; sum = 1.0; sum += e; Iteration Single precision Higher precision

12 8 Error Measurement relative error = exact value approximate value exact value

13 8 Error Measurement relative error = exact value approximate value exact value Approximate exact value with higher precision value

14 8 Error Measurement relative error = exact value approximate value exact value Approximate exact value with higher precision value =

15 8 Error Measurement relative error = exact value approximate value exact value Approximate exact value with higher precision value = Relative errors smaller than machine epsilon are unavoidable float double

16 9 Insufficient Precision Floating-Point Arithmetic Problems Finite Precision Cancellation Finding the Cause

17 10 Problem: Cancellation Benign

18 10 Problem: Cancellation Benign exact inexact

19 10 Problem: Cancellation Benign exact inexact relative error

20 10 Problem: Cancellation Benign exact inexact relative error Catastrophic exact inexact

21 10 Problem: Cancellation Benign exact inexact relative error Catastrophic exact inexact relative error

22 11 Solution: Cancellation Badness Benign exact inexact relative error (canceled) 7 (exact) + 1 = 0 0

23 11 Solution: Cancellation Badness Benign Catastrophic exact inexact relative error (canceled) 7 (exact) + 1 = 0 exact inexact relative error (canceled) 6 (exact) + 1 = 1

24 12 Insufficient Precision Floating-Point Arithmetic Problems Finite Precision Cancellation Finding the Cause

25 13 Problem: Finding the Cause 1 float e = f; 2 float x = 0.5f; 3 float y = 1.0f + x; 4 float more = y + e; 5 float diff e = more - y; 6 float diff 0 = diff e - e; 7 float zero = diff 0 + diff 0; 8 float result = 2 * zero; result = Higher precision yields result = 0

26 13 Solution: Light-Weight Slicing 1 float e = f; 2 float x = 0.5f; 3 float y = 1.0f + x; 4 float more = y + e; 5 float diff e = more - y; 6 float diff 0 = diff e - e; 7 float zero = diff 0 + diff 0; 8 float result = 2 * zero; Add (4) Sub (5) Sub (6) Add (7) 0 0 Add (8) result = Higher precision yields result = 0

27 14 Insufficient Precision Floating-Point Arithmetic Problems Finite Precision Cancellation Finding the Cause

28 Problem: Finite Precision 1 2 if n = 0, u n = 4 if n = 1, u n u n 1 u n 2 if n > 1. Mathematically correct lim u n = 6 n 1 Muller et al.: Handbook of Floating-Point Arithmetic, Birkhäuser,

29 Problem: Finite Precision 1 2 if n = 0, u n = 4 if n = 1, u n u n 1 u n 2 if n > 1. Mathematically correct For all finite precisions lim u n = 6 n lim u n = 100 n 1 Muller et al.: Handbook of Floating-Point Arithmetic, Birkhäuser,

30 16 Analysis of the Problem 100 u Double precision (53 bit) Higher precision (here: 128 bit) Correct n Fully automatic analysis detects no error

31 16 Analysis of the Problem 100 u Double precision (53 bit) Higher precision (here: 128 bit) Correct n Fully automatic analysis detects no error Can only be discovered with intermediate results

32 17 Solution: Stages int i; double u, v, w; u = 2; v = -4; for (i = 3; i <= 50; i++) { w = 111. u = v; v = w; /v /(v*u); }

33 17 Solution: Stages int i; double u, v, w; u = 2; v = -4; for (i = 3; i <= 50; i++) { FPDEBUG BEGIN STAGE(0); w = 111. u = v; v = w; /v /(v*u); } FPDEBUG END STAGE(0);

34 18 Case Study: Biochemical Algorithms Library (BALL) > 400, 000 lines of code

35 18 Case Study: Biochemical Algorithms Library (BALL) > 400, 000 lines of code double StretchComponent::updateEnergy() {... double distance = atom1->getdistance(*atom2); energy += stretch [i].values.k * (distance - stretch [i].values.r0) * (distance - stretch [i].values.r0);... }

36 18 Case Study: Biochemical Algorithms Library (BALL) > 400, 000 lines of code double StretchComponent::updateEnergy() {... double distance = atom1->getdistance(*atom2); energy += stretch [i].values.k * (distance - stretch [i].values.r0) * (distance - stretch [i].values.r0);... } Catastrophic cancellation At most 24 bits canceled (double: 53 bit precision)

37 18 Case Study: Biochemical Algorithms Library (BALL) > 400, 000 lines of code double StretchComponent::updateEnergy() {... double distance = atom1->getdistance(*atom2); energy += stretch [i].values.k * (distance - stretch [i].values.r0) * (distance - stretch [i].values.r0);... } Catastrophic cancellation At most 24 bits canceled (double: 53 bit precision) float Atom::getDistance(const Atom& a) const

38 Case Study: GNU Linear Programming Kit (GLPK) 2 > 100, 000 lines of code min x 20 s.t. (s + 1) x 1 x 2 s 1, sx i 1 + (s + 1) x i x i+1 ( 1) i (s + 1) for i = 2 : 19, sx 18 (3s 1) x x 20 (5s 7), 0 x i 10 for i = 1 : 13, 0 x i B for i = 14 : 20, all x i integers, 2 Neumaier and Shcherbina: Safe bounds in linear and mixed-integer linear programming, Mathematical Programming,

39 Case Study: GNU Linear Programming Kit (GLPK) 2 > 100, 000 lines of code min x 20 s.t. (s + 1) x 1 x 2 s 1, sx i 1 + (s + 1) x i x i+1 ( 1) i (s + 1) for i = 2 : 19, sx 18 (3s 1) x x 20 (5s 7), 0 x i 10 for i = 1 : 13, 0 x i B for i = 14 : 20, all x i integers, Unique solution if B 2 x = (1, 2, 1, 2,..., 1, 2) T 2 Neumaier and Shcherbina: Safe bounds in linear and mixed-integer linear programming, Mathematical Programming,

40 Case Study: GNU Linear Programming Kit (GLPK) Binary search B = x = (1, 2, 1, 2,..., 1, 2) T B = Problem has no integer feasible solution 20

41 Case Study: GNU Linear Programming Kit (GLPK) Binary search B = x = (1, 2, 1, 2,..., 1, 2) T B = Problem has no integer feasible solution Compared the runs Found variable that differs Bound Shadow Original

42 Case Study: Calculix (SPEC CFP2006) double DVdot (int size, double y[], double x[]) { double sum = 0.0; int i; for (i = 0; i < size; i++) { sum += y[i] * x[i]; } } return sum; 21

43 Case Study: Calculix (SPEC CFP2006) double DVdot (int size, double y[], double x[]) { FPDEBUG BEGIN(); double sum = 0.0; int i; for (i = 0; i < size; i++) { sum += y[i] * x[i]; } } FPDEBUG END(); return sum; 21

44 Case Study: Calculix (SPEC CFP2006) double DVdot (int size, double y[], double x[]) { FPDEBUG BEGIN(); double sum = 0.0; int i; for (i = 0; i < size; i++) { sum += y[i] * x[i]; } } FPDEBUG INSERT SHADOW(&sum); FPDEBUG END(); return sum; 21

45 Case Study: Calculix (SPEC CFP2006) double DVdot (int size, double y[], double x[]) { FPDEBUG BEGIN(); } double sum = 0.0; int i; for (i = 0; i < size; i++) { sum += y[i] * x[i]; } double errorbound = 1e-2; if (FPDEBUG ERROR GREATER(&sum, &errorbound)) { /* Print arrays x, and y */ } FPDEBUG INSERT SHADOW(&sum); FPDEBUG END(); return sum; 21

46 22 Performance: SPEC CFP2006 Benchmark Original Analyzed Slowdown bwaves s s 167 x gamess 0.70 s s 544 x milc s s 224 x gromacs 2.10 s s 472 x cactusadm 4.70 s s 1016 x leslie3d s s 292 x namd s s 957 x soplex 0.03 s 5.00 s 185 x povray 0.90 s s 444 x calculix 0.07 s s 244 x GemsFDTD 5.50 s s 208 x tonto 1.26 s s 321 x lbm 9.55 s s 303 x wrf 7.68 s s 342 x sphinx s s 213 x

47 23 Conclusion Dynamic program analysis Detects floating-point accuracy problems Detects catastrophic cancellations Works on large-scale programs Finds real-world problems Is open source: github.com/fbenz/fpdebug

48 23 Conclusion Dynamic program analysis Detects floating-point accuracy problems Detects catastrophic cancellations Works on large-scale programs Finds real-world problems Is open source: github.com/fbenz/fpdebug Thank You! Questions?