Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods

Size: px

Start display at page:

Download "Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods"

Justina Ward
5 years ago
Views:

1 Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods Victor Magron, CNRS VERIMAG Jointly Certified Upper Bounds with G. Constantinides and A. Donaldson Jointly Certified Lower Bounds with M. Farid SWIM ENS Lyon, 20 June 2016 Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 1 / 27

2 Errors and Proofs Mathematicians and Computer Scientists want to eliminate all the uncertainties on their results. Why? Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 2 / 27

3 Errors and Proofs Mathematicians and Computer Scientists want to eliminate all the uncertainties on their results. Why? M. Lecat, Erreurs des Mathématiciens des origines à nos jours, ; 130 pages of errors! (Euler, Fermat,... ) Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 2 / 27

4 Errors and Proofs Mathematicians and Computer Scientists want to eliminate all the uncertainties on their results. Why? M. Lecat, Erreurs des Mathématiciens des origines à nos jours, ; 130 pages of errors! (Euler, Fermat,... ) Ariane 5 launch failure, Pentium FDIV bug Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 2 / 27

5 Errors and Proofs Mathematicians and Computer Scientists want to eliminate all the uncertainties on their results. Why? M. Lecat, Erreurs des Mathématiciens des origines à nos jours, ; 130 pages of errors! (Euler, Fermat,... ) Ariane 5 launch failure, Pentium FDIV bug U.S. Patriot missile killed 28 soldiers from the U.S. Army s Internal clock: 0.1 sec intervals Roundoff error on the binary constant 0.1 Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 2 / 27

6 Errors and Proofs GUARANTEED OPTIMIZATION Input : Linear problem (LP), geometric, semidefinite (SDP) Output : solution + certificate numeric-symbolic formal Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 3 / 27

7 Errors and Proofs GUARANTEED OPTIMIZATION Input : Linear problem (LP), geometric, semidefinite (SDP) Output : solution + certificate numeric-symbolic formal VERIFICATION OF CRITICAL SYSTEMS Reliable software/hardware embedded codes Aerospace control molecular biology, robotics, code synthesis,... Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 3 / 27

8 Errors and Proofs GUARANTEED OPTIMIZATION Input : Linear problem (LP), geometric, semidefinite (SDP) Output : solution + certificate numeric-symbolic formal VERIFICATION OF CRITICAL SYSTEMS Reliable software/hardware embedded codes Aerospace control molecular biology, robotics, code synthesis,... Efficient Verification of Nonlinear Systems Automated precision tuning of systems/programs analysis/synthesis Efficiency sparsity correlation patterns Certified approximation algorithms Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 3 / 27

9 Roundoff Error Bounds Real : f (x) := x 1 x 2 + x 3 Floating-point : ˆf (x, e) := [x1 x 2 (1 + e 1 ) + x 3 ](1 + e 2 ) Input variable constraints x X Finite precision bounds over e E: e i 2 53 (double) Guarantees on absolute round-off error ˆf f? max ˆf f min ˆf f Upper Bounds Lower Bounds Lower Bounds Upper Bounds max ˆf f min ˆf f Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 4 / 27

10 Nonlinear Programs Polynomials programs : +,, x 2 x 5 + x 3 x 6 + x 1 ( x 1 + x 2 + x 3 x 4 + x 5 + x 6 ) Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 5 / 27

11 Nonlinear Programs Polynomials programs : +,, x 2 x 5 + x 3 x 6 + x 1 ( x 1 + x 2 + x 3 x 4 + x 5 + x 6 ) Semialgebraic programs:,, /, sup, inf 4x 1 + x 1.11 Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 5 / 27

12 Nonlinear Programs Polynomials programs : +,, x 2 x 5 + x 3 x 6 + x 1 ( x 1 + x 2 + x 3 x 4 + x 5 + x 6 ) Semialgebraic programs:,, /, sup, inf 4x 1 + x 1.11 Transcendental programs: arctan, exp, log,... log(1 + exp(x)) Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 5 / 27

13 Existing Frameworks Classical methods: Abstract domains [Goubault-Putot 11] FLUCTUAT: intervals, octagons, zonotopes Interval arithmetic [Daumas-Melquiond 10] GAPPA: interface with COQ proof assistant Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 6 / 27

14 Existing Frameworks Recent progress: Affine arithmetic + SMT [Darulova 14] rosa: sound compiler for reals (SCALA) Symbolic Taylor expansions [Solovyev 15] FPTaylor: certified optimization (OCAML/HOL-LIGHT) Guided random testing s3fp [Chiang 14] Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 6 / 27

15 Contributions Maximal Roundoff error of the program implementation of f : r := max ˆf (x, e) f (x) Decomposition: linear term l w.r.t. e + nonlinear term h max l(x, e) + max h(x, e) r max l(x, e) max h(x, e) Coarse bound of h with interval arithmetic Semidefinite programming (SDP) bounds for l: Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 7 / 27

16 Contributions Maximal Roundoff error of the program implementation of f : r := max ˆf (x, e) f (x) Decomposition: linear term l w.r.t. e + nonlinear term h max l(x, e) + max h(x, e) r max l(x, e) max h(x, e) Coarse bound of h with interval arithmetic Semidefinite programming (SDP) bounds for l: Upper Bounds Lower Bounds Lower Bounds Upper Bounds Sparse SDP relaxations Robust SDP relaxations Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 7 / 27

17 Contributions 1 General SDP framework for upper and lower bounds 2 Comparison with SMT and linear/affine/taylor arithmetic: Efficient optimization + Tight upper bounds 3 Extensions to transcendental/conditional programs 4 Formal verification of SDP bounds 5 Open source tool Real2Float (in OCAML and COQ) Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 7 / 27

18 Introduction Semidefinite Programming for Polynomial Optimization Upper Bounds with Sparse SDP Lower Bounds with Robust SDP Conclusion

19 What is Semidefinite Programming? Linear Programming (LP): min z c z s.t. A z d. Linear cost c Linear inequalities i A ij z j d i Polyhedron Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 8 / 27

20 What is Semidefinite Programming? Semidefinite Programming (SDP): min z s.t. c z F i z i F 0. i Linear cost c Symmetric matrices F 0, F i Linear matrix inequalities F 0 (F has nonnegative eigenvalues) Spectrahedron Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 9 / 27

21 What is Semidefinite Programming? Semidefinite Programming (SDP): min z s.t. c z F i z i F 0, A z = d. i Linear cost c Symmetric matrices F 0, F i Linear matrix inequalities F 0 (F has nonnegative eigenvalues) Spectrahedron Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 10 / 27

22 Applications of SDP Combinatorial optimization Control theory Matrix completion Unique Games Conjecture (Khot 02) : A single concrete algorithm provides optimal guarantees among all efficient algorithms for a large class of computational problems. (Barak and Steurer survey at ICM 14) Solving polynomial optimization (Lasserre 01) Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 11 / 27

23 SDP for Polynomial Optimization Prove polynomial inequalities with SDP: Find z s.t. f (a, b) = f (a, b) := a 2 2ab + b 2 0. ( a ) ( ) z 1 z 2 b z 2 z 3 }{{} 0 ( ) a b Find z s.t. a 2 2ab + b 2 = z 1 a 2 + 2z 2 ab + z 3 b 2 (A z = d) ( ) ( ) ( ) ( ) ( ) z1 z = z z 2 z z z }{{}}{{}}{{}}{{} F 1 F 2 F 3 F 0. Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 12 / 27

24 SDP for Polynomial Optimization Choose a cost c e.g. (1, 0, 1) and solve: min z s.t. ( ) z1 z Solution 2 = z 2 z 3 c z F i z i F 0, A z = d. i ( ) (eigenvalues 0 and 2) 1 1 a 2 2ab + b 2 = ( a b ) ( ) ( ) 1 1 a = (a b) b }{{} 0 Solving SDP = Finding SUMS OF SQUARES certificates Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 13 / 27

25 SDP for Polynomial Optimization General case: Semialgebraic set X := {x R n : g 1 (x) 0,..., g m (x) 0} p := min f (x): NP hard x X Sums of squares (SOS) Σ[x] (e.g. (x 1 x 2 ) 2 ) Q(X) := { } σ 0 (x) + m j=1 σ j(x)g j (x), with σ j Σ[x] Fix the degree 2k of products: { } Q k (X) := σ 0 (x) + σ j (x)g j (x), with deg σ j g j 2k m j=1 Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 14 / 27

26 SDP for Polynomial Optimization Hierarchy { of SDP relaxations: } λ k := sup λ : f λ Q k (X) λ Convergence guarantees λ k f [Lasserre 01] Can be computed with SDP solvers (CSDP, SDPA) No Free Lunch Rule: ( n+2k ) SDP variables n Extension to semialgebraic functions r(x) = p(x)/ q(x) [Lasserre-Putinar 10] Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 15 / 27

27 Sparse SDP Optimization [Waki, Lasserre 06] Correlative sparsity pattern (csp) of variables x 2 x 5 + x 3 x 6 x 2 x 3 x 5 x 6 + x 1 ( x 1 + x 2 + x 3 x 4 + x 5 + x 6 ) Maximal cliques C 1,..., C l 2 Average size κ ( κ+2k κ ) variables C 1 := {1, 4} C 2 := {1, 2, 3, 5} C 3 := {1, 3, 5, 6} Dense SDP: 210 variables Sparse SDP: 115 variables Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 16 / 27

28 Introduction Semidefinite Programming for Polynomial Optimization Upper Bounds with Sparse SDP Lower Bounds with Robust SDP Conclusion

29 Polynomial Programs Upper Bounds Upper Bounds Input: exact f (x), floating-point ˆf (x, e), x X, e i 2 53 Output: Bound for f ˆf 1: Error r(x, e) := f (x) ˆf (x, e) = r α (e)x α α 2: Decompose r(x, e) = l(x, e) + h(x, e), l linear in e 3: l(x, e) = m i=1 s i(x)e i 4: Maximal cliques correspond to {x, e 1 },..., {x, e m } 5: Bound l(x, e) with sparse SDP relaxations (and h with IA) Dense relaxation: Sparse relaxation: ( n+m+2k n+m m( n+1+2k n+1 ) SDP variables ) SDP variables Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 17 / 27

30 Preliminary Comparisons Upper Bounds Upper Bounds f (x) := x 2 x 5 + x 3 x 6 x 2 x 3 x 5 x 6 + x 1 ( x 1 + x 2 + x 3 x 4 + x 5 + x 6 ) x [4.00, 6.36] 6, e [ ɛ, ɛ] 15, ɛ = 2 53 Dense SDP: ( ) = variables Out of memory Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 18 / 27

31 Preliminary Comparisons Upper Bounds Upper Bounds f (x) := x 2 x 5 + x 3 x 6 x 2 x 3 x 5 x 6 + x 1 ( x 1 + x 2 + x 3 x 4 + x 5 + x 6 ) x [4.00, 6.36] 6, e [ ɛ, ɛ] 15, ɛ = 2 53 Dense SDP: ( ) = variables Out of memory Sparse SDP Real2Float tool: 15( ) = ɛ Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 18 / 27

32 Preliminary Comparisons Upper Bounds Upper Bounds f (x) := x 2 x 5 + x 3 x 6 x 2 x 3 x 5 x 6 + x 1 ( x 1 + x 2 + x 3 x 4 + x 5 + x 6 ) x [4.00, 6.36] 6, e [ ɛ, ɛ] 15, ɛ = 2 53 Dense SDP: ( ) = variables Out of memory Sparse SDP Real2Float tool: 15( ) = ɛ Interval arithmetic: 922ɛ (10 less CPU) Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 18 / 27

33 Preliminary Comparisons Upper Bounds Upper Bounds f (x) := x 2 x 5 + x 3 x 6 x 2 x 3 x 5 x 6 + x 1 ( x 1 + x 2 + x 3 x 4 + x 5 + x 6 ) x [4.00, 6.36] 6, e [ ɛ, ɛ] 15, ɛ = 2 53 Dense SDP: ( ) = variables Out of memory Sparse SDP Real2Float tool: 15( ) = ɛ Interval arithmetic: 922ɛ (10 less CPU) Symbolic Taylor FPTaylor tool: 721ɛ (21 more CPU) Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 18 / 27

34 Preliminary Comparisons Upper Bounds Upper Bounds f (x) := x 2 x 5 + x 3 x 6 x 2 x 3 x 5 x 6 + x 1 ( x 1 + x 2 + x 3 x 4 + x 5 + x 6 ) x [4.00, 6.36] 6, e [ ɛ, ɛ] 15, ɛ = 2 53 Dense SDP: ( ) = variables Out of memory Sparse SDP Real2Float tool: 15( ) = ɛ Interval arithmetic: 922ɛ (10 less CPU) Symbolic Taylor FPTaylor tool: 721ɛ (21 more CPU) SMT-based rosa tool: 762ɛ (19 more CPU) Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 18 / 27

35 Preliminary Comparisons Upper Bounds Upper Bounds 1, ɛ 762ɛ 721ɛ Error Bound (ɛ) Real2Float CPU Time rosa FPTaylor Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 18 / 27

36 Extensions: Transcendental Programs Upper Bounds Upper Bounds Reduce f := inf x K f (x) to semialgebraic optimization y par + par + a 2 a 1 arctan par a 2 m a 1 a 2 M a par a 1 Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 19 / 27

37 Extensions: Conditionals Upper Bounds Upper Bounds if (p(x) 0) f (x); else g(x); DIVERGENCE PATH ERROR: r := max{ } max ˆf (x, e) g(x) p(x) 0,p(x,e) 0 max ĝ(x, e) f (x) p(x) 0,p(x,e) 0 max ˆf (x, e) f (x) p(x) 0,p(x,e) 0 max ĝ(x, e) g(x) p(x) 0,p(x,e) 0 Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 20 / 27

38 Comparison with rosa Upper Bounds Upper Bounds Relative execution time 1 f 10 m 0.5 e d x o t c y v g a w z b i h 0.5 Relative bound precision u p r jk 1 l q Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 21 / 27

39 Comparison with FPTaylor Upper Bounds Upper Bounds Relative execution time β t w f x c d u α v e b i 0 10 g n o a 100 m Relative bound precision h r p q 1 jk δ l γ Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 22 / 27

40 Introduction Semidefinite Programming for Polynomial Optimization Upper Bounds with Sparse SDP Lower Bounds with Robust SDP Conclusion

41 Method 1: geneig [Lasserre 11] Lower Bounds Lower Bounds Generalized eigenvalue problem: f := min x X f (x) λ k := sup λ s.t. λ M k (f y) λm k (y). Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 23 / 27

42 Method 1: geneig [Lasserre 11] Lower Bounds Lower Bounds Generalized eigenvalue problem: f := min x X f (x) λ k := sup λ s.t. λ M k (f y) λm k (y). Uniform distribution moments: y α := X xα dx Localizing matrices M k (f y): M 1 (f y) = 1 x 1 x 2 1 X f (x)dx X f (x)x 1dx X f (x)x 2dx x 1 X f (x)x 1dx X f (x)x2 1 dx X f (x)x 1x 2 dx x 2 X f (x)x 2dx X f (x)x 2x 1 dx X f (x)x2 2 dx Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 23 / 27

43 Method 1: geneig [Lasserre 11] Lower Bounds Lower Bounds Generalized eigenvalue problem: f := min x X f (x) λ k := sup λ s.t. λ M k (f y) λm k (y). Uniform distribution moments: y α := X xα dx Localizing matrices M k (f y): M 1 (f y) = 1 x 1 x 2 1 X f (x)dx X f (x)x 1dx X f (x)x 2dx x 1 X f (x)x 1dx X f (x)x2 1 dx X f (x)x 1x 2 dx x 2 X f (x)x 2dx X f (x)x 2x 1 dx X f (x)x2 2 dx Theorem [Lasserre 11] λ k f Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 23 / 27

44 Method 2: mvbeta [DeKlerk et al. 16] Lower Bounds Lower Bounds Elementary calculation with f (x) = α f α x α : f := min x X f (x) f H k := min η+β 2k γ η+α,β f α α γ η,β Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 24 / 27

45 Method 2: mvbeta [DeKlerk et al. 16] Lower Bounds Lower Bounds Elementary calculation with f (x) = α f α x α : f := min x X f (x) f H k := min η+β 2k Multivariate beta distribution moments: γ η,β := x η (1 x) β dx. X γ η+α,β f α α γ η,β Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 24 / 27

46 Method 2: mvbeta [DeKlerk et al. 16] Lower Bounds Lower Bounds Elementary calculation with f (x) = α f α x α : f := min x X f (x) f H k := min η+β 2k Multivariate beta distribution moments: γ η,β := x η (1 x) β dx. X γ η+α,β f α α γ η,β Theorem [DeKlerk et al. 16] f H k f Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 24 / 27

47 Method 3: robustsdp Lower Bounds Lower Bounds Robust SDP with l(x, e) = m s i (x)e i : i=1 l := min l(x, e) λ k := sup (x,e) X E λ s.t. λ e E, M k (l y) λm k (y). Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 25 / 27

48 Method 3: robustsdp Lower Bounds Lower Bounds Robust SDP with l(x, e) = m s i (x)e i : i=1 l := min l(x, e) λ k := sup (x,e) X E λ s.t. λ e E, M k (l y) λm k (y). Linearity M k (l y) = m e i M k (s i y) i=1 Factorize M k (s i y) = L i k Ri k, L k := [L 1 k Lm k ], R k := [R 1 k Rm k ]T Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 25 / 27

49 Method 3: robustsdp Lower Bounds Lower Bounds Robust SDP with l(x, e) = m s i (x)e i : i=1 l := min l(x, e) λ k := sup (x,e) X E λ s.t. λ e E, M k (l y) λm k (y). Linearity M k (l y) = m e i M k (s i y) i=1 Factorize M k (s i y) = L i k Ri k, L k := [L 1 k Lm k ], R k := [R 1 k Rm k ]T Theorem following from [El Ghaoui et al. 98] λ k l and λ k = sup λ,s,g s.t. λ ( λmk (y) L k S L T k R T ) k + L k G R k G L T 0, k S S T = S, G T = G. Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 25 / 27

50 Benchmark kepler0 with k = 2 Lower Bounds Lower Bounds ɛ Error Bound (ɛ) ɛ 395ɛ 71ɛ 0 geneig mvbeta s3fp robustsdp CPU Time Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 26 / 27

51 Introduction Semidefinite Programming for Polynomial Optimization Upper Bounds with Sparse SDP Lower Bounds with Robust SDP Conclusion

52 Conclusion Sparse/Robust SDP relaxations for NONLINEAR PROGRAMS: Polynomial and transcendental programs Certified Formal roundoff error bounds (Joint work with T. Weisser and B. Werner) Real2Float open source tool: Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 27 / 27

53 Conclusion Further research: Automatic FPGA code generation Roundoff error analysis with while/for loops Master / PhD Positions Available! Victor Magron Certification of Roundoff Errors with SDP Relaxations and Formal Interval Methods 27 / 27

54 End Thank you for your attention! V. Magron, G. Constantinides, A. Donaldson. Certified Roundoff Error Bounds Using Semidefinite Programming, arxiv.org/abs/

Enclosures of Roundoff Errors using SDP

Enclosures of Roundoff Errors using SDP Victor Magron, CNRS Jointly Certified Upper Bounds with G. Constantinides and A. Donaldson Metalibm workshop: Elementary functions, digital filters and beyond 12-13