Enclosures of Roundoff Errors using SDP

Transcription

1 Enclosures of Roundoff Errors using SDP Victor Magron, CNRS Jointly Certified Upper Bounds with G. Constantinides and A. Donaldson Metalibm workshop: Elementary functions, digital filters and beyond March 2018 Victor Magron Enclosures of Roundoff Errors using SDP 0 / 20

2 Errors and Proofs GUARANTEED OPTIMIZATION Input : Linear problem (LP), geometric, semidefinite (SDP) Output : solution + certificate numeric-symbolic formal Victor Magron Enclosures of Roundoff Errors using SDP 1 / 20

3 Errors and Proofs GUARANTEED OPTIMIZATION Input : Linear problem (LP), geometric, semidefinite (SDP) Output : solution + certificate numeric-symbolic formal VERIFICATION OF CRITICAL SYSTEMS Reliable software/hardware embedded codes Aerospace control molecular biology, robotics, code synthesis,... Victor Magron Enclosures of Roundoff Errors using SDP 1 / 20

4 Errors and Proofs GUARANTEED OPTIMIZATION Input : Linear problem (LP), geometric, semidefinite (SDP) Output : solution + certificate numeric-symbolic formal VERIFICATION OF CRITICAL SYSTEMS Reliable software/hardware embedded codes Aerospace control molecular biology, robotics, code synthesis,... Efficient Verification of Nonlinear Systems Automated precision tuning of systems/programs analysis/synthesis Efficiency sparsity correlation patterns Certified approximation algorithms Victor Magron Enclosures of Roundoff Errors using SDP 1 / 20

5 Roundoff Error Bounds Real : f (x) := x 1 x 2 + x 3 Floating-point : ˆf (x, e) := [x1 x 2 (1 + e 1 ) + x 3 ](1 + e 2 ) Input variable constraints x X Finite precision bounds over e E: e i 2 53 (double) Guarantees on absolute round-off error ˆf f? max ˆf f min ˆf f Upper Bounds Lower Bounds Lower Bounds Upper Bounds max ˆf f min ˆf f Victor Magron Enclosures of Roundoff Errors using SDP 2 / 20

6 Nonlinear Programs Polynomials programs : +,, x 2 x 5 + x 3 x 6 + x 1 ( x 1 + x 2 + x 3 x 4 + x 5 + x 6 ) Victor Magron Enclosures of Roundoff Errors using SDP 3 / 20

7 Nonlinear Programs Polynomials programs : +,, x 2 x 5 + x 3 x 6 + x 1 ( x 1 + x 2 + x 3 x 4 + x 5 + x 6 ) Semialgebraic programs:,, /, sup, inf 4x 1 + x 1.11 Victor Magron Enclosures of Roundoff Errors using SDP 3 / 20

8 Nonlinear Programs Polynomials programs : +,, x 2 x 5 + x 3 x 6 + x 1 ( x 1 + x 2 + x 3 x 4 + x 5 + x 6 ) Semialgebraic programs:,, /, sup, inf 4x 1 + x 1.11 Transcendental programs: arctan, exp, log,... log(1 + exp(x)) Victor Magron Enclosures of Roundoff Errors using SDP 3 / 20

9 Existing Frameworks Classical methods: Abstract domains [Goubault-Putot 11] FLUCTUAT: intervals, octagons, zonotopes Interval arithmetic [Daumas-Melquiond 10] GAPPA: interface with COQ proof assistant Victor Magron Enclosures of Roundoff Errors using SDP 4 / 20

10 Existing Frameworks Recent progress: Affine arithmetic + SMT [Darulova 14], [Darulova 17], [Izycheva 17] rosa: sound compiler for reals (SCALA) Daisy: roundoff error + rewriting + multi-precision Symbolic Taylor expansions [Solovyev 15], [Chiang 17] FPTaylor: certified optimization (OCAML/HOL-LIGHT) Improve numerical accuracy of programs [Damouche 17] Salsa: program transformation Guided random testing s3fp [Chiang 14] Victor Magron Enclosures of Roundoff Errors using SDP 4 / 20

11 Contributions Maximal Roundoff error of the program implementation of f : r := max ˆf (x, e) f (x) Decomposition: linear term l w.r.t. e + nonlinear term h max l + max h r max l max h Coarse bound of h with interval arithmetic Semidefinite programming (SDP) bounds for l: Victor Magron Enclosures of Roundoff Errors using SDP 5 / 20

12 Contributions Maximal Roundoff error of the program implementation of f : r := max ˆf (x, e) f (x) Decomposition: linear term l w.r.t. e + nonlinear term h max l + max h r max l max h Coarse bound of h with interval arithmetic Semidefinite programming (SDP) bounds for l: Upper Bounds Lower Bounds Lower Bounds Upper Bounds Sparse SDP relaxations Robust SDP relaxations Victor Magron Enclosures of Roundoff Errors using SDP 5 / 20

13 Contributions 1 General SDP framework for upper and lower bounds 2 Comparison with SMT & affine/taylor arithmetic: Efficient optimization + Tight upper bounds 3 Extensions to transcendental/conditional programs 4 Formal verification of SDP bounds 5 Open source tools: Upper Bounds Real2Float (in OCAML and COQ) FPSDP (in MATLAB) Upper Bounds Lower Bounds Lower Bounds Victor Magron Enclosures of Roundoff Errors using SDP 5 / 20

14 Introduction Semidefinite Programming for Polynomial Optimization Upper Bounds with Sparse SDP Lower Bounds with Robust SDP Conclusion

15 What is Semidefinite Programming? Linear Programming (LP): min z c z s.t. A z d. Linear cost c Linear inequalities i A ij z j d i Polyhedron Victor Magron Enclosures of Roundoff Errors using SDP 6 / 20

16 What is Semidefinite Programming? Semidefinite Programming (SDP): min z s.t. c z F i z i F 0. i Linear cost c Symmetric matrices F 0, F i Linear matrix inequalities F 0 (F has nonnegative eigenvalues) Spectrahedron Victor Magron Enclosures of Roundoff Errors using SDP 6 / 20

17 What is Semidefinite Programming? Semidefinite Programming (SDP): min z s.t. c z F i z i F 0, A z = d. i Linear cost c Symmetric matrices F 0, F i Linear matrix inequalities F 0 (F has nonnegative eigenvalues) Spectrahedron Victor Magron Enclosures of Roundoff Errors using SDP 6 / 20

18 SDP for Polynomial Optimization Prove polynomial inequalities with SDP: Find z s.t. f (a, b) = f (a, b) := a 2 2ab + b 2 0. ( a ) ( ) z 1 z 2 b z 2 z 3 }{{} 0 ( ) a b Find z s.t. a 2 2ab + b 2 = z 1 a 2 + 2z 2 ab + z 3 b 2 (A z = d) ( ) ( ) ( ) ( ) ( ) z1 z = z z 2 z z z }{{}}{{}}{{}}{{} F 1 F 2 F 3 F 0. Victor Magron Enclosures of Roundoff Errors using SDP 7 / 20

19 SDP for Polynomial Optimization Choose a cost c e.g. (1, 0, 1) and solve: min z s.t. ( ) z1 z Solution 2 = z 2 z 3 c z F i z i F 0, A z = d. i ( ) (eigenvalues 0 and 2) 1 1 a 2 2ab + b 2 = ( a b ) ( ) ( ) 1 1 a = (a b) b }{{} 0 Solving SDP = Finding SUMS OF SQUARES certificates Victor Magron Enclosures of Roundoff Errors using SDP 8 / 20

20 SDP for Polynomial Optimization NP hard General Problem: f := min x X f (x) Semialgebraic set X := {x R n : g 1 (x) 0,..., g m (x) 0} Victor Magron Enclosures of Roundoff Errors using SDP 9 / 20

21 SDP for Polynomial Optimization NP hard General Problem: f := min x X f (x) Semialgebraic set X := {x R n : g 1 (x) 0,..., g m (x) 0} := [0, 1] 2 = {x R 2 : x 1 (1 x 1 ) 0, x 2 (1 x 2 ) 0} Victor Magron Enclosures of Roundoff Errors using SDP 9 / 20

22 SDP for Polynomial Optimization NP hard General Problem: f := min x X f (x) Semialgebraic set X := {x R n : g 1 (x) 0,..., g m (x) 0} := [0, 1] 2 = {x R 2 : x 1 (1 x 1 ) 0, x 2 (1 x 2 ) 0} f {}}{ x 1 x = σ 0 {}}{ ( 1 x 1 + x ) σ 1 {}}{ 1 2 g 1 {}}{ x 1 (1 x 1 ) + σ 2 {}}{ 1 2 g 2 {}}{ x 2 (1 x 2 ) Victor Magron Enclosures of Roundoff Errors using SDP 9 / 20

23 SDP for Polynomial Optimization NP hard General Problem: f := min x X f (x) Semialgebraic set X := {x R n : g 1 (x) 0,..., g m (x) 0} := [0, 1] 2 = {x R 2 : x 1 (1 x 1 ) 0, x 2 (1 x 2 ) 0} f {}}{ x 1 x = σ 0 {}}{ ( 1 x 1 + x ) σ 1 {}}{ 1 2 g 1 {}}{ x 1 (1 x 1 ) + σ 2 {}}{ 1 2 g 2 {}}{ x 2 (1 x 2 ) Sums of squares (SOS) σ i Victor Magron Enclosures of Roundoff Errors using SDP 9 / 20

24 SDP for Polynomial Optimization NP hard General Problem: f := min x X f (x) Semialgebraic set X := {x R n : g 1 (x) 0,..., g m (x) 0} := [0, 1] 2 = {x R 2 : x 1 (1 x 1 ) 0, x 2 (1 x 2 ) 0} f {}}{ x 1 x = σ 0 {}}{ ( 1 x 1 + x ) σ 1 {}}{ 1 2 g 1 {}}{ x 1 (1 x 1 ) + σ 2 {}}{ 1 2 g 2 {}}{ x 2 (1 x 2 ) Sums of squares (SOS) σ i Bounded degree: { } Q k (X) := σ 0 + m j=1 σ jg j, with deg σ j g j 2k Victor Magron Enclosures of Roundoff Errors using SDP 9 / 20

25 SDP for Polynomial Optimization Hierarchy { of SDP relaxations: } λ k := sup λ : f λ Q k (X) λ Convergence guarantees λ k f [Lasserre 01] Can be computed with SDP solvers (CSDP, SDPA) No Free Lunch Rule: ( n+2k ) SDP variables n Extension to semialgebraic functions r(x) = p(x)/ q(x) [Lasserre-Putinar 10] Victor Magron Enclosures of Roundoff Errors using SDP 10 / 20

26 Sparse SDP Optimization [Waki, Lasserre 06] Correlative sparsity pattern (csp) of variables x 2 x 5 + x 3 x 6 x 2 x 3 x 5 x 6 + x 1 ( x 1 + x 2 + x 3 x 4 + x 5 + x 6 ) Maximal cliques C 1,..., C l 2 Average size κ ( κ+2k κ ) variables C 1 := {1, 4} C 2 := {1, 2, 3, 5} C 3 := {1, 3, 5, 6} Dense SDP: 210 variables Sparse SDP: 115 variables Victor Magron Enclosures of Roundoff Errors using SDP 11 / 20

27 Introduction Semidefinite Programming for Polynomial Optimization Upper Bounds with Sparse SDP Lower Bounds with Robust SDP Conclusion

28 Polynomial Programs Upper Bounds Upper Bounds Input: exact f (x), approx ˆf (x, e), x X, e i 2 53 Output: Bound for f ˆf 1: Error r(x, e) := f (x) ˆf (x, e) = r α (e)x α α 2: Decompose r(x, e) = l(x, e) + h(x, e) 3: l(x, e) = m i=1 s i(x)e i 4: Maximal cliques correspond to {x, e 1 },..., {x, e m } Dense relaxation: Sparse relaxation: ( n+m+2k n+m m( n+1+2k n+1 ) SDP variables ) SDP variables Victor Magron Enclosures of Roundoff Errors using SDP 12 / 20

29 Preliminary Comparisons Upper Bounds Upper Bounds f (x) := x 2 x 5 + x 3 x 6 x 2 x 3 x 5 x 6 + x 1 ( x 1 + x 2 + x 3 x 4 + x 5 + x 6 ) x [4.00, 6.36] 6, e [ ɛ, ɛ] 15, ɛ = 2 53 Dense SDP: ( ) = variables Out of memory Victor Magron Enclosures of Roundoff Errors using SDP 13 / 20

30 Preliminary Comparisons Upper Bounds Upper Bounds f (x) := x 2 x 5 + x 3 x 6 x 2 x 3 x 5 x 6 + x 1 ( x 1 + x 2 + x 3 x 4 + x 5 + x 6 ) x [4.00, 6.36] 6, e [ ɛ, ɛ] 15, ɛ = 2 53 Dense SDP: ( ) = variables Out of memory Sparse SDP Real2Float tool: 15( ) = ɛ Victor Magron Enclosures of Roundoff Errors using SDP 13 / 20

31 Preliminary Comparisons Upper Bounds Upper Bounds f (x) := x 2 x 5 + x 3 x 6 x 2 x 3 x 5 x 6 + x 1 ( x 1 + x 2 + x 3 x 4 + x 5 + x 6 ) x [4.00, 6.36] 6, e [ ɛ, ɛ] 15, ɛ = 2 53 Dense SDP: ( ) = variables Out of memory Sparse SDP Real2Float tool: 15( ) = ɛ Interval arithmetic: 922ɛ (10 less CPU) Victor Magron Enclosures of Roundoff Errors using SDP 13 / 20

32 Preliminary Comparisons Upper Bounds Upper Bounds f (x) := x 2 x 5 + x 3 x 6 x 2 x 3 x 5 x 6 + x 1 ( x 1 + x 2 + x 3 x 4 + x 5 + x 6 ) x [4.00, 6.36] 6, e [ ɛ, ɛ] 15, ɛ = 2 53 Dense SDP: ( ) = variables Out of memory Sparse SDP Real2Float tool: 15( ) = ɛ Interval arithmetic: 922ɛ (10 less CPU) Symbolic Taylor FPTaylor tool: 721ɛ (21 more CPU) Victor Magron Enclosures of Roundoff Errors using SDP 13 / 20

33 Preliminary Comparisons Upper Bounds Upper Bounds f (x) := x 2 x 5 + x 3 x 6 x 2 x 3 x 5 x 6 + x 1 ( x 1 + x 2 + x 3 x 4 + x 5 + x 6 ) x [4.00, 6.36] 6, e [ ɛ, ɛ] 15, ɛ = 2 53 Dense SDP: ( ) = variables Out of memory Sparse SDP Real2Float tool: 15( ) = ɛ Interval arithmetic: 922ɛ (10 less CPU) Symbolic Taylor FPTaylor tool: 721ɛ (21 more CPU) SMT-based rosa tool: 762ɛ (19 more CPU) Victor Magron Enclosures of Roundoff Errors using SDP 13 / 20

34 Preliminary Comparisons Upper Bounds Upper Bounds 1, ɛ 762ɛ 721ɛ Error Bound (ɛ) Real2Float CPU Time rosa FPTaylor Victor Magron Enclosures of Roundoff Errors using SDP 13 / 20

35 Extensions: Transcendental Programs Upper Bounds Upper Bounds Reduce f := inf x K f (x) to semialgebraic optimization y par + par + a 2 a 1 arctan par a 2 m a 1 a 2 M a par a 1 Victor Magron Enclosures of Roundoff Errors using SDP 14 / 20

36 Extensions: Conditionals Upper Bounds Upper Bounds if (p(x) 0) f (x); else g(x); DIVERGENCE PATH ERROR: r := max{ } max ˆf (x, e) g(x) p(x) 0,p(x,e) 0 max ĝ(x, e) f (x) p(x) 0,p(x,e) 0 max ˆf (x, e) f (x) p(x) 0,p(x,e) 0 max ĝ(x, e) g(x) p(x) 0,p(x,e) 0 Victor Magron Enclosures of Roundoff Errors using SDP 15 / 20

37 Introduction Semidefinite Programming for Polynomial Optimization Upper Bounds with Sparse SDP Lower Bounds with Robust SDP Conclusion

38 Method 1: geneig [Lasserre 11] Lower Bounds Lower Bounds Generalized eigenvalue problem: f := min x X f (x) λ k := sup λ s.t. λ M k (f y) λm k (y). Victor Magron Enclosures of Roundoff Errors using SDP 16 / 20

39 Method 1: geneig [Lasserre 11] Lower Bounds Lower Bounds Generalized eigenvalue problem: f := min x X f (x) λ k := sup λ s.t. λ M k (f y) λm k (y). Uniform distribution moments: y α := X xα dx Localizing matrices M k (f y): M 1 (f y) = 1 x 1 x 2 1 X f (x)dx X f (x)x 1dx X f (x)x 2dx x 1 X f (x)x 1dx X f (x)x2 1 dx X f (x)x 1x 2 dx x 2 X f (x)x 2dx X f (x)x 2x 1 dx X f (x)x2 2 dx Victor Magron Enclosures of Roundoff Errors using SDP 16 / 20

40 Method 1: geneig [Lasserre 11] Lower Bounds Lower Bounds Generalized eigenvalue problem: f := min x X f (x) λ k := sup λ s.t. λ M k (f y) λm k (y). Uniform distribution moments: y α := X xα dx Localizing matrices M k (f y): M 1 (f y) = 1 x 1 x 2 1 X f (x)dx X f (x)x 1dx X f (x)x 2dx x 1 X f (x)x 1dx X f (x)x2 1 dx X f (x)x 1x 2 dx x 2 X f (x)x 2dx X f (x)x 2x 1 dx X f (x)x2 2 dx Theorem [Lasserre 11, de Klerk et al. 17] λ k f and λ k f = O (1/ k) Victor Magron Enclosures of Roundoff Errors using SDP 16 / 20

41 Method 2: mvbeta [DeKlerk et al. 17] Lower Bounds Lower Bounds Elementary calculation with f (x) = α f α x α : f := min x X f (x) f H k := min η+β 2k γ η+α,β f α α γ η,β Victor Magron Enclosures of Roundoff Errors using SDP 17 / 20

42 Method 2: mvbeta [DeKlerk et al. 17] Lower Bounds Lower Bounds Elementary calculation with f (x) = α f α x α : f := min x X f (x) f H k := min η+β 2k Multivariate beta distribution moments: γ η,β := x η (1 x) β dx. X γ η+α,β f α α γ η,β Victor Magron Enclosures of Roundoff Errors using SDP 17 / 20

43 Method 2: mvbeta [DeKlerk et al. 17] Lower Bounds Lower Bounds Elementary calculation with f (x) = α f α x α : f := min x X f (x) f H k := min η+β 2k Multivariate beta distribution moments: γ η,β := x η (1 x) β dx. X γ η+α,β f α α γ η,β Theorem [DeKlerk et al. 17] f H k f and λ k f = O (1/ k) Victor Magron Enclosures of Roundoff Errors using SDP 17 / 20

44 Method 3: robustsdp Lower Bounds Lower Bounds Robust SDP with l(x, e) = m s i (x)e i : i=1 l := min l(x, e) λ k := sup (x,e) X E λ s.t. λ e E, M k (l y) λm k (y). Victor Magron Enclosures of Roundoff Errors using SDP 18 / 20

45 Method 3: robustsdp Lower Bounds Lower Bounds Robust SDP with l(x, e) = m s i (x)e i : i=1 l := min l(x, e) λ k := sup (x,e) X E λ s.t. Linearity M k (l y) = m e i M k (s i y) i=1 Factorization M k (s i y) = L i k Ri k λ e E, M k (l y) λm k (y). Victor Magron Enclosures of Roundoff Errors using SDP 18 / 20

46 Method 3: robustsdp Lower Bounds Lower Bounds Robust SDP with l(x, e) = m s i (x)e i : i=1 l := min l(x, e) λ k := sup (x,e) X E λ s.t. Linearity M k (l y) = m e i M k (s i y) i=1 Factorization M k (s i y) = L i k Ri k Theorem following from [El Ghaoui et al. 98] λ e E, M k (l y) λm k (y). λ k l and λ k = sup λ,s,g s.t. λ ( λmk (y) L k S L T k R T ) k + L k G R k G L T 0, k S S T = S, G T = G. Victor Magron Enclosures of Roundoff Errors using SDP 18 / 20

47 Benchmark kepler0 with k = 2 Lower Bounds Lower Bounds ɛ Error Bound (ɛ) ɛ 71ɛ 0 geneig mvbeta CPU Time robustsdp Victor Magron Enclosures of Roundoff Errors using SDP 19 / 20

48 Introduction Semidefinite Programming for Polynomial Optimization Upper Bounds with Sparse SDP Lower Bounds with Robust SDP Conclusion

49 Conclusion Sparse/Robust SDP relaxations for NONLINEAR PROGRAMS: Polynomial and transcendental programs Certified Formal roundoff error bounds (Joint work with T. Weisser and B. Werner) Real2Float and FPSDP open source tools: Victor Magron Enclosures of Roundoff Errors using SDP 20 / 20

50 Conclusion Further research: Automatic FPGA code generation Handling while/for loops Master / PhD Positions Available! Victor Magron Enclosures of Roundoff Errors using SDP 20 / 20

51 End Thank you for your attention! V. Magron, G. Constantinides, A. Donaldson. Certified Roundoff Error Bounds Using Semidefinite Programming, ACM Trans. Math. Softw, Upper Bounds Upper Bounds V. Magron. Interval Enclosures of Upper Bounds of Roundoff Errors using Semidefinite Programming, arxiv.org/abs/ Lower Bounds Lower Bounds