Turbo-Charging Lemmas on Demand with Don t Care Reasoning

Transcription

1 Turbo-Charging Lemmas on Demand with Don t Care Reasoning Aina Niemetz, Mathias Preiner and Armin Biere Institute for Formal Models and Verification (FMV) Johannes Kepler University, Linz, Austria FMCAD 204 October 2-24, 204 Lausanne, Switzerland

2 Introduction Lemmas on Demand so-called lazy SMT approach our SMT solver Boolector implements Lemmas on Demand for the quantifier-free theory of fixed-size bit vectors arrays recently: Lemmas on Demand for Lambdas [DIFTS 3] generalization of Lemmas on Demand for Arrays [JSAT 09] arrays represented as uninterpreted functions array operations represented as lambda-terms reads represented as function applications

3 Lemmas on Demand Workflow: Original Procedure LOD LOD φ Preprocessing π consistent Refinement inconsistent Consistency Check σp(α(π) ξ) ξ = {l} ξ σ(α(π) ξ) Partial Model Extraction sat Formula Abstraction α(π) α(π) ξ DPB unsat bit vector formula abstraction (bit vector skeleton) enumeration of truth assignments (candidate models) iterative refinement with lemmas until convergence sat unsat

4 Lemmas on Demand Workflow: Original Procedure LOD LOD φ Preprocessing π consistent sat Refinement inconsistent Consistency Check σp(α(π) ξ) ξ = {l} ξ σ(α(π) ξ) Partial Model Extraction sat Full Candidate Model Formula Abstraction α(π) α(π) ξ DPB unsat unsat each candidate model is a full truth assignment of the formula abstraction full candidate model needs to be checked for consistency w.r.t. theories

5 Lemmas on Demand Workflow: Original Procedure LOD LOD φ Preprocessing π consistent Refinement inconsistent Consistency Check σp(α(π) ξ) ξ = {l} ξ σ(α(π) ξ) Partial Model Extraction sat Formula Abstraction α(π) α(π) ξ DPB unsat abstraction refinement usually the most costly part of LOD cost generally correlates with number of refinements checking the full candidate model often not required small subset responsible for satisfying formula abstraction sat unsat

6 Lemmas on Demand Workflow: Optimized Procedure LOD opt LOD φ Preprocessing π consistent Refinement inconsistent Consistency Check σp(α(π) ξ) ξ = {l} ξ σ(α(π) ξ) Partial Model Extraction sat Formula Abstraction α(π) α(π) ξ DPB Optimization unsat focus LOD on the relevant parts of the input formula exploit a posteriori observability don t cares partial model extraction prior to consistency checking subsequently reduces the cost for consistency checking sat Partial Candidate Model unsat

7 Lemmas on Demand Example: Input Formula Example. ψ i k (f(i) = e f(k) = v) v = ite(i = j, e, g(j)) and and eq eq or 2 ite 3 eq eq eq apply e apply 2 v apply 3 j i k f g

8 Lemmas on Demand Example: Formula Abstraction Example. Bit Vector Skeleton and and 2 eq eq 2 and 3 2 ite 3 eq 5 eq 4 eq 3 α(apply ) e α(apply 2) v α(apply 3) j i k

9 Lemmas on Demand Example: Formula Abstraction Example. Full Candidate Model and 0 and 2 eq eq 2 eq 5 and eq 4 2 ite 3 eq 3 α(apply ) e α(apply 2) v α(apply 3) j i k 0

10 Lemmas on Demand Example: Formula Abstraction Example. Full Candidate Model Check consistency: {apply, apply 2, apply 3} and 0 and 2 eq eq 2 eq 5 and eq 4 2 ite 3 eq 3 α(apply ) e α(apply 2) v α(apply 3) j i k 0

11 Lemmas on Demand Example: Formula Abstraction Example. Partial Candidate Model Check consistency: {apply } and 0 and 2 eq eq 2 eq 5 and X 2 ite 3 eq 4 eq 3 α(apply ) e α(apply 2) v α(apply 3) j i k 0

12 Partial Model Extraction Most intuitive: use justification-based approach Justification-based techniques in the context of SMT prune the search space of DPLL(T) [ENTCS 05, MSRTR 07] Model checking prune the search space of BMC [CAV 02] generalize proof obligations in PDR [EénFMCAD, ChoFMCAD ] generalize candidate counter examples (CEGAR) [LPAR 08]

13 Partial Model Extraction Our approach: Dual propagation-based partial model extraction exploiting the duality of a formula abstraction ψ assignments satisfying ψ (the primal channel) falsify its negation ψ (the dual channel) motivated by dual propagation techniques in QBF [AAAI 0] one solver with two channels (online approach) symmetric propagation between primal and dual channel here: offline dual propagation two solvers, one solver per channel consecutive propagation between primal and dual channel primal generates full assignment before dual enables partial model extraction based on the primal assignment

14 Partial Model Extraction Dual Propagation-Based Approach Example. Boolean Level Primal channel: ψ 2 (a b) (c d) Dual channel: ψ 2 ( a b) ( c d)

15 Partial Model Extraction Dual Propagation-Based Approach Example. Boolean Level Primal channel: ψ 2 (a b) (c d) Dual channel: ψ 2 ( a b) ( c d) Primal assignment: σ(ψ 2) {σ(a) =, σ(b) =, σ(c) =, σ(d) = }

16 Partial Model Extraction Dual Propagation-Based Approach Example. Boolean Level Primal channel: ψ 2 (a b) (c d) Dual channel: ψ 2 ( a b) ( c d) Primal assignment: σ(ψ 2) {σ(a) =, σ(b) =, σ(c) =, σ(d) = } Fix values of inputs via assumptions to the dual solver: Dual assumptions: {a=, b=, c=, d= }

17 Partial Model Extraction Dual Propagation-Based Approach Example. Boolean Level Primal channel: ψ 2 (a b) (c d) Dual channel: ψ 2 ( a b) ( c d) Primal assignment: σ(ψ 2) {σ(a) =, σ(b) =, σ(c) =, σ(d) = } Fix values of inputs via assumptions to the dual solver: Dual assumptions: {a=, b=, c=, d= } Failed assumptions: {a=, b= } sufficient to falsify ψ 2 sufficient to satisfy ψ 2

18 Partial Model Extraction Dual Propagation-Based Approach Example. Boolean Level Primal channel: ψ 2 (a b) (c d) Dual channel: ψ 2 ( a b) ( c d) Primal assignment: σ(ψ 2) {σ(a) =, σ(b) =, σ(c) =, σ(d) = } Fix values of inputs via assumptions to the dual solver: Dual assumptions: {a=, b=, c=, d= } Failed assumptions: {a=, b= } sufficient to falsify ψ 2 sufficient to satisfy ψ 2 Partial Model

19 Partial Model Extraction Dual Propagation-Based Approach structural don t care reasoning simulated via the dual solver no structural SAT solver necessary Example. (ctd) Input formula: ψ 2 (a b) (c d) Primal SAT solver: CNF(ψ 2) ( o x y) ( x o)? ( y o) ( x a) ( x b) ( a b x) ( y c) ( y d) ( c d y) Dual SAT solver: CNF( ψ 2) ( a b) ( c d) Dual assumptions: {a=, b=, c=, d= } Partial Model: {a =, b = } in contrast to partial model extraction techniques based on iterative removal of unnecessary assignments on the CNF level [FMCAD 3]

20 Partial Model Extraction Dual Propagation-Based Approach we lift this approach to the word level Primal channel: Dual channel: Γ Γ α(π) ξ α(π) l... l i one SMT solver per channel one single dual solver instance to maintain Γ over all iterations

21 Partial Model Extraction Dual Propagation-Based Approach Example. Word Level ψ i k (f(i) = e f(k) = v) v = ite(i = j, e, g(j)) α(ψ ) i k (α(apply ) = e α(apply 2) = v) v = ite(i = j, e, α(apply 3)) } Primal solver: α(ψ ) Formula abstraction and its negation Dual solver: α(ψ ) Primal assignment: σ(ψ 2) {σ(i) =, σ(j) =, σ(e) =, σ(v) =, σ(k) = 0, α(apply ) =, α(apply 2) =, α(apply 3) = } Fix values of inputs via assumptions to the dual solver: Dual assumptions: σ(ψ 2) {i =, j =, e =, v =, k = 0, α(apply ) =, α(apply 2) =, α(apply 3) = } Failed assumptions: {i =, j =, e =, v =, k = 0, α(apply ) = }

22 Partial Model Extraction Dual Propagation-Based Approach Example. Word Level ψ i k (f(i) = e f(k) = v) v = ite(i = j, e, g(j)) α(ψ ) i k (α(apply ) = e α(apply 2) = v) v = ite(i = j, e, α(apply 3)) } Primal solver: α(ψ ) Formula abstraction and its negation Dual solver: α(ψ ) Primal assignment: σ(ψ 2) {σ(i) =, σ(j) =, σ(e) =, σ(v) =, σ(k) = 0, α(apply ) =, α(apply 2) =, α(apply 3) = } Fix values of inputs via assumptions to the dual solver: Dual assumptions: σ(ψ 2) {i =, j =, e =, v =, k = 0, α(apply ) =, α(apply 2) =, α(apply 3) = } Failed assumptions: Partial Model {i =, j =, e =, v =, k = 0, α(apply ) = }

23 Partial Model Extraction Dual Propagation-Based Approach Example. Word Level ψ i k (f(i) = e f(k) = v) v = ite(i = j, e, g(j)) α(ψ ) i k (α(apply ) = e α(apply 2) = v) v = ite(i = j, e, α(apply 3)) } Primal solver: α(ψ ) Formula abstraction and its negation Dual solver: α(ψ ) Primal assignment: σ(ψ 2) {σ(i) =, σ(j) =, σ(e) =, σ(v) =, σ(k) = 0, α(apply ) =, α(apply 2) =, α(apply 3) = } Fix values of inputs via assumptions to the dual solver: Dual assumptions: σ(ψ 2) {i =, j =, e =, v =, k = 0, α(apply ) =, α(apply 2) =, α(apply 3) = } Failed assumptions: {i =, j =, e =, v =, k = 0, α(apply ) = } Consistency Check

24 Experimental Evaluation Configuration Four Configurations: Boolector sc version entering SMTCOMP 2, winner of the QF AUFBV track Boolector ba current Boolector base version (new LOD for Lambdas engine) Boolector dp with dual propagation-based partial model extraction enabled Boolector ju justification-based partial model extraction approach for comparison determine a posteriori observability don t cares skip lines that do not influence the output of an and-gate under its current assignment if both inputs of an and-gate are controlling ( ) skip either one based on a minimum cost heuristic

25 Experimental Evaluation Configuration Two Benchmark Sets: SMT 2: 49 benchmarks all non-extensional QF AUFBV benchmarks in SMTCOMP 2 Selected: 73 benchmarks all non-extensional QF AUFBV benchmarks (3696) in the SMT-LIB (pre-smtcomp 4) for which Boolector sc required at least 0 seconds 58 benchmarks shared between both sets all experiments on 2.83 GHz Intel Core 2 Quad machines with 8GB RAM running Ubuntu 2.04 time limit: 2 seconds, memory limit: 7GB

26 Experimental Evaluation Overview Overall results on sets SMT 2 and Selected. SMT 2 Selected Solver Solved (sat/unsat) TO MO Time [s] DS [s] Boolector sc 40 (83/57) Boolector ba 4 (83/58) Boolector ju 42 (84/58) Boolector dp 42 (84/58) Boolector sc 6 (72/44) Boolector ba 2 (76/45) Boolector ju 30 (85/45) Boolector dp 30 (85/45) TO... time out Time... total CPU time MO... memory out DS... dual solver overhead

27 Experimental Evaluation Overview Overall results on sets SMT 2 and Selected. SMT 2 Selected Solver Solved (sat/unsat) TO MO Time [s] DS [s] Boolector sc 40 (83/57) Boolector ba 4 (83/58) Boolector ju 42 (84/58) Boolector dp 42 (84/58) Boolector sc 6 (72/44) Boolector ba 2 (76/45) Boolector ju 30 (85/45) Boolector dp 30 (85/45) TO... time out Time... total CPU time MO... memory out DS... dual solver overhead SMT 2: additional instance (sat) Selected: 9 additional instances (all sat)

28 Experimental Evaluation Commonly Solved Instances Results for commonly solved instances on sets SMT 2 and Selected. SMT 2 Selected Solver Time [s] SAT [s] DS overhead [s] LOD Total Avg. Med. Total Avg. Med. Total Avg. Med. Total Avg. Med. Boolectorsc Boolectorba Boolectorju Boolectordp Boolectorsc Boolectorba Boolectorju Boolectordp Time... total CPU time DS overhead... dual solver overhead SAT... SAT solver runtime (primal solver) LOD... number of lemmas generated SMT 2: 39 (out of 49) benchmarks, 82 sat, 57 unsat not representative: 50% solved without a single refinement iteration Selected: 3 (out of 73) benchmarks, 70 sat, 43 unsat

29 Experimental Evaluation Commonly Solved Instances Results for commonly solved instances on sets SMT 2 and Selected. SMT 2 Selected Solver Time [s] SAT [s] DS overhead [s] LOD Total Avg. Med. Total Avg. Med. Total Avg. Med. Total Avg. Med. Boolectorsc Boolectorba Boolectorju Boolectordp Boolectorsc Boolectorba Boolectorju Boolectordp Time... total CPU time DS overhead... dual solver overhead SAT... SAT solver runtime (primal solver) LOD... number of lemmas generated Boolector sc implements old LOD engine new engine (Boolector ba ) struggles on a small set of benchmarks needs further investigation

30 Experimental Evaluation Commonly Solved Instances Results for commonly solved instances on sets SMT 2 and Selected. SMT 2 Selected Solver Time [s] SAT [s] DS overhead [s] LOD Total Avg. Med. Total Avg. Med. Total Avg. Med. Total Avg. Med. Boolectorsc Boolectorba Boolectorju Boolectordp Boolectorsc Boolectorba Boolectorju Boolectordp Time... total CPU time DS overhead... dual solver overhead SAT... SAT solver runtime (primal solver) LOD... number of lemmas generated sat solver runtime (SAT) Boolector dp most notable improvement on both sets

31 Experimental Evaluation Commonly Solved Instances Results for commonly solved instances on sets SMT 2 and Selected. SMT 2 Selected Solver Time [s] SAT [s] DS overhead [s] LOD Total Avg. Med. Total Avg. Med. Total Avg. Med. Total Avg. Med. Boolectorsc Boolectorba Boolectorju Boolectordp Boolectorsc Boolectorba Boolectorju Boolectordp Time... total CPU time DS overhead... dual solver overhead SAT... SAT solver runtime (primal solver) LOD... number of lemmas generated number of lemmas generated (LOD) SMT 2: Boolector ju least number of lemmas Boolector dp and Boolector ba approx. the same on 4 instances x more lemmas than Boolector ba Selected: Boolector dp most notable improvement

32 Experimental Evaluation Commonly Solved Instances Results for commonly solved instances on sets SMT 2 and Selected. SMT 2 Selected Solver Time [s] SAT [s] DS overhead [s] LOD Total Avg. Med. Total Avg. Med. Total Avg. Med. Total Avg. Med. Boolectorsc Boolectorba Boolectorju Boolectordp Boolectorsc Boolectorba Boolectorju Boolectordp Time... total CPU time DS overhead... dual solver overhead SAT... SAT solver runtime (primal solver) LOD... number of lemmas generated dual solver overhead 30-40% in total on 0% of the benchmarks 50-70% of the total runtime on >50% of the benchmarks <0% of the total runtime Boolector dp outperforms others disregarding DS overhead online dual propagation approach: DS overhead negligible

33 Experimental Evaluation Boolector dp vs Boolector ba 0 0 Boolector dp runtime [s] Boolector ba runtime [s] 0 0 Boolector ba runtime [s] DS overhead included DS overhead not included

34 Conclusion dual propagation-based optimization for Lemmas on Demand don t care reasoning on full candidate models improves performance our offline dual propagation-based approach competitive (in spite of introducing considerable overhead) Boolector ju won QF ABV track of SMTCOMP 4 Boolector dp came in close second Future work: online dual propagation approach, promises negligible or no dual solver overhead further improvment of overall performance by enabling partial model extraction even before a full candidate model has been generated requires interleaved execution between primal and dual solver

35 Appendix Boolector dp vs Boolector ju 0 0 Boolector dp runtime [s] Boolector ju runtime [s] 0 0 Boolector ju runtime [s] DS overhead included DS overhead not included

36 Appendix Boolector dp vs Boolector sc 0 0 Boolector dp runtime [s] Boolector sc runtime [s] 0 0 Boolector sc runtime [s] DS overhead included DS overhead not included

37 References I J. D. Bingham and A. J. Hu. Semi-formal bounded model checking. In CAV 02, volume 2404 of LNCS. Springer, 22. C. Barrett and J. Donham. Combining sat methods with non-clausal decision heuristics. ENTCS, 25(3), 25. L. de Moura and N. Bjørner. Relevancy propagation. Technical Report MSR-TR-27-40, Microsoft Research, 27. Z. S. Andraus, M. H. Liffiton, and K. A. Sakallah. Reveal: A formal verification tool for Verilog designs. In LPAR 08, volume 5330 of LNCS. Springer, 28. R. Brummayer and A. Biere. Lemmas on demand for the extensional theory of arrays. JSAT, 6(-3), 29. H. Chockler, A. Ivrii, A. Matsliah, S. Moran, and Z. Nevo. Incremental formal verification of hardware. In FMCAD. FMCAD Inc., 20. N. Eén, A. Mishchenko, and R. K. Brayton. Efficient implementation of property directed reachability. In FMCAD. FMCAD Inc., 20.

38 References II D. Déharbe, P. Fontaine, D. Le Berre and B. Mazure. Computing prime implicants. In FMCAD 3. IEEE, 203. A. Goultiaeva and F. Bacchus. Exploiting QBF duality on a circuit representation. In AAAI 0. AAAI Press, 2. M. Preiner, A. Niemetz and A. Biere. Lemmas on Demand for Lambdas. In DIFTS 3, volume 30 of CEUR Workshop Proceedings, 203.