Minimum Satisfying Assignments for SMT. Işıl Dillig, Tom Dillig Ken McMillan Alex Aiken College of William & Mary Microsoft Research Stanford U.

Transcription

1 Minimum Satisfying Assignments for SMT Işıl Dillig, Tom Dillig Ken McMillan Alex Aiken College of William & Mary Microsoft Research Stanford U. 1 / 20

2 Satisfiability Modulo Theories (SMT) Today, SMT solvers are underlying engine of most verification and program analysis tools 2 / 20

3 Satisfiability Modulo Theories (SMT) Today, SMT solvers are underlying engine of most verification and program analysis tools SMT Formula SMT Solver SAT UNSAT 2 / 20

4 Satisfiability Modulo Theories (SMT) Today, SMT solvers are underlying engine of most verification and program analysis tools SMT Formula SMT Solver + satisfying SAT assignment x=1, y=0, z=-3, w=5 UNSAT An assignment σ for formula φ is a mapping from free variables of φ to values 2 / 20

5 Partial Satisfying Assignments SMT Formula SMT Solver + satisfying SAT assignment x=1, y=0, z=-3, w=5 UNSAT Satisfying assignments provided SMT solver are full assignments assign every free variable to a value 3 / 20

6 Partial Satisfying Assignments SMT Formula SMT Solver + satisfying SAT assignment x=1, y=0, z=-3, w=5 UNSAT Satisfying assignments provided SMT solver are full assignments assign every free variable to a value But sometimes we want partial satisfying assignments 3 / 20

7 Partial Satisfying Assignments SMT Formula SMT Solver + satisfying SAT assignment x=1, y=0, z=-3, w=5 UNSAT Satisfying assignments provided SMT solver are full assignments assign every free variable to a value But sometimes we want partial satisfying assignments A partial satisfying assignment only assigns values to a subset of free variables, but is sufficient to make formula true 3 / 20

8 Partial Satisfying Assignments SMT Formula SMT Solver + satisfying SAT assignment x=1, y=0, z=-3, w=5 UNSAT Satisfying assignments provided SMT solver are full assignments assign every free variable to a value But sometimes we want partial satisfying assignments A partial satisfying assignment only assigns values to a subset of free variables, but is sufficient to make formula true For formula x < 0 x + y 0, x = 1 is a partial satisfying assignment 3 / 20

9 Minimum Satisfying Assignments (MSA) Special class of partial sat assignments: minimum sat assignments 4 / 20

10 Minimum Satisfying Assignments (MSA) Special class of partial sat assignments: minimum sat assignments Given cost function C from variables to costs, minimum sat assignment (MSA) is a partial sat assignment minimizing C C(x) = 1 C(y) = 50 C(z) = 1 C(w) = 4 4 / 20

11 Minimum Satisfying Assignments (MSA) Special class of partial sat assignments: minimum sat assignments Given cost function C from variables to costs, minimum sat assignment (MSA) is a partial sat assignment minimizing C Observe: Cost of assignment does not depend on values, but only to variables used in assignment! C(x) = 1 C(y) = 50 C(z) = 1 C(w) = 4 4 / 20

12 Minimum Satisfying Assignments (MSA) Special class of partial sat assignments: minimum sat assignments Given cost function C from variables to costs, minimum sat assignment (MSA) is a partial sat assignment minimizing C Observe: Cost of assignment does not depend on values, but only to variables used in assignment! Assignments x = 1 and x = 50 have same cost C(x) = 1 C(y) = 50 C(z) = 1 C(w) = 4 4 / 20

13 Minimum Satisfying Assignments (MSA) Special class of partial sat assignments: minimum sat assignments Given cost function C from variables to costs, minimum sat assignment (MSA) is a partial sat assignment minimizing C Observe: Cost of assignment does not depend on values, but only to variables used in assignment! Assignments x = 1 and x = 50 have same cost C(x) = 1 C(y) = 50 C(z) = 1 C(w) = 4 If variables have equal cost, an MSA is partial sat assignment with fewest variables 4 / 20

14 Example and Applications Consider cost function assigning every variable to 1 and Presburger arithmetic formula: φ : x + y + w > 0 x + y + z + w < 5 5 / 20

15 Example and Applications Consider cost function assigning every variable to 1 and Presburger arithmetic formula: φ : x + y + w > 0 x + y + z + w < 5 A minimum sat assignment for φ is z = 0 5 / 20

16 Example and Applications Consider cost function assigning every variable to 1 and Presburger arithmetic formula: φ : x + y + w > 0 x + y + z + w < 5 A minimum sat assignment for φ is z = 0 MSAs have many applications in verification: Finding small counterexamples in BMC Classifying and diagnosing error reports Abductive inference Minimizing # of predicates in pred abstraction 5 / 20

17 Contributions In propositional logic, MSAs known as min-sized prime implicants 6 / 20

18 Contributions In propositional logic, MSAs known as min-sized prime implicants Known algorithms for computing min prime implicants 6 / 20

19 Contributions In propositional logic, MSAs known as min-sized prime implicants Known algorithms for computing min prime implicants No work on computing MSAs for richer theories 6 / 20

20 Contributions In propositional logic, MSAs known as min-sized prime implicants Known algorithms for computing min prime implicants No work on computing MSAs for richer theories First algorithm for computing min sat assignments for SMT formulas Our algorithm applicable to any theory for which full first-order logic including quantifiers is decidable 6 / 20

21 Maximum Universal Subset Our algorithm exploits duality between MSAs and maximum universal subsets (MUS) MSA MUS 7 / 20

22 Maximum Universal Subset Our algorithm exploits duality between MSAs and maximum universal subsets (MUS) An MUS for φ is a set of variables X s.t.: MSA MUS 7 / 20

23 Maximum Universal Subset Our algorithm exploits duality between MSAs and maximum universal subsets (MUS) An MUS for φ is a set of variables X s.t.: 1 X.φ is satisfiable MSA MUS 7 / 20

24 Maximum Universal Subset Our algorithm exploits duality between MSAs and maximum universal subsets (MUS) An MUS for φ is a set of variables X s.t.: 1 X.φ is satisfiable 2 X maximizes cost function C MSA MUS 7 / 20

25 Maximum Universal Subset Our algorithm exploits duality between MSAs and maximum universal subsets (MUS) An MUS for φ is a set of variables X s.t.: 1 X.φ is satisfiable 2 X maximizes cost function C MSA MUS X is an MUS of φ MSA is a sat assignment of X.φ 7 / 20

26 Maximum Universal Subset Our algorithm exploits duality between MSAs and maximum universal subsets (MUS) An MUS for φ is a set of variables X s.t.: 1 X.φ is satisfiable 2 X maximizes cost function C MSA MUS X is an MUS of φ MSA is a sat assignment of X.φ Our approach first computes an MUS X and extracts an MSA from a sat assignment of X.φ 7 / 20

27 Algorithm to Compute MUS Recursive branch-and-bound style algorithm with input: 8 / 20

28 Algorithm to Compute MUS Recursive branch-and-bound style algorithm with input: Current formula φ 8 / 20

29 Algorithm to Compute MUS Recursive branch-and-bound style algorithm with input: Current formula φ Cost function C 8 / 20

30 Algorithm to Compute MUS Recursive branch-and-bound style algorithm with input: Current formula φ Cost function C Remaining variables V 8 / 20

31 Algorithm to Compute MUS Recursive branch-and-bound style algorithm with input: Current formula φ Cost function C Remaining variables V Lower bound L 8 / 20

32 Algorithm to Compute MUS Recursive branch-and-bound style algorithm with input: Current formula φ Cost function C Remaining variables V Lower bound L L is used to prune the search 8 / 20

33 Algorithm to Compute MUS Recursive branch-and-bound style algorithm with input: Current formula φ Cost function C Remaining variables V Lower bound L L is used to prune the search At each recursive call, considers new variable x and decides if x is in or out of MUS 8 / 20

34 Algorithm to Compute MUS Recursive branch-and-bound style algorithm with input: Current formula φ Cost function C Remaining variables V Lower bound L L is used to prune the search At each recursive call, considers new variable x and decides if x is in or out of MUS We do this by comparing cost of universal subsets with and without x 8 / 20

35 Algorithm to Compute MUS, cont. First, check if possible to include x is in universal subset 9 / 20

36 Algorithm to Compute MUS, cont. First, check if possible to include x is in universal subset If so, recursively compute cost of universal subset containing x 9 / 20

37 Algorithm to Compute MUS, cont. First, check if possible to include x is in universal subset If so, recursively compute cost of universal subset containing x If cost of current universal subset is better than previous best cost, update lower bound 9 / 20

38 Algorithm to Compute MUS, cont. First, check if possible to include x is in universal subset If so, recursively compute cost of universal subset containing x If cost of current universal subset is better than previous best cost, update lower bound Next, compute cost of universal subset not containing x 9 / 20

39 Algorithm to Compute MUS, cont. First, check if possible to include x is in universal subset If so, recursively compute cost of universal subset containing x If cost of current universal subset is better than previous best cost, update lower bound Next, compute cost of universal subset not containing x Compare the two costs and return whichever is best 9 / 20

40 Discussion of Algorithm This algorithm is branch-and-bound style 10 / 20

41 Discussion of Algorithm This algorithm is branch-and-bound style Branches on whether a given variable is in or out of MUS 10 / 20

42 Discussion of Algorithm This algorithm is branch-and-bound style Branches on whether a given variable is in or out of MUS Bounds search by checking if current max possible cost cannot improve previous estimate Cost(V) < L 10 / 20

43 Discussion of Algorithm This algorithm is branch-and-bound style Branches on whether a given variable is in or out of MUS Bounds search by checking if current max possible cost cannot improve previous estimate Also bounds search by checking when X.φ becomes unsat Cost(V) < L 10 / 20

44 Discussion of Algorithm This algorithm is branch-and-bound style Branches on whether a given variable is in or out of MUS Bounds search by checking if current max possible cost cannot improve previous estimate Also bounds search by checking when X.φ becomes unsat Cost(V) < L These two pruning strategies eliminate many search paths, but still exponential 10 / 20

45 Discussion of Algorithm This algorithm is branch-and-bound style Branches on whether a given variable is in or out of MUS Bounds search by checking if current max possible cost cannot improve previous estimate Also bounds search by checking when X.φ becomes unsat Cost(V) < L These two pruning strategies eliminate many search paths, but still exponential To make algorithm practical, must consider more optimizations 10 / 20

46 Improvements over Basic Algorithm Two important ways to improve over basic algorithm: 1 Initial cost estimate 11 / 20

47 Improvements over Basic Algorithm Two important ways to improve over basic algorithm: 1 Initial cost estimate Basic algorithm initially sets lower bound on MUS cost to 0 11 / 20

48 Improvements over Basic Algorithm Two important ways to improve over basic algorithm: 1 Initial cost estimate Basic algorithm initially sets lower bound on MUS cost to 0 Improves as algorithm progresses, but initially ineffective 11 / 20

49 Improvements over Basic Algorithm Two important ways to improve over basic algorithm: 1 Initial cost estimate Basic algorithm initially sets lower bound on MUS cost to 0 Improves as algorithm progresses, but initially ineffective If we can quickly compute a better initial lower bound estimate, pruning can be much more effective 11 / 20

50 Improvements over Basic Algorithm Two important ways to improve over basic algorithm: 1 Initial cost estimate Basic algorithm initially sets lower bound on MUS cost to 0 Improves as algorithm progresses, but initially ineffective If we can quickly compute a better initial lower bound estimate, pruning can be much more effective 2 Variable order 11 / 20

51 Improvements over Basic Algorithm Two important ways to improve over basic algorithm: 1 Initial cost estimate Basic algorithm initially sets lower bound on MUS cost to 0 Improves as algorithm progresses, but initially ineffective If we can quickly compute a better initial lower bound estimate, pruning can be much more effective 2 Variable order Basic algorithm chooses variables randomly 11 / 20

52 Improvements over Basic Algorithm Two important ways to improve over basic algorithm: 1 Initial cost estimate Basic algorithm initially sets lower bound on MUS cost to 0 Improves as algorithm progresses, but initially ineffective If we can quickly compute a better initial lower bound estimate, pruning can be much more effective 2 Variable order Basic algorithm chooses variables randomly But some variable orders much better than others 11 / 20

53 Improvements over Basic Algorithm Two important ways to improve over basic algorithm: 1 Initial cost estimate Basic algorithm initially sets lower bound on MUS cost to 0 Improves as algorithm progresses, but initially ineffective If we can quickly compute a better initial lower bound estimate, pruning can be much more effective 2 Variable order Basic algorithm chooses variables randomly But some variable orders much better than others Turns out better to consider variables likely to be in MSA first 11 / 20

54 Finding Initial Cost and Variable Order If we can find good approximation for MSA, can obtain good initial cost and variable order for MUS MSA MUS 12 / 20

55 Finding Initial Cost and Variable Order If we can find good approximation for MSA, can obtain good initial cost and variable order for MUS MUS Cost = Cost of free vars - MSA cost MSA MUS 12 / 20

56 Finding Initial Cost and Variable Order If we can find good approximation for MSA, can obtain good initial cost and variable order for MUS MUS Cost = Cost of free vars - MSA cost Thus, good estimate on MSA cost gives good estimate on MUS cost MSA MUS 12 / 20

57 Finding Initial Cost and Variable Order If we can find good approximation for MSA, can obtain good initial cost and variable order for MUS MUS Cost = Cost of free vars - MSA cost Thus, good estimate on MSA cost gives good estimate on MUS cost Good approximate MSA gives good variable order b/c if x is in MSA, x.φ more likely unsat MSA MUS 12 / 20

58 Finding Initial Cost and Variable Order If we can find good approximation for MSA, can obtain good initial cost and variable order for MUS MUS Cost = Cost of free vars - MSA cost Thus, good estimate on MSA cost gives good estimate on MUS cost Good approximate MSA gives good variable order b/c if x is in MSA, x.φ more likely unsat MSA MUS How can we "quickly" find good enough approximate MSA? 12 / 20

59 Approximate MSA from Prime Implicants Use min prime implicant to boolean structure of formula to approximate MSA SMT MSA SAT MinPI 13 / 20

60 Approximate MSA from Prime Implicants Use min prime implicant to boolean structure of formula to approximate MSA Prime implicant of boolean formula is conjunction of literals that implies it SMT MSA SAT MinPI 13 / 20

61 Approximate MSA from Prime Implicants Use min prime implicant to boolean structure of formula to approximate MSA Prime implicant of boolean formula is conjunction of literals that implies it SMT MSA First, compute φ + by replacing every literal in φ by boolean variable SAT MinPI 13 / 20

62 Approximate MSA from Prime Implicants Use min prime implicant to boolean structure of formula to approximate MSA Prime implicant of boolean formula is conjunction of literals that implies it SMT MSA First, compute φ + by replacing every literal in φ by boolean variable Extend techniques for computing MinPI to find a theory-satisfiable MinPI SAT MinPI 13 / 20

63 Approximate MSA from Prime Implicants Use min prime implicant to boolean structure of formula to approximate MSA Prime implicant of boolean formula is conjunction of literals that implies it SMT MSA First, compute φ + by replacing every literal in φ by boolean variable Extend techniques for computing MinPI to find a theory-satisfiable MinPI Theory-sat PI implies boolean structure of formula and is satisfiable modulo theory SAT MinPI 13 / 20

64 Approximate MSA from Prime Implicants Use min prime implicant to boolean structure of formula to approximate MSA Prime implicant of boolean formula is conjunction of literals that implies it SMT MSA First, compute φ + by replacing every literal in φ by boolean variable Extend techniques for computing MinPI to find a theory-satisfiable MinPI Theory-sat PI implies boolean structure of formula and is satisfiable modulo theory SAT MinPI Approximate MSA as variables in MinPI 13 / 20

65 Summary of First Optimization SAT MinPI Optimize basic B&B algorithm by finding good lower bound estimate on MUS and variable order Approximate MSA Cost estimate Variable x1 order x5 x2 Better pruning 14 / 20

66 Summary of First Optimization SAT MinPI Optimize basic B&B algorithm by finding good lower bound estimate on MUS and variable order Approximate MSA To find good estimate and variable order, compute approximate MSA Cost estimate Variable x1 order x5 x2 Better pruning 14 / 20

67 Summary of First Optimization SAT MinPI Optimize basic B&B algorithm by finding good lower bound estimate on MUS and variable order To find good estimate and variable order, compute approximate MSA Approximate MSA is obtained from theory-satisfiable min PI of boolean structure Approximate MSA Cost estimate Better pruning Variable x1 order x5 x2 14 / 20

68 Another Improvement: Non-Universal Subsets Suppose we knew a set of variables V is a non-universal set (i.e., V.φ is UNSAT) 15 / 20

69 Another Improvement: Non-Universal Subsets Suppose we knew a set of variables V is a non-universal set (i.e., V.φ is UNSAT) Can use non-universal subsets to improve algorithm because can avoid branching without SAT( X.φ) check 15 / 20

70 Another Improvement: Non-Universal Subsets Suppose we knew a set of variables V is a non-universal set (i.e., V.φ is UNSAT) Can use non-universal subsets to improve algorithm because can avoid branching without SAT( X.φ) check Furthermore, if V is a non-universal subset of implicate of φ, it is also non-universal subset of of φ. 15 / 20

71 Another Improvement: Non-Universal Subsets Suppose we knew a set of variables V is a non-universal set (i.e., V.φ is UNSAT) Can use non-universal subsets to improve algorithm because can avoid branching without SAT( X.φ) check Furthermore, if V is a non-universal subset of implicate of φ, it is also non-universal subset of of φ. How can we "quickly" find implicates with small non-universal subsets? 15 / 20

72 Finding Non-Universal Subsets For complete theories, such as Presburger arithmetic, if ψ sat, then free(ψ).ψ unsat UNSAT! x1 x10 x7 x4 16 / 20

73 Finding Non-Universal Subsets For complete theories, such as Presburger arithmetic, if ψ sat, then free(ψ).ψ unsat Thus, if ψ is an implicate of φ whose negation is sat, free(ψ) is a non-universal set UNSAT! x1 x10 x7 x4 16 / 20

74 Finding Non-Universal Subsets For complete theories, such as Presburger arithmetic, if ψ sat, then free(ψ).ψ unsat Thus, if ψ is an implicate of φ whose negation is sat, free(ψ) is a non-universal set Can quickly find implicates with this property from boolean structure of simplified form x1 x10 x7 x4 UNSAT! 16 / 20

75 Finding Non-Universal Subsets For complete theories, such as Presburger arithmetic, if ψ sat, then free(ψ).ψ unsat Thus, if ψ is an implicate of φ whose negation is sat, free(ψ) is a non-universal set Can quickly find implicates with this property from boolean structure of simplified form When all variables in ψ are -quantified, backtrack without checking satisfiability x1 x10 x7 x4 UNSAT! 16 / 20

76 Experimental Evaluation Implemented algorithm in Mistral SMT solver 17 / 20

77 Experimental Evaluation Implemented algorithm in Mistral SMT solver Evaluated algorithm on 400 Presburger arithmetic formulas 17 / 20

78 Experimental Evaluation Implemented algorithm in Mistral SMT solver Evaluated algorithm on 400 Presburger arithmetic formulas Formulas taken from static analysis tool that uses MSAs for performing abduction, in turn used for diagnosing error reports 17 / 20

79 Experimental Evaluation Implemented algorithm in Mistral SMT solver Evaluated algorithm on 400 Presburger arithmetic formulas Formulas taken from static analysis tool that uses MSAs for performing abduction, in turn used for diagnosing error reports Formulas contain up to 40 variables and several hundred boolean connectives 17 / 20

80 Experimental Results Basic (no opt) Basic + min. implicants Basic + non-univ-subsets Both optimizations 30 Time (s) Number of variables 18 / 20

81 Experimental Results Basic (no opt) Basic + min. implicants Basic + non-univ-subsets Both optimizations Basic algorithm very sensitive to # vars 30 Time (s) Number of variables 18 / 20

82 Experimental Results Time (s) Basic (no opt) Basic + min. implicants Basic + non-univ-subsets Both optimizations Basic algorithm very sensitive to # vars Optimizations have dramatic impact on performance Number of variables 18 / 20

83 Experimental Results Time (s) Basic (no opt) Basic + min. implicants Basic + non-univ-subsets Both optimizations Basic algorithm very sensitive to # vars Optimizations have dramatic impact on performance Number of variables Optimized version grows slowly in # of variables 18 / 20

84 Experimental Results Time (s) Basic (no opt) Basic + min. implicants Basic + non-univ-subsets Both optimizations Basic algorithm very sensitive to # vars Optimizations have dramatic impact on performance Number of variables Optimized version grows slowly in # of variables Even with both optimizations, computing MSAs 25 times more expensive than checking satisfiability 18 / 20

85 Experimental Results, cont Basic (no opt) Basic + min. implicants Basic + non-univ-subsets Both optimizations 20 Time (s) MSA size / # vars 19 / 20

86 Experimental Results, cont. Time (s) Basic (no opt) Basic + min. implicants Basic + non-univ-subsets Both optimizations Problem easier if # vars in MSA very small or very large MSA size / # vars 19 / 20

87 Experimental Results, cont. Time (s) Basic (no opt) Basic + min. implicants Basic + non-univ-subsets Both optimizations MSA size / # vars Problem easier if # vars in MSA very small or very large Problem hardest for formulas when ratio of vars in MSA to free vars is / 20

88 Summary First algorithm for finding MSAs of SMT formulas Recursive branch-and-bound style algorithm with two crucial optimizations MSAs can be computed in reasonable time for a set of benchmakrs obtained from static analysis But finding MSAs much more expensive than finding full sat assignment We believe significant improvements are still possible 20 / 20