A Scalable and Nearly Uniform Generator of SAT Witnesses

Transcription

1 1 A Scalable and Nearly Uniform Generator of SAT Witnesses Supratik Chakraborty 1, Kuldeep S Meel 2, Moshe Y Vardi 2 1 Indian Institute of Technology Bombay, India 2 Department of Computer Science, Rice University Motivators: R. Bodik (UCB), P. Madhusudhan(UIUC), A. Solar-Lezama(MIT), S. Seshia(UCB) Aug 20, 2013 NSF Site Visit 2013

2 2 Computer-Aided Education (Motivation: P. Madhusudhan) Automatic Problem Generation Formulate a template for questions Constraints for template parameters (correctness and difficulty level) Uniformly generate parameters that satisfy the constraints Uniform Generation of solutions to set of constraints

3 3 Sketch-based Synthesis (Motivation: R. Bodik, A. Solar-Lezama, S. Seshia) Given: Sketch and correctness condition Large space of programs that satisfy the correctness conditions Goal: Get the optimal program (running time, memory) Uniformly sample from the space of programs Uniform Generation of solutions to set of constraints

4 4 Sketch based Synthesis Automatic Problem Generation Uniform Generation

5 5 Sketch based Synthesis Design Methodology Automatic Problem Generation Uniform Generation Computational Engines Tools and Evaluation Challenge Problems Education and Knowledge Transfer Apps for Multicore Engines Multicore Protocols Networked Systems Robotic Systems

6 Problem Formulation 6 Set of Constraints CNF SAT Formula Given a SAT formula, can one uniformly sample solutions without enumerating all solutions

7 Problem Formulation 7 Set of Constraints CNF SAT Formula Given a SAT formula, can one uniformly sample solutions without enumerating all solutions while scaling to real world problems?

8 Overview 8 Prior Work & Our Approach Theoretical Results Experimental Results Where do we go from here?

9 Prior Work 9 BDD-based Poor performance SAT-based heuristics No guarantees INDUSTRY Theoretical Work Guarantees: strong Performance: weak Heuristic Work Guarantees: weak Performance: strong ACADEMIA BGP Algorithm (Bellare, Goldreich & Petrank,98) XORSample (Gomes, Sabhrawal & Selman, 07)

10 Our Contribution 10 BDD-based Poor performance UniWit Guarantees : strong Performance: strong SAT-based heuristics No guarantees INDUSTRY Theoretical Work Guarantees: strong Performance: weak Heuristic Work Guarantees: weak Performance: strong ACADEMIA BGP Algorithm (Bellare, Goldreich & Petrank,98) XORSample (Gomes, Sabhrawal & Selman, 07)

11 11 Central Idea

12 12 Central Idea

13 13 Central Idea

14 Central Idea 14 How can you partition into roughly equal cells of solutions when you don t know the distribution of solutions? Universal hash functions [Carter-Wegman (1979), Sipser (1983)]

15 The Power of Hashing 15 F : Input CNF Formula with n variables X: Set of input variables H(n,m,r): Family of r-wise independent hash functions mapping {0,1} n to {0,1} m A randomly picked h from H(n,m,r) treats upto r objects as independent Lower independence => lower complexity Solving for one cell: Solve F (h(x)=a)

16 Prior Hashing-Based Approaches 16 Small : Small : n-independent hashing N-independent N-independent Hashing Hashing R F : Solution space R F : Solution space 3-independent 3-independent Hashing Hashing Random Random Prior Work BGP Algorithm Prior Our Work Our Approach Approach Partitioned space Partitioned space Partitioned All space cells are small Partitioned A random space cells is small All cells are small A random cells is small All cells should be small Uniform Generation

17 Hashing-Based Approaches 17 Small : R F : Solution space R F : Solution space R F : Solution space Small : Small : N-independent 3-independent 2-independent hashing N-independent Hashing 3-independent Hashing Hashing N-independent Hashing 3-independent Random Hashing Hashing Random Random Random n-independent hashing Prior Work BGP Algorithm Prior Our Work Our Approach Prior Approach Our Partitioned space Partitioned space Work Approach Partitioned All space cells are small Partitioned A random space cells is small Partitioned space Partitioned space All cells are small A random cells is small All cells are small A random cells is small All cells should be small Only a randomly chosen cells needs to be small UniWit Uniform Generation Near Uniform Generation

18 Hashing-Based Approaches 18 Small : R F : Solution space R F : Solution space R F : Solution space Small : Small : N-independent 3-independent 2-independent hashing N-independent Hashing 3-independent Hashing Hashing N-independent Hashing 3-independent Random Hashing Hashing Random Random Random n-independent hashing From tens of variables to Prior Work BGP Algorithm thousands of variables! Prior Our Work Our Approach Prior Approach Our Partitioned space Partitioned space Work Approach Partitioned All space cells are small Partitioned A random space cells is small Partitioned space Partitioned space All cells are small A random cells is small All cells are small A random cells is small All cells should be small Only a randomly chosen cells needs to be small UniWit Uniform Generation Near Uniform Generation

19 Highlights 19 Employs XOR-based hash functions instead of computationally infeasible algebraic hash functions Uses off-the-shelf SAT solver CryptoMiniSAT (MiniSAT+XOR support)

20 Theorems 20 Uniformity For every solution y of R F Pr [y is output] = 1/ R F

21 Theorems 21 Near Uniformity For every solution y of R F Pr [y is output] >= 1 /8 x 1/ R F Success Probability Algorithm UniWit succeeds with probability at least 1/8 Polynomial calls to SAT Solver

22 Experimental Methodology 22 Benchmarks (over 200) Bit-blasted versions of word-level constraints from VHDL designs Bit-blasted versions from SMTLib version and ISCAS 85 Objectives Comparison with algorithms BGP & XORSample Uniformity Performance

23 Results: Uniformity XORSample Uniform Uniform/ Uniwit Uniform Uniform/8 Frequency Frequency Solutions XORSample Solutions UniWit Benchmark: case110.cnf; #var: 287; #clauses: 1263 Total Runs: 1.08x10 8 ; Total Solutions : XORSample could not find 772 solutions and more than 250 solutions were generated only once

24 case47 case_3_b14_3 case105 case8 case203 case145 case61 case9 case15 case140 case_2_b14_1 case_3_b14_1 squaring14 squaring7 case_2_ptb_1 case_1_ptb_1 case_2_b14_2 case_3_b14_2 Results : Performance Time(s) UniWit XORSample' Benchmarks

25 case47 case_3_b14_3 case105 case8 case203 case145 case61 case9 case15 case140 case_2_b14_1 case_3_b14_1 squaring14 squaring7 case_2_ptb_1 case_1_ptb_1 case_2_b14_2 case_3_b14_2 Results : Performance Time(s) UniWit is is 2-3 orders of magnitude faster than XORSample 10 Observed success probability = 0.6 ( >> theoretical guarantee UniWit of 0.125) 1 XORSample' 0.1 Benchmarks

26 Key Takeaways 26 Uniform sampling is an important problem Prior work either didn t scale or offered weak guarantees We use 2-wise independent hash function to divide solution space into small cells Only a randomly chosen cell has to be small Theoretical guarantees of near uniformity Major improvements in running time and uniformity compared to the existing generators Tool is available at

27 Where Do We Go From Here? 27 Extension to more expressive constraints (SMT) Extending the technique to model counting (CP 13) Efficient hash functions Deployment in automatic problem generation and Sketch-based tools

28 Discussion 28 Thank You for your attention!

29 UniWit 29 R F

30 UniWit 30 R F NO

31 31 UniWit

32 UniWit 32 NO

33 33 UniWit

34 UniWit 34 YES

35 UniWit 35 YES Select a solution randomly with probability c from the partition. If no solution is picked, return Failure