Programs that write themselves: Program synthesis for the masses. Yu Feng UT Austin

Size: px
Start display at page:

Download "Programs that write themselves: Program synthesis for the masses. Yu Feng UT Austin"

Transcription

1 Programs that write themselves: Program synthesis for the masses Yu Feng UT Austin

2 New platforms impose demand for programming 2

3 New platforms impose demand for programming 2

4 New platforms impose demand for programming 2

5 New platforms impose demand for programming 2

6 New platforms impose demand for programming 2

7 New platforms impose demand for programming 2

8 Program Synthesis Program synthesis make programming more accessible 3

9 Program Synthesis Program synthesis make programming more accessible Malware Signature Astroid, NDSS 7 3

10 Program Synthesis Program synthesis make programming more accessible Malware Signature Algorithmic Program Astroid, NDSS 7 SyPet, POPL 7 3

11 Program Synthesis Program synthesis make programming more accessible Malware Signature Algorithmic Program Data Wrangling Astroid, NDSS 7 SyPet, POPL 7 3 Morpheus, PLDI 7

12 Outline 4

13 Outline Morpheus: automating data wrangling tasks in data science (Completed) 4

14 Outline Morpheus: automating data wrangling tasks in data science (Completed) SyPet: generating complex programs from nontrivial APIs (Completed) 4

15 Outline Morpheus: automating data wrangling tasks in data science (Completed) SyPet: generating complex programs from nontrivial APIs (Completed) Neo: a CDCL constraint solver for general synthesis problems (Ongoing) 4

16 Data science is important in many application domains 5

17 AI Data science is important in many application domains 5

18 AI Market analysis Data science is important in many application domains 5

19 AI Market analysis Data science is important in many application domains Intrusion Detection 5

20 AI Market analysis Data science is important in many application domains Intrusion Detection Recommender System 5

21 Data science in reality 6

22 Data extraction Data science in reality 6

23 Data extraction Data cleaning Data science in reality 6

24 Data extraction Data cleaning Table transformation Data science in reality 6

25 Data extraction Data cleaning Table transformation Data science in reality Data scientists spend 80% of time on tedious data wrangling tasks 6

26 Morpheus at a glance Morpheus Synthesizer 7

27 Morpheus at a glance Morpheus Synthesizer 7

28 Morpheus at a glance Morpheus Synthesizer 7

29 Morpheus at a glance Table transformation Table reshaping Data consolidation Data computation Morpheus Synthesizer 7

30 8

31 8

32 A general framework End user Morpheus Synthesizer 9

33 A general framework End user Morpheus Synthesizer Morpheus Framework 9

34 A general framework End user Morpheus Synthesizer Morpheus Framework 9

35 A general framework End user Morpheus Synthesizer Morpheus Framework Synthesizer designer 9

36 A general framework End user Morpheus Synthesizer Morpheus Framework 9 Synthesizer designer Can be easily instantiated for any set of components!

37 Key contributions 0

38 Key contributions Novel component-based synthesis algorithm that utilizes overapproximate component specifications 0

39 Key contributions Novel component-based synthesis algorithm that utilizes overapproximate component specifications Core algorithm based on enumerative search, but uses SMT-based deduction over component specs to reject partial programs 0

40 Key contributions Novel component-based synthesis algorithm that utilizes overapproximate component specifications Core algorithm based on enumerative search, but uses SMT-based deduction over component specs to reject partial programs Further increases power of deductive reasoning by performing partial evaluation 0

41 Why incomplete specs

42 Why incomplete specs Unlike prior approaches (Brahma, Synquid), Morpheus doesn t require precise specs

43 Why incomplete specs Unlike prior approaches (Brahma, Synquid), Morpheus doesn t require precise specs Difficult or impossible to write precise specs in many cases (e.g. table pivoting, reshaping)

44 High-level synthesis algorithm 2

45 High-level synthesis algorithm Input Hypothesis Generation 2

46 High-level synthesis algorithm Input Hypothesis Generation Partial program SMT-based Deduction 2

47 High-level synthesis algorithm Input Prune partial programs Hypothesis Generation Partial program SMT-based Deduction 2

48 High-level synthesis algorithm Input Prune partial programs Hypothesis Generation Partial program Partial program SMT-based Deduction Type-directed Completion 2

49 High-level synthesis algorithm Input Prune partial programs Hypothesis Generation Partial program Partial program SMT-based Deduction Strengthen deduction via PE Type-directed Completion Partial Evaluation 2

50 High-level synthesis algorithm Input Prune partial programs Hypothesis Generation Partial program Partial program SMT-based Deduction Strengthen deduction via PE Type-directed Completion Partial Evaluation Full program 2

51 High-level synthesis algorithm Input Prune partial programs Hypothesis Generation Partial program Partial program SMT-based Deduction Strengthen deduction via PE Type-directed Completion Partial Evaluation Full program 2

52 Hypotheses Input Prune partial programs Hypothesis Generation Partial program Partial program SMT-based Deduction Strengthen deduction via PE Type-directed Completion Partial Evaluation Full program 3

53 Hypotheses Input Prune partial programs Hypothesis Generation Partial program Partial program SMT-based Deduction Strengthen deduction via PE Type-directed Completion Partial Evaluation Full program 3

54 Hypotheses Input Prune partial programs Hypothesis Generation Partial program Partial program SMT-based Deduction Strengthen deduction via PE Type-directed Completion Partial Evaluation Full program 3

55 SMT-based deduction Input Prune partial programs Hypothesis Generation Partial program Partial program SMT-based Deduction Strengthen deduction via PE Type-directed Completion Partial Evaluation Full program 4

56 SMT-based deduction Input Prune partial programs 3 Hypothesis Generation Partial program Partial program SMT-based Deduction Strengthen deduction via PE 2 Type-directed Completion Partial Evaluation Full program 4

57 SMT-based deduction Input Prune partial programs 3 Hypothesis Generation Partial program Partial program SMT-based Deduction Strengthen deduction via PE 2 Type-directed Completion Partial Evaluation Full program out.row = in.row & out.col < in.col 4

58 SMT-based deduction Input Prune partial programs 3 Hypothesis Generation Partial program Partial program SMT-based Deduction Strengthen deduction via PE 2 Type-directed Completion Partial Evaluation Full program out.row = in.row & out.col < in.col out.row < in.row & out.col = in.col 4

59 SMT-based deduction 3 2 5

60 SMT-based deduction 3 2 T3.row = T2.row & T3.col < T2.col & T2.row < T.row & T2.col = T.col 5

61 SMT-based deduction Id Name Age GPA 2 Bob Tom T3 (output): 2X4 T3.row=2,T3.col=4 T3.row = T2.row & T3.col < T2.col & T2.row < T.row & T2.col = T.col 5

62 SMT-based deduction Id Name Age GPA 2 Bob Tom T3 (output): 2X4 T3.row=2,T3.col=4 T3.row = T2.row & T3.col < T2.col & T2.row < T.row & T2.col = T.col 5

63 SMT-based deduction Id Name Age GPA 2 Bob Tom T3 (output): 2X4 2 T3.row=2,T3.col=4 Id Name Age GPA Alice Bob T3.row = T2.row & T3.col < T2.col & T2.row < T.row & T2.col = T.col 3 Tom T (input): 3X4 5

64 SMT-based deduction Id Name Age GPA 2 Bob Tom T3 (output): 2X4 2 T3.row=2,T3.col=4 Id Name Age GPA Alice Bob Tom T3.row = T2.row & T3.col < T2.col & T2.row < T.row & T2.col = T.col T.row=3, T.col=4 T (input): 3X4 5

65 SMT-based deduction Id Name Age GPA 2 Bob Tom T3 (output): 2X4 2 T3.row=2,T3.col=4 Id Name Age GPA Alice Bob Tom T3.row = T2.row & T3.col < T2.col & T2.row < T.row & T2.col = T.col T.row=3, T.col=4 T (input): 3X4 5

66 SMT-based deduction SMT-based deduction sketch > 00 programs 3 Id Name Age GPA 2 Bob Tom T3 (output): 2X4 Id Name Age GPA Alice Bob Tom T (input): 3X4 2 5 T3.row=2,T3.col=4 T3.row = T2.row & T3.col < T2.col & T2.row < T.row & T2.col = T.col T.row=3, T.col=4

67 Partial evaluation Given hole of type T, complete hole with possible inhabitants of T Input Prune partial programs Hypothesis Generation Partial program Partial program SMT-based Deduction Strengthen deduction via PE Type-directed Completion Partial Evaluation Full program 6

68 Partial evaluation Given hole of type T, complete hole with possible inhabitants of T Input Prune partial programs Hypothesis Generation Partial program Partial program SMT-based Deduction Strengthen deduction via PE Type-directed Completion Partial Evaluation Full program 6

69 Partial evaluation 3 id name age 2 Bob 8 3 Tom 2 2 id name age gpa Alice Bob Tom 2 3.0? 7

70 Partial evaluation 3 id name age 2 Bob 8 3 Tom 2 2 id name age gpa Alice Bob Tom age? > 8 7

71 Partial evaluation id name age 2 Bob 8 3 Tom 2 id name age gpa Alice Bob Tom age? > 8 7 T3.row=2, T3.col=3 T3.row = T2.row & T3.col < T2.col &

72 Partial evaluation id name age 2 Bob 8 3 Tom 2 id name age gpa Alice Bob Tom age? > 8 7 T3.row=2, T3.col=3 T3.row = T2.row & T3.col < T2.col &

73 Partial evaluation id name age 2 Bob 8 3 Tom 2 id name age gpa 2 Bob T3.row=2, T3.col=3 T3.row = T2.row & T3.col < T2.col & Partial evaluation to strengthen 7 current constraints

74 Partial evaluation id name age 2 Bob 8 3 Tom 2 id name age gpa 2 Bob T3.row=2, T3.col=3 T3.row = T2.row & T3.col < T2.col & T2.row=, T2.col=4 Partial evaluation to strengthen 7 current constraints

75 Partial evaluation Partial evaluation + SMT-based deduction Prune more programs! id name age gpa 2 Bob id name age 2 Bob 8 3 Tom 2 T3.row=2, T3.col=3 T3.row = T2.row & T3.col < T2.col & T2.row=, T2.col=4 Partial evaluation to strengthen 7 current constraints

76 Experiments 8

77 Experiments RQ: Can Morpheus efficiently automate realworld data wrangling tasks? 8

78 Experiments RQ: Can Morpheus efficiently automate realworld data wrangling tasks? RQ2: How effective are SMT-based deduction and partial evaluation? 8

79 Experiments RQ: Can Morpheus efficiently automate realworld data wrangling tasks? RQ2: How effective are SMT-based deduction and partial evaluation? RQ3: How does Morpheus compare with other synthesis tools? 8

80 Data wrangling tasks How to write a program to generate this table How to reshape table using dplyr 9

81 Data wrangling tasks How to write a program to generate this table How to reshape table using dplyr Collect 80 data wrangling tasks from Stackoverflow 9

82 Data wrangling tasks How to write a program to generate this table How to reshape table using dplyr Collect 80 data wrangling tasks from Stackoverflow Morpheus successfully synthesized the correct implementation of 78 benchmarks 9

83 Data wrangling tasks How to write a program to generate this table How to reshape table using dplyr Collect 80 data wrangling tasks from Stackoverflow Morpheus successfully synthesized the correct implementation of 78 benchmarks Median synthesis time 3.59 seconds 9

84 Deduction & Partial evaluation Cumulative running time of Morpheus 20

85 Deduction & Partial evaluation Cumulative running time of Morpheus 20

86 Deduction & Partial evaluation Deduction and PE are important! Cumulative running time of Morpheus 20

87 Compare with other tool

88 Compare with other tool No existing tool for synthesizing data wrangling tasks in R

89 Compare with other tool No existing tool for synthesizing data wrangling tasks in R Evaluate Morpheus on 28 benchmarks used in evaluating SQLSynthesizer (ASE 3)

90 Compare with other tool No existing tool for synthesizing data wrangling tasks in R Evaluate Morpheus on 28 benchmarks used in evaluating SQLSynthesizer (ASE 3) Each benchmark requires synthesizing SQL queries from input-output examples

91 Compare with other tool 22

92 Compare with other tool Comparison with SQLSynthesizer SQLSynthesizer Morpheus 0.75 Percentage SQL 22

93 Compare with other tool Comparison with SQLSynthesizer SQLSynthesizer Morpheus 96.4% Percentage % 0 SQL 22

94 Compare with other tool Morpheus outperforms state-of-the-art! Comparison with SQLSynthesizer SQLSynthesizer Morpheus 96.4% Percentage % 0 SQL 22

95 Component-based Synthesis 23

96 Component-based Synthesis Morpheus Incomplete FOL 23

97 Component-based Synthesis Morpheus Incomplete FOL SyPet 23

98 Outline Morpheus: automating data wrangling tasks in data science (Completed) SyPet: generating complex programs from nontrivial APIs (Completed) Neo: a CDCL constraint solver for general synthesis problems (Ongoing)

99 A Motivating Example Consider rotating some object using a Java API 25

100 A Motivating Example Consider rotating some object using a Java API Possible to do this using java.awt.geom, but not trivial: 25

101 A Motivating Example Consider rotating some object using a Java API Possible to do this using java.awt.geom, but not trivial: Must know this class 25

102 A Motivating Example Consider rotating some object using a Java API Possible to do this using java.awt.geom, but not trivial: Deconstruct Point object 25

103 A Motivating Example Consider rotating some object using a Java API Possible to do this using java.awt.geom, but not trivial: Call impure method 25

104 A Motivating Example Consider rotating some object using a Java API Possible to do this using java.awt.geom, but not trivial: Finally get the object 25

105 Our Goal 26

106 Our Goal Using an API to achieve a conceptually simple task may be quite hard 26

107 Our Goal Using an API to achieve a conceptually simple task may be quite hard Our goal: Use type- and example-directed program synthesis to help programmers use APIs 26

108 Our Goal Using an API to achieve a conceptually simple task may be quite hard Our goal: Use type- and example-directed program synthesis to help programmers use APIs 26

109 Our Goal Using an API to achieve a conceptually simple task may be quite hard Our goal: Use type- and example-directed program synthesis to help programmers use APIs Want to use this approach to synthesize programs over any API (any set of components) 26

110 Why is this hard? 27

111 Why is this hard? Number of components: Much previous work on component-based synthesis, but typically work for a very small (< 20) number of components 27

112 Why is this hard? Number of components: Much previous work on component-based synthesis, but typically work for a very small (< 20) number of components Generality: Unlike many API completion tools, want to synthesize multi-statement code snippets and allow components with side effects and multiple arguments 27

113 Why is this hard? Number of components: Much previous work on component-based synthesis, but typically work for a very small (< 20) number of components Generality: Unlike many API completion tools, want to synthesize multi-statement code snippets and allow components with side effects and multiple arguments Given type signature T, need smart way to enumerate all well-typed programs of type T over the given components! 27

114 Our Solution 28

115 Our Solution Use Petri net reachability analysis to look for well-typed programs of the desired type 28

116 Our Solution Use Petri net reachability analysis to look for well-typed programs of the desired type Model relationships between components using Petri net 28

117 Our Solution Use Petri net reachability analysis to look for well-typed programs of the desired type Model relationships between components using Petri net Use type signature of desired method to mark initial and target configurations 28

118 Our Solution Use Petri net reachability analysis to look for well-typed programs of the desired type Model relationships between components using Petri net Use type signature of desired method to mark initial and target configurations Perform reachability analysis to find valid sequences of method calls 28

119 Primer on Petri nets 2 T2 P T P2 T3 P3 29

120 Primer on Petri nets 2 T2 P T P2 T3 P3 Petri net is a generalized graph with two kinds of nodes: places and transitions 29

121 Primer on Petri nets 2 T2 P T P2 T3 P3 Petri net is a generalized graph with two kinds of nodes: places and transitions Each place contains zero or more tokens; edges are labeled with a number (of tokens) 29

122 Primer on Petri nets 2 T2 P T P2 T3 P3 Petri net is a generalized graph with two kinds of nodes: places and transitions Each place contains zero or more tokens; edges are labeled with a number (of tokens) A transition T can fire if, for each edge (p,t) with label n, place p contains at least n tokens 29

123 Primer on Petri nets 2 T2 P T P2 T3 P3 Petri net is a generalized graph with two kinds of nodes: places and transitions Each place contains zero or more tokens; edges are labeled with a number (of tokens) A transition T can fire if, for each edge (p,t) with label n, place p contains at least n tokens Firing a transition T consumes (resp. produces) the indicated number of tokens at the source (resp. target) nodes 29

124 Primer on Petri nets 2 T2 P T P2 T3 P3 Petri net is a generalized graph with two kinds of nodes: places and transitions Each place contains zero or more tokens; edges are labeled with a number (of tokens) A transition T can fire if, for each edge (p,t) with label n, place p contains at least n tokens Firing a transition T consumes (resp. produces) the indicated number of tokens at the source (resp. target) nodes 29

125 Reachability Problem in Petri nets 2 T2 P T P2 T3 P3 30

126 Reachability Problem in Petri nets 2 T2 P T P2 T3 P3 Reachability problem: Given a Petri net with initial marking M and a target marking M, is it possible to obtain M by firing a sequencing of transitions? 30

127 Reachability Problem in Petri nets 2 T2 P T P2 T3 P3 Reachability problem: Given a Petri net with initial marking M and a target marking M, is it possible to obtain M by firing a sequencing of transitions? Example: Consider marking M : [P 0,P2 0,P3 ]. 30

128 Reachability Problem in Petri nets 2 T2 P T P2 T3 P3 Reachability problem: Given a Petri net with initial marking M and a target marking M, is it possible to obtain M by firing a sequencing of transitions? Example: Consider marking M : [P 0,P2 0,P3 ]. This marking is reachable, and accepting run is T,T,T2. 30

129 Reachability Problem in Petri nets 2 T2 P T P2 T3 P3 Reachability problem: Given a Petri net with initial marking M and a target marking M, is it possible to obtain M by firing a sequencing of transitions? Example: Consider marking M : [P 0,P2 0,P3 ]. This marking is reachable, and accepting run is T,T,T2. 30

130 Reachability Problem in Petri nets 2 T2 P T P2 T3 P3 Reachability problem: Given a Petri net with initial marking M and a target marking M, is it possible to obtain M by firing a sequencing of transitions? Example: Consider marking M : [P 0,P2 0,P3 ]. This marking is reachable, and accepting run is T,T,T2. 30

131 Reachability Problem in Petri nets 2 T2 P T P2 T3 P3 Reachability problem: Given a Petri net with initial marking M and a target marking M, is it possible to obtain M by firing a sequencing of transitions? Example: Consider marking M : [P 0,P2 0,P3 ]. This marking is reachable, and accepting run is T,T,T2. 30

132 Algorithm Overview 3

133 Algorithm Overview APIs Construct Petri net 3

134 Algorithm Overview APIs Construct Petri net Init/target markings Signature 3

135 Algorithm Overview APIs Construct Petri net Init/target markings Reachability analysis Candidate Sketch Signature 3

136 Algorithm Overview APIs Construct Petri net Init/target markings Reachability analysis Candidate Sketch Sketch Completion Candidate Program Signature 3

137 Algorithm Overview APIs Construct Petri net Init/target markings Reachability analysis Candidate Sketch Sketch Completion Candidate Program Check Candidate Signature 3

138 Algorithm Overview APIs Construct Petri net Init/target markings Reachability analysis Candidate Sketch Sketch Completion Backtrack Candidate Program Check Candidate Signature 3

139 Algorithm Overview APIs Construct Petri net Init/target markings Reachability analysis Candidate Sketch Sketch Completion Backtrack Candidate Program Check Candidate Signature 3

140 Petri net Construction class CPt { CPt(Int x, Int y, Color c); Int getx(); void setcolor(color c);... } Int CPt void Color 32

141 Petri net Construction class CPt { CPt(Int x, Int y, Color c); Int getx(); void setcolor(color c);... } Int 2 CPt void Color CPt(..) 32

142 Petri net Construction class CPt { CPt(Int x, Int y, Color c); Int getx(); void setcolor(color c);... } Int Color 2 getx CPt(..) CPt void 32

143 Petri net Construction class CPt { CPt(Int x, Int y, Color c); Int getx(); void setcolor(color c);... } Int Color 2 getx CPt(..) CPt setcolor void 32

144 Clone Transitions getx Int 2 CPt void Color CPt(..) setcolor 33

145 Clone Transitions Our construction so far views objects as resources every method consumes and produces objects getx Int 2 CPt void Color CPt(..) setcolor 33

146 Clone Transitions Our construction so far views objects as resources every method consumes and produces objects But in conventional languages, we can reuse objects! getx Int 2 CPt void Color CPt(..) setcolor 33

147 Clone Transitions Our construction so far views objects as resources every method consumes and produces objects But in conventional languages, we can reuse objects! Therefore, augment Petri net model with clone transitions getx Int 2 CPt void Color CPt(..) setcolor 33

148 Clone Transitions Our construction so far views objects as resources every method consumes and produces objects But in conventional languages, we can reuse objects! Therefore, augment Petri net model with clone transitions getx Int 2 CPt 2 void Color CPt(..) setcolor 33

149 Clone Transitions Our construction so far views objects as resources every method consumes and produces objects But in conventional languages, we can reuse objects! Therefore, augment Petri net model with clone transitions 2 Int 2 getx CPt 2 void Color CPt(..) setcolor 33

150 Clone Transitions Our construction so far views objects as resources every method consumes and produces objects But in conventional languages, we can reuse objects! Therefore, augment Petri net model with clone transitions getx Int CPt void 2 setcolor CPt(..) Color 33

151 Clone Transitions Our construction so far views objects as resources every method consumes and produces objects But in conventional languages, we can reuse objects! Therefore, augment Petri net model with clone transitions getx Int CPt void 2 setcolor CPt(..) Color 2 33

152 Initial and Target Markings Use signature to determine initial and target markings of Petri net 2 2 Int Color 2 getx CPt(..) CPt 2 void 2 setcolor 34

153 Initial and Target Markings Use signature to determine initial and target markings of Petri net CPt shift (CPt p, Int shiftx, Int shifty) 2 2 Int Color 2 getx CPt(..) CPt 2 void 2 setcolor 34

154 Initial and Target Markings Use signature to determine initial and target markings of Petri net CPt shift (CPt p, Int shiftx, Int shifty) 2 2 Int Color 2 getx CPt(..) CPt 2 void 2 setcolor 34

155 Initial and Target Markings Use signature to determine initial and target markings of Petri net CPt shift (CPt p, Int shiftx, Int shifty) 2 2 Int Color 2 getx CPt(..) CPt 2 void 2 setcolor 34

156 Initial and Target Markings Use signature to determine initial and target markings of Petri net CPt shift (CPt p, Int shiftx, Int shifty) 2 2 Int Color 2 getx CPt(..) CPt 2 void 2 setcolor 34

157 Initial and Target Markings Use signature to determine initial and target markings of Petri net CPt shift (CPt p, Int shiftx, Int shifty) Target marking: 2 2 Int Color 2 getx CPt(..) CPt 2 void 2 setcolor 34

158 Initial and Target Markings Use signature to determine initial and target markings of Petri net CPt shift (CPt p, Int shiftx, Int shifty) Target marking: Cpt = getx Int CPt void 2 setcolor CPt(..) Color 2 34

159 Initial and Target Markings CPt shift (CPt p, Int shiftx, Int shifty) 2 2 Use signature to determine initial and target markings of Petri net Int Color 2 getx CPt(..) CPt Target marking: Cpt = void = * 2 void 2 setcolor 34

160 Initial and Target Markings CPt shift (CPt p, Int shiftx, Int shifty) 2 2 Use signature to determine initial and target markings of Petri net Int Color 2 getx CPt(..) CPt Target marking: Cpt = void = * int = 0 2 void 2 setcolor 34

161 Initial and Target Markings CPt shift (CPt p, Int shiftx, Int shifty) 2 2 Use signature to determine initial and target markings of Petri net Int Color 2 getx CPt(..) CPt Target marking: Cpt = void = * int = 0 color = 0 2 void 2 setcolor 34

162 Initial and Target Markings CPt shift (CPt p, Int shiftx, Int shifty) 2 2 Use signature to determine initial and target markings of Petri net Int Color 2 getx CPt(..) All args must be used! CPt Target marking: Cpt = void = * int = 0 color = 0 2 void 2 setcolor 34

163 Next Step APIs Construct Petri net Init/target markings Reachability analysis Candidate Sketch Sketch Completion Backtrack Candidate Program Check Candidate Signature 35

164 Reachability Analysis 36

165 Reachability Analysis All accepting runs of Petri net correspond to method call sequences with desired type signature! 36

166 Reachability Analysis All accepting runs of Petri net correspond to method call sequences with desired type signature! Need to perform reachability analysis to identify accepting runs of the Petri Net 36

167 Reachability Analysis All accepting runs of Petri net correspond to method call sequences with desired type signature! Need to perform reachability analysis to identify accepting runs of the Petri Net Furthermore, need to do this lazily because there may be many accepting runs 36

168 Reachability Analysis All accepting runs of Petri net correspond to method call sequences with desired type signature! Need to perform reachability analysis to identify accepting runs of the Petri Net Furthermore, need to do this lazily because there may be many accepting runs Our solution reduces reachability analysis to integer linear programming (ILP) solution corresponds to shortest sequence of method calls 36

169 Accepting Run as Program Sketch APIs Construct Petri net Init/target markings Reachability analysis Candidate Sketch Sketch Completion Backtrack Candidate Program Check Candidate Signature 37

170 Accepting Run as Program Sketch APIs Accepting run of Petri net corresponds to program sketch because it does not specify arguments of each method call. Construct Petri net Init/target markings Reachability analysis Candidate Sketch Sketch Completion Backtrack Candidate Program Check Candidate Signature 37

171 Sketch Completion 38

172 Sketch Completion Given a program sketch with holes, need to instantiate each hole with a program variable such that program type checks 38

173 Sketch Completion Given a program sketch with holes, need to instantiate each hole with a program variable such that program type checks Encode this as a boolean satisfiability problem: 38

174 Sketch Completion Given a program sketch with holes, need to instantiate each hole with a program variable such that program type checks Encode this as a boolean satisfiability problem: 38

175 Sketch Completion Given a program sketch with holes, need to instantiate each hole with a program variable such that program type checks each hole filled with one variable Encode this as a boolean satisfiability problem: 38

176 Sketch Completion Given a program sketch with holes, need to instantiate each hole with a program variable such that program type checks each hole filled with one variable Encode this as a boolean satisfiability problem: each variable used at least once 38

177 Evaluation Collected 30 API-usage questions from Stackoverflow involving six different libraries with methods: 39

178 Evaluation Collected 30 API-usage questions from Stackoverflow involving six different libraries with methods: Extracted signature and test case from post if available, otherwise wrote it ourselves 39

179 Evaluation Collected 30 API-usage questions from Stackoverflow involving six different libraries with methods: Extracted signature and test case from post if available, otherwise wrote it ourselves Used SyPet to automatically synthesize the implementation 39

180 Results 40

181 Results Our technique was able to successfully synthesize the correct implementation of all 30 benchmarks 40

182 Results Our technique was able to successfully synthesize the correct implementation of all 30 benchmarks Median synthesis time.57 seconds 40

183 Results Our technique was able to successfully synthesize the correct implementation of all 30 benchmarks Median synthesis time.57 seconds Our synthesis technique is useful to programmers 40

184 Comparison with Other Tools 4

185 Comparison with Other Tools Also compared SyPet with two other synthesis tools, CodeHint and InSynth 4

186 Comparison with Other Tools Also compared SyPet with two other synthesis tools, CodeHint and InSynth 4

187 Comparison with Other Tools Also compared SyPet with two other synthesis tools, CodeHint and InSynth CodeHint (resp. InSynth) synthesizes 3/30 (resp. /30) benchmarks 4

188 Where are we? 42

189 Where are we? State-of-the-art tools (Deepcoder, Morpheus) still can not scale to large programs (< 5 components) 42

190 Where are we? State-of-the-art tools (Deepcoder, Morpheus) still can not scale to large programs (< 5 components) Machine learning (Deepcoder) Deduction (L2, Synquid) 42

191 Where are we? State-of-the-art tools (Deepcoder, Morpheus) still can not scale to large programs (< 5 components) Most of existing tools focus on pruning search space Machine learning (Deepcoder) Deduction (L2, Synquid) 42

192 Where are we? State-of-the-art tools (Deepcoder, Morpheus) still can not scale to large programs (< 5 components) Most of existing tools focus on pruning search space Machine learning (Deepcoder) Deduction (L2, Synquid) How do we learn from mistakes? 42

193 Outline Morpheus: automating data wrangling tasks in data science (Completed) SyPet: generating complex programs from nontrivial APIs (Completed) Neo: a CDCL constraint solver for general synthesis problems (Ongoing)

194 Conflict-driven clause learning Decide Current decision New lemma No conflict Analyze Conflict Deduce Conflict SAT 44 UNSAT

195 Conflict-driven clause learning Decide Deduce New lemma Analyze Conflict Conflict 44

196 Conflict-driven clause learning Specification DSL grammar & semantics Knowledge base Decide All holes filled New lemma Partial program New partial program Analyze Conflict Deduce Conflict Solution No solution 44

197 Conflict-driven clause learning Synthesizing P that satisfies spec Φ = Solving the second-order constraint: 9P. (P ) Specification Decide Partial program New partial program Deduce DSL grammar & semantics Knowledge base All holes filled New lemma Analyze Conflict Conflict Solution No solution 44

198 Conflict-driven clause learning Specification DSL grammar & semantics Knowledge base Synthesizing P that satisfies spec Φ = Solving the second-order constraint: Decide All holes filled 9P. (P ) New lemma Natural integration with machine learning Partial program New partial program Analyze Conflict Deduce Conflict Solution No solution 44

199 A motivating example 45

200 A motivating example [,2,3] [,2] 45

201 A motivating example [,2,3] [,2] map [,2,3] g( ) h( ) 45

202 A motivating example [,2,3] [,2] map [,2,3] g( ) h( ) 45

203 A motivating example [,2,3] [,2] map f( ) [,2,3] g( ) [,2,3]?( ) h( )?( ) 45

204 A motivating example [,2,3] [,2] map f( ) [,2,3] g( ) [,2,3]?( ) h( )?( ) 45

205 A motivating example [,2,3] [,2] map f( ) [,2,3] g( ) [,2,3]?( ) h( )?( ) spec(map) = spec(f) 45

206 A motivating example [,2,3] [,2] map f( ) [,2,3] g( ) [,2,3]?( ) h( )?( ) spec(map) = spec(f) Analyze the root cause of failure and avoid making the same mistake! 45

207 Acknowledgement Isil Dillig Ruben Martins Yuepeng Wang Jacob Van Geffen Osbert Bastani Saswat Anand Alex Aiken Swarat Chaudhuri 46 Tom Reps Xinyu Wang Calvin Lin

208 47

209 47

210 47

211 Thank You!

Program Synthesis using Conflict-Driven Learning

Program Synthesis using Conflict-Driven Learning Program Synthesis using Conflict-Driven Learning Abstract Yu Feng University of Texas at Austin U.S.A yufeng@cs.utexas.edu Osbert Bastani Massachusetts Institute of Technology U.S.A obastani@csail.mit.edu

More information

Component-Based Synthesis for Complex APIs

Component-Based Synthesis for Complex APIs Component-Based Synthesis for Complex APIs Yu Feng University of Texas at Austin, USA yufeng@cs.utexas.edu Ruben Martins University of Texas at Austin, USA rmartins@cs.utexas.edu Yuepeng Wang University

More information

Component-Based Synthesis of Table Consolidation and Transformation Tasks from Examples

Component-Based Synthesis of Table Consolidation and Transformation Tasks from Examples Component-Based Synthesis of Table Consolidation and Transformation Tasks from Examples Yu Feng University of Texas at Austin, USA yufeng@cs.utexas.edu Ruben Martins University of Texas at Austin, USA

More information

Pluggable SAT-Solvers for SMT-Solvers

Pluggable SAT-Solvers for SMT-Solvers Pluggable SAT-Solvers for SMT-Solvers Bas Schaafsma DISI, University of Trento & Fondazione Bruno Kessler May 29, 2013 http://mathsat.fbk.eu/ 1 The SAT/SMT problem Applications of SAT/SMT-Solvers Motivation

More information

Bottom-up Context-Sensitive Pointer Analysis for Java

Bottom-up Context-Sensitive Pointer Analysis for Java Bottom-up Context-Sensitive Pointer Analysis for Java Yu Feng, Xinyu Wang, Isil Dillig and Thomas Dillig UT Austin 1 What is this talk about? Pointer analysis Given a program variable v, what are the heap

More information

EECS 219C: Formal Methods Syntax-Guided Synthesis (selected/adapted slides from FMCAD 13 tutorial by R. Alur) Sanjit A. Seshia EECS, UC Berkeley

EECS 219C: Formal Methods Syntax-Guided Synthesis (selected/adapted slides from FMCAD 13 tutorial by R. Alur) Sanjit A. Seshia EECS, UC Berkeley EECS 219C: Formal Methods Syntax-Guided Synthesis (selected/adapted slides from FMCAD 13 tutorial by R. Alur) Sanjit A. Seshia EECS, UC Berkeley Solving SyGuS Is SyGuS same as solving SMT formulas with

More information

Abstractions and small languages in synthesis CS294: Program Synthesis for Everyone

Abstractions and small languages in synthesis CS294: Program Synthesis for Everyone Abstractions and small languages in synthesis CS294: Program Synthesis for Everyone Ras Bodik Emina Torlak Division of Computer Science University of California, Berkeley Today Today: we describe why high-level

More information

arxiv: v1 [cs.se] 27 Aug 2016

arxiv: v1 [cs.se] 27 Aug 2016 Type-Directed Code Reuse using Integer Linear Programming arxiv:1608.07745v1 [cs.se] 27 Aug 2016 Yuepeng Wang University of Texas at Austin ypwang@cs.utexas.edu Arati Kaushik University of Texas at Austin

More information

Interactively Verifying Absence of Explicit Information Flows in Android Apps

Interactively Verifying Absence of Explicit Information Flows in Android Apps Interactively Verifying Absence of Explicit Information Flows in Android Apps Osbert Bastani, Saswat Anand, and Alex Aiken Stanford University OOPSLA 2015 Problem Google Play Store > 1 million apps on

More information

Formally Certified Satisfiability Solving

Formally Certified Satisfiability Solving SAT/SMT Proof Checking Verifying SAT Solver Code Future Work Computer Science, The University of Iowa, USA April 23, 2012 Seoul National University SAT/SMT Proof Checking Verifying SAT Solver Code Future

More information

Automatically Synthesizing SQL Queries from Input-Output Examples

Automatically Synthesizing SQL Queries from Input-Output Examples Automatically Synthesizing SQL Queries from Input-Output Examples Sai Zhang University of Washington Joint work with: Yuyin Sun Goal: making it easier for non-expert users to write correct SQL queries

More information

COUNTEREXAMPLE-GUIDED MODEL SYNTHESIS

COUNTEREXAMPLE-GUIDED MODEL SYNTHESIS COUNTEREXAMPLE-GUIDED MODEL SYNTHESIS Mathias Preiner, Aina Niemetz and Armin Biere Institute for Formal Models and Verification (FMV) Johannes Kepler University, Linz, Austria http://fmv.jku.at/ TACAS

More information

COUNTEREXAMPLE-GUIDED MODEL SYNTHESIS

COUNTEREXAMPLE-GUIDED MODEL SYNTHESIS COUNTEREXAMPLE-GUIDED MODEL SYNTHESIS Mathias Preiner, Aina Niemetz and Armin Biere Johannes Kepler University Stanford University SMT Workshop July 22-23, 2017 Heidelberg, Germany Introduction Counterexample-Guided

More information

Minimum Satisfying Assignments for SMT. Işıl Dillig, Tom Dillig Ken McMillan Alex Aiken College of William & Mary Microsoft Research Stanford U.

Minimum Satisfying Assignments for SMT. Işıl Dillig, Tom Dillig Ken McMillan Alex Aiken College of William & Mary Microsoft Research Stanford U. Minimum Satisfying Assignments for SMT Işıl Dillig, Tom Dillig Ken McMillan Alex Aiken College of William & Mary Microsoft Research Stanford U. 1 / 20 Satisfiability Modulo Theories (SMT) Today, SMT solvers

More information

LEARNING TO INSTANTIATE QUANTIFIERS

LEARNING TO INSTANTIATE QUANTIFIERS LEARNING TO INSTANTIATE QUANTIFIERS Armin Biere 1 joint work with Mathias Preiner 1,2, Aina Niemetz 1,2 TACAS 17, SMT 17, PhD Thesis Mathias Preiner in 2017 1 Johannes Kepler University Linz 2 Stanford

More information

Small Formulas for Large Programs: On-line Constraint Simplification In Scalable Static Analysis

Small Formulas for Large Programs: On-line Constraint Simplification In Scalable Static Analysis Small Formulas for Large Programs: On-line Constraint Simplification In Scalable Static Analysis Isil Dillig, Thomas Dillig, Alex Aiken Stanford University Scalability and Formula Size Many program analysis

More information

Learning Programs from Noisy Data

Learning Programs from Noisy Data Learning Programs from Noisy Data Veselin Raychev Pavol Bielik Martin Vechev Andreas Krause ETH Zurich Why learn programs from examples? Input/output examples often easier to provide examples than specification

More information

Motivation. CS389L: Automated Logical Reasoning. Lecture 17: SMT Solvers and the DPPL(T ) Framework. SMT solvers. The Basic Idea.

Motivation. CS389L: Automated Logical Reasoning. Lecture 17: SMT Solvers and the DPPL(T ) Framework. SMT solvers. The Basic Idea. Motivation Lecture 17: SMT rs and the DPPL(T ) Framework şıl Dillig n previous lectures, we looked at decision procedures for conjunctive formulas in various first-order theories This lecture: How to handle

More information

CAV Verification Mentoring Workshop 2017 SMT Solving

CAV Verification Mentoring Workshop 2017 SMT Solving CAV Verification Mentoring Workshop 2017 SMT Solving Alberto Griggio Fondazione Bruno Kessler Trento, Italy The SMT problem Satisfiability Modulo Theories Given a (quantifier-free) FOL formula and a (decidable)

More information

VS 3 : SMT Solvers for Program Verification

VS 3 : SMT Solvers for Program Verification VS 3 : SMT Solvers for Program Verification Saurabh Srivastava 1,, Sumit Gulwani 2, and Jeffrey S. Foster 1 1 University of Maryland, College Park, {saurabhs,jfoster}@cs.umd.edu 2 Microsoft Research, Redmond,

More information

Automated Migration of Hierarchical Data to Relational Tables using Programming-by-Example

Automated Migration of Hierarchical Data to Relational Tables using Programming-by-Example Automated Migration of Hierarchical Data to Relational Tables using Programming-by-Example Navid Yaghmazadeh University of Texas at Austin nyaghma@cs.utexas.edu Xinyu Wang University of Texas at Austin

More information

Colored Petri Net Evaluation Tool. Stephen Rojcewicz CS 2310

Colored Petri Net Evaluation Tool. Stephen Rojcewicz CS 2310 Colored Petri Net Evaluation Tool Stephen Rojcewicz CS 2310 Motivating Example (Colored Petri Nets) Consider a gesture-driven application interface. The system must detect three kinds of gestures and respond

More information

COUNTEREXAMPLE-GUIDED MODEL SYNTHESIS

COUNTEREXAMPLE-GUIDED MODEL SYNTHESIS COUNTEREXAMPLE-GUIDED MODEL SYNTHESIS Mathias Preiner?y, Aina Niemetz?y and Armin Biere?? Johannes Kepler University y Stanford University SMT Workshop July 22-23, 2017 Heidelberg, Germany Introduction

More information

Bringing SQL to the Masses with Program Synthesis. Chenglong Wang, Alvin Cheung, Ras Bodik University of Washington

Bringing SQL to the Masses with Program Synthesis. Chenglong Wang, Alvin Cheung, Ras Bodik University of Washington Bringing SQL to the Masses with Program Synthesis Chenglong Wang, Alvin Cheung, Ras Bodik University of Washington 1 End-User SQL Select rows with maximum value for each group. Find rows containing duplicate

More information

Automated Migration of Hierarchical Data to Relational Tables using Programming-by-Example

Automated Migration of Hierarchical Data to Relational Tables using Programming-by-Example Automated Migration of Hierarchical Data to Relational Tables using Programming-by-Example Navid Yaghmazadeh University of Texas at Austin nyaghma@cs.utexas.edu Xinyu Wang University of Texas at Austin

More information

Programming by Examples: Logical Reasoning meets Machine Learning

Programming by Examples: Logical Reasoning meets Machine Learning Programming by Examples: Logical Reasoning meets Machine Learning Sumit Gulwani Microsoft NAMPI Workshop July 2018 Joint work with many collaborators Example-based help-forum interaction 300_w30_aniSh_c1_b

More information

Automatic Software Verification

Automatic Software Verification Automatic Software Verification Instructor: Mooly Sagiv TA: Oded Padon Slides from Eran Yahav and the Noun Project, Wikipedia Course Requirements Summarize one lecture 10% one lecture notes 45% homework

More information

Programming by Examples: PL meets ML

Programming by Examples: PL meets ML Programming by Examples: PL meets ML Summit on Machine Learning meets Formal Methods Sumit Gulwani Microsoft July 2018 Joint work with many collaborators Example-based help-forum interaction 300_w30_aniSh_c1_b

More information

arxiv: v2 [cs.pl] 3 Apr 2018

arxiv: v2 [cs.pl] 3 Apr 2018 1 Scheduling Constraint Based Abstraction Refinement for Multi-Threaded Program Verification arxiv:1708.08323v2 [cs.pl] 3 Apr 2018 LIANGZE YIN, School of Computer, National University of Defense Technology,

More information

Semantics as a Foreign Language. Gabriel Stanovsky and Ido Dagan EMNLP 2018

Semantics as a Foreign Language. Gabriel Stanovsky and Ido Dagan EMNLP 2018 Semantics as a Foreign Language Gabriel Stanovsky and Ido Dagan EMNLP 2018 Semantic Dependency Parsing (SDP) A collection of three semantic formalisms (Oepen et al., 2014;2015) Semantic Dependency Parsing

More information

Combining the Logical and the Probabilistic in Program Analysis. Xin Zhang Xujie Si Mayur Naik University of Pennsylvania

Combining the Logical and the Probabilistic in Program Analysis. Xin Zhang Xujie Si Mayur Naik University of Pennsylvania Combining the Logical and the Probabilistic in Program Analysis Xin Zhang Xujie Si Mayur Naik University of Pennsylvania What is Program Analysis? int f(int i) {... } Program Analysis x may be null!...

More information

Instructions to use PIPE+

Instructions to use PIPE+ Instructions to use PIPE+ PIPE+ is implemented under the environment of Eclipse. User can import it into eclipse to view source code as well as run the program. The user can choose RunGUI to run the tool.

More information

arxiv: v2 [cs.pl] 24 Nov 2018

arxiv: v2 [cs.pl] 24 Nov 2018 73 FrAngel: Component-Based Synthesis with Control Structures arxiv:1811.05175v2 [cs.pl] 24 Nov 2018 KENSEN SHI, Stanford University, USA JACOB STEINHARDT, Stanford University, USA PERCY LIANG, Stanford

More information

Programming by Examples: Applications, Algorithms, and Ambiguity Resolution

Programming by Examples: Applications, Algorithms, and Ambiguity Resolution Programming by Examples: Applications, Algorithms, and Ambiguity Resolution Sumit Gulwani Microsoft Corporation, Redmond, WA, USA sumitg@microsoft.com Abstract. 99% of computer end users do not know programming,

More information

Programmable Networks with Synthesis

Programmable Networks with Synthesis Programmable Networks with Synthesis Ahmed ElHassany Petar Tsankov Laurent Vanbever Martin Vechev Network Misconfigurations are Common What Example Makes Network Configuration Hard? Low-level, local router

More information

Satisfiability (SAT) Applications. Extensions/Related Problems. An Aside: Example Proof by Machine. Annual Competitions 12/3/2008

Satisfiability (SAT) Applications. Extensions/Related Problems. An Aside: Example Proof by Machine. Annual Competitions 12/3/2008 15 53:Algorithms in the Real World Satisfiability Solvers (Lectures 1 & 2) 1 Satisfiability (SAT) The original NP Complete Problem. Input: Variables V = {x 1, x 2,, x n }, Boolean Formula Φ (typically

More information

Structure of Abstract Syntax trees for Colored Nets in PNML

Structure of Abstract Syntax trees for Colored Nets in PNML Structure of Abstract Syntax trees for Colored Nets in PNML F. Kordon & L. Petrucci Fabrice.Kordon@lip6.fr Laure.Petrucci@lipn.univ-paris13.fr version 0.2 (draft) June 26, 2004 Abstract Formalising the

More information

MajorSat: A SAT Solver to Majority Logic

MajorSat: A SAT Solver to Majority Logic MajorSat: A SAT Solver to Majority Logic Speaker : Ching-Yi Huang Authors: Yu-Min Chou, Yung-Chih Chen *, Chun-Yao Wang, Ching-Yi Huang National Tsing Hua University, Taiwan * Yuan Ze University, Taiwan

More information

Synthesis of Data Completion Scripts using Finite Tree Automata

Synthesis of Data Completion Scripts using Finite Tree Automata Synthesis of Data Completion Scripts using Finite Tree Automata XINYU WANG, University of Texas at Austin, USA ISIL DILLIG, University of Texas at Austin, USA RISHABH SINGH, Microsoft Research, USA In

More information

Hybrid Constraint Solvers

Hybrid Constraint Solvers Hybrid Constraint Solvers - An overview Why Hybrid Solvers CP and SAT: Lazy Clause Generation CP and LP: Reification of Linear Constraints Conclusions 9 November 2011 Pedro Barahona - EPCL - Hybrid Solvers

More information

CS 343: Artificial Intelligence

CS 343: Artificial Intelligence CS 343: Artificial Intelligence Bayes Nets: Independence Prof. Scott Niekum The University of Texas at Austin [These slides based on those of Dan Klein and Pieter Abbeel for CS188 Intro to AI at UC Berkeley.

More information

Sciduction: Combining Induction, Deduction and Structure for Verification and Synthesis

Sciduction: Combining Induction, Deduction and Structure for Verification and Synthesis Sciduction: Combining Induction, Deduction and Structure for Verification and Synthesis (abridged version of DAC slides) Sanjit A. Seshia Associate Professor EECS Department UC Berkeley Design Automation

More information

Range Restriction for General Formulas

Range Restriction for General Formulas Range Restriction for General Formulas 1 Range Restriction for General Formulas Stefan Brass Martin-Luther-Universität Halle-Wittenberg Germany Range Restriction for General Formulas 2 Motivation Deductive

More information

CSC410 Program Synthesis AZADEH FARZAN FALL 2017

CSC410 Program Synthesis AZADEH FARZAN FALL 2017 CSC410 Program Synthesis AZADEH FARZAN FALL 2017 The idea Produce small code fragments that satisfy the given specification As a programming aid: It helps you write programs. The insight is yours, the

More information

versat: A Verified Modern SAT Solver

versat: A Verified Modern SAT Solver Computer Science, The University of Iowa, USA Satisfiability Problem (SAT) Is there a model for the given propositional formula? Model: assignments to the variables that makes the formula true. SAT if

More information

Multi-event IDS Categories. Introduction to Misuse Intrusion Detection Systems (IDS) Formal Specification of Intrusion Signatures and Detection Rules

Multi-event IDS Categories. Introduction to Misuse Intrusion Detection Systems (IDS) Formal Specification of Intrusion Signatures and Detection Rules Formal Specification of Intrusion Signatures and Detection Rules By Jean-Philippe Pouzol and Mireille Ducassé 15 th IEEE Computer Security Foundations Workshop 2002 Presented by Brian Kellogg CSE914: Formal

More information

Semantics-Based Program Verifiers for All Languages

Semantics-Based Program Verifiers for All Languages Language-independent Semantics-Based Program Verifiers for All Languages Andrei Stefanescu Daejun Park Shijiao Yuwen Yilong Li Grigore Rosu Nov 2, 2016 @ OOPSLA 16 Problems with state-of-the-art verifiers

More information

Array Basics: Outline. Creating and Accessing Arrays. Creating and Accessing Arrays. Arrays (Savitch, Chapter 7)

Array Basics: Outline. Creating and Accessing Arrays. Creating and Accessing Arrays. Arrays (Savitch, Chapter 7) Array Basics: Outline Arrays (Savitch, Chapter 7) TOPICS Array Basics Arrays in Classes and Methods Programming with Arrays Searching and Sorting Arrays Multi-Dimensional Arrays Static Variables and Constants

More information

New Encodings of Pseudo-Boolean Constraints into CNF

New Encodings of Pseudo-Boolean Constraints into CNF New Encodings of Pseudo-Boolean Constraints into CNF Olivier Bailleux, Yacine Boufkhad, Olivier Roussel olivier.bailleux@u-bourgogne.fr boufkhad@liafa.jussieu.fr roussel@cril.univ-artois.fr New Encodings

More information

Coverability Graph and Fairness

Coverability Graph and Fairness Coverability Graph and Fairness prof.dr.ir. Wil van der Aalst www.vdaalst.com Recall reachability analysis or1 x r1 rg1 g1 go1 o1 r2 rg2 g2 go2 o2 or2 Petri net standard properties Boundedness Terminating

More information

Program Synthesis for Forth Forth Day 2012

Program Synthesis for Forth Forth Day 2012 Program Synthesis for Forth Forth Day 2012 Computer Science UC Berkeley Ras Bodik Mangpo Phitchaya Phothilimthana Tikhon Jelvis Rohin Shah Synthesis with sketches Extend your language with two constructs

More information

Optimizing Synthesis with Metasketches

Optimizing Synthesis with Metasketches Optimizing Synthesis Artifact with Metasketches * POPL * Consistent * Complete * Well Documented * Easy to Reuse * Evaluated * AEC * James Bornholt Emina Torlak Dan Grossman Luis Ceze University of Washington

More information

assertion-driven analyses from compile-time checking to runtime error recovery

assertion-driven analyses from compile-time checking to runtime error recovery assertion-driven analyses from compile-time checking to runtime error recovery sarfraz khurshid the university of texas at austin state of the art in software testing and analysis day, 2008 rutgers university

More information

CS-E3200 Discrete Models and Search

CS-E3200 Discrete Models and Search Shahab Tasharrofi Department of Information and Computer Science, Aalto University Lecture 7: Complete and local search methods for SAT Outline Algorithms for solving Boolean satisfiability problems Complete

More information

SAT Based Efficient Directed Test Generation Techniques

SAT Based Efficient Directed Test Generation Techniques SAT Based Efficient Directed Test Generation Techniques Presented by Mingsong Chen Software Engineering Institute East China Normal University May 5, 2011 Outline Introduction Model Checking Based Test

More information

Course Overview CS294: Program Synthesis for Everyone

Course Overview CS294: Program Synthesis for Everyone Course Overview CS294: Program Synthesis for Everyone Ras Bodik Emina Torlak Division of Computer Science University of California, Berkeley The name of the course this CS294 topics course has been listed

More information

xtc Robert Grimm Making C Safely Extensible New York University

xtc Robert Grimm Making C Safely Extensible New York University xtc Making C Safely Extensible Robert Grimm New York University The Problem Complexity of modern systems is staggering Increasingly, a seamless, global computing environment System builders continue to

More information

Efficient SAT-based Boolean Matching for FPGA Technology Mapping

Efficient SAT-based Boolean Matching for FPGA Technology Mapping Efficient SAT-based Boolean Matching for FPGA Technology Mapping Sean Safarpour, Andreas Veneris Department of Electrical and Computer Engineering University of Toronto Toronto, ON, Canada {sean, veneris}@eecg.toronto.edu

More information

Constraint Solving. Systems and Internet Infrastructure Security

Constraint Solving. Systems and Internet Infrastructure Security Systems and Internet Infrastructure Security Network and Security Research Center Department of Computer Science and Engineering Pennsylvania State University, University Park PA Constraint Solving Systems

More information

Xuandong Li. BACH: Path-oriented Reachability Checker of Linear Hybrid Automata

Xuandong Li. BACH: Path-oriented Reachability Checker of Linear Hybrid Automata BACH: Path-oriented Reachability Checker of Linear Hybrid Automata Xuandong Li Department of Computer Science and Technology, Nanjing University, P.R.China Outline Preliminary Knowledge Path-oriented Reachability

More information

Scalable Synthesis with Symbolic Syntax Graphs

Scalable Synthesis with Symbolic Syntax Graphs Scalable Synthesis with Symbolic Syntax Graphs Rohin Shah, Sumith Kulal, Ras Bodik UC Berkeley, IIT Bombay and UW 18 July 2018, Oxford UK [Solar-Lezama et al. ASPLOS06] Combinatorial Sketching for Finite

More information

DPLL(Γ+T): a new style of reasoning for program checking

DPLL(Γ+T): a new style of reasoning for program checking DPLL(Γ+T ): a new style of reasoning for program checking Dipartimento di Informatica Università degli Studi di Verona Verona, Italy June, 2011 Motivation: reasoning for program checking Program checking

More information

18733: Applied Cryptography S17. Mini Project. due April 19, 2017, 11:59pm EST. φ(x) := (x > 5) (x < 10)

18733: Applied Cryptography S17. Mini Project. due April 19, 2017, 11:59pm EST. φ(x) := (x > 5) (x < 10) 18733: Applied Cryptography S17 Mini Project due April 19, 2017, 11:59pm EST 1 SMT Solvers 1.1 Satisfiability Modulo Theory SMT (Satisfiability Modulo Theories) is a decision problem for logical formulas

More information

COMP 250 Winter 2011 Reading: Java background January 5, 2011

COMP 250 Winter 2011 Reading: Java background January 5, 2011 Almost all of you have taken COMP 202 or equivalent, so I am assuming that you are familiar with the basic techniques and definitions of Java covered in that course. Those of you who have not taken a COMP

More information

Object-oriented Compiler Construction

Object-oriented Compiler Construction 1 Object-oriented Compiler Construction Extended Abstract Axel-Tobias Schreiner, Bernd Kühl University of Osnabrück, Germany {axel,bekuehl}@uos.de, http://www.inf.uos.de/talks/hc2 A compiler takes a program

More information

SAT/SMT Solvers and Applications

SAT/SMT Solvers and Applications SAT/SMT Solvers and Applications University of Waterloo Winter 2013 Today s Lecture Lessons learnt so far Implementation-related attacks (control-hazard, malware,...) Program analysis techniques can detect

More information

PLDI 2016 Tutorial Automata-Based String Analysis

PLDI 2016 Tutorial Automata-Based String Analysis PLDI 2016 Tutorial Automata-Based String Analysis Tevfik Bultan, Abdulbaki Aydin, Lucas Bang Verification Laboratory http://vlab.cs.ucsb.edu Department of Computer Science Common Usages of Strings } Input

More information

The Design Recipe Fall 2017

The Design Recipe Fall 2017 CS17 Integrated Introduction to Computer Science Hughes The Design Recipe Fall 2017 Contents 1 Design Recipe Steps 1 2 An OCaml Example 6 1 Design Recipe Steps This PDF outlines the steps to writing the

More information

Synthesis of Domain Specific Encoders for Bit- Vector Solvers

Synthesis of Domain Specific Encoders for Bit- Vector Solvers Synthesis of Domain Specific Encoders for Bit- Vector Solvers Jeevana Priya Inala! with Rohit Singh, Armando Solar-Lezama To appear at SAT 16 High-level constraint to CNF clauses SMT solver High-level

More information

Autograder RISHABH SINGH, SUMIT GULWANI, ARMANDO SOLAR-LEZAMA

Autograder RISHABH SINGH, SUMIT GULWANI, ARMANDO SOLAR-LEZAMA Autograder RISHABH SINGH, SUMIT GULWANI, ARMANDO SOLAR-LEZAMA Test-cases based feedback Hard to relate failing inputs to errors Manual feedback by TAs Time consuming and error prone Feedback on Programming

More information

Chapter 4 Defining Classes I

Chapter 4 Defining Classes I Chapter 4 Defining Classes I This chapter introduces the idea that students can create their own classes and therefore their own objects. Introduced is the idea of methods and instance variables as the

More information

Complex Combinational circuits in Bluespec

Complex Combinational circuits in Bluespec Complex Combinational circuits in Bluespec Arvind Computer Science & Artificial Intelligence Lab M.I.T. L05-1 2-bit Ripple-Carry Adder cascading full adders x[1] y[1] x[0] y[0] c[2] fa c[1] fa 0 Use fa

More information

Nina Narodytska Samsung Research/CMU

Nina Narodytska Samsung Research/CMU Recent trends in MaxSAT solving Nina Narodytska Samsung Research/CMU Outline 1. Boolean satisfiability 2. Boolean (un)satisfiability (MaxSAT) 3. Counting (un)satisfiability 2 Outline Boolean satisfiability

More information

Pooya Saadatpanah, Michalis Famelis, Jan Gorzny, Nathan Robinson, Marsha Chechik, Rick Salay. September 30th, University of Toronto.

Pooya Saadatpanah, Michalis Famelis, Jan Gorzny, Nathan Robinson, Marsha Chechik, Rick Salay. September 30th, University of Toronto. Comparing the Pooya Michalis Jan Nathan Marsha Chechik, Rick Salay University of Toronto September 30th, 2012 MoDeVVa 12 1 / 32 in software modeling : pervasive in MDE Models with uncertainty: Represent

More information

Ufo: A Framework for Abstraction- and Interpolation-Based Software Verification

Ufo: A Framework for Abstraction- and Interpolation-Based Software Verification Ufo: A Framework for Abstraction- and Interpolation-Based Software Verification Aws Albarghouthi 1, Yi Li 1, Arie Gurfinkel 2, and Marsha Chechik 1 1 Department of Computer Science, University of Toronto,

More information

On Computing Minimum Size Prime Implicants

On Computing Minimum Size Prime Implicants On Computing Minimum Size Prime Implicants João P. Marques Silva Cadence European Laboratories / IST-INESC Lisbon, Portugal jpms@inesc.pt Abstract In this paper we describe a new model and algorithm for

More information

Active Learning of Points-To Specifications

Active Learning of Points-To Specifications arxiv:1711.03239v2 [cs.pl] 17 Nov 2017 Abstract Osbert Bastani Stanford University USA obastani@cs.stanford.edu Alex Aiken Stanford University USA aiken@cs.stanford.edu When analyzing programs, large libraries

More information

SMT-Style Program Analysis with Value-based Refinements

SMT-Style Program Analysis with Value-based Refinements SMT-Style Program Analysis with Value-based Refinements Vijay D Silva Leopold Haller Daniel Kröning NSV-3 July 15, 2010 Outline Imprecision and Refinement in Abstract Interpretation SAT Style Abstract

More information

CS 305j Midterm 2 Fall 2006

CS 305j Midterm 2 Fall 2006 Exam Number: Points off 1 2 3 4 5 6 Total off Net Score CS 305j Midterm 2 Fall 2006 Your Name Your UTEID Circle you TA's name: Brad Jacob Instructions: 1. Please turn off your cell phones 2. There are

More information

Synthesizing Pattern Programs from Examples

Synthesizing Pattern Programs from Examples Synthesizing Pattern Programs from Examples Sunbeom So Korea University sunbeom so@korea.ac.kr Hakjoo Oh Korea University hakjoo oh@korea.ac.kr Abstract We describe a programming-by-example system that

More information

Data Structure Synthesis. Calvin Loncaric Emina Torlak Michael D. Ernst

Data Structure Synthesis. Calvin Loncaric Emina Torlak Michael D. Ernst Data Structure Synthesis Calvin Loncaric Emina Torlak Michael D. Ernst Data structures are everywhere 2 Data structures are everywhere 2 Data structures are everywhere 2 Data structures are everywhere

More information

Synthesizing Data Structure Transformations from Input-Output Examples

Synthesizing Data Structure Transformations from Input-Output Examples Synthesizing Data Structure Transformations from Input-Output Examples John K. Feser Rice University, USA feser@rice.edu Swarat Chaudhuri Rice University, USA swarat@rice.edu Isil Dillig UT Austin, USA

More information

Seminar decision procedures: Certification of SAT and unsat proofs

Seminar decision procedures: Certification of SAT and unsat proofs Seminar decision procedures: Certification of SAT and unsat proofs Wolfgang Nicka Technische Universität München June 14, 2016 Boolean satisfiability problem Term The boolean satisfiability problem (SAT)

More information

Overview. CS389L: Automated Logical Reasoning. Lecture 6: First Order Logic Syntax and Semantics. Constants in First-Order Logic.

Overview. CS389L: Automated Logical Reasoning. Lecture 6: First Order Logic Syntax and Semantics. Constants in First-Order Logic. Overview CS389L: Automated Logical Reasoning Lecture 6: First Order Logic Syntax and Semantics Işıl Dillig So far: Automated reasoning in propositional logic. Propositional logic is simple and easy to

More information

variables hold values (e.g., int, double, char) Primitive and Object Variables Primitive Types Primitive Types Summer 2010 Margaret Reid-Miller

variables hold values (e.g., int, double, char) Primitive and Object Variables Primitive Types Primitive Types Summer 2010 Margaret Reid-Miller Primitive and Object Variables 15-110 Summer 2010 Margaret Reid-Miller variables hold values (e.g., int, double, char) Summer 2009 15-100 (Reid-Miller) 2 Variables of primitive types a storage location

More information

HySAT. what you can use it for how it works example from application domain final remarks. Christian Herde /12

HySAT. what you can use it for how it works example from application domain final remarks. Christian Herde /12 CP2007: Presentation of recent CP solvers HySAT what you can use it for how it works example from application domain final remarks Christian Herde 25.09.2007 /2 What you can use it for Satisfiability checker

More information

Program Synthesis. SWE 795, Spring 2017 Software Engineering Environments

Program Synthesis. SWE 795, Spring 2017 Software Engineering Environments Program Synthesis SWE 795, Spring 2017 Software Engineering Environments Today HW3 is due next week in class! Part 1 (Lecture)(~50 mins) Break! Part 2 (Discussion)(~60 mins) Discussion of readings Part

More information

CS 387: GAME AI TACTIC AND STRATEGY

CS 387: GAME AI TACTIC AND STRATEGY CS 387: GAME AI TACTIC AND STRATEGY 5/16/2017 Instructor: Santiago Ontañón so367@drexel.edu Class website: https://www.cs.drexel.edu/~santi/teaching/2017/cs387/intro.htm Outline Projects 1/2 Tactic and

More information

Reduction of nite linear CSPs to SAT using dierent encod

Reduction of nite linear CSPs to SAT using dierent encod Reduction of nite linear CSPs to SAT using dierent encodings Mirko Stojadinovi mirkos@matf.bg.ac.rs Department of Computer Science Faculty of Mathematics University of Belgrade Fifth Workshop on Formal

More information

Syntax-Guided Program Synthesis. Rajeev Alur. University of Pennsylvania

Syntax-Guided Program Synthesis. Rajeev Alur. University of Pennsylvania Syntax-Guided Program Synthesis Rajeev Alur University of Pennsylvania 1 Goal: Programming computers easier than communicating with people Can programming be liberated, period. David Harel, IEEE Computer,

More information

Syntax-Guided Program Synthesis. Rajeev Alur

Syntax-Guided Program Synthesis. Rajeev Alur Syntax-Guided Program Synthesis Rajeev Alur University of Pennsylvania 1 Goal: Programming computers easier than communicating with people Can programming be liberated, period. David Harel, IEEE Computer,

More information

CSE 403: Software Engineering, Fall courses.cs.washington.edu/courses/cse403/16au/ Static Analysis. Emina Torlak

CSE 403: Software Engineering, Fall courses.cs.washington.edu/courses/cse403/16au/ Static Analysis. Emina Torlak CSE 403: Software Engineering, Fall 2016 courses.cs.washington.edu/courses/cse403/16au/ Static Analysis Emina Torlak emina@cs.washington.edu Outline What is static analysis? How does it work? Free and

More information

Formal Support for QVT-Relations with Coloured Petri Nets

Formal Support for QVT-Relations with Coloured Petri Nets Formal Support for QVT-Relations with Coloured Petri Nets Juan de Lara Univ. Autónoma de Madrid (Spain) MODELS 2009 Denver, Colorado, USA Esther Guerra 1 Univ. Carlos III de Madrid (Spain) Motivation Model-to-Model

More information

Topics in Software Testing

Topics in Software Testing Dependable Software Systems Topics in Software Testing Material drawn from [Beizer, Sommerville] Software Testing Software testing is a critical element of software quality assurance and represents the

More information

Copyright by Navid Yaghmazadeh 2017

Copyright by Navid Yaghmazadeh 2017 Copyright by Navid Yaghmazadeh 2017 The Dissertation Committee for Navid Yaghmazadeh certifies that this is the approved version of the following dissertation: Automated Synthesis of Data Extraction and

More information

Frequently Asked Questions

Frequently Asked Questions Frequently Asked Questions This PowerTools FAQ answers many frequently asked questions regarding the functionality of the various parts of the PowerTools suite. The questions are organized in the following

More information

NOTES ON OBJECT-ORIENTED MODELING AND DESIGN

NOTES ON OBJECT-ORIENTED MODELING AND DESIGN NOTES ON OBJECT-ORIENTED MODELING AND DESIGN Stephen W. Clyde Brigham Young University Provo, UT 86402 Abstract: A review of the Object Modeling Technique (OMT) is presented. OMT is an object-oriented

More information

Program Synthesis for Hierarchical Specifications

Program Synthesis for Hierarchical Specifications Program Synthesis for Hierarchical Specifications Thibaud Hottelier Ras Bodik Electrical Engineering and Computer Sciences University of California at Berkeley Technical Report No. UCB/EECS-2014-139 http://www.eecs.berkeley.edu/pubs/techrpts/2014/eecs-2014-139.html

More information

Chapter 9: Constraint Logic Programming

Chapter 9: Constraint Logic Programming 9. Constraint Logic Programming 9-1 Deductive Databases and Logic Programming (Winter 2007/2008) Chapter 9: Constraint Logic Programming Introduction, Examples Basic Query Evaluation Finite Domain Constraint

More information

Yices 1.0: An Efficient SMT Solver

Yices 1.0: An Efficient SMT Solver Yices 1.0: An Efficient SMT Solver AFM 06 Tutorial Leonardo de Moura (joint work with Bruno Dutertre) {demoura, bruno}@csl.sri.com. Computer Science Laboratory SRI International Menlo Park, CA Yices: An

More information