Program Synthesis for Data Manipulation. Gustavo Soares Federal University of Campina Grande

Size: px

Start display at page:

Download "Program Synthesis for Data Manipulation. Gustavo Soares Federal University of Campina Grande"

Piers Baker
5 years ago
Views:

1 Program Synthesis for Data Manipulation Gustavo Soares Federal University of Campina Grande 1

2 2

3 3

4 4

5 ... Customer Pull Up Method Customer... createbill() Regular Customer... createbill() Preferred Customer... createbill() Regular Customer Preferred Customer 5

6 Manually performing these transformations are error prone and time consuming 6

7 Program Synthesis as a key for developing better data manipulation tools Automated Testing Refactoring Engines FlashProg: UI Framework for PBE 7

8 Automated Testing Refactoring Engines 8

9 Pull Up Method... Customer Refactoring engines Customer... createbill() Regular Customer... createbill() Preferred Customer... createbill() Regular Customer Preferred Customer Condition The createbill method is not declared in class Customer... 9

10 It s difficult to implement refactorings! Implemented conditions Overly weak conditions allow unsafe transformations 10

11 A int k() : 10 [SPLASH 10] JRRT A int k() : 10 int m() : this.k() B int k() : 20 int m() : super.k() int test() : m() Pull up method B int k() : 20 int test() : m()

12 Developers have invested on testing... 12

13 Expected Output Input program Developers have invested on testing... 13

14 2 classes 3 methods 2 fields Identifying bugs due to missing conditions... Failures/bugs Bug Report [Soares et al., IEEE TSE 13] 1 4

15 How do we generate programs? 2 classes 3 methods 2 fields Failures/bugs Bug Report 1 5

16 2 Class, 2 Method, 2 Field One subclass declaring a field pred pullupfield[] { some c1,c2 : Class, f : Field c1 in c2.extends and f in c2.fields } 1 6

17 2 Class, 2 Method, 2 Field One subclass declaring a field Java Meta model Alloy Solutions Type extends 0..1 Alloy Analyzer Class Field Method 1 7

18 2 Class, 2 Method, 2 Field One subclass declaring a field Alloy Solutions Java programs Alloy to Java public public Class0 { public public class intclass Class0 method1(){ Class0 { { public public int return int method1(){ 10; method1(){ return } return 10; 10; } } } } class } class Class1 Class1 extends extends Class0 Class0 { { public class public int Class1 int field; extends Class0 { field; public public int int method0(){ field; method0(){ return public return 20; int method0(){ 20; } return 20; } } } } } 1 8

19 How do we detect behavioral changes? 2 classes 3 methods 2 fields Failures/bugs Bug Report 1 9

20 A int k() : 10 A.k(), B.k(), B.test() A int k() : 10 int m() : this.k() B int k() : 20 int m() : super.k() int test() : m() 0/100 98/100 B int k() : 20 int test() : m() Behavioral change void test() { B var = new B(); int var1 = var.test(); AssertTrue(var1 == 10); } 2 0 [Soares et al., IEEE Software 10]

21 How do we classify failures into distinct faults? 2 classes 3 methods 2 fields Failures/bugs Bug Report 2 1

22 Evaluation 2 2

23 Can our approach identify bugs in real Java refactoring engines? JRRT bugs

24 Can our approach identify bugs in real Java refactoring engines? JRRT JRRT bugs

25 FlashProg 25

26 FlashProg team Generous producers FlashProg UI Mikael Mayer Rico Malvar Ben Zorn Overhead director FlashMeta FlashExtract - FlashFill Sumit Gulwani Alex Polozov Maxim Grechkin Rishabh Singh Vu le 26

27 FlashProg Framework for Data Manipulation 27

28 User Study 28

29 Research questions Do Program Navigation and Conversational Clarification contribute to correctness? Which of Program Navigation and Conversational Clarification is perceived more useful for data extraction? 29

30 Metrics We collected data from user interaction Number of Errors Time Survey Perceived usefulness of each interaction model Confidence about the extraction of each file 30

31 User study design 3 tasks 3 environments Basic interface (BI) BI + Program navigation (BI + PN) BI + Conversational clarification (BI + CC) 3 configurations Video tutorial 31

32 Results 29 users years old from 0 to 20 years of programming 32

33 Do Program Navigation and Conversational Clarification contribute to correctness? Configuration Errors (mean) BI 2.7 BI + PN 1.2 BI + CC 1.1 We performed the Wilcoxon signed-rank test on the instrumentation data. Based on the data collected, we have found significant reduction of number of errors with each of these new interaction modes 33

34 Which of Program Navigation and Conversational Clarification is perceived more useful for data extraction? Model Score (mean) PN 4.2 CC

35 Which of Program Navigation and Conversational Clarification is perceived more useful for data extraction? [I]n the last example, in which I didn t have [Conversational Clarification] as an option, I felt like I miss it so much It always helps me to find the right matching I absolutely loved [regular expression highlighting]. I think that perfectly helps one understand what the computer is thinking at the moment and to identify things that were misunderstood the program [is] quite understandable but it was not clear how to modify the program 35

36 Future work 36

37 Composition: FlashExtract + FlashFill 37

38 FlashExtract + FlashFill + Nlyze 38

39 Systematic changes by example 39

40 Program Synthesis for Data Manipulation Gustavo Soares Federal University of Campina Grande 40

Automated Behavioral Testing of Refactoring Engines

Automated Behavioral Testing of Refactoring Engines Gustavo Soares Federal University of Campina Grande, Campina Grande, PB, Brazil gsoares@dsc.ufcg.edu.br Abstract Refactoring is a transformation that