Automatic Programming: How Far Can Machines Go? Hila Peleg Technion

Transcription

1 Automatic Programming: How Far Can Machines Go? Hila Peleg Technion

2

3 Laziness

4 Laziness Impatience

5 Laziness Impatience Hubris

6 Automatic Programming Clean up my spreadsheet! Right away, boss!

7 Program synthesis to the rescue

8 Program Synthesis

9 What we really want This is what I want

10 What if we have automatic programming? Hyper-intelligent program generation for your every need

11 What if we have automatic programming? Hyper-intelligent program generation for your every need Self-aware, self-augmenting AI

12 What if we have automatic programming? Hyper-intelligent program generation for your every need Self-aware, self-augmenting AI The singularity

13 What if we have automatic programming? Hyper-intelligent program generation for your every need Self-aware, self-augmenting AI The singularity???

14 What if we have automatic programming? Hyper-intelligent program generation for your every need Self-aware, self-augmenting AI The singularity??? THE ROBOT APOCALYPSE

15 The Robot Apocalypse

16 But it s ok!

17 But it s ok! Understand users Build a program

18 But it s ok! Understand users Build a program Generalizing partial intent is hard

19 But it s ok! Understand users Build a program Generalizing partial intent is hard Solving HALT is hard*

20

21 User intent is hard Clean up my spreadsheet! Right away, boss!

22 User intent is hard Clean up my spreadsheet! Right away, boss! What s clean? Is this right? What am I doing with my life?

23 Intent via examples

24 Intent via examples

25 Intent via examples

26 Intent via examples

27 Query knowledge about (some kind of) code I want to apply foo() to requests to my http server

28 Query knowledge about (some kind of) code I want to apply foo() to requests to my http server val routes : Route =??? val bindingfuture = Http(). bindandhandle(routes, "localhost", 8080)

29 Commit Strip said it best

30 Commit Strip said it best

31 Commit Strip said it best

32 Commit Strip said it best

33 Building a program is also hard* Get me a program that takes a program and an input and tells me if that program stops on that input.

34 Building a program is also hard* Get me a program that takes a program and an input and tells me if that program stops on that input. (i.e., the halting problem)

35 Adjusting our expectations Here s a grammar of 20 functions and 10 constants, get me a program that I m certain is in this space.

36 Adjusting our expectations Here s a grammar of 20 functions and 10 constants, get me a program that I m certain is in this space.

37 Adjusting our expectations Here s a grammar of 20 functions and 10 constants, get me a program that I m certain is in this space.

38 What does this all mean?

39 What does this all mean? Generally, find me a program that cannot be solved

40 What does this all mean? Generally, find me a program that cannot be solved Still, we re not giving up

41 What does this all mean? Generally, find me a program that cannot be solved Still, we re not giving up Realistic expectations for realistic program synthesis

42 Realistic expectations for realistic synthesis We still want Partial specifications To not have to know everything A result! But we ll have to live without Checking every possible program Fully automatic solution Single-step solution

43 Realistic expectations for realistic synthesis We still want Partial specifications To not have to know But we ll have to live without Checking every possible program everything A result! Synthesis Engine Fully automatic solution Interaction Model Single-step solution

44 The synthesis engine Predicts code for intent Draws its understanding from language syntax and/or crowd wisdom and/or semantic specifications Reduce the number of programs seen

45 Reducing equivalent programs We ve seen x+y, so we don t want y+x

46 Reducing equivalent programs We ve seen x+y, so we don t want y+x But how do we know they re the same?

47 Reducing equivalent programs We ve seen x+y, so we don t want y+x But how do we know they re the same? 1. Heuristics

48 Reducing equivalent programs We ve seen x+y, so we don t want y+x But how do we know they re the same? 1. Heuristics 2. Solvers (e.g., Z3)

49 Reducing equivalent programs We ve seen x+y, so we don t want y+x But how do we know they re the same? 1. Heuristics 2. Solvers (e.g., Z3) 3. Observational Equivalence

50 Reducing equivalent programs We ve seen x+y, so we don t want y+x But how do we know they re the same? 1. Heuristics 2. Solvers (e.g., Z3) 3. Observational Equivalence Equivalence: p 1 p 2 i.f.f. for every possible input i ever, p 1 i = p 2 (i)

51 Reducing equivalent programs We ve seen x+y, so we don t want y+x But how do we know they re the same? 1. Heuristics 2. Solvers (e.g., Z3) 3. Observational Equivalence Equivalence: p 1 p 2 i.f.f. for every possible input i ever, p 1 i = p 2 (i) Observational equivalence: p 1 OE p 2 i.f.f. for every input i the user cares about, p 1 i = p 2 (i)

52 Trying a different strategy altogether Use knowledge bases instead of the language grammar They no longer contain every program (neither limited grammars) Searchable via graph algorithms or probability equations

53 The Interaction Model Aimed at programmers Specify intent Express yourself Think like a programmer

54 Specifying (and re-specifying) intent Task: find the median of a list User: examples! 1. [1,2,3] 2 2. [7,8,7,3] 7 Synthesis engine: input[input.length/2] User:

55 Specifying (and re-specifying) intent Task: find the median of a list User: examples! 1. [1,2,3] 2 2. [7,8,7,3] 7 Synthesis engine: input[input.length/2] User: It managed to find a single formula, let s make a counterexample

56 Specifying (and re-specifying) intent Task: find the median of a list User: examples! Principle #1: 1. [1,2,3] 2 Cost of communicating intent + 2. [7,8,7,3] 7 consuming result << cost of Synthesis engine: manually performing the task input[input.length/2] User: It managed to find a single formula, let s make a counterexample

57 Programming Not Only by Example input[input.length/2] [] input /. 2 input length A programmer can talk at the level of the program Read debug info Reason about subtrees or sequences of methods Even rewrite the program But also give examples, if those happen to be easier

58 Programming Not Only by Example input[input.length/2] Exclude [] input / Retain input. 2 length A programmer can talk at the level of the program Read debug info Reason about subtrees or sequences of methods Even rewrite the program But also give examples, if those happen to be easier

59 Programming Not Only by Example input[input.length/2] Exclude [] input / Retain input A programmer can talk at the level of the program Principle #2: Read debug info Let developers be Reason developers about subtrees or sequences of methods. 2 Even rewrite the program But also give examples, if those happen to be easier length

60 When models out-think the programmer def counts(l : List[String]) : Map[String,Int]= l.

61 When models out-think the programmer def counts(l : List[String]) : Map[String,Int]= l. groupby( map( fold(

62 When models out-think the programmer def counts(l : List[String]) : Map[String,Int]= l.groupby( identity) x => x.length) x => x[0])

63 When models out-think the programmer def counts(l : List[String]) : Map[String,Int]= l.groupby(identity). map(x => x._1 -> x._2 x => x.length) x => x[0])

64 When models out-think the programmer def counts(l : List[String]) : Map[String,Int]= l.groupby(identity). map(x => x._1 -> x._2. filter(y => y.startswith( x._2.length x._2[0])

65 When models out-think the programmer def counts(l : List[String]) : Map[String,Int]= l.groupby(identity). map(x => x._1 -> x._2. filter(y => y.startswith(

66 When models out-think the programmer def counts(l : List[String]) : Map[String,Int]= l.groupby(identity). map(x => x._1 -> x._2. filter(y => y.startswith(

67 Keep understanding what s going on??

68 Keep understanding what s going on??

69 Keep understanding what s going on?? Principle #3: Results must be explainable

70 % correct answer Programmers aren t as good as they think 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% histogram no. lines with text most frequent word PBE Only new operations All operations

71 Read-Eval-Synth Loops

72 What if we have automatic programming? Hyper-intelligent program generation for your every need Self-aware, self-augmenting AI The singularity??? THE ROBOT APOCALYPSE