Scaling Program Synthesis by Exploiting Existing Code

Transcription

1 Scaling Program Synthesis by Exploiting Existing Code James Bornholt Emina Torlak University of Washington

2 Synthesis: write programs automatically Syntax Semantics

3 Synthesis: write programs automatically Syntax Semantics

4 Synthesis: write programs automatically Target Behavior f(x) =4x +1 Syntax Semantics

5 Synthesis: write programs automatically 4*x + 1 Target Behavior f(x) =4x +1 Syntax Semantics

6 Synthesis: write programs automatically 4*x + 1 (x<<2) + 1 Target Behavior f(x) =4x +1 x + x + x + x + 1 Syntax Semantics

7 Synthesis: write programs automatically Target Behavior Permutation(L, f(l)) ^ Sorted(f(L)) Syntax Semantics

8 Synthesis: write programs automatically Quicksort Target Behavior Permutation(L, f(l)) ^ Sorted(f(L)) Bubble Sort Syntax Semantics

9 The success of synthesis End-user programming by example [FlashFill, POPL 11] Cache coherence protocols [Transit, PLDI 13] Parallel browser layout engines [PPoPP 13] Compilers for new spatial architectures [Chlorophyll, PLDI 14]

10 The success of synthesis End-user programming by example [FlashFill, POPL 11] Cache coherence protocols Synthesis [Transit, PLDI 13] turns high-level intent Parallel browser layout into engines [PPoPP 13] low-level detail Compilers for new spatial architectures [Chlorophyll, PLDI 14]

11 The success of synthesis End-user programming by example [FlashFill, POPL 11] Approximate Computing quality bounds into approximate programs Cache coherence protocols Synthesis [Transit, PLDI 13] turns high-level intent Parallel browser layout into engines [PPoPP 13] low-level detail Compilers for new spatial architectures [Chlorophyll, PLDI 14]

12 The success of synthesis End-user programming by example [FlashFill, POPL 11] Cache coherence protocols Synthesis [Transit, PLDI 13] turns high-level intent Parallel browser layout into engines [PPoPP 13] low-level detail Compilers for new spatial architectures [Chlorophyll, PLDI 14] Approximate Computing quality bounds into approximate programs Hardware Synthesis programs into hardware designs

13 The success of synthesis End-user programming by example [FlashFill, POPL 11] Cache coherence protocols Synthesis [Transit, PLDI 13] turns high-level intent Parallel browser layout into engines [PPoPP 13] low-level detail Compilers for new spatial architectures [Chlorophyll, PLDI 14] Approximate Computing quality bounds into approximate programs Hardware Synthesis programs into hardware designs Black Box Systems observed behaviors into specifications

14 Approximate Computing quality bounds into approximate programs Not all applications require perfect accuracy

15 Approximate Computing quality bounds into approximate programs Not all applications require perfect accuracy

16 Approximate Computing quality bounds into approximate programs Not all applications require perfect accuracy Video and image quality

17 Approximate Computing quality bounds into approximate programs Not all applications require perfect accuracy Video and image quality Sensors and simulation Machine learning

18 Approximate Computing quality bounds into approximate programs Not all applications require perfect accuracy Precise Implementation

19 Approximate Computing quality bounds into approximate programs Not all applications require perfect accuracy Precise Implementation Desired Quality

20 Approximate Computing quality bounds into approximate programs Not all applications require perfect accuracy Precise Implementation Approximate Compiler Desired Quality

21 Approximate Computing quality bounds into approximate programs Not all applications require perfect accuracy Precise Implementation Approximate Compiler Approximate Program Desired Quality

22 Approximate Computing quality bounds into approximate programs Not all applications require perfect accuracy Precise Implementation Desired Quality Approximate Program Synthesis Approximate Program Bornholt, Torlak, Ceze, Grossman. Approximate Program Synthesis. At WAX 15, collocated with PLDI 15.

23 Hardware Synthesis programs into hardware designs Synthesizing circuits from high-level languages

24 Hardware Synthesis programs into hardware designs Synthesizing circuits from high-level languages module example(input a, b, c, output y); assign y = ~a & ~b & ~c a & ~b & ~c a & ~b & c; endmodule

25 Hardware Synthesis programs into hardware designs Synthesizing circuits from high-level languages module example(input a, b, c, output y); assign y = ~a & ~b & ~c a & ~b & ~c a & ~b & c; endmodule

26 Hardware Synthesis programs into hardware designs Synthesizing circuits from high-level languages module example(input a, b, c, output y); assign y = ~a & ~b & ~c a & ~b & ~c a & ~b & c; endmodule

27 Hardware Synthesis programs into hardware designs Synthesizing circuits from high-level languages High-Level Synthesis (HLS)

28 Hardware Synthesis programs into hardware designs Synthesizing circuits from high-level languages Place and route Timing closure High-Level Synthesis (HLS) Crosstalk and feedback Quantum!

29 Hardware Synthesis programs into hardware designs Synthesizing circuits from high-level languages Mark Wyse UW grad student and HLS extraordinaire

30 Hardware Synthesis programs into hardware designs Synthesizing circuits from high-level languages Mark Wyse UW grad student and HLS extraordinaire float dist(float a[3], float b[3]) { float r = 0; r += (a[0] - b[0]) * (a[0] - b[0]); r += (a[1] - b[1]) * (a[1] - b[1]); r += (a[2] - b[2]) * (a[2] - b[2]); return sqrt(r); }

31 Black Box Systems observed behaviors into specifications Defining system behavior by observation

32 Black Box Systems observed behaviors into specifications Defining system behavior by observation

33 Black Box Systems observed behaviors into specifications Defining system behavior by observation

34 Black Box Systems observed behaviors into specifications Defining system behavior by observation

35 Black Box Systems observed behaviors into specifications Defining system behavior by observation Programming by example

36 Black Box Systems observed behaviors into specifications Defining system behavior by observation Programming by example Synthesizing x86 instruction specs Godefroid and Taly. Automated Synthesis of Symbolic Instruction Encodings from I/O Samples. PLDI 12.

37 Approximate Computing quality bounds into approximate programs Hardware Synthesis programs into hardware designs Black Box Systems observed behaviors into specifications

38 Machine Learning and Synthesis

39 Learning programs from examples Black Box Systems observed behaviors into specifications 9P. ^ x i 2X '(x i,p(x i ))

40 Learning programs from examples Black Box Systems observed behaviors into specifications 9P. ^ x i 2X '(x i,p(x i )) FlashFill

41 Learning programs from examples FlashFill Version Space Algebra

42 Machine learning and synthesis Program Synthesis Statistical Machine Learning

43 Machine learning and synthesis Millions of examples/parameters Program Synthesis Statistical Machine Learning

44 Machine learning and synthesis 100 instructions Millions of examples/parameters Program Synthesis Statistical Machine Learning

45 Machine learning and synthesis Approximate Computing Hardware Synthesis Black Box Systems 100 instructions Millions of examples/parameters Program Synthesis Statistical Machine Learning

46 Machine learning and synthesis Approximate Computing Hardware Synthesis Black Box Systems 100 instructions Millions of examples/parameters Program Synthesis Statistical Machine Learning

47 Programs are not uniformly distributed.

48 Programs are not uniformly distributed Stack Overflow mentions malloc longjmp

49 Programs are not uniformly distributed Stack Overflow mentions malloc longjmp

50 Programs are not uniformly distributed Stack Overflow mentions malloc longjmp

51 Programs are not uniformly distributed. Stack Overflow mentions malloc longjmp mov call push mov call test jmp lea push sub

52 Component-Based Synthesis

53 Synthesis of Loop-Free Programs Gulwani, Jha, Tiwari, and Venkatesan. Synthesis of loop-free programs. PLDI 11.

54 Synthesis of Loop-Free Programs f(x, y) = x + y 2 Gulwani, Jha, Tiwari, and Venkatesan. Synthesis of loop-free programs. PLDI 11.

55 Synthesis of Loop-Free Programs f(x, y) = x + y 2 and xor add >> 1 Gulwani, Jha, Tiwari, and Venkatesan. Synthesis of loop-free programs. PLDI 11.

56 Synthesis of Loop-Free Programs f(x, y) = x + y 2 and xor add >> 1 x y Gulwani, Jha, Tiwari, and Venkatesan. Synthesis of loop-free programs. PLDI 11.

57 Synthesis of Loop-Free Programs f(x, y) = x + y 2 and xor add >> 1 x y f(x,y) Gulwani, Jha, Tiwari, and Venkatesan. Synthesis of loop-free programs. PLDI 11.

58 Synthesis of Loop-Free Programs f(x, y) = x + y 2 x y xor and >> 1 add f(x,y) Gulwani, Jha, Tiwari, and Venkatesan. Synthesis of loop-free programs. PLDI 11.

59 Synthesis of Loop-Free Programs f(x, y) = x + y 2 and xor add >> 1 Gulwani, Jha, Tiwari, and Venkatesan. Synthesis of loop-free programs. PLDI 11.

60 Synthesis of Loop-Free Programs f(x, y) = x + y 2 and xor add >> 1 sub div rem or not mul udiv urem nand neg << 1 >>> 1 << 2 eq le ge lt Gulwani, Jha, Tiwari, and Venkatesan. Synthesis of loop-free programs. PLDI 11. gt

61 Synthesis of Loop-Free Programs f(x, y) = x + y 2 and xor add >> sub mul Solving time (secs) div udiv rem urem or nand not neg << 1 >>> Number of components << 2 eq le ge lt Gulwani, Jha, Tiwari, and Venkatesan. Synthesis of loop-free programs. PLDI 11. gt

62 lt gt Higher-level components f(l) =min{l i l i 2 L} and xor add >> 1 sub div rem or not mul udiv urem nand neg << 1 >>> 1 << 2 eq le ge

63 Higher-level components f(l) =min{l i l i 2 L}

64 Higher-level components f(l) =min{l i l i 2 L} quicksort 0 [ ]

65 Higher-level components f(l) =min{l i l i 2 L} f(l) = quicksort(l)[0] quicksort 0 [ ]

66 Higher-level components f(l) =min{l i l i 2 L} f(l) = quicksort(l)[0] quicksort 0 [ ] Mine existing code bases for common idioms

67 Higher-level components f(l) =min{l i l i 2 L} f(l) = quicksort(l)[0] quicksort 0 [ ] Mine existing code bases for common idioms Hardware Synthesis programs into hardware designs

68 Producing candidate programs f(x, y) = x + y 2 and xor add >> 1 x y f(x,y)

69 Producing candidate programs f(x, y) = x + y 2 xor add >> 1 x y and

70 Producing candidate programs f(x, y) = x + y 2 xor add >> 1 x y and mov call mov call test jmp lea push push sub

71 Producing candidate programs f(x, y) = x + y 2 xor add >> 1 x y and mov call mov call test jmp lea push push sub Learn search heuristics from existing code

72 Producing candidate programs f(x, y) = x + y 2 xor add >> 1 x y and mov call mov call test jmp lea push Learn search heuristics from existing code push sub Programming by example [FlashFill, POPL 11] Autocomplete with synthesis [CodeHint, ICSE 14]

73 Producing candidate programs f(x, y) = x + y 2 xor add >> 1 x y and Mine existing code bases for common idioms Learn search heuristics from existing code mov call push mov call test jmp lea push sub Programming by example [FlashFill, POPL 11] Autocomplete with synthesis [CodeHint, ICSE 14]

74 Producing candidate programs Mine existing code bases for common idioms Learn search heuristics from existing code Black Box Systems observed behaviors into specifications Selecting among many candidate solutions

75 Sketch-Based Synthesis

76 Sketches make synthesis more tractable

77 Sketches make synthesis more tractable bit[w] popcount(bit[w] x) { x = (x & 0x5555) + ((x >> 1) & 0x5555); x = (x & 0x3333) + ((x >> 2) & 0x3333); x = (x & 0x0077) + ((x >> 8) & 0x0077); x = (x & 0x000F) + ((x >> 4) & 0x000F); return x; }

78 Sketches make synthesis more tractable bit[w] popcount(bit[w] x) { x = (x & 0x5555) + ((x >> 1) & 0x5555); x = (x & 0x3333) + ((x >> 2) & 0x3333); x = (x & 0x0077) + ((x >> 8) & 0x0077); x = (x & 0x000F) + ((x >> 4) & 0x000F); return x; } bit[w] popsketched(bit[w] x) { loop (??) { x = (x &??) + ((x >>??) &??); } return x; }

79 Sketches make synthesis more tractable bit[w] popsketched(bit[w] x) { loop (??) { x = (x &??) + ((x >>??) &??); } return x; }

80 Sketches make synthesis more tractable bit[w] popsketched(bit[w] x) { loop (??) { x = (x &??) + ((x >>??) &??); } return x; } int[] map(int[] xs) { int[] ys = {}; for (int i=0; i<xs.length; i++) { ys.append(??(xs[i])); } return ys; int[] foldl(int[] xs) { int acc =??; for (int i=0; i<xs.length; i++) { acc =??(acc, xs[i]); } return ys; } } int[] filter(int[] xs) { int[] ys = {}; for (int i=0; i<xs.length; i++) { if (??(xs[i])) ys.append(xs[i]); } }

81 Sketches make synthesis more tractable Identify common sketches from existing code

82 Sketches make synthesis more tractable Mine existing code bases for common idioms Identify common sketches from existing code

83 Sketches make synthesis more tractable Mine existing code bases for common idioms Identify common sketches from existing code Approximate Computing quality bounds into approximate programs

84 Sketches make synthesis more tractable Mine existing code bases for common idioms Identify common sketches from existing code Approximate Computing quality bounds into approximate programs Precise Implementation Approximate Program Synthesis

85 Approximate Computing Hardware Synthesis Black Box Systems Program Synthesis Statistical Machine Learning

86 Approximate Computing Hardware Synthesis Black Box Systems Program Synthesis Statistical Machine Learning

87 Approximate Computing Hardware Synthesis Black Box Systems Program Synthesis Statistical Machine Learning Thanks!