Correctness of Speculative Optimizations with Dynamic Deoptimization

Transcription

1 Correctness of Speculative Optimizations with Dynamic Deoptimization Olivier Flückiger, Gabriel Scherer, Ming-Ho Yee, Aviral Goel, Amal Ahmed, Jan Vitek Northeastern University, Boston, USA October 12,

2 Our work Just-in-time (JIT) compilation is essential to efficient dynamic language implementations. (Javascript, Lua, R... Java) There is a blind spot in our formal understanding of JITs: speculation. We present a language design to study speculative optimizations and prove them correct. 2

3 Just-in-time compilation Source program JITs: 3

4 Just-in-time compilation Source program Critical region JITs: Profiling 3

5 Just-in-time compilation Source program Critical region Machine code JITs: Profiling + High/Low languages + Dynamic code generation/mutation 3

6 Just-in-time compilation Source program Critical region Machine code JITs: Profiling + High/Low languages + Dynamic code generation/mutation + Speculation 3

7 Just-in-time compilation Source program Critical region Machine code Checkpoint JITs: Profiling + High/Low languages + Dynamic code generation/mutation + Speculation and bailout 3

8 JITs: profiling high- and low-level languages (or multi-tiers, etc.) dynamic code generation + mutation speculation and bailout 4

9 JITs: profiling high- and low-level languages (or multi-tiers, etc.) dynamic code generation + mutation eliminates interpretation overhead (constant factor) speculation and bailout 4

10 JITs: profiling high- and low-level languages (or multi-tiers, etc.) dynamic code generation + mutation eliminates interpretation overhead (constant factor) speculation and bailout eliminates dynamic features overhead: dispatch (OO languages), type checks (Java), code loading (Java), redefinable primitives (R...) 4

11 JITs: profiling high- and low-level languages (or multi-tiers, etc.) dynamic code generation + mutation eliminates interpretation overhead (constant factor) speculation and bailout eliminates dynamic features overhead: dispatch (OO languages), type checks (Java), code loading (Java), redefinable primitives (R...) JIT formalization: Myreen [2010] Stack language and x86 assembly dynamic code generation code mutation mechanized in HOL! 4

12 JITs: profiling high- and low-level languages (or multi-tiers, etc.) dynamic code generation + mutation eliminates interpretation overhead (constant factor) speculation and bailout eliminates dynamic features overhead: dispatch (OO languages), type checks (Java), code loading (Java), redefinable primitives (R...) JIT formalization: Myreen [2010] Stack language and x86 assembly dynamic code generation code mutation mechanized in HOL! 4

13 JITs: profiling high- and low-level languages (or multi-tiers, etc.) dynamic code generation + mutation eliminates interpretation overhead (constant factor) speculation and bailout eliminates dynamic features overhead: dispatch (OO languages), type checks (Java), code loading (Java), redefinable primitives (R...) JIT formalization: Myreen [2010] Stack language and x86 assembly dynamic code generation code mutation mechanized in HOL! What about speculation? 4

14 JITs: profiling high- and low-level languages (or multi-tiers, etc.) dynamic code generation + mutation eliminates interpretation overhead (constant factor) speculation and bailout eliminates dynamic features overhead: dispatch (OO languages), type checks (Java), code loading (Java), redefinable primitives (R...) JIT formalization: Myreen [2010] Stack language and x86 assembly dynamic code generation code mutation mechanized in HOL! What about speculation? This work. 4

15 Sourir high- and low-level languages a single bytecode language dynamic code generation one unrolled multi-version program speculative optimization and bailout 5

16 Sourir high- and low-level languages a single bytecode language dynamic code generation one unrolled multi-version program speculative optimization and bailout 5

17 Sourir high- and low-level languages a single bytecode language dynamic code generation one unrolled multi-version program speculative optimization and bailout 5

18 Sourir high- and low-level languages a single bytecode language dynamic code generation one unrolled multi-version program speculative optimization and bailout a checkpoint instruction (See Myreen [2010] for the first two.) 5

19 Sourir high- and low-level languages a single bytecode language dynamic code generation one unrolled multi-version program speculative optimization and bailout a checkpoint instruction (See Myreen [2010] for the first two.) fun(c) tough L 1 luck var o = 1 print c + o assume c = 41 else fun.tough.l 1 [c = c, o = 1] print 42 5

20 Contribution A language design to model speculative optimization: Sourir A kit of correct program transformations and optimizations A methodology to reason about correct speculative optimizations 6

21 A simple bytecode language i ::= var x = e drop x x e array x[e ] array x = [e ] x[e 1] e 2 branch e L 1 L 2 goto L print e read x call x = e(e ) return e assume e else ξ ξ stop e ::= se x[se ] length(se) primop (se ) se ::= lit F x lit ::=..., 1, 0, 1,... nil true false 7

22 Versions P ::= F (x ) D F,... program: a list of named functions D F ::= V I,... function definition: list of versioned instruction streams I ::= L i,... instruction stream with labeled instructions fun(c) tough L 1 luck var o = 1 print c + o assume c = 41 else fun.tough.l 1 [c = c, o = 1] print 42 8

23 Checkpoints Checkpoint: guards + bailout data. assume c = 41 else fun.tough.l 1 [c = c, o = 1] Guards: just a list of expressions returning booleans. Bailout data: where to go: F.V.L in what state: [x 1 = e 1,.., x n = e n ] (plus more: see inlining) 9

24 Checkpoints Checkpoint: guards + bailout data. assume c = 41 else fun.tough.l 1 [c = c, o = 1] Guards: just a list of expressions returning booleans. Bailout data: where to go: F.V.L in what state: [x 1 = e 1,.., x n = e n ] (plus more: see inlining) Not a branch. (inlining) Checkpoints simplify optimizations... 9

25 Checkpoints Checkpoint: guards + bailout data. assume c = 41 else fun.tough.l 1 [c = c, o = 1] Guards: just a list of expressions returning booleans. Bailout data: where to go: F.V.L in what state: [x 1 = e 1,.., x n = e n ] (plus more: see inlining) Not a branch. (inlining) Checkpoints simplify optimizations...and correctness proofs! 9

26 Speculative optimization pipeline Critical version 10

27 Speculative optimization pipeline Critical version copy 10

28 Speculative optimization pipeline Critical version copy Add checkpoints and assumptions 10

29 Speculative optimization pipeline Critical version copy Add checkpoints and assumptions Optimize 10

30 Speculative optimization pipeline Critical version Final program: Two versions Active version 10

31 Speculative optimization pipeline Critical version 10

32 Execution: Operational semantics Configurations: C ::= P I L K M E Actions: A ::= read lit print lit A τ := A τ T ::= A. Reduction: C 1 A τ C 2 C 1 T C 2 11

33 Equivalence: (weak) bisimulation Relation R between the configurations over P 1 and P 2. R is a weak simulation if: A τ C 1 C 1 A τ C 1 C 1 R = R R C 2 A τ C 2 C 2 R is a weak bisimulation if R and R 1 are simulations. 12

34 Bailout invariants Version invariant: All versions of a function are equivalent. (Necessary to replace the active version) Bailout invariant: Bailing out more than necessary is correct. (Necessary to add new assumptions) 13

35 Branch pruning from the kit base L 1 branch tag = INT int nonint int... nonint... Use a line here please! 14

36 Branch pruning from the kit base L 1 branch tag = INT int nonint int... nonint... opt L 1 assume tag = INT else F.base.L 1 [...] branch tag = INT int nonint int... nonint... Checkpoint + guard inserted Bailout invariant! 14

37 Branch pruning from the kit base L 1 branch tag = INT int nonint int... nonint... constant folding opt L 1 assume tag = INT else F.base.L 1 [...] branch true int nonint int... nonint... 14

38 Branch pruning from the kit base L 1 branch tag = INT int nonint int... nonint... opt L 1 assume tag = INT else F.base.L 1 [...] int... unreachable code elimination 14

39 Conclusion All you need for speculation: versions + checkpoints. Future work: bidirectional transformations. Thanks! Questions? 15

40 Magnus O. Myreen. Verified just-in-time compiler on x86. In Principles of Programming Languages (POPL), doi: /

41 Bonus: inlining main( ) inlined array pl = [1, 2, 3, 4] array vec = [length(pl), pl] var size = nil var obj = vec assume obj nil else... var len = obj[0] size len 32 drop len drop obj ret base... goto ret print size stop main( ) base ret size(obj) opt base... array pl = [1, 2, 3, 4] array vec = [length(pl), pl] call size = size(vec) print size stop assume obj nil else... var len = obj[0] return len 32 17

42 Bonus: inlining main( ) inlined array pl = [1, 2, 3, 4] array vec = [length(pl), pl] var size = nil var obj = vec assume obj nil else... var len = obj[0] size len 32 drop len drop obj ret base... goto ret print size stop main( ) base ret size(obj) opt base... array pl = [1, 2, 3, 4] array vec = [length(pl), pl] call size = size(vec) print size stop assume obj nil else... var len = obj[0] return len 32 main (inlined) main(base) size assume obj nil else ξ main.base.ret size [vec = vec] 17