BEAMJIT: An LLVM based just-in-time compiler for Erlang. Frej Drejhammar

Transcription

1 BEAMJIT: An LLVM based just-in-time compiler for Erlang Frej Drejhammar

2 Who am I? Senior researcher at the Swedish Institute of Computer Science (SICS) working on programming languages, tools and distributed systems.

3 Acknowledgments Project funded by Ericsson AB. Joint work with Lars Rasmusson

4 What this talk is About Automatic synthesis of a JIT-compiling VM for Erlang. Our experiences with LLVM and MCJIT.

5 Outline Background Just-In-Time Compilation Erlang BEAM: Specification & Implementation Project Goals JIT:ing as it applies to BEAM Profiling Tracing Generating and Calling Native Code Future Work LLVM s Strengths and Weaknesses Questions

6 Just-In-Time (JIT) Compilation Decide at runtime to compile hot parts to native code. Fairly common implementation technique. Python (Psyco, PyPy) Smalltalk (Cog) Java (HotSpot) JavaScript (SquirrelFish Extreme, SpiderMonkey)

7 Erlang Functional language developed by Ericsson. Soft real-time. Multi-threaded with share-nothing semantics. Message passing. Powerful supervision primitives. Hot code loading and replacement. OTP: Framework for writing fault-tolerant applications. Compiled to virtual machine (VM), BEAM.

8 BEAM: Specification & Implementation BEAM is the name of the Erlang VM. A register machine. Approximately 150 instructions which are specialized to around 450 macro-instructions using a peephole optimizer during code loading. Instructions are CISC-like. Hand-written C (mostly) directly threaded interpreter. No authoritative description of the semantics of the VM except the implementation source code! HiPE a ahead-of-time native compiler Traditional back-end for x86, PowerPC, SPARC, ARM ErLLVM back-end based on LLVM

9 Motivation A JIT compiler increases flexibility. Compiled BEAM modules are platform independent. Cross-module optimization. Integrates naturally with code upgrade.

10 Project Goals Goals: Plan: Do as little manual work as possible. Preserve the semantics of plain BEAM. Automatically stay in sync with the plain BEAM, i.e. if bugs are fixed in the interpreter the JIT should not have to be modified manually. Have a native code generator which is state-of-the-art. Parse and extract semantics from the C implementation. Transform the parsed C source to C fragments which are then reassembled into a replacement VM which includes a JIT-compiler.

11 Outline Background Just-In-Time Compilation Erlang BEAM: Specification & Implementation Project Goals JIT:ing as it applies to BEAM Profiling Tracing Generating and Calling Native Code Future Work LLVM s Strengths and Weaknesses Questions

12 Just-In-Time (JIT) Compilation as it Applies to BEAM Use light-weight profiling to detect when we are at a place which is frequently executed. Trace the flow of execution until we get back to the same place. Compile trace to native code. NOTE: We are tracing the execution flow in the interpreter, the granularity is finer than BEAM opcodes. Profile Trace Generate Native Code Run Native

13 BEAMJIT: What is Needed? Three basic execution modes Profiling Tracing Native Interpreter loop has to be modified to support mode switching: Turn on/off tracing. Passing state to/from native code. Native code generation: Need the semantics for each instruction.

14 Profiling First step in figuring out what to JIT-compile Let Erlang compiler insert profile instructions at locations which can be the head of a loop Maintain a time stamp and counter for each location Measure execution intensity by incrementing a counter if the location was visited recently, reset otherwise Trigger tracing when count is high enough Blacklist locations which: Never produce a successful trace. Where we, when executing native code, leave the trace without executing the loop at least once.

15 Outline Background Just-In-Time Compilation Erlang BEAM: Specification & Implementation Project Goals JIT:ing as it applies to BEAM Profiling Tracing Generating and Calling Native Code Future Work LLVM s Strengths and Weaknesses Questions

16 Extracting the Semantics of the BEAM Opcodes bb_3944 bb_3950 i_bs_get_utf8_xfd is_function_fr bb_3784 bb_3788 bb_4130 bb_3776 bb_3783 i_bs_get_utf8_rfd bb_3936 bb_3794 bb_3792 bb_3790 i_bs_get_binary2_fxisid bb_4126 bb_3921 bb_4138 bb_4162 bb_3775 bb_4141 bb_4149 bb_4128 bb_4097 bb_4164 bb_3919 bb_3923 bb_4155 bb_3911 bb_4140 bb_4143 bb_3925 bb_4372 bb_4161 i_bs_get_binary2_frisid bb_4063 bb_4099 bb_4178 bb_4096 bb_4174 bb_4061 i_bs_skip_bits2_fxyi bb_4176 bb_4132 bb_4565 bb_4568 bb_4374 bb_4073 bb_4371 bb_4065 bb_4180 bb_4566 bb_4574 bb_4561 bb_4109 i_bs_get_float2_fxisid bb_4356 bb_4360 bb_4358 bb_4580 bb_4365 bb_4384 bb_4115 bb_4113 bb_4111 bb_4067 bb_4521 bb_4557 bb_4090 bb_4527 bb_4517 bb_4390 bb_4075 bb_4077 bb_4540 bb_4401 bb_4362 i_bs_skip_bits2_fxxi bb_4525 bb_4526 bb_4406 bb_4533 bb_4397 bb_4386 bb_4076 bb_4084 bb_4388 bb_4413 bb_4408 bb_4420 bb_2488 bb_3851 i_bs_skip_bits2_frxi bb_4405 bb_4486 bb_4493 i_bs_start_match2_rfiid i_bs_skip_bits2_fxri bb_4485 bb_4500 bb_3854 bb_4488 bb_4481 bb_4477 i_bs_get_integer_fiid i_bs_start_match2_yfiid i_bs_skip_bits2_fryi bb_4446 bb_4445 bb_4454 bb_4460 bb_3538 bb_4441 bb_4437 i_bs_start_match2_xfiid bb_4448 bb_3866 bb_3860 i_bs_match_string_xfii bb_4008 bb_3900 bb_3852 bb_4009 bb_4015 bb_4014 bb_4313 bb_4016 bb_3898 bb_4311 i_bs_match_string_rfii i_bs_get_float2_frisid bb_3872 bb_3894 bb_3896 bb_3883 bb_4343 bb_4315 bb_4320 bb_3884 bb_4326 bb_4339 bb_4327 bb_4329 bb_3878 do_is_ne_exact_literal bb_4345 bb_4317 bb_3847 bb_1636 bb_3880 i_is_ne_exact_literal_yfc bb_4341 bb_3762 bb_3745 bb_3746 bb_3752 bb_3758 bb_3764 bb_3584 bb_3753 bb_3760 Use libclang to parse and simplify the interpreter source: Use Erlang binding for libclang. Flatten variable scopes. Remove loops, replace by if + goto. Make fall-troughs explicit. Turn structured C into a spaghetti of Basic Blocks (BB), CFG Control Flow Graph. Do liveness-analysis of variables.

17 Naïve Tracing Use a new version of the interpreter, generated from the CFG. Generate a tracing and non-tracing version of each opcode. For each basic block we pass through, record basic block identity and PC. Abort trace if too long. If we reach the profile instruction we started the trace from We have found a loop!

18 Naïve Profiling to Tracing Mode Switch Direct threading PC 0xCDAADEF0 0xCDAADEFF 0xCDAACAF8 0xCDAAD432 Indirect threading PC xDEADBEEF 0XCDAACAF8 x = x + 1;... Profiling Opcodes 95: 0xCBAADEF0 96: 0xCDAEDEFF 97: 0xCDAACAF8 98: 0xCDAAD432 99: 0xDEADBEEF Tracing Opcodes 95: 0xCBBADEF0 96: 0xCBAEDEFF 97: 0xCBAACAF8 98: 0xCBAAD432 99: 0xDBADBEEF 0XCDAACAF8 x = x + 1;... 0XCBAACAF8 Have two implementations of each opcode. Switch the table of opcodes. record_trace_bb(4711, PC) x = x + 1;... Compiler has to assume that a mode switch can take place at any block performance suffers

19 Refined Tracing Modify the interpreter loop as little as possible. Have separate trace interpreter. Limit entry to the interpreter at instruction boundaries. Have separate cleanup-interpreter to continue execution to the next instruction boundary. Reuse cleanup-interpreter when returning from native mode. Profiling Tracing Native Cleanup

20 Further Tracing Refinements Ensure that we have a representative trace: Follow along a previously created trace. Allow multi-path traces. Generate native code when the trace has not grown for N successive iterations. Enforce limit on total size of trace.

21 Trace Compression Large CFGs slow down LLVM optimization and native code generation significantly. Solution: Compress traces to remove shared segments. BB=0 ip=0x4567 BB=0 ip=0x4567 BB=1 BB=2 BB=1 BB=2 ip=0x4567 ip=0x4567 ip=0x4567 ip=0x4567 BB=3 ip=0x4568 BB=3 ip=0x4568 BB=3 ip=0x4568 BB=4 ip=0x4569 BB=4 ip=0x4569 BB=4 ip=0x4569

22 Outline Background Just-In-Time Compilation Erlang BEAM: Specification & Implementation Project Goals JIT:ing as it applies to BEAM Profiling Tracing Generating and Calling Native Code Future Work LLVM s Strengths and Weaknesses Questions

23 Native-code Generation Glue together LLVM-IR-fragments for the trace. Guards are inserted to make sure we stay on the traced path. Fragments are extracted from the CFG as C-source, compiled to IR using clang (at build-time) and loaded during system initialization. Hand the resulting IR off to LLVM for the rest. jit_emu.c Trace beam_emu.c CFG Bitcode IR generator LLVM optimizer Native code fragments.c

24 Calling Native Code Switching from interpreter to native code: Use liveness information from the CFG. Package native-code as a function where the arguments are the live variables. Switching from native code to interpreter: The cleanup-interpreter is a set of functions, one for each BB, which tail-recursively calls the next BB. Arguments are the live variables. Cleanup-interpreter packs up live variables in a structure which the interpreter unpacks on return.

25 Performance Improvements Run native-code generation in separate thread. Erlang-aware constant propagation: Eliminate loads from code (constant at compile time). Will eliminate loading of immediates. Will eliminate many of the guards.

26 Performance Steady state: Eliminates most of the instruction decode overhead. Up to 50% reduction in runtime. Well-behaved programs: around 25%. Programs using cute tricks such as unrolling: up to 200% increase in runtime.

27 Future Work Full SMP support. Box/unboxing-aware constant propagation. Extend JIT support to fold in primitives.

28 Outline Background Just-In-Time Compilation Erlang BEAM: Specification & Implementation Project Goals JIT:ing as it applies to BEAM Profiling Tracing Generating and Calling Native Code Future Work LLVM s Strengths and Weaknesses Questions

29 LLVM s Strengths Access to the C AST via libclang. Quality of generated code. The optimization framework.

30 LLVM s Weaknesses No thread-safety Extra housekeeping to do things in the correct context. Compilation could be much faster. Native code has to be packaged as C-function Would really like to have enough control to patch generated native code. Clang/LLVM does bad job on the main interpreter: Allocates the VM s stack- and instruction-pointers on the stack. Inserts extra instruction sequence before indirect jumps (when dispatching to next VM instruction), GCC appears to insert it at the branch target and only if needed. Costs us 15-20%

31 Outline Background Just-In-Time Compilation Erlang BEAM: Specification & Implementation Project Goals JIT:ing as it applies to BEAM Profiling Tracing Generating and Calling Native Code Future Work LLVM s Strengths and Weaknesses Questions

32 Questions?