BEAMJIT, a Maze of Twisty Little Traces

Transcription

1 BEAMJIT, a Maze of Twisty Little Traces A walk-through of the prototype just-in-time (JIT) compiler for Erlang. Frej Drejhammar <frej@sics.se>

2 Who am I? Senior researcher at the Swedish Institute of Computer Science (SICS) working on programming tools and distributed systems.

3 What this talk is About A brief introduction to the BEAM just-in-time compiler followed by a walk-through of last year s development.

4 Outline Background Just-In-Time Compilation BEAM: Specification & Implementation Project Goal Tools JIT:ing as it applies to BEAM The BEAM JIT Prototypes Future Work Acknowledgements & Questions

5 Just-In-Time (JIT) Compilation Decide at runtime to compile hot parts to native code. Fairly common implementation technique Python (Psyco, PyPy) Smalltalk (Cog) Java (HotSpot) JavaScript (SquirrelFish Extreme, SpiderMonkey)

6 BEAM: Specification & Implementation BEAM is the name of the Erlang VM. A register machine. Approximately 150 instructions which are specialized to approximately 450 macro-instructions using a peephole optimizer during code loading. Hand-written C (mostly) directly threaded interpreter. No authoritative description of the semantics of the VM except the implementation source code!

7 Project Goal Goals: Do as little manual work as possible. Preserve the semantics of plain BEAM. Automatically stay in sync with the plain BEAM, i.e. if bugs are fixed in the interpreter the JIT should not have to be modified manually. Have a native code generator which is state-of-the-art. Plan: Parse and extract semantics from the C implementation. Transform the parsed C source to C fragments which are then reassembled into a replacement interpreter which includes a JIT-compiler.

8 HiPE vs JIT Why would Erlang need a JIT-compiler, we already have HiPE? Cross module optimization. Native-code much larger than BEAM-code. Tracing does not require switching to full emulation. Modules stay target independent, simplifies deployment: No need for cross compilation. Binaries not strongly coupled to a particular build of the emulator.

9 Tools LLVM A Compiler Infrastructure, contains a collection of modular and reusable compiler and toolchain technologies. Uses a low-level assembler-like representation called LLVM-IR. Clang A mostly gcc-compatible front-end for C-like languages, produces LLVM-IR. libclang A C library built on top of Clang, allows the AST of a parsed C-module to be accessed and traversed.

10 Just-In-Time (JIT) Compilation as it Applies to BEAM Use light-weight profiling to detect when we are at a place which is frequently executed. Trace the flow of execution until we get back to the same place. Compile trace to native code. NOTE: We are tracing the execution flow in the interpreter, the granularity is finer than BEAM opcodes. Profile Trace Generate Native Code Run Native

11 BEAMJIT: What is Needed? Three basic execution modes Profiling Tracing Native Interpreter loop has to be modified to support mode switching: Turn on/off tracing. Passing state to/from native code. Native code generation: Need the semantics for each instruction.

12 Extracting the Semantics of the BEAM Opcodes bb_3944 bb_3950 i_bs_get_utf8_xfd is_function_fr bb_3784 bb_3788 bb_4130 bb_3776 bb_3783 i_bs_get_utf8_rfd bb_3936 bb_3794 bb_3792 bb_3790 i_bs_get_binary2_fxisid bb_4126 bb_3921 bb_4138 bb_4162 bb_3775 bb_4141 bb_4149 bb_4128 bb_4097 bb_4164 bb_3919 bb_3923 bb_4155 bb_3911 bb_4140 bb_4143 bb_3925 bb_4372 bb_4161 i_bs_get_binary2_frisid bb_4063 bb_4099 bb_4178 bb_4096 bb_4174 bb_4061 i_bs_skip_bits2_fxyi bb_4176 bb_4132 bb_4565 bb_4568 bb_4374 bb_4073 bb_4371 bb_4065 bb_4180 bb_4566 bb_4574 bb_4561 bb_4109 i_bs_get_float2_fxisid bb_4356 bb_4360 bb_4358 bb_4580 bb_4365 bb_4384 bb_4115 bb_4113 bb_4111 bb_4067 bb_4521 bb_4557 bb_4090 bb_4527 bb_4517 bb_4390 bb_4075 bb_4077 bb_4540 bb_4401 bb_4362 i_bs_skip_bits2_fxxi bb_4525 bb_4526 bb_4406 bb_4533 bb_4397 bb_4386 bb_4076 bb_4084 bb_4388 bb_4413 bb_4408 bb_4420 bb_2488 bb_3851 i_bs_skip_bits2_frxi bb_4405 bb_4486 bb_4493 i_bs_start_match2_rfiid i_bs_skip_bits2_fxri bb_4485 bb_4500 bb_3854 bb_4488 bb_4481 bb_4477 i_bs_get_integer_fiid i_bs_start_match2_yfiid i_bs_skip_bits2_fryi bb_4446 bb_4445 bb_4454 bb_4460 bb_3538 bb_4441 bb_4437 i_bs_start_match2_xfiid bb_4448 bb_3866 bb_3860 i_bs_match_string_xfii bb_4008 bb_3900 bb_3852 bb_4009 bb_4015 bb_4014 bb_4313 bb_4016 bb_3898 bb_4311 i_bs_match_string_rfii i_bs_get_float2_frisid bb_3872 bb_3894 bb_3896 bb_3883 bb_4343 bb_4315 bb_4320 bb_3884 bb_4326 bb_4339 bb_4327 bb_4329 bb_3878 do_is_ne_exact_literal bb_4345 bb_4317 bb_3847 bb_1636 bb_3880 i_is_ne_exact_literal_yfc bb_4341 bb_3762 bb_3745 bb_3746 bb_3752 bb_3758 bb_3764 bb_3584 bb_3753 bb_3760 Use libclang to parse and simplify the interpreter source: Flatten variable scopes. Remove loops, replace by if + goto. Make fall-troughs explicit. Turn structured C into a spaghetti of Basic Blocks (BB), CFG Control Flow Graph. Do liveness-analysis of variables.

13 Outline Background Just-In-Time Compilation BEAM: Specification & Implementation Project Goal Tools JIT:ing as it applies to BEAM The BEAM JIT Prototypes Future Work Acknowledgements & Questions

14 BEAMJIT Evolution Evolution since last year Mk. I (EUC 12) Mk. Ib MK. II MK. III Mk. IV (EUC 13)

15 BEAMJIT Mk. I: Profiling First step in figuring out what to JIT-compile Let Erlang compiler insert profile instructions at places which can be the head of a loop. Count the number of times a function is executed. Trigger tracing when count is high enough. Eventually everything is compiled, this is BAD. Requires implementing (by hand) the profile-instruction in the interpreter.

16 BEAMJIT Mk. I: Tracing Switch to a new version of the interpreter, generated from the CFG. For each basic block we pass through, record basic block identity and PC. Abort trace if too long. If we reach the profile instruction we started the trace from We have found a loop!

17 BEAMJIT Mk. I: Profiling to Tracing Mode Switch Direct threading PC 0xCDAADEF0 0xCDAADEFF 0xCDAACAF8 0xCDAAD432 0xDEADBEEF Indirect threading PC XCDAACAF8 x = x + 1;... Profiling Opcodes 95: 0xCBAADEF0 96: 0xCDAEDEFF 97: 0xCDAACAF8 98: 0xCDAAD432 99: 0xDEADBEEF Tracing Opcodes 95: 0xCBBADEF0 96: 0xCBAEDEFF 97: 0xCBAACAF8 98: 0xCBAAD432 99: 0xDBADBEEF 0XCDAACAF8 x = x + 1;... Have two implementations of each opcode. Switch the table of opcodes. 0XCBAACAF8 record_trace_bb(4711, PC) x = x + 1;...

18 BEAMJIT Mk. I: Cost of Indirect Threading

19 BEAMJIT Mk. I: Native-code Generation Glue together LLVM-IR-fragments for the trace. Guards are inserted to make sure we stay on the traced path. Hand the resulting IR off to LLVM. Fragments are extracted from the CFG as C-source, compiled to IR using clang (at build-time) and loaded during system initialization. jit_emu.c Trace beam_emu.c CFG Bitcode IR generator LLVM optimizer Native code fragments.c

20 BEAMJIT Mk. I: Calling Native Code Interpreter Native: Interpreter: Copy live variables to a structure. Native: Load vars into temporaries. Native Interpreter: The reverse. Jump to the BB to continue from.

21 BEAMJIT Mk. I: Performance Depressing performance. Running in pure interpreting mode, 6-7 times slower. x =...; if (x!= 0) x = x+ 1; else x = x - 1;... bb0: regx =... brz bb1, bb0... bb1: regx =... brz bb1, bb0 bb0: regx = load(@x) inc regx bb1: regx = load(@x) dec regx

22 BEAMJIT Mk. Ib: First Useful First version that could compile OTP without crashing and pass the test suite. Make profiler time-aware. Measure execution intensity by including timestamp, count is incremented if the function was executed recently, reset otherwise. Blacklist locations which: Never produce a successful trace. Where we leave the trace without executing the loop at least once. GC traces when they are no longer needed. Minor performance improvements.

23 BEAMJIT Mk. II: Make it Easy for the Compiler Modify the interpreter loop as little as possible. Have separate trace interpreter. Limit entry to the interpreter at instruction boundaries. Have separate cleanup-interpreter to continue execution to the next instruction boundary. Profiling Tracing Native Cleanup

24 BEAMJIT Mk. II: Implementation Tricks Use liveness information from the CFG. Package native-code as a function where the arguments are the live variables. The cleanup-interpreter is a set of functions, one for each BB, which tail-recursively calls the next BB. Arguments are the live variables.

25 BEAMJIT Mk. II: Performance Performance not stellar. Sensitive to placement in source-code. Should be possible to optimize further.

26 BEAMJIT Mk. III: Trace-Along Appears that we quite often compile a trace which is not representative. Ensure that we have a representative trace: Trace-Along Follow along a previously created trace. Abort trace if we diverge. Generate code when succeeded multiple times.

27 BEAMJIT Mk. IV: Multi-path We blacklist many locations where trace-along repeatedly fails to find a representative trace. Allow multi-path traces. Generate native code when the trace has not grown for N successive iterations. Slows down LLVM optimization and native code generation significantly.

28 BEAMJIT Mk. IV: Trace Compression LLVM slowdown appears to be related to the size of the CFG. Inspection of traces shows loops and common segments. Compress traces to remove shared segments. BB=0 ip=0x4567 BB=0 ip=0x4567 BB=1 ip=0x4567 BB=2 ip=0x4567 BB=1 ip=0x4567 BB=2 ip=0x4567 BB=3 ip=0x4568 BB=3 ip=0x4568 BB=3 ip=0x4568 BB=4 ip=0x4569 BB=4 ip=0x4569 BB=4 ip=0x4569

29 BEAMJIT Mk. IV: Performance Compilation overhead dwarfs everything else (-O2). Future work: Figure out which optimization passes are needed.

30 BEAMJIT Mk. IV: Performance (cont.)

31 BEAMJIT Mk. IV: Performance (cont.) Guards costly. Not good where the common case cannot be on the fast path

32 Future Work Do not fixate on finding loops Allow traces which are runs rather than loops, ring benchmarks. Erlang-aware constant propagation: Eliminate loads from code (constant at compile time). Will eliminate loading of immediates. Will eliminate many of the guards. Increase performance in plain interpreting mode. Run native-code generation in separate thread. Extend trace outside the main interpreter loop, inside BIFs.

33 Outline Background Just-In-Time Compilation BEAM: Specification & Implementation Project Goal Tools JIT:ing as it applies to BEAM The BEAM JIT Prototypes Future Work Acknowledgements & Questions

34 Acknowledgements This work is funded by Ericsson AB.

35 Questions?