Optimising large dynamic code bases.

Transcription

1 Optimising large dynamic code bases.

2 Who am I? Duncan MacGregor Lead Software Engineer on the Magik on Java project at General Electric in Cambridge Aardvark179 on twitter 2

3 What is Magik? Dynamic, weakly typed message passing based language Does everything "wrong" i.e. numerical type promotion, closures over mutable values, etc. Not that special in terms of the language. 3

4 So what does make it special? It s tightly integrated with a long transaction version managed data store. There are large applications built using it. No, larger than that, I mean really large 4

5 So what is large? A typical application with an open database has 200,000 methods defined About 10% of those are record field accessors and other code generated by the ORM 5

6 It gets worse There are bigger data models out there, some with over 4000 tables Many customers will use third party add ons and their own customisations which may be almost as large as the base apps 6

7 What are our main concerns? Performance must be good enough Need to startup and open the database as fast as possible Must not use ridiculous amounts of memory as the application is used Warmup time needs to be as short as possible 7

8 What I m going to talk about today Memory usage Startup time Smoothing the path of moving from an old VM to a new one 8

9 Memory usage

10 Where we started on Java 8

11 First steps running on Java 8 Measure startup speed relative to Java 7 Examine memory relative to Java 7 Take heap dumps as the application is used Project memory usage of fully exercised application 11

12 What's using the memory? 1. LambdaForms 2. LambdaForms 3. LambdaForms 4. Concurrent Collections (!) 5. Atomics 6. Everything else 12

13 LambdaForms! Binding followed by adapting generated many many LFs Likewise drop arguments These combined to make SwitchPoint GWT surprising inefficient 13

14 Concurrent collections? We want a run time as lock free as possible a bad application shouldn't lock up core language infrastructure Concurrent collections are great in small numbers If every method has a little one to hold type adapted versions then the memory adds up 14

15 Atomics Again, we want things lock free if possible Methods have SwitchPoints to handle invalidation Flags Other meta-data All held in atomics to support update by multiple threads 15

16 What is everything else? Antlr s lexer generation produces large short arrays Our globals are stored in a prefix tree, we need to move them to a more compact structure Reflection caches can be huge Many other small things, but they add up 16

17 Reducing memory usage

18 Prototyping / proof of concept Change MH adaption order Cache unbound adapters for CallSites Reduce the PIC of each site to 1 Write our own SwitchPoint class 18

19 Old CallSite fallback adaption Exact Invoker Fold Type adaption Bind to CallSite Fallback 19

20 Old CallSite fallback adaption Exact Invoker Fold Type adaption Bind to CallSite Fallback 20

21 New CallSite fallback adaption Exact Invoker Drop argument Bind to CallSite Fold Type adaption Fallback 21

22 New CallSite fallback adaption Exact Invoker Drop argument Bind to CallSite Fold Type adaption Fallback 22

23 GuardWithTest issue Guard With Test Bind to Class Target Fallback ClassCheck 23

24 GuardWithTest issue Guard With Test Drop arguments Target Fallback Bind to Class ClassCheck 24

25 GuardWithTest solution Guard With Test Bind to Class Target Fallback Drop arguments ClassCheck 25

26 GuardWithTest solution Guard With Test Bind to Class Target Fallback Drop arguments ClassCheck 26

27 GuardWithTest solution Guard With Test Bind to Class Target Fallback Drop arguments ClassCheck 27

28 SwitchPoint dynamicinvoker Exact Invoker Fold Type adaption Bind to CallSite GET_TARGET 28

29 SwitchPoint dynamicinvoker Exact Invoker Fold Type adaption Bind to CallSite GET_TARGET 29

30 SwitchPoint guardwithtest Guard With Test Drop Arguments Target Fallback Invoker 30

31 SwitchPoint guardwithtest Guard With Test Bind to CallSite Target Fallback Fold Type adaption ExactInvoker GET_TARGET 31

32 SwitchPoint guardwithtest Guard With Test Bind to CallSite Target Fallback Fold Type adaption ExactInvoker GET_TARGET 32

33 Did it help? Yes! More than halved the LFs Applications feel faster when first started Memory usage significantly reduced 33

34 Putting it into production Started by tackling easy parts. Concurrent collections, Atomics etc. Then refactor call sites This work is still on going 34

35 CallSites: optimising for the common cases Method calls are complex Normal, super, self How many results? All sites supported instrumentation Added complexity to each instance 35

36 Refactor into many classes Normal Call Call To Self Super Call Single Result Single PrivateSingle SuperPrivateSingle Tuple Result Tuple PrivateTuple SuperPrivateTuple Unknown Number of Results Private SuperPrivate 36

37 Move towards functional composition Hard to build a good class hierarchy without repetition Some functionality based on choices at runtime (instrumentation etc.) Allows megamorphic costs to only be paid by the sites that need it 37

38 Unexpected advantages MethodSendPrivateSingleCallSite Number 0 5,000 10,000 15,000 20,000 25,000 30,000 35,000 40,000 45,000 MethodSendSingleCallSite MethodSendTupleCallSite MethodSendPrivateTupleCallSite MethodSendCallSite MethodSendPrivateCallSite MethodSendSuperPrivateSingleCallSite MethodSendSuperPrivateCallSite MethodSendSuperPrivateTupleCallSite 38

39 Start up and database open

40 Bootstrapping the system Bootstrapping the system involves loading a lot of code that is only run once Loading a class looks up MethodHandle constants repeatedly Loading a module causes resources to be scanned on disk 40

41 Improving this Improving Java s MH constant caching improves our start up time by over 10% Emitting fewer larger classes may also help Restructure our resource system to stop using the file system We may look at more radical solutions depending on where the time is spent 41

42 Opening the database Lots of meta-programming Subclasses for record types Field access methods Join navigation Old VM allowed images to be saved after all this had been initialised 42

43 Opening the database faster Creating new classes requires reflection Cache data on classes you know you ll need again Be very careful about the reflection calls you make, some are faster than others Be very focused about invalidating CallSites 43

44 Why? Unoptimised class/method creation After performance work MagikSF Alchemy (Java 7u25) Alchemy (Java 8) 0 MagikSF Alchemy (Java 7u25) Alchemy (Java 8) 44 GE Title or job number 7/28/2014

45 Serialisation Need to turn the whole data dictionary into a series of blobs Some things are hard to serialise Want to restore database connections in parallel trial by fire for thread safety Additional work then needed to stitch everything back together 45

46 How much faster is it? 80 Time to open database Without serialisation With serialisation 46

47 What s still to do? Fix some concurrency bugs Test with more databases 47

48 Easing the transition

49 Need to help people moving from our old VM Used to running tests with some form of coverage analysis Not used to threads being truly concurrent 49

50 Coverage of non-java languages Jacoco can compile coverage stats without any problems We ve run large test suites through it and nothing terrible happened Displaying those results is a different story 50

51 Coverage results Presenting coverage data for a large code base requires hierarchical display The packages for your classes need to be in a good hierarchy as well or the results still won t be managable If the class file contains info about the source file then use that, don t depend on Java source conventions 51

52 How much work is this? Prototyping changes to Jacoco took an afternoon Haven t turned those into proper patches yet Changes to our compiler still to be done 52

53 Threading We can instrument thread creation and use of atomic queues and locks That only catches the cases where people thought about thread safety at all 53

54 Summary

55 MethodHandle caching and reuse is vital Serialisation and optimising metaprogramming really help startup speeds No easy answers for finding concurrency issues 55

56