Bytecode Manipulation Techniques for Dynamic Applications for the Java Virtual Machine

Transcription

1 Bytecode Manipulation Techniques for Dynamic Applications for the Java Virtual Machine Eugene Kuleshov, Terracotta Tim Eck, Terracotta Tom Ware, Oracle Corporation Charles Nutter, Sun Microsystems, Inc. TS JavaOne SM Conference Session TS-1326

2 Goal Bytecode manipulation isn t difficult and it is very cool Understand how dynamic frameworks for Java platform do their job and how these ideas can be used in other applications JavaOne SM Conference Session TS

3 Agenda Java Virtual Machine, Bytecode and ASM Framework Lazy Attributes in Java Persistence API (JPA) (TopLink) Terracotta DSO Ruby to Java Compiler in JRuby Summary The terms Java Virtual Machine and JVM mean a Virtual Machine for the Java platform JavaOne SM Conference Session TS

4 Java Virtual Machine (JVM ) Proven platform for running reliable and high-performance applications Built for the statically-typed language Class-loading architecture and reflection API enables dynamic code Many frameworks need more Introduce additional logic into the existing code Increase performance Non-Java programming languages The terms Java Virtual Machine and JVM mean a Virtual Machine for the Java platform JavaOne SM Conference Session TS

5 The Class File Format Constant Pool Field and method names, type descriptors String literals and other constants Attributes Fields Methods Code (ordered list of instructions) Debug information (line numbers, local variable names) Exceptions User defined attributes Source: The Java Virtual Machine Specification 2007 JavaOne SM Conference Session TS

6 Class File Modification Problems Lots of serialization and deserialization details Constant pool management Missing or unused constants Managing constant pool indexes/references Jump offsets Inserting or removing instructions from the method Computation of stack size and StackMapTable Requires a control flow analysis 2007 JavaOne SM Conference Session TS

7 ASM Bytecode Framework Goal: Dynamic class generation and modification Very small and very fast tool Tool primarily adapted for simple transformations Complete control over the produced classes is not needed Approach Use the Visitor pattern without using in-memory object model Hide the (de)serialization and constant pool management details Represent jump offsets by Label marker objects Checker and ASMifier tools helps with the method code Automatic computation of the max stack size and StackMapTable Source: ASM project JavaOne SM Conference Session TS

8 ASM Bytecode Framework Main idea ClassReader ClassAdapter ClassWriter accepts ClassVisitor implements ClassVisitor implements ClassVisitor accept(v) visit("c", "Object", ) visit("c", "Object", ) serialized class (byte array) visitfield("i", "byte", ) visitfieldinsn(getfield, "i") visitfield("i", "byte", ) visitfield("_i", "int", ) visitmethodinsn( INVOKEVIRTUAL, "_geti") serialized class (byte array) tobytearray() Source: ASM project JavaOne SM Conference Session TS

9 ASM Bytecode Framework Example ClassReader cr = new ClassReader(bytecode); ClassWriter cw = new ClassWiter(cr, ClassWriter.COMPUTE_MAXS); FooClassAdapter cv = new FooClassAdapter(cw); cr.accept(cv, 0); // load the new class final byte[] bytes = cw.tobytearray(); Class newclass = new ClassLoader(parent) { Class c = defineclass(name, bytes, 0, bytes.length); }.c; Source: ASM project JavaOne SM Conference Session TS

10 ASM Bytecode Framework Framework Organization Core (only 36Kb) Generate classes Basic transformations Tree and Analysis In-memory representation and analysis algorithms Commons Renaming, sort local variables, inline subroutines (JSR/RET instructions), calculate serialversionuid, advice adapter, etc. Util Checker, decompiler/tracer and ASMifier utils XML XSLT-based transformations and querying Source: ASM project JavaOne SM Conference Session TS

11 Agenda Java Virtual Machine, Bytecode and ASM Framework Lazy Attributes in JPA API (TopLink) Terracotta DSO Ruby to Java Compiler in JRuby Summary 2007 JavaOne SM Conference Session TS

12 TopLink JPA Java Persistence API (JPA) Provides a standard API for Object/Relational mapping Most common use: Mapping Java objects to relational databases TopLink is an advanced object mapping library Open source JPA API reference implementation: TopLink Essentials (GlassFish project) Open source: EclipseLink Eclipse Java Persistence Platform Project Oracle TopLink 2007 JavaOne SM Conference Session TS

13 JPA API Lazy Loading 1 Employee 1 address 1 Address managed m Loading Employee could result in address and managed employees being loaded Potentially loads a large amount of data JPA API allows these relationships to be LAZY Lazy relationships are fetched only when needed Weaving is used to make 1-to-1 relationships LAZY (address) 2007 JavaOne SM Conference Session TS

14 JPA API Lazy Loading JPA API allows both field access and property access For field access, we replace any access to the field with a call to a method we add For property access, we weave the getter and setter methods to add some additional code The weaving code inserts a proxy object called a ValueHolder to represent the relationship Employee address_vh ValueHolder value Address 2007 JavaOne SM Conference Session TS

15 Employee.java /** * A simple Employee class using field access public class Employee private Address address; public Address getaddress() { return address; } } public void setaddress(address address) { this.address = address; } 2007 JavaOne SM Conference Session TS

16 Employee.java woven /** * Employee class using field access after weaving public class Employee private Address private ValueHolder _toplink_address_vh; public Address getaddress() { return _toplink_getaddress(); } public void setaddress(address address) { _toplink_setaddress(address); } 2007 JavaOne SM Conference Session TS

17 Employee.java woven (Cont.) // added through weaving public Address _toplink_getaddress() { address = _toplink_address_vh.getvalue(); return address; } // added through weaving public void _toplink_setaddress(address address) { _toplink_address_vh.setvalue(address); this.address = address; } } 2007 JavaOne SM Conference Session TS

18 Replacing a Variable Reference // this method is called by the visitfieldinsn() callback public void weaveattributesifrequired(int opcode, String owner, String name, String desc) {... if (opcode == GETFIELD && attributedetails!= null) { cv.visitmethodinsn( INVOKEVIRTUAL, tcw.classdetails.getclassname(), "_toplink_get" + name, "()L" + attributedetails.getreferenceclasstype().getdescriptor() ); } else { super.visitfieldinsn(opcode, owner, name, desc); } } 2007 JavaOne SM Conference Session TS

19 Bytecode Transformation in TopLink Benefits/suggestions Allows us to add features that can be used in more intuitive way Combination of the ASMifier and your favourite decompiler make it fairly easy to prototype weaving code Strong need for very well commented code Challenges Designing to avoid unintended side effects Do not benefit from some compiler features (e.g., primitive wrapping) Additional uses Optimizing change set calculation Fetch groups Read-only validation 2007 JavaOne SM Conference Session TS

20 Agenda Java Virtual Machine, Bytecode and ASM framework Lazy attributes in JPA API (TopLink) Terracotta DSO Ruby to Java Compiler in JRuby Summary 2007 JavaOne SM Conference Session TS

21 Terracotta DSO Distributed Shared Objects What is DSO? Object distribution and thread coordination across VMs Open source It s just Java code No APIs plain objects Existing language threading primitives synchronized, wait() / notify() Why ASM for DSO? Fast/small Actively maintained and supported Widely adopted, open source 2007 JavaOne SM Conference Session TS

22 Basic Terracotta Concepts DSO root objects Root objects are the top most object nodes of a distributed object graph Roots are bound to fields in your classes Objects referenced by the graph starting from a root become distributed class MyAppType { Map m; // root field } 2007 JavaOne SM Conference Session TS

23 Terracotta ASM Use Field/Array Operations PUTFIELD / AASTORE Object state mutations are recorded and broadcast to other VMs that contain the same object GETFIELD / AALOAD Allow lazy loading of portions of the object graph Record access frequency for use in eviction policy 2007 JavaOne SM Conference Session TS

24 Terracotta ASM Use Field Operations class Foo { private Bar bar; public Bar get() { return bar; } } public void set(bar bar) { this.bar = bar; } 2007 JavaOne SM Conference Session TS

25 Terracotta ASM Use Field Operations class Foo { private Bar bar; public Bar get() { if (isshared() && bar==null) bar = ManagerUtil.resolveReference(this, Foo.bar"); return bar; } } public void set(bar bar) { if (isshared()) ManagerUtil.fieldChanged(this, Foo.bar, bar); this.bar = bar; } 2007 JavaOne SM Conference Session TS

26 Terracotta ASM Use Distributed Object Monitors MONITORENTER / MONITOREXIT synchronized(obj) { // MONITORENTER... } // MONITOREXIT If a distributed object instance is synchronized on, the monitor can configured to be cluster wide Thread safe data structures can thus be made cluster safe 2007 JavaOne SM Conference Session TS

27 Terracotta ASM Use Interesting methods java.lang.object: wait(), notify(), notifyall() When invoked on distributed objects, thread signals can be used across VMs If you know how to use these methods, you already know how to use them in DSO Logical object actions For data structures like HashMap that depend on local hashcode() values Distributed Method Invocation DMI can be used to create event style notifications using your own methods 2007 JavaOne SM Conference Session TS

28 Agenda Java Virtual Machine, Bytecode and ASM framework Lazy attributes in JPA API (TopLink) Terracotta DSO Ruby to Java Compiler in JRuby Summary 2007 JavaOne SM Conference Session TS

29 JRuby Ruby to Java Compiler Ruby is a very dynamic language Many methods/types generated at runtime Interpretation remains important but code is largely static JVM implementation optimizes simpler code better Interpreter is not simple New code spat out every few seconds is not simple Compilation allows JVM implementation to work its magic Compiled code is bare metal Eliminate overhead of repeated interpreter logic 2007 JavaOne SM Conference Session TS

30 JRuby Ruby to Java Compiler Mixed-mode execution Code potentially generated with every operation Don t compile everything AOT (ahead-of-time) mode Dump out Java bytecode for atomic pieces Straight-through run of a script file Methods defined during that run Single class file for single.rb file One Java method per Ruby method plus run-through Simple distribution 2007 JavaOne SM Conference Session TS

31 Ruby to Java Compiler JRuby JIT (just-in-time) compilation Generated code is used either once or many times Interpret at first Count usages, compile when appropriate Very much like JVM implementation operation Lower overhead than compile always No continuous compilation cost No continuous class-gen/class-load cost Fall back on pure interpretation where compile won t work Incremental development 50% of code compiles? No problem, interpret the rest 2007 JavaOne SM Conference Session TS

32 JRuby Compiler Code walkthrough and perf tests 2007 JavaOne SM Conference Session TS

33 For More Information ASM project TopLink JPA Terracotta DSO JRuby JavaOne SM Conference Session TS

34 Q&A Eugene Kuleshov, Terracotta Tim Eck, Terracotta Tom Ware, Oracle Corporation Charles Nutter, Sun Microsystems, Inc JavaOne SM Conference Session TS

35 Bytecode Manipulation Techniques for Dynamic Applications for the Java Virtual Machine Eugene Kuleshov, Terracotta Tim Eck, Terracotta Tom Ware, Oracle Corporation Charles Nutter, Sun Microsystems, Inc. TS JavaOne SM Conference Session TS-1326