Living in the Matrix with Bytecode Manipulation

Similar documents
Program Dynamic Analysis. Overview

3/15/18. Overview. Program Dynamic Analysis. What is dynamic analysis? [3] Why dynamic analysis? Why dynamic analysis? [3]

The Dark Sides of Integration. Anton Arhipov JRebel,

Bytecode Manipulation with a Java Agent and Byte Buddy. Koichi Sakata PONOS Corporation

JVML Instruction Set. How to get more than 256 local variables! Method Calls. Example. Method Calls

Course Overview. PART I: overview material. PART II: inside a compiler. PART III: conclusion

JVM. What This Topic is About. Course Overview. Recap: Interpretive Compilers. Abstract Machines. Abstract Machines. Class Files and Class File Format

The Java Virtual Machine. CSc 553. Principles of Compilation. 3 : The Java VM. Department of Computer Science University of Arizona

Let s make some Marc R. Hoffmann Eclipse Summit Europe

Why GC is eating all my CPU? Aprof - Java Memory Allocation Profiler Roman Elizarov, Devexperts Joker Conference, St.

Michael Rasmussen ZeroTurnaround

Java Instrumentation for Dynamic Analysis

Compiling Techniques

Improving Java Code Performance. Make your Java/Dalvik VM happier

CS2110 Fall 2011 Lecture 25. Under the Hood: The Java Virtual Machine, Part II

Run-time Program Management. Hwansoo Han

Java Class Loading and Bytecode Verification

Improving Java Performance

Compiler construction 2009

Programming Language Systems

Taming the Java Virtual Machine. Li Haoyi, Chicago Scala Meetup, 19 Apr 2017

Mnemonics Type-Safe Bytecode Generation in Scala

Bytecode Manipulation Techniques for Dynamic Applications for the Java Virtual Machine

CSE P 501 Compilers. Java Implementation JVMs, JITs &c Hal Perkins Winter /11/ Hal Perkins & UW CSE V-1

Scala Benchmark: Invariant Verifier

Agenda. CSE P 501 Compilers. Java Implementation Overview. JVM Architecture. JVM Runtime Data Areas (1) JVM Data Types. CSE P 501 Su04 T-1

Static Analysis of Dynamic Languages. Jennifer Strater

02 B The Java Virtual Machine

How do you create a programming language for the JVM?

Java Code Coverage Mechanics

The Java Virtual Machine

Space Exploration EECS /25

Implement detection of duplicate test coverage for Java applications

invokedynamic IN 45 MINUTES!!! Wednesday, February 6, 13

Under the Hood: The Java Virtual Machine. Lecture 23 CS2110 Fall 2008

COMP3131/9102: Programming Languages and Compilers

Java Code Coverage Mechanics. by Evgeny Mandrikov at EclipseCon Europe 2017

Recap: Printing Trees into Bytecodes

Java Code Coverage Mechanics Evgeny Mandrikov Marc Hoffmann #JokerConf 2017, Saint-Petersburg

Exercise 7 Bytecode Verification self-study exercise sheet

Problem: Too Many Platforms!

Understand Every Line of Your Codebase

The Java Language Implementation

Java Security. Compiler. Compiler. Hardware. Interpreter. The virtual machine principle: Abstract Machine Code. Source Code

Under the Hood: The Java Virtual Machine. Problem: Too Many Platforms! Compiling for Different Platforms. Compiling for Different Platforms

CSc 372 Comparative Programming Languages. Getting started... Getting started. B: Java Bytecode BCEL

Over-view. CSc Java programs. Java programs. Logging on, and logging o. Slides by Michael Weeks Copyright Unix basics. javac.

CS263: Runtime Systems Lecture: High-level language virtual machines. Part 1 of 2. Chandra Krintz UCSB Computer Science Department

CSc 620 Debugging, Profiling, Tracing, and Visualizing Programs. Getting started... Getting started. 2 : Java Bytecode BCEL

CSc 620 Debugging, Profiling, Tracing, and Visualizing Programs

Java byte code verification

Compiling for Different Platforms. Problem: Too Many Platforms! Dream: Platform Independence. Java Platform 5/3/2011

Today. Instance Method Dispatch. Instance Method Dispatch. Instance Method Dispatch 11/29/11. today. last time

SOFTWARE ARCHITECTURE 7. JAVA VIRTUAL MACHINE

Tutorial 3: Code Generation

Java and C II. CSE 351 Spring Instructor: Ruth Anderson

CSE 431S Final Review. Washington University Spring 2013

Java 1.6 Annotations ACCU Tony Barrett-Powell

CSC 4181 Handout : JVM

Advanced Programming Introduction

JSR 292 backport. (Rémi Forax) University Paris East

Advances in Programming Languages: Generics, interoperability and implementation

Run-time Performance Testing in Java

JAM 16: The Instruction Set & Sample Programs

CSc 620 Debugging, Profiling, Tracing, and Visualizing Programs. Compiling Java. The Java Class File Format 1 : JVM

javac 29: pop 30: iconst_0 31: istore_3 32: jsr [label_51]

Chapter 5. A Closer Look at Instruction Set Architectures. Chapter 5 Objectives. 5.1 Introduction. 5.2 Instruction Formats

301AA - Advanced Programming [AP-2017]

Chapter 5. A Closer Look at Instruction Set Architectures

COMP3131/9102: Programming Languages and Compilers

Building a Compiler with. JoeQ. Outline of this lecture. Building a compiler: what pieces we need? AKA, how to solve Homework 2

Joeq Analysis Framework. CS 243, Winter

Compiling Faster, Compiling Better with Falcon. Iván

LINGI2252 PROF. KIM MENS REFLECTION (IN JAVA)*

Introduction Basic elements of Java

An Introduction to Multicodes. Ben Stephenson Department of Computer Science University of Western Ontario

The Future of Code Coverage for Eclipse

COMP 520 Fall 2009 Virtual machines (1) Virtual machines

JoeQ Framework CS243, Winter 20156

Compilation 2012 Code Generation

Roadmap. Java: Assembly language: OS: Machine code: Computer system:

Java: framework overview and in-the-small features

Introduction to Programming Using Java (98-388)

Stacks. stacks of dishes or trays in a cafeteria. Last In First Out discipline (LIFO)

CSCE 314 Programming Languages

JSR 292 Cookbook: Fresh Recipes with New Ingredients

Selected Questions from by Nageshwara Rao

Java Intro 3. Java Intro 3. Class Libraries and the Java API. Outline

Copyright 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 16

Index COPYRIGHTED MATERIAL

Chapter 5. A Closer Look at Instruction Set Architectures. Chapter 5 Objectives. 5.1 Introduction. 5.2 Instruction Formats

Chapter 5. A Closer Look at Instruction Set Architectures

Class, Variable, Constructor, Object, Method Questions

/* Copyright 2012 Robert C. Ilardi

1 Shyam sir JAVA Notes

Bugs in software. Using Static Analysis to Find Bugs. David Hovemeyer

Simple Data Source Crawler Plugin to Set the Document Title

Secure Method Calls by Instrumenting Bytecode with Aspects

LaboratoriodiProgrammazione III

Making Java more dynamic: runtime code generation for the JVM

Transcription:

Living in the Matrix with Bytecode Manipulation QCon NY 2014

Ashley Puls Senior Software Engineer New Relic, Inc.

Follow Along http://slidesha.re/1kzwcxr

Outline What is bytecode? Why manipulate bytecode? What frameworks can be used to manipulate bytecode? Examine two frameworks in depth

What is Java bytecode?.java file Java compiler (javac).class file Java Virtual Machine Native OS Java Runtime System Bytecode Verifier Class Loader Source: http://www.techlila.com/write-programs-linux/

What is Java bytecode?.java file Java compiler (javac).class file Java Virtual Machine Native OS Java Runtime System Bytecode Verifier Class Loader Source: http://www.techlila.com/write-programs-linux/

What is Java bytecode? Instruction set of the Java Virtual Machine.java file Java compiler (javac).class file Java Virtual Machine Native OS Java Runtime System Bytecode Verifier Class Loader Source: http://www.techlila.com/write-programs-linux/

Why learn about Java bytecode manipulation frameworks?

Why learn about Java bytecode manipulation frameworks? A bytecode manipulation framework is likely already on your stack Spring Hibernate Groovy Clojure Eclipse JRuby

Why learn about Java bytecode manipulation frameworks? A bytecode manipulation framework is likely already on your stack Spring Hibernate Groovy Clojure Eclipse JRuby Its really fun!!!

Why learn about Java bytecode manipulation frameworks? program analysis find bugs in code examine code complexity generate classes proxies remove access to certain APIs compiler for another language like Scala transform classes without Java source code profilers optimization and obfuscation additional logging

Why learn about Java bytecode manipulation frameworks? program analysis find bugs in code examine code complexity generate classes proxies remove access to certain APIs compiler for another language like Scala transform classes without Java source code profilers optimization and obfuscation additional logging

Logging Source: http://www.vforteachers.com/about_netsupport.htm

Logging Source: http://www.vforteachers.com/about_netsupport.htm, http://inzolo.com/blog/tutorials/bank-accounts

Logging Source: http://www.vforteachers.com/about_netsupport.htm, http://upload.wikimedia.org/wikipedia/commons/d/d3/49024-sos-atm.jpg

Logging Audit Log: a message should be logged every time an important method is called Source: http://www.vforteachers.com/about_netsupport.htm

Logging Audit Log: a message should be logged every time an important method is called log important input parameters to the method Source: http://www.vforteachers.com/about_netsupport.htm

Logging public class BankTransactions { public static void main(string[] args) { BankTransactions bank = new BankTransactions(); // login and add i dollars to account for (int i = 0; i < 100; i++) { String accountid = "account" + i; bank.login("password", accountid, "Ashley"); bank.unimportantprocessing(accountid); bank.finalizetransaction(accountid, Double.valueOf(i)); System.out.println( Transactions completed );

Logging public class BankTransactions { public static void main(string[] args) { BankTransactions bank = new BankTransactions(); // login and add i dollars to each account for (int i = 0; i < 100; i++) { String accountid = "account" + i; bank.login("password", accountid, "Ashley"); bank.unimportantprocessing(accountid); bank.finalizetransaction(accountid, Double.valueOf(i)); System.out.println( Transactions completed );

Logging /** * A method annotation which should be used to indicate important methods whose * invocations should be logged. * */ public @interface ImportantLog { /** * The method parameter indexes whose values should be logged. For example, * if we have the method hello(int parama, int paramb, int paramc), and we * wanted to log the values of parama and paramc, then fields would be ["0", * "2"]. If we only want to log the value of paramb, then fields would be * ["1"]. */ String[] fields();

Logging public void login(string password, String accountid, String username) { // login logic public void finalizetransaction(string accountid, Double moneytoadd) { // transaction logic

Logging @ImportantLog(fields = { "1", "2" ) public void login(string password, String accountid, String username) { // login logic @ImportantLog(fields = { "0", "1" ) public void finalizetransaction(string accountid, Double moneytoadd) { // transaction logic

Logging @ImportantLog(fields = { "1", "2" ) public void login(string password, String accountid, String username) { // login logic A call was made to method "login" on class "com/example/qcon/mains/banktransactions". Important params: Index 1 value: ${accountid Index 2 value: ${username @ImportantLog(fields = { "0", "1" ) public void finalizetransaction(string accountid, Double moneytoadd) { // transaction logic

Logging @ImportantLog(fields = { "1", "2" ) public void login(string password, String accountid, String username) { // login logic A call was made to method "login" on class "com/example/qcon/mains/banktransactions". Important params: Index 1 value: ${accountid Index 2 value: ${username @ImportantLog(fields = { "0", "1" ) public void finalizetransaction(string accountid, Double moneytoadd) { // transaction logic A call was made to method "finalizetransaction" on class "com/example/qcon/ mains/banktransactions". Important params: Index 0 value: ${accountid Index 1 value: ${moneytoadd

Java Agent Ability to modify the bytecode without modifying the application source code Added in Java 1.5 Docs: http://docs.oracle.com/javase/8/docs/api/ java/lang/instrument/package-summary.html

Typical Java Process java com/example/qcon/mains/banktransactions JVM Classloader BankTransactions.class public static void main(string[] args) Transactions completed

Java Agent java -javaagent:/path/to/agent.jar com/example/qcon/mains/ BankTransactions Agent JVM

Java Agent java -javaagent:/path/to/agent.jar com/example/qcon/mains/ BankTransactions JVM

Java Agent java -javaagent:/path/to/agent.jar com/example/qcon/mains/ BankTransactions Agent JVM

Java Agent java -javaagent:/path/to/agent.jar com/example/qcon/mains/ BankTransactions Agent Agent.class void premain(string agentargs, Instrumentation inst) JVM 1. call Agent premain in manifest 2. JVM registers my transformer MyTransformer.class byte[] transform(..., byte[] banktransbytes)

Java Agent java -javaagent:/path/to/agent.jar com/example/qcon/mains/ BankTransactions Agent Agent.class void premain(string agentargs, Instrumentation inst) MyTransformer.class byte[] transform(..., byte[] banktransbytes) JVM 1. call Agent premain in manifest 2. JVM registers my transformer 3. Give BankTransactions bytes to MyTransformer 4. MyTransformer provides bytes to load

Java Agent java -javaagent:/path/to/agent.jar com/example/qcon/mains/ BankTransactions Agent Agent.class void premain(string agentargs, Instrumentation inst) MyTransformer.class byte[] transform(..., byte[] banktransbytes) JVM 1. call Agent premain in manifest 2. JVM registers my transformer 3. Give BankTransactions bytes to MyTransformer 4. MyTransformer provides bytes to load BankTransactions.class void main(string[] args) 5. BankTransactions loaded and main runs Transactions completed

lets code Manifest: Premain-Class: com.example.qcon.agent.agent

lets code Manifest: Premain-Class: com.example.qcon.agent.agent package com.example.qcon.agent; public class Agent { public static void premain(string agentargs, Instrumentation inst) { System.out.println("Starting the agent"); inst.addtransformer(new ImportantLogClassTransformer());

lets code Manifest: Premain-Class: com.example.qcon.agent.agent package com.example.qcon.agent; public class Agent { public static void premain(string agentargs, Instrumentation inst) { System.out.println("Starting the agent"); inst.addtransformer(new ImportantLogClassTransformer());

lets code package com.example.qcon.agent; import java.lang.instrument.classfiletransformer; import java.lang.instrument.illegalclassformatexception; public class ImportantLogClassTransformer implements ClassFileTransformer { public byte[] transform(classloader loader, String classname, Class classbeingredefined, ProtectionDomain protectiondomain, byte[] classfilebuffer) throws IllegalClassFormatException { //TODO return null;

lets code package com.example.qcon.agent; import java.lang.instrument.classfiletransformer; import java.lang.instrument.illegalclassformatexception; public class ImportantLogClassTransformer implements ClassFileTransformer { public byte[] transform(classloader loader, String classname, Class classbeingredefined, ProtectionDomain protectiondomain, byte[] classfilebuffer) throws IllegalClassFormatException { //TODO return null;

lets code package com.example.qcon.agent; import java.lang.instrument.classfiletransformer; import java.lang.instrument.illegalclassformatexception; public class ImportantLogClassTransformer implements ClassFileTransformer { public byte[] transform(classloader loader, String classname, Class classbeingredefined, ProtectionDomain protectiondomain, byte[] classfilebuffer) throws IllegalClassFormatException { System.out.println(" Loading class: " + classname); return null;

lets code Starting the agent Loading class: java/lang/invoke/methodhandleimpl Loading class: java/lang/invoke/membername$factory Loading class: java/lang/invoke/lambdaform$namedfunction Loading class: java/lang/invoke/methodtype$concurrentweakinternset Loading class: java/lang/invoke/methodhandlestatics Loading class: java/lang/invoke/methodhandlestatics$1 Loading class: java/lang/invoke/methodtypeform Loading class: java/lang/invoke/invokers Loading class: java/lang/invoke/methodtype$concurrentweakinternset$weakentry Loading class: java/lang/void Loading class: java/lang/illegalaccessexception Loading class: sun/misc/postvminithook Loading class: sun/launcher/launcherhelper Loading class: java/util/concurrent/concurrenthashmap$forwardingnode Loading class: sun/misc/urlclasspath$fileloader$1 Loading class: com/example/qcon/mains/banktransactions Loading class: sun/launcher/launcherhelper$fxhelper

lets code package com.example.qcon.agent; import java.lang.instrument.classfiletransformer; import java.lang.instrument.illegalclassformatexception; public class ImportantLogClassTransformer implements ClassFileTransformer { public byte[] transform(classloader loader, String classname, Class classbeingredefined, ProtectionDomain protectiondomain, byte[] classfilebuffer) throws IllegalClassFormatException { System.out.println(" Loading class: " + classname); return null;

lets code package com.example.qcon.agent; import java.lang.instrument.classfiletransformer; import java.lang.instrument.illegalclassformatexception; public class ImportantLogClassTransformer implements ClassFileTransformer { public byte[] transform(classloader loader, String classname, Class classbeingredefined, ProtectionDomain protectiondomain, byte[] classfilebuffer) throws IllegalClassFormatException { // manipulate the bytes here and then return them return null;

Bytecode Manipulation Frameworks ASM: http://asm.ow2.org/ BCEL: http://commons.apache.org/proper/commons-bcel/ CGLib: https://github.com/cglib/cglib Javassist: http://www.csg.ci.i.u-tokyo.ac.jp/~chiba/javassist/ Serp: http://serp.sourceforge.net/ Cojen: https://github.com/cojen/cojen/wiki Soot: http://www.sable.mcgill.ca/soot/

Bytecode Manipulation Frameworks ASM: http://asm.ow2.org/ BCEL: http://commons.apache.org/proper/commons-bcel/ CGLib: https://github.com/cglib/cglib Javassist: http://www.csg.ci.i.u-tokyo.ac.jp/~chiba/javassist/ Serp: http://serp.sourceforge.net/ Cojen: https://github.com/cojen/cojen/wiki Soot: http://www.sable.mcgill.ca/soot/

Javassist Java Programming Assistant Subproject of JBoss Actively updated Version 3.18.2 released May 2014 Source code: https://github.com/jboss-javassist/ javassist Documentation: http://www.csg.ci.i.u-tokyo.ac.jp/ ~chiba/javassist/tutorial/tutorial.html

Javassist CtClass: represents a compile time class ArrayList HashMap

Javassist CtClass: represents a compile time class ClassPool: container of CtClass objects ArrayList HashMap

Javassist CtClass: represents a compile time class ClassPool: container of CtClass objects ArrayList ArrayList HashMap HashMap

Javassist CtClass: represents a compile time class ClassPool: container of CtClass objects ArrayList ArrayList HashMap HashMap BankTrans

Javassist CtClass: represents a compile time class ClassPool: container of CtClass objects ByteArrayClassPath: converts a byte[] to a CtClass ArrayList ArrayList HashMap HashMap BankTrans

Javassist CtClass: represents a compile time class ClassPool: container of CtClass objects ByteArrayClassPath: converts a byte[] to a CtClass Classpool.getDefault().insertClassPath(new ByteArrayClassPath( BankTrans, classbytes); ArrayList ArrayList HashMap HashMap BankTrans

Javassist CtClass: represents a compile time class ClassPool: container of CtClass objects ByteArrayClassPath: converts a byte[] to a CtClass Classpool.getDefault().insertClassPath(new ByteArrayClassPath( BankTrans, classbytes); ArrayList ArrayList HashMap HashMap BankTrans BankTrans

Javassist CtClass: represents a compile time class ClassPool: container of CtClass objects ByteArrayClassPath: converts a byte[] to a CtClass Classpool.getDefault().insertClassPath(new ByteArrayClassPath( BankTrans, classbytes); CtClass bank = Classpool.getDefault().get( BankTrans ); ArrayList ArrayList HashMap HashMap BankTrans BankTrans

Javassist CtClass: represents a compile time class ClassPool: container of CtClass objects ByteArrayClassPath: converts a byte[] to a CtClass Classpool.getDefault().insertClassPath(new ByteArrayClassPath( BankTrans, classbytes); CtClass bank = Classpool.getDefault().get( BankTrans ); ArrayList ArrayList HashMap HashMap BankTrans BankTrans BankTrans

Javassist CtClass: represents a compile time class BankTrans

Javassist CtClass: represents a compile time class CtConstructor: represents a compile time constructor BankTrans init

Javassist CtClass: represents a compile time class CtConstructor: represents a compile time constructor CtMethod: represents a compile time class BankTrans init login process

Javassist CtClass: represents a compile time class CtConstructor: represents a compile time constructor CtMethod: represents a compile time method CtMethod loginmethod = loginctclass.getdeclaredmethod( login ); BankTrans init login process

Javassist CtClass: represents a compile time class CtConstructor: represents a compile time constructor CtMethod: represents a compile time method CtMethod loginmethod = loginctclass.getdeclaredmethod( login ); BankTrans init login process login

Javassist CtMethod: represents a compile time method login

Javassist CtMethod: represents a compile time method Modifications: login

Javassist CtMethod: represents a compile time method Modifications: putmethod.insertbefore( System.out.println(\ before method\ ); ); putmethod.insertafter( System.out.println(\ after method\ ); ); login

Javassist CtMethod: represents a compile time method Modifications: putmethod.insertbefore( System.out.println(\ before method\ ); ); putmethod.insertafter( System.out.println(\ after method\ ); ); login System.out.println( before method ); System.out.println( after method );

Javassist CtMethod: represents a compile time method Modifications: CtClass exceptiontype = classpool.get( java.io.ioexception ); putmethod.addcatch( {System.out.println($e); throw $e, exceptiontype); login

Javassist CtMethod: represents a compile time method Modifications: CtClass exceptiontype = classpool.get( java.io.ioexception ); putmethod.addcatch( {System.out.println($e); throw $e, exceptiontype); login System.out.println( exception ); throw exception;

Javassist CtMethod: represents a compile time method Modifications: putmethod.insertat(5, true, System.out.println(\ hello\ ); ); login

Javassist CtMethod: represents a compile time method Modifications: putmethod.insertat(5, true, System.out.println(\ hello\ ); ); login System.out.println( hello );

Javassist CtMethod: represents a compile time method Modifications: insertbefore insertafter insertat addcatch login System.out.println( before method ); System.out.println( hello ); System.out.println( exception ); throw exception; System.out.println( after method );

Javassist CtMethod: represents a compile time method Modifications: insertbefore insertafter insertat addcatch Freeze Class: login System.out.println( before method ); System.out.println( hello ); System.out.println( exception ); throw exception; System.out.println( after method );

Javassist CtMethod: represents a compile time method Modifications: insertbefore insertafter insertat addcatch Freeze Class: writefile() toclass() tobytecode() login System.out.println( before method ); System.out.println( hello ); System.out.println( exception ); throw exception; System.out.println( after method );

lets code

lets code package com.example.qcon.agent; import java.lang.instrument.classfiletransformer; import java.lang.instrument.illegalclassformatexception; public class ImportantLogClassTransformer implements ClassFileTransformer { public byte[] transform(classloader loader, String classname, Class classbeingredefined, ProtectionDomain protectiondomain, byte[] classfilebuffer) throws IllegalClassFormatException { //TODO return null;

lets code package com.example.qcon.agent; import java.lang.instrument.classfiletransformer; import java.lang.instrument.illegalclassformatexception; public class ImportantLogClassTransformer implements ClassFileTransformer { public byte[] transform(classloader loader, String classname, Class classbeingredefined, ProtectionDomain protectiondomain, byte[] classfilebuffer) throws IllegalClassFormatException { //TODO return null;

lets code public byte[] transform(classloader loader, String classname, Class classbeingredefined, ProtectionDomain protectiondomain, byte[] cfbuffer) throws IllegalClassFormatException { 1. convert byte array to a ct class object 2. check each method of ct class for annotation @ImportantLog 3. if @ImportantLog annotation present on method, then a. get important method parameter indexes b. add logging statement to beginning of the method return null;

lets code public byte[] transform(classloader loader, String classname, Class classbeingredefined, ProtectionDomain protectiondomain, byte[] cfbuffer) throws IllegalClassFormatException { pool.insertclasspath(new ByteArrayClassPath(className, classfilebuffer)); CtClass cclass = pool.get(classname.replaceall("/", ".")); 2. check each method of ct class for annotation @ImportantLog 3. if @ImportantLog annotation present on method, then a. get important method parameter indexes b. add logging statement to beginning of the method return null;

lets code public byte[] transform(classloader loader, String classname, Class classbeingredefined, ProtectionDomain protectiondomain, byte[] cfbuffer) throws IllegalClassFormatException { pool.insertclasspath(new ByteArrayClassPath(className, classfilebuffer)); CtClass cclass = pool.get(classname.replaceall("/", ".")); if (!cclass.isfrozen()) { for (CtMethod currentmethod : cclass.getdeclaredmethods()) { Annotation annotation = getannotation(currentmethod); 3. if @ImportantLog annotation present on method, then a. get important method parameter indexes b. add logging statement to beginning of the method return null;

lets code public byte[] transform(classloader loader, String classname, Class classbeingredefined, ProtectionDomain protectiondomain, byte[] cfbuffer) throws IllegalClassFormatException { pool.insertclasspath(new ByteArrayClassPath(className, classfilebuffer)); CtClass cclass = pool.get(classname.replaceall("/", ".")); if (!cclass.isfrozen()) { for (CtMethod currentmethod : cclass.getdeclaredmethods()) { Annotation annotation = getannotation(currentmethod); if (annotation!= null) { List<String> parameterindexes = getparamindexes(annotation); return null; b. add logging statement to beginning of the method

lets code public byte[] transform(classloader loader, String classname, Class classbeingredefined, ProtectionDomain protectiondomain, byte[] cfbuffer) throws IllegalClassFormatException { pool.insertclasspath(new ByteArrayClassPath(className, classfilebuffer)); CtClass cclass = pool.get(classname.replaceall("/", ".")); if (!cclass.isfrozen()) { for (CtMethod currentmethod : cclass.getdeclaredmethods()) { Annotation annotation = getannotation(currentmethod); if (annotation!= null) { List<String> parameterindexes = getparamindexes(annotation); currentmethod.insertbefore(createjavastring( currentmethod, classname, parameterindexes)); return cclass.tobytecode(); return null;

lets code public byte[] transform(classloader loader, String classname, Class classbeingredefined, ProtectionDomain protectiondomain, byte[] cfbuffer) throws IllegalClassFormatException { pool.insertclasspath(new ByteArrayClassPath(className, classfilebuffer)); CtClass cclass = pool.get(classname.replaceall("/", ".")); if (!cclass.isfrozen()) { for (CtMethod currentmethod : cclass.getdeclaredmethods()) { Annotation annotation = getannotation(currentmethod); if (annotation!= null) { List<String> parameterindexes = getparamindexes(annotation); currentmethod.insertbefore(createjavastring( currentmethod, classname, parameterindexes)); return cclass.tobytecode(); return null;

lets code private Annotation getannotation(ctmethod method) { MethodInfo minfo = method.getmethodinfo(); // the attribute we are looking for is a runtime invisible attribute // use Retention(RetentionPolicy.RUNTIME) on the annotation to make it // visible at runtime AnnotationsAttribute attinfo = (AnnotationsAttribute) minfo.getattribute(annotationsattribute.invisibletag); if (attinfo!= null) { // this is the type name meaning use dots instead of slashes return attinfo.getannotation("com.example.qcon.mains.importantlog"); return null;

lets code public byte[] transform(classloader loader, String classname, Class classbeingredefined, ProtectionDomain protectiondomain, byte[] cfbuffer) throws IllegalClassFormatException { pool.insertclasspath(new ByteArrayClassPath(className, classfilebuffer)); CtClass cclass = pool.get(classname.replaceall("/", ".")); if (!cclass.isfrozen()) { for (CtMethod currentmethod : cclass.getdeclaredmethods()) { Annotation annotation = getannotation(currentmethod); if (annotation!= null) { List<String> parameterindexes = getparamindexes(annotation); currentmethod.insertbefore(createjavastring( currentmethod, classname, parameterindexes)); return cclass.tobytecode(); return null;

lets code private List<String> getparamindexes(annotation annotation) { ArrayMemberValue fields = (ArrayMemberValue) annotation.getmembervalue( fields ); if (fields!= null) { MemberValue[] values = (MemberValue[]) fields.getvalue(); List<String> parameterindexes = new ArrayList<String>(); for (MemberValue val : values) { parameterindexes.add(((stringmembervalue) val).getvalue()); return parameterindexes; return Collections.emptyList();

lets code public byte[] transform(classloader loader, String classname, Class classbeingredefined, ProtectionDomain protectiondomain, byte[] cfbuffer) throws IllegalClassFormatException { pool.insertclasspath(new ByteArrayClassPath(className, classfilebuffer)); CtClass cclass = pool.get(classname.replaceall("/", ".")); if (!cclass.isfrozen()) { for (CtMethod currentmethod : cclass.getdeclaredmethods()) { Annotation annotation = getannotation(currentmethod); if (annotation!= null) { List<String> parameterindexes = getparamindexes(annotation); currentmethod.insertbefore(createjavastring( currentmethod, classname, parameterindexes)); return cclass.tobytecode(); return null;

lets code private String createjavastring(ctmethod currentmethod, String classname, List<String> indexparameters) { StringBuilder sb = new StringBuilder(); sb.append("{stringbuilder sb = new StringBuilder"); sb.append("(\"a call was made to method '\");"); sb.append("sb.append(\""); sb.append(currentmethod.getname()); sb.append("\");sb.append(\"' on class '\");"); sb.append("sb.append(\""); sb.append(classname); sb.append("\");sb.append(\"'.\");"); sb.append("sb.append(\"\\n Important params:\");");...

lets code private String createjavastring(ctmethod currentmethod, String classname, List<String> indexparameters) { StringBuilder sb = new StringBuilder(); sb.append("{stringbuilder sb = new StringBuilder"); sb.append("(\"a call was made to method '\");"); sb.append("sb.append(\""); sb.append(currentmethod.getname()); sb.append("\");sb.append(\"' on class '\");"); sb.append("sb.append(\""); sb.append(classname); sb.append("\");sb.append(\"'.\");"); sb.append("sb.append(\"\\n Important params:\");");... Put multiple statements inside brackets {

lets code private String createjavastring(ctmethod currentmethod, String classname, List<String> indexparameters) {... for (String index : indexparameters) { try { int localvar = Integer.parseInt(index) + 1; sb.append("sb.append(\"\\n Index: \");"); sb.append("sb.append(\""); sb.append(index); sb.append("\");sb.append(\" value: \");"); sb.append("sb.append($" + localvar + ");"); catch (NumberFormatException e) { e.printstacktrace(); sb.append("system.out.println(sb.tostring());"); return sb.tostring();

lets code private String createjavastring(ctmethod currentmethod, String classname, List<String> indexparameters) {... for (String index : indexparameters) { try { int localvar = Integer.parseInt(index) + 1; sb.append("sb.append(\"\\n Index: \");"); sb.append("sb.append(\""); sb.append(index); sb.append("\");sb.append(\" value: \");"); sb.append("sb.append($" + localvar + ");"); catch (NumberFormatException e) { e.printstacktrace(); sb.append("system.out.println(sb.tostring());"); return sb.tostring(); $0, $1, $2,... can be used to access this and the actual method parameters

lets code public byte[] transform(classloader loader, String classname, Class classbeingredefined, ProtectionDomain protectiondomain, byte[] cfbuffer) throws IllegalClassFormatException { pool.insertclasspath(new ByteArrayClassPath(className, classfilebuffer)); CtClass cclass = pool.get(classname.replaceall("/", ".")); if (!cclass.isfrozen()) { for (CtMethod currentmethod : cclass.getdeclaredmethods()) { Annotation annotation = getannotation(currentmethod); if (annotation!= null) { List<String> parameterindexes = getparamindexes(annotation); currentmethod.insertbefore(createjavastring( currentmethod, classname, parameterindexes)); return cclass.tobytecode(); return null;

Javassist Positives Simplicity Do not have to write actual bytecode Decent documentation Negatives Generally slower than ASM Less functionality

ASM Released in Open Source in 2002 Actively updated Version 5.0.3 released May 2014 Two ASM libraries Event based (SAX like) Visitor design pattern Object based (DOM like) Documentation: http://download.forge.objectweb.org/ asm/asm4-guide.pdf

ASM Source: http://download.forge.objectweb.org/asm/asm4-guide.pdf

ASM Source: http://download.forge.objectweb.org/asm/asm4-guide.pdf

ASM Source: http://download.forge.objectweb.org/asm/asm4-guide.pdf

ASM ClassReader: given a byte[], parses a compiled class ClassReader: event producer ClassVisitor: event filter ClassWriter: event consumer BankTrans

ASM ClassReader: given a byte[], parses a compiled class ClassReader: event producer ClassVisitor: event filter ClassWriter: event consumer BankTrans visitfield visitmethod

ASM ClassReader: given a byte[], parses a compiled class ClassVisitor: delegates class events, event filter visitattribute visitfield visitmethod visitinnerclass visitend ClassReader: event producer ClassVisitor: event filter ClassWriter: event consumer BankTrans visitfield visitmethod

ASM ClassReader: given a byte[], parses a compiled class ClassVisitor: delegates class events, event filter visitattribute visitfield visitmethod visitinnerclass visitend ClassWriter: produces output byte[] ClassReader: event producer ClassVisitor: event filter ClassWriter: event consumer BankTrans visitfield BankTrans visitmethod

lets code package com.example.qcon.agent; import java.lang.instrument.instrumentation; public class Agent { public static void premain(string agentargs, Instrumentation inst) { System.out.println("Starting the agent"); inst.addtransformer(new ImportantLogClassTransformer());

lets code package com.example.qcon.agent; import java.lang.instrument.instrumentation; public class Agent { public static void premain(string agentargs, Instrumentation inst) { System.out.println("Starting the agent"); inst.addtransformer(new ImportantLogClassTransformer());

lets code package com.example.qcon.agent; import java.lang.instrument.classfiletransformer; import java.lang.instrument.illegalclassformatexception; public class ImportantLogClassTransformer implements ClassFileTransformer { public byte[] transform(classloader loader, String classname, Class classbeingredefined, ProtectionDomain protectiondomain, byte[] classfilebuffer) throws IllegalClassFormatException { //TODO return null;

lets code package com.example.qcon.agent; import java.lang.instrument.classfiletransformer; import java.lang.instrument.illegalclassformatexception; public class ImportantLogClassTransformer implements ClassFileTransformer { public byte[] transform(classloader loader, String classname, Class classbeingredefined, ProtectionDomain protectiondomain, byte[] classfilebuffer) throws IllegalClassFormatException { //TODO return null;

lets code public byte[] transform(classloader loader, String classname, Class<?> classbeingredefined, ProtectionDomain protectiondomain, byte[] classfilebuffer) throws IllegalClassFormatException { ClassReader cr = new ClassReader(classfileBuffer); ClassWriter cw = new ClassWriter(cr, ClassWriter.COMPUTE_FRAMES); cr.accept(cw, 0); return cw.tobytearray();

lets code public byte[] transform(classloader loader, String classname, Class<?> classbeingredefined, ProtectionDomain protectiondomain, byte[] classfilebuffer) throws IllegalClassFormatException { ClassReader cr = new ClassReader(classfileBuffer); ClassWriter cw = new ClassWriter(cr, ClassWriter.COMPUTE_FRAMES); ClassVisitor cv = new LogMethodClassVisitor(cw, classname); cr.accept(cv, 0); return cw.tobytearray();

ASM ClassReader: given a byte[], parses a compiled class ClassVisitor: delegates class events, event filter visitattribute visitfield visitmethod visitinnerclass visitend ClassWriter: produces output byte[] BankTrans ClassReader: event producer LogMethod ClassVisitor: event filter visitmethod ClassWriter: event consumer BankTrans

ASM Source: http://download.forge.objectweb.org/asm/asm4-guide.pdf

lets code public class LogMethodClassVisitor extends ClassVisitor { private String classname; public LogMethodIfAnnotationVisitor(ClassVisitor cv, String pclassname) { super(opcodes.asm5, cv); classname = pclassname; @Override public MethodVisitor visitmethod(int access, String name, String desc, String signature, String[] exceptions) { // put our logic in here

lets code public class LogMethodClassVisitor extends ClassVisitor { private String classname; public LogMethodIfAnnotationVisitor(ClassVisitor cv, String pclassname) { super(opcodes.asm5, cv); classname = pclassname; @Override public MethodVisitor visitmethod(int access, String name, String desc, String signature, String[] exceptions) { // put our logic in here

ASM Source: http://download.forge.objectweb.org/asm/asm4-guide.pdf

ASM Source: http://download.forge.objectweb.org/asm/asm4-guide.pdf

ASM Source: http://download.forge.objectweb.org/asm/asm4-guide.pdf

ASM MethodVisitor: method event filter visitannotation visitend visitcode visittrycatchblock MethodVisitor MethodVisitor login method visitor visitattribute visitcode login method visitor visitend

ASM Source: http://download.forge.objectweb.org/asm/asm4-guide.pdf

lets code public class LogMethodClassVisitor extends ClassVisitor { private String classname; public LogMethodIfAnnotationVisitor(ClassVisitor cv, String pclassname) { super(opcodes.asm5, cv); classname = pclassname; @Override public MethodVisitor visitmethod(int access, String name, String desc, String signature, String[] exceptions) { MethodVisitor mv = super.visitmethod(access, name, desc, signature, exceptions); return new PrintMessageMethodVisitor(mv, name, classname);

lets code public class PrintMessageMethodVisitor extends MethodVisitor { private String methodname; private String classname; private boolean isannotationpresent; private List<String> parameterindexes; public PrintMessageMethodVisitor(MethodVisitor mv, String methodname, String classname) { @Override public AnnotationVisitor visitannotation(string desc, boolean visible) { @Override public void visitcode() {

lets code public class PrintMessageMethodVisitor extends MethodVisitor { private String methodname; private String classname; private boolean isannotationpresent; private List<String> parameterindexes; public PrintMessageMethodVisitor(MethodVisitor mv, String methodname, String classname) { // initialize instance variables @Override public AnnotationVisitor visitannotation(string desc, boolean visible) { 1. check method for annotation @ImportantLog 2. if annotation present, then get important method param indexes @Override public void visitcode() { 3. if annotation present, add logging to beginning of the method

lets code public AnnotationVisitor visitannotation(string desc, boolean visible) { if ("Lcom/example/qcon/mains/ImportantLog;".equals(desc)) { isannotationpresent = true; return new AnnotationVisitor(Opcodes.ASM5, super.visitannotation(desc, visible)) { public AnnotationVisitor visitarray(string name) { if ( fields.equals(name)) { return new AnnotationVisitor(Opcodes.ASM5, super.visitarray(name)) { public void visit(string name, Object value) { parameterindexes.add((string) value); super.visit(name, value); ; else { return super.visitarray(name); ; return super.visitannotation(desc, visible);

lets code public class PrintMessageMethodVisitor extends MethodVisitor { private String methodname; private String classname; private boolean isannotationpresent; private List<String> parameterindexes; public PrintMessageMethodVisitor(MethodVisitor mv, String methodname, String classname) { // initialize instance variables @Override public AnnotationVisitor visitannotation(string desc, boolean visible) { 1. check method for annotation @ImportantLog 2. if annotation present, then get important method param indexes @Override public void visitcode() { 3. if annotation present, add logging statement to beginning of the method

lets code public void visitcode() { if (isannotationpresent) { // create string builder mv.visitfieldinsn(opcodes.getstatic, "java/lang/system", "out", "Ljava/io/PrintStream;"); mv.visittypeinsn(opcodes.new, "java/lang/stringbuilder"); mv.visitinsn(opcodes.dup);... // add everything to the string builder mv.visitldcinsn("a call was made to method \""); mv.visitmethodinsn(opcodes.invokespecial, "java/lang/stringbuilder", "<init>", "(Ljava/lang/String;)V", false); mv.visitldcinsn(methodname); mv.visitmethodinsn(opcodes.invokevirtual, "java/lang/stringbuilder", "append", "(Ljava/lang/String;)Ljava/lang/StringBuilder;", false);

What is Java bytecode? use javap to examine the bytecode

What is Java bytecode? public class PrintMessage { private String methodname; private String classname; public PrintMessage(String mname, String cname) { methodname = mname; classname = cname; public void print() { StringBuilder sb = new StringBuilder(); sb.append("a call was made to method \""); sb.append(methodname); sb.append("\" on class \""); sb.append(classname); sb.append("\"."); System.out.println(sb.toString());

What is Java bytecode? javap -c bin/com/example/qcon/mains/printmessage

What is Java bytecode? javap -c bin/com/example/qcon/mains/printmessage public void print(); Code: 0: new #38 // class java/lang/stringbuilder 3: dup 4: invokespecial #40 // Method java/lang/ StringBuilder."<init>":()V 7: astore_1 8: aload_1 9: ldc #41 // String A call was made to method \" 11: invokevirtual #43 // Method java/lang/ StringBuilder.append:(Ljava/lang/String;)Ljava/lang/StringBuilder; 14: pop 15: aload_1 16: aload_0 17: getfield #14 // Field methodname:ljava/lang/string; 20: invokevirtual #43 // Method java/lang/ StringBuilder.append:(Ljava/lang/String;)Ljava/lang/StringBuilder;...

What is Java bytecode? each line represents one JVM instruction public void print(); Code: 0: new #38 // class java/lang/stringbuilder 3: dup 4: invokespecial #40 // Method java/lang/ StringBuilder."<init>":()V 7: astore_1 8: aload_1 9: ldc #41 // String A call was made to method \" 11: invokevirtual #43 // Method java/lang/ StringBuilder.append:(Ljava/lang/String;)Ljava/lang/StringBuilder; 14: pop 15: aload_1 16: aload_0 17: getfield #14 // Field methodname:ljava/lang/string; 20: invokevirtual #43 // Method java/lang/ StringBuilder.append:(Ljava/lang/String;)Ljava/lang/StringBuilder;...

What is Java bytecode? JVM instruction = 1 opcode with 0 or more operands public void print(); Code: 0: new #38 // class java/lang/stringbuilder 3: dup 4: invokespecial #40 // Method java/lang/ StringBuilder."<init>":()V 7: astore_1 8: aload_1 9: ldc #41 // String A call was made to method \" 11: invokevirtual #43 // Method java/lang/ StringBuilder.append:(Ljava/lang/String;)Ljava/lang/StringBuilder; 14: pop 15: aload_1 16: aload_0 17: getfield #14 // Field methodname:ljava/lang/string; 20: invokevirtual #43 // Method java/lang/ StringBuilder.append:(Ljava/lang/String;)Ljava/lang/StringBuilder;...

What is Java bytecode? Opcode load store lcd return invokevirtual invokespecial Description load a reference onto the stack from a local variable iload, lload, fload, dload, aload stores a value from the stack into a local variable istore, lstore, fstore, dstore, astore push a constant from a constant pool onto the stack method return instruction ireturn, lreturn, freturn, dreturn, areturn, return invoke an instance method on an object invokes an instance method requiring special handling such as an initialization method See the spec for more info: http://docs.oracle.com/javase/specs/jvms/se8/jvms8.pdf

What is Java bytecode? Opcode invokestatic athrow new newarray if<cond> dup Description invoke a class (static) method on a named class throws an exception create a new class create a new array branch if int comparison with 0 succeeds ifeq, ifne, iflt, ifge, ifgt, ifle duplicate the top operand stack value See the spec for more info: http://docs.oracle.com/javase/specs/jvms/se8/jvms8.pdf

What is Java bytecode?

What is Java bytecode?

What is Java bytecode?

lets code public void visitcode() { if (isannotationpresent) { // create string builder mv.visitfieldinsn(opcodes.getstatic, "java/lang/system", "out", "Ljava/io/PrintStream;"); mv.visittypeinsn(opcodes.new, "java/lang/stringbuilder"); mv.visitinsn(opcodes.dup);... // add everything to the string builder mv.visitldcinsn("a call was made to method \""); mv.visitmethodinsn(opcodes.invokespecial, "java/lang/stringbuilder", "<init>", "(Ljava/lang/String;)V", false); mv.visitldcinsn(methodname); mv.visitmethodinsn(opcodes.invokevirtual, "java/lang/stringbuilder", "append", "(Ljava/lang/String;)Ljava/lang/StringBuilder;", false);

ASM Positives Speed and Size Decent documentation Depth of Functionality Number of people using framework Negatives Need to write actual byte code longer developer ramp-up

Why learn about Java bytecode manipulation frameworks? program analysis find bugs in code examine code complexity generate classes proxies remove access to certain APIs compiler for another language like Scala transform classes without Java source code profilers optimization and obfuscation additional logging

Bytecode Manipulation Frameworks ASM: http://asm.ow2.org/ BCEL: http://commons.apache.org/proper/commons-bcel/ CGLib: https://github.com/cglib/cglib Javassist: http://www.csg.ci.i.u-tokyo.ac.jp/~chiba/javassist/ Serp: http://serp.sourceforge.net/ Cojen: https://github.com/cojen/cojen/wiki Soot: http://www.sable.mcgill.ca/soot/

Make bytecode manipulation frameworks your friend. They might save your job one day. Source: http://www.kulfoto.com/dog-pictures/15799/mans-best-friend

Questions? Source: http://alangregerman.typepad.com/.a/6a00d83516c0ad53ef017c3329aba8970b-800wi