WHO AM I Christoph Engelbert (@noctarius2k) 8+ years of professional Java development Specialized to performance, GC, traffic topics Apache DirectMemory PMC Previous companies incl. Ubisoft and HRS Official Inhabitant of Hazelcastia CastMapR MapReduce for Hazelcast 3 Co-Author Java Off-heap JEP Proposal
TOPICS Java Memory Layout Definition of Off-Heap? Direct Memory Access? Why do we need Off-Heap Memory? Options for Off-Heap Storage Advantages and Disadvantages of (Off- )Heap Available Frameworks (Not so) Secret Future Small demonstration
JAVA MEMORY LAYOUT A FAST WALK-THROUGH
JAVA MEMORY LAYOUT (1/5)
JAVA MEMORY LAYOUT (2/5) YOUNG GENERATION - EDEN SPACE Only used for Object-Allocation TLAB¹ allocation available Affected by Minor and Major GC Objects moved to Survivor on Minor GC ¹ TLAB: Thread Local Allocation Buffer
JAVA MEMORY LAYOUT (3/5) YOUNG GENERATION - SURVIVOR SPACES Always one From-Space and one To-Space From-Space contains living objects on last GC Minor GC switches the Survivor Spaces Alive Objects are moved to To-Space Affected by Minor and Major GC Long living objects eventually moved to Tenured
JAVA MEMORY LAYOUT (4/5) TENURED SPACE Contains only long living objects Affected only by Major GC More objects means more GC spend time Concurrent object inspectation available Long Stop-The-World pauses possible Not optimal for big datasets
JAVA MEMORY LAYOUT (5/5) PERMGEN / META SPACE Class bytecodes Class metadata Runtime information Code Compilation Cache etc.
G1 (GARBAGE FIRST COLLECTOR) Generational GC Heap splitted into same sized regions Every region is either Eden, Survivor or Tenured space
DEFINITION OF OFF-HEAP?
DEFINITION OF OFF-HEAP? Off-Heap is a (continuously) self allocated, managed and freed direct memory region. It is not under control of the Java GarbageCollector and needs custom written allocation and cleanup of data-regions. Off-Heap can not be used for allocation of Java objects.
I'M DUKE SKYWALKER I'M HERE TO RESCUE YOU!
DIRECT MEMORY ACCESS? GIVE YOURSELF TO THE DARK SIDE
DIRECT MEMORY ACCESS? ARE YOU SERIOUS? Fast memory area Not affecting the GC Officially available since Java 1.4 Serialization overhead when storing objects Limited by -XX:MaxDirectMemorySize
WHY DO WE NEED OFF-HEAP MEMORY?
WHY DO WE NEED OFF-HEAP MEMORY? Storing of huge datasets Zero-Copy writes to channels Lower pressure on GC / less GC pauses Storage Space only limited by RAM Compact data representation possible IPC SharedMemory with Memory-Mapped-Files
OPTIONS FOR OFF-HEAP STORAGE
OPTIONS FOR OFF-HEAP STORAGE (1/3) JNI (Java Native Interface) int* buffer = ( int* ) malloc( 1024 * sizeof(int) ); if( buffer == NULL ) throwoutofmemoryexception(); for( int i = 0; i < 1024; i++ ) buffer[sizeof(int) * i] = i; free( buffer );
OPTIONS FOR OFF-HEAP STORAGE (2/3) (Direct)ByteBuffer ByteBuffer buffer = ByteBuffer.allocateDirect( 1024 * 4 ); for( int i = 0; i < 1024; i++) buffer.putint( i ); // buffer is automatically freed by GC
OPTIONS FOR OFF-HEAP STORAGE (3/3) sun.misc.unsafe Unsafe unsafe = tricktoretrieveunsafe(); long address = unsafe.allocatememory( 1024 * 4 ); for( int i = 0; i < 1024; i++ ) unsafe.putint( address + 4 * i, i ); unsafe.freememory( address );
ADVANTAGES AND DISADVANTAGES OF (OFF-)HEAP
ADVANTAGES ON-HEAP No messing with malloc Automatic Garbage Collection No need for Serialization
DISADVANTAGES ON-HEAP Slowdown on high allocation rate Big overhead on small objects (header) Heavily depending on GC Combination
ADVANTAGES OFF-HEAP No limitation* in size Compact data layout Zero-Copy socket read/write possible Frameworks available to manage memory
DISADVANTAGES OFF-HEAP Allocation is up to you Deallocation is up to you Data layout is up to you Serialization may be required JNI requires native library
AVAILABLE FRAMEWORKS
AVAILABLE FRAMEWORKS (1/5) Apache DirectMemory Terracotta BigMemory Hazelcast ElasticMemory MapDB.org etc.
AVAILABLE FRAMEWORKS (2/5) Apache DirectMemory CacheService<String, String> cacheservice = new DirectMemory<...>().setNumberOfBuffers( 10 ).newcacheservice(); cacheservice.put( "MyKey", "SomeValue" ); String value = cacheservice.get( "MyKey" );
AVAILABLE FRAMEWORKS (3/5) Terracotta BigMemory <ehcache xml:nonamespaceschemalocation="..." name="mycachedefinition"> <cache name="myoffheapcache" maxbyteslocaloffheap="2g" /> </ehcache> CacheManager cachemanager = new CacheManager(); Cache datastore = cachemanager.get( "MyOffheapCache" ); Element element = new Element( "MyKey", "SomeValue" ); datastore.put( element ); String value = (String) datastore.get( "MyKey" ).getobjectvalue();
AVAILABLE FRAMEWORKS (4/5) Hazelcast Offheap <hazelcast xml:nonamespaceschemalocation="..."> <map name="mymap"> <storage-type>offheap</storage-type> </map> </hazelcast> HazelcastInstance hz = Hazelcast.newInstance(); // Returns a IMap extends ConcurrentMap Map<String, String> map = hz.getmap( "MyMap" ); map.put( "MyKey", "SomeValue" ); String value = map.get( "MyKey" );
AVAILABLE FRAMEWORKS (5/5) MapDB.org DB db = DBMaker.newDirectMemoryDB().sizeLimit( 2 ).make(); // Returns a HTreeMap extends ConcurrentMap Map<String, String> map = db.createhashmap( "cache" ).make(); map.put( "MyKey", "SomeValue" ); String value = map.get( "MyKey" );
FURTHER INFORMATION AVAILABLE The Must-Read for Off-Heap http://bit.ly/must-read-off-heap PacketObject Description from IBM http://bit.ly/packet-objects Compact Datastructures (Martin Thompson) http://bit.ly/compact-datastructures C++ Like Performance for Serialization http://bit.ly/serialization-performance Tricks with Direct Memory Access in Java http://bit.ly/direct-memory-tricks
THE OFF-HEAP JEP PROPOSAL MAY THE FORCE BE WITH YOU. THE FORCE IS STRONG WITH THIS ONE.
OFF-HEAP JEP PROPOSAL ByteBuffer like interface Fully 64-bit sizes and offsets Compare-And-Swap operations Volatile and ordered operations Optional bounding checks Supports Memory mapping Support Foreign Function Interface (FFI) JEP
OFF-HEAP JEP PROPOSAL - CODE EXAMPLE DISCLAIMER: PROVISIONAL API import javax.direct.*; BytesFactory factory = createbytesfactory(); Bytes bytes = factory.boundschecking( false ).deallocationchecks( false ).freeongc( false ).create( ByteOrder.LITTLE_ENDIAN, 1024 * 4 ); for( int i = 0; i < 1024; i++ ) bytes.putvolatileint( i ); bytes.release();
OFF-HEAP JEP PROPOSAL Proposal Discussion Group http://bit.ly/offheap-ml Proposal Text http://bit.ly/proposal-text FFI JEP 191 http://bit.ly/ffi-jep
THANK YOU! ANY QUESTIONS? @noctarius2k @hazelcast http://www.sourceprojects.com http://github.com/noctarius Images: www.clipartist.info, Gnome Nebula Theme, KDE theme, www.grabsteine-klnt.de