WHO AM I.

Transcription

1

2 WHO AM I Christoph Engelbert (@noctarius2k) 8+ years of professional Java development Specialized to performance, GC, traffic topics Apache DirectMemory PMC Previous companies incl. Ubisoft and HRS Official Inhabitant of Hazelcastia CastMapR MapReduce for Hazelcast 3 Co-Author Java Off-heap JEP Proposal

3 TOPICS Java Memory Layout Definition of Off-Heap? Direct Memory Access? Why do we need Off-Heap Memory? Options for Off-Heap Storage Advantages and Disadvantages of (Off- )Heap Available Frameworks (Not so) Secret Future Small demonstration

4 JAVA MEMORY LAYOUT A FAST WALK-THROUGH

5 JAVA MEMORY LAYOUT (1/5)

6 JAVA MEMORY LAYOUT (2/5) YOUNG GENERATION - EDEN SPACE Only used for Object-Allocation TLAB¹ allocation available Affected by Minor and Major GC Objects moved to Survivor on Minor GC ¹ TLAB: Thread Local Allocation Buffer

7 JAVA MEMORY LAYOUT (3/5) YOUNG GENERATION - SURVIVOR SPACES Always one From-Space and one To-Space From-Space contains living objects on last GC Minor GC switches the Survivor Spaces Alive Objects are moved to To-Space Affected by Minor and Major GC Long living objects eventually moved to Tenured

8 JAVA MEMORY LAYOUT (4/5) TENURED SPACE Contains only long living objects Affected only by Major GC More objects means more GC spend time Concurrent object inspectation available Long Stop-The-World pauses possible Not optimal for big datasets

9 JAVA MEMORY LAYOUT (5/5) PERMGEN / META SPACE Class bytecodes Class metadata Runtime information Code Compilation Cache etc.

10 G1 (GARBAGE FIRST COLLECTOR) Generational GC Heap splitted into same sized regions Every region is either Eden, Survivor or Tenured space

11 DEFINITION OF OFF-HEAP?

12 DEFINITION OF OFF-HEAP? Off-Heap is a (continuously) self allocated, managed and freed direct memory region. It is not under control of the Java GarbageCollector and needs custom written allocation and cleanup of data-regions. Off-Heap can not be used for allocation of Java objects.

13 I'M DUKE SKYWALKER I'M HERE TO RESCUE YOU!

14 DIRECT MEMORY ACCESS? GIVE YOURSELF TO THE DARK SIDE

15 DIRECT MEMORY ACCESS? ARE YOU SERIOUS? Fast memory area Not affecting the GC Officially available since Java 1.4 Serialization overhead when storing objects Limited by -XX:MaxDirectMemorySize

16 WHY DO WE NEED OFF-HEAP MEMORY?

17 WHY DO WE NEED OFF-HEAP MEMORY? Storing of huge datasets Zero-Copy writes to channels Lower pressure on GC / less GC pauses Storage Space only limited by RAM Compact data representation possible IPC SharedMemory with Memory-Mapped-Files

18 OPTIONS FOR OFF-HEAP STORAGE

19 OPTIONS FOR OFF-HEAP STORAGE (1/3) JNI (Java Native Interface) int* buffer = ( int* ) malloc( 1024 * sizeof(int) ); if( buffer == NULL ) throwoutofmemoryexception(); for( int i = 0; i < 1024; i++ ) buffer[sizeof(int) * i] = i; free( buffer );

20 OPTIONS FOR OFF-HEAP STORAGE (2/3) (Direct)ByteBuffer ByteBuffer buffer = ByteBuffer.allocateDirect( 1024 * 4 ); for( int i = 0; i < 1024; i++) buffer.putint( i ); // buffer is automatically freed by GC

21 OPTIONS FOR OFF-HEAP STORAGE (3/3) sun.misc.unsafe Unsafe unsafe = tricktoretrieveunsafe(); long address = unsafe.allocatememory( 1024 * 4 ); for( int i = 0; i < 1024; i++ ) unsafe.putint( address + 4 * i, i ); unsafe.freememory( address );

22 ADVANTAGES AND DISADVANTAGES OF (OFF-)HEAP

23 ADVANTAGES ON-HEAP No messing with malloc Automatic Garbage Collection No need for Serialization

24 DISADVANTAGES ON-HEAP Slowdown on high allocation rate Big overhead on small objects (header) Heavily depending on GC Combination

25 ADVANTAGES OFF-HEAP No limitation* in size Compact data layout Zero-Copy socket read/write possible Frameworks available to manage memory

26 DISADVANTAGES OFF-HEAP Allocation is up to you Deallocation is up to you Data layout is up to you Serialization may be required JNI requires native library

27 AVAILABLE FRAMEWORKS

28 AVAILABLE FRAMEWORKS (1/5) Apache DirectMemory Terracotta BigMemory Hazelcast ElasticMemory MapDB.org etc.

29 AVAILABLE FRAMEWORKS (2/5) Apache DirectMemory CacheService<String, String> cacheservice = new DirectMemory<...>().setNumberOfBuffers( 10 ).newcacheservice(); cacheservice.put( "MyKey", "SomeValue" ); String value = cacheservice.get( "MyKey" );

30 AVAILABLE FRAMEWORKS (3/5) Terracotta BigMemory <ehcache xml:nonamespaceschemalocation="..." name="mycachedefinition"> <cache name="myoffheapcache" maxbyteslocaloffheap="2g" /> </ehcache> CacheManager cachemanager = new CacheManager(); Cache datastore = cachemanager.get( "MyOffheapCache" ); Element element = new Element( "MyKey", "SomeValue" ); datastore.put( element ); String value = (String) datastore.get( "MyKey" ).getobjectvalue();

31 AVAILABLE FRAMEWORKS (4/5) Hazelcast Offheap <hazelcast xml:nonamespaceschemalocation="..."> <map name="mymap"> <storage-type>offheap</storage-type> </map> </hazelcast> HazelcastInstance hz = Hazelcast.newInstance(); // Returns a IMap extends ConcurrentMap Map<String, String> map = hz.getmap( "MyMap" ); map.put( "MyKey", "SomeValue" ); String value = map.get( "MyKey" );

32 AVAILABLE FRAMEWORKS (5/5) MapDB.org DB db = DBMaker.newDirectMemoryDB().sizeLimit( 2 ).make(); // Returns a HTreeMap extends ConcurrentMap Map<String, String> map = db.createhashmap( "cache" ).make(); map.put( "MyKey", "SomeValue" ); String value = map.get( "MyKey" );

33 FURTHER INFORMATION AVAILABLE The Must-Read for Off-Heap PacketObject Description from IBM Compact Datastructures (Martin Thompson) C++ Like Performance for Serialization Tricks with Direct Memory Access in Java

34 THE OFF-HEAP JEP PROPOSAL MAY THE FORCE BE WITH YOU. THE FORCE IS STRONG WITH THIS ONE.

35 OFF-HEAP JEP PROPOSAL ByteBuffer like interface Fully 64-bit sizes and offsets Compare-And-Swap operations Volatile and ordered operations Optional bounding checks Supports Memory mapping Support Foreign Function Interface (FFI) JEP

36 OFF-HEAP JEP PROPOSAL - CODE EXAMPLE DISCLAIMER: PROVISIONAL API import javax.direct.*; BytesFactory factory = createbytesfactory(); Bytes bytes = factory.boundschecking( false ).deallocationchecks( false ).freeongc( false ).create( ByteOrder.LITTLE_ENDIAN, 1024 * 4 ); for( int i = 0; i < 1024; i++ ) bytes.putvolatileint( i ); bytes.release();

37 OFF-HEAP JEP PROPOSAL Proposal Discussion Group Proposal Text FFI JEP 191

38 THANK YOU! Images: Gnome Nebula Theme, KDE theme,