Heap Off Memory WTF?? Reducing Heap memory stress

Transcription

1 Heap Off Memory WTF?? Reducing Heap memory stress

2 Abstract * Java memory fundamental * heap off memory principles * heap-off cache with Apache DirectMemory

3 /me Olivier Lamy * Open Source Apache Team * Apache Member : Maven, Archiva, Tomcat, DirectMemory, Build Infra, etc.. * Jenkins * Add bugs in various OpenSource projects * twitter.com/olamy olamy.blogspot.com

4 Java Memory Fundamental

5 Java Memory Fundamental The cool part * Automatic memory allocation (do you remember when you used malloc) * Garbage collector (GC) (no more free)

6 Java Memory Fundamental The bad part * stop-the-world mode * proportionnal to memory's size * unpredictable unresponsive application : complicated with tight SLA

7 Heap Off Memory Not in the Heap process Limited GC stop «Unlimited storage»

8 Heap Off Memory How to use that? Native libraries with JNA with nio ByteBuffer.allocate( size ) (not off heap) ByteBuffer.allocateDirect( size ) You can use sun.misc.unsafe but can cause issue and not portable

9 Heap Off Memory How to use that? You can manipulate only Buffer (ByteBuffer byte[] etc..)! So you must serialize/deserialize your datas Maximum size with -XX:MaxDirectMemorySize=

10 Our use case : Cache RAM : ns Network : ns SSD Disk : ns Disk : ns Cache must use memory!

11 On-Heap Cache Objects stored by reference (no de/serialisation) GC storm effect when refreshing/removing objects!

12 Off-Heap Cache de/serialisation overhead (hopefully some very performant libraries exist for that) Cache objects payload no more affecting GC

13 Solutions : * Terracotta BigMemory (off-heap storage on top of ehcache) * Infinispan (by Jboss) * Huge collections * Apache DirectMemory

14 Apache Direct Memory Goals : Apache Direct Memory is a multi layered cache implementation featuring off-heap memory storage to enable caching of java objects without degrading jvm performance.

15 Apache Direct Memory Joined Apache Incubator end developpers ATM Under development : Memory allocation service just rewrite APIs are subject to be changed and bugs to be found!

16 Design & principles ByteBuffer.allocateDirect is the foundation of the cache ByteBuffers are allocated in big chunk and splitted for internal use

17 Build on layers Design & principles CachingService : serialize object (pluggable) MemoryManagerService: Compute ByteBuffer access ByteBufferAllocatorService: Eventually deals with ByteBuffer

18 Merging Strategy ByteBuffers Allocation No memory wasted Suffers from fragmentation Need synchronization at de/allocation Fixed Size buffers allocation Memory wasted if size not correctly configured No fragmentation

19 Use case (1) Multi layers cache Most used objects are cached on heap, the rest off-heap (maybe overflow to disk) Sounds like ehcache with BigMemory Hard coded class to use : net.sf.ehcache.store.offheap.offheapstore So same package class name in Apache DirectMemory ( Demo

20 Use case (2) Cache Server «à la» memcache tru http(s) But with a REST Api and written in Java Client API available Demo

21 > PUT /dm/cache/bordeaux HTTP/1.1 > Content-Type:text/plain {"millesime":"2003","description":"so good so good"} < HTTP/ OK < X-DirectMemory-SerializeSize: 58 < Content-Length: 0 > GET /dm/cache/bordeaux HTTP/1.1 > Accept:text/plain < HTTP/ OK < Content-Type: text/plain < Content-Length: 51 {"millesime":"2003","description":"so good so good"} > DELETE /dm/cache/foo HTTP/1.1

22 Next Steps JSR 107? Benchmarks Components Integration (Cassandra, Tomcat etc..) Dynamic cache size modification Management/Monitoring

23 Links Demo project : Slides :

24 Thanks && Questions

25 Heap Off Memory WTF?? Reducing Heap memory stress

26 Abstract * Java memory fundamental * heap off memory principles * heap-off cache with Apache DirectMemory

27 /me Olivier Lamy * Open Source Apache Team * Apache Member : Maven, Archiva, Tomcat, DirectMemory, Build Infra, etc.. * Jenkins * Add bugs in various OpenSource projects * twitter.com/olamy olamy.blogspot.com

28 Java Memory Fundamental

29 Java Memory Fundamental The cool part * Automatic memory allocation (do you remember when you used malloc) * Garbage collector (GC) (no more free)

30 Java Memory Fundamental The bad part * stop-the-world mode * proportionnal to memory's size * unpredictable unresponsive application : complicated with tight SLA

31 Heap Off Memory Not in the Heap process Limited GC stop «Unlimited storage»

32 Heap Off Memory How to use that? Native libraries with JNA with nio ByteBuffer.allocate( size ) (not off heap) ByteBuffer.allocateDirect( size ) You can use sun.misc.unsafe but can cause issue and not portable

33 Heap Off Memory How to use that? You can manipulate only Buffer (ByteBuffer byte[] etc..)! So you must serialize/deserialize your datas Maximum size with -XX:MaxDirectMemorySize=

34 Our use case : Cache RAM : ns Network : ns SSD Disk : ns Disk : ns Cache must use memory!

35 On-Heap Cache Objects stored by reference (no de/serialisation) GC storm effect when refreshing/removing objects!

36 Off-Heap Cache de/serialisation overhead (hopefully some very performant libraries exist for that) Cache objects payload no more affecting GC

37 Solutions : * Terracotta BigMemory (off-heap storage on top of ehcache) * Infinispan (by Jboss) * Huge collections * Apache DirectMemory

38 Apache Direct Memory Goals : Apache Direct Memory is a multi layered cache implementation featuring off-heap memory storage to enable caching of java objects without degrading jvm performance.

39 Apache Direct Memory Joined Apache Incubator end developpers ATM Under development : Memory allocation service just rewrite APIs are subject to be changed and bugs to be found!

40 Design & principles ByteBuffer.allocateDirect is the foundation of the cache ByteBuffers are allocated in big chunk and splitted for internal use

41 Build on layers Design & principles CachingService : serialize object (pluggable) MemoryManagerService: Compute ByteBuffer access ByteBufferAllocatorService: Eventually deals with ByteBuffer

42 Merging Strategy ByteBuffers Allocation No memory wasted Suffers from fragmentation Need synchronization at de/allocation Fixed Size buffers allocation Memory wasted if size not correctly configured No fragmentation

43 Use case (1) Multi layers cache Most used objects are cached on heap, the rest off-heap (maybe overflow to disk) Sounds like ehcache with BigMemory Hard coded class to use : net.sf.ehcache.store.offheap.offheapstore So same package class name in Apache DirectMemory ( Demo

44 Use case (2) Cache Server «à la» memcache tru http(s) But with a REST Api and written in Java Client API available Demo

45 > PUT /dm/cache/bordeaux HTTP/1.1 > Content-Type:text/plain {"millesime":"2003","description":"so good so good"} < HTTP/ OK < X-DirectMemory-SerializeSize: 58 < Content-Length: 0 > GET /dm/cache/bordeaux HTTP/1.1 > Accept:text/plain < HTTP/ OK < Content-Type: text/plain < Content-Length: 51 {"millesime":"2003","description":"so good so good"} > DELETE /dm/cache/foo HTTP/1.1

46 Next Steps JSR 107? Benchmarks Components Integration (Cassandra, Tomcat etc..) Dynamic cache size modification Management/Monitoring

47 Links Demo project : Slides :

48 Thanks && Questions