Gemeinsam mehr erreichen.

Transcription

1 Gemeinsam mehr erreichen. Bring the process to the cached data in Oracle Coherence September 2015

2 Agenda Currrent Situation Coherence in the CAF What is Coherence? Characteristics of Coherence Data Grid Coherence Services Coherence Topologies Coherence Integration Coherence Entry Processors Case 1: Dedicate Storage Nodes Case 2: Locking Steps to create an agent Live Demo Conclusions

3 Current Situation 3

4 Coherence in the Cloud Application Foundation 4

5 What is Coherence? Oracle Coherence is an in-memory data grid solution that enables organizations to predictably scale mission-critical applications by providing fast access to frequently used data. (Oracle Coherence Datasheet) Enterprise Applications Real Time Clients Application Tier Data Services Coherence Data Grid Data Sources 5

6 Characteristics of the Coherence Data Grid Reliable: Universal: Guarantee Continous Operation Single view of data Fault Tolerance Single Management view Self-Diagnosis Healing Simple programming model Once and only once Processing Scalable: Data: Dynamically Expandable Caching No data loss Transaction Processing No interruption of service Analytics Event processing 6

7 Coherence Goals A fully coherent, single system image (SSI) Scalability for both read and write access Fast, transparent failover and failback Linear scalability for storage and processing No Single-Points-of-Failure (SPOFs) Cluster-wide locking and transactions 7

8 Coherence Services HTTP session management Grid agent invocation Database caching Coherence Distributed queries Object caching Event processing 8

9 How Does the Coherence Data Grid work? 1. Data stored and load balanced across a cluster of servers 2. Data automatically replicated to at least another cluster node for fault tolerence 3. Cluster nodes monitor the health of each other 4. Cluster nodes cooperate to check the state of an unresponsive node 5. Healthy cluster nodes assume the responsibilities of a failed node. No loss of service or data!! X 9

10 Cache Topology: Local Cache 10

11 Cache Topology: Replicated Cache 11

12 Cache Topology: Distributed (Partitioned) Cache 12

13 Cache Topology: Near Cache 13

14 Replicated Cache Optimistic Cache Partitioned Cache Near Cache (backed by partitioned cache ) Local Cache Topology Replicated Replicated Partitioned Cache Local + Partitioned Local Cache Read Performance Instant Instant Locally cached: Instant. remote network speed Locally cached: Instant. remote network speed Instant Fault Tolerance Extremely High Extremely High Configurable: Zero to Extremely High Configurable: Zero to Extremely High Zero Write Performance Fast Fast Extremely Fast Extremely Fast Instant Memory Usage (Per JVM) Data Size Data Size Data Size/#JVMs * Redundancy Local Cache + (Data Size / #JVMs) Data Size Memory Usage Total JVMs*DataSize JVMs*DataSize Redundancy*Data Size (Redundancy*DataSi ze)+ (Local Cache ) JVMs*DataSize Coherency Fully coherent Fully coherent Fully coherent Fully coherent n/a Locking Fully transactional none Fully transactional Fully transactional Fully transactional Typical Uses Metadata Red heavy caches with access afinity Read Write Caches Red heavy caches with access afinity Local data 14

15 Coherence Data Storage options On-heap The fastest option, though it can affect JVM garbage collection times. NIO RAM No impact on garbage collection, though it does require serialization/deserialization. NIO Disk Similar to NIO RAM, but using memory-mapped files. Basic Concepts Introduction to Coherence 1-3 Journal A combination of RAM storage and disk storage, optimized for solid state disks, that uses a journaling technique. Journal-based storage requires serialization/deserialization. File-based Uses a Berkeley Database JE storage system. 15

16 Coherence Integration Web Coherence Web Web Middelware Coherence API Events Agents CoQL Middelware Database GoldenGate Hot Cache Database 16

17 Coherence Entry Processors ( or Coherence Agents) Send the process to data vs Send data to process. Coherence Agent use a weak migration paradigm. Low level lock on entries that are being processed. Fast parallel execution. A Process A B B Application C Result C Logical Primary LocalStorage= true LocalStorage= false JVM 1 JVM 2 17

18 Case 1: Dedicated Storage Nodes 18

19 Case 2: Locking // *requires network access* if (cache.lock(key, 0)) { try { // *requires network access* Integer i = (Integer) cache.get(key); // *requires network access* cache.put(key, new Integer(i.intValue() + 1)); } finally { // *requires network access* cache.unlock(key); } } // end processing VS // *requires network access* cache.invoke(key, new MyCounterProcessor()); // end processing... public static class MyCounterProcessor extends AbstractProcessor { // this is executed on the node that owns the data, // no network access required public Object process(invocablemap.entry entry) { Integer i = (Integer) entry.getvalue(); entry.setvalue(new Integer(i.intValue() + 1)); return null; } } 19

20 Steps to create a Coherence agent 1. Create a Java Class that extends com.tangosol.util. processor.abstractprocessor and implements java.io.serializable 2. Override Object process(invocablemap.entry entry) and Map processall(set set) methods 3. Generate a serialversionuid for the class 4. Distribute the agent through the cluster. 20

21 Code Demo public class CoherenceAgent extends AbstractProcessor implements Serializable { private static final long serialversionuid = 16789L; public Map processall(set set) { StockInfo stockinfo=new StockInfo(); JAXBContext context = getinstance().getcontext(); Map values = new HashMap(); long frommils = fromdate.gettime(); long untilmils = untildate.gettime(); int entries=0; float value=0; for (Object obj : set) { InvocableMap.Entry entry = (InvocableMap.Entry) obj; byte[] bytes = (byte[]) entry.getvalue(); Node node = binxmltodocument(bytes); StockType stocktype = getstocktype(node, context); long stockmils = stocktype.gettimestamp().togregoriancalendar().gettimeinmillis(); if (stockmils < untilmils && stockmils frommils) { entries++; value= stocktype.getvalue() + value; } } stockinfo.setnumberentries(entries); if(entries==0){ entries++; } stockinfo.setaveragevalue(value/entries); values.put("1", stockinfo); return values; 21

22 Live demo 22

23 Performance Comparison Agent Heap Coherence 1 Heap Coherence 2 Performance CPU Coherence 1 CPU Coherence mb 370mb ,4 22% mb 280mb ,4 17, mb 180mb 500 0, mb 170mb 220 0,4 3, mb 80mb 110 0,3 2, mb 23mb 31 0,2 0,9 203mb 11mb 12 0,3 0,4 Query Heap Coherence 1 Heap Coherence 2 Performance CPU Coherence 1 CPU Coherence mb 50mb ,2 1, mb 40mb ,7 1, mb 18mb , mb 17mb 720 1,4 1, mb 15mb 680 1,2 1, mb 20mb 600 1,2 1,5 205mb 20mb

24 Performance Comparison Coherence Agents are a nice alternative for dedicated storage nodes architectures and locking scenarios. By using agents, the perfomance overhead (Heap + CPU) is taken from logical node to the storage node. Perfect for filtering data that cannot be queried wit CohQL Fast parallel execution Agents have to be Serializable and have a consistent serial version UID, therefore it is important to have a version and rollout plan Always consider the network!! 24

25 Your Speaker Nicolas Fonnegra Senior Consultant Treffen Sie mich am scope-stand auf Ebene 1! 25