Introduction to Oracle NoSQL Database Anand Chandak Ashutosh Naik
Agenda NoSQL Background Oracle NoSQL Database Overview Technical Features & Performance Use Cases 2
Why NoSQL? 1. The four V s of Big Data More Data, More Queries, Low Latency Operations (SLAs) Driving Business Innovation 2. Cost Cost effective horizontally scalable simple operations on simple data, Requires less HW, Lower license costs than traditional RDBMS databases Lower COO 3. Time to Market Developer-centric Data Management, Data Variety, Rapid Evolution Shorter Time to Market anand.chandak@oracle.com ashutosh.s.naik@oracle.com 3
Where is NoSQL used? ERP EAM CRM Inventory Control Accting & Payroll Process Mgmt Simple Data Management Globally Distributed, Always On data Competitive Advantages of Fast Data Application Driver Customer Portals Real Time Event Processing Mobile Data Management Time Series & Sensor Data Mgmt Business Analytics Lower TCO, commodity HW scale-out Online Banking 4
The NoSQL Landscape NoSQL Developer- centric APIs Flexible schemas Partitioned/sharded data Horizontally scalable High Availability via Replication Integrated with Hadoop Key/Value Columnar Document Graph Keyspaces, Tables & Records Easy to Model Key-based access Limited Transactions Broad set of use cases Data stored in cells grouped in columns Stored in continuous disk Data aggregation use cases Collections Document-based access JSON & XML Objects as documents use cases Interconnected graphs Relatedness-based access Properties and Graphs, RDF Specific use cases What we hear from our customers: There is no one size fits all NoSQL solution NoSQL and SQL (RDBMS) must work together Application version 2.0 is the one that really works 5
Agenda NoSQL Background Oracle NoSQL Database Overview Technical Features & Performance Use Cases 6
What is Oracle NoSQL Database? less is more 101100101001001 001101010101011 100101010100100 101 Simple Fast Flexible Reliable Cost-effective and horizontally scalable solution that addresses the needs of developers, IT and the business. 7
Architecture and features Features Elastic Configurable transaction semantics Tables / JSON / Binary Online management Differentiators Oracle technology integrated Engineered Systems and Commodity HW Online rolling upgrades Streaming large object support Data Center Support Secondary Indexes Secure Access Flexible schema Application Application NoSQL DB Driver Storage Nodes Datacenter A Application Application NoSQL DB Driver Storage Nodes Datacenter B 9
Enterprise ready -- Integrated out of the box Query NoSQL data from Oracle Database Sync data using Database Mobile Server Access NoSQL data from Hadoop and Spark for DW and analytics Share data with Oracle Coherence for extensible inmemory cache grid Persist history & event streams for processing with Oracle Event Processing Store & query RDF data using Oracle RDF for NoSQL Replicate changes in Oracle Database to NoSQL DB using Oracle Golden Gate Monitor your NoSQL cluster using Oracle Enterprise Manager 10
Forrester Wave TM : NoSQL Key-Value Databases, Q3 2014 Comments from Forrester: Oracle s NoSQL database, based on BerkelyDB, has strong adoption and maturity. Oracle continues to do well in the NoSQL category with a comprehensive solution to support most enterprise needs. Oracle NoSQL key-value database is a mature, high-performing, scale-out storage engine and provides transactional semantics, fine-grained concurrency, primary and secondary indexes, and high availability features. Source: Forrester Oracle Confidential Internal/Restricted/Highly Restricted 11
Agenda NoSQL Background Oracle NoSQL Database Overview Technical Features & Performance Use Cases 13
Architecture (logical) Linear scaling and replication Application Elastic Auto Sharding (split, add, contract) NoSQL Driver Writes to elected node with flexible durability Shard M Shard R Shard R Shard M Reads from any node in system R M R R Auto re-balance of data on expansion R R M Expand Store and Rebalance R 14
Architecture (physical) Linear scaling and replicated D Application NoSQL Driver D Agents Machine1 Elastic Auto Sharding D M 1 R 2 R 3 A Elected Master Machine2 Read from any node R 1 R 2 M 3 D A Machine3 Data rebalance on expand M 2 R 1 R 3 A Shard 1 Shard 2 Shard 3 15
Data Center Support Availability Zones Flexible configuration Primary Zones Durability guarantees Low latency writes, HA 2 nd ary Read-Only Zones Asynchronous replication Analytic workloads Report generation Topology Aware Client Driver Provides business continuity and distributed workload management PrimaryZones DC1 DC2 DC3 Reports Batch Analytics 17
Configurable Durability and Consistency Greater Flexibility Configurable Durability per operation Consistency per operation Transaction scope is single API call Records must share same shard key Multiple operations supported 18
Application NoSQL DB Driver Large Object Support Parallel Streaming Interfaces Shard 1 Efficient storage and retrieval of large objects Client side streaming interface for low memory consumption Server side splitting and distribution of object chunks across nodes for better read/write latency Automatic partial LOB detection Large Object Shard 2 Shard N 19
Oracle NoSQL Database Flexible Data Model Primary key & Shard key Automatic Data sharding & local indexing ACID Transactions advanced key-value database 1. KV Application API, Application specific specific opaque opaque values values 2. JSON API, JSON Structures 3. Table API, Tables 20
Security Data Access Protection Configurable enforcement Authentication User/Password Configurable client time-outs Oracle Wallet integration Internal components self-authenticate Encryption over the wire All channels SSL encrypted Authorization Read/Write Access Administration Kerberos Integration Shard M R R SSL SSL Application NoSQL Driver Shard R M R Store SSL Username Password Read/Write or Admin Shard R R M 22
Time to Upgrade (min) Online Rolling Upgrades Linear time impact for upgrade Admin commands available to describe safe upgrade order Scripted available hands-free upgrade experience Read/Write availability throughout the upgrade process 17.5 15 12.5 10 7.5 Online Rolling Upgrade 5 200 Node version upgrade in 15 minutes 2.5 0 72 (24x3) 144 (48x3) 216 (72x3) Total Nodes (Shards x Rep. Factor) 23
Developer Centric APIs Java API for Key-Value, JSON and Table data models R, JRuby, Jython community drivers C API for Key-Value & JSON C API for Table data model Implementing using a new Thrift, JNI-less protocol Additional Thrift-based APIs to follow (JavaScript, Python) REST API available through Oracle Rest Data Services Deployments using Oracle Web Logic Server, Glassfish, Tomcat Directs Web Service calls to NoSQL DB Client-side DDL API Create and Alter tables and schemas inside an application Node.js and Python clients 24
Integration with Apache Hive and Oracle Big Data SQL Oracle Big Data SQL Common SQL access layer for data stored in Hadoop HDFS, Hive and Oracle NoSQL Database Defines new external table access driver types. ORACLE_HIVE for accessing data stored in or referenced by Hive Tables Query Oracle NoSQL Data from Hive or Oracle Database Runs Map Reduce Jobs against data stored in Oracle NoSQL Database Support for primitive and complex data types Common data definition between Oracle NoSQL DB, Hive and Oracle Database BIG DATA SQL 25
NoSQL As Mobile/Embedded Data Repository Complete Mobile/Embedded to NoSQL Integration Embedded data store (BDB) Automatic rule-based sync Device Management Database Mobile Server Secure data transport and conversion Mobile Device Management Scalable data repository Fast data analytics Devices NoSQL 26
Developer and Admin Tools Standards based tooling SNMP / JMX metrics Oracle Enterprise Manager Cloud ready HTML5 browser admin Command line interface Cluster Configuration & Status DDL & DML Scripting Data load Aggregates 27
Predictability, Reliability & Support Global, mission-critical application deployment experience Decades of enterprise-grade non-relational database technology Oracle Support available for both Enterprise and Community Edition Designed for Predictability and Manageability Bulk Insert Test Cluster Expansion Test Rolling Upgrade Test 29
Throughput (ops/sec) Average Latency (ms) YCSB on SSD-backed commodity servers What s the big deal Twitter sees ~500M tweets/day This is 350K a minute Capture all tweets with 3 commodity servers 1.25M ops/sec 2 billion records 2 TB of data 95% read, 5% update Low latency, High Scalability 1,400,000 1,200,000 1,000,000 800,000 600,000 400,000 200,000 0 Mixed Throughput 6 (2x3) 12 (4x3) 24 (8x3) 30 (10x3) Cluster Size ( 3->15 servers ) 4 3 2 1 0 Throughput (ops/sec) Write Latency (ms) Read Latency (ms) 30
What s New in Oracle NoSQL DB 2014 & 2015 Data Management Table Data Model* Secondary Indices* Client-side, SQL-like DDL* Application Enablement Maven Repository C API Binaries APIs C API for Tables & Indices REST API* Thrift Proxy Javascript (Node.js)* Python* Server & Performance Admin Repair Diagnostic enhancements Security Authentication* Authorization* User-Defined Roles* Table-Level Permissions* Wire-Level Encryption* Administration Secondary Data Centers Integration OEM* Oracle Big Data SQL* Oracle Database Mobile Server Oracle Golden Gate Apache Spark Apache Hive (*) Customer-driven requests 32
Hardware, License, Delivery Customer Installs Commodity servers (x86 or SPARC including M7) Big Data Appliance (BDA) Two Types of License Community Edition (AGPLv3 license) Support available Enterprise Edition (Commercial license) includes integration and security over CE. Cloud: Monthly subscription pricing (coming soon) 33
Agenda NoSQL Background Oracle NoSQL Database Overview Technical Features & Performance Use Cases 35
Oracle NoSQL Database customer successes Finance Large credit card network provider Fraud rule and history management 6,500 ops/sec, 85% update on read Telecoms Large mobile telecom operator Mobile personalized advertizing 24,000 ops/sec, 100% update on read Manufacturing (Airbus) Large airline manufacturer Developing next generation aircraft 750K key inserts/sec, 6TB/hour Customer Loyalty Operationalize analytics to NoSQL Customer Portal Addressed processing conflicts 20M+ customers in multiple countries many more 36
Use case growth in the last 12 months Web-Scale Personalization (Customer Portals & Digital Commerce) Direct-to-Consumer web services Real-time transaction capture High volume, Low latency Retail, Telco, Financial Svcs Time Series & Sensor Data Mgmt (aka Internet of Things) Efficient, scalable data persistence Requires analytics and visualization Scalability, Cost management Utilities, Oil & Gas, Financial Svcs Oracle CGBU BRM uses NoSQL Database Oracle s IoT cloud service uses NoSQL Database 37
High Level NoSQL DB Roadmap Cloud Oracle NoSQL Database Cloud Services Integration Oracle Stack Open Source Stack Product SQL for NoSQL Full Text Search Additional Security 46
Join NoSQL Database Community Oracle.com/BigData Twitter https://twitter.com/#!/oraclenosql LinkedIn http://www.linkedin.com/groups?gid=4147754 Oracle s NoSQL DB blog https://blogs.oracle.com/nosql Oracle Technology Network http://bit.ly/1f0d8wu Developer Webcast Series http://bit.ly/1dov2jl 47
Q&A 48