Copyright 2013, Oracle and/or its affiliates. All rights reserved.

Transcription

1 1

2 Oracle NoSQL Database: Release 3.0 What s new and why you care Dave Segleau NoSQL Product Manager

3 The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle s products remains at the sole discretion of Oracle. 3

4 Agenda NoSQL Background Release 3.0 Highlights Customer Response 4

5 How Does NoSQL Fit With Big Data? SQL JDBC, ODBC General Purpose More Flexible Schema Management Globally Distributed, Always On data Distributed, Batch Processing Distributed, Replicated File System Managed Schemas Security, Backups Analytics Competitive Advantages of Fast Data Lower TCO, commodity HW scale-out Application Driver NoSQL databases Flexible Schemas Sharded, Replicated Database High Speed, Simple Ops 5

6 Why NoSQL? Cost Flexible schema development paradigm Simple, typically not managed by IT Scalable, reliable, distributed architecture 6

7 NoSQL or Relational?? NoSQL DBMS Schema private to application Collections of data objects Basic Availability, Soft-state, Eventual consistency Relational DBMS Schema shared by multiple applications Inter-related collections of data objects Atomic, Consistent, Isolated, Durable 7

8 NoSQL and Relational Working Together Documentcentric data access Operational transactions Low-latency web-scale access Reporting and Analysis NoSQL Relational 8

9 What is Oracle NoSQL Database? less is more Simple Fast Flexible Reliable Non-Relational Key-Value database designed as cost effective, high performance solution for simple operations on collections of data with built in high availability and elastic scale-out. 10

10 Oracle NoSQL Database NoSQL for Developers and IT Simple: Setup, Admin, API & Integration Fast: Parallel Access & Scale-out Flexible: Flexible schema & Agile development Reliable: Built-in HA, Predictable Performance 11

11 Oracle NoSQL Database Scalable, Highly Available, Key-Value Database Features Key-value, JSON & RDF data Large Object API BASE & ACID Transactions Data Center Support Online Rolling Upgrade Online Cluster Management Release Table data model 3.0 Secondary Indices Secondary Zones (Data Centers) Security Application Application NoSQL DB Driver Storage Nodes Datacenter A Application Application NoSQL DB Driver Storage Nodes Datacenter B 13

12 Agenda NoSQL Background Release 3.0 Highlights Customer Response 14

13 Oracle NoSQL DB Release 3.0 Enterprise Ready Ease of Adoption Business Continuity Security 15

14 Oracle NoSQL DB Release 3.0 Table Data Model Benefits Lower barrier to adoption Simplifies application development shorter time to market Allows secondary indexing of multiple fields Sets foundation for future SQL query access from Oracle DB Previous Key/Value and JSON schema APIs still supported Simplified application modeling Uses familiar table concepts Introduces strongly typed fields Automatic mapping of Major/Minor key structure Defines JSON schema automatically Supports table evolution Easy to/from JSON objects ACID & BASE transactions Compatible with Release 2.0 JSON schemas 16

15 Oracle NoSQL DB Release 3.0 Simple Table Example table create -name Users add-field -name userid -type integer add-field -name lastname -type string add-field -name firstname -type string add-field -name -type string primary-key -field userid shard-key -field userid exit plan add-table -name Users -wait Primary Key Can be specified as a JSON string Must be proper subset of primary-key By default shard-key == primary-key Value userid lastname firstname 17

16 Oracle NoSQL DB Release 3.0 Nested Table Example table create -name Users table create -name Users.Folders add-field -name userid -type integer add-field -name foldername -type string add-field -name lastname -type string add-field -name msgcount -type integer add-field -name firstname -type string add-field -name favorite type boolean -default 'F' add-field -name -type string primary-key -field foldername primary-key -field userid exit exit plan add-table -name Users.Folders -wait plan add-table -name Users wait Primary Key [Value] Tables userid lastname firstname Users Shard Key Primary Key [Value] userid foldername msgcount favorite Users.Folders 18

17 Oracle NoSQL DB Release 3.0 Secondary Index Benefits More applications, more use cases Primary and Secondary Indexes (expanded data access) Indexes automatically maintained (fast and consistent) Provides sorted results Higher performance, scalable implementation, low overhead Shard-local indexing low latency, transactionally consistent record access Parallel B-tree scans, locally ordered, merged result set Improved index cache utilization 19

18 Oracle NoSQL DB Release 3.0 Secondary Indexes plan add-index table Users -name Users_idx1 field lastname field firstname userid firstname lastname Table Users Equality search on lastname + firstname get all records for Jones, Tom Equality search on lastname and range on firstname get all records for Jones where firstname starts with T Range or Equality search on lastname by itself get all records for Jones get all records where lastname starts with Johns Get() API specifies index Equality or range searches Sorted, multi-column indexes Composite index on values Parallel Index Scans Ordered results Indexes on arrays Can t search on just firstname 20

19 Writes Secondary Index Storage Primary Records Pkey A Value A F1 F2 F3 V1 V2 V3 Pkey B Value B F1 F2 F3 V1 V2 V3 Index on (V1) Index on (V2, V3) Shard 1 Shard 2 Skey A Skey A Pkey A V2 V3 F1 F2 F3 Skey B Pkey A V1 F1 F2 F3 Skey B Shard N Secondary Indexes Pkey B V1 F1 F2 F3 Pkey B V2 V3 F1 F2 F3 Reads 21

20 Oracle NoSQL DB Release 3.0 Secondary Data Center (Zones) Benefits Flexible configuration Primary Zone: low latency, durable operations (for production) Secondary Zone: asynchronous replication (for batch or globally distributed data centers) Better application workload management Topology-aware driver knows where to send queries Flexible application-level control, if needed 22

21 Oracle NoSQL DB Release 3.0 Zones Primary Zones Durability guarantees Low latency writes, HA 2 nd ary Read-Only Zones Asynchronous replication Analytic workloads Report generation Topology Aware Client Driver Application controls Can restrict operations to specific zones New Read Consistency: No Master Primary Zones DC1 DC2 DC3 Reports Batch Analytics 23

22 Oracle NoSQL DB Release 3.0 Security Benefits Protected Data Access Simple, flexible user authentication Comprehensive data encryption over the wire 24

23 Oracle NoSQL DB Release 3.0 Security Overview Data Access Protection Application NoSQL Driver Username Password Configurable enforcement SSL Authentication Shard Shard Shard User/Password Configurable client time-outs M M M Oracle Wallet integration Internal components self-authenticate Encryption over the wire R R SSL R R SSL R R All channels SSL encrypted Store 25

24 Best Practices Choosing a Data Model Tables Highest level abstraction, simple to model, familiar to developers System managed Secondary indices, Table evolution support Future: Security per table, Query Language JSON Schemas Medium level abstraction, need to model keys (strings), appealing to JSON-centric applications Application managed Index Views, Schema evolution support Future: Limited security Raw Key-Value Lowest level abstraction, need to model keys (strings), application serialized data, maximum flexibility Application managed Index Views, record evolution and security roll your own 26

25 Agenda NoSQL Background Release 3.0 Highlights Customer Response 27

26 What We Have Heard from Customers James Anthony, CTO, Passoker Oracle NoSQL already forms a critical component of our production environments, and the enhancements in this new version take the product to a new level. We are particularly excited about the secondary indexes and table functionality, and how this will provide significant productivity and performance gains, as well as simplifying the code we use. In addition the security enhancements allow us to ensure that we can meet the stringent security requirements demanded by our client base Senior Data Architect, customer loyalty services provider Oracle NoSQL Database release 3.0 addresses the needs of Enterprises that want to leverage NoSQL technology. For our company, the addition of security features like Encryption and Authentication help us to fulfill our security requirements. 28

27 What We Have Heard from Customers Alok Pareek, Founder and Executive Vice President, WebAction The new release of Oracle NoSQL Database adds compelling value for developers and IT, like security and data center monitoring enhancements which are critical for enterprise deployments. For WebAction's data center automation application, we need to have a robust NoSQL back end data store and Oracle NoSQL Database is a natural fit. The new table model and indexing features are also a welcome addition to the latest release, and we plan to take advantage of these features Various customers Combines best features from MongoDB and Cassandra 29

28 Oracle NoSQL DB Resources Oracle.com Oracle Technology Network Downloads, Documentation, Tutorials, White Papers are on OTN Oracle Big Data Handbook on Amazon, Safari and others Oracle NoSQL Database: Real-Time Big Data Management for the Enterprise on Amazon, Safari and others 30

29 31

30 Appendix 32

31 Big Data Architecture Where does NoSQL fit? Data Reservoir + Data Warehouse Cloudera Hadoop Oracle NoSQL Database Oracle R Distribution Oracle Big Data Connectors Oracle Data Integrator Oracle Database Oracle Database Oracle Industry Models Oracle Industry Models Oracle Advanced Oracle Advanced Analytics Analytics Oracle Oracle Spatial Spatial & Graph & Graph Oracle Event Processing Apache Flume Oracle GoldenGate Oracle Data Integrator Oracle GoldenGate Oracle Event Processing 33

32 The NoSQL Landscape NoSQL Developer- centric APIs Flexible schemas Partitioned/sharded data Horizontally scalable High Availability via Replication Integrated with Hadoop Columnar & Key/Value Keyspaces, Tables & Records Key-based access Limited Transactions Broad set of use cases Document Collections Document-based access JSON & XML Objects as documents use cases Graph Interconnected graphs Relatedness-based access Properties and Graphs, RDF Specific use cases 34

33 Choose the RIGHT storage option for the job Hadoop Distributed File System (HDFS) Oracle NoSQL Database Oracle Database File System Key-Value Database Relational Database No inherent structure Simple data structure Complex data structures, rich SQL High volume writes Limited functionality, roll-your-own applications Batch Oriented High volume random reads and writes Simple get/put high speed storage, flex configuration Real-Time, web-scale specialized applications High volume OLTP with 2-PC Security, Backup/Restore, Data life cycle mgmt, XML, etc. General purpose SQL platform, multiple applications, ODBC, JDBC 35

34 NoSQL Technology Choices What s Really Important? Technical Feature Importance Why Storage Model Not really Will merge over time Specific Features Somewhat Application requirements? Performance Somewhat Rapid changes, YMWV Integration Critical Long term, Repetitive cost Reliability/Support Critical Early products, Product direction Predictability Critical Production reqs & SLAs 36

35 Scalability Architecture Applications View Elastic Shards (split, add, contract) Application NoSQL Driver Writes to elected node Shard M Shard M Shard M Reads from any node in system R R R R R R Store 37

36 Metrics and Best Practices Tables No overhead, performs same as JSON schemas with AVRO serialization Use nested tables to encapsulate record types in a hierarchy Use Arrays and Arrays of Records to store self contained sets Record types can t be indexed (future) -> flatten into simple types if indexing is required UserName (FirstName, MI, LastName) -> FirstName, MI, LastName 38

37 Metrics and Best Practices Indexes Same rationale as an RDBMS Reduces time for lookups/range scans Increases overhead for updates Provides sorted results Indexes are not free -- add only when benefitial No optimizer -- application picks index to be used Remember to calculate cache size requirement New CLI option for get: -reportsize tells you the size of a key. Use output with DBCacheSize 39

38 Metrics and Best Practices Data Centers/Zones Data Centers may add write latency, depending on durability policy and inter-data center latency Options for tuning: Consider reducing durability policy ACK requirement if latency is high Simple rules 1. Allow Client Driver to perform load balancing 2. Use No-Master read consistency to restrict reads to replicas only 3. Use specific Secondary Zones to restrict read-only, batch, analytical workloads 40