Relational to NoSQL: Getting started from SQL Server. Shane Johnson Sr. Product Marketing Manager Couchbase

Transcription

1 Relational to NoSQL: Getting started from SQL Server Shane Johnson Sr. Product Marketing Manager Couchbase

2 Today s agenda Why NoSQL? Identifying the right application Modeling your data Accessing your data Installing and scaling your database Monitoring and managing your deployment Q & A 2015 Couchbase Inc. 2

3 About the speakers Shane Johnson Shane Johnson Senior Product Marketing Manager Couchbase (since Dec 2013) Experience: - Proud Red Hatter - Consulting (Architect) - Transitioned to Marketing (Technical Manager) - Passionate about Open Source - Expertise - Java and Distributed Systems 2015 Couchbase Inc. 3

4 What s Couchbase? Couchbase is the company behind Couchbase Server & Couchbase Mobile Open source JSON database Founded enterprise customers globally Some of our customers: Couchbase Server can be deployed as: Document database Key-value store Distributed cache 2015 Couchbase Inc. 4

5 What is NoSQL? No SQL? No. Not only SQL? Not really. Non relational? Yes. Distributed (most) Scaled out, not up Elasticity and commodity hardware Partitioned and replicated Scalability, performance, availability Schema-less (most) Flexible model JSON (some) 2015 Couchbase Inc. 5

6 Who is using NoSQL? Enterprises are adopting NoSQL for mission critical applications Media & Publishing ecommerce Hospitality 2015 Couchbase Inc. 6

7 Who is using NoSQL? Gannett, publisher of USA Today and 90+ media properties, replaced relational database tech- nology with NoSQL to power its digital publishing platform. Marriott deployed NoSQL to modernize its hotel reservation system that supports $38 billion in annual bookings. FHLBank Topeka leverages NoSQL on top of SQL Server to speed up access to customer financial data for its 770 member banks Cars.com, with over 30 million visits per month, replaced SQL Server with NoSQL to store customer and vehicle data Vente-privee.com improved the customer experience of its 18 million members by replacing SQL Server with NoSQL for better performance 2015 Couchbase Inc. 7

8 Why are they using NoSQL? Technology Drivers Customers are going online The Internet is connecting everything Big Data is getting bigger Applications are moving to the cloud The world has gone mobile Technical Needs Develop with agility Flexibility + Simplicity Easier + Faster Operate at any scale Elasticity + Availability Performance at scale Always-on, global deployment Business Needs Innovate and compete Faster time to market Reduced costs (operational + hardware) Increased revenue 2015 Couchbase Inc. 8

9 Why migrate from SQL Server? Easier to scale 3 nodes to 100s, 1 data center to many, commodity hardware Better performance integrated caching, memory-optimized indexes, memory-based replication Up to 40x lower cost open source, subscription-based, per instance (not per core) Cross-platform runs on Windows or Linux (Red Hat, Ubuntu, Debian, etc) Greater agility JSON-based data model, SQL-based query language 2015 Couchbase Inc. 9

10 How do you get started? Identify the right application Model your data Access your data Install and scale your database Monitor and manage your deployment 2015 Couchbase Inc. 10

11 Identifying the right application Have one or more of the following characteristics or requirements: Iterate faster Send and receive JSON Provide low latency at any throughput Support many concurrent users Examples: High performance, high availability caching service Small, independent application with a narrow scope Logical or physical service within a large application Global service that powers multiple applications Supports users anywhere and everywhere Be available 24x7 Store terabytes of data Read and write to multiple data centers Service Application Service Service RDMBS NoSQL 2015 Couchbase Inc. 11

12 Concepts: Terminology Relational (SQL Server) Failover Cluster Availability Group Database Table Row (Tuple) Primary Key IDENTITY Indexed View SQL NoSQL (Couchbase) Cluster Cluster Bucket Bucket Document (JSON) Object ID Counter View N1QL 2015 Couchbase Inc. 12

13 Model your data 2015 Couchbase Inc. 13

14 Modeling your data: Fixed vs. self-describing schema 2015 Couchbase Inc. 14

15 Modeling your data: The flexibility of JSON Same document type, Different fields Different types Optional On-demand Tip: Add a version field to track changes. { doctype : user, docversion : 1, } { doctype : user, docversion : 2, } 2015 Couchbase Inc. 15

16 Modeling your data: Changing the data model Relational database Modify the database schema Modify the application code (e.g. Java) Modify the interface (e.g. HTML5/JS) Document database Modify the interface (e.g. HTML5/JS) 2015 Couchbase Inc. 16

17 Modeling your data: Object IDs Best Practices Natural Keys Human Readable Deterministic Semantic Examples author::shane author::shane::blogs blog::nosql_fueled_hadoop blog::nosql_fueled_hadoop::comments What about identity columns? 1. Document<Long> nextauthoriddoc= bucket.counter( authoridcounter, 1); 2. Long nextauthorid = nextauthoriddoc.content(); 3. String authdocid = author:: + nextauthorid; // author::101 Tip: Increment the counter by 10, 20, etc. instead of doing it for every insert Couchbase Inc. 17

18 Modeling your data: Relationships Bottom up Top down Author Author (FK x2) Blog (FK) Blog (FK) Blog (FK x2) Blog Comment (FK) Comment (FK) Comment Comment 2015 Couchbase Inc. 18

19 Modeling your data: Relationships 2015 Couchbase Inc. 19

20 Modeling your data: Strategies and best practices If Relationship is one-to-one or one-to-many Relationship is many-to-one or many-to-many Data reads are mostly parent fields Data reads are mostly parent + child fields Data writes are mostly parent or child (not both) Data writes are mostly parent and child (both) Then Store related data as nested objects Store related data as separate documents Store children as separate documents Store children as nested objects Store children as separate documents Store children as nested objects 2015 Couchbase Inc. 20

21 Modeling your data: Strategies and best practices Are they a lot of concurrent writes, continuous updates? Store children as separate documents Blog Thread Comment Comment Thread Comment Comment Blog { doctype : blog, } author : author::shane, title : Couchbase Wins, threads : [ blog::couchbase_wins::threads::001, blog::couchbase_wins::threads::002 Thread { doctype : thread, comments : [ { visitor : Brendan Bond, text : This blog is amazing! replies : [ { user : Dustin Johnson, text : No, it is not. }] 2015 Couchbase Inc. 21 } }

22 Access your data 2015 Couchbase Inc. 22

23 Accessing your data: Options We ll focus on these three for now. Key-Value (CRUD) N1QL (Query) Views (Query) Full Text (Search) Geospatial (Search) Indexes MapReduce Indexes MapReduce Documents 2015 Couchbase Inc. 23

24 Accessing your data: Connecting to the database Access data via topology-aware smart clients Maintains an up-to-date cluster map Communicates directly with database nodes no proxies, no routers, etc. Available for Java, Node.js, PHP,.NET, Python, C, Go, and more With standard, certified JDBC/ODBC drivers (if you want to) 2015 Couchbase Inc. 24

25 Accessing your data: Domain objects vs. document objects Working with document objects requires less code, provides more flexibility. * JSON serialization via Boon Couchbase Inc. 25

26 Accessing your data: Key-value operations referenced data 2015 Couchbase Inc. 26

27 Accessing your data: Key-value operations nested data 2015 Couchbase Inc. 27

28 Accessing your data: Subdocument operations 2015 Couchbase Inc. 28

29 Accessing your data N1QL queries: Capabilities Feature SQL N1QL JOIN TRANSFORM FILTER AGGREGATE SORT SUBQUERIES PAGINATION OPERATORS FUNCTIONS 2015 Couchbase Inc. 29

30 Accessing your data: N1QL queries referenced data 2015 Couchbase Inc. 30

31 Accessing your data: N1QL queries nested data 2015 Couchbase Inc. 31

32 Accessing your data: N1QL queries CRUD 2015 Couchbase Inc. 32

33 Accessing your data: LINQ _context.query<users>().where(u => u.status == Platinum ) from user in _context.query<users>() join account in _context.query<account>() on user.accountidequals N1QlFunctions.Key(account) into usergroup from accounts in usergroup.defaultifempty() where (account.type == Visa account.type== MasterCard ) select new { firstname = user.firstname, lastname = user.lastname }; from user in _context.query<users>() join address in _context.query<address>() on user.addresses.shipping.addressid equals N1QlFunctions.Key(address) into addressgroup from address in addressgroup.defaultifempty() where address.state == CA select new { firstname = user.firstname, lastname = user.lastname }; 2015 Couchbase Inc. 33

34 Accessing your data: N1QL queries indexes Simple Functional Compound Partial 2015 Couchbase Inc. 34

35 Accessing your data: Views What if indexed views worked great with write intensive workloads? And you could use COUNT, ORDER BY, and everything else 2015 Couchbase Inc. 35

36 Accessing your data: Views COUNT?!? 2015 Couchbase Inc. 36

37 Accessing your data: Views Incremental MapReduce 2015 Couchbase Inc. 37

38 Accessing your data: Views queries 2015 Couchbase Inc. 38

39 Accessing your data: Strategies and best practices Concept Key Value Operations provide the best possible performance Strategies & Best Practices Create an effective key naming strategy Create an optimized data model Incremental MapReduce (Views) are well suited to aggregation Ideal for large data sets, the entire data set Can be used to create complex secondary indexes N1QL queries provide the most flexibility everything else Query data regardless of how it is modeled Remember to create indexes, leverage covering indexes where possible 2015 Couchbase Inc. 39

40 Install and scaling your database 2015 Couchbase Inc. 40

41 Installing and scaling your database 1. Download 2. Install 3. Configure 4. Cluster / Scale 5. Rebalance Admin screenshot add node 2015 Couchbase Inc. 41

42 Installing and scaling your Couchbase database: XDCR Multiple Data Centers Cluster per data center Replicate between clusters Unidirectional / bidirectional Master / Master Local reads and writes Ring Hub-and-spoke Mesh Combination Built-in 2015 Couchbase Inc. 42

43 Managing and monitoring your deployment 2015 Couchbase Inc. 43

44 Managing and monitoring your deployment Management, monitoring, and configuration Cluster / node / bucket / views Cross data center replication (XDCR) Database performance, network utilization, resource utilization Tasks Add and remove nodes Failover nodes Rebalance cluster Configuration Authentication and authorization Auditing Monitoring View and collect log information Tooling cbbackup & cbrestore cbcollect_info & cbdstats cbq cbtransfer and more * Web, REST API, and CLI 2015 Couchbase Inc. 44

45 Where to go next? Conduct a Successful Proof of Concept 1. Select a use case and application 2. Define the success criteria 3. Understand the data 4. Identify the access patterns 5. Review the architecture Measure your Return on Investment Greater agility? Faster time to market? Easier scalability? Better performance? Better availability? Lower costs? 2015 Couchbase Inc. 45

46 You re just in time Couchbase Server 4.5 Beta Easier, more efficient querying Integrated query editor Indexed joins Integrated full-text search Ready your own writes Simpler, more advanced data access Partial updates, reads, etc. Faster, more powerful indexing Memory-optimized indexes Array indexing Circular reuse Better, more comprehensive admin Query monitoring Faster backup and restore X.509 certificates Role-based access control 2015 Couchbase Inc. 46

47 Query Workbench: ad-hoc queries 2015 Couchbase Inc. 47

48 Query Workbench: schema browser 2015 Couchbase Inc. 48

49 Questions? 2015 Couchbase Inc. 49

50 WANT TO LEARN MORE? How to Get Started White Paper 4.0/sql_servertonosql.pdf Download Couchbase software: Getting Started guide: 2015 Couchbase Inc Free Online Training

51 Thank you Couchbase Inc. 51