CS-580K/480K Advanced Topics in Cloud Computing. NoSQL Database

Transcription

1 CS-580K/480K dvanced Topics in Cloud Computing NoSQL Database 1

2 1 Where are we? Cloud latforms 2 VM1 VM2 VM3 3 Operating System Operating System Virtualization Layer 3 Operating System VM1 VM2 VM3 3 Operating System Operating System Virtualization Layer Storage Virtualization Operating System Network Virtualization 4 1 VM1 VM2 VM3 2 3 Operating System Operating System Virtualization Layer 3 Operating System 4 Cloud pplications New databases technologies (e.g., Keyvalue store, and Object store) New rogramming Models (e.g., severless, microservices, Hadoop, and Spark)

3 simple example web-based applications Front-end Middle-Tier SQL Databases Storage 3

4 Scaling problem? Stateless Front-end Front-end Front-end Stateless Middle-Tier Middle-Tier Middle-Tier Stateful SQL Databases Storage 4

5 Scaling roblem RDBMS (relational database management system) were not designed to be distributed 5

6 Solution 1 dding a cache layer Stateless Front-end1 Front-end2 Front-end3 Stateless Middle-Tier Middle-Tier Middle-Tier Stateless Cache Cache Cache Stateful SQL Databases s datasets grew, the simple memcache model started to become problematic. Storage 6

7 Scaling roblem RDBMS (relational database management system) were not designed to be distributed Began to look at multi-node database solutions Known as scaling out or horizontal scaling Different approaches include: Master-slave Sharding or artitioning 7

8 Scaling RDBMS Master/Slave Master-Slave ll writes are written to the master. ll reads performed against the replicated slave databases Critical reads may be incorrect as writes may not have been propagated down Large data sets can pose problems as master needs to duplicate data to slaves Master Slave Slave Slave 8

9 Scaling RDBMS - artitioning Sharding Divide data amongst many cheap databases Manage parallel access in the application Scales well for both reads and writes Not transparent, application needs to be partition-aware 9

10 Other ways to scale RDBMS Multi-Master replication INSERT only, not UDTES/DELETES In-memory databases 10

11 In Summary Traditional relational database does not scale well, when dataset grows dding a cache layer scales well for reads, but not for writes Master/slave also scales well for reads, but not for writes Sharding scales well for both reads and writes, but not transparent 11

12 What is NoSQL Stands for No-SQL or Not Only SQL?? Class of non-relational data storage systems E.g. BigTable, and Dynamo Do not require a fixed table schema nor do they use the concept of joins (no relationships) Distributed data storage systems ll NoSQL offerings relax one or more of the CID properties (i.e., C theorem) 12

13 Why NoSQL? Relational databases offer a very good general purpose solution to many different data storage needs. But it cannot fit all Just as there are different programming languages, need to have other data storage tools in the toolbox 13

14 Why NoSQL? Explosion of social media sites (Facebook, Twitter) with large data needs Explosion of storage needs in large web sites such as Google, Yahoo Much of the data is not files Shift to dynamically-typed data with frequent schema changes 14

15 Dynamo and BigTable Three major papers were the seeds of the NoSQL movement BigTable (Google) Dynamo (mazon) C Theorem (counterpart of CID) 15

16 C Theorem (I) Three properties of a system Consistency (all copies have same value) vailability (system can run even if parts have failed) via replication artitions (network can break into two or more parts, each with active systems that can t talk to other parts) 16

17 C Theorem (II) Brewer s C Theorem : You can have at most two of these three properties for any system Very large systems will partition at some point Choose one of consistency or availability Traditional database choose consistency Most Web applications choose availability Except for specific parts such as order processing 17

18 vailability Traditionally, thought of as the server/process available five 9 s ( %). However, for large node system, at almost any point in time there s a good chance that a node is either down or there is a network disruption among the nodes. Want a system that is resilient in the face of network disruption Fault tolerance 18

19 Consistency Model consistency model determines rules for visibility and order of updates. For example: X is replicated on nodes M and N Client writes X to node N Some period of time t elapses. Client B reads X from node M For NoSQL, the answer would be: maybe Client B C Theorem states: Strict Consistency can't be achieved at the same time as availability and partition. M X X N Client 19

20 Eventual Consistency When no updates occur for a long period of time, eventually all updates will propagate through the system and all the nodes will be consistent Known as BSE (Basically vailable, Soft state, Eventual consistency) Who builds application based C Google, Yahoo, Facebook, mazon, ebay, etc 20

21 What kinds of NoSQL (1) Key/Value or the big hash table. mazon S3 (Dynamo)

22 Key/Value ros: Very fast Very scalable Simple model ble to distribute horizontally Cons: Many data structures (objects) can't be easily modeled as key value pairs

23 What kinds of NoSQL (2) Schema-less which comes in multiple flavors, columnbased, document-based or graph-based. Cassandra (column-based) CouchDB (document-based) Neo4J (graph-based) HBase (column-based) 23

24 Document-based Can model more complex objects Data model: collection of documents Document: JSON JavaScript Object Notation is a data model, which supports objects, records, structs, list, array, maps, dates, with nesting. 24

25 Schema-Less ros: - Schema-less data model is richer than key/value pairs - still provide excellent performance and scalability Cons: - typically no CID transactions or joins

26 Typical NoSQL I Basic I access: get(key) -- Extract the value given a key put(key, value) -- Create or update the value given its key delete(key) -- Remove the key and its associated value execute(key, operation, parameters) -- Invoke an operation to the value (given its key) which is a special data structure (e.g. List, Set, Map... etc). 26

27 dvantages of NoSQL Systems Cheap, easy to implement (open source) Data are replicated to multiple nodes (therefore identical and fault-tolerant) and can be partitioned When data is written, the latest version is on at least one node and then replicated to other nodes No single point of failure Easy to distribute Don't require a schema 27

28 What does NoSQL not rovide? Joins Group by CID transactions SQL Integration with applications that are based on SQL 28

29 Which One to use? NoSQL Data storage systems makes sense for applications that need to deal with very very large semi-structured data Log nalysis Social Networking Feeds Most of our work is on organizational databases, which are not that large and have low update/query rates regular relational databases are the correct solution for such applications 29

30 In Summary -- NoSQL Not Only SQL 1. Scale horizontally simple operations 2. Replicate/distribute data over many servers 3. Simple call level interface (contrast w/ SQL) 4. Weaker concurrency model than CID 5. Flexible schema 30

31 Demos Google Big Tables: 31