Key Value Store. Yiding Wang, Zhaoxiong Yang

Transcription

1 Key Value Store Yiding Wang, Zhaoxiong Yang

2 Outline Part 1 Definitions/Operations Compare with RDBMS Scale Up Part 2 Distributed Key Value Store Network Acceleration

3 Definitions A key-value database, or key-value store, is a data storage paradigm designed for storing, retrieving, and managing associative arrays, a data structure more commonly known today as a dictionary or hash. Dictionaries contain a collection of objects, or records, which in turn have many different fields within them, each containing data. These records are stored and retrieved using a key that uniquely identifies the record, and is used to quickly find the data within the database.

4 Simple definitions Simple Hash Table/Associative Array Collection of Key-Value Pair The Key Unique Within Collection No Relation/Query Among Data

5 Basic Operations Insert Pair Find Value Associate With a Particular Key Update Value Of Existing Pair Delete Pair Create, Read, Update and Delete (CRUD)

6 Basic Operations Table T: key k1 k2 k3 k4 keys are sorted value v1 v2 v3 v4 API: lookup(key)! value lookup(key range)! values getnext! value insert(key, value) delete(key) Each row has timestamp Single row actions atomic No query language

7 KVS and NoSQL: A Brief History KVS was brought by NoSQL movement Relational model and RDBMSs are too restrictive: 1. Flat tables with few data/attribute types 2. Restricted language interface (SQL) 3. Need to know schema first 4. Optimised for static dataset Is relational model sufficient and able to be extended to support new applications? Source: UCSD CSE190D

8 KVS and NoSQL: A Brief History KVS was brought by NoSQL movement Relational model and RDBMSs are too restrictive: 1. Flat tables with few data/attribute types Object-Relational DBMSs: UDT, UDFs, text, multimedia, etc. 2. Restricted language interface (SQL) PL/SQL; recursive SQL; embedded SQL; QBE; visual interfaces 3. Need to know schema first Schema-later semi-structured XML data model; XQuery 4. Optimised for static dataset Stream data model; standing queries; time series DB

9 KVS and NoSQL: A Brief History DB folks underappreciated 4 key concerns of Web folks: 1. Developability 2. Fault Tolerance 3. Elasticity 4. Cost

10 KVS and NoSQL: A Brief History DB folks underappreciated 4 key concerns of Web folks: 1. Developability RDBMS extensibility mechanisms (UDTs, UDFs, etc.) are too painful to use for programmers Need simpler APIs and DBMSs

11 KVS and NoSQL: A Brief History DB folks underappreciated 4 key concerns of Web folks: 2. Fault Tolerance Web giants gave a question: What if we run on 100Ks of machines? DB companies: our customers (banks, retails, etc.) do not need more than a few dozen machines to store and analyse their data Web companies: we need hundreds of thousands of machines for planetary-scale Web services If a machine fails, user should not have to rerun entire query. DBMS should take care of fault tolerance, not user/application

12 KVS and NoSQL: A Brief History DB folks underappreciated 4 key concerns of Web folks: 3. Elasticity Resources should adapt to query workload DB companies: our customers have fairly predictably sized datasets and workloads Web companies: our workloads could vary widely and the datasets they need vary widely Need to be able to upsize and downsize clusters easily on-thefly, based on current query workload

13 KVS and NoSQL: A Brief History DB folks underappreciated 4 key concerns of Web folks: 4. Cost Commercial RDBMS licenses are too costly Newly-built tools were free & open source and began popular

14 KVS and NoSQL: A Brief History Google Simple problem: index, store, and search the Web Large and unstructured data, lots of random reads and writes Developability, data model, fault tolerance, scale, cost, Engineers started with MySQL then abandoned it Cloud web applications require fast requests and the use of a relational database can sometimes be major bottleneck in the architecture NoSQL, no for no or not only

15 Popular KVS DynamoDB, hosting many Amazon Web Services, like S3 MemcacheDB, distributed memory object caching system whose largest user is Facebook. Facebook use it to cache queries/results Redis, distributed and in-memory key-value store with optional durability which is the most popular KVS. Offered on AWS and Azure etcd is a distributed reliable key-value store for the most critical data of a distributed system, used in Kubernetes MXNet, using distributed KVS for data sharing >>> kv = mx.kv.create('local') # create a local kv store. >>> kv.push(3, [mx.gpu(i) for i in range(4)]) >>> kv.pull(3, out = a)

16 Scaling Sharding/Horizontal Partition server 1 server 2 server 3 key k1 k2 k3 k4 k5 k6 k7 k8 k9 k10 value v1 v2 v3 v4 v5 v6 v7 v8 v9 v10 key k1 k2 k3 k4 value v1 v2 v3 v4 key k5 k6 value v5 v6 key k7 k8 k9 k10 tablet value v7 v8 v9 v10 Consistent Hashing

17 Scaling Sharding/Horizontal Partition server 1 server 2 server 3 key k1 k2 k3 k4 k5 k6 k7 k8 k9 k10 value v1 v2 v3 v4 v5 v6 v7 v8 v9 v10 key k1 k2 k3 k4 value v1 v2 v3 v4 key k5 k6 value v5 v6 key k7 k8 k9 k10 tablet value v7 v8 v9 v10 Consistent Hashing When a hash table is resized, only K/n keys need to be remapped on average, where K is #keys, and n is #slots.

18 Source: Consistent hashing, a guide & Go library

19 In consistent hashing a node is responsible for keys with ids from itself to its successor Consistent hashing with virtual nodes A virtual node looks like a single node in the system, but each node can be responsible for more than one virtual node. Effectively, when a new node is added to the system, it is assigned multiple positions in the ring. Used in DynamoDB and Cassandra

20 Scaling Tablet Replication server 3 server 4 server 5 key k7 k8 k9 k10 value v7 v8 v9 v10 key k7 k8 k9 k10 value v7 v8 v9 v10 key k7 k8 k9 k10 value v7 v8 v9 v10 primary backup backup Write request, sent to all replicas; read request, the most recent data item (based on timestamp) will be forwarded back. Inconsistent replicas handled and updated background Some DBMSs rely on distributed file systems to manage replication

21 Fault Detection Membership protocol, keep all nodes informed at all times of the other nodes in the ring Periodically, the node polls the membership protocol to bring its membership list up to date Failure detection is done by periodic random probing (ping). If no ACK from node, an indirect probe asks some random nodes to probe the this node. If still fail, mark it suspect and inform cluster. If the suspect node of the cluster does not dispute the suspicion, mark it faulty and spread the word to the whole cluster.

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41