Why NoSQL? Why Riak?

Transcription

1 Why NoSQL? Why Riak? Justin Sheehy 1

2 What's all of this NoSQL nonsense? Riak Voldemort HBase MongoDB Neo4j Cassandra CouchDB Membase Redis (and the list goes on...) 2

3 What went wrong with SQL databases? Nothing! They are great tools. 3

4 What went wrong with SQL databases? Nothing! They are great tools. 4

5 NoSQL came before SQL. VAX DBMS Honeywell IDS dbm IBM IMS MUMPS Cincom TOTAL PICK (and the list goes on...) 5

6 But that's not really NoSQL, is it? VAX DBMS Honeywell IDS dbm IBM IMS MUMPS Cincom TOTAL PICK (and the list goes on...) 6

7 The start of today's NoSQL Amazon: Dynamo (2007) Google: Bigtable (2006) Technology is not the important part! 7

8 The start of today's NoSQL Amazon sells books. (esp. 3+ years ago) Why not just use Oracle/MySQL/etc? most Technology is not the important part! 8

9 The start of today's NoSQL Amazon sells books. (esp. 3+ years ago) Why not just use Oracle/MySQL/etc? Business needs (re)created demand for alternative database technologies. 9

10 What's all of this NoSQL nonsense? Riak Voldemort HBase MongoDB Neo4j Cassandra CouchDB Membase Redis Do we really need all of these? 10

11 It's still about business needs. Riak use cases drive decisions! Redis 11

12 It's still about business needs. choose Riak what solves your real problem Redis 12

13 how do you know what solves your real problem? two places to start: innate/exposed data model distribution/operational model 13

14 data model differences requirements usually from app developers Semi-Structured Documents Graph Traversal Key/Value Hierarchical Tabular/Relational Indexed Query Native Data Structures Bigtable Column Family 14

15 data model differences not much useful ordering Graph Traversal which one is more powerful? Bigtable Column Family 15

16 data model differences Voldemort Neo4j Cassandra K/V Graph ColFam etc etc 16

17 distribution model differences requirements usually from biz or operations Distributed System Server Replication Single Server Locally Embedded 17

18 distribution model differences requirements usually from biz or operations Distributed System Server Replication Single Server Locally Embedded 18

19 starting to decide Distributed System Server Replication Single Server Locally Embedded K/V Graph ColFam etc etc 19

20 starting to decide We need a K/V or ColFam store. Distributed System Server Replication Single Server Locally Embedded K/V Graph ColFam etc etc 20

21 starting to decide We need a K/V or ColFam store. We need to recover quickly from server failure. Distributed System Server Replication Single Server Locally Embedded K/V Graph ColFam etc etc 21

22 now we're getting somewhere! Distributed System Server Replication K/V 22 ColFam

23 we can make a shorter list... Riak, Voldemort, CouchDB, Cassandra, HBase... Distributed System Server Replication K/V ColFam 23

24 we can make a shorter list... Riak, Voldemort, CouchDB, Cassandra, HBase... then we can narrow it more... protocols, licensing, benchmarking, simplicity... Distributed System Server Replication K/V ColFam 24

25 we can make a shorter list... Riak, Voldemort, CouchDB, Cassandra, HBase... then we can narrow it more... protocols, licensing, benchmarking, simplicity... and make a choice! Distributed System Server Replication K/V ColFam 25

26 we can make a shorter list... Riak, Voldemort, CouchDB, Cassandra, HBase... then we can narrow it more... protocols, licensing, benchmarking, simplicity... and make a choice! Distributed System How about MySQL? Server Replication K/V ColFam 26 I believe we could, Bob.

27 How about MySQL? I believe we could, Bob. wait, WHAT? 27

28 Great tools are still great tools. How about MySQL? I believe we could, Bob. wait, WHAT? 28

29 Great tools are still great tools. Understand your needs before you choose. 29

30 So what's so special about? 30

31 Riak KV Distributed System Server Replication Single Server Locally Embedded K/V Graph ColFam etc etc 31

32 Riak KV Riak Search Distributed System Server Replication Single Server Locally Embedded K/V Graph ColFam etc etc 32

33 Riak Core Riak KV Riak Search Distributed System Server Replication Single Server Locally Embedded K/V Graph ColFam etc etc 33

34 improvements flow upward Riak Core can flow sideways Distributed System Server Replication Single Server Locally Embedded K/V Graph ColFam etc? 34

35 Will the market need this? Distributed System Server Replication Single Server Locally Embedded K/V Graph ColFam etc? 35

36 Will the market need this? I know how to get there. Distributed System Server Replication Single Server Locally Embedded K/V Graph ColFam etc? 36

37 client application protobuf http The Riak key/value stack: riak_client dynamo model FSMs riak core vnode master k/v vnode storage engine 37

38 client application client application client application protobuf http protobuf http protobuf http riak_client riak_client riak_client dynamo model FSMs dynamo model FSMs dynamo model FSMs the cluster nodes are united by riak core via gossip, consistent hashing, etc vnode master vnode master vnode master k/v vnode k/v vnode k/v vnode storage engine storage engine storage engine 38

39 client application protobuf http riak_client dynamo model FSMs riak core vnode master just a local k/v store: k/v vnode storage engine 39

40 client application just an abstract k/v store: protobuf http riak_client dynamo model FSMs riak core vnode master k/v vnode storage engine 40

41 client application protobuf http riak_client dynamo model FSMs a distributed system at heart: riak core vnode master k/v vnode storage engine 41

42 client application failure detection protobuf http virtual nodes sloppy quorums riak_client dynamo model FSMs a distributed system at heart: vector clocks riak core vnode master dynamic membership gossip remote dispatch k/v vnode storage engine 42

43 client application protobuf http riak_client dynamo model FSMs carefully managed complexity... riak core vnode master k/v vnode storage engine 43

44 client application protobuf http riak_client dynamo model FSMs allows simplicity at the edges riak core vnode master k/v vnode storage engine 44

45 let's make a dist. search system! client application protobuf http riak_client dynamo model FSMs riak core vnode master k/v vnode k/v storage engine 45

46 let's make a dist. search system! client application protobuf http riak_client adapted FSMs dynamo model FSMs riak core vnode master k/v vnode k/v storage engine 46

47 let's make a dist. search system! client application protobuf http riak_client adapted FSMs dynamo model FSMs riak core vnode master search vnode k/v vnode search storage k/v storage engine 47

48 solr lucene protobuf http done! search client adapted FSMs riak core riak_client dynamo model FSMs vnode master search vnode k/v vnode search storage k/v storage engine 48

49 solr lucene protobuf http the sum is greater than the parts search client riak_client adapted FSMs dynamo model FSMs riak core vnode master search vnode k/v vnode search storage k/v storage engine 49

50 I don't know what's next 50

51 another I don't know what's next, but I know how to build it. interface solr lucene protobuf http X client search client riak_client more FSMs adapted FSMs riak core vnode master dynamo model FSMs X vnode search vnode k/v vnode X storage search storage k/v storage engine 51

52 fault-tolerance I don't know what's next, but I know how to build it. interoperability predictable availability scalability riak core ease of operations pluggable protocols flexible storage 52

53 interoperability scalability fault-tolerance predictable availability ease of operations pluggable protocols flexible storage I know how to build it, but you don't have to. 53

54 our earlier evaluation criteria: innate/exposed data model distribution/operational model 54

55 innate/exposed data model distribution/operational model fully distributed system 55

56 innate/exposed data model distribution/operational model flexible and growing fully distributed system 56

57 Justin Sheehy 57