Dr. Chuck Cartledge. 14 Sept. 2015

Transcription

1 CS-695 NoSQL Database Riak (part 2 of 2) Dr. Chuck Cartledge 14 Sept /21

2 Table of contents I 5 Summary 1 Miscellanea 2 Assignment #2 3 Extensions 4 Break 6 Conclusion 7 References 8 Backup slides 2/21

3 Corrections and additions since last lecture. Be sure to look at assignment #02. 3/21

4 Words of explanation. The full text is available at: ~ccartled/ Teaching/2015-Fall/NoSQL/ Assignments/02/ Requires a YouTube video. In general terms: 1 Parse data 2 Create some key/value pair 3 Query database 4 Create movie 4/21

5 Things added to the Dynamo design Fast key/value can seen sort of limiting Connecting keys (links) Querying data Building on distribution (MapReduce) Image from [1]. 5/21

6 Links; or how to get from one key to another Links Links are exactly like links in web pages. Links have a name (tag) Links are unidirectional (i.e., there is a source and a target) Links are created when a node is created A node can have multiple links Links are sort of like edges in a graph. They are not hyper-edges. 6/21

7 Links; or how to get from one key to another Creating a link It all happens in an HTTP header. Link: <target bucket key>; riaktag=somethingmeannigfull Like: Link: </riak/bucket/key>; riaktag="whatever" A link is created from this source bucket key to the target bucket key. For example: curl -X PUT -H "Content-Type: application/json" -H "Link: </riak/animals/polly>; riaktag="contains" -d {"room" : 101} A link is created from cages/1 (source) to animals/polly (target). Link is unidirectional, so target has no knowledge of linkage. 7/21

8 Links; or how to get from one key to another Walking links Once unidirectional links are created, they can be walked, and the Riak graph can be traversed. curl URL to key/,, Where,, are bucket name, and link name respectively (may be blank). Where is 0, 1, or blank where 0 is last link only, 1 is all intermediate links, and blank is current link curl Returns links that are attached to the animals bucket. 8/21

9 Break time. Take about 10 minutes. 9/21

10 Riak s implementation of MapReduce Its not the same as Hadoop s MapReduce Somethings are the same: Application moved to data Map functions are run in parallel Somethings are different: No Map output sort No Map output shuffle Limited language support No STDIN, STDOUT support See Daniel Reverri s slide show at: 10/21

11 Searching Riak values Riak can organize values Requires changes to the default installation: Based on Lucene 1 search technology Enable searching in /etc/riak/riak.conf Change bucket properties curl curl curl -X PUT -H "Content-Type: application/json" -d {"props":{"precommit": [{"mod": "riak search kv hook","fun":"precommit"}]}} /21

12 Searching Riak values Search operators and terms: 2 Fields: title:"the Right Way" AND text:go Wildcards: te?t test* te* Fuzzy: roam~ roam~0.8 Proximity: jakarta apache ~10 Range: mod date:[ TO ] title:{aida TO Carmen} Booleans: AND and OR: "jakarta apache" OR jakarta "jakarta apache" AND "Apache Lucene" NOT: "jakarta apache" NOT "Apache Lucene" Grouping: (jakarta OR apache) AND website /queryparsersyntax.html 12/21

13 Searching Riak values Example searches (1 of 2) curl urltolocalhost/solr/bucket/select?q=query curl q=breed:shepherd 13/21

14 Searching Riak values Example searches (2 of 2) Depends on configuring Riak as per backup slides 3 : curl "$RIAK HOST/search/query/famousAlso? wt=json&q=leader b:true" curl "$RIAK HOST/search/query/famousAlso? wt=json&q=age i:%5b30%20to%20*%5d" curl "$RIAK HOST/search/query/famousAlso? wt=json&q=name s:lion*" /21

15 Searching Riak values JSON field naming requirements. The default search and indexer has special suffix requirements for JSON field names. Searchable fields must end in s Indexable fields must end in i Boolean fields must end in b Among lots of others. Image from [3]. The complete schema is fairly long default schema.xml 15/21

16 Strengths and weaknesses Good and not so good Strengths Designed for Internet scale operations Removes single points of failure Maximizes changing in size Supports about a dozen different languages Extensible (via Erlang) Weaknesses Queryability Complex data structures Some limitations using Javascript vs. Erlang...not every problem needs ACID compliance or the ability to enforce a schema. Redmond and Wilson [2] 16/21

17 Applicabilities Good for, and not so good for Good fit Storing session data User profile and preferences Shopping cart data When indices are not needed Not so good fit Relationships between data Multioperation transactions Query by data Operations on sets 17/21

18 What have we covered? Reviewed assignment #02 Covered Riak extensions Remember Assignment #02 due before next class Next time: CRUDy HBase 18/21

19 References I [1] Vlad Mackevic, Academic writing. why do you need to acknowledge your weaknesses?, [2] Eric Redmond and Jim R Wilson, Seven databases in seven weeks, Pragmatic Bookshelf, [3] Unknown Staff, Hatsu basho 2013, yokozuna dohyo iri, /21

20 These steps are needed to create and activate search capabilities. Setup 1 of 2 1 export RIAK HOST= 2 Created a bucket type called animals: riak-admin bucket-type create animals {"props":{} 3 Activated the new bucket type: riak-admin bucket-type activate animals 4 Created a default search index (search indices are global): curl -XPUT $RIAK HOST/search/index/famousAlso -H Content-Type: application/json -d {"schema":" yz default"} 20/21

21 These steps are needed to create and activate search capabilities. Setup 2 of 2 1 Tied the search index to the bucket type: curl -XPUT $RIAK HOST/types/animals/buckets/cats2/props -H Content-Type: application/json -d {"props":{"search index":"famousalso"}} 2 Populated the bucket type curl -XPUT $RIAK HOST/types/animals/buckets/cats2/keys/panthro -H Content-Type: application/json -d {"name s":"panthro", "age i":36} 3 Queried the index curl "$RIAK HOST/search/query/famousAlso?wt=json&q=leader b:true" 21/21