introduction to riak 1.2 Good data need good storage POWERED BY SmartRecruiters

Transcription

1 introduction to riak 1.2 Good data need good storage POWERED BY SmartRecruiters

2 pawel

3 hiringwww.smartrecruiters.com made easy.

4

5 introduction to riak 1.2 how we do it at smartrecruiters what is your data closer look at riak 1.2 basic concepts configuration management java client and API overview searching with riak

6 I won't say much about mapreduce links walking secondary Indexes alternative persistent store backend Bitcask, InnoDB, LevelDB, Memory and more managing cluster command line tools (not all) multiple distribution riak, riak enterprise, riak cs

7 at smartrecruiters one oracle instance for storing data. mongodb with replica for storing data. one solr instance for search. one neo4j graph instance for relations. there is no much data - we are not facebook - yet!

8 mongodb is great replication and fault tolerance better performance better locking auto sharding read/write concerns for tuning it has embedded mongo for testing good clients we are moving away from morphia to mongo-jackson-mapper

9 mongodb is great but it is a bit complex when you need it to scale

10 preparing for growth

11 what is your data model? - how do you represent data? - can model objects be bound in size? - do you have lots of concurrent updates? - can you detect conflicts and resolve them in runtime? - is your data logically monotonic?

12 logically monotonic data we can model data in way it is always possible to merge without losing data ex. two objects representing keywords mentioned today on twitter can be merged into one objects with all unique keywords ;] ex. two objects representing keywords not mentioned yesterday on twitter cannot be merged... :/

13 a key-value distributed store

14

15 BASIC CONCEPTS distributed, highly available system

16 CAP theorem It is impossible for a distributed computer system to simultaneously provide all three of the following guarantees Consistency (all nodes see the same data at the same time) Availability (a guarantee that every request receives a response about whether it was successful or failed) Partition tolerance (the system continues to operate despite arbitrary message loss or failure of part of the system) You need to choose two! By default Riak team had chosen Availability and Partition tolerance in expense of Consistency.

17 CAP theorem hopefully you can tune this behaviour, see tunable CAP...

18 nodes, vnodes and partitions in this example cluster consists: 32 partitions in cluster (this is configurable) 4 physical nodes 8 vnodes per node 8 partitions per node

19 vnodes and replication object is replicated to N separate partitions on the Riak Ring HINTED HANDOFF - if target vnode is unavailable Riak will try to choose other node which will spawn temporary process to replace failed node and keep operating as usual. this temporary process will try to communicate with failed node and pass data.

20 vnodes and replication For cases where the number of nodes is less than the N value, data will likely be duplicated on some nodes. For example, with N=3 and 2 nodes in the cluster, one node will likely have one replica, and the other node will have two replicas. In general N copies of each piece of data will be stored in the cluster. There are no guarantees that the N replicas will go to N separate physical nodes. Riak will rebalance/re-replicate data after each successful read (if is needed) so called read repair. No master node to manage read/write operation.

21 replication in depth 1. each bucket/key pair is hashed by node handling the request 2. node is requesting a list of preferred vnodes for the hash 3. node is choosing first N vnodes and sends object to parent node of the vnode 4. if node is unavailable requesting node gets next vnodes from preferred list and sends the same info to the next node - HINTED HANDOFF 5. if operation is successful, the requesting node responds to client

22 architecture build on top of erlang ecosystem erlang distribution needs to be installed separately

23 node configuration riak/etc/app.config [ %% Riak Client APIs config {riak_api, [ {pb_ip, " " }, {pb_port, 8087}]}, %% Riak Core config {riak_core, [ %% Default location of ringstate {ring_state_dir, "./data/ring"}, {http, [{" ", 8098}]}, {https, [{" ", 8090}]}, {ssl, [ {certfile, "./etc/cert.pem"}, {keyfile, "./etc/key.pem"} ]}, {handoff_port, 8099 },... ]}, %% Riak KV config {riak_kv, [ {storage_backend, riak_kv_bitcask_backend},... ]

24 node configuration riak/etc/vm.args ## Name of the riak node -name ## Cookie for distributed erlang. All nodes in the same cluster ## should use the same cookie or they will not be able to communicate. -setcookie riak67c e5c9de9bd24f8da591d5f7 ## Heartbeat management; auto-restarts VM if it dies or becomes unresponsive ## (Disabled by default..use with caution!) ##-heart ## Enable kernel poll and a few async threads +K true +A 64 ## Treat error_logger warnings as warnings +W w...

25 buckets organise keys and values in buckets, think of it as tables in SQL

26 default bucket configuration riak/etc/app.config default bucket values [ %% Riak Core config {riak_core, [ {default_bucket_props, [ { n_val,3}, { allow_mult,false}, { last_write_wins,false}, {r, all}, %% n {w, quorum}, %% n/2+1 {dw, 1}, {rw, 2}, {precommit, []}, {postcommit, []}, {chash_keyfun, {riak_core_util, chash_std_keyfun}}, {linkfun, {modfun, riak_kv_wm_link_walker, mapreduce_linkfun}} ]} ]} ] more

27 working with buckets curl -v -X PUT -H "Content-Type: application/json" -d '{"props":{"n_val":2, "dw":1, "w":2, "r":1}}' curl -v curl -v curl -v more in general there is no limit to buckets number. that is: buckets comes for free as long as default configuration is used, no need to gossip custom bucket's properties.

28 tunable CAP N default to 3 - number of replicas R, W, DW, RW - defaults to quorum PR, PW - defaults to 0 rw - quorum for both operations (get and put) involved in deleting an object (default is set at the bucket level) r - (read quorum) how many replicas need to agree when retrieving the object w - (write quorum) how many vnodes must confirm receiving write request before returning a successful response dw - (durable write quorum) how many replicas to commit to durable storage before returning a successful response pr - (primary read quorum) works like r but requires that the nodes read from are not fallback nodes pw - (primary write quorum) how many replicas to commit to primary nodes before returning a successful response possible values 1..N, one, all (N), quorum (N/2 + 1)

29 tunable CAP how many down nodes cluster can tolerate when performing operations? Nodes N R W store operation read operation

30 MANAGEMENT

31 Web console riak/etc/app.config [ {riak_core, [ {https, [{ " ", 8090 }]}, {ssl, [ {certfile, "./etc/cert.pem"}, {keyfile, "./etc/key.pem"} ]} ]}, {riak_control, [ %% Set to false to disable the admin panel. {enabled, true}, {auth, userlist}, %% If auth is set to 'userlist' then this is the %% list of usernames and passwords for access to the %% admin panel. {userlist, [{"user", "pass"}]} ]} ] more

32

33

34 cluster status curl -v or riak/bin/riak-admin status node_gets Number of GETs coordinated by this node, including GETs to non-local vnodes on this node within the last minute node_gets_total Number of GET operations coordinated by vnodes on this node since node was started node_puts Number of PUTs coordinated by this node, including PUTs to non-local vnodes on this node within the last minute node_puts_total Number of PUT operations coordinated by vnodes on this node since node was started vnode gets Number of GET operations coordinated by vnodes on this node within the last minute vnode_gets_total Number of GET operations coordinated by vnodes on this node since node was started vnode_puts Number of PUT operations coordinated by vnodes on this node within the last minute vnode_puts_total Number of PUT operations coordinated by vnodes on this node since node was started read_repairs Number of read repair operations this this node has coordinated in the last minute read_repairs_total: Number of read repair operations this node has coordinated since node was started request-response time statisitics number of siblings statisitics object size statistics system cpu/threads/memory usage statistics cluster operating statistics more

35 cluster backup centralized backups riak/bin/riak-admin backup riak1.srv riak67c e5c9de9bd24f8da591d5f7 log all riak/bin/riak-admin restore riak1.srv riak67c e5c9de9bd24f8da591d5f7 log all one node backup - copy/archive files from following directories riak/data/bitcask riak/data/ring riak/etc

36 DEVELOPMENT fetching and storing

37 HTTP API basic api to communicate with cluster, supports: storing objects fetching objects deleting objects searching objects link walking map reduce

38 GET Object curl -v HTTP/ OK X-Riak-Vclock: a85hygbgzgdkbvicnmcuy5reyvfd6ejivcwa= Vary: Accept-Encoding Server: MochiWeb/1.1 WebMachine/1.9.0 (someone had painted it blue) Link: </riak/cars>; rel="up" Last-Modified: Wed, 03 Oct :58:23 GMT ETag: "3mf1qgFRZiIL02ilCVZHVr" Date: Fri, 05 Oct :00:14 GMT Content-Type: application/json; charset=utf-8 Content-Length: 390

39 PUT/POST Object curl -v -X PUT -d '{"name":"baz", friends:["max"]}' -H "Content-Type: application/json" -H "X-Riak-Vclock: IszMk55zKYEhnzWBlKIniO8mUBAA==" -H "X-Riak-Meta-MyCustomHeader:.." returnbody=true&r=2&w=5&dw=1 to generate unique around cluster, sortable keys with POST key can be auto generated Riak generates keys using an Erlang-generated unique ID and a timestamp hashed with SHA-1 and base-62 encoded for URL safety. Thus Riak doesn't make any guarantees of uniqueness for keys it generates, but will be extremely unlikely to get collisions.

40 java PB* client * final PBClientConfig config = new PBClientConfig.Builder().withPort( 8097).withConnectionTimeoutMillis( 200).build(); // round robined between nodes for requests and retries final PBClusterConfig pbclusterconfig = new PBClusterConfig(200); pbclusterconfig.addhosts(config, " ", " ",..); final IRiakClient riakclient = RiakFactory.newClient(pbClusterConfig); riakclient.setclientid( ClientId.generate()); final Bucket storebucket = riakclient.createbucket( "cars").allowsiblings( true).lazyloadbucketproperties().execute(); Car carsobject = //create any Car object carsobject = storebucket. store(carsobject). withoutfetch().returnbody( true).execute(); final Bucket fetchbucket = riakclient.fetchbucket( "cars").execute(); final Car fetched = bucket. fetch(new Car(id)).execute();

41 helpers com.basho.riak.client.cap.* public interface Retrier { <T> T attempt(callable<t> command) throws RiakRetryFailedException; } public interface Converter<T> { IRiakObject fromdomain(t domainobject, VClock vclock) throws ConversionException; T todomain(iriakobject riakobject) throws ConversionException; } public interface ConflictResolver<T> { T resolve(final Collection<T> siblings); } public interface Mutation<T> { T apply(t original); }

42 balancing requests? YEP:)

43 DEVELOPMENT commit hooks

44 hooks works like triggers in good old SQL RDBMS can be triggered before or after object is persisted hooks can modify object before persisting prevent object from being persisted allow object to be persisted post-commit hooks shouldn't modify target object hooks are stored per bucket in its properties pre-commit hooks can be written in javascript or in erlang post-commit hooks can be written in erlang only %% Riak Core config {riak_core, [ {default_bucket_props, [ {n_val,3}, {allow_mult,false}, {precommit, [ { "mod": "riak_search_kv_hook", "fun": "precommit" } ]}, {postcommit, []}, ]}

45 DEVELOPMENT searching and Yokozuna

46 search hooks search index is persisted on different vnodes than data can be stored data is stored in merge-index inspired by Lucene file format, Bitcask (Riak KV), and SSTables (Google's file structure) Riak Search uses timestamps, rather than vector clocks, for performance reasons

47 search hooks Riak Search does not use quorum values when writing (indexing) data. The data is written in a fire and forget model. Riak Search does use hinted-handoff to remain write-available when a node goes offline Riak Search does not use quorum values when reading (querying) data. Only one copy of the data is read, and the partition is chosen based on what will create the most efficient query plan overall

48 search hooks Riak Search indexes currently have no form of anti-entropy (such as read-repair). Furthermore, for performance and load balancing reasons, Search reads from 1 random node. This means that when a replica loss has occurred, inconsistent results may be returned.

49 configuration search is disabled by default %% Riak Search Config {riak_search, [ %% To enable Search functionality set this 'true'. {enabled, false} ]}, %% Merge Index Config {merge_index, [ %% The root dir to store search merge_index data {data_root, "./data/merge_index"}, %% Size, in bytes, of the in-memory buffer. {buffer_rollover_size, }, %% This is the maximum number of segments that will be %% compacted at once. {max_compact_segments, 20} ]}, next install hook for bucket riak/bin/search-cmd install cars OR curl -X PUT -H "content-type:application/json " /riak/friend -d {"props":{"precommit":[{"mod":"riak_search_kv_hook ","fun":" precommit"}]}}

50 indexing curl -v -X PUT friend/ d '{"name":"baz", friends:{"name": "max milus", "age_int":"21st"}}' -H "Content-Type: application/json" <?xml version="1.0" encoding="utf-8"?> <response> <lst name="responseheader"> <int name="status">0</int> <int name="qtime">21</int> <lst name="params"> <str name="indent">on</str> <str name="start">0</str> <str name="q">mil*</str> <str name="q.op">or</str> <str name="filter"></str> <str name="df">friends_name</str> <str name="wt">standard</str> <str name="version">1.1</str> <str name="rows">1</str> </lst> </lst><result name="response" numfound="1" start="0"> <doc> <str name="id"> </str> <str name="name">baz</str> <str name="friends_name">max milus</str> <str name="friends_age">21</str> </doc>

51 schema riak/bin/search-cmd show-schema riak/bin/search-cmd show-schema friend FIELDNAME_num - Numeric field. Uses Integer analyzer. Values are padded to 10 characters. FIELDNAME_int - Numeric field. Uses Integer analyzer. Values are padded to 10 characters. FIELDNAME_dt - Date field. Uses No-Op analyzer. FIELDNAME_date - Date field. Uses No-Op analyzer. FIELDNAME_txt - Full text field. Uses Standard Analyzer. FIELDNAME_text - Full text field. Uses Standard Analyzer. All other fields use the Whitespace analyzer. { schema, [ {version, "1.1"}, {n_val, 1}, {default_field, "friends_name"}, {analyzer_factory, {erlang, text_analyzers, noop_analyzer_factory}} ], [ %% Field names ending in "_num" are indexed as integers {dynamic_field, [ {name, "*_num"}, {type, integer}, {skip, false}, {analyzer_factory, {erlang, text_analyzers, integer_analyzer_factory}} ]},

52 analyzers white space analyzer including spaces, tabs, newlines, carriage returns, etc. standard analyzer uses dictionary to split english sentences integer analyzer no-op analyzer doesn't tokenize a field custom analyzer written in Erlang NOTE: white space analyzer preserves capitalization and punctuation

53 data types JSON Content-Type: application/json { "name":"alyssa P. Hacker", "bio":"i'm an engineer, making awesome things.", "favorites":{ "book":"the Moon is a Harsh Mistress", "album":"magical Mystery Tour" } } indexed fields: name, bio, favorites_book, favorites_album XML Content-Type: application/xml PLAIN TEXT Content-Type: text/plain ERLANG TERM Content-Type: application/x-erlang CUSTOM FORMAT

54 search quick overview Support for various mime types (JSON, XML, plain text, Erlang, Erlang binaries) for automatic data extraction Support for various analyzers (to break text into tokens) including a white space analyzer, an integer analyzer, and a no-op analyzer Robust, easy-to-use query language Exact match queries Wildcards Inclusive/exclusive range queries o AND/OR/NOT support Grouping Prefix matching Proximity searches Term boosting Solr-like interface via HTTP (not Solr compatible) Protocol buffers interface Scoring and ranking for most relevant results Search queries as input for MapReduce jobs

55 Yokozuna overview very new project - just released its alpha architecture Jetty/Solr operating system process per node. All vnodes on a node share that one instance. includes Solr handles all JVM management for you.

56 how yammer did it? com/blog/technical/2011/03/28/riakand-scala-at-yammer/

57 FIN Questions?