Building a Cassandra based applica1on from scratch. Patrick McFadin Cassandra Summit 2012 #cassandra12

Size: px

Start display at page:

Download "Building a Cassandra based applica1on from scratch. Patrick McFadin Cassandra Summit 2012 #cassandra12"

Ralf Lloyd
6 years ago
Views:

1 Building a Cassandra based applica1on from scratch Patrick McFadin Cassandra Summit 2012 #cassandra12

2 This is me Chief Architect at Hobsons Hobsons is an educa1on services company. More here: Cassandra user since.7 Follow me

3 Goals Take a new concept What s the data model?!?! Some sample code You get homework! (If you want)

4 Here s the plan Conceptualize a new applica1on Iden1fy the en1ty tables Iden1fy query tables Code. Rinse. Repeat. Deploy Profit! * I ll be using the term Tables which is equivalent to Column Families

5 Recommended Video Title Description Username Start with a concept Meow Ads by Google Video Sharing Website Rating: Tags: Foo Bar Comments Upload New! *Cat drawing by goodrob13 on Flickr

6 Break down the features Post a video* View a video Add a comment Rate a video Tag a video * Not talking about transcoding! Check out zencoder.com, it s pre^y sweet.

7 Create En1ty Tables Basic storage unit

8 Users!"#$%&'# (&"")*$+,-$"./&'# 0&"./&'# Similar to a RDBMS table. Fairly fixed columns Username is unique Use secondary indexes on firstname and lastname for lookup Adding columns with Cassandra is super easy CREATE TABLE users ( username varchar PRIMARY KEY, firstname varchar, lastname varchar, password varchar );

9 Users: The set code static void setuser(user user, Keyspace keyspace) { // Create a mutator that allows you to talk to casssandra Mutator<String> mutator = HFactory.createMutator(keyspace, stringserializer); try { // Use the mutator to insert data into our table mutator.addinsertion(user.getusername(), "users", HFactory.createStringColumn("firstname", user.getfirstname())); mutator.addinsertion(user.getusername(), "users, HFactory.createStringColumn("lastname", user.getlastname())); mutator.addinsertion(user.getusername(), "users", HFactory.createStringColumn("password", user.getpassword())); // Once the mutator is ready, execute on cassandra mutator.execute(); } } catch (HectorException he) { } he.printstacktrace(); You can implement the get

10 Videos!"#$%&# '((&)*!"#$%+,-$ (.$/+,-$ )$.0/"12"%3 4,5. Use a UUID as a row key for uniqueness Allows for same video names Tags should be stored in some sort of delimited format Index on username may not be the best plan CREATE TABLE videos ( videoid uuid PRIMARY KEY, videoname varchar, username varchar, description varchar, tags varchar );

11 Videos: The get code static Video getvideobyuuid(uuid videoid, Keyspace keyspace){ Video video = new Video(); //Create a slice query. We'll be getting specific column names SliceQuery<UUID, String, String> slicequery = HFactory.createSliceQuery(keyspace, uuidserializer, stringserializer, stringserializer); slicequery.setcolumnfamily("videos"); slicequery.setkey(videoid); slicequery.setcolumnnames("videoname","username","description","tags"); // Execute the query and get the list of columns ColumnSlice<String,String> result = slicequery.execute().get(); // Get each column by name and add them to our video object video.setvideoname(result.getcolumnbyname("videoname").getvalue()); video.setusername(result.getcolumnbyname("username").getvalue()); video.setdescription(result.getcolumnbyname("description").getvalue()); video.settags(result.getcolumnbyname("tags").getvalue().split(",")); } return video; You can implement the set

12 Comments!"#$%&# '((&)* (+$,-./$0'1"/$+1./2*.. (+$,-./$0'1"/$+1./2* Time Order Videos have many comments Use Composite Columns to store user and 1me Value of each column is the text of the comment Order is as inserted Use getslice() to pull some or all of the comments CREATE TABLE comments ( videoid uuid PRIMARY KEY comment varchar );

13 Ra1ng a video!"#$%&# '((&)* +,-"./01%2.- +,-"./0-%-,3 '1%2.-$+* '1%2.-$+* Use counter for single call update rating_count is how many ra1ngs were given rating_total is the sum of ra1ng Ex: ra1ng_count = 5, ra1ng_total = 23, avg ra1ng = 23/5 = 4.6 CREATE TABLE video_rating ( videoid uuid PRIMARY KEY, rating_counter counter, rating_total counter);* * Only valid in CQL 3+

14 Video Event!"#$%&#'()$*+,-$ ).,*./0."-$).,-12 ).%1/0."-$).,-12 ).,*./0."-$).,-12 3"#$%/0."-$).,-12 Time Order Track viewing events Combine Video ID and Username for a unique row Stop 1me can be used to pick up where they lei off Great for usage analy1cs later CREATE TABLE video_event ( videoid_username varchar PRIMARY KEY, event varchar );

15 Create Query Tables Indexes to support fast lookups

16 Lookup Video By Username!"#$%&'# ()*#+,*-./)'#"/&'01.. ()*#+,*-./)'#"/&'01 Username is unique One column for each new video uploaded Column slice for 1me span. From x to y VideoId is added the same 1me a Video record is added CREATE TABLE username_video_index ( username varchar PRIMARY KEY, videoid_timestamp varchar );

17 Videos by Tag!"# $%&'()&.. $%&'()& Tag is unique regardless of video Great for List videos with X tag Tags have to be updated in Video and Tag at the same 1me Index integrity is maintained in app logic CREATE TABLE tag_index ( tag varchar PRIMARY KEY, videoid varchar );

18 Deployment strategies Measure your risk Replica1on factor? Mul1- datacenter? Cost? Performance Today!= tomorrow. Scale when needed Have a expansion plan ready

19 Wrap up Similar data model process to RDBMS to start Query - > Index table Don t be afraid to write in mul1ple tables at once Bonus points: Hadoop and Solr!

20 Go play! Go to: h^p://github.com/pmcfadin Look for projects with cassandra12 Clone or fork my examples Implement stubbed methods Send me your solu1ons: Follow me for

21 Thank You! Connect with me Or linkedin Conference tag #cassandra12

Big Data Development CASSANDRA NoSQL Training - Workshop. November 20 to (5 days) 9 am to 5 pm HOTEL DUBAI GRAND DUBAI

Big Data Development CASSANDRA NoSQL Training - Workshop November 20 to 24 2016 (5 days) 9 am to 5 pm HOTEL DUBAI GRAND DUBAI ISIDUS TECH TEAM FZE PO Box 9798 Dubai UAE, email training-coordinator@isidusnet