Column-Family Stores: Cassandra

Transcription

1 NDBI040: Big Data Management and NoSQL Databases h p:// svoboda/courses/ ndbi040/ Lecture 10 Column-Family Stores: Cassandra Mar n Svoboda svoboda@ksi.mff.cuni.cz Charles University in Prague, Faculty of Mathema cs and Physics Czech Technical University in Prague, Faculty of Electrical Engineering

2 Lecture Outline Column-family stores General introduc on Apache Cassandra Data model Cassandra query language DDL statements DML statements NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

3 Wide Column Stores Data model Column family (table) Table is a collec on of similar rows (not necessarily iden cal) Row Row is a collec on of columns Should encompass a group of data that is accessed together Associated with a unique row key Column Column consists of a column name and column value (and possibly other metadata records) Scalar values, but also flat sets, lists or maps may be allowed NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

4 Wide Column Stores Query pa erns Create, update or remove a row within a given column family Select rows according to a row key or simple condi ons Warning Wide column stores are not just a special kind of RDBMSs with a variable set of columns! NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

5 Wide Column Stores Suitable use cases Event logging, content management systems, blogs, I.e. for structured flat data with similar schema When not to use ACID transac ons are required Complex queries: aggrega on (SUM, AVG, ), joining, Early prototypes: i.e. when database design may change Representa ves Apache Cassandra, Apache HBase, Apache Accumulo, Hypertable, Google Bigtable NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

6 Wide Column Stores Representa ves NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

7 Apache Cassandra

8 Apache Cassandra Column-family database h p://cassandra.apache.org/ Features Open-source, high availability, linear scalability, sharding (spanning mul ple datacenters), peer-to-peer configurable replica on, tunable consistency, MapReduce support Developed by Apache So ware Founda on Originally at Facebook Implemented in Java Opera ng systems: cross-pla orm Ini al release in 2008 NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

9 Data Model Database system structure Instance keyspaces tables rows columns Keyspace Table (column family) Collec on of (similar) rows Table schema must be specified, yet can be modified later on Row Collec on of columns Rows in a table do not need to have the same columns Each row is uniquely iden fied by a primary key Column Name-value pair + addi onal data NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

10 Data Model Column values Empty value Atomic value Na ve data types such as texts, integers, dates, Tuples Tuple of anonymous fields, each of any type (even different) User defined types (UDT) Set of named fields of any type Collec ons Lists, sets, and maps Nested tuples, UDTs, or collec ons are allowed, but currently only in frozen mode (such elements are serialized when stored) NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

11 Data Model Collec ons List Sorted collec on of non-unique values List elements are ordered by their posi ons Not always recommended because of performance issues Internal read-before-write opera ons have to be executed Set Sorted collec on of unique values Map Sorted collec on of key-value pairs Map elements are ordered by their keys Keys must be unique NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

12 Sample Data Table of actors id trojan name ( Ivan, Trojan ) year 1964 movies { samotari, medvidek } machacek name ( Jiří, Macháček ) year 1966 movies { medvidek, vratnelahve, samotari } schneiderova name ( Jitka, Schneiderová ) year 1973 movies { samotari } sverak name ( Zdeněk, Svěrák ) year 1936 movies { vratnelahve } NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

13 Sample Data Table of movies id samotari tle Samotáři year 2000 actors null genres [ comedy, drama ] medvidek tle Medvídek director ( Jan, Hřebejk ) actors { trojan: Ivan, machacek: Jirka } year 2007 vratnelahve tle Vratné lahve year 2006 actors { machacek: Robert Landa } zelary tle Želary year 2003 actors {} genres [ romance, drama ] NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

14 Data Model Addi onal data associated with the whole column in case of atomic values, or every element of a collec on Time-to-live (TTL) A er a certain amount of me (number of seconds) a given value is automa cally deleted Timestamp (write me) Timestamp of the last value modifica on Assigned automa cally or manually as well Both the records can be queried Unfortunately not in case of collec ons and their elements NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

15 Cassandra API CQLSH Interac ve command line shell Uses CQL (Cassandra Query Language) Client drivers Provided by the community Available for various languages Java, Python, Ruby, PHP, C++, Scala, Erlang, NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

16 Query Language CQL = Cassandra Query Language Declara ve query language Inspired by SQL DDL statements creates a new keyspace creates a new table DML statements selects and projects rows from a single table inserts rows into a table updates columns of rows in a table removes rows from a table NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

17 Keyspaces CREATE KEYSPACE CREATE KEYSPACE IF NOT EXISTS keyspace name WITH option name = option value Creates a new keyspace Replica on op on is mandatory (one replica on factor) (individual replica on factor for each data center) NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

18 Keyspaces USE Changes the current keyspace USE keyspace name DROP KEYSPACE Removes a keyspace, all its tables, data etc. DROP KEYSPACE keyspace name IF EXISTS ALTER KEYSPACE Modifies op ons of an exis ng keyspace NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

19 Tables CREATE TABLE Creates a new table within the current keyspace Each table must have exactly one primary key specified CREATE TABLE IF NOT EXISTS table name ( column name type PRIMARY KEY,, primary key ) NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

20 Tables Examples tables for actors and movies NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

21 Tables Primary key has two parts: Compulsory par on key Single column or mul ple columns Describes how table rows are distributed among par ons Op onal clustering columns Defines the clustering order, i.e. how table rows are locally stored within a par on PRIMARY KEY ( column name ( column name, ), column name ) NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

22 Tables DROP TABLE Removes a table together with all data it contains DROP TABLE table name IF EXISTS TRUNCATE TABLE Preserves a table but removes all data it contains TRUNCATE TABLE table name ALTER TABLE Allows to alter, add or drop table columns NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

23 Types Types of columns Na ve types Tuples Collec on types: lists, sets, and maps User-defined types native type TUPLE < type > LIST < type > SET < type > MAP < type, type >..., NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

24 Types Na ve types,,, Signed integers (1B, 2B, 4B, 8B) Arbitrary-precision integer Variable-precision decimal, Floa ng point numbers (4B, 8B) Boolean values and NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

25 Types Na ve types, UTF8 encoded string Enclosed in single quotes (not double quotes) Escaping sequence: ASCII encoded string,, Dates, mes and mestamps E.g.,, NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

26 Types Na ve types 8B signed integer Only 2 opera ons supported: incremen ng and decremen ng I.e. value of a counter cannot be set to a par cular number Restric ons in usage Counters cannot be a part of a primary key Either all table columns (outside the primary key) are counters, or none of them TTL is not supported arbitrary bytes IP address (both IPv4 and IPv6) NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

27 Literals Tuple and collec on literals Literals for tuples, lists, sets, and maps ( term ) [, term ] {, term }, { term : term }, NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

28 Selec on SELECT statement Selects matching rows from a single table SELECT clause FROM clause WHERE clause GROUP BY clause ORDER BY clause LIMIT clause ALLOW FILTERING NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

29 Selec on Clauses of SELECT statements columns or values to appear in the result single table to be queried filtering condi ons to be applied on table rows columns used for grouping of rows criteria defining the order of rows in the result number of rows to be included in the result Example NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

30 Selec on FROM clause Defines a single table to be queried From the current / specified keyspace I.e. joining of mul ple tables is not possible FROM keyspace name. table name NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

31 Selec on WHERE clause One or more rela ons a row must sa sfy in order to be included in the query result WHERE relation AND Only simple condi ons can be expressed and not all rela ons are allowed, e.g.: only primary key columns can be involved unless secondary index structures exist non-equal rela ons on par on keys are not supported NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

32 Selec on WHERE clause: rela ons column name = term ( column name, )!= < <= => > IN ( term ), CONTAINS term CONTAINS KEY NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

33 Selec on WHERE clause: rela ons Comparisons,,,,, IN Returns true if the actual value is one of the enumerated CONTAINS May only be used on collec ons (lists, sets, and maps) Returns true if a collec on contains a given element CONTAINS KEY May only be used on maps Returns true is a map contains a given key NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

34 Selec on SELECT clause Defines columns or values to be included in the result = all the table columns Aliases can be defined using SELECT DISTINCT * selector AS identifier, DISTINCT duplicate rows are removed NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

35 Selec on SELECT clause: selectors column name term COUNT ( * ) WRITETIME ( column name ) TTL ( column name ) COUNT(*) Number of all the rows in a group (see aggrega on) WRITETIME and TTL Selects mestamp / remaining me-to-live of a given column Cannot be used on collec ons and their elements Cannot be used in other clauses (e.g. WHERE) NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

36 Selec on ORDER BY clause Defines the order of rows returned in the query result Only orderings induced by clustering columns are allowed! ORDER BY column name ASC DESC, LIMIT clause Limits the number of rows returned in the query result LIMIT integer NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

37 Selec on GROUP BY clause Groups rows of a table according to certain columns Only groupings induced by primary key columns are allowed! GROUP BY column name, When a non-grouping column is selected without an aggregate func on, the first value encounter is always returned NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

38 Selec on GROUP BY clause: aggregates Na ve aggregates column Number of all the values in a given column values are ignored column, column Minimal / maximal value in a given column Sum of all the values of a given column Average of all the values of a given column User-defined aggregates NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

39 Selec on ALLOW FILTERING modifier By default, only non-filtering queries are allowed I.e. queries where the number of rows read the number of rows returned Such queries have predictable performance They will execute in a me that is propor onal to the amount of data returned enables (some) filtering queries NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

40 Inser ons INSERT statement Inserts a new row into a given table When a row with a given primary key already exists, it is updated At least primary key columns must be specified and they have to always be explicitly enumerated INSERT INTO keyspace name. table name ( column name, ) VALUES ( term, ) IF NOT EXISTS USING update parameters NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

41 Inser ons Example NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

42 Inser ons and Updates Update parameters TTL: me-to-live or or simply missing for persistent values TIMESTAMP: write me TIMESTAMP integer TTL AND Only newly inserted / updated values are really affected NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

43 Updates UPDATE statement Updates exis ng rows within a given table When a row with a given primary key does not yet exist, it is inserted All primary key columns must be specified in the WHERE clause UPDATE keyspace name. table name USING update parameters SET assignment, WHERE clause NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

44 Updates UPDATE statement: assignments Describe modifica ons to be applied Allowed assignments: Value of a whole column is replaced Value of a list or map element is replaced Value of an UDT field is replaced column name = term column name [ term ] column name. field name NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

45 Updates Examples NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

46 Updates Examples NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

47 Dele ons DELETE statement Removes exis ng rows / columns / collec on elements from a given table DELETE column name column name [ term ] column name. field name, FROM clause WHERE clause NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra

48 Lecture Conclusion Cassandra Column-family store Cassandra query language DDL statements DML statements SELECT, INSERT, UPDATE, DELETE NDBI040: Big Data Management and NoSQL Databases Lecture 10: Column-Family Stores: Cassandra