Dr. Chuck Cartledge. 1 Oct. 2015

Transcription

1 CS-695 NoSQL Database HBase (part 2 of 2) Dr. Chuck Cartledge 1 Oct /14

2 Table of contents I 1 Miscellanea 2 Assignment #3 3 Extensions 5 Conclusion 6 References 4 Summary 2/14

3 Corrections and additions since last lecture. Assignment #03 is available1. Corrected typos in lecture #005 3/14

4 Words of explanation. The full text is available at: ~ccartled/ Teaching/2015-Fall/NoSQL/ Assignments/03/ In general terms: 1 Parse data 2 Create columnar database 3 Alter the database for new data 4 Query database 5 Create histograms 4/14

5 They are minimal. HBase is more of a data store than a database. Large omission is lack join capability Can be implemented via MapReduce Can be implemented via client app. Nested loop, scanning each table entry against all table other entries Lots of IO means slow 5/14

6 They are minimal. A variety and selection of HBase shell commands 1 General status, version, whoami Management alter, create, describe, disable( all), drop( all), enable( all), list, show filters (part of scan) Data manipulation count, delete(all), get, get counter, incr, put, scan, truncate Surgery tools assign, balancer( switch), close region, compact, flush, major compact, move, split, unassign,zk dump Cluster replication a raft of commands Security tools grant, revoke, user permission 1 6/14

7 They are minimal. Example scan commands 2 scan command applies any filters and returns rows and columns that pass the filter scan hbase:meta scan hbase:meta, LIMIT => 1 scan hbase:meta, LIMIT => 10, COLUMNS => info:regioninfo scan hbase:meta, LIMIT => 1, STARTROW => bethu 2 7/14

8 They are minimal. Example filters 3 Filters return rows that pass the filter criteria (this may be a different type operation than some people are used to performing). Syntax: FilterName (argument, argument,..., argument) Compound operators: AND, OR, SKIP, and WHILE Execution order: parens, SKIP and WHILE, AND, OR Operators, <,<=,=,! =,>=,> PrefixFilter ( Row ) AND PageFilter (1) AND FirstKeyOnlyFilter () A complete list of filters is described in Chapter 75 [2]. Available filters are found using the HBase show filters command /14

9 They are minimal. Misc. sampling of companies using HBase 4 Adobe small cluster Caree.rs high tech hiring firm, backend data, machine learning, hire recommendations Facebook power messages infrastructure Infolinks In-Text ad provider to optimize ad placement OCLC catalog collections from 72,000 libraries OpenLogic world s Open Source packages Pacific Northwest National Laboratory biological data warehouse Stumbleupon real-time data storage and analytics Twitter back up of all mysql tables in production backend WorldLingo machine translation 4 9/14

10 They are minimal. Bloom Filters A space efficient probabilistic data structure designed to answer the question: have I seen this data before?? Small number of hash functions each set single bit in a bit vector Same hash functions used to see if bits are set May return FALSE POSITIVE Will not return FALSE NEGATIVE As filter becomes fuller, more FALSE POSITIVEs Image from [1]. Conceived by Burton Howard Bloom in 1970 as part of a spell checker. 10/14

11 Strengths and weaknesses Good and not so good Strengths Autoversioning of data Scaleability (gigabyte or terabyte databases) Community support Weaknesses Scaleability (gigabyte or terabyte databases) not small databases Small data sets (A few thousand/million rows[2]) Lack of strong query language Requires Hadoop and Zookeeper type infrastructure (i.e., hardware) Lack of sorting and indexing capabilities Lack of data types 11/14

12 Applicabilities Good for, and not so good for Good fit Event logging Content management systems, blogging platforms Counters, and expiring usage cases Not so good fit If ACID is required Aggregating data (SUM, AVG, etc.) Prototyping applications 12/14

13 What have we covered? Reviewed assignment #03 Covered some of the HBase querying capabilities Remember Assignment #03 due before next class Next time: MongoDB 13/14

14 References I [1] Ben Podgursky, Bloomjoin: Bloomfilter + cogroup, [2] Apache Staff, Apache hbase reference guide, /14