The State of Apache HBase. Michael Stack

Transcription

1 The State of Apache HBase Michael Stack

2 Michael Stack Chair of the Apache HBase PMC* Caretaker/Janitor Member of the Hadoop PMC Engineer at Cloudera in SF * Project Management Committee

3 What is it?

4 ... is an open source, distributed, scalable, consistent, low latency, non-relational, random access database

5 Built on Apache Hadoop core: Distributed file system (HDFS) App MR ZK HDFS MapReduce HBase persists all data to HDFS Uses Apache ZooKeeper Cluster coordination Goal: Billions of rows X millions of columns on clusters of 'commodity hardware'

6 Inspiration A Google Technology described in a 2006 paper, Bigtable: A Distributed Storage System for Structured Da ta by Chang et al.?

7 First commit... commit 454a9dbe046194f8eef3dddc3e dd5b7a1 Author: Douglass Cutting Date: Tue Apr 3 20:34: HADOOP Add contrib/hbase, a BigTable-like online database.

8 DISTRIBUTIONS

9 When to use it?

10 BIG Data

11 le! ca s

12 Low-latency, online, random read/writes

13 Datamodel* *Very like Google Bigtable model only different nomenclature

14 DataModel: A Bigtable! 0-N Bigtable(s) Rows x Column Families Column Families Has columns CF prefix and qualifier e.g. attribute:mimetype

15 Datamodel: Regions Table splits into regions Automatically as table grows Region has contiguous rows [startrow, endrow)

16 DataModel: Sorted & Versioned All is byte [] No native 'types' Schema-less (NoSQL) All is SORTED Rows in byte-lexographical order Columns sorted along row VERSIONED Cells are versioned 3D (timestamp)

17 Datamodel: Strongly consistent Row modifications are atomic Even if thousands of columns on a row Favors consistency over availability Designing applications to cope with concurrency anomalies in their data is very error-prone, timeconsuming, and ultimately not worth the performance gains -- F1: A Distributed SQL Database That Scales

18 Architecture: Physical Cluster is made of a Master and Slaves Nodes HDFS NameNodes HBase Masters ZooKeeper Quorum Slave Boxes (DN + RS)

19 Features Classes to MapReduce HBase tables Query predicate push down via server side filters Coprocessors (stored procedures/triggers) Extensible jruby-based (JIRB) shell Replication Security Table/Column Family Kerberos Authentication, ACLs

20 What to expect Writes: 1-3ms, 1k-20k writes/sec per node Reads: 0-3ms cached, 10-30ms disk 10-40k reads / second / node from cache > if SSD Cell size 0-3MB preferred Column-orientated so wide tables are OK Sparsely populated rows OK

21 Who uses it?

22

23 In Production

24 OLTP & Batch Messages 1B+ users Tens of PBs (compressed) Thousands of machines, Pods of ~200 ODS/Real-time monitoring/timeseries Dual write two clusters Critical eyes and ears

25 All on AWS 5 production clusters and growing Mix of SSD and SATA Billions of page views per month

26 Users Long time HBase user Two clusters of 1k nodes each Master-Master replicating Separate low-latency cluster Up to 1M reads a second

27 Cassini Ebay item search indexing 600M active items in HBase tables 1.4TB of data processed each day 400M puts to HBase each day 250M search metrics per day Two datacenters Growing clusters >1k

28 Deploy types Multitenant multifarious feature store o a.k.a dumping ground o Stumbleupon, Y!, SalesForce Reconciliation store o ebay Timeseries o SalesForce, FB ODS Lots-o-entities store o Flurry, genome o Lots-o-entities BLOBs, FB Messages

29 Who runs the project?

30 Diverse team* COMMITTERS! Preferably ALIVE! *

31 Dev Rate

32 # of commits Total Files 2021 Total Lines of Code Total Commits 6615 (~ 3/day) Authors 39 (

33 JIRA:

34 Commits/Month Over Time (0.94/trunk)

35 HBase Today

36

37 Release every month Each more stable & more performant Some features Currently at Wire compatible between releases

38

39 th Released October 19, months in the making >2000 fixes

40 Big Themes Stability Operability Insight, tools Scalability Evolvability

41 Sampler Pluggable Compression Smarter triggers Hadoop1 AND Hadoop2 Smarter Region Balancer Region Assignment Hardened Coprocessors More hooks

42

43

44 System tables Filesystem Up in zookeeper Over the wire

45 Namespaces Grouping of tables Like database in mysql System/User hbase:meta Quota Coming Security by namespace Grouping on cluster by namespace

46 And more... X-row (in-region) Transactions Query tracing New UI Online Merge Hardened Replication Off-heap bucket cache Metrics2 o Radical revamp

47

48 By the end of the year Rolling upgrade from In-line Cell-tags Security++ ACL down to the Cell-level Cell-level visibility labels Reverse Scan

49 HBase 2014 HBase th Reining in the 99 percentiles Multi-WAL Speculative replica reads More support for multi-tenancy Off-heap

50 HBaseEcosystem

51 OpenTSDB Timeseries Store, index and serve metrics at large scale Make data easily accessible and graphable

52 Haeinsa Haeinsa 란 무엇인가? Is a linearly scalable multi-row, multi-table transaction library for HBase. Haeinsa uses two-phase locking and optimistic concurrency control for implementing transaction. The isolation level of transaction is serializable. Inspired by Google Percolator VCNC

53 Chasm

54

55 How to make it easier writing applications against HBase?

56 Frameworks: Kiji.org Entity-centric, simple model o Types, complex, compound types. Each cell is schema versioned Works across MR & REST, etc. Machine-learning libs Examples, tutorials Production users Open-source

57 Frameworks: CDK APIs providing Dataset abstraction get/put/delete API in AVRO objects Highlights: Supports multiple components flume, morphlines, hive, crunch, hcat Types using Avro and parquet formats Manages schema evolution Open source by Cloudera

58

59 Client-embedded JDBC driver Connection conn = DriverManager.getConnection("jdbc:phoenix:localhost"); Alternate HBase Client API (SQL) Fast! Exploits HBase Coprocessors/Filters Types Aggregations Skip scans Secondary indices

60 Beyond... Hadoop Family evolving, growing No longer just Batch Real-time Streaming October Apache Hadoop 2.0 release an inflection point O'Reilly Strata + Hadoop World NYC 2013 Coming out party New distributions Enterprise

61 Beyond: No longer just batch YARN Distributed scheduling Resource management More than just MR on the cluster Arbitrary Apps Hive speedup Tez/Stinger Storm Streaming Hadoop Storm on YARN

62 Beyond: No longer just batch Apache Apache Cluster management Cloudera Impala Scalable low-latency SQL query HDFS (& HBase) Apache Drill & HBase!

63 Thank You!

64 TODO DBA: R (read), W (write), C (create), X (execute), A (admin). cell-level security. Every cell in an Accumulo store can have a label, stored effectively as part of the key, which is used to determine whether a value is visible to a given subject or not. The label is not an ACL, it is a different way of expressing security policy. A label instead turns this on its head and describes the sensitivity of the information to a decision engine that then figures out if the subject is authorized to view data of that sensitivity based on (potentially, many) factors. Then, as of HBASE-7662, HBase can store into and apply ACLs from cell tags, extending the current HBase ACL model down to the cell. Finally, we have also contributed transparent server side encryption, as HBASE-7544, for additional assurance against accidental leakage of data at rest, which is at this time an HBaseonly feature. Auto-manages partitioning Storage machinery in the RS I like the Latency/Throughput/Read/Write axis in Nick