HBASE + HUE THE UI FOR APACHE HADOOP

Transcription

1 HBASE + HUE THE UI FOR APACHE HADOOP Abraham Elmahrek LA HBase User Group - Dec 12, 2013

2 WHAT IS HUE? WEB INTERFACE FOR MAKING HADOOP EASIER TO USE Suite of apps for each Hadoop component, like Hive, Pig, Impala, Oozie, Solr, Sqoop2, HBase...

3 VIEW FROM 30K FEET Hadoop Web Server You and even that friend that uses IE9 ;)

4 ECOSYSTEM FILE BROWSER PIG JOB BROWSER JOB DESIGNER OOZIE HIVE IMPALA METASTORE BROWSER SEARCH HBASE BROWSER SQOOP ZOOKEEPER USER ADMIN DB QUERY SPARK HOME...

5 TARGET OF HUE GETTING STARTED WITH HADOOP BEING PRODUCTIVE EXPLORING DIFFERENT ANGLES OF THE PLATFORM LET ANY USER FOCUS ON BIG DATA PROCESSING BEING COMPATIBLE WITH ANY HADOOP VERSION (0.20/1.2.0/2.3.0)

6 OPEN SOURCE COMMITS 31 CONTRIBUTORS 477 STARS 157 FORKS github.com/cloudera/hue

7 THE CORE TEAM PLAYERS ABRAHAM ELMAHREK ROMAIN RIGAUX ENRICO BERTI CHANG BEER team.gethue.com

8 GROWING COMMUNITY Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Posts 0 Topics hue-user@

9 HISTORY HUE 1 Desktop-like in a browser, did its job but pretty slow, memory leaks and not very IE friendly but definitely advanced for its time ( ).

10 HISTORY HUE 2 The first flat structure port, with Twitter Bootstrap all over the place.

11 HISTORY HUE 2.5 New apps, improved the UX adding new nice functionalities like autocomplete and drag & drop.

12 HISTORY HUE 3 ALPHA Proposed design, didn t make it.

13 HISTORY HUE 3 Transition to the new UI, major improvements and new apps.

14 HISTORY HUE 3.5 Where we are now, new UI, several new apps, the most user friendly features to date.

15 BIG DATA MEANS HARDER SEARCH PROBLEMS. HBASE APPLICATION DEVELOPMENT WOES. UNDERSTANDING DATA BETTER.

16 HBASE BROWSER PROBLEMS LACK OF ACCESSIBILITY LACK OF FAMILIARITY USABILITY CHALLENGES

17 HBASE BROWSER DESIGN CHALLENGES PROBLEM HBase is a key-value store, not a traditional RDB SOLUTION Collapse Sparse Data across HBase Tables

18 HBASE BROWSER FLEXIBLE SEARCH BAR PROBLEM Need high-level tool for complex data fetching & analysis Row$Key$ Prefix$Scan$ Thri=$Filterstring$ SOLUTION Simple custom query language Supports HBase filter language Supports selection & Copy + Paste, gracefully degrades in IE Autocomplete Help Menu Scan$Length$ Column/Family$Filters$ Searchbar(Syntax(Breakdown(

19 HBASE BROWSER DOES IT SCALE? PROBLEM Performance on Millions of Columns SOLUTION Lazy loading & Truncation using Thrift FilterString Stream raw data cached MEM > DOM elements dynamically generated b64encode binary data to preserve during ASCII dump Detect schema on preview reading byte headers Capped, Lazy Loaded & Bound to DOM via MVVM Pattern IT DOES.

20 ZOOKEEPER BROWSER TECHNICAL DETAILS Communicate with Contrib. REST API View Node Stats View and Edit Node Data And and Remove ZNodes

21 QUERY HBASE WITH HIVE HIVE EDITOR

22 QUERY HBASE WITH PIG PIG EDITOR

23 INDEX HBASE AND SEARCH IT HUE LILY INDEXER CLOUDERA SEARCH

24 DEMO TIME

25 MISSED SOMETHING? learn.gethue.com

26 WHAT S MISSING? SECURITY STARGATE? ANYTHING ELSE? Kerberos SSL HBASE-8818 Thrift V2 Please! Use of Thrift interface over REST interface for HBase Browser You tell us!

27 LINKS WEBSITE BLOG USER GROUP

28 GET HUE CLOUDERA S CDH TARBALL CLOUDERA S DEMO VM Stable and highly tested releases perfectly integrated with the Hadoop ecosystem, automagically configured Try in advance the latest and greatest but you ll have to configure everything on your own. Get to play with Hue and various Hadoop components in 5 minutes. It s a self contained CDH by Cloudera Manager. HORTONWORKS* MAPR* HP CLOUD* environment ready to In HDP there s an old forked version of Hue 2.3. Newer version than HDP, close to the original 2.5 minus apps like HBase, Impala, Sqoop, Search. The newest addition, ships Hue 3.0 through the GreenButton products. * YOUR MILEAGE MAY VARY.

29 GETHUE.COM WHAT ARE YOUR USE CASES? WHICH COMPONENTS DO YOU USE? WHAT WOULD YOU LIKE TO SEE IN HUE? INTERESTED IN CONTRIBUTING? WANNA SAY HELLO? DO YOU WANT A TAILOR MADE TEAM RETREAT?

30 THANKS.