C5##54&6*"6*1%2345*D&'*E2)2*F"4G)&"69

Transcription

1 C5##54&6*"6*1%2345*D&'*E2)2*F"4G)&"69!"#$%&'%(?2%3"9*<%263&H&2 I%"7G3)*?262'5% E2)2*F3&5635*267*D&'*E2)2 13)"H5%*,J0*,-./!"#$%&'()*+*,-./0 1%2345*2678"%*&)9*2::&4&2)59;*<44*%&'()9*%595%=57;**>

2 Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle s products remains at the sole discretion of Oracle. Copyright 2018, Oracle and/or its affiliates. All rights reserved. 2

3 1%2345*D&'*E2)2*?262'5% J

4 !"635#)G24 <3)&"62H45 R=56)9 <3)&"62H45 E2)2*F5)9 <3)&"62H45?5)%&39 N6#G) R=56)9 F)%52L&6'*R6'&65 R6)5%#%&95*E2)2*S*B5#"%)&6' F)%G3)G%57 R6)5%#%&95* E2)2 RT53G)&"6 N66"=2)&"6 E2)2 E&93"=5%$* 1G)#G) Q

5 I%23)&324 <3)&"62H45 R=56)9 <3)&"62H45 E2)2*F5)9 <3)&"62H45?5)%&39 N6#G) R=56)9 1HW53)*F)"%5 K27""#8KEXF F)%52L&6'*R6'&65 R6)5%#%&95*E2)2*S*B5#"%)&6' F)%G3)G%57 R6)5%#%&95* E2)2 RT53G)&"6 N66"=2)&"6 E2)2 V")5H""O98<624$)&3*F5%=&359 E&93"=5%$* 1G)#G) U

6 D&'*E2)2*?262'5%! N634G757 A&)(*244*D&'*E2)2*"::5%&6'9*YDE<0* DE!!*267*DE!FZ! R62H459*L299&=5*#2%24454*3"#$*":*72)2* 45=5%2'&6'*<#23(5*F#2%O " KEXF*[\\] KEXF " KEXF*[\\]*1HW53)*F)"%2'5*Y!4"G7Z " X&45*E&::\&6' 267*3(53O&6'*2:)5%*3"#&59! DG&47*267*L262'5*#&#54&659! RLH57757*C5##54&6*V")5H""O " <624$^5*72)2*&69)26)4$ " <624$^5*2)*93245*A&)(*1%2345*B*<7=26357*<624$)&39* :"%*K27""# _

7 N634G757 A&)(*244*D&'*E2)2*"::5%&6'9*YDE<0* DE!!*267*DE!FZ! R62H459*L299&=5*#2%24454*3"#$*":*72)2* 45=5%2'&6'*<#23(5*F#2%O " KEXF*[\\] KEXF " KEXF*[\\]*1HW53)*F)"%2'5*Y!4"G7Z " X&45*E&::\&6' 267*3(53O&6'*2:)5%*3"#&59! DG&47*267*L262'5*#&#54&659! RLH57757*C5##54&6*V")5H""O <G)"6"L"G9*KGH*` D&'*E2)2*!4"G7*F5%=&35?"93"65*F"G)( Y?"672$*` c $z " <624$^5*72)2*&69)26)4$ " <624$^5*2)*93245*A&)(*1%2345*B*<7=26357*<624$)&39* :"%*K27""# a

8 /

9 X&45*H%"A95%*562H459*954:\ 95%=&359*72)2*L"=5L56)*:%"L* :"%*5T2L#45*KEXF*)"*1HW53)* F)"%2'5 d

10 NL#"%)26)4$*)(&9*7%2'S7%"# "%*3"#$*&9*)G%657*&6)"*2*F#2%O* #%"'%2L0*A(&3(*&9*5T53G)57* "%*93(57G457.-

11

12 I&3O*$"G%*:2="%&)5*6")5H""O* 56=&%"6L56)*267*9)2%)*)"* 3"75*&6*(5%5*2'2&69)*$"G%* 2624$)&39*4&H%2%&59 M(5*529&59)*A2$*)"*HG&47*"G)*2* 42H*&9*)"*45=5%2'5*9"L5* O6"A6*H29&39*267*34G9)5%&6'* )"*%G6*W"H9*&6*#2%24454 f95*4&h%2%&59*4&o5*b0* M569"%:4"A 267*!2::5*:"%*$"G%* &:*#"99&H45* &6*#2%24454 K27""#8KEXF.,

13 D&'*E2)2*?262'5%*A&)(*B*N6)5%#%5)5% <H&4&)$*)"*=&9G24&^5*95=5%24*9"G%359*:%"L*V")5H""O9*"6*DE<0*DE!F*267*DE!! )*+,-,./) ,6:2;,<,=6,*83./;=9.J

14 <66"G63&6'b*1B<<K*,;/;-*:"%*F#2%O*,;T.Q

15 What is ORAAH (Oracle R Advanced Analytics for Hadoop) ORAAH is a set of R packages and Java libraries that provide: An R interface for manipulating data stored in a local File System, HDFS, HIVE, Impala or JDBC sources, and creating Distributed Model Matrices across a Cluster of Hadoop Nodes in preparation for ML. A general computation framework where users invoke parallel, distributed MapReduce jobs from R, writing custom mappers and reducers in R while also leveraging open source CRAN packages. Parallel and distributed Machine Learning algorithms that take advantage of all the nodes of a Hadoop cluster for scalable, high performance modeling on big data. Functions use the expressive R formula object optimized for Spark parallel execution. ORAAH's custom LM/GLM/MLP NN algorithms on Spark scale better and run faster than the open-source Spark MLlib functions, but ORAAH provides interfaces to MLlib as well. Copyright 2018, Oracle and/or its affiliates. All rights reserved. 15

16 Where is ORAAH available? On premises: Part of the Oracle Big Data Connectors license for the Oracle Big Data Appliance, DIY Cloudera clusters and DIY Hortonworks clusters. On Oracle Cloud: Part of the Oracle Big Data Connectors license that is included with the Oracle Big Data Cloud Service and the Oracle Big Data Cloud at Customer Included as part of the Big Data Cloud (formerly known as Compute Edition) Copyright 2018, Oracle and/or its affiliates. All rights reserved. 16

17 ORAAH Benefits: Making Spark MLlib better for R users ORAAH Formula parser can handle the full set of open-source R formula transformations, so it can be used with any Spark MLlib algorithm supported by ORAAH. Even in newer Spark releases (Oct 2018) SparkR fails to process a simple interaction between attributes. Using SparkMLlib Logistic Regression model in SparkR fails: R> model <- glm( Kyphosis ~ (Age + Number)^2, df, family = "binomial") ERROR RBackendHandler: fitrmodelformula on org.apache.spark.ml.api.r.sparkrwrappers failed Error in invokejava(isstatic = TRUE, classname, methodname,...) :java.lang.illegalargumentexception: Could not parse formula: Kyphosis ~ (Age + Number)^2 Using Spark MLlib Logistic Regression model via ORAAH R> model <- orch.ml.logistic( Kyphosis ~ (Age + Number)^2, data = data) OBX Model Matrix: processed 1 factor variables, sec OBX Model Matrix: created MLlib LabeledPoint RDD (81 rows) sec OBX Machine Learning: MLlib Logistic Regression elapsed time sec R> model$coefficients [1] produces the same exact result from open-source R glm( Kyphosis ~ (Age + Number)^2, data = kyphosis, family = "binomial")$coefficients (Intercept) Age Number Age:Number Copyright 2018, Oracle and/or its affiliates. All rights reserved. 17

18 ORAAH and Python: Simple and clean code: building a Spark MLlib Random Forest model from HIVE source Python user steps 47 lines Load Libraries Establish Spark Session Process Formula Copy data from HIVE Create 3 rd copy of Data for vectors Build Model Single Vector of Predictions ORAAH user steps 14 lines Load Libraries Establish HIVE and Spark Session Build Model directly against HIVE (also HDFS, IMPALA,, JDBC or Spark DF) data with full formula support Predictions exported with desired columns, no need to glue back original columns Copyright 2018, Oracle and/or its affiliates. All rights reserved. 18

19 Machine Learning Algorithms and Utilities in ORAAH Spark 2.1+ Algorithms by Oracle, interfaces to Spark MLlib, plus HIVE, Impala and Spark DF interfaces RED indicates new in release Classification Clustering Extreme Learning Machines (Oracle s MPI/Spark-based) Hierarchical k-means (Spark MLlib) Hierarchical-ELM (Oracle s MPI/Spark-based) Gaussian Mixture Models (Spark MLlib) Multi-Layer Neural Nets (Oracle s Spark-based) Hierarchical k-means (also available in Map-Red) Logistic Regression (Oracle s Spark-based) Feature Extraction & Creation Gradient Boosted Trees (Spark MLlib) Logistic Regression (Spark MLlib) Distributed Stochastic PCA (Oracle s MPI/Spark-based) Decision Trees (Spark MLlib) Distributed Stochastic SVD (Oracle s MPI/Spark-based) Random Forest (Spark MLlib) Principal Component Analysis (Spark MLlib) Nonnegative Matrix Factorization (Map-Red) Regression Low Rank Matrix Factorization (Map-Red) Multi-Layer Neural Nets (Oracle s Spark-based) Transparency Functions with IMPALA and HIVE Linear Regression Model (Oracle s Spark-based) Gradient Boosted Trees (Spark MLlib) Aggregations, Table Joins, summarization Linear Regression Model (Spark MLlib) Variable Creation, Push & Pull data from IMPALA and HIVE Support Vector Machine (SVM) (Spark MLlib) Ability to push and pull data from Oracle Database LASSO (Spark MLlib) JDBC Driver interface - build Spark DataFrames for ORAAH Ridge Regression (Spark MLlib) Open Source R Algorithms Random Forest (Spark MLlib) Ability to run any R package via our hadoop.run Decision Trees (Spark MLlib) function in Map-Reduce mode Copyright 2018, Oracle and/or its affiliates. All rights reserved.

20 :"%*E2)2*)(2)*:&)9*&6*L5L"%$l lhg)*249"*2h45*)"*9"4=5*2*.-h&*%"a*l"754*ya(&3(*3266")*:&)*56)&%54$*&6*l5l"%$z* <44*)59)9*%G6*"6*2*_\V"75*D&'*E2)2*<##4&2635*qa\,*A&)(*,U_jD*":*B<?*#5%*V"75 X"%LG42b* *r*7&9)2635*s*"%&'&6*s*759) s*29;:23)"%yl"6)(z*s*29;:23)"%y$52%z*s*29;:23)"%y72$":l"6)(z*s*29;:23)"%y72$":a55oz*s*29;:23)"%y:4&'()6glz,-

21

22 1B<<KP9*V5A*E&9)%&HG)57*FhE D563(L2%O*":*1B<<KP9*YF#2%Os?INZ*=9*F#2%O*?44&Hb*_T*:29)5%*s*4&652%*93245*G# F&6'G42%*h53)"%*E53"L#"9&)&"6*,-O*T*,-O*75695*&6#G)*YJ;,jHZ.-*)(%5279*Y.,*23)G24*3"%590*\ qltq-jhz B26O ;U90;2 9+,6:MQ") VWI QR-?O 90;P2 MF8<-?OU X,Y,.-- HT Qd. 9+,6:2QR-?O $#AA/29+,6:MQ").U- I&Z a-a,-- IZ' d/,,,

23 V5A*1B<<K*f)&4&)$*:G63)&"69*:"%*E2)2* I%"3599&6'*267*N6'59) [;\]2,FJ29+,6:2;,<,2=6,*83,J

24 1B<<K*,;/;-*\ V5A*F#2%O*EX*72)2*L26&#G42)&"6*:G63)&"69,Q

25 1B<<K*,;/;-*\ V5A*F#2%O*EX*72)2*L26&#G42)&"6*:G63)&"69 467EL73YKJ> N6)5%#%5)9*2*!Fh*:&45*267*4"279* &)*&6*L5L"%$*&6)"*2*F#2%O*EX;* 9$9)5L*"%*KEXF,U

26 1B<<K*,;/;-*\ V5A*F#2%O*EX*72)2*L26&#G42)&"6*:G63)&"69 467ELJ8376?O8 j565%2)59*2*9&l#45*9gll2%$* ":*)(5*&6:"%L2)&"6*&6*2* F#2%OEX,_

27 1B<<K*,;/;-*\ V5A*F#2%O*EX*72)2*L26&#G42)&"6*:G63)&"69 467EL74--87< D%&6'9*2*F#2%OEX &6)"*BP9*4"324* L5L"%$*:"%*:G%)(5%* L26&#G42)&"6*"%*%59G4)*#%&6)&6',a

28 1B<<K*,;/;-*\ V5A*F#2%O*EX*F324&6'*:G63)&"6,/

29 1B<<K*,;/;-*\ V5A*F#2%O*EX*F324&6'*:G63)&"6 467EL37,-8 F32459*2*F#2%O*EX*G9&6'*"65*":*.U*

30 1B<<K*,;/;-*\ V5A*F#2%O*EX*F324&6'*:G63)&"6 ^3E4S F#2%O*EX*2335#)*L26$*%g2=2 75:2G4)* :G63)&"690*:%"L*9("AYZ*)"*3"G6)YZ J-

31 V5A*gED!*&6)5%:235!!%52)59*2*F#2%O*E2)2X%2L5 :%"L*2*gED!*9"G%35*)(2)*326*H5*G957*"6*26$*":* J.

32 ?"754*#5%:"%L2635*92L#45*:"%*<449)2)5*I%57&3)&"6*!(24456'5 D *92L#45*"G)*":*)(5*"%&'&624*.J;QL&*%53"%790*_-t8Q-t*9#4&)*:"%*M59)&6' $#AA/29+,6:MQ") 1RQ! <4L"9)*29*'""7*29*)(5* JT*:29)5%*)"*HG&47 $#AA/29+,6:2_RQK! X29)59)*)"*DG&47*267* F3"%5 J,

33 ?"754*#5%:"%L2635*92L#45*:"%*<449)2)5*I%57&3)&"6*!(24456'5 D *92L#45*"G)*":*)(5*"%&'&624*.J;QL&*%53"%790*_-t8Q-t*9#4&)*:"%*M59)&6' JJ

34 @&=5*E5L" D&'*E2)2*?262'5%*V")5H""O9 `46:?FN2S?<E2R,6N829+,6:2]-53<863

35 JU

36 J_

37 Ja

38 J/

39 Jd

40 Q-

41 Q.

42 Q,

43 QJ

44 QQ

45 QU

46 L")&=2)&"6*:"%*)(5*:G)G%5 B5'%599&"6 E53&9&"6*M%559 ())#b88aaa;l4$52%6&6';"%'8 Q_

47 34"G73G9)"L5%3"6653);"%2345;3"L

48

49