<Insert Picture Here>

Transcription

1 1

2 <Insert Picture Here> Oracle R Enterprise Training Sessions Session 1: Getting Started with Oracle R Enterprise Mark Hornick, Senior Manager, Development Oracle Advanced Analytics

3 The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle s products remain at the sole discretion of Oracle. 3

4 Oracle R Enterprise Training Sessions Date / Time Session Topics Thursday, December 1, 2:00 PM ET Tuesday, December 6, 11:00 AM ET Getting Started with Oracle R Enterprise (ORE) Introduction to the R Language and Environment Oracle R Enterprise Overview Installation of R Installation of Oracle R Enterprise Connecting to Exadata from R R Language Basics Producing Graphs in R Thursday, December 8, 2:00 PM ET ORE Transparency Layer Interacting with Database Tables Manipulating and transforming data through ORE Tuesday, December 13, 11:00 AM ET ORE Embedded R Script Execution Thursday, December 15, 2:00 PM ET Operationalizing R Scripts Tuesday, December 20, 11:00 AM ET Advanced Topics Execution through R interface Execution through SQL interface From Analyst to Production Integration with OBIEE XML graph generation using SQL Base SAS equivalent functionality ORE support for Hadoop and Map-Reduce Use of ORE in Exadata and BDA environments 4

5 Getting Started with Oracle R Enterprise (ORE) Oracle R Enterprise Overview Installing R Installing Oracle R Enterprise Connecting to Exadata from R 5

6 Oracle R Enterprise Overview 6

7 What is R? R is an Open Source language and environment for statistical computing and graphics Started in 1994 as an Alternative to SAS, SPSS & Other proprietary Statistical Environments The R environment R is an integrated suite of software facilities for data manipulation, calculation and graphical display Around 2 million R users worldwide Widely taught in Universities Many Corporate Analysts know and use R Hundreds of open sources packages to enhance productivity such as: Bioinformatics with R Spatial Statistics with R Financial Market Analysis with R Linear and Non Linear Modeling 7

8 Why statisticians/data analysts use R R environment is R is a statistics language similar to Base SAS or SPSS statistics Powerful Extensible Graphical Extensive statistics OOTB functionality with many knobs but smart defaults Ease of installation and use Free Statisticians typically are Not SQL literate Not familiar with DBA tasks 8

9 Limitations of R R is a client and server bundled together as 1 executable - like Excel Single user tool Not multi-threaded Cannot leverage CPU capacity even on a user's laptop/desktop R requires data it operates on to be first loaded into memory Loading data may not be a limitation given RAM available on laptops/desktops R s call by value semantics means as data flows into functions, for each function invocation, many copies of the data are made As a result you quickly run into memory limits 9

10 Why should you be interested in R? Emerging trends It s the next big thing in advanced analytics Colleges and universities use R for statistics classes (replacing more traditional software tools) Advanced Analytics as a critical differentiator of the DWH technology stack Augment Oracle deployments Enhance results with powerful graphics Integrate R results and graphics with BI Publisher documents and OBIEE dashboards A scalable R via Oracle R Enterprise Leverage Oracle-engineered solutions A worthy alternative to SAS 10

11 Rexer Analytics Survey

12 R Ecosystem Resources Many helpful books Installation and Administration User Groups UseR Conferences Huge libraries of reusable code Visit CRAN: Well-documented packages With sample data and code 12

13 Default R GUI 13

14 RStudio Third Party, Open Source IDE 14

15 Data Visualization with R 15

16 Graph examples 16

17 Oracle Strategy for R Deliver enterprise-level advanced analytics based on R environment 1. Oracle s Distribution of Open Source R Enterprise support for open-source R Enhanced performance with Intel MKL libraries for x86 hardware Packaged with Oracle Linux distribution 2. Oracle R Enterprise Embedded component of the RDBMS Eliminates R s memory constraint by enabling R to work transparently on database resident data Brings R users closer to Oracle Database by transparently leveraging in-database analytics via R Enables R users to leverage Oracle Database and Big Data Appliance for scalable R Analytics Enables integration of R scripts into enterprise production applications and OBIEE dashboards Fully leverages the latest R algorithms and models contributed to the Comprehensive R Archive Network (CRAN) Integrated with Hadoop and HDFS write map-reduce style R scripts Enables migration away from Base SAS and helps reduce SA$ Annual Usage Fees 17

18 Oracle R Distribution Features Supported by Oracle Optimized for high performance analytics using Intel MKL Library Platforms Windows 32-bit, 64-bit Linux 64-bit Integrated with Oracle Enterprise Linux distribution 18

19 R Licensing and Distribution Open source R is licensed under GPL-v2 Oracle will redistribute open source R providing free support for Oracle Database and Oracle Linux customers 19

20 Oracle R Enterprise R workspace console Oracle statistics engine Function push-down data transformation & statistics OBIEE, Web Services No changes to the user experience Scale to large data sets Embed in operational systems 20

21 Oracle R Enterprise R workspace console Oracle statistics engine Function push-down data transformation & statistics OBIEE, Web Services Transparently leverage Hadoop for High Performance Analytics to Big Data Appliance 21

22 Big Data Appliance An engineered system optimized for capturing and integrating low density data into Exadata High-performance Hardware 18 Sun X4270 M2 Servers 48 GB memory per node = 864 GB memory 12 Intel cores per node = 216 cores 24 TB storage per node = 432 TB storage 40 Gb p/sec InfiniBand 10 Gb p/sec Ethernet Optimized for Hadoop and NoSQL workloads InfiniBand Networking for integration with Exadata Software: Oracle Hadoop Oracle R Enterprise Oracle NoSQL DB Oracle Data Integrator (Hadoop capabilities) Oracle Loader for Hadoop 22

23 BDA and Exadata BDA Oracle Database HDFS Hadoop Load Data to Exadata ORE ORE ORHC R Engine Analyze Data Exadata 23

24 What R programmers should know Oracle R Enterprise provides a familiar environment to operate on data in Oracle Database overloads base R functions for data in Oracle Database to address big data as a single big data set problem enables embedded execution of existing R scripts in Oracle Database to address big data enables embedding advanced analytics into production infrastructure and business processes / applications provides database-controlled data-parallel execution framework removes need to manage data outside the database no knowledge of SQL required 24

25 What statisticians should know Oracle R Enterprise expands the kinds of problems that can be solved using readily available technologies through in-database statistical calculations Reduces the SQL skills required to get results reduces time programmers spend on building infrastructure to manage data outside of Oracle Database reduces the complexity of analytic solutions to be placed into production reduces costs by replacing SAS, and its high licensing fees, in the lab provides exhaustive support for statistical techniques and functions found in Base SAS and Base R implements select techniques from SAS libraries: STAT, ETS and OR 25

26 What a SQL user or DBA should know Oracle R Enterprise is an advanced analytical engine that can be used in addition to SQL in Oracle Database can be used to put an R script into production through SQL invocation reduces or eliminates the need to move data outside Oracle Database facilitates auditing of data flows reduces the number of LOB help requests for SQL queries to obtain data of interest enables embracing high impact advanced analytic workloads to run within production database infrastructure allows DBAs to control what R code runs within a deployment by requiring all R code reside in a secure, audited database table 26

27 What is Oracle R Enterprise? R packages, database library, and SQL extensions that bring Oracle Database closer to Advanced Analytics users in an Enterprise Transparency Layer Set of packages mapping R data types to Oracle Database objects Transparent SQL generation for R expressions on mapped data types Enables direct interaction with database-resident data using R language constructs Enables R users to work with data larger than desktop system memory Statistics Engine Set of statistical functions and procedures for commonly-used statistical libraries Executes in Oracle Database SQL extensions Enables database server execution of R code to facilitate embedding R in operational systems Eliminates client data loading and result write-back to Oracle Database Enables dataflow parallelism, generation of rich XML output, SQL access to R, and parallel simulations capability Hadoop Connector R packages that enable working directly with an Oracle Hadoop cluster using R Data resides in HDFS, Oracle Database,or local files 27

28 Oracle R Enterprise Compute Engines R Engine Other R packages SQL Oracle Database R R Engine Other R packages Oracle R Enterprise packages Results User tables Results Oracle R Enterprise packages User R Engine on desktop R-SQL Transparency Layer intercepts R functions for scalable in-db execution Function intercept for data transforms, statistical functions and advanced analytics Interactive display of graphical results and flow control as in standard R Submit entire R scripts for embedded execution by Oracle Database Connect to Oracle Hadoop Cluster and interact with HDFS data Database Compute Engine Scale to large datasets Access tables, views, and external tables, as well as data through DB LINKS Leverage database SQL parallelism Leverage new and existing in-database statistical and data mining capabilities R Engine(s) spawned by Oracle DB Database can spawn multiple R engines for database-managed parallelism Efficient data transfer to spawned R engines Emulate map-reduce style algorithms and applications Enables lights-out execution of R scripts 28

29 Oracle R Enterprise Data Sources R user on desktop Other databases Direct R access RODBC, DBI, etc R Engine Oracle R package Other R packages Direct R access File systems Results SQL Oracle Database User tables OLAP cubes Database Links External Tables Bulk import Other databases File systems 29

30 Installing R 30

31 Installing R Go to and choose Linux, Mac or Windows If Windows Click install R for the first time Click previous releases Click Download R for Windows If Linux Click R Sources under software Click R tar.gz to download Build R on your Linux system If Mac note ORE not supported on Mac Repeat the same process to install R on Oracle Database node(s) For more info than you d want, see 31

32 32

33 Other R GUIs, e.g., installing RStudio Go to Click Click for desktop Consider server install if you run R on a Linux server and want to enable users to remotely access RStudio using a web browser Installation instructions online 33

34 Running R on Windows Double click the desktop icon Run from C:\Program Files\R\R \bin\x64\Rgui.exe At the command prompt R> help.start() brings up help in your default browser plot(mtcars) get a quick pairs plot of the variables in mtcars Quit R quit() Or use: File Exit Save workspace image? 34

35 Installing R on Linux Download R source tar file from CRAN Untar the file tar -zxvf R tar.gz Change directory to R cd R Configure for shared library generation./configure --with-icu=no --enable-r-shlib make 35

36 Setting Environment Variables on Linux Set R_HOME environment variable to R root installation If RPM-based install using Oracle R Distribution setenv R_HOME /usr/lib64/r If manual R installation setenv R_HOME <dir-where-ran-make> Set PATH environment variable Add $R_HOME/bin to PATH If RPM-based instlal using Oracle R Distribution setenv PATH /usr/bin:$path If manual R installation setenv PATH <dir-where-ran-make>/bin:$path 36

37 Running R on Linux Run R $R_HOME/bin/R, or just R For the GUI, R --gui=tk At the command prompt R> help.start() brings up help in your default browser plot(mtcars) get a quick pairs plot of the variables in mtcars Quit R quit() Or use: Cntrl-D Cntrl-D Save workspace image? 37

38 Installing Oracle R Enterprise 38

39 Supported Configurations Oracle Database 11.2 series on Oracle Linux 5, 64 bit Exadata machines Oracle Database Oracle R Enterprise client Windows XP/Vista/7 32 bit or 64 bit Oracle Enterprise Linux 64 bit Hadoop Connector only deployed on Oracle Hadoop cluster See ORE Beta Release Notes 39

40 ORE Installation steps Install Oracle Database 11.2 series Install R ORE certified with R Install ORE Set up Rprofile 40

41 Prerequisites for ORE Oracle Database 11.2 series Version , or If using or , must include patch that fixes Bug Number R installed on user s desktop to connect to and work with Oracle Database Oracle Database node(s) used as database-embedded R engine Two R package must be installed Download DBI: or R> install.packages( DBI ) Download PNG: or R> install.packages( png ) 41

42 Database Patches Download the patch set for Enterprise Edition from My Oracle Support OR Linux 64-bit For Oracle : Patch number for Linux x86-64 For Oracle : Patch number for Linux x86-64 Windows XP 32-bit For Oracle Patch 11 fixes Bug Number Windows 7 32-bit and 64-bit For Oracle Patch 11 fixes Bug Number

43 Applying the Database Patch for Linux Install the patch to Oracle Database, if needed cd DBpatch setenv PATH $ORACLE_HOME/OPatch:$PATH unzip -d. p _112010_linux-x86-64.zip cd opatch apply 43

44 Installing the ORE Package From R command line for both ORE client and database server Choose URL appropriate for platform R R> remove.packages(c("orexml", "ORExml", "OREeda", "OREgraphics", "OREstats", "OREbase", "ORE", "ROracle")) R> install.packages("ore", repos = " Check package installation by invoking R> library(roracle) Note: installing ORE package on database server is only required if planning to use embedded R execution 44

45 Installing Oracle Instant Client on Linux Obtain the RPM files oracle-instantclient11.2-basic x86_64.rpm oracle-instantclient11.2-devel x86_64.rpm Install the RPMs sudo rpm ivh oracle-instantclient11.2-basic x86_64.rpm sudo rpm ivh oracle-instantclient11.2-devel x86_64.rpm 45

46 Installing ORE Server Components on Windows Download ORE server components from location specified when you enrolled in the Beta test Internal: rqe_install_win.zip Unzip the download into an empty directory, e.g., C:\rqe_install_win 46

47 Set Windows Environment Variables for ORE server Set R_HOME to the directory where R is installed for example, C:\Program Files\R\R , if you installed R in the default location Set R_PROFILE_USER to the location where the Rprofile is stored Default value is C:\Program Files\R\Rprofile This can be changed Bounce the database As sysdba in SQL*Plus execute shutdown immediate startup 47

48 Running the Windows install script for ORE server In a Command Window, change directory to where downloaded file unzipped Change directory to the server folder Run install.bat Change directory to data folder Run install.bat The installation scripts automatically create User account rquser (password rquser) in your database Note: If an account named rquser already exists, rename the account Demo tables in the database Rprofile file which tells R to connect to a specific database when starting R 48

49 Location of Windows Installed Files Client-side packages in $R_HOME/library Database Connectivity packages: DBI, ROracle ORE Client: ORE, OREbase, OREeda, OREgraphics, OREstats, ORExml ORE Hadoop client: ORHC Server-side (database) ORE libraries in $ORACLE_HOME/bin ore.dll and rqe.dll 49

50 Start R on Windows with ORE Click desktop icon, or Click, e.g., C:\Program Files\R\R \bin\i386\Rgui.exe Type ore.ls() to see a list of available tables Confirms database connectivity 50

51 .Rprofile options(prompt='ore> ') options(width=132).first <- function() { # Load Oracle R libraries. library(ore) } # Database credentials: user = my_schema password = my_password sid = MY_ORACLE_SID" host = MY_DB_HOST" cat("connecting to ORE\n") cat(" User:", user, "\n") cat(" SID: ", sid, "\n") cat(" Host:", host, "\n") ore.connect(user, sid, host, password) ore.sync() ore.attach() cat("connected.\n") 51

52 Installing ORE Server Components on Linux Download ORE from the location specified when you enrolled in the Beta test Unzip the download into an empty directory, e.g., ~/OREbeta Install database server-side components Log in as user oracle cd server./install.sh 52

53 Location of Linux Installed Files Client-side packages in in $R_HOME/library Database Connectivity packages: DBI, ROracle ORE Client: ORE, OREbase, OREeda, OREgraphics, OREstats, ORExml ORE Hadoop client: ORHC Server-side (database) ORE libraries in $ORACLE_HOME/lib ore.so and librqe.so 53

54 Enable and Check Database Connectivity Enable database connectivity on R startup Copy provided Rprofile file to the home directory cp Rprofile ~/.Rprofile Set R_PROFILE USER environment variable setenv R_PROFILE_USER ~/.Rprofile Bounce the database, as sysdba execute shutdown immediate startup Check database connectivity Start R Execute ore.ls() 54

55 Sample Code ORE shipped with examples illustrating how to use ORE A collection of self-contained R scripts Scripts located in <unzip-directory>/samples.tar Where <unzip-directory> is the directory where you unzipped the ORE download Unzip samples.tar 55

56 Troubleshooting See log.txt file created in directory where you executed install program View file to review errors Install program can be re-executed after correcting errors 56

57 Uninstalling ORE Client Components Start R and invoke remove.packages( OREeda ) remove.packages( OREgraphics ) remove.packages( OREstats ) remove.packages( OREbase ) remove.packages( ORExml ) remove.packages( ORE ) remove.packages( ROracle ) remove.packages( DBI ) Database Server Components As sysdba, execute in SQL*Plus SQL> drop user rqsys cascade; On linux command line, execute rm $ORACLE_HOME/bin/ore.so rm $ORACLE_HOME/bin/librqe.so Unset environment variable unsetenv R_PROFILE_USER 57

58 Connecting to Exadata (Oracle Database) from R 58

59 Connecting to Exadata (Oracle Database) from R You ve already done it by starting R with Rprofile 59

60 Oracle R Enterprise Initial connection and basic operations R> ore.connect("rquser", "orcl", "machine-1") R> ore.sync("rquser") R> ore.attach ("rquser") R> ore.ls() R> names(ontime_s) R> dim(ontime_s) Implicitly invoked through R_PROFILE_USER Sync database objects with R client, e.g., add/remove tables/views added via SQL*Plus) Add schema objects in R env search path List tables visible in schema rquser as ore.frame objects List columns associated with table ONTIME_S View the number of rows and columns associated with this ore.frame 60

61 Working with ORE Connections ore.connect("rquser","sid","host","password",1521) Connect to a specific schema and database One connection active at a time ore.create( ONTIME_S, table = "NEW_ONTIME_S") ore.create( ONTIME_S, view = "NEW_ONTIME_S_VIEW") Create a database table from a data.frame, ore.frame. Create a view from an ore.frame. ore.drop(table="new_ontime_s") ore.drop(view="new_ontime_s_view") v <- ore.push(c(1,2,3,4,5)) ore.sync() ore.sync("rquser") ore.sync(table=c("ontime_s", "NARROW")) ore.sync("rquser", table=c("ontime_s", "NARROW")) ore.exists("ontime_s", "RQUSER") Drop table or view in database Store R object in database as temporary object, returns handle to object. Data frame, matrix, and vector to table, list/model/others to serialized object Synchronize ORE proxy objects in R with tables/views available in database, on a per schema basis Returns TRUE if named table or view exists in schema 61

62 Working with ORE Connections ore.ls() ore.ls("rquser") ore.ls("rquser",all.names=true) ore.ls("rquser",all.names=true, pattern= "NAR") t <- ore.get("ontime_s","rquser") ore.attach("rquser") ore.attach("rquser", pos=2) List the objects available in ORE environment mapped to database schema. All.names=FALSE excludes names starting with a. Obtain object to named table/view in schema. Make database objects visible in R for named schema. Can place corresponding environment in specific position in env path. ore.detach("rquser") ore.rm("df1") ore.rm(list("table1","table2"), "RQUSER") ore.disconnect() Remove schema s environment from the object search path. Remove table or view from schema s R environment. Disconnect from the database. Clean up all associated R objects and temporary database schema objects ore.exec("create table F2 as select * from ONTIME_S") Execute SQL or PL/SQL without return value 62

63 Demonstration Connecting to ORE Manipulating database objects 63

64 Wrapping up What s Oracle R Enterprise Install R and ORE Interacting with Oracle Database through R 64

65 65

66 66