Why 12c How and why upgrade now

Transcription

1 Why 12c How and why upgrade now Andy Cleverly Director Database Technology Oracle EMEA Technology

2 Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle s products remains at the sole discretion of Oracle. Oracle Confidential Internal/Restricted/Highly Restricted 2

3 Release Roadmap Today Oracle 11.1 (GA: Aug 2007) AUG 2012 AUG 2015 Oracle 11.2 (GA: Sep 2009) JAN 2015 JAN 2018 Oracle 12.1 (GA: Jun 2013) JUN 2018 JUN 2021 Premier Support Waived Extended Extended Support Sustaining Support Oracle Database 12c Release 2 Targeted for Release 1HCY2016 (MOS Note )

4 Extended Support for Oracle Database 11.2

5 Database Patch Set Roadmap Release Schedule of Current Database Releases MOS Note x 3x 1x

6 Oracle Database 12c Release 1 ( ) Oracle Database 12c Overview Oracle Database In-Memory Oracle Database 12c for the Developer Oracle Database 12c for Big Data Oracle Multitenant Other Improvements 6

7 Oracle Database 12c ( ) Oracle Multitenant Database consolidation Fast Provisioning Manage many as one Oracle Automatic Data Optimization Smart Compression Automate Tiering Data Guard Far Sync Zero data loss over large distances 7

8 Oracle Database 12c ( ) Application Continuity Replay of failed transaction Data Redaction Masks application data dynamically Transparent to application Pattern Matching Sophisticated inter row pattern analysis 8

9 Oracle Database 12c Release 1 ( ) Exploit memory to improve performance Simplify access to Big Data Improve application developers experience Continue to improve consolidation 9

10 Oracle Database 12c Release 1 ( ) Oracle Database 12c Overview Oracle Database In-Memory Oracle Database 12c for the Developer Oracle Database 12c for Big Data Oracle Multitenant Other Improvements 10

11 Oracle Database In-Memory Goals Real Time Analytics Accelerate OLTP No Changes to Applications Exploit latest generation hardware CPU

12 Row Format Databases vs. Column Format Databases Row SALES Transactions run faster on row format Example: Insert or query a sales order Fast processing few rows, many columns Column SALES Analytics run faster on column format Example : Report on sales totals by region Fast accessing few columns, many rows Until Now Must Choose One Format and Suffer Tradeoffs 12

13 Oracle In-Memory Columnar Technology Pure In-Memory Columnar SALES Pure in-memory column format Not persistent, and no logging Quick to change data: fast OLTP 2x to 20x compression Enabled at table or partition level Available on all hardware platforms 13

14 Scans Billions of Rows per Second per CPU Core Memory CPU Load multiple region values REGION Vector Register CA CA CA CA Example: Find all sales in region of CA Vector Compare all values an 1 cycle > 100x Faster Each CPU core scans local in-memory columns Scans use fast SIMD vector instructions Originally designed for graphics & science Billions of rows/sec scan rate per CPU core 14

15 Oracle Database 12c Release 1 ( ) Oracle Database 12c Overview Oracle Database In-Memory Oracle Database 12c for the Developer Oracle Database 12c for Big Data Oracle Multitenant Other Improvements 15

16 Oracle Database 12c Release 1 ( ) Flexible Schema development Oracle Database 12c JSON Via RESTful service SQL JSON Via Native APIs Data persisted in database In JSON Data analyzed via SQL

17 Querying JSON Sample JSON in column customers.document { "firstname": "John", lastname : "Smith", "age": 25, "address": { "phonenumbers": [ "streetaddress": "21 2nd Street", "city": "New York", "state": "NY", "postalcode": "10021, "isbusiness" : false}, { "type": "home", "number": " }, { "type": "fax", "number": " } ] Simplified syntax for simple queries select c.document.firstname, c.document.lastname, c.document.address.city, c.document.phonenumbers from customers c; }

18 Oracle REST Data Services Enabling RESTful access to Oracle Database Provides data access consistent with modern App Dev frameworks Can map standard http(s) URI RESTful gets and posts to SQL Can declaratively returns results in JSON format JavaScript framework friendly Can support high numbers of end users Services HTTP(s) relational data access Oracle JSON collection based schemaless access Oracle NoSQL access over HTTP Oracle APEX mid-tier, web toolkit applications, mod_plsql replacement Formally known as Oracle APEX Listener Supported feature of the Oracle Database since 2010 Ships with Oracle Database 12c Release 1 ( )

19 Oracle REST Data Services Schemaless development using JSON Collection API ( ) URI JSON Transform Oracle REST Data Services JSON Collection API Pass Back Auto Generated SQL JSON HTTP(S) client Oracle Database Data stored in Oracle Database as JSON documents App Developer make standard HTTP(S) calls to JSON Collection APIs

20 Oracle REST Data Services Serving JSON results from relational data Oracle REST Data Services URI JSON Transform Map and Bind Transform to JSON SQL SQL Result Set HTTP(S) client Oracle Database Data stored in standard relational tables and columns Oracle REST Data Services (ORDS) Developer defines URI<>SQL mapping App Developer calls named URI over HTTP(S) gets and posts

21 Oracle REST Data Services Serving JSON results with Oracle NoSQL Database (ORDS 3.1) Oracle REST Data Services URI JSON Transform NoSQL API Pass Back Key-Value Lookup JSON HTTP(S) client Oracle REST Data Services provides HTTP(s) access to Oracle NoSQL Oracle NoSQL is embedded within ORDS and can also access remote DB Oracle NoSQL Database

22 Oracle Database 12c Release 1 ( ) Oracle Database 12c Overview Oracle Database In-Memory Oracle Database 12c for the Developer Oracle Database 12c for Big Data Oracle Multitenant Other Improvements 23

23 SQL is Critical.the complexity of dealing with a non-acid data store in every part of our business logic would be too great, and there was simply no way our business could function without SQL queries. Google, VLDB 2013 [Facebook] started in the Hadoop world. We are now bringing in relational to enhance that....[we] realized that using the wrong technology for certain kinds of problems can be difficult. Ken Rudin, Facebook, TDWI

24 Oracle Support for Any Data Management System Hadoop Change the Business Scale-out, low cost store Collect any data Map-reduce, SQL Analytic applications NoSQL Scale the Business Scale-out, low cost store Collect key-value data Find data by key Web applications Run the Business Relational Scale-out and scale-up Collect any data SQL Transactional and analytic applications for the enterprise Secure and highly available 25

25 Overcoming Barriers to Adoption of New Technologies INTEGRATION SKILLS SECURITY SQL Engineered Systems SQL on All Data Database Security on All Data Confidential 26

26 Oracle Big Data SQL One fast SQL query, onallyour data. Oracle SQL on Hadoop and beyond WithaSmartScanserviceasinExadata With native SQL operators With the security and certainty of Oracle Database Oracle Confidential Internal/Restricted/Highly Restricted 27

27 Accessing Big Data Without SQL Push Down Select w.sess_id, w.cust_id, w.page_id From web_logs w Where w.source_country = Brazil And w.category = TV And w.channel = Mobile Request for Data WEB_LOGS 100 s of Terabytes of Data Hadoop Cluster Low utilization of available resources All columns and rows from the table are returned High load on database server 28

28 Big Data SQL Push Down WithSQL Push Down Big Data SQL Server Select w.sess_id, w.cust_id, w.page_id From web_logs w Where w.source_country = Brazil And w.category = TV And w.channel = Mobile SQL shipped to BDA WEB_LOGS 10 s of Gigabytes of Data Hadoop Cluster Only columns and rows needed to answer query are returned Good utilization of available resources. SQL executed on Hadoopcluster Lower load on Server, Faster response 29

29 Big Data SQL Push Down WithSQL Push Down Big Data SQL Server Select w.sess_id, w.cust_id, w.page_id, c.name From web_logs w, customers c Where w.source_country = Brazil And w.category = TV And w.channel = Mobile And c.customer_id = w.cust_id SQL shipped to BDA WEB_LOGS 10 s of Gigabytes of Data CUSTOMERS Hadoop Cluster Only columns and rows needed to answer query are returned Good utilization of available resources. SQL executed on Hadoopcluster Data joined between CUSTOMERS and WEB_LOGS on server 30

30 Big Data SQL Push Down WithSQL Push Down WEB_LOGS Big Data SQL Server SQL Push Downs supported by Big Data SQL SQL shipped to BDA Hadoop scans (InputFormat, SerDe) JSON 10 s parsing of Gigabytes of Data WHERE clause evaluation Column projection Bloom filters for faster join Hadoop Cluster Only columns and rows needed to answer query are returned Select w.sess_id, w.cust_id, w.page_id, c.name From web_logs w, customers c Where w.source_country = Brazil And w.category = TV And w.channel = Mobile And c.customer_id = w.cust_id CUSTOMERS Good utilization of available resources. SQL executed on Hadoopcluster Data joined between CUSTOMERS and WEB_LOGS on server 31

31 New Data Sources for Oracle External Tables CREATE TABLE web_logs (click VARCHAR2(4000)) ORGANIZATION EXTERNAL ( TYPE ORACLE_HIVE DEFAULT DIRECTORY Dir1 ACCESS PARAMETERS ( com.oracle.bigdata.tablename logs com.oracle.bigdata.cluster mycluster) ) REJECT LIMIT UNLIMITED New set of properties ORACLE_HIVE and ORACLE_HDFS access drivers Identify a Hadoopcluster, data source, column mapping, error handling, overflow handling, logging New table metadata passed from Oracle DDL to Hadoop readers at query execution Architected for extensibility StorageHandlercapability enables future support for other data sources Examples: MongoDB, HBase, Oracle NoSQL DB 32

32 Oracle Database 12c Release 1 ( ) Oracle Database 12c Overview Oracle Database In-Memory Oracle Database 12c for the Developer Oracle Database 12c for Big Data Oracle Multitenant Other Improvements 33

33 Oracle Multitenant New architecture for consolidating databases and simplifying operations ERP CRM DW Self-contained PDB for each application Applications run unchanged Rapid provisioning (via clones) Portability (via pluggability) Common operations performed at CDB level Manage many as one (upgrade, HA, backup) Granular control when appropriate Shared memory and background processes More applications per server 34

34 Multitenant Provides Agility for New Projects Fast Provisioning, Snapshot Clones Seed PDB New PDB From Seed Original PDB Clone of PDB Snapshot Cloneof PDB Time Taken to Provision New Database New GL GL-2 GL-3 0 Non CDB PDB Snapshot Clone 35

35 Multitenant for Provisioning Fast cloning of PDBs GL-1 GL-2 AP-1 AP-2 PO-1 PDBs can be cloned from within the same CDB PDBs can be cloned from remote CDBs GL OE AP PO PDBs can be cloned from non-cdbs Thinly provision snapshot clones in seconds 36

36 Multitenant Cloning Types and Platforms Flavors Types Metadata Only clone Subset Clone Remote Clone over DB link Multit tenant Cloning Full Clones Snapshot clones Snapshot clone syntax: create pluggable database PDB2 from PDB1 snapshot copy Supported on all platforms Source in Read-only if child exists Copy-on-write -Source opened in read write after clone is created File System Agnostic (CloneDB=TRUE) Exadata Sparse clones ACFS ZFSSA Netapp EMC 37

37 Copy-on-Write Basics Much more efficient storage of substantially similar files my_file my_file_copy File is collection of data blocks And headers Copy file requires only copy of header Much more efficient storage Minimal IO to create copy Only copy block when changed cp my_file my_file_copy

38 Multitenant New Features in Cloning Subset by tablespace Metadata-only clone Remote clone (including snapshots) File system-agnostic cloning via dnfs(clonedb = true) SQL Cross PDB Queries New SQL clause to aggregate data across PDBs select ENAME from containers(scott.emp) where CON_ID in (45, 49); PRIMARY STANDBY Standby & Logging New standbys clause (all none) Nologging clause at PDB level Additional Features Flashback data archive Temporal SQL Support Compatible with DB In-Memory Maintains state of PDBs between CDB restarts 39

39 Oracle Database 12c Release 1 ( ) Oracle Database 12c Overview Oracle Database In-Memory Oracle Database 12c for the Developer Oracle Database 12c for Big Data Oracle Multitenant Other Improvements 40

40 Performance Improvements 41

41 Advanced Index Compression Compresses indexes to reduce their overall storage requirement Less space required on disk Better use of the database cache Indexes are compressed by between 1 3 times Little or no discernable overhead Compression Advisor extended to describe the possible benefits of this feature 42

42 Attribute Clustering Ordering of data so that rows are stored near one another based on column values Benefits : Improved query performance and concurrency Reduced physical data access trough smart IO Significant IO reduction for highly selective operations Optimized space utilization Less need for indexes Improved compression ratios through data clustering Full application transparency Any application will benefit 43

43 Zone Maps Persisted storage index X Stores minimum and maximum of specified columns Analogous to a coarse index structure Much more compact than an index Zone maps filter out what you don t need, indexes find what you do need Significant performance benefits with complete application transparency IO reduction for table scans with predicates on the table itself or even a joined table using join zone maps (a.k.a. hierarchical zone map ) Benefits are most significant with ordered data Used in combination with attribute clustering or data that is naturally ordered 44

44 Approximate Count Distinct 1,2,3... Not every query requires a completely accurate result How many distinct individuals visited our website last week? New SQL function for approximate results for COUNT DISTINCT aggregates APPROX_COUNT_DISTINCT() Approximate results can be significantly faster and use less resources than exact calculations 5x to 50x ++ times faster (depending upon number of distinct values and complexity of SQL) Accuracy > 97% (with 95% confidence) 45

45 Rapid Home Provisioning

46 Oracle Rapid Home Provisioning Install Once Use Many Patching is complex and time consuming Even when automated New way to deploy upgrades Create reference homes on Centralized Home Server Apply patches once (Enterprise) on Home Server Distribute homes on-demand or policy Fast and Efficient Rapid distribution (network efficient) Space efficient (snapshots) Local caches 47

47 Oracle Rapid Home Provisioning Oracle Enterprise Manager Provisioning Monitoring and Configuration Capacity and Resource Cluster Database Cloud Cluster Patching Performance and Tuning Cluster Service Level Application Database NFS Mount Cluster Cluster Cluster Grid Service Catalog Grid Home Server Local Gold Image S/W Distribution Differential Copy 48

48 Some of Tom Kytes favourite 12c features More online operations Easier Reorgs 32k Strings Application continuity Adaptive exectution plans SqlCL***New***

49 More Online Operations DROP INDEX ONLINE DROP CONSTRAINT ONLINE SET UNUSED COLUMN ONLINE ALTER INDEX UNUSABLE ONLINE ALTER INDEX [VISIBLE INVISIBLE]

50 Easier Reorgs In a single command you can Move table tablespaces Move index tablespaces Move lob tablespaces Implement table compression Implement index key compression Implement lob compression Convert to securelfile

51 Easier Reorgs ops$tkyte%ora12cr1> begin 2 dbms_redefinition.redef_table 3 ( uname => user, 4 tname => 'T', 5 table_compression_type => 'row store compress advanced', 6 table_part_tablespace => 'TEST' ); 7 end; 8 / PL/SQL procedure successfully completed.

52 32k Strings Varchar2, NVarchar2 and Raw datatypesmay be upto32k in size, like in PL/SQL Compatible = or higher Max_String_Size init.ora set to EXTENDED (default is not this) Not supported in clustered and index organized tables Will be stored out of line (LOB) but work just like long strings to your program

53 Transaction Guard FIRST RDBMS to preserve COMMIT outcome 1. Request 2. Calls 5. Guaranteed Outcome 4. Enforce Outcome 3. FAN or errors Known outcome for every transaction At-most-once transaction execution Used by Application Continuity Available for JDBC-thin, OCI, OCCI, ODP.net Without Transaction Guard, retries can cause logical corruption Oracle Database 12c Release 1 54

54 Application Continuity First RDBMS to mask planned/unplanned outages 1. Request 2. Calls 5. Success 4. Replay 3. FAN or errors Improves end user experience Improves developer productivity Application transparent when using Oracle stack Enabled with WebLogicServer, Peoplesoft, Fusion Apps, Siebel(possibly) Oracle Database 12c Release 1

55 Adaptive Execution Plans Table scan T1 NL Index Scan T2 HJ Table scan Threshold T2 exceeded, plan switches Table scan T1 HJ Table scan T2 Plan decision deferred until runtime Final decision is based on statistics collected during execution If statistics prove to be out of range, sub-plans can be swapped Bad effects of skew eliminated

56 Time to turn back the clocka bit to 1985 We release Oracle Database version 5 The client-server model is born And a little program makes it debut with roots going back to 1982

57 SQL*Plus? Statement completion? SqlCL

58 SQL History? SqlCL

59 Upgrade/Migrate Older Oracle Releases Oracle 5/6/7/8 Oracle 8i exp/imp Less Downtime? Near-Zero Downtime? Oracle 9i Oracle 9.2 Transportable Tablespaces Same platform only + Golden Gate Oracle 10.1

60 Upgrade Options to Oracle Database 12c Oracle 10.2 Oracle 11.1 Oracle Oracle /4 expdp/impdp DBUA HW catctl.pl Less Downtime? Transient Standby Logical Standby Transportable Tablespaces Full Transportable Export/Import Near-Zero Downtime? + Golden Gate RMAN Inc Bck +

61 Migration Options to Oracle Database 12c Oracle 10.2 expdp/impdp Less Downtime? Near-Zero Downtime? Oracle 11.1 Oracle /2 CTAS, COPY SQL*Loader TDB Only same Endianness Transportable Tablespaces + Golden Gate Oracle /4 Full Transportable Export/Import RMAN Inc Bck +

62 Concept Transportable Tablespaces impdp "'"sys/sys as sysdba sysdba"'" "'" expdp "'"sys/sys as sysdba"'" TRANSPORT_DATAFILES= TRANSPORT_TABLESPACES=TS1,TS2 TTS feature available since Oracle 8i Cross platform support since Oracle 10g Read Write Self Contained? Read Only 75TB 75TB Oracle hrs IBM AIX Oracle OL5.8 64bit

63 Concept Transportable Tablespaces xtts LITTLE ENDIAN PLATFORMS HP IA Open VMS HP Open VMS HP Tru64 UNIX Linux IA (32-bit) Linux IA (64-bit) Linux x86 64-bit Microsoft Windows IA (64-bit) Microsoft Windows x86 64-bit Microsoft Windows IA (32-bit) Solaris Operating System (x86) Solaris Operating System (x86-64) Cross platform support V$TRANSPORTABLE_PLATFORM FILE COPY RMAN CONVERT BIG ENDIAN PLATFORMS FILE COPY Apple Mac OS HP-UX (64-bit) HP-UX IA (64-bit) AIX-Based Systems (64-bit) IBM zseries Based Linux IBM Power Based Linux Solaris[tm] OE (32-bit) Solaris[tm] OE (64-bit)

64 Upgrade/Migration: Transportable Tablespaces Rebuild meta information (views, synonyms, trigger, roles etc) SOURCE Database DESTINATION Database SCOT T HUGO Read Only SYSTEM SYSAUX UNDO TEMP VIEWS CODE PRIVS SCOT T HUGO SYSTEM SYSAUX UNDO TEMP VIEWS CODE PRIVS

65 Speed Up Transportable Tablespaces Usually the biggest pain points with TTS Downtimedue to: Duration to copy very large amounts of data Duration to convert many tablespaces cross Endianness New technique: Avoid the copy & convert phase RMAN can convert incremental backups cross platform Available since Oracle for Exadata only Available for Linux x86-64 with Oracle Available on all platforms starting with Oracle 12c See MOS Note: for description and Linux perl scripts

66 Transportable Tablespace with Incremental Backups expdp impdp "'"sys/sys as as sysdba"'" TRANSPORT_TABLESPACES=TS1,TS2 TRANSPORT_DATAFILES= Downtime!!! Convert and apply backups SOURCE Database SCOT T HUGO Read Only SYSTEM SYSAUX UNDO TEMP VIEWS CODE PRIVS Data Pump DESTINATION Database SCOT T HUGO Read Write SYSTEM SYSAUX UNDO TEMP VIEWS CODE PRIVS

67 Full Transportable Export/Import Combining: Transportable Tablespaces with Data Pump taking care of all meta information with optional RMAN incremental backups to decrease downtime One impdp Command VERSION=12 Migration FULL=Y TRANSPORTABLE=ALWAYS This works: Cross platform With or without Oracle Multitenant Sourcecan be Oracle /4 or newer Targetmust be at least Oracle

68 Full Transportable Export/Import with Copies SOURCE Database DESTINATION Database VIEWS VIEWS SCOT T HUGO Read Only SYSTEM SYSAUX UNDO TEMP CODE PRIVS Data Pump SCOT T HUGO SYSTEM SYSAUX UNDO TEMP CODE PRIVS

69 Full Transportable Export/Import with Backups Convert and apply backups SOURCE Database DESTINATION Database VIEWS VIEWS SCOT T HUGO Read Only SYSTEM SYSAUX UNDO TEMP CODE PRIVS Data Pump SCOT T HUGO SYSTEM SYSAUX UNDO TEMP CODE PRIVS

70 Summary Oracle Database 12c ( ) will have premier support into 2018 Upgrading to Oracle Database 12cgets you lots of new features Oracle Database 12c is stable and in production at customers worldwide

71 Resources Download slides from:

72 Oracle Database 12c New Features Guide

73 Oracle Confidential Internal/Restricted/Highly Restricted 74

74