Copyright 2013, Oracle and/or its affiliates. All rights reserved.

Transcription

1 2 Copyright 23, Oracle and/or its affiliates. All rights reserved.

2 Oracle Database 2c Heat Map, Automatic Data Optimization & In-Database Archiving Platform Technology Solutions Oracle Database Server Technologies

3 Growth in Data Diversity and Usage,8 Exabytes of Data in 2, 2x Growth by 22 Today s Drivers Emerging Growth Factors Enterprise 45% per year growth in database data Cloud 8% of new applications and their data Regulation 3 exabytes in archives by 25 Mobile # Internet access device in 23 Big Data Large customers top 5PB Social Business $3B/year in commerce by 25 4 Copyright 23, Oracle and/or its affiliates. All rights reserved.

4 Managing Storage Challenges Manage more data without incurring additional cost Compress data, without impacting performance Tier and compress data based on usage 5 Copyright 23, Oracle and/or its affiliates. All rights reserved.

5 Value at Risk Information Lifecycle Management Managing Data Over its Lifetime High Value Medium Value Low Value $$ $ $$$ Total Cost of Ownership (TCO) The policies, processes, practices, and tools used to align the business value of information with the most appropriate and cost effective IT infrastructure from the time information is conceived through its final disposition. Storage Networking Industry Association (SNIA) Data Management Forum 6 Copyright 23, Oracle and/or its affiliates. All rights reserved.

6 Automatic Data Optimization Optimize data storage based on usage Smart Compression Heat Map Automated Tiering Network Compression In Database Archiving 7 Copyright 23, Oracle and/or its affiliates. All rights reserved.

7 Data Compression Reduce storage footprint, read compressed data faster Hot Data 3X Advanced Row Compression Warm Data X Columnar Query Compression Archive Data 5X Columnar Archive Compression 8 Copyright 23, 22, Oracle and/or its affiliates. All rights reserved. Confidential Oracle Restricted

8 Oracle Advanced Compression Transparent, Smaller, Faster % Application Transparent End-to-end Cost/Performance Benefits across CPU, DRAM, Flash, Disk & Network Runs Faster: OLTP Apps (Transactional & Analytics) & DW Reduces Database Footprint CapEx & OpEx savings Increases Cloud ROI through Database Footprint reduction in DRAM Memory 9 Copyright 23, Oracle and/or its affiliates. All rights reserved.

9 Oracle Advanced Compression Oracle Advanced Compression New Features, New Feature Names Oracle Database g OLTP Compression Secure Files Compression Secure Files De-duplication Hybrid Columnar Compression NEW NEW NEW Oracle Database 2c Advanced Row Compression Advanced LOB Compression Advanced LOB Deduplication Hybrid Columnar Compression Heat Map (Object and Row Level) Automatic Data Optimization Temporal (Advancements) Copyright 23, Oracle and/or its affiliates. All rights reserved.

10 Compression New features in Oracle Database 2c Logminer and GoldenGate support Capture side changes completed in. logminer Apply side changes in.2 Faster queries on advanced row (OLTP) compression Wide tables (>255 columns) for advanced row (OLTP) compression Network Compression Copyright 23, Oracle and/or its affiliates. All rights reserved.

11 Automatic Data Optimization Simplifying the life cycle of data An in memory heat map tracks access to segments and blocks Data is periodically written to disk Information is accessible by views or stored procedures Po licy Uses can attach policies to tables to compress or tier data based on access to data Tables or Partitions can be moved between compression levels whilst data is still being accessed 2 Copyright 23, Oracle and/or its affiliates. All rights reserved.

12 Heat Map What it tracks Active Frequent Access Occasional Access Dormant HOT Actively updated Infrequently updated, Frequently Queried Infrequent access for query and updates Long term analytics & compliance COLD Heat Map tracking Database level Heat Map shows which tables and partitions are being used Block level Heat Map shows last modification at the block level Comprehensive Segment level shows both reads and writes Distinguishes index lookups from full scans Automatically excludes stats gathering, DDLs or table redefinitions High Performance Object level at no cost Block level < 5% cost 3 Copyright 23, Oracle and/or its affiliates. All rights reserved.

13 Heat Map How to enable Active Enabling Heat Map SQL> alter system set heat_map = ON scope=both; Frequent Access Occasional Access Disabling Heat Map SQL> alter system set heat_map = OFF scope=both; Dormant 4 Copyright 23, Oracle and/or its affiliates. All rights reserved.

14 Understanding Data Usage Patterns Database heat map 5 Copyright 23, Oracle and/or its affiliates. All rights reserved.

15 Understanding Data Usage Patterns Database heat map 6 Copyright 23, Oracle and/or its affiliates. All rights reserved.

16 Heat Map for Tables and Partitions segment level tracking ORDER S 7 Copyright 23, Oracle and/or its affiliates. All rights reserved.

17 Heat Map for Blocks row level tracking ORDER S 8 Copyright 23, Oracle and/or its affiliates. All rights reserved.

18 Viewing Heat Map Data Data Dictionary Views V$HEAT_MAP_SEGMENTS DBA_HEAT_MAP_SEGMENTS OBJECT_NAME SEGMENT_READ_TIME SEGMENT_WRITE_TIME FULL_SCAN LOOKUP_SCAN DEPT 2/MAR/23 :9:3 9/MAR/23 :9:3 2/MAR/23 4:9:3 EMP 2/MAR/23 22::36 2/MAR/23 22:9:3 2/MAR/23 9:49:47 BONUS 2/MAR/23 :9:3 2/MAR/23 :9:3 SALGRADE 2/MAR/23 :9:3 EMPLOYEE 8/FEB/23 9:33:4 2/MAR/23 9:49:47 8/FEB/23 9:33:4 ORDERS 2/MAR/23 5:: 9/MAR/23 ::2 9 Copyright 23, Oracle and/or its affiliates. All rights reserved.

19 Heat Map Enterprise Manager 2 Copyright 23, Oracle and/or its affiliates. All rights reserved.

20 Automatic Data Optimization Usage Based Data Compression Hot Data Warm Data Archive Data 3X Advanced Row Compression X Columnar Query Compression 5X Columnar Archive Compression 2 Copyright 23, 22, Oracle and/or its affiliates. All rights reserved. Confidential Oracle Restricted

21 Automatic Data Optimization Add compression and tiering policies to tables Po licy Po licy 2 POLICY : Compress Partitions with row compression if they haven t been modified in 3 days POLICY 2: Compress Partitions with columnar compression if they haven t been modified in 8 days Oldest Data Most Recent Data 22 Copyright 23, 22, Oracle and/or its affiliates. All rights reserved.

22 Automatic Data Optimization A heat map tracks the activity of segments and blocks Po licy Po licy 2 Oldest Data Most Recent Data 23 Copyright 23, 22, Oracle and/or its affiliates. All rights reserved.

23 Automatic Data Optimization Policies are automatically applied to tables Po licy Po licy 2 Oldest Data Most Recent Data 24 Copyright 23, 22, Oracle and/or its affiliates. All rights reserved.

24 Automatic Data Optimization Policies are automatically applied to tables Po licy Po licy 2 Oldest Data Most Recent Data 25 Copyright 23, 22, Oracle and/or its affiliates. All rights reserved.

25 Automatic Data Optimization Policies are automatically applied to tables Po licy Po licy 2 Oldest Data Most Recent Data 26 Copyright 23, 22, Oracle and/or its affiliates. All rights reserved.

26 Automatic Data Optimization Reduce storage footprint, read compressed data faster Po licy Po licy 2 Oldest Data Most Recent Data 27 Copyright 23, 22, Oracle and/or its affiliates. All rights reserved. Confidential Oracle Restricted

27 Automatic Data Optimization Automatically tier data to lower cost storage Po licy Po licy 2 Po licy 3 Oldest Data POLICY 3: If the tablespace is nearly full compress the oldest partition with archive Most compression Recent Data and move it to Tier 2 Storage 28 Copyright 23, 22, Oracle and/or its affiliates. All rights reserved.

28 Automatic Data Optimization OLTP Reporting Compliance & Reporting This Quarter This Year Prior Years Row Store for fast OLTP x compressed Compressed Column Store for fast analytics 5x compressed Archive Compressed Column Store for max compression As data cools down, Advanced Data Optimization automatically converts data to columnar compressed Online 29 Copyright 23, Oracle and/or its affiliates. All rights reserved.

29 Up to 5x Smaller Footprint & Faster Queries Automatic Data Optimization for OLTP Both Columnar & Archive Compression now complement Advanced Row Compression Best Practice: Step : Use Advanced Row Compression for entire DB and then Step 2: ADO automatically converts into columnar compressed once the updates cool down, and is used mainly for reporting => Query speed of Columnar & x smaller footprint Step 3: ADO automatically converts into archive compressed once data cools down further and is no longer frequently queried => 5-5x smaller footprint 3 Copyright 23, Oracle and/or its affiliates. All rights reserved.

30 Optimizes Data Based on Heat Map Automatic Data Optimization for DW Data generally comes in via Bulk Loading Workload dominated by queries, even during loading Step : Bulk Load directly into Columnar Compressed x smaller footprint, Query speed of Columnar Step 2: ADO automatically converts to Archive Compressed and moves to Lower Cost Storage once its queried infrequently Data remains online, with 5-5x smaller footprint, & lower storage cost 3 Copyright 23, Oracle and/or its affiliates. All rights reserved.

31 Fast, Flexible Loads & Queries on Columnar Automatic Data Optimization Mixed Use Fastest Load with uncompressed & Fastest Queries with columnar Mixed workloads often use Java app or 3rd party tools to insert and update data that does not use Bulk Loads, so cannot use Columnar Step : Load into uncompressed, conventional inserts & updates Fast loading, & flexibility of using a regular OLTP app for loading Step 2: ADO moves to Row Compressed or Columnar Compressed or Low Cost Storage once updates cool down Faster Queries, 3-x smaller footprint 32 Copyright 23, Oracle and/or its affiliates. All rights reserved.

32 Powerful Policy Specification Automatic Data Optimization Declarative Policy Specification: Condition Action alter table orders ilm add policy row store compress advanced row after 3 days of no modification; Conditions are time period after creation, no access or no modification of data Actions can be Compression Tiering or Tablespace Tiering Policies are inherited from the tablespace or table New tables inherit from tablespace; can also be applied to existing tables New partitions (including interval partitions) inherit from table 33 Copyright 23, Oracle and/or its affiliates. All rights reserved.

33 Simple Declarative SQL extension Automatic Data Optimization ALTER TABLE orders Active Frequent Access Occasional Access Dormant OLTP Compressed (2-4x) Affects ONLY Candidate Rows Cached in DRAM & FLASH Warehouse Compressed (x) High Performance Storage Warehouse Compressed (x) Low Cost Storage Archive Compressed (5-5X) Archival Storage ILM add row store compress advanced row after 2 days of no update compress for query low after week of no update tier to SATA Tablespace compress for archive high after 6 months no access 34 Copyright 23, Oracle and/or its affiliates. All rights reserved.

34 Scheduled Policy Execution Automatic Data Optimization Immediate and background policy execution Row level policies are executed periodically (Users can configure the frequency of execution) Segment level policies are executed in maintenance windows Policies can be extended to incorporate Business Rules Users can add custom conditions to control placement (e.g. 3 months after the ship date of an order) 35 Copyright 23, Oracle and/or its affiliates. All rights reserved.

35 Automatic Data Optimization Optimized Back up OLTP Reporting Compliance & Reporting Read / Write Tablespace x compressed Read-mostly data is moved to a READONLY Tablespace 5x compressed READONLY TBS As data cools down, Automatic Data Optimization automatically moves it to a READONLY TBS, it s backed up only once after that 36 Copyright 23, Oracle and/or its affiliates. All rights reserved.

36 Automatic Data Optimization Optimized Backups with Automated READONLY data movement ORDER S. As tables grow in size ILM policies compress data 2. Tablespace containing partitions reaches ILM tiering threshold 3. Partitions are moved to new read only tablespace on lower spec disk group 37 Copyright 23, Oracle and/or its affiliates. All rights reserved.

37 Automatic Data Optimization Optimized Backups with Automated READONLY data movement SQL> ALTER TABLE ORDERS ILM ADD POLICY TIER TO DATA2 READ ONLY AFTER 8 DAYS OF NO MODIFICATION 38 Copyright 23, Oracle and/or its affiliates. All rights reserved.

38 Automatic Data Optimization Data Classification Automatic Detection Automatic Action WHAT IF and WHEN Then DO Scope Tablespace level Group level Segment level: Table/Partition/ Subpartition Clustered table Row level: Table If conditions met Which operation to track? Creation Low access No data modification Validity expired When? After 3 days After year Tablespace full Then Actions Compression Types: OLTP Move to other storage Both compress + move 39 Copyright 23, Oracle and/or its affiliates. All rights reserved.

39 Automatic Data Optimization Why Oracle? Oracle Storage Vendors Disk Saving Extensive Partial Performance Speeds Queries Adds Overhead Integration with Database Deep integration excludes maintenance tasks includes memory access Integrated with RMAN and Active Data Guard Zero integration maintenance tasks considered real access 4 Copyright 23, Oracle and/or its affiliates. All rights reserved.

40 Automatic Data Optimization Summary Automatic Data Optimization Active Recently inserted, actively updated OLTP compressed (2-4x) Cached in DRAM & Flash Frequent Access Infrequent updates, frequent reports High compression (x) High performance storage Occasional Access Infrequent access High compression (x) Low cost storage Dormant Retained for long-term analytics and compliance with corporate policies Archive compressed (5-5x) Archival storage (database or tape) 4 Copyright 23, Oracle and/or its affiliates. All rights reserved.

41 In-Database Archiving Speed up upgrade and reports Applications typically work with recent data But often need to retain data for 5 to years In-Database Archiving provides the ability to archive infrequently used data within the database Archived data is invisible by default Works with partition pruning and Exadata storage indexes to eliminate I/O for archived data Archived data remains online for SQL Query & DMLs 42 Copyright 23, Oracle and/or its affiliates. All rights reserved.

42 In-Database Archiving How to enable Easily enabled for a table: alter table row archival Application can marks rows as archived: update SALES_ORDERS set ORA_ARCHIVE_STATE = Sessions can set default visibility to see all data or active data only (default) alter session set row archival visibility = [all active] 43 Copyright 23, Oracle and/or its affiliates. All rights reserved.

43 In-Database Archiving Why Oracle? Application Knowledge Cost Oracle Required No cost for functionality (included in EE) Others Supply Knowledge Packs for various Oracle Apps. Supports custom rules. Consultant cost usually required. Typical deal $$$ Schema Changes Minimal Required (typically shadow table) Operational Impact Minimal Access to archive data requires special support, app dev effort 44 Copyright 23, Oracle and/or its affiliates. All rights reserved.

44 In-Database Archiving Vs Flashback Archive Flashback Data Archive provides the ability to track and store transactional changes to a table over its lifetime. A Flashback Data Archive is useful for compliance with record stage policies and audit reports. It archives previous states of rows, the current state of a record is always visible in the table. In-Database Archiving only keeps the current state of a record but allows the application to hide infrequently used data within the database. 45 Copyright 23, Oracle and/or its affiliates. All rights reserved.

45 Heat Map, Automatic Data Optimization and In-Database Archiving Summary Heat Map Automatically tracks access Database-aware: maintenance jobs, backups, etc don t affect heat map Automatic Data Optimization Declarative easy-to-use syntax to define data compression & movement policies Extensible with business-specific logic In-Database Archiving Automatically hide archive data from normal users Keep archive data accessible, minimize impact on storage and performance 46 Copyright 23, Oracle and/or its affiliates. All rights reserved.

46 Oracle Database 2c Partitioning

47 Oracle Partitioning in Oracle Database 2c Over a decade of development and better than ever before Oracle 8. Oracle 8i Core functionality Performance Manageability Range partitioning Global Range indexes Hash partitioning Composite partitioning (Range-Hash) Static partition pruning Partition-wise joins Dynamic partition pruning Basic maintenance: ADD, DROP, EXCHANGE Expanded maintenance: MERGE Oracle 9i List partitioning Global index maintenance Oracle 9i R2 Range-List partitioning Fast partition SPLIT Oracle g Global Hash indexes Local Index maintenance Oracle g R2 M partitions per table Multi-dimensional pruning Fast DROP TABLE Oracle g Oracle g R2 Virtual column based partitioning More composite choices REF partitioning Hash-Hash partitioning Expanded REF partitioning AND pruning Oracle 2c Interval-REF partitioning - Partition Maintenance on multiple partitions - Partial local and global indexes - Interval partitioning - Partition Advisor - Incremental statistics management Multi-branch execution - Asynchronous global index maintenance for DROP/TRUNCATE - Online partition MOVE - Cascading TRUNCATE/EXCHANGE 48 Copyright 23, Oracle and/or its affiliates. All rights reserved.

48 Oracle Partitioning in Oracle Database 2c Make a robust and successful feature even better Improved business modeling Interval-Reference Partitioning Advanced partition maintenance for Interval-Reference Partitioning More efficient data maintenance Enhanced partition maintenance operations Asynchronous global index maintenance Partial indexing 49 Copyright 23, Oracle and/or its affiliates. All rights reserved.

49 Partitioning Improvements Asynchronous Global Index Maintenance for DROP and TRUNCATE partition Cascade Functionality for TRUCATE and EXCHANGE partition Multiple partition operations in a single DDL Online move of a partition (without DBMS_REDEFINITION) Interval + Reference partitioning 5 Copyright 23, Oracle and/or its affiliates. All rights reserved.

50 Improved Business Modeling Interval-Reference Partitioning New partitions are automatically created when new data arrives All child tables will be automatically maintained Combination of two successful partitioning strategies for better business modeling INSERT INTO orders VALUES ( -APRIL-22,... ); 5 Copyright 23, Oracle and/or its affiliates. All rights reserved.

51 Improved Business Modeling Interval-Reference Partitioning 52 Copyright 23, Oracle and/or its affiliates. All rights reserved.

52 Improved Business Modeling Cascading TRUNCATE and EXCHANGE PARTITION Cascading TRUNCATE and EXCHANGE for improved business continuity Single atomic transaction preserves data integrity Simplified and less error prone code development ALTER TABLE orders TRUNCATE PARTITION APRIL_22 CASCADE; 53 Copyright 23, Oracle and/or its affiliates. All rights reserved.

53 Improved Business Modeling Cascading TRUNCATE PARTITION Parent Parent Child Child2 Child Child2 Grandchild Grandchild 2 Grandchild3 Grandchild Grandchild 2 Grandchild3 Great Grandchild Great Grandchild Proper bottom-up processing required Seven individual truncate operations One truncate operation 54 Copyright 23, Oracle and/or its affiliates. All rights reserved.

54 Improved Business Modeling Cascading EXCHANGE PARTITION Parent Child Child2 Parent Grandchild Grandchild 2 Grandchild3 Child Great Grandchild Grandchild Exchange (clear) out of target bottom-up Exchange (populate) into target top-down 55 Copyright 23, Oracle and/or its affiliates. All rights reserved.

55 Improved Business Modeling Cascading EXCHANGE PARTITION Parent Parent Child Child2 Parent Child Child2 Parent Grandchild Grandchild 2 Grandchild3 Child Grandchild Grandchild 2 Grandchild3 Child Great Grandchild Grandchild Great Grandchild Grandchild Exchange (clear) out of target bottom-up Exchange (populate) into target top-down Exchange complete hierarchy tree One exchange operation 56 Copyright 23, Oracle and/or its affiliates. All rights reserved.

56 Partitioning Improved business modeling Interval-Reference Partitioning Advanced partition maintenance for Interval-Reference Partitioning More efficient data maintenance Enhanced partition maintenance operations Asynchronous global index maintenance Partial indexing 57 Copyright 23, Oracle and/or its affiliates. All rights reserved.

57 Enhanced Partition Maintenance Operations Online Partition Move Transparent MOVE PARTITION ONLINE operation Concurrent DML and Query Index maintenance for local and global indexes 58 Copyright 23, Oracle and/or its affiliates. All rights reserved.

58 Enhanced Partition Maintenance Operations Maintenance on Multiple Partitions Partition Maintenance on multiple partitions in a single operation Full parallelism Transparent maintenance of local and global indexes ALTER TABLE orders MERGE PARTITIONS Jan22, Feb22, Mar22 INTO PARTITION Quarter_22 COMPRESS FOR ARCHIVE HIGH; 59 Copyright 23, Oracle and/or its affiliates. All rights reserved.

59 Enhanced Partition Maintenance Operations Asynchronous Global Index Maintenance DROP and TRUNCATE complete immediately We maintain a list of invalid data object ids and ignore those entries in the index from then on Automatic scheduler job PMO_DEFERRED_GIDX_MAINT_JOB will run to clean up all global indexes Can be run manually Alter index [partition] CLEANUP is another approach DROP and TRUNCATE become fast, metadata-only operations Delayed Global index maintenance 6 Copyright 23, Oracle and/or its affiliates. All rights reserved.

60 Enhanced Partition Maintenance Operations Asynchronous Global Index Maintenance Copyright 23, Oracle and/or its affiliates. All rights reserved.

61 Enhanced Indexing with Oracle Partitioning Indexing prior to Oracle Database 2c Local indexes Non-partitioned or partitioned global indexes Usable or unusable index segments Non-persistent status of index, no relation to table 62 Copyright 23, Oracle and/or its affiliates. All rights reserved.

62 Enhanced Indexing with Oracle Partitioning Indexing with Oracle Database 2c Local indexes Non-partitioned or partitioned global indexes Usable or unusable index segments Non-persistent status of index, no relation to table Partial local and global indexes Partial indexing introduces table and [sub]partition level metadata Leverages usable/unusable state for local partitioned indexes Policy for partial indexing can be overwritten 63 Copyright 23, Oracle and/or its affiliates. All rights reserved.

63 Enhanced Indexing with Oracle Partitioning Partial Local and Global Indexes Global Non-Partitioned Index Partial indexes span only some partitions Full Indexing Global Partitioned Index Applicable to local and global indexes Complementary to full indexing Enhanced business modeling Indexing on Partial Indexes Table Partition Local Partitioned Index Table Partition Table Partition Partial Local Partitioned Index Partial Global Partitioned Index Indexing off No Indexing Partial Global Index 64 Copyright 23, Oracle and/or its affiliates. All rights reserved.

64 Enhanced Indexing with Oracle Partitioning Partial Local and Global Indexes 65 Copyright 23, Oracle and/or its affiliates. All rights reserved.

65 66 Copyright 23, Oracle and/or its affiliates. All rights reserved.

66 Graphic Section Divider 67 Copyright 23, Oracle and/or its affiliates. All rights reserved.