Implementing Oracle database12c s Heat Map and Automatic Data Optimization to optimize the database storage cost and performance

Transcription

1 Implementing Oracle database12c s Heat Map and Automatic Data Optimization to optimize the database storage cost and performance Session #267: Prepared by: Kai Yu, Senior Principal Architect, Dell Oracle Solutions REMINDER Check in on the COLLABORATE mobile app

2 About Author Kai Yu, Senior Principal Architect, Dell Database Engineering 20 years Oracle DBA/Apps DBAS and Solutions Engineering Specializing in Oracle RAC, Storage, Oracle VM,Oracle EBS Oracle ACE Director, Oracle papers author/presenter 2011 OAUG Innovator of Year, 2012 Oracle Excellence Award: Technologist of the Year: Cloud Architect by Oracle Magazine My Oracle Blog: Co-author Apress Book Expert Oracle RAC 12c

3 Force10 Force W 750W KMM FPM W 750W My Recent Work: Architect of Dell Integrated Systems for Oracle Database -Ready Infrastructure Solution (1 x S60) 1GbE Switch (Management) Management Server (R630) Top of Rack (2 x S6000) 40GbE Switches (Public & Private) Oracle Database 12c (2 x R920) 2 x Brocade Gbps Fibre Channel SAN Switches Fibre Channel Network Two Dual Port 16GB HBAs U42 U41 U40 U39 U38 U37 U36 U35 U34 U33 U32 U31 U30 U29 U28 U27 U26 U25 U24 U23 U22 U21 U20 U19 U18 U17 U16 U15 U14 U13 U12 U11 U10 U09 U08 U07 U06 U05 U04 U03 U02 U01 19U 2 x S GbE TOR Switches (Public & Private) 1 x S55 Mgmt. Switch Database Servers (2 x R920) KMM Management Server (R630) 2 x Brocade Gbps FC Switches Dell Acceleration Appliances for Databases (DAAD-HA) Four IODrive2 iomemory Cards Mellanox Connect X3 Dual Port 40GB Network Adapaters Four IODrive2 iomemory Cards

4 Agenda Database Storage Challenges for IT Organizations Oracle 12c Information Lifecycle Management (ILM) Heat Map Automatic Data Optimization Examples of using Oracle ILM Questions

5 Database Storage Challenges for IT Organizations This is a subtitle or bulleted list

6 Database Storage Challenges Exponential data growth posts another Challenge to IT organization. Exposition in online access and contents such as pictures and videos. Mobile, Big Data, Social Medial, etc Compliance with government regulation on data retention Very common that your database performance is blocked by IOs Volumes of data makes the database IO bottleneck worse and worse IT Budgets Challenge Need more hard disks spindles or add SSDs to meet the database IO throughput and low latency requirement. More storage is very costly. Faster storage is even costly even the price is coming down IT budgets are not much increasing in most organization How to improve the performance, how to increase capacity without much growing cost Adapting storage tiering is the way to improve the storage efficience

7 Database Storage Challenges Storage IOs presents major bottlenecks to database performance As shown the following example, user I/O is the major bottleneck Even the configuration event is related to IO as it was the free buffer wait caused by the slow writing dirty data from the buffer cache to disks

8 Storage tiering: performance vs cost Pricing of high performance storage such as SSD is still higher. Only small percentage data is frequently accessed transactional data, and majority of data is less frequently accessed. An example of the tiered storage design: Tier1: small set of hot data (frequently accessed read/write intensive), using SLC SSD Tier 2: less frequently accessed or Read Intensive workloads: using MLC SSDs or 15k SAS for sequential workloads Tier 3: large volume of archival data :7.2k SATA/NL-SAS for low performance Tier1 SSD $$$ Tier 2 $$ $$ $$ $$ Tier 3 $ $ $ $ $ $ $ $

9 Implementation of Storage tiering Methods to implement storage tiering Manual method: manually assign or move the objects to storage tier. Build-in storage tiering in the storage product By Information management feature of database product. Manual method by Storage admin/dbas Manage the tired storage based on the performance, access pattern, sizing and storage cost. Identify IO performance bottleneck and move the objects to different storage tier. Example of two tiers storage with manual method: Tier 1: 4 PCI-SSD, Tier 2: MD3220 SAS with 24 X 15k rpms HDs Tier 3 $ $ $ $ $ $ $ $

10 Implementation of Storage tiering Performance of four configurations Config1: all objects stored in Tier 2(HDs) Config2: all indexes stored in Tier the rest stored in Tier 2 Config3: all indexes + one active table stored in Tier 1, the rest in Tier2 Config4: all indexes + four active tables stored in Tier 1 the rest stored in Tier 2 Tier 3 $ $ $ $ $ $ $ $

11 Implementation of Storage tiering Build-in storage tiering feature in the storage product The storage appliance moves blocks or pages of data from one type of disk to another type of disk based on rules or usage. Examples of storage Products: Dell Compellent Dell EqualLogic EMC FAST NetApp Advantages: Fully automated process. The simplest way to implement the storage tiering Disadvantages: Not application aware. Can t combine different vendors. Can t work if the tiered storage on multiple vendors storage Tier 3

12 Implementation of Storage tiering By Information management feature of applications. Applications keep track of the usage of the data System admin can define the rules for data moving base on the cost Applications moves data between these storage tiers based on the data usages and performance Oracle Information Lifecycle Management provides this feature. Advantages: The data moving process can be automated by setting the rules. Work well storage tiering among multiple storages products Tier 3 Conventional FC SAN Flash based storage

13 Oracle Information Lifecycle Management (ILM) This is a subtitle or bulleted list

14 Information Lifecycle Management (ILM) ILM is the practice of applying specific policies for effective information management. With respect to databases, ILM refers to the processes and practices of aligning digital records table rows with the most appropriate and cost effective IT infrastructure at each phase of its useful life. Examples: Moving transaction data to a data warehouse for improved analytics performance Compressing tables or partitions as data ages to reduce storage capacity Moving data for completed transactions to read-only storage Active Less Active Historical Archive

15 New features in Oracle Database 12c While ILM has been practiced for decades the explosion of new data and the proliferation of storage technologies has lead to innovations in improving management of automated ILM. Heat Map Stores system-generated data usage statistics at the block and segment levels Segment level Heat Map tracks the time of last modification and access of tables and partitions Row level Heat Map tracks modification times for individual rows (aggregated to the block level) Heat Map skips internal access done for system tasks automatically excluding Stats Gathering, DDLs or Table Redefinitions Automatic Data Optimization Allows you to create policies for data compression (Smart Compression) and data movement, to implement storage and compression tiering Smart Compression refers to the ability to utilize Heat Map information to associate compression policies, and compression levels, with actual data usage

16 Automatic Data Optimization (ADO) Compression types: Compression algorithm COMPRESS; ROW STORE COMPRESS BASIC: ROW STORE COMPRESS ADVANCED: COLUMNSTORE COMPRESS FOR QUERY: COLUMNSTORE COMPRESS FOR ARCHIVE: NOCOMPRESS: Description Basic table compression. The same as ROW STORE COMRESS BASIC The standard option (see above) Oracle DB compresses the data during Data Manipulation Language (DML) operations. This is recommended for Online Transaction Processing Applications (OLTP) Enables hybrid columnar compression in which the data is transformed into a column-oriented format then compressed. This is useful for Data Warehouses (OLAP) Same as above but compressed at a higher level. This is useful for archiving as it has a negative impact on performance. Disables table compression. The default. Data can be compressed at the table, partition/segment, or row level (rows are aggregated to the block level) Data can be tiered at the table or partition level (there is no row-based tiering)

17 Oracle ILM - Administration DBMS_ILM_ADMIN Provides an interface to customized Automatic Data Optimization (ADO) policy execution. customize_ilm Allows you to set parameters for ILM. disable_ilm Procedure to turn off background ILM enable_ilm Procedure to turn on background ILM clear_heat_map_all Procedure to delete all rows except dummy row set_heat_map_all Procedure to update/insert a row for all tables set_heat_map_table Procedure to update/insert a row for this table/segment clear_heat_map_table Procedure to clear all/some statistics for table set_heat_map_start Procedure to set start date of heat map data

18 Automatic Data Optimization Examples This is a subtitle or bulleted list

19 Automatic Data Optimization Examples Heat Map tracks usage information at the row and segment level : Views: V$/All/DBA/USER_HEAT_MAP_SEGMENT Add ADO policy: three parts Ado action: data compression or data movement Level: segment or row level for table or partition Condition: what condition will initiate an ADO action. ADO policy examples:

20 Automatic Data Optimization Examples Ado policy for data movement Thresholds at which ADO data movement policy will be started/stopped Tablespace percent used Tablespace percent free %used > 85% start %free < 30% stop Tablespaces on tier storage (in this example) T1DATA tablespace is built on 12x SLC SSDs in RAID10 T2DATA tablespace is built on 12x 1TB 7.2k rpm Hard disks in RAID10

21 Automatic Data Optimization Examples Example 1 : Create segment-level ADO policy to automatically compress table after 10 days no modification Create compression policy Check the policy via USER_ILMDataMovementPolicies Display Heat map tracking data via user_heat_map_segment

22 Automatic Data Optimization Examples Execute the ADO policy immediately Check the task status: policy_name: P102, task_id: 2146 Verify the tables compression

23 Automatic Data Optimization Examples Example 2 : Create ADO policy to move objects from T1DATA to T2DATA Current TBS: Check Tablespaces space usage: PCT_FREE, PCT_USED

24 Automatic Data Optimization Examples Check ADO parameters for ADO data migration threshold T1DATA: PCT_USED: 91.62% > 85% threshold Start migration when pct_used > 85%,stop migration when pct_free < 30% Create data movement policy: Check the policy: policy name: P122

25 Automatic Data Optimization Examples Execute the ADO policy immediately Check the task status : policy_name: P122, task_id: 2214

26 Automatic Data Optimization Examples Check the result Tablespace space usage: free space on T1DATA tablespace: T1DATA 8.38% free T1DATA: 15.38% free Before Aftersage:

27 Automatic Data Optimization Examples Example 3 : Compress partitions Check partitions and their write time Create policy

28 Automatic Data Optimization Examples Check policy Check execution only 6 days since last update, less than 10 days requirement. Recreate Policy

29 Automatic Data Optimization Examples Check policy: policy_name: P145, task_id: 2358 Check execution of task Check the result

30 Automatic Data Optimization Examples Example 4: Move partition to tier 2 storage T2DATA: Current tablespace: Create policy: Execute the policy:

31 Automatic Data Optimization Examples Why?? T1DATA s PCT_USED (84.62%) < 85% threshold Force the policy to get executed by changing T1DATa PCT_USED parameter Execute the policy again:

32 Automatic Data Optimization Examples Check the task: Check the result Tablespace before the data movement

33 References Heat Map, Automatic Data Optimization and ILM with Oracle Database - DMBS_ILM_ADMIN Documentation -

34 Contact us at or visit my Oracle Blog at

35 Please complete the session evaluation We appreciate your feedback and insight You may complete the session evaluation either on paper or online via the mobile app