Five Things You Might Not Know About Oracle Database

Transcription

1 Five Things You Might Not Know About Oracle Database Maria Colgan Oracle Database Product Management February

2 Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle s products remains at the sole discretion of Oracle. 2

3 Program Agenda Indexing Clustering Data Online Operation JSON Plan Stability 3

4 Indexing 4

5 Oracle Offers a Variety of Different Indexing Techniques B-Tree Indexes Root Block Branch Blocks leave Blocks Available since 5, they are the most common type of index Goal: minimize time to find small amounts of data Including function-based, Reverse Key Indexes, & IOTs One to one mapping, little degradation in retrieval performance as underlying table grows Bitmap Indexes Available since 7.3, designed for read-mostly workloads Goal: Improve performance of ad-hoc analytic queries A single bitmap key entry points to many rows Multiple indexes can be used together to determine rowids Bitmap Join indexes replaces need for two table join Text Search Indexes Available since 8i designed for text retrieval Goal: Quickly identify large coherent documents Can index documents of different formats such as MS Word, HTML or plain text A single entry points to many documents 5

6 Reverse Key Indexes Relieving Index contention Reverse key indexes physically stores the bytes of the keys in reverse order CREATE INDEX sales_prod_reverse_idx ON sales(prod_id) REVERSE; Used to avoid hot block syndrome or buffer busy waits On columns with monotonically increasing values Eg: sequence populated fields or dates/timestamps At a cost Prevents some index range scans as 5 is not stored next to 6 is not stored next to 7 Additional CPU evident mostly in single user mode, with multi-users you see less overall CPU because you are not spinning on a buffer wait

7 Partially Useable Indexes Relieving Index contention Enables the creation of local or global indexes on just a subset of partitions Indexing only the stable partitions (little or no DML) minimizes the impact on ingest performance Queries accessing data only within the indexed partitions will use the index Queries that access data only in the non-indexed partitions will do full scan Queries accessing partitions with and without indexes can take advantage of the query transformation called Table Expansion Supported by definition on Partition views 7

8 Partially Useable Indexes Relieving Index contention CREATE TABLE sales(order_status char(6) not null, order_date ) INDEXING ON PARTITION BY LIST (order_status)( PARTITION p1 VALUES( OPEN ) INDEXING OFF, PARTITION p2 VALUES( CLOSED )); 8

9 Table Expansion Transformation SELECT * FROM sales WHERE ORDER_DATE between and ; SALES P1: ORDER_STATUS= OPEN P2: ORDER_STATUS= CLOSED Local index on ORDER_DATE IDX_SALES: ORDER_DATE P1: Best access path is full table scan P2: Best access path is index scan 9

10 Table Expansion Transformation Id Operation Name Pstart Pstop SELECT STATEMENT 1 SORT AGGREGATE 2 VIEW VW_TE_2 3 UNION-ALL 4 PARTITION RANGE SINGLE 1 1 * 5 TABLE ACCESS FULL SALES PARTITION RANGE SINGLE 2 2 * 7 INDEX RANGE SCAN IDX_SALES access("order_date >=TO_DATE(' :00:00', 'syyyy-mm-dd hh24:mi:ss') AND "ORDER_DATE"<=TO_DATE(' :00:00', 'syyyy-mm-dd hh24:mi:ss')) filter("order_date >=TO_DATE(' :00:00', 'syyyy-mm-dd hh24:mi:ss') AND "ORDER_DATE"<=TO_DATE(' :00:00', 'syyyy-mm-dd hh24:mi:ss')) 7 - access("order_date >=TO_DATE(' :00:00', 'syyyy-mm-dd hh24:mi:ss') AND "ORDER_DATE"<=TO_DATE(' :00:00', 'syyyy-mm-dd hh24:mi:ss'))

11 How Do Nulls And Indexes Work Together? Bitmap indexes always index nulls B*Tree cluster indexes always index nulls B*Tree indexes do not if and only if the entire key is null (all columns) CREATE INDEX i ON t(object_id,0); SELECT * FROM t WHERE object_id is NULL; Id Operation Name Rows Bytes Cost (%CPU) Time SELECT STATEMENT K 96 (0) 00:00:02 1 TABLE ACCESS BY INDEX ROWID T K 96 (0) 00:00:02 * 2 INDEX RANGE SCAN I (0) 00:00:

12 Index Usage Statistics How do I know which indexes are actually needed? New DBA_INDEX_USAGE views provide usage histograms and access details for all indexes Track index usage with no overhead Public 12

13 Index Usage Statistics Index entries with no statistics indicate indexes that currently are not being used These are potential candidates to be deleted Public 13

14 INVISIBLE Indexes Ability to create and maintains an index that the optimizer is blind to Optimizer can t use an invisible index as an access method Easy mechanism to begin the process of removing unused indexes 1. Mark the index invisible ALTER INDEX i INVISIBLE; 2. If no one misses it for a complete business cycle, its safe to drop it Also useful to test out the benefit of a new index before publishing it 1. Create invisible index CREATE INDEX i ON t(x, object_id) INVISIBLE; 2. Set parameter OPTIMIZER_USE_INVISIBLE_INDEXES to true in 1 session to test ALTER SESSION SET optimizer_use_invisible_indexes=true;

15 Clustering Data 15

16 Clustering Data : Bringing it closer together Table Clusters Data Compression Attribute Clustering A data block contains data from multiple tables based around a cluster key Reduce I/O for commonly joined tables Index or Hash Shouldn t be used on tables that are commonly full table scanned or updated heavily Compression of indexes and data results in more efficient I/O and denser caching of data Specifies how data should be ordered and clustered together Also Consider. Data closer together results Less storage required in more efficient I/O for Transparent to application Partitioning scanning, joining and index lookups Big improvements in Potentially Oracle a more Database 12c Release flexible 2 model. Improves compression ratios 16

17 Index Compression First form of compression introduced in Oracle 8i Most operational Databases might have as much as 50% of their space dedicated to indexes Remove redundant leading edge values in index keys Break key into prefix and suffix Repeating prefix values are stored only once per leaf block Non costed feature (EE only) 17

18 Index Compression High 1.8X 4.6X 1.2 TB 679 GB 270 GB New Index High compression enables further compression of indexes Significant space savings for index heavy applications (i.e. OLTP) Index compression can also result in I/O improvements as a higher proportion of index can be cached No Compression Low Compression 10,000 Indexes High Compression 13% Improvement in workload throughput (for low compression) 18

19 Attribute Clustering Orders data so that it is in close proximity based on selected columns values: attributes Linear or Interleaved (Z-Order) Significant IO pruning when used with zone maps Also, reduced block IO for table lookups in index range scans Able to cluster data during MOVE PARTITION Queries that sort and aggregate can benefit from pre-ordered data Enable improved compression ratios

20 Attribute Clustering CREATE TABLE sales ( product_id store_id sales_date NUMBER NOT NULL, NUMBER NOT NULL, DATE NOT NULL, loyalty_card_id NUMBER, quantity_sold value_sold ) NUMBER(3) NOT NULL, NUMBER(10,2) NOT NULL CLUSTERING BY LINEAR ORDER (sales_date, product_category, store_id); 20

21 Online Operations 21

22 Online Operations DBAs often have a long list of maintenance activities & data model changes Goal to improve overall system performance In the past these tasks have required exclusive locks Forcing DBAs to wait until application downtime to complete them With online operations these tasks can occur when necessary without delay Online operations Wait for active transactions to end before beginning Do not block any new DML statements (parallel DML is the only exception) 22

23 Adding a Column to a Table Online From Oracle Database 11g onwards adding a column with or without a default value is a metadata only operation Fast online operation that allocates no space & generates no redo or undo Other changes to a table can also be done online: Marking a column or an index invisible Dropping a constraint or an index Building or rebuilding an index 23

24 Online Segment Shrink Over time tables can become sparely populated as rows are inserted & deleted Removing the whitespace would Speed up full scan Reduce the memory require to cache the table Online segment shrink removes whitespace 1. Enable row movement so ROWIDs can change without invalidating indexes ALTER TABLE sales ENABLE ROW MOVEMENT; 2. Shrink the table ALTER TABLE sales SHRINK SPACE CASCADE; SALES

25 Online Statistics Gathering Traditionally statistics need to be manually gathered after a large data load With online statistics gathering automatically gathered as part of direct path load operations Create Table As Select Insert As Select commands into empty objects Statistics available directly after load Similar to how index statistics are maintained No additional table scan to gather statistics All internal maintenance operations that use CTAS operations benefit from this Note: Histograms are not created as they require an additional scan of the data 25

26 Online Table Move SALES tablespace: active Closed orders Open orders SALES tablespace: ts_data Ability to move a table or partition from one segment to another online ALTER TABLE sales MOVE ONLINE TABLESPACE ts_data COMPRESS INCLUDING ROWS WHERE status = OPEN UPDATE INDEXES; Opportunity to modify storage attributes and to filter out some of the data as part of the move Global indexes automatically maintained NOTE: It s also possible to do an online datafile move 26

27 Online Table Conversion to Partitioned Table SALES Ability to convert a non-partitioned table to a partitioned table online SALES USA GERMANY JAPAN DEFAULT ALTER TABLE sales MODIFY PARTITION BY LIST (region) (partition p1 values ( USA ), partition p2 values ( Germany ), partition p3 values ( Japan ), partition p4 values (DEFAULT)) UPDATE INDEXES ONLINE; 27

28 JSON 28

29 Native SQL Support for JSON JSON is the most popular data format for new web applications JSON in database greatly simplifies app development Same schema-less data representation in App and DB Oracle stores JSON in table columns, with native SQL support SELECT c.json_column.address.city FROM customers c; JSON supported by all Oracle features Analytics, Encryption, In-Memory, RAC, Replication, Parallel SQL, Plus can manage JSON documents, and index any JSON element { } JSON Better JSON than JSON Only Databases 29

30 Oracle Database 12c as a Document Store 12.2 JSON Table containing JSON documents CREATE TABLE movie_tickets (booking_id number(16), booking details varchar2(4000) CONSTRAINT is_json CHECK (booking_details IS JSON) ); { "Theater":"AMC 15", Movie": La La Land", "Time : :45:00", "Tickets":{ "Adults":2 "show" : "Evening" }} IS_JSON constraint ensures only legal JSON documents can be inserted JSON can be queried using simple SQL dot notation SELECT m.booking_details.movie FROM movie_tickets m; Movie Fences Hacksaw Ridge Hell or High Water Hidden Figures La La Land Public 30

31 Data Guide : Understanding your JSON documents Metadata discovery: discovers the structure of collection of JSON documents Optional: deep analysis of JSON for List of Values, ranges, sizing etc. Automatically Generates Virtual columns Relational views De-normalized relational views for arrays Reports/Synopsis of JSON structure 31

32 Data Guide : Understanding Your JSON Documents SQL> SELECT JSON_DATAGUIDE(m.booking_details) FROM movie_tickets m; JSON_DATAGUIDE(O.ORDER_DETAILS) [ {"o:path": "$.Theater", "type": "string", "o:length": 132 }, {"o:path": "$.Movie", "type": "string", "o:length": 256 }, {"o:path": "$.Time", "type": string", "o:length": 32 }, {"o:path": "$.Tickets", "type": object", "o:length": 64 }, {"o:path": "$.Tickets. Adults","type": number", "o:length": 2 },... ] 32

33 Oracle Database 12c as a Document Store 12.2 JSON DataGuide Automatic Schema Inference Table containing JSON documents JSON DataGuide DBMS_JSON.AddVC( MOVIE_TICKETS, BOOKING_DETAILS ); Table enhanced with virtual columns SQL> desc MOVIE_TICKETS NAME TYPE BOOKING_ID RAW(16) BOOKING_TIME TIMESTAMP(6) BOOKING_DETAILS VARCHAR2(4000) BOOKING_DETAILS$Movie VARCHAR2(16) BOOKING_DETAILS$Theater VARCHAR2(16) BOOKING_DETAILS$Adults NUMBER BOOKING_DETAILS$Time VARCHAR2(32) Public 33

34 JSON Search Index : A universal index for JSON content CREATE SEARCH INDEX json_search_index ON customers (json_document) FOR JSON; Search on JSON attribute names and path values Range searches on numeric values within JSON documents Full text searches with all the power of Oracle Text Full boolean search capabilities (and, or, and not) together with phrase search, proximity search and "within field" searches. Inexact queries: fuzzy match, soundex and name search. Automatic linguistic stemming for 32 languages 34

35 Plan Stability 35

36 Plan Stability is Critical For Predictable Performance Unpredictable changes can happen to an execution plan Avoiding plan changes becomes the only method to avoid performance regression Lock statistics to prevent them from changing does guartenee the plan won t change Freezing an execution plan with a Stored Outline, which have been deprecated! No mechanism for plans to evolve! Solution use SQL Plan Management Optimizer automatically manages execution plans Only known and verified plans are used Plan changes are verified Only comparable or better plans are used going forward Now available in Standard Edition 36

37 How SPM Works SELECT count(empno) tot FROM emp e, dept d WHERE e.deptno=d.deptno AND d.dname= SALES ; 2 During hard parse Optimizer determines execution plan EMP NL DEPT Execute 1 SQL statement is submitted Plan history Plan baseline 3 Acceptable plan Before execution, the plan is compared to the plan in the baseline to confirm it s acceptable NL EMP DEPT 37

38 How SPM Works SELECT count(empno) tot FROM emp e, dept d WHERE e.deptno=d.deptno AND d.dname= SALES ; 1 SQL statement is submitted Plan history Plan baseline 2 During hard parse Optimizer determines execution plan 3 EMP HJ DEPT Plan Unacceptable Before execution, the plan is compared to the plan in the baseline to confirm it s acceptable EMP HJ DEPT EMP NL DEPT 4 If the plan does not match an accepted plan in the SQL plan baseline it is added to the plan baseline but not executed 38

39 How SPM Works SELECT count(empno) tot FROM emp e, dept d WHERE e.deptno=d.deptno AND d.dname= SALES ; EMP NL DEPT Execute Plan history Plan baseline 5 Acceptable plan Only an accepted plan will be use HJ NL EMP DEPT EMP DEPT 39

40 How SPM Works HJ Plan history Plan baseline NL Execute evolve process either automatically via an autotask or manually using DBMS_SPM package EMP DEPT EMP DEPT Performance of the new plan is compared to the accepted plan both in terms of elapse time & buffer gets If new plan is 50% better it will be accepted Otherwise it remains an unaccepted plan 40

41 Using SPM to Influence Execution Plan Without Adding Hints Example Overview Simple two table join between the SALES and PRODUCTS tables SELECT p.prod_name,sum(s.amount_sold) FROM products p, sales s WHERE p.prod_id = s.prod_id AND p.supplier_id = :sup_id GROUP BY p.prod_name; INDEX TABLE RANGE ACCESS SCAN PROD_SUPP_ID_INDX PRODUCTS GROUP BY HASH JOIN TABLE ACCESS SALES Current Desired Plan

42 Using SPM to Influence Execution Plan Without Adding Hints Step 1. Execute the non-hinted SQL statement SELECT p.prod_name,sum(s.amount_sold) FROM products p, sales s WHERE p.prod_id = s.prod_id AND p.supplier_id = :sup_id GROUP BY p.prod_name; PROD_NAME SUM(S.AMOUNT_SOLD) Baseball trouser Kids 91 Short Sleeve Rayon Printed Shirt $

43 Using SPM to Influence Execution Plan Without Adding Hints Default plan uses full table scans followed by a hash join

44 Using SPM to Influence Execution Plan Without Adding Hints Step 2. Find the SQL_ID for the non-hinted statement in V$SQL SELECT sql_id, sql_fulltext FROM v$sql WHERE sql_text LIKE 'SELECT p.prod_name, %'; SQL_ID SQL_FULLTEXT akuntdurat7yr SELECT p.prod_name, SUM(s.amount_sold) FROM products p, sales s WHERE p.prod

45 Using SPM to Influence Execution Plan Without Adding Hints Step 3. Create a SQL plan baseline for the non-hinted SQL statement VARIABLE cnt NUMBER EXECUTE :cnt := dbms_spm.load_plans_from_cursor_cache(sql_id=>'akuntdurat7yr'); PL/SQL PROCEDURE successfully completed. SELECT sql_handle, sql_text, plan_name, enabled FROM WHERE dba_sql_plan_baselines sql_text LIKE 'SELECT p.prod_name, %'; SQL_HANDLE SQL_TEXT PLAN_NAME ENA SQL_8f876d cf SELECT p.prod_name, sum(s.amount_sold) SQL_PLAN_8z1vdhk1176 YES FROM products p, sales s g

46 Using SPM to Influence Execution Plan Without Adding Hints Step 4. Disable plan in SQL plan baseline for the non-hinted SQL statement PL/SQL PROCEDURE successfully completed. SELECT sql_handle, sql_text, plan_name, enabled FROM dba_sql_plan_baselines WHERE sql_text LIKE 'SELECT p.prod_name, %'; EXECUTE :cnt := dbms_spm.alter_sql_plan_baseline(sql_handle=>'sql_8f876d cf,- plan_name=>'sql_plan_8z1vdhk11766g ',- attribute_name => 'enabled, - attribute_value => 'NO'); SQL_HANDLE SQL_TEXT PLAN_NAME ENA SQL_8f876d cf SELECT p.prod_name, sum(s.amount_sold) SQL_PLAN_8z1vdhk1176 NO FROM products p, sales s g

47 Using SPM to Influence Execution Plan Without Adding Hints Step 5. Manually modify the SQL statement to use the hint(s) & execute it SELECT /*+ index(p) */ p.prod_name,sum(s.amount_sold) FROM products p, sales s WHERE p.prod_id = s.prod_id AND p.supplier_id = :sup_id GROUP BY p.prod_name; PROD_NAME SUM(S.AMOUNT_SOLD) Baseball trouser Kids 91 Short Sleeve Rayon Printed Shirt $

48 Using SPM to Influence Execution Plan Without Adding Hints Step 6. Find SQL_ID & PLAN_HASH_VALUE for hinted SQL stmt in V$SQL SELECT sql_id, plan_hash_value, sql_fulltext FROM v$sql WHERE ; sql_text LIKE 'SELECT /*+ index(p) */ p.prod_name, %' SQL_ID PLAN_HASH_VLAUE SQL_FULLTEXT avph0nnq5pfc SELECT /*+ index(p) */ p.prod_name, SUM( s.amount_sold) FROM products p, sales

49 Using SPM to Influence Execution Plan Without Adding Hints Step 7. Associate hinted plan with original SQL stmt s SQL HANDLE VARIABLE cnt NUMBER EXECUTE :cnt := dbms_spm.load_plans_from_cursor_cache(sql_id=>'avph0nnq5pfc2,- plan_hash_value=>' ', - sql_handle=>'sql_8f876d cf ); PL/SQL PROCEDURE successfully completed. SQL_ID & PLAN_HASH_VALUE belong to hinted statement SQL_HANDLE is for the non-hinted statement

50 Using SPM to Influence Execution Plan Without Adding Hints Step 8. Confirm SQL statement has two plans in its SQL plan baseline SELECT sql_handle, sql_text, plan_name, enabled FROM WHERE dba_sql_plan_baselines sql_text LIKE 'SELECT p.prod_name, %'; SQL_HANDLE SQL_TEXT PLAN_NAME ENA SQL_8f876d cf SELECT p.prod_name, sum(s.amount_sold) SQL_PLAN_8z1vdhk1176 FROM products p, sales s g SQL_8f876d cf SELECT p.prod_name, sum(s.amount_sold) SQL_PLAN_8z1vdhk1176 YES FROM products p, sales s 6ge1c67f67 NO Hinted plan is the only enabled plan for non-hinted SQL statement

51 Using SPM to Influence Execution Plan Without Adding Hints Step 9. Confirm hinted plan is being used Non-hinted SQL text but it is using the plan hash value for the hinted statement Note section also confirms SQL plan baseline used for statement

52 If you have more questions later, feel free to ask

53 Additional Resources Related White Papers Oracle Database 12c Whitepaper JSON Whitepaper SQL Plan Management Related Videos Dominic s YouTube Channel Maria s YouTube Channel Oracle Database PM Channel Join the Conversation Any Additional Questions Our s: maria.colgan@oracle.com dominic.giles@oracle.com 53