Real-World Performance Training SQL Performance

Size: px
Start display at page:

Download "Real-World Performance Training SQL Performance"

Transcription

1

2 Real-World Performance Training SQL Performance Real-World Performance Team

3 Agenda SQL and the Optimizer You As The Optimizer Optimization Strategies Why is my SQL slow? Optimizer Edges Cases Top SQL Mistakes

4 Agenda SQL and the Optimizer You As The Optimizer Optimization Strategies Why is my SQL slow? Optimizer Edges Cases Top SQL Mistakes

5 SQL and the Optimizer

6 All Roads Lead to SQL The following items have more impact on the performance than any other aspect of Relational Database Design. Schema Design SQL Design Execution Strategy Optimization Technique Improvements to all aspects of SQL allow for orders of magnitude improvement to all SQL metrics

7 The Optimizer The cost-based optimizer is responsible for selecting a strategy for the execution of a query Important decisions Access method Join mechanism Join order Distribution method (for parallel execution)

8 Agenda SQL and the Optimizer You As The Optimizer Optimization Strategies Why is my SQL slow? Optimizer Edges Cases Top SQL Mistakes

9 You As The Optimizer If a query retrieves 10 rows from a 50 million row table, what is the correct access method? INDEX If a query retrieves 40 million rows from a 50 million row table, what is the correct access method? TABLE SCAN Did you make this decision based on rules, or an estimate of cost?

10 You As The Optimizer Determining the Access Method In the previous example we were given the number of rows to be retrieved. But usually the optimizer is not given this information, so let s look at a query and discuss how the optimizer decides which access method to use The query will ask What is the total quantity of orders in the week of July 4 th 1997 sold with a zero tax rate?

11 You As The Optimizer What information do you need to determine the access method? SELECT SUM(lo_quantity) FROM lineorder WHERE lo_orderdate BETWEEN to_date(' ','yyyymmdd') AND to_date(' ','yyyymmdd') AND lo_tax = 0 ;

12 You As The Optimizer What information do you need to determine the access method? SELECT SUM(lo_quantity) FROM lineorder WHERE lo_orderdate BETWEEN to_date(' ','yyyymmdd') AND to_date(' ','yyyymmdd') AND lo_tax = 0 ; Number of rows in the table Existence of indexes If the table is partitioned, how many partitions will need to be accessed Estimate of the number of rows to be retrieved from the table NDV Number of distinct values in a column How selective are the filter columns? If there are lots of distinct values the filter predicates will probably have fewer rows If there are few distinct values the filter predicates will probably have more rows

13 What are Statistics Basics The statistics contain information which the optimizer uses when developing the execution plan Statistics are kept at different levels Table Partition Column Index

14 What are Statistics Main Table and Partition Statistics Statistic Rows Blocks Empty blocks Avg Row Length Description Number of rows in the object Number of blocks in the object Number of empty blocks in the object Average Row Length in the object 14

15 What are Statistics Main Column Statistics Statistic NDV Low Value High Value Density Num Nulls Description Number of distinct values Low value in column High value in column Shows distribution of values Number of nulls in the column 15

16 You As The Optimizer Determining the Access Method The optimizer uses statistics to get an estimate of the number of rows retrieved from the table or index This is called the cardinality estimate The optimizer estimates how much data, in bytes, will be retrieved The optimizer estimates the cost for the access The optimizer evaluates the executions plans and determines their cost The optimizer picks the plan with the lowest cost

17 You As The Optimizer Determining the Access Method SELECT SUM(lo_quantity) FROM lineorder WHERE lo_orderdate BETWEEN to_date(' ','yyyymmdd') AND to_date(' ','yyyymmdd') AND lo_tax = 0 ; lineorder rows 53,986,608 lineorder partitioned no lo_orderdate indexed yes, btree lo_tax indexed no lo_orderdate NDV 2406 lo_tax NDV 9

18 You As The Optimizer Determining the Access Method Index Scan Id Operation Name Rows Bytes Cost (%CPU) Time SELECT STATEMENT (1) 00:00:02 1 SORT AGGREGATE 1 14 * 2 TABLE ACCESS BY INDEX ROWID BATCHED LINEORDER K (1) 00:00:02 * 3 INDEX RANGE SCAN LO_DATE_N 179K 480 (1) 00:00:

19 You As The Optimizer Determining the Access Method Table Scan Id Operation Name Rows Bytes Cost (%CPU) Time SELECT STATEMENT K (1) 00:00:08 1 SORT AGGREGATE 1 14 * 2 TABLE ACCESS FULL LINEORDER K 187K (1) 00:00:

20 You As The Optimizer Determining the Access Method Access Method Cost Index Scan 47,908 Table Scan 187,519 20

21 Execution Plan Basics Execution Plan Row Source Access method Join method Join order Cardinality Shows the detailed steps necessary to execute a SQL statement Execution plan operators that produce and/or consume rows The way in which the data is being accessed The method used to join tables with each other. The order in which the tables are joined to each other Number of rows from a row source 21

22 Execution Plan DBMS_XPLAN.DISPLAY_* Overview Gives the actual execution plan from various sources Cursor cache dbms_xplan.display_cursor AWR dbms_xplan.display_awr SQL Tuning Set dbms_xplan.display_sqlset

23 Execution Plan DBMS_XPLAN.DISPLAY_CURSOR Output Plan hash value: Id Operation Name Rows Bytes Cost (%CPU) Time SELECT STATEMENT 187K(100) 1 SORT AGGREGATE 1 14 * 2 TABLE ACCESS STORAGE FULL LINEORDER K 187K (1) 00:00: Predicate Information (identified by operation id): storage(("lo_tax"=0 AND "LO_ORDERDATE">=TO_DATE(' :00:00', 'syyyy-mm-dd hh24:mi:ss') AND "LO_ORDERDATE"<=TO_DATE(' :00:00', 'syyyy-mm-dd hh24:mi:ss'))) filter(("lo_tax"=0 AND "LO_ORDERDATE">=TO_DATE(' :00:00', 'syyyy-mm-dd hh24:mi:ss') AND "LO_ORDERDATE"<=TO_DATE(' :00:00', 'syyyy-mm-dd hh24:mi:ss')))

24 The Execution Plan Actual and Estimated Cardinality Each line of an execution plan can be considered a row source Estimated cardinality is the number of rows the Cost-Based Optimizer expects from a row source Determined at compile time Actual cardinality is the number of rows from a row source Determined at runtime

25 Execution Plan Actual and Estimated Cardinality In the SQL Monitor Report 2/14/2016

26 You As The Optimizer Determining the Join Order and Method Now let s look at a query that has more than one table We will ask the same question but instead of putting date predicates directly on the lineorder table, we join the lineorder table to date_dim and we specify our date range using date_dim What is the total quantity of orders in the week of July 4 th 1997 sold with a zero tax rate?

27 You As The Optimizer Determining the Join Order and Method SELECT SUM(lo_quantity) FROM lineorder JOIN date_dim ON lo_orderdate = d_datekey WHERE d_year = 1997 AND d_weeknuminyear = 27 AND lo_tax = 0 ;

28 You As The Optimizer Determining the Join Order and Method Consider the join order If we retrieve X number of rows from lineorder, how many rows will we then retrieve from date_dim And vice versa, if we retrieve X number of rows from date_dim, how many rows will we then retrieve from lineorders How many rows from each table overlap or join?

29 You As The Optimizer Determining the Join Order and Method Table Statistic Value lineorder rows 53,986,608 date_dim rows 2,556 lineorder lo_orderdate low 01-JAN-92 lineorder lo_orderdate high 02-AUG-98 date_dim d_datekey low 31-DEC-91 date_dim d_datekey high 29-DEC-98 29

30 You As The Optimizer Determining the Join Order and Method Table Statistic Value lineorder lo_orderdate NDV 2,406 lineorder lo_tax NDV 9 date_dim d_datekey NDV 2,556 date_dim d_year NDV 8 date_dim d_weeknuminyear NDV 53 30

31 You As The Optimizer Determining the Join Order and Method For each order, we then consider which join method to use Hash Nested Loop Merge

32 You As The Optimizer Determining the Join Order and Method Nested Loops Join Nested loops joins work best when retrieving a small number of rows for the join condition A nested loop join involves the following steps: 1. The optimizer determines the driving table and designates it as the outer table. 2. The other table is designated as the inner table. 3. For every row in the outer table, retrieve matching rows in the inner table. If 10 rows are retrieved from the outer table we need to perform 10 lookups in the inner table If 10M rows are retrieved from the outer table we need to perform 10M lookups in the inner table These lookups can be performed via index or table scans

33 You As The Optimizer Determining the Join Order and Method Nested Loops Join Id Operation Name Rows Bytes Cost (%CPU) Time SELECT STATEMENT (1) 00:00:02 1 SORT AGGREGATE NESTED LOOPS K (1) 00:00:02 3 NESTED LOOPS 157K 486K (1) 00:00:02 4 TABLE ACCESS BY INDEX ROWID BATCHED DATE_DIM (0) 00:00:01 * 5 INDEX RANGE SCAN DATE_DIM_N1 7 1 (0) 00:00:01 * 6 INDEX RANGE SCAN LO_DATE_N (0) 00:00:01 * 7 TABLE ACCESS BY INDEX ROWID LINEORDER (1) 00:00: Outer Table Inner Table

34 You As The Optimizer Determining the Join Order and Method Nested Loops Join Id Operation Name Rows Bytes Cost (%CPU) Time SELECT STATEMENT K (1) 00:04:02 1 SORT AGGREGATE NESTED LOOPS K 6187K (1) 00:04:02 3 NESTED LOOPS 5998K 486K 6187K (1) 00:04:02 * 4 TABLE ACCESS FULL LINEORDER 5998K 80M 187K (1) 00:00:08 * 5 INDEX UNIQUE SCAN DATE_DIM_PK 1 0 (0) 00:00:01 * 6 TABLE ACCESS BY INDEX ROWID DATE_DIM (0) 00:00: Outer Table Inner Table

35 You As The Optimizer Determining the Join Order and Method HASH Join Hash joins are used for joining large data sets. 1. The optimizer uses the smaller of the two tables or data sources to build a hash table on the join key in memory, using a deterministic hash function to specify the location in which to store each row in the hash table. This is called the build side. 2. It then scans the larger table, called the probe table, probing the hash table to find the joined rows. This method is best used when the smaller table fits in available memory. The cost is then limited to a single read pass over the data for the two tables.

36 You As The Optimizer Determining the Join Order and Method HASH Join Id Operation Name Rows Bytes Cost (%CPU) Time SELECT STATEMENT K (1) 00:00:08 1 SORT AGGREGATE 1 29 * 2 HASH JOIN K 187K (1) 00:00:08 3 TABLE ACCESS BY INDEX ROWID BATCHED DATE_DIM (0) 00:00:01 * 4 INDEX RANGE SCAN DATE_DIM_N1 7 1 (0) 00:00:01 * 5 TABLE ACCESS FULL LINEORDER 5998K 80M 187K (1) 00:00: Build Table Probe Table

37 You As The Optimizer Determining the Join Order and Method HASH Join Id Operation Name Rows Bytes TempSpc Cost (%CPU) Time SELECT STATEMENT K (1) 00:00:08 1 SORT AGGREGATE 1 29 * 2 HASH JOIN K 148M 194K (1) 00:00:08 * 3 TABLE ACCESS FULL LINEORDER 5998K 80M 187K (1) 00:00:08 4 TABLE ACCESS BY INDEX ROWID BATCHED DATE_DIM (0) 00:00:01 * 5 INDEX RANGE SCAN DATE_DIM_N1 7 1 (0) 00:00: Build Table Probe Table

38 You As The Optimizer Determining the Join Order and Method Merge Join Merge joins are used for joining large data sets. Requires the data to be sorted not as efficient as a hash join Used with inequalities where a hash join is not able to be used How it works: 1. A sort merge join reads two data sets and sorts them when they are not already sorted. 2. For each row in the first data set, the database finds a starting row in the second data set, and then reads the second data set until it finds a nonmatching row. 3. For large data sets that do not fit into memory the sorts will spill to temp, which has a noticeable performance impact while performing physical IO

39 You As The Optimizer Determining the Join Order and Method Merge Join Id Operation Name Rows Bytes TempSpc Cost (%CPU) Time SELECT STATEMENT K (1) 00:00:09 1 SORT AGGREGATE MERGE JOIN K 217K (1) 00:00:09 3 SORT JOIN (34) 00:00:01 4 TABLE ACCESS BY INDEX ROWID BATCHED DATE_DIM (0) 00:00:01 * 5 INDEX RANGE SCAN DATE_DIM_N1 7 1 (0) 00:00:01 * 6 SORT JOIN 5998K 80M 275M 217K (1) 00:00:09 * 7 TABLE ACCESS FULL LINEORDER 5998K 80M 187K (1) 00:00:

40 You As The Optimizer Determining the Join Order and Method Merge Join Id Operation Name Rows Bytes TempSpc Cost (%CPU) Time SELECT STATEMENT K (1) 00:00:09 1 SORT AGGREGATE MERGE JOIN K 217K (1) 00:00:09 3 SORT JOIN 5998K 80M 275M 217K (1) 00:00:09 * 4 TABLE ACCESS FULL LINEORDER 5998K 80M 187K (1) 00:00:08 * 5 SORT JOIN (34) 00:00:01 6 TABLE ACCESS BY INDEX ROWID BATCHED DATE_DIM (0) 00:00:01 * 7 INDEX RANGE SCAN DATE_DIM_N1 7 1 (0) 00:00:

41 You As The Optimizer Determining the Join Order and Method The optimizer will associate a cost with each of these based on it s estimate of the amount of work The goal is to reduce the number of rows early so it will perform less work throughout the execution of the SQL statement We basically have a matrix of join orders and methods and the cost associated with each

42 You As The Optimizer Determining the Join Order and Method Join Method Join Order date_dim, lineorder lineorder, date_dim Nested Loops 39,480 6,187,540 Hash 187, ,909 Merge 217, ,129 42

43 Serial vs Parallel Execution

44 Serial and Parallel Execution Serial Execution SQL is executed by one process The correct solution when: the query references a small data set high concurrency efficiency is important Parallel Execution SQL is executed by many processes working together The correct solution when: the query references a large data set low concurrency elapsed time is important

45 Parallel Execution Basics Query Coordinator (QC) Parallel Execution Server (PX) Degree of Parallelism (DoP) Parallel server group Distribution method The top level process for the parallel query An (OS) process that operates on part of a parallel query The number of parallel execution servers used in each parallel server group during parallel execution The group of parallel server processes that operate on a row source The method by which data is sent from one set of PX servers to another 45

46 Distribution Methods

47 Parallel Query Lets look at 2 parallel execution plans for the same query: select count(*) from yellow y, green g where y.deptno = g.deptno

48 Parallel Query Hash Distribution Green Table Scan PQ1 PQ2 f(x) f(x) Hash Distribution PQ3 Yellow Table Scan PQ1 PQ2 f(x) f(x) PQ4 Join and Hash Distribution Aggregate QC Result Set

49 Parallel Query Hash Distribution

50 Hash Distribution Producers Consumers

51 Hash Distribution Producers Consumers Sent 16 rows Received 16 rows

52 Parallel Query Broadcast Distribution Green Table Scan PQ1 Broadcast Distribution PQ2 PQ3 Yellow Table Scan, Join and Aggregate PQ4 QC Result Set

53 Parallel Query Broadcast Distribution

54 Broadcast Distribution Producers Consumers

55 Broadcast Distribution Producers Consumers Sent 16 rows Received 64 rows

56 Parallel Execution What may go wrong Skew Distribution method A PX process will do much more work than the others SELECT DISTINCT when there are very few distinct value A hash join with 1 or 2 values having most of the rows Using broadcast when it should use hash Usually happens when row estimate is much lower than actual Using hash when it should broadcast: Too few values, row distribution is skewed Usually, better/extended statistics help solve the problem

57 You As The Optimizer Determining the Distribution Method SELECT SUM(lo_quantity) FROM lineorder JOIN date_dim ON lo_orderdate = d_datekey WHERE d_year = 1997 AND d_weeknuminyear = 27 AND lo_tax = 0 ;

58 You As The Optimizer Determining the Distribution Method What information do you need to determine the distribution method? Size of the tables and rows retrieved Selectivity of join predicates How many rows do we expect to join from each table How many rows do we expect to retrieve, and thus distribute, from each table Skew in the data Hash join does not work well with data skew

59 You As The Optimizer Determining the Distribution Method Hash Distribution Id Operation Name Rows Bytes Cost (%CPU) Time TQ IN-OUT PQ Distrib SELECT STATEMENT K (1) 00:00:05 1 SORT AGGREGATE PX COORDINATOR 3 PX SEND QC (RANDOM) :TQ Q1,02 P->S QC (RAND) 4 SORT AGGREGATE 1 29 Q1,02 PCWP * 5 HASH JOIN K 104K (1) 00:00:05 Q1,02 PCWP 6 JOIN FILTER CREATE :BF (0) 00:00:01 Q1,02 PCWP 7 PX RECEIVE (0) 00:00:01 Q1,02 PCWP 8 PX SEND HASH :TQ (0) 00:00:01 Q1,00 P->P HASH 9 PX BLOCK ITERATOR (0) 00:00:01 Q1,00 PCWC * 10 TABLE ACCESS FULL DATE_DIM (0) 00:00:01 Q1,00 PCWP 11 PX RECEIVE 5998K 80M 104K (1) 00:00:05 Q1,02 PCWP 12 PX SEND HASH :TQ K 80M 104K (1) 00:00:05 Q1,01 P->P HASH 13 JOIN FILTER USE :BF K 80M 104K (1) 00:00:05 Q1,01 PCWP 14 PX BLOCK ITERATOR 5998K 80M 104K (1) 00:00:05 Q1,01 PCWC * 15 TABLE ACCESS FULL LINEORDER 5998K 80M 104K (1) 00:00:05 Q1,01 PCWP

60 You As The Optimizer Determining the Distribution Method Broadcast Distribution Id Operation Name Rows Bytes Cost (%CPU) Time TQ IN-OUT PQ Distrib SELECT STATEMENT K (1) 00:00:05 1 SORT AGGREGATE PX COORDINATOR 3 PX SEND QC (RANDOM) :TQ Q1,01 P->S QC (RAND) 4 SORT AGGREGATE 1 29 Q1,01 PCWP * 5 HASH JOIN K 104K (1) 00:00:05 Q1,01 PCWP 6 JOIN FILTER CREATE :BF (0) 00:00:01 Q1,01 PCWP 7 PX RECEIVE (0) 00:00:01 Q1,01 PCWP 8 PX SEND BROADCAST :TQ (0) 00:00:01 Q1,00 P->P BROADCAST 9 PX BLOCK ITERATOR (0) 00:00:01 Q1,00 PCWC * 10 TABLE ACCESS FULL DATE_DIM (0) 00:00:01 Q1,00 PCWP 11 JOIN FILTER USE :BF K 80M 104K (1) 00:00:05 Q1,01 PCWP 12 PX BLOCK ITERATOR 5998K 80M 104K (1) 00:00:05 Q1,01 PCWC * 13 TABLE ACCESS FULL LINEORDER 5998K 80M 104K (1) 00:00:05 Q1,01 PCWP

61 You As The Optimizer Determining the Distribution Method Small Table Replication Id Operation Name Rows Bytes Cost (%CPU) Time TQ IN-OUT PQ Distrib SELECT STATEMENT K (1) 00:00:05 1 SORT AGGREGATE PX COORDINATOR 3 PX SEND QC (RANDOM) :TQ Q1,00 P->S QC (RAND) 4 SORT AGGREGATE 1 29 Q1,00 PCWP * 5 HASH JOIN K 104K (1) 00:00:05 Q1,00 PCWP 6 JOIN FILTER CREATE :BF (0) 00:00:01 Q1,00 PCWP * 7 TABLE ACCESS FULL DATE_DIM (0) 00:00:01 Q1,00 PCWP 8 JOIN FILTER USE :BF K 80M 104K (1) 00:00:05 Q1,00 PCWP 9 PX BLOCK ITERATOR 5998K 80M 104K (1) 00:00:05 Q1,00 PCWC * 10 TABLE ACCESS FULL LINEORDER 5998K 80M 104K (1) 00:00:05 Q1,00 PCWP

62 You As The Optimizer Determining the Execution Plan The optimizer makes decisions for every step in the plan based on available information For more complex queries, the matrix of decisions becomes larger and the difficulty of producing accurate cardinality estimates increases There are lots of tools which can be used to increase performance but they are only effective if the optimizer chooses the correct plan Partitioning Compression Exadata Database In-Memory

63 The Optimizer Poor plans The vast majority of bad plans result from poor cardinality estimates Verification of cardinality estimates should always be the starting point for diagnosis Check for order of magnitude differences Consider whether gathering new statistics fits the scope of the problem Is there one problem query? Are there many problem queries? Not every poor cardinality estimate results in a bad plan

64 Agenda SQL and the Optimizer You As The Optimizer Optimization Strategies Why is my SQL slow? Optimizer Edges Cases Top SQL Mistakes

65 The Optimizer There are a number features to facilitate the optimizer choosing the correct plan To formulate the execution plan for a SQL statement, the optimizer uses The cost model defined by Oracle development Includes algorithms used in SQL execution Information about the data and SQL statements Such as statistics, dynamic sampling, SQL profiles

66 The Optimizer The optimizer has become increasingly more dynamic with each release We can classify the information the optimizer uses into the following categories Things to help get the right plan the first time Things to help get the right plan during execution Things to help get the right plan on subsequent executions Things to help get the right plan if the others fail

67 Optimizer Information Things to help get the right plan the first time Statistics Gathering Histograms Extended statistics Dynamic sampling Materialized views

68 System Statistics Three Options 1. Gather system statistics System statistics describe the system's hardware characteristics, such as I/O and CPU performance Recommended in the Upgrade Guide 2. Set Exadata statistics Exadata specific MOS Note Do not gather or set system statistics RWP Recommendation System statistics add another variable for the optimizer to consider, which can lead to suboptimal plans

69 Gathering Statistics Basics Use the DBMS_STATS procedure; do not use ANALYZE Use AUTO_SAMPLE_SIZE This uses a faster and more accurate NDV algorithm Allows the use of Incremental Statistics Allows the use of Concurrent Statistics Allows the use of new histogram types

70 Gathering Statistics Bulk Loads A new feature in 12c is to gather stats while loading data into a table Works in the following cases CREATE TABLE... AS SELECT INSERT INTO... SELECT into an empty table using a direct-path insert On by default and can be controlled with the GATHER_OPTIMIZER_STATISTICS hint

71 Gathering Statistics Incremental Stats Partitioned tables typically need both partition level and global statistics For queries that prune to a single partition, the partition level stats are used For queries that prune to more than one partition, the global level stats are used With incremental stats The statistics for a newly loaded partition are gathered and recorded in a synopsis Global level stats are computed from the synopsis 12c supports incremental stats on non-partitioned tables Creates synopsis on table to facilitate partition exchange

72 Gathering Statistics Incremental Stats Object Column Values NDV Partition #1 1,3,3,4,5 4 Partition #2 2,3,4,5,6 5

73 Gathering Statistics Incremental Stats Object Column Values NDV Partition #1 1,3,3,4,5 4 Partition #2 2,3,4,5,6 5 NDV by Addition WRONG 9

74 Gathering Statistics Incremental Stats Object Column Values NDV Partition #1 1,3,3,4,5 4 Partition #2 2,3,4,5,6 5 NDV by Addition WRONG 9 NDV by Synopsis CORRECT 6

75 Gathering Statistics Incremental Stats Comparison of Incremental vs non-incremental stats Simulates a daily load over 60 days 1,000,000 Rows Per Day 60 Partitions

76 Gathering Statistics Incremental Stats Begin DBMS_STATS.SET_TABLE_PREFS(USER, 'TABLE NAME', 'INCREMENTAL','TRUE'); End; /

77 Gathering Statistics Concurrent Stats By default, each partition of a partition table is gathered sequentially Concurrent stats enables partitions of a partitioned table to be gathered concurrently Enabled through job scheduler Stats jobs only run on one partitioned table at a time Full details:

78 Gathering Statistics Concurrent Stats Gather Schema Stats Job1 Table 1 Global Stats Job2 Table 2 Global Stats Job3 Table 3 Coord Job Job4 Table 4 Global Stats Job3.1 Table 3 Partition 1 Stats Job3.2 Table 3 Partition 2 Stats Job3.3 Table 3 Partition 3 Stats

79 Gathering Statistics Concurrent Stats Begin DBMS_STATS.SET_GLOBAL_PREFS('CONCURRENT','TRUE'); End; / Number of jobs limited by job_queue_processes Coordinate job_queue_processes and parallel setting on stats job

80 Gathering Statistics Histograms Designed to give more detail on data distribution Histograms are built based on SQL Usage Hence the need to seed the optimizer sys.col_usage$ keeps track of columns used in predicates and joins These columns are then candidates for histograms New in 12c Up to 2,048 buckets but default values still use 254 Uses a full table scan to create histograms with AUTO_SAMPLE_SIZE more accurate for unpopular values Types Frequency for columns with NDV <= 254 Top-frequency frequency histogram where NDV > 254 but majority of the rows <= 254 values Height Balanced no longer used with AUTO_SAMPLE_SIZE Replaced by Hybrid Still created with estimate_percent Remain after upgrades Hybrid for columns with NDV > 254 and lots of popular values

81 Gathering Statistics Extended Statistics Two Types of Extended Statistics Column Groups Provides the optimizer with information regarding the relationship (correlation) between the data stored in different columns of the same table City and airport code Month and zodiac sign Allows the optimizer to compute a better cardinality estimate when several the columns from the same table are use d together in a where clause of a SQL statement Expression Statistics Helps estimate the cardinality of a where clause predicate that has columns embedded inside expressions UPPER(EMP_LAST_NAME)=:B1

82 Gathering Statistics Extended Statistics Use the dbms_stats.create_extended_stats function to create extended statistics Select dbms_stats.create_extended_stats(user, AIRPORTS, (CITY,CODE) ) from dual; Select dbms_stats.create_extended_stats(user, EMP, (UPPER(EMP_LAST_NAME)) ) from dual;

83 Gathering Statistics Extended Statistics Restrictions Only used when the where the clause predicates are equalities or in-lists. Not used if there are histograms present on the underlying columns and there is no histogram present on the column group.

84 Dynamic Statistics Previously called dynamic sampling Statistics gathered at parse time to augment the existing statistics Used when regular statistics are not sufficient to get good quality cardinality estimates. Table level stats prior to 12 enhanced in 12c to include joins and group-by predicates Controlled by the OPTIMIZER_DYNAMIC_SAMPLING parameter Dynamic statistics can greatly enhance the quality of statistics and lead to better execution plans But there is a cost better statistics are usually derived with larger sample sizes leading to increased parse times

85 Materialized Views Stores the results of a query Can be used to quickly get the results of common and long running queries Periodically refreshed Can be used as a technique to aid the optimizer obtain a better cardinality estimate for related tables A snowflake schema contains related dimensions that are usually static Snowflake schemas are challenging for the optimizer to get an accurate cardinality estimate A materialized view on the related tables allows the optimizer to use statics on the materialized view to obtain better cardinality estimates

86 Gathering Statistics Strategy for initial stats gathering 1. Create tables 2. If using Auto Column Group Detection Gather stats on empty table Run dbms_stats.seed_col_usage 3. Run (or explain) queries on empty tables Prime / Seed the optimizer for histogram creation 4. Create column groups Dbms_stats.report_col_usage Dbms_stats.create_extended_stats 5. Enable incremental statistics For large partitioned tables 6. Load data 7. Gather statistics Use defaults 8. Create indexes (if required!) 2/14/2016

87 Optimizer Information Things to help get the right plan during execution Adaptive Plans Enable the optimizer to defer the final plan decision for a statement until execution Uses statistics collectors to compare the actual cardinality with the estimate If there is a significant difference, then the plan or a portion of it can be automatically adapted to avoid suboptimal performance Adaptive Join Methods Optimizer can switch between nested loops and hash joins Adaptive Parallel Distribution Methods Optimizer can switch between broadcast and hash distribution methods Displays as Hybrid Hash

88 Optimizer Information Things to help get the right plan on subsequent executions Statistics Feedback (aka Cardinality Feedback) Performance Feedback SQL Plan Directives

89 Optimizer Information Things to help get the right plan if the others fail Hints SQL Plan Management SQL Profiles

90 Optimizer Information Things not to do Change the optimizer_* parameters from their default values Underscore parameters to enable or disable features or behavior Use the DoP as a query tuning technique

91 Agenda SQL and the Optimizer You As The Optimizer Optimization Strategies Why is my SQL slow? Optimizer Edges Cases Top SQL Mistakes

92 Why is My SQL Slow?

93 Problem Query Table has 1B rows and is 55 GB.

94 Problem Query Query 1 consists of two subqueries. The first subquery finds all of the Ferraris.

95 Problem Query The second subquery finds all of the Ferrari 458s.

96 Problem Query Outer query performs aggregations. Outer query joins the results of the subqueries.

97 Problem Query Query 2 is the same but has different predicate values.

98 Default Statistics

99 Default Statistics Query 1 takes 40 seconds with default statistics

100 Default Statistics

101 Default Statistics Query 2 takes 3 seconds with default statistics

102 Default Statistics

103 Default Statistics Development Findings Baseline Performance for Query 1 exceeds target Baseline Performance for Query 2 meets target

104 Initial Optimization Steps More Predicate Values

105 More Predicate Values Increase the list of predicate values Now query 1 takes 2 seconds

106 More Predicate Values

107 More Predicate Values Development Findings Query runs faster just by changing the list of values in the select list Plan changed from a broadcast to a hash distribution due to the higher but inaccurate cardinality estimate Get correct plan with wrong cardinality estimate can lead to inconsistent plans and performance

108 Initial Optimization Steps Increase Degree of Parallelism

109 Degree of Parallelism Change DoP from 32 to 128 Now query takes 2 seconds

110 Degree of Parallelism

111 Degree of Parallelism Development Findings Changing DoP from 32 to 128 improves performance and meets the target; 4X more resources yields a 20X performance improvement Plan has changed from a broadcast distribution to a hash distribution due to DoP change DoP is a resource management technique, not a query tuning tool

112 Indexes

113 Indexes Development Findings Indexes Indexes on columns: owner_id country make model country, make, model

114 Indexes Add indexes and query takes longer 58 seconds!

115 Indexes Query performance varies depending on whether the index is cached or not Index lookups on millions of rows is slow

116 Indexes Development Findings Indexes Not understanding the big/little data challenge Indexes are not efficient for operations on a large numbers of rows Full table scan is faster with predictable performance

117 To Index or Not Indexing is an OLTP technique for operations on a small number of rows A table scan may consume more resources but it will be predictable no matter how many rows are returned Indexes impact DML operations If I/O bandwidth went from 70MB/sec to 70GB/sec would you change your optimization/execution strategy?

118 To Index or Not Index driven query retrieving 1,000,000 rows Assume the index is cached and the data is not. 1,000,000 random 5ms per I/O This would require 5000 Seconds ( or over 1 hour ) to Execute How much data could you scan in 5000 Seconds with a fully sized I/O system able to scan 25 GB/Sec? Over 100 TB!

119 Histograms

120 Histograms Rerun stats to get histograms no change in plan or run time

121 Histograms Lots of wait time on temp IO

122 Histograms Development Findings Re-gathered stats to automatically create histograms Frequency histograms on country, make and model columns No change in plan query still exceeds target

123 Flash Temp

124 Flash Temp Query time reduced to 23 seconds

125 Flash Temp Now IO accounts for a smaller percentage of database time

126 Flash Temp Development Findings Most of the wait time was spent performing IO on temp, so move temp to flash disks Improved performance but still does not meet target Not a good use of flash Incorrect use of tools/products

127 Manual Memory Parameters

128 Manual Memory Parameters

129 Manual Memory Parameters Very little IO in database time Poor cardinality estimate 256K estimated rows vs 40M actual rows Increased memory size manually now there is no use of temp

130 Manual Memory Parameters Development Findings Set sort_area_size and hash_area_size to 2G Eliminated temp usage but still did not meet target Memory is allocated per parallel server process, which can quickly exceed resources Moving to a solution before understanding the problem

131 Cardinality Estimates

132 Cardinality Estimates

133 Cardinality Estimates Plan switches from a broadcast to a hash distribution Use cardinality hint to specify correct number of rows

134 Cardinality Estimates Cardinality Hint SQL Monitor showed poor cardinality estimates Cardinality hint gives optimizer the correct number of rows for the table scan Plan changed from a broadcast to hash distribution Query time now meets target Now temp is not an issue

135 Disable Broadcast Distribution

136 Disable Broadcast Distribution

137 Disable Broadcast Distribution Disable broadcast distribution and now we have the hash distribution as with the cardinality hint

138 Disable Broadcast Distribution Development Findings Google reveals a hidden parameter to disable broadcast distribution Plan and run times are similar to cardinality hint, meeting target Moving to a solution before understanding the problem

139 Second Query with Broadcast Distribution Disabled

140 Query 2: Broadcast Distribution Disabled

141 Query 2: Broadcast Distribution Disabled Query 2 also uses a hash distribution but no longer meets the target

142 Query 2: Broadcast Distribution Disabled Development Findings Plan uses a hash distribution Exceeds target

143 Second Query with Broadcast Distribution Enabled

144 Query 2: Broadcast Distribution Enabled

145 Query 2: Broadcast Distribution Enabled Reset parameter to enable broadcast distribution now query 2 uses a broadcast and meets the target

146 Query 2: Broadcast Distribution Enabled Development Findings Reset _parallel_broadcast_enabled Plan now uses a broadcast distribution Meets target Should not change system parameters to tune one query

147 Extended Stats

148 Extended Stats

149 Extended Stats Created column group but still have a poor cardinality estimate

150 Extended Stats Development Findings High correlation between Country, Make and Model columns Created column group Query still exceeds target Still have poor cardinality estimate

151 Histogram on Column Group

152 Histogram on Column Group

153 Histogram on Column Group With a histogram on the column group we now have a good cardinality estimate Now we get a hash distribution and meet the target

154 Histogram on Column Group Development Findings Re-gathered stats after running the query with the column groups Frequency Histogram on the column group Accurate cardinality estimates Optimizer now uses a hash distribution

155 Second Query with Histogram on Column Group

156 Query 2: Histogram Column Group

157 Query 2: Histogram Column Group Query 2 also has a good cardinality estimate And uses a broadcast distribution

158 Query 2: Histogram Column Group Development Findings Accurate cardinality estimates Optimizer uses a broadcast distribution on second query

159 Histogram on Column Groups Now we have the correct solution! Both queries have good cardinality estimates Correct plans Meet targets

160 Auto Column Group Creation: Seed Column Usage

161 Auto Column Group Creation

162 Auto Column Group Creation Back to the default stats while seeding column usage poor cardinality estimate as seen earlier and a broadcast distribution for query 1

163 Auto Column Group Creation: Seed Column Usage Development Findings Start with default statistics Execute dbms_stats.seed_col_usage to monitor column usage Run query

164 Auto Column Group Creation: Create Extended Stats

165 Auto Column Group Creation

166 Auto Column Group Creation With the column group identified and created, we have a good cardinality estimate And we get a hash distribution

167 Auto Column Group Creation: Create Extended Stats Development Findings dbms_stats.report_col_usage shows column groups identified during Seed Column Usage dbms_stats.create_extended_stats creates column groups identified Automatically identifies usage of Country, Make and Model columns together and creates column group

168 Auto Column Group Creation: Create Extended Stats Development Findings Regather stats Automatically creates Histogram on the column group Query meets target

169 What Did We Learn?

170 Root Causes of Suboptimal Database Performance Using DoP for query tuning Indexes for large data sets Temp on flash Forcing use of more memory Disable broadcast distribution 2/14/2016

171 Agenda SQL and the Optimizer You As The Optimizer Optimization Strategies Why is my SQL slow? Optimizer Edges Cases Top SQL Mistakes

172 Optimizer Edge Cases Always stale statistics Arbitrary high and low values Correlation Functions

173 Optimizer Edge Cases Always stale statistics Scenario: Data is loaded into a partitioned table as it is received Statistics are gathered by nightly maintenance process Resulting stats indicate 0 rows in partition Poor cardinality estimates and plans Resolution: Copy statistics from the previous partition, or set statistics to appropriate values Lock statistics to prevent updates by maintenance job Optimizer is able to use representative stats to get more accurate cardinality estimates Optional: Unlock and gather new statistics after the partition is fully populated

174 Optimizer Edge Cases Always stale statistics TABLE_NAME PARTITION_NAME INT GLOBAL_STATS NUM_ROWS SAMPLE_SIZE LAST_ANALYZED LINEORDER_STALE R1992 NO YES OCT-14 LINEORDER_STALE R1993 NO YES OCT-14 LINEORDER_STALE R1994 NO YES OCT-14 LINEORDER_STALE R1995 NO YES OCT-14 LINEORDER_STALE R1996 NO YES OCT-14 LINEORDER_STALE R1997 NO YES OCT-14 LINEORDER_STALE R1998 NO YES 0 24-OCT-14 7 rows selected.

175 Optimizer Edge Cases Always stale statistics

176 Optimizer Edge Cases Representative statistics TABLE_NAME PARTITION_NAME INT GLOBAL_STATS NUM_ROWS SAMPLE_SIZE LAST_ANALYZED LINEORDER_YR R1992 NO YES OCT-14 LINEORDER_YR R1993 NO YES OCT-14 LINEORDER_YR R1994 NO YES OCT-14 LINEORDER_YR R1995 NO YES OCT-14 LINEORDER_YR R1996 NO YES OCT-14 LINEORDER_YR R1997 NO YES OCT-14 LINEORDER_YR R1998 NO YES OCT-14 7 rows selected.

177 Optimizer Edge Cases Representative statistics

178 Optimizer Edge Cases Arbitrary High and Low Values C1 C2 1 date' ' 2 date' ' 3 date' ' 4 date' ' 5 date' ' 6 date' '

179 Optimizer Edge Cases With Arbitrary High Value for Unknown Dates CREATE TABLE ( LINEORDER "LO_ORDERKEY","LO_LINENUMBER","LO_CUSTKEY","LO_PARTKEY","LO_SUPPKEY" NUMBER NOT NULL ENABLE NUMBER NUMBER NOT NULL ENABLE NUMBER NOT NULL ENABLE NUMBER NOT NULL ENABLE,"LO_ORDERDATE" DATE NOT NULL ENABLE...,"LO_CLOSEDATE" DATE DEFAULT '31-DEC-2999' ) ;

180 Optimizer Edge Cases With Arbitrary High Value for Unknown Dates

181 Optimizer Edge Cases With Nulls to Represent Unknown Dates CREATE TABLE ( LINEORDER "LO_ORDERKEY","LO_LINENUMBER","LO_CUSTKEY","LO_PARTKEY","LO_SUPPKEY" NUMBER NOT NULL ENABLE NUMBER NUMBER NOT NULL ENABLE NUMBER NOT NULL ENABLE NUMBER NOT NULL ENABLE,"LO_ORDERDATE" DATE NOT NULL ENABLE...,"LO_CLOSEDATE" ) ; DATE

182 Optimizer Edge Cases With Nulls to Represent Unknown Dates

183 Optimizer Edge Cases Correlation When different columns in a given table have values that are correlated Make and model of a car, a phone,. Month and zodiac sign City and airport Correlation causes an under-estimate in the cardinality because predicates are seen as independent by the Optimizer

184 Optimizer Edge Cases Correlation

185 Optimizer Edge Cases Functions and Wildcards Functions and wildcards make it very difficult to obtain good cardinality estimates In many cases the optimizer will simply guess at a cardinality estimate based upon 1% or 5% of the rows in a table Techniques such as dynamic sampling and extended statistics should be evaluated to improve the cardinality estimates.

186 Optimizer Edge Cases Rounding Function SELECT d_sellingseason, p_category, s_region FROM lineorder JOIN customer ON lo_custkey JOIN date_dim ON lo_orderdate = c_custkey = d_datekey JOIN part ON lo_partkey = p_partkey JOIN supplier ON lo_suppkey = s_suppkey WHERE lo_orderdate between to_date('15-jun-1993','dd-mon-yyyy') and to_date('15-jun-1995','dd-mon-yyyy') AND d_monthnuminyear in (12, 1) AND p_container AND p_color in ('JUMBO PACK') in ('red') AND round(lo_extendedprice) > ORDER BY d_sellingseason, p_category, s_region

187 Optimizer Edge Cases Rounding Function

188 Optimizer Edge Cases Substring Function SELECT d_sellingseason, p_category, s_region FROM lineorder JOIN customer ON lo_custkey = c_custkey JOIN date_dim ON lo_orderdate = d_datekey JOIN part ON lo_partkey = p_partkey JOIN supplier ON lo_suppkey = s_suppkey WHERE lo_orderdate between to_date('15-jun-1993','dd-mon-yyyy') and to_date('15-jun-1995','dd-mon-yyyy') AND d_monthnuminyear in (12, 1) AND SUBSTR(p_container,-4,4) in ('DRUM') AND p_color in ('red') ORDER BY d_sellingseason, p_category, s_region

189 Optimizer Edge Cases Substring Function

190 Optimizer Edge Cases Using Extended Statistics for Substring Function BEGIN DBMS_STATS.gather_table_stats( ownname => DW', tabname => PART', method_opt => 'for all columns size skewonly for columns (SUBSTR(P_CONTAINER,-4,4)) ); END; / PL/SQL procedure successfully completed. select extension_name, extension from user_stat_extensions where table_name='part'; EXTENSION_NAME EXTENSION SYS_STUW_AMVY8N$KU59A 847#Z7P (SUBSTR("P_CONTAINER",(-4),4))

191 Optimizer Edge Cases Using Extended Statistics for Substring Function

192 Agenda SQL and the Optimizer You As The Optimizer Optimization Strategies Why is my SQL slow? Optimizer Edges Cases Top SQL Mistakes

193 Top SQL Mistakes Missing Joins Implicit or Wrong Data Type Conversions More Top SQL Mistakes shown in the Reference Materials

194 Top SQL Mistakes Missing Joins There should be n-1 join conditions in a query, where n is the number of tables in the query block, otherwise Cartesian products will occur Often a problem with programmers new to SQL developing/testing on small datasets where a DISTINCT is used to reduce the rows This is often not seen as a performance problem until the datasets become large! If the query aggregates the data, the total number of rows returned would be unchanged, but the values are likely to be incorrect

195 Top SQL Mistakes Missing Join: 5 tables, 3 joins SELECT d_sellingseason, p_category, s_region, sum(lo_extendedprice) FROM lineorder, customer, date_dim, part, supplier WHERE -- lo_custkey lo_orderdate AND lo_partkey AND lo_suppkey = c_custkey AND = d_datekey = p_partkey = s_suppkey AND d_year in (1993, 1994, 1995) AND d_monthnuminyear in (12, 1) AND p_container AND p_color in ('JUMBO PACK') in ('red') GROUP BY d_sellingseason, p_category, s_region ORDER BY d_sellingseason, p_category, s_region

196 Top SQL Mistakes Missing Join: 5 tables, 3 joins

197 Top SQL Mistakes With All Joins: 5 tables, 4 joins SELECT d_sellingseason, p_category, s_region, sum(lo_extendedprice) FROM lineorder, customer, date_dim, part, supplier WHERE lo_custkey AND lo_orderdate AND lo_partkey AND lo_suppkey = c_custkey = d_datekey = p_partkey = s_suppkey AND d_year in (1993, 1994, 1995) AND d_monthnuminyear in (12, 1) AND p_container AND p_color in ('JUMBO PACK') in ('red') GROUP BY d_sellingseason, p_category, s_region ORDER BY d_sellingseason, p_category, s_region

198 Top SQL Mistakes With All Joins: 5 tables, 4 joins

199 Top SQL Mistakes Implicit or Wrong Data Type Conversions SQL does not enforce much data type checking on SQL statements Where there are data type mismatches SQL will automatically cast/convert data into the appropriate type to execute the SQL statement This may result in the following effects Increased resource usage converting data types Poor execution plans avoiding indexes and partition pruning Failed SQL statements as data values may not convert correctly

200 Top SQL Mistakes Implicit or Wrong Data Type Conversions Common in columns that contain numeric data but are never used for arithmetic operations telephone numbers credit card numbers check numbers When a programmer references these columns care must be made to ensure the bind variables are type VARCHAR2 and not numbers

201 Top SQL Mistakes Implicit or Wrong Data Type Conversions SELECT d_sellingseason, p_category, s_region, sum(lo_extendedprice) FROM lineorder JOIN customer ON lo_custkey = c_custkey JOIN date_dim ON lo_orderdate = d_datekey JOIN part ON lo_partkey = p_partkey JOIN supplier ON lo_suppkey = s_suppkey WHERE d_year AND p_container in ('1993', '1994', '1995') in ('JUMBO PACK') GROUP BY d_sellingseason, p_category, s_region ORDER BY d_sellingseason, p_category, s_region

202 Top SQL Mistakes Implicit or Wrong Data Type Conversions SQL> desc date_dim Name Null? Type D_DATEKEY D_DATE D_DAYOFWEEK D_MONTH D_YEAR D_YEARMONTHNUM D_YEARMONTH D_DAYNUMINWEEK D_DAYNUMINMONTH... NOT NULL DATE VARCHAR2(18) VARCHAR2(10) VARCHAR2(9) NUMBER NUMBER VARCHAR2(7) NUMBER NUMBER

203 Top SQL Mistakes No Data Type Conversion SELECT d_sellingseason, p_category, s_region, sum(lo_extendedprice) FROM lineorder JOIN customer ON lo_custkey = c_custkey JOIN date_dim ON lo_orderdate = d_datekey JOIN part ON lo_partkey = p_partkey JOIN supplier ON lo_suppkey = s_suppkey WHERE d_year in (1993, 1994, 1995) AND p_container in ('JUMBO PACK') GROUP BY d_sellingseason, p_category, s_region ORDER BY d_sellingseason, p_category, s_region

204 Top SQL Mistakes Implicit or Wrong Data Type Conversions SELECT d_sellingseason, p_category, s_region, sum(lo_extendedprice) FROM lineorder JOIN customer ON lo_custkey = c_custkey JOIN date_dim ON lo_orderdate = d_datekey JOIN part ON lo_partkey = p_partkey JOIN supplier ON lo_suppkey = s_suppkey WHERE d_year in (1993, 1994, 1995) AND c_phone = GROUP BY d_sellingseason, p_category, s_region ORDER BY d_sellingseason, p_category, s_region This SQL statement is vulnerable to: Bad plans because index cannot be used 1722 errors if there is non numerical characters Increased resource usage converting data Poor cardinality estimates

205 Top SQL Mistakes Implicit or Wrong Data Type Conversions SQL> desc customer Name Null? Type C_CUSTKEY C_NAME C_ADDRESS C_CITY C_NATION C_REGION C_PHONE C_MKTSEGMENT NOT NULL NUMBER VARCHAR2(25) VARCHAR2(25) VARCHAR2(10) VARCHAR2(15) VARCHAR2(12) VARCHAR2(15) VARCHAR2(10)

206 Top SQL Mistakes Implicit or Wrong Data Type Conversions SELECT d_sellingseason, * ERROR at line 1: ORA-12801: error signaled in parallel query server P00E, instance scao08adm01.us.oracle.com:imtst1 (1) ORA-01722: invalid number

207 Top SQL Mistakes No Data Type Conversion SELECT d_sellingseason, p_category, s_region, sum(lo_extendedprice) FROM lineorder JOIN customer ON lo_custkey = c_custkey JOIN date_dim ON lo_orderdate = d_datekey JOIN part ON lo_partkey = p_partkey JOIN supplier ON lo_suppkey = s_suppkey WHERE d_year in (1993, 1994, 1995) AND c_phone = GROUP BY d_sellingseason, p_category, s_region ORDER BY d_sellingseason, p_category, s_region

208 Top SQL Mistakes No Data Type Conversion

209 Top SQL Mistakes Implicit or Wrong Data Type Conversions SELECT /*+ MONITOR */ ; p_category, s_region, sum(lo_extendedprice) FROM lineorder_mon JOIN part ON lo_partkey = p_partkey JOIN supplier ON lo_suppkey = s_suppkey WHERE lo_orderdate between to_timestamp('15-jun-1993','dd-mon-yyyy') AND p_container GROUP BY p_category, s_region ORDER BY p_category, s_region and to_timestamp('15-jun-1995','dd-mon-yyyy') in ('JUMBO PACK') This SQL statement is vulnerable to: Bad plans because partition pruning may not take place Poor cardinality estimates

210 Top SQL Mistakes Implicit or Wrong Data Type Conversions SQL> desc lineorder Name Null? Type LO_ORDERKEY LO_LINENUMBER LO_CUSTKEY LO_PARTKEY LO_SUPPKEY LO_ORDERDATE LO_ORDERPRIORITY LO_SHIPPRIORITY LO_QUANTITY LO_EXTENDEDPRICE LO_ORDTOTALPRICE... NOT NULL NUMBER NUMBER NOT NULL NUMBER NOT NULL NUMBER NOT NULL NUMBER NOT NULL DATE VARCHAR2(15) VARCHAR2(1) NUMBER NUMBER NUMBER

211 Top SQL Mistakes Implicit or Wrong Data Type Conversions

212 Top SQL Mistakes Implicit or Wrong Data Type Conversions

213 Top SQL Mistakes Correct Data Type Conversion SELECT /*+ MONITOR */ p_category, s_region, sum(lo_extendedprice) FROM lineorder_mon JOIN part ON lo_partkey = p_partkey JOIN supplier ON lo_suppkey = s_suppkey WHERE lo_orderdate between to_date('15-jun-1993','dd-mon-yyyy') GROUP BY p_category, s_region ORDER BY p_category, s_region and to_date('15-jun-1995','dd-mon-yyyy')

214 Top SQL Mistakes Correct Data Type Conversion

215 Top SQL Mistakes Correct Data Type Conversion

216 Reference Section

217 Tools for SQL Statement Analysis

218 Tools For SQL Analysis Static Analysis Dynamic Analysis Tracing and Debugging

219 Tools for SQL Analysis Static Analysis Explain Plan and dbms_xplan.display SQL*Plus Autotrace

220 Static Analysis Explain Plan Overview Provides an indication of the possible execution plan DDL e.g. CREATE TABLE AS SELECT DML Queries Strengths Available in every installation Does not require execution of the SQL statement

221 Static Analysis Explain Plan Weaknesses Bind variables All binds treated as VARCHAR No bind peeking Beware of PLAN_TABLE inherited from previous releases Based on expectations rather than reality

222 Static Analysis Explain Plan Predicate Information Filters Transformations Candidates for Offload Bloom Filters Notes Section Automatic Degree of Parallelism (aka Auto DoP) Rationale for choice of DoP Dynamic Sampling When no statistics are available When the optimizer chooses to use dynamic sampling in the evaluation of parallel queries Cardinality Feedback

223 Static Analysis Explain Plan needs to be formatted Plan hash value: Id Operation Name Rows Bytes Cost (%CPU) Time SELECT STATEMENT (0) 00:00:01 1 TABLE ACCESS BY INDEX ROWID EMP (0) 00:00:01 * 2 INDEX RANGE SCAN EMP_N1 3 1 (0) 00:00: Predicate Information (identified by operation id): access("deptno"=10)

224 Static Analysis Explain Plan Formatting SET TAB OFF SET TRIMSPOOL ON Use fixed width font Preserve spaces Avoid truncating or wrapping long lines

225 Static Analysis Explain Plan Plan hash value: Id Operation Name Rows Bytes Cost (%CPU) Time SELECT STATEMENT (0) 00:00:01 1 TABLE ACCESS BY INDEX ROWID EMP (0) 00:00:01 * 2 INDEX RANGE SCAN EMP_N1 3 1 (0) 00:00: Predicate Information (identified by operation id): access("deptno"=10)

226 Static Analysis Explain Plan More Background Explain the Explain Plan Written by Maria Colgan

227 Static Analysis SQL*Plus Autotrace SQL> set linesize 132 tab off SQL> set autotrace traceonly explain SQL> SELECT * FROM EMP WHERE DEPTNO = 10; Execution Plan Plan hash value: Id Operation Name Rows Bytes Cost (%CPU) Time SELECT STATEMENT (0) 00:00:01 1 TABLE ACCESS BY INDEX ROWID EMP (0) 00:00:01 * 2 INDEX RANGE SCAN EMP_N1 3 1 (0) 00:00: Predicate Information (identified by operation id): access("deptno"=10)

228 Static Analysis SQL*Plus Autotrace SQL> set autotrace traceonly explain statistics SQL> SELECT * FROM EMP WHERE DEPTNO = 10; Execution Plan Statistics recursive calls 0 db block gets 4 consistent gets 0 physical reads 0 redo size 1159 bytes sent via SQL*Net to client 525 bytes received via SQL*Net from client 2 SQL*Net roundtrips to/from client 0 sorts (memory) 0 sorts (disk) 3 rows processed

229 Static Analysis SQL*Plus Autotrace Overview Provides an indication of the possible execution plan DML Queries Strengths Built in functionality with some useful shortcuts May not require execution of queries Weaknesses Much the same as Explain Plan

230 Tools for SQL Analysis Dynamic Analysis dbms_xplan.display_* SQL Monitor

231 Dynamic Analysis DBMS_XPLAN.DISPLAY_* Overview Gives the actual execution plan from various sources Cursor cache dbms_xplan.display_cursor AWR dbms_xplan.display_awr SQL Tuning Set dbms_xplan.display_sqlset

232 Dynamic Analysis dbms_xplan.display_cursor() SQL> set feedback off linesize 132 pagesize 0 tab off SQL> SELECT * FROM EMP WHERE DEPTNO = :n; SQL> select * from table(dbms_xplan.display_cursor()); SQL_ID 5xt3urx81f9th, child number SELECT * FROM EMP WHERE DEPTNO = :n Plan hash value: Id Operation Name Rows Bytes Cost (%CPU) Time SELECT STATEMENT 2 (100) 1 TABLE ACCESS BY INDEX ROWID EMP (0) 00:00:01 * 2 INDEX RANGE SCAN EMP_N1 3 1 (0) 00:00: Predicate Information (identified by operation id): access("deptno"=:n)

233 Dynamic Analysis dbms_xplan.display_cursor() Various formatting options Default is usually sufficient ALLSTATS LAST can be useful when using the GATHER_PLAN_STATISTICS hint to gather execution statistics Beware of overhead Ignore the following artifact in parallel queries storage(:z>=:z AND :Z<=:Z)

234 Dynamic Analysis SQL Monitor Overview Feature of SQL Tuning Pack Generation Oracle Enterprise Manager dbms_sqltune.report_sql_monitor Text, HTML or Active Active is preferred

235 Dynamic Analysis SQL Monitor set trimspool on set trim on set pagesize 0 set linesize 1000 set long set longchunksize spool sqlmon_previous.html select dbms_sqltune.report_sql_monitor( session_id=>sys_context('userenv','sid'), report_level=>'all', type=>'active') from dual; spool off

236 Dynamic Analysis SQL Monitor By default SQL Monitor is limited to 300 plan lines. If the report exceeds 300 lines, it will not show up in v$sql_monitor or the EM SQL Monitoring Page. This value can be increased using the_sqlmon_max_planlines parameter alter session set _sqlmon_max_planlines =500; 300 is usually sufficient and should only be changed if needed

237 Dynamic Analysis SQL Monitor

238 Performance Diagnosis and Tuning EM Access to Reports SQL Monitor can be accesses by navigating Performance Tab SQL Monitoring Link Select SQL Statement of Interest to drill down Show recent SQL statements running for more than 5 seconds. SQL statement details can quickly highlight issues such as skew in PQ execution non-representative statistics (large difference between estimated rows and actual rows)

239 EM Monitor Report Select SQL Monitoring

240 EM Monitor Report Select SQL of Interest

241 EM Monitor Report

242 Tools for SQL Analysis Tracing Active Session History Discussed in OLTP session

243 Tracing Overview Tracing of calls and optionally wait events and bind values Raw tracefile can be useful to identify anomalies Generally post-process using tkprof (trace kernel profile) Strengths Difficult to get some of the information any other way Weaknesses Tracing needs to be enabled Multiple tracefiles generated for parallel execution, making almost useless Runtime and space overhead

244 Tracing Embed an identifier in the name of the tracefile SQL> alter session set tracefile_identifier = my_10046_trace; Level 1 is equivalent to SQL_TRACE = TRUE SQL> alter session set events = '10046 trace name context forever, level 1';

245 Tracing Use dbms_monitor to set it Easy way to enable/disable tracing in other sessions SQL> execute dbms_monitor.session_trace_enable() SQL> execute dbms_monitor.session_trace_disable()

246 Tracing Trace Level Functionality Off (specified in place of forever ) Disable Tracing 1 Equivalent to SQL_TRACE=TRUE 4 Include Bind Values 8 Include Wait Events 12 Include Bind Values and Wait Events

247 Tracing select * from emp where deptno = :n call count cpu elapsed disk query current rows Parse Execute Fetch total Misses in library cache during parse: 1 Optimizer mode: ALL_ROWS Parsing user id: 88 (TEACHER) Number of plan statistics captured: 2 Rows (1st) Rows (avg) Rows (max) Row Source Operation TABLE ACCESS BY INDEX ROWID EMP (cr=4 pr=0 pw=0 time=152 us cost=2 size=114 card=3) INDEX RANGE SCAN EMP_N1 (cr=2 pr=0 pw=0 time=120 us cost=1 size=0 card=3)(object id 64363)

248 Tracing Rows Execution Plan SELECT STATEMENT MODE: ALL_ROWS 3 TABLE ACCESS MODE: ANALYZED (BY INDEX ROWID) OF 'EMP' (TABLE) 3 INDEX MODE: ANALYZED (RANGE SCAN) OF 'EMP_N1' (INDEX) Elapsed times include waiting on following events: Event waited on Times Max. Wait Total Waited Waited Disk file operations I/O SQL*Net message to client SQL*Net message from client ********************************************************************************

249 Tools for SQL Analysis Debugging Superseded by SQL_COMPILER trace SQL Test Case Builder SQLTXPLAIN my.oracle.support Doc ID

250 Debugging SQL_COMPILER Embed an identifier in the name of the tracefile: SQL> alter session set tracefile_identifier = SQLCOMP_trace; For all SQL statements: SQL> alter session set events = 'trace [SQL_COMPILER.*]'; For a specific SQL Identifier: SQL> alter session set events = 'trace [sql_compiler] [SQL:7h35uxf5uhmm1]';

251 Debugging SQL Test Case Builder Overview Export the execution environment for a SQL statement Generation Oracle Enterprise Manager dbms_sqldiag.export_sql_testcase dbms_sqldiag.import_sql_testcase TIP: Avoid the need to quote quotes, for example: SQL> variable i clob SQL> exec :i := q'#select * from emp where ename = 'KING'#'

252 Debugging SQL Test Case Builder variable i CLOB variable o CLOB BEGIN :i := q'#select * from emp where deptno = 10#'; END; / BEGIN dbms_sqldiag.export_sql_testcase ( directory => 'TEACHER_TCB_EXPORT',sql_text => :i,testcase => :o ); END; /

253 Debugging SQL Test Case Builder BEGIN dbms_sqldiag.import_sql_testcase ( directory => 'TEACHER_TCB_IMPORT',filename => 'oratcb1_171f07e50002main.xml' ); END; /

254 More Top SQL Mistakes 2/14/2016

255 More Top SQL Mistakes Parse Errors and Incorrect SQL Sending SQL that is syntactically incorrect is very expensive to the database and very historically difficult to diagnose exception handling/throwing errors is expensive Bad SQL leaves no trace within the shared pool because it is bad and hence unshareable Statistic parse count(failures) indicates it may be happening The danger is clever programmer that sends bad SQL to see if it parsed to determine version or other information

256 More Top SQL Mistakes Parse Errors and Incorrect SQL SQL> select pk_id from history_version_2 where pk_id is NULL; select pk_id from history_version_2 where pk_id is NULL; * ERROR at line 1: ORA-00942: table or view does not exist

257 ANSI Outer Join

Real-World Performance Training SQL Performance

Real-World Performance Training SQL Performance Real-World Performance Training SQL Performance Real-World Performance Team Agenda 1 2 3 4 5 6 The Optimizer Optimizer Inputs Optimizer Output Advanced Optimizer Behavior Why is my SQL slow? Optimizer

More information

Real-World Performance Training Star Query Prescription

Real-World Performance Training Star Query Prescription Real-World Performance Training Star Query Prescription Real-World Performance Team Dimensional Queries 1 2 3 4 The Dimensional Model and Star Queries Star Query Execution Star Query Prescription Edge

More information

Real-World Performance Training Dimensional Queries

Real-World Performance Training Dimensional Queries Real-World Performance Training al Queries Real-World Performance Team Agenda 1 2 3 4 5 The DW/BI Death Spiral Parallel Execution Loading Data Exadata and Database In-Memory al Queries al Queries 1 2 3

More information

Real-World Performance Training Star Query Edge Conditions and Extreme Performance

Real-World Performance Training Star Query Edge Conditions and Extreme Performance Real-World Performance Training Star Query Edge Conditions and Extreme Performance Real-World Performance Team Dimensional Queries 1 2 3 4 The Dimensional Model and Star Queries Star Query Execution Star

More information

Oracle Optimizer: What s New in Oracle Database 12c? Maria Colgan Master Product Manager

Oracle Optimizer: What s New in Oracle Database 12c? Maria Colgan Master Product Manager Oracle Optimizer: What s New in Oracle Database 12c? Maria Colgan Master Product Manager PART 3 2 Program Agenda Adaptive Query Optimization Statistics Enhancements What s new in SQL Plan Management 3

More information

1 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

1 Copyright 2011, Oracle and/or its affiliates. All rights reserved. 1 Copyright 2011, Oracle and/or its affiliates. All rights 2 Copyright 2011, Oracle and/or its affiliates. All rights Optimizer Statistics CPU & IO DATA DICTIONARY OPTIMIZER STATISTICS Index Table Column

More information

Top 10 Features in Oracle 12C for Developers and DBA s Gary Bhandarkar Merck & Co., Inc., Rahway, NJ USA

Top 10 Features in Oracle 12C for Developers and DBA s Gary Bhandarkar Merck & Co., Inc., Rahway, NJ USA Top 10 Features in Oracle 12C for Developers and DBA s Gary Bhandarkar Merck & Co., Inc., Rahway, NJ USA Agenda Background ORACLE 12c FEATURES CONCLUSION 2 Top 10 Oracle 12c Features Feature 1: ADAPTIVE

More information

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Copyright 2012, Oracle and/or its affiliates. All rights reserved. 1 Oracle safe harbor statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not

More information

Oracle Database 11gR2 Optimizer Insights

Oracle Database 11gR2 Optimizer Insights Oracle Database 11gR2 Optimizer Insights Marcus Bender Distinguished Sales Consultant Presales Fellow Strategic Technical Support (STU) ORACLE Deutschland GmbH, Geschäftsstelle Hamburg Parallel Execution

More information

<Insert Picture Here> Inside the Oracle Database 11g Optimizer Removing the black magic

<Insert Picture Here> Inside the Oracle Database 11g Optimizer Removing the black magic Inside the Oracle Database 11g Optimizer Removing the black magic Hermann Bär Data Warehousing Product Management, Server Technologies Goals of this session We will Provide a common

More information

Optimizer with Oracle Database 12c Release 2 O R A C L E W H I T E P A P E R J U N E

Optimizer with Oracle Database 12c Release 2 O R A C L E W H I T E P A P E R J U N E Optimizer with Oracle Database 12c Release 2 O R A C L E W H I T E P A P E R J U N E 2 0 1 7 Table of Contents Introduction 1 Adaptive Query Optimization 2 Optimizer Statistics 13 Optimizer Statistics

More information

Copyright 2015, Oracle and/or its affiliates. All rights reserved.

Copyright 2015, Oracle and/or its affiliates. All rights reserved. DB12c on SPARC M7 InMemory PoC for Oracle SPARC M7 Krzysztof Marciniak Radosław Kut CoreTech Competency Center 26/01/2016 Agenda 1 2 3 4 5 Oracle Database 12c In-Memory Option Proof of Concept what is

More information

Using Druid and Apache Hive

Using Druid and Apache Hive 3 Using Druid and Apache Hive Date of Publish: 2018-07-12 http://docs.hortonworks.com Contents Accelerating Hive queries using Druid... 3 How Druid indexes Hive data... 3 Transform Apache Hive Data to

More information

Die Wundertüte DBMS_STATS: Überraschungen in der Praxis

Die Wundertüte DBMS_STATS: Überraschungen in der Praxis Die Wundertüte DBMS_STATS: Überraschungen in der Praxis, 14. Mai 2018 Dani Schnider, Trivadis AG @dani_schnider danischnider.wordpress.com BASEL BERN BRUGG DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. GENEVA

More information

Pitfalls & Surprises with DBMS_STATS: How to Solve Them

Pitfalls & Surprises with DBMS_STATS: How to Solve Them Pitfalls & Surprises with DBMS_STATS: How to Solve Them Dani Schnider, Trivadis AG @dani_schnider danischnider.wordpress.com BASEL BERN BRUGG DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. GENEVA HAMBURG COPENHAGEN

More information

Oracle DB-Tuning Essentials

Oracle DB-Tuning Essentials Infrastructure at your Service. Oracle DB-Tuning Essentials Agenda 1. The DB server and the tuning environment 2. Objective, Tuning versus Troubleshooting, Cost Based Optimizer 3. Object statistics 4.

More information

SQL Gone Wild: Taming Bad SQL the Easy Way (or the Hard Way) Sergey Koltakov Product Manager, Database Manageability

SQL Gone Wild: Taming Bad SQL the Easy Way (or the Hard Way) Sergey Koltakov Product Manager, Database Manageability SQL Gone Wild: Taming Bad SQL the Easy Way (or the Hard Way) Sergey Koltakov Product Manager, Database Manageability Oracle Enterprise Manager Top-Down, Integrated Application Management Complete, Open,

More information

Answer: Reduce the amount of work Oracle needs to do to return the desired result.

Answer: Reduce the amount of work Oracle needs to do to return the desired result. SQL Tuning 101 excerpt: Explain Plan A Logical Approach By mruckdaschel@affiniongroup.com Michael Ruckdaschel Affinion Group International My Qualifications Software Developer for Affinion Group International

More information

<Insert Picture Here> DBA s New Best Friend: Advanced SQL Tuning Features of Oracle Database 11g

<Insert Picture Here> DBA s New Best Friend: Advanced SQL Tuning Features of Oracle Database 11g DBA s New Best Friend: Advanced SQL Tuning Features of Oracle Database 11g Peter Belknap, Sergey Koltakov, Jack Raitto The following is intended to outline our general product direction.

More information

Understanding Optimizer Statistics With Oracle Database 12c Release 2 O R A C L E W H I T E P A P E R M A R C H

Understanding Optimizer Statistics With Oracle Database 12c Release 2 O R A C L E W H I T E P A P E R M A R C H Understanding Optimizer Statistics With Oracle Database 12c Release 2 O R A C L E W H I T E P A P E R M A R C H 2 0 1 7 Table of Contents Introduction 1 What are Optimizer Statistics? 2 Statistics on Partitioned

More information

Top 7 Plan Stability Pitfalls & How to Avoid Them. Neil Chandler Chandler Systems Ltd UK

Top 7 Plan Stability Pitfalls & How to Avoid Them. Neil Chandler Chandler Systems Ltd UK Top 7 Plan Stability Pitfalls & How to Avoid Them Neil Chandler Chandler Systems Ltd UK Keywords: SQL Optimizer Plan Change Stability Outlines Baselines Plan Directives Introduction When you write some

More information

Effect of Stats on Two Columns Optimizer Statistics on tables and indexes are vital. Arup Nanda

Effect of Stats on Two Columns Optimizer Statistics on tables and indexes are vital. Arup Nanda Stats with Intelligence Arup Nanda Longtime Oracle DBA Effect of Stats on Two Columns Optimizer Statistics on tables and indexes are vital for the optimizer to compute optimal execution plans If there

More information

The Oracle Optimizer Explain the Explain Plan O R A C L E W H I T E P A P E R A P R I L

The Oracle Optimizer Explain the Explain Plan O R A C L E W H I T E P A P E R A P R I L The Oracle Optimizer Explain the Explain Plan O R A C L E W H I T E P A P E R A P R I L 2 0 1 7 Table of Contents Introduction 1 The Execution Plan 2 Displaying the Execution Plan 3 What is Cost? 7 Understanding

More information

Data Modeling and Databases Ch 7: Schemas. Gustavo Alonso, Ce Zhang Systems Group Department of Computer Science ETH Zürich

Data Modeling and Databases Ch 7: Schemas. Gustavo Alonso, Ce Zhang Systems Group Department of Computer Science ETH Zürich Data Modeling and Databases Ch 7: Schemas Gustavo Alonso, Ce Zhang Systems Group Department of Computer Science ETH Zürich Database schema A Database Schema captures: The concepts represented Their attributes

More information

Parallel Execution Plans

Parallel Execution Plans Parallel Execution Plans jonathanlewis.wordpress.com www.jlcomp.demon.co.uk My History Independent Consultant 33+ years in IT 28+ using Oracle (5.1a on MSDOS 3.3 Strategy, Design, Review, Briefings, Educational,

More information

ORACLE TRAINING CURRICULUM. Relational Databases and Relational Database Management Systems

ORACLE TRAINING CURRICULUM. Relational Databases and Relational Database Management Systems ORACLE TRAINING CURRICULUM Relational Database Fundamentals Overview of Relational Database Concepts Relational Databases and Relational Database Management Systems Normalization Oracle Introduction to

More information

Cost Based Optimizer CBO: Configuration Roadmap

Cost Based Optimizer CBO: Configuration Roadmap Cost Based Optimizer CBO: Configuration Roadmap Christian Antognini Sandro Crepaldi DOAG Regionaltreffen Hamburg/Nord 13.09.05, Hamburg Disclaimer > The CBO changes from release to release > It s difficult

More information

Slide 2 of 79 12/03/2011

Slide 2 of 79 12/03/2011 Doug Burns Introduction Simple Fundamentals Statistics on Partitioned Objects The Quality/Performance Trade-off Aggregation Scenarios Alternative Strategies Incremental Statistics Conclusions and References

More information

Advanced Oracle SQL Tuning v3.0 by Tanel Poder

Advanced Oracle SQL Tuning v3.0 by Tanel Poder Advanced Oracle SQL Tuning v3.0 by Tanel Poder /seminar Training overview This training session is entirely about making Oracle SQL execution run faster and more efficiently, understanding the root causes

More information

1/3/2015. Column-Store: An Overview. Row-Store vs Column-Store. Column-Store Optimizations. Compression Compress values per column

1/3/2015. Column-Store: An Overview. Row-Store vs Column-Store. Column-Store Optimizations. Compression Compress values per column //5 Column-Store: An Overview Row-Store (Classic DBMS) Column-Store Store one tuple ata-time Store one column ata-time Row-Store vs Column-Store Row-Store Column-Store Tuple Insertion: + Fast Requires

More information

Oracle Database 11g: SQL Tuning Workshop

Oracle Database 11g: SQL Tuning Workshop Oracle University Contact Us: Local: 0845 777 7 711 Intl: +44 845 777 7 711 Oracle Database 11g: SQL Tuning Workshop Duration: 3 Days What you will learn This Oracle Database 11g: SQL Tuning Workshop Release

More information

Oracle Database 18c New Performance Features

Oracle Database 18c New Performance Features Oracle Database 18c New Performance Features Christian Antognini @ChrisAntognini antognini.ch/blog BASEL BERN BRUGG DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. GENEVA HAMBURG COPENHAGEN LAUSANNE MUNICH STUTTGART

More information

Oracle Sql Tuning- A Framework

Oracle Sql Tuning- A Framework Oracle Sql Tuning- A Framework Prepared by Saurabh Kumar Mishra Performance Engineering & Enhancement offerings (PE2) Infosys Technologies Limited (NASDAQ: INFY) saurabhkumar_mishra@infosys.com This paper

More information

Course Modules for MCSA: SQL Server 2016 Database Development Training & Certification Course:

Course Modules for MCSA: SQL Server 2016 Database Development Training & Certification Course: Course Modules for MCSA: SQL Server 2016 Database Development Training & Certification Course: 20762C Developing SQL 2016 Databases Module 1: An Introduction to Database Development Introduction to the

More information

Adaptive Optimization. Presented by: Kerry Osborne Red Gate Webinar, Nov. 2013

Adaptive Optimization. Presented by: Kerry Osborne Red Gate Webinar, Nov. 2013 Adaptive Optimization Presented by: Kerry Osborne Red Gate Webinar, Nov. 2013 whoami Never Worked for Oracle Worked with Oracle DB Since 1982 (V2) Working with Exadata since early 2010 Work for Enkitec

More information

Exadata Implementation Strategy

Exadata Implementation Strategy Exadata Implementation Strategy BY UMAIR MANSOOB 1 Who Am I Work as Senior Principle Engineer for an Oracle Partner Oracle Certified Administrator from Oracle 7 12c Exadata Certified Implementation Specialist

More information

Interpreting Explain Plan Output. John Mullins

Interpreting Explain Plan Output. John Mullins Interpreting Explain Plan Output John Mullins jmullins@themisinc.com www.themisinc.com www.themisinc.com/webinars Presenter John Mullins Themis Inc. (jmullins@themisinc.com) 30+ years of Oracle experience

More information

Triangle SQL Server User Group Adaptive Query Processing with Azure SQL DB and SQL Server 2017

Triangle SQL Server User Group Adaptive Query Processing with Azure SQL DB and SQL Server 2017 Triangle SQL Server User Group Adaptive Query Processing with Azure SQL DB and SQL Server 2017 Joe Sack, Principal Program Manager, Microsoft Joe.Sack@Microsoft.com Adaptability Adapt based on customer

More information

Oracle 9i Application Development and Tuning

Oracle 9i Application Development and Tuning Index 2NF, NOT 3NF or BCNF... 2:17 A Anomalies Present in this Relation... 2:18 Anomalies (Specific) in this Relation... 2:4 Application Design... 1:28 Application Environment... 1:1 Application-Specific

More information

University of Waterloo Midterm Examination Sample Solution

University of Waterloo Midterm Examination Sample Solution 1. (4 total marks) University of Waterloo Midterm Examination Sample Solution Winter, 2012 Suppose that a relational database contains the following large relation: Track(ReleaseID, TrackNum, Title, Length,

More information

Oracle 1Z Oracle Database 11g Release 2- SQL Tuning. Download Full Version :

Oracle 1Z Oracle Database 11g Release 2- SQL Tuning. Download Full Version : Oracle 1Z0-117 Oracle Database 11g Release 2- SQL Tuning Download Full Version : http://killexams.com/pass4sure/exam-detail/1z0-117 OracleDatabase Data Warehousing Guide,Star Transformation with a Bitmap

More information

Exadata X3 in action: Measuring Smart Scan efficiency with AWR. Franck Pachot Senior Consultant

Exadata X3 in action: Measuring Smart Scan efficiency with AWR. Franck Pachot Senior Consultant Exadata X3 in action: Measuring Smart Scan efficiency with AWR Franck Pachot Senior Consultant 16 March 2013 1 Exadata X3 in action: Measuring Smart Scan efficiency with AWR Exadata comes with new statistics

More information

Data Vault Partitioning Strategies WHITE PAPER

Data Vault Partitioning Strategies WHITE PAPER Dani Schnider Data Vault ing Strategies WHITE PAPER Page 1 of 18 www.trivadis.com Date 09.02.2018 CONTENTS 1 Introduction... 3 2 Data Vault Modeling... 4 2.1 What is Data Vault Modeling? 4 2.2 Hubs, Links

More information

An Oracle White Paper April Best Practices for Gathering Optimizer Statistics

An Oracle White Paper April Best Practices for Gathering Optimizer Statistics An Oracle White Paper April 2012 Best Practices for Gathering Optimizer Statistics Introduction... 2 How to gather statistics... 3 Automatic statistics gathering job... 3 Manually statistics collection...

More information

Oracle Database 12c: SQL Tuning for Developers

Oracle Database 12c: SQL Tuning for Developers Oracle Database 12c: SQL Tuning for Developers Student Guide Volume I D79995GC10 Edition 1.0 December 2016 D84109 Learn more from Oracle University at education.oracle.com Author Dimpi Rani Sarmah Technical

More information

Introduction to column stores

Introduction to column stores Introduction to column stores Justin Swanhart Percona Live, April 2013 INTRODUCTION 2 Introduction 3 Who am I? What do I do? Why am I here? A quick survey 4? How many people have heard the term row store?

More information

7. Query Processing and Optimization

7. Query Processing and Optimization 7. Query Processing and Optimization Processing a Query 103 Indexing for Performance Simple (individual) index B + -tree index Matching index scan vs nonmatching index scan Unique index one entry and one

More information

Database statistics gathering: Synopsis

Database statistics gathering: Synopsis Database statistics gathering: Synopsis Introduction It is known that having proper database statistics is crucial for query optimizer. Statistics should properly describe data within the database. To

More information

Chapter 13: Query Processing

Chapter 13: Query Processing Chapter 13: Query Processing! Overview! Measures of Query Cost! Selection Operation! Sorting! Join Operation! Other Operations! Evaluation of Expressions 13.1 Basic Steps in Query Processing 1. Parsing

More information

MANAGING COST-BASED OPTIMIZER STATISTICS FOR PEOPLESOFT DAVID KURTZ UKOUG PEOPLESOFT ROADSHOW 2018

MANAGING COST-BASED OPTIMIZER STATISTICS FOR PEOPLESOFT DAVID KURTZ UKOUG PEOPLESOFT ROADSHOW 2018 MANAGING COST-BASED OPTIMIZER STATISTICS FOR PEOPLESOFT DAVID KURTZ UKOUG PEOPLESOFT ROADSHOW 2018 WHO AM I Accenture Enkitec Group Performance tuning PeopleSoft ERP Oracle RDBMS Book www.go-faster.co.uk

More information

D B M G Data Base and Data Mining Group of Politecnico di Torino

D B M G Data Base and Data Mining Group of Politecnico di Torino Database Management Data Base and Data Mining Group of tania.cerquitelli@polito.it A.A. 2014-2015 Optimizer operations Operation Evaluation of expressions and conditions Statement transformation Description

More information

Advanced Database Systems

Advanced Database Systems Lecture IV Query Processing Kyumars Sheykh Esmaili Basic Steps in Query Processing 2 Query Optimization Many equivalent execution plans Choosing the best one Based on Heuristics, Cost Will be discussed

More information

Copyright 2016 Ramez Elmasri and Shamkant B. Navathe

Copyright 2016 Ramez Elmasri and Shamkant B. Navathe CHAPTER 19 Query Optimization Introduction Query optimization Conducted by a query optimizer in a DBMS Goal: select best available strategy for executing query Based on information available Most RDBMSs

More information

Demystifying SQL Tuning: Tips and Techniques for SQL Experts

Demystifying SQL Tuning: Tips and Techniques for SQL Experts Demystifying SQL Tuning: Tips and Techniques for SQL Experts Mughees A. Minhas Director of Product Management, Database and Systems Management Sergey Koltakov Product Manager, Database Manageability Outline

More information

<Insert Picture Here> Oracle Database 11g: Neue Features im Oracle Optimizer

<Insert Picture Here> Oracle Database 11g: Neue Features im Oracle Optimizer Oracle Database 11g: Neue Features im Oracle Optimizer Hermann Bär, Oracle Director Product Management, Server Technologies Data Warehousing Inside the Oracle Database 11g Optimizer

More information

Oracle Hyperion Profitability and Cost Management

Oracle Hyperion Profitability and Cost Management Oracle Hyperion Profitability and Cost Management Configuration Guidelines for Detailed Profitability Applications November 2015 Contents About these Guidelines... 1 Setup and Configuration Guidelines...

More information

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Copyright 2012, Oracle and/or its affiliates. All rights reserved. 1 Oracle Partitioning für Einsteiger Hermann Bär Partitioning Produkt Management 2 Disclaimer The goal is to establish a basic understanding of what can be done with Partitioning I want you to start thinking

More information

Introduction to Query Processing and Query Optimization Techniques. Copyright 2011 Ramez Elmasri and Shamkant Navathe

Introduction to Query Processing and Query Optimization Techniques. Copyright 2011 Ramez Elmasri and Shamkant Navathe Introduction to Query Processing and Query Optimization Techniques Outline Translating SQL Queries into Relational Algebra Algorithms for External Sorting Algorithms for SELECT and JOIN Operations Algorithms

More information

20 Essential Oracle SQL and PL/SQL Tuning Tips. John Mullins

20 Essential Oracle SQL and PL/SQL Tuning Tips. John Mullins 20 Essential Oracle SQL and PL/SQL Tuning Tips John Mullins jmullins@themisinc.com www.themisinc.com www.themisinc.com/webinars Presenter John Mullins Themis Inc. (jmullins@themisinc.com) 30+ years of

More information

OpenWorld 2018 SQL Tuning Tips for Cloud Administrators

OpenWorld 2018 SQL Tuning Tips for Cloud Administrators OpenWorld 2018 SQL Tuning Tips for Cloud Administrators GP (Prabhaker Gongloor) Senior Director of Product Management Bjorn Bolltoft Dr. Khaled Yagoub Systems and DB Manageability Development Oracle Corporation

More information

Query Optimizer, Who Influences & How it works ++ optimization techniques

Query Optimizer, Who Influences & How it works ++ optimization techniques Query Optimizer, Who Influences & How it works ++ optimization techniques AIOUG : ODevC Yatra 2018, India Chandan Tanwani Senior Application Engineer Oracle Financial Services Software Ltd. Copyright 2018

More information

! A relational algebra expression may have many equivalent. ! Cost is generally measured as total elapsed time for

! A relational algebra expression may have many equivalent. ! Cost is generally measured as total elapsed time for Chapter 13: Query Processing Basic Steps in Query Processing! Overview! Measures of Query Cost! Selection Operation! Sorting! Join Operation! Other Operations! Evaluation of Expressions 1. Parsing and

More information

Chapter 13: Query Processing Basic Steps in Query Processing

Chapter 13: Query Processing Basic Steps in Query Processing Chapter 13: Query Processing Basic Steps in Query Processing! Overview! Measures of Query Cost! Selection Operation! Sorting! Join Operation! Other Operations! Evaluation of Expressions 1. Parsing and

More information

Oracle. Exam Questions 1Z Oracle Database 11g Release 2: SQL Tuning Exam. Version:Demo

Oracle. Exam Questions 1Z Oracle Database 11g Release 2: SQL Tuning Exam. Version:Demo Oracle Exam Questions 1Z0-117 Oracle Database 11g Release 2: SQL Tuning Exam Version:Demo 1.You ran a high load SQL statement that used an index through the SQL Tuning Advisor and accepted its recommendation

More information

Oracle Database Performance Tuning, Benchmarks & Replication

Oracle Database Performance Tuning, Benchmarks & Replication Oracle Database Performance Tuning, Benchmarks & Replication Kapil Malhotra kapil.malhotra@software.dell.com Solutions Architect, Information Management Dell Software 2 11/29/2013 Software Database Tuning

More information

Firebird in 2011/2012: Development Review

Firebird in 2011/2012: Development Review Firebird in 2011/2012: Development Review Dmitry Yemanov mailto:dimitr@firebirdsql.org Firebird Project http://www.firebirdsql.org/ Packages Released in 2011 Firebird 2.1.4 March 2011 96 bugs fixed 4 improvements,

More information

Column-Stores vs. Row-Stores: How Different Are They Really?

Column-Stores vs. Row-Stores: How Different Are They Really? Column-Stores vs. Row-Stores: How Different Are They Really? Daniel J. Abadi, Samuel Madden and Nabil Hachem SIGMOD 2008 Presented by: Souvik Pal Subhro Bhattacharyya Department of Computer Science Indian

More information

Chapter 12: Query Processing

Chapter 12: Query Processing Chapter 12: Query Processing Database System Concepts, 6 th Ed. See www.db-book.com for conditions on re-use Overview Chapter 12: Query Processing Measures of Query Cost Selection Operation Sorting Join

More information

Chapter 12: Query Processing. Chapter 12: Query Processing

Chapter 12: Query Processing. Chapter 12: Query Processing Chapter 12: Query Processing Database System Concepts, 6 th Ed. See www.db-book.com for conditions on re-use Chapter 12: Query Processing Overview Measures of Query Cost Selection Operation Sorting Join

More information

Tuning with Statistics

Tuning with Statistics Tuning with Statistics Wolfgang Breitling breitliw@centrexcc.com Agenda The DBMS_STATS Package Uses of DBMS_STATS beyond analyze The stattab Table Transferring Statistics with the stattab Table Means to

More information

Exploring Best Practices and Guidelines for Tuning SQL Statements

Exploring Best Practices and Guidelines for Tuning SQL Statements Exploring Best Practices and Guidelines for Tuning SQL Statements Ami Aharonovich Oracle ACE & OCP Ami@DBAces.co.il Oracle ACE Who am I Oracle Certified Professional DBA (OCP) Founder and CEO, DBAces Oracle

More information

Querying Data with Transact SQL

Querying Data with Transact SQL Course 20761A: Querying Data with Transact SQL Course details Course Outline Module 1: Introduction to Microsoft SQL Server 2016 This module introduces SQL Server, the versions of SQL Server, including

More information

PS2 out today. Lab 2 out today. Lab 1 due today - how was it?

PS2 out today. Lab 2 out today. Lab 1 due today - how was it? 6.830 Lecture 7 9/25/2017 PS2 out today. Lab 2 out today. Lab 1 due today - how was it? Project Teams Due Wednesday Those of you who don't have groups -- send us email, or hand in a sheet with just your

More information

.. Spring 2008 CSC 468: DBMS Implementation Alexander Dekhtyar..

.. Spring 2008 CSC 468: DBMS Implementation Alexander Dekhtyar.. .. Spring 2008 CSC 468: DBMS Implementation Alexander Dekhtyar.. Tuning Oracle Query Execution Performance The performance of SQL queries in Oracle can be modified in a number of ways: By selecting a specific

More information

Chapter 12: Query Processing

Chapter 12: Query Processing Chapter 12: Query Processing Overview Catalog Information for Cost Estimation $ Measures of Query Cost Selection Operation Sorting Join Operation Other Operations Evaluation of Expressions Transformation

More information

New Optimizer Features in Oracle 12c

New Optimizer Features in Oracle 12c SAGE Computing Services Customised Oracle Training Workshops and Consulting New Optimizer Features in Oracle 12c Penny Cookson SAGE Computing Services www.sagecomputing.com.au penny@sagecomputing.com.au

More information

Explaining the Explain Plan:

Explaining the Explain Plan: Explaining the Explain Plan: Interpre'ng Execu'on Plans for SQL Statements Maria Colgan Master Product Manager June 2017 @SQLMaria Safe Harbor Statement The following is intended to outline our general

More information

Tuning Considerations for Different Applications Lesson 4

Tuning Considerations for Different Applications Lesson 4 4 Tuning Considerations for Different Applications Lesson 4 Objectives After completing this lesson, you should be able to do the following: Use the available data access methods to tune the logical design

More information

Exadata Implementation Strategy

Exadata Implementation Strategy BY UMAIR MANSOOB Who Am I Oracle Certified Administrator from Oracle 7 12c Exadata Certified Implementation Specialist since 2011 Oracle Database Performance Tuning Certified Expert Oracle Business Intelligence

More information

Query Processing & Optimization

Query Processing & Optimization Query Processing & Optimization 1 Roadmap of This Lecture Overview of query processing Measures of Query Cost Selection Operation Sorting Join Operation Other Operations Evaluation of Expressions Introduction

More information

Teradata. This was compiled in order to describe Teradata and provide a brief overview of common capabilities and queries.

Teradata. This was compiled in order to describe Teradata and provide a brief overview of common capabilities and queries. Teradata This was compiled in order to describe Teradata and provide a brief overview of common capabilities and queries. What is it? Teradata is a powerful Big Data tool that can be used in order to quickly

More information

Course Outline and Objectives: Database Programming with SQL

Course Outline and Objectives: Database Programming with SQL Introduction to Computer Science and Business Course Outline and Objectives: Database Programming with SQL This is the second portion of the Database Design and Programming with SQL course. In this portion,

More information

<Insert Picture Here> DBA Best Practices: A Primer on Managing Oracle Databases

<Insert Picture Here> DBA Best Practices: A Primer on Managing Oracle Databases DBA Best Practices: A Primer on Managing Oracle Databases Mughees A. Minhas Sr. Director of Product Management Database and Systems Management The following is intended to outline

More information

Oracle SQL Tuning for Developers Workshop Student Guide - Volume I

Oracle SQL Tuning for Developers Workshop Student Guide - Volume I Oracle SQL Tuning for Developers Workshop Student Guide - Volume I D73549GC10 Edition 1.0 October 2012 D78799 Authors Sean Kim Dimpi Rani Sarmah Technical Contributors and Reviewers Nancy Greenberg Swarnapriya

More information

Oracle DB-Tuning Essentials

Oracle DB-Tuning Essentials Infrastructure at your Service. Oracle DB-Tuning Essentials Agenda 1. The DB server and the tuning environment 2. Objective, Tuning versus Troubleshooting, Cost Based Optimizer 3. Object statistics 4.

More information

MTA Database Administrator Fundamentals Course

MTA Database Administrator Fundamentals Course MTA Database Administrator Fundamentals Course Session 1 Section A: Database Tables Tables Representing Data with Tables SQL Server Management Studio Section B: Database Relationships Flat File Databases

More information

Bloom Filters DOAG Webinar, 12 August 2016 Christian Antognini Senior Principal Consultant

Bloom Filters DOAG Webinar, 12 August 2016 Christian Antognini Senior Principal Consultant DOAG Webinar, 12 August 2016 Christian Antognini Senior Principal Consultant BASEL BERN BRUGG DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. GENEVA HAMBURG COPENHAGEN LAUSANNE MUNICH STUTTGART VIENNA ZURICH

More information

Database System Concepts

Database System Concepts Chapter 13: Query Processing s Departamento de Engenharia Informática Instituto Superior Técnico 1 st Semester 2008/2009 Slides (fortemente) baseados nos slides oficiais do livro c Silberschatz, Korth

More information

Query optimization. Elena Baralis, Silvia Chiusano Politecnico di Torino. DBMS Architecture D B M G. Database Management Systems. Pag.

Query optimization. Elena Baralis, Silvia Chiusano Politecnico di Torino. DBMS Architecture D B M G. Database Management Systems. Pag. Database Management Systems DBMS Architecture SQL INSTRUCTION OPTIMIZER MANAGEMENT OF ACCESS METHODS CONCURRENCY CONTROL BUFFER MANAGER RELIABILITY MANAGEMENT Index Files Data Files System Catalog DATABASE

More information

MASSACHUSETTS INSTITUTE OF TECHNOLOGY Database Systems: Fall 2008 Quiz II

MASSACHUSETTS INSTITUTE OF TECHNOLOGY Database Systems: Fall 2008 Quiz II Department of Electrical Engineering and Computer Science MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.830 Database Systems: Fall 2008 Quiz II There are 14 questions and 11 pages in this quiz booklet. To receive

More information

Oracle Database 11g: SQL Tuning Workshop. Student Guide

Oracle Database 11g: SQL Tuning Workshop. Student Guide Oracle Database 11g: SQL Tuning Workshop Student Guide D52163GC10 Edition 1.0 June 2008 Author Jean-François Verrier Technical Contributors and Reviewers Muriel Fry (Special thanks) Joel Goodman Harald

More information

Column Stores vs. Row Stores How Different Are They Really?

Column Stores vs. Row Stores How Different Are They Really? Column Stores vs. Row Stores How Different Are They Really? Daniel J. Abadi (Yale) Samuel R. Madden (MIT) Nabil Hachem (AvantGarde) Presented By : Kanika Nagpal OUTLINE Introduction Motivation Background

More information

DB2 9 for z/os Selected Query Performance Enhancements

DB2 9 for z/os Selected Query Performance Enhancements Session: C13 DB2 9 for z/os Selected Query Performance Enhancements James Guo IBM Silicon Valley Lab May 10, 2007 10:40 a.m. 11:40 a.m. Platform: DB2 for z/os 1 Table of Content Cross Query Block Optimization

More information

INFSCI 2711 Database Analysis and Design Example I for Final Exam: Solutions

INFSCI 2711 Database Analysis and Design Example I for Final Exam: Solutions Dr. Stefan Brass July 26, 2001 School of Information Sciences University of Pittsburgh INFSCI 2711 Database Analysis and Design Example I for Final Exam: Solutions General Remarks The average was 22.2

More information

DATABASE PERFORMANCE AND INDEXES. CS121: Relational Databases Fall 2017 Lecture 11

DATABASE PERFORMANCE AND INDEXES. CS121: Relational Databases Fall 2017 Lecture 11 DATABASE PERFORMANCE AND INDEXES CS121: Relational Databases Fall 2017 Lecture 11 Database Performance 2 Many situations where query performance needs to be improved e.g. as data size grows, query performance

More information

Common Performance Monitoring Mistakes

Common Performance Monitoring Mistakes Common Performance Monitoring Mistakes Virag Saksena CEO Auptyma Corporation peakperformance@auptyma.com Tuning Approach BUS X SYS Identify slow business actions Correlate the two Find system bottlenecks

More information

A Unit of SequelGate Innovative Technologies Pvt. Ltd. All Training Sessions are Completely Practical & Real-time

A Unit of SequelGate Innovative Technologies Pvt. Ltd. All Training Sessions are Completely Practical & Real-time SQL Basics & PL-SQL Complete Practical & Real-time Training Sessions A Unit of SequelGate Innovative Technologies Pvt. Ltd. ISO Certified Training Institute Microsoft Certified Partner Training Highlights

More information

Query Processing. Debapriyo Majumdar Indian Sta4s4cal Ins4tute Kolkata DBMS PGDBA 2016

Query Processing. Debapriyo Majumdar Indian Sta4s4cal Ins4tute Kolkata DBMS PGDBA 2016 Query Processing Debapriyo Majumdar Indian Sta4s4cal Ins4tute Kolkata DBMS PGDBA 2016 Slides re-used with some modification from www.db-book.com Reference: Database System Concepts, 6 th Ed. By Silberschatz,

More information

Infrastructure at your Service. In-Memory-Pläne für den 12.2-Optimizer: Teuer oder billig?

Infrastructure at your Service. In-Memory-Pläne für den 12.2-Optimizer: Teuer oder billig? Infrastructure at your Service. In-Memory-Pläne für den 12.2-Optimizer: Teuer oder billig? About me Infrastructure at your Service. Clemens Bleile Senior Consultant Oracle Certified Professional DB 11g,

More information

Optimizing Testing Performance With Data Validation Option

Optimizing Testing Performance With Data Validation Option Optimizing Testing Performance With Data Validation Option 1993-2016 Informatica LLC. No part of this document may be reproduced or transmitted in any form, by any means (electronic, photocopying, recording

More information