Oracle DB-Tuning Essentials

Transcription

1 Infrastructure at your Service. Oracle DB-Tuning Essentials

2 Agenda 1. The DB server and the tuning environment 2. Objective, Tuning versus Troubleshooting, Cost Based Optimizer 3. Object statistics 4. Access paths I 5. Access paths II 6. Monitoring Performance I 7. Monitoring Performance II 8. Application setup Page 2

3 Tuning the Application > Storage clauses > Indexing > Partitioning > Result Cache > Materialized views > Star transformation > Redo logging > Parallel Query Page 3

4 Storage clauses Do we care? Do we care about how rows are stored in tables? Is it something for development or operations? Yes > Only development knows its data > Only development knows the lifecycle of the rows Do you want this default for create table? PCTFREE 10 PCTUSED 40 INITRANS 1 MAXTRANS 255 Do you want this default for create index? PCTFREE 10 INITRANS 2 MAXTRANS 255 COMPUTE STATISTICS Page 4

5 Storage clauses Space in a block Block HEADER [7369 SMITH CLERK 800 ][7499 ALLEN SALESMA N 1600][7521 WARD SAL ESMAN 1250][7566 JON ES MANAGER 2975][765 4 MARTIN SALESMAN 1 250][7698 BLAKE MANA GER 2850] FREE SPACE > We insert rows in a block that has enough space (chosen from freelist or ASSM) > But the size of the row can change (updates) > We must keep space for already inserted rows: PCTFREE > Not used only for rows, but also for transaction information (locks) > Only when lots of space is released we can insert again in that block Page 5

6 Storage clauses What if there is no space? When locking a row > We wait at block level instead of waiting at row level > Wait event: enq: TX - allocate ITL entry > more contention When updating a row > We cannot move a row because it s block location (rowid) is referenced in indexes > We can migrate a row, leaving only a pointer > Access by rowid must follow the pointer > more logical (and physical) reads > No difference for full table scan Page 6

7 Storage clauses What are chained rows? Row migration is a cause of a chained row > The pointer chains to another row piece in another block > Proactive solution: have higher PCTFREE > Reactive solution: reorganize A row may be split because it is larger than block size > The row pieces will be chained > more logical reads (and physical) when accessing to other parts > Solution: have larger block size, put frequently selected columns in front A row may be split because it has more than 255 columns > This is intra-block chaining. > more logical reads but for same block. Page 7

8 Storage clauses How to detect chained rows? Forget USER_TABLES.CHAIN_CNT > Updated only by ANALYZE > Does not show the consequence > Don t count intra-block chaining > But can be used to know the ROWID Session statistics NAME VALUE table scan rows gotten 3674 table fetch by rowid table fetch continued row 2584 index fetch by key 2 > Good to check when number of logical reads seems too high Page 8

9 Storage clauses Demo: Migrated rows scenario 1. Create a table with PCTFREE 10 (default) 2. Update to increase all rows 3. Do a FULL TABLE SCAN > Migrated rows don t have any consequence 4. Do an INDEX ACCESS > Migrated rows increase the table fetch continued row 5. Reorganize the table 6. Do an INDEX ACCESS > No migrated rows anymore Page 9

10 Storage clauses PCTFREE Define PCTFREE to avoid > Migrated rows > ITL contention > Hot blocks (too many rows in one block) > But don t waste space (block space is also space in buffer cache) If you have no updates: PCTFREE 0 If you update large columns > Have a large PCTFREE: > try to guess the number of rows per block that you will have at max > calculate PCTFREE from the size of initial inserts > Or use LOB instead of large datatypes Page 10

11 Storage clauses Block Size Default block size is 8k > Is the right size for most cases > Can be larger (16k) when having very large rows Tablespaces can have different block sizes > But buffer cache size has to be managed manually > Mainly used to transport tablespace, not for performance LOB objects can be smaller or bigger > When the application knows the size of the object > A distinct tablespace can be used to store the LOB to save space Page 11

12 Storage clauses Compression Basic compression in EE > Only compressed for full blocks during direct-path inserts > In 12c is called ROW STORE COMPRESS BASIC OLTP in EE + Advanced Compression > In 12c is called ROW STORE COMPRESS ADVANCED > Is the same a basic but automatically triggered when DML fills a block HCC Compression > COLUMN STORE COMPRESS FOR QUERY/ARCHIVE LOW/HIGH > Only for Oracle storage (Exadata, ZFS Appliance) > This is not for DML Page 12

13 Storage clauses Temporary tablespaces Multiple tempfiles > Do not help for concurrency Temporary tablespace groups > Helps to stripe across multiple files Key for efficient sorting (or index creation) > Temporary tablespace groups > Manual workarea with big sort_area_size (up to 1 or 2 GB) > Parallel Query > For index creation: NOLOGGING followed by a backup Page 13

14 Tuning the Application > Storage clauses > Indexing > Partitioning > Result Cache > Materialized views > Star transformation > Redo logging > Parallel Query Page 14

15 Indexing Indexing for WHERE clause Equality > Get directly to the rows through a unique index > Get directly to the rows for one value Select * from WHERE COL1=:value Inequality > Get rows starting from a point Select * from WHERE COL1>:value Is Null > Indexes (non bitmap) do not store entire null entries Select * from WHERE COL1 is null > But in a compound index one value can be null Select * from WHERE COL1:=value and COL2 is null Page 15

16 Indexing Indexing for WHERE clause Functions > Functions prevent index usage > Except if we index the function Functions may me implicit > Bad data types introducing TO_CHAR, TO_DATE > NLS_COMP=LINGUISTIC Id Operation Name Rows Bytes Cost (%CPU) SELECT STATEMENT (2) * 1 TABLE ACCESS FULL LIEN_DOC_ (2) Predicate Information (identified by operation id): filter("s"."type_lien">0 AND NLSSORT( "S"."DOC_BPM_UUID", 'nls_sort=''binary_ai''')=nlssort(:1,'nls_sort=''binary_ai''')) Page 16

17 Indexing Indexing for other reasons ORDER BY: > Indexes are sorted > May prevent a SORT operation for an order by > For binary data (number, date) not linguistic (varchar2) GROUP BY: > It s easier to deduplicate when the input is sorted LOCKS: > When we delete (or update the key) all child tables have to be locked > Without index starting with the foreign key, whole table is locked > With an index to access the child, index block manages concurrency Page 17

18 Indexing Column order Index on (A,B) > Can be used to access A=:value > Can be used to access A=:value and B=:value > Can be used to access A=:value and B>:value > Cannot be used to access B=:value (except with INDEX SKIP SCAN) Choose the order that fit to most of the queries Most selective first? > Yes if it is used alone > But there is also INDEX SKIP SCAN when first column has a bad selectivity > Compression is better with less selective first Page 18

19 Indexing Too many indexes Indexes must be maintained > 2 or 3 blocks for each insert > 2 or 3 blocks for each delete > doubles for each update of the indexed columns Too many indexes slows down DML (+ redo generation) Choose indexes carefully > Better to add a column to an existing index > Index only what is selective enough to be used > Difficult to monitor usage Use invisible indexes > To bring it back quickly if needed Page 19

20 Indexing Bitmap indexes For reporting / BI > We don t know all criteria combinations > We create an index on each column > Bitmap index is not expensive to merge Bitmap index are well suited when > Each column is not very selective > The combination (with OR and AND) is selective No bitmap index for OLTP not suited for conventional DML Regular indexes can be transformed to bitmap > But BITMAP CONVERSION FROM ROWIDS is expensive > And regular indexes are much bigger than bitmap indexes Page 20

21 Indexing Index Organized tables IOT : > Logically it is a table > Physically it is an index with all columns (indexed by PK) > No table segment: all in the index leaves but without a rowid > Some columns can go to an overflow segment IOT are well suited for > Access by primary key > It s the best clustering factor But secondary indexes > Reference the PK instead of a rowid > Access to non-indexed columns is not optimal IOT is well suited for association tables where PK covers all columns Page 21

22 Tuning the Application > Storage clauses > Indexing > Partitioning > Materialized views > Star transformation > Redo logging > Parallel Query Page 22

23 Partitioning Partitioning by List ORDER_ID CUST_ID SALES_DATE CHANNEL JAN-2014 WEB JUL-2014 SHOP MAR-2013 WEB JAN-2014 SHOP DEC-2014 WEB JAN-2013 WEB By LIST (CHANNEL) values ( WEB ) ORDER_ID CUST_ID SALES_DATE CHANNEL JAN-2014 WEB MAR-2013 WEB DEC-2014 WEB JAN-2013 WEB List partitioning: > To separate physically discrete values > Logically in one table but physically as separate tables By LIST (CHANNEL) values ( SHOP ) ORDER_ID CUST_ID SALES_DATE CHANNEL JUL-2014 SHOP JAN-2014 SHOP Page 23

24 Partitioning Partitioning by Range ORDER_ID CUST_ID SALES_DATE CHANNEL JAN-2014 WEB JUL-2014 SHOP MAR-2013 WEB JAN-2014 SHOP DEC-2014 WEB MAR-2013 WEB By RANGE (SALES_DATE) values less than (01-JAN-2014) ORDER_ID CUST_ID SALES_DATE CHANNEL MAR-2013 WEB MAR-2013 WEB values less than (01-JAN-2015) Range partitioning: > Often used for historical data > Easy to purge old data > Group recent data ORDER_ID CUST_ID SALES_DATE CHANNEL JAN-2014 WEB JUL-2014 SHOP JAN-2014 SHOP DEC-2014 WEB Page 24

25 Partitioning Partitioning by Hash ORDER_ID CUST_ID SALES_DATE CHANNEL JAN-2014 WEB JUL-2014 SHOP MAR-2013 WEB JAN-2014 SHOP DEC-2014 WEB MAR-2013 WEB By HASH (CUST_ID) partitions 2 ORDER_ID CUST_ID SALES_DATE CHANNEL MAR-2013 WEB MAR-2013 WEB JAN-2014 SHOP Hash partitioning: > Distribute data (performance) > Always power of 2 for better distribution (2,4,8,16,64) ORDER_ID CUST_ID SALES_DATE CHANNEL JAN-2014 WEB JUL-2014 SHOP DEC-2014 WEB Page 25

26 Partitioning Partitioning Composite Partition by RANGE (SALES_DATE) Subpartition by HASH (CUST_ID) values less than (01-JAN-2014) ORDER_ID CUST_ID SALES_DATE CHANNEL MAR-2013 WEB MAR-2013 WEB ORDER_ID CUST_ID SALES_DATE CHANNEL MAR-2013 WEB MAR-2013 WEB values less than (01-JAN-2015) ORDER_ID CUST_ID SALES_DATE CHANNEL JAN-2014 WEB JUL-2014 SHOP JAN-2014 SHOP DEC-2014 WEB ORDER_ID CUST_ID SALES_DATE CHANNEL JAN-2014 SHOP ORDER_ID CUST_ID SALES_DATE CHANNEL JAN-2014 WEB JUL-2014 SHOP DEC-2014 WEB Page 26

27 Partitioning Partitioning Reference and Interval When a parent table is partitioned > Child tables are partitioned automatically > Example: ORDERS partitioned by CUSTOMER then ORDER_ITEMS also partitioned by CUSTOMER > Even when the parent column is not present in the child Interval partitioning is a range partitioning > New partitions are created during inserts Reference and Interval partitioning > Both available from 11g > Can be used together since 12c Page 27

28 Partitioning Partitioning Why? Maintenance > Truncate old partition to purge old data > Exchange partition to load new data > Move current partition to faster storage Performance (scalability) > Partition Pruning > The where clause avoids to scan all partitions > Partition-wise Joins > When two tables are partitioned on the join columns, we can join partition pairs and then concatenate the result > Less temp space needed for Hash joins > Parallel query > Data distribution done at partition level Page 28

29 Partitioning Partitioning Indexes Indexes can be partitioned as well > LOCAL: same partitioning as the table > GLOBAL: independent partitioning Local indexes are easy to manage but bad performance if no partition pruning on the index Global indexes have to be maintained during partition maintenance > UPDATE GLOBAL INDEX > Can be done asynchronously in 12c Page 29

30 Partitioning Partitioning alternatives Partitioning is an EE option We have an alternative to partitioning option > Partition views are still usable Implementation: > Several tables with same structure > A UNION ALL view over them CBO can do partition pruning But we need to insert into the right partition Page 30

31 Tuning the Application > Storage clauses > Indexing > Partitioning > Result Cache > Materialized views > Star transformation > Redo logging > Parallel Query Page 31

32 Result Cache Why and When? The buffer cache is only for the blocks > Will avoid I/O if we run the same query frequently > But will still do the same work in CPU (buffer gets, join, sorts) Result Cache can keep the result for > A PL/SQL function > A SQL statement It keeps dependencies > Whole result is invalidated if a dependent table has modifications > Cache misses are expensive (exclusive latch for the whole cache) It is for static data that we query often > Example: Result of a Materialized View that is used frequently on query rewrites Page 32

33 Result Cache Parameters result_cache_mode > MANUAL: only when enabled by hint or by table attribute > FORCE: all queries (unless NO_RESULT_CACHE hint) result_cache_max_size > 0 disables the result cache (not the default) result_cache_max_result > Defines the maximum % for one result The result can even be cached at client level > Avoids frequent roundtrips > But not invalidated immediately > client_result_cache_lag And dependant objects can be remote (db link) > result_cache_remote_expiration Page 33

34 Result Cache Usage for queries ALTER TABLE RESULT_CACHE (mode force) > Result cache enabled for queries where all tables are in mode force SELECT /*+ RESULT_CACHE */ > Force result cache at statement level Id Operation Name Rows SELECT STATEMENT 14 1 RESULT CACHE 5rfy9nm195pn63q4u7mxm69x7a * 2 HASH JOIN 14 3 TABLE ACCESS FULL DEPT 4 4 TABLE ACCESS FULL EMP Result Cache Information (identified by operation id): column-count=10; dependencies=(demo.emp, DEMO.DEPT); name="select * from EMP join DEPT using(deptno)" Page 34

35 Result Cache Scalability The Result Cache is shared, protected by only one latch > In 11.1 always exclusive -> scalability issues > From 11.2 shared for cache hits but exclusive for cache misses Event: enq: RC - Result Cache: Contention > When lots of invalidations (updates on dependencies) > When lots of cache misses (function call for different values) It is for static data that we query often Page 36

36 Result Cache Exercise 1. Connect as demo/demo and flush the buffer cache SQL> alter system flush buffer_cache; SQL> connect demo/demo 2. Run the following query several times > Check logical and physical reads (autotrace, gather_plan_statistics) SQL> select max(prod_desc) from products; 3. Run the same with result cache > Use hint, alter session, alter table, > Check logical reads (after several executions) 4. Do some DML on underlying table SQL> insert into products values(0,'product'); 5. Run the same select again Page 37

37 Tuning the Application > Storage clauses > Indexing > Partitioning > Result Cache > Materialized views > Star transformation > Redo logging > Parallel Query Page 38

38 Materialized views Why? An index > Stores redundant data for performance reasons > Is maintained when table has DML > Is sorted and stored in a B-Tree > Is transparent (we still query on the view) A bitmap join index > Can index over a join (value from one table points to a rowid in a joined table). Often used in Star schemas. A materialized view > Can store redundant data from joins, aggregations, etc. > Is defined as a view (a SELECT) but the result is stored in a table > Result can be updated or refreshed > Often used in DWH (aggregates), replication (db link) Page 39

39 Materialized views Refresh data Refreshed manually > Calling dbms_mview > Can be scheduled > Can be truncate + insert but table will be seen empty in between Refreshed on commit > Changes are propagated to the MVIEW Fast refresh > Changes are stored in MVIEW LOGS. > Applied on refresh > Deleted when all MVIEWS are refreshed Real-Time Materialized Views in 12.2 > Builds actual result by joining the MVIEW Log with the stale MV Page 40

40 Materialized views Usage Materialized view as a table > We can query it > We can add constraints on it Materialized view to optimize > Query rewrite: The optimizer can choose to access the MVIEW even if we query the table. > Allowed by query_rewrite_enabled=true > Shown in execution plan as MAT_VIEW REWRITE > Even if MVIEW is not fresh: query_rewrite_integrity=stale_tolerated (or Real-Time MV in 12.2) Dbms_mview or Enterprise Manager > Check rewrite and refresh capabilities > EM can suggest modifications Page 41

41 Tuning the Application > Storage clauses > Indexing > Partitioning > Result Cache > Materialized views > Star transformation > Redo logging > Parallel Query Page 42

42 Star transformation Query transformations Question: Is it better to write Or select distinct dname from dept where deptno in ( select deptno from emp where sal>1000 ); select distinct dname from dept join emp using(deptno) where sal>1000; Answer: It s two ways to do the same Id Operation Name SELECT STATEMENT 1 HASH UNIQUE 2 HASH JOIN SEMI 3 TABLE ACCESS FULL DEPT 4 TABLE ACCESS FULL EMP ? Page 43

43 Star transformation Query transformations The optimizer tries several transformations in order to find new access paths (i.e. better execution plans) > Eliminate unnecessary predicates, joins, order by > Merge subqueries > Push down predications to subqueries > Move distinct and group by > In-Memory aggregation (vector transformation) > OR expansion to UNION ALL Can be driven by rules or cost estimation Can be controlled by hints > Eliminate unnecessary predicates, joins, order by Page 44

44 Star transformation Star Transformation Join back CUSTOMER Name, Addess, City VIP Y/N COUNTRY Name Area PRODUCT Code Descriptio Show SALES SALES Amount Quantity > Per Month, Product description, Country > With Amount and quantity DAY Day of week Date Month Holiday Y/N 2. Join back to dimensions double access to dimension > Only VIP customers in EMEA area 1. Join dimensions to sales fact Page 45

45 Tuning the Application > Storage clauses > Indexing > Partitioning > Result Cache > Materialized views > Star Transformation > Redo logging > Parallel Query Page 50

46 Redo logging Log writer LGWR writing idle Session 1 wait commit on log at successful SCN file sync 0010 Session 2 wait commit on log at successful SCN file sync 0020 Session 3 wait commit on log at successful SCN file sync 0030 Session 4 wait commit on log at successful SCN file sync 0040 Session 5 wait commit on log at successful SCN file sync 0050 Each commit sends a message to lgwr: > Redo up to commit SCN must be on disk > If lgwr is not busy, it starts to write > If lgwr is already busy, message is queued When lgwr completes its write > It acknowledges to the session > Then writes for all the queue > And acknowledges all sessions This is piggy back commit Optimization: > We can choose not to wait acknowledgement: COMMIT NOWAIT > We can choose to group all writes: COMMIT BATCH Page 51

47 Redo logging COMMIT_WAIT and COMMIT_LOGGING Page 52

48 Redo logging COMMIT_WAIT and COMMIT_LOGGING When can you use NOWAIT BATCH? > You are doing intermediate commits during bulk load > You are migrating data using the transactional web services (which commit every record) But > You must do a COMMIT WAIT at the end when sending the successful status to the user > You must manage failure without relying on successful commit status > You must know that another session may see commited changes which can be lost in case of instance failure Page 53

49 Redo logging COMMIT_WAIT and COMMIT_LOGGING NOLOGGING > Avoids redo generation for direct-path operations > On tables only (still have redo for indexes and undo) > Is what happens in NOARCHIVELOG mode Use nologging > only when you accept to loose data > and you accept to have some manual operations to do after recovery Recommended to take Backup at the end > Backup the unrecoverable tablespaces as soon as you can after your nologging operations Standby databases (physical or logical replication) > If you need redo for other goals (such as standby database) use force logging. Page 54

50 I/O mechanisms Demo: redo size On a 1MB table in NOARCHIVELOG > 1MB redo for conventional and direct-path insert > 2MB redo for delete (because undo generates redo) > No redo for /*+ append */ insert > With indexes more redo is generated > Updates that change values have lot of redo for indexes > Commit generates redo > Rollback generates redo Page 55

51 Tuning the Application > Storage clauses > Indexing > Partitioning > Result Cache > Materialized views > Star Transformation > Redo logging > Parallel Query Page 56

52 Parallel Query Parallelism In serial mode (default) > Each session is served by one process > The process is either: > Idle (SQL*Net message from client) > In CPU > Doing a system call (wait event) > Some I/O calls can be parallelized by the OS (async I/O) > But one session cannot use more than one CPU In parallel query > Several background processes can do some work > The session process coordinates them and run serial operations Parallel query gives all server resources to one session. Page 57

53 Parallel Query Parallelism A Degree Of Parallelism (DOP) is defined by the optimizer > Either manually > At session level with ALTER SESSION > At statement level with hints > At table/index level with PARALLEL clause > Or automatically > Depends on the number of CPUs > Several instances parameters to manage it > At execution time the DOP may be downgraded > Depends on the workload (available resources) > We can choose to downgrade or to queue Each Query can use 2xDOP processes > Some operations have one producer and one consumer Page 58

54 Parallel Query Configuration Parallel degree policy > AUTO: Auto DOP + Statement queuing + IM parallel query > LIMITED: only Auto DOP parallel_adaptive_multi_user boolean TRUE parallel_automatic_tuning boolean FALSE parallel_degree_level integer 100 parallel_degree_limit string CPU parallel_degree_policy string MANUAL parallel_execution_message_size integer parallel_force_local boolean FALSE parallel_instance_group string parallel_io_cap_enabled boolean FALSE parallel_max_servers integer 80 parallel_min_percent integer 0 parallel_min_servers integer 8 parallel_min_time_threshold string AUTO parallel_server_instances integer 1 parallel_servers_target integer 32 parallel_threads_per_cpu integer 2 Page 59

55 Parallel Query Configuration Message size > Was too low before 11g (was 2k, now 16k, can be increased to 32k) > Allocated in shared pool parallel_adaptive_multi_user boolean TRUE parallel_automatic_tuning boolean FALSE parallel_degree_level integer 100 parallel_degree_limit string CPU parallel_degree_policy string MANUAL parallel_execution_message_size integer parallel_force_local boolean FALSE parallel_instance_group string parallel_io_cap_enabled boolean FALSE parallel_max_servers integer 80 parallel_min_percent integer 0 parallel_min_servers integer 8 parallel_min_time_threshold string AUTO parallel_server_instances integer 1 parallel_servers_target integer 32 parallel_threads_per_cpu integer 2 Page 60

56 Parallel Query Configuration parallel_adaptive_multi_user boolean TRUE parallel_automatic_tuning boolean FALSE parallel_degree_level integer 100 parallel_degree_limit string CPU parallel_degree_policy string MANUAL parallel_execution_message_size integer parallel_force_local boolean FALSE parallel_instance_group string parallel_io_cap_enabled boolean FALSE parallel_max_servers integer 80 parallel_min_percent integer 0 parallel_min_servers integer 8 parallel_min_time_threshold string AUTO parallel_server_instances integer 1 parallel_servers_target integer 32 parallel_threads_per_cpu integer 2 > Default DOP: cpu_count x parallel_threads_per_cpu x instances Page 61

57 Parallel Query Parallel DML Parallel DML must be activated SQL> alter session enable parallel DML; > Reason: it locks the whole table Id Operation Name IN-OUT INSERT STATEMENT 1 PX COORDINATOR 2 PX SEND QC (RANDOM) :TQ10000 P->S 3 LOAD AS SELECT TEST PCWP 4 OPTIMIZER STATISTICS GATHERING PCWP 5 PX BLOCK ITERATOR PCWC 6 TABLE ACCESS FULL TEST PCWP > LOAD is below PX SEND when the DML is parallelized Page 62

58 Tuning the Application Core Message Lot of possibilities > Keep default when no reason to do otherwise > Know the possibilities before having to use them Enterprise Edition only? > Still can have good performance in SE when having good design Know the application behavior > DSS: bulk load, read only ad-hoc queries on large volume > OLTP: frequent and concurrent execution of same DML Any questions? Please do ask. Page 63