Resource Optimization

Transcription

1 Resource Optimization Compression Space Reclamation Scan Sharing Tim Vincent DB2 LUW Chief Architect June 18, 2010

2 Lower Storage Costs with Deep Compression With DB2 9, we re seeing compression rates up to 83% on the Data Warehouse. The projected cost savings are more than $2 million initially with ongoing savings of $500,000 a year. - Michael Henson We are saving anywhere between 60 to 65% in storage and we ve actually found the performance has improved - Bashir Khan Best in industry Minimize storage costs Improve performance Easy to implement Advances with Index compression Temp space compression XML compression 1.5 Times Better 3.3 Times Better 2.0 Times Better 8.7 Times Better DB2 9 Other 2

3 How Compression Works Compression looks for repeating patterns across the entire table When a pattern is found, string is replaced with 12-bit symbol Symbols are stored in a dictionary for fast lookup Data resides compressed on pages (both on-disk and in memory) Significant I/O bandwidth savings better performance Significant memory savings more efficient memory utilization Name Dept Salary City Province Postal_Code Zikopoulos Whitby ONT L4N5R4 Dictionary 01 opoulos 02 WhitbyONTL4N5R4 Katsopoulos Whitby ONT L4N5R4 Zikopoulos Whitby ONT L4N5R4 Katsopoulos Whitby ONT L4N5R4 Unique to DB2 Zik (01) (02) Kats (01) (02) 3

4 Compression Improvements Multiple algorithms for automatic index compression Unique in the industry Automatic compression for temporary tables Table Temp Table Temp Unique in the industry Order By Order By Compression of large objects and XML Replication of Compressed Tables Log db2readlog API Dictionary Compressed user data in logs Uncompressed user data in logs IBM Corporation

5 Index Compression Algorithms implemented by the Database Engine (under-the-covers): RID List Compression, Prefix Compression, and variable slot directory Applies to all indexes except: Catalog indexes, MDC block indexes, XML path indexes and meta indexes, Index specifications Activated: When row compression is activated on a table CREATE INDEX with the new COMPRESS YES option ALTER INDEX COMPRESS [YES NO] statement, followed by an index reorg Savings ADMIN_GET_INDEX_COMPRESS_INFO to estimate compression savings for uncompressed index COMPRESS and PCTPAGESSAVED in the SYSINDEXES catalog show if an index is defined as compressed and the percentage saved respectively IBM Corporation

6 I n d e x C o m p r e s s i o n E a r l y C u s t o m e r R e s u l t s S a v in g s D B E a rly C u s to m e r D a ta b a s e S iz e D a ta C o m p r e s s io n R a tio In d e x C o m p re s s io n R a tio T o ta l D a ta b a s e S a v in g W o rld le a d in g c o n s tru c tio n m a c h in e ry m a n u fa c tu re r, U S A G B 7 2 % 4 9 % 6 8 % G lo b a l c o n s u m e r a n d c o m m e rc ia l p ro d u c t m a rk e te r, U S A 1.4 T B 5 8 % 4 9 % 5 6 % H a ie r G ro u p, C h in a 5 2 % J o h n D e e re, C h in a 5 8 % E n e rg y d e liv e ry c o m p a n y, U S A 6 2 G B 5 2 % In s u ra n c e c o m p a n y, G e rm a n y G B 5 0 % T -S ys te m s, G e rm a n y G B 6 0 % 7 3 % 6 5 % M e d tro n ic, U S A 3.6 T B 6 5 % E R P S y s te m s B W S y s te m s IB M P age 7 SA P IBM Corporation

7 Temp Table Compression Compression of temporary tables aims to: Reduce the amount of temporary disk space required Have no performance penalty as a result of the extra processing required for row compression. Applicable to User temporary tables and System temps (DGTT/CGTT) Sorts, MGJN,NLJN, utilities, If Deep Compression is licensed, then temporary tables will be compressed by default. There is no additional action required by the user in order to use it. DB2 will evaluate the query and apply compression where appropriate. db2pd will report on temp tablespace usage IBM Corporation

8 Deep Compression: Warehouse Results Customer POC Compressed data from 15.3 to 7.9 TB Table compression rates were between 80-85% Aggregate Build throughput improved 15% Query Response time decreased 23% 8 8

9 Deep Compression: Warehouse Results System CPU decreased from 16.5% to 12.3% Wait time decreased from 23.9% to 5.7% User CPU increased from 30.7% to 53% BUT combination of: Increased throughput on aggregate build (15%) Decreased response time (23%) Reduction in wait time Compress/uncompress 9 9

10 Temp Compression: Measurements Space Savings for TPC-DS Queries with Temp Compression Elapsed Time for TPC-DS Queries with Temp Compression Size (Gigabytes) % less space Minutes % Faster 0.0 Without Temp Comp Total Bytes Stored With Temp Comp Bytes Stored Without Temp Comp Runtime With Temp Comp Runtime * Lower is better * Lower is better TPC-DS CPU Analysis for Temp Compression Effective CPU Usage I/O Wait User CPU Baseline Index Com pression 10 10

11 Simple Space Reclamation New tablespace format to allow automated extent remapping Allow extents that are not assigned to any object (eg. table, index) to be used by other tablespaces ALTER TABLESPACE REDUCE XXX MAX All new tablespaces will have this format Storage in an MDC table is tracked through a block map which extents have data and which don t When a block is emptied the storage remains with the table and is available for later reuse by that table New option on reorg table command to not reorg the table but reclaim these empty blocks/extents REORG TABLE <mdc table> RECLAIM EXTENTS ON [table partition clause] ALLOW WRITE ACCESS ALLOW READ ACCESS ALLOW NOACCESS 11 11

12 DB2 Compresses all Aspects of the Database Percentage of Original Database Size 15% 30% 55% 1x 1.2x? 2x? Oracle Exadata Compression Range 1x 1.5x? 3x? 1.5x? 2x? 10x? DB2 Compression Range 2.5x 3x 4x 3x 4x 7x 12 12

13 Automatic Storage Migration Support ALTER DATABASE command for non-auto AS database Allow existing tablespaces to grow into auto storage containers ALTER TABLESPACE <table_space_name> MANAGED BY AUTOMATIC STORAGE Existing containers can no longer be altered. Support redirected tablespace restore to AS tablespace RESTORE DB <dbname> REDIRECT SET TABLESPACE CONTAINERS FOR <tablespaceid> USING AUTOMATIC STORAGE REBALANCE support after a new path is added to the database Allows existing tablespaces to use new path Ability to DROP a path from an automatic storage database. Can be used to migrate to new containers 13 13

14 Scan Sharing for DB2 Scan Sharing Performance Test TPCH Q1 : CPU Intensive, Slow Query On Lineitem Table Using A Table Scan TPCH Q6 : IO Intensive, Fast Query On Lineitem Table Using A Table Scan Test Scenario : Queries executed in parallel in the following sequence Q1 30 Secs Q6 60 Secs 90 Secs Q1 Q6 Results : 34% Improvement In End to End Timing C um ilat iv e R eads M illions Reads on a disk: 42% Reduction Scan Sharing Base % Tim e S pe nt CPU Usage Base ScanSharing 0 Tim e 0 User System Idle IO Wait IBM Corporation IBM Confidential