Oracle Database 11g Release 2 for SAP Advanced Compression Christoph Kersten Oracle Database for SAP Global Technology Center (Walldorf, Germany)
Implicit Compression <Insert Picture Here>
Efficient Use of Disk Space Disk Space Allocation (SAP ERP) Structured Data Database too large? Oracle saves disk space even without explicit compression Indexes Unstructured Data
Efficient Use of Disk Space Implicit Compression (Oracle Database 10g) Indexes Structured Data Secure Files Database too large? Oracle saves disk space even without explicit compression Internal Data Representation Variable length representation of number values Variable length representation of character strings Highly efficient bitmap index technology
Efficient Use of Disk Space Implicit Compression (Oracle Database 11g) Indexes Structured Data Secure Files Database too large? Oracle saves disk space even without explicit compression Internal Data Representation Variable length representation of number values Variable length representation of character strings Highly efficient bitmap index technology Object Space Allocation Deferred Segment Creation Dictionary Only Add Column
Explicit Compression Overview <Insert Picture Here>
Data/Index Compression Setting Realistic Expectations
Data/Index Compression Setting Realistic Expectations Type of Data % Disk Space Opt. Savings Min. Savings Index data 33% 23% 15% Structured data 55% 39% 25% Unstructured data 12% 8% 6% Overall 100% 70% 46% Assumptions Optimal Savings: - all objects (100%) compressed - compression ratio of 70% on object level - result: disk space savings on database level Average Savings: - largest objects only compressed (80% of disk space used for this type) - compression ratio of 60% on object level - result: disk space savings on database level
Index Key Compression <Insert Picture Here>
Persistent Bitmap Index Table Bitmap Index Key Bitmap <Blue: 1000100100010010100> <Green: 0001010000100100000> <Red: 0100000011000001001> <Yellow: 0010001000001000010>
Oracle Index Key Compression Example 1 A A A A A A A B B B B B B ROWID01 ROWID02 ROWID03 ROWID04 ROWID05 ROWID06 ROWID07 ROWID08 ROWID09 ROWID10 ROWID11 ROWID12 ROWID13
Oracle Index Key Compression Example 1 A A A A A A A B B B B B B ROWID01 ROWID02 ROWID03 ROWID04 ROWID05 ROWID06 ROWID07 ROWID08 ROWID09 ROWID10 ROWID11 ROWID12 ROWID13
Oracle Index Key Compression Example 2 1 A X 1 A 1 ROWID01 1 A X 1 B 2 ROWID02 1 A X 2 A 3 ROWID03 1 A Y 1 B 4 ROWID04 1 A Y 3 C 5 ROWID05 1 A Y 3 C 6 ROWID06 1 A Y 3 D 7 ROWID07 1 B X 1 A 1 ROWID08 1 B X 1 A 2 ROWID09 1 B X 1 C 3 ROWID10 1 B X 3 A 4 ROWID11 1 B X 3 C 5 ROWID12 1 B X 3 C 6 ROWID13
Oracle Index Key Compression Example 2: Prefix = 1 1 A X 1 A 1 ROWID01 A X 1 B 2 ROWID02 A X 2 A 3 ROWID03 12 values A Y 1 B 4 ROWID04 eliminated A Y 3 C 5 ROWID05 A Y 3 C 6 ROWID06 A Y 3 D 7 ROWID07 B X 1 A 1 ROWID08 B X 1 A 2 ROWID09 B X 1 C 3 ROWID10 B X 3 A 4 ROWID11 B X 3 C 5 ROWID12 B X 3 C 6 ROWID13 Prefix Suffix
Oracle Index Key Compression Example 2: Prefix = 2 1 A X 1 A 1 ROWID01 X 1 B 2 ROWID02 X 2 A 3 ROWID03 22 values Y 1 B 4 ROWID04 eliminated Y 3 C 5 ROWID05 Y 3 C 6 ROWID06 Y 3 D 7 ROWID07 1 B X 1 A 1 ROWID08 X 1 A 2 ROWID09 X 1 C 3 ROWID10 X 3 A 4 ROWID11 X 3 C 5 ROWID12 X 3 C 6 ROWID13 Prefix Suffix
Oracle Index Key Compression Example 2: Prefix = 3 1 A X 1 A 1 ROWID01 1 B 2 ROWID02 2 A 3 ROWID03 30 values 1 A Y 1 B 4 ROWID04 eliminated 3 C 5 ROWID05 3 C 6 ROWID06 3 D 7 ROWID07 1 B X 1 A 1 ROWID08 1 A 2 ROWID09 1 C 3 ROWID10 3 A 4 ROWID11 3 C 5 ROWID12 3 C 6 ROWID13 Prefix Suffix
Oracle Index Key Compression Example 2: Prefix = 4 1 A X 1 A 1 ROWID01 B 2 ROWID02 1 A X 2 A 3 ROWID03 28 values 1 A Y 1 B 4 ROWID04 eliminated 1 A Y 3 C 5 ROWID05 C 6 ROWID06 D 7 ROWID07 1 B X 1 A 1 ROWID08 A 2 ROWID09 C 3 ROWID10 1 B X 3 A 4 ROWID11 C 5 ROWID12 C 6 ROWID13 Prefix Suffix
Oracle Index Key Compression Example 2: Prefix = 5 1 A X 1 A 1 ROWID01 1 A X 1 B 2 ROWID02 1 A X 2 A 3 ROWID03 15 values 1 A Y 1 B 4 ROWID04 eliminated 1 A Y 3 C 5 ROWID05 6 ROWID06 1 A Y 3 D 7 ROWID07 1 B X 1 A 1 ROWID08 2 ROWID09 1 B X 1 C 3 ROWID10 1 B X 3 A 4 ROWID11 1 B X 3 C 5 ROWID12 6 ROWID13 Prefix Suffix
Oracle Index Key Compression Example 2: Prefix = 6 1 A X 1 A 1 ROWID01 1 A X 1 B 2 ROWID02 1 A X 2 A 3 ROWID03 0 values 1 A Y 1 B 4 ROWID04 eliminated 1 A Y 3 C 5 ROWID05 1 A Y 3 C 6 ROWID06 1 A Y 3 D 7 ROWID07 1 B X 1 A 1 ROWID08 1 B X 1 A 2 ROWID09 1 B X 1 C 3 ROWID10 1 B X 3 A 4 ROWID11 1 B X 3 C 5 ROWID12 1 B X 3 C 6 ROWID13 Prefix Suffix
Computing the Optimal Prefix Length
Efficient Use of Disk Space Index Key Compression Indexes Structured Data Secure Files Available in Oracle Database 10g and 11g Recommended as first step of compression implementation Best practices: Compress indexes on <n> largest tables Use utility IND_COMP (see SAP Note 1109743)
Efficient Use of Disk Space Index Key Compression Available in Oracle Database 10g and 11g Structured Data Recommended as first step of compression implementation Best practices: Compress indexes on <n> largest tables Indexes Free Space Secure Files Use utility IND_COMP (see SAP Note 1109743) Disk Space Savings: up to 75% on object level 15-20% on DB level
Table Compression <Insert Picture Here>
Data Compression Algorithm Step 1: Find Repeating Values 0001 0002 0003 0004 0005 0006 0007 0008 0009 0010 0011 0012 January Customer 1 500.00 January Customer 2 480.00 January Customer 3 360.00 January Customer 1 475.00 January Customer 2 600.00 January Customer 3 555.00 January Customer 1 200.00 February Customer 1 365.00 February Customer 3 510.00 February Customer 3 120.00 February Customer 2 800.00 February Customer 1 380.00
Data Compression Algorithm Step 2: Create Symbol Table 0001 January Customer 1 500.00 0002 0003 0004 0005 0006 0007 January January January January January January Customer 2 Customer 3 Customer 1 Customer 2 Customer 3 Customer 1 480.00 360.00 475.00 600.00 555.00 200.00 1 2 3 4 5 January February Customer 1 Customer 2 Customer 3 0008 February Customer 1 365.00 0009 February Customer 3 510.00 0010 February Customer 3 120.00 0011 February Customer 2 800.00 0012 February Customer 1 380.00
Data Compression Algorithm Step 3: Replace Values by References 0001 1 3 500.00 0002 0003 0004 0005 0006 0007 1 1 1 1 1 1 4 5 3 4 5 3 480.00 360.00 475.00 600.00 555.00 200.00 1 2 3 4 5 January February Customer 1 Customer 2 Customer 3 0008 2 3 365.00 0009 2 5 510.00 0010 2 5 120.00 0011 2 4 800.00 0012 2 3 380.00
Table Data Compression The Main Issue You cannot allow every single INSERT or UPDATE to modify the symbol table too much overhead for this process (performance problem) too much locking (concurrency problem) The Oracle and DB2 implementations address this issue in different ways different advantages different disadvantages
Table Data Compression Implementation Option 1 xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx Symbol Table Data Block Data Block Data Block One global symbol table per data segment (table), stored in dedicated blocks
Table Data Compression Implementation Option 2 Symbol Tables xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx Data Block Data Block Data Block Many local symbol tables, each of them built from data in a single block, and stored in that particular block
Oracle 10g Data Compression Benefits & Limits Benefits Unlimited number of compressed values All data required for decompression in one single block Compression ratio does not degrade due to new values No manual rebuild of symbol table required Limits Bulk load operations only (no single-record INSERT, UPDATE) Certified for SAP BI only
Efficient Use of Disk Space OLTP Compression (Adv. Compression Option) xxx xxxxxxx xxxxxxxx xxx xxx xxxxxxx xxxxxxxx xxx xxx xxxxxxx xxxxxxxx xxx Users insert new records These records are stored uncompressed This happens until a predefined threshold is reached xxx xxxxxxx xxxxxxxx xxx xxx xxxxxxx xxxxxxxx xxx xxx xxxxxxx xxxxxxxx xxx
Efficient Use of Disk Space OLTP Compression (Adv. Compression Option) xxx xxxxxxx xxxxxxxx xxx xxx xxxxxxx xxxxxxxx xxx xxx xxxxxxx xxxxxxxx xxx xxx xxxxxxxx xxxxxxxx x x xxx xxx xxxxxxxx xxxxxxxx x x xxx xxx xxxxxxxx xxxxxxxx x x xxx If the threshold is reached, data are compressed in batch mode The process scans the data and builds the symbol table in the block header compresses the records in this block After that, the block is again in phase 1 (available for new inserts)
Efficient Use of Disk Space Structured Table Data Compression Indexes Free Space Structured Data Secure Files Oracle Database 10g: Batch compression Works with batch loads only Supported for SAP BW Oracle Database 11g: Dynamic online compression (OLTP compression) Works for conventional DML (Insert, Update) Supported for all SAP applications Completely transparent to applications
Efficient Use of Disk Space Structured Table Data Compression Structured Data Indexes Secure Files Free Space Oracle Database 10g: Batch compression Works with batch loads only Supported for SAP BW Oracle Database 11g: Dynamic online compression (OLTP compression) Works for conventional DML (Insert, Update) Supported for all SAP applications Completely transparent to applications
SecureFiles Compression <Insert Picture Here>
SecureFiles Significantly faster access times compared to LOBs in SAP environments Increased transaction throughput on SAP cluster tables especially with RAC Prerequisite for compression of SAP tables containing LOBs (e.g. cluster tables) Performance Improvement Factor 2,5 2 1,5 1 0,5 0 SAP VBDATAThroughput (Insert/Read/Delete) 1 2 4 8 16 32 64 row size in KB LONGs (9.2,10.2) LOBs (10.2,11.2) Securefiles 11.2
SecureFiles Compression for SAP Part of Oracle Advanced Compression securefiles Compression for inline and out-of-line LOB data unique to Oracle for out-of-line LOBs (IBM can only do inline compression) out-of-line LOBs are more used than inline LOB data in SAP products such as SAP CRM SAP XI SAP Portal SAP R/3 (almost all non-cluster tables use out-of-line LOBs
Efficient Use of Disk Space Unstructured Table Data Compression Structured Data Indexes Secure Files Free Space Secure Files replaces previous data types (LONG, LOB) dramatically faster prerequisite for compression Secure Files Compression the only real compression feature no inlining of data required
Efficient Use of Disk Space Unstructured Table Data Compression Structured Data Indexes Secure Files Free Space Secure Files replaces previous data types (LONG, LOB) dramatically faster prerequisite for compression Secure Files Compression the only real compression feature no inlining of data required
Additional Compression Features <Insert Picture Here>
RMAN Backup Compression Compression levels are BASIC, LOW, MEDIUM and HIGH Each affords a trade off related to backup throughput and the degree of compression Run the following RMAN commands: RMAN> CONFIGURE DEVICE TYPE DISK BACKUP TYPE TO COMPRESSED BACKUPSET; RMAN> CONFIGURE COMPRESSION ALGORITHM <'HIGH', 'MEDIUM', 'LOW', 'BASIC'>; RMAN>BACKUP AS COMPRESSED BACKUPSET DATABASE; RMAN>BACKUP AS COMPRESSED BACKUPSET ARCHIVELOG ALL; RMAN>CONFIGURE COMPRESSION ALGORITHM 'ZLIB'; (ZLIB new 11.2 / BZIP2 <=11.2)
Data Guard Redo Transport Services Compress network traffic between primary and standby databases Lower bandwidth networks (<100Mbps) 15-35% less time required to transmit 1 GB of data Bandwidth consumption reduced up to 35% High bandwidth networks (>100 Mbps) Compression will not reduce transmission time But will reduce bandwidth consumption up to 35% Syntax: LOG_ARCHIVE_DEST_3='SERVICE=denver SYNC COMPRESSION=ENABLE [DISABLE]
Data Pump Compression Metadata compression available since Oracle Database 10g Oracle Database 11g extends compression to table data during exports No need to decompress before import COMPRESSION={ALL DATA_ONLY [METADATA_ONLY] NONE} Single step compression of both data and metadata Compressed data directly hits disk resulting in reduced disk space requirements Internal tests reduced dump file size up to 75% Application transparent Complete Data Pump functionality available on compressed files
Compression Benefits <Insert Picture Here>
Space Savings (Customer Production Databases) Database (fully reorged) ECC 6.0 BI 7.0 CRM 7.0 RMAN Backup (for compressed database) Export Data Pump Without Compression 4,782 GB 1,413 GB 950 GB With Compression and Other Space Optimizations 1,976 GB 543 GB 334 GB Factor Space Savings 2.4 2.6 2.8 2,817 GB 667 GB 4.2 973 GB 156 GB 6.2
Compression Storage vs. Performance? Disk Space Savings: Up to 75% disk space can be saved on object level DB size can be reduced by ca. 60% (avg.) Higher CPU consumption More than compensated by reduced I/O Overall throughput increased by 10-20%