Oracle Database 11g Release 2 for SAP Advanced Compression. Christoph Kersten Oracle Database for SAP Global Technology Center (Walldorf, Germany)

Transcription

1 Oracle Database 11g Release 2 for SAP Advanced Compression Christoph Kersten Oracle Database for SAP Global Technology Center (Walldorf, Germany)

2 Implicit Compression <Insert Picture Here>

3 Efficient Use of Disk Space Disk Space Allocation (SAP ERP) Structured Data Database too large? Oracle saves disk space even without explicit compression Indexes Unstructured Data

4 Efficient Use of Disk Space Implicit Compression (Oracle Database 10g) Indexes Structured Data Secure Files Database too large? Oracle saves disk space even without explicit compression Internal Data Representation Variable length representation of number values Variable length representation of character strings Highly efficient bitmap index technology

5 Efficient Use of Disk Space Implicit Compression (Oracle Database 11g) Indexes Structured Data Secure Files Database too large? Oracle saves disk space even without explicit compression Internal Data Representation Variable length representation of number values Variable length representation of character strings Highly efficient bitmap index technology Object Space Allocation Deferred Segment Creation Dictionary Only Add Column

6 Explicit Compression Overview <Insert Picture Here>

7 Data/Index Compression Setting Realistic Expectations

8 Data/Index Compression Setting Realistic Expectations Type of Data % Disk Space Opt. Savings Min. Savings Index data 33% 23% 15% Structured data 55% 39% 25% Unstructured data 12% 8% 6% Overall 100% 70% 46% Assumptions Optimal Savings: - all objects (100%) compressed - compression ratio of 70% on object level - result: disk space savings on database level Average Savings: - largest objects only compressed (80% of disk space used for this type) - compression ratio of 60% on object level - result: disk space savings on database level

9 Index Key Compression <Insert Picture Here>

10 Persistent Bitmap Index Table Bitmap Index Key Bitmap <Blue: > <Green: > <Red: > <Yellow: >

11 Oracle Index Key Compression Example 1 A A A A A A A B B B B B B ROWID01 ROWID02 ROWID03 ROWID04 ROWID05 ROWID06 ROWID07 ROWID08 ROWID09 ROWID10 ROWID11 ROWID12 ROWID13

12 Oracle Index Key Compression Example 1 A A A A A A A B B B B B B ROWID01 ROWID02 ROWID03 ROWID04 ROWID05 ROWID06 ROWID07 ROWID08 ROWID09 ROWID10 ROWID11 ROWID12 ROWID13

13 Oracle Index Key Compression Example 2 1 A X 1 A 1 ROWID01 1 A X 1 B 2 ROWID02 1 A X 2 A 3 ROWID03 1 A Y 1 B 4 ROWID04 1 A Y 3 C 5 ROWID05 1 A Y 3 C 6 ROWID06 1 A Y 3 D 7 ROWID07 1 B X 1 A 1 ROWID08 1 B X 1 A 2 ROWID09 1 B X 1 C 3 ROWID10 1 B X 3 A 4 ROWID11 1 B X 3 C 5 ROWID12 1 B X 3 C 6 ROWID13

14 Oracle Index Key Compression Example 2: Prefix = 1 1 A X 1 A 1 ROWID01 A X 1 B 2 ROWID02 A X 2 A 3 ROWID03 12 values A Y 1 B 4 ROWID04 eliminated A Y 3 C 5 ROWID05 A Y 3 C 6 ROWID06 A Y 3 D 7 ROWID07 B X 1 A 1 ROWID08 B X 1 A 2 ROWID09 B X 1 C 3 ROWID10 B X 3 A 4 ROWID11 B X 3 C 5 ROWID12 B X 3 C 6 ROWID13 Prefix Suffix

15 Oracle Index Key Compression Example 2: Prefix = 2 1 A X 1 A 1 ROWID01 X 1 B 2 ROWID02 X 2 A 3 ROWID03 22 values Y 1 B 4 ROWID04 eliminated Y 3 C 5 ROWID05 Y 3 C 6 ROWID06 Y 3 D 7 ROWID07 1 B X 1 A 1 ROWID08 X 1 A 2 ROWID09 X 1 C 3 ROWID10 X 3 A 4 ROWID11 X 3 C 5 ROWID12 X 3 C 6 ROWID13 Prefix Suffix

16 Oracle Index Key Compression Example 2: Prefix = 3 1 A X 1 A 1 ROWID01 1 B 2 ROWID02 2 A 3 ROWID03 30 values 1 A Y 1 B 4 ROWID04 eliminated 3 C 5 ROWID05 3 C 6 ROWID06 3 D 7 ROWID07 1 B X 1 A 1 ROWID08 1 A 2 ROWID09 1 C 3 ROWID10 3 A 4 ROWID11 3 C 5 ROWID12 3 C 6 ROWID13 Prefix Suffix

17 Oracle Index Key Compression Example 2: Prefix = 4 1 A X 1 A 1 ROWID01 B 2 ROWID02 1 A X 2 A 3 ROWID03 28 values 1 A Y 1 B 4 ROWID04 eliminated 1 A Y 3 C 5 ROWID05 C 6 ROWID06 D 7 ROWID07 1 B X 1 A 1 ROWID08 A 2 ROWID09 C 3 ROWID10 1 B X 3 A 4 ROWID11 C 5 ROWID12 C 6 ROWID13 Prefix Suffix

18 Oracle Index Key Compression Example 2: Prefix = 5 1 A X 1 A 1 ROWID01 1 A X 1 B 2 ROWID02 1 A X 2 A 3 ROWID03 15 values 1 A Y 1 B 4 ROWID04 eliminated 1 A Y 3 C 5 ROWID05 6 ROWID06 1 A Y 3 D 7 ROWID07 1 B X 1 A 1 ROWID08 2 ROWID09 1 B X 1 C 3 ROWID10 1 B X 3 A 4 ROWID11 1 B X 3 C 5 ROWID12 6 ROWID13 Prefix Suffix

19 Oracle Index Key Compression Example 2: Prefix = 6 1 A X 1 A 1 ROWID01 1 A X 1 B 2 ROWID02 1 A X 2 A 3 ROWID03 0 values 1 A Y 1 B 4 ROWID04 eliminated 1 A Y 3 C 5 ROWID05 1 A Y 3 C 6 ROWID06 1 A Y 3 D 7 ROWID07 1 B X 1 A 1 ROWID08 1 B X 1 A 2 ROWID09 1 B X 1 C 3 ROWID10 1 B X 3 A 4 ROWID11 1 B X 3 C 5 ROWID12 1 B X 3 C 6 ROWID13 Prefix Suffix

20 Computing the Optimal Prefix Length

21 Efficient Use of Disk Space Index Key Compression Indexes Structured Data Secure Files Available in Oracle Database 10g and 11g Recommended as first step of compression implementation Best practices: Compress indexes on <n> largest tables Use utility IND_COMP (see SAP Note )

22 Efficient Use of Disk Space Index Key Compression Available in Oracle Database 10g and 11g Structured Data Recommended as first step of compression implementation Best practices: Compress indexes on <n> largest tables Indexes Free Space Secure Files Use utility IND_COMP (see SAP Note ) Disk Space Savings: up to 75% on object level 15-20% on DB level

23 Table Compression <Insert Picture Here>

24 Data Compression Algorithm Step 1: Find Repeating Values January Customer January Customer January Customer January Customer January Customer January Customer January Customer February Customer February Customer February Customer February Customer February Customer

25 Data Compression Algorithm Step 2: Create Symbol Table 0001 January Customer January January January January January January Customer 2 Customer 3 Customer 1 Customer 2 Customer 3 Customer January February Customer 1 Customer 2 Customer February Customer February Customer February Customer February Customer February Customer

26 Data Compression Algorithm Step 3: Replace Values by References January February Customer 1 Customer 2 Customer

27 Table Data Compression The Main Issue You cannot allow every single INSERT or UPDATE to modify the symbol table too much overhead for this process (performance problem) too much locking (concurrency problem) The Oracle and DB2 implementations address this issue in different ways different advantages different disadvantages

28 Table Data Compression Implementation Option 1 xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx Symbol Table Data Block Data Block Data Block One global symbol table per data segment (table), stored in dedicated blocks

29 Table Data Compression Implementation Option 2 Symbol Tables xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx xxxx xxxxx Data Block Data Block Data Block Many local symbol tables, each of them built from data in a single block, and stored in that particular block

30 Oracle 10g Data Compression Benefits & Limits Benefits Unlimited number of compressed values All data required for decompression in one single block Compression ratio does not degrade due to new values No manual rebuild of symbol table required Limits Bulk load operations only (no single-record INSERT, UPDATE) Certified for SAP BI only

31 Efficient Use of Disk Space OLTP Compression (Adv. Compression Option) xxx xxxxxxx xxxxxxxx xxx xxx xxxxxxx xxxxxxxx xxx xxx xxxxxxx xxxxxxxx xxx Users insert new records These records are stored uncompressed This happens until a predefined threshold is reached xxx xxxxxxx xxxxxxxx xxx xxx xxxxxxx xxxxxxxx xxx xxx xxxxxxx xxxxxxxx xxx

32 Efficient Use of Disk Space OLTP Compression (Adv. Compression Option) xxx xxxxxxx xxxxxxxx xxx xxx xxxxxxx xxxxxxxx xxx xxx xxxxxxx xxxxxxxx xxx xxx xxxxxxxx xxxxxxxx x x xxx xxx xxxxxxxx xxxxxxxx x x xxx xxx xxxxxxxx xxxxxxxx x x xxx If the threshold is reached, data are compressed in batch mode The process scans the data and builds the symbol table in the block header compresses the records in this block After that, the block is again in phase 1 (available for new inserts)

33 Efficient Use of Disk Space Structured Table Data Compression Indexes Free Space Structured Data Secure Files Oracle Database 10g: Batch compression Works with batch loads only Supported for SAP BW Oracle Database 11g: Dynamic online compression (OLTP compression) Works for conventional DML (Insert, Update) Supported for all SAP applications Completely transparent to applications

34 Efficient Use of Disk Space Structured Table Data Compression Structured Data Indexes Secure Files Free Space Oracle Database 10g: Batch compression Works with batch loads only Supported for SAP BW Oracle Database 11g: Dynamic online compression (OLTP compression) Works for conventional DML (Insert, Update) Supported for all SAP applications Completely transparent to applications

35 SecureFiles Compression <Insert Picture Here>

36 SecureFiles Significantly faster access times compared to LOBs in SAP environments Increased transaction throughput on SAP cluster tables especially with RAC Prerequisite for compression of SAP tables containing LOBs (e.g. cluster tables) Performance Improvement Factor 2,5 2 1,5 1 0,5 0 SAP VBDATAThroughput (Insert/Read/Delete) row size in KB LONGs (9.2,10.2) LOBs (10.2,11.2) Securefiles 11.2

37 SecureFiles Compression for SAP Part of Oracle Advanced Compression securefiles Compression for inline and out-of-line LOB data unique to Oracle for out-of-line LOBs (IBM can only do inline compression) out-of-line LOBs are more used than inline LOB data in SAP products such as SAP CRM SAP XI SAP Portal SAP R/3 (almost all non-cluster tables use out-of-line LOBs

38 Efficient Use of Disk Space Unstructured Table Data Compression Structured Data Indexes Secure Files Free Space Secure Files replaces previous data types (LONG, LOB) dramatically faster prerequisite for compression Secure Files Compression the only real compression feature no inlining of data required

39 Efficient Use of Disk Space Unstructured Table Data Compression Structured Data Indexes Secure Files Free Space Secure Files replaces previous data types (LONG, LOB) dramatically faster prerequisite for compression Secure Files Compression the only real compression feature no inlining of data required

40 Additional Compression Features <Insert Picture Here>

41 RMAN Backup Compression Compression levels are BASIC, LOW, MEDIUM and HIGH Each affords a trade off related to backup throughput and the degree of compression Run the following RMAN commands: RMAN> CONFIGURE DEVICE TYPE DISK BACKUP TYPE TO COMPRESSED BACKUPSET; RMAN> CONFIGURE COMPRESSION ALGORITHM <'HIGH', 'MEDIUM', 'LOW', 'BASIC'>; RMAN>BACKUP AS COMPRESSED BACKUPSET DATABASE; RMAN>BACKUP AS COMPRESSED BACKUPSET ARCHIVELOG ALL; RMAN>CONFIGURE COMPRESSION ALGORITHM 'ZLIB'; (ZLIB new 11.2 / BZIP2 <=11.2)

42 Data Guard Redo Transport Services Compress network traffic between primary and standby databases Lower bandwidth networks (<100Mbps) 15-35% less time required to transmit 1 GB of data Bandwidth consumption reduced up to 35% High bandwidth networks (>100 Mbps) Compression will not reduce transmission time But will reduce bandwidth consumption up to 35% Syntax: LOG_ARCHIVE_DEST_3='SERVICE=denver SYNC COMPRESSION=ENABLE [DISABLE]

43 Data Pump Compression Metadata compression available since Oracle Database 10g Oracle Database 11g extends compression to table data during exports No need to decompress before import COMPRESSION={ALL DATA_ONLY [METADATA_ONLY] NONE} Single step compression of both data and metadata Compressed data directly hits disk resulting in reduced disk space requirements Internal tests reduced dump file size up to 75% Application transparent Complete Data Pump functionality available on compressed files

44 Compression Benefits <Insert Picture Here>

45 Space Savings (Customer Production Databases) Database (fully reorged) ECC 6.0 BI 7.0 CRM 7.0 RMAN Backup (for compressed database) Export Data Pump Without Compression 4,782 GB 1,413 GB 950 GB With Compression and Other Space Optimizations 1,976 GB 543 GB 334 GB Factor Space Savings ,817 GB 667 GB GB 156 GB 6.2

46 Compression Storage vs. Performance? Disk Space Savings: Up to 75% disk space can be saved on object level DB size can be reduced by ca. 60% (avg.) Higher CPU consumption More than compensated by reduced I/O Overall throughput increased by 10-20%

47