Oracle Database 10g: Implement and Administer a Data Warehouse

Transcription

1 Oracle Database 10g: Implement and Administer a Data Warehouse Student Guide Volume 1 D18957GC10 Edition 1.0 November 2005 D22685

2 Authors Donna Keesling Jean Francois Verrier Jim Womack Technical Contributors and Reviewers Hermann Baer Herbert Bradbury Harald van Breederode Joel Goodman Paul Lane Jim Spiller Andreas Walter Mark van de Wiel Editors Arijit Ghosh Atanu Raychaudhuri Graphic Designers James Hans Samir Mozumdar Copyright 2005, Oracle. All rights reserved. Disclaimer This document contains proprietary information and is protected by copyright and other intellectual property laws. You may copy and print this document solely for your own use in an Oracle training course. The document may not be modified or altered in any way. Except where your use constitutes "fair use" under copyright law, you may not use, share, download, upload, copy, print, display, perform, reproduce, publish, license, post, transmit, or distribute this document in whole or in part without the express authorization of Oracle. The information contained in this document is subject to change without notice. If you find any problems in the document, please report them in writing to: Oracle University, 500 Oracle Parkway, Redwood Shores, California USA. This document is not warranted to be error-free. Restricted Rights Notice If this documentation is delivered to the United States Government or anyone using the documentation on behalf of the United States Government, the following notice is applicable: U.S. GOVERNMENT RIGHTS The U.S. Government s rights to use, modify, reproduce, release, perform, display, or disclose these training materials are restricted by the terms of the applicable Oracle license agreement and/or the applicable U.S. Government contract. Trademark Notice Oracle, JD Edwards and PeopleSoft are registered trademarks of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Publisher Giri Venugopal

3 Contents Preface 1 Data Warehouse Design Objectives 1-2 Characteristics of a Data Warehouse 1-3 Comparing OLTP and Data Warehouses 1-4 Data Warehouse Architectures 1-6 Data Warehouse Design 1-9 Logical Design 1-10 Oracle Warehouse Builder 1-12 Data Warehousing Schemas 1-13 Schema Characteristics 1-14 Data Warehousing Objects 1-16 Fact Tables 1-17 Dimensions and Hierarchies 1-18 Physical Design 1-20 Data Warehouse Physical Structures 1-21 Managing Large Volumes of Data 1-23 I/O Performance in Data Warehouses 1-24 Performance of Sequential I/Os 1-25 Minimizing I/O Requests 1-26 I/O Scalability 1-30 Typical Data Warehouse Cluster 1-35 Parallel Execution with RAC 1-36 Summary Data Warehouse Schemas Objectives 2-2 Star Schema Model 2-3 Snowflake Schema Model 2-5 Creating Dimensions 2-7 Creating Hierarchy Relationships 2-8 Third Normal Form Schemas 2-9 Tuning Star Queries 2-10 Star Transformation 2-11 Contrasting Star Transformation and Conventional Joins 2-12 Star Transformation Hints 2-13 Star Query: Example 2-14 Star Transformation: Rewrite Example 2-15 Star Transformation Execution Plan 2-17 Retrieving Fact Rows from One Dimension 2-18 Retrieving Fact Rows from All Dimensions 2-19 iii

4 Joining the Result Set with Dimensions 2-20 Star Transformation: Further Optimization 2-21 Bitmap Index Storage Performance 2-22 Static Partition Pruning and Star Query 2-23 Static Partition Pruning Plan: Example 2-24 Dynamic Partition Pruning and Star Query 2-25 Dynamic Partition Pruning Determination 2-26 Summary 2-27 Practice 2: Overview Partitioning Basics Objectives 3-2 Partitioned Tables and Indexes 3-3 Manageability: Rolling Window Operations 3-4 Benefits of Partitioning: Performance Considerations 3-5 Performance Consideration: Partition Pruning 3-6 Table Versus Index Partitioning 3-7 Partitioning Methods 3-8 CREATE TABLE Statement with Partitioning 3-10 Logical and Physical Attributes 3-11 Partitioning Type 3-12 Specifying Partition Attributes 3-13 Partition Key Value 3-14 Range Partitioning 3-15 Range Partitioning: Example 3-16 List Partitioning 3-17 Hash Partitioning, Named Partitions 3-18 Hash-Range Partitioning 3-19 Range-List Partitioning 3-20 Composite Partitioning Using a Template 3-21 Composite Partitioning and Template: Example 3-22 Partition Extended Table Names 3-23 Create Partitions with Enterprise Manager 3-24 Equipartitioning 3-25 Full Partitionwise Joins 3-26 Partial Partitionwise Joins 3-27 Summary 3-28 Practice 3: Overview Extraction, Transformation, and Loading (ETL): Extraction and Transportation Objectives 4-2 Overview 4-3 What Is ETL? 4-4 Extraction Methods 4-6 iv

5 Logical Extraction Methods 4-7 Physical Extraction Methods 4-9 Offline Extraction 4-10 Implementing Methods of Extraction 4-11 Incremental Extraction Using CDC 4-13 Publish and Subscribe Model 4-14 Synchronous CDC 4-18 Asynchronous CDC 4-19 Asynchronous AutoLog Mode 4-20 Asynchronous HotLog Configuration 4-22 Asynchronous Distributed HotLog Mode 4-23 Preparing to Publish Change Data 4-24 Creating a Publisher User 4-25 Synchronous Publishing 4-27 Asynchronous Distributed HotLog Publishing 4-29 Subscribing to Change Data 4-39 Asynchronous Distributed HotLog Source Database Initialization Parameters 4-43 Asynchronous Distributed HotLog Staging Database Initialization Parameters 4-45 Data Dictionary Views Supporting CDC 4-46 Transportation in a Data Warehouse 4-47 Transportable Tablespaces 4-48 Transportable Tablespaces: Example 4-50 Summary 4-52 Practice 4: Overview Extraction, Transformation, and Loading (ETL): Loading Objectives 5-2 Data-Loading Mechanisms 5-3 Loading Mechanisms 5-4 SQL*Loader: Example 5-5 Loading Mechanisms 5-6 Applications of External Tables 5-7 Example of Defining External Tables 5-9 Populating External Tables with Data Pump 5-11 Privileges for External Tables 5-12 Defining External Tables Using SQL*Loader 5-13 Data Dictionary Information for External Tables 5-14 Changing External Data Properties 5-16 Other Loading Methods 5-18 Summary 5-19 Practice 5: Overview Extraction, Transformation, and Loading (ETL): Transformation Objectives 6-2 Data Transformation 6-3 v

6 Multistage Data Transformation 6-4 Pipelined Data Transformation 6-5 Transformation Mechanisms 6-6 Transformation Using SQL 6-7 CREATE TABLE AS SELECT and UPDATE 6-8 MERGE Statement: Overview 6-10 Data Warehousing MERGE: Example 6-11 Data Maintenance with MERGE/DELETE 6-12 Overview of Multitable INSERT Statements 6-13 Example of Unconditional INSERT 6-14 Example of Conditional ALL INSERT 6-15 Example of Pivoting INSERT 6-16 Example of Conditional FIRST INSERT 6-17 Overview of Table Functions 6-18 Creating Table Functions 6-19 Using Table Functions 6-21 DML Error Logging: Overview 6-22 DML Error Logging Concepts 6-23 Error-Logging Table 6-25 Error-Logging Table Format 6-27 Inserting into a Table with Error Logging 6-28 Summary 6-29 Practice 6: Overview Parallelism Concepts Objectives 7-2 Introduction to Parallel Execution 7-3 Operations That Can Be Parallelized 7-4 How Parallel Execution Works 7-5 The Granule 7-6 Parallel Operations 7-7 Parallel Execution with Real Application Clusters 7-8 How Parallel Execution Servers Communicate 7-9 Degree of Parallelism (DOP) 7-10 Default Degree of Parallelism 7-11 Parallel Execution Plan 7-12 OTHER_TAG Column 7-13 Serial and Parallel Execution Plans 7-14 Parallel Plan Interpretation 7-15 Parallel Execution Server Pool 7-20 Minimum Number of Parallel Execution Servers 7-21 Object s PARALLEL Clause 7-22 PARALLEL Clause: Examples 7-23 vi

7 Enabling Parallel DML/DDL/QUERY 7-24 Using Parallelization Hints 7-26 Parallelization Rules 7-27 Summary 7-28 Practice 7: Overview Parallel Operations in Data Warehouses Objectives 8-2 Parallel Query 8-3 Parallel Partitioned Table Scan 8-4 Parallel Partitioned Index Scan 8-5 Partitionwise Joins 8-6 Parallel DDL 8-7 Creating Indexes in Parallel 8-8 Parallel DDL: Example 8-11 Parallel DML: Overview 8-12 Performance Benefits of Parallel DML 8-13 Enabling Parallel DML 8-14 Parallel DML: Example 8-15 When to Use Parallel DML 8-16 Restrictions on Parallel DML 8-18 Tuning Parameters for Parallel Execution 8-19 Using Default Parameter Settings 8-20 Balancing the Workload 8-21 Adaptive Multiuser and DOP 8-23 Resource Manager and the DOP 8-24 Are There Execution Problems? 8-26 Data Distribution and V$PQ_TQSTAT 8-27 Using Other Dynamic Performance Views 8-29 Using V$PX_SESSION 8-31 Using V$PX_SESSTAT 8-32 Using V$PX_PROCESS 8-33 Using V$SYSSTAT 8-34 Tuning PDML 8-36 Summary 8-37 Practice 8: Overview Materialized Views Objectives 9-2 The Need for Summary Management 9-3 Full Notes Page 9-4 Using Summaries to Improve Performance 9-5 Using Summaries 9-6 Using Materialized Views for Summary Management 9-7 vii

8 Summary Management Components 9-9 Using Summary Management 9-10 How Many Materialized Views? 9-11 One Materialized View for Multiple Queries 9-12 Determining Which Materialized View to Create 9-15 Types of Materialized Views 9-16 The Need for Nested Materialized Views 9-17 Using Nested Materialized Views 9-18 Nested Materialized Views: Restrictions 9-19 Materialized View: Example 9-20 Materialized Views Storage 9-21 Estimating Materialized View Size 9-22 Specifying Build Methods 9-23 Specifying Refresh Options 9-24 Specifying Refresh Execution Modes 9-25 Using Column Aliases in Materialized Views 9-26 Using Materialized View Column Alias Lists 9-27 Registering User-Defined Materialized View Tables 9-28 Partitioning and Materialized Views 9-29 Partitioned Materialized View: Example 9-30 Using Enterprise Manager to Create Materialized Views 9-31 Privileges Required to Create Materialized Views 9-32 Additional Privileges Required to Create Materialized Views 9-33 Globalization and Materialized Views 9-34 Globalization Parameters Significant for Materialized Views 9-35 Adding Comments to Materialized Views 9-36 Altering Materialized Views 9-37 Maintaining Partitions of a Materialized View 9-39 Dropping Materialized Views 9-40 Viewing Staleness Information 9-41 Materialized View Integrity 9-42 Invalidating Materialized Views 9-43 Summary 9-44 Practice 9: Overview Dimensions Objectives 10-2 What Are Dimensions? 10-3 Dimensions and Hierarchies 10-4 Dimension Table: Example 10-6 Benefits of Dimensions 10-7 Defining Dimensions and Hierarchies 10-8 Specifying the SKIP WHEN NULL Clause 10-9 Creating Dimensions Using Enterprise Manager viii

9 Dimensions Based on Multiple Tables Dimensions with Multiple Hierarchies Dimensions and Privileges Dimension Restrictions Using VALIDATE_DIMENSION to Verify Relationships in a Dimension Verifying Relationships in a Dimension Viewing Dimensions Using Enterprise Manager Viewing Dimensions in the Data Dictionary Viewing the Definition of a Dimension Dimension Invalidation Dropping Dimensions Selectively Dropping Dimension Components Constraints or Dimensions? Summary Practice 10: Overview Refreshing Materialized Views Objectives 11-2 Refresh Options 11-3 Refresh Modes 11-5 Refresh at Scheduled Time 11-6 Manual Refresh Using DBMS_MVIEW 11-7 Refresh Specific Materialized Views with REFRESH 11-9 Refresh All Materialized Views with REFRESH_ALL_MVIEWS Refresh Dependent Materialized Views with REFRESH_DEPENDENT Finding Dependent Materialized Views When Does a Complete Refresh Occur? Conditions That Effect Possibility of Fast Refresh Explain Materialized View Using DBMS_MVIEW.EXPLAIN_MVIEW Materialized View Logs: General Concepts Creating a Materialized View Log Restrictions for Fast-Refreshable Materialized Views Indexes and Fast-Refreshable Materialized Views Specifying REFRESH FORCE Partition Change Tracking (PCT) Refresh PCT Refresh Requirements When Is PCT Refresh Used? Partition Key or Partition Marker? PCT Using ROWID Consider Fresh: Fastest Refresh Scenario CONSIDER FRESH Refresh Implications General Tips for Materialized View Refresh Refresh Tips for Nested Materialized Views ix

10 Summary Practice 11: Overview Query Rewrite Objectives 12-2 Query Rewrite: Overview 12-3 Cost-Based Query Rewrite Process 12-4 What Can Be Rewritten? 12-5 Setting Initialization Parameters for Query Rewrite 12-6 Using Trusted Constraints 12-8 Enabling Query Rewrite 12-9 Query Rewrite Requirements Query Rewrite Hints Did Query Rewrite Occur? Verifying That Query Rewrite Occurred Using DBMS_MVIEW.EXPLAIN_REWRITE Query Rewrite Methods Text Match Rewrite Methods Expression Matching Query Rewrite Expression Matching: Example General Query Rewrite Methods Partition Change Tracking (PCT) and Query Rewrite PCT and Query Rewrite: Example Query Rewrite and Bind Variables Creating Indexes for Materialized Views Query Rewrite Restrictions Summary Practice 12: Overview Using the SQL Access Advisor Objectives 13-2 What Does the SQL Access Advisor Do? 13-3 SQL Access Advisor: Overview 13-4 SQL Access Advisor: Usage Model 13-6 Possible Recommendations 13-7 Using the SQL Access Advisor 13-8 SQL Access Advisor Repository 13-9 Privileges Needed to Use the SQL Access Advisor Invoking the SQL Access Advisor Wizard Setting Initial Options Specifying the Workload Source Specifying Recommendation Options Viewing Results Reviewing Recommendations x

11 Using the DBMS_ADVISOR Package SQL Access Advisor Procedure Flow Creating a Task Setting SQL Access Advisor Parameters Using Templates Setting Template Parameters Using a Template to Create a Task Defining the Workload Defining Workload Contents Importing a SQL Tuning Set Loading User-Defined Workloads Loading SQL Cache Workloads Using Hypothetical Workloads Adding SQL Statements to a Workload Removing Workloads Linking the Workload to a Task Generating Recommendations Viewing and Implementing Recommendations SQL Access Advisor Actions Generating SQL Scripts Performing a Quick Tune Tuning Materialized Views for Fast Refresh and Query Rewrite Using DBMS_ADVISOR.TUNE_MVIEW TUNE_MVIEW Output Results Implementation Process Output Undo Process Output Accessing TUNE_MVIEW Output Results Summary Practice 13: Overview System Management Tasks and Techniques Objectives 14-2 Are There Execution Problems? 14-3 Performance Monitoring and Tuning 14-4 Statistics Collection 14-5 End to End Application Tracing 14-6 End to End Application Tracing Using EM 14-7 Compression 14-8 What Can Be Compressed? Compression: Examples Maximizing Compression Ratios Resumable Sessions Data Warehouse Backup and Recovery Recovery Time Objective (RTO) xi

12 Recovery Point Objective (RPO) More Backup and Recovery Considerations Backup and Recovery Tools Backup and Recovery: Best Practices NOLOGGING Backup and Recovery Strategies Employing Incremental Backups Data Warehouse Security Layered Security Model Transparent Data Encryption Implementing TDE Summary Practice 14: Overview Appendix A: Practices Appendix B: Solutions xii