Oracle9i Database: Data Mining Techniques

Size: px

Start display at page:

Download "Oracle9i Database: Data Mining Techniques"

Marsha Ward
5 years ago
Views:

1 Oracle9i Database: Data Mining Techniques Volume 1 - Student Guide D16245GC10 Edition 1.0 September 2003 D38053

2 Author Ramaa Mani Technical Contributors and Reviewers Gina Abeles Charlie Berger Franck Boss Mary Bryksa Winfred Deering Robert Haberstroh Rosita Hanoman Steven Kahn Vincent Kirkwood Jayne Obrien Patrick Odell Joyce Peng Vijay Pillai Joseph Rayman Raymond Roccaforte Richard Solari Robert Stackowiak Susie Stephens Publisher Copyright 2003, Oracle. All rights reserved. This documentation contains proprietary information of Oracle Corporation. It is provided under a license agreement containing restrictions on use and disclosure and is also protected by copyright law. Reverse engineering of the software is prohibited. If this documentation is delivered to a U.S. Government Agency of the Department of Defense, then it is delivered with Restricted Rights and the following legend is applicable: Restricted Rights Legend Use, duplication or disclosure by the Government is subject to restrictions for commercial computer software and shall be deemed to be Restricted Rights software under Federal law, as set forth in subparagraph (c)(1)(ii) of DFARS , Rights in Technical Data and Computer Software (October 1988). This material or any portion of it may not be copied in any form or by any means without the express prior written permission of Oracle Corporation. Any other copying is a violation of copyright law and may result in civil and/or criminal penalties. If this documentation is delivered to a U.S. Government Agency not within the Department of Defense, then it is delivered with Restricted Rights, as defined in FAR , Rights in Data-General, including Alternate III (June 1987). The information in this document is subject to change without notice. If you find any problems in the documentation, please report them in writing to Education Products, Oracle Corporation, 500 Oracle Parkway, Box SB-6, Redwood Shores, CA Oracle Corporation does not warrant that this document is error-free. All references to Oracle and Oracle products are trademarks or registered trademarks of Oracle Corporation. All other products or company names are used for identification purposes only, and may be trademarks of their respective owners. Glenn Austin

3 Contents Preface 1 Overview of Oracle9i Data Mining Concepts Objectives 1-2 Overview 1-3 The Role of Data Mining 1-4 Data Mining Concepts 1-5 Data Mining Examples 1-6 Oracle9i Data Mining Architecture 1-8 Overview of the Data Mining Process 1-9 Data Mining Models 1-10 Classification 1-11 Clustering 1-13 Association Rules 1-14 Attribute Importance 1-16 Implementing Data Mining 1-17 Data Mining Phase 1: Defining the Problem 1-18 Data Mining Phase 2: Restating the Problem 1-19 Restating the Problem in Terms of the Available Data 1-20 Data Mining Phase 3: Data Preparation 1-21 Preparing the Data 1-22 Data Mining Phase 4: Building the Model 1-24 Data Mining Phase 4: Testing the Model 1-25 Building and Testing the Model 1-26 Data Mining Phase 5: Deploying the Model 1-28 Data Mining Phase 6: Applying the Model 1-30 Summary Configuration Requirements Objectives 2-2 Oracle9i Data Mining Infrastructure 2-3 Introducing JDeveloper 2-5 Working with JDeveloper 2-6 Creating a New Project Workspace 2-7 Working with JDeveloper 2-10 Establishing a Connection to the Database 2-11 Opening the Database Connection 2-16 iii

4 Starting ODM 2-17 Data Mining for Java (DM4J) 2-21 Viewing Data Mining Components 2-23 Generating DM4J Components 2-24 DM4J Component Management 2-25 Oracle Data Mining Browser 2-26 Viewing ODM Browser 2-27 Summary Data Mining Tasks Objectives 3-2 Data Mining Tasks 3-3 Model Building 3-4 Testing the Model 3-5 Computing Lift 3-6 Applying the Model 3-8 Overview of Data Mining Tasks per Function 3-9 Summary Preparing Data for Data Mining Objectives 4-2 Data Source for Data Mining 4-3 Data Source for Clustering 4-5 Data Source for Attribute Importance 4-6 Data Source for Association Models 4-7 Data Formats Used in Data Mining 4-8 Nontransactional Data Format 4-9 Transactional Data Format 4-10 Mining Attributes 4-11 What Is Binning? 4-12 Binning Schemes 4-13 Summary Naive Bayes Algorithm Objectives 5-2 Naive Bayes Algorithm 5-3 Creating a Naive Bayes Model Using DM4J 5-6 Selecting Data Mining Components 5-8 Establishing a Connection to the Database 5-9 Entering the Model Details 5-10 Selecting the Input Data 5-11 Selecting the Target Attribute 5-13 Providing the Target Information 5-14 Selecting the Attributes 5-16 Selecting Naive Bayes Algorithm 5-17 Viewing Naive Bayes Model Details 5-18 iv

5 Naive Bayes Model Components 5-19 Executing the Naive Bayes Model 5-20 Executing Generated Code 5-21 Invoking the ODM Browser 5-22 ODM Browser 5-23 Viewing NBModel Details 5-24 Viewing Mining Model Information 5-25 Testing the Naive Bayes Model 5-26 Establishing a Connection to the Database 5-27 Providing Component Details 5-28 Selecting the Model 5-29 Selecting the Data 5-30 Viewing Classification Model Test Details 5-31 Executing the Model 5-32 Verifying the Mining Task Status 5-33 Viewing the Test Results 5-34 Calculating Naive Bayes Model Lift 5-35 Connecting to the ODM schema 5-36 Describing the Model 5-37 Selecting the Model 5-38 Describing the Input Data 5-39 Selecting the Target Attribute Value 5-40 Viewing the Wizard Component Details 5-41 Executing the Model 5-42 Viewing the Lift Chart: Cumulative Lift Chart 5-43 Viewing the Lift Chart: Cumulative Target Chart 5-44 Applying the Model to New Data 5-45 Selecting the Database and Data 5-46 Selecting the Model and Input Data 5-47 Selecting the Apply Attributes 5-48 Selecting the Apply Option 5-49 Storing Apply Results and Viewing Details 5-50 Executing the Apply Model 5-51 Viewing the Apply Results 5-52 Summary Attribute Importance Objectives 6-2 Overview of Attribute Importance 6-3 Overview of Predictor Variance Algorithm 6-4 Building an Attribute Importance Model 6-5 Selecting Schema and Describing Data 6-7 Identifying Input Data 6-8 Setting a Target Attribute 6-9 Viewing Attributes 6-10 v

6 Verify Component Details 6-11 Executing the Model 6-12 Viewing the Settings in the ODM Browser 6-13 Viewing the Results in the ODM Browser 6-14 Summary Adaptive Bayes Network Algorithm Objectives 7-2 Overview of Adaptive Bayes Network (ABN) Algorithm 7-3 Building an ABN Model 7-5 Describing the Model 7-6 Selecting Input Data 7-7 Selecting a Target Attribute 7-8 Setting the Weight for Scoring 7-9 Including Attributes 7-10 Selecting Algorithm Settings 7-11 Algorithm Settings 7-12 Verify the Model Details 7-15 Executing the Model 7-16 Viewing a Model Build Task 7-17 Viewing ABN Model Details 7-18 Viewing ABN Model Rules 7-19 Summary Enhanced k-means Algorithm Objectives 8-2 Overview of k-means Algorithm 8-3 Usability 8-5 Binning for k-means 8-6 How Does the k-means Algorithm Work? 8-7 Scalability Through Summarization 8-8 Scoring 8-9 Building a k-means Clustering Model 8-10 Connect to the Database and Define Data 8-11 Define Input Data 8-12 Selecting Attributes 8-13 Setting the Algorithm 8-14 Verify Component Details 8-15 Execute the Component 8-16 Viewing Cluster Information 8-17 Viewing Cluster Rules 8-21 Applying the Clustering Model to New Data 8-22 Connecting to Database and Defining Data 8-23 Selecting a Clustering Model 8-24 Selecting Input Data 8-25 vi

7 Include Attributes in Apply Table 8-26 Selecting the Apply Option 8-27 Selecting the Output Table 8-28 Verify Component Details 8-29 Execute the Component 8-30 Viewing the Results in the ODM Browser 8-31 Summary O-Cluster Algorithm Objectives 9-2 Clustering of High-Dimensional Data Sets 9-3 Overview of the Orthogonal Partitioning Cluster (O-Cluster) Algorithm 9-4 Usability 9-6 Binning for O-Cluster 9-7 Attribute Types 9-8 Scoring 9-9 Building an O-Cluster Algorithm 9-10 Connecting to the Database and Describing Data 9-11 Selecting Input Data 9-12 Selecting Attributes 9-13 Setting O-Cluster Algorithm Parameters 9-14 Verifying Model Details 9-15 Executing the Model 9-16 Viewing Model Status in ODM Browser 9-17 Viewing Cluster Settings 9-18 Viewing Cluster Details 9-19 Viewing Cluster Rules 9-20 Comparison of Enhanced k-means and O-Cluster Algorithms 9-21 Summary Association Rules Algorithm Objectives 10-2 Overview of the Association Rules Function 10-3 What Is the Association Rules Function? 10-5 What Is the Apriori Algorithm? 10-6 Building an Association Rules Model 10-7 Database Connection and Describing Data 10-8 Identifying Input Data 10-9 Selecting Attributes Viewing Algorithm Settings Verifying Component Details Executing the Model Viewing the Model Build Process Viewing Model Results Viewing Association Rules Summary vii

8 11 Deploying Java Components Objectives 11-2 Deploying Oracle9i Data Mining Java Components 11-3 Production: Build Model Production: Score Predictive Model Markup Language (PMML) Summary Sample Program for Discretization Objectives 12-2 Oracle9i Data Mining Sample Programs 12-3 Sample_Global.property File 12-4 Individual Property Files 12-5 Compiling Sample Programs 12-6 Executing Sample Programs 12-7 Compiling and Executing Sample Programs from the Command Prompt 12-8 Sample Program to Create Bin Boundaries Sample_Discretization_CreateBinBoundaryTables.property Sample_Discretization_CreateBinBoundaryTables.java Quantile Binning Example Explicit Specification for Categorical Data Top N Most Frequent Items Explicit Specification for Numerical Data Sample_Discretization_CreateBinBoundaryTables.java Binning the CENSUS_2D_BUILD_UNBINNED Data Set Setting the CLASSPATH Variable Modifying the Sample_Global.property File Compiling Sample_Discretization_CreateBinBoundaryTables.java Executing Sample_Discretization_CreateBinBoundaryTables.java Viewing the Results Viewing Intermediate Discretization Tables Sample_Discretization_UseBinBoundaryTables.property Sample_Discretization_UseBinBoundaryTables.java Creating a Binned View Viewing the Binned Data Summary Model Seeker Objectives 13-2 Overview 13-3 Model Seeker Functionality 13-4 Figure of Merit 13-5 Process for Finding the Best Model 13-6 Sample_ModelSeeker.property 13-8 viii

9 Working with Model Seeker Sample Program Modifying the Sample_Global.property File Compile Sample_ModelSeeker.java Executing Sample_ModelSeeker.java Viewing the Results Viewing Model Seeker Results Summary Cross Validation Objectives 14-2 Leave-one-out Cross Validation 14-3 Sample_NaiveBayesCrossValidate.property 14-4 Cross Validating a Naive Bayes Model 14-5 Summary 14-8 Appendix A Appendix B Practices and Solutions ix

Oracle9i Database: Using OLAP

Oracle9i Database: Using OLAP Student Guide D16001GC10 Edition 1.0 October 2003 D38689 Author Karen Kehn Technical Contributors and Reviewers William Endress Patricia Fay Nancy Greenberg Rosita Hanoman