MINING VERY LARGE DATABASES WITH PARALLEL PROCESSING

Transcription

1 MINING VERY LARGE DATABASES WITH PARALLEL PROCESSING

2 The Kluwer International Series on ADVANCES IN DATABASE SYSTEMS Series Editor Ahmed K. Elmagarmid Purdue University West Lafayette, IN Other books in the Series: DATABASE CONCURRENCY CONTROL: Methods, Performance, and Analysis by Alexander Thomasian ISBN: X TIME-CONSTRAINED TRANSACTION MANAGEMENT Real-Time Constraints in Database Transaction Systems by Nandit R. Soparkar, Henry F. Korth, Abraham Silberschatz ISBN: SEARCHING MULTIMEDIA DATABASES BY CONTENT by Christos Faloutsos ISBN: REPLICATION TECHNIQUES IN DISTRIBUTED SYSTEMS by Abdelsalam A. Helal, Abdelsalam A. Heddaya, Bharat B. Bhargava ISBN: VIDEO DATABASE SYSTEMS: Issues, Products, and Applications by Ahmed K. Elmagarmid, Haitao Jiang, Abdelsalam A. Helal, Anupam Joshi, Magdy Ahmed ISBN: DATABASE ISSUES IN GEOGRAPHIC INFORMATION SYSTEMS by Nabil R. Adam and Aryya Gangopadhyay ISBN: INDEX DATA STRUCTURES IN OBJECT-ORIENTED DATABASES by Thomas A. Mueck and Martin L. Polaschek ISBN: INDEXING TECHNIQUES FOR ADVANCED DATABASE SYSTEMS by Elisa Bertino, Beng Chin Ooi, Ron Sacks-Davis, Kian-Lee Tan, Justin Zobel, Boris Shidlovsky and Barbara Catania ISBN:

3 MINING VERY LARGE DATABASES WITH PARALLEL PROCESSING by Alex A. Freitas University of Essex Colchester, United Kingdom and Simon H. Lavington University of Essex Colchester, United Kingdom SPRINGER SCIENCE+BUSINESS MEDIA, LLC

4 Library of Congress Cataloging.in.Publication Data Freitas, Alex A., Mining very large databases with parallel processing / by Alex A. Freitas and Simon H. Lavington. p. cm. -- (The Kluwer international series on advances in database systems) Includes bibliographical references and index. ISBN ISBN (ebook) DOI / Database management. 2. Data mining. 3. Transaction systems (Computer systems) 4. Parallel processing (Electronic computers) I. Lavington, S. H. (Simon Hugh), II. Title. III. Series. QA76.9.D3F dc CIP Copyright by Springer Science+Business Media New York Origina1ly published by Kluwer Academic Publishers in 2000 Softcover reprint of the hardcover 1 st edition 2000 AII rights reserved. No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, mechanical, photocopying, recording, or otherwise, without the prior written permission of the publisher, Springer Science+Business Media, LLC. Printed an acid-free paper.

5 This book is dedicated to all the people who believe that learning is not only one of the most necessary but also one of the noblest human activities.

6 CONTENTS. PREFACE... xi ACKNOWLEDGMENTS... xiii INTRODUCTION... 1 The Motivation for Data Mining and Knowledge Discovery... 1 The Inter-disciplinary Nature of Knowledge Discovery in Databases (KDD)... 2 The Challenge of Efficient Knowledge Discovery in Large Databases and Data Warehouses... 3 Organization of the Book... 4 Part I KNOWLEDGE DISCOVERY AND DATA MINING KNOWLEDGE DISCOVERY TASKS Discovery of Association Rules Classification Other KDD Tasks KNOWLEDGE DISCOVERY PARADIGMS Rule Induction (RI) Instance-Based Learning (IBL) Neural Networks (NN) Genetic Algorithms (GA) On-Line Analytical Processing (OLAP) Focus on Rule Induction THE KNOWLEDGE DISCOVERY PROCESS An Overview of the Knowledge Discovery Process Data Warehouse (DW) Attribute Selection Discretization Rule-Set Refinement DATA MINING Decision-Tree Building Overfitting Data-Mining-Algorithm Bias Improved Representation Languages Integrated Data Mining Architectures... 49

7 viii 5 DATA MINING TOOLS Clementine Darwin MineSet Intelligent Miner Decision-Tree-Building Tools Part II PARALLEL DATABASE SYSTEMS BASIC CONCEPTS ON PARALLEL PROCESSING Temporal and Spatial Parallelism Granularity, Level and Degree of Parallelism Shared and Distributed Memory Evaluating the Performance of a Parallel System Communication Overhead Load Balancing Approaches for Exploiting Parallelism DATA PARALLELISM, CONTROL PARALLELISM AND RELATED ISSUES Data Parallelism and Control Parallelism Easy of Use and Automatic Paralle1ization Machine-Architecture Independence Scalability Data Partitioning Data Placement (Declustering) PARALLEL DATABASE SERVERS Architectures of Parallel Database Servers From the Teradata DBC 1012 to the NCR WorldMark ICL Goldrush Running Oracle Parallel Server IBM SP2 Running DB2 Parallel Edition (DB2-PE) Monet PartIII PARALLEL DATA MINING APPROACHES TO SPEED UP DATA MINING Overview of Approaches to Speed up Data Mining Discretization Attribute Selection Sampling and Related Approaches Fast Algorithms Distributed Data Mining Parallel Data Mining Discussion

8 ix 10 PARALLEL DATA MINING WITHOUT DBMS FACILITIES Parallel Rule Induction Parallel Decision-Tree Building Parallel Instance-Based Learning Parallel Genetic Algorithms O.S Parallel Neural Networks Discussion PARALLEL DATA MINING WITH DATABASE FACILITIES An Overview ofintegrated Data MininglData Warehouse Frameworks The Case for Integrating Data Mining and the Data Warehouse Server-Based KDD Systems... lsi 11.4 Hybrid Client/Server-Based KDD Systems... IS Generic, Set-Oriented Primitives for the Hybrid Client/Server-Based KDD Framework... IS A Generic, Set-Oriented Primitive for Candidate-Rule (CR) Evaluation in Rule Induction A Generic, Set-Oriented Primitive for Computing Distance Metrics in Instance-Based Learning Parallel Data Mining with Specialized-Hardware Parallel Database Servers SUMMARY AND SOME OPEN PROBLEMS Data-Parallel vs. Control-Parallel Data Mining Client/Server Frameworks for Parallel Data Mining Open Problems REFERENCES INDEX

9 PREFACE. This book addresses the problem of large-scale data mining. It is an interdisciplinary text, describing advances in the integration of three computer science areas, namely: "intelligent" (machine learning-based) data mining techniques; relational databases and parallel processing. The basic idea is to use concepts and techniques of the latter two areas - particularly parallel processing - to speed up and scale up data mining algorithms. The book is divided into three parts. The first part presents a comprehensive review of intelligent data mining techniques such as rule induction, instance-based learning, neural networks and genetic algorithms. Likewise, the second part presents a comprehensive review of parallel processing and parallel databases. Each of these parts includes an overview of commercially-available, state-of-the-art tools. The third part deals with the application of parallel processing to data mining. The emphasis is on finding generic, cost-effective solutions for realistic data volumes. Two parallel computational environments are discussed, firstly excluding the use of commercialstrength DBMS, and then using parallel DBMS servers. It is assumed that the reader has a knowledge roughly equivalent to a first degree (B.Sc.) in accurate sciences, so that (s)he is reasonably familiar with basic concepts of statistics and computer science. The primary audience for this book is industry data miners and practitioners in general, who would like to apply intelligent data mining techniques to large amounts of data. The book will also be of interest to academic researchers and post-graduate students, particularly database researchers interested in advanced, intelligent database applications and artificial intelligence researchers interested in industrial, real-world applications of machine learning.

10 ACKNOWLEDGMENTS. Since we started to work on data mining we have had the help of several good people. We are grateful to all of them, for their support. In particular, we would like to express our thanks to the following people: To Dominicus R. Thoen and Neil EJ. Dewhurst, for their help in some data mining experiments and for their support in general. To Paul Scott, for interesting discussions about data mining and machine learning. To Steve Hassan, for his help in using the White Cross WX90lO parallel database server. To Foster Provost, Richard Kufrin, and Sarabjot Anand, for interesting discussions about parallel data mining and for their encouragement. During the project that led to the writing up of this book, the first author was financially supported by a grant from the Brazilian government's National Council of Scientific and Technological Development (CNPq), process number /93-7.