Data Mining for Association Rules and Sequential Patterns

Transcription

1 Data Mining for Association Rules and Sequential Patterns

2 Springer-Science+Business Media, LLC

3 Jean-Marc Adamo Data Mining for Association Rules and Sequential Patterns Sequential and Parallel Algorithms With 54 Illustrations, Springer

4 Jean-Marc Adamo Universile de Lyon 43 bd. II novembre 1918 Bat. 308, B.P Vi lleurbanne, cedelt France Ubrary ofcongress Cataloging-in-Publication Data Adamo, Jean-Marc, Data mining for association rule.s and sequential pancms : sequential and par.lllel algorithms I Jean-Marc Adamo p. cm. Includes bibliographical rcferem:e.s and index. ISBN ISBN (ebook) DOI / Data mining. 2. Col1ţlute r algorithms. 1. Title. QA76.9.D343A dc2\ (}O.{)S6267 Printed on acid-free paper. C 2001 Springe r Sciencc+ Busincss Media New York Originally publishcd by Springel"'-Ve rla g New YOI k, Ine in 2001 Softcover reprint ofthe hardcover l at edition 2001 AII rights reserved. This work may RO( be tmnslated or copied in whole or in pari without the wrinen permission ofme publisher (Springer Science+Business Media, LLC), except for brief excerpts in connection wim rcvicws Of scllolarly analysis. Use in connection with any form of infonnation stomge ancl retrieval, electronic adaptation, computer software, CIr by similar Of dissimilar metllodology now known or hercafter developcd is forbidden. The use ofgeneral descriptive names. U"ade names, trademaooi, etc., in mis publicat ion, even ifme former are RO( e.special1y idenlified, is noi to be laken IS I sign that such names. as unclerstood by the Trade Marks and Merchandise Marks Act, may aecordingly be uscd frecl y byanyonc:. Production manage<! by Steven Pisano; manufacturing supervised by Jeffrey Taub. Camcra-rcady pagts prcpared ITom the authors' Microsoft Word files ISUN

5 Preface Data mining includes a wide range of activities such as classification, clustering, similarity analysis, summarization, association rule and sequential pattern discovery, and so forth. The book focuses on the last two previously listed activities. It provides a unified presentation of algorithms for association rule and sequential pattern discovery. For both mining problems, the presentation relies on the lattice structure of the search space. All algorithms are built as processes running on this structure. Proving their properties takes advantage of the mathematical properties of the structure. Part of the motivation for writing this book was postgraduate teaching. One of the main intentions was to make the book a suitable support for the clear exposition of problems and algorithms as well as a sound base for further discussion and investigation. Since the book only assumes elementary mathematical knowledge in the domains of lattices, combinatorial optimization, probability calculus, and statistics, it is fit for use by undergraduate students as well. The algorithms are described in a C-like pseudo programming language. The computations are shown in great detail. This makes the book also fit for use by implementers: computer scientists in many domains as well as industry engineers. Mining for association rules and sequential patterns is known to be a problem with large computational complexity. Most mining algorithms typically take hours to complete when performed on large real-life datasets. The issue of designing efficient parallel algorithms should be considered as critical. Most algorithms in the book are devised for both sequential and parallel execution. Parallel algorithm design takes advantage of the lattice structure of the search space. Partitioning is performed via lattice recursive bisection. Database partitioning is also used as an additional source of parallelism. The book contains ten chapters including the introduction. Chapter 2 is dedicated to search space partitioning and to mining with partitioned search spaces. Chapter 3 contains a review of all rule-mining algorithms that have been presented thus far in the literature. Chapter 4 extends the search space partition-based algorithm so that it can also deal with taxonomies. Chapter 5 investigates the problem of rule mining under Boolean constraints. Chapter 6 presents a database partitioning method based

6 vi Preface on sampling. Methods for merging the search space and database partitioning are proposed, leading to new sequential and parallel algorithms. Chapter 7 investigates the problem of mining rules with categorical and metric attributes. The latter problem deals with exhaustive enumeration. Another way of drawing useful information from quantitative association rules leads to optimization problems. Chapter 8 proposes a unified presentation of the problems and solutions. Chapter 9 describes new measures aimed at improving the predictive ability of the rules and new algorithms aimed at limiting combinatorial explosion. Finally, Chapter 10 deals with sequential pattern mining. The problem is investigated by using a method similar to the one used for rule mining. The implementation of all algorithms presented in the book is currently under development. The work is carried out on a cluster of SMP machines and uses the ARCH library (run above MPI) as a development tool. Progress reports will be found at the address Acknowledgments The creation of this book benefited from the assistance of a number of people. I am grateful to Yves Kodratoff for reading the manuscript and making constructive comments. I am also grateful to my wife, Monique, for reading the manuscript so professionally and making many comments that helped improve the syntax significantly. Many thanks also go to my editors, Steven Pisano, Wayne Wheeler, and Wayne Yuhasz at Springer-Verlag, for their assistance. The Ecole Supeneure de Chimie Physique et Electronique de Lyon hosted me and provided logistic support and facilities. The cover photograph is by Olivier Lernout, CINES, Montpellier, France. Permission was granted by Olivier Lernout and Alain Quere, Director of CINES. The photograph represents the shot of a largescale sand counter designed by Jean Bernard Metais for counting the time between two eclipses: August 11, 1999 and June 22, The work of art is being exhibited at the Museum National d 'Histoire Naturelle de Paris. The sand flows down multiple holes, thereby creating a series of patterns, which triggers a striking evocation of parallel data mining--one of the main topics of this book. Jean-Marc Adamo

7 I Contents Preface... v 1. Introduction Search Space Partition-Based Rule Mining Problem Statement Canonical Attribute Sequences (cas) Database Support Association Rule Problem Statement Search Space Splitting Procedure Enumerating a-frequent Attribute Sets (cass) Sequential Enumeration Procedure Parallel Enumeration Procedure Initial Load Balancing Computing the Starting Sets Enumeration Procedure Dynamic Load Balancing Generating the Association Rules Sequential Generation Parallel Generation Apriori and Other Algorithms Early Algorithms AlS SETM The Apriori Algorithms Apriori AprioriTid Direct Hashing and Pruning Filtering Candidates... 41

8 viii Contents Database Trimming The DHP Algorithm Dynamic Set Counting Mining for Rules over Attribute Taxonomies Association Ru1es over Taxonomies Problem Statement and Algorithms Pruning Uninteresting Rules Measure o/interest Rule Pruning Algorithm Attribute Presence-Based Pruning Constraint-Based Rule Mining Boolean Constraints Syntax Semantics Propagation o/boolean Constraints Prime Implicants Problem Statement and Algorithms Data Partition-Based Rule Mining Data Partitioning Building a Probabilistic Model Bounding Large Deviations for One cas (Chernoff bounds) Bounding Large Deviations for Sets of cass cas Enumeration with Partitioned Data Data Partitioning Local CF-Frequent cas Generation Global CF-Frequent cas Generation Mining for Rules with Categorical and Metric Attributes Interval Systems and Quantitative Ru1es k-partial Completeness Pruning Uninteresting Rules Measure of Interest Attribute Presence-Based Pruning Enumeration Algorithms Optimizing Rules with Quantitative Attributes Solving I-I-Type Rule Optimization Problems Problem Statement MCISProblem MSIC Problem MG Problem

9 Data Miningfor Association Rules and Sequential Patterns ix 8.2 Solving d-l-type Rule Optimization Problems Solving l-q-type Rule Optimization Problems Problem Statement MSIC Problem MG Problem Solving d-q-type Rule Optimization Problems Problem Statement Basic Enumeration Enumeration with Pruning Pruning the Instantiation Set Beyond Support-Confidence Framework A Criticism ofthe Support-Confidence Framework Conviction Pruning Conviction-Based Rules Analyzing Conviction Transitivity-Based Pruning Improvement-Based Pruning One-Step Association Rule Mining Building a Procedure for One-Step Mining Building a Procedure for Improvement-Based Pruning Correlated Attribute-Set Mining Collective Strength Correlated Attribute-Set Enumeration Refining Conviction: Association Rule Intensity Measure Construction Properties Relating a-int(s.=;> u) to conv(s.=;> u) Mining with the Intensify Measure a-intensify Versus Intensity as Defined in [G96] Search Space Partition-Based Sequential Pattern Mining Problem Statement Sequences of cass Database Support Problem Statement Search Space Splitting the Search Space Splitting Procedure Sequence Enumeration Extending the Support Set Notion Join Operations Sequential Enumeration Procedure Parallel Enumeration Procedure

10 x Contents Appendix 1. Chernoff Bounds Appendix 2. Partitioning in Figure 10.5: Beyond 3rd Power Appendix 3. Partitioning in Figure 10.6: Beyond 3rd Power References Index