Contents. List of Figures. List of Tables. List of Algorithms. I Clustering, Data, and Similarity Measures 1

Transcription

1 Contents List of Figures List of Tables List of Algorithms Preface xiii xv xvii xix I Clustering, Data, and Similarity Measures 1 1 Data Clustering Definition of Data Clustering The Vocabulary of Clustering Records and Attributes Distances and Similarities Clusters, Centers, and Modes Hard Clustering and Fuzzy Clustering Validity Indices Clustering Processes Dealing with Missing Values Resources for Clustering Surveys and Reviews on Clustering Books on Clustering Journals Conference Proceedings Data Sets Summary Data Types Categorical Data Binary Data Transaction Data Symbolic Data Time Series Summary v

2 vi Contents 3 Scale Conversion Introduction Interval to Ordinal Interval to Nominal Ordinal to Nominal Nominal to Ordinal Ordinal to Interval Other Conversions Categorization of Numerical Data Direct Categorization Cluster-based Categorization Automatic Categorization Summary Data Standardization and Transformation Data Standardization Data Transformation Principal Component Analysis SVD The Karhunen-Loève Transformation Summary Data Visualization Sammon s Mapping MDS SOM Class-preserving Projections Parallel Coordinates Tree Maps Categorical Data Visualization Other Visualization Techniques Summary Similarity and Dissimilarity Measures Preliminaries Proximity Matrix Proximity Graph Scatter Matrix Covariance Matrix Measures for Numerical Data Euclidean Distance Manhattan Distance Maximum Distance Minkowski Distance Mahalanobis Distance... 72

3 Contents vii Average Distance Other Distances Measures for Categorical Data The Simple Matching Distance Other Matching Coefficients Measures for Binary Data Measures for Mixed-type Data A General Similarity Coefficient A General Distance Coefficient A Generalized Minkowski Distance Measures for Time Series Data The Minkowski Distance Time Series Preprocessing Dynamic Time Warping Measures Based on Longest Common Subsequences Measures Based on Probabilistic Models Measures Based on Landmark Models Evaluation Other Measures The Cosine Similarity Measure A Link-based Similarity Measure Support Similarity and Dissimilarity Measures between Clusters The Mean-based Distance The Nearest Neighbor Distance The Farthest Neighbor Distance The Average Neighbor Distance Lance-Williams Formula Similarity and Dissimilarity between Variables Pearson s Correlation Coefficients Measures Based on the Chi-square Statistic Measures Based on Optimal Class Prediction Group-based Distance Summary II Clustering Algorithms Hierarchical Clustering Techniques Representations of Hierarchical Clusterings n-tree Dendrogram Banner Pointer Representation Packed Representation Icicle Plot Other Representations...115

4 viii Contents 7.2 Agglomerative Hierarchical Methods The Single-link Method The Complete Link Method The Group Average Method The Weighted Group Average Method The Centroid Method The Median Method Ward s Method Other Agglomerative Methods Divisive Hierarchical Methods Several Hierarchical Algorithms SLINK Single-link Algorithms Based on Minimum Spanning Trees CLINK BIRCH CURE DIANA DISMEA Edwards and Cavalli-Sforza Method Summary Fuzzy Clustering Algorithms Fuzzy Sets Fuzzy Relations Fuzzy k-means Fuzzy k-modes The c-means Method Summary Center-based Clustering Algorithms The k-means Algorithm Variations of the k-means Algorithm The Continuous k-means Algorithm The Compare-means Algorithm The Sort-means Algorithm Acceleration of the k-means Algorithm with the kd-tree Other Acceleration Methods The Trimmed k-means Algorithm The x-means Algorithm The k-harmonic Means Algorithm The Mean Shift Algorithm MEC The k-modes Algorithm (Huang) Initial Modes Selection The k-modes Algorithm (Chaturvedi et al.)...178

5 Contents ix 9.10 The k-probabilities Algorithm The k-prototypes Algorithm Summary Search-based Clustering Algorithms Genetic Algorithms The Tabu Search Method Variable Neighborhood Search for Clustering Al-Sultan s Method Tabu Search based Categorical Clustering Algorithm J-means GKA The Global k-means Algorithm The Genetic k-modes Algorithm The Selection Operator The Mutation Operator The k-modes Operator The Genetic Fuzzy k-modes Algorithm String Representation Initialization Process Selection Process Crossover Process Mutation Process Termination Criterion SARS Summary Graph-based Clustering Algorithms Chameleon CACTUS A Dynamic System based Approach ROCK Summary Grid-based Clustering Algorithms STING OptiGrid GRIDCLUS GDILC WaveCluster Summary Density-based Clustering Algorithms DBSCAN BRIDGE DBCLASD...222

6 x Contents 13.4 DENCLUE CUBN Summary Model-based Clustering Algorithms Introduction Gaussian Clustering Models Model-based Agglomerative Hierarchical Clustering The EM Algorithm Model-based Clustering COOLCAT STUCCO Summary Subspace Clustering CLIQUE PROCLUS ORCLUS ENCLUS FINDIT MAFIA DOC CLTree PART SUBCAD Fuzzy Subspace Clustering Mean Shift for Subspace Clustering Summary Miscellaneous Algorithms Time Series Clustering Algorithms Streaming Algorithms LSEARCH Other Streaming Algorithms Transaction Data Clustering Algorithms LargeItem CLOPE OAK Summary Evaluation of Clustering Algorithms Introduction Hypothesis Testing External Criteria Internal Criteria Relative Criteria...304

7 Contents xi 17.2 Evaluation of Partitional Clustering Modified Hubert s Ɣ Statistic The Davies-Bouldin Index Dunn s Index The SD Validity Index The S_Dbw Validity Index The RMSSTD Index The RS Index The Calinski-Harabasz Index Rand s Index Average of Compactness Distances between Partitions Evaluation of Hierarchical Clustering Testing Absence of Structure Testing Hierarchical Structures Validity Indices for Fuzzy Clustering The Partition Coefficient Index The Partition Entropy Index The Fukuyama-Sugeno Index Validity Based on Fuzzy Similarity A Compact and Separate Fuzzy Validity Criterion A Partition Separation Index An Index Based on the Mini-max Filter Concept and Fuzzy Theory Summary III Applications of Clustering Clustering Gene Expression Data Background Applications of Gene Expression Data Clustering Types of Gene Expression Data Clustering Some Guidelines for Gene Expression Clustering Similarity Measures for Gene Expression Data Euclidean Distance Pearson s Correlation Coefficient A Case Study C++ Code Results Summary IV MATLAB and C++ for Clustering Data Clustering in MATLAB Read and Write Data Files Handle Categorical Data...347

8 xii Contents 19.3 M-files, MEX-files, and MAT-files M-files MEX-files MAT-files Speed up MATLAB Some Clustering Functions Hierarchical Clustering k-means Clustering Summary Clustering in C/C The STL The vector Class The list Class C/C++ Program Compilation Data Structure and Implementation Data Matrices and Centers Clustering Results The Quick Sort Algorithm Summary A Some Clustering Algorithms 371 B The kd-tree Data Structure 375 C MATLAB Codes 377 C.1 The MATLAB Code for Generating Subspace Clusters C.2 The MATLAB Code for the k-modes Algorithm C.3 The MATLAB Code for the MSSC Algorithm D C++ Codes 385 D.1 The C++ Code for Converting Categorical Values to Integers D.2 The C++ Code for the FSC Algorithm Bibliography 397 Subject Index 443 Author Index 455