Clustering. Cluster Analysis of Microarray Data. Microarray Data for Clustering. Data for Clustering

Size: px

Start display at page:

Download "Clustering. Cluster Analysis of Microarray Data. Microarray Data for Clustering. Data for Clustering"

Doris Benson
6 years ago
Views:

1 Clustering Cluster Analysis of Microarray Data 4/3/009 Copyright 009 Dan Nettleton Group obects that are siilar to one another together in a cluster. Separate obects that are dissiilar fro each other into different clusters. The siilarity or dissiilarity of two obects is deterined by coparing the obects with respect to one or ore attributes that can be easured for each obect. Data for Clustering Microarray Data for Clustering attribute obect n attribute tie points obect n estiated expression levels genes 4 Microarray Data for Clustering attribute tissue types obect n estiated expression levels genes 5 Microarray Data for Clustering treatent attribute conditions obect n estiated expression levels genes 6

2 Microarray Data for Clustering attribute genes obect n estiated expression levels saples 7 Clustering: An Exaple Experient Researchers were interested in studying gene expression patterns in developing soybean seeds. Seeds were harvested fro soybean plants at 5, 30, 40, 45, and 50 days after flowering (daf). One RNA saple was obtained for each level of daf. 8 An Exaple Experient (continued) Diagra Illustrating the Experiental Design Each of the 5 saples was easured on two two-color cdna icroarray slides using a loop design. The entire process we repeated on a second occasion to obtain a total of two independent biological replications. Rep Rep The daf eans estiated for each gene fro a ixed linear odel analysis provide a useful suary of the data for cluster analysis. 400 genes exhibited significant evidence of differential expression across tie (p-value<0.0, FDR=3.%). We will focus on clustering their estiated ean profiles. Noralized Data for One Exaple Gene Estiated Means + or SE Noralized Data for One Exaple Gene Estiated Means + or SE Noralized Log Signal Noralized Log Signal daf daf daf daf

3 We build clusters based on the ost significant genes rather than on all genes because... Estiated Mean Profiles for Top 36 Genes Much of the variation in expression is noise rather than biological signal, and we would rather not build clusters on the basis of noise. Soe clustering algoriths will becoe coputationally expensive if there are a large nuber of obects (gene expression profiles in this case) to cluster. 3 4 Dissiilarity Measures Parallel Coordinate Plots When clustering obects, we try to put siilar obects in the sae cluster and dissiilar obects in different clusters. Scatterplot Parallel Coordinate Plot We ust define what we ean by dissiilar. There are any choices. Let x and y denote diensional obects: x Value x=(x, x,..., x ) y=(y,y,..., y ) e.g., estiated eans at =5 five tie points for a given gene. 5 x Coordinate 6 These are parallel coordinate plots that each show one point in 5-diensional space. Euclidean Distance d E (x,y) = x - y = (x - y ) = -Correlation d cor (x,y) = - r xy = - = = (x - x) (x - x)(y - y) = (y - y) 7 8 3

4 Euclidean Distance -Correlation Dissiilarity Scatterplot Parallel Coordinate Plot d E (black,green) x d E (black,red) d E (red,green) Value The black and green obects are close together and far fro the red obect. x Coordinate 9 0 Relationship between Euclidean Distance and -Correlation Dissiilarity Let x~ = Let y~ = ~ x - ~ y = = = x - x s s x y - y y = ~ x = and let x~ and let y~ ( ~ x - ~ y ) = + ( - ) - r y = xy = (x ~,x ~,...,x ~ = (y ~,y ~,...,y ~ ~ - = ~ ~ y x = ). ( ~ x + ~ y - ~ x ~ y ) ). Thus Euclidean distance for standardized obects is proportional to the square root of the -correlation dissiilarity. We will standardize our ean profiles so that each profile has ean 0 and standard deviation (i.e., we will convert each x to ~ x). We will cluster based on the Euclidean distance between standardized profiles. Original ean profiles with siilar patterns are close to one another using this approach. Clustering ethods are often divided into two ain groups.. Partitioning ethods that attept to optially separate n obects into K clusters.. Hierarchical ethods that produce a nested sequence of clusters. Soe Partitioning Methods. K-Means. K-Medoids 3. Self-Organizing Maps (SOM) (Kohonen, 990; Toayo, P. et al., 998) 3 4 4

5 K Medoids Clustering Exaple of K Medoids Clustering 0. Choose K of the n obects to represent K cluster centers (a.k.a., edoids).. Given a current set of K edoids, assign each obect to the nearest edoid to produce an assignent of obects to K clusters.. For a given assignent of obects to K clusters, find the new edoid for each cluster by finding the obect in the cluster that is the closest on average to all other obects in its cluster. 3. Repeat steps and until the cluster assignents do not change. 5 6 Start with K Medoids Assign Each Point to Closest Medoid 7 8 Assign Each Point to Closest Medoid Assign Each Point to Closest Medoid

6 Assign Each Point to Closest Medoid Find New Medoid for Each Cluster New edoids have sallest average distance to other points in their cluster. 3 3 Reassign Each Point to Closest Medoid Reassign Each Point to Closest Medoid Find New Medoid for Each Cluster Reassign Each Point to Closest Medoid No reassignent is needed, so the procedure stops

7 Cluster of 3 fro K-Medoids Algorith Applied to the Top 400 Genes fro Public the Cluster Two-Color of 3 Array Data Cluster of 3 fro K-Medoids Algorith Applied to the Top 400 Genes fro Public the Cluster Two-Color of 3 Array Data Standardized Mean Profile Standardized Mean Profile DAF 37 DAF 38 Cluster 3 of 3 fro K-Medoids Algorith Applied to the Top 400 Genes fro Public the Cluster Two-Color 3 of 3 Array Data Choosing the Nuber of Clusters K Standardized Mean Profile Choose K that axiizes the average silhouette width. Rousseeuw, P.J. (987). Journal of Coputational and Applied Matheatics, 0, Kaufan, L. and Rousseeuw, P.J. (990). Finding Groups in Data: An Introduction to Cluster Analysis. Wiley, New York. Choose K according to the gap statistic. DAF 39 Tibshirani, R., Walther, G., Hastie, T. (00). Journal of the Royal Statistics Society, Series B-Statistical Methodology, 63, Silhouette Width Silhouette Width = (B-W)/ax(B,W) The silhouette width of an obect is (B-W)/ax(B,W) where W=average distance of the obect to all other obects within its cluster and B=average distance of the obect to all obects in its nearest neighboring cluster. Values near indicate that an obect is near the center of a tight cluster. Values near 0 indicate that an obect is between clusters. Negative values indicate that an obect ay be in the wrong cluster. The silhouette width will be between - and

8 Silhouette Width = (B-W)/ax(B,W) For a Given K Copute Silhouette Width for Each Point The silhouette widths of clustered obects can be averaged. A clustering with a high average silhouette width is preferred. For a given ethod of clustering, we ay wish to choose the value of K that axiizes the average silhouette width. Find W=average distance fro point to all others within its cluster. Find B=average distance fro point to all others in its nearest neighboring cluster. 43 Silhouette width is B-W ax(b,w) 44 Choice of K Silhouette width is coputed for all points and averaged. Average Silhouette Width vs. K for the K-Medoids Algorith Applied to the Top 400 Genes fro the Two-Color Array Data K with largest average silhouette width is preferred. K=3: Average Silhouette Width=0.640 K=: Average Silhouette Width=0.646 Slight preference for K= in this case Cluster of fro K-Medoids Algorith Applied to the Top 400 Genes fro the Two-Color Array Data Cluster of fro K-Medoids Algorith Applied to the Top 400 Genes fro the Two-Color Array Data Standardized Mean Profile Standardized Mean Profile DAF 47 DAF 48 8

9 Gap Statistic Gap Statistic (continued) For a given clustering of n obects x,...,x n; the distance d(x i,x ) between obects x i and x is called a within-cluster distance if x i and x are within the sae cluster. Let D r = the su of all within-cluster distances in the r th cluster, and let n r denote the nuber of obects in the r th cluster. For a given clustering of n obects into k clusters, k let W k = Σ r= D r / n r. 49 For a given clustering ethod, copute log W, log W,..., log W K. Let in denote the iniu of the th coponent of all n obects clustered. Let ax denote the axiu of the th coponent of all n obects to be clustered. Generate n rando obects uniforly distributed on the diensional rectangle [in,ax ] x... x [in,ax ]. 50 Gap Statistic (continued) Estiate of Best K Using the Gap Statistic Using the rando unifor data, copute log W *, log W *,..., log W K *. Randoly generate new unifor data ultiple ties (0 or ore) and use the results to obtain log W *, log W *,..., log W K * and S, S,...,S K ; the averages and standard deviations of the siulated log W values. Let G(k) = log W k * log W k. An approxiate standard error for G(k) is S k +/N where N denotes the nuber of randoly generated data sets. An estiate of the best K is given by K ^ = in { k : G(k) G(k+) S k+ +/N }. 5 5 Siple Exaple Data Revisited Gap Analysis for the Siple Exaple (N=000) log W k * and log W k vs. k G(k)=log W k * log W k vs. k (+ or standard error) log W k G(k) k=nuber of Clusters k=nuber of Clusters

The Gap Statistic Suggests K=3 Clusters The Gap Statistic Suggests K=3 Clusters G(3) >= G(4)-SE G() less than G(3)-SE G() less than G()-SE 55 56 Gap Analysis for Two-Color Array Data (N=00) Gap

10 The Gap Statistic Suggests K=3 Clusters The Gap Statistic Suggests K=3 Clusters G(3) >= G(4)-SE G() less than G(3)-SE G() less than G()-SE Gap Analysis for Two-Color Array Data (N=00) Gap Analysis for Two-Color Array Data (N=00) log W k * and log W k vs. k G(k)=log W k * log W k vs. k (+ or standard error) zooed in version of previous plot Gap Analysis Estiates K= Clusters log W k G(k) G(k) k=nuber of Clusters k=nuber of Clusters 57 k=nuber of Clusters

67 68 Plot of Cluster Medoids 69 70 Principal Coponents Principal coponents can be useful for providing low-diensional views of high-diensional data.

12 67 68 Plot of Cluster Medoids Principal Coponents Principal coponents can be useful for providing low-diensional views of high-diensional data.... nuber of variables Principal Coponents (continued) Each principal coponent of a data set is a variable obtained by taking a linear cobination of the original variables in the data set. variable or attribute x x... x x Data x. Matrix. X =... x n or Data Set x n..... x n.. n observation or obect nuber of observations 7 A linear cobination of variables x, x,..., x is given by c x + c x c x. For the purpose of constructing principal coponents, the vector of coefficients is restricted to have unit length, i.e., c + c c =. 7

Principal Coponents (continued) The Siple Data Exaple The first principal coponent is the linear cobination of the variables that has axiu variation across the observations in the data set.

13 Principal Coponents (continued) The Siple Data Exaple The first principal coponent is the linear cobination of the variables that has axiu variation across the observations in the data set. The th principal coponent is the linear cobination of the variables that has axiu variation across the observations in the data set subect to the constraint that the vector of coefficients be orthogonal to coefficient vectors for principal coponents,..., -. x 73 x 74 The First Principal Coponent Axis The First Principal Coponents x x st PC for this point is signed distance between its proection onto the st PC axis and the origin. x 75 x 76 The Second Principal Coponent Axis The Second Principal Coponent x x nd PC for this point is signed distance between its proection onto the nd PC axis and the origin. x 77 x 78 3

Plot of PC vs. PC Copare the PC plot to the plot of the original data below.

PC is ust a rotation of the original plot.

Proection of Two-Color Array Data with -Medoid Clustering Consider the principal coponents for

Our data atrix has n=400 rows and =5 coluns.

14 Plot of PC vs. PC Copare the PC plot to the plot of the original data below. PC x Because there are only two variables here, the plot of PC vs. PC is ust a rotation of the original plot. PC 79 x 80 There is ore to be gained when the nuber of variables is greater than. Proection of Two-Color Array Data with -Medoid Clustering Consider the principal coponents for the 400 significant genes fro our two-color icroarray experient. Our data atrix has n=400 rows and =5 coluns. We have looked at this data using parallel coordinate plots. What would it look like if we proected the data points to -diensions? PC a= b= c=3 d=4 e=5 f=6 g=7 h=8 i=9 =0 k= 8 PC 8 Proection of Two-Color Array Data with -Medoid Clustering PC 3 a= b= c=3 d=4 e=5 f=6 g=7 h=8 i=9 =0 k= PC PC

Proection of Two-Color Array Data with -Medoid Clustering Hierarchical Clustering Methods PC 3 a= b= c=3 d=4 e=5 f=6 g=7 h=8 i=9 =0 k= Hierarchical clustering ethods build a nested sequence of

15 Proection of Two-Color Array Data with -Medoid Clustering Hierarchical Clustering Methods PC 3 a= b= c=3 d=4 e=5 f=6 g=7 h=8 i=9 =0 k= Hierarchical clustering ethods build a nested sequence of clusters that can be displayed using a dendrogra. We will begin with soe siple illustrations and then ove on to a ore general discussion. PC The Siple Exaple Data with Observation Nubers Dendrogra for the Siple Exaple Data root node Tree Structure a parent node x nodes daughter nodes (daughter nodes with sae parent are sister nodes) x 87 terinal nodes or leaves corresponding to obects 88 A Hierarchical Clustering of the Siple Exaple Data Dendrogra for the Siple Exaple Data x Scatterplot of Data Dendrogra The height of a node represents the dissiilarity between the two clusters erged together at the node. x clusters within clusters within clusters These two clusters have a dissiilarity of about.75. 5

16 The appearance of a dendrogra is not unique. Dendrogra The appearance for of the a dendrogra Siple Exaple is not unique. Data Any two sister nodes could trade places without changing the eaning of the dendrogra. By convention, R dendrogras show the lower sister node on the left. Ties are broken by observation nuber. Thus 4 next to 7 does not iply that these obects are siilar. 9 e.g., 3 is to the left of 4 9 The appearance of a dendrogra is not unique. Cutting the tree at a given height will correspond to a partitioning of the data into k clusters. The lengths of the branches leading to terinal nodes have no particular eaning in R dendrogras. k= Clusters Cutting the tree at a given height will correspond to a partitioning of the data into k clusters. Cutting the tree at a given height will correspond to a partitioning of the data into k clusters. k=3 Clusters k=4 Clusters

17 Cutting the tree at a given height will correspond to a partitioning of the data into k clusters. Aggloerative (Botto-Up) Hierarchical Clustering Define a easure of distance between any two clusters. (An individual obect is considered a cluster of size one.) Find the two nearest clusters and erge the together to for a new cluster. k=0 Clusters Repeat until all obects have been erged into a single cluster Coon Measures of Between-Cluster Distance Coon Measures of Between-Cluster Distance Single Linkage a.k.a. Nearest Neighbor: the distance between any two clusters A and B is the iniu of all distances fro an obect in cluster A to an obect in cluster B. Coplete Linkage a.k.a Farthest Neighbor: the distance between any two clusters A and B is the axiu of all distances fro an obect in cluster A to an obect in cluster B. Average Linkage: the distance between any two clusters A and B is the average of all distances fro an obect in cluster A to an obect in cluster B. Centroid Linkage: the distance between any two clusters A and B is the distance between the centroids of cluster A and B. (The centroid of a cluster is the coponentwise average of the obects in a cluster.) x Aggloerative Clustering Using Average Linkage for the Siple Exaple Data Set Scatterplot of Data x N L J F Dendrogra P E D O K H A C M I G B 0 Aggloerative Clustering Using Average Linkage for the Siple Exaple Data Set P A. - N L J F O K H E D A C B. 9-0 C. 3-4 D. 5-6 E. 7-(5,6) F. 3-4 M G. - I G H. (,)-(3,4) B I. (9,0)-(,) etc

18 Aggloerative Clustering Using Single Linkage for the Siple Exaple Data Set Aggloerative Clustering Using Coplete Linkage for the Siple Exaple Data Set Aggloerative Clustering Using Centroid Linkage for the Siple Exaple Data Set Centroid linkage is not onotone in the sense that later cluster erges can involve clusters that are ore siilar to each other than earlier erges. Aggloerative Clustering Using Centroid Linkage for the Siple Exaple Data Set The erge between 4 and (,,3,5) creates a cluster whose centroid is closer to the (6,7) centroid than 4 was to the centroid of (,,3,5) Aggloerative Clustering Using Single Linkage for the Two-Color Microarray Data Set Aggloerative Clustering Using Coplete Linkage for the Two-Color Microarray Data Set

19 Aggloerative Clustering Using Average Linkage for the Two-Color Microarray Data Set Aggloerative Clustering Using Centroid Linkage for the Two-Color Microarray Data Set 09 0 Which Between-Cluster Distance is Best? Depends, of course, on what is eant by best. Single linkage tends to produce long stringy clusters.. Conduct aggloerative hierarchical clustering for this data using Euclidean distance and coplete linkage.. Display your results using a dendrogra. 3. Identify the k= clustering using your results. Coplete linkage produces copact spherical clusters but ight result in soe obects that are closer to obects in clusters other than their own. (See next exaple.) Average linkage is a coproise between single and coplete linkage. Centroid linkage is not onotone. Results of Coplete-Linkage Clustering Divisive (Top-Down) Hierarchical Clustering Results for k= Clusters Start with all data in one cluster and divide it into two clusters (using, e.g., -eans or -edoids clustering). At each subsequent step, choose one of the existing clusters and divide it into two clusters. Repeat until there are n clusters each containing a single obect

20 Potential Proble with Divisive Clustering 5 5 Macnaughton-Sith et al. (965). Start with obects in one cluster A.. Find the obect with the largest average dissiilarity to all other obects in A and ove that obect to a new cluster B. 3. Find the obect in cluster A whose average dissiilarity to other obects in cluster A inus its average dissiilarity to obects in cluster B is axiu. If this difference is positive, ove the obect to cluster B. 4. Repeat step 3 until no obects satisfying 3 are found. 5. Repeat steps through 4 to one of the existing clusters (e.g., the one with the largest average within-cluster dissiilarity) until n clusters of obect each are obtained. 6 Macnaughton-Sith Divisive Clustering Macnaughton-Sith Divisive Clustering A B A B Macnaughton-Sith Divisive Clustering Macnaughton-Sith Divisive Clustering A B A B B

21 Macnaughton-Sith Divisive Clustering Dendrogra for the Macnaughton-Sith Approach A B B 5 Next continue to split each of these clusters until each obect is in a cluster by itself. Aggloerative vs. Divisive Clustering Divisive clustering has not been studied as extensively as aggloerative clustering. Divisive clustering ay be preferred if only a sall nuber of large clusters is desired. Aggloerative clustering ay be preferred if a large nuber of sall clusters is desired. 3

Cluster Analysis for Microarray Data

Cluster Analysis for Microarray Data Seventh International Long Oligonucleotide Microarray Workshop Tucson, Arizona January 7-12, 2007 Dan Nettleton IOWA STATE UNIVERSITY 1 Clustering Group objects that