A systematic performance evaluation of clustering methods for single-cell RNA-seq data Supplementary Figures and Tables

Transcription

1 A systematic performance evaluation of clustering methods for single-cell RNA-seq data Supplementary Figures and Tables Angelo Duò, Mark D Robinson, Charlotte Soneson

2 Supplementary Table : Overview of the datasets used in the study. Data set Total # cells Total # features Filtering Retained # cells Retained # features Koh 8,98 KohTCC 8,98 Kumar 6,9 KumarTCC 6 8, SimKumareasy,66 SimKumarhard 99,68 SimKumar8hard 99,6 Trapnell, TrapnellTCC 7 68,9 Zhengmixeq,99,68 Zhengmixuneq 6,98 6, Zhengmix8eq,99,76,898,898 MDrop,898 8,9 8,9 MDrop 8,9 6, 6,6 MDrop 6,6 6 8, 6 8, MDrop 6 8,,6,6 MDrop,6 99,6 99,6 MDrop 99,6 99,6 99,6 MDrop 99,6,, MDrop, 7 68,9 7 68,9 MDrop 7 68,9,,6,,7 MDrop,,7 6,,6,79,6 MDrop,8,6,97,7,798,7 MDrop,66,7

3 Zhengmixeq Zhengmixuneq Zhengmix8eq cd.monocytes cd.monocytes dimension dimension dimension cd.monocytes cd.t.helper cd6.nk memory.t naive.t dimension dimension Kumar dimension Trapnell Koh H7_derived_APS Dgcr8 knockout mouse serum+lif v6. mouse i+lif v6. mouse serum+lif H7_derived_DLtM T T T8 dimension dimension dimension H7_derived_DCntrlDrmmtm H7_derived_DLLpPXM H7_derived_ESMT H7_derived_MPS H7_derived_Sclrtm H7_dreived_D._Smtmrs H7hESC dimension dimension SimKumareasy dimension SimKumarhard SimKumar8hard Group Group Group Group Group Group dimension Group dimension dimension Group Group Group Group Group Group Group6 Group7 Group8 dimension dimension KohTCC dimension TrapnellTCC KumarTCC H7_derived_APS H7_derived_DCntrlDrmmtm H7_derived_DLLpPXM H7_derived_ESMT H7_derived_MPS H7_derived_Sclrtm T T T8 Dgcr8 knockout mouse serum+lif v6. mouse i+lif v6. mouse serum+lif H7_dreived_D._Smtmrs dimension H7_derived_DLtM dimension dimension H7hESC dimension dimension dimension Supplementary Figure : t-sne representations of all data sets used in the study (before gene filtering). The cells are colored by the annotation used to define the ground truth, to which all clusterings are compared.

4 Zhengmixeq Zhengmixuneq MDrop MDrop.6% 6.8% MDrop 7.%.%.%.7% 9.% 9.8%.6%.6%.8%.%.%.%.88%.% 6%.9% Kumar Trapnell MDrop Koh MDrop.9% 9.% MDrop.%.9% 8.9%.% 6.%.% 7.%.%.9%.%.89% 6.%.86%.% 8.7%.%.%.8%.7% SimKumareasy SimKumarhard MDrop %.% 7.%.6% Zhengmix8eq SimKumar8hard MDrop.99% 6%.% 6.9%.6%.%.7%.9% KohTCC 7.% MDrop % MDrop MDrop 8.%.7%.%.6%.%.% 6.8%.79% 8% 6%.%.7%.% KumarTCC.69% 8.% % TrapnellTCC MDrop.9%.7% 6.% 6.%.%.% 9.7% 7.9% % Supplementary Figure : Overlaps between the features retained by the different gene filtering approaches, for each of the data sets. The numbers represent percentages of the total number of features selected by at least one of the approaches.

5 Koh KohTCC Kumar KumarTCC SimKumareasy SimKumarhard SimKumar8hard Trapnell TrapnellTCC Zhengmixeq Zhengmixuneq Zhengmix8eq..7 Median Adjusted Rand Index filtered filtered filteredmdrop Number of clusters SC SCsvm Supplementary Figure : Adjusted Rand Index (ARI) as a function of the number of clusters (k), for each of the data sets (columns) and gene filterings (rows). The vertical lines indicate the true number of populations in the respective data sets.

6 A sce_filtered_zhengmixeq B sce_filtered_zhengmixeq True class True class cd.monocytes cd.monocytes dimension Clusters dimension Clusters C dimension sce_filtered_zhengmixuneq D dimension sce_filtered_trapnell SC True class cd.monocytes True class T T dimension Clusters dimension T8 Clusters dimension dimension Supplementary Figure : t-sne plots showing examples of data set/clustering method combinations with a large disagreement between the inferred and true clusters. The respective data sets, gene filtering methods and clustering methods are indicated above each plot. The true clusters are indicated with different shapes, and the inferred clusters in different colors. 6

7 Kumar KumarTCC SimKumareasy KohTCC Koh Zhengmixuneq SimKumarhard Zhengmixeq SimKumar8hard Zhengmix8eq TrapnellTCC Trapnell filtered filtered filteredmdrop Median ARI..7.. SCsvm SC SCsvm SC SCsvm SC Supplementary Figure : Adjusted Rand Index (ARI) at the number of clusters that gives the best performance. Some methods failed to return any clustering for certain data sets (indicated by white squares). Association between number of detected features and cluster assignment Koh KohTCC Kumar KumarTCC 7 SimKumareasy SimKumarhard SimKumar8hard Trapnell 8 6 TrapnellTCC Zhengmixeq Zhengmixuneq Zhengmix8eq SC SCsvm SC SCsvm SC SCsvm SC SCsvm SC SCsvm log(adjusted p value) filtering filtered filtered filteredmdrop Supplementary Figure 6: Association between the number of detected features and the cluster assignment, for all data sets and filterings, with the true number of clusters. Only one of the five runs is shown. Values are adjusted p-values (using the Benjamini-Hochberg method) from a Kruskal-Wallis test. 7

8 Koh KohTCC Kumar KumarTCC SimKumareasy SimKumarhard SimKumar8hard Trapnell TrapnellTCC Zhengmixeq Zhengmixuneq Zhengmix8eq Elapsed time (s) filtered filtered filteredmdrop SC SCsvm Number of clusters Supplementary Figure 7: Actual run times versus the imposed number of clusters, for each data set and gene filtering method. 8

9 Koh KohTCC Kumar KumarTCC SimKumareasy SimKumarhard SimKumar8hard Trapnell TrapnellTCC Zhengmixeq Zhengmixuneq Zhengmix8eq Elapsed time (s) filtered filtered filteredmdrop SC SCsvm SC SCsvm SC SCsvm SC SCsvm SC SCsvm SC SCsvm SC SCsvm SC SCsvm SC SCsvm SC SCsvm SC SCsvm SC SCsvm Supplementary Figure 8: Actual run times, across all numbers of clusters, for each data set and gene filtering method. 9

10 Median ARI. MDrop_MDrop.8 _.6 MDrop MDrop. SCsvm SC Supplementary Figure 9: Comparison of the clusterings obtained after different gene filterings. The values represent the median Adjusted Rand Index (ARI) across the data sets and filterings, for the true number of clusters, for each pair of filterings. Koh KohTCC Kumar KumarTCC SimKumareasy SimKumarhard SimKumar8hard Trapnell TrapnellTCC Zhengmixeq Zhengmixuneq Zhengmix8eq Entropy filtered filtered filteredmdrop SC SCsvm Number of clusters Supplementary Figure : Entropy for different imposed numbers of clusters. The true number of clusters is indicated by the vertical line, and the cross represents the entropy of the true partition.

11 filtered filtered filteredmdrop Difference between estimated and true k SC SCsvm SC SCsvm SC SCsvm Supplementary Figure : Difference between estimated and true number of clusters, for methods including a function to estimate the number of clusters. Kumar KumarTCC filtered filtered filteredmdrop SimKumareasy KohTCC Koh SimKumar8hard SimKumarhard Zhengmixeq Zhengmixuneq Zhengmix8eq Median ARI TrapnellTCC Trapnell SCsvm SC SCsvm SC SCsvm SC Supplementary Figure : Adjusted Rand Index (ARI) at the number of clusters estimated by the indicated methods. Empty squares indicate that the estimated number of clusters was outside the evaluated range (- for data sets with less than or equal to 7 true clusters, - for data sets with more than 7 true clusters.