Interpreting predictive models in terms of anatomically labelled regions

Transcription

1 Interpreting predictive models in terms of anatomically labelled regions

2 Data Feature extraction/selection Model specification Accuracy and significance Model interpretation 2

3 Data Feature extraction/selection Model specification Accuracy and significance Where in the brain is the information of interest? i.e. Which brain areas contribute most to the model s prediction? Model interpretation 3

4 Computing weight maps : - From linear (kernel) methods - Attributing one weight per feature - For an SVM model: w = n i=1 y i α i x i 4

5 Thresholding weight maps: Can we choose to look only at the largest contributions? - Linear combination f x i = w, x i + b - As in an election - Each vote counts

6 Interpreting weight maps: What do the model parameters represent? - Voxel 1: s n + d(n) - Voxel 2: d(n) w(1) w(2) Weights: 1-1 Voxel 1 Voxel 2 Haufe et al., 2014: weight patterns versus weight filters (forward versus backward model) 6

7 Different strategies: 1. A priori knowledge: ROI selection 2. Searchlight 3. Feature selection 4. Sparse algorithms 5. Multiple Kernel Learning (MKL) 6. Permutation tests 7. A posteriori knowledge: weight summarization 7

8 Data Feature extraction/selection Model specification Accuracy and significance Model interpretation Different strategies: 1. A priori knowledge: ROI selection 2. Searchlight 3. Feature selection 4. Sparse algorithms 5. Multiple Kernel Learning (MKL) 6. Permutation tests 7. A posteriori knowledge: weight summarization 8

9 1. A priori knowledge: ROI selection e.g. Choosing voxels/features based on literature or on previous univariate results (independent dataset). 9

10 1. A priori knowledge: ROI selection Good accuracy? + - Anatomically/ functionally relevant Need an anatomical hypothesis Cannot threshold obtained map Arbitrary choice of region number and size Multiple models = multiple comparisons 10

11 2. Searchlight (Kriegeskorte et al., 2006) 11

12 2. Searchlight (Kriegeskorte et al., 2006) Threshold for each neighborhood Not a weight filter + - Review: Etzel et al., 2013 Not anatomically/functionally relevant (spheres) Locally multivariate Arbitrary choice of neighborhood size Multiple models = multiple comparisons Time-consuming 12

13 3. Feature selection Rank the features according to a specific criterion and select a subset. Example: - Univariate t-test - Recursive Feature Elimination (RFE, De Martino, 2008) based on SVM weights 13

14 3. Feature selection Data-driven + - Not anatomically/functionally relevant (voxel sparse) Time-consuming: nested CV Depends on selected technique: flaws in feature selection are reflected in model. user choice must be informed. Different strategies = different patterns with same accuracy. 14

15 4. Sparse algorithms Algorithms with regularization constraints such that the weight of some features is perfectly null. Examples: - LASSO (Tibshirani, 1996) - Elastic-net (Zou and Hastie, 2005) 15

16 4. Sparse algorithms Data-driven Embedded in model + - Not anatomically/functionally relevant (voxel sparse) Stability of selected voxels (Baldassarre, et al., 2012): different regularization terms = different patterns for same accuracy. 16

17 4. Sparse algorithms Data-driven Embedded in model + - Not anatomically/functionally relevant Stability of selected voxels (Baldassarre, et al., 2012): different regularization terms = different patterns for same accuracy. Note: stability selection (Varoquaux et al., 2012, Rondina et al., 2014). 17

18 5. Multiple Kernel Learning (MKL) Build one kernel per anatomically defined region as input of a (sparse) MKL algorithm (Filippone, 2012). K x, x = M m=1 d m K m (x, x ) with d m 0, M m=1 d m = 1 18

19 5. Multiple Kernel Learning (MKL) Decision function of an MKL problem: f x i = m w m, x i + b Considered algorithm (Rakotomamonjy et al., 2008): min 1 1 w 2 2 d m + C ξ i m m s. t. y i m ξ i 0 i, w m, x i + b 1 ξ i i m d m = 1, i d m 0 m 19

20 5. Multiple Kernel Learning (MKL) Weight map of an MKL model: n w m = d m i=1 y i α i x i But also d m Can be considered as a hierarchical approach 20

21 5. Multiple Kernel Learning (MKL) 21

22 5. Multiple Kernel Learning (MKL) 22

23 5. Multiple Kernel Learning (MKL) + - Anatomically/ functionally relevant Used the full pattern to build the model Thresholded and sorted list of regions according to d m Hierarchical view of the model parameters Stability of the selected regions Depends on atlas 23

24 6. Permutation test Build weight maps from permutation tests (e.g. Mourão- Miranda et al., 2005) and associate a p-value to each weight. Simulated data. Image from Gaonkar and Davatzikos,

25 6. Permutation test Map can be thresholded Used the full pattern to build the model + - Computationally expensive Need to correct for multiple comparisons (cannot make the hypothesis of independent voxels/weights) Note: analytic approximation of null distribution for SVM (Gaonkar and Davatzikos, 2013). 25

26 7. A posteriori knowledge: weight summarization (Schrouff et al., 2013) Build weight maps then summarize weights according to anatomically defined regions (e.g. based on an atlas). Mathematically: NW ROI = v ROI W v #v ROI 26

27 7. A posteriori knowledge: weight summarization AAL atlas (Tzourio-Mazoyer, 2002). 27

28 7. A posteriori knowledge: weight summarization + - Anatomically/ functionally relevant Used the full pattern to build the model Sorted list of regions according to their proportion of NW ROI No thresholding of the sorted list of regions Depends on atlas 28

29 How about electrophysiological recordings? - All strategies explained previously apply - For MKL, can be applied on each dimension (i.e. channels, time point, frequency band) or combination (e.g. channel + frequency band) 29

30 How about non-linear models? - Statistical sensitivity maps (Rasmussen et al., 2011) for SVM, kernel logistic regression and kernel Fisher discriminant. sensitivity map (linear) = (weight filter) 2 - Use of non-linear kernels in neuroimaging? (Kragel et al., 2012) 30

31 Conclusions: - Interpreting machine learning based models can be complex. - Different strategies exist, but always a trade-off between different parameters: Thresholded list of voxels/regions contributing Anatomical/functional relevance Multivariate power Stability of the pattern Computational time Your (informed) choice! 31

32 Acknowledgments: - Belgian American Educational Foundation (BAEF) - Medical Foundation of the Liège Rotary Club - Stanford University - University College London - Wellcome Trust Figures created using PRoNTo version 2.0 ( 32

33 References: - Baldassarre, L., J. Mourao-Miranda, M. Pontil. «Structured Sparsity Models for Brain Decoding from fmri data.» Proceedings of the 2nd conference on Pattern Recognition in NeuroImaging Gaonkar, B., C. Davatzikos. «Analytic estimation of statistical significance maps for support vector machine based multi-variate image analysis and classification.» NeuroImage 78 (2013): Etzel, J., A. Jeffrey, M. Zacks, T.S. Braver. «Searchlight analysis: promise, pitfalls, and potential.» Neuroimage 78 (2013): Filippone, M., A. Marquand, C. Blain, C. Williams, J. Mourao-Miranda, M. Girolami. «Probabilistic prediction of neurological disorders with a statistical assessement of neuroimaging data modalities.» Annals of Applied Statistics 6 (2012): Haufe, S., F. Meinecke, K. Görgen, S. Dhäne, J.-D. Haynes, B. Blankertz, F. Biessmann. «On the interpretation of weight vectors of linear models in multivariate neuroimaging.» NeuroImage 87 (2014): Kragel, P., R. Carter, S. Huettel. «What makes a pattern? Matching decoding methods to data in multivariate pattern analysis.» Front Neurosci. 6 (2012): Kriegeskorte, N., R. Goebel, P. Bandettini. «Information-based functional brain mapping.» PNAS 103 (2006): Mourão-Miranda, J., A. Bokde, C. Born, H. Hampel, M. Stetter. «Classifying brain states and determining the discriminating activation patterns: Support Vector Machine on functional MRI data.» NeuroImage 28 (2005):

34 References: - Rakotomamonjy, A., F. Bach, S. Canu, et Y. Grandvalet. «SimpleMKL.» Journal of Machine Learning 9 (2008): Rasmussen, P., K. Madsen, T. Lund, L.K. Hansen. «Visualization of nonlinear kernel models in neuroimaging by sensitivity maps.» NeuroImage 55 (2011): Schrouff, J., J. Cremers, G. Garraux, L. Baldassarre, J. Mourão-Miranda, et C. Phillips. «Localizing and comparing weight maps generated from linear kernel machine learning models.» Proceedings of the 3rd workshop on Pattern Recognition in NeuroImaging Tibshirani, R. «Regression shrinkage and selection via the lasso.» Journal of the Royal Statistical Society. 58 (1996): Tzourio-Mazoyer, N., et al. «Automated Anatomical Labeling of activations in SPM using a Macroscopic Anatomical Parcellation of the MNI MRI single-subject brain.» NeuroImage 15 (2002): Varoquaux, G., A. Gramfort, B. Thirion. «Small-sample brain mapping: sparse recovery on spatially correlated designs with randomization and clustering.» Proceedings of ICML (2012). icml.cc/discuss/2012/688.html - Zou, H., et T. Hastie. «Regularization and variable selection via the elastic net.» J. R. Statist. Soc. B 67 (2005):