Multi-Voxel Pattern Analysis (MVPA) for fmri

Transcription

1 MVPA for - and Multi-Voxel Pattern Analysis (MVPA) for 7 th M-BIC Workshop 2012 Maastricht University, Department of Cognitive Neuroscience Maastricht Brain Imaging Center (M-Bic), Maastricht, The Netherlands 22 March 2012

2 Overview MVPA for - and 1 2 Machine learning for : Principles 3 and

3 Univariate Statistics for MVPA for - and Functional images Time Conventional (voxel-by-voxel) statistics ~2s Signal (% change) Voxel Time Course Time Condition Map superimposed on anatomical MRI image Voxel/ Region of interest (ROI) ~ 5 min

4 s for MVPA for - and

5 s for MVPA for - and Does the activity in the two voxels covary? Functional connectivity

6 s for MVPA for - and Does the activity in the two voxels covary? Functional connectivity Does the activity in one voxel influence the activity in the other? Effective connectivity

7 s for MVPA for - and Does the activity in the two voxels covary? Functional connectivity Does the activity in one voxel influence the activity in the other? Effective connectivity Do they jointly convey information on a stimulus or a brain state? Multi-voxel pattern analysis (MVPA), brain reading

8 Multi-Voxel Pattern Analysis MVPA for - and To deal with multiple independent variables and multiple dependent variables, it is possible to employ multivariate statistical tests, e.g. Analysis of Variance (MANOVA) or Canonical Variate Analysis (CVA). Each independent variable takes up a degree of freedom, therefore in case of large amount of voxels, it is troublesome to employ them.

9 Multi-Voxel Pattern Analysis MVPA for - and To deal with multiple independent variables and multiple dependent variables, it is possible to employ multivariate statistical tests, e.g. Analysis of Variance (MANOVA) or Canonical Variate Analysis (CVA). Each independent variable takes up a degree of freedom, therefore in case of large amount of voxels, it is troublesome to employ them. Reduce the amount of voxels using a searchlight approach, [Kriegeskorte06]) Reduce the amount of voxels using PCA, PLS, MLM, [Friston95a, McIntosh96, Worsley97] Test for a multivariate effect in a different way... (MVPA)

10 Multi-Voxel Pattern Analysis MVPA for - and Distributed representation of faces and objects in ventral temporal cortex [Haxby01] Object photos presented in category blocks Haxby et al. (2001)

11 Multi-Voxel Pattern Analysis MVPA for - and L R Correlation analysis of spatial response patterns Haxby et al. (2001)

12 Multi-Voxel Pattern Analysis MVPA for - and Pattern comparison by category pair Haxby et al. (2001)

13 Multi-Voxel Pattern Analysis MVPA for - and analysis can handle weak-at-every-voxel distributed effects opposite selectivity in adjacent voxels

14 Multi-Voxel Pattern Analysis MVPA for - and Patterns as points

15 Multi-Voxel Pattern Analysis MVPA for - An Aid to Intuition about Analysis: Part 1 and Ideal Univariate Data Voxel 2 Activity Condition A Condition B Baseline Activity David Cox and Robert Savoy

16 Multi-Voxel Pattern Analysis MVPA for - and An Aid to Intuition about Analysis: Part 2 Linearly Separable, but Problematic in Univariate Analyses Condition A Condition B Baseline Voxel 2 Activity Activity David Cox and Robert Savoy

17 Multi-Voxel Pattern Analysis MVPA for - An Aid to Intuition about Analysis: Part 3 and Nonlinearly Separable Data Voxel 2 Activity Voxel 1 Activity Condition A Condition B Baseline Activity Voxel 1 Voxel 2 David Cox and Robert Savoy

18 : an example from [Cox03] MVPA for - and Basic Format of the Experiments and Analysis: Part 1 Brain Reading : Classifying experience using patterns of neural activity TRAINING classifier label: basket David Cox and Robert Savoy voxels

19 : an example from [Cox03] MVPA for - Basic Format of the Experiments and Analysis: Part 2 and CLASSIFICATION of data from a subsequent session best guess: basket Trained classifier David Cox and Robert Savoy voxels

20 : an example from [Cox03] MVPA for - of Stimuli and Stimuli We have begun by using complex, real world, visual objects, from 10 different groups, presented as gray-scale images. David Cox and Robert Savoy

21 norm. MR signal : an example from [Cox03] MVPA for - Comparing 10 classifiers (from one subject) and No one voxel or small subset of voxels is significantly different between category for single trials voxel number David Cox and Robert Savoy

22 : an example from [Cox03] MVPA for - Effect of Voxel Set Size and David Cox and Robert Savoy

23 example: PBAIC 2007 competition MVPA for - and PBAIC 2007: Pittsburgh Brain Activity Interpretation Competition Interpreting subject-driven actions and sensory experience in a rigorously characterized virtual world,organized by Walter Schneider and Greg Siegle of the University of Pittsburgh, Figure 1: c 2007 University of Pittsburgh

24 example: PBAIC 2007 competition MVPA for - and 3 subject operating in a virtual reality world Novel Virtual Reality world task: searching for and collecting objects, interpreting changing instructions, and avoiding a threatening dog Learn from Run1 and Run2 and in Run 3 predict feature vectors from brain activity data Figure 2: c 2007 University of Pittsburgh

25 example: PBAIC 2007 competition MVPA for - and Use the data from Run1 and Run2 to develop the ability to go from the brain activation data to the feature data Figure 3: c 2007 University of Pittsburgh

26 example: PBAIC 2007 competition MVPA for - Predict unknown ratings from data of Run3 and Figure 4: c 2007 University of Pittsburgh

27 example: PBAIC 2007 competition MVPA for - Using Relevance Vector Machine [Tipping01, Valente11] and

28 Overview MVPA for - and 1 2 Machine learning for : Principles 3 and

29 for MVPA for - and Consider the following datasets: t can be Dataset1: Data X and labels t (Training dataset) Dataset2: Data X and labels t (Test dataset, not available during training!) Discrete: (or ordinal regression) Continuous: With machine learning we aim at finding, on the training data X and t, a suitable function f = f(x, θ) where θ denotes a set of parameters, such that t = f(x, θ) is a good estimate of t

30 for MVPA for - and The learnt f should be such that it minimizes a measure of error on the test dataset (generalization). The most employed measures are: : accuracy, Area under the Curve : Correlation, Root Mean Square Error (RMSE)

31 for MVPA for - and The learnt f should be such that it minimizes a measure of error on the test dataset (generalization). The most employed measures are: : accuracy, Area under the Curve : Correlation, Root Mean Square Error (RMSE) How do we learn the?

32 for MVPA for - and The learnt f should be such that it minimizes a measure of error on the test dataset (generalization). The most employed measures are: : accuracy, Area under the Curve : Correlation, Root Mean Square Error (RMSE) How do we learn the? Minimizing the error on the training data?

33 MVPA for - and Consider a simple 1-D example (from [Bishop06]). To learn a functional relationship (ideal: GREEN) on training data (RED) and generalize on test data (BLUE), we use a polynomial : y = a 0 +a 1 x+a 2 x a n x n Real TRAINING data TESTING data

34 MVPA for - and We optimize the polynomial coefficients by minimizing the least-square error ɛ (Maximum Likelihood (ML) solution, if the noise has a Gaussian distribution) (a) order 2 (b) order 3 (c) order 5

35 MVPA for - and The RMS error decreases with the polynomial order increasing (d) order 7 (e) order 8 (f) order 10

36 MVPA for - and Examining the performances on the test dataset: (a) order 2 (b) order 3 (c) order 5

37 MVPA for - and A better fit on the training dataset does not always imply a better fit on the test dataset (d) order 7 (e) order 8 (f) order 10

38 MVPA for - and Error on Training Dataset Error on Testing Dataset With a high order polynomial we fit perfectly the training data, but have higher error on test data. In this case training data have been OVERFITTED: we have considered noise as interesting signal

39 MVPA for - and Sample size ESTIMATION PROCEDURE Overfitting Model complexity Given the available samples, our was too complex. A simpler would have better performances if more samples were available, there will be less overfitting with the same polynomial order. The estimation procedure (Least Square) tends to favor overfitting of the training data.

40 MVPA for - and Linear s are the preferred choice: simple and they makes it easy to map voxel s relevance or sensitivity

41 MVPA for - and Linear s are the preferred choice: simple and they makes it easy to map voxel s relevance or sensitivity In, there can be around 10 5 voxels, and typically few hundreds (regression) or few tens (classification) examples extremely high risk of overfitting, even with linear s.

42 MVPA for - and Linear s are the preferred choice: simple and they makes it easy to map voxel s relevance or sensitivity In, there can be around 10 5 voxels, and typically few hundreds (regression) or few tens (classification) examples extremely high risk of overfitting, even with linear s. Avoid Least Square fitting, but use regularized s (Shrinkage, Tikhonov Regularization, Automatic Relevance Determination).

43 Overview MVPA for - and 1 2 Machine learning for : Principles 3 and

44 MVPA MVPA for - and

45 and MVPA for - and Pre-processing similar to standard data analyses Little or no spatial smoothing (see [OpdeBeeck10] and responses)

46 and MVPA for - and Pre-processing similar to standard data analyses Little or no spatial smoothing (see [OpdeBeeck10] and responses) Pre-processing of single trials Normalization (% signal change) Linear trend removal Artifact and outliers rejection.

47 MVPA for - and For Blocked design or slow-event related design, information from single trials can be summarized with: Average in a specific time window (e.g. from 2 to 4 volumes after stimulus onset), eventually normalized by a baseline estimation. Fitting a single trial GLM (with a standard HRF and an intercept) and considering β values or t-values. No clear-cut criterion to choose feature extraction. For a comparison of feature extraction strategies (on two regions in the visual cortex during a visual experiment) see [Misaki10].

48 MVPA for - and For fast-event related designs (no separated trials) Considerably more troublesome for MVPA than blocked or slow-event related designs Particular care in separating events (possibly leave a whole run out), otherwise biased results Consider the acquisition 4-6 sec. after stimulus onset [Beauchamp09], or more complex deconvolution approaches [Mumford12].

49 MVPA for - and Kernel methods (such as SVM) can deal with large dimensions (e.g. whole brain). However, when only few voxels convey discriminative information, it is important to reduce the amount of dimensions prior to training.

50 MVPA for - and Kernel methods (such as SVM) can deal with large dimensions (e.g. whole brain). However, when only few voxels convey discriminative information, it is important to reduce the amount of dimensions prior to training. Anatomical information, e.g. ROI approach. Strong a priori assumptions are needed

51 MVPA for - and Kernel methods (such as SVM) can deal with large dimensions (e.g. whole brain). However, when only few voxels convey discriminative information, it is important to reduce the amount of dimensions prior to training. Anatomical information, e.g. ROI approach. Strong a priori assumptions are needed Univariate (on the training data!). Fast, but discards multivariate nature of patterns

52 MVPA for - and Kernel methods (such as SVM) can deal with large dimensions (e.g. whole brain). However, when only few voxels convey discriminative information, it is important to reduce the amount of dimensions prior to training. Anatomical information, e.g. ROI approach. Strong a priori assumptions are needed Univariate (on the training data!). Fast, but discards multivariate nature of patterns Wrappers: Recursive Elimination [Guyon03, DeMartino08] Embedded: Sparse Bayes Learning [Li02, Yamashita08] Computationally more demanding, but better to identify discriminative regions, when compared to univariate

53 Recursive Elimination [DeMartino08] MVPA for - 46 F. De Martino et al. / NeuroImage 43 (2008) and Fig.1. General description of the proposed SVM/RFE iterative procedure to brain mapping. After single trial-response estimation functional time series are divided in training and test data sets (Train k; Test k). An optional step of voxel reduction can be performed prior to RFE using only the training data (Train k). For each voxel selection level the recursive procedure (RFE; red dashed box in figure) consists of two steps. First an SVM classifier is trained on a subset of the training data (Train RFEi) using the current set of voxels. Second a set of voxels is

54 Model Training MVPA for - and Training MUST be done on separate data. Several strategies can be employed Leave Run Out: Most conservative option Problems due to across-run variability of BOLD signal (and noise) Leave One Out (LOO) cross-validation: Uses a larger training dataset (n 1 examples) more complex, better generalization Small bias, large variance Computationally demanding K-fold Cross Validation: Computationally attractive Uses less training samples than LOO, therefore less accurate (more bias, but less variance)

55 MVPA for - and lgorithms, Generative: everything about classes Discriminative: learn a rule (hyperplane in the linear case) to discriminate between the two classes. Learn a smaller problem, better suited with small sample size. Many algorithms currently available Linear Discriminant Analysis Fisher Discriminant Analysis Support Vector Machines Gaussian Naïve Bayes classifier Logistic Relevance Vector Machine Gaussian Processes

56 Discriminative maps MVPA for - and Figure 5: Separating hyperplanes, source: Discriminative linear (e.g. SVM), decision rule: y i = sign ( w T x i + b ) The coefficient w i denotes the inclination of the hyperplane in the i-th dimension discriminative maps. With non-linear kernels it is not possible to have discriminative maps Sensitivity maps [Rasmussen11]

57 MVPA for - and For balanced classes, average accuracy (or error) can be used as a measure of goodness of classification. Chance level is equal to 1/ classes (for instance, 50 % with two classes, 25 % with four classes). For unbalanced classes there is no unique chance level, and average accuracy may not be meaningful. It is better to use class-specific accuracies, or Recall and Precision.

58 MVPA for - and For balanced classes, average accuracy (or error) can be used as a measure of goodness of classification. Chance level is equal to 1/ classes (for instance, 50 % with two classes, 25 % with four classes). For unbalanced classes there is no unique chance level, and average accuracy may not be meaningful. It is better to use class-specific accuracies, or Recall and Precision. Consider a learning that has a LOO accuracy of 60 % on a two-class problem. Is this accuracy enough (i.e. significantly different from chance level)?

59 MVPA for - and Single subject Parametric : characterizing the null distribution (Binomial or Multinomial). It is very fast and easy to calculate, but it requires i.i.d. samples. With cross-validation approaches this is not true (each test sample is also used to train other s). Non parametric s: permutation test, that estimates the empirical distribution of the null hypothesis. More computationally demanding, but less assumptions.

60 MVPA for - and Single subject Parametric : characterizing the null distribution (Binomial or Multinomial). It is very fast and easy to calculate, but it requires i.i.d. samples. With cross-validation approaches this is not true (each test sample is also used to train other s). Non parametric s: permutation test, that estimates the empirical distribution of the null hypothesis. More computationally demanding, but less assumptions. Performing statistics on the cross-validation accuracies (in K-fold cross-validation) is not a correct procedure, as the variance depends on the chosen K

61 MVPA for - and Single subject Parametric : characterizing the null distribution (Binomial or Multinomial). It is very fast and easy to calculate, but it requires i.i.d. samples. With cross-validation approaches this is not true (each test sample is also used to train other s). Non parametric s: permutation test, that estimates the empirical distribution of the null hypothesis. More computationally demanding, but less assumptions. Performing statistics on the cross-validation accuracies (in K-fold cross-validation) is not a correct procedure, as the variance depends on the chosen K Group study: Random effect analysis, comparing the accuracies of the subjects and their theoretical (empirical) null distribution. t-tests are used, but non-parametric statistics are a better option (accuracies are not normally distributed)

62 MVPA for - and Thank you for your attention! Q&A

63 Bibliography I MVPA for - and Michael S. Beauchamp, Stephen LaConte, and Nafi Yasar. Distributed representation of single touches in somatosensory and visual cortex. Human Brain Mapping, 30(10): , Christopher M. Bishop. Pattern Recognition and (Information Science and Statistics). Springer, David D. Cox and Robert L. Savoy. Functional magnetic resonance imaging () brain reading : detecting and classifying distributed patterns of activity in human visual cortex. NeuroImage, 19(2): , 2003.

64 Bibliography II MVPA for - and Federico De Martino,, Noel Staeren, John Ashburner, Rainer Goebel, and Elia Formisano. Combining multivariate voxel selection and support vector machines for mapping and classification of fmri spatial patterns. NeuroImage, 43:44 58, K.J. Friston, C. Frith, R.S.J. Frackowiak, and R. Turner. Characterizing dynamic brain responses with : A multivariate approach. NeuroImage, 2: , Keyword: connectivity,dynamical,multivariate, Isabelle Guyon and Andr Elisseeff. An introduction to variable and feature selection. Journal of Research, 3: , 2003.

65 Bibliography III MVPA for - and James V. Haxby, M. Ida Gobbini, Maura L. Furey, Alumit Ishai, Jennifer L. Schouten, and Pietro Pietrini. Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science, 293(5539): , N. Kriegeskorte, R. Goebel, and P. Bandettini. Information-based functional brain mapping. Proceedings of the National Academy of Sciences of the United States of America, 103: , Y. Li, C. Campbell, and M. Tipping. Bayesian automatic relevance determination algorithms for classifying gene expression data. Bioinformatics, 18: , 2002.

66 Bibliography IV MVPA for - and Anthony Randal McIntosh and Nancy J. Lobaugh. Partial least squares analysis of neuroimaging data: applications and advances. NeuroImage, 23(Supplement 1):S250 S263, Mathematics in Brain Imaging. Masaya Misaki, Youn Kim, Peter A. Bandettini, and Nikolaus Kriegeskorte. Comparison of multivariate classifiers and response normalizations for pattern-information fmri. NeuroImage, 53(1): , Jeanette A. Mumford, Benjamin O. Turner, F. Gregory Ashby, and Russell A. Poldrack. Deconvolving bold activation in event-related designs for multivoxel pattern classification analyses. NeuroImage, 59(3): , 2012.

67 Bibliography V MVPA for - and Hans P. Op de Beeck. Probing the mysterious underpinnings of multi-voxel fmri analyses. NeuroImage, 50(2): , Peter Mondrup Rasmussen, Kristoffer Hougaard Madsen, Torben Ellegaard Lund, and Lars Kai Hansen. Visualization of nonlinear kernel s in neuroimaging by sensitivity maps. NeuroImage, 55(3): , Michael E. Tipping. Sparse bayesian learning and the relevance vector machine. Journal of Research, 1: , 2001.

68 Bibliography VI MVPA for - and, Federico De Martino, Fabrizio Esposito, Rainer Goebel, and Elia Formisano. Predicting subject-driven actions and sensory experience in a virtual world with relevance vector machine regression of data. NeuroImage, 56(2): , Decoding and Brain Reading. K.J. Worsley, J-B. Poline, K.J. Friston, and A.C. Evans. Characterizing the response of PET and data using multivariate linear s. NeuroImage, 6(4): , O. Yamashita, M.-A. Sato, T. Yoshioka, F. Tong, and Y. Kamitani. Sparse estimation automatically selects voxels relevant for the decoding of activity patterns. NeuroImage, 42: , 2008.