Statistical Methods in functional MRI. Classification and Prediction. Data Processing Pipeline. Machine Learning. Machine Learning

Transcription

1 Statstcal Methods n functonal MRI Lecture 10: Predcton and Bran Decodng 05/0/13 Martn Lndqust Department of Bostatstcs Johns Hopkns Unversty Data Processng Ppelne Classfcaton and Predcton Expermental Desgn There s a growng nterest n usng fmri data for classfcaton of mental dsorders and predctng the early onset of dsease. Data Acqus5on Reconstruc5on Preprocessng Slce-tme Correcton Data Analyss Localzng Bran Actvty In addton, there s nterest n developng methods for predctng stmul drectly from functonal data. Moton Correcton, Co-regstraton & Normalzaton Spatal Smoothng Connectvty Predcton Ths opens the possblty of nferrng nformaton about subjectve human experence drectly from bran actvaton patterns. Machne Learnng Predctng bran states s a challengng process that requres the applcaton of novel statstcal and machne learnng technques. Varous technques have successfully been appled to fmri data n whch a classfer s traned to dscrmnate between dfferent bran states and then used to predct the bran states n a new set of fmri data. Machne Learnng When appled to fmri data the result s often a pattern of weghts across bran regons that can be appled prospectvely to new bran actvaton maps to quantfy the degree to whch the pattern responds to a partcular type of event. w T x > 0 Group A w T x < 0 Group B x =(x 1,.x V ) w =(w 1,.w V ) 1

2 MVPA Multvarate Analyss The applcaton of machne learnng methods to fmri data s often referred to as mult-voxel pattern analyss (MVPA) Instead of focusng on sngle voxels, MVPA uses pattern-classfcaton algorthms appled to multple voxels to decode the patterns of actvty. Haynes et al. MVPA vs GLM In MVPA the goal s to determne the model parameters that allow for the most accurate predcton of new observatons. Seek to create rules that can be used to categorze new observatons. In contrast, the GLM seeks to determne the model parameters that best ft the data at hand. Classfers A classfer s a functon f(.) that takes the values of observed features (e.g., voxels) and predcts to whch class the observaton belongs (e.g., dsease state). Let us denote the set of features x=(x 1, x V ) and the class label y. ^ Predcted class: y = f(x) Tranng Data A classfer has a number of parameters w that needed to be estmated, or learned. The learnng s typcally performed on a subset of the observatons called the tranng data. The learned classfer models the relatonshp between the features and class labels n the tranng data set. Test Data Once traned, the classfer s evaluated usng an ndependent set of observatons called the test data. If the classfer truly captures the relatonshp between features and classes, t should be able to predct the class label for data t hasn t seen before. The accuracy of the classfer measures the fracton of observatons n the test data for whch the correct label was predcted.

3 Illustraton Illustraton Features (voxels) x =(x 1,.x V ) Class Labels y The full data set s splt nto two parts: tranng and test data Tranng Data Observa5ons Data Test Data Illustraton Performng MVPA The process of performng MVPA follows a seres of steps: Defnng features and classes Tranng Data Feature selecton Test Data Classfer Predcted labels True labels Choosng a classfer Tranng and testng the classfer Examnng results Defnng Features There are many possble choces of what nformaton should be used as features. Raw fmri data over both space and tme Averaged fmri data over a block Beta values from a GLM analyss Defnng Classes The choce of whch class labels to use depends upon the research queston. Stmulus class Subject response or decson Any measurement that can be ted to an observaton Average of several voxels n an ROI 3

4 Feature Selecton In fmri the number of features s typcally many tmes larger than the number of observatons. Hence, t s usually benefcal to reduce the number of features through feature selecton. Ths could nvolve usng only voxels from a partcular ROI, dmenson reducton technques (SVD or PCA) or meta-analyss data. Feature Selecton Note that t s not permssble to select voxels that appear to dstngush between classes usng nformaton from the entre data set. Informaton n the test data set may affect the learnng of the classfer and bas subsequent accuracy measures. Classfers There are many types of classfers to choose between that vary n the knds of statstcal relatonshps they can detect. We often dscrmnate between lnear and non-lnear classfers. Lnear Classfers Lnear classfer: w T x + b > 0 In V-dmensons ths defnes a V-1 dmensonal hyperplane w s a V-dmensonal vector of weghts b s a threshold The nner product s zero when vectors are orthogonal, so the equaton w T x = 0 defnes a lne orthogonal to w. Lnear Classfers There exst many types of lnear classfers. w = [1 1] T, b =.5 Some examples nclude: Lnear Support Vector Machnes (SVM) Logstc Regresson (LR) Gaussan Nave Bayes (GNB) Fsher's Lnear Dscrmnant Analyss (LDA) 4

5 SVM SVMs maxmze the margn around the separatng hyperplane. If there are no ponts near the decson surface, then there are no uncertan classfcaton decsons SVM The decson functon s fully specfed by a subset of tranng samples, the support vectors. Support Vector Support Vector Support Vector Support Vector Machnes Consder (x, y ), =1, N, where x R p and y {-1,1}. Solve the convex optmzaton problem: Separable Data sets that can be separated exactly by a lnear boundary are sad to be lnearly separable. Mnmze: 1 w X 1/ w Lnearly separable Not lnearly separable Subject to T y ( w x b) 1 Solvng SVMs s a quadratc programmng problem X 1 Slack Varable When the data s not lnearly separable, we may stll use a lnear classfer by allowng certan data ponts to be on the wrong sde of the boundary. However, they ncur a penalty that ncreases wth ther dstance from the boundary. Support Vector Machnes Consder (x, y ), =1, N, where x R p and y {-1,1}. Solve the convex optmzaton problem: Mnmze: 1 w + C N ξ = 1 X 1/ w To mplement ths we can ntroduce slack varables ξ 0 to allow msclassfcaton of dffcult or nosy observatons. Subject to y T ( w x b) 1 ξ 0 ξ ξ ξ j X 1 5

6 Incorporatng Nonlnearty Data sets that are lnearly separable tend to be easy to work wth. If the data sn t lnearly separable we can always ntroduce slack varables. Another opton s to map the data to a hgherdmensonal space where the tranng set s separable wth a lnear classfer. Nonlnear Classfcaton 0 x 0 x x 0 x Nonlnear Classfcaton Φ: x φ(x) Tranng and Testng To accurately assess the performance of a classfer when appled to a new data set, t s crtcal to use separate data to tran and test the classfer. Ideally, we would lke to tran a classfer usng as much of the avalable data as possble. However, ths leaves lttle to test wth. One approach to balance ths problem s to use cross valdaton. K-fold Cross-valdaton Procedure: Dvde dataset nto K parts, or folds. Leave one fold out Tran on the remanng K-1 folds Predct observatons on omtted fold Repeat for each fold n turn Compute accuracy of all predctons made Choosng K If unbased accuracy assessment s mportant, than use more folds. Less error usng large tranng data. If parameter optmzaton s mportant, than use fewer folds. More stable estmate of error usng large test data. In practce, 5 or 10 fold s often used as a compromse. 6

7 Comments When performng cross-valdaton t s mportant that each fold contans observatons from each class. The classes should be roughly balanced n the cross-valdaton procedure. Stratfcaton can be used to guarantee that each class s adequately represented. One should be careful to nclude correlated observatons n the same fold. Comments Cross-valdaton provdes a method for choosng between dfferent types of classfers and determnng certan parameter estmates. Fnal classfer weghts can be obtaned n a number of ways, such as averagng the weghts across folds. Assessng Accuracy We often want to determne the accuracy of a classfer to determne whether t works better than chance. A smple approach s to use a bnomal test wth p(success)=0.5 per tral. A more accurate way to quantfy performance s to use resamplng methods. Comments It s mportant to examne the accuracy for all classes of observatons, rather than computng the overall accuracy across observatons. If almost all of the observatons fall n the same class than assgnng all new observatons to that class wll gve a hgh overall accuracy. Comments For example, f 90% of all observatons fall n class A then a classfer that always assgns new observatons to class A wll have 90% overall accuracy. It wll also have a 100% accuracy of correctly classfyng class A observaton. However, t wll have a 0% accuracy of correctly classfyng class B observatons. Weght Maps An mportant queston s determnng whch voxels drve the classfcaton. The classfer weghts can be mapped onto the bran to provde nformaton about each voxels contrbuton to the classfer performance. Dfferent classfers may provde dfferent maps as they are senstve to dfferent features. w =(w 1,.w V ) 7

8 Interpretablty In fmri t s mportant to make analyss choces that balance nterpretablty wth predctve power. Certan methods may gve good predctons but the resultng voxel weghts may be dffcult to nterpret and may not generalze to new subjects. Partcpants (n=0) receved a seres of thermal stmulatons for 1 trals at each of four ntenstes: nnocuous warmth and three levels of ncreasngly panful heat. Each tral conssted of separate perods of antcpaton, thermal pan and pan recall. The outcome measure was a tral-specfc pan ratng reported on a contnuous vsual analogue scale. Cue Pan ISI 1 Ratng ISI 8 s 10 s 14 s 4 s 10 s Analyss Feature selecton: Performed a meta-analyss of 4 prevous studes to select ~33,000 voxels assocated wth pan. Averaged bran actvty at each temperature for each partcpant over a 16s post-stmulus wndow. Averaged the 1 repettons at each temperature for each partcpant to yeld 4 unque maps per subject. Analyss Classfer: Support vector regresson. Other classfer technques gave comparable results. Tranng and testng: Used leave-one-subject out cross-valdaton. Labels: Average pan response for each partcpant n each condton. Weght Maps Results Pan- predc5ve bomarker pazern Precuneus SMA dacc dacc Hy sthal vmpfc CB LeX Rght IFJ MOG AIns CB Fus IFJ S S dpins MOG Nega5ve predc5ve weghts PAG IOG dacc mdins sthal SPL Pos5ve predc5ve weghts vmpfc Hy vlthal SMG MTG ains PCC Predcted pan report Cross-valdated predcton of pan Pan vs. other affectve events Physcal Heat Antcpaton Bomarker Response Pan recall dpins IOG mdins Z Z x = - 40 x = 44 Pan report Stmulus ntensty level 8

9 Analyss Partcpants (n=41; Kross et al., 011) had all recently been rejected n a romantc relatonshp. Test accuracy usng bomarker from Study They were asked to brng pctures of the former partner and a non-rejectng frend. Accuracy 0.9 Experment conssted of four randomly ntermxed condtons: (1) vewng the rejector; () vewng the frend; (3) recevng hgh thermal pan; (4) recevng low thermal pan Subjects reported perceved pan after each tral. Test Reject vs. Frend Test Hot vs. Warm Weght maps from cross- valdated analyss Hot vs. Warm 5 13 Cross- valdated analyss Study x = x = x = - 3 Test Hot vs. Warm Test Reject vs. Frend 0.9 Accuracy Dfference [H- W], [R- F] correla5on 1.0 Reject vs. Frend Regon number Rej - Frend Tran Hot vs. Warm Rej - Frend Rej - Frend Tran Reject vs. Frend Hot - Warm z = - 6 Hot - Warm Hot - Warm #1 medal thalamus #4 lex anteror nsula #7 anteror cngulate z = - 33 Partcpants (n=1) receved two ntravenous nfusons of remfentanl. In the open nfuson run partcpants were told they receved the drug. In the hdden nfuson run partcpants were ncorrectly told they receved no drug. A seres of ndvdually calbrated panful and warm stmul were delvered before, durng and after drug admnstraton. How s the the bomarker affected by drug nfuson? Results demonstrate a strong remfentanlnduced modulaton of the bomarker response, but no modulaton by psychologcal context. 9