Modeling Low-Frequency Fluctuation and Hemodynamic Response Timecourse in Event-Related fmri

Transcription

1 Human Bain Mapping 29: (2008) TECHNICAL REPORT Modeling Low-Fequency Fluctuation and Hemodynamic Response Timecouse in Event-Related fmri Kendick N. Kay, 1 Stephen V. David, 2 Ryan J. Penge, 3 Kathleen A. Hansen, 1 and Jack L. Gallant 1,4 * 1 Depatment of Psychology, Univesity of Califonia, Bekeley, Califonia 2 Depatment of Bioengineeing, Univesity of Califonia, Bekeley, Califonia 3 Depatment of Physics, Univesity of Califonia, Bekeley, Califonia 4 Helen Wills Neuoscience Institute, Univesity of Califonia, Bekeley, Califonia Abstact: Functional magnetic esonance imaging (fmri) suffes fom many poblems that make signal estimation difficult. These include vaiation in the hemodynamic esponse acoss voxels and low signal-to-noise atio (SNR). We evaluate seveal analysis techniques that addess these poblems fo event-elated fmri. (1) Many fmri analyses assume a canonical hemodynamic esponse function, but this assumption may lead to inaccuate data models. By adopting the finite impulse esponse model, we show that voxel-specific hemodynamic esponse functions can be estimated diectly fom the data. (2) Thee is a lage amount of low-fequency noise fluctuation (LFF) in blood oxygenation level dependent (BOLD) time-seies data. To compensate fo this poblem, we use polynomials as egessos fo LFF. We show that this technique substantially impoves SNR and is moe accuate than high-pass filteing of the data. (3) Model ovefitting is a poblem fo the finite impulse esponse model because of the low SNR of the BOLD esponse. To educe ovefitting, we estimate a hemodynamic esponse timecouse fo each voxel and incopoate the constaint of time-event sepaability, the constaint that hemodynamic esponses acoss event types ae identical up to a scale facto. We show that this technique substantially impoves the accuacy of hemodynamic esponse estimates and can be computed efficiently. Fo the analysis techniques we pesent, we evaluate impovement in modeling accuacy via 10-fold coss-validation. Hum Bain Mapp 29: , VC 2007 Wiley-Liss, Inc. Key wods: hemodynamic esponse function; low-fequency noise; model evaluation; coss-validation; evese coelation Contact gant sponsos: National Institute of Mental Health; The National Eye Institute. Stephen V. David is cuently at Institute fo Systems Reseach, Univesity of Mayland, College Pak, MD 20742, USA. Kathleen A. Hansen is cuently at Laboatoy of Bain and Cognition, NIMH, Bethesda, MD 20892, USA. *Coespondence to: Jack L. Gallant, Univesity of Califonia at Bekeley, 3210 Tolman Hall No. 1650, Bekeley, CA 94720, USA. gallant@bekeley.edu. Received fo publication 10 Apil 2006; Revision 13 Octobe 2006; Accepted 2 Januay 2007 DOI: /hbm Published online 29 Mach 2007 in Wiley InteScience (www. intescience.wiley.com). VC 2007 Wiley-Liss, Inc.

2 Event-Related fmri Modeling INTRODUCTION Event-elated functional magnetic esonance imaging (fmri) expeimental designs offe seveal impotant advantages ove block designs: moe efficient estimates of the timing and shape of the hemodynamic esponse (HDR), inceased flexibility in expeimental design and analysis, and eduction of anticipation and adaptation effects [Josephs and Henson, 1999; Zaahn et al., 1997a]. Howeve, event-elated fmri has educed statistical powe fo detecting signal activations [Liu, 2004]. In addition, event-elated fmri inceases the complexity of the data and the assumptions undelying the data analysis (e.g. tempoal lineaity of the BOLD esponse). It is theefoe citical to maximize pecision and accuacy in the analysis of event-elated fmri data. In this study we addess thee poblems in the analysis of event-elated fmri data. Many of the specific techniques we pesent have been published peviously. The goal of the pesent study is to evaluate igoously and systematically the value of these techniques, applied in concet, on empiical data. We emphasize coss-validation pedictive pefomance as an objective metic fo quantifying model accuacy. (This is in contast to such metics as epoducibility and statistical significance, which ae impotant but not diectly elated to model accuacy.) We also emphasize single voxel modeling, which is likely to become inceasingly impotant as the spatial esolution and signal-tonoise atio (SNR) of fmri impove. One poblem in event-elated fmri analysis is vaiation in the HDR acoss voxels [Aguie et al., 1998; Handweke et al., 2004; Miezin et al., 2000; Neumann et al., 2003; Saad et al., 2001]. Although the assumption of a canonical HDR function (HRF) is common in fmri analyses, this assumption may lead to incoect data infeences [Buock and Dale, 2000; Handweke et al., 2004]. We avoid the assumption of an a pioi HRF by adopting the famewok of the finite impulse esponse (FIR) model [Dale, 1999]. Unde the FIR model, a HDR is estimated fo each voxel to each event type, and thee is no constaint on the shape of the esponses. A second poblem is the lage amount of low-fequency noise fluctuation (LFF) in blood oxygenation level dependent (BOLD) time-seies data [Aguie et al., 1997; Pudon and Weisskoff, 1998; Zaahn et al., 1997b]. LFF has been attibuted to scanne and physiological noise [Smith et al., 1999; Zaahn et al., 1997b]. We compensate fo LFF by using polynomials [Liu et al., 2001] as egessos fo the baseline signal level, i.e. the signal level associated with the absence of the stimulus. We show that this technique impoves the SNR and is moe accuate than high-pass filteing of the time-seies data. Moeove, we show that polynomials can poduce moe accuate esults than Fouie basis functions. A thid poblem is model ovefitting. Ovefitting tends to occu when a model has a lage numbe of paametes elative to the amount of available data. To educe ovefitting by the FIR model, we incopoate the constaint of time-event sepaability. This is the constaint that HDR estimates acoss event types ae identical up to a scale facto, and is easonable fo many expeimental paadigms. In a elated study, Hinichs et al. [2000] confimed inceased estimation efficiency unde the time-event sepaable model. We extend thei esults by demonstating a simple, fast method fo fitting the time-event sepaable model and by confiming impoved coss-validation pedictive pefomance. We evaluate the poposed analysis techniques on empiical data. These data wee obtained fom occipital cotex duing bief pesentations of a checkeboad patten at diffeent locations in the visual field. Data fom this expeiment ae especially useful fo methodological development, because the stimulus is tightly contolled, the SNR is obust, and the data ae ichly stuctued. In addition, the shee numbe of activated voxels makes it easy to discen population effects. To maximize pecision, we analyze the data at the single voxel level, with no spatial smoothing o spatial aveaging. We also summaize esults fom data involving othe stimulus designs. MATERIALS AND METHODS Stimulus The stimulus design was simila to that of a pevious study fom ou laboatoy [Hansen et al., 2004]. The stimulus consisted of a 7.5-Hz contast-evesing checkeboad patten pesented within 12 wedges of 308 pola angle width (Fig. 1). The patten had a adial spatial fequency Figue 1. Schematic of visual stimulus. The stimulus consisted of a 7.5-Hz contast-evesing checkeboad patten pesented within 12 wedges in the visual field. A cyclically shifted binay m-sequence contolled the pesentation timing fo each wedge. Each tial lasted 4 s, and thee wee 255 consecutive tials. Fo data analysis we define 12 event types, one event type pe wedge. 143

3 Kay et al. of 12 cycles pe evolution and was scaled with eccenticity. At any given time, the patten was pesented within each wedge at 0% (OFF) o 99% (ON) Michelson contast. The pesentation timing was contolled by an m-sequence of level 2, ode 8, and length ¼ 255. The m- sequence was cyclically shifted by 21 elements to poduce the ON OFF patten fo each wedge. The bin duation of the m-sequence was 4 s, and the total stimulus duation was 255 tials 4s¼ 17 min. Fo each wedge, thee was a total of 128 ON states and 127 OFF states. The minimum, maximum, and mean stimulus onset asynchony fo a given wedge was 4 s, 32 s, and 8 s, espectively. The use of an m-sequence minimizes coelations between wedges and enables efficient estimation of HDRs [Buacas and Boynton, 2002; Liu, 2004]. m-sequences have been used in othe fmri studies [de Zwat et al., 2005; Hansen et al., 2004; Kellman et al., 2003]. Code fo m- sequence geneation was povided by T. Liu ( The stimulus was displayed by an Epson PoweLite 7700p LCD pojecto (Epson Ameica, Long Beach, CA) fitted with a custom zoom lens (Buhl Optical, Rocheste, NY). The image was focused onto a semitanslucent backpojection sceen (Aeoview 100 mateial, Stewat Filmsceen, Toance, CA). The subject viewed the sceen via a fist-suface mio. The viewing distance was 38 cm, and the stimulus subtended of visual angle. An occluding device pevented the subject fom seeing the uneflected image of the sceen. Duing stimulus pesentation, the subject pefomed a change detection task at a cental fixation dot (the mean inteval between changes was 2 s). An optical button esponse box (Cuent Designs, Philadelphia, PA) ecoded subject esponses. The pojecto opeated at a esolution of 1, at 60 Hz. Luminance output was measued using a Minolta LS-110 photomete (Konica Minolta Photo Imaging, Mahwah, NJ), and the luminance esponse was lineaized via a lookup table. The mean luminance of the stimulus was 550 cd/m 2. The stimulus was time-locked to the pojecto efesh ate and synchonized to scanne data acquisition. A Macintosh PoweBook G4 compute (Apple Compute, Cupetino, CA) contolled stimulus pesentation and logged button esponses, using softwae witten in MAT- LAB (The Mathwoks, Natick, MA) and Psychophysics Toolbox 2.53 [Bainad, 1997; Pelli, 1997]. Data Collection The expeimental potocol was appoved by the UC Bekeley Committee fo the Potection of Human Subjects. MRI data wee collected at the Bain Imaging Cente at UC Bekeley using a 4 T INOVA MR scanne (Vaian, Palo Alto, CA) with a whole-body gadient set capable of 35 mt/m with a ise time of 300 ms (Tesla Engineeing, Sussex, UK). A cuvilinea quadatue tansmit/eceive suface coil (Midwest RF, LLC, Hatland, WI) was positioned ove the occipital pole fo enhanced MR SNR. Head motion was minimized with foam padding. Manual shimming of the magnetic field was used to impove image quality and educe image distotion. Coonal slices coveing occipital cotex wee selected: 16 slices, slice thickness 1.8 mm, slice gap 0.2 mm, field-ofview mm 2, matix size 64 64, and nominal esolution 2 2 2mm 3. Fo BOLD data, a T2*-weighted, single-shot, slice-inteleaved, gadient-echo echo plana imaging (EPI) sequence was used: TR 1 s, TE s, flip angle 208. An initial dummy peiod was included to allow magnetization to each steady-state. Duing stimulus pesentation, the fist eight tials wee epeated afte the end of the 255-tial sequence. BOLD data wee collected up though the last tial, and data collected duing the initial 8 s 4 s ¼ 32 s wee ignoed [Kellman et al., 2003]. This stategy avoids potential attentional atifacts at the beginning and end of stimulus pesentation, compensates fo the delay in the HDR, and allows complete sampling of the m-sequence. Data Pepocessing A nonlinea phase coection was applied to the image data to educe Nyquist ghosts and image distotion. Diffeences in slice acquisition times wee coected via sinc intepolation. To compensate fo slow changes in head position, SPM99 motion coection was pefomed with the following modification: motion paamete estimates wee low-pass filteed at 1/20 Hz to emove high-fequency modulations caused by signal activations [Feie and Mangin, 2001]. No additional spatial o tempoal filteing was applied. FIR Model Ou analysis appoach is based on the FIR model fo event-elated fmri [Dale, 1999]. Ou ealie evese coelation appoach [Hansen et al., 2004] is a special case of the FIR model, applicable when stimulus events ae uncoelated. In the FIR model, the BOLD signal is assumed to be a linea, time-invaiant system with espect to the stimulus. A HDR is estimated fo each stimulus event type using a set of shifted delta functions as egessos. No assumption on the shape of HDRs is made. Additional egessos ae used to model the baseline signal level, i.e. the signal level associated with the absence of the stimulus. The model chaacteizes two types of effects in the data: stimulus effects consist of the tansient HDRs to stimulus events, and nuisance effects consist of the pesistent baseline signal level that may vay ove time. Let e be the numbe of event types, l be the numbe of time points in one HDR, m be the numbe of nuisance tems, and t be the numbe of time-seies data points. The time-seies data ae modeled as y¼xh+sb+n, whee y is the data (t 1), X is the stimulus matix (t el), h is the concatenation of the HDR associated with each event type 144

4 Event-Related fmri Modeling Schematic of data models. (A) Stimulus matix X (FIR model). The matix dimensions ae 1,020 time points 252 paametes. The matix is the concatenation of the stimulus convolution matix fo each of the 12 event types. The stimulus convolution matix fo a given event type consists of shifted vesions of a binay sequence, whee ones indicate event occuences. Thee ae 21 shifts, one shift fo each time point in the HDR estimate. The inset (uppe-left) depicts an enlaged view of the paametes fo the fist two event types. (B) Nuisance matix S (polynomial vesion). The matix dimensions ae 1,020 time points 5 paametes. The matix consists of Legende polynomials of Figue 2. degees 0 though 4. The inset (uppe-left) depicts the polynomials in a line fomat. (C) Convolution of stimulus matix X 2 and time kenel k (time-event sepaable model). The matix dimensions ae 1,020 time points 12 paametes. Stimulus matix X 2 (1,020 12) consists of one paamete fo each of the 12 event types. The paamete fo a given event type is a binay sequence, whee ones indicate event occuences. Time kenel k (21 1) is a voxel-specific esponse timecouse estimated fom the data. The inset (uppe-left) depicts an enlaged view of the paametes fo the fist two event types. The inset (left) depicts the time kenel in a line fomat. (el 1), S is the nuisance matix (t m), b is a set of nuisance paametes (m 1), and n is a noise tem (t 1). The stimulus matix is the concatenation of the stimulus convolution matix fo each event type. The stimulus convolution matix fo a given event type consists of shifted vesions of a binay sequence, whee ones indicate event occuences (Fig. 2). Stimulus effects ae given by Xh, and nuisance effects ae given by Sb. Fo ou data, thee ae a total of 1,020 time-seies data points (t ¼ 1,020). We define 12 event types, one event type pe wedge (e ¼ 12). We teat the ON state (99% contast) of a wedge at the beginning of a tial as an event occuence. We estimate a HDR of duation 20 s fo each event type (l ¼ 21). The baseline signal level is the signal level associated with viewing the fixation dot against the gay backgound. Modeling LFF We evaluate seveal vesions of the FIR model. These vesions diffe in how they compensate fo LFF. In the simple vesion of the FIR model, LFF is ignoed and nuisance matix S consists of only a constant tem. This constant tem chaacteizes the baseline signal level as a DC offset in the time-seies data. HDR estimates obtained unde this vesion of the FIR model will be poo if the magnitude of LFF is lage. This is because LFF adds noise to the time-seies data. One stategy fo compensating fo LFF is to include in nuisance matix S egessos that model the timecouse of LFF. This stategy enables the modeled baseline signal level to vay ove time. Fouie basis functions ae commonly used as egessos; in this case, the nuisance matix consists of a constant tem and a set of sine and cosine functions. A diffeent choice of egessos is a set of polynomials of inceasing degee (Fig. 2). We use Legende polynomials [Liu et al., 2001] which ae paiwise othogonal. Equivalent model fits can be obtained with othe sets of polynomials (e.g., 1, t, t 2, etc.) that span the same subspace as Legende polynomials. Anothe stategy fo compensating fo LFF is to detend the time-seies data as a pepocessing step [Kuggel et al., 1999; Machini and Ripley, 2000; Skudlaski et al., 1999; Tanabe et al., 2002]. We use a high-pass filteing technique: we fist emove a linea tend to avoid wap-aound 145

5 Kay et al. effects and then high-pass filte the data. On the filteed data, we fit the simple vesion of the FIR model in which nuisance matix S consists of a constant tem. Time-Event Sepaable Model The FIR model uses a lage numbe of paametes to chaacteize stimulus effects. In ou case, thee ae (e ¼ 12) (l ¼ 21) ¼ 252 paametes in stimulus matix X and only 1,020 data points. Given the limited amount of data available in a typical fmri expeiment, the FIR model isks ovefitting the data. To educe the numbe of model paametes, we incopoate the constaint of time-event sepaability. This is the condition that HDR estimates acoss event types ae identical up to a scale facto. (Moe loosely, time-event sepaability is the condition that the shape of the HDR is the same fo any event type.) Unde the time-event sepaable model, stimulus effects ae chaacteized by a single esponse timecouse the time kenel and an amplitude value fo each event type. The HDR to an event type is the poduct of the time kenel and the amplitude value associated with the event type. The time-seies data ae modeled as y¼(x 2 * k)h 2 +Sb+n, whee X 2 is the stimulus matix (t e), k is the time kenel (l 1), * epesents convolution, h 2 is a set of event amplitudes (e 1), and S, h, andn aeasinthefirmodel.the stimulus matix consists of one paamete fo each event type. The paamete fo a given event type is a binay sequence, whee ones indicate event occuences (Fig. 2). Stimulus effects ae given by (X 2 * k)h 2, and nuisance effects ae given by Sb. Fo ou data, the time-event sepaable model uses (l ¼ 21) + (e ¼ 12) ¼ 33 paametes to chaacteize stimulus effects. This is much fewe than the 252 paametes used in the FIR model. Model Fitting We fit the FIR model by obtaining the odinay leastsquaes estimate ^h 1W ¼ W ^b T W T y whee W¼[X S]. This poduces ĥ, a set of HDR estimates, and bˆ, a set of nuisance paamete estimates. We fit the time-event sepaable model using two diffeent methods. In the fist method (SEPNL), we use an iteative fitting appoach [Hinichs et al., 2000]. We estimate the time kenel, event amplitudes, and nuisance paametes using nonlinea least-squaes optimization (MATLAB Optimization Toolbox, Levenbeg-Maquadt method). This method detemines all model paametes simultaneously, minimizing the squaed eo between the model fit and the data. A disadvantage of the iteative fitting method is that it is computationally intensive the method may be impactical given that thousands of voxels ae analyzed in a typical fmri expeiment. Also, the fitting method may convege to a local minimum of the eo function. In the second method fo fitting the time-event sepaable model (SEPSVD), we estimate the time kenel befoe the othe model paametes. This appoach avoids iteative computation but may not poduce an optimal model fit (in the least-squaes sense). The method poceeds as follows. We obtain HDR estimates ĥ fom the FIR model. We eshape ĥ into a matix with ows coesponding to event types and columns coesponding to time points (e l). We pefom singula value decomposition on this matix to obtain the singula vecto associated with the lagest singula value. This vecto is the l-dimensional vecto along which vaiance in ĥ is maximized; this is the time kenel estimate ^k. (Anothe way to conceptualize ^k is as the l-dimensional vecto that best econstucts ĥ in the least-squaes sense.) Using ^k, we obtain the odinay least-squaes estimate ^h 2 ^b 1W h ¼ W T W T y whee W ¼ ðx 2 ^kþs i. This poduces ĥ 2, a set of event amplitude estimates, and bˆ,a set of nuisance paamete estimates. Note that the time kenel estimate is based on the FIR model fit. Thus, ovefitting by the FIR model has some effect on the time kenel estimate. In pactice, howeve, the SEPSVD method pefoms quite well (see Results). To obtain standad eos on the paamete estimates of a model, we use a nonpaametic jackknife pocedue [Efon and Tibshiani, 1993]. We andomly divide the time-seies data points into 10 subsets and fit the model 10 times, each time with a diffeent subset excluded. (To exclude data points, we delete ows of y and the coesponding ows of X, S, andx 2 * k.) Standad eos ae calculated fom the distibutions of paamete estimates acoss the 10 model fits. To quantify the amplitude of a HDR, we sum ove a time window coesponding to the peak of the positive BOLD esponse [de Zwat et al., 2005]. (Fo ou data, we use the time window of 3 7 s based on inspection of HDR estimates acoss voxels and event types (Fig. 3).) We quantify the SNR of an event type as the absolute value of the HDR amplitude divided by the standad eo of the HDR amplitude. (The standad eo is calculated via a jackknife pocedue; see ealie.) We quantify the SNR of a voxel as the maximum SNR achieved ove all event types. We calculate pecent BOLD change elative to the DC paamete estimate (i.e. the paamete estimate fo the constant tem included in the nuisance matix). In one instance we use an altenative SNR metic, which we denote by SNR alt. This metic is useful fo compaing the SNR of diffeent models. Fo a given voxel, we calculate the maximum absolute HDR amplitude (MAX) obtained unde any of the models. We then quantify the SNR alt fo each model as MAX divided by the median standad eo on HDR amplitudes acoss events. This metic pevents vaiability in HDR amplitude estimates fom influencing SNR values. Note that the SNR metics descibed ealie ae simila to the conventional t-statistic. Thus, one can intepet changes in SNR in tems of statistical significance and 146

6 Event-Related fmri Modeling Figue 3. Inspection of HDR estimates acoss voxels and event types. Using the POLY model (FIR model combined with polynomials), we obtained fo each voxel an estimate of the HDR to each of the 12 event types. The figue depicts positive HDR estimates with a SNR of at least 30 (n ¼ 216 fom 212 unique voxels). The x-axis indicates time elative to event onset; the y-axis indicates the BOLD signal. Fo display puposes, each HDR estimate is nomalized by dividing by its maximum value. The inset indicates the median standad eo fo the depicted data points. The obustness of the shapes of the timecouses indicates the high SNR in the data, despite the small voxel size (2 mm) and the modeate amount of data (17 min). effect size. Fo example, suppose we wish to detect a signal change whose magnitude is fou times the magnitude of the noise, given a fixed amount of data. At an a value of and 9 degees of feedom (10 jackknifes wee taken), the powe to detect such a change is With a 50% incease in SNR, the powe to detect such a change inceases to To quantify the magnitude of LFF, we calculate the median absolute deviation (elative to the mean) of the time points of the estimated nuisance effects. We convet the aw BOLD units to standad deviation units, whee one standad deviation unit equals the standad deviation of the time-seies data with the nuisance effects subtacted. We define the esulting quantity as the LFF magnitude index. Intuitively, this index quantifies the typical deviation of the baseline signal level ove the couse of the timeseies. Fo example, a value of 0.4 indicates that, on aveage, the baseline signal level is 0.4 standad deviation units away fom the mean baseline signal level. Model Evaluation We quantify the fit accuacy of a model as the coefficient of multiple detemination (R 2 ) between the data and the model fit to the data. This value is the amount of vaiance in the data explained by the model fit. When compaing models, an impovement in fit accuacy could eflect impovement in model accuacy, but could also eflect model ovefitting. To measue the accuacy of a model while contolling fo ovefitting, we use a nonpaametic n-fold coss-validation pocedue whee n ¼ 10. We andomly divide the time-seies data points into 10 subsets. We exclude one subset and fit the model on the emaining data points. (To exclude data points, we delete ows of y and the coesponding ows of X, S, and X 2 *k.) We use the obtained model paamete estimates to pedict the data in the excluded subset. The pocess is epeated 10 times, such that each subset is excluded once. We theeby obtain a pediction fo each data point. We quantify the pediction accuacy of a model as the coefficient of multiple detemination (R 2 ) between the data and the model pediction of the data. This value is the amount of vaiance in the data explained by the model pediction. The pediction accuacy of a model is how well the model genealizes to new data, i.e. data not used in the fitting of the model. LFF often dominate the vaiance in the time-seies data. In these cases, the coefficient of multiple detemination is atificially high (e.g., 0.9) and eflects pimaily how well LFF is modeled. To obtain a pediction accuacy metic that eflects stictly how well stimulus effects ae modeled, we pefom the following pocedue. We subtact the pedicted nuisance effects fom both the oiginal data and the model pediction. We then calculate the coefficient of multiple detemination between the adjusted data and adjusted pediction. We define the esulting value as the LFF-adjusted pediction accuacy. (Because the pedicted nuisance effects ae only estimates and not the tue nuisance effects, the metic is potentially biased. Howeve, we obseve the same tends in model pefomance with eithe pediction accuacy metic.) In the pesent context, the coefficient of multiple detemination (R 2 ) diectly quantifies how well a given model explains the obseved data. Repoting R 2 values is not common in the liteatue [one exception is Razavi et al., 2003]. When compaing models with espect to R 2 values, a diffeence of 1 2% can be consideed a small effect, a diffeence of 5% can be consideed a modeate effect, and a diffeence of 10% can be consideed a lage effect. Additional Data Sets We also collected data sets using diffeent subjects, imaging paametes, and stimulus designs. Fom the pespective of the pesent study, thee is no specific motivation fo the paticula chaacteistics of these othe data sets. The pupose of these additional data sets is to show that esults ae not specific to a paticula expeiment. Data set 1 is the pimay data set descibed ealie, and involved subject KH (an autho). Data set 2 involved subject KK (an autho), a volume coil, a two-shot EPI sequence (TR 1 s pe shot), and a 3 3 3mm 3 voxel size. The stimulus was the same as in data set 1. Data set 3 involved subject TN and a mm 3 voxel size. The stimulus consisted of achomatic sinusoidal gatings of eight diffeent oientations. One tial consisted 147

7 Kay et al. TABLE I. Summay of data models Model Stimulus effects Nuisance effects DC Finite impulse esponse Constant tem FOURIER Finite impulse esponse Constant tem, sine and cosine functions with 1, 2, and 3 cycles POLY Finite impulse esponse Polynomials of degees 0 though 4 FILTER Finite impulse esponse Constant tem, afte emoving a linea tend and high-pass filteing at 1/60 Hz SEPNL Time-event sepaable, Polynomials of degees 0 though 4 iteative fitting method SEPSVD Time-event sepaable, singula value decomposition fitting method Polynomials of degees 0 though 4 This table lists how each model chaacteizes stimulus effects (i.e. hemodynamic esponses to stimulus events) and how each model compensates fo nuisance effects (i.e. the baseline signal level). of the pesentation of a gating fo 1 s followed by 3 s of a gay backgound. The eight oientations wee epeated 15 times each, and the pesentation ode was andomly chosen. The stimulus altenated between 16-s peiods duing which a gay backgound was pesented and 80 s peiods duing which tials wee pesented. The stimulus duation was 9.9 min. Fo data analysis we used eight event types, one event type fo each gating oientation. Data set 4 involved subject TN and a mm 3 voxel size. The stimulus consisted of 12 gayscale natual photos. One tial consisted of the pesentation of a photo fo 1 s followed by 3 s of a gay backgound. Each photo was epeated 13 times; the pesentation ode was contolled by an m-sequence of level 13, ode 2, and length ¼ 168. The stimulus duation was 11.2 min. Fo data analysis we used 12 event types, one event type fo each distinct photo. RESULTS We collected data using multiple subjects, imaging paametes, and stimulus designs. Ou analysis esults wee lagely consistent acoss data sets. In this section we pesent in-depth esults fo a single data set (Figs. 1 8), indicate which esults wee vaiable in othe data sets, and summaize esults fo all data sets (Fig. 9). Basic Data Inspection We conducted an event-elated fmri expeiment involving bief (4 s) pesentations of a checkeboad patten within 12 wedges in the visual field (Fig. 1). Ou analysis appoach is based on the FIR model fo event-elated fmri [Dale, 1999]. We define 12 event types, one event type pe wedge. Fo each voxel, a HDR to each of the 12 event types is estimated using a set of shifted delta functions. No assumption on the shape of HDRs is made. Additional egessos ae used to model the time-vaying baseline signal level (Fig. 2). We obtained stong BOLD activations in occipital cotex. Figue 3 depicts positive HDR estimates obtained unde the POLY model (Table I) with a SNR of at least 30. (This stict citeion selects only those estimates that ae nealy noise-fee.) The obustness of the shapes of the timecouses confims the high SNR in the data, despite the small voxel size (2 mm) and the modeate amount of data (17 min). The high SNR is due to the high magnetic field (4 T), the use of a suface coil, the use of an expeienced fmri subject, the m-sequence expeimental design, and the high-contast visual stimulus. Compensation fo LFF We evaluated seveal stategies fo compensating fo LFF in the time-seies data. (1) The DC model ignoes LFF and uses only a constant tem to model DC offset. (2) The FOURIER model uses a constant tem and Fouie basis functions with 1, 2, and 3 cycles to model LFF. (3) The POLY model uses Legende polynomials of degees 0 though 4 to model LFF. (The spectal content of these polynomials appoximately match those of the Fouie basis functions.) (4) The FILTER model emoves a linea tend and high-pass filtes the time-seies data at 1/60 Hz as a pepocessing step. Panel A of Figue 4 shows that the POLY model geatly inceased pediction accuacy compaed to the DC model (median incease 14.8%; P < 0.001). This indicates ignoing LFF esulted in model fits with poo genealizability. This also indicates that a substantial amount of LFF exists in the time-seies data. Panel B of Figue 4 shows that the POLY model somewhat inceased pediction accuacy compaed to the FOURIER model (median incease 2.3%; P < 0.001). This indicates that polynomials moe accuately chaacteized LFF compaed to Fouie basis functions. Howeve, in othe data sets, the POLY and FOURIER models had compaable pefomance (Fig. 9). Panel C of Figue 4 shows the POLY model substantially inceased LFF-adjusted pediction accuacy compaed to 148

8 Event-Related fmri Modeling Modeling LFF with polynomials maximizes pediction accuacy. In these gaphs we compae diffeent stategies fo LFF compensation (Table I). Fo each gaph, we selected voxels with a minimum SNR of 10 unde eithe of the models being compaed. Each point in a gaph epesents pediction accuacy fo a single voxel. (A) DC vs. POLY. The x- and y-axes indicate pediction accuacy unde the DC and POLY models, espectively. Thee was a lage incease in accuacy unde the POLY model compaed to the DC model (n ¼ 1904; median incease 14.8%; P < 0.001). This indicates that ignoing LFF esulted in model fits with poo genealizability, and that a substantial amount of LFF exists in the time-seies data. Some voxels exhibited vey lage inceases in pediction accuacy; in these cases, the contibution of LFF to vaiance in the time-seies data is much lage than Figue 4. the contibution of stimulus effects. (B) FOURIER vs. POLY. The x- and y-axes indicate the pediction accuacy unde the FOU- RIER and POLY models, espectively. Thee was a small incease in accuacy unde the POLY model compaed to the FOURIER model (n ¼ 1,971; median incease 2.3%; P < 0.001). This indicates polynomials moe accuately chaacteized LFF compaed to Fouie basis functions in this data set. (C) FILTER vs. POLY. The x- and y-axes indicate the LFF-adjusted pediction accuacy unde the FILTER and POLY models, espectively. Thee was a lage incease in accuacy unde the POLY model compaed to the FILTER model (n ¼ 1,880; median incease 6.5%; P < 0.001). This indicates that stimulus effects wee bette chaacteized when polynomials wee used to model LFF compaed to when the time-seies data wee high-pass filteed to emove LFF. the FILTER model (median incease 6.5%; P < 0.001). The use of the LFF-adjusted pediction accuacy metic (see Methods) ensues that inceased accuacy unde the POLY model is not simply due to the modeling of LFF. The esult indicates that stimulus effects wee bette chaacteized when polynomials wee used to model LFF compaed to when the time-seies data wee high-pass filteed to emove LFF. (We also evaluated the FILTER model using a fequency cutoff of 1/500 Hz; compaed to this model, the POLY model still povided a median incease of 2.8% LFF-adjusted pediction accuacy.) Chaacteistics of LFF We investigated in moe detail the timecouses of LFF. Panel A of Figue 5 illustates the effect of manipulating the maximum degee of the polynomials included in the POLY model. Damatic inceases in LFF-adjusted pediction accuacy wee obtained by inceasing the maximum degee fom 0 (median accuacy 5.8%) to 4 (median accuacy 10.8%). Polynomials with degee geate than 4 only maginally inceased accuacy; moeove, these inceases wee inconsistent acoss voxels (data not shown). Panel A also illustates the effect of maximum polynomial degee on the SNR. Substantial inceases in SNR wee obtained by inceasing the maximum degee fom 0 (median SNR 9.4) to 3 (median SNR 11.6), beyond which SNR did not incease appeciably. Panel B of Figue 5 depicts the spectal content of Legende polynomials of degees 0 though 4. These polynomials consist pedominantly of vey low fequencies ( Hz). With each additional polynomial degee, highe fequencies in the time-seies data can be modeled. Panel C of Figue 5 illustates seveal example LFF timecouses. Note that the shape and magnitude of LFF vay acoss voxels. We quantified the magnitude of LFF with the LFF magnitude index. The index quantifies the typical deviation of the baseline signal level ove the time-seies data, and is in standad deviation units (see Methods). The 25th and 75th pecentiles of the index ae 0.18 and 0.57, espectively. (These pecentiles wee calculated fo voxels with a minimum SNR of 10 unde the POLY model.) This indicates that noise due to LFF accounts fo a substantial faction of the vaiation in the time-seies data. Ovefitting by the FIR Model Ovefitting tends to occu when a model has a lage numbe of paametes elative to the amount of available data. Two lines of evidence show that the FIR model suffes fom ovefitting. 149

9 Kay et al. C O L O R Chaacteistics of LFF. (A) The effect of the maximum polynomial degee on model pefomance. We manipulated the maximum degee of the polynomials included in the POLY model (x-axis) and evaluated the effect on LFF-adjusted pediction accuacy (y-axis; ed line) and SNR (y-axis; geen line). Fo this gaph we selected voxels with a minimum SNR of 10 unde any of the model vaiants (n ¼ 2,890). Dots indicate the median acoss voxels, and eo bas indicate 61 SE(bootstap pocedue). With inceasing polynomial degee, both LFFadjusted pediction accuacy and SNR damatically inceased. (B)Spectal content of Legende polynomials of degees 0 though 4. The polynomials extend ove the couse of the time-seies data (17 min). Figue 5. We calculated the discete Fouie tansfom of each polynomial afte applying a Hanning window to avoid edge atifacts and subtacting the mean value. The coelation (y-axis) between the time-seies data and the Fouie component at each fequency (x-axis) is plotted. Fo display puposes the zeo-fequency point is omitted. Note that the polynomials consist pedominantly of vey low fequencies ( Hz). (C) Example timecouses of LFF. Fo 25 voxels we plot nuisance effects as detemined unde the POLY model. These voxels wee andomly selected fom voxels with a minimum SNR of 10 (n ¼ 1,730). The x-axis indicates time; the y-axis indicates standad deviation units (see Methods). Fo display puposes, the mean of each timecouse is emoved. The specific HDR window used in the FIR model substantially affected the quality of model fits. Panel A of Figue 6 shows that fit accuacy monotonically inceased with window duation. This eflects the fact that, with a longe window duation, additional model paametes ae available to fit the data. Howeve, pediction accuacy did not monotonically incease, but was maximized at a duation of 9 s. This indicates that on aveage, estimating HDRs beyond 9 s Ovefitting by the FIR model. We manipulated two chaacteistics of the POLY model (Table I) and evaluated the effect on fit accuacy (gay line) and pediction accuacy (black line). Fo these gaphs we selected voxels with a minimum SNR of 10 unde the POLY model (n ¼ 1,730). Dots indicate the median acoss voxels, and eo bas indicate 61 SE(bootstap pocedue).(a) The effect of HDR window duation on fit accuacy and pediction accuacy. The x-axis indicates the HDR window duation used in the model; the y-axis indicates explained vaiance. Pediction accuacy was maximized at a duation of 9 s. This indicates that, on aveage, estimating HDRs beyond 9 s esulted in ovefitting and educed model genealizability. (B) The effect of the numbe of event types Figue 6. on fit accuacy and pediction accuacy. Based on SNR estimates obtained unde the POLY model, we efit the model including only the top event types with espect to SNR. (Because diffeent voxels espond to diffeent event types, the included event types vaied on a voxel-by-voxel basis.) The x-axis indicates the numbe of event types; the y-axis indicates explained vaiance. Pediction accuacy was maximized at thee event types. This indicates that, on aveage, estimating moe than thee event types esulted in ovefitting and educed model genealizability. This esult is explained by the fact that voxels in visual cotex ae often highly selective fo spatial position, in such a way that stimuli positioned at nonpefeed locations poduce no discenable activation. 150

10 Event-Related fmri Modeling esulted in ovefitting and educed model genealizability. This esult is consistent with the obsevation that HDRs have mostly died off by 9 s afte event onset (see Fig. 3). The numbe of event types included in the FIR model also substantially affected the quality of model fits. We evaluated vaiants of the model in which only the top event types with espect to SNR ae included. (The top event types wee detemined on a voxel-by-voxel basis.) Panel B of Figue 6 indicates that fit accuacy monotonically inceased with numbe of event types. This eflects the fact that, with moe event types, additional model paametes ae available to fit the data. Howeve, pediction accuacy did not monotonically incease, but was maximized at thee event types. This indicates that on aveage estimating moe than thee event types esulted in ovefitting and educed model genealizability. This esult is explained by the fact that voxels in visual cotex ae often highly selective fo spatial position, in such a way that stimuli positioned at nonpefeed locations poduce no discenable activation. Time-Event Sepaability To educe ovefitting by the FIR model, we incopoated the constaint of time-event sepaability. Unde the time-event sepaable model, stimulus effects ae chaacteized by a single esponse timecouse the time kenel and an amplitude value fo each event type (Fig. 2). This educes the numbe of model paametes that need to be estimated. We evaluated two methods fo fitting the time-event sepaable model, SEPNL and SEPSVD (see Methods). Panel A of Figue 7 shows that the SEPSVD model geatly inceased LFF-adjusted pediction accuacy compaed to the POLY model (median incease 9.9%; P < 0.001). This indicates that voxel esponses wee lagely time-event sepaable, and that time-event sepaability impoved the accuacy of HDR estimates. Panel B of Figue 7 shows that the SEPNL model slightly inceased LFFadjusted pediction accuacy compaed to the SEPSVD model (median incease 0.5%; P < 0.001). This indicates that the two fitting methods poduced vey simila esults. Howeve, in one of the othe data sets, the SEPNL model pefomed substantially bette than the SEPSVD model (Fig. 9). The incopoation of time-event sepaability also inceased the SNR. We selected voxels with a minimum SNR of 10 unde eithe the POLY o SEPSVD model. Of these voxels, the median SNR alt fo the POLY model was 14.3, while the median SNR alt fo the SEPSVD model was This incease was statistically significant (P < 0.001). Example Voxels We have pesented population esults thus fa, but it is also useful to inspect esults fo individual voxels. Panels A E of Figue 8 show model paamete estimates fo a Figue 7. Time-event sepaability educes ovefitting and inceases pediction accuacy. In these gaphs we compae the FIR model to the time-event sepaable model (Table I). Each point in a gaph epesents pediction accuacy fo a single voxel. (A) POLY vs. SEPSVD. The x- and y-axes indicate the LFF-adjusted pediction accuacy unde the POLY and SEPSVD models, espectively. The gaph depicts voxels with a minimum SNR of 10 unde eithe data model (n ¼ 1,884). Thee was a lage incease in accuacy unde the SEPSVD model compaed to the POLY model (median incease 9.9%; P < 0.001). This indicates that voxel esponses wee lagely time-event sepaable, and that time-event sepaability impoved the accuacy of HDR estimates. (B) SEPSVD vs. SEPNL. The x- and y-axes indicate the LFF-adjusted pediction accuacy unde the SEPSVD and SEPNL models, espectively. The gaph depicts voxels with a minimum SNR of 10 unde the POLY model (n ¼ 1,730). Thee was a tiny incease in accuacy unde the SEPNL model compaed to the SEPSVD model (median incease 0.5%; P < 0.001). This indicates that the singula value decomposition fitting method compaed favoably against the iteative fitting method in this data set. typical voxel. Notice the DC model poduced vey noisy HDR estimates; the FILTER model poduced HDR estimates consideably diffeent fom those poduced by othe models; and the SEPSVD model poduced the most accuate HDR estimates. Panel F of Figue 8 depicts the spectal content of stimulus effects fo the voxel. Notice powe is distibuted ove a wide ange of fequencies. Panel G of Figue 8 shows model paamete estimates fo anothe typical voxel. Again, the SEPSVD model poduced the most accuate HDR estimates. Model Pefomance Summay Figue 9 summaizes the LFF-adjusted pediction accuacy of the data models we evaluated, and includes esults fom additional data sets. Acoss fou data sets, the same basic tend in accuacy was obseved: the SEPNL and SEPSVD models wee the most accuate, the FOURIER and POLY models wee modeately accuate, and the DC and FILTER models wee the least accuate. Thee wee two inteesting anomalies. Fist, wheeas the POLY model outpefomed the FOURIER model fo data 151

11 Kay et al. C O L O R Compaison of data models fo two typical voxels in occipital cotex. Panels A F depict one voxel, and panel G depicts a second voxel. In panels A E and G, the main axes show HDR estimates fo the 12 event types. The x-axis indicates time elative to event onset; the y-axis indicates pecent BOLD change. A thick black hoizontal line indicates zeo pecent BOLD change. Eo bas indicate 61 SE(jackknife pocedue). Indicated in paentheses is the LFF-adjusted pediction accuacy, which is calculated via 10-fold coss-validation. The inset axes above the main axes depict the time-seies data (ed line) and nuisance effects (geen line). (A) DC model. This model ignoes LFF and uses only a constant tem to chaacteize the baseline signal level. Unde this model, HDR estimates wee vey noisy and pediction accuacy was poo. (B) FOURIER model. This model uses a constant tem and Fouie basis functions with 1, 2, and 3 cycles to model LFF. Compaed to the DC model, HDR estimates wee less noisy and pediction accuacy was bette. Note that the nuisance effects pooly tack the timeseies data at the beginning and end of the time-seies. (C) POLY model. This model uses polynomials of degees 0 though 4 to model LFF. Compaed to the FOURIER model, HDR estimates wee slightly less noisy and pediction accuacy was bette. Notice the nuisance Figue 8. effects tack the time-seies data well. The LFF magnitude index is (D) FILTER model. This model high-pass filtes the time-seies data at 1/60 Hz to emove LFF as a pepocessing step. The filteed data ae shown in ed in the inset (above). HDR estimates wee consideably diffeent fom those obtained unde othe data models. (E) SEPSVD model. This model incopoates the constaint of time-event sepaability and uses polynomials of degees 0 though 4 to model LFF. Time-event sepaability is the condition that HDR estimates acoss event types ae identical up to a scale facto. Pediction accuacy was highest unde the SEPSVD model. The LFF magnitude index is (F) Spectal content of stimulus effects. We obtained the estimated timecouse of stimulus effects unde the SEPSVD model. We calculated the discete Fouie tansfom of this timecouse afte subtacting the mean value. The coelation (y-axis) between the timeseies data and the Fouie component at each fequency (x-axis) is plotted. Fo display puposes the zeo-fequency point is omitted. Note that the powe is distibuted ove a wide ange of fequencies. (G) Compaison of data models fo a second voxel. The fomat is identical to that of panels A E, except that the y-axis anges fom 3 to 7. Again, the SEPSVD model had the highest pediction accuacy. 152

12 Event-Related fmri Modeling Figue 9. Summay of data model pefomance. This gaph summaizes esults fom fou data sets involving diffeent subjects, imaging paametes, and stimulus designs: the pimay data set (illustated in Figs. 1 8) and thee additional data sets. Fo each data set, we selected voxels (n ¼ 2,223, 699, 236, 825, espectively) passing a minimum SNR theshold (¼ 10, 10, 7, 10, espectively) unde any of the models that do not involve iteative fitting (this excludes the SEPNL model). (The SNR theshold is loweed fo data set 3 due to low signal in that data set.) The x-axis indicates the data models we evaluated; and the y-axis indicates the LFFadjusted pediction accuacy. The height of each ba indicates the median acoss voxels, and the ba shading indicates the data set. Eo bas indicate 61 SE (bootstap pocedue). The same basic tend in pefomance was obseved acoss the fou data sets: the SEPNL and SEPSVD models wee the most accuate, the FOURIER and POLY models wee modeately accuate, and the DC and FILTER models wee the least accuate. set 1 (see also Fig. 4), the two models had simila pefomance in othe data sets. The only diffeence between the two models is the choice of egessos fo LFF. The vaiable esults acoss data sets suggest that LFF chaacteistics ae dependent on the subject, imaging paametes, and/o stimulus design. Second, wheeas in data sets 1 3, the SEPNL and SEPSVD models had simila pefomance, in data set 4, the SEPNL model substantially inceased accuacy compaed to the SEPSVD model (median incease acoss voxels 5.4%; P < 0.001). The eason fo the lage but inconsistent incease in accuacy unde the SEPNL model is an issue fo futhe investigation. We speculate that the vaiable esults may be due to stong tempoal nonlineaities in data set 4. DISCUSSION Lineaity Assumptions The FIR and time-event sepaable models assume that the BOLD esponse is a linea, time-invaiant system with espect to the stimulus. Howeve, violations of time-invaiance have been widely documented [Boynton et al., 1996; Buxton et al., 2004; Dale and Buckne, 1997; Fiston et al., 2000b; Glove, 1999; Huettel and McCathy, 2001; Logothetis, 2003; Miezin et al., 2000; Wage et al., 2005]. In geneal, the esponse to an event closely peceded by anothe event has a geate delay and lowe amplitude than expected. This tempoal nonlineaity may be neual in oigin (e.g. adaptation) and/o elated to the coupling between neual activity and the BOLD esponse [Bandettini et al., 2002; Bin et al., 2001; Boynton and Finney, 2003; Huettel et al., 2004; Janz et al., 2001; Ogawa et al., 2000]. We dampened the impact of tempoal nonlineaities in ou expeimental design by the use of a 4-s bin duation. This is because deviations fom lineaity ae lage only at shot-stimulus duations [Bin et al., 2001; Boynton et al., 1996; Pfeuffe et al., 2003; Vazquez and Noll, 1998]. Wheeas the esponse to a modeate-length stimulus (4 s) well pedicts the esponse to a longe stimulus (8 s), the esponse to a shot stimulus (1 s) pooly pedicts the esponse to a longe stimulus (2 s). It may be possible to devise models to account fo tempoal nonlineaities when they exist [Fiston et al., 2000b; Wage et al., 2005]. Ou expeimental design involves simultaneous pesentation of diffeent event types (i.e. multiple wedges in the visual field at any given time). The pimay pupose of simultaneous pesentation is to incease the numbe of event epetitions and theeby incease the SNR. Note that both the FIR and time-event sepaable models assume that the BOLD esponse is additive acoss events: that is, the esponse to events pesented simultaneously is equal to the sum of the esponses to the events pesented in isolation. The validity of this assumption depends on the expeimental paadigm [Hansen et al., 2004]. Howeve, the analysis techniques we pesent ae not specific to expeimental designs using simultaneous event pesentation. Low-Fequency Fluctuation We found that using polynomials to model LFF esulted in moe accuate HDR estimates than those obtained with othe stategies. We used an event-elated expeimental design, and ou study complements studies that investigated LFF fo block designs [LaConte et al., 2003; Razavi et al., 2003]. The poo pefomance of high-pass filteing is explained by the fact that stimulus effects in ou data exist at low fequencies. High-pass filteing emoves LFF but also emoves a potion of the stimulus effects [Kuggel et al., 1999; Ollinge et al., 2001; Skudlaski et al., 1999; Smith et al., 1999]. Moeove, the emoval of stimulus effects induces bias in HDR estimates (Fig. 8). Detending techniques (of which high-pass filteing is one instance) ae appopiate only when stimulus effects can be assumed to be absent at low fequencies (e.g. a peiodic ON OFF block expeimental design). Using egessos to model LFF is not equivalent to emoving these egessos fom the time-seies data befoe fitting the data model [Liu et al., 2001]. The latte is effec- 153