Lightweight, flexible MATLAB-based visualization and analysis of FreeSurfer surfaces

Transcription

1 Lightweight, flexible MATLAB-based visualization and analysis of FreeSurfer surfaces Ionatan J. Kuperwajs Macalester College Saint Paul, Minnesota A thesis submitted for the degree of Undergraduate Honors May 2018 Advisor: Andrew Beveridge Reader: Kendrick Kay, University of Minnesota Reader: Susan Fox, Macalester College

2 Abstract Functional magnetic resonance imaging allows cognitive neuroscientists to noninvasively measure neural activity. Due to recent advancements in field strength and spatial resolution, more efficient statistical and image processing tools are required to properly interpret highly-detailed experimental datasets. We provide AutoVizFS, a lightweight software package built on the prominent FreeSurfer suite, to analyze preprocessed cortical surfaces. AutoVizFS offers visualization of anatomical surfaces, manual definition of differently configured regions-of-interest, and analytical exploration of functional values. We demonstrate how AutoVizFS can be utilized to accurately decode fine-scale neural activity in a simple localizer experiment while considering persistent issues in fmri such as distortion from cortical curvature and the darkening effect of veins.

3 Contents 1 Introduction Motivation fmri Background Statistical Analysis Foundational High-Resolution fmri Studies AutoVizFS: Software Design Loading Preprocessed fmri Data Display Options Line Profile Definition and Analysis ROI Definition and Analysis Application in Neuroscience Scientific Promise Localizer Experiment Accurate Measurement of Fine-Scale Neural Activity Conclusion Current Project Scope Future Work Bibliography 34 i

4 Chapter 1 Introduction 1.1 Motivation Automated Visualization of FreeSurfer Surfaces (AutoVizFS) is a lightweight, flexible MATLAB-based software package for visualization and analysis of FreeSurfer[2] surfaces. The toolbox provides the following core functionality: 1. the capability to quickly and efficiently visualize anatomical surfaces and functional data from fmri scans, 2. a graphical user interface (GUI) to interactively set parameters and analysis options, and 3. the ability to modify and customize the underlying code, which is free and open-source. This project was especially driven by the need for detailed investigation of highresolution fmri datasets. These typically consist of high levels of detail and would benefit from both automation within and throughout subjects and careful control over interpolation. There are many other great software options for fmri visualization and analysis, so it is important to justify the need for AutoVizFS. One of the most prominent examples is AFNI, which has a program (SUMA) that allows for cortical surface based functional imaging analysis[1]. BrainVoyager contains advanced analysis and visualization tools for structural and functional magnetic resonance imaging (MRI) data and for combined EEG and MEG distributed source imaging, but is expensive to obtain a license[5]. Pycortex is a software library built for use with ipython that allows you to visualize fmri or other volumetric neuroimaging data on cortical surfaces[3]. 1

5 FreeSurfer also has a native visualization tool, FreeView. Given the existing tool plurality and diversity, it is important to justify the addition of AutoVizFS to this current landscape. AutoVizFS is particularly well-suited for certain use cases and lab pipelines. The primary advantages of AutoVizFS are integration with FreeSurfer and MATLAB, flexibility in terms of input and output data, portability and ease of installation, and automation of large scale visualization and analysis options. 1.2 fmri Background Functional magnetic resonance imaging (fmri) is a neuroimaging technique that uses MRI technology to noninvasively measure brain activity by detecting hemodynamicrelated changes. This technique relies on the fact that cerebral hemodynamics and neural activity are coupled. In other words, when an area of the brain starts to increase in neural activity, blood flow to that region also increases. fmri has dominated cognitive neuroscience research since the early 1990s because it is non-invasive and has relatively high spatial resolution. Thus far, spatial resolution in typical fmri experiments is on the order of 2 4 mm. Thanks to the recent advent of ultra-high field strengths (e.g. 7 T and higher), ultra-high spatial resolution (e.g. 0.8 mm) fmri has garnered more widespread use. In particular, multiband pulse sequences now allow for relatively high spatial coverage while also acquiring signals at high spatial resolution, as shown in Figure 1.1. The technically impressive, cutting edge datasets that neuroscientists can now acquire pose some substantial challenges. The size of the resultant datasets are often difficult to handle (memory requirements, computational time), and experimenters now have much more data to sift through, visualize, and analyze. Many neuroscience labs around the world have acquired large amounts of data from various visual and other experiments that, at present, has not undergone detailed analysis. Researchers need efficient tools in order to expedite this process. A major feature of current datasets is that they have been acquired at a sufficient resolution to potentially resolve different functional properties of distinct cortical layers. Addressing this backlog of data analysis efficiently could lead to important scientific insights. 2

6 Figure 1.1: Demonstration of the level of image resolution in a slice of fmri data for a sample subject. The left figure is 2.5 mm (3T) while the right figure is 0.8 mm (7T). In addition to superior spatial detail, high-resolution fmri provides excellent spatial coverage (many slices over a large region in cortex) and signal-to-noise (SNR) ratio. 1.3 Statistical Analysis Many techniques are available for statistical analysis of fmri data. These methods aim to produce an image identifying regions of the brain that show significant signal change in response to the experimental task. Each pixel is assigned a value dependent on the likelihood of the falsity of the null hypothesis, that the observed signal changes can be explained purely by random variation in the data consistent with its variance. The goal of this section is to provide a mathematical intuition for many of the underlying computational challenges that arise when creating such statistical parametric maps. One of the simplest methods for obtaining results from a simple two-state fmri experiment is to perform a subtraction. This is carried about by taking the average of the images acquired during the on phase of the task and subtracting the average of the off images. This method can be extended to multi-state experiments by subtracting different combinations of conditions, the default being the condition of interest versus the rest of the conditions grouped together. It is more common to use a t-test that weights the difference in means by the standard deviation in the on or off values. This yields a statistic that can be tested against the null hypothesis, giving high scores to large differences with small standard deviations and low scores to small 3

7 differences with large standard deviations. The t-statistic is calculated in a pixelwise manner for a given time series X using the following formula: where and s 2 y is the pooled variance s 2 y = t = X 1 X 2 s X1 X 2 (1.1) s X1 X 2 = s 2 y + s2 y n 1 n 2 (1.2) (X1 X 1 ) 2 + (X 2 X 2 ) 2. (1.3) n 1 + n 2 2 The index 1 refers to the n 1 images acquired during the on period of the task, while index 2 refers to the n 2 images acquired during the rest period. It is possible to improve the detection of activations by predicting the shape of the response to the stimulus, as we know that BOLD response is mediated by blood flow. Calculating correlation coefficients between each pixel time course and this reference waveform is less sensitive than the aforementioned subtraction method to physiological changes during the experiment. For a time course X and reference waveform Y, we calculate the correlation coefficient as follows: (X X)(Y Ȳ ) r = (X X). (1.4) 2 (Y Ȳ )2 As expected, r has a value of 1 for perfect correlation, zero for no correlation, and 1 for perfect anti-correlation. The choice of an appropriate reference waveform is vital for the success of this technique in finding activations, where the response to a single stimulus informs a general construction of a waveform that predicts hemodynamic response in a general case. As an alternative, the response can be modeled by a mathematical function, such as a Poisson distribution: f(x) = λx e λ. (1.5) x! In general, a Poisson with width λ = 6 seconds seems to fit well to observed hemodynamic responses. This process is detailed in Figure 1.2. Both the t-statistic and correlation calculations are parametric tests that are special cases of the general linear model (GLM), which is a framework for analyzing functional imaging data. Therefore, 4

8 Figure 1.2: General method for statistical processing of fmri data. (a) First, a waveform is selected that approximates high values for scans acquired during the task and low values for scans acquired during rest. (b) A delay of 3-6 seconds is then added along with smoothing of the hemodynamic response which regulates the BOLD contrast. (c) Correlation coefficients are calculated between this waveform and that of every other pixel in the image. (d) Response to each stimulus is averaged together, producing a mean response to the single cycle which is repeated over time. (e) Finally, a Poisson model approximates the observed waveform. 5

9 providing an outline of this model is useful for the context of this paper. In its current iteration, AutoVizFS assumes that a GLM has been run on the user s fmri data during FreeSurfer preprocessing. The aim of the general linear model is to explain the variation of the time course in terms of a linear combination of explanatory variables and an error term. If we have explanatory variables x 1,..., x n and time course y 1,..., y n, the GLM can be written as: y i = x i β + ɛ i (1.6) where β is the scaling parameter and ɛ is the error term. We can rewrite this same equation in matrix form in the case that we have more variables: Y = Xβ + ɛ (1.7) where Y is the vector of observed pixel values, β is the vector of parameters, and ɛ is the vector of error terms. The design matrix X has one row for every time point in the original data and one column for every explanatory variable in the model. Specifically in fmri, the columns of X contain vectors corresponding to the on and off elements of the stimulus presented. We can find the magnitude of the parameter β corresponding to these column vectors in order to detect the presence or absence of activation. We do this by solving the normal equations: X T Y = (X T X) ˆβ (1.8) where ˆβ is the best linear estimate of β. Provided that X T X is invertible, ˆβ is given by ˆβ = (X T X) 1 X T Y (1.9) Such parameter estimates are normally distributed, and statistical inference can be made as to whether the β parameter corresponding to the model of an activation response is significantly different from the null hypothesis. The GLM provides a framework for most kinds of modeling of fmri data and can eliminate effects that may confound the analysis. There exist, of course, much more complicated and nuanced statistical computations that are done in addition to the GLM to extract meaning from functional data, but the general statistical methodology that is assumed throughout the rest of this paper is encapsulated by this section. 6

10 1.4 Foundational High-Resolution fmri Studies A fundamental background of fmri studies in the field of neuroscience is critical to understanding why tool development for high-dimensional datasets can further our understanding of the brain. We highlight some landmark studies in the following section, particularly in the sphere of high-resolution fmri, and provide justification for their relevance to this paper. Figure 1.3 illustrates some example cognitive models built from similar high-resolution neuroimaging data, again displaying the scientific promise of such data acquisition techniques. Yacoub et al. (2008) employs high-resolution fmri in order to characterize the existence and spatial features of orientation-selective columns in humans[12]. A larger number of these orientation columns are devoted to processing orientations around 90 degrees (vertical stimuli with horizontal motion), and can be linked to the spatial features of these columns in monkeys. Orientation columns in visual cortex cannot be discerned via anatomical characteristics, but instead require functional imaging methods that were traditionally limited by insufficient sensitivity, spatial accuracy, and image resolution. This study included the following two contributions: 1) proposing that unmapped or unknown columnar level functional organizations may exist in the human brain, and (2) making use of the precise advantages that high-resolution fmri offers in order to detect these small neuronal ensembles. Polimeni et al. (2010) provides a precursor to how we conceptualize and analyze high-resolution fmri data by introducing a laminar surface-based analysis method, studying the relationship between spatial localization and activation strength as a function of laminar depth[9]. This was accomplished by acquiring 1 mm isotropic, single-shot EPI at 7 T and sampling the BOLD signal exclusively from the superficial, middle, or deep cortical laminae. High-resolution fmri studies are limited by the biological point-spread of the hemodynamic signal, and the extent of this spread is determined by the local vascular distribution and the spatial specificity of blood flow regulation. Highly-accelerated EPI can limit image distortions to the point where a boundary-based registration algorithm accurately aligns the EPI data to the surface reconstruction. Avoiding surface laminae improved spatial localization, which implies that optimal spatial resolution in functional imaging of the cortex can be achieved using anatomically-informed spatial sampling to avoid large pial vessels. The group utilized FreeSurfer for preprocessing. 7

11 While we build tools to effectively analyze individual subject scans at high resolutions, an equally challenging task is to integrate group analysis of many subjects at a moderate spatial resolution, such as is the case in the Human Connectome Project (HCP)[4]. This clearly requires a distinct approach to preprocessing and data formats, but nonetheless represents a modern effort to use cutting-edge statistical techniques to explain fmri data that takes into account many similar considerations in the design of analytical tools. Specifically, the HCP has developed pipelines for structural, functional, and diffusion MRI to accomplish spatial artifact/distortion removal, surface generation, cross-modal registration, and alignment to standard space. This final standard space allows for combined cortical surface and subcortical volume analyses while reducing the storage and processing requirements for high spatial and temporal resolution data. Waehnert et al. (2014) began a trend in the field of analyzing intracortical structures, such as heavily myelinated layers in 3D[11]. Currently, these serve as a prerequisite for in-vivo parcellation of individual human brains. This parcellation can only be performed precisely if the profiles used in cortical analysis are anatomically meaningful. Profiles are often constructed as traverses that are perpendicular to computed laminae. Many methods for modeling cortical laminae exist, and include methods that construct laminae maintaining a constant distance to the cortical boundaries (equidistant laminae), compute equipotentials between the cortical boundary surfaces with the Laplace equation, and an equi-volume model that preserves the volume of cortical segments while layer thickness changes to compensate for cortical folding. In general, geometric and anatomical issues are critical when thinking about computing intricate statistics for distinct cortical layers. Mucki et al. (2015) employ high-resolution fmri to provide empirical evidence for theoretical feedback models, such as predictive coding[8]. Examples include the layered cortical organization of external versus internal visual processing streams during perception in healthy human subjects. Superficial layers of visual cortex V1 receive information when they are not directly stimulated, with information originating from contextual feedback from higher visual areas. The main contribution of this paper the potential of high-resolution fmri to access internal processing in sub-millimeter human cortex. 8

12 Finally, a recent review by Polimeni et al. (2017) surveys advanced data analysis strategies developed for high-resolution fmri. These new fmri studies acquire data reaching sub-millimeter scales, enabling new classes of experiments to probe the functional organization of the human brain[10]. These include strategies designed to mitigate distortion and artifacts associated with higher fields in order to preserve spatial resolution of the fmri data. Particular focus is placed on anatomically-informed analyses, including cortical surface-based analysis, which are powerful techniques that can guide each step of the analysis from preprocessing to statistical analysis to interpretation and visualization. Techniques for laminar and columnar fmri as well as single-subject individualized analyses are also discussed. This highlights the importance of proper analysis strategies when characterizing and understanding valuable data about the human brain. 9

13 Figure 1.3: Recent models in cognitive neuroscience related to high-resolution fmri. (a) Model of bottom-up computations in VTC [7]. (b) Distinct levels of explanatory sophistication and detail for models that characterize levels of neural activity [6]. 10

14 Chapter 2 AutoVizFS: Software Design 2.1 Loading Preprocessed fmri Data Most of the data processing from high-resolution fmri data in the lab is conducted using FreeSurfer, a software package for the analysis of structural and functional neuroimaging data from cross-sectional or longitudinal studies. FreeSurfer is developed by the Laboratory for Computational Imaging at the Athinoula A. Martinos Center for Biomedical Imaging. FreeSurfer is the software of choice for the Human Connectome Project as well as many other fmri labs around the world. FreeSurfer provides a wide range of tools for different use cases, from structural, functional, and diffusion MRI to multimodal and PET data computations. FreeSurfer preprocessing for the scans that the user wants to visualize and analyze is required for AutoVizFS. The code repository is built assuming that this is the case. The GUI is launched by running the autovizfs.m file from the MATLAB terminal. The starting point for the GUI is to select the Browse button in the Data Selection panel to locate the folder containing the GLM analysis for the desired fmri data. Typically, it is best to use the most recent GLM variant. In our case, the GLM included condition splitting, assumed canonical HRF, and noise regressors fixed to those derived from the layer mean. Upon folder selection, the results are then loaded in from the folder layer-by-layer, as shown by a progress bar. The path of the selected results directory is displayed in the text box. By default, AutoVizFS thresholds the tstatistics to the minimum and maximum range present in the data, displaying them on an inflated ventral surface of the brain using the hot colormap. 11

15 Figure 2.1: GUI layout for AutoVizFS. The directory containing the GLM results is displayed in the top left, while the visualization and analysis options sit in the right panel of the interface. The visual output is a ventral inflated view for a sample participant, with tstastics between face stimuli and all other experimental categories in the first FreeSurfer layer. 2.2 Display Options The GUI allows for relative flexibility in visualizing fmri results. Figure 2.1 shows the current GUI interface, and the available display options are detailed below. The Viewpoint drop-down menu allows the user to rotate the view of the cortical surface, viewing the data from the following anatomical options: ventral inflated, ventral sphere, occipital sphere, occipital inflated, parietal inflated, medial inflated, lateral inflated, medial-ventral inflated, ventral flat VTC, and flat EVC. Figure 2.2 contains a set of six distinct viewpoints. The Background option designates the map on which the fmri data will be overlayed on top of. Curvature provides an anatomical map of sulci (dark grey) and gyri (light grey). Mean EPI is a method to form a complete image from a single data sample, or snapshot, in fmri. This provides large speed advantages and a subject-specific anatomical background. Sulci, meanwhile, is 12

16 Figure 2.2: Different viewing options available within AutoVizFS for tstatistics plotted on the cortical surface with inflated ventral, inflated parietal, and ventral flat VTC rotations (top row) and mean EPI, Kastner atlas, and HCP-MMP atlas plotted as the background of the image (bottom row). similar to curvature but emphasizes sulci structures. The remaining options all superimpose different atlases on the standard curvature, offering the ability to view which atlas regions contain certain activity. The available atlases are Kastner, HCP-MMP1, and Visual Sulci. Figure 2.3 shows an example of how to utilize an atlas to make sense of stimulus-selective beta weights. Foreground is utilized to define which experimental conditions are computed for the functional results. The second text box is subtracted from the first text box, and multiple conditions can be grouped together using an underscore. For example, to view face and object trials versus the rest of the categories, the first text box would read faces objects and the second text box would read all. The experimental categories must be headers in the loaded fmri results. Colormap codes the functional results in the color scheme selected. Contrast Metric defines the statistical quantity plotted from the fmri data: tstatistics are the ratio of the departure of the estimated value of a parameter from its hypothesized value to its standard error, beta weights are the standardized 13

17 regression coefficient (the slope of a line in a regression equation), and normalized beta weights simply normalize the beta weights to be centered around zero. Threshold will constrain the fmri values plotted to a certain minimum and maximum range, which will allow functional data to be visualized correctly based on the contrast metric that is used. Finally, the Layer Display option allows the user to flip through the results with the aforementioned visual settings for each of the six layers created by preprocessing in Freesurfer (from pial to white), as well as the average across layers. 2.3 Line Profile Definition and Analysis By selecting the Draw Line button, the user is allowed to define vertices on the cortical surface. Simultaneously, instructions appear in the MATLAB terminal. The user clicks to create a new successive vertex that is connected to the previous vertex, and left-clicks to end the line-drawing process. The pixel coordinates corresponding to each vertex in the high-resolution image are displayed in the MATLAB terminal. Underneath the interface, the tool translates pixel coordinates to meaningful vertex values that map to functional data. Post-processing accounts for the fact that small pixel values might stay within one vertex value. Ultimately, non-repeating vertex values are stored to compose the defined line segment, which is achieved via a simple nearest-neighbor algorithm. Some important implementational caveats have been considered within AutoVizFS s line definition process. For instance, sequences of vertices may not be joined by edges, sequences of vertices may be jagged rather than a smooth line in 3-dimensional space, and cumulative distance is an overestimate of how long an ideal line would be. Therefore, the user must be careful to not define vertices near the edge of the cortical surface. After left-clicking, the user can again select Draw Line to clear the current line and begin defining a new one, or Clear Line to simply clear the current line. Selecting Analyze Line Profile generates a few plots that explain the fine-scale activity at the given vertices. These are detailed in Figure 2.4 and Figure

18 Figure 2.3: Posterior view on a cortical sphere in AutoVizFS, with areas selective for visual stimuli highlighted and corroborated by a Kastner atlas overlay (top row) and percent BOLD change elicited by face stimuli (bottom row). 15

19 Figure 2.4: (a) Line definition in AutoVizFS between the fusiform gyrus and occipital temporal sulcus and near the midfusiform gyrus. (b) Cortical layers from white to pial split by FreeSurfer and plotted in terms of cortical thickness along the aforementioned brain region, and (c) Bias-corrected EPI intensity and mean absolute beta roughly following the cortical thickness. 16

20 Figure 2.5: Sample line profile analysis done using AutoVizFS for the same region as Figure 2.3. Beta weights for each layer are plotted and separated by experimental condition, with anatomical curvature displayed along the top of the figures. Here, the fusiform face area (FFA) drives higher amplitude signal changes for experimental conditions that show images of faces as opposed to other stimuli. Note that overall amplitude decreases in deeper layers. 17

21 2.4 ROI Definition and Analysis Defining a region-of-interest (ROI) on the cortical surface can be insightful for identifying brain areas that are selective for a certain stimulus condition. The Draw ROI button invokes a built-in MATLAB function called roipoly that allows the user to define a polygonal region over the image, as in Figure 2.6. The user defines any number of vertices successively, selecting the initial vertex to close the polygon. The polygon s vertices can then be moved around to adjust the geometric shape. The user right-clicks to create the final mask. This polygon will most likely include cortical regions that are selective for more than just the desired stimulus, assuming that the edges encompass more than where the functional values exist. The Shrinkwrap button will restrict the polygonal region to where functional values exist inside the polygon, thereby resulting in the correct stimulus selectivity upon analysis. Making use of Toggle Overlay will clearly show where that ROI is defined on the cortical surface. Upon selecting Analyze ROI, the interface will open a new MATLAB window. This will preliminarily plot the beta weights for each experimental condition in a bar chart, allowing the user to flip through the same plot for each layer. This can be used to verify that the ROI is indeed selective for the desired stimulus, and can be corroborated by some of the aforementioned atlas background options. Finally, the user can use Save ROI to save out a defined ROI. A text box will appear, where the name of the ROI and directory for it to be saved to can be selected. These are.m files that can be loaded into MATLAB or back into the interface using the Load ROI button. This can be useful for comparison between subjects, sessions, or different stimulus-selective regions. 18

22 Figure 2.6: An example set of ROIs drawn in AutoVizFS. For each hemisphere, visually selective regions, marked by beta weights selective for face stimuli, are surrounded by a polygon on the posterior view of a cortical sphere. 19

23 Chapter 3 Application in Neuroscience 3.1 Scientific Promise From the point of view of a neuroscientist, the recent developments in in high-resolution fmri technology are particularly exciting, as they suggest that we can noninvasively measure activity at the columnar and laminar level across large swaths of human cortex. This promises to bridge the spatial scale between such experiments and recordings in animals. Ultimately, we must consider how best to make use of high-resolution fmri in order to answer scientific questions. This means scrutinizing and trying to make sense of the nature of these large-scale datasets, assessing whether they can practically support some of the hypothesis that they claim to. AutoVizFS is part of an effort to develop methods and tools towards this goal, with careful attention given to physical units, how to properly visualize results, and how to consider signal and noise characteristics. In the next few sections, we show how AutoVizFS is a step forward in terms of accurately measuring fine-scale neural activity within the framework of a simple localizer experiment. 3.2 Localizer Experiment The dataset we ll use to highlight the scientific promise of AutoVizFS was collected by members of the Computational Visual Neuroscience Laboratory at the University of Minnesota s Center for Magnetic Resonance Research in St. Paul, MN. Human participants were placed in the scanner and shown five distinct stimulus categories: faces, characters, body parts, objects, and places. A sample block of stimuli is shown in Figure 3.1. Stimuli come in 4-s trials, meaning that 8 images are displayed consecutively. Additionally, we used 144 trials per stimulus and ran each participant for 60 minutes. The data we collected included EPI at 7T (0.8mm, 2.2-s, 84 slices), which is 20

24 Figure 3.1: Sample series of eight images that were consecutively shown to participants in the localizer experiment. the slice prescription, and fieldmaps in the same slices at 7T. We also collected T1 and T2 anatomical data at 3T (0.8mm, whole-brain). In order to improve SNR, we averaged a number of T1 slices. This allows us to, for example, separate cortical layers because of the accurate surface estimation. Then, we took the averaged T1 slice and fed it to FreeSurfer with the hires option, doing some manual editing to increase the density of the surfaces and create a resultant six different depths. In our preprocessing of the functional data, we also conduct space and time corrections and run a basic GLM where we assume an HRF shape to extract beta weights. The beta weights are quantified by a percent signal change, as the raw signal amplitudes are divided by the mean signal intensity times 100. Examining both the high quality and high spatial density of our reconstructions is worthwhile at this point. Zooming in on a coronal T1 slice of a subject, toggling with the overlay of our surfaces, and smoothing the underlay, we can see the individual vertices on the actual surfaces, as shown in Figure 3.5. To get the functional volumes onto anatomical surfaces, we use an affine transformation which registers the functional T1 values to T2. Doing a cubic interpolation, as shown in Figure 3.2, deals with the effects of head motion, EPI spatial distortion, and registration to anatomy. Finally, we have to visualize the results after mapping the functional data onto surfaces. In order to do this, we implement a nearest neighbor strategy as in Figure 3.3 to ensure 21

25 Figure 3.2: Preprocessing step for functional data where EPI slices are recorded at different times, and each voxel is resampled across its own time course. This fixes the temporal offset of the slices, and the spatial offset is aligned using a cubic interpolation. 22

26 Figure 3.3: Nearest neighbor implementation used during preprocessing of the experimental data where (a) values are assigned to pixels based on the nearest vertex and (b) the algorithm is displayed on a larger scale. that pixel values correspond to known vertices with a experimental value attached to them. In summary, we handle spatial issues of resultant high-resolution fmri datasets in three main ways: 1) use functional volumes, 2) use a single cubic interpolation to place the data on a densely sampled cortical surface, and 3) visualize surfaces using a nearest neighbor lookup. In addition, AutoVizFS allows for plotting of beta weights for a given experimental condition. This means that for each vertex on the cortical surface we can visualize the percent bold change elicited by face stimuli divided into six distinct layers with reproducible fine scale detail. To observe this more quantitatively, we developed a Fourier-based analysis (Figure 3.4) and divided the spectrum into spatial frequency bins. We then sum the power within each bin to quantify the information difference. 3.3 Accurate Measurement of Fine-Scale Neural Activity The previous section shows an already nontrivial result that can be achieved with AutoVizFS: it is possible to measure fine-scale BOLD activity. Next, we ll take a 23

27 Figure 3.4: Spatial frequency Fourier analysis. The top panel shows the amount of power present at different spatial frequencies (i.e. one cycle per 8mm). The lower plot summarizes cortical curvature (sulci and gyri structures) where most of the variance exists at low frequencies, while the functional data exhibits image variance at 4 and 2 mm that is only captured by high-resolution fmri. 24

28 Figure 3.5: Conical columns and surfaces from a subject s T1 scan. (a) Zoomed-in view of the midfusiform gyrus and the fusiform gyrus (left) and expanded FreeSurfer layers from white to pial proportional to cortical thickness (right). (b) Inflated view of the right hemisphere of the same subject, with the same anatomical region marked with a line. (c) 3D coordinates of each of the six layers with their putative normals highlighting the increase and decrease of vertex density in relation to curvature. closer look at some issues with the results, and then discuss how AutoVizFS attempts to circumvent these. One main problem is that cortical curvature causes distortion in the data. Our method takes a volume with known spatial dimensions and samples it onto surfaces via a nearest neighbor interpolation. We typically get approximately square regions that confirm that our physical units were preserved well. However, when we zoom in on distinct cortical depths, it becomes clear that there are regions of high compression (gyri) as well as high expansion (sulci). These distort the shape of the sampled voxels, and can be seen in Figure 3.5. Starting with the gyrus as we move from deep to superficial depths, cortical surface area increases. So as we visualize these maps, gyri 25

29 have to compress. In contrast, looking at sulci and moving from deep to superficial depths, surface area decreases. This causes the opposite effect: sulci have to expand in our visualizations. This effect is not limited to spatial distortion. Resolution is altered as a voxel is forced to correspond to a single vertex rather than many vertices. These effects impact how we can interpret high resolution data, as much of the spatial structure is dependent on precise sulcal and gyral patterns in the subject. A second important problem relates to the influence of veins on functional data. For instance, veins cause extensive darkness in cortex that becomes more apparent in superficial layers (Figure 3.6). Moreover, these dark voxels are in sensible locations in the grey matter and are not artifacts of surface problems or misregistration. In order to assess this darkening effect, we can use a minimum intensity projection to see the veins more clearly. It is important to remember that venous structures exist in the brain. This is visually clear when we acquire high-resolution data, but the effects also exist in low-resolution data, even if they are not immediately visible. In addition to the extensive (and directly observable) susceptibility effects, veins also introduce local biases. After correcting for coil bias, we can plot the histogram of EPI intensity values. At the most superficial cortical depths, the tail of each distribution is extremely heavy, meaning that there is anatomical bias for veins in superficial cortical depths. Finally, we can show that veins corrupt our functional measurements. This is manifested particularly in a zoomed-in view of the cortex (Figure 3.7). When looking at functional responses, we have two main quantities of interest: beta weights, which represent the percent signal change evoked by the stimuli, and error on beta weights, which indicates the trial-to-trial variability. Looking at Figure 3.8 and starting with the error on the beta weights, there exists a high correspondences between dark vertices and large errors. The beta weight magnitudes portray the same effects, which are highly reproducible in all subjects. In conclusion, AutoVizFS can make accurate and reliable measurements of finescale BOLD activity. Given the systematic influence of veins, it is not clear that this is local neural activity. Our results indicate that we should take curvature seriously as it causes systematic changes in distortion and resolution. Additionally, we can use EPI intensity as a simple and convenient metric to track where vein effects are occurring. Analyzing functional data has to be done carefully when compensating for noise strength, and typical approaches such as t-statistics and multivariate classifiers tend to highlight the veins and therefore lead to bias. 26

30 Figure 3.6: Darkness and anatomy. (a) Histogram of EPI intensity where darkness covaries with lower depths, indicated by heavier tails in the distribution. This is particularly true in depth 1. (b) Individual subject data where the percentage of dark pixels is higher in gyri than sulci. 27

31 Figure 3.7: Zoomed-in view of cortex, with EPI intensity for coil bias. Intricate darkening patterns that corrupt the functional data are visible in the percent BOLD change due to veins. Given these analytical challenges, we believe that AutoVizFS can be an invaluable tool for the fmri community at large. A few methods for combating these difficulties are purposefully integrated into the software package: identifying cortical curvature, exploiting EPI intensity, and utilizing proper metrics to account for variations in signal and noise. Fundamentally, the package provides an automated way to visualize these complicated FreeSurfer surfaces and any corresponding functional activity on anatomically detailed maps. This provides a cursory check of fmri results. Standard atlases are available to overlay as references, while EPI and curvature are background options for any viewpoint. When the input data is split by experimental condition, AutoVizFS performs careful calculation of normalized beta weights and controls for tstatistics. Perhaps a more specific example is that we can observe that veins tend to amplify the measured beta weights in a given line profile (Figure 2.5). The user can visualize certain cortical regions to ensure that the stimulus is the source of these signal changes rather than any local anatomical features. We can further analyze a small part of a given line profile, as in Figure 3.9, to gain a better sense for category selectivity in the localizer experiment. For every vertex in the depth we divide the 28

32 Figure 3.8: EPI intensity plotted against beta weight magnitude (percent BOLD) in a histogram of all vertices for all subjects (left). Notice the symmetry in the figure: dark vertices indicating veins show extremely high signal values. The same pattern can be observed for the error on the beta weights (right). functional values by their L2 norm, which is the length of the vectors of the five category values in the given voxel. After this statistical manipulation, it becomes more clear that category selectivity is well-behaved. We can also tie this type of analysis back to the localizer experiment in a broad, scientific manner by asking the following question: does FFA, a known face-selective brain region, have any sort of substructure? In other words, are there parts of FFA that are more or less selective to faces, or that are more selective to other categories of stimuli? In theory, datasets like the one presented in this paper can answer questions like this. Ultimately, we envision AutoVizFS contributing towards providing an answer for scientific questions, such as characterization of regional substructures, but also assisting in the necessary awareness and careful control for curvature and darkness that enables confident observation of these effects. This is of critical importance, as many cognitive neuroscientists in the field of fmri are starting to conduct more depth-related analyses, especially with the advent of high-resolution fmri. One of the ongoing lines of work in our lab is a complete fix for these vein and darkness effects, but awareness of anatomical issues is a critical step for the field at large. 29

33 Figure 3.9: Detailed view of a small section of depth 2 from Figure 2.3 showing original beta weights and L2 normalized beta weights for the localizer experiment. The latter end of the plot shows almost complete face selectivity that isn t accentuated unless the right area of cortex is analyzed. 30

34 Chapter 4 Conclusion 4.1 Current Project Scope In its current form, AutoVizFS is a MATLAB-based toolbox for analyzing preprocessed FreeSurfer surfaces and functional data that is passed through a GLM. The core functionality of the package can be summarized in three distinct purposes: visualization, manual definition of cortical areas, and exploration of data based off of a given definition and recorded neural activity. In terms of visualization, we provide the ability to quickly view data on real surfaces with differing viewpoints, change colormap, access standard atlases, calculate tstatistics, beta weights, and normalized beta weights, select EPI intensity as a background option, and compare experimental conditions and FreeSurfer layers. Meanwhile, line and ROI definition via nearest-neighbor pixel mapping allows for inspection of a manually drawn region with predefined analysis tools. 4.2 Future Work In order for AutoVizFS to be a viable tool for the larger neuroscience and fmri community, we plan on integrating several key features. Critically, the current iteration of the GUI was built using GLM results from FreeSurfer. This is relatively straightforward when using built-in FreeSurfer utilities, but we don t want to assume that every user will have the same pipeline. Therefore, our tool should have the capability of performing the same detailed analysis for a standard FreeSurfer output that contains anatomical surfaces as well as functional data. Additionally, every visualization and analysis option available in the GUI should have the capability to be run from the command line. This will increase user productivity, as rendering images with high resolution takes time and isn t necessary when the user has a specific visualization in mind a priori or after some familiarity with the options provided in 31

35 AutoVizFS. This leads into the next feature that we believe is the toolbox s strongest selling point: batch automation. To our knowledge, it is either difficult or impossible for most other software packages to repeat a given analytical configuration within subjects for slightly differing options, such as viewpoints or colormaps, or across subjects. In future versions, we want to make large-scale generation of figures a focal point of AutoVizFS. As mentioned previously in this paper, we provide canned data exploration options for line profiles and ROIs. The output of either of these is a collection of vertices with some functional values attached. The user will be able to save these out in a.m file and conduct further analysis by modifying our code or using locally-based scripts. In order to continue integrating ease of thinking about the effects of curvature and darkness on functional measurements, we ll add the capability to interactively zoom-in within any viewpoint, generate a spatially accurate three-dimensional representation of any defined line, and provide the option of selecting surface voxels (Figure 4.1) as a background option to visualize curvature expansion and compression differences throughout cortical layers. Finally, we want to combine ROIs and line profiles in an intuitive way that allows the user to define a polygonal region, and then carefully interpolate evenly spread line segments throughout that region in either the horizontal or vertical direction. Then, further line profile analyses can be done within that given region. These extensions will be completed over the coming months, after which we plan to upload the code to a GitHub repository. The code will be accompanied by a website that details all of the visualization and analysis options in AutoVizFS, and provide how-to videos that will guide the user through various hypothetical scenarios (i.e. I want to automate x number of some type of visualization for y number of subjects and z number of viewpoints ). Finally, we ll publish a paper explaining the toolbox, its features, and its value to the broader neuroscience community. 32

36 Figure 4.1: Inflated cortical view with surface voxels plotted on top to help identify anatomical curvature distortion. (a) Color changes indicate a new voxel. The crosssections are relatively square when sampled from the white surface, but become topologically distorted when the pial surface is sampled instead. (b) Zoomed in view of the surface voxels for each FreeSurfer layer. Again, there is a gradual compression effect when we move from the white to pial layer for gyri, and an analogous expansion effect at sulci. 33

37 Bibliography [1] Cox, R.W. (1996) AFNI: Software for analysis and visualization of functional magnetic resonance neuroimages. Computers and Biomedical Research 29, /cbmr [2] Fischl, B. (2012) FreeSurfer. NeuroImage 62, [3] Gao, J.S., Hugh, A.G., Lescroart, M.D., and Gallant, J.L. (2015) Pycortex: an interactive surface visualizer for fmri. Front. Neuroinform /fninf [4] Glasser, M.F., Sotiropoulos, S.N., Wilson, J.A., Coalson, T.S., Fischl, B., Andersson, J.L.,... Jenkinson, M. (2013) The minimal preprocessing pipelines for the Human Connectome Project. NeuroImage 80, [5] Goebel, R. (2012) BrainVoyager past, present, future. NeuroImage. 62, /j.neuroimage [6] Kay, K.N. (2017) Principles for models of neural information processing. NeuroImage /j.neuroimage [7] Kay, K.N, and Yeatman, J.D. (2017) Bottom-up and top-down computations in world- and face-selective cortex. elife, /eLife [8] Muckli, L., De Martino, F., Vizioli, L., Petro, L.S., Smith, F.W., Ugurbil, K., Goebel, R., and Yacoub, E. (2015) Contextual feedback to superficial layers of V1. Current Biology 25, [9] Polimeni, J.R., Fischl, B., Greve, D.N., and Wald L.L. (2010) Laminar analysis of 7 T BOLD using an imposed spatial activation pattern in human V1. NeuroImage 52,

38 [10] Polimeni, J.R., Renvall, V., Zaretskaya, N., and Fischl, B. (2017) Analysis strategies for high-resolution UHF-fMRI data. NeuroImage, [11] Waehnert, M.D., Dinse, J., Weiss, M., Streicher, M.N., Waehnert, P., Geyer, S., Turner, R., and Lazin, P.L. (2014) Anatomically motivated modeling of cortical laminae. NeuroImage 93, [12] Yacoub, E., Harel, N., and Uğurbil, K. (2008) High-field fmri unveils orientation columns in humans. PNAS 105,