Introduction of a Quantitative Evaluation Method for White Matter Tractography using a HARDI-based Reference

Transcription

1 Introduction of a Quantitative Evaluation Method for White Matter Tractography using a HARDI-based Reference Peter F. Neher 1, Bram Stieltjes 2, Hans-Peter Meinzer 1, and Klaus H. Fritzsche 1,2, 1 German Cancer Research Center (DKFZ), Medical and Biological Informatics, Heidelberg, Germany 2 German Cancer Research Center (DKFZ), Quantitative Image-based Disease Characterization, Heidelberg, Germany Abstract. The diffusion tensor imaging (DTI) Tractography Challenge MICCAI 2011 featured a review board of highly recognized medical experts and prominent research groups working in the field of tractography. The workshop revealed the limitations of current tractography algorithms in crossing fiber regions when clinical image data quality is provided and spotlighted a critical need for objective and quantitative evaluation methods to stimulate a clinically relevant advancement in the field. Here we address this issue by proposing a reference-based validation procedure that allows for objective and quantitative assessment of tractography results obtained from DTI data of clinical quality. The method employs high angular resolution diffusion images (HARDI) as reference datasets and is evaluated by applying it to two well-known tractography algorithms. Their different capabilities in the reconstruction of crossing fiber regions were successfully revealed by the validation procedure. We conclude the paper with an outlook on open access of data and public validation opportunities. Keywords: Tractography Evaluation, Fiber Tracking, Diffusion-weighted Imaging, Diffusion Tensor Imaging, HARDI, Reference-based Validation Procedure 1 Introduction Diffusion based tractography noninvasively provides insight into the course of white matter pathways in the living human brain. However, good tract representation in research settings with high quality data is strongly hampered by a reduced image quality in a clinical setting. This issue was confirmed during the DTI Tractography Challenge MICCAI 2011 where a neurosurgically highly relevant task had to be solved on clinical data [1]. According to the workshops medical review board, none of the participants could satisfactorily solve the task Corresponding author. address: k.fritzsche@dkfz-heidelberg.de.

2 2 Peter F. Neher of reconstructing the corticospinal tract, one of the most prominent tracts in the brain. More than one third of all image voxels contain crossing fibers [2] and thus challenge tensor-based tractography with regions of uncertainty that have to be resolved by some kind of intelligent mechanism or apriori knowledge. We believe, also motivated by the success story of the organ segmentation challenges at MICCAI, that the key aspect in making a significant step forward in the field of diffusion tractography research will be the development of quantitative tools for automatic evaluation of these mechanisms. Most current validation techniques are either based on synthetic [3] or physical [4] phantoms that can only represent limited aspects of in-vivo data. Existing approaches for quantitative tractography evaluation on real patient data have so far been limited to the evaluation of reproducibility, e.g. by bootstrap analysis [5], or the comparison of fiber envelopes by the STAPLE algorithm [6]. Evaluation by medical experts suffers from a high inter and intra rater variability. Due to the complexity and diversity of fiber bundles in the brain, manually defined gold standards are difficult to establish and not commonly used in tractography. An optimal evaluation method would be automated and generically applicable to all fiber structures in the brain, yielding quantitative measures of reproducibility, validity and integrity of tracking algorithms especially in complex fiber configurations. While many of the advanced diffusion acquisition and modeling techniques like high angular resolution diffusion imaging (HARDI) are not applicable in clinical settings due to extensive acquisition and processing times, we believe that they can provide important means for the validation of tractography algorithms. We propose a reference-based validation procedure that uses HARDI data as basis of comparison for tractography obtained from DTI. The method proposed here follows the validation framework for medical image processing introduced by Jannin et al. [7, 8] and is particularly useful for the evaluation of tractography results in crossing regions. 2 Methods The proposed reference-based validation framework is depicted in Fig. 1 and discribed in more detail in section 2.1. In our experiments the procedure was applied to two well known tractography algorithms (section 2.2). 2.1 Validation framework Validation objective: The objective is the validation of tractography algorithms with a focus on their ability to correctly reconstruct crossing regions in the brain from DTI datasets that do not explicitely model mutliple directions per voxel. Validation datasets: The validation dataset consists of HARDI and corresponding DTI datasets. The DTI data can either be acquired in the same session as the HARDI dataset or it can be subsampled from the HARDI dataset

3 Reference-based Evaluation of White Matter Tractography 3 Fig. 1. Schematic depiction of the proposed reference-based validation procedures for quantitative tractography evaluation. The main contributions of this work are highlighted in blue. Adapted from [8]. using the desired b-value, spatial resolution, number of gradient directions and gradient distribution scheme. Computation of the reference: In order to obtain a reference, diffusion orientation distribution functions (dodf) are reconstructed from the HARDI dataset [9]. Comparison: We propose two options to compare the DTI tractography to the reference: an analysis of the voxel-wise fiber directions (fodf) as illustrated in Fig. 2a or a comparison based on the dodf images as shown in Fig. 2b. Fig. 2. Illustration of the comparison of the tractography result with the reference image based on directions (a) and dodf images (b). Direction-based comparison: For a direction-based comparison, fodfs have to be calculated from the reference image as well as from the tractography result.

4 4 Peter F. Neher The relation between dodf, R and fodf is illustrated in Fig. 3. The fodfs are calculated from the reference image using spherical deconvolution as described by Tournier et al. [10]. The deconvolution kernel R is estimated by aligning and averaging the most anisotropic dodfs in the image. The fodfs of the fiber tractography result at a spatial position x are calculated by averaging the fiber fragments in the neighborhood of x that is defined by an isotropic Gaussian with variance σ: fodf trac (x, n) = 1 2π f F f e 1 x x 2 2 σ 2 δ(t(x ) n)dx, (1) where δ is the Dirac delta function, F is the set of all fibers and t(x) is the fiber tangent at position x on fiber f. An angular threshold is used to combine clusters of fodf maxima that indicate a common underlying fiber direction. Fig. 3. Illustration of the relation between the kernel R, fodf and dodf. Adapted from [11]. Several measures can be evaluated for comparisons based on the fodf directions: 1. Sensitivity and specificity of the detection of crossing fiber voxels. 2. Root-mean-square (RMS) error of the number of detected directions per voxel. 3. RMS angular error of detected direction angles. 4. Visual inspection. Image-based comparison: For an image-based comparison, an artificial dodf image has to be calculated from the tracking result. This can be achieved by a convolution of the fodf (c.f. equation 1) with the convolution kernel R as illustrated in Fig. 3. The resulting dodf images can either be compared directly or via derived quantities like the generalized fractional anisotropy: 1. RMS error of derived quantities or of the dodfs. 2. Jensen-Shannon divergence [12] of the dodfs. 3. Distance measures based on euclidean metrics [13] of the dodfs. 4. Visual inspection.

5 Reference-based Evaluation of White Matter Tractography Experiments The images used in our experiments were obtained using a single-shot EPI sequence on a 3T MR scanner, 64 different gradient directions with three repetitions, 2.5 mm isotropic resolution and two shells at b-values of 1000 mm/s 2 and 3500 mm/s 2. All images were corrected for head motion and eddycurrent effects using an affine registration to the first baseline image employing FSL-FLIRT ( The gradient directions were corrected according to these transformations. The 192 images at b=3500 mm/s 2 were used as HARDI reference image. A DTI image of clinical quality was extracted from the b=1000 mm/s 2 images by selecting 30 unique gradient directions. The validation procedure was tested on a local streamlining approach based on the well known FACT algorithm [14] and a global probabilistic approach called Gibbs tracking [15]. The local algorithm was automatically seeded in every voxel and the FA stopping criterion was set to 0.2. The Gibbs tracking was run with 10 8 iterations. These two tracking algorithms were chosen to illustrate the capability of our method to detect the well known limitations of FACT to resolve crossing fiber configurations as well as the expected better performance of a global approach in this situation that was already shown during MICCAI s Fiber Cup 2009 [16]. We illustrate the direction-based part of our validation approach (c.f. Fig. 2a) by calculating the sensitivity and specificity of the algorithms capabilities to detect voxels containing crossing fibers and the image-based part (c.f. Fig. 2b) by generating an artificial dodf image for each tracking result and comparing it visually to the reference. In all experiments we focused on one representative crossing of the corpus callosum and the corona radiata. This area was manually labelled by a radiologist. 3 Results Fig. 4 shows both fiber tractography results in the region of interest. Both algorithms show a high specificity in detecting crossing voxels (Tab. 1). The low sensitivity of the FACT algorithm shows that almost no fiber crossings could be resolved. The global Gibbs tracking was capable of resolving about 26% of all crossing voxels. The most intuitive way to evaluate the tracking result in the image domain is by visual inspection. The different dodf images are depicted in Fig. 5 and are discussed in the next section. Algorithm Sensitivity Specificity Gibbs 26% 86% FACT 3% 97% Table 1. Sensitivity and specificity of the two tracking algorithms to detect voxels containing crossing fibers.

6 6 Peter F. Neher (a) ROI (b) FACT tracking (c) Gibbs tracking Fig. 4. Reconstructed tensor image overlaid with the region of interest (red) containing the crossing of corpus callosum and corona radiata (a) and the tractography results of FACT (b) and Gibbs (c) tracking in the selected ROI. 4 Discussion This paper proposes a reference-based approach to the automatic evaluation of fiber tracking algorithms using high quality HARDI data as basis of comparison for results generated from DTI data of clinical quality. The proposed method can be used to evaluate several different measures of tracking accuracy and integrity. It is especially valuable for the examination of the algorithms performance in crossing regions and other situations that cannot be resolved by clinical DTI data. To our knowledge this is the first approach to quantitatively evaluate the performance of fiber tracking algorithms on clinical DTI data with a HARDI reference yielding objective and easy to interpret results like sensitivity and specificity. The method was demonstrated by comparing a local streamlining approach based on the well known FACT algorithm to a global tractography method called Gibbs tracking. Their different capabilities in the reconstruction of crossing fiber regions were successfully revealed by the validation procedure. The crossing sensitivity of the streamline approach shows that almost no crossing fiber was detected. The global approach shows a somewhat higher detection rate. These findings are fortified by the visual analysis of the fiber tracts and visual comparison of dodf images and are consistent with the experiences made during the DTI Tractography Challenge MICCAI While the presented evaluation procedure and corresponding results are very promising, there is still room for improvement considering the current analysis pipeline: The reconstruction of dodfs and fodfs from the HARDI acquisition would potentially be more accurate when employing a multi-shell q-ball reconstruction scheme instead of selecting only the higher q-shell as reference image. The analytical calculation of dodf maxima from the spherical harmonic coefficients should also be considered as an alternative to the deconvolution of the signal. In future experiments we will start incorporating more regions of the brain and consider the complete set of proposed indices of comparison in our experiments. Finally, one important limitation should be noted: the proposed validation scheme is, of course, limited by the quality and potential ambiguities

7 Reference-based Evaluation of White Matter Tractography (a) Reference image dodfs (b) Tensor image dodfs (c) dodfs based on FACT (d) dodfs based on Gibbs 7 Fig. 5. Reference (a) and DTI (b) images in comparison with the artificial dodf images generated from the two tractography results (c and d). of the reference images (e.g. if crossing and kissing fiber configurations should be distinguished). Besides the further exploration of the methods proposed here we are planning to achieve a general improvement of the evaluation and comparability of tracking methods developed by groups around the world. Following the model of the Fiber Cup where phantom datasets and access to evaluation methods was made publicly available, we plan to establish a similar framework for the evaluation of in-vivo datasets. The first step will be the acquisition of multiple high quality datasets from different subjects and the construction of a large image database containing quality pyramids of test images with multiple levels of spatial resolution, number of gradient directions and number of q-shells. It is planned to provide open access to this database, allowing scientists in the whole community to test their tracking algorithms on common datasets and to submit their results for evaluation.

8 8 Peter F. Neher References 1. Pujol, S., Kikinis, R., Golby, A., Gerig, G., Styner, M., Wells, W., Westin, C.F., Gouttard, S. In: DTI Tractography for Neurosurgical Planning: A Grand Challenge, MICCAI (September 2011) 2. Behrens, T.E.J., Berg, H.J., Jbabdi, S., Rushworth, M.F.S., Woolrich, M.W.: Probabilistic diffusion tractography with multiple fibre orientations: What can we gain? Neuroimage 34(1) (Jan 2007) Delputte, S., Fieremans, E., Dedeene, Y., D Asseler, Y., Achten, R., Lemahieu, I., de Walle, R.V.: Quantitative validation of white matter fiber tractography by use of an anatomically realistic synthetic diffusion tensor phantom. In: Proc. Intl. Soc. Mag. Reson. Med. 14. (2006) 4. Moussavi-Biugui, A., Stieltjes, B., Fritzsche, K., Semmler, W., Laun, F.B.: Novel spherical phantoms for q-ball imaging under in vivo conditions. Magnetic Resonance in Medicine 65 (2011) Jones, D.K., Pierpaoli, C.: Confidence mapping in diffusion tensor magnetic resonance imaging tractography using a bootstrap approach. Magnetic Resonance in Medicine 53 (2005) Pujol, S., Westin, C.F., Whitaker, R., Gerig, G., Fletcher, T., Magnotta, V., Bouix, S., Kikinis, R., III, W.M.W., Gollub, R.: Preliminary results on the use of staple for evaluating dt-mri tractography in the absence of ground truth. In: Proc. Intl. Soc. Mag. Reson. Med. 17. (2009) 7. Jannin, P., Fitzpatrick, J., Hawkes, D., Pennec, X., Shahidi, R., Vannier, M.: Validation of medical image processing in image-guided therapy. IEEE Trans Med Imaging 21(12) (2002) Jannin, P., Grova, C., Maurer, C.: Model for defining and reporting referencebased validation protocols in medical image processing. Int J CARS 1 (2006) /s Aganj, I., Lenglet, C., Sapiro, G.: Odf reconstruction in q-ball imaging with solid angle consideration. In: Sixth IEEE International Symposium on Biomedical Imaging. (2009) 10. Tournier, J.D., Calamante, F., Gadian, D.G., Connelly, A.: Direct estimation of the fiber orientation density function from diffusion-weighted mri data using spherical deconvolution. NeuroImage 23 (2004) Lenglet, C., Campbell, J., Descoteaux, M., Haro, G., Savadjiev, P., Wassermann, D., Anwander, A., Deriche, R., Pike, G., Sapiro, G., Siddiqi, K., Thompson, P.: Mathematical methods for diffusion mri processing. NeuroImage 45 (2009) Cohen-Adad, J., Descoteaux, M., Wald, L.L.: Quality assessment of high angular resolution diffusion imaging data using bootstrap on q-ball reconstruction. Journal of Magnetic Resonance Imaging 33 (2011) Pasternak, O., Sochen, N., Basser, P.J.: The effect of metric selection on the analysis of diffusion tensor mri data. NeuroImage 49 (2010) Fillard, P., Gerig, G.: Analysis tool for diffusion tensor mri. In: In Proc. of Medical Image Computing and Computer-Assisted Intervention, Springer (2003) Reisert, M., Mader, I., Anastasopoulos, C., Weigel, M., Schnell, S., Kiselev, V.: Global fiber reconstruction becomes practical. Neuroimage 54 (2011) Fillard, P., Descoteaux, M., Goh, A., Gouttard, S., Jeurissen, B., Malcolm, J., Ramirez-Manzanares, A., Reisert, M., Sakaie, K., Tensaouti, F., Yo, T., Mangin, J.F., Poupon, C.: Quantitative evaluation of 10 tractography algorithms on a realistic diffusion mr phantom. Neuroimage 56(1) (May 2011)