Acceleration of Probabilistic Tractography Using Multi-GPU Parallel Processing. Jungsoo Lee, Sun Mi Park, Dae-Shik Kim

Transcription

1 Acceleration of Probabilistic Tractography Using Multi-GPU Parallel Processing Jungsoo Lee, Sun Mi Park, Dae-Shik Kim Introduction In particular, probabilistic tractography requires relatively long computation time. Computation time is a key element. We would therefore like to modify sequential algorithms to become parallel algorithms and take advantage of parallel processing with the GPU. With a much higher number of cores available as compared to that of the CPU, the GPU is better suited to parallel processing and displays even better performance when processing larger amounts of data. Methods Data Acquisition DW images were acquired by a Philips 3-tesla scanner and obtained with the following parameters: TR = ms; 129 volumes; TE = 80 ms; 128 non-colinear diffusion gradient directions with a b- factor of 3000 s/mm 2 ; a b-factor image of 0 s/mm 2 is one; matrix size is ; mm 3 field of view as acquired. Parallelized Algorithm of Probabilistic Tractography (BEDPOSTX of FSL) This process makes probability density function (pdf) for every voxel Bayesian estimation of diffusion parameters obtained using sampling techniques [1-3]. The goal of this process is to infer optimal predicted values μ i from eight free parameters using MCMC. P(w Y, M) = P(Y w, M)P(w M) P(Y M) where w (θ, φ, ψ, λ 1, λ 2, λ 3, S 0, σ) is the set of parameters of data. Y and M are data and model. The first step of parallelizing is to find the dependencies in the algorithm. It is important that it considers how to separate algorithm. When probability distribution function is obtained in a voxel, MCMC is used. In order to obtain mean and variance of θ and φ distributions, algorithm runs 3000 steps because of burnin process and every second jump sampling. Obtaining mean and variance of distributions in each voxel is not affected by results of the different each voxel and this is same process with different data in each voxel. This is a key parallel algorithm. As in Figure 1, sequential algorithm is calculated at one by one voxel and this algorithm generates many loops. On the other hand, parallel algorithm can simultaneously be calculated at many voxels; as many pdfs in voxels may

2 be generated as there are threads, because mean and variance of probability distribution function (θ, φ) of one voxel corresponds to one thread. We used 3 GPUs to provide a significant number of cores (720 cores). To simultaneously operate 3 GPUs, CPU parallel processing is used based on OpenMP. Approximately, 250,000 voxels are processed per a GPU respectively.

3 Results To compare results of sequential algorithm and GPU parallel algorithm, we used two hardware. First hardware is desktop PC featuring Intel Xeon E5520, 8 GB RAM. Second hardware is desktop server featuring same CPU, RAM and 3 NVIDIA TESLA C1060. The result is as follows: When we ran sequential algorithms using CPU, we had a total time of approximately 34 hours. On the other hand, when we ran parallel algorithms based on one GPU, the total computation time is about 100 minutes, and when we ran it using 3 GPUs, it took about 30 minutes which is approximately 60 times faster than using sequential processing. This performance result was obtained to maximize performance through optimization process (Figure 3) Using shared and constant memories are a great help to raise performance. In this study, shared memory was used for some variables by using coalesced memory access to reduce processing time.

4 Conclusions Probabilistic tractography using multi-gpu parallel processing can dramatically reduce computation time. In particular, GPU parallel processing is very suitable for tractography algorithms, because the greater part of the tractography algorithm is used to obtain pdf of each voxel and the algorithm has highly parallelizable properties. GPU processing is also much more cost effective than CPU processing. Through this study, the results show us the possibilities of realtime processing of probabilistic tractography using GPU parallel processing. Although parallel programming using GPU has not yet become widespread, it will be applied across the brain imaging research as well as DTI tractography in the near future.

5 References [1] T. E. J. Behrens, H. Johansen-Berg, M. W. Woolrich, S. M. Smith, C. A. M. Wheeler-Kingshott, P. A. Boulby, G. J. Barker, E. I. Sillery, K. Sheehan, O. Ciccarelli, A. J. Thompson, J. M. Brady, P. M. Matthews. (2003). Non-invasive mapping of connections between human thalamus and cortex using diffusion imaging. Nature Neuroscience. 6(7), pp [2] T. E. J. Behrens, M. W. Woolrich, M. Jenkinson, H. Johansen-Berg, R. G. Nunes, S. Clare, P. M. Matthews, J. M. Brady, and S. M. Smith. (2003). Characterization and Propagation of Uncertainty in Diffusion-Weighted MR Imaging. Magnetic Resonance in Medicine, 50, pp [3] M. W. Woolrich, S. Jbabdi, B. Patenaude, M. Chappell, S. Makni, T. E. J. Behrens, C. Beckmann, M. Jenkinson, and S. M. Smith. (2009). Bayesian analysis of neuroimaging data in FSL. NeuroImage, 45, pp. S173-S186. [4] D. B. Kirk, W. W. Hwu. (2010). Programming Massively Parallel Processors A Hands-on Approach. Elsevier, Morgan Kaufmann Publishers.