From Image to Video: Real-time Medical Imaging with MRI

Transcription

1 From Image to Video: Real-time Medical Imaging with MRI Sebastian Schaetz, Martin Uecker BiomedNMR Forschungs GmbH at the MPI for biophysical Chemistry, Goettingen, Germany Electrical Engineering and Computer Sciences, University of California, Berkeley GTC 2013 (S3236) March 20th, 2013

2 About Us Computer scientist and physicist, working on Magnetic Resonance Imaging (MRI) in close collaboration with clinicians

3 Magnetic Resonance Imaging (MRI) Noninvasive imaging of living organisms Visualization of structure and function Image slices of arbitrary location and orientation No ionizing radiation Applications: Radiology, clinical research, and neuro science Limitations: Strong magnetic eld (contraindication e.g. pacemaker) Long measurement times

4 Real-time MRI Real-time imaging Imaging of dynamic processes Adequate spatial and temporal resolution No cardiac or respiratory gating, no stop motion Real-time processing Fast and low-latency reconstruction Requirement: bounded processing delay

5 Principles of MR Imaging

6 Principles of MR Imaging

7 Principles of MR Imaging

8 Principles of MR Imaging

9 Principles of MR Imaging

10 Principles of MR Imaging

11 Principles of MR Imaging

12 Principles of MR Imaging

13 Principles of MR Imaging

14 Principles of MR Imaging

15 Principles of MR Imaging

16 Principles of MR Imaging

17 Principles of MR Imaging

18 Principles of MR Imaging

19 Principles of MR Imaging

20 Principles of MR Imaging

21 Principles of MR Imaging

22 Principles of MR Imaging

23 Principles of MR Imaging?

24 Image Reconstruction as Inverse Problem Forward problem: y = Fx + n y data, F (nonlinear) operator, x image (and more), n noise Regularized solution: Advantages: x = argmin x Fx y 2 2 }{{} data consistency Modelling of physical eects (coil sensitivities) + αr(x) }{{} regularization Prior knowledge via suitable regularization terms

25 Nonlinear Inverse Reconstruction Inverse problem: Unknown image ρ and coil sensitivities c j Nonlinear equation Fx = y Reconstruction: Iteratively regularized Gauss-Newton method (IRGNM) x n+1 x n = argmin δx DF H (x n )δx + F (x n ) y α n δx + x n 2 2 Smoothness penalty for the coil sensitivities Uecker et al., Magn Reson Med 60:674682, 2008

26 Challenge: Real-time Reconstruction Measurements with up to 50 fps Images must be available immediately Iterative algorithm Fixed problem size: "strong scaling" Size and cost of computing system relevant Zhang S et al, JMRI 2012;35:

27 Multi-GPU System Memory b CPU GPU GPU GPU GPU c a CPU d GPU GPU GPU Memory 4HE TYAN FT72B7015 a: Host 2 x 6 core Intel Xeon, 96 GB b: Host 8 x GeForce GTX 580 c: Device 12,7 TFLOPS and 1,53 TB/s d: Device Host Device GPU 19 GB/s 6 GB/s Device 2,5 GB/s Device 6 GB/s

28 Parallelization of Algorithm: Data Parallelism irgn cg

29 Parallelization of Algorithm: Data Parallelism irgn cg

30 Parallelization of Algorithm: Data Parallelism Communication n n g

31 Results

32 Results - Details

33 Results - Details

34 P2P Transfer Optimization naive 1.28 ms 4 GPUs, matrix 384x384

35 P2P Transfer Optimization naive 2D 1.28 ms 4 GPUs, matrix 384x ms

36 P2P Transfer Optimization naive 2D optimized sync 1.28 ms 0.43 ms 0.37 ms 4 GPUs, matrix 384x384 Speedup = 3.5

37 Generalization: A multi-gpu programming library C++ Library based on Boost Identical code for 1...N GPUs Full control over application Minimal overhead Full access to performance-relevant hardware features Integration of established algorithms and libraries CUDA backend (OpenCL backend in preparation)

38 Concept: Segmented Container data segmented vector pointer0 pointer1 pointer2 size0 size1 size2 GPU0 GPU1 GPU2

39 Segmented Container Usage 1 // coil profile size (e.g. 512 * 512 * 10) 2 std::size_t size = dim_x * dim_y * coil_profiles; 3 4 // create runtime environment, use 4 GPUs 5 environment e(create_dev_group(0, 4)); 6 7 // create vector in CPU main memory to store coil profiles 8 std::vector<cfloat> Coils_host(size); 9 10 // allocate memory for coil profiles across all 4 GPUs 11 seg_dev_vector<cfloat> Coils_device(size, dim_x * dim_y); // copy from host to device 14 copy(coils_host, Coils_device.begin()); // copy back from device to host 17 copy(coils_device, Coils_host.begin());

40 Intra-System Communication copy copy broadcast GPU0 GPU0 GPU0 GPU1 GPU1 GPU1 CPU/GPU CPU/GPU GPU2 GPU2 CPU/GPU GPU2 scatter/gather reduce all-reduce GPU0 GPU0 GPU0 GPU0 CPU/ GPU GPU1 GPU1 GPU1 GPU1 GPU2 CPU/GPU GPU2 GPU2 GPU2

41 Interface and Integration All memory transfers (except reduce) through one function: copy(source_range, destination_iterator); Interoperability with other libraries and frameworks such as Boost ublas, Matlab: 1 #include <mgpu/container/dev_vector.hpp> 2 #include <mgpu/transfer/copy.hpp> 3 void mexfunction(int nlhs, mxarray * plhs[], int nrhs, const mxarray *prhs[]) 4 { 5 double const * i_real = mxgetpr(prhs[0]); 6 mgpu::dev_vector<double> dev(size); 7 mgpu::copy(mgpu::make_range(i_real, i_real+size), dev.begin()); 8 plhs[0] = mxcreatenumericmatrix(size, 1, mxdouble_class, mxcomplex); 9 double * o_imag = mxgetpi(plhs[0]); 10 mgpu::copy(dev, o_imag); 11 }

42 MGPU Library Available here (alpha version) Schätz, Sebastian and Uecker, Martin. "A Multi-GPU Programming Library for Real-Time Applications." Algorithms and Architectures for Parallel Processing (2012):

43 Parallelization of Algorithm: Task Parallelism Problem: initial value and temporal continuity

44 Parallelization of Algorithm: Task Parallelism Problem: initial value and temporal continuity GPU GPU 0 GPU t in s

45 Beating Heart Example

46 Singing Example

47 From Image to Video: Real-time Medical Imaging with MRI