Fast, near real-time, Monte Carlo dose calculations using GPU Xun Jia Ph.D. xun.jia@utsouthwestern.edu Outline GPU Monte Carlo Clinical Applications Conclusions 2 Outline GPU Monte Carlo Clinical Applications Conclusions 3 1
GPU A specialized accelerator (personal super-computer) >1000 cores -16 cores Comparison CPU/GPU peak processing power Comparison CPU/GPU performance in linear algebra solver GPU Advantages High processing power: 3- TFLOPS per card nowadays Low cost: order of magnitude lower than CPU with similar processing power Easy to set up, maintain, and access Full system control: do not rely on 3 rd party management Low burden of data communication: card is local at the computer Suitable for Medical Physics problems Jia et. al. Review Article, PMB, 59, R151(201) 5 GPU Monte Carlo project Since 2009 in UCSD Utilize GPU to accelerate MC simulations in medical physics Use appropriate physics model to maintain accuracy Design GPU-friendly implementations for high efficiency Apply developed codes to solve medical physics problems 6 2
GPU Monte Carlo project Published >10 papers on peer-reviewed journals, with 6 more under review/in preparation, and >20 conference presentations Four GPU training courses Upcoming 5 th one in Oct 201 at UTSW 7 GPU Monte Carlo project A wide spectrum of GPU-based MC packages gdpm: MV photon/electron (2009) gmcdrr/gctd: kv photon (2011) gpmc: proton (+electron) (2012) gbmc: brachytherapy dose calculations (201) g????: MV photon/electron, opencl (201) Developing new GPU packages Unifying code interface Solve clinical problems 8 gdpm gdpm Maintain DPM physics and hence accuracy Seek for optimal implementation and high efficiency Jia et. al., Phys. Med. Biol., 55, 3077 (2010) Jia et. al., Phys. Med. Biol., 56, 7107 (2011) Phase space source model with GPU-friendly implementation Results Townson et. al., Phys. Med. Biol., 58, 31(2013) 9 3
Dose difference(gy) Difference(Gy) Dose difference(gy) Difference(Gy) Difference(Gy) Difference(Gy) Difference(Gy) Difference(Gy) Dose(Gy) Dose(Gy) Difference(Gy) Difference(Gy) Difference(Gy) Difference(Gy) Dose(Gy) 8 6 2 0 7/2/201 gdpm Average relative uncertainty, t-statistical test passing rate, execution time T, and speed-up factor Source type # of Histories Case D/D (%) P all (%) T CPU (sec) T GPU (sec) T CPU/T GPU 20MeV e 2.5 10 6 water-lung-water 0.98 99.9 117.5 1.71 69.7 20MeV e 2.5 10 6 water-bone-water 0.99 99.8 127.0 1.65 77.0 6MV p 2.5 10 8 water-lung-water 0.72 97.7 103.7 16.1 87.2 6MV p 2.5 10 8 water-bone-water 0.6 97.5 171.0 20.5 8.9 6MV p 2.5 10 8 VMAT Prostate patient 0.78 N/A N/A 39.6 N/A 6MV p 2.5 10 8 IMRT HN patient 0.57 N/A N/A 36.1 N/A Multi-GPU CPU: Intel Xeon processor with 2.27GHz GPU: NVIDIA Tesla C2050 Source type # of Histories Case T GPU (sec) T GPU (sec) T GPU/T GPU 6MV Photon 10 9 water-lung-water 312.82 78. 3.99 6MV Photon 10 9 water-bone-water 03.75 101.19 3.99 10 gdpm desired uncommissioned commissioned Automatic commissioning Tian et. al., submitted to Phys. Med. Biol. (201) Phase spacelet (PSL) Adjust PSL weights to match calculation with measurements 1.1 0.5 0.9 0. 0.3 0.7 1 0. 10cm 0. 10cm 0.5 1.5x1.5 0.1 0.1 5x5 0.3 20cm 20cm 0 (a1) (b1) 0 (a) -120-610 0 6 12 0 10-12 -6 0 6 12 0.02 5 6 desired 7 8 9 10 11 0.03 5 6 7 8 9 10 11 0.8 0.1 10 Off-axis distance (cm) Off-axis distance (cm) Off-axis uncommissioned distance (cm) Depth (cm) 0 desired0.01 Off-axis distance (cm) 0.0 8 commissioned uncommissioned -0.01 8-0.02 8 commissioned 0.02 1.5x1.5-0.0 (a2) 6 2.5cm -0.05 (b2) 6 2.5cm -0.06 0.02 6-0.07 0.01 0.020 0.02 0.01 0.01 0.6 0 (b) -0.01 0-0.02 0-0.01-0.01 0.06-0.02-0.02-0.02 (a3) 22 10cm -0.0 (b3) 2 10cm -0.05 0.01 (a1) 0.01 (b1) 15x15 0 00 0 0 0. 0-0.01 0.1-0.01-0.02 0.1-0.1-0.02 0 (c) (a) -0.3 20cm 0.0 (b) -0.1 20cm 0 5 10 15 20-0.5 25 30-0.0-5 (a2) 6 7 8 9 10 11 5 (b2) 6 7 8 9 10 11-0.3 Depth (cm) -0.7 Off-axis distance (cm) 0 5 Off-axis -10 distance -5 (cm) -15-10 -10-5 -5 0 5-15 0 5 z z position in in CT CT coordinate(cm) z position in CT coordinate(cm) 11 0 5 10 15 20 25 30 10x10 0.8 0. 15x15 0.6 0.3 a desired desired uncommissioned uncommissioned commissioned commissioned 2.5cm 10 15 20 25 30 Depth (cm) 0.5 1 b 1 0.8 0.6 10(Gy) 2.5cm desired uncommissioned commissioned gctd/mcdrr Realistic simulations of CBCT imaging process Facilitate CBCT related projects Evaluate patient-specific imaging dose (a) (a) (b) (b) Primary Scatter Noise CBCT Ray-tracing (c) MC (d) Time: Catphan phantom 1 min per projection simulation 20sec for dose calculations Jia, et. al., PMB, 57, 577 (2012) Jia, et. al., Med. Phys., 39, 7368 (2012) 12
gpmc In collaboration with MGH group Combines the physics from literatures Time: 6~22 sec of computation time In the process of clinical evaluations Jia, et. al., Phys. Med. Biol., 57, 7783 (2012) Jia et. al., AAPM 2013 Giantsoudi, et. al., AAPM 201 13 Other GPU packages GPUMCD ~2 min dose calc + optimization, one GPU/beam Hissoiny, et. al., Phys. Med. Biol., 56, 5119 (2011) Bol et. al., Phys. Med. Biol., 57, 1375 (2012) GMC: Geant implementation 39 sec on GTX 580 ARCHER RT 60~80 sec on Tesla M2090 Jahnke, et. al., Phys. Med. Biol., 57, 1217 (2012) Su, et. al., Med. Phys. 1, 071709 (201) 1 Outline GPU Monte Carlo Clinical Applications Conclusions 15 5
Treatment (re)-planning Pencil beam dose calculations using MC Adaptively sample particles based on optimized MU Acceleration X (in addition to acceleration of MC by GPU) TH-E-BRE-8, Li, et. al. Optimized fluence map with 1 10 6 / beamlet and 1 10 / beamlet/iteration Dash Ground truth: 1 10 6 / beamlet Solid - 5 10 3 / beamlet with 9 iteration 16 Pre-treatment A web-based service for treatment plan verification User upload a plan gdpm runs on a GPU server Plan dose is verified and a report is generated 17 In/post-treatment Real-time simulation of photon dose delivery using (simulated) online machine log 18 6
Hu Value 7/2/201 CBCT dose calculations Accurate evaluation of patient-specific dose in CT scans (a) (b) (c) (a) (b) z (c) (a) (b) z x (c) x x y y z (d) (e) (f) y (d) (e) (f) Montanari, et. al., PMB. 59, 1239 (201) 19 CBCT scatter Estimate scatter signals using MC 2 FDK reconstruction + 1 MC in 30 sec Xu, et. al., submitted to PMB. (201) 0-200 -00-600 -800-1000 -1200 0 200 00 600 Pixel Position of the profile 20 Outline GPU Monte Carlo Clinical Applications Conclusions 21 7
Conclusions GPU is a great tool for MC simulations Fast, or almost near real-time dose calculation is possible Enable new and novel applications 22 Acknowledgement Research team Xun Jia, Ph.D. Assistant Professor Collaborators MGH group Steve Jiang, Ph.D. Professor Zhen Tian, Ph.D. Instructor Feng Shi, Ph.D. Postdoctoral fellow Michael Folkerts Ph.D. student Yongbao Li Yuan Xu Ph.D. student Ph.D. student 23 8