Diffusion Imaging Visualization

Diffusion Imaging Visualization Thomas Schultz URL: http://cg.cs.uni-bonn.de/schultz/ E-Mail: schultz@cs.uni-bonn.de 1

Outline Introduction to Diffusion Imaging Basic techniques Glyph-based Visualization Streamline Visualization Direct Volume Rendering Current topics of research Uncertainty Visualization Comparative Visualization 2

Introduction to Diffusion MRI Goal: Investigate the microstructure of biological tissue using Magnetic Resonance Imaging (MRI) Challenge: Voxel size is far too large to resolve the structures of interest 2 mm 1 mm 1 mm Voxel size ø 0.1-20μm Axon (nerve fiber) size 4

Diffusion Propagator Diffusion propagator = Probability distribution of displacement vectors Computation based on huge number of MR images <1% of the images acquired for a single subject Images by [Garyfallidis et al. 2014] / own work 5

Models of Diffusion Diffusion Tensor Imaging (DTI) Gaussian approximation High Angular Resolution Diffusion Imaging (HARDI) Angular structure of propagator Diffusion Spectrum Imaging (DSI) Full propagator Zoo of related models Images by [Garyfallidis et al. 2014] / own work 6

Glyph Visualization A glyph is a geometric object whose shape, size, orientation, and color conveys the data Common for diffusion tensors: Ellipsoid Axes aligned with eigenvectors scaled with eigenvalues Implicit Equation: x T D 2 x 1 8

Superquadric Tensor Glyphs Ellipsoids suffer from visual ambiguities: Superquadric Glyphs greatly reduce them: Images taken from Kindlmann [2004] 9

The Idea Behind Superquadric Glyphs Ellipsoids are transformations of the sphere Superquadrics smoothly interpolate between sphere, cylinder, and box Ellipsoids Superquadrics 10

Standard HARDI Glyph Orientation Distribution Function (ODF) Standard ODF glyph = Polar Plot Each point on a sphere is scaled by the ODF value in the corresponding direction Visualizing diffusion tensors in this way leads to a peanut shape, not to the standard ellipsoid Polar Plot of Tensor Tensor Ellipsoid 11

HOME Glyph Schultz/Kindlmann [2010]: HOME Glyph Generalizes diffusion ellipsoid to ODFs Same extrema, but sharper maxima Polar Plot HOME Glyph 12

Polar Plot vs. HOME Glyph Direct comparison on real data: Polar Plot 13

Polar Plot vs. HOME Glyph Direct comparison on real data: HOME Glyph 14

Image taken from Kindlmann et al. [2006] Glyph Packing Packing glyphs densely reduces occlusion and enhances perception of continuous structures Grid-based Layout Glyph Packing 15

Efficient Rendering Interactive performance can be achieved using Implicit representation and GPU-based raycasting [Peeters et al. 2009] Explicit geometry with Viewport Culling + Levels of Detail [Schultz/Kindlmann 2010] 16

Deterministic Tractography Seed Points on Mid-Sagittal Plane 18

Image taken from Zhang et al. [2003] Streamtubes Stream tubes [Zhang et al. 2003] also encode second and third eigenvector Elliptical cross-section reflects second/third eval Fix maximum radius, preserve aspect ratio 19

Images taken from Wiens et al. [2014] Superquadric Streamtubes Superquadric streamtubes [Wiens et al. 2014] Superquadric instead of elliptical crosssection Shape index σ = λ 3 λ 2 γ Spherical for λ 2 = λ 3 Clear edges for λ 2 λ 3 20

ODF Streamtubes Glyphs along streamtubes [Prckovska et al. 2011] Multi-fiber hyperstreamlines and streamribbons [Vos et al. 2013] 21

Illustrative Rendering Techniques Depth-Dependent Halos [Everts et al. 2009] 22

Direct Volume Rendering Diffusion Tensors [Kindlmann et al. 2000] Diffusional Kurtosis [Bista et al. 2014] 24

Uncertainty Visualization Sources of uncertainty in diffusion MRI Measurement noise and artifacts Choice of parameters Selection of models Reasons to visualize uncertainty Avoid misinterpretation Increase reproducibility and trustworthiness Reduce uncertainty 26

Uncertainty from Measurement Noise Measurement 1 27

Uncertainty from Measurement Noise Measurement 2 28

Image taken from [Abbasloo et al. 2016] Visualizing Tensor Normal Distributions 29

[Abbasloo et al. 2015] Image taken from [Abbasloo et al. 2016] Visualizing Tensor Normal Distributions Confidence intervals at particular locations 30

Visualizing Local Measurement Uncertainty Cones of Uncertainty [Jones 2003] 95% confidence interval around main direction Limitation: Quite dissimilar distributions map to the same cone Images taken from [Jones 2003] / [Schultz et al. 2013] 31

Visualizing Local Measurement Uncertainty HiFiVE: Main direction + residual [Schultz et al. 2013] Hilbert-Space Fiber Variability Estimate Application to Uncertainty Reduction 32

Visualizing Local Measurement Uncertainty Uncertainty in HARDI Multi-Fiber HiFiVE [Wiens et al. 2014] ODF Ensembles [Jiao et al. 2012] 33

Visualizing Global Measurement Uncertainty PASTA ( Pointwise Assessment of Streamline Tractography Attributes ) [Jones et al. 2005] Superquadric Streamtubes [Wiens et al. 2014] 34

Visualizing Global Measurement Uncertainty Probabilistic Tractography produces a distribution of possible streamlines based on local probability distribution of directions Images taken from [Koch 2002] / [Jones 2010] 35

Visualizing Global Measurement Uncertainty Wrapped Streamlines [Enders et al. 2005] Topology-Based Fuzzy Fiber Geometry [Schultz et al. 2007] 36

Visualizing Global Measurement Uncertainty Illustrative Confidence Intervals [Brecheisen et al. 2013] 37

Visualizing Parameter Uncertainty Exploration Tool for Parameter Sensitivity [Brecheisen et al. 2009] 38

Comparative Visualization Tender glyphs compare tensor fields using reorientation and a checkerboard pattern [Zhang et al. 2016] 40

Conclusion and Outlook Diffusion imaging poses interesting challenges for visualization High information density Complementary and integrated tools required Open questions and ongoing work Advanced diffusion models Visualization of uncertainty Visualization of ensembles Comparative visualization 41

Further Reading In: C. Hansen et al. (Eds): Scientific Visualization: Uncertainty, Multifield, Biomedical, and Scalable Visualization, Springer, 2014 E-Mail schultz@cs.uni-bonn.de for a copy 42

References: Glyphs Kindlmann: Superquadric tensor glyphs, EG/IEEE Symposium on Visualization (SymVis), pages 147-154, 2004 Kindlmann and Westin: Diffusion Tensor Visualization with Glyph Packing, IEEE Transactions on Visualization and Computer Graphics 12(5):1329-1335, 2006 Peeters, Prčkovska, et al.: Fast and Sleek Glyph Rendering for Interactive HARDI Data Exploration, IEEE Pacific Visualization Symposium, pages 153-160, 2009 Schultz and Kindlmann: A Maximum Enhancing Higher-Order Tensor Glyph, Computer Graphics Forum 29(3):1143-1152, 2010 43

References: Streamlines Zhang, Demiralp, Laidlaw: Visualizing Diffusion Tensor MRI Images Using Streamtubes and Streamsurfaces IEEE Trans. Vis. Comp. Graphics 9(4):454-462, 2003 Schultz, Seidel: Estimating Crossing Fibers: A Tensor Decomposition Approach IEEE Trans. Vis. Comp. Graphics 14(6):1635-1642, 2008 Everts, Bekker et al.: Depth-dependent halos: Illustrative rendering of dense line data IEEE Trans. Vis. Comp. Graphics 15(6):1299-1306, 2009 Prčkovska, Peeters et al.: Fused DTI/HARDI Visualization IEEE Trans. Vis. Comp. Graphics 17(10):1407-19, 2011 Vos, Viergever, Leemans: Multi-Fiber Tractography Visualization for Diffusion MRI Data PLOS ONE 8(11):e81453, 2013 44

References: Volume Rendering Kindlmann, Weinstein, Hart: Strategies for Direct Volume Rendering of Diffusion Tensor Fields IEEE Trans. Vis. Comp. Graphics 6(2):124-138, 2000 Bista et al.: Visualization of Brain Microstructure through Spherical Harmonics Illumination of High Fidelity Spatio-Angular Fields IEEE Trans. Vis. Comp. Graphics 20(12):2516-2525, 2014 45

References: Uncertainty (I) Koch, Norris, et al.: An Investigation of Functional and Anatomical Connectivity Using Magnetic Resonance Imaging NeuroImage 16:241-250, 2002 Jones: Determining and Visualizing Uncertainty in Estimates of Fiber Orientation from Diffusion Tensor MRI Magnetic Resonance in Medicine 49(1):7-12, 2003 Jones et al.: PASTA: Pointwise assessment of streamline tractography attributes Magnetic Resonance in Medicine 53(6):1462-67, 2005 Enders et al.: Visualization of White Matter Tracts with Wrapped Streamlines Proc. IEEE Visualization, pp. 51-58, 2005 46

References: Uncertainty (II) Schultz, Theisel, Seidel: Topological Visualization of Brain Diffusion MRI Data IEEE Trans. Vis. Comp. Graphics 13(6):1496-1503, 2007 Brecheisen et al.: Parameter Sensitivity Visualization for DTI Fiber Tracking IEEE Trans. Vis. Comp. Graphics 15(6):1441-48, 2009 Jones: Challenges and limitations of quantifying brain connectivity in vivo with diffusion MRI, Future Medicine 2(3):341-355, 2010 Jiao et al.: Uncertainty Visualization in HARDI based on ensembles of ODFs Proc. IEEE PacificVis, pp. 193-200, 2012 47

References: Uncertainty (III) Brecheisen et al.: Illustrative Uncertainty Visualization of DTI fiber pathways The Visual Computer 29(4):297-309, 2013 Schultz et al.: HiFiVE: A Hilbert Space Embedding of Fiber Variability Estimates for Uncertainty Modeling and Visualization Computer Graphics Forum 32(3):121-130, 2013 Wiens et al.: Visualizing Uncertainty in HARDI Tractography Using Superquadric Streamtubes Proc. EuroVis Short Papers 2014 Abbasloo et al.: Visualizing Tensor Normal Distributions at Multiple Levels of Detail IEEE Trans. Vis. Comp. Graphics 22(1), 2016 48

Other References Garyfallidis et al.: Dipy, a library for the analysis of diffusion MRI data. Frontiers in Neuroinformatics 8, 2014 Zhang et al.: Glyph-based Comparative Visualization for Diffusion Tensor Fields IEEE Trans. Vis. Comp. Graphics 22(1), 2016 49