Direct Volume Rendering

Transcription

1 Direct Volume Rendering CMPT 467/767 Visualization Torsten Möller Weiskopf/Machiraju/Möller

2 Overview Volume rendering equation Compositing schemes Ray casting Acceleration techniques for ray casting Texture-based volume rendering Shear-warp factorization Splatting Pre-integration Cell projection (Shirley-Tuchman) Weiskopf/Machiraju/Möller 2

3 Readings The Visualization Handbook: Chapter 7 (Overview of Volume Rendering) Chapter 8 (Volume Rendering Using Splatting) Chapter 10 (Pre-Integrated Volume Rendering) Chapter 11 (Hardware-Accelerated Volume Rendering) Engel et al: Real-time Volume Graphics Chapter 1 (Theoretical Background and Basic Approaches) Chapter 3 (Basic GPU-Based Volume Rendering) Chapter 7 (GPU-Based Ray Casting) Chapter 9 (Improving Image Quality) Weiskopf/Machiraju/Möller 3

4 Overview 2D visualization slice images (or multi-planar reformating MPR) Indirect 3D visualization isosurfaces (or surface-shaded display SSD) Direct 3D visualization (direct volume rendering DVR) Weiskopf/Machiraju/Möller 4

5 Overview Directly get a 3D representation of the volume data The data is considered to represent a semi-transparent light-emitting medium Also gaseous phenomena can be simulated Approaches are based on the laws of physics (emission, absorption, scattering) The volume data is used as a whole (look inside, see all interior structures) Weiskopf/Machiraju/Möller 5

6 Overview Optical model Emission and absorption of lightd Participating media Volume rendering equation s 0 Volume integral Transparency Weiskopf/Machiraju/Möller 6

7 Overview Numerical approach to compute the volume rendering integral Riemann sum Sampling of light rays D Δx s 0 Weiskopf/Machiraju/Möller 7

8 Overview Backward methods Image space, image order algorithms Performed pixel-by-pixel Example: Ray casting Weiskopf/Machiraju/Möller 8

9 Overview Forward methods Object-space, object-order algorithm Cell projection Performed voxel by voxel Examples: Slicing, shear-warp, splatting Weiskopf/Machiraju/Möller 9

10 Volume Rendering Equation Goal: physical model for volume rendering Emission-absorption model Density-emitter model [Sabella 1988] Leads to volume rendering equation More general approach: Linear transport theory Equation of transfer for radiation Basis for all rendering methods Important aspects: Absorption Emission Scattering Participating medium Weiskopf/Machiraju/Möller 10

11 Volume Rendering Equation Assumptions: Based on a physical model for radiation Geometrical optics Neglect: Diffraction Interference Wave-character Polarization Interaction of light with matter at the macroscopic scale Describes the changes of specific intensity due to absorption, emission, and scattering Based on energy conservation Expressed by equation of transfer Weiskopf/Machiraju/Möller 11

12 Volume Rendering Equation Basic quantity of light: radiance I Sometimes called specific intensity n projected area element da θ dω radiative energy radiance position frequency direction solid angle time Weiskopf/Machiraju/Möller 12

13 Volume Rendering Equation Contributions to radiation at a single position: Absorption Emission Scattering Weiskopf/Machiraju/Möller 13

14 Volume Rendering Equation Absorption Total absorption coefficient or total extinction coefficient χ(x,n,ν) Loss of radiative energy through a cylindrical volume element: di dω ds 1/ χ is called mean free path dω di Weiskopf/Machiraju/Möller 14

15 Volume Rendering Equation Total absorption coefficient consists of: True absorption coefficient κ(x,n,ν) Scattering coefficient σ(x,n,ν) di dω ds dω di removal of radiative energy by true absorption (conversion to thermal energy) scattering out of solid angle dω Weiskopf/Machiraju/Möller 15

16 Volume Rendering Equation [Pharr, Humphreys, Physically Based Rendering, 2004] Effect of absorption Weiskopf/Machiraju/Möller 16

17 Volume Rendering Equation Emission Emission coefficient η(x,n,ν) Emission of radiative energy within a cylindrical volume element: Consists of two parts: Thermal part or source term q(x,n,ν) Scattering part j(x,n,ν) Weiskopf/Machiraju/Möller 17

18 Volume Rendering Equation [Pharr, Humphreys, Physically Based Rendering, 2004] Effect of emission Weiskopf/Machiraju/Möller 18

19 Volume Rendering Equation [Pharr, Humphreys, Physically Based Rendering, 2004] Effect of in-scattering Weiskopf/Machiraju/Möller 19

20 Volume Rendering Equation Integral form of the equation of transfer Integral equation because η contains I Interpretation: Radiation consists of Sum of photons emitted from all points along the line segment, Attenuated by the integrated absorptivity of the intervening medium, and Additional, attenuated contribution from radiation entering the boundary surface Weiskopf/Machiraju/Möller 20

21 Volume Rendering Equation Special case for most volume rendering approaches: Emission-absorption model Density-emitter model [Sabella 1988] Volume filled with light-emitting particles Particles described by density function Simplifications: No scattering Emission coefficient consists of source term only: η = q Absorption coefficient consists of true absorption only: χ = κ Weiskopf/Machiraju/Möller 21 No mixing between frequencies (no inelastic effects)

22 Equation of Transfer for Light Volume rendering equation with optical depth Weiskopf/Machiraju/Möller 22

23 Equation of Transfer for Light Discretization of volume rendering equation Discrete steps s k Often equidistant Weiskopf/Machiraju/Möller 23

24 Equation of Transfer for Light Discretization of volume rendering equation Define: Transparency part Emission part Discretized volume integral: Weiskopf/Machiraju/Möller 24

25 Equation of Transfer for Light Discretization of volume rendering equation Often use the following approximation Transparency part Emission part Approximation by piecewise constant functions Riemann sum Weiskopf/Machiraju/Möller 25

26 Compositing Compositing = iterative computation of discretized volume integral Traversal strategies Front-to-back Back-to-front Back-to-front compositing Directly derived from discretized integral Just different notation: Colors C and opacity α are assigned with transfer function Weiskopf/Machiraju/Möller 26

27 Compositing Back-to-front compositing (cont.) Over operator [Porter & Duff 1984] Used, e.g., in texture-based volume rendering Compositing equation: C out = (1 - α) C in + C C in C(i) in = C(i-1) out C(0) in C out C, α C i, α i Weiskopf/Machiraju/Möller 27 C(N) out

28 Compositing Front-to-back compositing Reverse the order of summation From obtain With accumulated transparency T n Weiskopf/Machiraju/Möller 28

29 Compositing Front-to-back compositing (cont.) Needs to maintain α in Most often used in ray casting Compositing equation: C out, α out C out = C in +(1 - α in ) C α out = α in +(1 - α in ) α C in, α in C, α Weiskopf/Machiraju/Möller 29

30 Compositing Associated colors Color contributions are already weighted by their corresponding opacity Also called pre-multiplied colors Non-associated colors: C Cα Just substitute in compositing equations Yields the same results as associated colors (on a cont. level) Differences occur when combined with interpolation + postclassification Ex.: back-to-front compositing with non-associated colors: C out = (1 - α) C in + Cα Standard OpenGL blending for semi-transparent surfaces Weiskopf/Machiraju/Möller 30

31 Compositing So far: accumulation scheme Variations of composition schemes First Average Maximum intensity projection Weiskopf/Machiraju/Möller 31

32 Compositing Intensity Max Accumulate Average First Depth Weiskopf/Machiraju/Möller 32

33 Compositing Compositing: First Extracts isosurfaces Intensity First Depth Weiskopf/Machiraju/Möller 33

34 Compositing Compositing: Average Produces basically an X-ray picture Intensity Average Depth Weiskopf/Machiraju/Möller 34

35 Compositing Maximum Intensity Projection (MIP) Often used for MR or CT angiograms Good to extract vessel structures Intensity Max Weiskopf/Machiraju/Möller 35

36 Compositing Compositing: Accumulate Emission-absorption model Make transparent layers visible (see classif.) Intensity Accumulate Weiskopf/Machiraju/Möller 36 Depth

37 Compositing Note: First and average are special cases of accumulate Weiskopf/Machiraju/Möller 37

38 Ray Casting Similar to ray tracing in surface-based computer graphics In volume rendering we only deal with primary rays; hence: ray casting Natural image-order technique As opposed to surface graphics - how do we calculate the ray/surface intersection? Weiskopf/Machiraju/Möller 38

39 Ray Casting Since we have no surfaces - need to carefully step through volume A ray is cast into the volume, sampling the volume at certain intervals Sampling intervals are usually equidistant, but don t have to be (e.g. importance sampling) At each sampling location, a sample is interpolated / reconstructed from the voxel grid Popular filters are: nearest neighbor (box), trilinear, or more sophisticated (Gaussian, cubic spline) First: Ray casting in uniform grids Implicit topology Simple interpolation schemes Weiskopf/Machiraju/Möller 39

40 Ray Casting Volumetric ray integration: Tracing of rays Accumulation of color and opacity along ray: compositing color 1.0 Weiskopf/Machiraju/Möller 40 object (color, opacity)

41 Ray Casting interpolation kernel volumetric compositing color opacity 1.0 object (color, opacity) Weiskopf/Machiraju/Möller 41

42 Ray Casting interpolation kernel volumetric compositing color c = α s c s (1 - α) + c opacity α = α s (1 - α) + α 1.0 object (color, opacity) Weiskopf/Machiraju/Möller 42

43 Ray Casting volumetric compositing color opacity 1.0 object (color, opacity) Weiskopf/Machiraju/Möller 43

44 Ray Casting volumetric compositing color opacity 1.0 object (color, opacity) Weiskopf/Machiraju/Möller 44

45 Ray Casting volumetric compositing color opacity 1.0 object (color, opacity) Weiskopf/Machiraju/Möller 45

46 Ray Casting volumetric compositing color opacity 1.0 object (color, opacity) Weiskopf/Machiraju/Möller 46

47 Ray Casting volumetric compositing color opacity object (color, opacity) Weiskopf/Machiraju/Möller 47

48 Ray Casting How is color and opacity at each integration step determined? Opacity and (emissive) color in each cell according to classification Additional color due to external lighting: according to volumetric shading (e.g. Blinn-Phong) No shadowing, no secondary effects Implementations Traditional CPU implementation straightforward, very efficient GPU implemenations Fragment shader loops (Shader Model 3 GPUs) Weiskopf/Machiraju/Möller 48

49 Ray Casting Ray casting in tetrahedral grids Linear interpolation within cells Slightly modify the traversal through the grid, compared to uniform grids Algorithm by M. P. Garrity [ Raytracing irregular volume data, VolVis, p , 1990] GPU implementations available Weiskopf/Machiraju/Möller 49

50 Ray Casting Ray casting in tetrahedral grids Traverse rays front-to-back Stop at intersected cell faces Comp. color + opacity for current ray segment Accumulate volume colors and opacities Pixels in view plane Weiskopf/Machiraju/Möller 50 Tetrahedral mesh

51 Acceleration Techniques for Ray Casting Problem: ray casting is time consuming Idea: Neglect irrelevant information to accelerate the rendering process Exploit coherence Early-ray termination Idea: colors from faraway regions do not contribute if accumulated opacity is to high Stop traversal if contribution of sample becomes irrelevant User-set opacity level for termination Front-to-back compositing Weiskopf/Machiraju/Möller 51

52 Acceleration Techniques for Ray Casting Effect of early-ray termination Example image # Sample points (semi-transparent) # Sample points (opaque) Weiskopf/Machiraju/Möller 52

53 Acceleration Techniques for Space leaping Ray Casting Skip empty cells Homogeneity-acceleration Approximate homogeneous regions with fewer sample points Weiskopf/Machiraju/Möller 53

54 Acceleration Techniques for Ray Casting Hierarchical spatial data structure Octree Mean value and variance stored in nodes of octree Weiskopf/Machiraju/Möller 54

55 Acceleration Techniques for Ray Casting Adaptive ray traversal Different velocities for traversal Different distance between samples Based on vicinity flag Layer of vicinity voxels around non-transparent parts of the volume Weiskopf/Machiraju/Möller 55

56 Acceleration Techniques for Ray Casting Modern GPUs can be used for ray casting Essential idea Fragment shader loop Implements ray marching Benefits from High processing speed of GPUs Built-in trilinear interpolation in 3D textures Requires Pixel Shader 3.0 compliant GPUs E.g. GeForce 6xxx or higher Weiskopf/Machiraju/Möller 56

57 Texture-Based Volume Rendering Object-space approach Based on graphics hardware: Rasterization Texturing Blending Proxy geometry because there are no volumetric primitives in graphics hardware Slices through the volume Supported by older graphics hardware No need for (advanced) fragment shaders Weiskopf/Machiraju/Möller 57

58 Texture-Based Volume Rendering Slice-based rendering color opacity object (color, opacity) Similar to ray casting with simultaneous rays Weiskopf/Machiraju/Möller 58

59 Texture-Based Volume Rendering scene description geometry processing rasterization fragment operations vertices primitives fragments pixels rendering pipeline Weiskopf/Machiraju/Möller 59

60 Texture-Based Volume Rendering Proxy geometry Stack of texture-mapped slices Generate fragments Most often back-to-front traversal Weiskopf/Machiraju/Möller 60

61 Texture-Based Volume Rendering 2D textured slices Object-aligned slices Three stacks of 2D textures Bilinear interpolation Weiskopf/Machiraju/Möller 61

62 Texture-Based Volume Stack of 2D textures: Rendering Artifacts when stack is viewed close to 45 degrees Locations of sampling points may change abruptly Weiskopf/Machiraju/Möller 62

63 Texture-Based Volume Rendering 3D textured slices View-aligned slices Single 3D texture Trilinear interpolation Weiskopf/Machiraju/Möller 63

64 3D texture: Texture-Based Volume Rendering Needs support for 3D textures Data set stored only once (not 3 stacks!) Trilinear interpolation within volume Slower Good image quality Constant Euclidean distance between slices along a light ray Constant sampling distance (except for perspective projection) Weiskopf/Machiraju/Möller 64

65 Texture-Based Volume Rendering 3D texture: No artifacts due to inappropriate viewing angles Increase sampling rate more slices Easy with 3D textures Weiskopf/Machiraju/Möller 65

66 Texture-Based Volume Rendering 2D textures axis-aligned texturing (bilinear interpolation) compositing (blending) 3D texture view-aligned texturing (trilinear interpolation) Weiskopf/Machiraju/Möller compositing 66 (blending)

67 Texture-Based Volume Rendering Representation of volume data by textures Stack of 2D textures 3D texture Typical choices for texture format: Luminance and alpha Pre-classified (pre-shaded) gray-scale volume rendering Transfer function is already applied to scalar data Change of transfer func. requires complete redefinition of texture data RGBA Pre-classified (pre-shaded) colored volume rendering Transfer function is already applied to scalar data Luminance Only the actual scalar data is stored Best solution! Weiskopf/Machiraju/Möller 67

68 Texture-Based Volume Post-classification? Rendering Data set represented by luminance texture (single channel) Dependent texture lookup in texture for color table Fragment or pixel shader program Weiskopf/Machiraju/Möller 68

69 Texture-Based Volume Rendering Compositing: Works on fragments Per-fragment operations After rasterization Blending of fragments via over operator OpenGL code for over operator glenable (GL_BLEND); glblendfunc (GL_ONE, GL_ONE_MINUS_SRC_ALPHA); Generate fragments: Render proxy geometry Slice Simple implementation: quadrilateral More sophisticated: triangulated intersection surface between slice plane Weiskopf/Machiraju/Möller and boundary of the volume data 69 set

70 Texture-Based Volume Rendering Outlook on more advanced texture-based volume rendering techniques: Exploit quite flexible per-fragment operations on modern GPUs (NVIDIA GeForce 6xxx or ATI Radeon X8xx) Pre-integration for very high-quality rendering (see later) Decompression of compressed volumetric data sets on GPU to save texture memory Weiskopf/Machiraju/Möller 70

71 Texture-Based Volume Rendering Advantages of texture-based rendering: Supported by consumer graphics hardware Fast for moderately sized data sets Interactive explorations Surface-based and volumetric representations can easily be combined mixture with opaque geometries Disadvantages: Limited by texture memory Solution: bricking at the cost of additional texture downloads to the graphics board Brute force: complete volume is represented by slices Rasterization speed + memory access can be problematic Weiskopf/Machiraju/Möller 71

72 Shear-Warp Factorization Object-space method Slice-based technique Fast object-order rendering Accelerated volume visualization via shear-warp factorization [Lacroute & Levoy 1994] CPU-based implementation Weiskopf/Machiraju/Möller 72

73 Shear-Warp Factorization General goal: make viewing rays parallel to each other and perpendicular to the image This is achieved by a simple shear shear warp Parallel projection (orthographic camera) is assumed Weiskopf/Machiraju/Möller 73

74 Shear-Warp Factorization Algorithm: Shear along the volume slices Projection + comp. to get intermediate image Warping transformation of intermediate image to get correct result shear view rays slices projection view plane transformation: warp Weiskopf/Machiraju/Möller 74

75 Shear-Warp Factorization Shear for parallel and perspective proj. shear perpendicular to z-axis shear and scale shear & scale perspective projection warp Weiskopf/Machiraju/Möller 75

76 Splatting Splatting [Westover 1990] Object-order method Project each sample (voxel) from the volume into the image plane splat image Weiskopf/Machiraju/Möller 76

77 Splatting Draw each voxel as a cloud of points (footprint) that spreads the voxel contribution across multiple pixels Footprint: splatted (integrated) kernel Approximate the 3D kernel h(x,y,z) extent by a sphere Weiskopf/Machiraju/Möller 77

78 Splatting Larger footprint increases blurring and used for high pixel-to-voxel ratio Footprint geometry Orthographic projection: footprint is independent of the view point Perspective projection: footprint is elliptical Pre-integration of footprint For perspective projection: additional Weiskopf/Machiraju/Möller 78 computation of the orientation of the ellipse

79 Splatting Volume = field of 3D interpolation kernels One kernel at each grid voxel Each kernel leaves a 2D footprint on screen Weighted footprints accumulate into image voxel kernels screen footprints = splats screen Weiskopf/Machiraju/Möller 79

80 Splatting Volume = field of 3D interpolation kernels One kernel at each grid voxel Each kernel leaves a 2D footprint on screen Weighted footprints accumulate into image voxel kernels screen footprints = splats screen Weiskopf/Machiraju/Möller 80

81 Splatting Volume = field of 3D interpolation kernels One kernel at each grid voxel Each kernel leaves a 2D footprint on screen Weighted footprints accumulate into image voxel kernels screen footprints = splats screen Weiskopf/Machiraju/Möller 81

82 Splatting Simple extension to volume data without grids Scattered data with kernels Example: SPH (smooth particle hydrodynamics) Needs sorting of sample points Weiskopf/Machiraju/Möller 82

83 Pre-integrated Rendering Slice-by-slice Slab-by-slab Projection s f s b Pre-integration of all combinations s b Volume integral in dependent texture Texturing Front Back s b s f s b Weiskopf/Machiraju/Möller s f 83 sf

84 Pre-integrated Rendering Assumptions: Linear interp. of scalar values within a slab Constant length of a slab: L Only an approximation, but gives good results in most cases Pre-computation of all potential contrib. from a slab apply TF (linear interpolation within a slab) RGB pre-integrated RGBA values Weiskopf/Machiraju/Möller 84

85 Pre-integrated Rendering Quality comparison 128 Slices 284 Slices 128 Slabs Weiskopf/Machiraju/Möller 85

86 Pre-integrated Rendering Quality comparison 128 Slices 284 Slices 128 Slabs Weiskopf/Machiraju/Möller 86

87 Pre-integrated Rendering Idea of pre-integrated volume rendering can be adopted for Ray casting Shear-warp volume rendering Weiskopf/Machiraju/Möller 87

88 Cell Projection For unstructured grids Alternative to ray casting (Garrity s alg.) Projected Tetrahedra (PT) algorithm [P. Shirley, A. Tuchman: A polygonal approximation to direct scalar volume rendering, Volvis 1990, p ] Weiskopf/Machiraju/Möller 88

89 Cell Projection Basic idea Final Image Blending Weiskopf/Machiraju/Möller 89

90 Cell Projection Spatial sorting for all tetrahedra in a grid Back-to-front or front-to-back strategies possible Compositing is not commutative MPVO algorithm: Meshed Polyhedra Visibility Ordering [P. Williams: Visibility Ordering Meshed Polyhedra, ACM Transactions on Graphics, 11(2), 1992, p ] Only for acyclic, convex grids convex Weiskopf/Machiraju/Möller 90 non-convex cyclic

91 Cell Projection Decomposition of non-tetrahedral unstructured grids into tetrahedra PT can be applied for all types of unstructured grids Weiskopf/Machiraju/Möller 91

92 Cell Projection Alternative to working directly on unstructured grids Resampling approaches, adaptive mesh refinement (AMR) Weiskopf/Machiraju/Möller 92