Computer Graphics. - Volume Rendering - Philipp Slusallek

Transcription

1 Computer Graphics - Volume Rendering - Philipp Slusallek

2 Overview Motivation Volume Representation Indirect Volume Rendering Volume Classification Direct Volume Rendering

3 Applications: Bioinformatics Image by [Chimera 08]

4 Image by [Salama 07] Applications: Entertainment

5 Applications: Industrial

6 Applications: Medical

7 Applications: Simulations Image by [RTVG 08]

8 Volume Processing Pipeline Acquisition Measure or computation the data Filtering Picking desired features, cleaning, noise-reduction, re-sampling, reconstruction, classification,... Mapping Map N-dimensional data to visual primitives Rendering Generate the image Post-processing Enhancements (gamma correction, tone mapping)

9 Volume Acquisition Measuring Computer Tomography (CT, X-Ray), Magnetic Resonance Imaging (MRI, e-spin) Positron-Emission Tomography (PET) Ultrasound, sonar Electron microscopy Confocal microscopy Cryo-EM/Light-Tomography Simulations Essentially everything > 2D Visualization of mathematical objects

10 Filtering Raw data usually unsuitable Selection of relevant aspects Cleaning & repairing Correcting incomplete, out-of-scale values Noise reduction and removal Classification Adaptation of format Re-sampling (often to Cartesian grids) Transformations Volume reconstructing of 3D data from projection

11 Mapping Create something visible Interpretation of measurement values Mapping to geometric primitives Mapping to parameters (colors, absorption coefficients,...) Rendering Surface extraction vs. direct volume rendering Single volume vs multiple (possibly overlapping) Object-based vs. image-based rendering Forward- or backward mappings (rasterization/rt)

12 Volume Rendering Our input? Representation of volume Our output? Colors for given samples (pixels) Our tasks? Map weird values to optical properties Project 1D data values within 3D context to 2D image plane

13 VOLUME ACQUISITION AND REPRESENTATION

14 Data Acquisition Simulated Data Fluid dynamics Heat transfer etc Generally Scientific Visualization Measured Data CT (Computed Tomography) scanner Reconstructed from rotated series of two-dimensional X-ray images Good contrast between high and low density media (e.g. fat and bones) MRI (Magnetic Resonance Imaging) Based on magnetic/spin response of hydrogen atoms in water Better contrast between different soft tissues (e.g. brain, muscles, heart) PET (Positron Emission Tomography) And many others (also here on campus, e.g. material science)

15 Data Acquisition CT vs. MRI

16 Volume Representations Definition 3D field of values: Essentially a 3D scalar or color texture Sometimes higher dimensional data (e.g. vector/tensor fields) Sampled representation 3D lattice of sample points (akin to an image but in 3D) Typically equal-distance in each directions Generally point cloud in space Point neighborhood information (topology) Data values at the points Procedural Mathematical description of values in space Sum of Gaussians (e.g. in quantum mechanics) Perlin noise (e.g. for non-homogeneous fog) Always convertible to sampled representation But with loss of information

17 Volume Organization Rectilinear Grids Common for scanned data May have different spacings Curvilinear Grids Warped rectilinear grids Unstructured Meshes Common for simulated data E.g. tetrahedral meshes Point clouds No topological/connection information Neighborhood computed on the fly

18 Reconstruction Filter Nearest Neighbor Cell-centered sample values Tri-Linear Interpolation Node-centered sample values

19 Tri-Linear Interpolation Compute Coefficients wx = (x x0) / (x1 x0) wy = (y y0) / (y1 y0) wz = (z z0) / (z1 z0) 3-D Scalar Field per Voxel f(x, y, z) = (1 - wz) (1 - wy) (1 - wx) c000 + (1 - wz) (1 - wy) wx c100 + (1 - wz) wy (1 - wx) c010 + (1 - wz) wy wx c110 + wz (1 - wy) (1 - wx) c001 + wz (1 - wy) wx c101 + wz wy (1 - wx) c011 + wz wy wx c111 z y x

20 Tri-Linear Interpolation Successive Linear Interpolations Along X c00 = (1 - wx) c000 + wx c100 c01 = (1 - wx) c001 + wx c101 c10 = (1 - wx) c010 + wx c110 c11 = (1 - wx) c011 + wx c111 Along Y c0 = (1 - wy) c00 + wy c10 c1 = (1 - wy) c01 + wy c11 Along Z c = (1 - wz) c0 + wz c1 z y Order of dimensions does not matter x

21 VOLUME MAPPING

22 Mapping / Classification Definition Map scalar data values to optical properties E.g. Optical density Albedo Emission Instances Analytical function Discrete representation Array of sample colors corresponding to sample data values Interpolate colors for data values in between sample points

23 Mapping / Classification Physical Mapping Physically-based mapping via optical properties of material Concentration of soot to optical density, albedo, etc Temperature to emitted blackbody radiation Allows for realistic rendering, often intuitively interpretable by us

24 Mapping / Classification Empirical or task-specific mapping (Transfer Function) User-defined mapping from data to colors Typically stored as an array sample correspondences (color map transfer function) Mapping may have no physical interpretation Assigning color to pressure, electrostatic potential, electron density, Highlight specific features of the data Isolate bones from fat

25 Pre/Post-Classification Pre-Classification First classify data values in sample cells Then interpolate classified optical properties Post-Classification First interpolate data values, then classify interpolated values

26 Cinematic Rendering Nominated for Deutsche Zukunftspreis 2017 Klaus Engel & Robert Schneider, Siemens Healthineers

27 DIRECT VOLUME RENDERING

28 Direct Volume Rendering Definition Directly render the volumetric data (only) as translucent material

29 Scattering in a Volume

30 Beer s Law Volumetric Attenuation Assume constant optical density κ 01 Transmittance: T x 0, x 1 = e κ 01(x 1 x 0 ) Transmitted radiance: L o x 0, ω = T x 0, x 1 L o x 1, ω x1 x0

31 Analytical Form Volumetric Attenuation Assume constant optical density κ 01 (extinction coefficient) Transmittance: T x 0, x 1 = e κ 01(x 1 x 0 ) Transmitted radiance: T x 0, x 1 L o x 1, ω Volumetric Contributions Also assume (constant) volume radiance L v x, ω [Watt/(sr m^3)] Contributed radiance: 1 T x 0, x 1 L v x 01, ω Volumetric Equation Radiance reaching the observer Emission within segment + transmitted background radiance L o x 0, ω = 1 T x 0, x 1 L v x 01, ω + T x 0, x 1 L o x 1, ω

32 Ambient Homogenous Fog Constant-Optical Density Volumetric Contributions Assume constant volumetric albedo ρ v x Assume constant ambient lighting L a (everywhere, no shadowing) Leads to constant volume radiance L v x, ω = L a ρ v Pervasive Fog Entry at camera, exit at intersection, or inf. x Algorithm 0 Compute surface illumination L o x 1, ω Modulate shadow visibility by transmittance between surface and light source Compute volume transmittance T x 0, x 1 and attenuate surface radiance Add contributions from volume radiance x 1

33 Ambient Homogeneous Fog Pros Simple Efficient Cons No true light contributions No volumetric shadows

34 Ray-Marching Riemann Summation Non-constant optical density / non-constant volume radiance Sample volume at discrete locations Assume constant density and volume radiance in each interval

35 Ray-Marching Homogeneous Segments L o x 0, ω = 1 e κ 01Δx L v x 01, ω + e κ 01Δx L o x 1, ω L o x 1, ω = 1 e κ 12Δx L v x 12, ω + e κ 12Δx L o x 2, ω L o x 2, ω = x0 x1 Recursive Substitution x2 x3 L o x 0, ω = 1 e κ 01Δx L v x 01, ω + e κ 01Δx 1 e κ 12Δx L v x 12, ω + e κ 12Δx = 1 e κ 01Δx L v x 01, ω + e κ 01Δx 1 e κ 12Δx L v x 12, ω + e κ 01Δx e κ 12Δx = n 1 i 1 e κ j,j+1δx 1 e κ i,i+1δx L v x i,i+1, ω + n 1 e κ j,j+1δx L o x n, ω i=0 j=0 j=0

36 Ray-Marching (front to back) L = 0; T = 1; t = 0; // t_enter; while(t < t_exit) dt = min(t_step, t_exit - t); P = ray.origin + (t + dt/2) * ray.direction; b = exp(- volume.density(p) * dt); L += T * (1 - b) * Lv(P); T *= b; // Optional early termination t += t_step; L += T * trace(ray.origin + t_exit * ray.direction, ray.direction); return L;

37 Homogeneous Fog Constant-optical density Non-constant volume radiance Similar to surface reflected radiance (i.e. rendering equation) Use phase function ρ x, Δω, (e.g. ρ v ) instead of BRDF*cosine 4 π Modulate shadow visibility by transmittance

38 Homogeneous Fog E.g. Anisotropic Point Light Modulate visibility at surfaces by transmittance L rl x, ω o = I( ω) x y 2 V x, y T(x, y)f r ω x, y, x, ω o cos θ i Modulate visibility at each volume sample by transmittance L v x, ω o = I( ω) x y 2 V x, y T(x, y) ρ v 4 π

39 Homogeneous Fog Inverse Square Law Volumetric Shadows Projective Light

40 Heterogeneous Fog Assumptions Non-constant-optical density Non-constant volume radiance Shadow visibility modulated by transmittance Ray-marched shadow rays at surface Ray-marched shadow rays at each volume sample!! T x 0, x n = n 1 j=0 e κ j,j+1δx

41 Heterogeneous Fog

42 Ray-Casting Early Ray Termination Abort ray-marching when subsequent contributions are negligible if (T < epsilon) return L; Very effective in dense volumes Also avoids ray-marching to infinity Grid Traversal 3-D DDA Ray-marching Adaptive Marching Bulk integration over homogeneous regions (e.g. octree, bricks) Pre-compute and store maximum step size separately Increasing step size with decreasing accumulated transmittance Vertex Connection and Merging & Joint Path Sampling [Siggraph 14]

43 Full Volumetric Light Simulation Taking into account multiple scattering in the volume

44 Full Volumetric Light Simulation Including Shadows, Caustics, etc.