Multiscale Techniques: Wavelet Applications in Volume Rendering

Transcription

1 Multiscale Techniques: Wavelet Applications in Volume Rendering Michael H. F. Wilkinson, Michel A. Westenberg and Jos B.T.M. Roerdink Institute for Mathematics and University of Groningen The Netherlands November 2002 FANTOM workshop

2 Volume visualization Large three-dimensional (3D) data sets arise from measurement by physical equipment, or from computer simulation. Scientific areas: computerized tomography (CT), astronomy, computational physics or chemistry, biology, fluid dynamics, seismology, environmental research, non-destructive testing, etc. For easy interpretation volume visualization techniques allow viewing the data from different view points. FANTOM workshop 1

3 Overview X-ray volume rendering Client/server visualization system Fourier-wavelet volume rendering Wavelet splatting Discussion FANTOM workshop 2

4 X-ray volume rendering Integrate the density along the line of sight. Applicable in medical applications, since physians are trained in interpreting X-ray like images. Mathematical concept is the X-ray transform. FANTOM workshop 3

5 X-ray volume rendering example Computerized tomography (CT) dataset FANTOM workshop 4

6 Client/server visualization Exchange of volume data through Internet requires fast and efficient methods of transfer and display. To relieve the demand on the server capacity, volume data may be stored on a central server. Rendering is (partly) performed on client systems. Not all of these clients have a high-bandwidth network connection, so we need to visualize data incrementally as it arrives. FANTOM workshop 5

7 Requirements Compression/simplification: visualize reduced version of data in controllable way. Progressive refinement: incremental visualization from low to high resolution. Progressive transmission: transmit data incrementally from server to client s workstation (data transfer is time-limiting factor) Wavelets meet these requirements FANTOM workshop 6

8 Mathematics tics Fourier volume rendering (Malzbender 1993) Fourier projection slice theorem allows efficient implementation of X-ray volume rendering z X-ray transform θ v u y P θ f(u, v) = R f(uu + vv + tθ) dt P (u,v) θ x projection slice theorem F 2 P θ f(ω u, ω v ) = F 3 f(ω u u + ω v v) FANTOM workshop 7

9 Fourier rendering algorithm Preprocessing. Compute the 3-D FFT of the volume data Rendering. For each direction θ do: 1. Interpolate the 3-D volume and resample on the slice plane. 2. Compute inverse 2-D FFT of the slice plane. technicalities: interpolation spatial zero-padding (20%) to reduce aliasing cubic spline interpolation real-to-complex/complex-to-real FFT to save time and space FANTOM workshop 8

10 Why wavelets? Wavelets allow systematic decomposition of data into versions at different levels of resolution. Wavelets allow incremental visualization of the data from low to high resolution. Wavelets perform well for compression tasks. FANTOM workshop 9

11 Wavelet transform Wavelet decomposition obtained by convolving signal with analysis filter, followed by downsampling results in approximation coefficients giving coarse features and detail coefficients giving finer structure Wavelet reconstruction obtained by upsampling approximation and detail coefficients, followed by convolution with a synthesis filter Haar basis Linear B-spline basis FANTOM workshop 10

12 2-D wavelet decomposition example CT image 2-level Haar wavelet decomposition FANTOM workshop 11

13 Wavelet X-ray transform Expansion of the X-ray transform on a 2-D wavelet basis P θ f(u, v) = k,l c M k,l(θ)φ 0 M,k,l(u, v) + M j=1 τ T k,l d j,τ k,l (θ)ψτ j,k,l(u, v) Computation of the approximation and detail coefficients c M k,l(θ) = F 1 2 d j,τ 1 k,l (θ) = F2 ( F 2 P θ f F 0 ) 2 Φ M (2 M k, 2 M l) ( F 2 P θ f F 2 Ψ τ ) j (2 j k, 2 j l) Φ 0 M(u, v) = Φ 0 M,0,0 ( u, v) Ψ τ j (u, v) = Ψ τ j,0,0 ( u, v) FANTOM workshop 12

14 Pyramid algorithm in Fourier domain upsampling and downsampling X down = 1 4 (X a + X b + X c + X d ) when X = ( ) X up X X = X X ( ) Xa X b X c X d decomposition (FWD) C j+1 = [ H j C j ] down D j+1,τ = [ G j,τ C j ] down reconstruction (FWR) C j = H j [C j+1 ] up + 3 G j,τ [D j+1,τ ] up τ=1 FANTOM workshop 13

15 Fourier-wavelet volume rendering algorithm Preprocessing. Compute the 3-D FFT of the volume data Rendering. For each direction θ do: 1. Interpolate the 3-D volume and resample on the slice plane, yielding the array C Perform a 2-D FWD of depth M, resulting in approximation coefficients Ck,l M and detail coefficients D j,τ k,l. 3. Perform a partial FWR from Ck,l M, followed by an inverse 2-D FFT, yielding an approximation in the spatial domain. 4. Refine approximation by partial FWR using the details D j,τ k,l, followed by a 2-D inverse FFT to obtain an approximation at a finer scale. FANTOM workshop 14

16 Mathematics tics Progressive refinement (1) Define operators H j C j+1 = H j [C j+1 ] up G j,τ D j+1,τ = G j,τ [D j+1,τ ] up and rewrite wavelet reconstruction into C j = H j C j G j,τ D j+1,τ τ=1 By iterating the above, the full reconstruction C 0 can be written as M C 0 = Ĉ0,M + D 0,M j j=0 Ĉ 0,M = H 0 H 1... H M 1 C M D 0,M j = H 0 H 1... H M j 2 3 G M j 1,τ D M j,τ τ=1 FANTOM workshop 15

17 Mathematics tics Progressive refinement (2) a) upsampling C 3 C 3 C 3 x = point-wise multiplication x C 2 H 2 b) upsampling C 2 C 2 C 2 replace C 2 C 3 C 3 C 3 C 3 C 3 C 3 C 3 C 3 x x x H 1 H 2 H 2 x x combined filter C 1 H 1 FANTOM workshop 16

18 Example renderings Haar wavelet Linear B-spline wavelet FANTOM workshop 17

19 Movie Fourier-wavelet renderings of CT head. FANTOM workshop 18

20 Mathematics tics Splatting (Westover 1990) Voxels are represented by 3-D reconstruction kernels. Kernels are integrated along the line of sight, resulting in footprints. Image is formed by mapping the footprints weighted by the voxel values to the view plane. FANTOM workshop 19

21 Splatting graphically FANTOM workshop 20

22 Mathematics tics Wavelet splatting (1) (Lippert Gross 1995) Modification of standard splatting Uses wavelets as reconstruction filters Provides a mechanism to visualize at different levels of detail Suitable for client/server system FANTOM workshop 21

23 Mathematics tics Wavelet splatting (2) Expand f on a 3-D wavelet basis and plug into X-ray transform P θ f(u, v) = R = + k,l,m M f(uu + vv + tθ) dt c M k,l,m j=1 τ T k,l,m R Φ 0 M,k,l,m(uu + vv + tθ) dt d j,τ k,l,m Ψ τ j,k,l,m(uu + vv + tθ) dt R Compute footprints by Fourier slicing FANTOM workshop 22

24 Wavelet splatting rendering algorithm 1. Preprocessing Perform a 3-D wavelet transform (depth M) of the volume data. Construct sequence of coefficients for each wavelet type and decomposition depth. Discard coefficients equal to zero. 2. Rendering Select viewing direction θ, and compute footprints. While a user interacts with the data, compute low resolution images from the approximation coefficients only. When interaction stops, refine the image incrementally with the detail coefficients. FANTOM workshop 23

25 Example renderings Haar wavelet Linear B-spline wavelet FANTOM workshop 24

26 Movies Wavelet splatting of CT volume with Haar wavelet Wavelet splatting of confocal microscopy data with linear B-spline wavelet FANTOM workshop 25

27 Differences Fourier-wavelet rendering Wavelet splatting wavelet coefficients depend on θ independent of θ time complexity O(N 2 log N) O(N 3 ) run-time depends on size of input only also depends on wavelet FANTOM workshop 26

28 Comparison Cumulative rendering times measured on an SGI Onyx (200 MHz CPU). CT head MR head Fourier-wavelet rendering slice extraction FWD level 3 approximation 7.40 level 2 approximation level 1 approximation full reconstruction Wavelet splatting level 3 approximation 3.49 level 2 approximation level 1 approximation full reconstruction FANTOM workshop 27