Using image data warping for adaptive compression

Transcription

1 Using image data warping for adaptive compression Stefan Daschek, Abstract For years, the amount of data involved in rendering and visualization has been increasing steadily. Therefore it is often desirable and sometimes necessary to use some sort of compression. In this paper I give a short overview of the technique of data warping. In simple terms, data warping means expanding important regions of the data while contracting unimportant regions. With data warping it is possible to reduce data size significantly without noticeably losing visual quality. I give examples for warping two-dimensional texture images and threedimensional voxel datasets. Keywords: data warping, texture mapping, volume rendering, texture compression, volume warping 1 Introduction As the size of data to be rendered increases, the need for efficient compression arises. Data warping addresses this problem. The main idea is to automatically segment the data into regions of different importance. 1 There is of course an interrelationship between importance and level of detail: Naturally, high importance corresponds with regions containing many details, while low importance corresponds with areas containing fewer details. Algorithms to automatically classify data according to its level of detail are explained in Section 2. After classifying the data a warping function is applied. The warping function is based on the importance map in such a way that areas of very low importance (i.e. low level of detail) are shrunk aggresively while important (i.e. high level of detail) areas are just downsampled slightly or even not at all. This form of adaptive compression yields remarkable good results. When rendering the warped and downsampled data, the warping has to be undone. For twodimensional texture image rendering the unwarping process can be incorporated dircetly into the texture mapping function thus leading to no additional cost at all (see Section 4.1 for a detailled description). For 1 Whereas it may be desirable to let the user change or specify importance interactively, this paper only describes methods for automatically generating importance maps (Section 2). However, addtional user interaction can be added easily. some volume rendering techniques unwarping is possible at low additional cost, see Section 4.2 for an overview and comparison of the different techniques. Using data warping for reducing data size consists of 3 steps: 1. Compute an importance map based on original texture image or volume dataset. 2. Compute a warping function (based on the importance map) and apply it to the original data. The warped data can then be downsampled without significant loss of visual quality. 3. Render image with regard to the inverse warping function. In the following sections I explain each of those steps in more detail. 2 Computing importance maps An importance map (based on the original data) is needed for computing the warping function (Section 3). There are two main possibilities of generating importance maps: automatically or by user interaction. As stated in Section 1, this paper focuses on methods for automatically generating importance maps. Different algorithms are needed for two-dimensional texture images and three-dimensional voxel data, respectively. 2.1 Importance maps for texture images For texture images, the importance map can be computed based on a frequency analysis of the original data. High frequencies correspond to areas of high detail, whereas low frequencies denote less detailed areas. The frequency analysis can be done by a wavelet packet decomposition usually implemented using an iterated filter bank (Figure 1). However, it is also necessary to take into account the distortion introduced by the surface parametrization (i.e. where the texture is stretched when applied to the model s geometry and where it is contracted). The mapping between texture points and surface points is described by following parametrization function: (u, v) Ω t(u, v) = (x(u, v), y(u, v), z(u, v)) S, (1)

2 Figure 1: Image fitler bank: The filter bank takes an Image I(u,v) as input and outputs four matrices of coefficients HH, HL, LH, and LL. The boxes denote convolution using low-pass (H 0) and high-pass (H 1) filters. The circles (2 ) denote downsampling by a factor of two. Figure 3: Neighborhood importance map calculated automatically for the bonsai tree dataset. The map is represented as an isosurface. The darker the shade of the isosurface region, the more important is the corresponding area in the volume dataset. Figure 2: Texture image (left) and calculated frequency map (right). Brighter colors denote higher frequencies. where (u, v) Ω [0, 1] 2 are the texture coordinates and (x, y, z) S R 3 are the surface points. The mapping distortion is then given by the singular values of the Jacobian matrix J = [ t u t v ]. (2) If no distortion occurs, then both singular values equal to 1. When any singular value is larger than 1, the texture is stretched when applied to the model s geometry. Consequently, the texture is contracted if any singular value is smaller than 1. Using those singular values, the model geometry can be taken into account by scaling the results of the wavelet packet decomposition accordingly to the measured distortion (for a detailed explanation see [2]). This leads to a modulated frequency map σ(u, v) which is then used as importance map. See Figure 2 for an example. 2.2 Importance maps for volume data For volume data, an importance map defines a measure of importance for each voxel. In [1] a method is given to automatically compute a neighborhoodcrossing map which can then be used as featuresensitve importance map for volume data. Figure 3 gives an illustration of such a neighborhood-crossing map. The map is computed by searching for intensity values within a user-defined range of the isolevel for each voxel of the original dataset. Such values are called crossings. For each crossing, voxels in a surrounding neighborhood 2 is checked for a crossing too. The total number of crossings found in each voxel s neighborhood builds the neighborhood-crossing map. 3 Warping the data 3.1 Computing the warping function Based on the importance map it is possible to compute a warping function ω : Q Q and its inverse warping function η = ω 1 : Q Q. The warping function ω is obtained by a discrete relaxation algorithm described in [2]. A rectangular (or cubic in case of volume data) grid is relaxed by minimizing the energy function E D = σ(ν i )σ(ν j ) ν i ν j 2 (3) e=(i,j) with the grid vertices v i and the grid edges e = (i, j). The energy functions takes into account the importance map σ(u, v) described in Section 2. To obtain ω, the energy function is minimized in successive steps 3. In each step any grid vertex νi n gets displaced to obtain the new vertex ν n+1 i. First, unconstrained displacement vectors δ U (ν) are computed: δ U (ν) i = µ ij (ν j ν i ) µ ij ν j ν i 2 j i j i σ(ν i ) 2σ(ν j ) (4) 2 The size of this neighborhood is specified by the user. 3 In fact the algorithm leads to an estimation of the inverse warping function η, because it relaxes the grid covering the original image (i.e. the domain of the warping function ω).

3 Figure 5: Deformation grids for a volume dataset (shown as two-dimensional slice). Figure 4: Warping texture images: (a) Original texture image. (b) Warped and downsampled texture image (this image is actually only one quarter its original size). (c)- (d) Deformation grids for defining the warping function ω. In this formula, i is the set of grid vertices connected to i, and σ(ν j ) µ ij = j i σ(ν j). (5) To keep the corners of the warped image fixed, additional boundary constraints are imposed: ω η = 0. (6) Q This means that boundary points can only move along the boundary, while corner points cannot move at all. Adhering to the boundary constraints finally leads to the constrained displacement vectors ν n+1 i = ν n i + δ(ν n ) i. (7) A detailed description can be found in [2], a similar algorithm was proposed by [7]. 3.2 Warping texture images The warping function ω can be applied directly to a texture image I(u, v). This yields the warped texture image I (u, v ). In practice it is more efficient to evaluate the inverse warping function η for each pixel of the warped texture image. See Figure 4 for an example of a texture image and the corresponding warped texture image. 3.3 Warping volume data For volume data, the warping function ω can be represented by two associated cubic grids G and G (Figure 5). Both grids share the same size and connectivity. The warped grid G is obtained by applying ω to each vertex of the regular grid G. For warping the volume dataset it is necessary to evaluate the inverse warping function η for each voxel in the warped dataset. This is done by determining which cell of G the voxel belongs to and calculating trilinear coordinates with respect to the eight vertices of the containing cell. Those trilinear coordinates are then evaluated at the eight vertices of the correspondending cell in the warped grid G to find the corresponding voxel in the original dataset. Since G is a regular grid, simple and efficient quantization can be used for determining the containing cell. 4 Rendering the data The effect of warping the orginial data must be undone in the rendering step to produce an image comparable to the image rendered directly from unwarped data. Different techniques are used for rendering texture images and volume data. 4.1 Rendering texture images As stated in Section 2.1, the mapping t(u, v) describes how the texture image is mapped onto the models geometry. Support for this mapping function is included in modern graphics hardware, as it is a standard part of the texturing pipeline. For rendering warped texture images we compute a new mapping function t (u, v ) which maps the warped texture image onto the model s geometry such that the visual appearance is similar to mapping the original image using t(u, v). Figure 6 shows the relationships between the original texture image, the warped texture image and the rendered image. The new mapping function t can be implemented in hardware exactly like t. Therefore rendering warped textures does not require additional costs compared to

4 Figure 6: Texture image warping: Original image I(u, v), warped image I (u, v ), textured model S(x, y, z), mapping function t, new mapping function t for warped texture. rendering unwarped textures. 4.2 Rendering volume data As there exist different techniques for rendering volume data, different approaches must be used to undo the warping. The following Sections deal with common volume rendering techniques and give an overview how unwarping may be incorporated into the rendering process Isosurface extraction Isosurface rendering is an indirect approach to volume rendering. Instead of directly rendering the volume data, an isosurface (usually a triangular mesh) is extracted at a specified isolevel and subsequently rendered using standard surface rendering methods. See [8] for more information. For generating a (warped) mesh M from a warped volume dataset, any isosurface extraction algorithm (see for example [9], [11], and [12]) can be used. After generating the mesh, each vertex location must be unwarped to recover the original visual appearance, resulting in the unwarped mesh M. Unwarping is done by applying the inverse warping function η to each vertex, i.e. M = η(m ). (8) The warped volume dataset is expanded in areas of high detail and contracted in areas of low detail. Generating the mesh from the warped volume and subsequently unwarping this mesh leads therefore to a mesh with adaptive tessellation. See Figure 7 for an example comparing evenly and adaptively tessellated meshes. When using an adaptively tessellated mesh, fewer vertices are necessary to achieve the same level of detail Splatting Splatting is an high-quality object-order approach to volume rendering. Each voxel is splatted onto the screen, leaving a specific footprint. The weighted footprints are then accumulated resulting in the final image. See [10] for more information. Figure 7: Comparison of meshes with even tessellation and adaptive tessellation, respectively. From the bonsai tree dataset. For splatting a warped volume dataset it is necessary to consider the inverse warping function η for each voxel, i.e. every footprint has to be recalculated 4. This requires additional cost compared to splatting the original dataset. However, as the warped volume dataset can be downsampled (shrunk) without significant loss of visual quality, the reduced number of voxels may outweigh this performance penalty Ray casting Ray casting is an image-order approach to volume rendering. For each pixel in the final image a ray is shot into the volume dataset yielding an intensity profile which is then composited into the image. See [6] for more information. Ray casting a warped volume dataset requires nonlinear rays to undo the warping. A ray shot into the warped volume dataset must be warped using the warping function ω. However, evaluating ω is more expensive than evaluating the inverse warping function η: While for evaluating η it is necessary to find the containing cell in the regular grid G which can be done efficiently by quantization (see Section 3.3), finding the containing cell in the warped (and hence nonregular) grid G for evaluating ω requires an expensive spatial search Shear warp factorization Shear warp factorization is a fast object-order approach to volume rendering. The volume dataset is divided into parallel slices. Two shear operations are applied to those slices before compositing is done along either the x-, y- or z-axis. Finally the intermediate image is warped to get rid of distortions. See [5] for more information. Using shear warp factorization for rendering warped volume datasets seems not to be applicable. First of all, the shear warp technique assumes that 4 Recalculation of every footprint is similar to perspective splatting described in [4].

5 each voxel correspondends to exactly one pixel, which is not the case for a warped volume dataset. Slicing the warped dataset does not give feasible results either Texture-based rendering Texture-based rendering is an approach to volume rendering that can be implemented in hardware. Using two- or three-dimensional textures is possible. Slices of textures are then composited into the final image using back-to-front compositing. See [3] for more details. Similar to the shear warp technique, texture-based rendering of warped volume datasets suffers from the problem of dividing the warped dataset into meaningful slices. Extending the compositing to undo the warping should be possible but expensive and may abandon the possibility of using standard graphics hardware. 5 Results Figure 8 and Figure 9 show a comparison of visual quality between warped data and original data at reduced data size for both cases addressed in this paper, two-dimensional texture images and threedimensional volume data. It is apparent that warping increases quality at high downsampling rates when compared with downsampling unwarped data. Especially for isosurface rendering it is known that aggresively downsampling the voxel data often drastically changes the geometry of the extracted mesh. Warping relieves this effect, as can be seen in Figure 9. 6 Conclusions Using data warping based on automatically generated importance maps makes it possible to significantly reduce data size (i.e. shrink textures and reduce number of voxels) while obtaining good visual appearance. The automatic generation of importance map works very well for two-dimensional texture images (see Section 2.1), as does computing the warping function using a iterative grid relaxation approach (Section 3). Future work may include improving the algorithm for automatically generation of importance maps for three-dimensional volume data (Section 2.2) and combining data warping with other volume rendering techniques than isosurface extraction (Sections 4.2.2, 4.2.3, 4.2.4, and 4.2.5). 7 Acknowledgments Figure 1, Figure 2, Figure 4, Figure 6, and Figure 8 are taken from [2]. Figure 3, Figure 5, Figure 7, Figure 9, and Figure 10 are taken from [1]. References [1] Laurent Balmelli, Christopher J. Morris, Gabriel Taubin, and Fausto Bernardini. Volume warping for adaptive isosurface extraction. In Proceeding of IEEE Visualization, [2] Laurent Balmelli, Gabriel Taubin, and Fausto Bernardini. Space-optimized texture maps. Computer Graphics Forum, 21(3): , [3] Allen Van Gelder and K. Kim. Direct volume rendering via 3d texture mapping hardwar. In 1996 Volume Rendering Symposium, pages 22 30, [4] J. Edward Swan II, Klaus Mueller, Torsten Möller, Naeem Shareef, Roger Crawfis, and Roni Yagel. Perspective splatting. [5] P. Lacroute and Marc Levoy. Fast volume rendering using a shear-warp factorization of the viewing transformation. In Proc. SIGGRAPH 94, pages , [6] Marc Levoy. Efficient ray tracing of volume data. ACM Transactions on Graphics, 9(3): , [7] S. Z. Li. Adaptive sampling and mesh generation. Computer Aided Design, 27(3): , [8] William Lorenson and Harvey Cline. Marching cubes: A high resolution 3d surface construction algorithm. Computer Graphics, 21(4): , [9] R. Shu, C. Zhou, and M. S. Kankanhalli. Adaptive marching cubes. The Visual Computer, 11(4): , [10] L. Westover. SPLATTING: A parallel, feedforward volume rendering algorithm. PhD thesis, UNC-Chapel Hill, [11] Jane Wilhelms and Allen Van Gelder. Octrees for faster isosurface generation. ACM Transactions on Graphics, 11(3): , [12] Z. J. Wood, M. Debrun, P. Schröder, and David Breen. Semiregular mesh extraction from volumes. In Visualization 2000, pages IEEE, 2000.

6 Figure 8: Comparison of the visual quality obtained at reduced texture sizes. Top row: original image; bottom row: warped and downsampled image. (a) Original texture size. In (b), (c), and (d) the image is reduced at 30%, 20%, and 10% of the original size, respectively. Figure 9: Comparison of the visual quality obtained at downsampled volume datasets. Volumes are rendered using isosurface extraction. Top row: original dataset; bottom row: warped and downsampled dataset (see Figure 5 for the importance map used). The volume dimensions are (from left to right) 1283, 963, 643, and 323, respectively.

7 Figure 10: Adaptive isosurface extraction. Images show (from left to right): Isosurface extracted from original dataset (128 3 ), isosurface extracted from warped dataset, unwarped isosurface. Closeups compare regular and adaptive tessellation.