Ray-Patch Intersection for Improving Rendering Quality of Per-pixel Displacement Mapping Ravish Mehra 1 Subodh Kumar 1 1 IIT Delhi

Transcription

1 Ray-Patch Intersection for Improving Rendering Quality of Per-pixel Displacement Mapping Ravish Mehra 1 Subodh Kumar 1 1 IIT Delhi Abstract Recent GPU advances have popularized pixel-shader based techniques for per-pixel displacement mapping. An important step in the process is to compute intersection of ray with a bilinear patch approximating the displacement map. Accurate ray-patch intersection can significantly enhance the quality of rendered images. However, the increase in quality comes at a significant computation cost resulting in a substantial drop in performance. This paper explores this trade-off between computation and quality. We propose several techniques for ray-patch intersection and compare it to exact ray-quadric/ray-bilinear intersection proposed previously. We show that our quadric based local-gradient-plane approximation achieves a comparable visual quality with a significant performance improvement. 1. Introduction Ray-patch intersection has traditionally been used in various ray-tracing approaches for rendering complex geometry. This geometry can be made up of either simple primitives like points and triangles or high order primitives like second order patches (bilinear, quadric, SLIM, Bezier). These patches are not only C 0 continuous but may also have smooth derivatives as well. This results in visibly better shading than is possible with simple primitives. These representations have numerous applications in scientific visualizations, CAD systems for modeling geometry, physical simulations of particles and biological applications for visualizing DNA s & molecules. Recently, ray-patch intersection has found a new application per-pixel displacement mapping (also called inverse displacement mapping). Based on programmability of modern GPU s, per-pixel displacement mapping is a ray-casting approach that tries to achieve the same visual effects as traditional displacement mapping [2] but without ever generating or drawing the detailed geometry. Incorporating raypatch intersection in the last steps of per-pixel displacement mapping algorithm (i.e. near the surface boundary) significantly improves the visual quality of the rendered results. However, since it requires expensive computation, it impacts the performance as well becoming the main bottleneck. This paper is a detailed study of ray-surface patch intersection. We propose a sequence of approximation techniques for this and test the validity of these approximations with chosen patch sizes. We provide a comparison between the exact and approximate techniques on the basis of quality and performance achieved by them. Finally, we propose an approximate ray-quadric intersection technique in which the quadric surface is approximated with a tangent plane whose normal is in direction of surface gradient. We show that the quality achieved by our method is comparable to exact ray-quadric intersection but is much faster. We also give a simple conversion formula for converting height field based bilinear patch into quadric. The main contributions of this paper are as follows: 1. A novel quadric based local-gradient-plane approximation & intersection technique that achieves high rendering quality and is much faster than numerically solving ray-quadric intersection. 2. A quality and speed based comparison of different exact and approximate intersection techniques. Figure 1. ICVGIP box: res , 45 fps

2 2. Related work In the last few years, there has been increasing interest in graphics community for using non-triangular primitives for displaying complex models. Several high order surface representations such as bilinear, quadric, SLIM, isosurfaces have been used providing superior quality shading as compared to flat triangles and points. Several point splatting approaches [1, 6, 9] have been proposed for rendering these primitives. Sigg et al [10] render each quadric primitives separately by representing them as single vertex and computing their bounding volume in vertex shader. For fragments generated via this bounding box, ray-quadric intersection is computed in fragment shader. Finally, depth buffer is used for finding closest quadric s fragment and blending the result. Per-pixel displacement mapping on the other hand, renders a single bounding volume for the entire geometry, which could be highly complex and detailed. Actual geometry is stored offline in the pre-processing stage either as height-map, cone-map [4] or relief-map [7, 8] in 2D texture or VDM [12], GDM [13] or distance-map [3] in 3D texture. This dataset is passed onto the fragment shader that computes the actual intersection of viewing ray with the detailed geometry. This intersection point is used for final shading. These techniques score over traditional displacement mapping [2] by not tessellating the surface and hence not rendering a large number of micro polygons generated, thus achieving real time performance. Since implicit representation of detailed geometry in all these techniques is still triangular, shading achieved is of the similar quality as rasterization. Recently, several per-pixel displacement mapping techniques changed the implicit surface representation to higher order primitives(surface patches) and performed ray-patch intersections as the last step of their algorithm for significantly improving their rendering results. Oh et al [5] represented the discrete height field as bilinearly interpolated height-map and sampled it linearly along the viewing ray to find the ray-bilinear intersection, thus removing blockiness caused by discrete height-map. But as pointed out in [11], this linear approximation of bilinear height-map is not always correct and fails at places where the height-map is not approximately linear. Tevs [11] create a hierarchical maximum mipmapped height-map with bilinear patch residing at the lowest hierarchy level. Ray-bilinear patch intersection is performed at the lowest level either analytically using the method proposed in [9] or approximately using linear & binary search steps to converge to the final intersection point. They drew a comparison between these two approaches and favored the second, approximate, technique due to the significant performance cost of the first. 3. Overview Per-pixel displacement mapping techniques for height field rendering store the detailed complex geometry as a height-map. This height-map is a uniformly sampled 2D grid that stores at each sample point the height of the surface at that point above a reference surface. Different techniques process this height-map for generating specialized datasets such as cone-map [4], relief-map [7, 8] or maximum mipmapped height-map [11]. These datasets are used by their respective algorithms in the rendering stage for finding point of intersection of the ray with the detailed geometry. For piecewise patch approximation of height mapped surface we do the following - Within each 2D grid cell we perform an approximation for the height-mapped surface passing above that cell. This results in a piece-wise approximation for the entire geometry. This cell approximation could be in the form of a triangular tessellation, bilinear or quadric, or simply planar. For the triangular tessellation, we first find the height of the surface patch at the center of the cell. The triangles are generated by connecting the surface points at the corners of the cell with that at the center yielding four triangles. Approximation by a planar patch is a plane fitting problem and can be solved by standard algorithms such as Levenberg-Marquardt algorithm (LMA). Conversion to a bilinear patch can be done in a straightforward manner as given in [9]. This bilinear expression can be converted to a quadric function directly(section 4.1). We compute the patch parameters offline and store them in a 2D texture that is passed onto the fragment shader. Triangular, bilinear and quadric patch parameters can also be computed online in the fragment shader by passing the heightmap. For ray-patch intersection, we will first discuss quadric based exact numerical roots technique and then propose three approximate techniques triangular patch, planar patch and local-gradient-plane approximation. Figure 2. Overview of algorithm A generic overview of the rendering pass employed by per-pixel displacement mapping techniques incorporating ray-patch intersection is as follows:

3 Algorithm 1 Per-pixel displacement mapping with raypatch intersection 1. Instead of generating and drawing the actual detailed geometry, we draw its bounding box. 2. For each fragment f generated by the bounding box, we trace a ray starting at its texture location r in the direction d (from eye to r) until it reaches close to detailed geometry. 3. We access the parameters of the patch corresponding to the cell reached in step 2 and compute ray-patch intersection exactly or approximately. 4. If the intersection lies within the cell, we have successfully found the intersection point and exit. Otherwise the ray hits the boundary of the cell and we must go back and continue step 2. The ray-geometry intersection reduces to first locating the intersection in a cell and then computing the exact intersection of the ray with the patch corresponding to that cell. Ray-tracing step and the exact definition of close to detailed geometry varies based on the actual algorithm being used. For maximum mipmapped heightmap [11] and pyramidal displacement mapping [5], we switch to ray-patch intersection when we reach the lowest level of the mipmap hierarchy. Sphere tracing [3] uses distance to the nearest point on the geometry as the safety distance for tracing the ray. When this safety distance becomes zero(or less than some ɛ), we perform ray-patch intersection. Ray-patch intersection, due to its high computational cost, drastically impacts the performance of per-pixel displacement mapping and becomes the main bottleneck. Table 2 compares the performance of a per-pixel displacement mapping algorithm sphere tracing, with and without ray-patch intersection. Figure 3 shows the visual quality achieved by both the methods. 4. Quadric surface & ray-quadric intersection Each height-map cell defined by its four corners can be exactly represented as a bilinear patch that maintains C 0 continuity. This bilinear patch is a second degree function and can be directly converted into a quadric. We use the full second degree quadric defined by following equation- Ax 2 + By 2 + Cz 2 + Dxy + Eyz + F zx + Gx + Hy + Iz + J = 0 (1) 4.1 Bilinear patch to quadric conversion This section presents the conversion of bilinear surface patch into corresponding quadric. The results are stated here while the derivation is given in Appendix. Bilinear patch formed by four points ( V A, V B, V C, V D ) described by weighing parameters (u, v) is : V (u, v) = (1 u)(1 v) V A + (1 u)vv B + (1 v)uv C + uvv (2) D Equation 34 gives us the quadric coefficients: A = B = C = E = F = 0 (3) ( ) ( ) a3 c3 b 2 d 2 a 3 D = G = (4) ( ) b3 c 1 a 3 d 1 H = I = 1 (5) ( ) ( ) d1 d 2 a 3 + d 3 d2 b 3 c 1 + d 1 c 3 b 2 J = (6) We store coefficients D, G, H, J as RGBα components in a 2D texture(shown in Figure 4). Figure 4. Quadric parameters (a) sphere tracing (b) sphere tracing + ray-patch Figure 3. Quality comparison We now discuss in detail several approaches of doing surface patch approximation and ray-patch intersection. 4.2 Ray-quadric Intersection For computing the intersection of ray r+td with quadric we will first discuss numerically solving ray-quadric intersection.

4 (a) 64 2 (b) (c) Figure 5. Statue surface: Rendering result for ray-quadric at different resolutions Numerical roots We solve the equation of the ray with quadric as follows: Ray equation : r + td : (x 0, y 0, z 0 ) + t(d x, d y, d z ) Solving with quadric, A(x 0 + td x ) 2 + B(y 0 + td y ) 2 + C(z 0 + td z ) 2 + D(x 0 + td x )(y 0 + td y ) + E(y 0 + td y )(z 0 + td z ) + F (z 0 + td z )(x 0 + td x ) + G(x 0 + td x ) + H(y 0 + td y ) + I(z 0 + td z ) + J = 0 Substituting, (7) Figure 7. Ray-quadric intersection P =Ad 2 x + Bd 2 y + Cd 2 z + Dd x d y + Ed y d z + F d z d x Q =2x 0 d x A + 2y 0 d y B + 2z 0 d z C + x 0 d y D + y 0 d x D + y 0 d z E + z 0 d y E + z 0 d x F + x 0 d z F + Gd x + Hd y + Id z R =Ax By Cz Dx 0 y 0 + Ey 0 z 0 + F z 0 x 0 + Gx 0 + Hy 0 + Iz 0 + J Finally, (8) P t 2 + Qt + R = 0 (9) Algorithm 2 Ray-quadric intersection if P = 0 then t = R Q else if Q 2 4P R < 0 then t = HIT CELL BOUNDARY(x 0, y 0, z 0 ) else t = Q± Q 2 4P R 2P Quadric patch defined in this manner truly represents the surface formed by the four corners of the height-map cell. Figure 5 shows the quality achieved at various resolutions of coefficient texture(figure 4). Note the superior quality even at low resolutions. Due to a large number of steps involved in the ray-quadric intersection step, this method is much slower. 5 Approximate intersection techniques In this section, we describe in detail the three approximate ray-patch intersection techniques Ray-triangular patch intersection In this approach, we approximate the surface patch in a height-map cell with four triangular sub-patches formed by the four corners and the midpoint of that cell. This computation can be done online eliminating the need for preprocessing. For finding ray-patch intersection, we compute ray-triangle intersection for the four triangular sub-patches and test the validity of the intersection point. For a valid intersection, the point should lie within the boundaries of the cell and the intersected triangle. Also the normal of this triangle(ñ) should be facing the ray(r + td) i.e. d.ñ < 0. We stop the algorithm if the intersection point is valid. Otherwise, we hit the wall and continue tracing the ray. Figure 8 illustrates the steps of the algorithm. Figure 6 shows the rendering result achieved by this method for various resolutions of height-map. Rendering

5 (a) 64 2 (b) (c) Figure 6. Rocky surface: Rendering result for triangular patch at different resolutions quality increases with resolution as approximation of surface with triangular patch becomes more accurate at higher resolution. t = (Ax 0 + By 0 + Cz 0 + D) (Ad x + Bd y + Cd z ) (11) Figure 8. Triangular patch intersection Here (A, B, C) is the direction of normal(n) to the plane. Small shader code resulting in increased performance is the main advantage of this technique. But this planar approximation is valid only at high resolutions. At lower resolutions, C 0 continuity is not maintained causing missed intersections and obvious cracks at cell boundaries. Figure 11 shows the quality achieved by this method for different resolutions of coefficient texture(figure 9). Planar approximation becomes more & more accurate as we increase the resolution Planar approximation and ray-plane intersection We approximate the surface of the height-map cell with the plane Ax+By+Cz+D = 0 by performing plane fitting and store the coefficients A, B, C, D in RGBα channels of 2D texture. This texture is passed to the fragment shader. Figure 10. Ray-plane intersection 5.3 Local-gradient-plane approximation Figure 9. Planar approximation parameters For finding ray-plane intersection, we numerically solve for intersection of the plane equation with the ray r + td as follows: Putting r = (x 0, y 0, z 0 ), d = (d x, d y, d z ) in the plane equation A(x 0 + td x ) + B(y 0 + td y ) + C(z 0 + td z ) + D = 0 (10) We now present an approximate ray-quadric intersection approach that achieves superior rendering quality comparable to exact ray-quadric intersection discussed before but at the same performance cost as ray-plane intersection. For finding the ray-quadric intersection of the ray r+td starting at point r(x 0, y 0, z 0 ) going in the direction d(d x, d y, d z ), we first find the corresponding point r (x 0, y 0, z ) located below r lying on the quadric(see Figure 12). We find the

6 (a) 642 (b) 1282 (c) 2562 Figure 11. ICVGIP surface: Rendering result for planar approximation at different resolutions gradient vector of the quadric surface Ax2 +By 2 +Cz 2 + Dxy + Eyz + F zx + Gx + Hy + Iz + J = 0 at r0. = ( x, y, z ) (12) 0 x = (2x0 A + y0 D + z F + G) 0 (13) y = (2y0 B + x0 D + z E + H) (14) z = (2z 0 C + y0 E + x0 F + I) (15) Figure 13 shows the rendered result of local-gradientplane approximation at various resolutions of quadric coefficient texture(figure 4). It can be seen that this method achieves high quality similar to numerically solving rayquadric. (a) 642 Figure 12. Local-gradient-plane intersection We then construct a plane tangential to the surface passing through r such that the normal vector N of the plane is ˆ Equation of plane can in the direction of the i.e N =. be written as N.(x, y, z) + M = 0 for constant M x x + y y + z z + M = 0 (b) 1282 (16) Figure 13. Glass surface: Gradient-plane Since the plane passes through r0, M = ( x x0 + y y0 + z z 0 ) Using equation 11, the solution becomes x (x0 ) + y (y0 ) + z (z0 ) + M t= x dx + y dy + z dz z (z0 z 0 ) = (Using eq. 17) ( x dx + y dy + z dz ) (17) (18) (19) 6. Comparison and Results We compare our proposed approximation techniques triangular, planar & gradient plane, with exact bilinear and quadric patch techniques in terms of performance and speed. Table 1 shows a comparison between their rendering performance for different test cases at varying resolutions.

7 Resolution 642 Triangular Bilinear Planar Quadric Gradient Statues surface Rocky surface Glass surface ICVGIP surface Table 1. Performance comparison of various techniques (in fps). Screen resolution 1024x768 In this test, we incorporate only steps 1,3,4 of the algorithm(section 3) and assume that we are always close to the surface. This gives us a direct comparison between these techniques as we only perform ray-patch intersection for tracing the ray. Among these, triangular, bilinear and quadric are the slowest as they require complicated root solving that requires longer shader code. Planar intersection is the fastest but the quality of rendering suffers at low resolution(figure 11a,b). On the other hand, gradient plane achieves faster frames per second with better quality even at low resolution. In Table 2, we show the rendering performance of a per-pixel displacement mapping algorithm - sphere tracing, and augment it with different ray-patch intersection techniques. We define close to surface when the safety distance of sphere tracing becomes zero(or less than ) and perform all the four steps of the algorithm. We can also see the impact of ray-patch intersection on performance of per-pixel displacement mapping. Simple sphere tracing( =0.01) with no ray-patch intersection is obviously the fastest but yields the least quality. With = 0, the quality improves but becomes extremely slow. Sphere tracing( =0.01) with raytriangular, ray-quadric and ray-bilinear is slow but yield the highest quality. Ray-planar is fast but gives good rendering quality only at higher resolutions. Ray-gradient-plane, on the other hand, is fast and gives rendering quality comparable to triangular, bilinear & quadric even at low resolutions. Figures 1 & 14 show the demos rendered using sphere tracing augmented with ray-gradient-plane intersection. We have implemented the proposed techniques as Shader Model 4.0 fragment programs. Timings reported in this paper were produced on Intel PentiumD 3.4GHz machine with Geforce 8800GTS graphics card and 2GB RAM. All images given in the paper were rendered at a screen resolution of 1280x Conclusion and Future Work We have presented several exact and approximate techniques of performing ray-patch intersection and explored its applicability in improving the rendering quality of per-pixel displacement mapping. We have provided a detailed quality & performance based comparison between these techniques and their impact on per-pixel displacement mapping. We have also tested the validity of the proposed approximations at different resolution. For future work, we would like to explore other surface patch approximations ( Bezier, SLIM ) along with efficient & accurate methods for their ray-patch intersection. (a) Tiled cylinder: texture 5122, 88fps (b) Earth: Coeff. texture 5122, 135fps Figure 14. Ray-gradient-plane intersection

8 Statues surface Rocky surface Glass surface ICVGIP surface Resolution ST(ɛ = 0.0) ST(ɛ = 0.01) Triangular Bilinear Planar Quadric Gradient Table 2. Performance comparison of various techniques (in fps). Screen resolution 1024x768 References A Bilinear patch to quadric conversion [1] M. Botsch, A. Hornung, M. Zwicker, and L. Kobbelt. Highquality surface splatting on today s gpus. Point-Based Graphics, Eurographics/IEEE VGTC Symposium Proceedings, pages , June [2] R. L. Cook. Shade trees. SIGGRAPH Comput. Graph., 18(3): , [3] W. Donnelly. Per-pixel displacement mapping with distance functions. GPU Gems 2, 22(3): , [4] Dummer. Cone step mapping: An iterative ray-heightfield intersection algorithm [5] K. Oh, H. Ki, and C.-H. Lee. Pyramidal displacement mapping: a gpu based artifacts-free ray tracing through an image pyramid. In VRST 06: Proceedings of the ACM symposium on Virtual reality software and technology, pages 75 82, [6] M. Pauly, R. Keiser, L. P. Kobbelt, and M. Gross. Shape modeling with point-sampled geometry. ACM Trans. Graph., 22(3): , [7] F. Policarpo and M. M. Oliveira. Relief mapping of nonheight-field surface details. In I3D 06: Proceedings of the 2006 symposium on Interactive 3D graphics and games, pages 55 62, New York, NY, USA, ACM. [8] F. Policarpo, M. M. Oliveira, and a. L. D. C. Jo Realtime relief mapping on arbitrary polygonal surfaces. In I3D 05: Proceedings of the 2005 symposium on Interactive 3D graphics and games, pages , New York, NY, USA, ACM. [9] S. D. Ramsey, K. Potter, and C. Hansen. Ray bilinear patch intersections. journal of graphics tools, 9(3):41 47, [10] C. Sigg, T.Weyrich, M.Botsch, M. GrossRamsey, K. Potter, and C. Hansen. Gpu-based ray-casting of quadratic surfaces. Eurographics symposium of point based rendering, [11] A. Tevs, I. Ihrke, and H.-P. Seidel. Maximum mipmaps for fast, accurate, and scalable dynamic height field rendering. In SI3D 08: Proceedings of the 2008 symposium on Interactive 3D graphics and games, pages , [12] L. Wang, X. Wang, X. Tong, S. Lin, S. Hu, B. Guo, and H.-Y. Shum. View-dependent displacement mapping. ACM Trans. Graph., 22(3): , [13] X. Wang, X. Tong, S. Lin, S. Hu, B. Guo, and H.-Y. Shum. Generalized displacement maps. Computer Graphics Forum, 22(3): , Bilinear patch formed by four adjacent cell points V A (x 1, y 1, z 1 ), V B (x 2, y 1, z 2 ), V C (x 1, y 2, z 3 ), V D (x 2, y 2, z 4 ) described by weighing parameters (u, v) is equation 2. Substituting a = (a 1, a 2, a 3 ) = V A + V D V B V C (20) = (0, 0, z 1 + z 4 z 2 z 3 ) (21) b = (b1, b 2, b 3 ) = V C V A (22) = (0, y 2 y 1, z 3 z 1 ) (23) c = (c 1, c 2, c 3 ) = V B V A (24) = (x 2 x 1, 0, z 2 z 1 ) (25) d = (d 1, d 2, d 3 ) = V A (26) And rearranging, = (x 1, y 1, z 1 ) (27) V (u, v) = uv a + u b + v c + d (28) (X, Y, Z) = uv(a 1, a 2, a 3 ) + u(b 1, b 2, b 3 ) (29) Comparing both sides, + v(c 1, c 2, c 3 ) + (d 1, d 2, d 3 ) (30) X = vc 1 + d 1 (31) Y = 0 + ub d 2 (32) Z = uva 3 + ub 3 + vc 3 + d 3 (33) Substituting value of u and v from Equations 31& 32 in Equation 33, we get ( ) ( ) ( ) a3 c3 b 2 d 2 a 3 b3 c 1 a 3 d 1 Z = XY + X + Y ( ) d1 d 2 a 3 + d 3 d 2 b 3 c 1 d 1 c 3 b 2 = + (34)