Efficient rendering of multiresolution meshes with guaranteed image quality

Transcription

1 Efficient rendering of multiresolution meshes with guaranteed image quality Reinhard Klein and Andreas Schilling Wilhelm-Schickard-Institut, GRIS, Universität Tübingen, Germany Abstract 4, 5, 13, 14, 16]. The generation of a MRM of an object Recently, output-sensitive rendering algorithms have gained a lot interest. While some years ago the number of triangles describing a scene was smaller than the number of pixels on the screne, this relation has changed to the opposite. The size of the models has become significantly larger while the screen resolution remained more or less constant. One way to get output-sensitivity, is the view-dependent use of multiresolution models. The key idea is to determine the lowest resolution for each part of the object that is necessary to get good images that are visually indistinguishable from rendering the original mesh. To take care of the illumination of the objects there are approaches that use cones of normal vectors to perform illumination dependent selective refinement. This allows the correct rendering of highlights and transitions between lit and unlit areas of an object. The drawback of these approaches is that using only the normal cone for the detection gives only a very poor criteria to detect the problematic cases. Therefore, too much refinement has to be performed which in most cases prevents real time update rates between successive frames. In this paper we describe a new approach for better estimation of normal deviations between different levels of detail. This allows for accurate lighting with a minimum number of triangles. 1 Introduction and previous work Multiresolution modeling (MRM) for simplicial geometric models (any dimension, non-orientable, non-manifold, non-regular) has become very popular. Multiresolution models provide the basis to handle, visualize and transmit over the network and edit very large datasets [15, 2, 9, 1, 1 Universität Tübingen, Auf der Morgenstelle 10 / C9, Tübingen, Germany. fandreasjreinhardggris.uni-tuebingen.de Figure 1: Example of a hierarchy of simplification operations. On the top the original triangulation is shown. The simplification algorithm performs the simplification operations in the sequence OP1, OP2, OP3, OP4. Since OP2 and OP3 can be performed independently from each other (as their regions do not overlap) they can be swapped in the sequence. This is shown in the right branch of the figure.

2 Figure 2: The acyclic directed graph of the simplification operations shown in Figure 1. Since the operations OP2 and OP3 depend only on OP1 they can be performed in arbitrary order. OP4 depends on both, OP2 and OP3. Therefore, both operations have to preceed OP4. generally involves a sequence of local simplification operations like vertex removal, edge collapse, triangle collapse and vertex clustering. This sequence of local simplification operations defines a sequence of coarser and coarser approximations of the original model, the MRM. In order to be able to refine a coarse approximation of the model, the inverse local simplification operations are needed and the MRM must contain the necessary information. Selective refinement becomes possible if the sequence defined by the simplification process is changed. This is only possible under certain circumstances which guarantee that the parts of the mesh on which the operation is performed are present in the current mesh. This requirement automatically defines a hierarchy of modification operations and the corresponding triangles. This hierarchy can be considered as a directed acyclic graph of modification operations or the associated triangles [12]. Figure 1 shows an example. Figure 2 shows the corresponding acyclic directed graph. For illumination depended selective refinement the surface normals are essential. For the decision if the mesh at a certain location has to be refined further, the normals contained in the actual coarse approximation cannot be used. Information about the normals in the corresponding area of the original model (which can be totally different) is required. Figure 3: To determine the normal cone for the light grey triangle in the triangulation the hierarchy shown in Figure 2 is travesed up to the finest level (OP1). Then the normalcone is calculated from the normals of all predecessor triangles in the hierarchy. The resulting normal cone is far too large. Therefore, in previous work the variation of the normals in a certain area is represented by a cone of normals represented by a vector together with an angle. The normal cone conservatively bounds all the normals of all the triangles in the corresponding area. This corresponding area is defined by several authors [17, 5, 10] as all the triangles in the original mesh which are descendents of the area being considered in the simplification hierarchy. However, the dependencies introduced by the simplification hierarchy are not related to the geometry of the

3 Figure 4: The upper two pictures show the illumination of a car bonnet without and with adaptive triangulation. The lower two pictures show the corresponding triangulations. Data courtesy of Volkswagen AG. relevant area. Therefore, in general the area considered for the calculation of the normal cone comprises in many cases more than twice the actual needed area, see Figure 3. This leads to much larger normal cones than required and therefore to unnecessary refinements. Therefore, we suggested in [6] not to use the hierarchy to determine the corresponding area but instead to use a geometric correspondence. Despite of this improvement there still remains a general problem when a normal cone is used: The cone of normals does not tell anything about the distribution of the normals in the relevant area. Figure 4 shows the result of lighting dependent simplification using normal cones. te that in the area of the highlight the car bonnet is refined down to triangles of pixel size although this would be unnecessary especially if Phong-interpolation is used. If we knew that the normals over the area of a triangle can be sufficiently well approximated by a linear interpolation of the vertex normal (which occurs very often) the refinement could be avoided in the following cases:

4 always, if Phong interpolation is used, if Gouraud interpolation is used and a linear brightness distribution can be assumed. In the remainder of the paper, we will introduce a suitable measure to be stored within the MRM, that allows us to detect these cases during the refinement process at rendering time and to obtain substantially smaller approximating meshes for the same high image quality. 2 The simplification algorithm The mesh simplification algorithm always starts with the finest triangulation in 3D space approximating the original model surface. In the following this starting triangulation is denoted with M, indicating that it contains M vertices. The simplification algorithm successively simplifies the starting triangulation by removing vertices from the current triangulation and retriangulating the resulting hole or by collapsing edges or triangles. This is done until no further vertices can be removed from the simplified triangulation without exceeding a predefined distance between the original triangulation and the simplified one. If after a single simplification step the simplified triangulation contains m vertices it is denoted by m : The order in which the simplification steps are performed is determined by a priority queue. For every single vertex v i of the simplified mesh an error value i is computed and stored. This value describes the potential error, that is the modified one-sided Hausdorff distance that would occur between the simplified and the original mesh if the simplification step was performed [9]. At the beginning of the algorithm this error value is computed for every vertex, edge or triangle in the original mesh. The vertices, edges or triangles of the original mesh are then sorted into a priority queue according to their potential error. Afterwards, in each simplification step we successively eliminate the vertex, edge or triangle with the smallest potential error on top of the priority queue. If a simplification step is actually performed the potential errors of the neighbor elements are recomputed and the priority queue is updated. Figure 5: Alias problems due to sampling of normals. 2.1 Generating information about the normals The normals in a simplified triangulation During the simplification process the normals at the vertices must be recalculated to avoid alias problems due to undersampling, see Figure 5. After a simplification step the normals at the vertices v i incident to the modified area are given by n i X = Pj 1 A n j A j ; j where j enumerates all triangles incident to vertex v i :. 1 te that as a consequence of this filtering procedure, the normals at the vertices of a given triangle may change during the simplification process Determination of deviation angles To get information about the deviation of the normals, the correspondence map between the vertices and triangles of the original triangulation and the triangles of the simplified triangulation used for the distance calculations [9] is exploited. For a given triangle (v k1 ;v k2 ;v k3 ) of the simplified triangulation this map delivers all vertices 1 In order to model sharp edges several normals at a vertex are necessary. These cases can be handled in a similar way taking into account only the respective incident triangles. j

5 n ij1 n i v k1 v' ij1 v' ij2 n ij2 v i v i v k3 V k2 Figure 6: Determination of deviation angles by projecting original vertices on plane of triangle Deviation of normals due to normal filtering As described above, normal filtering is performed by recalculating the normals at the vertices of a simplified triangulation based on the adjacent triangles. This filtering does not only occur during the simplification process but also during the extraction of selectively refined meshes from the MRM. During the recalculation of these normals, different constellations of adjacent triangles must be taken into account. The deviation between the normal vector used to determine the deviation angles (see previous paragraph)and an actual normal vector varies. To get a conservative estimation for this deviation after the simplification process all possible normals have to be computed. This can be done with a simple recursive algorithm operating on the directed graph of the hierarchy. 3 The extraction algorithm v i 2 M of the original triangulation that are nearer to than to any other triangle of the simplified triangulation. Each of these vertices corresponding to is projected onto the triangle : In addition to these vertices we determine all vertices which are connected to one of the vertices v i via an edge and project them onto the plane defined by : The projected vertices are denoted with v 0 : They are connected by the same edges as their counterparts in the original triangulation. For each projected triangle ij of which at least one vertex does not belong to ; the two intersection points vij 0 1 and vij 0 2 of their projected edges with the edges of triangle are determined and corresponding normal vectors n ij1 and n ij2 of these points in the original mesh are interpolated, see Figure 6. In the next step for each of these vertices vi 0 ; and intersection points vij 0 k the barycentric coordinates with respect to are computed, and using these coordinates, linearly interpolated normals n i based on the current normals at the vertices v k1 ;v k2 ;v k3 are computed. In a last step the deviation angles i between the original normals n i and corresponding interpolated normals n i in the vertices and intersection points are computed. After computing all these angles a linear deviation function f is computed, such that i f (vi 0 ): The function f is defined by its values 1 ; 2 ; 3 at the vertices v k1 ;v k2 ;v k3 : Figure 7: Simplified model of a sphere. The goal of the multiresolution model is to minimize the number of polygons that are sent to the rendering pipeline, without affecting the quality of the resulting image. To achieve this, the multiresolution model allows the extraction of selectively refined models either by coarsening or refining the current triangulation. The decision whether a given triangle may remain in the triangulation or if its surrounding area must be refined further or if a coarser triangulation of its surrounding area is possible, is the most important step in this process. Figures 7 to 10 serve as illustrations for the different criteria for this decision. For an implementation this can be reduced to the single question of whether a given triangle fulfills the quality requirements or not [8]. To answer this question we have to investigate certain characteristics of that part

6 of the original surface that corresponds to the triangle in question. In the following we call this part of the original surfacecorresponding surface for simplicity. The flow diagram of the algorithm we use for this decision is shown in Figure Handling backfacing surface parts The first question in the algorithm is to decide if a triangle (v i1 ;v i2 ;v i3 )and its corresponding part on the original surface is on the backside with respect to the current viewing position. It can easily be answered in a conservative way using the normals n i1 ;n i2 ;n i3 at the vertices of the triangle together with the corresponding deviation angles i1 ; i2 ; i3.lete i1 ;e i2 ;e i3 be the vectors pointing from the vertices to the observer then we get the following criteria. If 3.3 Controlling the shading errors Figure 9: Geometrically refined model with highlight: shading artefacts! 6 (,e ij ;n ij )<=2, ij ;j =1;2;3; then the surface is backfacing and the triangle need not be refined any more. In the other cases we proceed with checking the geometric and shading errors. 3.2 Controlling the geometric error Figure 10: Illumination dependent selective refinement. In the following we assume that the Blinn-Phong shading model is used. For one light source it is given by: 8< L = : L a k a + L d k d (n l) +L s k s (n h) m ; L a k a if (n l) > 0; else, Figure 8: Refined model: screen space error < 1/2 pixel. To control the geometric error the distance (Hausdorffdistance) between original and simplified triangulation [9] must be converted to a screen space error. Several approaches to this problem have been published and can be used [7, 17, 5, 10]. A further significant reduction can be achieved by avoiding the refinement outside of the view frustum. However, refinement outside the view frustum is inevitable to support fast changes of the viewing direction in VR-applications. where k a ;k d and k s are the ambient, diffuse and specular material coefficients, L a ;L d ;L s the corresponding luminaces, l is the direction from the surface point to the light l+e jjl+ejj source, and h = the so called halfway vector between the vector to the light source l and the vector to the observer e Only ambient light contributions The first test determines for a given triangle if the corresponding surface faces away from the light source and therefore, gets only (constant) ambient lighting. To perform this test we determine if the normal cones at the vertices defined by the normals n i1 ;n i2 ;n i3 at the vertices

7 Backside? Do not refine and the corresponding deviation angles i1 ; i2 ; i3 are facing away from the direction to the light source. Let l i1 ;l i2 ;l i3 be the vectors pointing from the vertices to the light source then we get the following criteria. If 6 (,l ij ;n ik )<=2, ij ;j;k =1;2;3; Projected Triangle Smaller than pixel? Do not refine only the constant ambient term has to be considered and the triangle is not refined further Is linear Phong interpolation sufficient? Refine Refine Geometry ok? Facing away from light source Lin. approx. of normals sufficient? α*dl/dα small enough? Do not refine Linear Phong Interpolation sufficient? In the next test we determine if linear Phong interpolation on the given triangle adequately describes the illumination of the corresponding surface. As a criteria we assume that it can be tolerated if the correct color in a pixel is replaced by a color that would be correct on a point, that is located in a one pixel-wide neighborhood of the pixel itself. In other words, the deviation angle, introduced by the varying normals on the corresponding surface must be less than the minimal deviation between Phong interpolated normals of two points that are one (screen-) pixels apart 3.Letn x ;n y respectively, be the partial derivatives of n in the directions of (screen-) x and y and cos() n x + sin() n y the derivative of n in the direction of (cos(); sin()). w, we have to guarantee that a deviation of the normal by an angle in an arbitrary direction is always less than the minimum absolute deviation value d determined by n x and n y.togetd (and with that ) we could solve the corresponding eigenvalue problem, see Figure 12. To avoid this computation we use the simple conservative bound for d (and therefore for tan()) based on a projection in the normal vector space, see Figure 12. If n x n y 0 we get the following formula Phong interpolation used? Do not refine n y, n x tan() jjn x,(n x (n y,n x )) ((n y, n x ) (n y, n x )) jj: Otherwise, n x is replaced by,n x : For sake of simplicity, for n x and n y in this calculation the Phong increments can be used. Up to this point the angular deviations are con- Refine Brightness sufficiently linear? Do not refine Figure 11: Flow diagram of the extraction algorithm. 2 Although cases can be constructed where these nine tests are required, they can normally be reduced to three tests 6 (,l ij ;n ij ) < =2, ij ;j =1;2;3with only marginal effects on the resulting illumination 3 To be correct the deviation of the halfway vector should also be considered. However, we neglect the deviation of this vector in a one pixelwide neighbourhood.

8 n n+ n/ x α Figure 12: Finding the smallest deviation in directional deviation space. trolled. Therefore, the resulting meshes cannot only be used for Phong-illumination but also in ray-tracing applications, or with environment mapping which is of special interest in the CAD-industry to visualize the quality and smoothness of free-form surfaces. If linear interpolation of the normals is not sufficient, there still may be cases where refinement may be unnecessary, since the gradient of the intensity may be so small that in larger neighborhoods of the pixel or even on the whole triangle the differences of intensities cannot be recognized by an observer. This cases are detected in a subsequent test. For this purpose let us assume that we use a Phong model. Than the original normals deviate from the interpolated normals at most by an angle of : w we have to find an upper bound for the changes in the brightness that might be caused by a variation of the normals by an angle of in arbitrary direction. To end up this we consider the derivatives L d of the diffuse term and L of the specular term, where cos() =(nl)and cos =(nh):then the changes that can be caused by an angle are bounded by (max L () + max L ()) ; where min, <=<= max + and min, <= <= max + : As L d = L d cos() d k d d =,L d k d sin() is a monotonic function of the maximum de- Figure 13: Plot of the function cos 10 (). The derivative 1 has a maximum absolute value at p 19:1 10,1 : viation can be found at = max + where is the angle between n and l at the darkest (diffuse) point in the triangle. For simplicity we make the reasonable assumption that the curvature of the lines of constant intensity never change sign 4. Under this assumption the darkest point in the triangle will be one of its vertices. For the specular term things are more complicated as L s = L d cos s k n () cos() d d =,L s k n n sin() s is not cos() monotonic any more, see Figure 13. On the range 0 the maximum derivative occurs at 0 = arctan( p n,1 2 1 ). But this value is only of interest if min 0 max. In the other cases the maximum derivative occurs either at min 0 (brightest point) or 0 max (darkest point). Again, the darkest point is located at one of the vertices and can be used to determine if max 0 : To find the brightest point is more complicated. Therefore, we determine min using the cone of the normals and the cone of the halfway vectors of the triangle [6], see Figure 14. In both cases it is now easy to check if the changes of the brightness on the triangle are acceptable or not. 4 te that this is not the case, if we have multiple highlights on one Phong-interpolated surface. But this is only the case, if the viewing point is close to the focal point or close to a caustics and then only slight changes of the viewing position will change the illumination dramatically. Errors introduced by coarser triangulations because of this assumption are hardly noticeable, even in animations.

9 applied to produce optimally simplified surface models of glossy surfaces for ray- tracing with normal interpolation. Despite of the already available algorithms to check if Gouraud-interpolation can replace Phong-interpolation further work in this area is needed to handle all cases without artifacts. 5 Acknowledgements We wish to thank two of our anonymous reviewers, who provided us with exceptionally thorough reviews. Figure 14: The minimal angle min between the cone of the normals and the cone of halfway vectors deliver a bound for at the brightest point Is Gouraud interpolation sufficient After performing the Phong test, we know that the normals of the corresponding surface are sufficiently approximated by the interpolated normals. If we have a renderer that supports Phong interpolation we are done. In the other case we suppose that we have graphics hardware supporting Gouraud shading. w, the only task left is to determine, if the Phong interpolation can be adequately replaced by a simple Gouraud interpolation. This task has been investigated by [3, 11]. Although their algorithms do not cover all possible cases in our context the algorithm of Harrison et. al. [3] delivers acceptable results. 4 Conclusions and future work In this paper we describe an algorithm for lightingdependent refinement of multiresolution surface models. The given algorithm is not restricted to a special representation of the multiresolution model and can be used in combination with all currently available MRMs, as long as the normal deviations are calculated as described in the paper. It is the first algorithm that can be used to check if Phong-interpolation on a triangle is sufficient. Moreover, it can be used to control refinement in combination with environment mapping, which is demanded by the automobile industry in order to check the quality and smoothness of their surfaces. Last but not least, the algorithm can be References [1] A. Ciampalini, P. Cignoni, C. Montani, and R. Scopigno. Multiresolution decimation based on global error. Technical Report CNUCE: C96-021, Istituto per l Elaborazione dell Informazione - Consiglio Nazionale delle Ricerche, Pisa, ITALY, July [2] J. Cohen, A. Varshney, D. Manocha, G. Turk, H. Weber, P. Agarwal, F. P. Brooks, Jr., and W. Wright. Simplification envelopes. In H. Rushmeier, editor, SIG- GRAPH 96 Conference Proceedings, Annual Conference Series, pages ACM SIGGRAPH, Addison Wesley, Aug held in New Orleans, Louisiana, August [3] K. Harrison, D. A. P. Mitchell, and A. H. Watt. The H- test, a method of high speed interpolative shading. In N. Magnenat-Thalmann and D. Thalmann, editors, New Trends in Computer Graphics (Proceedings of CG International 88), pages Springer-Verlag, [4] H. Hoppe. Progressive meshes. In Computer Graphics Proceedings, Annual Conference Series, 1996 (ACM SIG- GRAPH 96 Proceedings), pages , [5] H. Hoppe. View-dependent refinement of progressive meshes. In Computer Graphics Proceedings, Annual Conference Series, 1997 (ACM SIGGRAPH 97 Proceedings), pages , [6] R. Klein. Multiresolution representations for surfaces meshes. Computers & Graphics, 22(1), Jan to appear. [7] R. Klein, D. Cohen-Or, and T. Hüttner. Incremental viewdependent multiresolution triangulation of terrain. In Pacific Graphics 97 Proceedings. IEEE Computer Society, IEEE Computer Society Press, [8] R. Klein and J. Krämer. Multiresolution representations for surface meshes. In W. Straßer, editor, Proceedings of

10 the Spring Conferece on Computer Graphics, pages 57 66, June held in Budmerice, Slovakia, June [9] R. Klein, G. Liebich, and W. Straßer. Mesh reduction with error control. In R. Yagel, editor, Visualization 96. ACM, vember [10] D. Luebke and C. Erikson. View-dependent simplification of arbitrary polygonal environments. In T. Whitted, editor, SIGGRAPH 97 Conference Proceedings, Annual Conference Series, pages ACM SIGGRAPH, Addison Wesley, Aug ISBN [11] J. Pöpsel and C. Hornung. Highlight shading: Lighting and shading in a PHIGS+/PEX-environment. Computers & Graphics, 14(1):55 64, [12] S. G. R. Klein. Data compression of multiresolution surfaces. In Visualization in Scientific Computing 98, pages Springer, May [13] R. Ronfard and J. Rossignac. Full-range approximation of triangulated polyhedra. Computer Graphics Forum, 15(3):C67 C76, C462, Sept [14] J. Rossignac and P. Borrel. Multi-resolution 3d approximation for rendering complex scences. In B. Falcidieno and T. L. Kunii, editors, Modeling in Computer Graphics: Methods and Applications, pages Springer Verlag, [15] W. J. Schroeder, J. A. Zarge, and W. E. Lorensen. Decimation of triangle meshes. In E. E. Catmull, editor, Computer Graphics (SIGGRAPH 92 Proceedings), volume 26, pages 65 70, July [16] G. Turk. Re-tiling polygonal surfaces. In E. E. Catmull, editor, Computer Graphics (SIGGRAPH 92 Proceedings), volume 26, pages 55 64, July [17] J. C. Xia and A. Varshney. Dynamic view-dependent simplification for polygonal models. In H. Rushmeier, editor, Visualization 96 Proceedings, volume 30(4), Aug