Automatic Instancing of Hierarchically Organized Objects

Automatic Instancing of Hierarchically Organized Objects R Schultz, H Schumann University of Rostock Abstract Instancing is an approved concept to save space for the storage of three dimensional scenes In this paper we propose an algorithm to automatically create instances from objects that are organized in a hierarchical scene graph We present a definition of instancing and derive an algorithm from that We discuss results of tests done with our implementation of the algorithm and give options for further development Keywords: efficient scene storage, instancing, automatic instancing 1 Introduction In order to reduce the memory consumption of complex scenes, which is an important aspect for rendering performance, different techniques may be used These are for example multiresolution techniques (see [6] and [7]), instancing [11], procedural objects (see [12], [1], and [5]), and tool objects [16] However, multiresolution techniques, procedural objects, and tool objects are not scope of this paper Instancing is an approved concept to save space that is not only used in computer graphics For scenes with many similar objects (eg architectural scenes or scenes with vegetation) it is desirable to save just the properties that are different between the similar objects and not to save eg the full geometric representation for each object Instancing realizes this desired behaviour Commercially available modelling and rendering packages (like Maya [19], 3D Studio Max [18], RenderMan [2]) offer this functionality The first use of instancing in computer graphics dates back to Sutherlands Sketchpad system that has been introduced 1963 [10] Sutherland defined the terms Master as an object which holds full geometric information and Instance as a transformed variant of the Master object In [3] and [4] automatic generation of instancing hierarchies for plant models was considered by J Hart and DeFanti They, additionally, extended the instancing paradigm from hierarchical structures to recursive (self similar) ones While Harts approach dealt with exact instancing, Fachbereich Informatik, Universität Rostock, A Einstein Str 21, 18051 Rostock, Germany; e-mail: rschultz@informatikuni-rostockde Deussen et al [11] used a new approach called approximate instancing to render large models of vegetation This approach is based on an observation by Smith [12] that the set of random numbers used in stochastic algorithms to create fractal mountains and particle-system trees can be reduced to a few representative values without affecting the perceived visual complexity of the generated images too much Deussen et al applied clustering algorithms to sets of vectors of real numbers (representing the parameters of procedural plant models) to find representative vectors (masters for instancing) Furthermore, Deussen points out that this approach can be stated in terms of vector quantization as used in colour image quantization algorithms [15] Snyder [13] and Kay [14] used the instancing paradigm to successfully render large models using ray tracing Instancing can also be used to improve radiosity rendering as was shown in [8] and [9] While Ouhyoung [9] shared precomputed geometric information between so called re-usable radiosity objects to speed up the calculation of form factors and visibility, Soler [8] proposed a hierarchical instancing technique that allowed radiosity solutions to be computed for complex objects such as entire trees While Deussen et al used automatic instantiation for special objects [11], the automatic instantiation of arbitrary objects is a currently unsolved problem In this paper we propose an algorithm to automatically create instances for static scenes with objects, that are organized in a scene graph For that, we define instancing and instantiation in the next two sections, then we show a general solution to the problem, present experimental results with our first implementation, and finally give some concluding remarks 2 Instancing In this section we will first discuss instancing in general and then give a formal definition of instancing The concept of instancing has been introduced in eg [16] We would like to point out, that the way we use the term instance conflicts with the terminology of object oriented programming, where an instance is a concrete representation of a class A better but seldom used term for the type of instancing in the computer graphics community would be referencing In the following sections we use the terms Master and Instance as follows: the Master object is an object which

U b R O b holds full information Instance objects just point to a Master object (and are therefore smaller as the Master object), share certain properties of the Master, but also have at least one property which differs from the Master Commonly, this differing property are affine transformations Instance objects can be used to save storage, speed up rendering, or to ease operations on objects that would normally have been done on many objects separately The amount of saved memory can be very high, but obviously, this depends highly on the scene If there are no objects that share certain properties in the scene one can hardly use instancing Properties can, for instance, be geometric information, textures or other material descriptions We will now give a more formal definition of instancing Let be a set of all objects of a scene Objects are eg CSG primitives, objects resulting from CSG operations, freeform curves and surfaces, procedural objects, and parameter objects of procedural objects A property vector! "$#% '&(*)+, is assigned to every $56'7/98:/9); -'0/213/4 object describing the properties $5 of Let the order of the properties for all objects in be uniform Consequently, for all object types the same property vector is in use >=?@BA We DCE < introduce a distance vector for two objects and : F G <' >=>H IJK >LJ # & J 5 NM6O P5 <H QP5 = for else 5 If the property J 5 is identical R < >=SJ 5 for the objects and is zero, otherwise is a value from the interval O LT The R value describes the amount of difference and is computed $5 differently depending on the type of the specific property To compute this value for certain geometric properties is a difficult task and is not solved in general For special cases, eg comparison of two spheres using their radii in normalized R A R coordinates, it is easier to compute a Given a value it makes sense to introduce a threshold vector 2V> V #W&, whose elements describe the tolerable difference of two objects with regard to a certain property This way, not only instancing of identical objects, but also V 5 of similar objects is possible The elements have either the value V X 5 or values from the interval Y O T A negative value for means that the corresponding property is not used for the comparison of two objects We further de- for the instancing of V 5 mand that at least one has to be X two objects, because all instances of a master have at least one property in which they < differ = We call two objects and instantiable if: Z V!5\[ J^5,/_V 5`a8\ 9L) O ] We define a function b for easier handling: <'c >=>H dm e < = else satisfy equation (2) (1) (2) (3) g= g=e'g/h1i/jh >= To handle all instances of a master object we introduce 1A kce the set f The elements of f are found as follows: Z l nm G >=>H 2 ] g=g f To avoid redundancies, we state D DqB 3 Automatic Instancing g=pò f In this section, we present an algorithm for automatic instancing for a hierarchically organized static scene (n- Tree) Automatic instantiation could eg be useful to improve rendering speed while rendering large scenes For example, scene files for film work with about 1GB per frame need to be distributed in rendering farms While introducing the instantiation formally, we abstracted from the fact, that the objects are given in a hierarchical structure (eg a CSG-tree or a scene graph) Our algorithm works on such a hierarchical scene description, that is given as a tree, a so called scene tree We identify the leafs of the scene tree as single objects and the nodes as grouping objects There are grouping objects that just group objects to inherit transformation attributes or objects that carry out CSG operations or other complex geometric operations (eg procedural objects) 31 An Algorithm for Automatic Instancing Our algorithm traverses the scene recursively and searches for every object (starting from the leafs of the scene graph) for instantiable objects If it finds instantiable objects, they will be converted to instances It does not try to instantiate already existing instances Moreover, it does not try to find instantiable objects for a grouping object among the children of this grouping object For a first approach, we demand that the scene does not contain any instances before our algorithm runs Our algorithm works as follows: (4) 1 Let the root object of a scene tree be the current object 2 If the current object has child objects that are not instances, put the current object onto the stack, make the first child that is not instance the current object, and go to 2) 3 Compare the current object with all other objects of the scene tree (except for the child objects of the current object, if there are any) The comparison of two objects itself is carried out by comparing the difference vector of the two objects with the threshold vector (see equation (2)) If the objects are instantiable, transform the second object to an instance of the current object

4 If there is another sister object of the current object, that is not an instance, make this object the current object and go to 2) If not, take an object from the stack, make it the current object and go to 2) If there are no objects on the stack the algorithm terminates 32 Automatic Instancing, Example In this section we will illustrate the work of the algorithm with an example Figure 1 shows a simple example of the Start: Step 1: order the nodes are processed Start: We start with a scene with 10 objects The objects A and D exist two and the object E three times, and A, B, and C are grouping objects Step 1: For the first object to be processed (D ) an instantiable object has been found in the second traversal (Dr ), which is then transformed to an instance Step 2: For the next object Es, the second traversal finds two instantiable objects (Et and Eu ), which are again transformed to instances Step 3: Now we process Av, which is a grouping object In this case Av and Aw are instantiable too, because of the child objects of Aw, which are instances of the child objects of Av The two instances that are child objects of Aw are deleted and Aw is transformed to an instance of A For all other candidate master objects (Bx, Fy, C {z ) no instantiable objects can be found and the algorithm terminates 33 Instantiating Grouping Objects Step 2: Step 3: Figure 1: Automatic Instancing, Example work of the algorithm At the top of figure 1 the initial scene tree of a scene is displayed Identical letters in the nodes denote instantiable (after equation (2)) objects The letter I is used to identify instance objects Asterisks show, which objects are instantiated in the respective steps of the algorithm The numbers beneath the nodes show in which In this section, we show how grouping objects may be instantiated Grouping objects may be put into two categories, those that expect a certain order of their child objects, and those that do not An examples for an object that expects a certain order is a grouping object that carries out a CSG difference An example for an object where the order does not matter is a group object that just inherits transformation attributes Since the shape of a grouping object does not only depend on its (eventually present) parameters but also on its child objects, for the comparison of two grouping objects normally all child objects would have to be compared too However, we found out that this is not necessary if we traverse the scene hierarchy as shown Because the algorithm traverses the scene starting from the leafs (objects without child objects), it is clear that if we process the parent object of some objects its child objects already have been compared by our algorithm Thus, if they are not already instances or there are no instances of them in the other part of the scene, we will not find a matching grouping object anyway This means, for all possible candidates of instances of a grouping object, we need to check for the following: 1 We first have to check, whether all child objects of the current object are instantiated objects or instances If they are not, the current group object is not instantiable The test can be done separately, checking reference counters and types of the child objects A better solution would be, to check a flag that has been set while

trying to instantiate the child objects The flag would be set if an object could not be instantiated 2 After that, we have to check, whether the child objects of the candidate group object are all instances of the child objects of the current object Or, if the child objects of the current group object are instances, we need to check, whether the child objects are instances of the same master objects If they are not, the current group object is not instantiable 3 Furthermore, we have to check the transformation attributes of the child objects of both objects If the attributes do not match, the current group object is not instantiable Note, that for grouping objects, where the exact order of child objects does matter, we have to do tests 1 and 2 in exact order for the child objects of both, current and candidate grouping object Furthermore, if the grouping objects have object type specific properties as eg procedural object parameters, they need to be compared also Note, that created instances may be deleted during the instantiation process of grouping objects 34 Complexity ) @ Our algorithm works on scene trees with objects, levels and instantiable grouping objects The complexity depends on all three values: } *),@W @; 2 It is clear, that for and O the complexity is } *) s However, if there are grouping @p~4 @,~9 objects ( ) or better instantiable grouping objects ( ), many comparisons of objects may be saved This is due to the fact, that the number of objects gets smaller as the algorithm creates instances from grouping objects Furthermore, instantiable objects for grouping objects are never searched for among the child objects of this object For example, while instancing the scene DOAP (see figure 6), which consists of ƒ >! objects, there were just comparisons of objects instead of ƒ > pˆ K Š ƒ > 35 Instantiating Scenes with already existing Instances In section 31 we demanded that no instances exist before our algorithm starts This is because already existing instances influence our algorithm in two ways: 1 Instances of instances might result: This occurs when already existing master objects are candidates for instances and are transformed to instances, but instances of instances might not be allowed by the target system 2 Grouping objects may not be instantiated: If instances and master objects are crosswise referencing (see figure 2) between the child objects of the candidate master and candidate instance grouping object, the grouping objects may not be instantiated A A E I I F Figure 2: Crosswise Instantiating Child Objects A solution to the first problem is to transform the already existing master object to an instance and redirect all existing instances to the new master object However, this redirection might be an expensive operation including a search through the complete scene, depending on the implementation method used for instancing To solve the second problem swapping of master objects to be child objects of the candidate master grouping object (possibly with the aforementioned high costs for redirecting existing instances) is necessary 4 Implementation We implemented the proposed algorithm for our modelling environment The Mops (see [17]), which is a free modeling environment for the RenderMan interface Object: next down Attributes RiAttributes Transform Tags Shaders refine Object Object object type specific data Figure 3: Object data structure of The Mops The Mops stores object properties in a hierarchically organized data structure In the top level of this hierarchy there are pointers to other objects in the scene tree, common attributes (eg type information, reference counter), RenderMan Interface specific attributes, as well as pointers to shaders, tags, and the object type specific properties (see figure 3) This is a lot of information and the reason, why instance objects do not save that much memory in the modeller as we would wish Due to this design of the data structures instance objects are simply not small enough Despite of this, we will show that considerable amounts of

X memory may be saved A different approach, which is utilized by scene graph APIs such as Java3D is to implement more specialized and smaller node types to build the scene graph Instance objects are realized by putting a pointer to another object (the Master) into the object type specific properties Master objects carry a reference counter to prevent their removal while instances exist We implemented two recursive functions to traverse the scene in The Mops The function TraverseTree is the outer traversal, that looks for potential master objects: 1 function TraverseTree(tree root, tree node): integer; 2 var num: integer; 3 begin 4 num := 0; 5 while exists(node) do 6 if exists(nodedown) then 7 TraverseTree(root, nodedown); 8 fi 9 if nodetype!= instance then 10 num := num + CreateInstances(node, 11 root); 12 fi 13 node := nodenext; 14 od; 15 TraverseTree := num; 16 end; The function CreateInstances is the inner traversal, that looks for candidate objects for instances, for an object ref : 1 function CreateInstances(tree ref, tree node): 2 integer; 3 var num: integer; 4 begin 5 num := 0; 6 while exists(node) do 7 if ref!= node then 8 if exists(nodedown) then 9 num := num + 10 CreateInstances(ref, nodedown); 11 fi; 12 if instantiable(ref, node) then 13 TransformToInstance(ref, node); 14 num := num + 1; 15 fi; 16 fi; 17 node := nodenext; 18 od; 19 CreateInstances := num; 20 end; This function uses two auxiliary functions, instantiable and TransformToInstance The function instantiable checks for the two objects ref and node, whether they are instantiable (see equation (2)) For that, it compares the types and the properties stored in the main object data structure (except for the transformation attributes) If they are identical it compares object type specific properties as well To compare the object type specific data structures, we store pointers to functions of proper type in a hash table This concept is used in many other places of The Mops as well New object types may easily profit from automatic instancing if the new object type specific compare functions are stored in this hash table Currently, the functions that compare the various properties do not calculate differences as suggested in equation (1), but return O if the properties to compare are identical or if they are not identical However, the comparison of RiAttributes, shaders and tags may be turned V 5 off globally in our implementation (the respective are set to X, otherwise they are always O ) TransformToInstance simply removes the object type specific data structures, tags, and shaders from the object to be turned into an instance (node) Then the type of the object is changed to instance, the refine pointer is set to point to the master (ref ), and the reference counter of the master is increased 5 Experimental Results Scene Cactus Spirit Swish DOAP Objects 310 277 223 474 Original Size 68475 90519 85970 228441 Inst created 243 208 183 381 Inst left 243 37 47 130 New Size 41976 33347 15855 48985 New Size in % 61 37 18 21 Table 1: Experimental Results from Automatic Instancing, see appendix A for rendered images of the scenes; Inst created is the total number of instance objects created during the instantiation process and Inst left is the number of instances that are finally left over in the scene after our algorithm finished Table 1 shows some experimental results, we achieved with our implementation of automatic instancing for our 5 modeling environment The Mops [17] No property V 5 A had been turned off for comparison in the instantiation (all, except for affine transformations) The memory savings differ considerably between the scenes, but this is nothing unexpected, as instancing is highly dependent on the scene The Cactus scene (figure 4) contains 310 objects, and 243 instances are created from them The final number of instances is not differing from the number of created in-

O stances, which means no grouping objects could be instantiated The prickles of the cactus are modelled using cones that are organized in two hierarchies of grouping objects Unfortunately the transformation attributes of the cones differ in the lowest level of the groups, so that the groups may not be instantiated We temporarily disabled the test of transformation attributes for instances and achieved the following results: 283 instances created, 43 left, new file size 15690 Bytes (23%) Of course, the scene changed a bit as can be seen in figure 4, but not that dramatically as one would expect With the next scene (Spirit, see figure 5) we achieved much better results (a reduction in size to 37%) This is because a lot of detail that has been modelled in this scene is in a radial motor (these details can not be seen clearly in the rendered image from the scene) Highly detailed identical cylinders and exhaust-pipes are present several times and instantiable as the number of instances shows The scene Swish (figure 5) shows three quite similar looking objects Two of them are indeed exactly the same, the third object is a bit different, but some parts of it may also be instantiated This leads to the best result of 18% Again, many grouping objects have been instantiated as may be derived from the difference of the number of created instances (183) and instances left (47) The last scene tested is DOAP (figure 6), which shows a lot of architectural details like columns, highly detailed capstones, and capitals The scene is very well organized in groups and may be instantiated to 21% of its original size In all examples, the algorithm instantiated the scene in far less than a second (eg Œ! for DOAP) on a SGI O2 (R10000, 225MHz) However, table 2 shows some re- (random) NURBS curves 500 1000 2000 Time (s) 16 63 31 Table 2: Results from instancing random NURBS curves sults we achieved, by trying our algorithm on a number of small (4 control points) NURBS curves that were put into a single level and were not instantiable, as their control vectors were filled with random numbers In this configuration (many not instantiable objects in a single level of a scene hierarchy), the algorithm has quadratic complexity, as pointed out in section 34 The results show clearly, that our algorithm is not applicable to large sets of polygons without preprocessing, as the execution times very fast raise to unacceptable levels It is clear, that if grouping objects are in use in a scene considerable memory savings may be achieved We further tested the impact of our instantiated scenes on rendering times in the RenderMan context (using rgl and rendrib from BMRT [20]) but achieved just very small differences in the rendering times This is, because both renderers, rendrib and rgl resolve instances while parsing and therefore do not profit from instances while rendering Moreover, because of the small size of the scenes we used the parsing time of our scenes had no impact on overall rendering time, even for rgl which uses OpenGL for rendering and could profit from small parsing times 6 Conclusions and Future Work In this paper we gave a formal definition of instancing We presented an algorithm for the automatic creation of instances in hierarchically organized scenes Experimental results showed that well organized (through the use of grouping objects) scenes may be automatically instantiated well and fast The number of necessary object comparisons is far below the worst case for the tested scenes Further work has to be done in the following areas: 1 Increasing the efficiency: Decrease the number of comparisons: Because of the order, in which we traverse the scene tree, we can limit the search for candidates for instancing to objects we did not visit in our traversal yet Faster comparisons using MD5 sums: If two objects are identified as candidates for instancing their internal data structures have to be compared This may be very time consuming in the case of complex and big objects, such as NURBS patches The time needed for the comparison could be reduced, if an MD5 checksum could be created over all data from an object A first test would check the MD5 sums, and only if they are identical we would need to check the data structures of the objects 2 Enhance the instantiation: Exclusion of single objects from the instantiation process: This is especially useful in conjunction with approximate instancing, because the user may identify certain objects as important and does not want them to be subject to approximate instancing Finer control of the comparisons in terms of approximate instancing: Currently, the user may just leave out tags, shaders or attributes from the comparisons in our implementation The use of thresholds for certain properties as used in equation (2) could improve that More flexible comparison of certain grouping objects: Our implementation currently does not take advantage of the fact that the order of the child objects for certain grouping objects does not influence the shape of the grouping object

Applicability: An automatic test could check the applicability of our algorithm based on, the existence of grouping objects, the relation between the number of grouping and non grouping objects, or Strahler numbers 3 Application in other fields: We implemented our algorithm just in the context of our own modelling system, however, application in other fields is worthwhile and should be tested Both, VRML and Java3D, support instancing and even instances of grouping objects but not automatic instancing The use of a scene graph instead of a scene tree leads to problems, because applications may interconnect object nodes and change object properties which might rely on a specific structure If these mechanisms are not in use, automatic instantiation for static geometry is applicable and should be tested Acknowledgements The authors would like to thank H Birkholz for the implementation of the instantiation and the reviewers for their constructive suggestions References [1] John M Snyder: Generative Modeling for Computer Graphics and CAD, Academic Press Inc, 1992 [2] Steve Upstill: The RenderMan Companion, Addison Wesley, 1989 [3] John C Hart, The Object Instancing Paradigm for Linear Fractal Modeling, Proceedings of Graphics Interfaces 1992, pp 224-231 [4] John C Hart, T A DeFanti: Efficient antialiased rendering of 3D-linear graftals, Computer Graphics (Proceedings of SIGGRAPH 1991) [5] JWeber, JPenn, Creation and Rendering of realistic Trees, Computer Graphics (Proceedings of SIG- GRAPH 1995) [6] H Hoppe, T DeRose, T Duchamp, J McDonald and W Stuetzle, Mesh Optimization, Computer Graphics (Proceedings of SIGGRAPH 1993) [7] M Eck, T DeRose, T Duchamp, H Hoppe, M Lounsberry and W Stuetzle, Multiresolution Analysis of arbitrary Meshes, Computer Graphics (Proceedings of SIGGRAPH 1995) [8] Cyril Soler, Francois Sillion: Hierarchical Instantiation for Radiosity, Eurographics Workshop on Rendering 2000 [9] M Ouhyoung, Y Chuang, R Liang: Reusable radiosity objects, Computer Graphics Forum 15(3):C347-C356, C483, Septemper 1996 [10] I E Sutherland: Sketchpad: A man-machine graphical communication system, Proceedings of the Spring Joint Computer Conference 1963 [11] O Deussen, P Hanrahan, B Lintermann, R Mech, M Pharr, P Prusinkiewicz: Realistic Modeling and Rendering of Plant Ecosystems, Computer Graphics (Proceedings of SIGGRAPH 1998) [12] A R Smith: Plants, fractals, and formal languages, Computer Graphics (Proceedings of SIGGRAPH 1984) [13] J M Snyder, A H Barr: Ray tracing complex models containing surface tesselations, Computer Graphics (Proceedings of SIGGRAPH 1987) [14] T L Kay, J T Kajiya: Ray tracing complex scenes, Computer Graphics (Proceedings of SIGGRAPH 1986) [15] P Heckbert: Color quantization for frame buffer display, Computer Graphics (Proceedings of SIG- GRAPH 1982) [16] R Schultz, H Schumann: Efficient Scene Descriptions Using Advanced Modelling Techniques in the RenderMan Context, Proceedings of Spring Conference on Computer Graphics 2000 [17] R Schultz, The Mops Home Page: http://wwwinformatikuni-rostock de/ rschultz/mops/ [18] 3D Studio Max R2 Reference Manual, Autodesk, 1998; Discreet: http://wwwdiscreetcom/ [19] Learning Maya, Alias Wavefront, 1998; Alias Wavefront: http://wwwaliaswavefrontcom/ [20] Blue Moon Rendering Tools (BMRT), Exluna: http://wwwexlunacom/

A Images Figure 4: Cactus, Cactus (prickles instantiated with approximative instancing) Figure 5: Swish (left), Spirit (right) Figure 6: Domestication Of A Pyramid (DOAP)