Configurable Instances of 3D Models for Declarative 3D in the Web

Transcription

1 Configurable Instances of 3D Models for Declarative 3D in the Web Felix Klein 1,2, Torsten Spieldenner 1,3, Kristian Sons 1,2,3, and Philipp Slusallek 1,2,3 1 Saarland University 2 Intel VCI 3 DFKI Figure 1: A Virtual World scene featuring several instances of the same robot character, configured to have individual poses and colors. Abstract The Declarative 3D for the Web initiative by the W3C [W3C 2011] connects 3D content to the Web document, intertwining it with other Web technologies known to millions of Web developers. The goal is to make 3D on the Web more accessible compared to lowlevel APIs such as WebGL. However, all proposals for Declarative 3D for the Web are missing an essential feature: configurable instances of structured 3D models. While instance mechanisms do exist, they all have limited capabilities to configure instances individually. In this paper we present a new approach for configurable instances of 3D models that is integrated into XML3D. Our approach comes with a compact interface, a powerful extension mechanism to handle configurations, and an efficient data structures for efficient instancing. We demonstrate how our instance mechanism simplifies the handling of 3D models in several different application areas, including Virtual Worlds, and provide several performance results for the instancing process. CR Categories: I.3.6 [Computer Graphics]: Methodology and Techniques Graphics data structures and data types I.3.6 [Computer Graphics]: Methodology and Techniques Languages Keywords: XML3D, instancing, assets, Web3D, WebGL klein@intel-vci.uni-saarland.de torsten.spieldenner@dfki.de kristian.sons@dfki.de slusallek@dfki.de 1 Introduction With the introduction of WebGL [Khronos Group 2011] the Web received a powerful technology for the creation of complex and plugin free 3D applications. WebGL is a low-level graphics API that provides much control over hardware details and therefore covers the needs of many potential 3D applications. However, the lowlevel design of WebGL also differs significantly from other Web Technologies. 3D content created with WebGL is disconnected from the Document Object Model (DOM) of the Web page and therefore detached from the DOM event model, Cascading Style Sheets (CSS), and other common Web technologies. As a result, WebGL is not a technology that is easy to learn and use for Web developers. In contrast to WebGL, the Declarative 3D for the Web initiative aims for a better integration of 3D graphics into Web technologies, by making 3D content part of the Web document. Since such a declaration of 3D content is comparable to HTML and compatible with many other Web technologies, it is potentially easier to pick up for Web developers and non-experts in computer graphics in general. Most 3D scenes are composed out of a number of 3D models, whereas individual models may be reused several times. In most cases, those 3D models are structured, containing several transformed meshes with attached materials. Further, structured 3D models may embed a diverse set of additional features such as dynamic meshes, skeleton animation, or several levels of detail (LOD). An essential feature every 3D technology has to provide is an instance mechanism for structured 3D models. With instancing, the same 3D model can be placed multiple times within the scene without duplicating its data. This does not only enhance performance by avoiding the duplication of hardware buffers, it also provides encapsulation. When instancing a 3D model, the model itself can be referenced as a black box: Authors do not need knowledge about the model s internal structure. However, the abstraction of a black box alone is not sufficient, because often it is required to have an interface to configure each instance individually, e.g., to change surface shading properties or the current pose of the animation. Many types of 3D applications make heavy use of configurable instances of 3D models. Examples range from tools for collaborative

2 3D content creation over visualization tools to Virtual World [Dionisio et al. 2013] and game scenarios. Workflow in content creation often consists of selecting a predefined 3D model, placing it into the scene, and then modifying the configuration of each instance. Applications for 3D traffic flow or crowd behavior visualization may employ a large number of potentially animated objects to represent individuals in these simulations. Virtual Worlds as known from popular applications such as Second Life or massively multiplayer online games such as World of Warcraft or Eve Online 1 are usually populated by objects that are rendered from the same 3D model, but need to be treated individually, e.g., for individual animation states, colors, or equipment. Despite its high relevance for 3D applications, configurable instances of 3D models remain a poorly supported feature among the current proposals of Declarative 3D for the Web. X3DOM [Behr et al. 2009] does support the instancing of 3D models with its DE- F/USE mechanism, effectively blackboxing their content, but lacks the option to configure instances individually. XML3D [Sons et al. 2010] only supports the reuse of generic data and shaders and therefore requires the replication of the 3D model s structure for each instance. In this paper we propose a new mechanism for configurable instances of 3D models for Declarative 3D in the Web. This mechanism is integrated into XML3D and leverages the data composition and processing capabilities of Xflow [Klein et al. 2012a]. Our approach supports the instancing of structured 3D models with a compact interface, provides a powerful extension mechanism to configure the content of each instance, and makes use of a data structure that allows for fast and efficient instancing. The paper is structured as follows: First, we explain different kinds of instancing approaches in Section 2, followed by an explanation on how the current XML3D version handles 3D models in Section 3. Afterwards we explain the design and interface of our instance mechanism in more detail in Section 4, compare it to other approaches in Section 5, and give a few insights about the implementation and optimizations in Section 6. Following that, we present several use-cases that make use of our new instance mechanism in Section 7. Finally, we will present several performance results in Section 8, followed by Conclusion and Future Work. 2 Related Work In computer graphics the reuse of resources can be achieved through many different techniques. Reasons for the the reuse of graphics resources include memory and runtime performance, reduction of overhead in the rendering pipeline, scene management, and rapid creation of complex scenes using encapsulation of functionality. In the following, we shortly discuss several techniques that allow for reuse of graphics resources and are relevant for declarative 3D in the browser. Geometry Instancing Geometry instancing is a technique that enables efficient rendering of multiple mesh instances with differentiating shader parameters. This technique is a batch processing optimization that reduces the overhead that comes with activating the differentiating shader parameters one by one and doing the draw calls separately. Geometry instancing is a concept orthogonal to other instancing techniques and can be used by every rendering system capable of identifying copies of meshes. Geometry instancing is available in WebGL through the ANGLE_instanced_arrays extension. It was first introduced in Direct3D 9 [Microsoft 2003]. Nowadays several similar but more flexible approaches with even less driver overhead exist, e.g., glmultidrawarraysindirect in OpenGL 4.3 [Khronos Group 2012b]. Data Level Instancing The next level of reuse is based on data blocks. This technique reduces memory consumption. In the best case, graphics data is stored in buffers in the graphics card s local memory and only referred by the application, possibly by multiple objects. X3D [Web3D Consortium 2011] supports the reuse of data with its DEF/USE mechanism. Compatible are all nodes that represent data entries such as <Coordinate>, <Normal>, or <Color>, as well as arbitrary vertex attributes via <X3DVertexAttributeNode>. Some X3D geometry nodes such as the commonly used <IndexFaceSet> allows multiple indices (position, normal and color indices) and per face colors and normals, which are all features not supported by recent graphics APIs. As a result, the data entries need to be pre-processed before being stored on the GPU, which makes it difficult to share GPU buffers. Only one specific configuration of IndexedTriangle- Set can be directly mapped to GPU buffers. XML3D on the other hand provides fine-granular data compositing based on generic <data> elements. If used as vertex attributes of a mesh, these entries directly correspond to OpenGL Vertex Buffer Objects (VBOs). The set of vertex attributes can then be packed into Vertex Array Objects (VAOs) and activated in a single API call to reduce overhead. In contrast to X3D s DEF/USE mechanism, a <data> element can also refer to resources contained in external documents. In that case, there is no interface to change the content of the external resource. This knowledge can be exploited to optimize the data structures internally. With the data composition model it is possible to compose mesh data from internal and external resources, and to override shader attributes from mesh instances. Sub Graph Instancing X3D supports the reuse of arbitrary subscene graphs using the DEF/USE mechanism. This extends the data level instancing, because it includes structure, transformations, materials, and even nodes with a higher abstraction level, e.g., humaniod animation nodes. Although this is an established mechanism in scene graphs, it comes with a number of issues in particular in the DOM context. At first, the scene structure represented in the DOM tree becomes a Directed Acyclic Graph (DAG). Apart from the fact that the complexity of CSS inheritance rules increases a lot, the main issue of a DAG is that picked objects do not map to a single DOM element and can only be differentiated by examining the path of picked objects. This does not match with the DOM event system, and is therefore not compatible with libraries build on top of DOM events. Moreover, it is a concept web developers are required to learn. In addition to the issues that come with sub graph instancing, the usefulness of this concept is very limited, because the instances can only be modified collectively. This makes sub graph instancing useless for all the many cases the instances shall be configured (e.g., animated) individually. Higher-Level Instancing A logically separated unit can also include behavior, i.e. intrinsic scripting and events. The idea of these higher-level instancing is to be able to define modular objects (components) in the sense of reusable building blocks. As long as the interface does not change, the component and the instancing scene can be modified independently. Several approaches exist that provide instancing capabilities based on components. In HTML, a well-known way to inline existing

3 HTML documents is the<iframe>. The referred page has its own run-time, the parent document has access to the page within the iframe and vice versa. A similar concept for 3D scenes provides the X3D Inline node. To address objects from the inlined scene, they need to be explicitly exported from the inlined scene and imported using the X3D IMPORT statement. However, this technique can only be used to connected events to the inlined scene, but not to configure its content. Another class of component models can be considered as facade pattern. X3D provides Prototypes to encapsulate functionality behind a facade. A Prototype is a scene in a separate namespace. All addressable objects are explicitly exposed by the Prototype s interface. Using X3D s routing concept, one can communicate and modify a Prototype instance. The concept is quite powerful, but not implemented in X3DOM. The XML Binding Language (XBL) [W3C 2012] is a facade pattern designed specifically for the DOM technology stack, but has never been implemented in any browser and never got beyond recommendation status in the W3C standardization process. Instead, it was replaced by Web Components [W3C 2013], which is a collection of technologies in order to achieve reusable components. Overall, higher-level instancing provides a good abstraction for arbitrary complex content and may even support configuration via compact interfaces. However, the complexity of these approaches makes them inherently inefficient to instantiate. Instead, implementations involve the replication of instanced structures, which is slow and memory consuming. 3D Model Formats There exist an abundance of individual 3D model formats that can be considered for instancing. However, only few of them are designed to be independent of a specific engine or API. Relevant examples for the Web include COLLADA [Khronos Group 2008] and gltf [Khronos Group 2012a]. COLLADA is a format intended for the interchange of 3D assets between 3D modelling programs and therefore not optimized for runtime delivery. In contrast, gltf is deliberately designed for runtime delivery of 3D assets, featuring a hardware-optimized description of 3D content with a combination of a JSON file and binary buffers. While these kind of 3D model formats provide a detailed definition of content, they do not define any interface to access and configure this content for each instance. Thus, by themselves these 3D model formats do not provide a solution for instantiation, as we still need to define how exactly they can be configured and placed into the scene. However, ideally an instantiation mechanism should be capable of supporting these kind of formats as possible input data. Instancing in the 3D Web For DOM-based 3D graphics an instancing mechanism is required, which is not API specific, allows for flexible configuration of instances but is still simple enough to be stored and instanced efficiently. Currently there is a large gap between efficient instancing of sub graphs, which provide no means for configurations, and full-fledged component models, which are memory consuming and inefficient to instantiate. Our approach tries to find a good trade-off between both approaches, by deliberately choosing simplified structures for instancing, focusing on the most relevant aspects of 3D models: structure, transformations, shading, geometry and animations. 3 3D Models in XML3D XML3D [Sons et al. 2010] is a minimal extension to HTML5 for interactive 3D content. It defines a small set of new elements to de- Figure 2: XML3D element connection for 3D models. Dashed lines are references via document id. scribe a scene graph with 3D geometry, surface shading, and lighting. In addition, it features the declaration of generic data structures as well as the declarative language Xflow [Klein et al. 2012a] used to perform processing on this data. Thanks to its generic design and its integration with existing Web standards, XML3D supports a wide range of 3D related features, such as skinned mesh animations and Augmented Reality [Klein et al. 2012b], without adding a large amount of new DOM elements. However, one major downside of XML3D is its lack of an instance mechanism for structured 3D models. While XML3D does provide the means to reuse generic data and shaders, it requires users to duplicate the entire structure of instanced 3D models. Our goal is to find a new approach to 3D model instancing that integrates well with the existing concepts of XML3D. Thus, we first have a look on how 3D models are currently handled in XML3D. We will highlight what elements are required to describe 3D models, how they are handled when instancing the model, and how they need to be modified to configure each instance. 3.1 Instancing of 3D models Structured 3D models in XML3D are declared using a combination of <mesh>, <group>, <shader>, and <transform> elements (see Figure 2). It is possible to assign <shader> and <transform> element to several <group> elements via a reference to their (local) URL and their document id. In addition, generic buffer data can be declared in separate <data> elements and shared among several meshes via external URL based references or internal references based on document ids. The first limitation becomes apparent when the user plans to instance the whole content of a <group>. To do so, it is required to duplicate each and every <group> and <mesh> node in the sub tree. On top of that, elements shared via documents ids need to be specially treated in two possible ways: Either they are shared among all instances and must not be duplicated, or they are duplicated, which makes it mandatory to adapt their document id and related references as such ids must be unique inside the document. It is possible to structure XML3D content in a way to simplify the instancing process, which involves extracting all shared content into external files such that the remaining content only includes external references and can be copied without further modifications. However, all XML3D content without this clean separation will be hard to instance. Overall, the explicit duplication of the 3D models structure is inconvenient for simple instancing.

4 4.1 Basic assets definition and instancing We introduce the concept of assets, a structure that describes a complete 3D model with transformed geometry, shading, and dataflow processing. A basic asset with a number of meshes is defined using the <asset> and <assetmesh> elements: Figure 3: XML3D element connection for 3D models with embedded dataflow processing. Dashed lines are references via document id. <asset id="exampleasset"> <assetmesh shader="#woodshader" > <!-- Mesh buffers declared here --> <assetmesh shader="#ironshader" transform="#xfma" > <!-- Mesh buffers declared here --> 3.2 Configuration of 3D models As previously discussed, many 3D applications require the configuration of 3D model instances beyond basic placement into the scene. One example is the configuration of shading properties. Usually, we only want to adapt the shading of specific parts of the 3D model. This can be achieved in two ways: Firstly, we can assign a new shader to an internal <group> element to change the shading for all meshes in its sub tree. Secondly, we can add shading parameters to the data content of a <mesh> element, which will overwrite the parameters of the attached <shader> element. Another important use-case are dynamic 3D models (e.g., models with mesh animations). Dynamic 3D models are created in XML3D using a combination of <data> elements and Xflow dataflows. Figure 3 shows a common example on how a skinned 3D model with skeletal animation is composed in XML3D. Note, that we have one <data> block with shared mesh data that is transformed by an Xflow dataflow. To drive the skeletal animation, we need to continuously modify a key value (e.g., time) inside that data block via JavaScript. Overall, it becomes clear that any kind of configuration requires a modification somewhere inside the structure of the 3D model. Thus, we need to be able to access the structure at least to some extent. 4 Design Our new instance mechanism for XML3D should provide the following: (1) an abstraction over the structure of the 3D model, (2) the option to configure the 3D model without exposing its full structure, and (3) a data structure optimized for a fast instancing process to reduce load times for large scenes. As we discussed in related work, a higher-level instancing approach such as X3D Prototypes would fulfill the first two requirements, but has the disadvantage of being less efficient to instance by effectively replicating the content for each instance, Since our goal is to have a flexible yet efficient mechanism for instancing, we choose a more simplified data structures, which includes a list of transformed meshes connected to shaders and dataflows. However, we combine this simplified structure with a powerful and generic configuration mechanism, which makes this approach applicable to many use-cases. The <assetmesh> element includes all attributes of a regular <mesh> element. In addition, shaders and transformations can be attached to each <assetmesh> directly. An asset can be instanced with the <model> element, which is placed inside the scene graph and refers to an internal or external asset via the src attribute: <group style="transform: translatex(5px)" > <model src="assets.xml#exampleasset" ></model> </group> Consequently, all meshes declared in the asset will be visible in the scene graph, correctly transformed, and with shaders attached. 4.2 Assets with shared data Commonly, 3D models are composed in a way to maximise the reuse of data. A common structure is one set of vertex buffers shared by several meshes with individual index buffers. For our asset declaration we can share buffers by using already existing data composition techniques of XML3D and Xflow: <data id="sharedattribs" > <!-- Shared vertex buffers --> </data> <asset id="sharedexampleasset"> <assetmesh shader="#wood" > <data src="#sharedattribs" /> <int name="index" > </int> <assetmesh shader="#iron" > <data src="#sharedattribs" /> <int name="index" > </int> However, in addition to these techniques we introduce a new data sharing concept for assets with the <assetdata> element and the name and includes attribute:

5 <asset id="sharedexampleasset2"> <assetdata name="shared" > <!-- Shared vertex buffers --> <assetmesh shader="#wood" includes="shared" > <int name="index" > </int> <assetmesh shader="#iron" includes="shared" > <int name="index" > </int> In this example, the content of <assetdata> is shared among the two declared <assetmesh> elements. It is also possible to refer multiple <assetdata> elements by having white space separated names in the includes attribute. For these basic examples, the two alternatives for data sharing behave exactly the same way and might seem redundant. However, one advantage of the name-based connections is, that they reside inside a name space enclosed by the asset, while document ids are in one name space for the whole document. Thus, with names we have the option to define several assets in the same document without the risk of having name collisions. In addition, there is an important difference in the semantics of the two connection methods, which is related to another important feature of assets: extensibility. 4.3 Extension of assets The concepts presented so far already allow for a compact definition and instancing of static 3D models. However, in order to support configurable 3D models as discussed in Section 3.2, we need to be able to modify our 3D model instances in certain ways, e.g., to set the key value of the animation. To support configuration, we add a powerful extension mechanism to our <asset> elements. An <asset> element can extend from another <asset> referred to by the src attribute. Consequently, all declared <assetdata> and <assetmesh> elements will extend equallynamed entries of the source assets. In general, this enables us to extend the generic data content of asset entries. For an <assetmesh> entry, we can also overwrite the attached shader or transformation attributes. In the following example, we extend the #sharedexampleasset2 asset of the previous subsection by adding a new color attribute to the shared vertex buffers: <asset src="#sharedexampleasset2"> <assetdata name="shared" > <float3 name="color" >...</float3> The original source asset had all <assetmesh> elements referring to the shared <assetdata> element via the includes attribute. Since we now extend the shared <assetdata>, the new color buffer will be assigned to all meshes. Note, that this behavior is due to the semantic of the connection via the includes attribute, which always refers to the fully resolved version of the asset entry. This means that, before we connect an asset entry to another via includes, we first recursively resolve its own includes-connections and extension chain. Finally, the src attribute of <model> can be used to extend the referred asset as well with exactly the same syntax. Thus, we can extend and configure an asset from within its instance declaration: <group style="transform: translatex(5px)" > <model src="assets.xml#sharedexampleasset2"> <assetdata name="shared" > <float3 name="color" >...</float3> </model> </group> Overall, this extension approach enables the enclosure of complex data connections within assets without losing the option to extend individual data entries and meshes, e.g., to give each avatar a different look. Instead of declaring individual buffers, one can also reference a whole data block (possibly external) that includes all configuration values packed together. 4.4 Dataflows in assets As with other XML3D elements, such as <data>, we support the use of the compute attribute for <assetdata> and <assetmesh>. With the compute attribute we can attach Xflow operators or dataflows for any kind of generic data processing, e.g., for mesh animations, LOD and other features. For example, this enables us to embed skeletal animations within an asset: <asset id="animatedasset"> <assetdata name="shared" compute="dataflows[ #skin ]"> <data src="#sharedvertexbuffers" /> <data src="#keyframeanimationdata" /> <float name="key" >0</float> <assetmesh shader="#woodshader" includes="shared" > <int name="index" > </int> <! > The skin dataflow attached to the the shared <assetdata> entry performs the skeletal animation based on the key frames provided by <data src="#keyframeanimationdata"/>. The currently visible pose of the animation is determined by the key input value. Note, that the skin dataflow is executed on the fully resolved version of the <assetdata> entry. Thus, we can modify input arguments of the dataflow, such as the key value, even within the very last extension inside the <model> element: <model src="assets.xml#animatedasset"> <assetdata name="shared" > <!-- Replace the key frames of the animation --> <data src="#otherkeyframedata" /> <!-- overwrite the previously declared key --> <float name="key" >3.7</float> </model> Overall, the combination of the dataflow processing capabilities of Xflow and the new asset extension mechanism allows for an instancing of complex animated 3D models with a minimal yet flexible interface for configurations. 5 Comparison to X3D Prototypes After presenting the design of our new instance mechanism, we compare it now to another high-level instancing approach of X3D: Prototypes. X3D Prototypes support the instancing of arbitrary

6 scene graph content with plenty of options for configuration. Our instance mechanism supports a similar degree of configuration, but is limited to structured 3D models. On the other hand, we support the attachment of dataflows to arbitrary parts of the 3D model, such as mesh data, shader parameters and transformations. This feature can be used in many ways to simulate functionality of Prototypes. For instance, we can simulate modifications in a transformation hierarchy by processing transformations in a dataflow and connecting the result to mesh entries. In general, we can compose our dataflow to provide the most compact interface for the desired functionality. For example, if we want to be able to rotate a wheel we accept a float value specifying the angle around a fixed axis, computing the matching transformation within the dataflow. Another limitation compared to Prototypes is the lack of embedded Sensors and Routing nodes. This is because XML3D doesn t define these kind of elements, leaving all interactive functionality to JavaScript. Consequently, our instance mechanism is a perfect fit, providing a compact, yet flexible interface for JavaScript programs to control any aspect of the instance. Figure 4: The mapping of a <model> definition to an Xflow graph. The order in which data entries are connected determines how duplicated data entries are overwritten. Thus, data entries are connected such that mesh data overwrites included data and extended data overwrites original data. In contrast to Prototypes, we do not support the instancing of view points and light sources. The instancing of view points is usually not relevant and in fact may result in strange behavior with respect to the selection of the active view point. The instancing of light sources may be more relevant and our instance mechanism might be extended to support them in a later version. Up to this point, however, light sources are usually carefully placed independent of 3D models to achieve a good lighting without putting too much stress on the performance. Finally, we have the advantage that our <asset> elements can be referred easily from external files. In contrast, for external Prototypes the user has to replicate the interface of the Prototype in the main document which is error prone. 6 Implementation We extended the xml3d.js [Sons et al. 2013] library to support our new instance mechanism, focusing on an efficient instancing process with little memory overhead. Our past experiences have shown that a great number of DOM elements will result in a large memory footprint and prolonged load times, even for structured content without embedded vertex buffer data. Fortunately, the design of our <asset> structures allows for an instancing process that doesn t require the replication of DOM elements. Instead, we map each instance to light-weight rendering structures, which were introduced in recent versions of xml3d.js and are separated from the DOM. For instance, just as each <mesh> element is mapped to a RenderObject and each <group> to a RenderGroup, a single <model> element is mapped to a whole sub graph of these two structures. In addition to the optimized creation of new instances, we also need an efficient process to update the configuration of each instance. This is critical e.g., for animated characters where the pose is modified each frame. Consequently, modifying the generic data content of an <assetmesh> or <assetdata> entry should quickly result in an update of the rendering structures of all connected <model> instances. However, with the options of extending asset entries and connecting them implicitly with the includes attribute, a <model> instance may contain a complex connection of generic data that needs to be considered during updates. We efficiently handle such updates by mapping these data connections to an internal Xflow graph. We structure this graph such that it correctly combines all the generic Figure 5: Collaborative content creation in a 3D Web-application (left), instanced user avatars meeting in a Virtual World (right). data of the asset entries according to their extension and connection properties. See Figure 4 for an example. With this approach, consequent data updates inside asset entries are automatically propagated by Xflow, which is designed to efficiently handle dataflow modifications. 7 Use-Cases In the following, we will discuss some use-cases in more detail and show how they profit from the simple and intuitive scene description introduced by our instancing implementation. 7.1 Collaborative Content Creation An type of application that became more and more popular in the scope of 3D Web-applications during the last years is collaborative 3D content creation. For example, DiFac [Sacco et al. 2007] showed how the design of industrial production sites benefits from virtual design and evaluation. Smparounis et al. [Smparounis et al. 2009] presented a browser-based solution of a collaborative prototype designer. Zinnikus et. al implemented a browser-based solution for factory design and evaluation, using declarative 3D for design and simulation [Zinnikus et al. 2013]. In these applications, a common step in the work flow is to choose a 3D model from a set of provided models as building block of the final 3D scene, and create an instance of the respective asset in the scene, comparable to recent game level editors. Obviously, this kind of use-case profits significantly from our improvement in XML3D. Placing an asset in the application then corresponds to nothing but creating a <model> node in the DOM. This in turn

7 means that the DOM structure reflects the actual scene content very well. Each <model> tag corresponds to one building block in the scene, other than before, when each building block introduced a full copy of a possibly complex declarative geometry representation. Our implementation of instancing in addition allows for custom changes to individual instances. Instances can for example be scaled individually or receive a distinct color by overriding the respective entry in the exposed <assetdata>. Figure 5 (left) shows an example of a collaborative construction site planning tool. Each of the containers, trucks, or cranes are instances of a shared asset definition. Although the underlying geometry may be rather complex, each object can be instanced by just a single <model> node. 7.2 Customizable User Avatars for Virtual Worlds There are also investigations of how to transfer Virtual World applications like Second Life to the Web browser: Byelozyorov et al. [Byelozyorov et al. 2012] presented an application that creates virtual versions of real world cities using a declarative 3D representation for the graphics. Dahl et al. [Dahl et al. 2013] presented a browser-based Virtual World Web-client that uses three.js for rendering. In such a Virtual World scenario, users who explore the world are usually represented as 3D avatars. As the number of connected users in a virtual environment can range from a few dozen to several hundreds [Dionisio et al. 2013], it is obvious that instancing geometry for user avatars is highly desirable. In addition, the instanced geometry needs to be customizable, not only concerning details that may vary between different users, but also considering individual poses of objects. With our approach, it is easy to re-use geometry for a number of avatars and still allow for various visual variations. We can for example override <assetdata> nodes to provide an interface for changing the color that is used by a certain shader (see Figure 5, bottom). Apart from configuring the color of skin or hair, users may also change the appearance of the clothing of their avatar by switching to different textures. Moreover, by exposing <assetmesh> nodes that contain references to mesh <data>, we can also change actual geometry that is part of the avatar mesh, as for example carried equipment in a gaming scenario. By exposing individual animation keys per avatar model, the different instances can be animated individually. This is obviously a strict requirement for a Virtual World scenario, as users also perform individual actions and interactions with the world. For this, we need means to display the interactions by selecting the respective avatar animations and display them independent of other entities. Instancing a customized avatar model could for example look like this: <model src="avatar.xml#asset"> <assetdata name="walkingkey"> <float name="key" >0.3</float> <assetmesh shader="sword.xml#shd" includes="handxfm"> <data src="sword.xml#mesh" /> </model> Using this code, the model would be capable of animation (by exposing the respective animation key) and have the option to change the object it is carrying in its hand by replacing the respective <assetmesh> entry. Including the handxfm entry provides the transformation to align the geometry with the hand s position and orientation of the current pose. Figure 6: Test assets used for performance measurement (from left to right): Fire Truck, Natalie, Sniper (model courtesy of Valve Software) 8 Performance Results In the previous section, we have already shown how different usecases may profit from the simpler scene description. However, the resulting smaller size of the DOM tree and the fewer number of DOM nodes that have to be inserted compared to copying the geometry representation should also influence both memory consumption and page load times. To verify this assumption and estimate the benefit of instancing compared to the previous method of copying the complete DOM representation of the instanced objects to the Web page, we load various instances of the same model, first with the previous approach, second with our new instancing method. We use 3 different models for this (see also Figure 6): Fire Truck, which is a static model that consists of 41 nodes for geometry definition. This rather complex representation results from various different shaders that all need to be assigned to distinct group nodes. However, as the geometry is completely static, and thus no animation keys need to be exposed, we can instance a Fire Truck asset with just one <model> node. Natalie, which consists of only 11 nodes for geometry definition and one <float> node as animation key. Thus we can instance Natalie with one <model> node that contains one <assetdata> node containing the animation key. Sniper, which is animated and consists of 29 nodes for geometry definition and one <float> node as animation key. With our instancing approach, we also need only one <model> and one <assetdata>, as for Natalie as well. The generated scene that consists only of the instanced assets is created on a server in an abstract representation. We connect to the server with an XML3D-based client. Upon receiving information about which assets are present in the remote scene, the client retrieves the XML3D representation of the models and adds them dynamically to the DOM, first by the old approach of copying the complete representation for each of them, second by using the presented instancing approach. This client also adds an enclosing group node to each instance to allow for individual positioning and orientation. This behavior corresponds to what one would expect in the use-cases described in Section 7. Measurements are done in Google Chrome Web Browser. Assets and test application are hosted on the same machine. This eliminates delays in the measurement from retrieving geometry definition from a remote location and allows for a better comparison of

8 DOM Nodes Memory Peak Page load time FireTruck Copy Instance Copy Instance Copy Instance Speedup 50 Inst MB 50 MB 2,36 s 0,285 s 87.9 % 500 Inst MB MB 79,3 s 5,15 s 93.5 % 1000 Inst N/A 178 MB N/A 19,0 s N/A Natalie Copy Instance Copy Instance Copy Instance Speedup 50 Inst MB 77 MB 1,34 s 0,44 s 67.1 % 500 Inst MB 235 MB 59,1 s 7,2 s 87.8 % 1000 Inst MB 384 MB 224,1 s 31,8 s 85.8 % Sniper Copy Instance Copy Instance Copy Instance Speedup 50 Inst MB 74 MB 3,2 s 0,43 s 86.5 % 500 Inst MB 241 MB 164 s 7,7 s 95.3 % 1000 Inst N/A 381 MB N/A 31,4 s N/A Table 1: Page size and load times for different scene complexity. The new approach keeps the DOM significantly smaller, which results in a speedup of page load times of at least 85 % in most cases. For the complex models of Fire Truck and Sniper, creating copies was not possible by cloning geometry due to process memory limitations of the Chrome browser. Natalie Copy Instance 5 Inst. 60 fps 60 fps 10 Inst. 39 fps 41 fps 20 Inst. 21 fps 23 fps Sniper 5 Inst. 60 fps 60 fps 10 Inst. 36 fps 37 fps 20 Inst. 19 fps 21 fps Table 2: Render frame rate for Xflow animations for copied geometry compared to instanced geometry: The new approach does not introduce a bottleneck in Xflow processing load times, as they are independent of the file size of the different asset definition files. Thus, the page load times we compare result only from DOM construction when building the scene dynamically. The respective resulting DOM sizes, peak memory usage reached by the browser process, and page load times are listed in Table 1. We can see that page load times decrease drastically when switching from copying to instancing geometry. The impact is largest for the complex, but static Fire Truck Model. All of the complex geometry can be hidden in the external asset definition from which we can create an instance with just one <model> node (plus one <group> node for individual positioning). Also complex animated models, like Team Fortress Sniper, profit tremendously. As we cannot reduce the number of nodes for Natalie much, the effect is lowest here. For Fire Truck and Sniper, we hit the browser tab process memory limit when trying to create 1000 copies of the model with the old approach 2. In these tests, the browser process crashed completely. Thus, the approach of actual instancing the models does not only result in a remarkable speed-up, but in the end also allows to display much larger scenes. Instancing should not influence existing features of XML3D. Especially Xflow data processing that is used for example for animating skinned characters should not suffer from dealing with the new format. Skinning operators in Xflow are entirely computed in JavaScript, thus evaluation needs to be efficient. To ensure that instancing does not introduce a bottleneck here, we have compared the frame rate for different numbers of instances, both when copying the geometry representation and when using the new instancing feature. The results are listed in Table 2. The frame rates are comparable for both approaches, thus our implementation of instancing does not introduce computational overhead. 2 The issue of limited browser tab process memory is discussed at 9 Conclusion and Future Work In this paper, we introduced a new mechanism for configurable instances of 3D models in the area of Declarative 3D for the Web. With the addition of merely 4 new types of DOM elements, the mechanism provides a compact interface to instance arbitrary 3D models with embedded dataflow processing for mesh animations and other features. The powerful extension mechanism allows for a flexible and detailed configuration of each instance, be it to specify the current pose of the animation, replace shading parameters, or to replace entire shaders or parts of the geometry. Additionally, we demonstrated how the flat structure of our assets simplifies the efficient implementation of the instancing process. Finally, we demonstrate how our new mechanism can be used in a range of use-cases and how it decreases memory usage and shortens page load times. Of course, there still remain use-cases that collide with certain limitations in our instancing interface. One limitation of the current implementation is lacking functionality to merge two independent assets. While it is possible to extend an <asset> element in arbitrary ways, e.g., by listing more <assetmesh> elements, it is not possible to merge all <assetmesh> entries of two or more <asset> elements. To tackle this limitation, we plan to integrate a concept of nesting <asset> elements within each other. This concept will also include an option to expose configuration properties of nested <asset> elements to keep the merged asset configurable as a whole. For instance we could place the asset of a propeller hat inside the asset of the person and expose the configuration value to rotate the propeller of the hat to the final asset. 10 Acknowledgement The character models of the robots were provided courtesy of Disney Research Zurich. The character model of Team Fortress Sniper was provided courtesy of Valve Software. The research presented in this paper has been generously supported by the Intel Visual Computing Institute, FI-CONTENT, FI-

9 WARE/FI-Core, and FITMAN, which are part of the EU Future Internet PPP program, and the EU FP7 project VERVE. References BEHR, J., ESCHLER, P., JUNG, Y., AND ZÖLLNER, M X3DOM: a DOM-based HTML5/X3D integration model. In Proceedings of the 14th International Conference on 3D Web Technology, ACM, New York, NY, USA, Web3D 09, BYELOZYOROV, S., JOCHEM, R., PEGORARO, V., AND SLUSALLEK, P From real cities to virtual worlds using an open modular architecture. Springer Berlin / Heidelberg, /s DAHL, T., KOSKELA, T., HICKEY, S., AND VATJUS-ANTTILA, J A virtual world web client utilizing an entity-component model. In NGMAST, IEEE, DIONISIO, J. D. N., III, W. G. B., AND GILBERT, R d virtual worlds and the metaverse: Current status and future possibilities. ACM, New York, NY, USA, vol. 45, 34:1 34:38. KHRONOS GROUP, COLLADA - 3D Asset Exchange Schema. URL: March. KHRONOS GROUP, WebGL - OpenGL ES 2.0 for the Web. URL: May. KHRONOS GROUP, gltf - the runtime asset format for WebGL, OpenGL ES, and OpenGL. URL: May. KHRONOS GROUP, OpenGL Graphics System Version 4.3. URL: August. KLEIN, F., SONS, K., JOHN, S., RUBINSTEIN, D., SLUSALLEK, P., AND BYELOZYOROV, S Xflow: Declarative data processing for the web. In Proceedings of the 17th International Conference on 3D Web Technology, ACM, New York, NY, USA, Web3D 12, KLEIN, F., SONS, K., JOHN, S., RUBINSTEIN, D., SLUSALLEK, P., AND BYELOZYOROV, S Xflow: Declarative data processing for the web. In Proceedings of the 17th International Conference on 3D Web Technology, ACM, New York, NY, USA, Web3D 12, MICROSOFT, Direct3D 9 Reference. URL: SACCO, M., REDAELLI, C., CONSTANTINESCU, C., LAWSON, G., D CRUZ, M., AND PAPPAS, M Difac: digital factory for human oriented production system. In Proceedings of the 12th international conference on Human-computer interaction: applications and services, Springer-Verlag, Berlin, Heidelberg, HCI 07, SMPAROUNIS, K., ALEXOPOULOS, K., AND XANTHAKIS, V A Web-based Platform for Collaborative Product Design and Evaluation. In 15th International Conference on Concurrent Enterprising (ICE). W3C, Declarative3D for the Web Community Group. June. W3C, XBL 2.0 Working Group Note. May. W3C, Introduction to Web Components Working Draft. June. WEB3D CONSORTIUM, X3d international specification standards. URL: specification.html, May. ZINNIKUS, I., SPIELDENNER, T., CAO, X., KLUSCH, M., KRAUSS, C., NONNENGART, A., AND SLUSALLEK, P A collaborative virtual workspace for factory configuration and evaluation. In Proceedings of CollaborateCom-13. IEEE International Conference on Collaborative Computing: Networking, Applications and Worksharing (CollaborateCom-13), 9th, October 20-23, Austin,, TX, USA, o.a. SONS, K., KLEIN, F., RUBINSTEIN, D., BYELOZYOROV, S., AND SLUSALLEK, P XML3D: Interactive 3D Graphics for the Web. In Proceedings of the 15th International Conference on Web 3D Technology, ACM, New York, NY, USA, Web3D 10, SONS, K., SCHLINKMANN, C., KLEIN, F., RUBINSTEIN, D., AND SLUSALLEK, P xml3d.js: Architecture of a polyfill implementation of xml3d. In SEARIS.