Exploration of Data from Modelling and Simulation through Visualisation

Transcription

1 Exploration of Data from Modelling and Simulation through Visualisation Tao Lin: CSIRO Mathematical and Information Sciences, PO Box 664, ACT 2601, Australia. Robert Cheung*: CRC for Advanced Computational Systems, ANU, ACT 0200, Australia. Zhen He*: CRC for Advanced Computational Systems, ANU, ACT 0200, Australia. Kevin Smith: CSIRO Mathematical and Information Sciences, PO Box 664, ACT 2601, Australia. { tao.lin, Summary Advanced simulator or simulation environments often require a number of 3D/4D modelling systems to be integrated together. This paper presents a data model structure called a Multiple-Layer and Multiple-Relationship-set graph (MLMR graph) which can be used to represent the complicated relationships between the data generated by modelling systems and their internal data models. Some visualisation techniques have been developed for displaying the data models represented by an MLMR graph. By using these visualisation tools users can browse and control the data models in an MLMR graph through their 3D renderings. 1. INTRODUCTION The simulation of human directed tasks in real world domains can be very difficult. For example, a simulator of underground mining vehicle operation would need a number of 3D modelling systems: 3D modelling of the geology including ore grade distribution and rock quality;, 3D modelling of the mine infrastructure eg drives, stopes and shafts; 3D/4D kinematic modelling of the vehicle; 4D mechanico-physical modelling of machine/rock interaction; as well as a system to drive the vehicle operator interface and user task models. Thus a simulator or a simulation environment can often require a number of existing software packages to be integrated together. In this paper, we present a new mechanism (a data model) describing and enabling the data interaction between the various internal data representations in the "legacy" packages and the overall data model of the integrated system. This data model can be visualised and explored through a 3D rendering. Often the various modelling systems were developed independently with little or no consideration given to the future need to integrate them into a more complex system and, in particular, a simulator. In an integrated environment, there are two important tasks: transforming the data between these individual systems, and dependencies and structures. A data model of a modelling process consists of the input data, the output data, and the data in the various intermediate representations. A data model structure defines the syntax of a data model for a particular application area. CSIRO developed an object-oriented geological data model structure [1] for modelling geological data. In this geological data model structure, the data structures and the operations in the data are implemented as object classes that also include the reference links to the mated objects. As part of the geological data model, modelling/analysis tools can generate new objects from existing objects. Since there are reference links between objects, the consistency of the model can be maintained. However, if a new data representation is introduced into the system, the implementation of an existing object class has to be modified in order to provide the links to the associated objects. In many situations the modelling/analysis tools belong to legacy packages and reference links cannot be supported between these packages. Thus a mechanism is required to provide and maintain links between any associated data outside of these packages. This paper describes a data maintenance and visualisation system for integrating and exploring interactive modelling systems, which consists of three items: maintaining a consistent data model in the sense of data * ACSys CRC Vacation Scholars in

2 1) a data model structure called a Multiple-layer and Multiple-Relationship-set graph (MLMR graph) for representing complicated data structures; 2) modelling tools for creating new data for the data model and analysis tools for producing results for the user; and 3) visualisation tools for helping users to understand the data model and the results produced by analysis tools. This paper mainly discusses the first and last items. Multiple layer graph models have been used widely for modelling complicated structures, such as knowledge construction [3, 5] and software construction [4]. The MLMR graph model differs from the other multiple layer graph models in three aspects: 1) An MLMR graph allows multiple edge sets in the same layer and between layers; 2) An MLMR graph supports multiple alternatives for edge and node sets; and 3) As the formats of the elements in an MLMR graph are generic, it can be used to represent the complicated data structures for a wide range of applications and can be used inside and outside of modelling packages. During the development or operation of a simulator or simulation environment, it is often necessary for humans to understand the state of the system as a whole and the various data based interactions between the various modelling systems. Using an MLMR graph, the between the data can be used for maintaining the consistency of the model as well as information about the data. Therefore, the visualisation of an appropriate MLMR graph can help a user to understand the model and invoke modelling/analysis operations on the model by direct manipulations of a 3D rendering of the graph. 2D visualisation techniques are limited in their capacity for providing views of the whole model of complicated structures [3, 7, 8]. 3D visualisation technology and techniques increase the capacity for information display [8]. Various systems have been developed to visualise 3D models that are produced by interactive modelling processes [2]. COMAIDE [3, 5] demonstrates that 3D visualisation can display more infatuation than can be done using 2D visualisation, such as links across layers. The implementation of COMAIDE is in 2.5D. Our implement of MLMR graphs is in 3D. Real-time 3D rendering provides a natural environment for users to browse and 'feel' the data model. As the data model for an interactive modelling process can be very complicated and large, we also provide a navigation tool through which users can control the portion of the model to be visualised and displayed. By this means, users can easily control their region of interest. The rest of this paper is organised as follows. Section 2 analyses the requirements for a data model structure and the mechanisms for users to interact with a data model. Section 3 gives a specification of the MLMR graph that we use to represent our data model structure. Section 4 Section 5 concludes this paper. 2. REQUIREMENTS Beginning with an example of interactive modelling in Section 2.1, this section discusses the requirements for modelling systems from two aspects: data modelling (in Section 2.2) and user interactions (in Section 2.3). 2.1 A Modelling Example Geologists can model and analyse a geological model derived by examining data from diverse sources, such as aeromagnetic data, gravity data and drillhole data through an interactive geological modelling system. Here we give an interactive modelling example that takes drillhole data as the input. A drillhole consists of a set of drill segments and each of them contains the information obtained by examining the samples contained in the associated drill segments. A common approach [2] for generating a geological model from drillhole data is shown in Figure 1. One first creates triangles from drillhole data, and then further builds the other objects until geological elements such as faults, bedding planes and stratigraphic-units are generated. The data from all of the representations listed in Figure 2 forms a geological data model but the output data only contains the geological elements. Figure 1. A Drillhole Drived Modelling Procedure. Figure 2 illustrates the creation of a triangle from three drillholes. Assume that the values of the material attribute above and below nodes a, b and c are the same in these three drillholes, therefore the triangle A (representing a geological contact surface) can be inferred. However, due to the complexity of geological structures, such a inference may not be valid. In many situations the geographical distribution of the drillholes is very sparse due to their cost. In order to generate geological models, geologists sometimes need to add some 'artificial drillholes according to their interpretation of the geological history and structure. describes some visualisation issues for MLMR graphs. 306

3 Thus there can be significant uncertainty in the geological modelling processes. 2.2 Data Modelling In the example shown in Figure 2, the triangle depends on the three drillholes. In a modelling process, some drillholes might be 'artificial' and for various reasons the sample data in the drillholes might not be correct. Thus a geologist may need to adjust the drillholes (the coordinates and the material values of the drillhole segments). If the drillholes are modified, the triangle generated from the drillholes needs to be adjusted as well. Figure 2. Creating Triangles from Drillholes. To maintain the consistency of the data model, links from the drillholes to the triangle need to be supported. The elements of the CSIRO geological data model [1] are implemented as objects. Both the drillhole and the triangle provide references to the same node object (for instance node a). However, it would not be easy to maintain such links if the drillholes and triangles are handled by different software packages or the software packages do not provide such links (most existing interactive modelling systems are like this). As discussed in Section 1, it is not a practical solution to modify existing packages to add support for links. Our solution is to provide a relational model "on top of' the data produced by an interactive modelling system Figure 3. An Integrated Interactive Modelling System. Figure 3 shows an integrated interactive modelling depends on data set a. A relational model is built on the top of data set a and b. This relational model maintains the relationships between the data in the same data set and also the dependency relationships between the data sets. Thus, through the visualisation of the relational model, users can understand the structure of the entire data model. A user can apply further modelling/analysis through the relational model as well. If the consistency of the data model cannot be maintained according to the relational model, the system can take certain actions. For example, if data in data set a has been modified, modelling package B will be invoked to update data set b. In this modelling system, the relational model plays a very important role. Users can understand the data modelled through the visualisation of the model and choose the region of the data to be further modelled and analysed by direct manipulations on the display of the model. Some requirements for such a relational model are listed below:. Generic. The format of the relational model structure should be application independent. Specific. The information for a specific system should be able to be represented in this model. Multi-Layer. This model should be able to represent multiple data sets and the relationships between Multi-Relation. It should be able to represent multiple relationships. Multi-Alternative. The relational model should be able to support multiple alternatives for its elements. 2.3 Interaction In. an interactive system, the mechanisms for users to interact with the system are very critical. In an integrated interactive modelling system, users can interact with the system through a relational model. The requirements for the visualisation of a relational model are listed below:. 3D view. The relational model discussed above might have multiple layers and each layer can contain several edge sets. To understand the entire structure of the relational model, a 3D view is required to display the relationships between layers. Navigation. The data associated with this relational model might be very large and complicated. When a user conducts a modelling operation, she only needs to see the relevant information and not the entire relational model. Therefore, a navigation tool is required to help users to control the portion of the data to be visualised and displayed. Then users can concentrate on specific regions. 3. RELATIONAL MODEL A relational model is not a data model. A relational model and the data set referred by the relational model Figure 3 shows an integrated interactive modelling system. The input and output of modelling package A is the data set a. Modelling package B takes the data set a as the input and produce data set b. Therefore data set b 306

4 form a data model. Based on the requirements discussed in Section 2.2, we developed a relational model structure called a Multiple-Layer and Multiple-Relationship-set graph (MLMR graph) for representing the data generated by integrated, interactive modelling processes. There are two stages for handling a MLMR graph based data model: declaration and utilisation. In the declaration stage, a user defines the data model and, in the utilisation stage, a uses inputs data that is produced by various modelling/analysis tools into the data model. The data model is constructed according to the rules of the data model structure defined in the declaration stage. The BNF for the elements of an MLMR graph is given in Table 1. Data Model = Layer-Sector-Key{Layer-Sector-Key},Inter-Edge Set- Key{, Inter-Edge-Set-Key}Declaration-Key, Utilisation-Key; Layer -Sector = Layer-Sector-Key, Active-Node-Set Key{Alternative- Node-Set-Key}, Intra-Edge-Set-Key{, Intra-Edge- Set-Key}, Declaration-Key, Utilisation-Key; Node-Set = Node-Set-Key{, Node-Key}, Header-Key; Declaration-Key Utilisation-Key; Declaration = Declaration-Key{, String}; Utilisation = Utilisation-Key{, Attribute}; Header = Header-Key{, Data-Type}; Intra-Edge-Set = Active-Edge-Set-Key{, Alternative-Edge-Set-Key}; Edge-Set = Edge-Set {, Edge-Key}, Header-Key, Declaration-Key, Utilisation-Key; Edge = Edge-Key, Attachment-Key, From-Node-Key, To- Node-Key; Inter-Edge-Set = From-Layer-Sector-Key, To-Layer-Sector-Key, Active-Edge-Set Key{, Alternative-Edge-Set-Key}. Table 1. BNF of the External Description for an MLMR Graph. A data model of an MLMR graph has a set of layer sectors and a group of inter-edge sets. Each inter. edge set contains the edges linking between two layer sectors. There can be more than one inter-edge set between any two layer sectors. A layer sector has a node set and a group of intraedge sets. Each intra-edge set contains the links for a particular relationship and all the intra-edge sets in a layer sector are built on the same node set. Thus a layer sector can be used to construct multiple different graphs with the same node set. The data to be modelled by an MLMR graph consists of two pans: relational information and non-relational information. The relational information is represented by the structure of an MLMR graph. All the links between objects are represented as binary relations in an MLMR graph. Through the attachments to the nodes and edges, nonrelational information can be also modelled by an MLMR graph. There are two stages in constructing a data model in an MLMR graph: Declaration stage: defining a data model structure for a particular application eg geological modelling. Utilisation stage: constructing a data model following the syntax of the data model structure defined in the declaration stage. The header defines the data types of the data in the attachments for a particular element. For instance, the header in the node set defines the types of all the attachments to the nodes. The header is defined in the declaration stage and the attachments are assigned in the utilisation stage. Declaration elements contain the rules and utilisation elements contain the description of the actual configuration. Declaration and utilisation elements appear as pairs within node set, edge set, layer sector and data model elements. The values of declaration elements are assigned in the declaration stage of a data model and the values of utilisation elements are assigned in the utilisation stage. Alternatives are allowed for both the node set and the edge set including the intra-edge set and the inter-edge set. Thus an MLMR graph allows us to represent a data model with alternative internal structures. To reduce complexity of our modelling experiments, we choose the granularity of elements with alternatives in the node or the edge set but not in the node and edge levels. 4. VISUALISATION According to the requirements discussed in Section 2.3, this section presents an implementation of the visualisation components for MR graph. A visualisation pipeline for information visualisation is given first in Section 4.1. The model display and the navigation of a MLMR graph are discussed in Sections 4.2 and 4.3 respectively. 4.1 A Visualisation Pipeline Kamada [11] first proposed a pipeline for the implementation of visualisation systems for relational information. This pipeline illustrated in Figure 4 consists of four representations: Application Representation. Data in an application representation is the application data, such as rock properties in a geological model. Logical Representation. Data in a logical representation is the data abstracted from the data in application representation and is relational information (which can be represented as a graph). Visual Representation. Data in a visual representation contains visual attributes, such as size and coordinates. Graphical Representation. Data in a graphical representation is associated with rendering functions for visual display. 306

5 The data transformations between representations follow some common rules. An object-oriented information visualisation architecture [12] was developed based on this pipeline. In that architecture, an object pipeline illustrated in Figure 5 are implemented based on the four representations. Bi-directional reference links are provided in the objects of the pipeline. Through the links, the changes of the objects in the application representation can be mapped to the display and the users' interactions with the display can also be mapped back to applications according to the mapping rules representation and the objects in the logical representation, the changes made in each representation can be sent to the other representation. After receiving a message, the other representation makes adjustments to maintain the consistency of the data in the different representations. We adopt 3D technology for the visualisation of MLMR graphs. At this stage, we just simply extended the Sugiyama method [15] for layout generation. There can be more than one edge set in a layer sector. We provide two ways to display a layer sector with multiple edge sets: 1. combine the edge sets into a single one and generate the display for the combined graph. 2. generate the display with only a single edge set and place the display in a plane and place the display of the other edge set on the other plane. The disadvantage of the first approach is that there may be too many edges crossing even for graphs with a quite reasonable size. This disadvantage Can be overcome by the second approach. However, if the number of edges is not small the first approach is better. Each layer sector is assigned a certain volume for its display. Using 3D technology, we can display multiple layer sectors in the same scene. The inter-edge sets can also be displayed. 4.3 Navigation Tools The 3D display of an MLMR graph can provide users with a view of the entire model. However, data mode]s arising from simulators or simulation environments tend to be large and complicated. For practical visualisation and interaction with the model, a navigation tool is needed. An abstract diagram is used to show the data model structure of an MLMR graph. Figure 6 illustrates the primitive elements of this abstract diagram. Three symbols represent three primitive elements of the abstract diagram: node set, intra-edge set and inter-edge set. Figure 5. An Object Visualisation Pipeline. Based on the visualisation pipeline, we also developed a generic multi-view architecture [2] for interactive visualisation systems and a framework [13, 14] for the integration and collaboration of interactive modelling systems. 4.2 Model Display We use an MLMR graph to represent the data extracted from a modelling system. The MLMR graph is implemented as the logical representation. To generate a display of a model represented by an MLMR graph, the objects in the MLMR graph are mapped into objects in the visual representation according to the rules for layout generation, and then the objects in graphical representation according to the dressing rules (ie the rules for assigning colours and shapes). Figure 6. Symbols for the Abstract Diagram. To generate the display for the objects in an MLMR graph, the associated objects in the other two representations (visual and graphical) need to be generated as well. Therefore each primitive element in this abstract diagram may be in one of four modes: The results produced by analysis tools' are generated directly as the objects in the visual representation. Because of the bi-directional links between the objects in the visual 307

6 Display: the graphical objects of the associated objects contained in the primitive element in the data model are displayed on the screen. Graphics Generated: the graphical objects of the associated objects contained in the primitive element in the data model have been created. Geometry Generated: the visual objects of the associated objects contained in the primitive element in the data model have been created. Original: the visual objects of the associated objects contained in the primitive element in the data model have not been created. These modes have some dependent relationships. We say mode A contains mode B if the objects in mode A must be in mode B. The modes listed above are contained by the previous mode; for example, the elements in display mode must also be in graphics generated mode. The mode of an intra-edge set must be contained by the mode of the node set to which the intra-edge set links. The mode of an extra edge set must be contained by modes of the two node sets to which this inter-edge set links. We provide a navigation tool based on this abstract diagram for users to control the portions of the data model to have their layout generated, the graphical objects generated and displayed. One can also decide the approach for handling multiple edge sets within a. layer sector and between layer sectors through this navigation tool. 5. CONCLUSION This paper presents a data model structure, an MLMR graph which can be used to represent the links on the top of the data generated by an interactive modelling processes. Several visualisation techniques for the visualisation of MLMR graph based models have also been discussed: 3D display; and visual navigation tool. An MLMR graph provides a powerful modelling mechanism for constructing models with complicated structures and the visualisation techniques for MLMR graphs provide the mechanisms for users to explore and model the data set with large and complicated structures. The system is implemented in Java (JDK 1.1 [9, 10]). The 3D visualisation was developed by using Kuhlua (a package of Java wrappers around Open Inventor developed by Brown University). We have implemented the MLMR graph and are currently improving the visualisation components and integrating some generic modelling/analysis components with the system. Besides geological modelling, we are also applying the techniques discussed in this paper to other application areas, such as program comprehension for the understanding and maintenance of large software systems, and data mining for discovering useful information from large and complicated data sets. The authors wish to acknowledge that this work was carried out within the Cooperative Research Centre for Advanced Computational Systems established under the Australian Government's ere Program. 6. REFERENCES [1] W. L. Power, P. Lamb and F.G. Horowitz. "From Databases to Visualisation - Data Transfer Standards and Data Structures for 3D Geological Modelling", Proc. of APCOM XXV pp , [2] T. Lin, M. O. Ward, W. L. Power and D. M. Landy. "From Databases to Visualisation - Providing a User Friendly Environment for Creating 3D Solid geology Models", Proceedings of APCOM XXV pp , [3] D. Dodson, "COMAIDE: Information Visualization Using Cooperative 3D Diagram Layout", Graph Drawing' 95, Lecture Notes in Computer Science 894, Springer [4] B. Lague, et al, "A Framework for the Analysis of Layered Software Architectures", Proceedings of the 2 nd International Workshop on Empirical Studies of Software Maintenance. pp , [5] D. Dodson, et al, "VIM: 3D Co-operative Diagrams as KBS Surfaces", Proc. of Expert Systems [6]M. D. Story, I-L A. Muller, and K. Wong, "Manipulating and. Documenting Software Structures", Software Visualisation, P. Eades and K Zhang (ed.), World Scientific. pp , [7] S. P. Reiss, "An Engine for the 3D Visualisation of Program Information", Journal of Visual Languages and Computing. Vol. 6, No. 3, pp , [8] G. Robertson, J. Mackinglay and S. Card, "Animated 3D Visualisation of Hierarchical Information", Proceedings of SIGCHI'91: Human Factors in Computing Systems (ACM). pp , [9] C. S. Horstmann arid G. Cornell, "CORE Java 1.1 Volume I: Fundamentals", The SunSoft Press, Java Series, Prentice Hall [10]C. S. Horstmann and G. Cornell, ''CORE Java 1.1 Volume II: Advanced Features", The SunSoft Press, Java Series, Prentice Hall [11]T. Kamada, "Visualising Abstract Objects and Relations", World Scientific [12]T. Lin, "A General Schema for Diagrammatic User Interfaces", PhD Thesis, The University of Newcastle [13]T. Lin and K. Smith, "A Generic Functional Architecture for the Integration of 3D Systems for Simulation", Proceedings of International Conference: SimTecT'97. pp 9-14, [14]T. Lin and K Smith, "A Generic Functional Architecture for the Development of Multi-user 3D Environments", Book Chanter in Virtual Worlds.on the Internet, IEEE Computer Society Press [15]K. Sugiyama, S. Tagawa, M. Toda, "Methods for Visual Understanding of Hierarchical System Structures", IEEE Transactions on Systems, Man and Cybernetics. Vol. SMC-11, No.2. pp , ACKNOWLEDGEMENT 308

7 308