Institutionen för datavetenskap Department of Computer and Information Science

Institutionen för datavetenskap Department of Computer and Information Science Examensarbete Integrated generic 3D visualization of Modelica models av Henrik Magnusson LIU IDA/LITH EX A 08/035 SE 2008 07 06 Linköpings universitet SE 581 83 Linköping, Sweden Linköpings universitet 581 83 Linköping

Final thesis Integrated generic 3D visualization of Modelica models by Henrik Magnusson LIU-IDA/LITH-EX-A 08/035 SE 2008-07-06 Supervisor: Examiner: Anders Sandholm Department of Computer and Information Science, Linköpings universitet Prof.Peter Fritzson Department of Computer and Information Science, Linköpings universitet

Contents 1 Introduction 5 1.1 Modelica.............................. 5 1.1.1 Modelica Standard Libraries.............. 5 1.1.2 Modelica MultiBody library............... 5 1.2 OpenModelica........................... 6 1.3 OMNotebook........................... 6 1.4 Glossary.............................. 6 2 Purpose 7 2.1 Requirements........................... 7 3 Annotation based visualization 9 3.1 Annotations in OpenModelica.................. 9 3.1.1 Documentation...................... 9 3.1.2 Icon............................ 10 3.1.3 Diagram.......................... 10 3.2 Visualization annotation..................... 10 3.2.1 Trial implementation................... 11 4 Object based visualization 13 4.1 Previous work........................... 13 4.2 Design............................... 13 4.3 Visualization package...................... 14 4.3.1 Object properties..................... 14 4.3.2 Object parameters.................... 15 4.4 Conclusions............................ 16 5 Integrating visualization in OMNotebook 17 5.1 Graphics.............................. 17 5.2 Alternatives............................ 18 5.2.1 Crystal Space....................... 18 5.2.2 Irrlicht Engine...................... 18 5.2.3 OGRE 3D......................... 18 5.2.4 Coin3D.......................... 18 iii

5.2.5 QT GLWidget with support libraries.......... 19 5.3 Conclusion............................ 19 6 Implementation 21 6.1 Execution flow.......................... 21 6.2 Extensions of the OpenModelica compiler........... 22 6.2.1 Implementation of visualize().............. 23 6.2.2 Implementation of getelementsofvistype()...... 23 6.2.3 The SendData package.................. 23 6.3 Limitations............................ 23 7 3D visualization 25 7.1 Design............................... 25 7.2 Simulation Classes........................ 26 7.2.1 SimulationData...................... 26 7.2.2 SimulationKeypoint................... 26 7.2.3 SimulationObject..................... 26 7.3 Creating the scene graph..................... 27 7.4 Running the visualization.................... 27 7.5 Interpolation........................... 27 7.5.1 Interpolating position.................. 28 7.5.2 Rotation.......................... 28 8 A simple example 31 8.1 BouncingBall........................... 31 8.1.1 Creating the model.................... 31 8.1.2 Simulating........................ 32 8.1.3 Adding visualization................... 32 8.1.4 Running the simulation................. 33 8.1.5 Starting the visualization................ 34 9 Advanced example 35 9.1 The Pendulum model...................... 35 9.2 Adding the visualization..................... 36 9.3 Loading a shape......................... 39 10 Related Work 41 10.1 MultiBody library........................ 41 10.2 3D Graphics and Modelica - an integrated approach..... 41 11 Future work 43 11.1 Support visualizing MultiBody simulations.......... 43 11.1.1 Replace type names................... 43 11.1.2 Consolidate parameter names and types........ 43 11.1.3 Extended shape support................. 44 11.2 Utilize Modelica better...................... 44

11.3 Improved user experience.................... 45 12 Summary 47 12.1 Result............................... 47 12.2 Conclusions............................ 48 A visualize() 49 B getelementsofvistype() 51

List of Figures 4.1 SimpleVisual class diagram.................... 14 4.2 Specifying up direction using two positions........... 15 6.1 Visualization call graph...................... 22 6.2 SendData protocol........................ 24 7.1 System graph............................ 25 7.2 SimulationData class hierarchy.................. 26 7.3 Syntax of object information sent by OMC........... 27 7.4 Linear interpolation of position (pseudo code)......... 28 7.5 SLERP versus linear interpolation................ 29 8.1 BouncingBall example....................... 34 9.1 Pendulum example......................... 37 9.2 Finished Pendulum........................ 39 9.3 Using a loaded mesh........................ 40 10.1 Visualization annotation example................ 42 1

Abstract OpenModelica is a complete environment for developing and simulating Modelica models based on free software. It is promoted and developed by the OpenModelica Consortium. This thesis details a method for describing and consequently displaying visualizations of Modelica models in OMNotebook, an application in the OpenModelica suite where models can be written and simulated in a document mixed with text, images and plots. Two different approaches are discussed; one based on Modelica annotations and one based on creating a simple object hierarchy which can be connected to existing models. Trial implementations are done which make it possible to discard the annotation approach, and show that an object based solution is the one best suited for a complete implementation. It is expanded into a working 3D visualization solution, embedded in OMNotebook. 3

Chapter 1 Introduction This chapter gives an introduction to the thesis by defining the actors and organizations working in the domain of the project, and gives a brief introduction to the environment within which the resulting software implementation will exist. 1.1 Modelica Modelica[1] is an object-oriented language for physical systems modeling. It was conceived in 1996 as an attempt to enable exchange and reuse of models and model libraries between different tools. The development of Modelica is promoted by the Modelica Association, a non-profit, non-governmental organization. The Modelica Association owns among other things the Modelica trademarks, the Modelica Language Specification and the Modelica Standard Libraries. 1.1.1 Modelica Standard Libraries The Modelica Association offers a large number of freely available libraries, ranging from electrical, thermal, and fluid to electromagnetic and mechanical applications. 1.1.2 Modelica MultiBody library The mechanical library, also known as the MultiBody library, is of particular interest for this thesis. Mechanical simulations lend themselves extremely well for visualization. Unfortunately, the OpenModelica compiler does not yet support the MultiBody library since it uses some features of the Modelica language that are not yet implemented. 5

1.2 OpenModelica The OpenModelica project was started with the intention of creating a complete Modelica modeling, compilation and simulation environment based on free software. In December 2007, the OpenModelica Consortium[2] was founded as a non-profit, non-government organisation aiming to promote the use and development of OpenModelica. The OpenModelica Project homepage[3] contains more information about OpenModelica. 1.3 OMNotebook OMNotebook is an application based on the idea of literal programming, described by Donald E. Knuth in [4]. In OMNotebook OpenModelica models can be constructed, simulated and plotted, together with text and images in a living document. OMNotebook was developed as a thesis project at PELAB, the Programming Environments Laboratory at the Institution for Computer Technology at Linköping University by Ingemar Axelsson and is described in [5]. It has since been extended in the master thesis of Anders Fernström[6]. In particular, Henrik Erikssons thesis[7] on improving the plotting functionality in OMNotebook has been most helpful for this thesis since it laid the foundation for communicating simulation results from OMC, the Open- Modelica compiler, to OMNotebook. 1.4 Glossary CORBA CORBA[8] is an acronym for Common Object Request Broker Architecture. CORBA is a framework for remote object invocation that is used for communication between OMNotebook and OMC. OMC The OpenModelica Compiler, called from OMNotebook using the Mico CORBA API[9]. QT QT[10] is a complete development framework in C++ developed by Trolltech. OpenGL OpenGL[11] is an environment for developing portable and interactive 2D and 3D application. It is the industry standard for scientific visualization. Coin3D Coin3D[12] is a high-level toolkit for developing interactive realtime 3D visualization. It is cross platform and built upon OpenGL. Widget A widget is a generic name for a component in a graphical user interface.

Chapter 2 Purpose The purpose of this thesis is to extend the OpenModelica suite with a 3D visualization package integrated in OMNotebook. The visualization should be completely described by the Modelica model, and not rely on external software or processing. The language or syntax used to create the visualization must be able to fully describe both the appearance of a visualized object as well as connect properties like position, rotation or size to a variable in the simulated model. 2.1 Requirements The following requirements on the visualization solution are defined: It should be possible to present the 3D visualization inline in an OM- Notebook document. Hardware acceleration should be used if available, but no effort will be made to support advanced graphical effects or lighting techniques. The embedded visualization will have a simple GUI, where a user can control the playback of the simulation and adjust the camera position to view the simulation from different angles. One typical example where visualization is often used is the simulation of rigid bodies using the Modelica MultiBody library. This library is not yet supported by OpenModelica, but a goal of this thesis is to create a package that can easily be adapted to support visualization of MultiBody simulations once support is added. 7

Chapter 3 Annotation based visualization This chapter describes the first approach that was made at a language for defining visualization for Modelica models. This first approach is based on Modelica annotations. 3.1 Annotations in OpenModelica An Annotation is a syntactic construct in the Modelica language that has no impact on the simulation of a model. An annotation contains information about the model, often useful for the program that presents the simulation result. The OpenModelica CORBA API supports querying a model for three common annotation types: Documentation Icon Diagram 3.1.1 Documentation The Documentation annotation is used to describe the model in more detail. HTML is often used. annotation ( Documentation ( i n f o= <html> <body> <h1>model</h1> <p>d e s c r i p t i o n </p> 9

</body> </html > ) ) 3.1.2 Icon The Icon annotation describes the appearance of an icon for the model. The appearance can be a link to a bitmap or a graphic annotation describing how to draw the icon, as in the example below. Icon ( Rectangle ( e x t e n t =[ 88,100; 90, 100], s t y l e ( c o l o r =0, r g b c o l o r ={0,0,0}, t h i c k n e s s =2, f i l l C o l o r =51, r g b f i l l C o l o r ={255,255,170}) ), Line ( p o i n t s = [ 4 0, 5 4 ; 4 0, 6 0 ; 40,60; 1 0, 0 ; 40, 60; 40, 60; 40, 52], s t y l e ( c o l o r =0, r g b c o l o r ={0,0,0}, f i l l P a t t e r n =1) ) ) 3.1.3 Diagram Like the Icon annotation, the Diagram annotation describes a graphical representation of a class. The difference between the two is that while the main objective of the Icon annotation is to represent an instance of the object, the Diagram annotation describes the composition of the class by showing subcomponents. 3.2 Visualization annotation None of the existing annotation types are suited for describing visualization information. To overcome this problem, a new visualization type has to be defined. What is needed is to connect a property of the visual representation with a variable in the simulation. For example, if we are visualizing a falling object, the variable containing the height of the object must be connected to the y position of a visualization. annotation ( V i s u a l i z a t i o n ( frame a= x, frame b= y, c o l o r = red, shape= cube ) )

3.2.1 Trial implementation A trial implementation of the annotation approach was done, but some serious flaws was discovered with the approach and the implementation was scrapped. The annotation system is very inflexible. Querying the OpenModelica compiler for an annotation yields only a list of values with no specification of the contents. To support many different kinds of visualizations, i.e. displaying cubes, spheres, with or without color etc, a new annotation would have to be defined for each type. Since there is no polymorphism in the annotation syntax, this quickly becomes problematic. Type checking is impossible when the only connection between a property (such as the position of an object) and the variable containing the information is the name of the variable. In the example above, there is no guarantee that x is of a type that is usable as a position. Syntax checking is not performed on annotations in the sense that it is impossible to know at compile time if the visualization annotation is correctly specified.

Chapter 4 Object based visualization In the former chapter visualization described using the Modelica annotation concept. It was discovered that using annotations in this way has a lot of drawbacks that make it unnecessarily complex. This chapter gives a detailed account of an alternative approach at how to define visualization of a Modelica model in a way that better utilizes the Modelica language. 4.1 Previous work Since one important goal of this thesis was to come up with a system for visualization that might be used for simulations done with the Modelica MultiBody library, it follows that much can be learned from investigating how visualization is used in conjunction with it. Visualizing MultiBody models is done by for example Dymola from Dynasim[13]. The MultiBody package is well suited for visualization. Entities in a MultiBody simulation correspond to physical entities in a real world and as such have many of the properties needed to correctly display them within a visualization of the simulation, such as position and rotation. The visualization solution of the MultiBody package is discussed in further detail in chapter 10. 4.2 Design Instead of using annotations to encode information about how a certain object is supposed to look when visualized, object based visualization creates additional Modelica objects of a predetermined type that can be known to the client actually doing the visualization. These objects contain variables such as position, rotation and size that can be connected to the simulated variables using ordinary Modelica equations. When asked to visualize a model, the OpenModelica compiler can find variables in the model that are 13

in the visualization package and only send only those datasets over to the client doing the visualization, in this case OMNotebook. 4.3 Visualization package Taking inspiration from the MultiBody library, a small package has been designed that provides a minimal set of classes that can be connected to variables in the simulation. It is created as a Modelica package and can be included in the Modelica Library. The package is called SimpleVisual, and consists of a small hierarchy of classes that in increasing detail can describe properties of a visualized object. Figure 4.1: SimpleVisual class diagram. For some simulations, it might not be relevant to include rotation of the visualized object. Since Modelica requires that a model has the same number of equations as unknowns, the user would still have to specify equations for rotation. While a simple assignment would suffice, having this structure means the user can apply the correct amount of detail to the visualization and minimize the amount of work needed. 4.3.1 Object properties The visualized objects have four properties that can be controlled from the simulation, Position, Rotation, Size and Offset. Position The position of an object is controlled by assigning equations to the class variable frame_a.

Rotation Rotation is controlled by assigning equations to a second position variable, called frame_b. The vector between frame_b and frame_a defines the up direction of the object, as described in figure 4.3.1. Size An object can be scaled in all three dimensions by assigning to the size variable of a SimpleVisual object. Offset By assigning to the offset variable, the origin of the visualized object can be moved, to allow for rotation around a position other than in the center of an object. Figure 4.2: Specifying up direction using two positions. The names frame_a and frame_b are chosen from their counterparts in the Multibody library. 4.3.2 Object parameters Parameters are aspects of an object that will not change during a simulation. Using the SimpleVisual package, parameters of an object are specified as a semicolon-separated list of <parameter>=<value> pairs in the comment of a variable declaration. This is far from optimal, but unfortunately it was the only solution that allowed object parameters to be included without sub-classing in absurdum that was possible to implement within the time frame of the thesis project. There are two alternative implementations that would fit much better within the Modelica language. Annotations Annotations, much like comments, are syntactical constructs whose contents do not affect the behavior of a model. Instead of using annotations to describe visualization, as described in section 3.1, they could be used for simply describing parameters of objects. Annotations are commonly used to encode information about a class or variable that can be interpreted by

external applications, such as HTML documentation for a class or defining an icon for a model. This makes them pretty well suited for encoding visualization parameters. Class parameters In the Modelica language, variables can be declared with the parameter keyword. This tells the compiler that it will not change during a simulation. These would be the most obvious to use for specifying parameters of the visualized object, but unfortunately it is not possible to extract the parameters of an object in an easy way in the OpenModelica compiler, and due to time contstraints this path was not pursued further. 4.4 Conclusions Using the Modelica language to specify new objects that essentially only contain visualization information turned out to be very straight forward. It is very easy to add visualization to an existing model as will be shown in the following two chapters where example implementations are given. There are of course drawbacks with this solution. Verbosity Since a model must contain the same number of equations as unknowns, each variable in a visualization object must have an equation, even if it isn t interesting for the visualization. This can have the effect that adding visualization to a model doubles or triples it in code size, impacting legibility and maintainability of the code. The SimpleVisual package attempts to minimize this problem by providing a class hierarchy where a class containing the least information needed can be chosen. Simulation size The verbose property of the visualization has another drawback, namely that each variable in visualization objects must also be simulated. Typically, equations for the objects are simple assignments to other variables in the simulation, which means that the running time of simulating the model is not terribly affected, but the resulting data will grow very large compared to the original. This is not a problem when using the MultiBody library, since the variables used for visualization is inherent in the rigid body objects in the library and would have been simulated anyway.

Chapter 5 Integrating visualization in OMNotebook This chapter explains the reasoning done when deciding how to integrate a visualization solution into the existing OMNotebook. It describes the criteria set for evaluating alternatives, and gives a brief presentation of the alternatives. Finally, one solution is selected and the implementation based on this solution is described in further detail in the following chapter. 5.1 Graphics OMNotebook uses the QT toolkit[10] created by Trolltech to implement GUI features. The QT libraries are available on a plethora of platforms, most importantly X11 and Windows. To be able to display 3D graphics in the OMNotebook application, a 3D graphics library of some sort is needed. The requirements for this application are quite modest. The alternatives have been evaluated with the following criteria. Platform independence OMNotebook is available for Windows, Mac OS X and Linux, and subsequently the selected graphics engine must support all three as well. Scene graph A scene graph is a tree structure defining the contents of a scene. It is a very good abstraction which makes implementing complex visualizations easier. Some type of scene management, most typically a scene graph, is available in almost all 3D engines. Support for importing 3D files The engine must be able to import polygon models in some format, preferably X3D or VRML which are open standards. 17

Ease of integration The easier the engine is to integrate with the legacy code the better. Input handling It must be possible to interact with the simulation, eg controlling the camera. License Since the OpenModelica project is open source, the license of the selected 3D package can not be in conflict with the OpenModelica license. 5.2 Alternatives 5.2.1 Crystal Space Crystal Space[14] is one of the oldest open source 3D engines. The project was started in 1997, but it is still under active development and has a thriving community. The engine is written in C++ and has a plug-in based architecture. Crystal Space is licensed under LGPL[15]. 5.2.2 Irrlicht Engine Irrlicht[16] is another popular open source 3D engine. Ports are officially available for Windows, Linux and Mac OS X, but it has been ported to many other system as well. It supports importing many mesh file formats, but not X3D or VRML. It is written in C++ and is released under the Zlib license[17]. The Zlib license allows commercial use without any requirements of releasing code changes. One criticism of the Irrlicht engine is that is is still in rapid development, leading to incompatibilities between different releases. 5.2.3 OGRE 3D OGRE[18] is yet another open source 3D engine, implemented in C++ with an active, large user base. It is licensed under LGPL. OGRE is different to the rest of the alternatives in that it is only a graphics engine, whereas the other alternatives all include features geared towards game development, such as collision detection or audio support. 5.2.4 Coin3D Coin3D[12] is a very complete and competent 3D visualization package. It is based on Open Inventor[19], a high level graphics library on top of OpenGL[11] delevoped by SGI[20]. It supports all the requirements set up for the visualization in OMNotebook. Conveniently enough, it can also be contained in a QT widget. This makes it extremely easy to integrate with the OMNotebook application. Coin3D is freely available as GPL[21], or by a license for possible proprietary usage.

5.2.5 QT GLWidget with support libraries The QT library contains an OpenGL widget. It offers some help when it comes to setting up and integrating OpenGL code in a QT application, but no scene graph, input handling or file type support. This could be supported using additional libraries. For example, there are plenty of open source scene graph or file import libraries available. 5.3 Conclusion A comparison of the different 3D engines resulted in Coin3D being selected for the thesis project. Coin3D is the easiest to integrate into OMNotebook of the alternatives thanks to the SoQT widget, it can import VRML97 as well as VRML 2.0 objects and the license is compatible with that of OpenModelica. Coin3D provides a scene graph and an interactive object viewer environment with support for controlling the camera with the mouse out of the box.

Chapter 6 Implementation This chapter will describe the design and implementation of the visualization package in OMNotebook. An overview is given of the flow of events that enable the visualization package to get information on what to display, and how to connect this to the simulation data. The extensions made to the OpenModelica language is described in detail. 6.1 Execution flow A visualization is initiated from OMNotebook, where a user has created a model, added visualization by creating objects and tied them to the model with equations. The visualization must be started from an InputCell which recognizes the command. 1 - The user calls the OpenModelica command visualize(), specifying a model as the function argument. OMNotebook connects to the Open- Modelica compiler via the CORBA API and issues the command. It also opens a listening socket and registers call-back handlers for connect events. The InputCell recognizes the called command and initializes the 3D graphics display. 2 - OMC finds the internal visualize command and runs it. The command searches the model for variables of a type that is defined within the SimpleVisual package. The list of variables are matched with the latest simulation result of the model, and the result is then sent to the socket opened by OMNotebook together with a description of the included variables. 3 - OMNotebook reads the description of the variables and the simulation data and builds an internal data structure from them. 4 - When all simulation data has been sent, OMC closes the connection. OMNotebook reads until the socket is closed. 21

Figure 6.1: Visualization call graph. 6.2 Extensions of the OpenModelica compiler The OpenModelica API has been extended with a number of functions to support visualization in OMNotebook. The most important is of course the visualize command which contacts OMC and asks for the results of the latest simulation and starts the animation. Table 6.1 contains an overview of the added API calls. Table 6.1: Extensions of the OpenModelica CORBA API Command visualize(model) getelementsofvistype() Description The visualize command searches the specified model for variables in the SimpleVisual package. The name and type of each object are sent to OMNotebook, followed by the simulation results for these variables. Searches a model for elements that are members of the Visualization package.

6.2.1 Implementation of visualize() The visualize method is the starting point of the visualization. When it is called from OMNotebook, it gathers the needed data and sends it back for the visualization GUI to display. It is implemented in the file Ceval.mo, and can be found in appendix A. 6.2.2 Implementation of getelementsofvistype() The function getelementsofvistype is called from visualize to traverse the model looking for element from the SimpleVisual package by extracting the type name of each element and comparing it to a set of hard coded type names. If a match is found, the variable, its comment and type is added to a list that is returned to visualize. It is implemented in the file Interactive.mo, and can be found in appendix B. 6.2.3 The SendData package The implementation of visualization in OMNotebook uses a slightly modified version of the senddata code developed by Henrik Eriksson. Most parameters to senddata have been removed. In addition, it has been modified to send descriptions of all variables included in the dataset. This is needed for telling the visualization about properties of the objects that are not simulated and thus not a part of the simulation dataset. Typical use for these descriptions are to transmit information regarding object color, material or paths to 3D geometry data. QDataStream The network transfer is implemented using a QDataStream, a class from the QT library. It makes transferring data over a network very simple. To use it, a standard C++ stream is opened, and any object or type can be sent to the stream using the << operator. The QDataStream class automatically wraps and unwraps objects on the stream. The sending and receiving end must still agree on a protocol for the receiver to know what to expect from the stream. This protocol is described in figure 6.2. The second block in the protocol contains the objectinformation. It is a string that describes the objects in the model that originate from the SimpleVisual package. The format of the string is described in further detail in chapter 7. 6.3 Limitations A technical limitation in this implementation is that all visualization must be declared in the top level of the hierarchy of classes that make up a

Figure 6.2: SendData protocol <block>ptolemydatastream </block> <block> <QString>o b j e c t i n f o r m a t i o n <quint32>n <QString>variablename [ 1 ]... <QString>variablename [ n ] </block> <block [1] > <quint32>n <QString>variablename [ 1 ] <double>v a r i a b l e v a l u e [ 1 ]... <QString>variablename [ n ] <double>v a r i a b l e v a l u e [ n ] </block [1] >... <block [m]> <quint32>n <QString>variablename [ 1 ] <double>v a r i a b l e v a l u e [m]... <QString>variablename [ n ] <double>v a r i a b l e v a l u e [m] </block [m]> </o b j e c t s > simulation. Objects from the SimpleVisual package will only be found if they are declared within the model that is sent as a parameter to visualize. This makes it difficult to create libraries of objects with included simulation, but as long as they are publicly accessible from the top level, any subcomponent variable could be linked to a simulation object. The problem is that no recursive search for SimpleVisual objects is done. This is purely a limitation of this implementation, and could be added in a future project.

Chapter 7 3D visualization In this chapter the details of the 3D graphics view of the simulation is described. 7.1 Design As discussed in chapter 5, the library used to create the 3D visualization is called Coin3D. It implements a scene graph on top of an OpenGL rendering subsystem. This in turn is embedded in a QT widget, which is displayed within OMNotebook. Figure 7.1: System graph. 25

7.2 Simulation Classes A small hierarchy of classes are responsible for storing the simulation values and the objects to visualize in the client. The main class is SimulationData, which acts as a container for the other two, SimulationKeypoint and SimulationObject. Figure 7.2: SimulationData class hierarchy. 7.2.1 SimulationData SimulationData has a list of all the key-points from the simulation, as well as a list of the objects. SimulationData also holds the scene graph. The visualization can ask SimulationData to setup the scene graph at a specific time. SimulationData will then iterate over all objects in the simulation and modify parameters such as position, rotation or size, and finally the visualization component will display the scene graph in the 3D window. 7.2.2 SimulationKeypoint A SimulationKeypoint is created for each point in time that is included in the network stream of simulation data. The time of the key-point is the unique property of a key-point in the simulation. Each key-point also contains a hash table of variables and their value at the key-point time. 7.2.3 SimulationObject The SimulationObject class contains information about an object in the visualization, such as position and rotation, but also static parameters such as color, shape or mesh filename.

7.3 Creating the scene graph When OMC sends the result from a simulation to the visualization, it first includes a description of all the objects included. This information is used by the SimulationData class to create the corresponding 3D objects with the correct shape and color and adding them to the scene graph. o b j e c t [ 1 ], property=value ; \ n... o b j e c t [ n ], property=value ; \ n Figure 7.3: Syntax of object information sent by OMC. The properties understood by the visualization are: color Set the color of an object. Valid values are blue, red and green. shape Control the shape of an object. Valid values are cube, sphere and cylinder. model Load the shape from a file. This property overrides any previously define shape value. The value is a string containing the complete path to a VRML model. 7.4 Running the visualization In the interface of the visualization window, the user can start and stop the visualization, and search for a specific time in the simulation using a slider. When play is pressed, the start time is saved, and then a timer is set that will be triggered every 40 ms. The call-back function of the timer calls a method called setframe in SimulationData, passing a parameter that contains the accumulated time since the simulation was started. The setframe function iterates over all objects and sets up the scene graph of the visualization according to the values found in the SimulationKeypoint, interpolating between key-points as necessary. There is a hardcoded 40 ms delay giving a steady frame-rate of 25 frames per second. 7.5 Interpolation When performing a simulation in OpenModelica, the user can supply a parameter controlling the resolution of the simulation. If the simulation

is very long, simulation with full resolution might not be practical due to limitations in memory or time. Simulating with a lower resolution sacrifices detail to minimize space/time complexity. 7.5.1 Interpolating position An object has four parameters that can be interpolated: frame_a (position), offset, size and frame_b (controlling rotation). The easiest interpolation to implement is linear interpolation. The simulation must consist of at least two key-points. Linear interpolation is done in the setframe function by taking the current simulation time, finding the two closest key-points and interpolating between them. f u n c t i o n setframe ( t i m e t y p e time ) { f o r e a c h ( o b j e c t i n s i m u l a t i o n o b j e c t s ) { s i m u l a t i o n k e y p o i n t k1 = s i m k e y p o i n t s. f i n d P r e v i o u s ( time ) ; s i m u l a t i o n k e y p o i n t k2 = s i m k e y p o i n t s. findnext ( time ) ; p o s i t i o n pos1 = k1. getvalue ( o b j e c t, frame a ) ; p o s i t i o n pos2 = k2. getvalue ( o b j e c t, frame a ) ; time t = ( s i m u l a t i o n t i m e k1. time ) / ( k2. time k1. time ) ; p o s i t i o n r e a l p o s = (1 t ) pos1 + t pos2 ; } } o b j e c t. setpos ( r e a l p o s ) ; Figure 7.4: Linear interpolation of position (pseudo code). 7.5.2 Rotation Interpolating rotation can be slightly more complex than interpolating a position. In the SimpleVisual package, the rotational direction of an objects up direction is specified by the vector between the position of the object, frame_a and a second position, frame_b. As frame_b is just a position, we can interpolate it linearly just like the other properties. However, in reality the movement of a point at the end of a rotating vector is not linear, but will follow an arc.

SLERP interpolation SLERP is shorthand for spherical linear interpolation. The difference between SLERP and linear interpolation is that SLERP approximates the movement of the point frame_b as if it was moving on the face of a sphere. SLERP interpolation is available as a method in the Coin3D library, making it very simple to include in the visualization. Figure 7.5: SLERP versus linear interpolation.

Chapter 8 A simple example This chapter will give a short introduction to how the SimpleVisual package is used. It describes how to add a simple visualization of an already existing Modelica model. 8.1 BouncingBall The bouncing ball model is a simple introduction to the Modelica language used in DrModelica[23], a Modelica learning tool implemented as a Modelica notebook. Adding visualization of the bouncing ball using the SimpleVisual package is very straightforward. 8.1.1 Creating the model Startup OMNotebook and create an empty notebook by selecting New from the file menu. First, we need an InputCell to have somewhere to describe our model. Add an InputCell from the Cell menu. The following simple model describes the movement of a bouncing ball. model BouncingBall parameter Real e =0.7 c o e f f i c i e n t o f r e s t i t u t i o n ; parameter Real g =9.81 g r a v i t y a c c e l e r a t i o n ; Real h ( s t a r t =10) h e i g h t o f b a l l ; Real v v e l o c i t y o f b a l l ; Boolean f l y i n g ( s t a r t=t r u e ) true, i f b a l l i s f l y i n g ; Boolean impact ; Real v new ; equation impact=h <= 0. 0 ; der ( v )=i f f l y i n g then g e l s e 0 ; der ( h )=v ; 31

when {h <= 0. 0 and v <= 0. 0, impact } then v new=i f edge ( impact ) then e pre ( v ) e l s e 0 ; f l y i n g=v new > 0 ; r e i n i t ( v, v new ) ; end when ; end BouncingBall ; 8.1.2 Simulating To run a simulation of the bouncing ball, create a new InputCell and type call the simulate method. The simulate method takes a model, start time, and an end time as arguments. s i m u l a t e ( BouncingBall, s t a r t =0, end=5s ) ; 8.1.3 Adding visualization The bouncing ball will be simulated with a red sphere. We will let the variable h control the y position of the sphere. Since the ball has a size and the model describes the bouncing movement of a point, we will use that size to translate the visualization slightly upwards. First, we must import the SimpleVisual package and create an object to visualize. That is done by adding a couple of lines to the beginning of the BouncingBall model. model BouncingBall import SimpleVisual. SimpleVisual. P o s i t i o n S i z e b a l l c o l o r=red ; shape=sphere ; ;... The string "color=red;" is used to set the color parameter of the object and the shape parameter controls how we will display this object in the visualization. The next step is to connect the position of the ball object to the simulation. Since Modelica is an equation based language, we must have the same number of variables as equations in the model. This means that even though the only aspect of the ball that is really interesting is its y-position, each variable in the ball object must be assigned to an equation. Setting a variable to be constant zero is a valid equation. The SimpleVisual library

contains a number of generic objects which gives the user an increasing amount of control. SimpleVisual.Position SimpleVisual.PositionSize SimpleVisual.PositionRotation SimpleVisual.PositionRotationSize SimpleVisual.PositionRotationSizeOffset Since we are really only interested in the position of the ball, we could use SimpleVisual.Position, but to make it a little bit more interesting we use SimpleVisual.PositionSize and make the ball a little bigger. When assigning static values to a propery, it is possible to use array initialization, but the OpenModelica compiler does not accept array initialization if the equation contains other variables. obj. s i z e = {5, 5, 5 } ; obj. frame a [ 1 ] = 0 ; obj. frame a [2]= h+obj. s i z e [ 2 ] / 2 ; obj. frame a [ 3 ] = 0 ; A SimpleVisual.PositionSize object has two properties; size and frame_a. All are three dimensional real numbers, or Real[3] in Modelica. size controls the size of the visual representation of the object. frame_a contains the position of the object. 8.1.4 Running the simulation To be able to simulate the model with the added visualization, OpenModelica must load the SimpleVisual package. l o a d L i b r a r y ( Modelica. SimpleVisual ) Now, call simulate once more. This time the simulation will generate values for the added SimpleVisual object that can be read by the visualization in OMNotebook.

s i m u l a t e ( BouncingBall, s t a r t =0, end=5s ) ; 8.1.5 Starting the visualization To display the visualization, create a InputCell and call the visualize method in the input part of the cell. v i s u a l i z e ( BouncingBall ) ; Figure 8.1: BouncingBall example.

Chapter 9 Advanced example This chapter expands on the previous, encompassing all the features available in the SimpleVisual package. Multiple objects Rotation Scaling Different shapes Loading mesh data 9.1 The Pendulum model The model used in this example is a simple ideal 2D pendulum, not modeling properties like friction, air resistance etc. c l a s s MyPendulum Planar Pendulum c o n s t a n t Real PI =3.141592653589793; parameter Real m=1, g =9.81, L=5; Real F ; Real x ( s t a r t =5), y ( s t a r t =0) ; Real vx, vy ; equation m der ( vx )= (x/l) F ; m der ( vy )= (y/l) F m g ; der ( x )=vx ; der ( y )=vy ; xˆ2+yˆ2=l ˆ 2 ; end MyPendulum ; 35

Start by identifying the variables in the model that will be needed to create a visual representation of the simulation. Real x and Real y holds the current position of the pendulum. Real L is a parameter which holds the length of the pendulum. 9.2 Adding the visualization As before, to be able to use the SimpleVisual package we must import it. c l a s s MyPendulum Planar Pendulum import Modelica. SimpleVisual ;... Adding a sphere to represent the weight of the pendulum is done in the same way the BouncingBall was visualized. The variables x and y hold the position.... Real vx, vy ; SimpleVisual. P o s i t i o n S i z e b a l l c o l o r=red ; shape=sphere ; ; equation b a l l. s i z e = { 1. 5, 1. 5, 1. 5 } ; b a l l. frame a [1]= x ; b a l l. frame a [2]= y ; b a l l. frame a [ 3 ] = 0 ; m der ( vx )= (x/l) F ;... The next step is to create a visualization of the thread that holds the pendulum. It will be represented by a small elongated cube connected to the ball in one end and in the fixed center of the pendulum movement. We will want the object to rotate with the pendulum motion so create a SimpleVisual.PositionRotationSize object. SimpleVisual. P o s i t i o n R o t a t i o n S i z e thread shape=cube ;

To specify the rotation of an object, the visualization package uses two points. One is the position of the object, frame_a, that has been demonstrated earlier. The other position, frame_b, is interpreted as the end point of a vector from frame_a. This vector is used as the new up direction for the object. In this example, defining frame_b is simple. The cube that represents the thread will always be pointing to (0, 0, 0). We already know the length of the thread from the parameter L. thread. s i z e [ 1 ] = 0. 0 5 ; thread. s i z e [2]=L ; thread. s i z e [ 3 ] = 0. 0 5 ; thread. frame a [1]= x ; thread. frame a [2]= y ; thread. frame a [ 3 ] = 0 ; thread. frame b = {0, 0, 0 } ; Figure 9.1: Pendulum example. Running this simulation and starting the visualization, we notice that everything is not quite right. The thread is centered around the pendulum. We could calculate a new position by translating the x and y coordinates along the rotation vector, but there is a better way. Change the object type to SimpleVisual.PositionRotationSizeOffset. The offset parameter is a translation within the local coordinate system of the object. To shift

the center of the object to be at the bottom of the thread we add an offset of L/2 to the y component of offset. thread. s i z e [ 1 ] = 0. 0 5 ; thread. s i z e [2]=L ; thread. s i z e [ 3 ] = 0. 0 5 ; thread. frame a [1]= x ; thread. frame a [2]= y ; thread. frame a [ 3 ] = 0 ; thread. frame b = {0, 0, 0 } ; thread. o f f s e t [ 1 ] = 0 ; thread. o f f s e t [2]=L / 2 ; thread. o f f s e t [ 3 ] = 0 ; In the final model, a simple static fixture has also been added. c l a s s MyPendulum Planar Pendulum import Modelica. SimpleVisual ; c o n s t a n t Real PI =3.141592653589793; parameter Real m=1, g =9.81, L=5; Real F ; Real x ( s t a r t =5), y ( s t a r t =0) ; Real vx, vy ; SimpleVisual. P o s i t i o n S i z e b a l l c o l o r=red ; shape=sphere ; ; SimpleVisual. P o s i t i o n S i z e f i x t u r e shape=cube ; ; SimpleVisual. P o s i t i o n R o t a t i o n S i z e O f f s e t thread shape=cube ; ; equation f i x t u r e. s i z e = { 0. 5, 0. 1, 0. 5 } ; f i x t u r e. frame a = {0, 0, 0 } ; b a l l. s i z e = { 1. 5, 1. 5, 1. 5 } ; b a l l. frame a [1]= x ; b a l l. frame a [2]= y ; b a l l. frame a [ 3 ] = 0 ; thread. s i z e [ 1 ] = 0. 0 5 ; thread. s i z e [2]=L ; thread. s i z e [ 3 ] = 0. 0 5 ; thread. frame a [1]= x ; thread. frame a [2]= y ; thread. frame a [ 3 ] = 0 ; thread. frame b = {0, 0, 0 } ; thread. o f f s e t [ 1 ] = 0 ; thread. o f f s e t [2]=L / 2 ; thread. o f f s e t [ 3 ] = 0 ; m der ( vx )= (x/l) F ; m der ( vy )= (y/l) F m g ; der ( x )=vx ;

der ( y )=vy ; xˆ2+yˆ2=l ˆ 2 ; end MyPendulum ; Figure 9.2: Finished Pendulum. 9.3 Loading a shape By adding a model parameter to an object it is possible to use a 3D model to represent the object instead of a basic geometric shape. The model parameter contains a filename or an URL to a VRML file. In the pendulum example, change the declaration of the ball object. SimpleVisual. P o s i t i o n S i z e b a l l model=/models /bunny. wrl ; ;

Figure 9.3: Using a loaded mesh.

Chapter 10 Related Work This chapter will describe previous research and similar solutions implemented in commercial packages. 10.1 MultiBody library The MultiBody library contains a number of different parts, such as BodyBox, BodyCylinder etc. These use objects in the Visualizer package that contain information about the position, rotation and size of the object. The objects in the Visualizer package in turn use a Shape object which contains information about how the object should be displayed, for example as a box, sphere or using an external DXF mesh file. Like the visualization extensions to the OpenModelica compiler described in this thesis, the output from the Shape object can be captured and used to create a 3D visualization. This is done by the Dymola package created by Dynasim[13]. The visualization is very tightly coupled with the MultiBody library, but it can also be used to add visualization to a generic Modelica model. Compared to using the SimpleVisual package, the amount of work needed is similar. The Visualizer object can be linked to variables in a model using the connect statement which is a big improvement. Writing equations manually for all variables in a SimpleVisual object is tedious and error prone in comparison. They also use SI units throughout. Both of these differences are noted as a possible future addition to the SimpleVisual package in chapter 11. 10.2 3D Graphics and Modelica - an integrated approach In an article called 3D Graphics and Modelica - an integrated approach[22], Vadim Engelsson describes an annotation based solution in favor of the 41

class based approach. Both approaches are discussed and pros and cons are weighed against each other. Two advantages with the annotation approach described in the paper are: Annotations can describe mesh data in-line in a Modelica Model, since they are ignored by the compiler. This is not possible using a class based approach, but using parameters external mesh data can be loaded instead. A simulation of a model can simply ignore the visualization data in annotations if visualization is not needed, to speed up simulation time. In a class based approach, the coupling between model and visualization is much stronger and this is harder to achieve. annotation ( PrimitiveGeometry ( { T r a n s l a t e ( p o i n t ={1,0,0}), // T r a n s l a t e s both nodes Node ({ T r a n s l a t e ( p o i n t ={0,0,1}), // T r a n s l a t e s only one sphere Sphere ( p o i n t ={0,0,0}, r a d i u s =1) }), Node ({ T r a n s l a t e ( p o i n t ={0,0, 1}), // T r a n s l a t e s only one sphere Sphere ( p o i n t ={0,0,0}, r a d i u s =1) }) }) ) Figure 10.1: Visualization annotation example. The article does not motivate why the annotation approach was chosen, and does acknowledge that both methods have their drawbacks. Compared to an object based approach, the annotation approach requires the user to write more code to add visualization to a model. However, both methods could be used to build a package of re-usable visualizations. In this thesis, the added requirement of a possible adaptation to the Multibody library improves the case for the object based approach.

Chapter 11 Future work This chapter describes the authors view of where any extended work on the visualization package described in this thesis would be best concentrated. It lists features that was originally planned for inclusion in the thesis but left out because of time constraints as well as ideas and requirements that was first identified during work on the thesis. 11.1 Support visualizing MultiBody simulations One of the goals of this thesis was to create a simulation package that could be extended to use the visualization information in the MultiBody library. This section will outline the steps needed to fulfill that goal once the MultiBody library is supported by OpenModelica. 11.1.1 Replace type names The type names that are recognized as usable for visualization by the Open- Modelica compiler are hard coded in Interactive.mo. They have to be updated and extended with the classes in MultiBody.Visualizers. 11.1.2 Consolidate parameter names and types While some care has been taken to make the SimpleVisual package use parameter names like frame_a for position and frame_b for the relative position indicating direction just like the MultiBody library, there are most likely omissions of this principle. The MultiBody library uses a variable of type SI Pos for position while SimpleVisual uses Real[3]. The MultiBody library also defines its own types for color information and other parameters. 43

11.1.3 Extended shape support The SimpleVisual package can support visualizing an object of any shape using models of mesh format, but the MultiBody library defines a bigger set of standard shapes that must be recognized and rendered. Adding a shape requires a few lines of code, but is very simple. It must be given a keyword that can be used as a value to the shape property, and the visualization must recognize the keyword and create a corresponding primitive using Coin3D. Coin3D can import mesh data or vector data could be procedurally generated within OMNotebook. Pipe Beam Gearwheel Spring 11.2 Utilize Modelica better There are a couple of issues with the way the visualization package utilizes the Modelica language that could be improved. Some of these are required to support the MultiBody library. Using class parameters The visualization package uses information encoded in object comments to find out about parameters such as the color or shape of an object. The comments are not the best place to do this, since the compiler doesn t check the syntax of comments, nor can type checking be done. Ideally, class parameters in the SimpleVisual package should be used. Use connectors Connectors is a feature of Modelica which enables easier integration of complex models. Connectors in Modelica are similar to interfaces in object oriented languages. Extending the classes in the SimpleVisual package to support connectors would enable users to simply use the connect statement to link a simulation variable to the visualization instead of specifying each equation individually. Use Modelica.Units The Units package available in the Modelica Library defines standard types such as SI position. The SimpleVisual package stores variables as Real[3], and does not contain any unit information. Extending the SimpleVisual package to with connectors for SI units could simplify integration with other models. Recursive search In the trial implementation, no recursive search for objects to simulate is done. Only SimpleVisual objects declared in the top-level model are visualized.