Instance generation from meta-models (for model transformation testing)

Size: px

Start display at page:

Download "Instance generation from meta-models (for model transformation testing)"

Collin Marshall Harris
5 years ago
Views:

1 Instance generation from meta-models (for model transformation testing) Robbe De Jongh University of Antwerp Abstract Testing model transformations is a tedious job. One needs to make a representative set out of the possibly infinite set of input models, run the transformation on it, and confirm that the output is the expected one. Generating input models automatically seems to be a good first step in automating the entire process. In this paper we discuss the model fragment strategy of this generation (Brottier et al. (2006)) and we try our hands on making an algorithm to generate instance models for the RPG in metadepth. This algorithm in no way can be used as a general approach to solve this problem however, as it highly depends on knowledge about the requirements and the designed meta model. Keywords: testing Meta Modelling, Instance generation, Model transformation 1. Introduction. Model Driven Engineering is widely spreading as the next best thing in making software. But a vital part of making software, is being able to ensure it will do what you want it to do. To be able to claim and validate that it does, we need to test everything about it. As model transformations are a key part in MDE, testing model transformations is very important. You might say: We already know how to test software, we did all that for Object-Oriented programs, I m sure we can do the same here. Baudry et al. (2009) discussed why it is not trivial to use known testing techniques to test model transformations as well as some possible solutions on how to solve the issues. In short, address: robbe.dejongh@student.ua.ac.be (Robbe De Jongh) Preprint submitted to Elsevier January 20, 2013

2 those issues come down to the complexity of the input and output (models in stead of strings, note how models can be represented in various ways), the lack of tool support, which is a consequence of the rather new approach of course, and the heterogeneity of transformations and techniques. (As we saw in the MoSIS ( course, a different situation/problem calls for a different modelling language).. We don t have to reinvent everything either, though, as testing a model transformation comes down to giving it input, and checking if the output is the desired output for the input model. This is black box testing, we don t know or care how the transformation happens, we just give it input and see what comes out. The first non trivial part of that strategy is giving the transformation input. Manually making instances that conform to some meta model is not a fun job to do, and it only gets worse when you think of the properties your input cases should have to make a good test bench. Therefore, we try to automate the generation of instance models to hand off to our model transformation. This article is about a few strategies on how to accomplish that. We also try to implement some of it to test our model transformation/simulation we made in the MDE ( course. More precisely we will try to generate instance models in Metadepth for the meta model created for assignment 1. In the following section we discuss some background information about the whole topic of model transformations, modelling and meta modelling. In section 3 we will discuss various strategies to generate models from a meta model, in section 4 we will elaborate on our experiment, related work will be discussed in section 5 and we will conclude and talk about future work in section Background. The goal in searching for different programming paradigms or designs is mostly the ease of use for the programmer, or even to eliminate the need of a programmer and to enable domain specific experts to use their knowledge without needing to know how to program. In many cases, big changes in the style of making software meant more abstraction. From machine code to assembler code. From assembler code to high level programming languages. From programming languages to... modelling languages. Expressing yourself and understanding things u see is a lot easier if those things are not plain 2

3 text. So making models is easier to do and easier to understand in many cases for those involved, as domain knowledge leads to a great understanding of the model, without the need to know what a pointer is.. How should we then make models? When making program code, you have a syntax and semantics to stick too if you want the code to do when you want and expect it to do. When modelling, you want your model to conform to a meta model in the same way. You want to be able to give a precise meaning to a circle, without necessarily describing the details of the circle. You want to be able to give a precise meaning to a for-loop, without describing what the body of the loop will do. This is what the meta model is for. It describes what kind of structures you can use in your model and how they can be connected or related. In most cases, this meta model again has its own meta model, called the meta meta model, describing the structure of the meta model. This meta meta model in most cases is a UML Class diagram, which can be seen as its own meta model as well.. We have models, but what can be do with them? There are various uses of models. Some modelling languages enable you to verify properties of your design, and afterwards can be transformed into different models to check different properties. Some models are transformed to program code and executed or simulated. This transformation to program code can be seen as the equivalent of a compiler, from program code, to assembler or machine code. After decades of using and optimizing compilers, we all feel like a compiler will create us a program that when executed will do what we coded it to do (most of the time, this will not be 100 percent what we thought it would be though...). This trust has yet to be reached in model transformations, as the ways to test the transformations is harder, and newer than the compiler. It is however a key issue in making modelling and model transformation a viable component of software engineering. To be able to get this validation of our transformation, we need ways to test it, and we need ways to show that those tests suffice in making sure that the transformation is correct. There are various testing techniques, like white-box testing, where one knows the implementation of the system under study and black-box testing, where one does not know. With white-box testing, it is possible to choose test cases that cover the whole implementation (think of the implementation as program code, possibly transformed/compiled from a model). In black-box testing, this is not possible. The only thing you can do, is to make instance models, 3

4 that span as many as possible variations that are allowed while conforming to the meta model. The set of allowed instance models might be infinite, so generating all instance models is not an option. You have to generate instance models, and preferably not a huge amount, but still enough to be able to say that u covered all possibilities. So you need something to be able to say how much of the meta model you cover with the instance models. Another big part of the transformation testing, is the oracle or the expected output for each model. Both of those last two things mentioned are out of the scope of this article. We will focus on simply generating instance models, although we keep in mind that we want to be able to use them for testing the transformation. 3. Generation Strategy 3.1. Model Fragments Brottier et al. (2006) propose a three-step way of going from a meta model to input models. They propose a partitioning of simple types from the meta model to create equivalence classes of simply typed attributes and association cardinalities. Those equivalence classes are then used to create model fragments, which correspond with the test objectives of the transformation. The last step is generating input models based on the model fragments. The meta model they start with is a meta model, with addition of extra constraints if necessary to make the possible input set of the transformation equal to the set of models that conform to the meta model. Category-Partition testing is a known technique in software engineering, which consists of making equivalence classes and using a value in each class as a test case. This technique is adapted to fit model transformations. The authors propose two partitioning strategies, namely default partitioning and knowledge-based partitioning. Default partitioning consists of defining a partition for each type or structure. A string type for example would lead to the partitioning: {{null}, { }, {s with length(s) larger than 0}}. Knowledge-based partitioning uses values provided by the tester or knowledge extracted from the model transformation to get representative values. To test a model transformation, one needs interesting instance models. Interesting models require interesting object structures. These interesting object structures are called model fragments and consist of multiple object fragments. An object fragment specifies a partial instance of a class. It relates equivalence classes with properties. It is a 4

5 set of property constraints, that associate a property of a class to an equivalence class. A model fragment comes down to a list of partitions with a specifically chosen value. To generate actual instance models from the model fragments, they provide an algorithm in pseudo code. The algorithm has various points where one can choose a different strategy, thereby changing the generated output Algorithm input : set of model fragments S, meta-model MM output : set of models L conformed to MM while there are uncovered model fragments in S do { create an empty model M while the model size limit is not reached (1) and M still can grow do { choose an uncovered model fragment MF in S for each object fragment OF in MF do { find an object O which is instance of the class partially specified by OF (3) for each constraint CT defined in OF on the property P do { if P is an attribute (value partition case) then choose a value and set it to P in O (5) else (multiplicity partition case) { choose a cardinality N according to CT (5) if the type of P is a class then find N objects with a P type and set them to P in O (3) else find N values in the partition of P and set them to P in O (5) }} add O to M completion of M until it is conformed to MM (2, 4) }} mark MF as covered add M to L} The numbers between brackets stand of those strategy options. The following list shows what they mean and what the different options are at each point. 1: Size of Models: max objects number, min model fragments number. 5

6 2: Model Completion: naive, path. 3: Object Choice (during model fragment coverage): always create, reuse whenever possible, new object for each OF. 4: Object Choice (during model completion): always create, reuse whenever possible. 5: EC choice in partition and value choice in EC: default, limits, cycle. (EC = Equivalent Class). Sen et al. (2008) introduce a tool called Cartier in which meta models and all of the necessary additions can be transformed to a constraint language, namely Alloy. Those additions to the meta model consist of transformation preconditions (OCL constraints) and test objectives, or model fragments (also in the form of OCL constraints). The meta models they work with are Ecore class diagrams, with additional OCL constraints. The Alloy model is then transformed into a boolean formula in conjunctive normal form, which is then solved by a satisfiability solver. This solution is then transformed back to its representation in the syntax of the meta model.. Sen et al. (2009) then go on to use the Cartier tool to test various strategies of which info to give the system. They then use mutation analysis to see how much of the input set of the transformation they covered. They did a random or unguided generation and a model fragment based generation. 4. Experiment: Generating Instance Models for RPG in Metadepth. For this experiment, we created an eol operation that adds objects to an initially empty metadepth model. The amount of objects of each class we have is a random number chosen between a minimum and maximum value. These values are hard-coded into the operation, but they could be derived from the meta model constraints, or the tester/ developer could provide good values that make sense for the transformation. Some of these values depend on the randomly chosen value of previous classes. The algorithm is highly dependant on knowledge of the meta model, and the requirements for defining meaningfull or interesting values for the minima and maxima. This knowledge is also critical in defining the order of making objects, and how to properly link them. Due to the time limits of this experiment, we 6

7 were not able to implement the generation of doors, as this leads to a lot more difficulties than any other tiles due to the fact that one needs information about multiple scenes. But this does not mean we can not make valid models that can test a big part of the transformation. If we only make one scene (by setting the minimum and maximum amount of scenes to one), there is no need for doors (there can not be a door as it must be a portal to a different scene), and we can test everything except for the movement of villains in different scenes, the actual travelling through doors and the key system for doors.. The algorithm puts serious limitations on the input domain of the transformation, partially by the values we chose for interesting models, as well as the simplicity or structure of the algorithm, an example of this is the scene layout. All scenes are rectangles, as this is very easy to do in two nested for loops (looping over the width and height). Another example of this is that the hero always gets placed on the first regular tile that is made. While this might be the most logical place to put it when u manually make a model, the meta model does not make a constraint that this must happen, while our generated instances will be like that. But the transformation, that we had already implemented, does not require this. So it is not a constraint coming from the meta model or the model transformation, it is our design choice when making the algorithm. But as one would like to have a general algorithm for any meta model, this is not good practice. Again, as we had a time limit, the ease of designing it so beats the overall goal to have a general algorithm for any meta model (for now).. Eol and metadepth unfortunately did not allow me to simply store the generated instance as a metadepth model that i could load again at a later time. The metadepth command dump does show all the objects created though, so one could use that information to make a file that contains that specific model, which would still be easier than thinking of a model myself though, as I could just see what was created by the generator and put the equivalent in a.mdepth file. 5. Related Work. The previously mentioned studies (Brottier et al. (2006), Sen et al. (2008), Sen et al. (2009) and Baudry et al. (2009)) seem to be the main researchers studying the issue of generating instance models from a meta model with the 7

8 purpose of testing model transformations. Wu et al. (2012) did a research on the topic of generating instance models from a meta model in general and found four main uses for the instance generation, one of which is model transformation testing. Ehrig et al. (2008) propose a graph grammar that generates instance models from meta models automatically. 6. Conclusion and future work As Baudry et al. (2009) show, there are plenty of things that need to or should be worked on if we want fully automated instance generation from a meta model. My own experiences give me the idea that the glaring needs are standardization of meta models, and the additional constraints both on the meta model and the transformation, which might already exist but never really showed up in the papers I read. Another need is a stand alone mechanism in stead of a system like Alloy, that is made for the purpose of testing model transformations. Like Sen et al. (2009) said that it is their goal or plan to eventually replace the translation to Alloy with a translation to a stand alone language that is made with the purpose of helping in the generation of instance models. Another need, or maybe better to call it a nice addition, would be a single tool, rather than the tool chain used. This would make things a lot easier and faster, as a developer only needs to learn or conform to one tool rather than using output of one tool as input for the next. This topic has gradually increased my interest in testing model transformations automatically, as well as generating instance models automatically, in a kind of software factory way, as our algorithm in the experiment can automatically generate a lot of similar, but still different enough playing fields that could be seen as different games or levels. As I do not have a master thesis topic yet, I will explore if this topic is possible to advance in, as I already have a small background of information, as well as a good memory of how much of a burden it is to manually make instance models. References Baudry, B., Ghosh, S., Fleurey, F., France, R., Traon, Y. L., Mottu, J.-M., Barriers to systematic model transformation testing. Brottier, E., Fleurey, F., Steel, J., Baudry, B., Traon, Y. L., Metamodel-based test generation for model transformations: an algorithm and a tool. 8

9 Ehrig, K., Kster, J. M., Taentzer, G., Winkelmann, J., Generating instance models from meta models. Sen, S., Baudry, B., Mottu, J.-M., On combining multi-formalism knowledge to select models for model transformation testing. Sen, S., Baudry, B., Mottu, J.-M., Automatic model generation strategies for model transformation testing. Wu, H., Monahan, R., Power, J. F., Metamodel instance generation: a systematic literature review. 9

DSM model-to-text generation: from MetaDepth to Android with EGL

DSM model-to-text generation: from MetaDepth to Android with EGL Rafael Ugaz Antwerp University (Belgium), rafaelugaz@gmail.com Abstract This article describes the process of a specific model-to-text transformation