Data generation framework for EMF models. Igor Palkoci

Transcription

1 Czech Technical University in Prague Faculty of Electrical Engineering Department of Computer Science and Engineering Bachelor s Project Data generation framework for EMF models Igor Palkoci Supervisor: Ing. Martin Tamáš Study Programme: Softwarové technologie a management, Bakalářský Field of Study: Softwarové inženýrství May 25, 2012

2 iv

3 v Acknowledgements I would like to thank to my current and former supervisors, Ing. Martin Tamáš and Yury Chernikov, for their help. I would also like to thank to Ing. Tomáš Pour and ALVILA SYSTEMS s.r.o. for providing the opportunity and conditions to work at the project. Last but not least, I would like to thank to my father, Ing. Emil Palkoci, for his support.

4 vi

5 vii Declaration I hereby declare that I have completed this thesis independently and that I have listed all the literature and publications used. I have no objection to usage of this work in compliance with the act 60 Zákon č. 121/2000Sb. (copyright law), and with the rights connected with the copyright act including the changes in the act. In Prague on May 25,

6 viii

7 Abstract The goal of this project is to create a data generation framework, written in Java, that will facilitate generation of instances of EMF models. The project designs and implements Data Definition Model, a decoration model of Ecore models, and Data Generator that generates data-filled instances of the input EMF domain models. The final product is implemented as Eclipse plug-ins installable into Eclipse IDE. The framework potential is demonstrated on the Company example. In addition, other existing open source solutions for synthetic data generation are outlined. Abstrakt Cieľom projektu je vytvoriť framework napísaný v programovacom jazyku Java, ktorý umožní generovanie inštancií EMF modelov. V projekte je navrhnutý a implementovaný Data Definition Model, dekorátor Ecore modelov, a Data Generator, ktorý generuje inštancie vstupných doménových modelov a naplňuje ich dátami. Výsledný produkt je implementovaný v podobe zásuvných modulov, ktoré je možno inštalovať do vývojového prostredia Eclipse. Potenciál frameworku je demonštrovaný na ukážkovom príklade Company. Sú spomenuté aj existujúce open source riešenia syntetických generátorov dát. ix

8 x

9 Contents 1 Introduction 1 2 Eclipse Modeling Framework EMF Model Ecore Creating an EMF Model Generating Code Reflective API Existing Data Generators Generatedata.com DBMonster Databene Benerator Analysis and Design Requirements Discussion Framework Components Data Definition Model Instance Definitions Data Type Definitions Dictionaries Root Elements Generating DDM DDM Editor Data Generator Data Generator API Implementation Built-In Dictionaries Custom Dictionaries Data Generator Wizards Data Definition Model Wizard Instance Wizard xi

10 xii CONTENTS 6 Company Example Company Ecore Model Company Data Definition Model Company Instance Conclusion 47 Bibliography 49 A List of Abbreviations 51 B Installation and User Guide 53 B.1 Requirements B.2 Installing Plug-Ins B.3 Creating a Domain Model B.4 Creating a Data Definition Model B.5 Editing a Data Definition Model B.6 Creating a Model Instance C Contents of Attached CD 57

11 List of Figures 2.1 Model Import and Generation[1] Ecore Kernel Instance of Ecore Generatedata.com - GUI Databene Benerator - Multi-Platform Generation [6] Framework Components DDM - Instance Definition DDM - Data Type Definition DDM - Enumerations DDM - Dictionaries DDM - Root Contents Data Definition Model Wizard Instance Wizard Company Ecore Diagram Company DDM - Name Data Type Definition Properties Company Data Definition Model Company DDM - Employee Properties Company Instance xiii

12 xiv LIST OF FIGURES

13 Chapter 1 Introduction Motivation This project was assigned by ALVILA SYSTEMS s.r.o. Almost every software system works with a data model, also called a domain model, even if not explicitly defined. At software development process, developers and testers need available data, instances of the data model, for purposes of software testing. It is often desirable that testing data resembles real valid data of a specific domain modelled by a data model. Such data is rarely available before or during development. A common approach is to prepare the data manually. This can be, however, a very timeconsuming and labour-intensive procedure. In addition, consider the potential complexity of the used data model which tends to increase over time as well as its frequent redesigning and adjusting. Creating valid instances of the entire model could be therefore a challenging task. For these reasons, an automatic data generator, capable of generating data out of the current data model at whichever point of the development process, can be a beneficial and time-saving tool. Eclipse Modeling Framework (EMF), a framework which adapts the MDA (Model Driven Architecture) concepts of using models as a central element of development, provides tools for creating manipulating and generating code of models. EMF-based applications, which model their data models with EMF, can utilize all the benefits that the EMF framework provides. Since ALVILA SYSTEMS uses the EMF framework for developing its products, there is a demand for automatic data generator for EMF-based data models, i.e., EMF models. Goals The goal of this project is to create a data generation framework, written in Java, that will facilitate generation of instances of EMF models. The first step will be to make an introduction of the EMF framework and describe the mechanism of EMF models creation. In addition, other existing solutions for synthetic data generation will be outlined. Afterwards, an analysis of the requirements of the project will 1

14 2 CHAPTER 1. INTRODUCTION be performed and their realization will be designed. Finally, a prototype of the framework, capable of processing simple EMF models, will be implemented. The functionality of the framework will be demonstrated by creating an example EMF domain model and performing all the necessary steps to generate its data-filled instances. The framework is expected to meet these additional requirements: The final product will be implemented as Eclipse plug-ins that will be installable into Eclipse IDE. Data generation will be easily configurable via GUI. There will be an option to persist the generated model instances in XMI format. Structure of the Document Chapter 2, introduces Eclipse Modeling Framework and EMF models. It also describes EMF s meta-model, the Ecore model, and other EMF facilities, such as creating Ecore models, generating Java code or EMF Reflective API. Chapter 3 provides an overview of other existing solutions for synthetic data generation. The analysis and design of the framework developed as part of this project are described in Chapter 4. The implementation of the framework is described in Chapter 5. Finally, the chapter 6 provides an illustrative step-by-step example for using of the framework on a sample Company model. The conclusion, Chapter 7, summarizes the entire project and outlines potential enhancements of the framework.

15 Chapter 2 Eclipse Modeling Framework In this chapter, we will make a brief overview of the EMF framework and its application. For more thorough information on everything the framework is capable of, you need to read the official documentation. The sources of the information found in this chapter were [2], [10], [1], [11] and [5]. Eclipse Modeling Framework (EMF) is an open source framework developed in Java, based on the Eclipse platform[9]. According to the EMF wiki, "The Eclipse Modeling Framework (EMF) project is a modeling framework and code generation facility for building tools and other applications based on a structured data model.[11]" Another source says "EMF is a simple, pragmatic approach to modeling.[1]" Evidently, modeling is the essential application of EMF and as the previous definitions imply, EMF is used to create models, generate code out of the models and build tools based on them. Additionally, it provides XMI serialization, change notifications and reflective API for generic manipulation of EMF models. We will describe the particular facilities in the subsequent text. 2.1 EMF Model An EMF model is a product of the EMF framework. It is essentially a unification of XML, UML and Java code. In other words, EMF models describe application data models and that description is available in all the mentioned forms at any time, simply by generating each one out of any other. For illustration see Figure Ecore Every EMF model is represented by the Ecore model, Ecore is therefore the meta-model of EMF models. The Ecore model itself is also represented by Ecore, being its own metamodel, thus meta-meta-model. Ecore is in the center of the whole EMF framework and every EMF model is described by it. A simplified core subset of Ecore, the Ecore kernel, is illustrated in Figure 2.2. As we can see, the Ecore model is very similar to UML Class Diagram. In fact, Ecore is essentially UML Class Diagram, leaving out a few features. Comparing to EMF models, 3

16 4 CHAPTER 2. ECLIPSE MODELING FRAMEWORK UML XML Schema I M P O R T Ecore Model GENERATE Java Model Java Edit Java Editor Figure 2.1: Model Import and Generation[1] UML models also the behaviour of a system, whereas the Ecore model defines only data structure. To understand the structure of Ecore more easily, we can describe particular elements comparing them to elements of UML Class Diagram. In UML Class Diagram, an entity is represented by a class; its Ecore equivalent is EClass. Each UML class has two sections; the first contains class attributes, the second class methods. The attributes store data belonging to the class, while the methods are the things that the class does. In Ecore, an attribute is represented by Eattribute and a method by EOperation. The relationships between classes are called associations in UML, represented by EReference in Ecore. There is a specific kind of EReference a containment reference that is an equivalent of an UML composition association. As in UML, references can be either one-way (uni-directional) or two-way (bi-directional). Finally, we define the reference multiplicity that specify a number of referred classes. Figure 2.2: Ecore Kernel Let us examine the structure of the Ecore model more closely. We can see the diagram of the Ecore kernel in Figure 2.2. The actual Ecore is of course far more complex, however,

17 2.1. EMF MODEL 5 the presented subset is sufficient to illustrate basic elements. EClass represents EMF model classes. Every class has zero, one, or several super classes. A class contains a collection of structural features, represented by EStructuralFeature, the common superclass of EReference and EAttribute. An attribute (EAttribute) has exactly one data type, modelled by EDataType; it can be a primitive type or a Java object. In the Ecore model, several data types are predefined, including all Java primitive data types plus, for example, java.lang.string or java.util.date. Every reference (EReference) has zero or exactly one opposite EReference; an association is two-way if the eopposite reference is not null, otherwise the association is one-way. The ereferencetype reference assigns an EClass, the reference type, to every EReference. The containment attribute specifies whether the reference is containment or not. EClasses and EStructuralFeatures are identified by name. The name attribute is actually an attribute of Ecore s ENamedEelement that is the ancestor of EClass, EAttribute and EReference. There are two remaining attributes in the diagram: lowerbound and upperbound. These attributes describe the multiplicity of an EStructuralFeature; lowerbound specify the minimum of allowed referred instances, upperbound the maximum. If upperbound is set to -1 the number of referred values is boundless. The black diamond on one end of the estructuralfeatures() reference in Figure 2.2 signifies that the reference is a containment. A containment reference, a composition association in UML, represents a whole-part relationship between objects. It says that the whole consists, and manages life-cycle, of the parts. In EMF, containment references play significant role in serialization. : EClass name = Company : EAttribute name = name = EDataType : EAttribute name = established : EReference name = employees eattributetype eattributetype ereferencetype : EDataType name = EString : EDataType name = EDate : EClass name = Employee Figure 2.3: Instance of Ecore

18 6 CHAPTER 2. ECLIPSE MODELING FRAMEWORK Ecore Instance Figure 2.3 shows an instance of the Ecore model, an extract of the Company model shown in Figure 6.1. There are two instances of Ecore s EClass, having the name attribute set to "Company" and "Employee", respectively. In addition, we have two EAttribute objects, the first having set name="name", the second name="established". The Ettribute objects have assigned the corresponding EDataType objects: EString that wraps Java String and EDate that wraps java.util.date. Finally, there is the EReference object, having set name="employees", referring to the Employee object Creating an EMF Model There are at least three ways in which an Ecore model can be defined, i.e., a meta-model of any EMF model: XML Schema, annotated Java interfaces or UML Class Diagram. The first option is suitable for XML experts; however, it can be quite tedious. The second approach might suit pure-java developers, without having to be concerned about UML or XML. The last option, UML Class Diagram, is probably the most convenient and user-friendly. For whole picture see 2.1. Java Interfaces If we choose the Java interfaces approach, we need to create Java interfaces and annotate them with appropriate EMF annotations. For example, if we pick the Company class from the Company example (Figure 6.1), we need to create the following Java interface: /** */ public interface Company { /** type="employee" containment="true" */ List<Employee> getemployees(); /** type="branch" containment="true" required="true" */ List<Branch> getbranches(); /** */ String getname(); /** */ java.util.date getestablished(); }

19 2.1. EMF MODEL 7 To ensure that all the elements of the interface will be parts of the model definition, we need to include annotation in their Javadoc comment. Otherwise, the element will be ignored by EMF. In the example, tags distribution has the following implications. Company is a modelled class, containing two attributes: name and established, and two references: branches and employees. The attributes do not need additional model information because the default are sufficient. However, additional non-default model information for the references is necessary to be defined explicitly. Setting up containment="true" defines that the instances of Company will be containers of Employee and Branch instances. The lowerbound attribute of the branches reference will be set to 1 because we set required="true". Finally, the type property specifies the data type of the referred objects, setting Employee for the employees reference and Branch for the branches reference. We need to declare the data type because the List return types indicate that the references are multiplicity-many. Since Java 5.0, the type property can be replaced by using generics to specify the types of referred objects. Notice that we do not need to declare any setter methods, EMF will generate them automatically where necessary. To get the complete model, we likewise need to specify all the remaining classes, such as Branch and Employee. UML Class Diagram As for the UML approach, there are several options. We can create an Ecore model directly. EMF provides a graphical tree-based editor, called Sample Ecore Model Editor, for creating and editing EMF models. Another very convenient tool is a UML-notation based editor, provided by the Ecore Tools project[15], called Graphical Ecore Editor. Actually, Graphical Ecore Editor was used to create the diagram shown in Figure 6.1. A little different approach is to import UML models from external UML tools. However, EMF directly supports only Rational Rose (.mdl files). The Eclipse UML2 project[17] provides an importer for UML 2.x models. Serialization Although an Ecore model can be represented in various formats, i.e. XML Schema, Java classes, UML Class Diagram, there is actually another, EMF-canonical, representation, XML Metadata Interchange (XMI)[14]. XMI bridges XML with EMF models. The structure of an XMI document respects naming and hierarchy of EMF models, following the containment hierarchy. As a result, the similarity of an EMF model and its XMI serialization is easily observable. A small snippet of the Company Ecore model in serialized XMI format may look like this: <?xml version="1.0" encoding="utf-8"?> <ecore:epackage xmi:version="2.0" xmlns:xmi=" xmlns:xsi=" xmlns:ecore=" name="company"

20 8 CHAPTER 2. ECLIPSE MODELING FRAMEWORK nsuri=" nsprefix="org.alvila.datagenerator.examples.company"> <eclassifiers xsi:type="ecore:eclass" name="company"> <estructuralfeatures xsi:type="ecore:ereference" name="employees" upperbound="-1" etype="#//employee" containment="true" resolveproxies="false"/> <estructuralfeatures xsi:type="ecore:ereference" name="branches" lowerbound="1" upperbound="-1" etype="#//branch" containment="true"/> <estructuralfeatures xsi:type="ecore:eattribute" name="name" etype="ecore:edatatype <estructuralfeatures xsi:type="ecore:eattribute" name="established" etype="ecore:edatatype </eclassifiers>... </ecore:epackage> 2.2 Generating Code Once we have an Ecore model ready, we can generate Java code out of it, with only a couple of mouse clicks. Model Classes For every Ecore class, i.e., EClass, EMF will generate an interface and its implementation class. For example, if we take the Company interface from Section 2.1.2, EMF will generate the following Company interface and CompanyImpl class: public interface Company extends EObject { EList<Employee> getemployees(); EList<Branch> getbranches(); String getname(); void setname(string value); Date getestablished(); void setestablished(date value); } public class CompanyImpl extends EObjectImpl implements Company {... } Let us examine the generated interface. There are two pairs of getter and setter methods: get/setname() and get/setestablished(). These methods manipulate the values of the name and established attributes. Methods getemployees() and getbranches() are

21 2.2. GENERATING CODE 9 generated for the references employees and branches. Since the references are multiplicitymany, having List return values, they do not need setter methods. Notice that Company extends another interface, EObject, the base interface of every generated model class. The EObject interface, the EMF equivalent of java.lang.object, contains, among others, the following methods: public interface EObject extends Notifier {... EClass eclass(); Resource eresource(); EObject econtainer(); Object eget(estructuralfeature feature); void eset(estructuralfeature feature, Object newvalue); boolean eisset(estructuralfeature feature); void eunset(estructuralfeature feature);... } The eclass() method returns the meta-object (i.e., an EClass) of the object. In addition, there are reflective accessors: eget(), eset(), eunset() and eisset(). The previous methods are useful for accessing the object s data generically. We will examine the approach more closely in Section 2.3. Finally, econtainer() returns the container object, i.e., the object which contain this object, and eresource() returns the object s resource. Factory and Package Besides the generated model classes, EMF will generate two important pairs of classes (a pair of an interface and its implementation): a factory and a package; e.g., CompanyPackage and CompanyFactory. The factory is usually used to create instances of the model classes, it contains methods to create each one of them. For example, to create an instance of Employee, we can use the statement: Employee employee = CompanyFactory.eINSTANCE.createEmployee(); A package provides access to all Ecore meta-data of the model as well as additional information about the package. Let us examine the portion of CompanyPackage: public interface CompanyPackage extends EPackage { String ename = "company"; String ens_uri = " CompanyPackage einstance =... EClass getcompany(); EReference getcompany_employees(); EAttribute getcompany_name(); EEnum getsex();

22 10 CHAPTER 2. ECLIPSE MODELING FRAMEWORK }... interface Literals { EClass COMPANY = einstance.getcompany(); EReference COMPANY EMPLOYEES = einstance.getcompany_employees();... } The ens_uri variable stores the namespace URI of the package; CompanyPackage s URI is You can see how the particular meta-objects, such as EReferences, EAttributes, EClasses and EEnums, can be accessed. Notice how the Literals subinterface provides the accessors for the meta-data through einstance, the accessor for the package singleton instance. 2.3 Reflective API The EMF Reflective API is a very powerful feature of the EMF framework for manipulating EMF objects generically. Since every generated model class extends the EObject interface, it inherits the set of reflective methods: EClass eclass(); Object eget(estructuralfeature feature); void eset(estructuralfeature feature, Object newvalue); boolean eisset(estructuralfeature feature); void eunset(estructuralfeature feature); It is possible to create instances of model classes, e.g., EClasses, reflectively. For example, if we want to create an instance of Employee, we can use the statements: EClass employeeclass = (EClass)companyPackage.getEClassifier("Employee"); Employee employee = (Employee)companyFactory.create(employeeClass); If we want to retrieve or set the value of an attribute or a reference, e.g. the firstname attribute of the Employee, we can do the following: EAttribute nameattribute = (EAttribute)employeeClass.getEStructuralFeature("firstName"); String name = (String)employee.eGet(nameAttribute); employee.eset(nameattribute, "Michael"); Another option is to obtain the meta-objects directly: EClass employeeclass = CompanyPackage.eINSTANCE.getEmployee(); EAttribute nameattribute = CompanyPackage.eINSTANCE.getEmployee_FirstName(); There are two methods remaining: eunset() and eisset(). As their names indicate, eisset() can find out whether the specified structural feature has been set, whereas eunset() unsets or resets it.

23 Chapter 3 Existing Data Generators This chapter will outline existing solutions for synthetic data generation. There are several frameworks or libraries that provide data generation[8]. Unfortunately, the complex, stable and eligible tools are in most cases commercial products. Another unfavourable phenomenon is that many are provided only for the Windows platform. In this preview, we will focus only on solutions in the open source world. The open source generators are in most cases very small single-developer projects; several of them ended before releasing a stable version. We will present three open source products: generatedata.com, DBMonster and Databene Benerator. 3.1 Generatedata.com Generatedata.com[12] is a free, open source (under GNU licence) script written in PHP and JavaScript, using the MySQL database. It is a web application with simple graphical user interface (GUI), shown in Figure 3.1; a demo application is available on-line. The application GUI has a form wherein a user specifies the following information: result type Provided are: HTML, Excel, XML, CSV or SQL country-specific data Data sets of four countries are available: Canada, Netherlands, United Kingdom and USA number of results form management (not available in the demo) A user can save the current form or load previously saved forms table of data definition Rows with data specification: column/node title, data type and type-specific options output-specific options For example, if selecting the SQL result type, the user can specify the database and table name; if XML, node names 11

24 12 CHAPTER 3. EXISTING DATA GENERATORS Figure 3.1: Generatedata.com - GUI

25 3.2. DBMONSTER 13 The application enables a user to generate a wide variety of data types. Human data, such as names, addresses, phone numbers, dates, address elements (city, street, zip code,...). Random numbers with constraints, such as range or increment. Lorem ipsum 1 text with a fixed or variable number of words. Alpha-numeric strings generated according to specific regular expressions. And finally, user-specific collections. The resulting data can have several formats: displayed as an HTML page, generated to an Excel, XML or CSV file, SQL statements for the MySQL or Oracle database. For example, the form in Figure 3.1 might produce the output: <records> <record> <name>remedios Mckenzie</name> < >placerat@at.com</ > <birthdate>may 30, 1966</birthDate> <maritalstatus>single</maritalstatus> <phone> </phone> </record> <record> <name>ori Gonzalez</name> < >diam@luctus.edu</ > <birthdate>jun 4, 1969</birthDate> <maritalstatus>divorced</maritalstatus> <phone> </phone> </record> </records> 3.2 DBMonster DBMonster[4] is a small and simple tool implemented in Java under Apache Software License 1.1. It is used to feed relational databases with random data. Since DBMonster is capable of generating high volumes of data, it is applicable for load or performance testing. The generation is table-oriented, which means that the tables are filled one by one. The order of the processed tables is computed dynamically at run time. If, for example, the currently processed table contains a column which with foreign keys referring to another table, the referred table is then processed primarily. DBMonster can work with several database systems: PostgreSQL, MySQL, Oracle, HSQLDB, MSSQL or any database providing a JDBC driver. Configuration DBMonster uses two kinds of configuration files. A dbmonster.properties file stores information about the connected database, such as the database name, driver, user and password as well as some additional properties. 1 a Latin placeholder text

26 14 CHAPTER 3. EXISTING DATA GENERATORS Database tables, its columns and value restrictions are defined via an XML-format schema, i.e., DBMonster s proprietary meta-data definition. DBMonster enables a user to grab the schema automatically from the database; it can be further overwritten. The other option is to create the schema manually from scratch. The following XML snippet is an example of such a schema: <dbmonster-schema> <name>test Schema</name> <table name="table01" rows="1500"> <key databasedefault="false"> <generator type="pl.kernelpanic.dbmonster.generator.maxkeygenerator"> <property name="columnname" value="id"/> </generator> </key> <column name="login"> <generator type="pl.kernelpanic.dbmonster.generator.stringgenerator"> <property name="nulls" value="0"/> <property name="minlength" value="4"/> <property name="maxlength" value="12"/> <property name="allowspaces" value="false"/> </generator> </column> </table> </dbmonster-schema> The schema defines one table named table01 with two columns: id and login. The id column is defined as a primary key by the <key> element and its value is to be generated using the MaxKeyGenerator generator. The value of the login column is to be generated by StringGenerator, with couple of additional restrictions: the resulting values will be non-null strings of length between 4 and 12 characters, eventually containing spaces. The number of rows will be 1500, as defined in the rows attribute. In addition to the previous configuration files, DBMonster can also execute SQL scripts before and after generation. The scripts can contain DDL commands to drop or create database tables. Generators DBMonster supports generation of several SQL data types, such as strings (char, varchar, text), integers (int4 and int8), numbers, boolean and timestamps (datetime and timestamp). Every column has its values generated by a factory, called Generator, according to its data type. There are couple of already implemented Generators, such as StringGenerator, NumberGenerator, BooleanGenerator, etc. StringKeyGenerator and MaxKeyGenerator are capable of generating key values, i.e., primary keys. However, these Generators are not

27 3.3. DATABENE BENERATOR 15 always satisfactory. Therefore, DBMonster provides a mechanism for creation of new userimplemented Generators. 3.3 Databene Benerator Databene Benerator[6][7] is a complex and highly extensible data generation framework written in Java, released under dual license GPL and commercial. Benerator s mission is to deliver generation of realistic and valid high-volume data for load and performance testing. It is capable of generating data for a variety of platforms, providing plug-in interfaces for custom extensions to eventually target whichever platform. Benerator provides an Eclipse plug-in, called Benclipse, as well as a Maven plug-in. Figure 3.2: Databene Benerator - Multi-Platform Generation [6] Figure 3.2 illustrates Benerator s platform-independent approach to generation. On the input is meta-data, read either from a descriptor of imported from systems like databases, which can be manually overwritten. In the next step, Benerator generates abstract entities, internally represented by Java objects, instances of class Entity. The abstract entities can be afterwards mapped to platform-dependent entities: relational data, graphs (JavaBeans 2 ) and various file formats, such as XML, XML Schema, CSV, Excel Sheets, flat files, etc. Output to multiple platforms at the same time is possible. Benerator supports database systems: Oracle, IBM DB2, MS SQL Server, MySQL, PostgreSQL, HSQL, Derby and Firebird. A variety of files can be imported or exported, e.g., CSV, flat files, DBUnit, XML (output only), SQL and Excel Sheets. Benerator directly supports generation of primitive data types, collections, arrays, strings generated according to regular expressions; in addition, custom generators can be implemented. The data types can be decorated with simple constraints to obtain more customized data as well as complex constraints; for example, if generating a person, his or her gender will correspond with name. 2 Java objects with specific conventions, such as being serializable, their properties being accessed via getter and setter methods, having a zero-argument constructor.

28 16 CHAPTER 3. EXISTING DATA GENERATORS The framework provides data sets of real-world domains. For example, address data elements (i.e., street, country, postal code), personal data (name, surname, gender, salutation), company names, addresses, etc. Benerator uses an XML-based file, a meta-data descriptor, to describe the structure of entities and attributes as well as to configure additional information of data processing. An example of a Benerator descriptor might be the following benerator.xml: <setup... <import defaults="true" platforms="xml" domains="person,address"/> <bean id="xmlwriter" class="xmlentityexporter" /> <generate type="users" count="1" consumer="xmlwriter"> <generate type="user" count="2" consumer="xmlwriter"> <variable name="person" generator="persongenerator"/> <id name="id" type="long"/> <attribute name="salutation" script="person.salutation"/> <attribute name="firstname" script="person.givenname"/> <attribute name="lastname" script="person.familyname"/> <attribute name=" " script="person. "/> <attribute name="age" type="int" min="18" max="60"/> <generate type="address" count="1" consumer="xmlwriter"> <variable name="address" generator="addressgenerator"/> <attribute name="street" script="address.street" /> <attribute name="number" script="address.housenumber"/> <attribute name="city" script="address.city"/> <attribute name="postalcode" pattern="[0-9]{5}"/> <attribute name="country" script="address.country"/> </generate> </generate> </generate> </setup> The platforms="xml" attribute setting causes that XMLEntityExporter will be imported for further usage, in this case to create the xmlwriter bean. Notice also the importing of the domain packages of person and address data, specified by domains="person,address". The descriptor defines one top-level entity represented by the generate element, users, that will be a container of user entities, the expected number of the entities is set in attribute count. There are several entity attributes, represented by the attribute element, defined within user. The id attribute specifies a unique identifier. Values of the following four attributes will be retrieved from a person bean provided by PersonGenerator. The age attribute defines an int value between 18 and 60. Within user is a nested entity, the generate element with type="address". Likewise, several attribute values will be generated by AdressGenerator, however, postalcode will by generated by the specified pattern [0-9]5, i.e., five digits.

29 3.3. DATABENE BENERATOR 17 A sample generation of the above description might be: <users> <user id="1" salutation="mrs." firstname="marion" lastname="snyder" ="marion_snyder@bhvkrcj.ch" age="47"> <address street="ridge Street" number="24" city="eddy" postalcode="08841" country="united States"/> </user> <user id="2" salutation="mr." firstname="cody" lastname="vargas" ="cody.vargas@dgkvsfvvgwn.org" age="24"> <address street="north Street" number="48" city="seville" postalcode="66409" country="united States"/> </user> </users>

30 18 CHAPTER 3. EXISTING DATA GENERATORS

31 Chapter 4 Analysis and Design The system requirements will be specified at the beginning of this chapter. Afterwards, we will make a discussion and solutions for particular requirements will be proposed. In the next step, framework components will be described. Subsequently, a detailed description of Data Definition Model design will be provided. Finally, basic principles of Data Generator functionality will be outlined as well as its API. 4.1 Requirements The following system requirements are either directly or indirectly derived from the project assignment s guidelines: R.1 The system will take an EMF domain model of the EMF framework on the input R.2 The system will use Data Definition Model (DDM) to specify the necessary information for the generation R.3 The system will generate an initial DDM from the corresponding domain model R.4 The system will provide an editor for editing DDMs R.5 The system will generate instances of the input domain model R.6 The system will enable a user to specify the amount of the generated data R.7 The system will be able to generate primitive data types as well as more complex higher-level data types R.8 The system will use dictionaries that will provide values of the data types R.9 The system will provide an option to persist the generated model instance into XMI file R.10 The system will provide wizards for creating new DDMs and domain model instances 19

32 20 CHAPTER 4. ANALYSIS AND DESIGN According to the requirements, a domain model is expected on the input of the framework; an instances of the domain model on the output. Before we discuss the requirements, we need to define more precisely what is meant by a domain model and its instance. Domain Model A domain model, also a conceptual model or a data model, is an abstract model that describes data of a specific domain that a software system works with. Data model consists of entities, its properties and operations, relationships between the entities and constraints of the relationships. If we have, for example, a system for company management, as in the Company example (Figure 6.1), its domain will contain entities such as Employee, Branch, Department, etc. The Employee entity will have properties, such as name, surname, , all being a string value. There might be a relationship between Employee and Department: "a department has employees" or the other way around "an employee works at a department". Relationships might have various multiplicities and directions. For example, the mentioned relationships are one-to-many and many-to-one, respectively. Instance of Domain Model While a domain model describes the structure of a domain, an instance is a realization of the domain model. For example, an instance of entity Employee will be a concrete employee; providing value "John" for his name property, we get employee John. Entity instances might be linked together according to their relationships defined in the domain model. For example, if we have one instance of entity Department, department D, and two instances of entity Employee, John and Michael, and link them together, the resulting relationship will be: "employees John and Michael work at department D". The set of all entity instances linked together makes the resulting instance of the provided domain model. It is important to respect relationship constraints and property data types to obtain a valid instance. 4.2 Discussion The system requirements imply a few additional requirements that have to be considered in the subsequent analysis and design. Let us discuss, and suggest solutions for, the particular requirements. 1. Requirement R.1 implies the necessity to utilize Eclipse Modeling Framework 2 and the Eclipse platform on which EMF is built. The input domain model must be created with, or imported into, the EMF framework; essentially, it must be an EMF model 2.1. The creation of EMF models is described in Section As stated in requirement R.2, we need another model, Data Definition Model (DDM), to specify information that will be necessary in order to generate instances of domain models. Therefore, an important step will be to design such a model and specify what kind of information will be useful for the generation process. DDM a decoration model 1 of the input EMF domain model will also be an EMF model. 1 Data Definition Model "decorates" Ecore, i.e., adds additional information to Ecore

33 4.2. DISCUSSION Requirement R.3 expects of the framework to generate initial DDMs. However, the generated DDMs will not be sufficient in most cases, so as to generate desirable data. Nevertheless, the framework will try to automatically generate as much portion of DDM as possible, according to the domain model structure. 4. The EMF.Edit framework, a part of EMF, enables a user to create editors for EMFbased models, utilizing the Eclipse UI framework. EMF is able to generate the base of the editor out of any EMF model, creating the initial implementation. EMF.Edit will be used to generate the initial editor that will be further tweaked to obtain a functional editor for Data Definition Models DDM Editor. 5. To generate domain model instances, i.e., to create instances of its classes, fill their attributes with data and link the instances together, the system will utilize the EMF s reflective API 2.3, a feature of the EMF framework. 6. The amount of the generated data is the number of the generated class instances. Therefore, to meet requirement R.6, Data Definition Model will store the information about the desired amount of the model s instances. 7. As requirement R.7 indicates, there will be two kinds of generated data types. The primitive data types will cover most of the primitive Java data types, such as String, various numeric data types, boolean, java.util.date, etc. The higher-level data types will define data from real world domains, such as an address, a person s name, surname, , etc. In addition, primitive data types with additional user-specified constraints will be provided. The definitions of the data types will be designed and included within Data Definition Model. 8. Requirement R.8 mentions dictionaries. A dictionary, in the scope of the framework, will be a collection of values of a specific data type that will match the defined data types. The framework will provide predefined set of dictionaries for the predefined higher-level data types. However, user will be allowed to create his own, easily pluggable, dictionaries, whether for the existing or user-created data types. 9. XMI is a canonical representation of EMF models. EMF provides a default XMI serializer for persisting EMF models into XMI files. Therefore, requirement R.9 is directly supported by EMF. 10. Two wizards will be created: the first for creating new DDMs; the second for creating instances of provided domain models. The wizards will be created using the standard Eclipse platform approach for creating wizards, utilizing the Eclipse UI framework.

34 22 CHAPTER 4. ANALYSIS AND DESIGN Designtime Domain Model input Runtime Data Generator generate generate Data Definition Model Instance edit persist DDM Editor XMI Figure 4.1: Framework Components 4.3 Framework Components The scheme of the framework elements is illustrated in Figure 4.1. There are two distinct phases in which the particular framework components and models play their parts: design time and run time. The design time, illustrated in the left rectangle, is a phase in which a user will create and edit the two models necessary for the further processing. 1. Firstly, the user will need to provide a domain model as described in Section In the next step, the user will need to create a Data Definition Model for the provided domain model. This is the time when the framework comes in. The framework will provide a wizard to accomplish the task. The "generate" action in the scheme indicates that the initial DDM model will be generated. 3. To extend the generated base of the DDM model, a graphical tree-based editor the DDM Editor component in the scheme will be provided. Once the DDM model is generated, edited and ready for the data generation, the next phase, the run time, takes place. 1. The Data Generator component will generate the data according to the information contained in the DDM which will be passed to Data Generator on the input. The DDM will contain a reference to the corresponding domain model, therefore Data

35 4.4. DATA DEFINITION MODEL 23 Generator will have access to the domain model through the DDM. As a result, the user will obtain an in-memory instance of the domain model, which will be in most cases the desired output of the framework. 2. There will be, however, a possibility to generate and persist the model instance at once, rather than get only an in-memory instance. For this task, the framework will provide a wizard which will generate and persist the instance into an XMI file. 4.4 Data Definition Model According to the requirements, Data Definition Model is supposed to be a decoration model of the EMF s Ecore model. It should define the necessary information that will be processed by Data Generator. The specific information on what will DDM define and how it will influence the data generation will be described in the subsequent sections Instance Definitions The portion of Data Definition Model that describes instance definitions is shown in Figure 4.2. The central element in the diagram is InstanceDefinition, the representation of an instance definition. InstanceDefinition is a superclass of two direct subclasses: ClassInstaceDefinition and ReferenceInstanceDefinition. The difference between the two classes is that the former refers to an Ecore s EClass, whereas the latter to an Ecore s EReference. Those references and classes are elements of the domain model. In practice, since we focus on class instances, we are interested in EClasses. While ClassInstaceDefinition has direct access, ReferenceInstanceDefinition access an EClass through the ereferencetype reference of the EReference. In addition, ReferenceInstanceDefinition contains the concretetype reference which refers to, or one of the subclasses of, the EClass accessed through the EReference assigned by ereference. A concrete type is such an EClass that has its interface and abstract attributes set to false. There is yet another instance definition class, ContainmentReferenceInstanceDefinition, which refers only to containment references, i.e., EReferences with the containment attribute set to true. Its parent, ReferenceInstanceDefinition, refers to noncontainment EReferences. The ContainerInstanceDefinition interface, the common parent of ClassInstanceDefinition and ContainmentReferenceInstanceDefinition, contains the referencedefinitions reference pointing to reference instance definitions. As a result of this design, the instances of the data definition classes form a tree, or a set of trees, i.e., a forest, where the ClassInstanceDefinitions are roots and the both reference instance definition classes are nodes of the trees. An ClassInstanceDefinitionContainer is a container of all the trees and DataDefinition is the root element of DDM. Every InstanceDefinition has an interval of numbers, an instance of the Interval class. The interval values are stored in the minnumber and maxnumber attributes. The EInt data type wraps the Java int type. The intervals have a slightly different application when they belong to an ClassInstanceDefinition and ContainmentReferenceInstanceDefinition, respectively. In the former case, the intervals express the desired counts of the

36 24 CHAPTER 4. ANALYSIS AND DESIGN Figure 4.2: DDM - Instance Definition EClasses instances, whereas in the latter the counts of instances of the EClass types of the corresponding containment EReferences Data Type Definitions The definition of the higher-level data types is illustrated in the scheme in Figure 4.3. Data Definition represents a DDM s root element, DataTypeDefinitionContainer a container of DataTypeDefinition objects. Since DataTypeDefinition is an abstract class, it will not be instantiated directly, only its subclasses. Every DataTypeDefinition is defined for at least one EAttributes, i.e., the attributes of the domain model, that are accessed by the domainattributed reference. The preferrednumberofvalues attribute is applicable when some of the domain model s attributes are multiplicity-many, i.e., they store a list of values. The preferrednumberofvalues attribute then determines how many values will be generated for the attribute. Data Definition Model predefines seven concrete subclasses of DataTypeDefinition. Every one represents a real-world higher-level data type. The first one, Number, defines numeric values. The scale attribute s value determines whether the generated number will be decimal or integer. If the value is 0, the number will be integer, if greater than zero, the value will be decimal, having the set scale. A negative

37 4.4. DATA DEFINITION MODEL 25 Figure 4.3: DDM - Data Type Definition scale is currently not supported. The min and max attributes, as the names indicate, define the restriction of expected values, i.e., the generated number must be greater than, or equal to, min and lesser than, or equal to, max. The restriction attributes are of the java.math.bigdecimal type in order to be applicable for all numeric types. A min value bigger than a max value will cause a validation failure. The class represents an address, respecting the standard format: localpart@domain, for example, joe.smith@somedomain.com. The expected values of will have the format: <string>@<string>.[a-za-z]{2,3}, i.e., a string followed by sign, an additional string, a dot, and two or three ASCII characters. The next class, Text, will serve for defining text values of various kinds and lengths. In Figure 4.4, we can see the TextType enum, enumerating all the possible types of text, which is accessed through the type attribute. Therefore we can have generated the following kinds of text elements: characters, words, sentences or paragraphs. The elementnumber attribute specifies the count of the chosen text element type. For example, if we want to generate six paragraphs, we will set the attributes to elementnumber=6 and type=paragraph. To define date values, we can use the Date element. The class has two attributes: before and after. the EDate data type is an encapsulation of the Java java.util.date object. The value of the before attribute will cause the generated dates to be prior to the before s value, whereas the value of the after attribute will cause the values to be older than the value of after. For example, the settings before= and after= will

38 26 CHAPTER 4. ANALYSIS AND DESIGN Figure 4.4: DDM - Enumerations restrict the generated values to dates between July 2000 and the beginning of year If the after value is older than the before value the validation of the DDM will fail. The Address element of DDM will be used to generate the particular elements of an address, such as street name, house number, post code, city and country values. All the possible element types are listed in the AddressType enum, shown in Figure 4.4. To generate various kinds of names is often a desirable ability of any data generator. DDM defines the Name class to provide such an option. As listed in the NameType enum, there are currently four name types: FIRST_NAME, LAST_NAME, FULL_NAME and OTHER. The first three types are supposed to generate human names, while the last type will be used in all other cases. The last class, Identifier, will define identifiers, unique values used as an identification of objects. For now, only java.lang.long values of identifiers are supported. The minvalue attribute, which takes a Long value, stores the initial value used for the generation; the subsequent identifiers will be generated by incremental method Dictionaries The last portion of Data Definition Model, centred around dictionary definitions, is illustrated in Figure 4.5. The central element, the Dictionary interface, represents a dictionary. A dictionary, in the scope of this project, is an object that offers a collection of values of a specific real-world domain, such as names, address elements, address, etc., for a defined data type, i.e., an instance of DataTypeDefinition. For example, if we have a Name of the FIRST_NAME type, the corresponding dictionary will offer a set of human first names. The Dictionary interface has one string attribute, name, by which is identified. In addition, it contains three operations: getallvalues() : JavaList, getrandomvalue() : EJavaObject and isfordefinition(datatypedefinition) : EBoolean. The first operation returns all the values a dictionary object provides, the second returns one random value and the last operation determines whether the particular dictionary stores values for the provided DataTypeDefinition, input as an argument, returning a boolean flag. Dictionary is generic, therefore Java types of values, i.e., JavaObject and values of JavaList, will vary depending on the provided generic type.

39 4.4. DATA DEFINITION MODEL 27 Figure 4.5: DDM - Dictionaries The Dictionary interface was designed in a way to be easily extendable, providing a plug for another EMF models that would eventually model the concrete dictionaries. There is yet another approach to provide dictionaries, independently of EMF models. For this purpose the CustomDictionary class will be used. Notice that CustomDictionary has one attribute, dictclass, that will store an instance of java.lang.class, a meta-object representing a Java class. ExtendedJavaClass is simply a wrapper of java.lang.class. Instances of the class represented by the provided java.lang.class object will have to implement the IDictionary interface, that looks like this: public interface IDictionary<T> { } List<T> getallvalues(); T getrandomvalue(); boolean isfordefinition(datatypedefinition<?> datatypedefinition); IDictionary is virtually the same interface as the Dictionary element in DDM. The difference between the two is that the former is DDM-independent and it can provide the values from whatever source. For example, an instance of IDictionary can define in-memory values or provide the values from a database, the options are unlimited, the approach is very flexible. Once a java.lang.class object is set, CustomDictionary will instantiate its instance and check whether it implements IDictionary. If everything pass without failure, the Custom- Dictionary object will afterwards forward invoking of its operations to the IDictionary object s operations of the same name.

40 28 CHAPTER 4. ANALYSIS AND DESIGN Notice that every DataTypeDefiniton can use several Dictionaries. On the other hand, dictionaries are not required, therefore the dictionaries reference might stay empty Root Elements Root elements of the entire Data Definition Model are shown in Figure 4.6. DataDefinition is the top of DDM, it contains a reference, domainmodel, to the decorated domain model, an instance of Ecore s EPackage. In addition, it is a container for the rest of DDM objects, contained within the containers of particular DDM parts described in the previous sections. Figure 4.6: DDM - Root Contents Generating DDM The framework will provide generating initial DDM models out of the input Ecore models. Such a functionality is highly desirable especially when having complex models with many classes and references. The question is "what data can be generated automatically?". InstanceDefinitions can be generated because they virtually copy the reference structure of the underlying Ecore model. The framework will therefore support generating the initial structure of InstanceDefinitions. The Ecore model provides too little information to automatically generate DataTypeDefinitions. There might be some options, however, very restricted and therefore the framework will not provide such generation. 4.5 DDM Editor As mentioned in Section 4.3, the framework will provide an editor for modifying DDM models, called DDM Editor. The EMF framework and its sub-framework, EMF.Edit, is capable of generating EMF model-based editors, utilizing the Eclipse UI framework and the Eclipse platform. An example of such an editor is Sample Ecore Model Editor, a part of EMF, for editing Ecore models.

41 4.6. DATA GENERATOR 29 The classes generated by the EMF.Edit framework will have to be further modified to customize the default editor in such a way that the final editor will support DDM-specific configuration of DDM models. For example, the editor will offer dictionaries only to the corresponding supported DataTypeDefinitions. 4.6 Data Generator The Data Generator component is the engine of the generation process. It takes a valid Data Definition Model on the input and generates data, an instance of the domain model decorated with the DDM, accordingly. There are several tasks Data Generator has to perform to return such an instance of the domain model that will meet all the defined expectations and constraints. 1. Data Generator must generate such numbers of instances of the particular domain model classes as are defined in DDM, by InstanceDefinitions (see Section 4.4.1). Since every InstanceDefiniton contains only an interval of numbers, it will be Data Generator s duty to compute the exact numbers of the to-be-generated instances. 2. The generated class instances must be linked together, following the references and restrictions defined by ReferenceInstanceDefinitions. Therefore not all classes nor all references of the domain model will be processed during the generation, only the classes and references defined by InstanceDefinition s subclasses. 3. Data Generator must be able to randomly generate values of both primitive Java types plus often used Java objects, such as java.lang.string and java.util.date, and the higher-level types defined by DataTypeDefinition s (see Section 4.4.2) subclasses. If Dictionary (see Section 4.4.3) objects are provided, instead of generating random values, it will choose a random value from the dictionaries. 4. Attributes of the input domain model, which are decorated by DataTypeDefinitions, must be filled with generated values according to the DataTypeDefinitions properties and provided Dictionaries Data Generator API In this section, the core API for generating data type values will be outline as well as the main interface of the component, i.e., interface that generates an instance of the input domain model. The framework will provide a factory that will create a new DataGenerator object for every DDM. public interface DataGenerator { } Collection<EObject> generate();

42 30 CHAPTER 4. ANALYSIS AND DESIGN The DataGenerator interface contains only a single method that returns a collection of EObjects. As mentioned in Section 2.2, EObject is a base of every generated EMF model class. Moreover, it is essentially a base of every EMF object, even if there is no generated class, when working only with dynamic EMF. The returned collection of the EObject instances is the instance of the EMF model. The objects contained within the collection are containers of the rest of instances, i.e., root objects that are not contained by another EMF object, they are roots of trees of EMF objects. All data type generators, whether for primitive data types or otherwise, will implement the common interface DataTypeGenerator: public interface DataTypeGenerator<T> { } boolean isfordatatype(edatatype edatatype); Collection<EDataType> getsupporteddatatypes(); The interface is generic and consists of two methods. The first method, isfordatatype(), returns a boolean value for the input edatatype. EDataType objects wrap Java primitive data types and non-ecore Java types. For example, EcorePackage.Literals.ESTRING is a wrapper of java.lang.string. The method should determine whether the provided edatatype is supported by the generator; in other words, whether the generator is able to generate values for such a data type. The getsupporteddatatypes() method will return a collection of all the supported data types for which the generator will be able to generate values. There will be two interfaces extending DataTypeGenerator. The first, PrimitiveDataType- Generator, will be a base for primitive data type generators and the second, DefinedData- TypeGenerator, for defined data type generators. A data type is defined when there is a corresponding concrete DataTypeDefinition subclass for it, whereas the primitive data types are primitive Java types, such as String, boolean, int, etc. public interface PrimitiveDataTypeGenerator<T> extends DataTypeGenerator<T> { T generate(edatatype edatatype) throws EDataTypeNotSupportedException; public interface Registry { List<PrimitiveDataTypeGenerator<?>> getgenerators(edatatype edatatype); void addgenerator(primitivedatatypegenerator<?> generator); } } Registry INSTANCE =...;

43 4.6. DATA GENERATOR 31 The PrimitiveDataTypeGenerator s singe method, generate(), is the method that will generate and return a value of the data type wrapped by the argument edatatype object. If the data type is not supported, the EDataTypeNotSupportedException will be thrown. The PrimitiveDataTypeGenerator.Registry interface defines methods for registering and accessing instances of PrimitiveDataTypeGenerator that will be registered in memory, in a singleton registry instance, accessible via the INSTANCE variable. The registry has two methods: getgenerators() for retrieving the generators registered for the edatatype argument and addgenerator() for registering a generator. Likewise, the DefinedDataTypeGenerator interface provides the generate() method as well. In this case, it takes two arguments instead of only one, specifically, DataTypeDefinition and EDataType objects. The reason why the both objects need to be provided is that it is possible that some DataTypeDefinition supports more than one EDataTypes. The method can throw two kinds of an exception: EDataTypeNotSupportedException and DataTypeDefinitionNotSupportedException. The later indicates that the provided Data-TypeDefinition is not supported by this generator. Whether the DataTypeDefinition is supported will be able to be checked with isfordatatypedefinition(). Additionally, there will be a registry for DefinedDataTypeGenerators that will implement the DefinedDataTypeGenerator.Registry interface. The register will work on the same basis as the PrimitiveDataTypeGenerator.Registry. The registering method, addgenerator(), will take two arguments, the generator and a DataTypeDefinition for which the generator will be registered. Unlike the previous registry, this registry interface defines one additional method, getsupporteddatatypes(), that will return a set of supported data types for the input DataTypeDefinition object passed as a parameter. public interface DefinedDataTypeGenerator<T> extends DataTypeGenerator<T> { T generate(datatypedefinition<t> datatypedefinition, EDataType edatatype) throws EDataTypeNotSupportedException, DataTypeDefinitionNotSupportedException; boolean isfordatatypedefinition(datatypedefinition<?> datatypedefinition); public interface Registry { <U> List<DefinedDataTypeGenerator<U>> getgenerators( DataTypeDefinition<U> datatypedefinition); <U> void addgenerator(datatypedefinition<u> datatypedefinition, DefinedDataTypeGenerator<U> generator); Set<EDataType> getsupporteddatatypes( DataTypeDefinition<?> datatypedefinition); } } Registry INSTANCE =...

44 32 CHAPTER 4. ANALYSIS AND DESIGN

45 Chapter 5 Implementation This chapter will outline technologies and tools connected to the development of the framework. Additionally, the particular framework plug-ins will be described. The rest of the chapter will overview basic implementation concepts adapted by the framework, illustrated with code samples. Since the framework is supposed to be a generation tool for EMF models, requirements on the used technologies were indirectly determined. Specifically, the Eclipse platform and Eclipse Modeling Framework (Chapter 2). The technologies are written in Java, therefore the Java platform is listed among essential technologies. The particular versions used for implementation were: EMF 2.6.1, Eclipse and Java SE JDK 6. The following tools were used during development: Eclipse Modeling Tools (Helios SR2) Eclipse Modeling Tools is an Eclipse IDE with additional Eclipse Modeling Project components, including EMF, GMF, MDT XSD/OCL/UML2, M2M, M2T, and EMFT. The package is available at All tools necessary for the implementation were contained within this single package, it served also as an IDE for writing Java code. Subversion Subversion[13] is an open source version control system (VCS) and it was used for maintaining source codes of the framework. Binary packages are available at The framework is divided into six separate plug-ins, each one having the prefix org.alvila.datagenerator: 1. org.alvila.datagenerator plug-in contains Data Generator, i.e., the engine of the framework. It consists of the API, data type generators and the generator/processor of model instances. Additionally, the plug-in provides a wizard for generating model instances. 2. org.alvila.datagenerator.ddm plug-in generated by the EMF generator, contains all the pieces that EMF generates out of Ecore models, see Section 2.2, and persisted models of the Data Definition Model, i.e., an Ecore, a GenModel 1 and Ecore Diagrams. 1 An EMF model that wraps an Ecore and stores generation-related information of the Ecore model 33

46 34 CHAPTER 5. IMPLEMENTATION 3. org.alvila.datagenerator.ddm.edit plug-in stores generated EMF.Edit classes that provide class-specific information for DDM Editor. 4. org.alvila.datagenerator.ddm.editor plug-in initially generated by EMF. It contains Eclipse UI framework classes that provide support for an Eclipse IDE based editor, i.e., DDM Editor. It also provides a wizard for DDM models creation and DDM builder for generating initial DDM models. 5. org.alvila.datagenerator.dictionaries plug-in containing yet another EMF model, the Dictionaries model, and its generated Java code. The Dictionaries model extends DDM, providing concrete classes that extend the DDM s Dictionary interface. Several instances of Dictionaries, persisted within the dictionaries folder, are read by the framework at run time and provided as dictionaries for data type definitions. 6. org.alvila.datagenerator.examples.company plug-in contains the Company EMF model. The Company example is described in more details in Chapter Built-In Dictionaries The Dictionaries model, contained within the org.alvila.datagenerator.dictionaries plug-in, extends Data Definition Model, specifically, its Dictionary interface. See It defines a couple of EClasses that extend the interface; their instances are used as buildin dictionaries for defined data types, i.e., DataTypeDefinition objects. The framework reads and registers dictionaries contained within the plug-in s dictionaries folder. Values of the existing dictionaries were generated by generatedata.com 3.1. A sample method from the CityDictionaryImpl class, isfordefinition(), determines whether the dictionary is for the argument datatypedefinition. public boolean isfordefinition(datatypedefinition<?> datatypedefinition) { return datatypedefinition instanceof Address && ((Address)dataTypeDefinition).getType() == AddressType.CITY; } 5.2 Custom Dictionaries The CustomDicationary element of DDM provides a mechanism for users to provide their own implementations of dictionaries. The whole approach is described in Section The following code, written within the CustomDicationaryImpl class, displays the updatedictadapter() method that is invoked when a new dictclass is set via setdict- Class(). The value of dictclass is then instantiated and checked whether it implements the org.alvila.datagenerator.ddm.dict.idictionaty interface. If everything passes without failure, the value is assigned to the dictadapter variable.

47 5.3. DATA GENERATOR private IDictionary<T> dictadapter; private void updatedictadapter() { dictadapter = null; if (getdictclass()!= null) { try { Object instance = getdictclass().newinstance(); if (instance instanceof IDictionary) { dictadapter = (IDictionary<T>)instance; } } catch (InstantiationException e) { DDMPlugin.INSTANCE.log(e); } catch (IllegalAccessException e) { DDMPlugin.INSTANCE.log(e); } } } public T getrandomvalue() { if (dictadapter == null) { return null; } return dictadapter.getrandomvalue(); }... It is evident from the getrandomvalue() method that if the dictadapter field is set, a method of the same name is invoked on dictadapter. Otherwise, null is returned. In the same way work also the remaining IDictionary methods, i.e., isfordefinition() and getallvalues(). 5.3 Data Generator Section 4.6 describes how Data Generator should operate, Section previews its API. A DataGenerator object, that returns the final instance of a domain model, is manufactured by the DataGeneratorFactory class: public class DataGeneratorFactory { public static DataGenerator createdatagenerator( DataDefinition datadefinition) { return new DataGeneratorImpl(dataDefinition); } }

48 36 CHAPTER 5. IMPLEMENTATION The factory creates a new DataGeneratorImpl, an implementation of DataGenerator, for every Data Definition Model. When generate() is invoked on a DataGeneratorImpl object, the following process proceeds: 1. Domain model classes are instantiated, having numbers generated from the intervals specified by InstanceDefinitions The instances are created with EMF reflective API 2.3. In this step, containment references are resolved. Instances, classes of which have containment references, have assigned instances of which they become containers. The reflective API serves for this task as well. 2. The generated instances are linked together according to their non-containment references decorated by ReferenceInstanceDefinitions. (In current version, the algorithm is not fully capable to resolve the non-containment references, there might be a scarcity of available instances.) 3. Attributes of all the instances are processed. If there is no DataTypeDefinition defined for an attribute, its value is generated by a PrimitiveDataTypeGenerator according to its Java data type. Otherwise, the framework checks whether there are dictionaries defined for the attribute. If not, the value is generated by DefinedDataType- Generator according to the DataTypeDefinition, however, only a random meaningless value. Otherwise, the dictionaries are used. If there are more than one dictionary, one is picked randomly and the value of the attribute is randomly chosen from the dictionary. The number of assigned values to the attribute is determined both by its multiplicity and the preferrednumberofvalues attribute value. 4. The generate() method returns a collection of EObects, the container instances. Primitive Data Type Generator PrimitiveDataTypeGenerators generate data for primitive Java data types and are registered in the PrimitiveDataTypeGenerator.Registry singleton object. Generators for almost all Java primitive types are implemented, as well as StringGenerator providing values of java.lang.string; or DateGenerator that generates values of java.util.date. StringGenerator contains Lorem Ipsum text which provides combination of its segments as demanded. An example subclass of PrimitiveDataTypeGenerator is the BooleanGenerator class: public class BooleanGenerator extends AbstractPrimitiveDataTypeGenerator<Boolean> { protected BooleanGenerator() { super(ecorepackage.literals.eboolean, EcorePackage.Literals.EBOOLEAN_OBJECT); }

49 5.3. DATA GENERATOR 37 protected Boolean dogenerate() { return GenerationUtil.getRandomBoolean(); } There is another class, AbstractPrimitiveDataTypeGenerator, that takes an array of parameters into its constructor, the array being the supported EDataTypes. The dogenerate() method is invoked by generate() after checking for potential null values or exceptions. The class supports EcorePackage.Literals.EBOOLEAN and EcorePackage.Literals. EBOOLEAN_OBJECT that wrap boolean and Boolean, respectively. A random boolean is generated by GenerationUtil.getRandomBoolean(). Defined Data Type Generator DefinedDataTypeGenerators generate data for attributes decorated by DataTypeDefinitions with no defined dictionaries. The generators cover all the DataTypeDefinition subclasses. For example, for the Address data type definition, DefinedAddressGenerator is prepared. The generators are registered in the DefinedDataTypeGenerator.Registry singleton object. The Defined Generator class: public class Defined Generator extends StringDefinedDataTypeGenerator { } protected Defined Generator() public boolean isfordatatypedefinition( DataTypeDefinition<?> datatypedefinition) { return datatypedefinition instanceof ; public String dogenerate(datatypedefinition<string> datatypedefinition, EDataType edatatype) { return getstringgenerator().generate(stringtype.word, 1) + "@" + getstringgenerator().generate(stringtype.word, 1) + "." + getstringgenerator().generate(stringtype.letter, 3); } generates a string having the format of an address. The isfordatatypedefinition() method determines whether the provided datatypedefinition is instance of the

50 38 CHAPTER 5. IMPLEMENTATION class. The class extends StringDefinedDataTypeGenerator that provides a StringGenerator, accessed by getstringgenerator(). 5.4 Wizards Wizard A wizard is a widget containing several pages that gather input from a user and guide him step by step to perform particular tasks. It has a standard set of buttons, such as Cancel, Back, Next to navigate through pages and Finish to complete the task. Every wizard has its title, eventually an icon, as well as an area for information or error messages. The Eclipse platform provides wizards, for example, to create resources, such as projects or files, or to import or export various kinds of resources into or out of the workspace Data Definition Model Wizard A new Data Definition Model is created via a wizard, called Data Definition Model Wizard, illustrated in Figure 5.1. The wizard consists of two pages: 1. Data Definition Model The first page collects information about the to-be-created DDM file, specifically, the name of the new file and its parent container. The page contains text fields to gather the values. Alternatively, a user can select the parent container in the widget contained within the page. The widget provides the workspace resource tree. The DDM file must ends with.ddm extension. 2. Import Ecore Domain Model The task of the second page, which is shown in Figure 5.1, is to import an Ecore domain model, necessary to initialize a new DDM. The Ecore model s URI can be imported from various sources, the page provides three dialogs to select the model s URI. In addition, the Model URI text field stores the model s URI, which can be also typed manually. There are four types of the Ecore model that the wizard is able to import: (a) run-time registered package The Ecore model imported from the global package registry EPackage.Registry.INSTANCE. The registry contains EPackages that are registered via the org.eclipse.emf.ecore.generated_package extension point in the running Eclipse. The URI is identical with the package s namespace URI, e.g., (b) development-time registered package Unlike the run-time registered package, in this case, the resource of the Ecore model is imported. The resource is stored within an installed Eclipse plug-in, having the URI format: platform:/plugin/ plugin_id/path_to_folder/model.ecore. (c) workspace resource The Ecore model is imported from a resource contained within the Eclipse workspace. The URI has the format: platform:/resource/project/ path_to_folder/model.ecore.

51 5.4. WIZARDS 39 (d) file system resource The Ecore model is imported from an.ecore file that has been selected from arbitrary location in the file system. The resource s URI has the format: file:path_to_file/model.ecore. Finally, the Load button performs the load action. If the Ecore model is not valid, a dialog with error list will appear and the wizard will not enable the Finish button. Clicking Finish, the wizard will generate the initial DDM Instance Wizard Instance Wizard, shown in Figure 5.2, is provided by the framework for persisting Ecore model instances in XMI format. It is very similar to Data Definition Model Wizard with a few differences: instead of an Ecore model, a DDM is imported. In addition, Instance Wizard does not contain the Browse Registered Packages button.

52 40 CHAPTER 5. IMPLEMENTATION Figure 5.1: Data Definition Model Wizard Figure 5.2: Instance Wizard

53 Chapter 6 Company Example In this chapter, we will demonstrate the whole process of generating instances of Ecore models. For an illustrative Company model, we will go through all necessary steps to obtain its instances, thus demonstrating the framework functionalities. The Company model is a simple domain model of a business company with branches, departments and employees. Its diagram is shown in Figure 6.1. Figure 6.1: Company Ecore Diagram 41

54 42 CHAPTER 6. COMPANY EXAMPLE To create the Company Example project: 1. Click File > New > Example. 2. In the New Example wizard, expand the Data Generation Examples category and select Company Example. 3. Click Next and Finish. In the model folder of the newly created workspace project are stored several files that represent various models of the Company example. An Ecore model is serialized within the company.ecore file, company.ecordiag contains an Ecore Diagram, company.ddm contains a Data Definition Model, and finally, company.xmi is the generated instance of Company, serialized in the XMI format. The models were created following the approach described in Appendix B. 6.1 Company Ecore Model Double-click the company.ecore file to open it in Sample Ecore Model Editor. You can examine the tree-like structure of the Company s Ecore model. The root element of the entire model is the company package, an instance of EPackage. Expanding the package, classifiers, such as the Employee and Address classes and the Sex enum, will appear. Expanding a particular class, its attributes and references will be revealed. In the Properties view, you can examine the properties of the element selected in the editor. Properties of particular tree nodes correspond with attributes of their classes. For example, the abstract attribute of EClass correspond to the Aabstract property of an arbitrary class, e.g., Emloyee. For more details on the Ecore model, see Section The Ecore Diagram of Company, shown in Figure 6.1, can be be opened by double-clicking the company.ecordiag file. Ecore Diagram resembles, and essentially is, UML Class Diagram. 6.2 Company Data Definition Model To examine Company s DDM, which is illustrated in Figure 6.3, open company.ddm in DDM Editor. In Dictionaries Container are created two custom dictionaries. You can find the implementations of the dictionaries within the dictionaries directory. CompanyNameDictionary and USDictionary implement the org.alvila.datagenerator.ddm.dict.idictionary interface; the former offers a single value, "United States", the latter provides a collection of company names. Dictionaries are described in Section In Data Type Definition Container, you can examine the data type definitions defined for the Company model. For Example, the Number [10000 <= x <= 50000; 0] element defines a numeric range between and and refers to the salary attribute of the domain model. Properties of Name [LAST_NAME], a Name data type definition, are shown in Figure 6.2. It has the Type property set to LAST_NAME and refers to the lastname attribute of the Employee class. In addition, it sets the Dictionary property

55 6.2. COMPANY DATA DEFINITION MODEL 43 Figure 6.2: Company DDM - Name Data Type Definition Properties Figure 6.3: Company Data Definition Model

56 44 CHAPTER 6. COMPANY EXAMPLE Figure 6.4: Company DDM - Employee Properties to refer to the English Surnames dictionary. More info on data type definitions can be found in Section Instance Definition Container contains a tree of instance definitions, subclasses of InstanceDefinition. The tree structure follows references of the domain model. A root element is always a ClassInstanceDefinition, in this case Class: Company, that represents a class that is not referred by any other class with a containment reference. There might be more than one root ClassInstanceDefinitions; as a result, Instance Definition Container would contain a forest, i.e., a set of trees. Notice that only ClassInstanceDefinitions and ContainmentReferenceInstanceDefinitions are permitted to have children. The class names in brackets display class types referred by references. For example, the Containment reference: employees (Employee) instance definition represents the employees reference of the Company class that has its containment attribute set to true. Its properties are shown in Figure 6.4. The interval of to-be-generated instances is set to minnumber=10 and maxnumber=20. The Concrete Type property sets a concrete type, i.e., the referred class itself or its concrete subclass, that will be instantiated. The description of instance definitions is available in Section Company Instance When a DDM is configured and pass validation, an instance of the Company model can be generated. A sample instance is persisted in the company.xmi file; it can be examined via Sample Reflective Ecore Model Editor. In Figure 6.5 is displayed an instance of a slightly modified Company DDM that consists of a root instance of Company, three Employee instances, each employee refers to exactly one instance of Address. Additionally, the company has one branch that contains address and two Department objects. In the Properties view can be seen properties, i.e., the generated data, of Employee Janice.

57 6.3. COMPANY INSTANCE 45 Figure 6.5: Company Instance