SWESE2011. Editors: Kalina Bontcheva Jeff Z. Pan Yuting Zhao

Transcription

1 SWESE2011 The 7th International Workshop on Semantic Web Enabled Software Engineering Co-located with ISWC2011 Bonn, Germany 24th October, 2011 Editors: Kalina Bontcheva Jeff Z. Pan Yuting Zhao

2 Table of Contents 1. Supporting Constructive and Exploratory Modes of Modeling in Multi-Level Ontologies... Colin Atkinson, Bastian Kennel and Björn Goß 2. Developing Efficient Ontology-based Systems Using A-box Data from Running Applications... Heiko Paulheim and Lars Meyer 3. Results of a Survey on Improving the Art of Semantic Web Application Development... Emanuele Della Valle, Gaetano Niro and Catalina Mancas 4. Towards Ontology-driven Requirements Engineering... Katja Siegemund, Edward J. Thomas, Yuting Zhao, Jeff Pan and Uwe Assmann 5. Identifying ontology design styles with metrics... Eleni Mikroyannidi, Robert Stevens and Alan Rector 6. Ontology-Driven Application Architectures with KOMMA... Ken Wenzel 7. A Visual Tool for Supporting Developers in Ontology-based Application Integration... Tobias Wieschnowsky and Heiko Paulheim 8. Semantic Web-driven Development of Services-oriented Systems Exploiting Linked Data for Services Annotation and Discovery... Stefan Dietze, Dong Liu, Hong Qing Yu and Carlos Pedrinaci

3 Supporting Constructive and Exploratory Modes of Modeling in Multi-Level Ontologies Colin Atkinson, Bastian Kennel, and Björn Goß University Mannheim A5,6, D Mannheim Abstract. In the long term, fully exploiting the potential synergies between ontology engineering and software engineering approaches to modeling is contingent on a complete understanding of their similarities and differences and the development of a unified modeling framework that can support both paradigms in a seamless way. In this paper we contribute to the former by characterizing the fundamental modes of modeling that the different approaches have evolved to support and we contribute to the latter by outlining how a promising unification paradigm can be enhanced to support them. This is the potency-based multi-level modeling paradigm which allows all ingredients of a comprehensive ontology (e.g. instances, classes and metaclasses) to be treated in a uniform way, and allows ontologies to include as many classification levels as needed to best represent the domain in hand. The paper explains how the basic notion of potency can be enhanced to support exploratory and unbounded modes of modeling alongside the constructive and bounded modes of modeling which underpin the software engineering approach to modeling for which it was originally developed. Keywords: Multi-level modeling, metamodeling, modeling modes, constructive modeling, exploratory modeling, ontology engineering, potency 1 Introduction As the potential synergy between ontology-based reasoning services (a la OWL) and software-engineering oriented specification capabilities (a la UML) becomes increasingly apparent, a growing number of researchers have started to explore ways of bringing the two schools of modeling together. In the short to medium term, bridging technologies that support mappings between the two paradigms offer the most practical way of leveraging these two forms of modeling together. Prime examples of such technologies include the Ontology Definition Metamodel standardized by the OMG [4] and the TwoUse interoperation tool [10] developed in the MOST project [6]. Nevertheless, in the long term, the cleanest solution is to develop a single unified modeling paradigm which incorporates the capabilities of existing modeling technologies as special cases.

4 2 C. Atkinson, B. Kennel, B. Goß Accommodating the different assumptions and interpretations underlying software engineering oriented modeling and ontology (i.e. semantic web) oriented modeling within a single unified framework presents several fundamental challenges, however. Some of these have been identified and widely discussed in the literature. Examples include different underlying logics to define the meaning of classes and taxonomies (e.g. description logic, basic set theory), different interpretation of missing information (e.g. closed-world versus open world assumptions) and different architectures used to organize model information (e.g. traditional four level OMG architecture), two level OWL architecture) [2, 14, 15]. Other differences, in contrast, have received relatively little attention and have yet to be fully elaborated. These relate to the fundamental modes by which models are developed and used in the software and ontology engineering communities and the criteria under which models are considered valid. One important difference relates to whether models are developed and used in a constructive or exploratory way. In constructive (i.e. software engineering oriented) modeling the role of class diagrams is to serve as templates from which populations of instances can be generated at a future point in time, while in exploratory (i.e. semantic web oriented) modeling the role of class models (i.e. ontologies) is to capture classification information wrapped up in an already existing population of instances. This is related to the distinction between prescriptive and descriptive models [18, 19], but focusses on the conditions under which model content can be considered complete or valid rather than on the purpose for which they are being deployed. Both prescriptive and descriptive models are often developed in exploratory mode and applied in constructive mode. Another important difference relates to whether the ways in which model content can be extended are bounded or unbounded. Bounded models can only be extended across a predefined number of classification levels, while unbounded models can be extended over an unlimited number of classification levels. In this paper we fully characterize these different modes of modeling and present a strategy for accommodating them seamlessly within a unified framework. This framework is based on the multi-level modeling paradigm that is emerging as the favored foundation for a unified modeling framework because it allows all ingredients of an ontology (e.g. instances, classes and metaclasses) to be treated uniformly as first class citizens, and allows an ontology to include as many classification levels as needed to best represent the domain in hand [13, 15]. The paper makes three concrete contributions (1) it elaborates and fully characterize the different modeling modes, (2) it introduces and consolidates new terminology to discuss the properties of models in the context of these modes and (3) it enhances the notion of potency, the key features that support arbitrary numbers of classification levels, to support the unification of these modes within a single, integrated modeling framework.

5 Reconciling Constructive and Exploratory Modeling 3 2 Fundamental Modes of Modeling As Figure 1 illustrates, there are two fundamental concerns that characterize the way models are developed and the assumptions under which they are used the order or direction in which elements in the classification hierarchy are created and the scope of models in terms of their intended completeness and extensibility. These concerns are orthogonal and both give rise to a dichotomy of choices. Scope Bounded Unbounded Direction Exploratory Constructive????? Fig. 1. Modeling Modes 2.1 Constructive versus Exploratory Modeling The dominant dichotomy is the direction dichotomy which distinguishes between constructive modeling and exploratory modeling according to the general direction in which the elements in the classification levels are populated and the criteria under which a completed model may be considered valid. Constructive Modeling The goal of constructive modeling is to create a complete, definitive description of all the types in a system (i.e. to create type models in the sense of Kühne [9]) so that instances can be generated from them at runtime. Since the instances are usually generated in a different runtime environment from that used to represent the types they are normally not regarded as being part of the model per se. For example, type models are frequently used to describe the objects that can populate a database or exist in the runtime system of a software application. The idea behind constructive modeling is illustrated in the top left hand part of Figure 1 which shows two classification levels separated by a vertical line. Those on the left hand side of the line are types while those on the right hand side are instances. The solid and dashed boundaries of the elements are intended to show the chronological direction (order) in which the elements are

6 4 C. Atkinson, B. Kennel, B. Goß created, with the solid elements being created before the dashed ones. Thus, this picture shows a type model from which a population of instances is subsequently created by instantiation. An important convention in constructive modeling is that a type model is considered consistent and valid without any of the instances described by the model actually being present in the modeling framework. In other words, in constructive modeling the solid-lined elements at the type level in the upper left hand corner of Figure 1 can be considered complete and valid before any of the dashed instances at the instance level actually exist. Because all instances are created mechanically at runtime from types, another fundamental characteristic of constructive modeling is that every instance has a special relationship to one individual type. In constructive modeling a concrete class is a class that can be instantiated while an abstract one is a class that cannot be instantiated. Since it evolved in the software engineering community, the UML is usually used in a constructive modeling mode and most of its features and terminology are optimized for this purpose. However, the UML can be, and often is, used in an exploratory way, for example during the analysis phase of a development project, although its semantics are not optimized for this mode of modeling. Exploratory Modeling One aim of exploratory modeling is to develop types that characterize the objects populating a domain of interest. As illustrated in the bottom left hand corner of Figure 1, in contrast with constructive modeling where the types come before the instances, in exploratory modeling the types are usually identified from, and therefore after, the instances they classify. Moreover, a body of model content (i.e. an ontology) is only considered complete and valid when the instances classified by types are also present. This distinction is not only reflected in the features of languages and tools used to create models, it is also reflected in the mood of natural language used to refer to the population of types and instances. Whereas in constructive modeling most of the terminology used to define language semantics and model properties is in the subjunctive form, in exploratory modeling it is in the assertive form. The derivation of types from instances is often performed by human ontology developers, but ontology engineering tools also typically provide reasoning services that can identify types and subtypes automatically. The direction of population and causality in exploratory modeling means that the notion of mechanical instantiation is no longer relevant. Instead, instanceof relationships need to be established based on the intention of types, applying the usual principle that an object is an instance of a type if it possesses all of the properties that the type defines for its instances. This means that instances no longer have a single, special type from which they were created. They are usually instances of multiple types, in different ways, depending on their properties. This also has an impact on the way subtyping and abstract/concrete classes are viewed. Because exploratory modeling focuses on describing typing concepts that exist in a domain, it is possible for two types to have the same intentions and extensions. In other words, two types can be equal in the terminology of

7 Reconciling Constructive and Exploratory Modeling 5 OWL. Also, since there is no notion of mechanical instantiation, the difference between abstract and concrete classes has to be defined in terms of the extent to which the instances of a type satisfy its intention. 2.2 Bounded versus Unbounded Modeling The difference beteween bounded and unbounded modeling revolves around whether a model is developed under the assumption that the number of classification levels is fixed (at the time the model is developed) or not fixed. Of course, it is always possible to change a model, once completed, and add more classification levels if desired. For example, a finished, bounded model with two levels can always be turned into a new, bounded model with three levels. However, this is not what is intended here. The key difference is that unbounded models can be extended over an arbitrary number of classification levels without any of the existing model elements being changed, whereas bounded models cannot. Bounded Modeling Bounded modeling takes place in the context of a fixed number of classification levels. This is illustrated in the two left hand diagrams of Figure 1. This is the main mode of modeling performed today, both in constructive and exploratory modeling and includes UML models in software engineering, ER models in data base design and OWL models (a.k.a. ontologies) in knowledge engineering. Moreover, most modeling projects are usually only concerned with two levels a type level and an instance level. Since the advent of the UML, the use of more classification levels has become common in the context of language definition. The UML infrastructure has always nominally been based on four classification levels, while tools that focus on the flexible definition of domain specific languages usually work with three levels. Unbounded Modeling Unbounded modeling takes place in the context of an unfixed or uncertain number of classification levels. In other words, an unbounded model is a completed model that explicitly indicates that the number of levels is unfixed or uncertain. This is indicated by the question marks appearing within the diagrams on the right hand side of Figure 1. The presence of the question marks is meant to symbolize the fact that one or more of the model levels have model elements that are explicitly indefinite about the number of classification levels in the subject of interest. In the case of constructive unbounded modeling (top right), this indefiniteness arises in terms of future instantiation of the types in a completed model. In other words, it exists when the model content is intended to represent a framework where future extension (by population of additional model classification level(s)) is anticipated but the exact number of levels is left undetermined. This kind of indeterminacy is common in libraries and frameworks, but the uncertainty is usually associated with specialization levels rather than instantiation. Standard programming languages such as Java, and language metamodels such as UML, possess a predefined set of types which are intended to be specialized over an indeterminate number of specialization

8 6 C. Atkinson, B. Kennel, B. Goß levels. The notion of unbounded modeling is based on the same idea but with the indeterminacy also associated with the number of classification levels. An important consequence of unbounded modeling is that the numbering of classification levels can no longer be fixed a priori. If the goal is to define model content to serve as an extensible framework, the predefined levels have to be numbered in such a way that any new levels can be added in a systematic way. A numbering scheme that fulfils this requirement is introduced at the end of Chapter Potency-based Multi-Level Modeling Various flavors of multi-level modeling have been developed in recent years [11 13]. One of the most widely known is based on the notions of clabjects and potency to represent multiple classification levels [8, 4]. In recent years, a growing number of researchers have recognized this paradigm s potential to unify the ontology-engineering and software engineering approaches to modeling [1, 2, 15]. Fig. 2. Orthographic Classification Architecture The key idea in this form of multi-level modeling is to recognize and separate two distinct forms of classification one (linguistic classification) capturing the linguistic type of model elements, the other (ontological classification) capturing the ontological type. Since this leads to two orthogonal hierarchies of classification levels, each conforming to the tenets of strict metamodeling, the resulting architecture is sometimes referred to as the orthogonal classification architecture (OCA). This is depicted schematically in Figure 2, where the vertically arranged levels, L0, L1 and L2 represent the linguistic levels, and the horizontally arranged levels (within L1) represent the ontological levels. The bottom level, L2, represents the real world as in the UML model hierarchy. Clabjects are the basic modeling elements used to populate multi-level models in the OCA. They are able to do so in a level-agnostic way because in general, they represent classes and objects at the same time. All the main model elements in Figure 2, in all three ontological levels, are clabjects, and the element in the middle level, Video, has both a type and an instance facet since it is an instance of VideoType

9 Reconciling Constructive and Exploratory Modeling 7 on the left hand side and a type for 2001aSpaceOdyssey on the right hand side. Clabjects can also have attributes (previously called fields [8]) and participate in connections. 3.1 Potency Since clabjects are essentially level and type agnostic, some way is needed to represent the typeness of a clabject - that is, the degree to which it is capable of having instances. To achieve this Atkinson and Kühne introduced the notion of potency to indicate how many levels a clabject can have instances over [8]). This original notion of potency is based on the following concepts - 1. potency is a non-negative integer associated with clabjects and attributes, 2. a clabject, i, instantiated from another clabject, t, has a potency one less than t, 3. the potency of an attribute of a clabject must be no greater than that of the clabject, 4. every attribute of a clabject, i, instantiated from a clabject t, must have a corresponding attribute in t, with a potency one higher, 5. a clabject, i, instantiated from another clabject, t, must have an attribute corresponding to every attribute of t, except those that have potency 0. Corresponding here means an attribute with the same name and datatype. Since potency values are defined as non-negative integers, it follows that a clabject of potency 0 cannot be instantiated (i.e. represents an object in classic UML terms) and a clabject of potency 1 represents a regular class in classic UML terms. By allowing clabjects to have potencies of 2 and higher, new kinds of classification behavior (spanning more than immediately adjacent levels) can be supported. This has been referred to as deep instantiation. This interpretation of potency also lends itself to a simple definition of abstract and concrete classes. Since abstract classes cannot be instantiated they have a potency of 0, while concrete classes which can be instantiated have a potency greater than 0. The basic problem with this original definition of potency is that it is inherently constructive and bounded. In other words it is inherently based on the principles that underpin the constructive and bounded modes of modeling. In fact, the very name potency conveys a constructive (type to instance) mentality. The fundamental definition of potency has always relied on the constructive notion of instantiation (instances created from types) and is not compatible with exploratory and unbounded modeling. In the following sections we therefore propose enhancements to the notion of potency to support all the modes of modeling identified above. 4 Constructive Multi-Level Modeling To avoid ambiguity in the remainder of this paper we refer to a given ontological level as a model and the total collection of models, across all the ontological levels in the L1 level, as an ontology. This conforms to the prevailing notions of type

10 8 C. Atkinson, B. Kennel, B. Goß and token models of Kühne [9] and the practice of considering instances as well as types when developing ontologies. The original definition of potency in the previous section is based on the notion that the extension of a type is characterized exclusively by attributes. However, in general this is an over simplification. If clabjects are allowed to have methods and enter into relationships with other clabjects these also have a bearing on the intention of a type. A full treatment of classification in the context of clabjects therefore needs to take these characteristics into account. However, these details are unimportant for understanding the modes of modeling discussed in this paper and the differences between them. Without loss of generality, therefore, in the remainder of the paper we assume that the only properties possessed by clabjects are attributes. 4.1 Bounded Constructive Modeling The definition provided in the previous section does not really need to be changed to support constructive multi-level modeling it already contains the required ingredients. However, in order to define a unified framework and better capture bounded constructive modeling s place within it, it is helpful to introduce new terminology to distinguish the different interpretations of classification relationships employed in each mode. As described previously, classification in constructive modeling is used to drive the automated generation of instances in a running system. Since every clabject therefore has one-and-only-one type from which it is created, we refer to this type as the blueprint of the clabject. Going further, if one clabject, b, is a blueprint of another clabject, o, we say that o is an offspring of b. Fig. 3. Bounded Constructive Modeling Figure 3 shows an example of a bounded, multi-level ontology that has been created in constructive mode. This is represented in the LML (Level-agnostic Modeling Language) that we have optimized to visualize multi-level ontologies [16]. The only aspect of this diagram that is not valid LML syntax is the representation of clabjects with dashed boundaries on the right hand side of the model. This notation is used here to convey the idea that the instances were instantiated from, and thus created after, the types in the middle. As always some exceptions are required at the extreme end of the model hierarchy and the clabjects in the left most model do not have blueprints. In terms of these new definitions we can interpret the potency of a clabject as defining the maximum potential depth of its deep offspring tree that is the tree formed by taking all its offsprings, and all of its offspings offsprings, and so on, recursively.

11 Reconciling Constructive and Exploratory Modeling Unbounded Constructive Modeling If it were ever deemed necessary to add a fourth model level to the bounded ontology in Figure 3 it would be necessary to change the potency values of many of the existing clabjects, leading to new versions of the type models in the two left hand levels. However, a common goal in constructive software engineering is to create reusable frameworks containing models that can be extended in arbitrary ways without requiring them to be changed. The extension is usually achieved by specialization, for example in the definition of Java class libraries or in the addition of new abstract syntax elements to language definitions. Extension by specialization is inherently unlimited and does not require changes to existing model elements because they do not carry any predetermined constraints on the depth of the specialization hierarchy. However, with the bounded form of potency described previously, it is not possible to add more ontological level to an ontology without changing the potencies of the existing model elements, defeating the goal of defining predefined frameworks of model content which can subsequently be extended as necessary in future projects. Fig. 4. Unbounded Constructive Modeling To address this problem we extend the potency concept defined previously to include the notion of unlimited or * potencies. These have the same basic meaning as * multiplicity constraints in UML. Instead of leaving the number of instances of an association open, however, * multiplicities leave the number of ontological levels open. The use of * multiplicities is illustrated in Figure 4 above. The stars can be thought of as the concrete incarnation of the question marks in the schematic representation of the modeling approaches in Figure 1. Figure 4 is quite similar to Figure 3, but obviously has four ontological levels. This is possible because the potencies of ProductType and Book are * rather than 2 and 1 respectively. Of course, it would be possible to create a well formed model by changing the potencies of ProductType and Book to 3 and 2, but this would give rise to another bounded model which could not be extended in the future. By making the potencies of these clabjects * an unlimited number of levels can be added without change. In Figure 4, the two left hand levels can be interpreted as defining an extensible framework. This framework can support the right hand model of Figure 3 as well as the two right hand models of Figure 4. To accommodate unbounded multi-level modeling using * potencies, the overall definition of potency needs to be enhanced as follows -

12 10 C. Atkinson, B. Kennel, B. Goß 1. Potency is a non-negative Integer or a * associated with clabjects and attributes, 2. An offspring of a clabject with numeric potency must also have a numeric potency with a value that is one lower, 3. An offspring of a clabject with * potency may also have * potency or any numeric potency, 4. the potency of an attribute of a clabject must be no greater than that of the clabject, 5. a clabject, i, instantiated from another clabject, t, must have an attribute corresponding to every attribute of t, except those that have potency 0. If the attribute of t has * potency the corresponding attribute of i can have * potency or any numeric potency. If the attribute of t has a (non-zero) numeric potency, the corresponding attribute of i must have a numeric potency that is one lower. Extending the potency mechanism in this way with the notion of * potencies has the desired effect of allowing predefined models to be extended in unpredicted ways, but it also raises a dilemma related to the numbering of the ontological levels. Since they all implicitly assumed a bounded mode of multi-level modeling, existing modeling infrastructures such as that of the UML and the original OCA chose a fixed numbering scheme in which the most general levels have the highest numbers and the least abstract levels instantiated from them have the lowest numbers, with the bottom level having the number 0. However, this is obviously not extensible across an arbitrary number of levels, unless one accepts levels having negative numbers which is somewhat counter intuitive. To support unlimited extension of generic models across an unlimited number of levels it is therefore necessary to adopt a number scheme in which the less abstract (instantiated) levels have a higher number. In terms of Figure 4, this would mean that the left hand model would have level number 0, the model to the right, instantiated from it, would have level number 1 and so on. In this way, the number of levels could grow in an unlimited way. This reverses the numbering convention popularized by the UML infrastructure. 5 Exploratory Multi-Level Modeling The notion of mechanically instantiating types to generate instances is not relevant in exploratory modeling. Instead, a more discovery-oriented notion of classification (instanceof) is required based on the properties that a clabject requires to be considered an instance (i.e the extension of a type). The basic requirement that an instance of a type needs to posses all the properties defined in the extension of that type remains unchanged. However, this notion alone is not sufficient to define useful semantics for potency in an exploratory context. It is also necessary to distinguish between two forms of instanceof relationship one in which an instance has exactly the properties needed to be an instance of the type, and no more, and one in which an instance has more properties than those needed to be an instance of a type. To this end we introduce two distinct forms

13 Reconciling Constructive and Exploratory Modeling 11 of classification (instanceof) relationship - if a clabject has exactly the required properties needed to be an instanceof a type but no more, we refer to it as an isonymic instance of that type, whereas if a clabject has more properties than those defined in the extension of a type, we refer to it as a hyponymic instance of that type. The set of instances of a type is thus partitioned into two subsets the set of isonymic instances and the set of hyponymic instances (i.e. isonyms and hyponyms for short). 5.1 Bounded Exploratory Modeling These specialized forms of instanceof relationship allow exploratory multi-level modeling to be supported in a simple and powerful way. Basically, instead of defining the semantics of potency in terms of the instantiatability of clabjects, in exploratory modeling it is characterized in terms of the depth of the isonymic instance tree emanating from a clabject. Fig. 5. Bounded Exploratory Modeling The detailed definition of potency for bounded exploratory modeling is as follows - 1. potency is a non-negative integer associated with clabjects and attributes, 2. an isonymic instance of a clabject must have a potency one lower than the clabject, 3. the potency of an attribute of a clabject must be no greater than that of the clabject, 4. every attribute of a clabject, i, that is an isonymic instance of a clabject t, must have a corresponding attribute in t, with a potency one higher, 5. a clabject, i, that is an instance of a clabject, t, must have an attribute corresponding to every attribute of t, except those that have potency 0. In contrast with the definitions in section 4.1, these definitions are driven by the isonymic instanceof relationship rather than the instantiation relationship. The main consequence of this difference is that in exploratory modeling a clabject can have more than one isonymic type (which by definition are then equal types in the sense of OWL). Also, as an abstract type is defined as a type

14 12 C. Atkinson, B. Kennel, B. Goß that has no isonymic instances (but possibly many hyponymic instances), while a concrete class is a type that has at least one isonymic instance. Because potency is defined in terms of isonymic instances, and as before always goes down by one across classification levels, it follows that it is a measure of the depth of the isonymic instance tree of a clabject. These concepts are illustrated in Figure 5 which actually contains the same clabjects as Figure 3. The difference is the direction in which the clabjects are interpreted as having been discovered and the ways in which the classification relationships are interpreted. In Figure 5 the clabjects in the two left hand models are dashed, indicating that they were derived from their instances in the model on the right hand side. Furthermore, the instanceof relationships in the ontology are classified as being either isonymic or hyponymic. Notice that the extension of a clabject (i.e. the properties that the clabject requires of its instances) is defined by its attributes of potency 1 or higher. The potency zero attributes of a clabject have no effect on the instanceof relationships it can participate in as a type. 5.2 Unbounded Exploratory Modeling By analogy with unbounded constructive modeling in section 4, unbounded exploratory modeling extends the bounded form with the notion of * potency. However, whereas all the other three modes of modeling have a clear and well established role in information modeling and are all used in practical IT projects, the role of unbounded, exploratory modeling is less clear. We have included it in the scheme for symmetry and completeness, and because it fits within a rounded and clean extension of potency. The value of this mode of modeling is yet to be explored. As indicated in Figure 6, the main difference to unbounded constructive modeling is that the ontology is not populated in a specific direction (or order). Fig. 6. Unbounded Exploratory Modeling The detailed definition of potency for unbounded exploratory modeling is adapted from that for constructive modeling by the same principle of using the isonymic instanceof relationship rather than instantiation -

15 Reconciling Constructive and Exploratory Modeling potency is a non-negative integer or a * associated with clabjects and attributes, 2. an isonymic instance of a clabject with numeric potency must have a numeric potency with a value that is one lower, 3. an isonymic instance of a clabject with * potency may also have * potency or any numeric potency, 4. the potency of an attribute of a clabject must be no greater than that of the clabject, 5. every attribute of a clabject, i, that is an isonymic instance of a clabject t, must have a corresponding attribute in t, with a potency one higher. If the attribute of i has * potency the corresponding attribute of t can have * potency or any numeric potency. 6. a clabject, i, that is an isonymic instance of a clabject, t, must have an attribute corresponding to every attribute of t, except those with potency 0. The interpretation of the * potency is different. Whereas in constructive modeling, * potency reflects a capacity to spawn offspring in an unbounded number of derived levels, in exploratory modeling it reflects a measure of uncertainty in how many levels there actually are in the subject of interest. This is related to the idea of open world interpretations of ontologies in knowledge engineering. However, the difference between open world and closed world interpretations of models revolves around how a modeling systems responds to queries in the absence of explicit information, while the uncertainty in exploratory, unbounded modeling revolves around the depth of isonym trees in multi-level ontologies. 6 Multi-Mode Modeling At first sight the different interpretations and definitions of potency presented in the previous two sections might seem incompatible. With the enhancements suggested in this paper they can be integrated within a single framework, in which constructive modeling is essentially a special case of exploratory modeling. For example, the blueprint/offspring notion in the constructive view of potency is a special case of the isonymic/hyponymic instance dichotomy since by definition an offspring is an isonym. Having the basis for a framework in which all four modeling modes can be applied to the same content does not necessarily make it a good idea. It begs the question: why would a unified framework supporting all four modes of modeling be of value? We believe there are many scenarios where such capabilities are of value, but for space reasons describe only two of them below. In most full spectrum software and database engineering projects the early phase of development starts out with an analysis or requirements elicitation phase where the basic properties of the problem to be solved and the systems requirements need to be understood. In other words, most projects start out in an exploratory mode where the goal is to learn and characterize the types in the domain of interest rather than define instantiatable types. But later on, once the

16 14 C. Atkinson, B. Kennel, B. Goß project moves into the design phase, the goal is to move into a constructive mode and actually define the types that will be instantiated in the final database or software system. In software engineering this distinction is often characterized in terms of the difference between the domain model and the design class diagrams [17]. Thus, there is an inherent and natural need to switch between exploratory and constructive modes of development in most software and database engineering projects. Ideally, engineers would like to be able to populate an ontology in an exploratory way near the beginning of a project and then at some point move to a constructive approach. Another important trend in the knowledge engineering and model-driven development communities is to explore how model content can be used at runtime, with associated reasoning and checking facilities, to drive the execution of software systems. This research goes under the name of models@runtime or ontology-driven systems, and the resulting applications are often referred to as intelligent because they are able to exhibit behavior that is not hard-wired into them. Once again, this not only requires the ability to switch between constructive and exploratory modes of modeling (since the run-time ontology probably needs to be manipulated under the mechanisms for exploratory modeling) it also requires unbounded exploratory modeling since the direction and extent of an ontologys growth is unknown a priori. 7 Conclusion In this paper we have identified four fundamentally distinct modes of modeling, based on the interaction of two orthogonal dichotomies that reflect the scope of models and the direction in which they are populated. We also presented examples of scenarios in which these modes are needed and where the ability to switch seamlessly between them would greatly increase the efficiency of software and data engineering projects. We then presented a generalization of the potency-based approach to multi-level modeling that equips it with the ability to accommodate the different modes within a single, unified, multi-mode modeling framework. This unified framework opens up the possibility of a truly fundamental unification of the ontology engineering and software/database engineering approaches to modeling in which the advantages of one community s technology can be exploited by the other and vice versa. In particular, the exploratory definition of potency based on the distinction between isonymic and hyponymic instanceof relationships provides the basis for reasoning services of the kind available with OWL DL. The described approach therefore for the first time makes potency-based multi-level modeling useful to the knowledge engineering community as well as to the software and data engineering communities. We are currently working on an EMF-based prototype of the infrastructure which should make the approach usable within the rich set of facilities provide by the Eclipse modeling tools. We are also working on identifying stronger uses case for the unbounded exploratory mode of modeling and on understanding how these modes relate to the open and closed querying techniques applied to models.

17 Reconciling Constructive and Exploratory Modeling 15 References 1. Saeki, M. and Kaiya, H.: On Relationships among Models, Meta Models and Ontologies. In: Proceedings of DSM OOPSLA Workshop on Domain-Specific Modeling, Portland, Oregon USA (2006) 2. Atkinson, C., Gutheil, M., and Kiko, K.: On the Relationship of Ontologies and Models. 2nd International Workshop on Meta-modeling and Ontologies (WoMM2006), Karlsruhe (2006) 3. Weishan Z.; Hansen, K.M.; Fernandes, J.: Towards OpenWorld Software Architectures with Semantic Architectural Styles, Components and Connectors. In : ICECCS 09 Proceedings of the th IEEE International Conference on Engineering of Complex Computer Systems, vol., pp IEEE Press, New York (2009) 4. Object Management Group, Inc.: Ontology Definition Metamodel. (2009) 5. Dong, J.S., Lee, C. H., Lee, H. B., Li, Y. F., Wang, H.: A combined approach to checking web ontologies. In: Proceedings of the 13th international conference on World Wide Web, New York, NY, USA (2004) 6. MOST Project - Marrying Ontology and Software Technology, 7. World Wide Web Consortium (W3C), 8. Atkinson, C., Kühne, T.: Model-Driven Development: A Metamodeling Foundation. In: IEEE Software, 20(5), 3641 (2003) 9. Kühne, T.: Matters of (Meta-)Modeling. In: Journal on Software and Systems Modeling, Volume 5, Number 4, (2006) 10. Parreiras, F., Fernando, Staab, S., Winter, A.: TwoUse: Integrating UML Models and OWL Ontologies. Institut für Informatik, Universität Koblenz-Landau. Nr. 16/2007. Arbeitsberichte aus dem Fachbereich Informatik (2007) 11. Neumayr, B., Grün, K., Schrefl, M.: Multi-level domain modeling with m-objects and m-relationships. In: Proceedings of the Sixth Asia-Pacific Conference on Conceptual Modeling, Wellington, New Zealand (2009) 12. de Lara, J., Guerra, E.: Deep meta-modeling with METADEPTH. In: Vittek, J. (ed) TOOLS 10 Proceedings of the 48th international conference on Objects, models, components, patterns. LNCS, pp Springer, Heidelberg (2010) 13. Atkinson, C, Gutheil, M, Kennel, B.: A Flexible Infrastructure for Multi-Level Language Engineering. In: IEEE Transactions on Software Engineering, vol. 35, no. 6, pp (2009) 14. Staab, S., Walter, T., Gröner, G.,Parreiras, F.. Model driven engineering with ontology technologies. In: Reasoning Web. Semantic Technologies for Software Engineering, volume 6325 of LNCS. Springer (2010) 15. Jekjantuk, N., Gröner, G., Pan, J. Z., Zhao, Y.. Modelling and validating multilevel models with owl fa. In: Proceedings of the 6th International Workshop on Semantic Web Enabled Software Engineering (2010) 16. Atkinson, C., Kennel, B., Goß, B.: The Level-agnostic Modeling Language. In: Proceedings of the SLE LNCS, vol. 6563, Springer, Heidelberg (2010) 17. Larman, C.: Applying UML and Patterns: An Introduction to Object-Oriented Analysis and Design and the Unified Process. Prentice Hall PTR Upper Saddle River, NJ, USA (2001) 18. Seidewitz, E.: What models mean. In: IEEE Software 20(5), (2003) 19. Assmann, U., Zschaler, S., & Wagner, G. (2006). Ontologies, Meta-models, and the Model-Driven Paradigm. Ontologies for Software Engineering and Software Technology,

18 Developing Efficient Ontology-based Systems Using A-box Data from Running Applications Heiko Paulheim 1 and Lars Meyer 2 1 Technische Universität Darmstadt Knowledge Engineering Group paulheim@ke.tu-darmstadt.de 2 SAP Research {lars.meyer}@sap.com Abstract. Today, information is typically contained in different IT systems. Ontologies and semantic integration have been proposed for integrating this information with the purpose of meaningful processing and providing useful services on top of that information. As applications, at the same time, modify the information during run-time, the information to be integrated has a dynamic nature. In this paper, we discuss performance aspects of integrating such dynamically changing A-box data from running applications, point out several technical alternatives, and present performance measures for those alternatives. We show how the findings are applied in two different application examples. 1 Introduction In a typical software landscape, information is contained in different systems databases, legacy systems, desktop applications, and so forth. As users have to work with information from those different systems, integration is required. For providing a meaningful, semantic integration, ontologies have been widely discussed. There are several possible utilizations of such an integration, each providing different benefits to end users [1]: On a semantic desktop, novel ways of searching for data in applications are made possible by extracting semantically annotated information from applications [2, 3]. User interfaces can be automatically adapted according to the users needs by having a reasoner analyze the UI components, the data they contain, and the user s needs [4]. Help on applications can be provided at run-time, adapted according to the system s current state and/or a user model [5, 6]. Software components can be automatically integrated by having a reasoner process events raised by different components and thereby coordinating user interactions across different, heterogeneous components [7]. Interactive, ontology-based visualizations of the information contained in different related applications can assist the user in fulfilling knowledge-intensive tasks [8].

19 In all of those cases, the information is contained in running applications, which means that it is highly dynamic and thus needs to be integrated at run-time. Furthermore, reasoning on that data is essential for providing valuable information to the end user. Thus, it is required that a reasoner has efficient access to the data as its A-box. At the same time, user interaction is involved all of the cases, which imposes strict requirements in terms of performance. Thus, high performance mechanisms for reasoning on dynamic data from running applications are needed. With this paper, we investigate different architectural alternatives of building a systems which support efficient integration and reasoning about running software applications, and we analyze the performance impact of the different alternatives with respect to dynamic data. In two examples, we show how the findings can be applied to improve the performance of real-world applications of semantic integration. The rest of this paper is structured as follows. In section 2, we introduce our basic reasoning framework. Section 3 discusses different approaches for optimization, which are evaluated in section 4. In section 5, we introduce two example use cases for our framework and discuss how they benefit from the optimization strategies. We conclude with a review on related work, a summary, and an outlook on future work. 2 Basic Architecture From our work in application integration, we have derived a generic architecture for reasoning on running software applications. In [9], we have analyzed two different basic architectural variants for providing A-box data from running software components to a reasoner: 1. In a pushing approach, software components inform the reasoner about updates, and the reasoner keeps an up-to-date representation in its own A-box, which duplicates the original data contained in the different components. 2. In a pulling approach, the reasoner does not maintain an A-box. Instead, it dynamically pulls instance data from the components whenever the evaluation of a query demands for that instance data. Our experiments have shown that only pulling approaches are feasible for building a scalable solution [9]. The reason is that given a highly dynamic component which changes its state quite frequently, the reasoner is kept busy with processing the updates on its A-box. Once the update frequency exceeds a certain threshold, the reasoner is overloaded and cannot answer any queries anymore. On the other hand, many of those updates are unnecessary, e.g., if the same facts are overwritten many times without being used in a query. A pulling approach avoids those unnecessary updates, which only create system load without any benefit. Furthermore, maintaining an A-box with information which is also kept in the system leads to an overhead due to double bookkeeping and can therefore

20 Component Container Component Container Component Container Component 1 Component 2 Component n O1 O2 O3 O4 O5 Objects O1 O2 O3 O4 O5 Objects O1 O2 O3 O4 O5 Objects Component Adapter register Component Adapter register... Component Adapter register URI Factory Object Registry URI Factory Object Registry URI Factory Object Registry Triple Factory Triple Cache Triple Factory Triple Cache Triple Factory Triple Cache query query query Query Client query about running system Query Interface Reasoning Module A-box Connector T-box Connector Ontology T-Box Fig. 1. General architecture of our framework for reasoning about running applications. cause consistency problems: with a pushing approach, the updates and queries need to be properly queued in order to guarantee correct answers to each query. Fig. 1 shows the basic building blocks of our framework. It depicts a number of software components which are to be integrated. Each component (which we treat as a black box) is encapsulated in a container, which provides an adapter to the reasoner. The adapter consists of four essential parts: Integrated components create and hold objects, which they register at the adapter s object registry to reveal them to the reasoner. Each component may process different types of objects, and each type of object may be processed by different components. There are different possible strategies of informing the registry of updates of the registered objects (e.g., using the observer pattern or change listeners); in our current implementation, the component actively sends updates to the object registry. From those objects, a triple factory creates data that is usable by the reasoner, i.e., RDF triples. The triple factory is also responsible for processing mappings between the component s class model and the ontology used for information exchange. Those mappings are defined in a flexible, rule-based language, which is also applicable for conceptually heterogeneous class models [10]. To create such triples, a URI is needed for each object. The URI factory is responsible for creating such URIs which are unambiguous and unique throughout the whole integrated system. To improve performance, triples may be cached by the component. There are different variants such as lazy or eager caches, which will we analyze in more detail in the subsequent sections.

21 These adapters are used by the reasoner s A-box connector to dynamically resolve queries. In addition to that A-box connector, a T-box connector provides the T-box part of the ontology (i.e., the definition of classes and relations) used as a common ground for integrating information from the different components. In contrast to the A-box, the T-box is considered as static, and the T-box connector loads it once when the system starts. The reasoner has a query interface that allows client components to pose queries about the running system. Client components may be, e.g., internal components, such as an event processing logic, as discussed in section 5.1, or a graphical user interface providing an endpoint for querying the system, as discussed in section 5.2. The query interface may use languages such as SPARQL or F-Logic. For the prototype implementation of our framework, we have used OntoBroker [11] as a reasoner, and F-Logic [12] as a query language. When a query runs, the results of that query have to be consistent. Thus, updates occuring between the start of a query and its end should not be considered when computing the result. To provide such a consistent query answering mechanism, the reasoner sends a lock signal to the component wrappers when a query is started. Updates coming from the component are then queued until the reasoner notifies the component that the query has been finished. 3 Aspects of Optimization In the previous section, we have sketched the basic architecture of our system. There are different variants of implementing that architecture. In [9], we have discussed two basic aspects: centralized and decentralized processing, and using a redundant A-box vs. using connectors for retrieving instance data at query time. These results led to the architecture introduced in Sect. 2, using a centralized reasoner and connectors for retrieving A-box data. In this section, we have a closer look at two design aspects which allow several variations: the design of the rules which make the reasoner invoke the A-box connector, and the use of caches. 3.1 Design of Connector Invocation Rules To make the reasoner invoke a connector, a rule is needed whose head indicates the type of information the connector will deliver, and whose body contains a statement for actually calling that connector. Technically, a connector is wired to the reasoner with a predicate. For example, a connector providing instance data for an object can be addressed with a generic rule as follows 1 : instance connector(?i,?c)?c(?i). (1) 1 We use the common SWRL human readable syntax for rules, although in SWRL, variables are not allowed for predicates. In our implemented prototype, we have used F-Logic for formulating the rules, which allows for using variables for predicates.

22 The reasoning framework, OntoBroker in our case, is responsible for dispatching the use of the predicate instance connector to an implementation of that connector, i.e. a Java method. This method then provides a set of bindings for the variables (in this case:?i and?c). If some of the variables are already bound, the contract is that the method returns the valid bindings for the unbound variables which yield a true statement given the already bound ones. Consider the following example query, asking for all instances of a class #Person: SELECT?I WHERE {?I rdf:type #Person } The resolution of this query leads to the invocation of rule 1 with the variable?c bound to #Person. The connector method now returns a set of bindings for the variable?i for which the statement #P erson(?i) is true. The reasoner then substitutes the results in the query and returns the result. The mechanism defined in rule 1 is the most basic way of integrating a connector which delivers information about instances and the classes they belong to. As it has to be evaluated in each condition in a rule s body where statements like?c(?i) occur (either unbound or with one or both of the variables bound), the connector is invoked very frequently. Since invoking a connector may be a relatively costly operation (even with caches involved, as described below), this is a solution which may imply some performance issues. A possible refinement is the use of additional constraints. The idea is that for each integrated software component, the set of possible ontology classes the data objects may belong to is known. Given that the union of those sets over all components is #Class1 through #ClassN, the above rule can be refined to an extended rule: (equal(?c, #Class1) equal(?c, #Class2)... equal(?c, #ClassN)) instance connector(?i,?c)?c(?i) (2) Assuming a left to right order of evaluation of the rule s body, the connector is now only invoked in cases where the variable?c is bound to one of the given values. Therefore, the number of the connector s invocations can be drastically reduced. A variant of that solution is the use of single rules instead of one large rule: instance connector(?i, #Class1) #Class1(?I) instance connector(?i, #Class2) #Class2(?I) instance connector(?i, #ClassN) #ClassN(?I) (3) In that case, the connector is not always invoked when evaluating a statement of type?c(?i). Instead, each rule is only invoked for exactly one binding of?c. In the example query above, only the first rule s body would be evaluated at all, invoking the connector once with one bound variable. On the other hand, the number of rules the reasoner has to evaluate for answering a query is increased....

23 O1 O2 O3 O4 O5 O1 O2 O3 O4 O5 O1 O2 O3 O4 O5 Component query Adapter query Queries are passed through to the original component update Component Adapter query Cache entries are replaced when updates occur update Component Adapter query query on cache miss Cache entries are invalidated when updates occur A-box Connector A-box Connector A-box Connector (a) Without cache (b) Eager cache (c) Lazy cache Fig. 2. Different variants for using caches The above example rules show how to invoke the instance connector wrapper, which returns statements about category membership of instances. The other important is relation connector(?x,?r,?y ), which has three variables. It returns the set of all triples where object?x is in relation?r with object Y. As for the instance connector wrapper, the corresponding types of invocation rules exist. For this paper, we have analyzed the performance impact of all three rule types: the generic rule (1), the use of an extended rule (2), and the use of single rules (3). 3.2 Distributed Caching of A-Box Fragments To speed up the answer of our connectors, partly caching instance data in the connector is a good strategy [13], although it slightly contradicts to the idea of avoiding double bookkeeping it is the classic trade-off of redundancy vs. performance. We have analyzed three different variants: using no caches at all, i.e., each query for instance data is directly passed to the underlying objects, and statements are assembled at query time (see Fig. 2(a)); and using eager and lazy caching. While the eager cache updates the required statements for each object when that object changes (see Fig. 2(b)), the lazy cache flags statements as invalid upon change of the represented object, and re-creates them only if they are requested (see Fig. 2(c)). While using no caches at all avoids unnecessary workload when an update occurs, eager caches are supposed to be the fastest to respond to queries. Lazy caches can provide a compromise between the two, allowing fast responses to queries as well as avoiding unnecessary workload. In the next section, we will analyze those effects in more detail.

24 Description Query in SPARQL 1 Get all objects of type C SELECT?I WHERE {?I rdf:type C.} 2a Get all objects of type C SELECT?I WHERE {?I rdf:type C.?I R O.} 2b in a relation R with SELECT?I WHERE {?I rdf:type C. O R?I.} 2a+b object O SELECT?I WHERE {{?I rdf:type C.?I R O.} UNION {?I rdf:type C. O R?I.}} 3a Get all objects of type C SELECT?I WHERE {?I rdf:type C.?I?R O.} 3b in any relation with SELECT?I WHERE {?I rdf:type C. O?R?I.} 3a+b object O SELECT?I WHERE {{?I rdf:type C.?I?R O.} UNION {?I rdf:type C. O?R?I.}} 4a Get all objects of any SELECT?I WHERE {?I R O.} 4b type in a relation R with SELECT?I WHERE {O R?I.} 4a+b object O SELECT?I WHERE {{?I R O.} UNION {O R?I.}} 5a Get all objects of any SELECT?I WHERE {?I?R O.} 5b type in any relation with SELECT?I WHERE {O?R?I.} 5a+b object O SELECT?I WHERE {{?I?R O.} UNION {O?R?I.}} Table 1. The different query types we used to analyze the performance impact of different variants, and their SPARQL representation. 4 Evaluation 4.1 Query Times In this section, we analyze the performance impact of the different connector rule and caching variants on different reasoning tasks which involve information about and in the running system. We have examined both the query times for different query types, as well as the maximum degree of dynamics, i.e. the maximum frequency of updates to the A-box which can be handled by the different implementation variants. 4.2 Setup To evaluate the impact of the different variants, we have analyzed various elementary query types, as shown in table 1. The queries were selected to cover a variety of cases with known or unknown predicate types, as well as known or unknown type restrictions for the queried objects. For each query, we have measured the average time to process that query. All tests have been carried out on a Windows XP 64Bit PC with an Intel Core Duo 3.00GHz processor and 4GB of RAM, using Java 1.6 and OntoBroker 5.3. The variables in our evaluation are the number of software components (5, 10, and 20), the number of object instances maintained by each component (250 and 500, randomly distributed over 10 classes per component), and the average update frequency (25 and 50 updates per second). Each instance has been given 5 relations to other instances. Therefore, our maximum test set has object

25 instances with relations. As our focus is on dynamic data, we altered the data at a frequency of 50 updates per second in the first set of evaluations. In Fig. 3, we show the results of selected typical queries, which illustrate the main findings of our analysis 2. While the figure depicts the results for the a + b flavor of each query type (see table 1), the results for the a and b flavor are similar. Generally, the query times using eager caching are faster than those using lazy caching. The actual factor between the two ranges from double speed (e.g., in case of type 4 queries) to only marginal improvements (e.g., in case of type 3 queries). Since lazy caches have to re-create the invalidated triples to answer a query, eager caches can always serve the requested triples directly. Therefore, the latter can answer queries faster. When looking at scalability, it can be observed that doubling the number of integrated components (which also doubles the number of A-box instances) about doubles the query answer time in most cases, thus, there is a linear growth of complexity in most cases. Multiple observations can be made when looking at the individual results regarding different queries. For type 3 queries, it is single rules which produce significant outliers (see Fig. 3(b)), while for type 5 queries, it is generic rules (see Fig. 3(d)). Thus, only only extended rules guarantee reasonable response times in all cases without any outliers, although they are outperformed by generic rules in type 2 and 4 queries. In case of type 1, 2 and 4 queries, the relation in which the objects sought is already fixed (e.g., find all persons which are married to another person ), while the case of type 3 and 5 queries, the relation is variable (e.g., find all persons which have any relation to another person ). The first type of query is rather target-oriented, while the latter is rather explorative. The key finding of the results is that target-oriented queries do not pose any significant problems, while explorative queries do. The bad behavior of single rules in the case of explorative queries can be explained by the fact that when an explorative query is answered, the various single rules fire, thus causing many potentially expensive invocations of the connector for N components and M types, an explorative query may cause up to O(N M) connector invocations. The generic and extended rules, on the other hand, invoke the connector less often. For a similar reason, the generic rule variant behaves badly for the explorative query types: here, the reasoner determines the possible relation types by invoking the wrapper multiple times, each time trying another relation type. Extended and single rules, on the other hand, already restrict the relation types in the rule body, thus requiring less invocations of the wrapper. 2 Type 1 queries are not shown; they are generally answered very quickly, and there are no significant differences between the approaches.

26 4.3 Maximum Frequency of A-box Updates Besides the time it takes to answer a query, another important performance criterion is the robustness of the system regarding A-box dynamics. While the rule design only influences the query times as such, a careful design of the wrappers caches has a significant impact on the system s scalability with respect to the maximum possible frequency of A-box updates, as shown in Fig. 4. The figure shows that while both eager and lazy caches do not drop in performance too strongly when scaling up the number of instances involved, lazy caching is drastically more robust regarding A-box dynamics. While several thousand updates per second on the A-box are possible with lazy caching, eager caching allows for less than 100. As assumed, lazy caches thus scale up much better regarding a-box dynamics, but at the trade-off of longer query response times, as shown above. 5 Examples To illustrate the relevance of the findings presented in the previous sections, we introduce two examples: one using goal-directed and one using explorative queries. 5.1 Example for Goal-Directed Queries: Semantic Event Processing In [14], we have discussed the use of ontologies for application integration on the user interface level. The approach relies on using ontologies for formally describing user interface components and the information objects they process. Reasoning is used to facilitate semantic event processing as an indirection for decoupling the integrated applications [9]. By annotating the events produced by different user interface components, a reasoner can analyze those events, compute possible reactions by other components, and notify those components for triggering those reactions. This reasoning process requires instance information about the different applications, their states, and the data they process, which is delivered by the framework explained in Sect. 2. An example for an integration rule could state the following: When the user performs a select action with an object representing a customer who has an address, the address book component will display that address, if it is visible on the screen. If this rule is evaluated by a reasoner, it has to be able to validate certain conditions, e.g. whether there is an address book component which is visible, or whether the customer in question has an address. It therefore needs access to information about both the system s components as well as the information objects they process. More sophisticated reasoning may come into place, e.g., when implementing different behaviors for standard and for premium customers, where the distinction between the two may involve the evaluation of different business rules.

27 Generic Rule, Eager Cache Generic Rule, Lazy Cache Extended Rule, Eager Cache Extended Rule, Lazy Cache Single Rules, Eager Cache Single Rules, Lazy Cache Generic Rule, Eager Cache Generic Rule, Lazy Cache Extended Rule, Eager Cache Extended Rule, Lazy Cache Single Rules, Eager Cache Single Rules, Lazy Cache (a) Query type 2a+b (b) Query type 3a+b Generic Rule, Eager Cache Generic Rule, Lazy Cache Extended Rule, Eager Cache Extended Rule, Lazy Cache Single Rules, Eager Cache Single Rules, Lazy Cache Generic Rule, Eager Cache Generic Rule, Lazy Cache Extended Rule, Eager Cache Extended Rule, Lazy Cache Single Rules, Eager Cache Single Rules, Lazy Cache (c) Query type 4a+b (d) Query type 5a+b Fig. 3. Query times for selected query types, each for 500 object instances per component, and 50 updates per second. The x axis shows the number of components (there have been no explicit measurements for 15 components), and the y axis shows the query time in seconds. A typical query used for event processing asks: given a particular event #E, which other events are triggered by that event: SELECT?E1 WHERE {?E1 #triggeredby #E.} Thus, the predicate is fixed, and the query is goal-directed. As discussed for the general results above, we have experienced that the different invocation rule variants do not affect performance too much, while eager caching leads to a significant speed-up. Details on the example can be found in [9]. 5.2 Example for Explorative Queries: Semantic Data Visualization Gathering and aggregating information from different IT systems can be a cumbersome and time consuming task for the end user. Combining that data with a reasoner can provide direct benefit for the end user. In [8], we have introduced the Semantic Data Explorer. The Semantic Data Explorer provides a uniform graphical visualization of the data contained from applications using a central reasoning module, using the architecture discussed

28 Max. updates/second Eager Cache Lazy Cache Fig. 4. Evaluation of robustness of different caching approaches regarding A-box dynamics. The graph shows the maximum number of updates per second that the system can process. Note that the y-axis has a logarithmic scale. above. The user can drag objects from connected applications to the Semantic Data Explorer and navigate the corresponding graph view. The Semantic Data Explorer uses the reasoner as an indirection for constructing the graph view. From an implementation point of view, this architecture provides a decoupling of the visualization and the data sources. More importantly, the reasoner may also reveal implicit knowledge gathered from the A-box information using T-box axioms and rules. This implicit knowledge is then included in the visualization as well, providing additional value to the end user. A user study has shown that the Semantic Data Explorer can lead to significantly faster task completion times when gathering information, as well as to enhanced user satisfaction 3. Implementation details on the tool, as well as a detailed description of the user study can be found in [8]. Displaying a node in the Semantic Data Explorer requires finding all incoming and outgoing edges to other objects and data properties. Thus, the underlying queries are explorative: SELECT?R?V WHERE {<#x>?r?v} SELECT?R?V WHERE {?V?R <#x>} The results above advise to use extended rules for this sort of queries. In fact, experiments with the SDE showed that using an extended rule leads to a perceivably faster system, which in turn increases the end users satisfaction. 3 A demo video is available at

29 6 Related Work One of the best researched approaches for reasoning on objects from integrated systems is the use of so-called wrappers [11] or mediators [13], which collect objects from databases or structured documents and provide them to a reasoner as instance data. D2RQ [15] is an example for a wrapper platform that integrates standard, non-rdf databases as RDF data sources and thus makes them available to a reasoner. Based on mapping rules, data entries from database tables are lifted as RDF instance data. The authors present an evaluation based on different query types that shows that retrieval of the data is feasible in reasonable time. Lixto [16] is an example that uses wrappers to gather RDF data from nonannotated web pages. It provides graphical interfaces for defining the mechanisms used to extract data from the HTML documents. The authors show different use cases where RDF data gathered from the web is utilized. Those applications do not perform the retrieval at run-time, but offline, i.e. they parse web sites and build an RDF data store. The user s queries are then posed against that RDF data store. OntoBroker [11] is a reasoning engine that provides different means for integrating data from various sources, including access to databases, web pages and web services via so-called connectors. As the API also foresees the integration of own connectors accessing arbitrary sources of instance data, we have based the prototype described in this paper on OntoBroker. Various approaches have been proposed for directly accessing objects of running software applications [17]. There are two main variants of making the instance data known to the reasoner. The first relies on semantic annotation of the underlying class models, such as sommer 4 or otm-j [18]. The second uses class models generated directly from an ontology, with the necessary access classes for reasoning access being generated as well, such as RDFReactor [19], or OntoJava [20]. With dynamically typed scripting languages, the corresponding classes may also be generated on the fly, as shown, e.g., with Tramp 5 for Python. A detailed comparison of such approaches is given in [10]. However, analyses of efficiency and scalability of these approaches are rarely found. Most of those approaches are not very flexible with respect to conceptual heterogeneity (i.e., class models that are different from the common ontology used for integration) as well as technological heterogeneity (i.e., using class models in different programming languages in parallel). The framework discussed in this paper uses flexible mapping rules and allows for containers for different programming languages [10, 21]. One of the best-known and most compelling application of making data from various applications known to a reasoner is the semantic desktop [3]. It allows users to browse, analyze, and relate data stored in different applications and provides new means of accessing data stored on a personal computer. Different

30 adapters exist which wrap data from databases, file systems, or clients. While there are various publications concerning impressive applications of the semantic desktop as well as various architectural aspects, systematic approaches of assessing the performance of the underyling technology are still hard to find. 7 Conclusion and Outlook In this paper, we have introduced a framework for integrating dynamic A-box data from running software system with a central reasoner. There are several use cases for applying such a framework, e.g. searching information from different applications on a semantic desktop, dynamically adapting user interfaces to users needs, or automatically integrating existing user interface components to a seamless application at run-time. In all of those approaches, a reasoner is used, which may need to have access to the data both contained in software components as well as about those software components as such. As reasoning is performed while those components are running, the A-box can be highly dynamic. In most of the use cases of reasoning on dynamic systems sketched above, good performance is an essential requirement, as user interactions are involved. Based on the prototype implementation of our architecture, we have conducted several experiments to evaluate the performance impact of different implementation variants. Those variants encompass different caching strategies as well as the design of the rules from which the connectors are called. We have tested the variants with 13 different query types. In this paper, we have analyzed the performance impact of three different rule types for rules invoking connectors to software components: generic rules, extended rules, and single rules. In some test cases, the query answering times even differ at a factor of 100 between the different approaches. This proves that the design of rules has a significant impact on the system performance. One major finding is that there is no solution that provides optimal results for each usage scenario. In summary, we have shown that there are significant differences between explorative and goal-directed queries: in the first case, queries contain a fixed relation (e.g. find all persons that are married to another person ), while in the latter case, the relation is a variable (e.g. find all persons that have any relation to another person ). While some queries are handled almost equally well by all three rule types, only extended rules guarantee reasonable and stable query answering times in all cases. To illustrate the significance of the results, we have introduced two example use cases, one using goal-directed and one using explorative queries. In addition, we have analyzed two different strategies for caching data in the wrappers: eager and lazy caching. Eager caching allows for response times up two twice as fast as lazy caching. On the other hand, lazy caching supports much more dynamic A-boxes: eager caching only works for less than 100 A-box updates per second, while with lazy caches, several thousand A-box updates per second can be processed. Therefore, a trade-off between A-box dynamics and

31 query times can be identified. When implementing an actual system, a solution should be chosen according to that system s actual requirements. In this paper, we have analyzed the performance effects using a set of elementary queries in this paper, and we have shown that different implementation variants perform better or worse with certain query types. More complex query types may reveal deeper insights into performance optimization of semantic applications. While semantic technologies and reasoning on running software applications allow for interesting and valuable functionality, poor performance can be and in fact often is a show stopper. Thus, such applications should be carefully designed in order to be adopted by end users on a larger scale. With this paper, we have given insight in some strategies which can be carried over to the development of high performance systems using semantic technology. We are confident that this contribution will help developers of semantic web based software in creating systems which be come widely accepted. Acknowledgements The work presented in this paper has been partly funded by the German Federal Ministry of Education and Research under grant no. 01ISO7009 and 01IA References 1. Paulheim, H., Probst, F.: Ontology-Enhanced User Interfaces: A Survey. International Journal on Semantic Web and Information Systems 6(2) (2010) Cheyer, A., Park, J., Giuli, R.: IRIS: Integrate. Relate. Infer. Share. [22] 3. Sauermann, L., Bernardi, A., Dengel, A.: Overview and Outlook on the Semantic Desktop. [22] 4. Karim, S., Tjoa, A.M.: Towards the Use of Ontologies for Improving User Interaction for People with Special Needs. In Miesenberger, K., Klaus, J., Zagler, W.L., Karshmer, A.I., eds.: ICCHP. Volume 4061 of Lecture Notes in Computer Science., Springer (2006) Gribova, V.: Automatic Generation of Context-Sensitive Help Using a User Interface Project. In Gladun, V.P., Markov, K.K., Voloshin, A.F., Ivanova, K.M., eds.: Proceedings of the 8th International Conference Knowledge-Dialogue-Solution. Volume 2. (2007) Kohlhase, A., Kohlhase, M.: Semantic Transparency in User Assistance Systems. In: Proceedings of the 27th annual ACM international conference on Design of Communication. Special Interest Group on Design of Communication (SIGDOC- 09), Bloomingtion,, IN, United States, ACM Special Interest Group for Design of Communication, ACM Press (2009) Paulheim, H.: Ontologies for User Interface Integration. In Bernstein, A., Karger, D.R., Heath, T., Feigenbaum, L., Maynard, D., Motta, E., Thirunarayan, K., eds.: The Semantic Web - ISWC Volume 5823 of LNCS., Springer (2009) Paulheim, H.: Improving the Usability of Integrated Applications by Using Visualizations of Linked Data. In: Proceedings of the International Conference on Web Intelligence, Mining and Semantics (WIMS 11). (2011)

32 9. Paulheim, H.: Efficient Semantic Event Processing: Lessons Learned in User Interface Integration. In Aroyo, L., Antoniou, G., Hyvönen, E., ten Teije, A., Stuckenschmidt, H., Cabral, L., Tudorache, T., eds.: The Semantic Web: Research and Applications (ESWC 2010), Part II. Volume 6089 of LNCS., Springer (2010) Paulheim, H., Plendl, R., Probst, F., Oberle, D.: Mapping Pragmatic Class Models to Reference Ontologies. In: The 2011 IEEE 27th International Conference on Data Engineering Workshops - 2nd International Workshop on Data Engineering meets the Semantic Web (DESWeb). (2011) Decker, S., Erdmann, M., Fensel, D., Studer, R.: Ontobroker: Ontology Based Access to Distributed and Semi-Structured Information. In Meersman, R., Tari, Z., Stevens, S.M., eds.: Database Semantics - Semantic Issues in Multimedia Systems, IFIP TC2/WG2.6 Eighth Working Conference on Database Semantics (DS- 8), Rotorua, New Zealand, January 4-8, Volume 138 of IFIP Conference Proceedings., Kluwer (1999) Angele, J., Lausen, G.: Ontologies in F-Logic. In Staab, S., Studer, R., eds.: Handbook on Ontologies. International Handbooks on Information Systems. 2nd edition edn. Springer (2009) Wiederhold, G., Genesereth, M.: The Conceptual Basis for Mediation Services. IEEE Expert 12(5) (sep/oct 1997) Paulheim, H., Probst, F.: Application Integration on the User Interface Level: an Ontology-Based Approach. Data & Knowledge Engineering Journal 69(11) (2010) Bizer, C., Seaborne, A.: D2RQ - Treating Non-RDF Databases as Virtual RDF Graphs. In: ISWC2004 Posters. (November 2004) 16. Baumgartner, R., Eiter, T., Gottlob, G., Herzog, M., Koch, C.: Information Extraction for the Semantic Web. In Eisinger, N., Maluszynski, J., eds.: Reasoning Web. Volume 3564 of Lecture Notes in Computer Science., Springer (2005) Puleston, C., Parsia, B., Cunningham, J., Rector, A.: Integrating Object-Oriented and Ontological Representations: A Case Study in Java and OWL. In Sheth, A.P., Staab, S., Dean, M., Paolucci, M., Maynard, D., Finin, T.W., Thirunarayan, K., eds.: The Semantic Web - ISWC Volume 5318 of LNCS., Springer (2008) Quasthoff, M., Meinel, C.: Semantic Web Admission Free - Obtaining RDF and OWL Data from Application Source Code. In Kendall, E.F., Pan, J.Z., Sabbouh, M., Stojanovic, L., Bontcheva, K., eds.: Proceedings of the 4th International Workshop on Semantic Web Enabled Software Engineering (SWESE). (2008) 19. Völkel, M., Sure, Y.: RDFReactor - From Ontologies to Programmatic Data Access. In: Posters and Demos at International Semantic Web Conference (ISWC) 2005, Galway, Ireland. (2005) 20. Eberhart, A.: Automatic Generation of Java/SQL Based Inference Engines from RDF Schema and RuleML. In Horrocks, I., Hendler, J.A., eds.: The Semantic Web - ISWC 2002, First International Semantic Web Conference, Sardinia, Italy, June 9-12, 2002, Proceedings. Volume 2342 of Lecture Notes in Computer Science., Springer (2002) Paulheim, H.: Seamlessly Integrated, but Loosely Coupled - Building UIs from Heterogeneous Components. In: ASE 10: Proceedings of the IEEE/ACM International Conference on Automated Software Engineering, New York, NY, USA, ACM (2010) Decker, S., Park, J., Quan, D., Sauermann, L., eds.: Proceedings of the ISWC 2005 Workshop on The Semantic Desktop - Next Generation Information Management & Collaboration Infrastructure. Volume 175 of CEUR-WS. (2005)

33 Results of a Survey on Improving the Art of Semantic Web Application Development Emanuele Della Valle 1, Gaetano Niro, and Catalina Mancas 2 1 Politecnico di Milano, Dipartimento di Elettronica e Informazione, Milano - Italy 2 University of Craiova, Software Engineering Department, Craiova - Romania emanuele.dellavalle@polimi.it, gaetano.niro@gmail.com, Catalina.Mancas@ucv.ro Abstract. While the demand for solutions based on Semantic Web Technologies (SWTs) is rapidly growing, engineering a Semantic Web Application (SWA) is still largely an art. We believe that the lack of an end-to-end methodology is a major barrier for industrial uptake of SWTs. In order to empirically prove our thesis, we run two surveys: a preliminary one in summer of 2010 with 26 participants and a broad one in winter and spring 2011 with 111 participants. In this paper, we report our main findings: we identified the the most frequent barriers in applying SWTs, we analyse the impact of SWT adoption on the various phases of a software project, we examined how risky it is to apply SWTs, and we report on the reaction of both the developers team and the customers towards SWTs. We believe that the analysis herein presented can cast some light on the art of engineering SWAs and help in understanding which methods and tools the community should focus on for fostering Semantic Web industrial uptake. 1 Introduction In the last decade, a growing amount of Semantic Web Applications (SWAs) have been identified [11,6,16] and developed using the Semantic Web Technologies (SWTs) standardized by the World Wide Web Consortium (W3C). The recent industrial uptake Linked Data [5] can lead to the fake assumption that every future software project employing SWTs will succeed. However, in software project management, successful projects are the result of the right processes correctly monitored. Unfortunately, ontology engineering [13,21,20] is the only deeply studied process in SWA development. Our thesis is that the peculiarities of SWAs require a specific end-to-end methodology. What we see, when we observe the Linked Data successes, are projects based on the art and craft of few early (and very skilled) adopters. In the past years, we discussed our thesis with several Semantic Web expert, and they generally disagree. The Semantic Web community appears to believe that a project manager leading a SWA project should chose an appropriate standard software process and integrate ad-hoc the ontology development process. In order to empirically prove our thesis, we run two surveys: a preliminary one in summer of 2010 with 26 participants chosen among a carefully selected

34 group of project managers that lead SWA projects and a broad one in winter and spring 2011 open to every body. The participants to the second survey were 111. The topics of the two surveys were similar. The main difference (apart from the number of interviewees) among the two surveys is that the most frequent answers to the open questions of the first survey, became possible choices for the close questions of the second one. In this paper we present step by step the research methodology adopted and we report our main findings: we identified the most frequent barriers in applying SWTs, we analyse the impact of SWT adoption on the various phases of a software project, we examined how risky it is to apply SWTs, and we report on the reaction of both the developers team and the customers. The rest of the paper is organized as follows. In Section 2, we present the research process. In Section 3 we briefly review the literature that we used to formulate our two surveys. Section 4 presents how the surveys were created the content of the two surveys. In Section 5, we describe how the two surveys were conducted and the relationships among them. In Section 6, we report the results of the second survey and we discuss our findings. Finally, in Section 7 we briefly conclude sketching a methodology supported by a case tool. 2 The Research Process The research process followed in conducting our work consists in four steps illustrated in Figure 1. In 18 months, we repeated those steps twice. Fig. 1. Our Research Process The very first step in our research process consists in reviewing the literature of Semantic Web and Software Engineering from the late 90s until today, in order to understand the current state of the art of SWA development in terms of available technologies, process, and tools. In the second step, the survey creation, we use the Goal, Question, Metric (GQM) Approach [2] to identify the goal, i.e., check whether people that lead SWA projects perceives the need for an end-to-end methodology for SWA de-

35 velopment, formulate the questions and decide the metric to use to collect the answer for an effective analysis. The third step consists in the survey carry-out and results collection. For the preliminary survey, which we conducted in summer of 2010, we collected the answer of the 26 interviewees by sending the questionnaire via and, where possible, personally interviewing them. For the broader survey, which we conducted in winter and spring 2011, we used SurveyMonkey, a Web based survey tool. The interviewees to the second survey were 111. The fourth and final step consists in the analysis of the results. We used pivot tables to perform both single-question and cross-question analysis. We used the results of the first survey to refine and focus the second one. For instance, the most frequent answers to the open questions of the first survey, become possible choices for the close questions of the second one. 3 Literature Review In this section, we present the results of our literature review clustered by the following topics: a) domains in which SWAs have been developed, b) expected benefits of SWTs, c) barriers, problems and risks commonly encountered in developing SWAs, d) impact on Software Development Lifecycle (SDLC) of the introduction of SWTs in a software project, e) resource consumption and reusability of SWTs, f) techniques for costs and effort estimations in software projects targeting the development of a SWA, g) impact on the developers team of the adoption of SWT, and, h) last but not least, customer satisfaction in software projects targeting SWAs. Domains. In our literature review we found many papers that attest successful results at applying SWTs in a large variety of domains. The trace starts back in the late 90s when papers like [10] recognized a common thread binding the industrial and research communities: the urge need to share meaning of terms and common understanding of certain domain. Given that such a sharing can be easily achieved by conceptualizing a domain in an ontology, any domain enough knowledge intensive can be a potential target for SWA development. According to an authoritative study [12] conducted in 2009 in Europe, the four prototypical application fields for SWAs are: Healthcare and Biotechnologies, Knowledge Management, ecommerce and ebusiness, and finally, Multimedia and Audiovisual Services. Benefits. The literature abounds in papers claiming that the Semantic Web is going to embrace the same success as the Web [3,6,16,8]. These claims are in many cases supported by industrial demonstrators. A rich collection of them is available in W3C Semantic Web Case Studies and Use Cases [1]. By analysing them we extracted the following list of often claimed benefits: improving search relevance, enabling automatic annotation and tagging, offering better control over data and documents, and easing information and data integration. Barriers, Problems and Risks. Few papers report on investigations focused on encountered barriers, problems and risks in developing SWAs. One of

36 the most authoritative papers under this perspective remains [10] that identifies: lacks in usability; lack of tools; lack of skilled programmers; lack of methods and of best practices; low capability to estimate costs; and inability to assess benefit in a clear way. While we are not saying that no progress at all has been made (see, for instance, the introduction of Linked Data bests practices [4]), we believe that the Semantic Web community is still under-estimating how high the barriers for broad industrial uptake are. Impact on SDLC. As we already discussed in Section 1, methodologies supporting the development of ontologies for SWAs [13,21] and general-purpose software development methodologies already exist [20]. To our best knowledge, no end-to-end methodology for developing SWAs has been proposed and no one has conducted a systematic study on the impact of SWTs on the different phases of development of a software project. Resource Consumption and Reusability. The literature supports the idea that SWTs produce cost savings. For example, the authors of Ontology Driven Architectures and Potential Uses of the Semantic Web in Systems and Software Engineering [22], identify two positive aspects which lead to a cost decrease: a reduced maintenance overhead achieved through increases in consistency and an increased potential for reuse, substitution or extension via content discovery on the Semantic Web. However, this claim is based on few use cases developed in research project. An in-the-field survey has not been conducted. Cost Estimations. The Software Engineering literature [15] correlates successful software projects with rigorous estimation of time, costs and resources. The Semantic Web community has not proposed a cost estimation tool for SWA development, yet. The most advanced tool, proposed so far, is ONTOCOM [19], a parametric tool that estimates the costs of ontologies. People Dimension. As books like Peopleware [7] remind us, people are the most valuable asset in a software projects. Even if the Software Engineering literature abounds of research on the topic we were not able to find any study tailored to Semantic Web related literature on this topic. Customer Satisfaction. As we learn from software project management literature [15] minding and negotiating customer s expectations is the key to a successful software development. Nevertheless SWA customers expectations, assumptions, needs and reactions are only recently getting the relevance that they deserve. The best quote we found on this topic is by Richard Zlade who, in Inventing the Semantic Web... again [23], states: The Semantic Web [...] potential is enormous and the enthusiasm around it is fully justified. But lets not fool ourselves. We still need legions of users to make the Semantic Web successful. To win them over, were going to need to speak to their goals [...]. 4 Creating the Surveys The literature review attests the rapid success of SWAs, but, at the same time, the lack of several typical software engineering researches targeted to SWA development. To the best of our knowledge no empirical study was conducted on

37 the impact of SWTs on SDLC and development team, on the actual reusability of Semantic Web artefacts, and on the satisfaction of the costumer. To formulate the questionnaires of the two surveys we applied the Goal, Question, Metric (GQM) approach [2]; one of the most suitable method in defining measurable goals. Defining measurable goals is, indeed, a crucial step in creating the questionnaire because we cannot simply ask: do you feel the need for an end-to-end methodology for SWA development? We have to ask questions that reveal such a need (if present) and we need to define what exactly we want to analyse, making sure that a high percentage of the interviewees will understand the questions and will be able to answer them. The GQM Approach has a hierarchical structure presenting three levels of refinement. The upper conceptual level is the Goal level, where the aim of the survey and the viewpoint are decided. It is followed by the Question or the operational level, where a set of questions is used to characterize the way the assessment of a specific goal is going to be performed. The questions try to characterize the object of measurement with respect to a selected quality issue and to determine its quality from the selected viewpoint. The third and last quantitative level is the Metric level. Once the questions have been developed they need to be associated with the appropriate metrics. The same metric can be used to answer different questions under the same goal. We identify as goal of our survey the analysis of the life cycle of a SWA development trying to understand if the interviewees perceive the need for an end-to-end methodology, and as view point the one of an industrial software project manager. The space we can dedicate in the paper to questions and metrics allow us only to present the seven areas they cover 3 : Personal information we used this information to check that the interviewees matches our viewpoint and to perform some cross-question analysis (e.g., breaking down the answers on the domain the interviewees developed their SWA for). Barriers in applying SWTs our intention was to assess the most often encountered obstacles in applying SWTs. As metric, we propose to choose among the barriers attested by the literature (e.g., the redundancy with regard to other systems; the lack of tools, methods and best practices; or the fear of losing control over one s data) and a list of SWTs (i.e., reasoners, SPARQL engines, ontologies, RDF stores, Linked Data, or other specified by the interviewee). Impact of SWTs on each phase of the SDLC our intention was to assess how much each software development phases is positively or negatively affected by the adoption of SWTs in the software project. As metrics, we propose interviewees to choose among two lists. The first includes the usual project phases, i.e., concept identification, requirements collection and analysis, design, coding and debugging, integration, testing, deployment and 3 At [14,17], we provide interested readers with the text of the two surveys.

38 maintenance. The second focuses on processes specific of SWA development, i.e., SWA design, ontology engineering, Linked Data integration, reasoner integration, RDF store integration, querying reasoners and RDF stores, and providing a tailored visual interface for such knowledge intensive application. Risks in SWAs development our purpose was to identify the most frequently encountered problems in developing SWAs. We collected the answers using three metrics. The first one is the simple choice between the risk of running a project with higher costs, longer duration and lower quality of the resulting application. The second metric is a list of SWTs (i.e., the same metric used in the question about the barriers). The third metric is a list of processes specific of SWA development (i.e., the same metric used in the question about the impact of SWTs on SDLC). Team s acceptance of SWTs our aims was to identify the team s behaviour when they have to deal with SWTs. In the first survey, we left this as an open question. In the second version, we were able to propose a metric based on the most frequent answer of the first questionnaire (e.g., teams requires additional training, or difficulties in meeting deadlines) as true/false statements. In addition to those true/false statements we also used as metrics the list of processes specific of SWA development (i.e., the same metric used in the questions about the impact of SWTs on SLDC and risks in developing SWAs) and the list of SWTs (i.e., the same metric used in the questions about the barriers and risks in developing SWAs) to understand the level of effort required to perform the processes and use the SWTs. Cost and effort estimation in SWAs our intention was to understand if interviewees perceive the usefulness of a cost estimation tool specific to SWAs. In particular, we asked the interviewees whether they estimate costs or not and what methods they use when performing the cost estimation. Moreover, the question asks whether there is a clear separation between the cost estimation for the part of the application that involves usage of SWTs and the rest of the application. Customer s satisfaction about uses of SWTs our intention was to study the customer reaction to the use of SWTs in the beginning and in the end of the project. Final remarks our purpose was to understand the general perception of the interviewees about the survey and collect indications about how to improve it. 5 Conducting the Surveys As a target for our first survey we chose team coordinators with at least 10 years of experience (i.e., project managers in industry and main investigators in academia) and developers with at least 5 years of experience. We specifically targeted people active in technology transfer, i.e. people from academia with a record in technology transfer (e.g., launch of academic start-ups), and people from industry with an attitude in transferring research results. As main source

39 of potential interviewees we used the authors of the W3C Semantic Web Use Cases and Case Studies [1]. In summer 2010 we contacted 51 people and we collect 26 questionnaires. The number of interviewees was small, but sufficient to understand strong and weak points of our first survey. Moreover, we were able to use the answers to the open questions to formulate close questions in the second version of the survey. For the second version of the survey, we used SurveyMonkey, a web based tool that allows to formulate complex questionnaires with switches and that randomizes order of answers in close questions. The survey was open in winter and spring In total we collected 111 answers, 52,6% of which were provided by people that exactly match our target (i.e., team coordinators with at least 10 years of experience) and 26,3% of which were closely matching our target (i.e., chief developer with at least 5 years of experience). We believe that this numbers are sufficient to claim a good level of generality for our findings. 6 The Results Analysis In this section, we run through the findings of our second survey. Since some of the questions were optional, we do not report results of analysis made on questions that received little attention by the interviewees. 6.1 Barriers in applying SWTs As shown in Figure 2), according to our interviewee target the top-4 barriers are: lack of tools, lack of skilled programmers, lack of usability of the available tools and lack of standard ontologies. The bottom-2 barriers are: redundancy with regard to other systems and lack of scalability. In terms of SWTs, reasoners and ontologies are largely the most complex to put at work, while RDF stores and Linked Data appears easy to include in a software project. Fig. 2. Barriers perceived by the target group of our survey. Figure 3 allows for a cross-analysis of the answers by breaking down the results according to the domain the SWA under analysis was developed for.

40 Fig. 3. A cross-analysis of the perceived barriers broken down by the domain the SWA under analysis was developed for.

41 While the top-4 barriers are (sort of) natural for innovative technologies, the bottom-2 sounds as an alert for all the ongoing research on scalable reasoning. We advance the hypothesis that to foster industrial uptake of reasoners and ontologies research should focus on support tools. 6.2 Impact of SWTs on each phase of the SDLC As readable in Figure 4, the target of our survey, in general, believes that SWTs have a positive impact on the early phases of a software project (with the exclusion of the requirement gathering phase) and on integration, while they negatively impact on code & debugging and testing phases. Fig. 4. Impact of SWTs on the different phases of a software project. The results for the requirement and maintenance phases are controversial, our target group appears to have experienced both positive and negative impacts. It is worth to note that this results contradicts evidence from literature [22]. If we have the opportunity to run a third version of this survey, we will include specific questions on those points. In terms of SWA development specific processes, consensus appears to be that ontology engineering, reasoning integration and developing appropriate user interfaces are the most complex process, while integrating RDF stores and developing the SWA are the least complex. The results in general confirm those of the question on barriers. A particular attention point emerges for user interfaces. 6.3 Risks in SWAs development. As visible in Figure 5, according to our target, the higher risks for a software project aiming at developing a SWA are to overrun (the opinion of the 44,8% of the target) and exceed budget (the opinion of the 37,9% of the target). Only

42 Fig. 5. Risks perceived by the target group of our survey. 10,3% of the target believes that the adoption of SWTs can expose the software project to the risk of lower quality of the resulting software artefact. The results are similar even when we take a closer look at the data breaking them down per domain. However, it is worth to note that our target group believes the risk of lower quality of the solution is significant (21,4%) in the enterprise application integration domain. As shown in Figure 6, In terms of SWA development specific process the top- 3 risky ones are: ontology engineering, developing user interfaces and integrating Linked Data. Fig. 6. Risks perceived by the target group of our survey. While the first and the third risk confirm the results of the answer on barriers and impact of SWTs on SDLC, the second one (i.e., integrating Linked Data) appears to contradict the results of the question about barriers, where Linked Data are believed to be easy to integrate. This point requires further investigation. The good news is that integrating reasoners and designing the SWA are perceived as low risk tasks.

43 6.4 Team s acceptance of SWTs. As shown in Figure 7, among the true/false statement, which we proposed to our interviewees, the target of our survey believes that it is difficult to find the developers having enough knowledge about SWTs and that, thus, the team requires additional training. In general, they do not believe that the morale of the team is negatively effected by the adoption of SWTs. They have mixed feeling about the statement that the team is not able to meet deadlines due to the adoption of SWTs. Fig. 7. Number of interviewees in the target group of the second survey that agreed or disagreed with the statement about team acceptance of SWTs, which we collected in the first survey. Moreover, as visible in Figure 8, the results of the question of the effort spent on the SWA development specific processes and the list of SWTs confirms the finding of the question about barriers and risks with a small difference on querying reasoners and RDF stores for which a medium/high effort is highlighted. 6.5 Cost and effort estimation in SWAs. The question about cost estimation was answered only by 26 interviewees and even less of them answered the subquestions. Apparently only few people in our target group estimate costs (i.e., 13 interviewees). We believe this to be a concrete example of the general immaturity of the processes about SWA development from a software engineering point of view, but we cannot claim the generality of this statement and we omit the analysis of the results.

44 Fig. 8. Effort perceived by the target group of our survey.

45 6.6 Customer s satisfaction about uses of SWTs. According to our target, only 48,4% of the customer are aware of the benefits claimed in literature when embracing the development of a SWA. In the beginning of the project only 40% have a positive impression of the SWTs. The good news is that after the project this percentage grows to 70%. These results should rise the Semantic Web community attention on the point of communicating more broadly (i.e., beyond scientific publications) the benefits that SWTs can bring. 7 Conclusions The SWT industry is rapidly growing, but we believe that a sustainable growth requires mature software development processes. SWA have their peculiarities, but they are still a software artefacts. If we measure, based on the results of the two surveys, the maturity of the SWT industry in terms of the Capability Maturity Model [18], we asses that it is placed at level 1. At this level, few or no processes are in place, no cost and time estimation tools are used, systematic testing is neglected and success depends on the heroism of the few. We believe that the results of the survey show that the target of our survey perceives the need for an end-to-end methodology. Actually, they tell us that the methodology should come with an appropriate case tool. The methodology should specifically support the project manager in the requirements gathering, testing and maintenance phases. Moreover, it should include a light-way cost and time estimation tool required to bring the SWT industry to level 2 of the Capability Maturity Model. We hazard the hypothesis that the best case tool in this historical moment of Semantic Web should exploit component base software development [9]. This would lower the integration barriers, it would assure the correct interoperability of the components in the framework and it can allow for a light cost estimation tool based on the selected components. In particular, the components to include in the case are an RDF store, a framework to invoke both local and remote SPARQL Engines, a reasoner and a set of ontologies that can serve as starting point for the development of the ontology at the core of the SWA. Last, but not least, a special attention should be put to tools that support the development of graphic user interfaces. References 1. T. Baker, T. Heath, N. Noy, R. Swick, and I. Herman. Semantic Web Case Studies and Use Cases. Available as 2. V. R. Basili, G. Caldeira, and H. D. Rombach. Encyclopedia of Software Engineering, pages John Wiley & Sons, T. Berners-Lee, J. Hendler, and O. Lassila. The Semantic Web: Scientific American. Scientific American, May 2001.

46 4. C. Bizer, R. Cyganiak, and T. Heath. How to publish linked data on the web. Web page, Revised Accessed 22/02/ C. Bizer, T. Heath, and T. Berners-Lee. Linked data - the story so far. Int. J. Semantic Web Inf. Syst., 5(3):1 22, M. d Aquin, E. Motta, M. Sabou, S. Angeletou, L. Gridinoc, V. Lopez, and D. Guidi. Toward a new generation of semantic web applications. IEEE Intelligent Systems, 23(3):20 28, T. De Marco and T. Lister. Peopleware: Productive Projects and Teams (Second Edition). Dorset House Publishing Company, Incorporated, J. Domingue and D. Fensel. Toward a service web: integrating the semantic web and service orientation. IEEE Intelligent Systems, 23(1):86 88, G. T. Heineman and W. T. Councill. Component based software engineering : putting the pieces together. Addison-Wesley, Boston, R. Jasper and M. Uschold. A framework for understanding and classifying ontology applications. In Proceedings of the IJCAI-99 Workshop on Ontologies and Problem- Solving Methods (KRR5): Stockholm, Sweden: 1999, August 2nd, K. Kozaki, Y. Hayashi, M. Sasajima, S. Tarumi, and R. Mizoguchi. Understanding semantic web applications. In J. Domingue and C. Anutariya, editors, ASWC, volume 5367 of Lecture Notes in Computer Science, pages Springer, A. Leger, J. Heinecke, L. J. Nixon, P. Shvaiko, J. Charlet, P. Hobson, and F. Goasdou. Semantic Web take-off in a European Industry Perspective. Idea Group Inc., M. F. Lopez, A. G. Perez, and N. Juristo. METHONTOLOGY: from Ontological Art towards Ontological Engineering. In Proceedings of the AAAI97 Spring Symposium, pages 33 40, Stanford, USA, March C. Mancas and E. Della Valle. Improving the art of semantic web application development - survey Available as S. McConnell. Rapid development-taming wild software schedules. Microsoft Press, E. Motta and M. Sabou. Next generation semantic web applications. In R. Mizoguchi, Z. Shi, and F. Giunchiglia, editors, ASWC, volume 4185 of Lecture Notes in Computer Science, pages Springer, G. Niro and E. Della Valle. Improving the art of semantic web application development - survey Available as M. C. Paulk, C. V. Weber, B. Curtis, and M. B. Chrissis. The capability maturity model : guidelines for improving the software process. Addison-Wesley, Boston, E. P. B. Simperl, I. Popov, and T. Buerger. Ontocom revisited: Towards accurate cost predictions for ontology development projects. In 6th Annual European Semantic Web Conference (ESWC2009), pages , June I. Sommerville. Software Engineering (8th Edition) (International Computer Science Series). Addison Wesley, May Y. Sure, S. Staab, and R. Studer. Methodology for development and employment of ontology based knowledge management applications. SIGMOD Record, 31(4):18 23, P. Tetlow, J. Z. Pan, D. Oberle, E. Wallace, M. Uschold, and E. Kendall. Ontology driven architectures and potential uses of the semantic web in systems and software engineering. W3C Working Draft Working Group Note 2006/02/11, W3C,

47 23. R. Zlade. Inventing the semantic web... again. Article on Semantic Universe, , Available as

48 Towards Ontology-driven Requirements Engineering Katja Siegemund 1, Edward J. Thomas 2, Yuting Zhao 2, Jeff Pan 2, and Uwe Assmann 1 1 Technische Universität Dresden, Germany 2 University of Aberdeen, UK Abstract. Requirements Engineering has a huge impact on success or failure of a software project. However, the acquisition, specification and evolution of goals and requirements from different stakeholders or sources often leads to incomplete, ambiguous, and faulty requirements. Therefore, the ability to detect and repair inconsistent and incomplete requirements is crucial to the successful modelling of requirements specifications. Ontologies provide the expressivity to sufficiently capture requirements and reasoning tasks can be performed to check the consistency and completeness of the requirements model. This paper presents a Meta Model for ontology-driven goal-oriented Requirements Engineering. Ontology consistency checking and rule driven completeness tests are combined to measure the validity and coverage of the evolving requirements model. Experiences from a first evaluation are described. Key words: Requirements Engineering, Ontologies, Reasoning 1 Introduction Usually, Requirements Engineering (RE) i.e., the identification, specification and documentation of requirements is the first task of a software development process. Results are documented in a Requirement Specification often treated as a contract between customer and developer. The importance of RE was already identified in several studies (e.g. [1]). Thus, an improved RE contributes to safer and better-quality software, safes time and money and also decreases the risk of overran budgets and project failures. An important challenge for requirements engineering is to cope with inconsistencies or incompleteness in requirements specifications. Such inconsistencies result from the acquisition, specification, and evolution of goals and requirements from multiple stakeholders and sources [2]. It is frequently the case that changes of requirements have a particularly significant impact on the consistency of specifications. In order to regain consistency, requirements are removed from the specification which often leads to incompleteness. Zowghi et. al. ([3]) describes this vicious circle as a causal relationship between consistency, completeness and correctness. From a formal point of view, correctness is usually meant to be the combination of consistency and completeness. Therefore, the

49 2 Towards Ontology-driven Requirements Engineering ability to detect and repair inconsistent and incomplete requirements is crucial to the successful development of requirements specifications and will lead to a more correct RSB. The paper is structured as follows: after some background information on RE in Section 2, we briefly sketch our solution approach in Section 3. In Section 4 we introduce Goal-Oriented Requirements Engineering and exemplify a way towards ontology-driven goal-oriented RE. Section 5 describes the implementation of the Requirements Ontology and the realisation of the completeness and consistency checks. First evaluation results are presented in Section 6. Finally, the paper concludes with an outlook and future work in Section 7. 2 Technical Background The following sections give a brief overview about the main problems of RE addressed in this approach. Completeness. Good requirements are meant to be consistent, necessary, unambiguous, and verifiable [4,5]. Additionally, they should be complete [6]. Missing or incomplete requirements propagate through the entire system development and lead to incomplete or incorrect designs, architectures, implementations and tests. Today, it is reasonably well known that missing or incomplete requirements lead to faulty software designs, implementations and tests resulting in software of improper quality or safety risks. Project cost and time overruns due to missing requirements and underestimated budgets may even cause a project to be aborted. Davis in [7] states completeness to be the most difficult of these (above mentioned) specification attributes to define and incompleteness the most difficult violation to detect. Complete metadata for requirements, that is data about that requirement rather than data listed in the requirement [6]), ensure completeness. Although some approaches exist that aim to ensure complete requirements (e.g. [8]), up to now there is no absolute way to determine the completeness of requirements in advance. It is reasonably well known, that requirement [specifications] will never be totally complete, finished or finalized as long as the system is in service and must evolve [6]. This may be a reason for today tools not to address this field. However, it is at least possible to improve and validate the completeness of information about these requirements (metadata). Though current RE tools provide means for capturing requirements, they fail in providing sufficient support for metadata about requirements and leave it to the requirements engineer to define them. Another shortcoming of RE tools is the lack of tests for completeness, that is, checking whether all important metadata are available. This way, the requirement engineer would detect missing but relevant information easily. Consistency. The vicious circle of completeness, consistency and correctness shows the importance of a consistent Requirement Specification in order to improve its correctness. Nuseibeh and Russo in [9] argue that detecting and re-

50 Ontology-driven Requirements Engineering 3 pairing of inconsistent requirements is crucial to a successful development of requirement specifications. Inconsistent specifications restrain deriving useful information. Following [9] and [3] the reasons for inconsistent requirements are changes of requirement during software development. Various approaches have been proposed to handle inconsistent goals and requirements during Requirements Engineering. Most of them use formal logics to detect and handle such inconsistencies. Yang et. al. separate consistency checking in ontology-based requirements elicitation methods into two steps: (1) verifying the domain knowledge for consistency and (2) checking whether the requirements model keeps consistent under the restriction of a consistent domain model [10]. Nuseibeh and Russo provide a formal technique for detecting and repairing inconsistencies, based on a specific type of reasoning, called abduction. If a particular consistency rule is violated, their proposed abductive reasoning technique identifies (evolutionary) changes to perform on the specification, such that a particular consistency rule is no longer violated [9]. Other approaches treat the detection of conflicts between requirements often only regarding refinement relationships or conceptual overlapping (e.g. [11]). Moreover, most techniques consider binary conflicts only, that is, conflicts among two requirements. Although these approaches and solutions exist, up to now hardly any RE tool makes use of them. Another problem are the numerous and often crosscutting 3 interrelationships among requirements that are not considered at all. Thus, there is no systematic support for detecting and resolving this kind of inconsistencies. One notable exception is [2]. To address these problems, our approach aims to detect such inconsistencies between various RE objects and provide solutions for repairing them. Deficiencies of RE Methods. Analysis of available RE Methods and approaches exposed a number of problems and shortcomings described above. Additionally, some have also been reported in other approaches. The deficiencies of current RE methods and tools can be summarized as follows: Requirement knowledge is not sufficiently covered. Intentions, risks, obstacles and decisions are not documented during RE and thus, are not available at later stages during software development. Relationships among requirements are inadequately captured and are often limited to binary relations between requirements instead of defining which kind of relation is meant (e.g. excluding, alternative, generalization). Requirement problems (e.g. conflicts, unstated information) are detected too late or not all. Completeness and consistency are not verified. Models for RE need richer and higher-level abstractions [13]. 3 Requirements are scattered through the whole system. The may appear in Use- Cases or Test-Cases and be a part of metrics. Additionally, crosscutting requirements provide a description of the overlap between requirements - the first step to managing any inconsistency that arises at such overlap [12].

51 4 Towards Ontology-driven Requirements Engineering 3 Solution Approach Ontologies provide a formal representation of knowledge and the relationships between concepts. Thus, they are well suited to tackle the above mentioned problems and shortcomings of Requirements Engineering. The Web Ontology Language in particular, which is standardised by W3C, supports the open world assumption, allowing incomplete knowledge to be reasoned over. This characteristic of the language makes it suitable for use in Requirements Engineering. The separation of the concepts of consistency and completeness mean that an evolving requirements model can be checked for consistency, without the incompleteness of the model causing a problem. In contrast, closed world systems such as relational databases make no distinction between incomplete and inconsistent knowledge; any fact not known is assumed to be false. The idea presented in this paper is to use an ontology for structuring the concepts, requirements and relationships captured during requirements elicitation. Here, goals play a crucial role for reaching a high level of completeness (see Section 4.1). This domain independent ontology can be instantiated with the requirement artefacts (goals, requirements, obstacles, etc.) for a particular project. This forms the Requirement Specification Base. TBox Reasoning techniques are applied for consistency checking. To validate the consistency, and subsequently the completeness of requirements we propose query answering techniques to detect incompleteness based on predefined completeness rules. The novel of this approach is twofold: (1) the Ontology-based Requirements Meta Model with a huge set of relevant meta data and requirement relationships and (2) the consistency and completeness rules for validating the Requirement Specification Base and providing concrete suggestions on how to solve causes for inconsistency and incompleteness. This approach allows for requirements reasoning based on formal semantics and thus, aims to resolve many of the shortcomings observed in other approaches. 4 Related Work This section describes the main concepts of goal-oriented RE approaches, related work and sketches the way to our solution towards ontology-driven goal-oriented RE. Conventional Requirements Engineering in its early times concentrated on what the system should do and how it should do it. This lead to fairly low-level requirements on data, operations, etc. [14]. Goals have become more and more popular for the early phases of Requirements Engieering (e.g. [15], [13], [16]). Yue showed in [17] that goals in requirements models provide a criterion for requirements completeness. Goal-oriented Requirements Engineering must not be understood as a different or new form of Requirements Engineering. It is rather a supplement of the RE process, in the meanwhile self widespread. Goal-driven concepts have been adopted in many Requirements Engineering frameworks in numerous domain

52 Ontology-driven Requirements Engineering 5 and stages of Requirements Engineering. This indicates that goals are a core concept for RE in general. Thus, we decided to adapt to this methodology and support not only a general RE method, but also a goal-oriented RE. The next section explains the main advantages and concepts of goal-oriented Requirements Engineering. Subsequently, we describe our approach to enable ontology-driven goal-oriented requirements engineering. 4.1 Goal-Oriented Requirements Engineering (GORE) As already manifested in its name, GORE puts much emphasize on goals which will be used to identify, describe and correlate requirements. Lamsweerde defines goals as declarative statements of intent to be achieved by the system under consideration [18]. Goals are formulated in terms of prescriptive assertions (as opposed to descriptive ones) [19]; they may refer to functional or non-functional properties and range from high-level concerns to lower-level ones [20]. Goals explore why certain requirements are necessary for the system to be. Thus, they capture stable information and provide means to separate stable from unstable information which enables a better reuse. There are several reasons to extend RE with the identification and formulation of goals. Lamsweerde describes in ([20]) the importance of goals for RE. Some of the main benefits of goal-orientation are: Goals provide a meaningful criterion for sufficient completeness of a requirement specification. A requirement specification can be denoted as complete with respect to a set of goals if all the goals can be proved to be achieved from the specification and the properties known about the domain considered. [17] Specification of pertinent requirements, that are requirements that serve at least one of the identified goals [17]. Goals may be satisfied by different alternative requirements. Thus, relationships between goals and requirements can help to choose the best one. Concrete requirements may change over time whereas goals pertain stable. Thus, stable information can be separated from volatile one. Goals drive the identification of requirements. 4.2 Enabling Ontology-Driven Requirements Engineering Knowledge-driven techniques seem promising to overcome some of the shortcomings of current RE-practices. Kossmann et. al. defines Knowledge-driven Requirements Engineering when Requirements Engineering is guided not only by a process but as well by knowledge about the process and the problem domain [21]. In order to use knowledge-driven techniques, it is necessary to apply knowledge repositories that can be easily updated and utilised. Furthermore, inferencing and decision support should be possible to apply on such a repository.

53 6 Towards Ontology-driven Requirements Engineering Ontologies are useful for representing and interrelating various knowledge. Since RE involves knowledge capturing and analysis, there is a clear synergy between the ontological modelling of a domain and the modelling that a Requirements Engineer will perform during the requirements process [22]. Due to this overlap, numerous works dating back have addressed the use of ontologies in RE, e.g. [23], [24]. All formalisms for RE need a particular conceptualisation, and almost all of them are reducible to first order logic [22]. Thus, they have much in common with ontologies that are constructed by using a formal language. Since the semantic web emerged, there has been a renewed interest in ontologies. There is an increasing amount of research devoted to utilising semantic web technologies in RE (e.g. [25], [21]) and software engineering in general. However, most of them concentrate only on specific requirement artefacts, e.g. goals, requirements or use-cases and do not support reasoning over relations between all concepts. Sancho et. al. ([26]) describe a method for ontology-aided performance engineering. They propose the structure of an ontological database to relate Performance Engineering issues among themselves and with other nonfunctional requirements. The NFR/i* Framework first proposed in [27] focuses on qualitative reasoning for goal contribution and evaluation of alternative goal refinements. The KAOS Framework ([28]) concentrates mainly on goal satisfaction. Lamsweerde describes in [29] an interesting method for reasoning about alternative requirements and inspired our work regarding some of the requirement relationships and ontology properties. Lamsweerde introduces influence relations among requirements, stating whether a requirement has a positive or negative influence on another. By applying formal reasoning techniques it becomes possible to identify alternative requirements for those with a negative impact. Other approaches aim to address a broader field of Requirements Engineering. The OntoRem approach ([21]) describes the development of a meta model for general RE concepts. Lee et. al. present an Ontology-based active Requirements Engineering Framework (Onto-ActRE) based on GORE [30]. While relationships of requirements with other concepts, such as goals and risks, are supported, they do not consider the variety of relations between requirements. 5 Ontology-Driven Requirements Engineering This section describes the main architecture decisions, concepts and relationships of the Requirements Ontology (the Requirements Meta Model), the rules for verifying the consistency and completeness of requirements metadata. 5.1 Requirements Ontology In Figure 5.1 we present the architecture of ODRE. A so-called Requirement Meta Model is generated from the knowledge of the requirements analysis and builds the TBox of the Requirements Ontology. It formalizes the RE concepts, as well as relationships between requirement artefacts. This domain independent

54 Ontology-driven Requirements Engineering 7 Requirement Engineering background knowledge Requirement Meta Model Requirements Requirements Requirements Specifica9ons Specifica9ons Specifica9ons of a project of a project of a project Tbox ABox Completeness checking rules Queries for checking completeness Consistency checking rules Queries for checking consistency Fig. 1. The architecture of ODRE (Ontology-Driven Requirement Engineering) ontology TBox can be instantiated with the requirement artefacts for a particular project. This way, the ABox is generated from the results of requirement analysis of an application project. It builds the requirement Specification Base. Once the Requirement Elicitation has been completed and the Requirements Ontology is filled with all information, we execute consistency and completeness queries, generated from consistency checking rules (cf. Section 5.2) and completeness checking rules (cf. Section 5.3), respectively. These rules are used to enable the validation of the Requirements Specification Base of the project. The following sections present the rules for completeness and consistency checking and exemplarily explain how they have been implemented so far. The Requirements Meta Model was mainly inspired by a UML Meta Model for GORE presented in [31]. Especially the requirement relationships have been adapted, comprehensively extended and transferred to ontological concepts. Furthermore, we adapted the categorization of non-functional requirements published by ISO/IEC 9126 standard to comply to this well-established and acknowledged categorization. In our approach, a huge number of GORE concepts are modelled as OWL Classes, while relations, relationships and most of the meta data are modelled as object properties. The Ontology consists of 102 classes, 39 object properties and 6 data properties, implemented with Protege 4.0. The ontology is available at The class Artifact is the most fundamental concept of this ontology. It subsumes the concepts Problem and RequirementArtifact. RequirementArtifact encapsulates the concepts Challenge, Goal, Story and Requirement. As already described in section 5 requirement artifacts have various relationships among each other. These relationships are mainly modelled as object prop-

55 8 Towards Ontology-driven Requirements Engineering Fig. 2. Extract of the main concepts in the Requirements Ontology erties. For example a requirement may negatively contribute to a goal, others positively. To capture this valuable information, we provide the object properties isnegativecontribution to and ispositivecontributionto between goals and requirements. This data is used for later queries and supports the user in recognizing the quality of his decisions regarding goal satisfaction. Functional and non-functional requirements are identified by means of Scenarios and Use Cases. Use Cases and Scenarios can be related to requirements via the object property isrelatedto. The class Attribute contains different attributes all requirements can be associated with, e.g. Priority and Cost. Additionally, this class has the subclass State. The requirement status will be automatically assigned after verification. Thus, the state can become Checked, Invalid, etc. We put much emphasis on the modelling of relationships between requirements. Therefore, we provide eight object properties to capture alternative, coexistent, excluding and conflicting requirements and to denote generalizations, specialisations and refinements of requirements. Decisions in the elicitation process may possibly derive in conflicts. These conflicts will have an impact on further decisions regarding requirements, Use Cases and the final design of the software. Thus, it is important to document such conflicts. This is realised by the OWL property isinconflictwith. Additionally, conflicts help in identifying inconsistencies between Project Work and Requirements. Finally, the class Refinement and RefinementReason as well as three associated object properties are used to track refinement changes (source and target) of goals and requirements. Furthermore, enabling the user to describe the reason for such a refinement eases further decisions and enables the traceability of them.

56 Ontology-driven Requirements Engineering Rules for Consistency Checking We base our notion of consistency and completeness on the violation of consistency and completeness rules. Therefore, we predefined two set of rules in natural language and realised the appropriate checks by querying techniques. We will have a brief look on the concrete realisation of these rules separately for consistency and completeness. The consistency rules check for valid relations between the requirement artifacts, e.g. whether a goal is indeed connected with at least one requirement. Moreover, requirement relationships are verified in order to prevent concurring requirement, conflicts, and so on. We predefined six consistency rules containing the rule that has to be met, the output if it is violated and possible solutions. To realise these rules in the requirements ontology, we have encoded them as a set of DL axioms which guarantee that if these rules are fulfilled, the ontology will be logically consistent. Any deviation from the rules will result in an inconsistent ontology. One such rule is modelled in as follows: ChosenRequirement isexclusionof.chosenrequirement This means that no requirement that was specified to exclude any other requirement can be included in one and the same requirement subset. Other consistency rules are specified in a similar way. The mapping between the axioms in the ontology and the rules is given in the ontology using annotation properties on the particular axioms. If one of these axioms is violated, we can use an explanation service to find which axioms caused this violation. The reasoner can determine the axioms in the inconsistency justification that represent a consistency rule. The additional axioms are used to find which individuals had caused the error. This allows us to explicitly mention the invalid requirements in the error message that is displayed to the user. For inconsistencies we distinguish errors and warnings. Errors must be resolved by the requirements engineer and warnings should be resolved. A template is used to define generic error messages for each consistency rule. For inconsistencies we distinguish errors and warnings. Errors must be resolved by the requirements engineer and warnings should be resolved. A template is used to define generic error messages for each consistency rule and suggestions to resolve found inconsistencies. Example We assume the following individuals and relationships as an extract of the ontology resulting from the Requirement Analysis: isexclusionof(f unctional Requirement5, F unctional Requirement7) ChosenRequirement(F unctional Requirement5) ChosenRequirement(F unctional Requirement7) This set of axioms results in an inconsistency with regard to the previously defined consistency rule. If one requirement excludes another, they cannot both be included in the set of requirements chosen for the final model.

57 10 Towards Ontology-driven Requirements Engineering We use the approach described above to derive errors and warnings displayed to the user. To support the requirements engineer in error elimination, the system suggests a list of possible solutions. The concrete error message and the suggestion to resolve our inconsistency is shown below: Excluding requirements must not be included in one chosen requirement subset. Error. The following requirements exclude others: Functional Requirement5. Please choose one of the following options: Suggestion. Exclude the following requirements from the chosen requirement set: Functional Requirement5 Find alternatives for: Functional Requirement5 or Revise the requirement relationships of (Functional Requirement5, Functional Requirement7). After the Requirements Engineer has done the appropriate actions to remove errors and warnings, the requirement subset has to be checked for consistency once more. In case the set is now consistent, all requirements will be automatically asserted Checked and Valid in the Ontology. If not, another revision and check becomes necessary until the the Requirement Specification Base is consistent or the analyst refuses to do any further action. In the latter case, all the requirements will be asserted Checked as well, but all invalid requirements will be asserted Invalid. This way it is possible to display all invalid requirements and change them manually or check for options without being forced to run too many consistency checks. However, it is always possible to have another consistency check. 5.3 Rules for Completeness Checking We defined 43 Completeness Rules to check the completeness of the requirement artifacts, e.g. whether goals, requirements, use-case descriptions and so on have been specified and if all required attributes and relations have been specified. According to Firesmith ([6]) the most important metadata to capture is: Project-Unique Identifier (PUID) Prioritization Rationale Source Status (may include more than one kind of status) Verification Method These metadata were integrated in our completeness rule set and extended by further completeness criteria regarding the existence of requirement artifacts and their relations among each other. An extract of these rules is shown below:

58 Ontology-driven Requirements Engineering 11 Every Functional Requirement (FR) must define whether it is mandatory or optional. IF FR is not mandatory AND not optional THEN print error: You did not specify whether the following FRs are mandatory or optional Please specify whether the following FRs are mandatory or optional: [FR n]. Every FR must specify AT LEAST ONE property hasrequirementrelationship. IF FR has NOT specified a property hasrequirementrelationship THEN print error: You did not specify any requirement relationship for the Functional Requirements [FR n]. Please check the relationships for the following Functional Requirements: [FR n]. Example We assume an extract of the following individuals and relationships: isrelatedt o(goal2, U secase7) N onf unctionalrequirement(n onf unctionalrequirement1) isoptional(n onf unctionalrequirement1, true) F unctionalrequirement(f unctionalrequirement1) This raises the following error messages and suggestions for our completeness rules: Every Requirement must state whether it is mandatory or optional. Error. You did not specify whether the following FR are mandatory or optional: FunctionalRequirement1. Please specify this attribute for the FR: FunctionalRequirement1. Every FR must specify AT LEAST ONE requirement relationship. Warning. You did not specify any requirement relationship for the following FR: FunctionalRequirement1. Please check whether there exists any relationship to another requirement for the FR FunctionalRequirement1. The completeness rules have been realised as SPARQL queries. If the patterns encoded in the rule match the entailed ontology, then the particular completeness rule being tested failed, and the requirements model is deemed incomplete. The results of the query identify those parts of the ontology which are incomplete. Since we are looking for incomplete requirements, we had to allow the query to work in a closed world environment. This requires the use of negation as failure in the query, which is available in the draft SPARQL 1.1 specification using the NOT EXISTS keyword and functionality. Generally, OWL uses the open world assumption (OWA), which states that every fact not known is undefined,

59 12 Towards Ontology-driven Requirements Engineering therefore it is difficult to find incomplete requirements, as these requirements are treated as present but unknown by an OWL reasoner. The SPARQL 1.1 specification supports querying under OWL, and it supports negation in queries, which is treated as negation as failure by the SPARQL specification. In this case, we used the SPARQL 1.1 support in the TrOWL reasoner (REF) to support our implementation. The implementation of these database style constraints is similar to that described by Sirin and Tao in [32]. Therefore, we use a combination of open world and closed world assumption. OWA allows us to check the logical consistency of incomplete requirements without being forced to have a complete set of metadata. Closed world reasoning using negation as failure is employed when the Requirements Engineer decides to check completeness during RE or he decides to complete RE and finalises the requirements. Verification of the Requirement Specification Base We implemented the ontology using Protege 4.0, and created the completeness constraints as queries in SPARQL, using the SPARQL 1.1 OWL profile and support for negation as failure against OWL knowledge bases. The reasoner and query engine used is the TrOWL tractable reasoner 4. This is particularly well suited to this application, as once a consistent and complete set of requirements is loaded, it is possible to make a SPARQL endpoint available permanently to allow traceability links to be established between the artifacts in the requirements ontology and the rest of the software engineering life cycle. The TrOWL reasoner also supports explanation of consistency checks for OWL-DL ontologies, and supports justification of query answers. The additional functionality for rewriting the reasoning explanations and justifications into english has been implemented into the RELib package. 6 Evaluation ODRE has primarily been evaluated within the MOST Project 5. The evaluation criterion for the discussed ontology are that (i) all requirement artefacts should be possible to capture, (ii) all inconsistencies and incomplete metadata are detected and (iii) all requirement metadata are finally complete and consistent. During development, we have simulated its usage by instantiating it against few case studies of the project, i.e. semantic modeling of network physical devices ([33]) and validating Component-Based Implementations of Business Processes ([34]). After instantiation, we evaluated our implementation by performing the completeness and consistency checks over this ontology. The evaluation was performed on a 2GHz dual core MacBook Pro. The evaluation was an excellent test of the ability of our approach to deal with incomplete and inconsistent specifications. The ontology was then stored in the Quill OWL2-QL repository using 4 Available at 5 Marrying Ontologies with Software Technology,

60 Ontology-driven Requirements Engineering 13 Semantic Approximation [35] and the completeness checks were performed. The Case Study ontology failed 14 of the 37 tests included in the RELib test suite, and a total of 132 different individual problems were identified. Thus, the evaluation proved the detection of incomplete and inconsistent requirement artefacts, which was one of our main goals. The total time required to test the completeness constraints and generate the English explanations from the justifications provided by the reasoner was 808ms. The performance of the system is therefore good enough to be used as part of an ongoing cycle of testing and revision of the ontology. However, in order to prove the applicability of our Ontology-driven Requirements Engineering method and the impact of reasoning over requirement data, another and exceeded evaluation will be conducted in the near future. In this evaluation we will not only test the quality of the developed requirement artefacts, but also the effort and time needed for specification until a certain predefined level of completeness and consistency is reached. 7 Outlook and Conclusion In this paper we have presented an approach to capturing and validating a set of software project requirements using OWL ontologies and reasoning technology. The approach combines the open world reasoning capabilities of OWL to allow the consistency of incomplete requirements to be validated during the requirements engineering process, and extends OWL with closed world constraints to further check the completeness of the requirements model against several wellestablished metrics such as ISO/IEC We have implemented our approach and have performed an in-depth evaluation using an extant set of project requirements. This evaluation has shown that the approach is capable of dealing with a reasonably complex set of requirements from a real-world problem, and can quickly identify where these requirements are inconsistent or incomplete. The performance is such that it can be integrated into the requirements engineering workflow without becoming a burden on the requirements engineer. Further work in this area concentrates on guidance of the requirements engineering process and traceability through other stages of the software development cycle. We developed a guidance ontology for our approach which is connected to the Requirements Ontology. This enables user guidance through the prescribed process of requirements engineering. This guidance includes not only a task list for the user but also incorporates the described consistency and completeness checks at the appropriate time as well as real-time display of detected errors and suggestions for resolving them. All these features have been integrated into a Eclipse Plugin and can be accessed through a yet rudimentary java user interface. The completed requirements ontology can be integrated with other ontological models which cover other aspects of the software development process, links between the artefacts in the requirement ontology and those artefacts generated at later stages can therefore be queried later for traceability purposes.

61 14 Towards Ontology-driven Requirements Engineering References 1. Tracy Hall, Sarah Beecham, and Austen Rainer. Requirements Problems in Twelve Software Companies: An Empirical Analysis. IEE Proceedings - Software, 149(5): , Axel van Lamsweerde, Robert Darimont, and Emmanuel Letier. Managing conflicts in goal-driven requirements engineering. IEEE Transactions on Software Engineering, 24: , Didar Zowghi and Vincenzo Gervasi. The Three Cs of Requirements: Consistency, Completeness, and Correctness. In Proceedings of 8th International Workshop on Requirements Engineering: Foundation for Software Quality, (REFSQ 02), Donald Firesmith. Specifying Good Requirements. Journal of Object Technology, 2:77 87, Ian Sommerville and Pete Sawyer. Requirements Engineering: A Good Practices Guide. John Wiley & Sons, Donald Firesmith. Are Your Requirements Complete? Journal of Object Technology, 4(1):27 44, Alan M. Davis. Software Requirements: Analysis and Specification. Prentice Hall Press, Upper Saddle River, NJ, USA, 2nd edition edition, Ronald S. Carson. Requirements Completeness: A Deterministic Approach, Bashar Nuseibeh and Alessandra Russo. Using Abduction to Evolve Inconsistent Requirements Specifications. In The Use of Logical Abduction in Software Engineering 25, Yang Ying-ying, Li Zong-yon, and Wang Zhi-xue. Domain knowledge consistency checking for ontology-based requirement engineering. In CSSE 08: Proceedings of the 2008 International Conference on Computer Science and Software Engineering, pages , Washington, DC, USA, IEEE Computer Society. 11. Xuefeng Zhu. Inconsistency Measurement of Software Requirements Specifications: An Ontology-Based Approach. In ICECCS 05: Proceedings of the 10th IEEE International Conference on Engineering of Complex Computer Systems, pages , Washington, DC, USA, IEEE Computer Society. 12. Bashar Nuseibeh. Crosscutting Requirements. In AOSD 04: Proceedings of the 3rd international conference on Aspect-oriented software development, pages 3 4, New York, NY, USA, ACM. 13. John Mylopoulos, Lawrence Chung, and Eric Yu. From Object-oriented to Goaloriented Requirements Analysis. Communications of the ACM, 42(1):31 37, Alexei Lapouchnian. Goal-oriented Requirements Engineering: An Overview of the Current Research, Annie I. Antón. Goal-Based Requirements Analysis. In ICRE 96: Proceedings of the 2nd International Conference on Requirements Engineering (ICRE 96), page 136, Washington, DC, USA, IEEE Computer Society. 16. L. Liu and E. Yu. From requirements to architectural design - using goals and scenarios, K. Yue. What does it mean to say that a specification is complete? Proc. IWSSD-4, Fourth International Workshop on Software Specification and Design, Axel van Lamsweerde. Requirements Engineering in the year 00: A Research Perspective. In International Conference on Software Engineering, pages 5 19, Pamela Zave and Michael Jackson. Four dark corners of requirements engineering. ACM Transactions on Software Engineering and Methodology, 6(1):1 30, 1997.

62 Ontology-driven Requirements Engineering Axel van Lamsweerde. Goal-Oriented Requirements Engineering: A Guided Tour. In RE 01: Proceedings of the Fifth IEEE International Symposium on Requirements Engineering, page 249, Washington, DC, USA, M. Kossmann, R. Wong, M. Odeh, and A. Gillies. Ontology-driven Requirements Engineering: Building the OntoREM Meta Model. In Information and Communication Technologies: From Theory to Applications, ICTTA rd International Conference on, pages 1 6, Glen Dobson and Peter Sawyer. Revisiting ontology-based requirements engineering in the age of the semantic web. In Dependable Requirements Engineering of Computerised Systems at NPPs, John Mylopoulos, Alex Borgida, Matthias Jarke, and Manolis Koubarakis. Telos: Representing Knowledge About Information Systems. ACM Trans. Inf. Syst., 8: , Anne Dardenne, Axel van Lamsweerde, and Stephen Fickas. Goal-directed Requirements Acquisition. Sci. Comput. Program., 20:3 50, Haruhiko Kaiya and Motoshi Saeki. Ontology based requirements analysis: lightweight semantic processing approach. In Proc. Fifth International Conference on Quality Software (QSIC 2005), Pere P. Sancho, Carlos Juiz, Ramon Puigjaner, Lawrence Chung, and Nary Subramanian. An approach to ontology-aided performance engineering through NFR framework. In WOSP 07: Proceedings of the 6th international workshop on Software and performance, pages , New York, NY, USA, ACM Press. 27. J. Mylopoulos, L. Chung, and B. Nixon. Representing and using nonfunctional requirements: A process-oriented approach. IEEE Trans. Softw. Eng, 18(6): , R. Darimont, E. Delor, P. Massonet, and A. van Lamsweerde. Grail/kaos: an environment for goal-driven requirements engineering. In Proceedings of the 19th international conference on Software engineering, ICSE 97, pages , New York, NY, USA, ACM. 29. Axel van Lamsweerde. Reasoning about alternative requirements options. In Alexander Borgida, Vinay K. Chaudhri, Paolo Giorgini, and Eric S. K. Yu, editors, Conceptual Modeling: Foundations and Applications, volume 5600 of Lecture Notes in Computer Science, pages Springer, Seok Won Lee and Robin A. Gandhi. Ontology-based Active Requirements Engineering Framework Rodrigo Cerón, Juan C. Dueñas, Enrique Serrano, and Rafael Capilla. A metamodel for requirements engineering in system family context for software process improvement using cmmi. In PROFES, pages , Evren Sirin and Jiao Tao. Towards integrity constraints in owl. In Rinke Hoekstra and Peter F. Patel-Schneider, editors, OWLED, volume 529 of CEUR Workshop Proceedings. CEUR-WS.org, Krzysztof Miksa, Marek Kasztelnik, Pawel Sabina, and Tobias Walter. Towards semantic modeling of network physical devices. In MoDELS Workshops, volume 6002 of Lecture Notes in Computer Science, pages , Jens Lemcke, Andreas Friesen, and Tirdad Rahmani. Validating component-based implementations of business processes. In Electronic Business Interoperability: Concepts, Opportunities and Challenges, chapter 7, pages IGI Global, J. Z. Pan and E. Thomas. Approximating OWL-DL Ontologies. In the Proc. of the 22nd National Conference on Artificial Intelligence (AAAI-07), pages , 2007.

63 Identifying ontology design styles with metrics Eleni Mikroyannidi, Robert Stevens and Alan Rector The University of Manchester Oxford Road, Manchester, M13 9PL Abstract. This paper presents a metrics framework for identifying highlevel ontology composition styles as an abstraction layer for ontology comprehension. The metrics are implemented on top of the OWL API metrics, which have been extended to capture various aspects of an ontology. The metrics are applied to an ontology repository, and using clustering methods underlying design styles have been revealed. We seek to give a high-level view of how an ontology has been put together through the identification of authoring style using a range of ontology metrics. This ontological style can be considered as a metamodel, giving an intuition of what lies under the hood of an ontology. This can be useful in the broader area of ontology comprehension. The results highlighted five design styles covered by the repository describing simple, as well as rich and complex ontologies. 1 Introduction We present the use of data-mining across a broad range of ontology metrics to describe the ontological design style of how a particular ontology has been authored in terms of its size, shape, number and disposition of axioms. Our aim is to use this guide to ontological style as part of a framework to facilitate ontology comprehension. OWL ontologies are often considered to be complex entities that require an excessive amount of time and effort to be used in conjunction with other ontologies or to be integrated into an application [16]. One important task in authoring, using and re-using an ontology is its comprehension [6,12]; that is, understanding the ontology at a range of levels. The process of understanding ontologies can be compared to the process of understanding code, its structure, organisation etc in software engineering. The OWL language can be compared with programming languages, as it represents the ontology as a series of statements or axioms. To comprehend an ontology, a reader has to take these statements and synthesise an understanding of how the ontology has been put together and how it works; without such a comprehension, extension and maintenance are difficult. On a second level, the user has to identify those parts of the ontology that have been modelled with the same combination or pattern of constructors and entities, together with their interconnection with the other parts of the ontology. This already difficult task is made harder still as ontologies that are modelled with logic axioms can be

64 difficult to understand [13]. In addition, ontologies that make use of advanced characteristics of OWL can lead to inferences that add complexity in their structure. As a consequence, a user can be faced with a logical artefact that is difficult to comprehend at many levels: The style of the modelling; the size and shape of the whole ontology; the patterns of axioms used; the entailments afforded by the ontology; and so on. Metamodels can be helpful towards the comprehension of an ontology, as they provide abstraction layers that hide the complexity of the ontology [5]. One such high-level abstraction is the ontological design style employed in a given ontology. By ontological design style we mean, very informally, how an ontology s axioms have been put together. More precisely, ontology style is the summary of the main characteristics of composition and structure of an ontology. The design style can be assessed by inspecting the ontology,; however, metrics are helpful for giving an overview of an ontology s design style. The workflow for the generation of the ontology design styles is presented in Figure 1. In this paper, we use a range of metrics measuring various aspects of an ontology, such as the size, the complexity of the structures in the ontology, the axiom types etc. to provide a series of dimensions that characterise the design style. These dimensions constitute a rating scheme for describing the design style from a high level of abstraction. We then use clustering methods to group ontologies in a repository according to these various dimensions. Intuitively, we expect to divide the repository into groups such as small and complex, large and simple etc. Indeed, the results showed five main design styles covered by the repository, describing ontologies, which for example are small but rich and complex; ontologies in which most information is expressed in the taxonomy; ontologies in which information is expressed with rich object property hierarchies and class expressions; etc. These proposed design styles should help to reveal the main characteristics of the ontologies. We believe that the detection and description of design styles can be helpful for comprehending an ontology, as it can give an intuition of its composition and an initial guide on how to inspect the ontology. For example, if there is a complex object property hierarchy and if all the entailments refer to property assertions, the user can start by inspecting these elements in the ontology. In general, these design styles could be used as the initial information that guides the choice of the comprehension tactic for an ontology. 2 Related work Existing work on metrics is related to the assessment of the ontology quality in the scope of ontology integration and ontology modularisation [11]. Recent work shows that metrics are used more for ontology analysis than for ontology comprehension by ontology authors or other users. OntoQA [15] is a metrics framework for assessing the quality in terms of reusing an ontology. Tartir et al. [15] define metrics for the schema inheritance tree and its instances to evaluate the success of it in terms of modelling a real-

65 OWL API Metrics Cohesion Metrics Selection and definition Normalisation Categorisation Processed Metric set generates Other metrics KMeans Clustering for defining the classes of the rating scheme Unprocessed Data TONES repository applied to generates Rating scheme Data described according to the rating scheme revealing Ontology Design Styles EM Clustering refinement of Fig. 1. Workflow for generating the ontology design styles world domain. Although the described metrics are indications of the schema and the population of the ontology, they do not explicitly refer to the asserted or inferred class graph. both the asserted and inferred structure of an ontology need to be understood by an ontology s users. In addition, the schema metrics focus only on the class hierarchy. Object and data property hierarchy structures can also exist in the ontology and should also be considered, as they can cause additional implications upon use of an automated reasoner. Yao et al. [18] describe cohesion metrics to measure the modular relatedness of OWL ontologies. The cohesion refers to the degree to which the elements in a module belong together. Strong cohesion is recognised as a desirable property of object-oriented classes, as it measures separation of responsibilities, independence of components and control of complexity. The authors focus only on the class hierarchy of the ontology, without explicitly defining the constraints that should be considered for the computation of the metrics. Alternative representations can derive from the same subsumptions that could alter the measures. Such an example is shown in Figure 2. The subsumption axioms D C, C B, B A can be represented differently depending on the consideration of direct and indirect inheritance. Therefore, the consideration of additional parameters are required to ensure the stability of the metrics [17]. In another direction is the use of statistical analysis to assess the design of an ontology from a high level. Ontology editors such as Protégé-4 and Swoop

66 A A is-a is-a B B is-a is-a is-a is-a C C is-a D is-a is-a D (a) (b) Fig. 2. Different representation of the hierarchy derived from the same subsumption axioms. provide simple metrics for an ontology. Protégé-4 has an introductory view pane that presents metrics that can give an indication of the size and some of the characteristics of the ontology, while Swoop provides some more details on the taxonomy and property hierarchy of the loaded ontology. Even though there are helpful tools that provide general statistics for ontologies, there is a lack of systematic and comprehensive metrics for characterising ontologies in terms of design. These could be further used as a metamodel that presents the main characteristics of the ontology. Work close to metrics and metamodels is presented in [5,4]. However, the structural metrics defined by the authors have the same instability in terms of their computation as the aforementioned frameworks. Efforts for describing ontologies and their complexity using metrics are reported in [9,10]. In [9] the authors use the OWL API metrics and additional ones on the entailments of the ontologies to describe a biomedical repository. The work presented here is also based on the OWL API metrics. However, we use additional machine learning techniques to pinpoint underlying design styles of an ontology; it is this ability to give a high-level overview of what kind of ontology style has been used in an ontology that is the contribution of this paper. 3 Ontology metrics framework The main question for analysing ontologies with metrics for design style is which metrics are appropriate to characterise an ontology and give an indication of the design style and complexity of the ontology? This cannot be answered easily, as there can be multiple metrics that can describe a particular aspect of an ontology, such as the taxonomy, and all of them might be equally informative as to design style.

67 Class usage ObjectProperty usage DataProperty usage Individual usage Class hierarchy wideness ObjectProperty hierarchy wideness DataProperty hierarchy wideness Class hierarchy complexity ObjectProperty hierarchy complexity DataProperty hierarchy complexity ClassExpression complexity Usage Avg. entity usage (assertedinferred) Avg. OWLClassExpressions per class (asserted-inferred) Size Number of axioms Number of entities Ontology aspects Structures Avg. subentities per entity (Class, ObjectProperty, DataProperty) (asserted-inferred) Avg. superentities per entity(class, ObjectProperty, DataProperty) (asserted-inferred) Avg. hierarchy depth (Class, ObjectProperty, DataProperty) (asserted-inferred) Avg. superentities per entity (Class, ObjectProperty, DataProperty) (asserted-inferred) Avg. hierarchy depth (Class, ObjectProperty, DataProperty) (asserted-inferred) Avg. OWLClassExpression depth Entailment richness Entailments and explanations Entailments per axioms Avg Justifications per entailment Justification richness OWLClassExpression usage Axiom types Avg defined classes per class Avg. OWLClassExpressions per class (asserted-inferred) Avg. DisjointClassAxioms per class Avg. DisjointUnion axioms per class Avg. OWLObjectPropertyRange axioms per ObjectProperty Avg. TransitiveObjectProperty axioms per ObjectProperty Avg. InverseFunctionalObjectProperty axioms per ObjectProperty Avg. ReflexiveObjectProperty axioms per ObjectProperty Avg. DisjointObjectProperties axioms per ObjectProperty Avg. AsymmetricObjectProperty axioms per ObjectProperty Avg. EquivalentObjectProperties axioms per ObjectProperty Avg. NegativeObjectPropertyAssertion axioms per ObjectProperty Avg. InverseObjectProperties axioms per ObjectProperty Avg. FunctionalObjectProperty axioms per ObjectProperty Avg. IrrefexiveObjectProperty axioms per ObjectProperty Avg. SubPropertyChainOf axioms per ObjectProperty Avg. SymmetricObjectProperty axioms per ObjectProperty Avg. EquivalentDataProperties axioms per DataProperty Avg. DisjointDataProperties axioms per DataProperty Avg. FunctionalDataProperty axioms per DataProperty Avg. DatatypeDefinition axioms per DataProperty Avg. NegativeDataPropertyAssertion axioms per DataProperty Avg. Types per individual Avg. OWLObjectPropertyAssertions axioms per individual Avg. OWLDataPropertyAssertions axioms per individual Avg. SameIndividual axioms per individual Avg. SubAnnotationPropertyOf axioms per axiom Avg. AnnotationAssertion axioms per axiom Avg. AnnotationPropertyDomain axioms per axiom Avg. AnnotationPropertyRangeOf axioms per axiom Class feature richness ObjectProperty feature richness ObjectProperty feature richness Individual feature richness Annotations feature richness Fig. 3. The metrics framework: Ontology aspects, metrics and dimensions. The full list of metrics and their descriptions covered by this framework can be found online 1. In Figure 3, the core of the metrics framework is presented. The figure shows five main aspects of an ontology that are considered by this framework. These are: 1. Size 2. Axiom types 3. Structures 4. Entity usage 5. Entailments and explanations. 1

68 As can be seen in Figure 3, these aspects are further analysed into 20 dimensions and each one is represented by a set of metrics (the dimensions are mapped with dashed arrows to the corresponding set of metrics). The size aspect is described by two metrics for characterising the ontology in terms of size. The Axiom type aspect is represented by five dimensions and sets of metrics, which try to assess the variety and type of OWL constructs and features that are used to develop the ontology. The structures we consider in this framework are the class, object and data property hierarchies and the nesting of class expressions. A corresponding set of metrics is used to assess the complexity of these structures. The entity usage metrics give a higher granularity of information related to the axiom type metrics, as they show how many entities of the ontology are used over and above their declaration. Finally, a set of metrics is related to the implications of the ontology. Theoretically, the inferences in an ontology are infinitely many. However, from a practical point of view, an ontology editor shows a finite number of entailments. In the proposed metric framework, we extract a finite subset of and provide metrics for capturing the underlined semantics. 3.1 OWL API metrics This work uses as a basis the metrics that have been implemented in the OWL API. All the additional metrics have been developed using the OWL API and have been implemented on top of the existing metrics package, by using and extending the same interfaces. The OWL API has already a set of 46 implemented metrics. These include all the axiom types in the ontology, the expressivity and import closure. We selected a subset of 28 metrics, and some of these were modified and computed for the inferred ontology, in order to take into account the implications of the ontology. These mainly refer to axiom types and are shown in Figure Structural metrics The purpose of the metrics related with the hierarchies (class, object property, data property) is to give an intuition of their shape, meaning its wideness, nesting and complexity. A nested class hierarchy might introduce a complexity in the ontology, but it also represents the level of organisations of the entities. This set of metrics is computed for both the asserted and inferred hierarchies. Changes between the asserted and inferred values reflect the changes in the hierarchy before and after the classification of the ontology. The type of entailments that are taken into account are direct atomic subsumptions. The interface of the reasoner in the OWL API handles hierarchies similarly to the model presented in [1]. We compute all the structural metrics using the reasoner. For the asserted ontology the OWL API provides the StructuralReasoner, which works with the told information only. The reasoner interface of the OWL API handles hierarchies as directed acyclic graphs (DAG), containing nodes connected via edges. Each node in the hierarchy represents a set of entities that are equivalent to each other. The hierarchy has the top node (in OWL is owl:thing,

69 owl:topobjectproperty, owl:topdataproperty) and the bottom node (in OWL is owl:nothing, owl:bottomobjectproperty, owl:bottomdataproperty). Figure 4(a) shows an example of a class hierarchy; each box in the hierarchy represents a Node. Class K is unsatisfiable, so it is equivalent to owl:nothing, and therefore appears in the bottom node. In addition, with this approach any cycles are excluded by the reasoner, as this would lead to an infinite number of inheritance levels in the ontology. For the computation of the structural metrics, a number of adjustments have been made to the representation of Figure 4(b). These are: Each node in the hierarchy is a single entity. Therefore, equivalent entities are represented as sibling nodes. Such a representation is also adopted by ontology editors. We adopt this kind of representation for having a standard way for computing metrics such as the wideness of the hierarchy. A Leaf node is considered as a node without any direct sub nodes, minus the bottom node. Therefore, for the computation of the metrics, the bottom node is excluded as it contains all the unsatisfiable classes, which are captured by a separate metric. owl:thing DirectSubClassOf DirectSubClassOf A B DirectSubClassOf DirectSubClassOf DirectSubClassOf DirectSubClassOf DirectSubClassOf DirectSubClassOf C D F DirectSubClassOf E (a) Example class hierarchy as it is (b) Example class hierarchy as it is represented by the structural metrics. handled by the reasoner in the OWL API. Fig. 4. Representation of the hierarchy by the reasoner and the structural metrics. Figure 4(b) shows how the hierarchy of Figure 4(a) is formulated according to the previous adjustments. It should be also mentioned that for the computation

70 of the structural metrics referring to superclasses and subclasses, only direct superclasses and subclasses are taken into account. Finally, for assessing the complexity of OWLClassExpressions, we use the average OWLClassExpression nesting. 3.3 Entity usage metrics This set of metrics gives an insight into the interconnectivity between entities in the ontology. For the computation of these metrics, we exclude usage that refers to declaration and subsumption axioms. The subsumption axioms are excluded as they are assessed with the structural metrics. We consider and measure the usage for the classes, object properties, data properties, individuals and class expressions in the ontology. The OWLClassExpressions usage metric tries to assess the number of universal and existential restrictions that enrich the description of classes. The inferred metric is computed by extracting for each class structural information from the class hierarchy. In the OWL API, the reasoner returns entailments referring to atomic subsumptions. Based on this information and on the inheritance that derives from the class hierarchy, we compute the usage of OWLClassExpressions. 3.4 Entailment metrics There are different metrics that are used to assess the inferences in the ontology. A set of metrics computes the entailments referring to relations between atoms. Authors in [2] describe parameters that need to be considered when extracting a finite set of entailments from an ontology. Although reasoners share the same interface in the OWL API, they can handle entailments differently and return subsets or supersets of the desired ones. Therefore, we need to set all the required parameters in order to extract and describe them precisely. For the computation of entailment metrics, we consider entailments that cover relations between atoms of the following types: Equivalent classes and object properties. Non-trivial direct subsumptions between entities (classes and object properties). Class and object property assertions. Based on the definition of finite sets of entailments, described in [2], for the computation of direct subsumptions between classes, we select the inferred transitive closure, with the exclusion of asserted of entailments. The main normalised metric that is used to assess entailments is the ratio of entailments to the total number of axioms. It should be noted that there are different metrics that also consider the entailments as they are computed for both the asserted and inferred ontology. However, they are used to highlight different aspects of the ontology. For example, for the computation of the entailment metrics, the asserted information of

71 the ontology is excluded. That is because these metrics are used to measure only the size and type of entailments in the ontology. On the other hand, the structural metrics are computed for both the asserted and inferred class hierarchy. These metrics take into account also the asserted information in the ontology and aims to show how the entailments affect the shape of the hierarchy (nesting, complexity and wideness). In order to assess the complexity of the entailments, we define the Justification richness, which is the average number of justifications per entailment. Justifications and their usage are described in [7,8]. 3.5 Design style dimensions We define 20 design style dimensions, which organise the metrics into groups for assessing the aspects of an ontology. These are presented in Figure 3. They comprise the rating scheme that is used to give the metamodel describing the design style of an ontology. Apart from the metrics that are used to assess the size of the ontology, the rest of them are normalised. We deliberately use normalised metrics as these are more convenient for the comparison of two ontologies, which might vary in size, but are similar in terms of richness and complexity. We selected these dimensions to pinpoint the main aspects of an ontology that normally a user will deal with on the initial inspection; These can be the size, the shape of the hierarchies, the entailments of the ontology etc. On the selected metrics for representing each dimension we applied feature selection algorithms in order to exclude from the set redundant metrics. For the final induction of a rating scheme for describing the design style of an ontology from a high level, we further normalise the metrics to have a single normalised metric representing each dimension. For example the class feature richness metric is defined as class f eature richness = CCi CC n, where CC i represents the normalised metrics of the class characteristics and in this case are the Avg. defined classes per class, Avg. OWLClassExpressions per class, Avg. DisjointClass axioms per class, Avg. DisjointUnion axioms per class and CC n is the total number of characteristics, which in this case is equal to 4. Details about the process that is followed to conclude the rating scheme are presented in the next section. 4 Results The TONES repository was selected for testing the metrics as it appears to include a variety of different ontologies with different design styles. The metric framework was applied to 112 ontologies from TONES 2 repository, which were classified in less than 50 minutes. For the classification of the ontologies we use the Pellet 3 reasoner. The results are further processed for the induction of the

72 ontology design styles that are covered in the repository. For this process we use clustering methods to define a rating scheme that divides the set of ontologies into different groups. All the results can be found in detail online 4. In this section we focus on the process that was followed, and present the main outcomes. After the application of the metrics in the repository, the next step is to separate the ontologies into groups according to their similar description. The results of the repository are expressed with normalised metrics. For the conclusion of the rating scheme we define three nominal values (classes) to describe each dimension in the rating scheme. The set of classes is S = {Small, Medium, High} characterising the ontology in terms of richness and complexity for every dimension. The reason for defining these classes is to make more practical use of metrics and how to interpret a value of a metric. We set the range of these classes into the corresponding values by applying the KMeans [14] clustering algorithm on the data and setting the parameter of clusters equal to 3. Based on the output from clustering, the normalised value of each dimension is mapped to the corresponding class. The results appear to show an intuitive description of the ontologies. An example is shown in Figure 5. The cell ontology 5 [3] is a medium size ontology, with many annotations, but few entailments, although with many justifications for those entailments. There is much information that is expressed in the class hierarchy, which is wide and complex. There is a medium usage of the class expressions and object properties, which do not have many characteristics. Such a description summarises the main characteristics of a single ontology. However, we do not yet know which design styles are covered by the repository and which ontologies have similar descriptions. High level description of the cell ontology: Size : Medium Class hierarchy wideness : High Class hierarchy complexity : Medium ObjectP roperty hierarchy wideness : Small ObjectP roperty hierarchy compexity : Medium DataP roperty hierarchy wideness : Small DataP roperty hierarchy complexity : Small Class usage : Small Class expression usage : Medium Class expression complexity : Small ObjectP roperty usage : Medium DataP roperty usage : Small Individual usage : Small Class feature richness : Small ObjectP roperty feature richness : Small DataP roperty feature richness : Small Individual feature richness : Small Annotation feature richness : High Entailment richness : Small Justification richness : Medium Fig. 5. Description of the Cell ontology based on the rating scheme //obo.cvs.sourceforge.net/*checkout*/obo/obo/ontology/anatomy/cell_ type/cell.obo

73 For further analysis of the repository, we performed additional clustering on the results generated from the TONES repository. The dimensions of the rating scheme give a high level metamodel of the ontology. The additional clustering is done using only the 20 dimensions of the rating scheme, which now are not separated into groups but are tested all together. The reason is to highlight the most important design styles that exist in the repository. These clusters use the combination of the metrics and can reveal possible similarities in design style between the ontologies. In this case we do not know how many design styles exist in the repository, thus we choose the EM [14] clustering algorithm, because it can also predict the number of clusters. For the generation of these clusters, we represent the set of classes (S = {Small, Medium, High}) in the dimensions with the values S = {1, 2, 3}. We make this transformation because the EM algorithm does not work with nominals. In addition, we do not use the initial normalised values for this clustering as it would be more difficult to cluster values that are relatively close to each other. We perform clustering with the absolute values because they are better separated. The clustering algorithm separates all the data into 5 clusters, the description for which is given in Table 1. In this way, the repository is analysed into five dominant design styles. Cluster 0 is of small to medium sized ontologies with most of the information expressed in the class hierarchy, which is complex and wide. There are not many class expressions, complex property hierarchies, data properties or individuals. The inferential richness is small to medium while the justification richness is medium to high. In addition, this cluster includes the ontologies with many annotations and annotation features. This cluster includes many bio-ontologies in OBO, such as the fly-taxonomy 6 and the environment 7 ontology. Cluster 1 is of medium sized ontologies, with a simple class hierarchy and an object property hierarchy of small to medium complexity. There is a small usage of class expressions and a medium object property usage. Also, this cluster seems to include ontologies with data properties as well as individuals with many features and the highest usage of all of the clusters. Finally, it has a small inferential and justification richness. Some ontologies in this cluster are the time 8, units 9 and substance 10. Cluster 2 includes small ontologies, but with a medium to high inferential and justification richness. Even though its ontologies have a simple class hier- 6 //obo.cvs.sourceforge.net/*checkout*/obo/obo/ontology/taxonomy/fly_ taxonomy.obo 7 //obo.cvs.sourceforge.net/*checkout*/obo/obo/ontology/phenotype/ environment/environment_ontology.obo 8 //sweet.jpl.nasa.gov/1.1/time.owl 9 //sweet.jpl.nasa.gov/1.1/units.owl 10 //sweet.jpl.nasa.gov/1.1/substance.owl

74 Table 1. Clusters of ontologies revealing design styles in the TONES Repository. Cluster Attribute Cluster 0 Cluster 1 Cluster 2 Cluster 3 Cluster 4 Ontologies 37 ( 33%) 15 (13%) 24 ( 21%) 17 ( 15%) 19 ( 17%) Size mean std. dev Class Hierarchy Wideness mean std. dev Class Hierarchy Complexity mean std. dev objectproperty Hierarchy Wideness mean std. dev objectprop Hierarchy Compexity mean std. dev Class Usage mean std. dev Class Expression Usage mean std. dev Class Expression Complexity mean std. dev ObjectProperty Usage mean std. dev DataPropery Usage mean std. dev Individual Usage mean std. dev Class Feature Richness mean std. dev Object Property Feature Richness mean std. dev DataProperty Feature Richness mean std. dev Individual Feature Richness mean std. dev Annotations Feature Richness mean std. dev Entailment Richness mean std. dev Justification Richness mean std. dev archy, there is a rich object property hierarchy having many object property characteristics.

75 Cluster 3 includes the big ontologies in the repository, with a wide class hierarchy of medium to high complexity and a quite complex object property hierarchy. Its ontologies also have a rich description of the classes, with complex class expressions and high class usage. However, the inferential richness is small and the justification richness is small to medium. Cluster 4 has the richest ontologies in terms of inferential and justification richness. However, its ontologies are of small size. The class hierarchies are wide, with a high complexity. There appears also to be rich object property hierarchies, but the properties do not have many property characteristics. This cluster also has the highest class expression usage. This cluster seems to have many toy ontologies, which have been used for demonstrating OWL usage, such as the family-tree 11, generations 12 and university 13 ontology. Some of these clusters share characteristics, and thus appear closer in design style. However, there is at least one dimension in the rating scheme that splits them into different groups. It should be noted though, that this clustering does not produce fully accurate results, as there is expected to be some incorrect classifications (e.g. an ontology might be very close to two clusters). However, the purpose of this clustering is to give a general view and analysis of the repository, highlighting the most common design styles of its ontologies. In order to test how well the data were clustered, we use classifications algorithms with 10 fold cross validation to classify the ontologies to the corresponding 5 design styles, which were identified by the clustering. The accuracy of the classification algorithms (SMO, BayesNet, RandomForest) [14] were between 80.4% to 83.9% with an F-Measure range between 0.75 and Conclusions The contribution of this paper is the definition and categorisation of metrics for the description of underlying design styles in ontologies. This high level abstract description of an ontology can be considered as a metamodel, giving an intuition of the main characteristics of the ontology as a collection of axioms and entailments. In particular, we defined a rating scheme consisting of 20 dimensions, covering five different aspects of an ontology. Each dimension is assessed using mainly normalised metrics. The metric framework presented here was built on top of the existing metric package that is implemented in the OWL API. The results of the application of the metric framework in 112 ontologies from the TONES repository revealed five important design styles in the repository. It should be noted that these design styles depend on the content of the repository, and these results do not exclude the existence of other design styles. However, 11 // // //

76 the identification of potential design styles should be helpful for the analysis of a repository, as it can show from a high level the type of ontologies that reside in the repository. It also reveals similarities between ontologies that are included in the same cluster of design style; information that should be useful for the integration and reuse of ontologies. The final five clusters that were detected using clustering algorithms in the repository, show five different design styles of ontologies. Their inspection showed that there were small but complex ontologies as well as relatively big and simple ontologies. In addition, ontologies of similar apparent design style were found in the same cluster (e.g. they were toy ontologies, or were using the same OWL features). We believe the approach taken here for providing a metamodel based on metrics, has a potential as a starting point when comprehending an ontology. Future work remains to demonstrate its utility for users of ontologies. This will involve the evaluation and refinement of the rating scheme for categorising and assessing ontologies in terms of their design. In addition, future work will also involve the testing of the metric framework with multiple repositories in order to highlight the most common design styles of their ontologies. Metrics are abstract descriptions that can give an intuition of the overall design style of the ontology. Based on this initial information the user has an indication of what to expect to find in the ontology. This should be helpful for the decision on what comprehension tactic to follow. This framework of metrics, together with the use of straight-forward data-mining techniques, thus offers the possibility of providing ontologists with a high-level abstraction of an ontology s design style as an aid to ontology comprehension. References 1. F. Baader, B. Hollunder, B. Nebel, H. Profitlich, and E. Franconi. An empirical analysis of optimization techniques for terminological representation systems. Applied Intelligence, 4(2): , S. Bail, B. Parsia, and U. Sattler. Extracting finite sets of entailments from owl ontologies. International Workshop on Description Logics (DL), J. Bard, S. Rhee, and M. Ashburner. An ontology for cell types. Genome Biology, 6(2):R21, A. Gangemi, C. Catenacci, M. Ciaramita, and J. Lehmann. Modelling ontology evaluation and validation. The Semantic Web: Research and Applications, pages , A. Gangemi and P. Mika. Understanding the semantic web through descriptions and situations. In CoopIS/DOA/ODBASE 03, pages , A. Gibson, K. Wolstencroft, and R. Stevens. Promotion of ontological comprehension: Exposing terms and metadata with web 2.0. In Proceedings of the Workshop on Social and Collaborative Construction of Structured Knowledge at 16th International World Wide Web Conference (WWW2007). Citeseer, M. Horridge, J. Bauer, B. Parsia, and U. Sattler. Understanding entailments in OWL. In Proc. of OWLED. Citeseer, M. Horridge, B. Parsia, and U. Sattler. Laconic and precise justifications in owl. The Semantic Web-ISWC 2008, pages , 2010.

77 9. M. Horridge, B. Parsia, and U. Sattler. The state of bio-medical ontologies. ISMB conference, Bio-Ontologies SIG workshop, D. Kang, B. Xu, J. Lu, and W. Chu. A complexity measure for ontology based on uml. In Distributed Computing Systems, FTDCS Proceedings. 10th IEEE International Workshop on Future Trends of, pages , May C. Keet. Enhancing comprehension of ontologies and conceptual models through abstractions. AI* IA 2007: Artificial Intelligence and Human-Oriented Computing, pages , N. Noy, A. Chugh, and H. Alani. The CKC challenge: Exploring tools for collaborative knowledge construction. Intelligent Systems, IEEE, 23(1):64 68, A. Rector, N. Drummond, M. Horridge, J. Rogers, H. Knublauch, R. Stevens, H. Wang, and C. Wroe. OWL pizzas: Practical experience of teaching OWL-DL: Common errors & common patterns. Engineering Knowledge in the Age of the SemanticWeb, pages 63 81, P.-N. Tan, M. Steinbach, and V. Kumar. Introduction to Data Mining. Addison- Wesley, S. Tartir, I. Arpinar, M. Moore, A. Sheth, and B. Aleman-Meza. OntoQA: Metricbased ontology quality analysis. In IEEE Workshop on Knowledge Acquisition from Distributed, Autonomous, Semantically Heterogeneous Data and Knowledge Sources, volume 9. Citeseer, M. Uschold, M. Healy, K. Williamson, P. Clark, and S. Woods. Ontology reuse and application. In Formal Ontology in Information Systems, volume 179, page 192. Citeseer, D. Vrandecic and Y. Sure. How to design better ontology metrics. In The Semantic Web: research and applications: 4th European Semantic Web Conference, ESWC 2007, Innsbruck, Austria, June 3-7, 2007: proceedings, page 311. Springer-Verlag New York Inc, H. Yao, A. Orme, and L. Etzkorn. Cohesion metrics for ontology design and application. Journal of Computer Science, 1(1): , 2005.

78 Ontology-Driven Application Architectures with KOMMA Ken Wenzel Fraunhofer-Institute for Machine Tools and Forming Technology IWU, Department for Production Planning and Resource Management, Reichenhainer Str. 88, Chemnitz, Germany Abstract. The Knowledge Modeling and Management Architecture is an application framework for Java software systems based on Semantic Web technologies. KOMMA supports all application layers by providing solutions for persistence with object triple mapping, management of ontologies using named graphs as well as domain-specic visualization and editing of RDF data. This paper gives a short introduction to KOMMA's architecture and illustrates its usage with examples. Keywords: application framework, eclipse, rdf, semantic web, object triple mapping, model-driven architecture 1 Introduction Semantic Web technologies have gained popularity in the elds of information integration and semantic search. Therefore research and development activities mainly focused on ontologies, ontology editors, storage and inference solutions for RDF data as well as on generic solutions for presenting, navigating and editing RDF data sets. We argue that the wide adoption of Semantic Web technologies can be accelerated by providing sophisticated application frameworks that lower the barriers for the development of RDF and OWL-based software systems. Existing frameworks for model-driven engineering (MDE) like the Eclipse Modeling Framework (EMF) 1 simplify the creation of model-based applications by providing tools that support the required transformation tasks. The EMF project shows that there is large industrial request for model-based application frameworks. Over recent years this project was able to acquire a large community that actively uses and develops a variety of solutions around the EMF core components. Because knowledge representation languages like RDFS or OWL can also be regarded as some type of modeling language in the sense of MDE, we've started the development of KOMMA by taking EMF as an example. 1

79 The goal of KOMMA is to combine RDF persistence providers with a sophisticated solution for object triple mapping and EMF-like exible visualization and editing capabilities into a unied application framework for RDF- and ontologybased software systems. 2 Architecture and object model KOMMA uses a layered architecture as depicted in Figure 1 to improve the reusability of its components. Edit Presentation Model ModelSet Named Graphs EntityManager DataManager Storage and Mapping Fig. 1. Architecture overview. The storage and mapping layer (Section 3) denes a small interface for pluggable persistence providers and implements an advanced solution for the mapping between Java objects and RDF triples. It is ontology language agnostic to ensure its reusability for dierent types of RDF based systems no matter which kind of ontology language, if any, is used. Currently, a persistence provider plugin for OpenRDF 2 Sesame is bundled with KOMMA but other triple stores can be easily added by implementing the IDataManager interface. The next layer on the stack deals with RDF named graphs (Section 4). It provides models to manage OWL ontologies along with their mutual dependencies and associated software components. In its current implementation the data of each ontology is mapped to one model that itself is contained in a set of interrelated models. Consequently, a model set contains these models that constitute the imports closure of all contained ontologies. The last layer implements mechanisms for the presentation and editing of RDF resources. It uses the adapter pattern to create views and editors for RDF resources and already provides many generic implementations and base classes to quickly create editable lists, tables or tree views. 2

80 IValue IReference IReference ILiteral IEntityManager * IEntity URI 0..1 metadatamanager IResource IModel * 1 IModelSet * IObject Fig. 2. Core object model. Figure 2 shows KOMMA's core object model. All objects representing RDF nodes implement the interface IValue. RDF resources are modeled by objects of type IReference while ILiteral is used denote RDF literals. In the case of named nodes an IReference is associated to a corresponding URI, for blank nodes this URI is null. The interfaces URI and IEntity are dened as specializations of IReference. This creates a unied class hierarchy where mapped objects of type IEntity can be used interchangeably with basic data-level references (URI for named nodes, IReference for blank nodes) to RDF nodes. Java objects representing RDF resources are managed by an IEntityManager which is responsible for object triple mapping as described in Section 3. IResource is a subject-oriented interface for reective access to properties of RDF nodes. Among other things, it can be used for determining the direct RDF types of a resource or for adding and removing of property values. The shaded classes in Figure 2 are core components of KOMMA's subsystem for the management of RDF named graphs and OWL ontologies which is covered by Section 4. 3 Object triple mapping KOMMA implements object triple mapping by using a role-based composition approach that is partially based on the work done for the OpenRDF projects Elmo and AliBaba. Figure 3 shows the main components of a KOMMA application where modules are used to group related roles (interfaces and behaviour classes) for object triple mapping.

81 OWL ontologies Ontology converter Generate interfaces Java interfaces «interface» Interface 1 «interface» Interface 2 KOMMA module Behaviour classes «mixin» Behaviour 1 «mixin» Behaviour 2 KOMMA application RDF store DataManager Sesame connector Jena connector... EntityManager Entity beans ModelSets + Models (optional) Adapter factories + Adapters Application user interface Fig. 3. Development workow and basic application structure. 3.1 Mapping of RDF types to Java classes RDF resources may have multiple associated types while Java supports only single inheritance. This fact imposes problems for the mapping of RDF resources to Java objects. One solution to circumvent this issue is by using the adapter pattern to create multiple views for a Java object that reect the dierent types of the underlying RDF resource. This approach is, for example, implemented by Jena's 3 ontology API. KOMMA follows another approach by using an engine that is able to combine multiple interfaces and behaviour classes into one class representing the union of multiple RDF types. These bean classes are generated dynamically at runtime of the application. Interfaces and classes can be mapped to RDF types by using annotation that expects an absolute IRI to associate its target with an RDF resource. When KOMMA needs to create a new Java class to represent an RDF resource as Java object, it retrieves its RDF types and determines all associated Java interfaces and classes. These are then combined into one class which embodies the resource's types. Figure 4 shows an example of mappings for the class Pet and its subtypes Cat and Dog. Each pet has the ability to yell. Since cats and dogs usually make dierent sounds, the method yell is implemented according to Figure 5 for cats by CatSupport and for dogs by DogSupport. 3

82 @ i r i ( " h t t p : / / example. org / p e t s#pet " ) p u b l i c i n t e r f a c e Pet i r i ( " h t t p : / / example. org / p e t s#name" ) S t r i n g getname ( ) ; v o i d setname ( S t r i n g name ) i r i ( " h t t p : / / example. org / p e t s#p a r e n t " ) Set<Pet> g e t P a r e n t s ( ) ; v o i d s e t P a r e n t s ( Set<Pet> p a r e n t s ) ; } v o i d y e l l ( ) i r i ( " h t t p : / / example. org / p e t s#cat " ) p u b l i c i n t e r f a c e Cat e x t e n d s Pet i r i ( " h t t p : / / example. org / p e t s#dog" ) p u b l i c i n t e r f a c e Dog e x t e n d s Pet {} Fig. 4. Mapping of RDF types to Java interfaces. A Java class representing an RDF resource with both types Cat and Dog (e.g. the famous CatDog 4 ) would get a yell method that chains both implementations of CatSupport and DogSupport. The order of chained calls can be dened by establishing a hierarchy between behaviour classes by using the a b s t r a c t p u b l i c c l a s s CatSupport i m p l e m e n t s Cat { p u b l i c v o i d y e l l ( ) { System. out. p r i n t l n ( getname ( ) + " s a y s MEOW" ) ; } p r e c e d e s ( CatSupport. c l a s s ) a b s t r a c t p u b l i c c l a s s DogSupport i m p l e m e n t s Dog { p u b l i c v o i d y e l l ( ) { System. out. p r i n t l n ( getname ( ) + " s a y s WUFF WUFF" ) ; } } Fig. 5. Extending mapped objects with user-dened behaviour. 4

83 3.2 Mapping of RDF properties to object methods The example in Figure 4 contains mappings for the properties name and parent of the type Pet. Both mappings are dened by annotating the corresponding getter methods with annotation. If a setter method is also present then the property is treated as writable, else as read-only. The mapping of properties is implemented by using PropertySet objects that encapsulate the required logic to convert between their RDF and Java representation. KOMMA contains a bytecode generator to implement interface methods annotated This generator uses a pluggable factory for property sets to enable the usage of various storage back ends. 4 Named graphs and models RDF named graphs are a means to structure RDF data sets. With named graphs it becomes possible to assign an IRI to a related set of RDF statements that can then be directly selected by its name within SPARQL queries. KOMMA provides models and model sets to manage data contained in RDF named graphs in combination with associated software components in form of Java interfaces and behaviour classes. Figure 6 shows two related models. Each model has its own module with interfaces and behaviour classes as well as an own entity manager. This manager is initialized with roles of the model's module and of all other modules contained in the model's imports closure. Hence, besides having read-only access to data from imported models, it is also able to use the imported roles for transformations of RDF nodes to Java objects. This enables the creation of ontology-driven architectures where composition of ontologies is accompanied by composition of related software components. RDF store with named graphs foaf: < foaf: rdf:type owl:ontology... p: < p: rdf:type owl:ontology p: owl:imports foaf:... read/write read read/write ModelSet Model < EntityManager EntityManager Model < Fig. 6. Managing named graphs and associated software components with models. Models implement the interface IModel that provides methods for loading the model's contents into a repository, for saving them to an external resource

84 and for deleting them from a repository. A model does also provide methods to access information about modication state or about diagnostics like warnings and errors. Loading and saving models is done with the help of a so-called URI converter that uses a set of rules for resolving model URIs to physical locations in the local le system or on a remote server. After resolving the correct location, a specialized URI handler for the encoded scheme (e.g. http, ftp or le) is responsible for accessing the corresponding resource. An IModelSet is a group of multiple interrelated models that is backed by an RDF repository where each model is stored in its own context (named graph). Figure 2 reveals that each IModelSet has an associated metadatamanager. The reason is that KOMMA implements models and model sets by using its own object triple mapping mechanism. This results in the ability to handle these objects in the same way as all other RDF resources and enables the usage of SPARQL to access metadata about models and model sets. 5 Editing framework and adapter factories KOMMA includes an Eclipse framework comprising generic reusable classes for building editors for RDF data models. This framework is strongly modeled after EMF.Edit, the editing framework for EMF. Likewise, it simplies the creation of GUI editors by providing content and label providers for JFace viewers, a command framework, including a set of generic commands for common editing operations as well as automatic undo and redo support for changes to models. Adapters are used to implement tailored presentation and editing behaviour for dierent model elements. These adapters are created by factories that may support dierent adapter types. In contrast to EMF where each model object keeps track of its associated adapters, KOMMA transfers this obligation to the adapter factories. This decision was made to decouple the domain model from transient applications objects. The reason is that KOMMA's domain objects are rather lightweight references to RDF resources and may be instantiated multiple times while EMF's domain objects usually exist only once in memory. 6 Using and supporting KOMMA KOMMA is developed as open source software, licensed under the terms of the Eclipse Public License (EPL) 5. The public community site for KOMMA is hosted at komma.sourceforge.net. This is the entry point to public forums, a bug tracker as well as KOMMA's source code repositories. Our code review system at git.iwu.fraunhofer.de enables users to participate in KOMMA's development by submitting patches for bugs or features. 5

85 Fig. 7. Editing framework in action. 7 Conclusions and future work KOMMA complements existing model-driven application frameworks by providing an integrated architecture for RDF and ontology-based software systems. Therefore, it is able to reduce the eort for the creation of semantic desktop and internet applications. Future developments of KOMMA include the extension of its editing framework towards more advanced domain-specic graphical and textual presentations of RDF data as well as seamless support of rich client and rich internet applications. We hope that these activities will improve KOMMA's applicability to a wide range of real-world usage scenarios. KOMMA already implements required core features to support innovative software design paradigms like Data, Context and Interaction (DCI) 6. Further research could investigate how KOMMA's role-based object triple mapping in combination with its notion of models can be used to implement context-based policies for user interaction to accomplish goals of the DCI principle. 8 Acknowledgements The work presented in this paper is co-funded within the Cluster of Excellence Energy-Ecient Product and Process Innovation in Production Engineering (eniprod R ) by the European Union (European Regional Development Fund) and the Free State of Saxony. 6

86 A Visual Tool for Supporting Developers in Ontology-based Application Integration Tobias Wieschnowsky 1 and Heiko Paulheim 2 1 SAP Research tobias.wieschnowsky@sap.com 2 Technische Universität Darmstadt Knowledge Engineering Group paulheim@ke.tu-darmstadt.de Abstract. Ontologies are increasingly used for a larger variety of tasks in software engineering. However, it turns out that many existing tools are directed rather at ontology experts than at software engineering experts, which often reduces their acceptance and hinders are larger breakthrough of semantic technologies in software engineering. Tools that speak the language of software engineers may help overcoming that gap. In this paper, we introduce an ontology and rule editor specifically tailored at the task of application integration, and demonstrate its usability for software engineers with a comparative user study. 1 Introduction Ontologies are used in software engineering for various purposes, e.g., in integration tasks, for capturing requirements, for building abstraction layers between systems, for improving model based development, or for enabling better development tools, such as component repositories with semantic annotations [1]. While those approaches solve different problems, they most often demand that the developer learns to handle ontologies and understands the underlying logic foundations. Experience has shown that the promises semantic technologies makes for software engineering are not always kept, and often, the reason is that software engineers lack the appropriate understanding for ontologies. While some concepts of ontology engineering, such as the definition of classes and sub classes, are rather intuitive for software engineers, working with more sophisticated concepts such as logic axioms or rules is not as trivial. Furthermore, existing ontology editors and tools most often demand a deeper insight into the underlying formalisms, and thus, lack good usability [2]. Therefore, in order to get semantic technologies accepted by software engineers, good tools are an essential requirement, which hide most of the logic complexity under the hood, yet are versatile enough for the task at hand, and speak the developers language. In this paper, we focus on the particular aspect of creating logic rules employed in an ontology-based application integration framework. We discuss a prototype for creating such rules, and compare it to other state-of-the-art tools in a comparative user study.

87 2 Application Integration on the User Interface Level Ontologies have been successfully used in application integration during the past 20 years. The best-known approaches are ontology-based integration on the database level, where an ontology-based abstraction layer is placed on top of different, heterogeneous databases, and on the business logic level, e.g., by employing semantic web services. In contrast to those well-researched approaches, ontology-based application integration on the user interface level is a comparatively new research direction. In [3], we have discussed a framework for employing ontologies and rules for integrating applications on the user interface level, i.e., facilitating crossapplication interactions between existing applications, while, at the same time, reusing existing user interfaces. When using the framework, a developer providing an application to be integrated has to write an ontology describing the application, as well as a set of rules defining the possible interactions with other applications. We have built a prototype implementation of that framework, using Java and OntoBroker 1, and F-Logic for describing both application ontologies and integration rules. The prototype of the framework was used in the large scale research project SoKNOS for developing a functional prototype for the domain of emergency management [4]. Each development team providing an application was asked to also develop the respective ontology and interaction rules. However, said task was not trivial for developers, even with a sophisticated tool such as OntoStudio. Especially the interaction rules in F-Logic proved difficult to create. Furthermore, it was hard for the developers to understand given rules, thus, the problem was not solved with providing a few example rules. In addition, the maintenance of existing rules was difficult due to that lack of understanding. These insights lead to the requirement of providing tool support to developers. 3 Prototype Existing tools for developing ontologies and rules are typically general-purpose tools, which provide many capabilities, at the price of being very complex. Furthermore, they typically speak the language of formal logics, rather than the language of software engineering and/or the domain for which the software is developed. The basic design rationales for developing a tool for supporting developers were therefore to reduce complexity at the cost of functionality, and to make the tool speak the developers language. Thus, we have developed a tool which can be used to create only the specific types of ontologies and rules that were needed in the context of the integration framework. The tool prototype is realized as an Eclipse plugin, since Eclipse is a wellknown software engineering tool. Fig. 1 provides an overview of the architecture 1

88 User Interfaces Application Ontology UI UI Rule Tool Application Logic Ontology Object Model OntoBroker Graphiti Diagram & EMF F-Logic File Filestorage Diagram File Fig. 1. Overview of the architecture of the UI Integration Tool. of the prototype. On the top level it consists of two user interfaces that each consist of a number of eclipse views and editors. The first user interface is the application ontology editing user interface and the second is the integration rule user interface. The later is the main focus of this paper and is also called the UI Rule Tool. The main editor of the UI Rule Tool is a Graphitti/EMF based visual rule editor, where developers can construct their integration rules without any knowledge of F-Logic. Underneath the integration rule user interface lies the Graphiti/EMF model which is supported by a Java representation of the ontologies, called the Ontology Object Model. All application ontology user interfaces are based directly on the Ontology Object Model. Any information in the Ontology Object Model is stored as F-Logic files through OntoBroker s Java API. Any existing integration rules are also saved in Graphiti s own Diagram format which includes additional information, like position of the parts in the diagram. This dual storage of the rule itself as well as the layout information allows for the exact reconstruction of the visual layout when a rule is reloaded, a feature which is not provided by editors that only store the logic rule itself. Figure 2 shows an example integration rule in the UI Rule Tool. The user interface consists of several utility views, like for example the ontology viewer (on the left) and the main editor (in the middle). The main editor window in turn, contains the drawing area and the tool palette. The two basic components of the visual integration rules are instances (displayed as boxes) and relations (displayed as the arrows connecting the boxes). This visual representation is very simple and should be intuitive. In addition there are more components like attributes (boxes on instances) and logic parts like OR and NOT. The most visually dominant feature of the UI Rule Tool is the clear visible split into cause and effect. This reflects the two parts of a logic rule, body and head, but has been renamed to make it clearer which role the two parts of the rule play within the framework. The first part of the rule describes the conditions to

89 Fig. 2. An example rule in the UI Rule Tool. the event that occurs within the framework and causes the reasoner to examine the rule (the cause). The second part describes what changes in the application that owns the rule if the cause part is found to be true (the effect). Thinking of the two parts of the rule in these terms should help developers to better understand how the rule works within the framework. Another feature that attempts to incorporate the framework into the rule editing process are the instance roles, which are derived from reoccurring patterns of rules used in the integration framework. This pattern can also be seen in the example rule diagram in figure 2: The rule is triggered by an instance of type Action (or one of its subtypes), which may involve one or more ApplicationObjects representing objects from the domain, and which may be performed with a Component. The effect side of a rule typically consists of an Interaction supported by the Component and triggered by the Action, which in turn causes the performing of an Action. All those instances are especially marked in the tool with specific symbols, thus providing a domain-specific visualization of the rule. Furthermore, the palette contains a set of tools for creating that basic pattern or typical subpatterns of rules used in the framework. 4 Evaluation In order to evaluate the rule editing interface, a user study was conducted, which compared it against those of three other ontology and/or rule editing tools. The goal of the user study was to determine if the UI Rule Tool is better suited for the creation of integration rule within the framework for UI integration. 4.1 Overview The user study was comprised of a total of 16 participants, who were undergraduate students, PhD candidates, and researchers from SAP Research in Darmstadt

90 or the TU Darmstadt. Each participants was given the same 4 example rules, taken from the SoKNOS prototype of the UI Integration Framework. For each participant, each tool was randomly assigned to one of the four example rules. Overall, each tool was assigned to each rule an equal number of times. After completing all tasks for a tool/rule combination, the participants were asked to fill out a User Experience Questionnaire that captured their attitude towards the tool they had just worked with. After all 16 participants completed the study, the results were evaluated by three criteria: Time needed to complete the tasks, number of errors made, and the responses to he questionnaires. 4.2 Setup All participants of the user study were volunteers from SAP Research and the TU Darmstadt and had a strong background in computer science. At the beginning of the study they were asked to self-asses their knowledge on ontologies and logic rules, as well as the three tools competing with the UI Rule tool. Depending on their response to the questions about their ontology and logic rule knowledge, the users were classified as either an expert, or a non-expert. This classification was later used for in-depth analysis of the results, but had no influence on the study design. Non of the participants indicated enough experience with any of the tools to warrant a separation into expert and non-expert groups. The four tools compared in the user study were: the our prototype of the UI Rule Tool, OntoStudio 2, Protégé 3, and Strelka 4. For each rule, participants had to complete a set of three tasks, two of them multiple choice and one modification task. The multiple choice questions were concerned with understanding the given rules: the first question asked users to identify the element in the rule that triggered the rule from the perspective of the framework, and the second question asked the users to identify the element in the rule that was the result of the rule and would lead to the framework modifying the application the rule belongs too. For both multiple choice tasks, error rate and completion time were recorded. The third task for each tool was a rather open task, where the user was asked to modify the rule and explore the capabilities of the tool. Participants were also allowed to ask questions to the interviewer when performing the third task. Error rate and completion time were not recorded for this task as it s focus was more to allow the user to work with the tools in order to increase the significance of the questionnaire results. After completing all tasks assigned to a tool, the participants were asked to fill out a User Experience Questionnaire for the corresponding tool. The User Experience Questionnaire [5] (UEQ) asks participants to judge their experience with a tool based on 26 adjective pairs. Each pair contains two polar opposites like for example, fast and slow. Users then marked which of the two adjectives more closely described their interaction with the program. The 26 attribute

91 Fig. 3. Errors per Task and Tool pairs are used to calculate six user experience factors: attractiveness, perspicuity, dependability, efficiency, novelty and stimulation. As part of the evaluation of the results, the statistical significance of the recorded differences between the tools was computed. For time and UEQ results, the two sample t test (or short t test) was used to compute the statistical significance, with a threshold of 95% before a result was considered significant. For the differences in errors, the chi-square test (or chi-test) was applied to compute the significance, again using 95% as the threshold. These test were used to ensure that the results of the user study were not just due to random variations in the rather small data set of the 16 participants. 4.3 Results Overall significantly more errors were made in all tools than expected, considering that all rules shared a common pattern and the correct responses were very similar for each rules tasks. Figure 3 shows a summary of errors made by task. The overall lowest number of errors were made in Protégé which was a surprising result, considering it is the only rule editor in the study that is text based. only the comparison between the errors made in Protégé (least amount of errors) and OntoStudio (most errors), was statistically significant (chi-test with threshold of 95%). Figure 4 provides an overview of the average time recorded for the completion of each multiple choice task during the study. The two visual rule editing tools, UI Rule Tool and OntoStudio, are on average faster than the other two tools. The differences in the recorded times were significant in three cases. Participants solved the tasks significantly faster with the UI Rule Tool than with Protégé or with Strelka. The final recorded metric of the user study were the participants responses to the UEQ for the individual tools. The UEQ yielded the most significant results of the three metrics. Figure 5 shows the computed results for all tools in the six attributes and the overall score. The UI Rule Tool achieved the highest score in every single attribute of the UEQ. All score differences between the UEQ and the other tools were significant,

92 Fig. 4. Average time per task and tool for the multiple choice tasks. Fig. 5. Overview of the UEQ scores for all four tools. with the exception of the novelty score in comparison to OntoStudio and Strelka. Especially the huge advantage in perspicuity was important for the tool, since perspicuity directly relates to how understandable and readable the integration rules are, which was one of the main goals of the tool. These overwhelmingly positive responses indicate that the participants of the user study clearly preferred the UI Rule Tool over the others for the given tasks. Finally the UEQ score for the two participant groups, ontology experts and non-experts, were compared. Overall the UEQ scores were a lot closer for the expert group between tools. Especially Protégé and Strelka received significantly higher scores on average. A possible explanation for the large difference is that the expert group is more likely to be familiar with the notation of Protégé and Strelka. 5 Conclusion and Outlook In this paper, we have introduced a tool which eases the creation and maintenance of ontologies and rules in a given software engineering setting, i.e., application integration on the user interface level. The key rationales were the reduction of complexity by moving from a general-purpose ontology and rule

93 editor to a tool specifically tailored to one particular task, i.e., the integration of applications. We have compared our prototype tool to three state of the art tools in a user study. That study showed that our approach was better suited to the task of editing and understanding integration rules then the more generic competitors, especially in terms of task completion time and perceived user experience. Some improvements are possible in the future. Especially the number of errors in all tools in the user study was higher then expected, suggesting that the rules are still complicated to understand and their role within the framework is at times unclear. Improvements that could be implemented include a split which divides the cause part of the rule into the two applications that are involved in the transaction. The first application is the one that triggers the event, the second the one is modified in response to the event. Since the rule can only modify the second one, the effect part of the rule should remain unsplit. From the comments of the users, we have learned that the display of OR and NOT parts of rules should be changed, since it was misleading in some cases. Additionally due to the lower error rate in Protégé, a hybrid representation of visual and textual rule editing could be considered. In summary, our prototype has shown that for specific software engineering tasks, it may be beneficial to step from general purpose ontology and rule editors to particularly tailored tools. As adequate tool support is an essential requirement for increasing the software engineers acceptance of semantic technologies, and the results of our user study have shown significant improvements in terms of user experience, we are confident that this is a fruitful direction for future research. References 1. Happel, H.J., Seedorf, S.: Applications of Ontologies in Software Engineering. In: Workshop on Semantic Web Enabled Software Engineering (SWESE) on the 5th International Semantic Web Conference (ISWC 2006), Athens, Georgia, November 5-9, (2006) 2. García-Barriocanal, E., Sicilia, M.A., Sánchez-Alonso, S.: Usability evaluation of ontology editors. Knowledge Organization 32(1) (2005) Paulheim, H., Probst, F.: Application Integration on the User Interface Level: an Ontology-Based Approach. Data & Knowledge Engineering Journal 69(11) (2010) Döweling, S., Probst, F., Ziegert, T., Manske, K.: SoKNOS - An Interactive Visual Emergency Management Framework. In Amicis, R.D., Stojanovic, R., Conti, G., eds.: GeoSpatial Visual Analytics. NATO Science for Peace and Security Series C: Environmental Security, Springer (2009) Laugwitz, B., Held, T., Schrepp, M.: Construction and evaluation of a user experience questionnaire. In: Proceedings of the 4th Symposium of the Workgroup Human-Computer Interaction and Usability Engineering of the Austrian Computer Society on HCI and Usability for Education and Work. USAB 08, Berlin, Heidelberg, Springer-Verlag (2008) 63 76

94 Semantic Web-driven Development of Services-oriented Systems Exploiting Linked Data for Services Annotation and Discovery Stefan Dietze 1, Dong Liu 2, Hong Qing Yu 2, Carlos Pedrinaci 2 1 L3S Research Center, Hanover, Germany dietze@l3s.de 2 Knowledge Media Institute, The Open University, Milton Keynes, MK76AA, UK {d.liu, h.q.yu, c.pedrinaci}@open.ac.uk Abstract. Within a service-oriented architecture (SOA), software components are accessible via well-defined interfaces. To this end, the discovery and integration of Web services and APIs is becoming an increasingly important task in present-day software engineering. Despite considerable research dedicated to Semantic Web Services (SWS), structured semantics are still not used significantly to facilitate services and API discovery. This is due to the complexity of comprehensive SWS models and has led to the emergence of a new approach dubbed Linked Service which adopt Linked Data principles to produce simplified, RDF-based service descriptions that are easier to create and interpret. However, current Linked Services tools assume the existence of services documentation (HTML, WSDL) and do not sufficiently support nonfunctional properties (NfP). Therefore, we introduce SmartLink, a Web-based editor and search environment which allows both humans as well as machines to produce light-weight service descriptions from scratch by addressing both, functional and non-functional service properties. Keywords: Software Engineering, Semantic Web, Linked Data, SmartLink. 1 Introduction An essential part of Software Engineering nowadays is concerned with the discovery of reusable software components which satisfy one or more requirements of the overall system to be implemented. The past decade has seen the emergence and largescale success of another fundamental paradigm: service-orientation. Within a serviceoriented architecture (SOA), components are accessible via well-defined interfaces and usually exchange messages via remote-procedure calls (RPC), HTTP or SOAP. Particularly the emergence of REST-ful services has led to the widespread availability of public and reusable Web APIs, such as the wide range of APIs offered by Google 1. To this end, the discovery and integration of Web services and APIs is becoming an increasingly important task in present-day software engineering. 1

95 Research efforts in the area of Semantic Web Services (SWS) were mainly aiming at the automation of Web service-related tasks such as discovery, orchestration or mediation. Several conceptual models, such as OWL-S [6], WSMO [3], and standards like SAWSDL [7] have been proposed, usually covering aspects such as service capabilities and interfaces. However, SWS research has for the most part targeted WSDL or SOAP-based Web services, which are not prevalent on the Web. Also, due to the inherent complexity required to fully capture computational functionality, creating SWS descriptions has represented an important knowledge acquisition bottleneck and required the use of rich knowledge representation languages and complex reasoners. Hence, so far there has been little take up of SWS technology within non-academic environments. That is particularly concerning since Web services nowadays including a range of often more light-weight technologies beyond the WSDL/SOAP approach, such as RESTful services or XML-feeds are in widespread use throughout the Web. That has led to the emergence of more simplified SWS approaches such as WSMO-Lite [9] SA-REST [7] and Micro-WSMO/hRESTs [4] which benefit from simpler models expressed in RDF(S). While the Semantic Web has successfully redefined itself as a Web of Linked (Open) Data (LOD) [1], the emerging Linked Services approach [7] exploits the established LOD principles for service description and publication. By supporting annotation of a variety of services, such as WSDL services as well as REST APIs, the Linked Services registry and discovery engine iserve 2 enables publishing of service annotations as linked data expressed in terms of a simple conceptual model: Minimal Service Model (MSM), a simple RDF(S) ontology able to capture (part of) the semantics of both Web services and Web APIs. However, while Linked Services appears to be a promising stream of research, we observe two major issues which hinder a large-scale take-up of the Linked Services approach: (i1) Lack of consideration of non-functional service properties and less formal metadata (i2) Lack of appropriate editors and annotation environments With respect to (i1), previous efforts have largely focused on formalizing the actual functionalities of a service (capabilities, interfaces). However, in order to allow assessment about suitability of individual services or APIs for a particular service consumer, non-functional properties (NfP) are of crucial importance. These include, for instance, basic metadata about the development status or the licensing model as well as information about the quality of service (QoS). In addition, less formal services annotations turned out to be very useful since one of the yet most established mode of using Linked Services aims at rather semi-automated service discovery where developers browse or navigate through Linked Services libraries based on filtering mechanisms, as opposed to fully automated services discovery and orchestration. While the latter is fundamentally dependent on complex and formal specifications of services capabilities and interfaces (i.e. functional properties) the former can be supported based on rather light-weight and often non-functional service 2

96 metadata, such as classifications, tags or development status information. However these are not sufficiently supported within current schemas such as MSM and WSMO-Lite. With regard to (i2), editors had been developed which support developers in creating semantic annotations for services: SWEET [5] (SemanticWeb services Editing Tool) and SOWER (SWEET is not a Wsdl EditoR). However, SWEET and SOWER build on the assumption that either HTML documentation of services/apis (SWEET) or WSDL files (SOWER) are available as starting point for annotation. While that holds for a certain set of services, a growing number of services on the Web neither provide a WSDL nor an HTML documentation and hence, current Linked Services editors cannot be deployed in a range of cases. In this regard, we particularly would like to promote an approach where services documentation relies exclusively on structured RDF(S) while additional human-readable documentation is not provided manually but automatically generated to avoid redundancies. Therefore, we introduce SmartLink 3 ("SeMantic Annotation environment for Linked services"), which addresses (i1) and (i2) by contributing: (a) an RDF schema and data store for service NfP (b) an integrated editing and browsing environment for Linked Services on the Web (taking into account both functional and non-functional data) In the following Section we provide some background information on Linked Services, while Section 3 introduces the SmartLink NfP schema. Section 4 describes overall architecture of SmartLink. We finally discuss our results in Section 6. 2 Non-functional properties for Linked Services Previous work dealing with the exploitation of SWS and Linked Services technologies in NoTube 4 and meducator 5, as described in [2][10], has shown that one of the most established and accepted use cases for Linked Services annotations aims at browsing and searching services in a meaningful way as opposed to automated services discovery and execution. To this end, Linked Services seem of particular use when aiding developers in finding APIs for a given software engineering task. In this regard, formal specifications turned out to be less important while lightweight service annotation with tags/keywords and classifications plaid a vital role. Particularly when supporting collaborative annotation of entities services like any documents, content or data by a multiplicity of service consumers and developers, formal correctness of the generated data can hardly be enforced and means are required to provide descriptions in a more loose and flexible way. For instance, in many cases, Linked Data resources can be roughly associated with a service for instance, by tagging it with a service category or keyword which might not provide formal enough semantics to facilitate automation of discovery-based execution, but might still be useful to facilitate users in finding appropriate services. For instance, an 3 &

97 API exposing metadata of resources could be associated with a keyword metadata or a reference to However, the current scope of SWS and Linked Services does not provide appropriate facilities to represent such rather lose relationships in an appropriate way but focuses on formal representations of service elements, such as message parts or operations. In that respect, a need for less formal services annotations was observed, to facilitate developers and service consumers to collaboratively annotate services based on Linked Data principles without constraining them by insisting on complete coherence of the provided annotations. Instead of enforcing non-contradictory data, collaborative annotation schemas need to embrace diversity even if that reduces the opportunities for reasoning-based automation. On a similar note, current service description schemas (e.g., MSM, OWL-S, WSMO-Lite) seem to be fundamentally focused on functional properties while not providing sufficient support for NfPs, which would, for instance, allow users to specify licensing schemes, quality of service information or development status descriptions. While some schemas already allow the association of additional service information with particular service instances, the use of dedicated Linked Data vocabularies to further specify NfPs is still underdeveloped. SmartLink NfP schema To this end, we have developed a dedicated schema that addresses the aforementioned issues by (a) focusing in particular on NfPs and (b) facilitating collaborative, naturally diverse and less formally coherent annotation. To ensure the widespread applicability and reusability of the NfP schema, we reuse existing ontologies and vocabularies rather than constructing new ontologies from scratch. As shown in Fig. 1, the schema captures four main aspects of the non-functional properties of Web services, i.e. social, technical, licensing and QoS. Social attributes include human factors such as developer, contact person, organisation, project. The FOAF 6 vocabulary is adopted to describe those personal and social factors. Furthermore, tags attached to Web services are also regarded as an important social attribute, which helps in service classification and organization. Thus, the CommonTag 7 vocabulary is adopted to support the tagging by ensuring interoperability of provided service tags. The technical NfPs refer to information about how to interact with the services and cover, for instance, the communication protocol (e.g. HTTP and SOAP), data (exchange) format (e.g. XML, RDF and JSON), status (e.g. testing, final, work-in-progress), authentication model (e.g. HTTP Basic, API Key, OAuth). It is worth noting that technical NfPs do not describe the behaviours of services, but clarify the prerequisites for consumers to invoke those Web services. The licensing properties indicate the terms and conditions with respect to the usage of individual Web services. As shown in Fig. 1, we currently define four concepts for the licensing properties, i.e. service license, data license, usage limits and fees. A service license authorizes and constrains invocation of the service, whereas a data license is for the reuse or repurpose of data generated or provided by the service. Usage limits cover the amount of times of service invocation within a certain time period, or the minimum interval between two times of invocation. Obviously, fees are

98 applicable to non-free services only and refer to the price a consumer needs to pay for consuming a service. Social foaf:person ctag:tag nfp:protocol nfp:dataformat Technical foaf:organisation foaf:project nfp:fee nfp:usage Limit nfp:hascontactperson nfp:hasdeveloper nfp:hasorganisation nfp:hasproject nfp:hasfees nfp:haslimit nfp:haslicense nfp:hastag nfp:status nfp:hasprotocol nfp:hasstatus nfp:dataformat nfp:authenticationmodel nfp:authenticatedby msm:service nfp:response nfp:hasqosparameter Time nfp:availability Licensing nfp:data License nfp:service License nfp:reliabilty nfp:throughput QoS Fig. 1. A partial view of the SmartLink NfP Schema. The licensing properties indicate the terms and conditions with respect to the usage of individual Web services. As shown in Fig. 1, we currently define four concepts for the licensing properties, i.e. service license, data license, usage limits and fees. A service license authorizes and constrains invocation of the service, whereas a data license is for the reuse or repurpose of data generated or provided by the service. Usage limits cover the amount of times of service invocation within a certain time period, or the minimum interval between two times of invocation. Obviously, fees are applicable to non-free services only and refer to the price a consumer needs to pay for consuming a service. With respect to the quality of Web services, we adopt the model from [12], where the QoS parameters are divided into two classes: objective parameters and subjective parameters. The former are quantitative measures like availability, reliability, throughput and response time, whereas the latter are qualitative measures like user ratings. Here, we only focus on the objective QoS parameters, because theses have been published on the Web 8, 9. Schema mapping and alignment We reuse existing vocabularies to represent the NfPs of Web services. It allows interoperability between individual service description repositories and facilitates the import of publicly available service NfP metadata into SmartLink. Here, we take ProgrammableWeb 10 as an example to demonstrate schema mapping and alignment. Parts of the mappings between our schema and the one of ProgrammableWeb are shown in the table below. In addition, API Status 8 provides the statistics of the availability and response time of public APIs. Similarly, Mashery 9 monitors on the availability and response time of a set of services. The metadata these repositories exploit can be completely mapped to SmartLink schema. Moreover, the data can also be imported to SmartLink

99 Table. 1. NfP schema mapping between SmartLink and ProgrammableWeb. SmartLink NfP Schema ServiceLicense ServiceLicense Fee Usage Limit Authentication Model foaf:organization foaf:company foaf:weblog ProgrammableWeb's Schema Commercial Licensing Non-Commercial Licensing Usage Fees Usage Limits Authentication Model Provider Company API Blog 3 SmartLink: a Linked Services editor and browser In order to provide a Linked Services editor which allows (a) the annotation of RESTful services without any pre-existing documentation and (b) annotation of services according to multiple schemas, in particular SmartLink NfP, we have developed the SmartLink editor. SmartLink provides editing and browsing facilities to interact with multiple RDF stores and data sets. It allows annotation of services from scratch, that is, without any pre-existing services documentation such as WSDL or HTML files, as assumed by existing annotation tools (Section 1). HTML Linked Data SPARQL SmartLink LInked Data Provider GUI SmartLink Editor REST API SmartLink Service Content Negotiator RDF Serializer Page Generator Service Manager SmartLink Core User Manager Service Invoker NfP Repository rd fs:isd e fin e d By Fig. 2. SmartLink overall architecture. As shown in Fig. 2, SmartLink operates on top of Linked Data stores that exploit the MSM and the SmartLink NfP schemas and are interlinked with other Linked Data sets. MSM-schema properties are directly stored in iserve, while additional properties are captured in our SmartLink NfP repository 11. The repository provides a SPARQL endpoint 12. Following rdfs:isdefinedby links from SmartLink to iserve, more information about the functionalities and behaviours of the services can be retrieved. Being an LOD-compliant environment, one of the core features of SmartLink is the capability to associate service descriptions with so-called model references which refer to RDF descriptions in external vocabularies defining the semantics of the

100 service or its parts. However, while this feature is useful and even necessary in order to provide meaningful service models, finding appropriate model references across the entire Web of data is a challenging task. Therefore, SmartLink uses established Linked Data APIs currently the WATSON 13 API - to identify and recommend suitable model references to the user. Fig. 3. SmartLink Service editor. After loading RDF triples from both iserve and SmartLink, the editor visualizes the description of a service as shown in Fig. 3. The left-hand side of the editor is the treebased overview of the service, which represents a hierarchy composed of a service, its operations and input/output messages. The right hand side displays more details about the selected element in a form, which essentially include the semantics, categories, and literal descriptions. To persistently store changes made to a service description, SmartLink publishes the descriptions as Linked Data by invoking the RESTful APIs provided by iserve and the SmartLink NfP repository. SmartLink currently provides mechanisms that enable the export of particular service instances as RDF or humanreadable HTML. In order to facilitate service model transformation between MSM and other SWS formalisms, current research deals with the establishment of an export mechanism of MSM/SmartLink NfP services. In addition, SmartLink also offers a simple UI for filtering services by NfPs. That way, developers can easily construct queries without having to formulate SPARQL queries to create specific views on the services data. 4 Discussion and conclusion In this paper, we have proposed SmartLink which provides (a) an RDF schema which allows to describe non-functional properties of Web APIs and services (SmartLink NfP) and (b) a public environment which enables developers to annotate services and store descriptions in a public Linked Data-compliant store, to interlink them with other service descriptions such as the ones offered by iserve, and to search for available services and APIs by exploiting the structured semantics of the SmartLink NfP repository. To this end, SmartLink facilitates software engineering processes, particularly in the context of the prevailing SOA paradigm, by supporting developers in annotation and discovery of software components, i.e., services and APIs, across the Web. 13