Learning Ontology Alignments using Recursive Neural Networks

Transcription

1 Learning Ontology Alignments using Recursie Neural Networks Alexandros Chortaras, Giorgos Stamou, Andreas Stafylopatis School of Electrical and Computer Engineering National Technical Uniersity of Athens Zografou 57 80, Athens, Greece Abstract. The Semantic Web is based on technologies that make the content of the Web machine-understandable. In that framework, ontological knowledge representation has become an important tool for the analysis and understanding of multimedia information. Because of the distributed nature of the Semantic Web howeer, ontologies describing similar fields of knowledge are being deeloped and the data coming from similar but non-identical ontologies can be combined only if a semantic mapping between them is first established. This has lead to the deelopment of seeral ontology alignment tools. We propose an automatic ontology alignment method based on the recursie neural network model that uses ontology instances to learn similarities between ontology concepts. Recursie neural networks are an extension of common neural networks, designed to process efficiently structured data. Since ontologies are a structured data representation, the model is inherently suitable for use with ontologies. Introduction The purpose of the Semantic Web is to introduce structure and semantic content in the huge amount of unstructured or semi-structured information aailable in the Web. The central notion behind the Semantic Web is that of ontologies, which describe the concepts and the concept relations in a particular field of knowledge. The data associated with an ontology acquire a semantic meaning that facilitates their machine interpretation and makes them reusable by different systems. Howeer, the distributed deelopment of domain-specific ontologies introduces a new problem: in the Semantic Web many independently deeloped ontologies, describing the same or ery similar fields of knowledge, will coexist. These ontologies will not be identical and will present from minor differences, such as different naming conentions, to higher leel differences in their structure and in the way they represent knowledge. Moreoer, legacy ontologies will hae to be used in combination with new ones. For this reason, before being able to combine similar ontologies, a semantic and structural mapping between them has to be established. The process of establishing such a mapping is called ontology alignment. It will become increasingly significant as the Semantic Web eoles, it is already an actie research area and seeral auto- The author is funded by the Alexander S. Onassis Public Benefit Foundation.

2 matic or semi-automatic ontology alignment tools hae been proposed (e.g. [5, 6, 7]). Most of the tools rely on heuristics that detect some sort of similarity in the description of the concepts and the structure of the ontology graphs, by using e.g. string and graph matching techniques. They usually work at the terminological leel of the ontologies without taking into account their instances. A different method is proposed in [], where a machine learning methodology is used. The approach is extensional, i.e. it exploits the information contained in the ontology instances. For all concepts in the ontologies to be aligned, a naïe Bayes classifier is built. The instances of each concept are then presented to the classifiers of the other ontology and, depending on the degree of oerlap of the classifications, a similarity measure for each concept pair is computed. The classifiers do not take into account the structure of the ontologies; this is considered at a subsequent stage by integrating a relaxation labelling technique. The method we propose follows a similar machine learning approach, but takes directly into account the structure of the ontologies, by relying on the use of recursie neural networks [3], which are a powerful tool for the processing of structured data. The rest of the paper is organized as follows: section discusses the basic ideas underlying the recursie neural network model, section 3 describes the details of our method and presents a simple example, and section 4 discusses future work and concludes. Recursie Neural Networks ch chp The recursie neural network model was proposed in [3, 8] as an extension to the recurrent neural networks, and is capable of efficiently processing structured data. The data are represented as labelled directed ordered acyclic graphs (DOAGs), on whose structure a neural network (encoding neural network) is repeatedly unfolded. Because the representation of the data as DOAGs is in many applications too restrictie, some extensions to the initial model hae been proposed, that generalize the type of graphs on which it can be applied. For example, in [4] the graphs are allowed to hae labels attached also to their edges. This extension, which we use in our method, lifts a significant constraint of the initial model that required the graphs to hae a maximum, a priori known out-degree as well as an ordering on their out-going edges. In the model the data are represented as directed acyclic graphs, each node of m which is assigned a label L R and each edge connecting nodes and w a label L k ( w, ) R. To each node an encoding neural network is attached, that computes a n state ector X R for. Let ch ( ) be the set of children of, p the cardinality of ch( ) and ch i ( ) the i -th child of. The input to the encoding neural network of is a) a ector, function of the state ectors X ( ), X ( ) of the node s children and of the corresponding edge labels L (,ch i ( ) ), and b) the label L of. The encoding neural network is usually an MLP and is identical for all the nodes of the graph. The output of the recursie neural network is obtained at the graph s supernode, a node from which a path to all other nodes of the graph exists. The output is computed by a common neural network applied on the state ector of the super-node. The strength of the model is that the state of each node, calculated by the encoding neural network and encoded in the state ector, is computed as a function not only of

3 the label of the node, but also of the states of all its children. On their turn, the states of the children depend recursiely on the states of their respectie children. As a result, the state ector of each node encodes both the structure and the label content of the sub-graph that stems from the node. Thus, if a recursie neural network classifier is trained appropriately on data represented as graphs of different structures then it will be able to identify data similar both in their content and structure. 3 Neural Extensional Ontology Alignment Our method is based on the fact that an ontology can be considered as a graph, with the ontology concepts (relations) corresponding to the nodes (edges) of the graph. The graph is directed because the ontology relations are in general not symmetric, and each node (edge) has a label consisting of the and the attributes of the corresponding concept (relation). This holds at the terminological leel of the ontology. At the instance leel, there are instances belonging to concepts and pairs of instances connected by relation instances. Thus, if we consider an instance I of a concept C, then by following the relation instances stemming at I, we obtain a tree that consists of I in its root (root instance), which is connected with nodes that correspond to the instances of the ontology concepts with which I is related. The tree can grow up to n leels, by recursiely following the relation instances of the new nodes that are added to the tree. We call this graph an instance tree of leel n of concept C for the instance I. Our tool computes similarities between the concepts of two ontologies by training and applying a recursie neural network classifier on such instance trees. In detail, the method has as follows: Let O be the ontology whose concepts we want to map to the concepts of a similar ontology O, and C i, i =, p, C i, i =, p be the concepts in O and O respectiely. We first decompose the graph of O (at the terminological leel) into a set of p sub-graphs, in particular into trees that we call concept trees, one for each concept in O. Each tree is constructed by setting as its root the concept that it corresponds to and its children are taken to be the concepts that are directly connected to it with a relation in the ontology. As in the case of the instance trees we define a maximum leel up to which the tree can grow by recursiely following the relations defined for the children that are added to it. In the concept trees we ignore the direction of the relations as well as the edges corresponding to hierarchical (is-a) relations. A problem that arises while constructing the concept trees is that the graph of the ontology in general contains cycles. This obstacle can be oercome by following a methodology like the one used in [] and appropriately expanding the graph into an equialent tree, by traersing it in a specific order and breaking the circles by duplicating the nodes that form them. An example of a concept tree is shown in Fig.. Once the p concept trees hae been constructed, they are used as templates for the generation of the instance trees, by assigning a unique instance to each one of their nodes. Each concept tree will in general gie birth to seeral instance trees, not only because of the seeral instances aailable for its root concept (root instances), but also because the ontology relations may hae cardinalities higher than one. In this case, each edge in a concept tree will correspond to seeral instance pairs. In fact, for

4 a particular instance in the role of the root instance, the total number of instance trees that can be produced is equal to the product of the number of instances that can be assigned to each node in the corresponding concept tree. In the instance trees we assume that all edges hae the same label, i.e. that all the relations are equialent and that all edges hae direction towards the root of the tree. The instance trees are used as training data in order to build a recursie neural network classifier. The desired output for each instance tree is the concept in O its root belongs to. Assuming a two layer perceptron with a linear output actiation function as the encoding neural network, the state ector of node is computed as: ( ) X = σ A X + B L + C () q n where σ () is a sigmoid function and A R q m, B R q, C R are parameters to be learned and q is the number of hidden neurons. L is the label of the node and X a ector, function of the state ectors of its children. A similar equation holds for the neural network that computes the output of the super-node. Since for simplicity we hae considered all edge labels in the graphs to be the same, we can write for X : ch( ) X = D X ch () i () ch () i= n n where D R is a matrix of parameters. The parameters can be learned by the back-propagation through structure algorithm [3]. What remains to be defined are the labels L, which must be descriptie of the instances contents. In the simplest case we can consider as label space the space of the attribute alues of all instances in O and use as label for each instance its term frequency ector in this space. Person is-a owned by is-a Agency Actor acts in ordered Company Director directed Adertisment ordered acts in adertises Company Actor Product Director directed Adertisement adertises sells Product Agency sells Product Agency Fig.. Left: A graph representing an ontology of the adertising domain. The sub-graph that corresponds to the concept Adertisement and includes the concepts related to it with direct relations is marked with thick lines. Right: The corresponding concept-tree of leel two. After the classifier for the concepts of O has been trained, the same procedure is followed for O, from which p concept trees and the corresponding instance trees are generated. The instance trees of O are presented to the classifier, which classifies Ci them to one of the concepts of O. Let t be the number of instance trees of O Ci, Cj belonging to concept C i and t those of them that hae been assigned by the classifier to concept C. Gien that in general seeral instance trees correspond to j

5 C, C Ci i j the same root instance, the alues of t and t are normalized with respect to the number of instance trees that correspond to the same root instance. Then, assuming that the instances that we use are a representatie sample of the instance space of the two ontologies, we estimate the conditional probabilities: Ci, Cj ˆ t PC ( j Ci) = ij, (3) Ci t that an instance of O belonging to concept C i is also an instance of C j of O. We use this probability estimate as the similarity measure sc ( i, C j) of C i with C j. Other similar similarity measures can also be computed, like the Jaccard coefficient. The final output of the tool is a set of similarity pairs for all concepts O and O. partofday Schedule hasschedule showson has Day date TV Station scheduledat ondate broadcasts Genre genre category TV Program title time duration ownerof haspg Parental Guide category presenterin directorin actorin Person TV Station Schedule hasschedule ownerof partofday scheduledat broadcasts actorin TV Program Series Film Show actorin directorin presenterin attime Actor Director Owner Presenter Person Time date starttime endtime title pgcategory Fig.. Two ontologies for the TV programs domain. Left: ontology A, Right: ontology B. Table. Computed similarities for the concepts of ontology B with the concepts of ontology A. A Parental TV TV Day Schedule Person B Guide Program Station Genre Actor Presenter Director Owner Series Film Show Schedule Time Station The proposed method has been implemented and tested on small-scale datasets with promising initial results. As an example, the similarities of the concepts of the two ontologies of Fig. computed by our method are presented in Table. The recursie neural network classifier has been trained with instance trees of ontology B of leel, using as label space the stemmed words of the attribute alues, excluding proper s and numbers. The dataset was taken from [9]. The results are intuitiely correct; it is howeer worth noticing that the classifier does not produce ery high simi-

6 larity scores. This reflects the fact that some of the distinctie attributes of the domain concepts are distributed oer different ontology concepts in the two ontologies. The recursie neural network classifier takes thus into account such inter-concept dependencies. Moreoer, the classifier performs well with those instances, the concept to which they belong can correctly be determined only if the information about the instances with which they are related is also considered. For each instance this information is proided to the classifier through the corresponding instance tree. 4 Conclusions We described a machine learning ontology alignment tool based on the use of recursie neural networks. Our method exploits the ability of recursie neural networks to efficiently process structured data and builds a classifier which is used to estimate a distribution-based similarity measure between the concepts of two ontologies. Our research is ongoing and we are at the stage of configuring and ealuating our method, haing some promising initial results. There are seeral points where the suggested method may be improed. Particularly important is the definition of the label space of the instance trees. Currently, we use the attribute alues, but it is desirable to reduce the label space dimensionality by extracting more general labels. For this purpose the Wordnet ontology could e.g. be used to map the indiidual attribute alues to more general features, moing in this way the attribute alues closer to the abstract attributes they represent and improing the generalization properties of the classifier. References. Bianchini, M., Gori, M., Scarselli, F.: Recursie Processing of Directed Cyclic Graphs, In: Proc. IEEE Int. Conf. Neural Networks, (00) Doan, A., Madhaan, J., Domingos, P., Haley A.: Ontology Matching: A Machine Learning Approach, In: Staab S., Studer, R., (eds.): Handbook on Ontologies in Information Systems, Springer-Velag (004) Frasconi, P., Gori, M., Sperduti M.: A General Framework for Adaptie Processing of Data Structures, In: IEEE Trans. Neural Networks, Vol. 9:5 (997) Gori, M., Maggini, M., Sarti, L.: A Recursie Neural Network Model for Processing Directed Acyclic Graphs with Labeled Edges, In: Proc. Int. J. Conf. Neural Networks, Vol. (003) Melnik, S., Garcia-Molina, H., Rahm, E.: Similarity Flooding: A Versatile Graph Matching Algorithm, Extended Technical Report (00). 6. Noy, N., Musen, M. A.: PROMPT: Algorithm and Tool for Automated Ontology Merging and Alignment: In Proc. 7th Nat. Conf. Artificial Intelligence (000). 7. Noy, N. Musen, M. A.: Anchor PROMPT: Using Non-Local Context for Semantic Mapping, In Proc. Int. Conf. Artificial Intelligence (00). 8. Sperduti, A., Starita A.: Superised Neural Network for the Classification of Structures, In: IEEE Trans. Neural Networks, Vol. 8 (997) Web-based Knowledge Representation Repositories,