Detecting Dependencies between Unsatisfiable Classes

Detecting Dependencies between Unsatisfiable Classes Aditya Kalyanpur and Bijan Parsia and Evren Sirin University of Maryland College Park, USA aditya@cs.umd.edu, bparsia@isr.umd.edu, evren@cs.umd.edu Abstract Determining the cause of concept unsatisfiability in an ontology axiomatized in an expressive description logic is a difficult task. As the ontology grows larger and more complex, modelers may be driven to underconstrain their concepts in order to avoid problems. Part of the difficulty in correcting satisfiability bugs lies in the poverty of tool support typically, reasoner only report that a class is unsatisfiable, not why. The unsatisfiability also destroys much of the other information (such as position in the subsumption graph) which may have helped the modeler. The traditional, if underexplored, solution is to generate explanations of the problem using some features of the internal state of the reasoner, so called glass box approaches. This requires reasoners to be modified to support the explanation, which often adversely affects performance. In this paper, we explore black box techniques for debugging unsatisfiable concepts expressed in the Web Ontology Language (OWL), that is, techniques which use the reasoner solely to determine which concepts are unsatisfiable, but use the asserted structure of the ontology to help isolate the source of the problems. Introduction Now that OWL is a W3C Recommendation, one can expect that a much wider community of users and developers will be exposed to the expressive description logic SHIF(D) and SHOIN(D) which are the basis of OWL-DL. These users and developers are likely not to have a lot of experience with knowledge representation (KR), much less logic-based KR, much less description logic based KR. For such people, having excellent documentation, familiar techniques, and helpful tools is a fundamental requirement. A ubiquitous activity in programming is debugging, that is, finding and fixing defects in a program. Ontologies too have defects, and a common activity is to find and repair these defects. Unfortunately, the tool and training support for debugging ontologies is fairly weak. 1 We have chosen Copyright c 2005, American Association for Artificial Intelligence (www.aaai.org). All rights reserved. 1 While, historically, good KR modeling practices have been developed and described, often with an emphasis on description logics(rector et al. 2004), tool support for good modeling remains elusive, especially given the lack of consensus on practice and the strong dependence of goodness on application and domain specifics. to focus on debugging unsatisfiable concepts (and contradictory ABoxes) because contradictions, in general, seem analogous to fatal errors in programs. Debugging fatal errors in programs can be relatively straightforward: the program crashes, there is a stack trace or similar information, and (one measure of) success is a running program. Current tools do support indicating the dramatic failure of a unsatisfiable class, and success is similarly clear, however, supporting the diagnosis and resolution of the bug is not supported at all. In (Parsia, Sirin, & Kalyanpur 2005), we investigated better support for debugging unsatisfiable concepts using both glass box (wherein the reasoner is modified return explanations of the unsatisfiability) and black box (wherein the only inference service is satisfiability checking) techniques. In this paper, we extend our investigation of black box techniques which have two advantages over glass box ones: reasoner independence (you do not need a specialized, explanation generating reasoner) and avoiding the performance penalty of glass box techniques. Root and Derived Unsatisfiabilities We categorize unsatisfiable classes into two types: 1. Root Class - this is an unsatisfiable class in which a clash or contradiction found in the class definition (axioms) does not depend on the unsatisfiability of another class in the ontology. More specifically, the unsatisfiability bug for a root class cannot be fixed by simply correcting the unsatisfiability bug in some other class, instead, it requires fixing some contradiction stemming from its own definition. Root classes can be further subdivided into two categories: (a) Pure Root: This is a (root) class whose unsatisfiability depends strictly on its own definition (axioms), and not on the unsatisfiability of any other (root) class in the ontology. For example: Student ( 2)hasAdvisor ( 1)hasAdvisor (b) Mutually Dependent Root: This is a (root) class whose unsatisfiability depends on one or more distinct (root) classes in the ontology. For example: if you have two classes Male and F emale, which are asserted to be disjoint from one other, but also incorrectly defined as equivalents. In this case, both classes are mutually dependent roots with the problem lying in the equivalence

and disjoint axioms. 2. Derived Class - this is an unsatisfiable class in which a clash or contradiction found in a class definition either directly (via explicit assertions) or indirectly (via inferences) depends on the unsatisfiability of another class (we refer to it as the parent unsatisfiable class). Hence, this is a less critical bug in the sense that (in most cases) it can be resolved by fixing the unsatisfiability of this parent dependency. Note that there may be cases in which fixing the dependency bug reveals yet another unsatisfiability bug in the class, which needs to be resolved separately (making the derived class necessarily, but not sufficiently, dependent on the parent). Example of a derived class is: Class GraduateStudent Student, where Student is an unsatisfiable class itself, in this case, its parent. Automating root class discovery In this section, we present an algorithm to separate the root unsatisfiable classes from the derived ones given the total set of unsatisfiable classes in an ontology as provided by a reasoner. The algorithm consists of two parts: Structural Tracing and Root Pruning and we describe each in detail in the following subsections. For each unsatisfiable class in the ontology, the algorithm obtains its dependencies as follows: For a root class, it returns itself. For a derived class, it returns all the parent dependency classes along with the corresponding axioms that link the derived class to the parent. More specifically, the data structure returned is a set of tuples, where each tuple τ is recursively defined as: τ = (τ, axiom), and the fixed point of τ is: τ = (dep, axiom), where, dep is a set of dependency sets dep, such that each dep is a set of parent unsatisfiable classes that together cause the bug (could be a singleton set), and axiom = associated axiom linking current class to dependency set For example, τ(a) = ({{D}, equclaaxiom}, {{C, E}, subclaaxiom}) implies the following: A has an equivalent class axiom relating it to parent dependency D i.e. (solely D) is unsatisfiable makes A unsatisfiable. E.g., A = D ( 1p) A has a subclass axiom relating it to parents (C and E) i.e. (both C and E) are unsatisfiable makes A unsatisfiable. E.g., A (C E) Structural Tracing The tracing algorithm is used to determine structural dependencies between unsatisfiable classes. It is divided into three stages: Stage 1: Pre-processing - given a class definition (considering its equivalence and subclass axioms), we obtain a set of all property-value chains inherent in these axioms, which terminate in a universal value restriction ( ) on an unsatisfiable class. For example, consider the class definition A: A = p. q.(b ( r.c s.d)) If classes B, D are unsatisfiable, the following propertyvalue chains (allp C) are obtained for A: allp C = (p.q., B), (p.q.s., D o ). Notes: The class D has a subscript o to denote that a union (or non-deterministic branch) exists in its property chain (p.q.s), making this value restriction an optional requirement (to be handled later in the postprocessing stage). We also consider role hierarchy in constructing property-value chains, i.e., in the example above, if the property p has ancestor properties p 1, p 2, we expand the chains in our allp C set to: ((p/p 1 /p 2 ).q., B), ((p/p 1 /p 2 ).q.s., D o ) The intuition for this stage is as follows: universal value restrictions on a property must be satisfied iff the property exists i.e. the class definition entails a 1 cardinality restriction on the property. In Stage 2 (dependency-tracing of a particular class), each time we discover the existence of a property, we check the allp C chains to ensure that the associated universal restriction is satisfied. However, note that we need to determine the set of allp C chains beforehand, since the non-localization of the class definition makes it difficult to verify all universal restrictions during tracing directly. Stage 2: Dependency-tracing - a recursive set of methods are used to extract all dependency unsatisfiable classes and the adjoining axioms given the original class definition. The output contains a mixture of definite and optional dependency cases. The basic tenets of the tracing approach are as follows: Class A is a derived unsatisfiability if: 1. A is equivalentto/subclassof an intersection set, any of whose elements are unsatisfiable, i.e., A = (B C.. D), and one of B,C..D is unsatisfiable (any such unsatisfiable class becomes its parent) 2. A is equivalentto/subclassof a union set, all of whose elements are unsatisfiable, i.e., A = (B C.. D), and all B,C..D are unsatisfiable (all such unsatisfiable classes become its parents) 3. A has an existential ( ) property restriction on an unsatisfiable class, i.e., A = (p, B) and B is unsatisfiable (B becomes its parent) 4. A entails a ( 1) cardinality restriction on a propertychain, and the universal ( ) value restriction on that chain is not satisfied (object/value of property chain becomes its parent) 5. A entails a ( 1) cardinality restriction on a property, and the domain of the property is unsatisfiable, i.e., A ( 1p), domain(p) = B, and B is unsatisfiable, making it the parent of A (similar domain check has to be made for every ancestor property of p)

6. A entails a ( 1) cardinality restriction on an object property, and the range of its inverse is unsatisfiable, i.e., A ( 1p), range(p ) = B, and B is unsatisfiable, making it the parent of A (similar range check has to be made for every ancestor property of p ) Notes: A key caveat of the tracing algorithm not mentioned in the pseudo code is: each time we find a pair of mutually-dependent derived unsatisfiabilities such that the cause for the dependency is an equivalence axiom, its unclear whether one of the classes is actually a parent of the other. E.g., if we have the equivalent class axiom A = B in the ontology and both classes, A and B, are unsatisfiable, its unclear whether A depends on B or vice versa. In this case, we simply remove the equivalence axiom from the ontology and test the satisfiability of A and B again using the reasoner. This gives us three possible outcomes: 1. Both classes become satisfiable: This implies that there is a strong interdependence between the two classes which makes each class unsatisfiable (e.g., A B). Hence we mark them both as roots. 2. One class becomes satisfiable while the other remains unsatisfiable: This implies that the former is derived from the latter (parent) class. 3. Both classes stay unsatisfiable: We run the structural tracing algorithm on each class again to find alternate distinct dependencies. The tracing approach described above does not consider nominals or transitive property relations. Hence, it can be considered as a heuristic to prune unsatisfiability results, which is sound (i.e. if it finds a derived unsatisfiability, its accurate based on the definition provided earlier), but not complete (does not find all derived unsatisfiable classes). Stage 3: Post-processing - if the final dependency set contains an optional unsatisfiable class C o, we check if adjoining (siblings within the same nested set) dependencies are definite i.e. without an o tag. If they are, we simply remove the optional dependency C o ; else (if C o is the sole dependency) we transform it to a C getting rid of the optional tag. We do this recursively, until all the optional unsatisfiable classes are either pruned out or transformed in the final dependency set. For example, if the final dependency set for a class is: dep = ({{C o, D}, equclaaxiom}, {{E o }subclaaxiom}) we reduce it to: dep = ({{D}, equclaaxiom}, {{E}, subclaaxiom}) We consider a few cases to explain the working of the entire structural tracing algorithm (especially highlighting the significance of the pre- and post-processing stages). In all cases below, A, B, C, D are named OWL classes (as opposed to complex class expressions) and p,q are OWL properties. Case 1: Given two axioms defining class A: (suppose A, B, C, D all unsatisfiable) (i)a (( p.c) B ( q.d)) (ii)a (( 1p) D) After pre-processing, we get the following propertyvalue chains: allp C = {(p.c)} During structural tracing of axiom (i), we obtain the following dependencies {{B}, {D}, (i)} (last element being the axiom pointer). B appears independently as its unsatisfiable and part of the intersection set, and D appears independently because the existential property restriction on q forces the unsatisfiable class D to be A s successor. Note that all p.c restrictions are ignored in this stage (read below to see how they are accounted for). Structural tracing of axiom (ii) returns {{C, D}, (ii)}. The {C} appears when the restriction on p (as stored in allp C) is checked because of the ( 1p) restriction, and D appears since its part of the union set. Note here that both C and D appear together in one set (as opposed to two independent elements as in the previous axiom) because union semantics of axiom (ii) imply that A is unsatisfiable if both C and D are unsatisfiable. In the post-processing stage, no additional transformations take place on the dependency sets because there are no optional dependencies to consider. Thus the final dependency sets for A are {{B}, {D}, (i)} and {{C, D}, (ii)}, which imply that fixing A requires correcting the unsatisfiability bugs in B and D. You don t need to correct C since its part of the union set with D. Case 2: Given two axioms defining class A: (suppose A, B, C, D all unsatisfiable) (i)a (( 1p) B) (ii)a (( p.c) ( q.d)) After pre-processing, we get the following propertyvalue chains: allp C = {(p.c o ), (q.d o )} (the o subscript denoting that the chain is optional as it terminates inside a union) During structural tracing of axiom (i), we obtain the following dependencies {{C o }, {B}, (i)}. The {C o } appears when the restriction on p (as stored in allp C) is checked because of the ( 1p) restriction, and B appears directly as its part of the intersection set. Structural tracing of axiom (ii) returns nothing since restrictions are basically ignored in this stage. In the final post-processing stage, since C o has a sibling B which is a definite root, we remove C o from the dependency set giving us just {{B}, (i)}. This is consistent with the model, since fixing the unsatisfiability of B will make class A satisfiable as well. Case 3: Given two axioms defining class A: (suppose A, C, D are unsatisfiable) (i)a (( 1p) ( 1q)) (ii)a (( p.c) ( q.d)) After pre-processing, we get the same property-value chains as in Case 2: allp C = {(p.c o ), (q.d o )} During structural tracing of axiom (i) we obtain the following dependencies {{C o }, {D o }, (i)} for similar reasons as explained in Case 2. Again structural tracing of axiom (ii) returns nothing since restrictions are basi-

cally ignored in this stage. In the final post-processing stage, since neither C o nor D o have definite root siblings we randomly remove the optional tag on any root, say C, and now since D o has a definite root sibling, we remove it from the dependency set giving us {{C}, (i)}. This is consistent with the model, since fixing the unsatisfiability of either C (or D as the case maybe) will make class A satisfiable as well. Complexity Analysis of Tracing : The recursive tracing algorithm for a given class explores the class definition fully by considering its equivalence and subclass axioms and probing all nested class expressions in each axiom. During tracing, any other structurally linked class is unfolded (i.e. replaced by its definition) iff its not unsatisfiable. This makes the tracing cross strict class definition boundaries per se. Yet though full expansion of a terminology is known to be exponential, since we deal with expansion only for a single class (with limited unfolding), the average case complexity is much better than the worst case. Drawbacks of Tracing : One of the main drawbacks of the structural tracing algorithm is that it does not consider inferred equivalence or subsumption between root classes. Consider two classes A and B that do not have an explicit subsumption relation between them but the reasoner can infer one, e.g., A = ( 1p) and B = ( 2p). Even though there is no subclass axiom relating the two classes, a reasoner can infer that B A (provided of course, that both are satisfiable). However, if the tracing algorithm returns both classes as root, it cannot find the hidden dependency of B on A. In this case, even using a reasoner to infer the subsumption relation will not work as both classes are unsatisfiable and hence effectively equivalent to the bottom concept, owl : Nothing. As a result, we need an alternate way to prune the root classes further by finding these hidden dependencies. Pruning potential roots We are working on two orthogonal approaches to detect hidden dependency (equivalence and/or subsumption) between potential root classes (root p ) found at the end of the structural tracing. The first is a simple, but optimized, brute force approach that involves removing a set of roots p from the ontology and testing the satisfiability of the remaining roots p. The intuition here is that if we can reduce the original set of roots p to the largest subset S, which satisfies the following condition: removing a specific class, say R, from S renders the remaining classes satisfiable, then we can conclude that R is a root and the remaining classes in S have this root as one of their dependencies. The second approach is a rather non-standard one: we modify the original ontology to remove all contradictions while trying to preserve the equivalence and/or subsumption relations as much as possible. The idea here, very similar in principle to concept approximation (Brandt, R, & Turhan 2002), is to bring the KB to a consistent state (by reducing its expressivity) upon which a reasoner can perform classification (note that the initial problem with using a reasoner to detect hidden dependencies was due to the unsatisfiability of the classes). Thus, after approximating the KB, we use the reasoner to classify only the potential root classes to discover any equivalence/subsumption relations between them and filter out all dependencies from the final root set. In this paper, we only present the details of the first approach as we are still investigating the second approach fully. Brute-force Pruning Before we present the details of this approach, we need to specify what we mean by removing a class from the ontology: here, removal of class C implies getting rid of all definitions of the class from the ontology, i.e., axioms in which C appears solely on the LHS. Now, consider the following example. Suppose structural tracing identifies the following potential root classes : {R 1, R 2, R 3, C 1, C 2, C 3, C 4 } such that {R 1, R 2, R 3 } are the actual roots (each mutually dependent) and {C 1, C 2, C 3, C 4 } are derived from the roots based on the following (non-asserted) dependencies: C 3 C 1 C 4 R 1 C 3 R 2 C 2 R 2 C 4 R 3 An algorithm for iteratively pruning the actual roots is as follows: 1. L = linear list of potential roots; 2. roots = {}; 3. repeat iteration { 4. dependents = {}; 5. L = L; 6. for each potential root, R, in L, { 7. remove R from the ontology; 8. L = L - {R}; 9. if (R dependents) continue; 10. boolean allsatisfiable = true; 11. for each class C in L, 12. if (C is not satisfiable), { 13. allsatisfiable = false; 14. break; 15. } 16. else 17. dependents += {C}; 18. if (allsatisfiable) roots += {R}; 19. } 20. L = L - roots - dependents; 21. (reorder L randomly if needed) 22. } 23. until no more roots found (i.e. L contains all satisfiable classes)

Explanation: One iteration of the algorithm takes a set of unsatisfiable classes and identifies a single root in it by gradually reducing it to one in which all classes are satisfiable. Thus, we have the following condition: given a set of unsatisfiable classes L, we split it into two parts, L 1 and L 2, such that: 1. removing L 1 from the ontology makes L 2 satisfiable, and 2. L 1 is minimal, i.e., there exists no other partitioning of the set L = {L 1, L 2} satisfying (1) such that L 1 L 1 Now, if R is the class at the boundary of the partition, i.e., R is the last class to be added to L 1 to satisfy the above two conditions, then we can infer the following: 1. R is a root class since removing it (along with the previously removed classes in L 1 ) from the ontology makes L 2 completely satisfiable. Note: L 1 (which contains R) should be minimal for this to hold 2. Conversely, each class in L 2 is a derived class which has atleast one root dependency: R (there may be additional dependencies on other classes in L 1 ) Since the above root identification process is generic for any list of unsatisfiable classes, we can remove the root R from the ontology and repeat the above process iteratively to prune out all the roots. An important optimization in the algorithm above is to track dependent sets dynamically, i.e., if during an intermediate partitioning stage, we find that a class becomes consistent, we tag it as a dependent class and skip it during later partitioning stages, thereby preventing numerous unnecessary satisfiability checks. To see how the algorithm works with the above case, consider an initial (random) ordering of potential roots: L = {C 4, R 3, C 1, R 1, C 3, R 2, C 2 }. The iterations of the root pruning process are shown in the table below. The total no. of satisfiability tests needed to identify all three roots {R 1, R 2, R 3 } is 11. Unsat. Class List / Partitions Roots Dependent L = {C 4, R 3, C 1, R 1, C 3, R 2, C 2 } R 2 C 1, C 2 L 1 = {C 4, R 3, C 1, R 1, C 3 } + {R 2 } L 2 = {C 2 } No. of Sat Tests = 6 L = {C 4, R 3, R 1, C 3 } R 1 C 3 L 1 = {C 4, R 3 } + {R 1 } L 2 = {C 3 } No. of Sat Tests = 3 L = {C 4, R 3 } R 3 - L 1 = {C 4 } + {R 3 } L 2 = {} No. of Sat Tests = 1 L = {C 4 } - C 4 No. of Sat Tests = 1 Evaluation: The Tambis Ontology For the purpose of evaluation, we needed an expressive, moderately-sized OWL ontology that had a large number of unsatisfiable classes. The Tambis ontology (Baker et al. 1998) fit our needs well - its an OWL DL ontology containing 395 Classes and its expressivity is SHIN. Moreover, the OWL version 2 was generated by a conversion script and a number of errors crept in during that process 144 unsatisfiable classes in all. Many of the unsatisfiable classes depend in simple ways on other unsatisfiable classes, so that a brute force going down the list correcting each class in turn is unlikely to produce correct results, or, at best, will be pointlessly exhausting. In one case, three changes repaired over seventy other unsatisfiable classes. Given the highly non-local effects of assertions in a logic like OWL, it is not sufficient to take on defects in isolation. We implemented the black-box techniques in the Swoop OWL ontology editor 3, used the default DL Tableaux Reasoner, Pellet, and carried out the analysis and debugging of Tambis. Running the structural tracing algorithm on the unsatisfiable classes in Tambis identified 111 derived classes and 33 potential roots. This was a significant result, the problem space was pruned by more than 75% enabling us to direct our attention on a narrow set of unsatisfiabile classes, and moreover, for each derived unsatisfiabile class, we obtained the dependency relation (via axioms) leading to its corresponding roots, which were presented in the Swoop UI. Out of the remaining 33 potential roots, we applied the brute-force root pruning technique (which performed an additional 51 satisfiability checks) and reduced the root set to just 2! This was both, a surprising and interesting result, and due in fact to the inferred equivalence between a large number of potential roots. The 33 classes all shared the same structure (defined equivalent to the same intersection set) out of which 2 were actual roots, (any pair from the set {metal, nonmetal, metalloid} 4 ), each asserted as mutually disjoint in the ontology thus causing the contradiction; while the remaining 30 classes were all inferred to be equivalent to the above 3 classes making them unsatisfiable as well. Thus, root pruning was able to reduce the problem space of the ontology even further, demonstrating the effectiveness of black-box techniques for debugging ontologies. The next step we intend working on is pinpointing problematic axioms in the ontology, a problem which is simplified now that we have identified the small set of root unsatisfiable classes. Related Work To our knowledge, black-box debugging to find and explain dependencies between unsatisfiable classes is a largely unexplored topic. The closest work we know of is (Schlobach & Cornet 2003), who propose non-standard reasoning algorithms (for ALC TBoxes) based on minimization of axioms using Boolean methods, and demonstrate promising results on the DICE terminology. Their approach which deals with axiom and concept pinpointing is related to our work, though they rely on glass box techniques as well. 2 http://www.cs.man.ac.uk/ horrocks/owl/ontologies/tambisfull.owl 3 http://www.mindswap.org/2004/swoop 4 Fixing any pair of unsat. classes from the set makes the third class satisfiable

Conclusion The black box debugging techniques described in this paper focus on separating the root from the derived unsatisfiable classes allowing the modeler to focus solely on the problematic parts of the ontology. Evaluation performed on the Tambis ontology has given us promising preliminary results. The structural tracing algorithm helped prune out a large chunk of unsatisfiable classes (111/144) based on simple (direct), as well non-local (indirect) dependencies on root classes; while further root pruning (30/33) helped reveal hidden equivalence/subsumption between potential roots. The Tambis use case scenario is quite reasonable, given that automated scripts for converting ontologies or schemas to OWL are likely to introduce errors, and modelers keen on using the rich expressivity of OWL-DL require good debugging support to fix them. References Baker, P.; Brass, A.; Bechhofer, S.; Goble, C.; Paton, N.; and Stevens, R. 1998. TAMBIS: Transparent Access to Multiple Bioinformatics Information Sources. An Overview. 25 34. Brandt, S.; R, K.; and Turhan, A. 2002. Approximation and difference in description logics. Proceedings of the Knowledge Representation KR02. Parsia, B.; Sirin, E.; and Kalyanpur, A. 2005. Debugging owl ontologies. In The 14th International World Wide Web Conference (WWW2005). Rector, A.; Drummond, N.; Horridge, M.; Rogers, J.; Knublauch, H.; Stevens, R.; Wang, H.; and Wroe, C. 2004. Owl pizzas: Common errors & common patterns from practical experience of teaching owl-dl. In European Knowledge Acquisition Workshop (EKAW). Schlobach, S., and Cornet, R. 2003. Non-standard reasoning services for the debugging of description logic terminologies. Proceedings of IJCAI, 2003.