IR Models based on predicate. logic. Norbert Fuhr, Henrik Nottelmann University of Duisburg-Essen. POOL: a probabilistic object-oriented logic

Transcription

1 IR Models based on predicate logic Norbert Fuhr, Henrik Nottelmann University of Duisburg-Essen Description logic Datalog Probabilistic Datalog POOL: a probabilistic object-oriented logic Mapping OWL onto Datalog Norbert Fuhr; Henrik Nottelmann 1/57

2 limitations of propositional logic: conventional indexing (based on propositional logic): d = {tree, house} query: Is there a picture with a tree on the left of the house? query cannot be expressed in propositional logic predicate logic: d: tree(t1). house(h1). left(h1,t1).?- tree(x) & house(y) & left(x,y). Norbert Fuhr; Henrik Nottelmann 2/57

3 1 Description logic 1.1 Thesaurus polygon regular polygon triangle quadrangle... rectangle regular triangle square Norbert Fuhr; Henrik Nottelmann 3/57

4 thesaurus knowledge: can be expressed in propositional logic square = quadrangle regular-polygon description logic based on semantic networks more expressive than thesauri instances of concepts roles between (instances of) concepts Norbert Fuhr; Henrik Nottelmann 4/57

5 1.2 Introduction into OWL Norbert Fuhr; Henrik Nottelmann 5/57

6 Semantic Web (ontology) languages RDF: Resource description language, semantic markup language, only resources and their properties, serialisation in XML RDFS: RDF Schema, schema definition language for RDF OWL: extends RDF/RDFS by richer modelling primitives, OWL Lite/DL/Full OWL Lite contains simple primitives OWL DL corresponds to expressive description logic OWL Full is OWL DL + RDF knowledge base can be modelled as collection of RDF triples (RDF/XML serialisation) alternative encoding: abstract syntax (easier to read) Norbert Fuhr; Henrik Nottelmann 6/57

7 Objects, classes, literals and datatypes Two distinct domains: Classes: for objects Data types: for literals owl:class owl:datatype rdf:type rdf:type Person xsd:decimal rdf:type rdf:type Peter 1,89 Norbert Fuhr; Henrik Nottelmann 7/57

8 Classes (1) owl:class rdf:type Animal Person rdfs:subclassof rdfs:subclassof rdfs:subclassof owl:disjointwith rdfs:subclassof Female Male Man Class(Female partial Animal) <owl:class rdf:id="female"> <rdfs:subclassof rdf:resource="#animal"/> </owl:class> Norbert Fuhr; Henrik Nottelmann 8/57

9 Classes (2) owl:class rdf:type Animal Person rdfs:subclassof rdfs:subclassof rdfs:subclassof owl:disjointwith rdfs:subclassof Female Male Man Class(Male partial Animal) DisjointClasses(Male Female) <owl:cass rdf:id="male"> <rdfs:subclassof rdf:resource="#animal"/> <owl:disjointwith rdf:resource="#female"/> </owl:class> Norbert Fuhr; Henrik Nottelmann 9/57

10 Object properties (1) owl:objectproperty rdf:type rdfs:domain rdfs:range Animal hasparent Animal owl:subpropertyof rdfs:range hasfather Male ObjectProperty(hasParent domain(animal) range(animal)) <owl:objectproperty rdf:id="hasparent"> <rdfs:domain rdf:resource="#animal"/> <rdfs:range rdf:resource="#animal"/> </owl:class> Norbert Fuhr; Henrik Nottelmann 10/57

11 Object properties (2) owl:objectproperty rdf:type rdfs:domain rdfs:range Animal hasparent Animal owl:subpropertyof rdfs:range hasfather Male ObjectProperty(hasFather super(hasparent) range(male)) <owl:objectproperty rdf:id="hasfather"> <rdfs:subpropertyof rdf:resource="#hasparent"/> <rdfs:range rdf:resource="#male"/> </owl:class> Norbert Fuhr; Henrik Nottelmann 11/57

12 Datatype properties owl:functionalproperty owl:datatypeproperty rdf:type rdf:type rdfs:domain rdfs:range Person shoesize xsd:decimal DatatypeProperty(shoesize Functional domain(animal) range(xsd:decimal)) <owl:datatypeproperty rdf:id="shoesize"> <rdfs:domain rdf:resource="#animal"/> <rdfs:range rdf:resource="xsd:decimal"/> <rdf:type rdf:resource="owl:functionalproperty"/> </owl:class> Norbert Fuhr; Henrik Nottelmann 12/57

13 Property restrictions Class(Person partial Animal restriction(hasparent allvaluesfrom(person)) restriction(hasparent cardinality(2))) <owl:class rdf:id="person"> <rdfs:subclassof rdf:resource="#animal"/> <rdfs:subclassof> <owl:restriction> <owl:onproperty rdf:resource="#hasparent"/> <owl:allvaluesfrom rdf:resource="#person"/> </owl:restriction> </rdfs:subclassof> <rdfs:subclassof> <owl:restriction> <owl:onproperty rdf:resource="#hasparent"/> <owl:cardinality>2</owl:cardinality> </owl:restriction> </rdfs:subclassof> </owl:class> Norbert Fuhr; Henrik Nottelmann 13/57

14 Instances Individual(Kain type(male) value(hasfather Adam) value(hasmother Eve) value(shoesize 10)) <Male rdf:id="kain"> <hasfather rdf:resource="#adam"/> <hasmother rdf:resource="#eve"/> <shoesize>10</shoesize> </Male> Norbert Fuhr; Henrik Nottelmann 14/57

15 Further modelling primitives owl:sameclassof,owl:samepropertyof: subpropertyof rdfs:subclass/rdfs:subpropertyof owl:inverseof: inverse property: p(a, b) r(b, a) owl:transitiveproperty: p(a, b), p(b, c) p(a, c) owl:symmetricproperty: p(a, b) p(b, a) owl:inversefunctionalproperty: inverse property is functional owl:hasvalue at least one property value equals object or datatype value owl:somevaluesfrom at least one property value is instance of class expression or datatype owl:intersectionof, owl:unionof, owl:complementof: boolean combinations of class expressions owl:oneof: define class by enumerating its instances Norbert Fuhr; Henrik Nottelmann 15/57

16 Problems with OWL OWL lacks support for uncertainty: only deterministic relationships possible, no weighting or probabilistic facts Pr(hasFather(lisa,thomas))=0.9 cannot be expressed rules: no general rules, only specific rules like subclassof, TransitiveProperty... if hasparent(a,b) and hasparent(c,d) and hassibling(b,d), then hascousion(a,c) cannot be expressed n-ary datatype predicates: OWL datatypes are based on XML Schema datatypes, thus providing only unary datatype predicates cannot be expressed IR queries cannot be expressed directly in OWL Norbert Fuhr; Henrik Nottelmann 16/57

17 OWL: Conclusion OWL extends RDF(s) by additional modelling primitives well-defined semantics, based on description logics does not support all RDF features (no reification, only three levels owl:class, classes and objects) lacks important features: only deterministic features, no probabilistic relationships no rules (should appear in SWRL) restricted datatype predicates (due to XML Schema) OWL and associated languages become standard in the Semantic Web Norbert Fuhr; Henrik Nottelmann 17/57

18 2 Modelling IR in Datalog Norbert Fuhr; Henrik Nottelmann 18/57

19 2.1 Introduction Datalog: horn predicate logic no functions restricted forms of negation allowed sound and complete evaluation algorithms docterm(d1,ir). docterm(d2,ir). docterm(d1,db). docterm(d2,oop).?- docterm(d,ir).?- docterm(d,ir) & docterm(d,db).?- docterm(d,ir) & not(docterm(d,db)). Norbert Fuhr; Henrik Nottelmann 19/57

20 2.2 Hypertext structure docterm(d1,ir). docterm(d1,db). link(d1,d2). link(d2,d3). link(d3,d1). about(d,t) :- docterm(d,t). about(d,t) :- link(d,d1) & about(d1,t). d3 docterm d1 d2 link ir db?- about(d,ir) Norbert Fuhr; Henrik Nottelmann 20/57

21 2.3 Aggregation book chapter section part(d,p) :- chapter(d,p). part(d,p) :- section(d,p). retrieve node if at least one part is about the search term: about(d,t) :- part(d,p) & about(p,t). retrieve node if all its parts are about the search term: about(d,t) :- part(d,x) & about(x,t) & not(anypart(d,t)). anypart(d,t):- part(d,p)& not(about(p,t)). Norbert Fuhr; Henrik Nottelmann 21/57

22 3 Probabilistic Datalog Norbert Fuhr; Henrik Nottelmann 22/57

23 3.1 Motivation powerful retrieval logic expressiveness: traditional IR models: propositional logic predicate logic recursion (structured documents, hypertext links, terminological structures) uncertain inference: probabilistic inference multimedia retrieval logical basis for high-level representation languages integration of information retrieval and database systems Norbert Fuhr; Henrik Nottelmann 23/57

24 3.1.1 Syntax ground facts with probabilistic weights 0.9 docterm(d1,ir). 0.5 docterm(d1,db). 0.8 docterm(d2,ir). 0.3 docterm(d2,oop).?- docterm(d,ir). gives d1 0.9 d2 0.8?- docterm(d,ir) & docterm(d,db). gives d Norbert Fuhr; Henrik Nottelmann 24/57

25 3.2 Semantics of probabilistic Datalog Extensional vs. intensional semantics 0.9 docterm(d1,ir). 0.5 docterm(d1,db). 0.7 link(d2,d1). about(d,t) :- docterm(d,t). about(d,t) :- link(d,d1) & about(d1,t) q(d) :- about(d,ir) & about(d,db). extensional semantics: weight of derived fact as function of weights of subgoals P(q(d2)) = P(about(d2,ir)) P(about(d2,db)) = ( ) ( ) Problem: improper treatment of correlated sources of evidence [Pearl] extensional semantics only correct for tree-like inference structures intensional semantics: weight of IDB fact as function of weights of underlying ground facts Norbert Fuhr; Henrik Nottelmann 25/57

26 3.2.2 Implementation of intensional semantics event keys and event expressions 0.9 docterm(d1,ir). [dt(d1,ir)] 0.5 docterm(d1,db). [dt(d1,db)] 0.7 link(d2,d1). [l(d2,d1)]?- docterm(d,ir) & docterm(d,db). gives d1 [dt(d1,ir) & dt(d1,db)] = 0.45 Norbert Fuhr; Henrik Nottelmann 26/57

27 about(d,t) :- docterm(d,t). about(d,t) :- link(d,d1) & about(d1,t)?- about(d,ir) & about(d,db). gives d2 [l(d2,d1) & dt(d1,ir) & l(d2,d1) & dt(d1,db)] = d1 [dt(d1,ir) & dt(d1,db)] = 0.45 Norbert Fuhr; Henrik Nottelmann 27/57

28 about(d,t) :- docterm(d,t). about(d,t) :- link(d,d1) & about(d1,t). d docterm 0.5 d1 d2 link ir db?- about(d,ir) d1 [dt(d1,ir) l(d1,d2) & l(d2,d3) & l(d3,d1) & dt(d1,ir)...] d3 [l(d3,d1) & dt(d1,ir)] Norbert Fuhr; Henrik Nottelmann 28/57

29 d2 [l(d2,d3) & l(d3,d1) & dt(d1,ir)] 0.288?- about(d,ir) & about(d,db) d1 [dt(d1,ir) & dt(d1,db)] d3 [l(d3,d1) & dt(d1,ir) & l(d3,d1) & dt(d1,db)] d2 [l(d2,d3) & l(d3,d1) & dt(d1,ir) & dt(d1,db)] Norbert Fuhr; Henrik Nottelmann 29/57

30 computation of probabilities for event expressions 1. transformation of expression into disjunctive normal form 2. application of sieve formula: ci conjunct of event keys P(c1... cn) = n ( 1) i 1 i=1 1 j1<...< ji n P(c j1... c ji ). Norbert Fuhr; Henrik Nottelmann 30/57

31 3.2.3 Interpretation of probabilistic weights possible worlds semantics 0.9 docterm(d1,ir). P(W1) = 0.9: {docterm(d1,ir)} P(W2) = 0.1: {} Norbert Fuhr; Henrik Nottelmann 31/57

32 0.9 docterm(d1,ir). 0.5 docterm(d1,db). possible interpretations: I1: P(W1) = 0.45: {docterm(d1,ir)} P(W2) = 0.45: {docterm(d1,ir), docterm(d1,db)} P(W3) = 0.05: {docterm(d1,db)} P(W3) = 0.05: {} I2: P(W1) = 0.5: {docterm(d1,ir)} P(W2) = 0.4: {docterm(d1,ir), docterm(d1,db)} P(W3) = 0.1: {docterm(d1,db)} I3: P(W1) = 0.4: {docterm(d1,ir)} Norbert Fuhr; Henrik Nottelmann 32/57

33 P(W2) = 0.5: {docterm(d1,ir), docterm(d1,db)} P(W3) = 0.1: {} probabilistic logic: 0.4 P(docTerm(d1, ir)&docterm(d1, db)) 0.5 probabilistic Datalog with independence assumptions: P(docTerm(d1, ir)&docterm(d1, db)) = 0.45 Norbert Fuhr; Henrik Nottelmann 33/57

34 Disjoint events example: imprecise attribute values # py(dk,av). 0.2 py(d3,89). 0.7 py(d3,90). 0.1 py(d3,91). interpretation: P(W1) = 0.2: {py(d3,89)} P(W2) = 0.7: {py(d3,90)} P(W3) = 0.1: {py(d3,91)}?- py(x,y) & Y > 89. d3 [p(d3,90) p(d3,91)] = 0.8 Norbert Fuhr; Henrik Nottelmann 34/57

35 Probabilistic search term weighting via disjoint events 0.8 docterm(d1,db). 0.7 docterm(d1,ir). # qtw(av). 0.4 qtw(db). 0.6 qtw(ir). s(d) :- qtw(x) & docterm(d,x).?- s(d). Norbert Fuhr; Henrik Nottelmann 35/57

36 0.4 qtw(db) 0.6 qtw(ir) 0.7 docterm(d1,ir) 0.8 docterm(d1,db) d1 [q(db) & dt(d1,db) q(ir) & dt(d1,ir)] = 0.74 Norbert Fuhr; Henrik Nottelmann 36/57

37 3.2.4 Probabilistic rules rules for deterministic facts: 0.7 likes-sports(x) :- man(x). 0.4 likes-sports(x) :- woman(x). man(peter). interpretation: P(W1) = 0.7: {man(peter), likes-sports(peter)} P(W2) = 0.3: {man(peter)} Norbert Fuhr; Henrik Nottelmann 37/57

38 rules for uncertain facts: # sex(dk,av). 0.7 l-s(x) :- sex(x,male). 0.4 l-s(x) :- sex(x,female). 0.5 sex(x,male) :- human(x). 0.5 sex(x,female) :- human(x). human(peter). interpretation: P(W1) = 0.35: {sex(peter,male), l-s(peter)} P(W2) = 0.15: {sex(peter,male)} P(W3) = 0.20: {sex(peter,female), l-s(peter)} P(W4) = 0.30: {sex(peter,female)} Norbert Fuhr; Henrik Nottelmann 38/57

39 sameauthor(d1,d2) :- author(d1,x) & author(d2,x). 0.5 link(d1,d2) :- refer(d1,d2). 0.2 link(d1,d2) :- sameauthor(d1,d2).?? link(d1,d2) :- refer(d1,d2) & sameauthor(d1,d2). P(l r), P(l s) P(l r s)? Norbert Fuhr; Henrik Nottelmann 39/57

40 0.7 link(d1,d2) :- refer(d1,d2) & sameauthor(d1,d2). 0.5 link(d1,d2) :- refer(d1,d2) & not(sameauthor(d1,d2)). 0.2 link(d1,d2) :- sameauthor(d1,d2) & not(refer(d1,d2)). probabilistic inference networks, rules define link matrix refer sameauthor link Norbert Fuhr; Henrik Nottelmann 40/57

41 3.2.5 Vague predicates pc(m1,486/dx50,8,540,900). pc(m2,pe60,16,250,1000). pc(m3,pe90,16,540,1100).?- pc(mod, CPU, MEM, DISK, PRICE), PRICE < 1000 vague predicate ˆ< (builtin) 1.00 ˆ<(900,1000) 1.00 ˆ<(950,1000) 0.99 ˆ<(1000,1000) 0.90 ˆ<(1050,1000) 0.60 ˆ<(1100,1000)?- pc(mod, CPU, MEM, DISK, PRICE), PRICE ˆ< pc(m1,486/dx50,8,540,900) pc(m2,pe60,16,250,1000) pc(m3,pe90,16,540,1100). Norbert Fuhr; Henrik Nottelmann 41/57

42 applications of vague predicates: vague fact conditions proper name search (string similarity) (also OCRed text) multimedia IR (e.g. audio retrieval, image retrieval) Norbert Fuhr; Henrik Nottelmann 42/57

43 4 Retrieval of structured documents: POOL goals: retrieval of structured documents hierarchical logical structure abstraction from node types contexts as untyped nodes multimedia retrieval expressiveness of restricted predicate logic Norbert Fuhr; Henrik Nottelmann 43/57

44 4.1 Structure of POOL programs object: identifier + content context: object with nonempty content (a1, s11, s12) program: set of clauses clause: context / proposition / rule proposition: term (image, presentation) classification (article(a1), section(s11)) attribute (s11.author(smith), a1.pubyear(1997)) Norbert Fuhr; Henrik Nottelmann 44/57

45 example: a1[ s11[ image 0.6 retrieval presentation ] s12[ ss121[ audio indexing ] ss122[ video not presentation ] ] ] s11.author(smith) s121.author(miller) s122.author(jones) a1.pubyear(1997) article(a1) section(s11) section(s12) subsection(ss121) subsection(ss122) Norbert Fuhr; Henrik Nottelmann 45/57

46 rule: head :- body head: proposition / context containing a proposition body conjunction of subgoals (propositions or contexts) docnode(d) :- article(d) docnode(d) :- section(d) docnode(d) :- subsection(d) mm-ir-doc(d) :- docnode(d) & D[audio & retrieval] german-paper(d) :- D.author.country(germany) query:?- body?- D[audio & indexing] Norbert Fuhr; Henrik Nottelmann 46/57

47 4.2 Contexts and augmentation clauses only hold for context where stated augmentation: propagation of propositions to surrounding contexts a1[ s11[ image 0.6 retrieval presentation ] s12[ ss121[ audio indexing ] ss122[ video not presentation ] ] ]?- D[audio & video] s12 a1 Norbert Fuhr; Henrik Nottelmann 47/57

48 augmentation with uncertainty:?- audio 1.00 ss s a1?- D[audio & video] 0.36 s a1 augmentation with uncertainty prefers most specific context! Norbert Fuhr; Henrik Nottelmann 48/57

49 4.3 Augmentation and inconsistencies d1[ s1[ audio indexing ] s2[ s21[ image retrieval] s22[ video not retrieval ] ] ]?- D[audio & indexing] s1 d1?- D[video & image] s2 d1?- D[video & retrieval] retrieval is inconsistent in s2 Norbert Fuhr; Henrik Nottelmann 49/57

50 four-valued logic truth values: unknown, true, false, inconsistent s22: video true image unknown retrieval false s2: image true video true retrieval inconsistent Norbert Fuhr; Henrik Nottelmann 50/57

51 5. Mapping OWL Lite onto probabilistic Datalog Norbert Fuhr; Henrik Nottelmann 51/57

52 Classes, properties, individuals and literals Basic idea: unary Datalog predicates c(x) for classes binary Datalog predicates p(x, y) for properties constants for individuals and literals Class(Male partial Animal) ObjectProperty(hasFather super(hasparent)) Individual(Kain type(male) value(hasfather Adam) value(hasmother Eve) value(shoesize 10)) animal(x) :- male(x). hasparent(x,y) :- hasfather(x,y) male(kain). hasfather(kain,adam). hasmother(kain,eve). shoesize(kain,"10"). Norbert Fuhr; Henrik Nottelmann 52/57

53 Properties revisited (1) Domain and range: ObjectProperty(hasFather domain(animal) range(male)) animal(x) :- hasfather(x,y). male(y) :- hasfather(x,y). Inverse properties: ObjectProperty(hasParent inverseof(haschild)) hasparent(x,y) :- haschild(y,x). haschild(x,y) :- hasparent(y,x). Symmetric and transitive properties: ObjectProperty(hasBrother Symmetric Transitive) hasbrother(x,y) :- hasbrother(y,x). hasbrother(x,z) :- hasbrother(x,y) & hasbrother(y,z). Norbert Fuhr; Henrik Nottelmann 53/57

54 Property restrictions (1) Universal quantifier: Class(Person partial Animal restriction(hasparent allvaluesfrom(person))) person(y) :- person(x) & hasparent(x,y). Existential quantifier: can only express inconsistencies (four-valued pdatalog) Class(Person partial Animal restriction(hasparent somevaluesfrom(person))) status(consistent). exists(x) :- hasparent(x,y) & person(y).!status(consistent) :- person(x) &!exists(x). Norbert Fuhr; Henrik Nottelmann 54/57

55 Property restrictions (2) Cardinality restrictions: can only express inconsistencies (four-valued pdatalog) Class(Person partial Animal restriction(hasfather cardinality(1))) status(consistent). exists(x) :- hasfather(x,y). diff2(x) :- hasfather(x,y) & hasfather(x,z) & Y<>Z.!status(consistent) :- Person(X) &!exists(x).!status(consistent) :- Person(X) & diff2(x). similar for other cardinality restrictions and (inverse) functional properties Norbert Fuhr; Henrik Nottelmann 55/57

56 Possible OWL extensions Probabilities in OWL: modelled by corresponding pdatalog program Individual(lisa 0.7 type(person) 0.9 value(hasparent thomas)) Class (Person partial 0.5 Man) ObjectProperty(foo 0.7 Symmetric) ObjectProperty(foo 0.4 domain(bar)) 0.7 person(lisa). 0.9 hasparent(lisa,thomas). 0.5 man(x) :- person(x). 0.7 foo(x,y) :- foo(y,x). 0.4 bar(x) :- foo(x,y). Rules: use pdatalog rules for cases where OWL is not sufficient hasolderbrother(x) :- hassibling(x,y) & male(y) & age(x,ax) & age(y,ay) & AX<AY. Norbert Fuhr; Henrik Nottelmann 56/57

57 Conclusion basic idea: map classes/properties onto unary/binary predicates many primitives cannot be mapped onto Datalog at least inconsistencies can be detected with four-valued Datalog mapping allows for OWL extensions probabilities rules Norbert Fuhr; Henrik Nottelmann 57/57