Implementation Techniques

Transcription

1 Web Science & Technologies University of Koblenz Landau, Germany Implementation Techniques Acknowledgements to Angele, Gehrke, STI

2 Word of Caution There is not the one silver bullet In actual systems, different techniques are combined 2 of 63

3 Many further techniques Truth maintenance Tabling Tail elimination Compilation into SAT solvers (especially for EDLP) Optimization of information passing (comparable to join ordering). Plus extensions for External predicates, connection to relational databases Combinations with first order logics Arithmetics. What may help in one case may cause harm in another one! 3 of 63

4 Forward Chaining w Naive Evaluation w Semi-naive evaluation Backward Chaining w SLDNF / Prolog w Dynamic Filtering Magic Sets Alternating Fixpoint 4 of 63

5 Example knowledge base The law says that it is a crime for an American to sell weapons to hostile nations. The country Nono, an enemy of America, has some missiles, and all of its missiles were sold to it by Colonel West, who is American. Prove that Col. West is a criminal 5 of 63

6 Example knowledge base contd.... it is a crime for an American to sell weapons to hostile nations: American(x) Weapon(y) Sells(x,y,z) Hostile(z) Criminal(x) Nono has some missiles, i.e., x Owns(Nono,x) Missile(x): Owns(Nono,M 1 ) Missile(M 1 ) all of its missiles were sold to it by Colonel West Missile(x) Owns(Nono,x) Sells(West,x,Nono) Missiles are weapons: Missile(x) Weapon(x) An enemy of America counts as "hostile : Enemy(x,America) Hostile(x) West, who is American American(West) The country Nono, an enemy of America Enemy(Nono,America) 6 of 63

7 Forward chaining algorithm / Naïve evaluation T ω 7 of 63

8 Forward chaining proof 8 of 63

9 Forward chaining proof 9 of 63

10 Forward chaining proof 10 of 63

11 Properties of forward chaining Sound and complete for first-order definite Horn clauses, which means that it computes all entailed facts correctly. Datalog = first-order definite clauses + no functions FC terminates for Datalog in finite number of iterations May not terminate in general if α is not entailed w This is unavoidable: entailment with definite clauses is semi-decidable 11 of 63

12 Efficiency of forward chaining Incremental forward chaining: no need to match a rule on iteration k if a premise wasn't added on iteration k-1 w match each rule whose premise contains a newly added positive literal Magic Sets: rewriting the rule set, using information from the goal, so that only relevant bindings are considered during forward chaining Magic_Criminal(x) American(x) Weapon(y) Sells(x,y,z) Hostile(z) Criminal(x) add Magic_Criminal(West) to the KB Matching itself can be expensive w Database indexing allows O(1) retrieval of known facts: for a given fact it is possible to construct indices on all possible queries that unify with it w Subsumption lattice can get very large; the costs of storing and maintaining the indices must not outweigh the cost for facts retrieval. Forward chaining is widely used in deductive databases 12 of 63

13 Backward chaining algorithm 13 of 63

14 Backward chaining example 14 of 63

15 Backward chaining example 15 of 63

16 Backward chaining example 16 of 63

17 Backward chaining example 17 of 63

18 Backward chaining example 18 of 63

19 Backward chaining example 19 of 63

20 Backward chaining example 20 of 63

21 Properties of backward chaining Depth-first recursive proof search: space is linear in size of proof Incomplete due to infinite loops w can be fixed by applying breadth-first search Inefficient due to repeated sub-goals (both success and failure) w can be fixed using caching of previous results (extra space) Widely used for logic programming 21 of 63

22 Resolution strategies Unit preference w clauses with just one literal are preferred w Unit resolution incomplete in general, complete for Horn KB Set of support w at least one of the clauses makes part from the set of support w complete if the remainder of the sentences are jointly satisfiable w Using the negated query as set-of-support Input resolution w at least one of the clauses makes part from the initial KB or the query w Complete for Horn, incomplete in the general case w Linear resolution: P and Q can be resolved if P is in the original KB or P is an ancestor of Q in the proof tree; complete Subsumption: all sentences subsumed by others in the KB are eliminated 22 of 63

23 SLDNF / Prolog - In each resolution step unification is applied! - green: goal - red: KB rules 23 of 63

24 Comparison Forward/Backward Chaining Advantage Forward Chaining: w Computing join is more efficient than nested-loop implicit in backward chaining Advantage Backward Chaining: w Avoiding the computation of the whole fixpoint 24 of 63

25 Summary Proof algorithms, are only semi-decidable i.e., might not terminate for non-entailed queries Propositionalization instantiating quantifiers: slow Unification makes the instantiation step unnecessary Complete for definite clauses; semi-decidable w Forward-chaining w Backward-chaining w decidable for Datalog Strategies for reducing the search space of a resolution system required 25 of 63

26 OPTIMIZATIONS 26 of 63

27 Example wheel 3 1 frame spoke tire seat pedal 1 1 rim tube Find components of! Are we running low on any parts needed to build a? What is total component and assembly cost to build at today's part prices? wheel 3 frame 1 frame seat 1 frame pedal 1 wheel spoke 2 wheel tire 1 tire rim 1 tire tube 1 Assembly instance of 63 part subpart number

28 Datalog Query that Does the Job Comp(Part, Subpt) :- Assembly(Part, Subpt, Qty). Comp(Part, Subpt) :- Assembly(Part, Part2, Qty), Comp(Part2, Subpt) of 63

29 Example Comp(Part, Subpt) :- Assembly(Part, Subpt, Qty). Comp(Part, Subpt) :- Assembly(Part, Part2, Qty), Comp(Part2, Subpt). For any instance of Assembly, we compute all Comp tuples by repeatedly applying two rules. Actually: we can apply Rule 1 just once, then apply Rule 2 repeatedly. Rule1 ~ projection Rule2 ~ cross-product with equality join of 63

30 wheel 3 frame 1 frame seat 1 frame pedal 1 wheel spoke 2 wheel tire 1 tire rim 1 tire tube 1 Assembly instance wheel wheel spoke tire seat pedal rim tube Comp tuples by applying Rule 2 once wheel wheel of 63 spoke tire seat pedal rim tube rim tube Comp tuples by applying Rule 2 twice Comp(Part, Subpt) :- Assembly(Part, Part2, Qty), Comp(Part2, Subpt).

31 Example For any instance of Assembly, we can compute all Comp tuples by repeatedly applying the two rules. (Actually, we can apply Rule 1 just once, then apply Rule 2 repeatedly.) wheel wheel spoke tire seat pedal rim tube Comp tuples got by applying Rule 2 once wheel wheel spoke tire seat pedal rim tube rim tube Comp tuples got by applying Rule 2 twice of 63

32 Evaluation of Datalog Programs Avoid Repeated inferences: Avoid Unnecessary inferences: of 63

33 Query Optimization # of 63

34 Evaluation of Datalog Programs Avoid Repeated inferences: When recursive rules are repeatedly applied in naïve way, we make same inferences in several iterations of 63

35 wheel 3 frame 1 frame seat 1 frame pedal 1 wheel spoke 2 wheel tire 1 tire rim 1 tire tube 1 Assembly instance wheel wheel spoke tire seat pedal rim tube Comp tuples by applying Rule 2 once wheel wheel of 63 spoke tire seat pedal rim tube rim tube Comp tuples by applying Rule 2 twice Comp(Part, Subpt) :- Assembly(Part, Part2, Qty), Comp(Part2, Subpt).

36 Avoiding Repeated Inferences Semi-naive Fixpoint Evaluation: Ensure that when rule is applied, at least one of body facts used was generated in most recent iteration. Such new inference could not have been carried out in earlier iterations of 63

37 Avoiding Repeated Inferences Idea: For each recursive table P, use table delta_p to store P tuples generated in previous iteration. w 1. Rewrite program to use delta tables w 2. Update delta tables between iterations. Comp(Part, Subpt) :- Assembly(Part, Part2, Qty), Comp(Part2, Subpt). Comp(Part, Subpt) :- Assembly(Part, Part2, Qty), delta_comp(part2, Subpt) of 63

38 Query Optimization # of 63

39 Evaluation of Datalog Programs Unnecessary inferences: w If we just want to find components of a particular part, say wheel, then first computing general fixpoint of Comp program and then at end selecting tuples with wheel in the first column is wasteful. This would compute many irrelevant facts of 63

40 Avoiding Unnecessary Inferences SameLev(S1,S2) :- Assembly(P1,S1,Q1), Assembly(P1,S2,Q2). SameLev(S1,S2) :- Assembly(P1,S1,Q1), SameLev(P1,P2), Assembly(P2,S2,Q2). 3 1 wheel frame spoke tire seat pedal 1 1 rim tube 40 of 63

41 Avoiding Unnecessary Inferences SameLev(S1,S2) :- Assembly(P1,S1,Q1), Assembly(P1,S2,Q2). SameLev(S1,S2) :- Assembly(P1,S1,Q1), SameLev(P1,P2), Assembly(P2,S2,Q2). Tuple (S1,S2) in SameLev if there is path up from S1 to some node and down to S2 with same number of up and down edges. wheel of 63 frame spoke tire seat pedal 1 1 rim tube

42 Avoiding Unnecessary Inferences Want all SameLev tuples with spoke in first column. Intuition: Push this selection into fixpoint computation. How do that? SameLev(S1,S2) :- Assembly(P1,S1,Q1), SameLev(P1,P2), Assembly(P2,S2,Q2). SameLev(spoke,S2) :- Assembly(P1,spoke,Q1), SameLev(P1?=spoke?,P2), Assembly(P2,S2,Q2) of 63

43 Avoiding Unnecessary Inferences Intuition: Push this selection with spoke into fixpoint computation. SameLev(S1,S2) :- Assembly(P1,S1,Q1), SameLev(P1,P2), Assembly(P2,S2,Q2). SameLev(spoke,S2) :- Assembly(P1,spoke,Q1), SameLev(P1,P2 Assembly(P2,S2,Q2). SameLev(spoke,seat) :- Assembly(wheel,spoke,2), SameLev(wheel,frame), Assembly(frame,seat,1). n Other SameLev tuples are needed to compute all such tuples with spoke, e.g. wheel of 63

44 Magic Sets Idea 1. Define filter table that computes all relevant values 2. Restrict computation of SameLev to infer only tuples with relevant value in first column of 63

45 Intuition Relevant values: contains all tuples m for which we have to compute all same-level tuples with m in first column to answer query. Put differently, relevant values are all Same-Level tuples whose first field contains value on path from spoke up to root. We call it Magic-SameLevel (Magic-SL) of 63

46 Magic Sets in Example Idea: Define filter table that computes all relevant values : Collect all parents of spoke. Magic_SL(P1) :- Magic_SL(S1), Assembly(P1,S1,Q1). Magic_SL(spoke) :-. Make Magic table as Magic-SameLevel of 63

47 Magic Sets Idea Idea: Use filter table to restrict the computation of SameLev. Magic_SL(P1) :- Magic_SL(S1), Assembly(P1,S1,Q1). Magic(spoke). SameLev(S1,S2) :- Magic_SL(S1), Assembly(P1,S1,Q1), Assembly(P1,S2,Q2). SameLev(S1,S2) :- Magic_SL(S1), Assembly(P1,S1,Q1), SameLev(P1,P2), Assembly(P2,S2,Q2) of 63

48 Magic Sets Idea Idea: Define filter table that computes all relevant values, and restrict the computation of SameLev correspondingly. Magic_SL(P1) :- Magic_SL(S1), Assembly(P1,S1,Q1). Magic(spoke). SameLev(S1,S2) :- Magic_SL(S1), Assembly(P1,S1,Q1), Assembly(P1,S2,Q2). SameLev(S1,S2) :- Magic_SL(S1), Assembly(P1,S1,Q1), SameLev(P1,P2), Assembly(P2,S2,Q2) of 63

49 The Magic Sets Algorithm 1. Generate an adorned program Program is rewritten to make pattern of bound and free arguments in query explicit 2. Add magic filters of form Magic_P for each rule in adorned program add a Magic condition to body that acts as filter on set of tuples generated (predicate P to restrict these rules) 3. Define new rules to define filter tables Define new rules to define filter tables of form Magic_P of 63

50 Step 1:Generating Adorned Rules Adorned program for query pattern SameLev bf, assuming left-to-right order of rule evaluation : SameLev bf (S1,S2) :- Assembly(P1,S1,Q1), Assembly(P1,S2,Q2). SameLev bf (S1,S2) :- Assembly(P1,S1,Q1), SameLev bf (P1,P2), Assembly(P2,S2,Q2). v v Argument of (a given body occurrence of) SameLev is: v b if it appears to the left in body, v or if it is a b argument of head of rule, v Otherwise it is free. Assembly not adorned because explicitly stored table. 50 of 63

51 Step 2: Add Magic Filters For every rule in adorned program add a magic filter predicate SameLev bf (S1,S2) :- Magic_SL (S1), Assembly(P1,S1,Q1), Assembly(P1,S2,Q2). SameLev bf (S1,S2) :- Magic_SL (S1), Assembly(P1,S1,Q1), SameLev bf (P1,P2), Assembly(P2,S2,Q2). Filter predicate: copy of head of rule, Magic prefix, and delete free variable of 63

52 Step 3:Defining Magic Tables Rule for Magic_P is generated from each occurrence of recursive P in body of rule: w Delete everything to right of P w Add prefix Magic and delete free columns of P w Move P, with these changes, into head of rule of 63

53 Step 3:Defining Magic Table Rule for Magic_P is generated from each occurrence O of recursive P in body of rule: w Delete everything to right of P SameLev bf (S1,S2) :- Magic_SL(S1), Assembly(P1,S1,Q1), SameLev bf (P1,P2), Assembly(P2,S2,Q2). w Add prefix Magic and delete free columns of P Magic-SameLev bf (S1,S2) :- Magic_SL(S1), Assembly(P1,S1,Q1), Magic-SameLev bf (P1 ). w Move P, with these changes, into head of rule Magic_SL(P1) :- Magic_SL(S1), Assembly(P1,S1,Q1) of 63

54 Step 3:Defining Magic Tables Rule for Magic_P is generated from each occurrence of P in body of such rule: SameLev bf (S1,S2) :- Magic_SL(S1), Assembly(P1,S1,Q1), SameLev bf (P1,P2), Assembly(P2,S2,Q2). Magic_SL(P1) :- Magic_SL(S1), Assembly(P1,S1,Q1). 54 of 63

55 Magic Sets Idea Define filter table that computes all relevant values: Restrict computation of SameLev Magic_SL(P1) :- Magic_SL(S1), Assembly(P1,S1,Q1). Magic(spoke). SameLev(S1,S2) :- Magic_SL(S1), Assembly(P1,S1,Q1), Assembly(P1,S2,Q2). SameLev(S1,S2) :- Magic_SL(S1), Assembly(P1,S1,Q1), SameLev(P1,P2), Assembly(P2,S2,Q2) of 63

56 DYNAMIC FILTERING 56 of 63

57 Dynamic Filtering (Kifer) q(f(a), a), q(a,c), q(f(c),c), r(a, c), r(c, d), r(e,c) p(x, Y) q(f(x), X) r(x, Y) p(x, c) p(x, c) X, c a a, c a, a a, c e, c p(x, Y) q(f(x), X) r(x, Y) f(a), a f(e), e f(a), a q(f(a), a) q(a,c) q(f(c),c) X, c a, c e, c r(a, c) r(c, d) r(e, c) 57 of 63

58 Dynamic Filtering: Problem Negation a f(c) a f(c) q(x, c) not p(x, c) X, c a, c a, a a, c p(x, Y) q(f(x), X) r(x, Y) X, c a, c f(c),c f(a), a f(a), a q(f(a), a) q(a,c) q(f(c),c) X, c r(a, c) r(c,d) r(e,f) 58 of 63

59 Dynamic Filtering: Stratification a f(c) a f(c) q(x, c) not p(x, c) Level 2 X, c a, c a, a a, c p(x, Y) q(f(x), X) r(x, Y) Level 1 X, c a, c f(c), c f(a), a f(a), a q(f(a), a) q(a,c) q(f(c),c) X, c r(a, c) r(c,d) r(e,f) Level 0 59 of 63

60 ALTERNATING FIXPOINT 60 of 63

61 What about the following? person(somebody). woman(x) <- person(x) and not man(x). man(x) <- person(x) and not woman(x). 61 of 63

62 Alternating Fixed Point (Well-Founded Semantics) Van Gelder et al., JACM 1991, If there exists some n: T n+1 (I) = T n-1 (I) and T n+2 (I) = T n (I) Then the true facts are defined by the smaller sets of T n-1 (I) and T n (I) Unknown facts are given by set difference. 62 of 63

63 Alternating Fixed Point: Example person(sb). woman(x) <- person(x) and not man(x). man(x) <- person(x) and not woman(x). T(I={person{sb}}) = {woman(sb), man(sb), person(sb)} T(T(I)) = {person(sb)} T(T(T(I))) = {woman(sb), man(sb), person(sb)} T(T(T(T(I)))) = {person(sb)} True Facts = {person(sb)} Unknown Facts = {woman(sb), man(sb)} Very expensive L 63 of 63