15-780: Graduate AI Lecture 2. Proofs & FOL. Geoff Gordon (this lecture) Tuomas Sandholm TAs Byron Boots, Sam Ganzfried

Transcription

1 15-780: Graduate AI Lecture 2. Proofs & FOL Geoff Gordon (this lecture) Tuomas Sandholm TAs Byron Boots, Sam Ganzfried 1

2 Admin 2

3 Audits 3

4 Review 4

5 What is AI? Lots of examples: poker, driving robots, RoboCup Things that are easy for us to do, but no obvious algorithm Search / optimization / summation Handling uncertainty 5

6 Propositional logic Syntax variables, constants, operators literals, clauses, sentences Semantics (model {T, F}) Truth tables, how to evaluate formulas Satisfiable, valid, contradiction 6

7 Propositional logic Manipulating formulas (e.g., de Morgan) Normal forms (e.g., CNF) Tseitin transformation to CNF How to translate informally-specified problems into logic (e.g., 3-coloring) 7

8 Proofs 8

9 Entailment Sentence A entails sentence B, A B, if B is True in every model where A is same as saying that (A B) is valid 9

10 Proof tree A tree with a formula at each node At each internal node, children parent Leaves: assumptions or premises Root: consequence If we believe assumptions, we should also believe consequence 10

11 Proof tree example 11

12 Proof by contradiction Assume opposite of what we want to prove, show it leads to a contradiction Suppose we want to show KB S Write KB for (KB S) Build a proof tree with assumptions drawn from clauses of KB conclusion = F so, (KB S) F (contradiction) 12

13 Proof by contradiction 13

14 Proof by contradiction 14

15 Inference rules 15

16 Inference rule To make a proof tree, we need to be able to figure out new formulas entailed by KB Method for finding entailed formulas = inference rule We ve implicitly been using one already 16

17 Modus ponens (a b c d) a b c d Probably most famous inference rule: all men are mortal, Socrates is a man, therefore Socrates is mortal Quantifier-free version: man(socrates) (man(socrates) mortal(socrates)) 17

18 Another inference rule (a b) b a Modus tollens If it s raining the grass is wet; the grass is not wet, so it s not raining 18

19 One more Resolution (a b c) ( c d e) a b d e Combines two sentences that contain a literal and its negation Not as commonly known as modus ponens / tollens 19

20 Resolution example Modus ponens / tollens are special cases Modus tollens: ( raining grass-wet) grass-wet raining 20

21 Resolution (a b c) ( c d e) a b d e Simple proof by case analysis Consider separately cases where we assign c = True and c = False 21

22 Resolution (a b c) ( c d e) Case c = True (a b T) (F d e) = (T) (d e) = (d e) 22

23 Resolution (a b c) ( c d e) Case c = False (a b F) (T d e) = (a b) (T) = (a b) 23

24 Resolution (a b c) ( c d e) Since c must be True or False, conclude (d e) (a b) as desired 24

25 Soundness and completeness An inference procedure is sound if it can only conclude things entailed by KB common sense; haven t discussed anything unsound A procedure is complete if it can conclude everything entailed by KB 25

26 Completeness Modus ponens by itself is incomplete Resolution is complete for propositional logic not obvious famous theorem due to Robinson caveat: also need factoring, removal of redundant literals (a b a) = (a b) 26

27 Variations Horn clause inference (faster) Ways of handling uncertainty (slower) CSPs (sometimes more convenient) Quantifiers / first-order logic 27

28 Horn clauses Horn clause: (a b c d) Equivalently, ( a b c d) Disjunction of literals, at most one of which is positive Positive literal = head, rest = body 28

29 Use of Horn clauses People find it easy to write Horn clauses (listing out conditions under which we can conclude head) happy(john) happy(mary) happy(sue) No negative literals in above formula; again, easier to think about 29

30 Why are Horn clauses important Modus ponens alone is complete So is modus tollens alone Inference in a KB of propositional Horn clauses is linear e.g., by forward chaining 30

31 Forward chaining Look for a clause with all body literals satisfied Add its head to KB (modus ponens) Repeat See RN for more details 31

32 Handling uncertainty Fuzzy logic / certainty factors simple, but don t scale Nonmonotonic logic also doesn t scale Probabilities may or may not scale more later Dempster-Shafer (interval probability) 32

33 Certainty factors KB assigns a certainty factor in [0, 1] to each proposition Interpret as degree of belief When applying an inference rule, certainty factor for consequent is a function of certainty factors for antecedents (e.g., minimum) 33

34 Problems w/ certainty factors Famously difficult to debug a KB with certainty factors, because it s hard to separate a large KB into mostly-independent chunks that interact only through a well-defined interface, because certainty factors are not probabilities (i.e., do not obey Bayes Rule) 34

35 Nonmonotonic logic Suppose we believe all birds can fly Might add a set of sentences to KB bird(polly) flies(polly) bird(tweety) flies(tweety) bird(tux) flies(tux) bird(john) flies(john) 35

36 Nonmonotonic logic Fails if there are penguins in the KB Fix: instead, add bird(polly) ab(polly) flies(polly) bird(tux) ab(tux) flies(tux) ab(tux) is an abnormality predicate Need separate ab i (x) for each type of rule 36

37 Nonmonotonic logic Now set as few abnormality predicates as possible Can prove flies(polly) or flies(tux) with no ab(x) assumptions If we assert flies(tux), must now assume ab(tux) to maintain consistency Can t prove flies(tux) any more, but can still prove flies(polly) 37

38 Nonmonotonic logic Works well as long as we don t have to choose between big sets of abnormalities is it better to have 3 flightless birds or 5 professors that don t wear jackets with elbow-patches? even worse with nested abnormalities: birds fly, but penguins don t, but superhero penguins do, but 38

39 First-order logic 39

40 First-order logic Bertrand Russell So far we ve been using opaque vars like rains or happy(john) Limits us to statements like it s raining or if John is happy then Mary is happy Can t say all men are mortal or if John is happy then someone else is happy too 40

41 Predicates and objects Interpret happy(john) or likes(joe, pizza) as a predicate applied to some objects Object = an object in the world Predicate = boolean-valued function of objects Zero-argument predicate x() plays same role that Boolean variable x did before 41

42 Distinguished predicates We will assume three distinguished predicates with fixed meanings: True / T, False / F Equal(x, y) We will also write (x = y) and (x y) 42

43 Equality satisfies usual axioms Reflexive, transitive, symmetric Substituting equal objects doesn t change value of expression (John = Jonathan) loves(mary, John) loves(mary, Jonathan) 43

44 Functions Functions map zero or more objects to another object e.g., professor(15-780), last-commonancestor(john, Mary) Zero-argument function is the same as an object John v. John() 44

45 The nil object Functions are untyped: must have a value for any set of arguments Typically add a nil object to use as value when other answers don t make sense 45

46 Types of values Expressions in propositional logic could only have Boolean (T/F) values Now we have two types of expressions: object and Boolean done(slides(15-780)) happy(professor(15-780)) Functions convert objects to objects; predicates convert objects to Booleans; connectives convert Booleans to Booleans 46

47 Definitions Term = expression referring to an object John left-leg-of(father-of(president-of(usa))) Atom = predicate applied to objects happy(john) raining at(robot, Wean-5409, 11AM-Wed) 47

48 Definitions Literal = possibly-negated atom happy(john), happy(john) Sentence = literals joined by connectives like raining done(slides(780)) happy(professor) 48

49 Semantics Models are now much more complicated List of objects (finite or countable) Table of function values for each function mentioned in formula Table of predicate values for each predicate mentioned in formula Meaning of sentence: model {T, F} Meaning of term: model object 49

50 For example 50

51 KB describing example alive(cat) ear-of(cat) = ear in(cat, box) in(ear, box) in(box, cat) in(cat, nil) ear-of(box) = ear-of(ear) = ear-of(nil) = nil cat box cat ear cat nil 51

52 Aside: avoiding verbosity Closed-world assumption: literals not assigned a value in KB are false avoid stating in(box, cat), etc. Unique names assumption: objects with separate names are separate avoid box cat, cat ear, 52

53 Aside: typed variables KB also illustrates need for data types Don t want to have to specify ear-of(box) or in(cat, nil) Could design a type system argument of happy() is of type animate Function instances which disobey type rules have value nil 53

54 Model of example Objects: C, B, E, N Assignments: cat: C, box: B, ear: E, nil: N ear-of(c): E, ear-of(b): N, ear-of(e): N, ear-of(n): N Predicate values: in(c, B), in(c, C), in(c, N), 54

55 Failed model Objects: C, E, N Fails because there s no way to satisfy inequality constraints with only 3 objects 55

56 Another possible model Objects: C, B, E, N, X Extra object X could have arbitrary properties since it s not mentioned in KB E.g., X could be its own ear 56

57 An embarrassment of models In general, can be infinitely many models unless KB limits number somehow Job of KB is to rule out models that don t match our idea of the world Saw how to rule out CEN model Can we rule out CBENX model? 57

58 Getting rid of extra objects Can use quantifiers to rule out CBENX model: x. x = cat x = box x = ear x = nil Called a domain closure assumption 58

59 Quantifiers Want all men are mortal, or closure Add quantifiers and object variables x. man(x) mortal(x) x. lunch(x) free(x) : no matter how we fill in object variables, formula is still true : there is some way to fill in object variables to make formula true 59

60 Variables Build atoms from variables x, y, as well as constants John, Fred, man(x), loves(john, z), mortal(brother(y)) Build formulas from these atoms man(x) mortal(brother(x)) New syntactic construct: term or formula w/ free variables 60

61 New syntax new semantics New part of model: interpretation Maps variables to model objects x: C, y: N Meaning of a term or formula: look up its free variables in the interpretation, then continue as before alive(ear(x)) alive(ear(c)) alive(e) T 61

62 Working with interpretations Write (M / x: obj) for the model which is just like M except that variable x is interpreted as the object obj M / x: obj is a refinement of M 62

63 Binding Adding quantifier for x is called binding x In ( x. likes(x, y)), x is bound, y is free Can add quantifiers and apply logical operations like in any order But must wind up with ground formula (no free variables) 63

64 Semantics of A sentence ( x. S) is true in M if S is true in (M / x: obj) for all objects obj in M 64

65 Example M has objects (A, B, C) and predicate happy(x) which is true for A, B, C Sentence x. happy(x) is satisfied in M since happy(a) is satisfied in M/x:A, happy(b) in M/x:B, happy(c) in M:x/C 65

66 Semantics of A sentence ( x. S) is true in M if there is some object obj in M such that S is true in model (M / x: obj) 66

67 Example M has objects (A, B, C) and predicate happy(a) = happy(b) = True happy(c) = False Sentence x. happy(x) is satisfied in M Since happy(x) is satisfied in, e.g., M/x:B 67

68 Scoping rules Portion of formula where quantifier applies = scope Variable is bound by innermost enclosing scope with matching name Two variables in different scopes can have same name they are still different vars 68

69 Scoping examples ( x. happy(x)) ( x. happy(x)) Either everyone s happy, or someone s unhappy x. (raining outside(x) ( x. wet(x))) The x who is outside may not be the one who is wet 69

70 Quantifier nesting English sentence everybody loves somebody is ambiguous Translates to logical sentences x. y. loves(x, y) y. x. loves(x, y) 70

71 Reasoning in FOL 71

72 Entailment, etc. As before, entailment, unsatisfiability, validity, equivalence, etc. refer to all possible models But now, can t determine by enumerating models since there could be infinitely many 72

73 Equivalences All transformation rules for propositional logic still hold In addition, there is a De Morgan s Law for moving negations through quantifiers x. S x. S x. S x. S And, rules for getting rid of quantifiers 73

74 Generalizing CNF Eliminate, move in w/ De Morgan but moves through quantifiers too Get rid of quantifiers (see below) Distribute, or use Tseitin 74

75 Do we really need? x. happy(x) happy(happy_person()) y. x. loves(y, x) y. loves(y, loved_one(y)) 75

76 Skolemization Called Skolemization (after Thoraf Albert Skolem) Thoraf Albert Skolem Eliminate using function of arguments of all enclosing quantifiers 76

77 Do we really need? 77

78 Getting rid of quantifiers Standardize apart (avoid name collisions) Skolemize Drop (free variables implicitly universally quantified) Terminology: still called free even though quantification is implicit 78

79 For example x. man(x) mortal(x) ( man(x) mortal(x)) x. (honest(x) happy(diogenes)) ( honest(y) happy(diogenes)) y. x. loves(y, x) loves(z, f(z)) 79

80 Exercise ( x. honest(x)) happy(diogenes) 80

81 Proofs 81

82 Proofs Proofs by contradiction work as before: add S to KB put in CNF run resolution if we get an empty clause, we ve proven S by contradiction But, CNF and resolution have changed 82

83 Generalizing resolution Propositional: ( a b) a b FOL: ( man(x) mortal(x)) man(socrates) mortal(socrates) Difference: had to substitute x = Socrates 83

84 Unification Two FOL sentences unify with each other if there is a way to set their variables so that they are identical man(x), man(socrates) unify using the substitution x = Socrates 84

85 Unification examples loves(x, x), loves(john, y) unify using x = y = John loves(x, x), loves(john, Mary) can t unify loves(uncle(x), y), loves(z, aunt(z)): 85

86 Unification examples loves(x, x), loves(john, y) unify using x = y = John loves(x, x), loves(john, Mary) can t unify loves(uncle(x), y), loves(z, aunt(z)): z = uncle(x), y = aunt(uncle(x)) loves(uncle(x), aunt(uncle(x))) 86

87 Most general unifier May be many substitutions that unify two formulas MGU is unique (up to renaming) Simple, fast algorithm for finding MGU (see RN) 87

88 First-order resolution Given clauses (a b c), ( c d e) And a variable substitution V If c : V and c : V are the same Then we can conclude (a b d e) : V 88

89 First-order factoring When removing redundant literals, we have the option of unifying them first Given clause (a b c), substitution V If a : V and b : V are the same Then we can conclude (a c) : V 89

90 Completeness First-order resolution (together with firstorder factoring) is sound and complete for FOL Famous theorem 90

91 Completeness 91

92 Proof strategy We ll show FOL completeness by reducing to propositional completeness 92

93 Propositionalization Given a FOL KB in clause form And a set of terms U (for universe) We can propositionalize KB under U by substituting elements of U for free variables in all combinations 93

94 Propositionalization example ( man(x) mortal(x)) mortal(socrates) favorite_drink(socrates) = hemlock drinks(x, favorite_drink(x)) U = {Socrates, hemlock, Fred} 94

95 Propositionalization example ( man(socrates) mortal(socrates)) ( man(fred) mortal(fred)) ( man(hemlock) mortal(hemlock)) drinks(socrates, favorite_drink(socrates)) drinks(hemlock, favorite_drink(hemlock)) drinks(fred, favorite_drink(fred)) mortal(socrates) favorite_drink(socrates) = hemlock 95

96 Choosing a universe To check a FOL KB, propositionalize it using some universe U Which universe? 96

97 Herbrand Universe Herbrand universe H of formula S: start with all objects mentioned in S or synthetic object X if none mentioned apply all functions mentioned in S to all combinations of objects in H, add to H repeat 97

98 Herbrand Universe E.g., loves(uncle(john), Mary) H = {John, Mary, uncle(john), uncle(mary), uncle(uncle(john)), uncle(uncle(mary)), } 98

99 Herbrand s theorem If a FOL KB in clause form is unsatisfiable And H is its Herbrand universe Then the propositionalized KB is unsatisfiable for some finite U H 99

100 Significance This is one half of the equivalence we want: unsatisfiable FOL KB unsatisfiable propositional KB 100

101 Converse of Herbrand A. J. Robinson proved lifting lemma Write PKB for a propositionalization of KB (under some universe) Any resolution proof in PKB corresponds to a resolution proof in KB and, if PKB is unsatisfiable, there is a proof of F (by prop. completeness); so, lifting it shows KB unsatisfiable 101

102 Proofs w/ Herbrand & Robinson So, FOL KB is unsatisfiable if and only if there is a subset of Herbrand universe making PKB unsatisfiable If we have a way to find proofs in propositional logic, we have a way to find them in FOL 102

103 Proofs w/ Herbrand & Robinson To prove S, put KB S in CNF: KB Build subsets of Herbrand universe in increasing order of size: U1, U2, Propositionalize KB w/ Ui, look for proof If U i unsatisfiable, use lifting to get a contradiction in KB If Ui satisfiable, move on to Ui+1 103

104 Making it faster Restrict semantics so we only need to check one finite propositional KB Unique names: objects with different names are different (John Mary) Domain closure: objects without names given in KB don t exist Restrictions also make entailment, validity feasible 104

105 Why FOL 105

106 What s so special about FOL Existence of a sound, complete inference procedure Additions to FOL break this procedure although sometimes there s a replacement 106

107 Equality Paramodulation is sound and complete for FOL+equals (see RN) 107

108 Second order logic SOL adds quantification over predicates E.g., principle of mathematical induction: P. P(0) ( x. P(x) P(S(x))) x. P(x) There is no sound and complete inference procedure for SOL (Gödel s famous incompleteness theorem) 108

109 Others Temporal logics ( P(x) will be true at some time in the future ) Modal logics ( John knows P(x) ) First-class functions (lambda operator, application) 109