Inductive Programming

Transcription

1 Inductive Programming A Unifying Framework for Analysis and Evaluation of Inductive Programming Systems Hofmann, Kitzelmann, Schmid Cognitive Systems Group University of Bamberg AGI 2009 CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 1 / 22

2 Inductive Program Synthesis (IP) Inductive Program Synthesis (IP) researches the automatic construction of (recursive) programs from incomplete specifications, i.e. input/ouput examples (I/O examples) Example (reverse) I/O-examples: reverse [] = [] reverse [a] = [a] reverse [a,b] = [b,a] reverse [a,b,c] = [c,b,a] Induced functional program: reverse [] = [] reverse (x:xs) = (reverse xs) ++ [x] CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 2 / 22

3 Key Concepts Preference Bias criteria to choose among (semantically different!) candidate solutions, i.e syntactic size, number of case distinctions, runtime (search strategy). Restriction Bias Restricts the inducable class of problems, through syntactic constraints, i.e. linear recursion as sole kind of recursion (hypothese language) Background Knowledge already implemented functions, which can by used for synthesis, i.e. append and partition for quicksort Sub Functions Functions neither defined as target functions nor in the background knowledge, but automaticallly introduced as auxiliary functions by the IP algorithm CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 3 / 22

4 Key Concepts Preference Bias criteria to choose among (semantically different!) candidate solutions, i.e syntactic size, number of case distinctions, runtime (search strategy). Restriction Bias Restricts the inducable class of problems, through syntactic constraints, i.e. linear recursion as sole kind of recursion (hypothese language) Background Knowledge already implemented functions, which can by used for synthesis, i.e. append and partition for quicksort Sub Functions Functions neither defined as target functions nor in the background knowledge, but automaticallly introduced as auxiliary functions by the IP algorithm CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 3 / 22

5 Key Concepts Preference Bias criteria to choose among (semantically different!) candidate solutions, i.e syntactic size, number of case distinctions, runtime (search strategy). Restriction Bias Restricts the inducable class of problems, through syntactic constraints, i.e. linear recursion as sole kind of recursion (hypothese language) Background Knowledge already implemented functions, which can by used for synthesis, i.e. append and partition for quicksort Sub Functions Functions neither defined as target functions nor in the background knowledge, but automaticallly introduced as auxiliary functions by the IP algorithm CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 3 / 22

6 Key Concepts Preference Bias criteria to choose among (semantically different!) candidate solutions, i.e syntactic size, number of case distinctions, runtime (search strategy). Restriction Bias Restricts the inducable class of problems, through syntactic constraints, i.e. linear recursion as sole kind of recursion (hypothese language) Background Knowledge already implemented functions, which can by used for synthesis, i.e. append and partition for quicksort Sub Functions Functions neither defined as target functions nor in the background knowledge, but automaticallly introduced as auxiliary functions by the IP algorithm CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 3 / 22

7 Different Approaches analytic generate & test systematic logic DIALOGS-II FOIL/FFOIL, functional THESYS, IGOR I, IGOR II GOLEM MAGIC- HASKELLER evolutionary ADATE Inductive Logic Programming (ILP) Inductive Functional Programming (IFP) CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 4 / 22

8 Different Approaches analytic generate & test systematic logic DIALOGS-II FOIL/FFOIL, functional THESYS, IGOR I, IGOR II GOLEM MAGIC- HASKELLER evolutionary ADATE Inductive Logic Programming (ILP) ILP is machine learning with representation and inference based on Computational Logic (PROLOG). IP as special case of ILP. Inductive Functional Programming (IFP) CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 4 / 22

9 Different Approaches analytic generate & test systematic logic DIALOGS-II FOIL/FFOIL, functional THESYS, IGOR I, IGOR II GOLEM MAGIC- HASKELLER evolutionary ADATE Inductive Logic Programming (ILP) Inductive Functional Programming (IFP) Based Term Rewriting or Combinatory Logic / λ-calculus primary objective is program learning CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 4 / 22

10 Different Approaches analytic generate & test systematic logic DIALOGS-II FOIL/FFOIL, functional THESYS, IGOR I, IGOR II GOLEM MAGIC- HASKELLER evolutionary ADATE Analytic Generate & Test CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 5 / 22

11 Different Approaches analytic generate & test systematic logic DIALOGS-II FOIL/FFOIL, functional THESYS, IGOR I, IGOR II GOLEM MAGIC- HASKELLER evolutionary ADATE Analytic different inputs are sub problems of each other so their output is included in other outputs as subterms analyze I/Os and fold regularities into a recursive definition Generate & Test CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 5 / 22

12 Different Approaches analytic generate & test systematic logic DIALOGS-II FOIL/FFOIL, functional THESYS, IGOR I, IGOR II GOLEM MAGIC- HASKELLER evolutionary ADATE Analytic Generate & Test (1): systematic enumerate all correct programs systematically constraints limit search space (type information, library, modes) I/Os are only used as filter CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 5 / 22

13 Different Approaches analytic generate & test systematic logic DIALOGS-II FOIL/FFOIL, functional THESYS, IGOR I, IGOR II GOLEM MAGIC- HASKELLER evolutionary ADATE Analytic Generate & Test (2): evolutionary heuristic use genetic coperators to traverse search space fitness function maps programs to numeric space evaluated program attributes are e.g. runtime, program size, etc. CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 5 / 22

14 Different Approaches analytic logic DIALOGS-II FOIL/FFOIL, functional THESYS, IGOR I, IGOR II generate & test systematic evolutionary GOLEM MAGIC- HASKELLER ADATE large diversity of underlying theoretical concepts and requirements hard to compare and evaluate CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 6 / 22

15 Need for Unifying Framework Provide system independent syntax and operational semantics Benefits + consistent representation of different target languages + gives a unifying ( normalised ) perspecitve on IP systems + helps identifying system specific strength and weaknesses + provide a transparent evaluation and comparison of IP systems + basis for a general IP algorithm + means for an abstract problem definition language (IP Problem Definition Language) Conditional Constructor (Rewrite) Systems (CCS) CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 7 / 22

16 Need for Unifying Framework Provide system independent syntax and operational semantics Benefits + consistent representation of different target languages + gives a unifying ( normalised ) perspecitve on IP systems + helps identifying system specific strength and weaknesses + provide a transparent evaluation and comparison of IP systems + basis for a general IP algorithm + means for an abstract problem definition language (IP Problem Definition Language) Conditional Constructor (Rewrite) Systems (CCS) CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 7 / 22

17 Need for Unifying Framework Provide system independent syntax and operational semantics Benefits + consistent representation of different target languages + gives a unifying ( normalised ) perspecitve on IP systems + helps identifying system specific strength and weaknesses + provide a transparent evaluation and comparison of IP systems + basis for a general IP algorithm + means for an abstract problem definition language (IP Problem Definition Language) Conditional Constructor (Rewrite) Systems (CCS) CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 7 / 22

18 Need for Unifying Framework Provide system independent syntax and operational semantics Benefits + consistent representation of different target languages + gives a unifying ( normalised ) perspecitve on IP systems + helps identifying system specific strength and weaknesses + provide a transparent evaluation and comparison of IP systems + basis for a general IP algorithm + means for an abstract problem definition language (IP Problem Definition Language) Conditional Constructor (Rewrite) Systems (CCS) CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 7 / 22

19 Need for Unifying Framework Provide system independent syntax and operational semantics Benefits + consistent representation of different target languages + gives a unifying ( normalised ) perspecitve on IP systems + helps identifying system specific strength and weaknesses + provide a transparent evaluation and comparison of IP systems + basis for a general IP algorithm + means for an abstract problem definition language (IP Problem Definition Language) Conditional Constructor (Rewrite) Systems (CCS) CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 7 / 22

20 Need for Unifying Framework Provide system independent syntax and operational semantics Benefits + consistent representation of different target languages + gives a unifying ( normalised ) perspecitve on IP systems + helps identifying system specific strength and weaknesses + provide a transparent evaluation and comparison of IP systems + basis for a general IP algorithm + means for an abstract problem definition language (IP Problem Definition Language) Conditional Constructor (Rewrite) Systems (CCS) CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 7 / 22

21 Need for Unifying Framework Provide system independent syntax and operational semantics Benefits + consistent representation of different target languages + gives a unifying ( normalised ) perspecitve on IP systems + helps identifying system specific strength and weaknesses + provide a transparent evaluation and comparison of IP systems + basis for a general IP algorithm + means for an abstract problem definition language (IP Problem Definition Language) Conditional Constructor (Rewrite) Systems (CCS) CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 7 / 22

22 The paper at one glance C F T F B F I E + E BK X 2 search strategy ADATE { } global search, g n t FLIP, sequential covering FFOIL c, sequential covering GOLEM { } sequential covering IGOR I { } 2-step, global search IGOR II global search MAGH. { } breadth first, g n t unrestricted / conditional rules restricted / unconditional rules { } singleton set empty set c constants built in predicates CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 8 / 22

23 Empirical Results isort reverse weave shiftr mult/add allodds ADATE FLIP FFOIL 0.4 < GOLEM IGOR II MAGH lasts last member odd/even multlast ADATE FLIP FFOIL < 0.1 < 0.1 GOLEM < < IGOR II MAGH not tested stack overflow timeout wrong all runtimes in seconds CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 9 / 22

24 Our Project Publications Downloads Links CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 10 / 22

25 inductive-programming.org Introduction to IP Systems overview Repository with benchmark problems IP related publications Mailing list... CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 11 / 22

26 Thank you for your attention! CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 12 / 22

27 ... estion Questions? Questions? Questions? Questions? Questions? Questions? Questions? Questions? Questions? Questions? CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 12 / 22

28 CCS in a nutshell given a set of function symbols Σ and a set of variables X terms over Σ and X denoted as T Σ (X ) constructors C and defined function symbols F Σ = F C, F C = programs are sets of rewrite rules lhs rhs lhs is of the form F (p 1,..., p n ) with F F and p i T C (X ) conditional rewrite rules lhs rhs cond where cond {v 1 = u 1,..., v n = u n } and v i, u i T Σ (X ) rewriting binds free variables in v i, modelling variable declaration, let- and case-expressions higher-order context with X = X 1 X 2 and abstraction operator [ ] CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 13 / 22

29 CCS in a nutshell given a set of function symbols Σ and a set of variables X terms over Σ and X denoted as T Σ (X ) constructors C and defined function symbols F Σ = F C, F C = programs are sets of rewrite rules lhs rhs lhs is of the form F (p 1,..., p n ) with F F and p i T C (X ) conditional rewrite rules lhs rhs cond where cond {v 1 = u 1,..., v n = u n } and v i, u i T Σ (X ) rewriting binds free variables in v i, modelling variable declaration, let- and case-expressions higher-order context with X = X 1 X 2 and abstraction operator [ ] CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 13 / 22

30 CCS in a nutshell given a set of function symbols Σ and a set of variables X terms over Σ and X denoted as T Σ (X ) constructors C and defined function symbols F Σ = F C, F C = programs are sets of rewrite rules lhs rhs lhs is of the form F (p 1,..., p n ) with F F and p i T C (X ) conditional rewrite rules lhs rhs cond where cond {v 1 = u 1,..., v n = u n } and v i, u i T Σ (X ) rewriting binds free variables in v i, modelling variable declaration, let- and case-expressions higher-order context with X = X 1 X 2 and abstraction operator [ ] CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 13 / 22

31 CCS in a nutshell given a set of function symbols Σ and a set of variables X terms over Σ and X denoted as T Σ (X ) constructors C and defined function symbols F Σ = F C, F C = programs are sets of rewrite rules lhs rhs lhs is of the form F (p 1,..., p n ) with F F and p i T C (X ) conditional rewrite rules lhs rhs cond where cond {v 1 = u 1,..., v n = u n } and v i, u i T Σ (X ) rewriting binds free variables in v i, modelling variable declaration, let- and case-expressions higher-order context with X = X 1 X 2 and abstraction operator [ ] CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 13 / 22

32 CCS in a nutshell given a set of function symbols Σ and a set of variables X terms over Σ and X denoted as T Σ (X ) constructors C and defined function symbols F Σ = F C, F C = programs are sets of rewrite rules lhs rhs lhs is of the form F (p 1,..., p n ) with F F and p i T C (X ) conditional rewrite rules lhs rhs cond where cond {v 1 = u 1,..., v n = u n } and v i, u i T Σ (X ) rewriting binds free variables in v i, modelling variable declaration, let- and case-expressions higher-order context with X = X 1 X 2 and abstraction operator [ ] CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 13 / 22

33 CCS in a nutshell given a set of function symbols Σ and a set of variables X terms over Σ and X denoted as T Σ (X ) constructors C and defined function symbols F Σ = F C, F C = programs are sets of rewrite rules lhs rhs lhs is of the form F (p 1,..., p n ) with F F and p i T C (X ) conditional rewrite rules lhs rhs cond where cond {v 1 = u 1,..., v n = u n } and v i, u i T Σ (X ) rewriting binds free variables in v i, modelling variable declaration, let- and case-expressions higher-order context with X = X 1 X 2 and abstraction operator [ ] CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 13 / 22

34 CCS Target Languages in the CCS Framework multlast([]) -> [] multlast([a]) -> [A] multlast([a,b C]) -> [D,D E] <= [D E] = multlast([b C]) Haskell multlast [] = [] multlast [A] = [A] multlast [A,B C] = let [D E] = multlast([b C]) in [D,D E] Prolog multlast([], []). multlast([a], [A]). multlast([a,b C],[D,D E]) :- multlast([b C],[D E]). CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 14 / 22

35 The IP task in CCS function symbols F = F T F B F I user defined rules R = E + E BK restriction bias (lhs, rhs, u, v T Σ (X )) preference bias ( ) IP Task CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 15 / 22

36 The IP task in CCS function symbols F = F T F B F I F T F B F I function symbols of target functions function symbols of background knowledge pool of function symbols for inventing sub functions user defined rules R = E + E BK restriction bias (lhs, rhs, u, v T Σ (X )) preference bias ( ) IP Task CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 15 / 22

37 The IP task in CCS function symbols F = F T F B F I user defined rules R = E + E BK E + positive evidence F(t 1,..., t n ) r E negative evidence as inequalities F(t 1,..., t n ) r BK background knowledge F(t 1,..., t n ) r {v 1 = u 1,..., v n = u n } restriction bias (lhs, rhs, u, v T Σ (X )) preference bias ( ) IP Task CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 15 / 22

38 The IP task in CCS function symbols F = F T F B F I user defined rules R = E + E BK restriction bias (lhs, rhs, u, v T Σ (X )) Allow only a subset of T Σ (X ) for lhss, rhss, and conditions preference bias ( ) IP Task CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 15 / 22

39 The IP task in CCS function symbols F = F T F B F I user defined rules R = E + E BK restriction bias (lhs, rhs, u, v T Σ (X )) preference bias ( ) Partial ordering on terms, lhss, rhss, conditions, rules, and programs IP Task CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 15 / 22

40 The IP task in CCS function symbols F = F T F B F I user defined rules R = E + E BK restriction bias (lhs, rhs, u, v T Σ (X )) preference bias ( ) IP Task Find a set of rules R T s.t. R T BK = E + R T BK = E and R T is optimal w.r.t. restriction and preference bias. CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 15 / 22

41 Higher-Order Rewriting map([u]z(u),nil) map([u]z(u),cons(x,y)) -> nil -> cons(z(x),map([u]z(u),y)) more Terese p 612 CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 16 / 22

42 ADATE F T F B F I C unrestricted singleton unrestricted E + unrestricted E unrestricted BK unrestricted X 2 restr. bias subset of SML pref. bias user defined fitness function search str. global search, generate and test CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 17 / 22

43 F T F B C unrestricted F I unrestricted unrestricted E + unconditional FLIP E unconditional (may be empty) BK unrestricted X 2 restr.bias lhs is a consistent (w.r.t. evidence) but restricted (no new variables on rhs least general generalisation of two positive examples rhs is derived via inverse narrowing from two lhss pref. bias minimum discription length and coverage search str. heuristic search with sequential covering CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 18 / 22

44 FFOIL F T C constants, including {true, false} singleton F B {=,, <,, >,, } F I E + unconditional E unconditional (may be empty) BK unconditional X 2 restr. bias l, v {F(i 1,..., i n ) i i X 1, F F} r, u T Σ (X ) pref. bias foil gain search str. sequential covering CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 19 / 22

45 F T F B C {true, false} F I singleton unrestricted E + unconditional E unconditional BK unrestricted X 2 GOLEM restr. bias l, v {F(i 1,..., i n ) i i T Σ (X ), F F} r, u T Σ (X ) pref. bias clause with highest coverage in a lattice of least general generalisations relative to BK of randomly picked examples search str. sequential covering CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 20 / 22

46 F T F B C unrestricted F I unrestricted unrestricted IGOR II domain of invented function equals domain of calling function (no variable invention) E + unconditional E BK unconditional X 2 restr. bias non-overlapping lhss, rhs = F(...), F F I, conditions model only let-expressions pref. bias fewer case distinctions, most specific patterns, fewer recursive calls or calls to BK search str. best first CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 21 / 22

47 MAGICHASKELLER F T F B F I C unrestricted singleton unrestricted E + unrestricted E unrestricted BK unrestricted X 2 only via paramorphisms from BK restr. bias type constraints, composition of functions from BK pref. bias smallest w.r.t. BK search str. breadth first, generate and test CogSys Group (Univ. Bamberg) Inductive Programming AGI 2009, Arlington 22 / 22