An Action Model Learning Method for Planning in Incomplete STRIPS Domains

Transcription

1 An Action Model Learning Method for Planning in Incomplete STRIPS Domains Romulo de O. Leite Volmir E. Wilhelm Departamento de Matemática Universidade Federal do Paraná Curitiba, Brasil Abstract The dependence on complete STRIPS domain models limits the applicability of much of planning technologies. Under incompleteness hypothesis, some knowledge discovery technique is required to aid a planner in the task of building robust plans. This paper presents an Action Model Learning method that consists on overestimating domain s knowledge and posteriorly eliminating eventual inconsistencies based on correct representation rules, in order to discover improved domain models to be delivered for a complete-knowledge-based planner. An example illustrates how the method is applied to a STRIPS planning problem. Keywords automated planning; action model learning; incomplete domains I. INTRODUCTION The problem of planning based on incomplete STRIPS domain models consists on building plans that are solutions for planning problems even when the set of preconditions and effects of the operators are not completely known or stated. This problem arises from the toughness of building complete domains from scratch, which is usually a non-trivial and timeconsuming task. Most work on planning relies on domain completeness. Because of that, current planning techniques show a lack of robustness when facing incompleteness, i.e., have low accuracy in building plans that are effective solutions in a complete context. Works like [1], [2] and [5] suggest that such dependence on completeness represents a bottleneck for the applications of new planning technologies. When completeness cannot be guaranteed, an auxiliary source of information is needed to provide complementary knowledge about the domain. In Case-Based Planning, such source of information consists on a set of solved planning problems that are known to be valid for the complete domain. The main objective of the present work is to propose an action model learning method for gathering knowledge from a set of plan cases, in order to enhance the knowledge about a given STRIPS domain model. This is done by, in a first step, overestimating preconditions and effects of the operators when those are not stated. After that, a sequence of reasonableness rules is applied to eliminate practical and logical inconsistencies. The output is a domain model such that, for all operators, the lists of preconditions and effects contain all literals that are valid for the complete domain model, with possible overkills which are eventually not eliminated based on the information gathered from the set of plan cases. The present paper is organized as follows: section II makes a brief description of related work; section III is concerned about the formal definition for the problem of planning in incomplete domains; sections IV and V present the proposed method s algorithm and an experiment applying it to a planning problem in blocks world domain, respectively; and, at last, section VI contains a brief discussion about results and conclusions from the experiment. II. RELATED WORK There are two main approaches for the planning problem under incompleteness assumption; both apply principles of Case-Based Planning, i.e. make use of a set of correctly solved problems to obtain complementary information about the domain. One is Model-Lite approach, proposed by Kambhampati [6], which is concerned about building plans even when the input domain model is assumed to be incomplete and mining for additional knowledge from the set of plan traces. More on this approach can be found in [8], [10] and [13]. Another approach is Action Model Learning, focused on learning characteristics of the operators in the domain model for the application of some complete-knowledge-based planning technique, making use of available valid plan traces. Zettlemoyer, Pasula and Kaelbling [12] presented an action learning algorithm to deal with noisy and stochastic domains where states are fully observable; ARMS algorithm, proposed by Yang, Wu and Jiang [11] gathers knowledge on the statistical distribution of frequent set of actions in a set of plan traces to build a domain model from scratch; Walsh and Littman [9] addressed web-service composition problem using

2 a raw experience algorithm that constructs pessimistic action models in a procedure that is somewhat similar to the inflation phase in our proposed method; Creswell, McCluskey and West [3] presented the LOCM system to learn action models based only on the set of cases, without any partial domain as input, but requires a relatively large training to be efficient; LAMP algorithm [15] deals with action models with quantifiers and logical implications, but, just like ARMS, tries to build action models from scratch; RIM method use plan traces to learn micro-actions to refine incomplete domain models[14]. In this context, the present work brings up an action model learning method that aims to turn an incomplete model into an overcomplete model, the Domain Inflation and Reduction (DIR) method. Assuming that the lack of knowledge leads to non-accurate plans, the research question proposed is whether using the set of plan traces to eliminate exceeding literals from action models, instead of using it to compose those action models starting from incomplete ones, leads to more efficient learning and, consequently, more accurate solutions for planning problems on incomplete domain models. III. PROBLEM DEFINITION Let L be a function-free first-order language with finitely many predicate symbols and finitely many constant symbols; a STRIPS domain on L is defined as a restricted state-transition system = (S,A, ), such that S is a finite set of STRIPS states, A is a set of ground instances of some STRIPS planning operators, and is the transition function [4]. Note that, in a restricted STRIPS domain as just defined, the current state remains unchanged until the application of some action in A, that is, no events occur in such domain. Let O be the set of operators in ; an operator o O is defined as a 3-tuple (name(o), pre(o), eff(o)), where name(o) is the operator s name, pre(o) is the set of preconditions of o and eff(o) is the set of effects of o. The name of the operator o, name(o), is a syntactic expression of the form n(x 1,, x n ), where n is a unique symbol and x 1,, x n are all the argument variables to o; pre(o) and eff(o) are sets of literals. The set of effects of o has two disjoint subsets: eff + (o) is the set of positive literals in eff(o), also called the add list, and eff - (o) is the set of negative literal, also called the delete list. Only variables that are arguments of o are allowed to figure in the literals in pre(o) and eff(o). In essence, a STRIPS domain is described by its set of operators; so, the completeness of a domain depends on the completeness of its operators. An operator is said to be complete if there are no missing literal in its sets of preconditions and effects; in opposite, an operator is incomplete if there are some missing literals in those sets. Consequently, a domain that contains some incomplete operator is said to be an incomplete domain [13]. A STRIPS planning problem P is a 3-tuple (, s 0, g), such that is a STRIPS domain model, s 0 S is an initial state and g is a goal. A state in S is a set of ground atoms; so, s 0 is described by the set of ground atoms that are assumed to hold at the starting point. On the other hand, g is a set of literals that must hold at the final point. The goal g is satisfied by any state in s such that all literals in g hold. A solution to a STRIPS planning problem is a plan = {a 1,, a k }, where a 1,, a k is an ordered sequence of ground instances of the operators in O, i.e., a i A for i = 1,..., k. The definitions given so far in this section allow to formally define the incomplete domain planning problem: given an incomplete domain *, a planning problem P = ( *, s 0, g) and a set C of successful plan cases with respect to the complete domain, the objective is to find a solution for P that is correct with respect to. An example of incomplete STRIPS domain for blocks world is proposed by Zhuo, Nguyen and Kambhampati [13], as Table I shows. Underlined literals are those omitted from the original complete domain model. In addition, Table I contains the initial state and the goal for a specific planning problem and also shows the set of plan cases. TABLE I. AN EXAMPLE OF INCOMPLETE DOMAIN PLANNING PROBLEM Operator Preconditions Effects pickup (?x not () not () putdown (?x not () unstack (?x?y stack (?x?y Initial state (s 0 ) ontable D on C A Problem ID Initial state (P i ) (s 0 ) P 1 ontable D P 2 on C B Goal (g) on D A on A C on C B on C A on A B not () not () not () not () Goal (g) on B A on C B on D C Plan trace ( i ) {pickup C, stack C B, pickup A, stack A C, pickup D, stack D A} {unstack C B, putdown C, pickup A, stack A B, pickup C, stack C A}

3 IV. THE ALGORITHM In essence, DIR algorithm works in two phases. The first phase aims to inflate the lists of preconditions and effect of all operators in the domain, filling them up with all literals that are candidates to be missing ones. In a first step, the sets of effects of every operator is filled up and, in a second step, every plan trace is used to guide the inflation of the sets of preconditions. The given preconditions and effects are referred as fixed preconditions/effects, whereas the ones added on inflation phase are referred as inflated preconditions/effects. For every solved planning problems in C, starting from the respective initial state and assuming that it meets all preconditions for the application of the first action in the plan, the algorithm parses every predicate that has at most the same number of parameters as the current action; if there exist some literal stating the predicate with respect to some argument in the current action and if such literal is not a ground fixed precondition, the respective unground precondition is added to the set of preconditions of the current operator. After that, the current state is modified through the application of the inflated effects of the current operator. The inflation applied in the first phase produces a set of overfilled operators. However, under the hypothesis of finite predicate and variable symbols, it is possible to state that the complete sets of preconditions and effects are subsets of the sets in those operators on this stage. The inflation phase pseudocode is described by the Algorithm 1 (Table II). Inflation phase is critical in terms of complexity because of the construction of all possible candidates to be missing literals. Let A be an operator in a domain with t types of objects; the set of parameters of A may be composed of objects of s types, 0 s t. Hence, candidate literals are related to predicates which have at most s types of objects. In the worst case scenario, where s = t and all predicates has parameters of t different types, the number c(a) of candidate literals for an operator A is equal to Algorithm 1 TABLE II. INFLATION PHASE ALGORITHM Input: ( *, C) Output: an inflated domain model inf 0: inf = * 1: for every operator o in O do 2: add all the literals that are supposed to be missing to the set of inflated effects of o 3: end for 4: for every plan case P i in C do 5: current state = s 0 (P i ) 6: for every action in i do 7: add all literals that are supposed to be missing to the set of inflated preconditions of o 8: current state = (current state, current action) 9: end for 10: end for 11: return inf The first reduction step imposes constraints to operators individually. According to Yang, Wu and Jiang [11], for an operator to be correctly represented the intersection of its set of preconditions and its set of positive effects must be the empty set. Hence, under the assumption that all the knowledge contained in the given incomplete domain model actually stands for the complete domain, the first reduction step eliminates all inflated preconditions which, when compared to fixed effects, violate such constraint; the same is applied to inflated effects related to fixed preconditions. The first reduction pseudocode is described by Algorithm 2 (Table III). The second reduction step uses plan traces in C to detect inconsistencies in 1. It aims to eliminate inflated preconditions and effects that are not according to the order actions are applied in every trace, if 1 is assumed to be true. If a preceding action a i produces some effect l over some subset of objects in the list of parameters, then the negation of that effect, l, must not be a precondition of a subsequent action a i + 1, and vice-versa; so, when such situation is found, l or l must be eliminated. t i c(a) 4k m i 1 where k is the number of predicates in the domain model (multiplied by 4, because of the inflations of both sets of preconditions and effects with positive and negative literals), m i is the number of objects of type i in the set of parameter of A and n i is the number of objects of type i in the set of parameters of the predicates. Although theoretically possible, such scenario is extremely pessimistic in practice. The second phase is called the reduction phase and it is composed of at least three steps. These steps consist on a systematic application of some rules to eliminate exceeding literals which were added on the first phase. At least means that there are three steps that are applicable independently of the domain; for each case, some domain-specific rules may be applied for a more efficient reduction. n i TABLE III. Algorithm 2 FIRST STEP ON REDUCTION PHASE ALGORITHM Input: inf Output: a partially inflated domain 1 0: 1 = inf 1: for every operator o in O do 2: if a literal l is a fixed precondition and is an inflated effect of o then 3: eliminate l from the set of effects of o 4: else if a literal l is a fixed effect and an inflated precondition of o Then 5: eliminate l from the set of preconditions of o 6: end if 7: end for 8: return 1 The way the inflated domain inf is constructed may yield some contradictions into the sets of preconditions and effects of an operator. For example, l and l may appear in a same set in

4 1. In order to get rid of these contradictions, the second reduction step of DIR algorithm parses the connections between two consecutive actions and eliminates the literals that cause inconsistency. The second reduction step pseudocode is described by Algorithm 3 (Table IV). TABLE IV. Algorithm 3 SECOND STEP ON REDUCTION PHASE ALGORITHM Input: 1 Output: a partially inflated domain 2 0: 2 = 1 1: for every action plan trace j in C do 2: for every pair of consecutive actions a i and a i + 1 in j do 3: if (l is an effect of a i and l is a precondition of a i + 1 ) and ( l is an effect of a i and l is not a precondition of a i + 1 ) then 4: eliminate l from the set of effects of o i a 5: else if (l is an effect of a i and l is a precondition of a i + 1 ) and ( l is not an effect of a i and l is a precondition of a i + 1 ) then 6: eliminate l from the set of preconditions of o i + 1 a 7: end if 8: end for 9: end for 10: return 2 a. oi and o i + 1 are the operators whose a i and a i + 1 are ground instances, respectively. The objective of the third reduction step is to eliminate contradictions that eventually remain in 2. In the inflation phase, absent predicates are candidates to be missing; however, this may not be the case for some predicate. There are three possibilities: a literal l is indeed a missing one or; l is the actual missing literal or; the predicate to what l refers is not a missing predicate. So, if an eventual contradiction remains at this point, the algorithm discards both l and l, assuming that the predicate does not make part of the sets of preconditions or effects of the operator. The third reduction step pseudocode is described by Algorithm 4 (Table V). The three reduction steps presented so far are applicable to any domain model that fits the hypothesis described in section III. An additional reduction step may be applied to provide a better refinement of the domain model; however, some domain-specific knowledge is needed. The next section presents an example of application of the proposed algorithm in a planning problem on the blocks world domain where a fourth reduction step is applied in order to eliminate inflated literals that can be deduced from fixed ones. V. EXPERIMENT Given the problem described in section III, the DIR algorithm begins applying the inflation phase over all operators in the domain. The operator pickup (?x), for example, has three literals in the set of effects: not (), and not (). So, there is no statement about the predicates clear and on. For the predicate clear, there are two candidates to be missing literals: and ; so, both are added to the set of inflated effects of pickup (?x). On the other hand, the predicate on has more arguments than the operator pickup (?x); so, there is no candidate to be missing literals with respect to this predicate in the current operator. The same is applied to the remaining operators: putdown (?x), unstack (?x?y) and stack (?x?y). TABLE V. Algorithm 4 THIRD STEP ON REDUCTION PHASE ALGORITHM Input: 2 Output: a partially inflated domain 3 0: 3 = 2 1: for every operator o in O do 2: if there exist some literal l such that l and l are inflated preconditions then 3: eliminate l and l from the set of preconditions of o 4: else if there exist some literal l such that l and l are inflated effects then 5: eliminate l and l from the set of effects of o 6: end if 7: end for 8: return 3 The initial state s 0 in P 1 meets all the preconditions for the application of the first action in 1 (pickup (C)). Searching for missing preconditions in pickup (?x), it s found that: holds for s 0 and is not a fixed precondition add to the set of inflated preconditions of pickup (?x); holds for s 0 and it is already a fixed precondition; the predicate on does not apply, because it has more arguments than the operator pickup (?x); holding C does not hold in s 0 ; holds for s 0 and it is already a fixed precondition. The application of the inflated action pickup (C) in s 0 leads to another partially observable state s 1 such that, compared to the fully observable state s 1 which would be obtained if the complete domain were given instead *, contains all the literals that should hold in s 1. However, there are eventually literals in s 1 that should not hold in s 1. The transition from s 0 to s 1 is described by Table VI. TABLE VI. EXAMPLE OF TRANSITION WITH AN INFLATED ACTION s 0 pickup (C) s 1 ontable D - fixed preconditions: - inflated preconditions: - fixed effects: not holding C not - inflated effects: not not holding C

5 TABLE VII. putdown (?x EXAMPLE OF DOMAIN AFTER THE INFLATION PHASE Operator Preconditions Effects not () Fixed pickup (?x not () Fixed unstack (?x?y stack (?x?y Fixed Fixed not () not () Handempty not () Handempty not () not () not () not () holding?y not (holding?y) not () not () holding?y not (holding?y) not () not () not () holding?y not (holding?y) After the inflation phase, inf will be as shown on Table VII. Note that there are many antagonistic literals; for example, the set of effects of pickup (?x) has and not () as elements. However, both must not be valid simultaneously for the correct action model. So, the reduction phase algorithm takes inf as input and aims to eliminate such inconsistencies. The first step on reduction phase is to eliminate all inflated literals which are in the intersection of fixed preconditions and inflated effects and also in the intersection of inflated preconditions and fixed effects. For example, the operator putdown (?x) has as an inflated precondition and a fixed effect; hence, it must be eliminated from the set of preconditions. The second step algorithm searches for inconsistencies in the sequences of actions in the plan cases. For example, pickup (C) and stack (C B) are actions a 1 and a 2 in the plan 1, respectively. The literals and are effects of pickup (?x); on the other hand, only is a precondition of stack (?x?y). This leads to the conclusion that must not be an effect of pickup (?x); otherwise, stack (C B) would not be applicable right after pickup (C). Hence, must be eliminated from the set of effects of pickup (?x). TABLE VIII. RESULTING DOMAIN MODEL Operator Preconditions Effects not () pickup (?x not () putdown (?x unstack (?x?y stack (?x?y not () b not () not () not () b not () not () b. Differs from the complete domain model The third step is to eliminate remaining inconsistencies inside the set of inflated preconditions and the set of inflated effects in each operator. For example, the set of inflated effects of the operator unstack (?x?y) remains with both literals and not (). The rule at this point is to assume that eventual remaining inconsistencies indicates that the referred predicate is not likely to be present in the set of effects of unstack (?x?y) in the complete domain and eliminate both literals. So, and not () are both discarded. In addition a fourth reduction step is applied. It consists on the elimination of unnecessary literals, based on domainspecific axioms: for every operator, if all fixed preconditions imply an inflated precondition, this must be eliminated; on the other hand, if all fixed effects imply an inflated effect, this must be eliminated. For example, the operator putdown (?x) has as fixed precondition and and not () as inflated preconditions. It is known from the domain axioms that if holding x is true for some block x, it implies that is also true; the same can be said about not (). So, these two negative preconditions are not necessary, since imply both. After the application of the four reduction steps, the resulting domain model is that described by Table VIII. The resulting domain model is then used to replace * as the input domain to SATPLAN 2006 solver [7], in order to find a solution to the planning problem presented in the section III. The plan built by the solver is the sequence of actions {unstack C A, putdown C, pickup B, stack B A, pickup C, stack C B, pickup D, stack D C}, which is indeed a solution to the same planning problem on the complete domain. VI. CONCLUSIONS This paper presents DIR action model learning method for incomplete STRIPS domain problems. The experiment

6 reported shows one case in which the algorithm starts with a 31% incomplete domain model (8 missing literals in a total of 26) and yields an inflated model with 7,6% of exceeding literals (28 over 26 literals). These exceeding literals remained only in one of the four operators, what guarantees that plans which do not apply the operator unstack (?x?y) are actual solutions in the complete model. Furthermore, only in cases when this operator is required to apply on a state such that block y is over some other block the solver will not yield a correct solution. Another remarkable aspect is that a low number of plan cases was sufficient to provide a reasonable improvement of complete domain s knowledge. In addition, correct plans generated contribute to compose the set of successful cases that may be reused for further applications. However, the results of the present research are too limited to provide means to assess accuracy and robustness of the proposed technique. DIR algorithm is at present under computational implementation. Future work consists on tests with more complex domain, models with different degrees of incompleteness and sets of plan traces with different sizes. It is also needed to assess the scalability of solutions and to compare performances to state-of-art action model learning algorithms. ACKNOWLEDGMENT The authors would like to thank Prof. Dr. Fabiano Silva, Prof. Dr. Luis Allan Künzle and Prof. Marcos Alexandre Castilho from DINF/UFPR for the support on this research and CAPES for the financial subvention. REFERENCES [1] P. Bertoli, M. Pistore, and P. Traverso, Automated composition of web services via planning in asynchronous domains, Artificial Intelligence Journal, vol. 174, pp , [2] J. Blythe, E. Deelman, and Y. Gil, Automatically composed workflows for grid environments, IEEE Intelligent Systems, vol. 19, pp , [3] S. N. Cresswell, T. L. McCluskey, and M. M. West, Acquisition of object-centered domain models from planning examples, in Proceedings of ICAPS 19, pp , [4] M. Ghallab, D. Nau, and P. Traverso, Automated planning: theory and practice, Morgan Kaufman: San Francisco, 2004, pp [5] J. Hoffmann, P. Bertoli, and M. Pistore, Web service compositions as planning, revisited: in between background theories and initial state uncertainty, in Proceedings of AAAI Conference on Artificial Intelligence 22, pp , [6] S. Kambhampati, Model-lite planning for the web age masses: the challenges of planning with incomplete and evolving domain models, in Proceedings of AAAI Conference on Artificial Intelligence 22, pp , [7] H. Kautz, B. Selman, and J. Hoffman, Satplan: planning as satisfiability, in Abstracts of International Planning Competition 5, [8] T. A. Nguyen, S. Kambhampati, and M. B. Do, Assessing and generating robust plans with partial domain models, in ICAPS Workshop on Planning under Uncertainty, [9] T. J. Walsh, and M. L. Littman, Efficient learning of action schemas and web-service descriptions, in Proceeding of AAAI Conference on Artificial Intelligence 23, pp , [10] C. Weber, and D. Bryce, Planning and acting in incomplete domains, in Proceedings of ICAPS 21, pp , [11] Q. Yang, K. Wu, and Y. Jiang, Learning action models from plan examples using weighted MAX-SAT, Artificial Intelligence Journal, vol. 171, pp , [12] L. S. Zettlemoyer, H. M. Pasula, and L. P. Kaelbling, Learning planning rules in noisy stochastic worlds, in Proceedings of AAAI Conference on Artificial Intelligence 20, pp , [13] H. H. Zhuo, T. Nguyen, and S. Kambhampati, Model-lite case-based planning, in Proceedings of AAAI Conference on Artificial intelligence 27, pp , [14] H. H. Zhuo, T. Nguyen, and S. Kambhampati, Refining incomplete planning domain models through plan traces, in Proceedings of IJCAI 23, pp , [15] H. H. Zhuo, Q. Yang, D. H. Hu, and L. Li, Learning complex action models with quantifiers and logical implications, Artificial Intelligence Journal, vol. 174, pp , 2010.