Milestone State Formulation Methods

Transcription

1 Milestone State Formulation Methods Hyungoo Han Department of Computer Science & Engineering Hankuk University of Foreign Studies 89 Wangsan-ri Mohyeon Cheoin-gu Yongin-si, Gyeonggi-do South Korea Abstract- An intelligent robot generates a plan to achieve a goal in a problem domain. A plan is a sequence of robot actions that accomplish a given mission by being successfully executed. However, in the real world, a robot may encounter unexpected situations and may not execute its actions. A plan repairing method is required in such situations for a robot to accomplish its given mission. Two basic procedures for handling such situations are generating a new plan and repairing an existing plan. The re-planning procedure can cause time to be lost generating a new plan while discarding an existing one. The repair procedure must allocate a large storage area to preserve every expected state transformation to adjust the unexpected changes to normal states. Plan repair with milestone states is an alternative procedure to cope with the situation. It retains the advantages of the other two procedures. This paper proposes progressive and regressive methods of formulating milestone states. A method of assigning weighting values on conditions that compose a milestone state is also proposed. The task to repair a plan employs the weighting values as its job priority. The regressive method formulates less complex milestone states and leads to the conditions of a milestone state to take pertinent weighting values for an efficient way to repair a plan. Keywords: planning, intelligent agents, plan repair, milestone states, weighting values, artificial. I. INTRODUCTION Intelligent robots, like humans, generate a plan to achieve a goal in a problem domain before they execute it [1,2,3]. A plan is a sequence of robotic actions that accomplishes a given goal by being executed successfully. The planning process is a state transformation process from an initial state to a goal state. I have modified the definition of actions of the Stanford Research Institute Problem Solver (STRIPS) slightly [4]. Each action consists of four components: a precondition formula, a set of positive post-conditions, a set of negative post-conditions, and a set of still-conditions. The precondition formula is a conjunction of prerequisite conditions for an action to be triggered. Both the set of positive and negative post-conditions are created, as the result of firing an action; the positive set consists of true or new conditions and the negative set consists of false or deleted conditions The success of a plan is based on the consecutive and successful executions of the individual actions of a robot. However, in the real world a robot may be confronted with unexpected situations termed error states in this paper - that may occur due to device failures or malfunctions, inconsistent sensing data, or unanticipated environmental changes [3]. Planning to consider all possible error states is almost impossible; a robot should be able to fix the error states whilst executing the plan [3,5]. The two basic approaches for handling error states are generating a new plan and repairing an existing plan. [3,6,7]. The re-planning approach discards the current plan and regenerates a plan in its current state, when an error state occurs [3,7]. This procedure builds a new plan that transforms the error state to the goal state; the plan repairing process is simple. However, this re-planning approach can lose time, since it builds a new plan, discarding the existing plan. The second approach is regionally repairing a plan [3,8]. A robot must know the exact descriptions of an expected normal state, which would have been produced if the error state had not occurred, to regionally repair a plan when an error state is encountered. The regional repair task is the process of building a partial plan, which transforms the error state to the expected normal state, without discarding the current plan. Even though this method has the advantage of reusing the existing current plan, it must incur a large cost to store those expected normal states. A robot may keep track of every state transformation and produce each expected normal state internally, whenever it triggers an action, not to store all the expected states. However, this process will confuse a robot in a real time environment of plan execution, because the process of tracking state transformations to produce expected states is not simple; it is time consuming. Plan repair with milestone states is another procedure [6]. This procedure has the benefits of the previous two procedures. The idea of this procedure is not to store all the expected normal states, unlike the regional plan repair procedure, but selecting and storing an appropriate number of expected normal states, as milestone states. With this procedure, a robot will choose a milestone state, which appears behind and is nearest to the current error state, and build a partial plan to transform the error state to

2 the chosen milestone state, discarding actions between the current state and the milestone state only. This paper proposes progressive and regressive methods to formulate milestone states for the last procedure. A method for assigning weighting values on conditions that compose a milestone state is also proposed. The process for repairing a plan employs the weighting values as its job priority. The regressive method formulates less complex milestone states and leads the conditions of a milestone state to take more pertinent weighting values for an effective and efficient handling procedure to repair a plan. II. RELATED WORK The three procedures to handle unexpected environmental changes are stated in section I. This section details the plan repair with milestone states. In general, the regional plan repair procedure is more efficient for fixing an error state than is the re-planning procedure. However, theoretically the former is not always better than the latter [8,9]. The re-planning procedure wastes the existing plan and consumes more time to generate a new plan. The regional plan repair procedure also must store all the expected normal states. In contrast, the procedure with milestone states reuses the existing plan, as much as possible, and stores as few of the expected normal states as possible. Fig. 1 depicts the expected normal states and selected milestone states of a plan. The circles, regardless of their size, denote the expected normal states. Executing the plan transforms the initial state S 1 to the goal state S n in a given problem domain, when all the actions of the plan are triggered consecutively and successfully. Action a i is triggered in state S i and produces S i+1. That is, if a robot has to consume much effort to generate a new lengthy plan. The robot should know the perfect descriptions of the expected normal state with the regional plan repair procedure, as mentioned in the previous section. In Fig. 1, every 6 th state is selected as a milestone state. Determination of the appropriate number of milestone states to repair error states is not within the scope of this paper. Note that when no milestone state is selected, the repair procedure with milestone states becomes the re-planning procedure and all the states are selected as milestone states, the procedure becomes the regional plan repair procedure. If a robot selects more milestone states it must allocate more storage and, if it selects less milestone states, it has to discard more actions and spend more time to generate a partial plan. Therefore, it is not a simple task to determine how many milestone states should be selected to repair error states; this may depend on different factors in problem domains. III. TWO METHODS TO FORMULATE MILESTONE STATES III-1. Problem domain and a blocks world I employed the blocks world in Fig. 2 as a problem domain to describe how the two methods work to formulate milestone states. It is also used to compare the efficiency of their roles in repairing error states and explain the mechanism of assigning weighting values on conditions in the problem domain. In the figure, the plan with 28 actions is generated from the given initial and goal state descriptions. Box B and box C are depicted with dotted lines, because the conditions related to the two Fig. 1. A plan space and milestone states recognizes that every precondition of a i is sound as expected in S i, it triggers a i in S i with transforming S i to S i+1. In Fig. 1, every 6 th state is selected as a milestone state for plan repair. Big circles with M including the goal state S n in the figure are milestone states and will be stored in the robot. Suppose that a robot finds that it cannot trigger action a k in the current state S k, i. e. the S k is an error state, it will choose the first following milestone state M i+1 and generate a partial plan that will transform the error state S k to M i+1. Under this situation, with the re-planning procedure the robot will throw away the entire plan and generate a new plan that transforms S k to the goal state S n. Note that when a k is in the forepart of the plan the robot Fig. 2. Blocks world problem domain

3 boxes are irrelevant to the goal state and the goal state description in the figure does not contain any conditions relevant to the two boxes. Some constraints are imposed in the problem domain. Two different actions, Pick and Unstack, are used to hold a box in the hand of the robot. Unstack(X,Y) is to remove X from Y when Y is a box and both robot and Y are on the same object. Conversely, Pick(X,Y) picks up X from Y when both the robot and X are on Y. Stack(X,Y) and Put(X,Y) are the actions to place X on Y. Stack(X,Y) stacks X on Y when the robot and Y are on the same object. Put(X,Y) is to place X on Y when the robot is on Y. The robot must empty its hand to hold a box, climb, or come down a box. Only one box can be placed on the shelf; to reach a box on the shelf, the robot must stack box A on box D and get on the pile of the two boxes. The robot cannot reach a box on a pile of multiple boxes. The robot, box A, and another box, or the robot and two boxes other than box A, can be placed on box D simultaneously. The robot and a box, or two boxes other than box D, can simultaneously be on box A. There is always room on the floor for the robot and boxes. III-2. State space modeling of a plan A conceptual model for state transition T is defined as a 4-tuple system, since the execution of a plan is concerned with firing actions to change the states of a problem domain. T = (S, A, C, τ)[10]. S is a finite set of domain states, A is a finite set of robot actions, C is a finite set of conditions that comprise the domain states, and τ: S A C S is a state transition function. τ is represented as τ(s i, a i, precond(a i )) s i+1. When the preconditions of an action a i, which are denoted as precond(a i ) are satisfied in the state s i, τ triggers a i in s i, and causes the state transition from s i to s i+1. This new state s i+1 consists of postcond(a i ), postcond - (a i ), and stillcond(a i ). Postcond(a i ) has conditions that are satisfied in compliance with action a i. Postcond - (a i ) has conditions that may be deleted or negated with action a i. Stillcond(a i ) has conditions that are members of state s i, and irrelevant to the execution of the action a i. This stillcond of an action has the conditions created by some previously executed actions and may have preconditions of following actions. The length of a plan is the number of the actions that compose a plan. The number of produced states exceeds that of the plan length by one, when the execution of all actions of the plan is carried out successfully. Both the progressive and the regressive methods produce the same number of states. A milestone state is one of the domain states produced by applying function τ to the actions of a plan. An appropriate number of domain states are selected as milestone states. Determination of an appropriate number may be a distinct research topic of the plan repair procedure with milestone states. Therefore, the methods to build a pool of domain states, from which the milestone states will be selected, are proposed in this paper. III-3. Progressive method The progressive method transforms domain states forward consecutively. A progressive transition function τ f to produce states forward is defined as τ f : S A C F. F is a finite set of domain states produced by applying τ f to actions of A and is a sub set of S. A progressive transition function τ f is defined as: τ f (f i, a i, precond(a i )) f i+1, where f i+1 = (postcond(a i ) postcond - (a i )) stillcond(a i ), f i+1 = f i+1 ( c i f i+1, (c i f i+1 ) ( c i f i+1 )). Note that f i and f i+1 are the elements of F, f 1 is an initial state, and f n+1 is a goal state that is produced with τ f (f n, a n, precond(a n )) when the length of a plan is n. The last expression states that a condition and its negation cannot reside in a state simultaneously. The stillcond of an action consists of both the irrelevant conditions to the action and the conditions unaffected by the execution of the following actions, up to the next milestone state. In some cases, a stillcond of an action may have conditions that remain unchanged by the following actions up to the goal state. These unaffected conditions naturally exist in the milestone states that are formulated in compliance with the progressive method. The unaffected conditions will appear more in the milestone states as the plan execution proceeds, since it is possible that some of the postcond or postcond - of an action can become the stillcond of following actions. Furthermore, these phenomena will be amplified, when a given problem domain is complex and the domain states become more complex. Some unaffected conditions may not be preconditions of the following actions or unnecessary conditions for a robot to accomplish its mission. Meanwhile, the plan repair task must generate a complete partial plan that satisfies all the conditions of the milestone state chosen to fix an error state, regardless of their necessity. Therefore, a robot may make additional effort to generate and execute a partial plan, with the milestone states formulated by the progressive method. It will be hard to expect an efficient way to repair a plan. Table 1 shows example states produced by the two methods. Both methods will produce 29 states with the example blocks world domain in section III-1. III-4. Regressive method The regressive method transforms domain states backward consecutively. A regressive transition function τ b to produce states backward is defined as τ b : S A C B. B is a finite set of domain states produced

4 byapplying τ b to actions of A and is a subset of S. When the plan length is n, b 1 and b n+1 are elements of B and the initial state and the final state, respectively, as for the progressive transition function in the previous section. TABLE 1. Example states States produced by States produced by progressive method regressive method on(r,fl) hempty on(a,fl) on(r,fl) hempty on(a, on(b,a) clear(b) on(c,sf) FL) on(b,a) clear(b) f 1 clear(c) on(d,fl) clear(d b 1 on(c,sf) clear(c) on(d, ) on(e,a) clear(e) FL) clear(d) on(e,a) clear(e) on(r,fl) on(a,fl) clear(a) on(r,fl) on(a,fl) clea f hold(b) on(c,sf) clear(c) 2 on(d,fl) clear(d) on(e,a) b r(a) hold(b) on(c,sf) 2 clear(c) on(d,fl) cle clear(e) ar(d) on(e,a) clear(e) on(r,fl) hempty on(a,d) on(r,fl) hempty on(a, f clear(a) on(b,fl) clear(b) 7 on(c,sf) clear(c) on(d,fl) b D) clear(a) on(c,sf) 7 clear(c) on(d,fl) clear( clear(d) on(e,fl) clear(e) D) on(e,fl) clear(e) f 8 on(r,d) hempty on(a,d) cl ear(a) on(b,fl) clear(b) o n(c,sf) clear(c) on(d,fl) clear(d) on(e,fl) clear(e) on(r,fl) hold(a) on(b,fl) f clear(b) on(c,fl) clear(c) 28 on(d,fl) clear(d) on(e,s b 28 F) clear(e) f 29 A regressive transition function τ b is defined as: τ b (b i, a i-1, precond(a i-1 )) b i-1, where b i-1 = precond(a i-1 ) ( c i b i, c i (postcond(a i-1 ) postcond - (a i-1 ))), b i-1 = b i-1 ( c i b i-1, (c i b i-1 ) ( c i b i-1 )). b i-1 must have precond(a i-1 ), which is the set of prerequisite conditions of ai i-1, since action a i-1 is to be triggered in state b i-1 to produce state b i. State b i-1 must not have the postcond(a i-1 ) or the postcond - (a i-1 ), which are in state b i, because the execution of a i-1 will create these conditions and produce state b i. The last expression states that a condition and its negation cannot reside in a state simultaneously. The stillconds of actions will not have unnecessary conditions, unlike the progressive method, but have the conditions that compose the goal state or preconditions of following actions instead, since the regressive function produces states backwards. Therefore, the unnecessary conditions may also disappear in the milestone states selected from those states produced by the regressive method. The plan repair task with this method will be much lighter and simpler than the task with the progressive method. The lengths of the partial plans to repair the error state in Fig. 3 reveal the efficiency of the regressive method. III-5. An example for repairing an error state Fig. 3 shows an example of an error state and a selected milestone state. The robot is about to trigger the 8 th action, Geton(D,A), of the plan in Fig. 2 and has encountered an error state, as in Fig. 3. Since the precondition, on(r, D), b 8 on(r,fl) hempty on(a,fl) clear(a) on(b,fl) clear(b) on(c,fl) clear(c) on(d,f b 29 L) clear(d) on(e,sf) clear( E) on(r,d) hempty on(a,d ) clear(a) on(c,sf) cl ear(c) on(d,fl) clear(d ) on(e,fl) clear(e) on(r,fl) hold(a) on(d, FL) clear(d) on(e,sf) clear(e) on(r,fl) hempty on(a, FL) clear(a) on(d,fl) clear(d) on(e,sf) cle ar(e) of the action is not true in the error state, the robot cannot trigger the 8 th action. The robot will choose the milestone state in Fig. 3 and generates a partial plan that will transform the error state to the milestone state. Two different forms, Mf 2 by the progressive method and Mb 2 by the regressive method, are described below the picture of the milestone state, from Table 3 in section IV-1. The partial plan for Mf 2 has more actions to repair the error state than the partial plan for Mb 2 Fig. 3. An error state and plan repair III-6. Weighting values of milestone state conditions A weighting value is assigned to each condition composing a milestone state to provide a guide to fix an error state with the milestone states. The task will be able to fix error states efficiently, since the order of fixing the conditions for the plan repair task can be determined by the weighting values. The principle of assigning weighting values on conditions is that for a condition appearing in two consecutive milestone states simultaneously, the condition appearing in the latter milestone state gets additional weighting values. Note that an initial state is employed as a milestone state to pair up to the first milestone state. The conditions appearing in two consecutive milestone states simultaneously may not be the preconditions of the actions between the two milestone states, but may be the preconditions of the following actions of the latter milestone state or the conditions that compose the goal state. These conditions are usually created by the actions that are executed before the first milestone state of the two consecutive milestone states. The continuously appearing conditions may be preserved during the execution of the actions between the two consecutive milestone states and have higher weighting values than other conditions of the rear milestone state, since the unnecessary conditions are removed from milestone states formulated by the

5 regressive method. The following expressions assign weighting values on conditions when it is assumed that W is a weighting value of a condition, RC is a condition related to a robot, and c is a condition of a milestone state M. c i M i, (c i M i+1 c i RC) W(c i M i+1 ) + 2 c i M i, (c i M i+1 c i RC) W(c i M i+1 ) + 1 c i M i+1, c i M i W(c i M i+1 ) = 0 RC is increased by 1 not 2, as a robot is the main agent handling the domain conditions and it has to handle the conditions that are not related to it, prior to handling the RC. When a new condition appears in the rear milestone state, a zero weighting value is given to it. IV. COMPARISON OF THE TWO METHODS IV-1. Complexity comparisons In Table 2, the length of the plan is the number of actions of the plan in Fig. 2, the number of states is the number of states produced by the simulated execution of the plan, and the number of conditions is the total number of conditions of 29 states. Note that the 29 states will be produced by the execution of the 28 actions in the plan. The number of conditions and the average number of state conditions in the table show that the regressive method produces states that are less complex than are those of the progressive method. That is, fewer conditions are to be repaired when a plan repair task is needed. Methods Progressive method Regressive method TABLE 2. Complexity of states Length of Number of Number of plan states conditions Average Table 3 shows the two types of milestone states selected from the 29 states that are produced by progressive and regressive transit functions, with the plan in Fig. 2. In this paper, every 4 th state, including the goal state, is chosen as a milestone state from the 29 states. The goal state is the last milestone state; there are 8 milestone states. In Table 3, Mf i and Mb i are the names of the milestone states formulated by the progressive and the regressive methods respectively, where i is an index of a milestone state and an integer between 1 and 8. There must be a method to pinpoint a milestone state, when an error state is encountered. The integer i becomes the index of the chosen milestone, when it satisfies the following expression, where k is the index of an action encountering an error state and p is a constant used to select milestone states. min(k < i p + 1) From the problem domain, if action 6 has encountered an error state, i will be 2 by the expression min(6 < i 4 + 1), and the second milestone state is chosen for plan repair with milestone states. TABLE 3. Milestone states Progressive method Regressive method on(r,fl) on(a,fl) clear( on(r,fl) on(a,fl) clear( Mf A) on(b,fl) clear(b) on 1 Mb (C,SF) clear(c) on(d,f), 1 A) on(c,sf) clear(c) on( D, clear(d) hold(e) FL) clear(d) hold(e) on(r,d) hempty on(a,d) clear(a) on(b,fl) clear on(r,d) hempty on(a,d) Mf 2 (B) on(c,sf) clear(c) o Mb clear(a) on(c,sf) clear(c) 2 on(d,fl) clear(d) on(e, n(d,fl) clear(d) on(e,f ) clear(e) FL) clear(e) Mf 3 on(r,d) hempty on(a,d) clear(a) on(b,fl) clear (B) on(c,a) clear(c) on( D,FL) clear(d) on(e,fl) clear(e) clear(sf) on(r,fl) on(a,d) clear( Mb 3 A) on(b,fl) clear(b) ho Mf 4 ld(c) on(d,fl) clear(d) Mb 4 on(e,fl) clear(e) clear(s F) on(r,d) hempty on(a,d) clear(a) on(b,fl) clear Mf 5 (B) on(c,fl) clear(c) o Mb 5 n(d,fl) on(e,d) clear(e) clear(sf) on(r,a) on(a,d) clear(a ) on(b,fl) clear(b) on( Mf 6 C,FL) clear(c) on(d,fl) clear(d) hold(e) clear(s F) on(r,fl) hold(a) on(b,f Mf L) clear(b) on(c,fl) cle 7 ar(c) on(d,fl) clear(d) on(e,sf) clear(e) on(r,fl) hempty on(a,f L) clear(a) on(b,fl) cle Mf 8 ar(b) on(c,fl) clear(c) on(d,fl) clear(d) on(e,s F) clear(e) on(r,d) hempty on(a,d) clear(a) on(c,a) clear(c) on(d,fl) clear(d) on(e, FL) clear(e) clear(sf) on(r,fl) on(a,d) clear(a) hold(c) on(d,fl) clear( D) on(e,fl) clear(e) clea r(sf) on(r,d) hempty on(a,d) clear(a) on(d,fl) clear(d) on(e,d) clear(e) clear(s F) Mb 6 on(r,a) on(a,d) clear(a) on(d,fl) clear(d) hold( E) clear(sf) Mb 7 on(r,fl) hold(a) on(d,fl ) clear(d) on(e,sf) clear( E) Mb 8 on(r,fl) hempty on(a,fl ) clear(a) on(d,fl) clear( D) on(e,sf) clear(e) IV-2. Weighting value comparisons The weighting values of the milestone states formulated by the progressive and the regressive methods are assigned in the same way as in section III-6. Table 4 shows the weighting values assigned to the conditions of the milestone states in Table 3. The conditions of states are the domain conditions that appear in all 29 states. In this table, each milestone state is composed only by the conditions with weighting values 0, 1, or 2. The highlighted part of the table shows that in the progressive method, high weighting values are assigned to the unnecessary conditions related to box B and box C. In contrast, in the regressive method, those conditions have low weighting values or do not appear in the milestone states. This means that the unnecessary conditions disappear in the list of conditions to fix an error state. The error state repair task will try to compose a partial plan for the conditions of a chosen milestone state in the order of their weighting values. Therefore, the regressive method will guide the error state repair task to be more efficient than will the

6 progressive method. Conditions TABLE 4. Weighting values of conditions Progressive method Milestone states Regressive method on(r,fl) on(r,a) 0 0 on(r,d) hempty hold(a) 0 0 hold(b) hold(c) 0 0 hold(e) on(a,fl) on(a,d) clear(a) on(b,fl) on(b,a) clear(b) on(c,fl) on(c,a) 0 0 on(c,d) on(c,sf) clear(c) on(d,fl) clear(d) on(e,fl) on(e,a) on(e,d) 0 0 on(e,sf) clear(e) clear(sf) future plan repairing methods with milestone states. REFERENCES [1] Yongtae Do et al., Artificial Intelligence Concepts and Applications, 3rd Edition, SciTech, Seoul Korea, 2009 [2] Malik Ghallab, Dana Nau, Paolo Traverso. Automated Planning Theory and Practice, Morgan Kaufmann Publishers, New York, 2004 [3] Hyungoo Han, Kai Chang, William Day, A Comparison of Failure Handling Approaches for Planning Systems Replanning vs. Recovery, Journal of Applied Intelligence, Vol. 3, Kluwer academic publishers, , 1993 [4] Richard E. Fikes, Nils J. Nilsson, STRIPS: A New Approach to the Application of Theorem Proving to Problem Solving, Artificial Intelligence (2), pp , 1971 [5] Jianming Guo, Liang Liu, A Study of Improvement of D* Algorithms for Mobile Robot Path Planning in Partial Unknown Environments, Kybernetes Volume 39, Emerald Group Publishing Limited, Issue 6, pp , 2010 [6] Hyungoo Han, Plan repair with Milestone States, Institute of Information Industrial Engineering, Hankuk University of Foreign Studies, Vol. 13, pp , 2009 [7] Bernhard Nebel, Jana Koehler, Plan Reuse versus Plan Generation: A Theoretical and Empirical Analysis, Artificial Intelligence, 76, pp , 1995 [8] Roman van der Krogt, Mathijs de Weerdt, Plan Repair as an Extension of Planning, ICAPS, pp , 2005 [9] C. A. Broverman and W. B. Croft, Reasoning about Exceptions during Plan Execution Monitoring, Proc. Natl. Conf. Artificial Intelligence, Seattle, WA, pp , 1987 [10] T. Dean and M. Wellman, Planning and Control, Morgan Kaufmann, 1991 V. CONCLUSION This paper proposes the progressive and regressive methods to formulate milestone states and the method to assign weighting values on the conditions that compose a milestone state. The progressive method forces a milestone state to have unnecessary conditions to repair error states, while the regressive method does not. Therefore, the regressive method usually formulates smaller and less complex milestone states to fix the error states. When a repair list of conditions becomes longer, the length of a new partial plan to repair the conditions on the list also becomes larger. A robot has to spend more time to generate a larger partial plan and to execute the larger partial plan. Unlike the progressive method, the regressive method causes the unnecessary conditions to assume low weighting values or disappear from the milestone states. The task to fix error states employs the weighting values as its job priority. Therefore, the regressive method to formulate milestone states provides an efficient way to repair plans. The proposed method is unsuited to path selection plans. A study on topics, such as the similarity between milestone states, the degree of proximity of locations and domain states, the heuristic approaches to repair error states, and so forth, will deserve a place in