Representation of Action Spaces in Multiple Levels of Detail

Transcription

1 Representation of Action Spaces in Multiple Levels of Detail Andreas Hasselberg Dirk Söffker Institute of Flight Guidance, German Aerospace Center, Braunschweig, Germany ( Chair of Dynamics and Control, University of Duisburg-Essen, Germany ( Abstract: Operators of complex dynamic systems like industrial process control or air traffic control should be able to predict future system states and also the actions needed to reach their goals. Models of human planning processes suggest that humans use plans with various levels of details to guide their actions. An assistance systems that would be able to propose plans in multiple levels of detail could lead to a better understanding of plans and thus to a better performance of the operator, because of the possibility to represent plans graphically similar to the operators mental model. In principal the visualization of abstract plan is more robust to changes of details, additionally less changes of visualized plans are necessary. This contribution describes a method to compute multiple representations of action spaces with different levels of detail, which could be used in an assistant system. The method assumes that the process can be described by discrete events and thus the action space of the process contains discrete states. The method is based on the state space generated by a Coloured Petri Net model. First the state space of this model will be computed, in the next step intermediate goal states will be identified, which will constitute the states of a more abstract state space. The arcs connecting these states will be generated according to the reachability of the states in the detailed state space. This procedure can be repeated and finally a representation of an action space in multiple levels of detail results. Keywords: human-centered design, state-space, petri-net, discrete systems, data reduction, search methods 1. INTRODUCTION In many domains human operators are working with complex dynamic systems. Working field examples are the supervision of industrial processes or power plants, air traffic control, or anesthesia. Human operators of complex dynamic systems should be able to predict future states of the system as well as the consequences of their actions to make good decisions, to avoid errors, malfunctions, and accidents, see for example Dörner (1989) and Söffker (2003). Thereby they have to deal with uncertainties when determining the current system state and especially when projecting future system states. Models of human planning processes described that one of many strategies to deal with uncertainties is to use flexible plans with an adaptable level of detail. For example in the model of Mumford et al. (2001), it is assumed that a non-specific plan is created first that is worked out in more detail later. The advantage of a non-specific plan is its good adaptability. Parts of a plan lying further ahead can be planned with a lower level of detail to deal with inaccurate forecasts of future system states. If it is possible to make more accurate predictions after some time, the details of a plan may be worked out. Through the use of adaptable plans additional cost can be avoided, which where necessary if parts of the plan would have to be abandoned and regenerated according to the updated predictions. The opportunistic model of human planning by Hayes-Roth and Hayes-Roth (1979) distinguishes between four levels of abstraction, which are arranged hierarchically. At the top level, the goals are defined (outcomes). The next level contains the design, which describes the general behavioral approaches. The procedures, which define sequences of actions, are following on the third level. On the last level, the actions are defined. Some studies (for example Amalberti (1992)) have reported that plans can even remain abstract. They can be used to lead acting without defining the actions in detail. In order to assist the human operator dealing with complex dynamic systems, various assistance systems have been developed in the last years, allowing the prediction of future states of the system and suggesting plans. If an assistance system gives an exact forecast and the controlled system is disturbed, this can unfortunately lead to a rejection of the plan so replaning becomes necessary. Frequent replanning may hinder the operator s understanding of the proposed plan and additionally may also increase the error rate. An assistance system imitating the human planning and proposing a flexible and adjustable plan can adapt to occurring disturbances and will be able to propose a plan according to the operator s mental model and thus make a better understanding of the plan possible. This could have

2 Fig. 1. A simple state space with two level of detail a positive effect on the operator s performance and reduce the error rate. The prerequisite for such an assistance system is the ability to represent the action space in different levels of detail. In this paper it will be demonstrated how this can be achieved. The method described in this contribution assumes, that the task can be modeled by discrete events and thus the action space can be described as a discrete state space. The method will be presented in section 2. In section 3 the method will be demonstrated on an example task. The Results of this demonstration are given in section 4. Possible improvements of the presented method are given in section 5. Section 6 draws a conclusion. 2. METHOD This section presents a method to calculate multiple representations of action spaces in different levels of detail according to the mental plans of human operators. As stated above, humans can use intermediate goals to structure their plans. Accordingly this method will also use possible intermediate goal states to structure the action space. This method assumes a Petri Net model of the process so that the complete state spaces can be calculated. A state space is a directed graph that represents all possible states of a system as nodes and the connections between them as arcs. How the presented method can be extended so that it can be applied without calculating a complete state space will be discussed in section 5. To get action spaces with multiple levels of details, in a first step the state space of the Petri Net model has to be computed. The state space is the set S of all markings m reachable form the initial marking m 0. The transitions in the Petri Net should model single actions so that the state space contains all possible action sequences and their consequences in detail. Such a state space describes the action space of the human operator and can be used to identify action sequences that could be applied to solve the given task (Hasselberg et al. (2009)). An example state space is shown in the lower part of figure 1. This will be called detailed state space or level 1 state space and denoted as S 1. To be able to generate a more abstract representation of the state space (S 2 ), a method to identify states which can be used to structure the state space has to be applied. For example in Möhlenbrink et al. (2008) the state space was represented graphically more abstract by only considering the states with more than on successor. Thus situations in which decisions were necessary were used to structure the abstract state space. In this contribution, possible goals will be used to structure the state space in multiple levels of detail according to the mental model of the human operator. Intermediate goals will be identified by specific characteristics of states. It may be required that the characteristics have specified values, but also that a function applied to the characteristics has a particular outcome. A state with the marking m will be identified as goal g, if the marking satisfy the condition C g (m). Additionally the characteristics of the surrounding states may be important. If for example the goal is to execute a specific step, this could only be identified by analyzing two adjacent states. This can also be included in the condition C g (m), which can demand that a state can be transformed into a certain successor state, which in turn must also meet a certain condition. In other cases the path on which the goal states are reached could be important. Here every state in the path from an initial state m gk to the intermediate goals state m g could have to meet a condition C gk (m gk ) with k = K... 0, whereby K is the number of states in the path. These conditions can be summarized to the condition C g. A number of goals can exist in a task. Therefore these multiple goals have to be defined. In addition, the goals may depend on the particular state, so that at some states specific goals are active, while other goals are active at different states. An activation criterion a(m) has to be defined which specifies the activated goals based on the marking of the actual state. When calculating a more abstract state space (level 2) firstly the initial state m 0 of the detailed state space will also be defined as the initial state of the abstract state space (see node 1 in figure 1). Based on the activation criterion a(m) the active goals have to be determined. The possible goal states are then identified by a search algorithm, which start from the initial state. Then the search algorithm finds all states meeting the condition C g of one of the active goals. These goals have to be reachable directly, that means without another intermediate goal

3 state in between. The states identified as possible intermediate goals will then be added to the more abstract state space. In the example in figure 1 node 5 is identified as goal state. The dotted lines indicate, that the characteristics of the predecessor node 4 und the successor node 13 where used to identify node 5 as goal state. Also the path between the initial state and the goal states is added to the abstract state space and represented by an arc between these two states (arc between node 1 and 5 in the example). Starting with the goals identified in the first step, further goals are searched starting from the already identified goals. The identified goals and paths are added as nodes and arcs to the abstract state space. In the example the search starts from node 5 and finds the next intermediate goal with node 9. In this case the characteristics of node 15 and 9 where used to identify this goal state. The steps can be repeated until all goals are identified and are added to the abstract state space. A state space results consequently, which can describe the action space, but does not specify the particular actions. The resulting abstract state space S 2 is thus a subset of the detailed state space, S 2 S 1. This process can be repeated using the calculated level 2 state space as input to calculate an even more abstract state space S 3 (level 3), whereby S 3 S 2 S 1. For this repetition intermediate goals have to be defined on an higher level. The amount of repetitions possible depends on the amount of different abstraction levels on which the goals can be defined. Thus the definition of goals essential influences the possible simplification of the state space. This repetition would start by the identification of higher level intermediate goals. Further steps will be identical to the process described above. The repetition of the method results in multiple representation of an action space with different levels of abstractions. In addition the arcs of the more abstract representations are described in more detail in the other state spaces. It is thus possible to describe an action space with different levels of detail at the same time without losing information. For example, the next steps necessary in a given situation may be described in great detail, while only the goals are indicated for later actions. 3. REALIZATION In this section the application of the presented method to an example task is presented. First the example task and the corresponding Petri Net Model will be described. Then the conditions used to identify intermediate goals are specified. In the next subsection the algorithm to calculation a more abstract state space is introduced. After that, the repetition of this method to generate a more abstract state space is illustrated. 3.1 Example Task To demonstrate the method, a simple example task was designed, which complexity is sufficient to explain the method. In this example task some containers have to be loaded by a crane. The task environment is shown in figure 2. The containers (C1 - C8) in the storage areas on the Fig. 2. The example task environment left-hand side have to be moved to the position P1 on the right-hand side. The containers have to be placed on that position in a specific demand-sequence. If a demanded container is placed on that position, it is removed from the task. To move the containers, the crane can move in four directions (up, down, right, left), grasp, and release one container. 3.2 Modeling of the Task with Coloured Petri Nets The task was modeled as Coloured Petri Net with the software CPN Tools, which is discribed in Jensen and Kristensen (2009). CPN Tools was used, because one of its advantages is the built-in functionality necessary to compute state spaces. In the Coloured Petri Net model one place is used to model the storage area. Each mark on that place represents one container, which is described by its coordinates and id (C1 to C8). On another place, one mark represents the crane, again with its coordinate and the id of the held container. On the third place, the demand-sequence is defined. Four transitions are used to model the possible movements of the crane, whereby each transition models the movement in a different direction. Further two transitions are used to model the grasping and releasing of containers. Another transition removes marks representing containers, if they were at the top of the demand sequence and placed on the position P1. In sum a state of the model describes the positions of the containers in the storage area, the position of the crane, and the demand-sequence. As the movements of the crane where implemented as transitions, each movement generates a new state. Thus the arcs between the states represent the individual movements and the state space contains all movements in detail. The complete state space of the Coloured Petri Net is generated and exported into a file using the software Access/CPN, for an introduction see Westergaard and Kristensen (2009). This file is then loaded into the software which was built to calculate representation of the action space in different levels of detail. 3.3 Identification of Intermediate Goals In the next step, possible intimidate goals have to be identified. As stated above, intermediate goal have to meet certain conditions which can additionally include characteristics of the successor states. Further conditions

4 can be defined for the states in the path on which the goal is reached, if necessary. In the task example an intermediate goal is to grasp a container. This goal is defined by a state, in which first the crane holds a container. Additionally, in the direct predecessor state the crane must not hold a container. This is modeled by the tow conditions C grasp0 = C id 0 (holding a container) and C grasp1 = C id=0 (not holding a container). Another intermediate goal may be to release a container. Again this goal is defined by the characteristics of a sequence of two states. In this case, the characteristics have to be opposite as for the goals of grasping. Here this is modeled by the two conditions C release0 = C id=0 and = C id 0 (m). C release1 Further the actual state defines which of the two intermediate goals may be the next goal. An activation function a(m) is defined, that determines releasing a container as the next goal, if the crane is holding a container. Grasping a container will be defined as the next intermediate goal, if the crane is not holding a container. Thus these two goals are alternating. As in the initial situation the crane is not holding a container the first intermediate goal will be to grasp a container. If this goal is reached the type of goal is changing and the next goal is to release that container. 3.4 Calculation of a Abstract State Space To generate a more abstract state space the following steps were implemented. At first the initial state m 0 of the detailed state space is defined as the initial state of the more abstract state space. Then the characteristics of this initial state are evaluated to determine the active goals. As in this state the crane is not holding a container, the next possible goals will be to grasps a container. A depth-first-search is performed to search in the detailed state space for possible action sequences to reach this intermediate goal. The search checks every state and its predecessors if they fulfill the above defined conditions C grasps of the goal. If such a state is found in a path, the search algorithm aborts the search in this path and returns to the last junction. In this way it finds all possible intermediate goals of grasping one container, which are reachable from the initial state. It also ensures that no other possible intermediate goal lies between the initial node and the goals found. Every state that is identified as intermediate goal is added to the abstract state space. Additionally a connection from the initial state to every state found is added to the state space. This procedure is repeated for every intermediate goals found in the first search. The characteristics of this state are first evaluated to determine the active goals, which is now to release the container. The search algorithm starts then at this intermediate goal and searches for the goal of releasing this container. The goal states found in this second run are also added to the state space together with the arcs, connecting them with the state the second search started from. The search algorithm then start from the intermediate goals found in the second step, now searching again for the state sequences meeting the condition of the first goal of grasping a container. This is repeated until all intermediate goals are found. The whole search is implemented as a depth-first search and allows defining a search depth. This procedure results in a state space which only contains the states meeting the defined goal definitions and thus represents the action space in a more abstract way (level 2). 3.5 Repetition for More Abstract Levels To reduce the complexity further and to allow representing the action space in a third way, this method is repeated to calculate an even more abstract state space (level 3). Again the procedure starts with the definition of the conditions necessary to identify goal states. As stated above the goal states have to be defined in a more abstract way as the in first application of the method. In the example task an intermediate goal on a higher level would be to place the container that is actually demanded on the place P1. The conditions (C placeonp 1 ) of this goal would be similar to the condition of the goal to release a container (C release ) except that additionally the position of the crane has to be on position P1 while releasing the container. The repetition uses the abstract state space calculated during the first run as input (level 2). It has access to the first level only to check if the path meets the conditions for the active intermediate goal. The search algorithm starts again with the initial node m 0 searching for sequences of states that meet the definition of the goal C placeonp 1. These nodes are added to the more abstract state space together with the arcs connecting them with the initial node. The search can be repeated until all goals are found and the level 2 state space is complete. In this second run, only one kind of goal is defined and used to generate the more abstract state space. This second execution results in a more abstract state space. The amount of repetitions possible depends on the different abstraction levels on which the goals can be defined. 4. RESULTS After the double execution of the process described above, the action space of the task is described by three states spaces with different levels of detail, whereby the amount of information is decreasing with every level. Because arcs in higher level state spaces are linked to paths in the lower level state spaces, the information is not lost but hidden. As an example figure 3 shows the first steps of the task represented in the detailed (level 1) and the abstract state space (level 2). The detailed state space describes the movements of the crane and is given in light gray. Possible sequences of movements with up to 25 movements are given. The states identified as intermediate goals and thus contained in the abstract state space as well as the arcs of the abstract state space are draw in black. Here possible sequences of the first 8 intermediate goals are given. Some states are reachable within the first ten intermediate goals but not within the first 25 movements. Thus these

5 could be used to calculate more abstract representations. The goals identified in the more abstract state space could be assessed and some of them selected so that in the next step more parts of the detailed state space could be calculated stating with selected goals. Thus the identified goals on the more abstract level could be used to limit the detailed state space to the relevant parts. Another possibility to improve the presented method would be to apply one of the variety of different existing search algorithms (see for example Russell and Norvig (2010)) to speed up the calculation. Further Monte-Carlo methods could be used to calculate parts of the state space. In multiple simulations starting with the initial state the actions would be chosen by random. The information from different runs could be combined. This would result in partial state spaces which could be used as described above. 6. CONCLUSION Fig. 3. A part of the state space of the example task in two levels of detail states are only present in the part of the given abstract state space and thus are only connected with black arcs. By calculating multiple representations of state spaces the complexity could be reduced. It can be seen in the example part of the state space given in figure 3 that the amount of nodes and arcs is decreasing from the detailed to the abstract state space. Additionally it is possible to combine the representations to describe the next steps in greater detail and to give only the goals for the next steps. 5. ADVANCED METHODS The method described above requires the complete calculation of the state space. This is not possible in many cases, because the state space of the models might be too large to be calculated. But different levels of abstraction could also be generated, if the underlying state space is not complete. A possible solution would be to calculate only the state space partially, a method which was for example applied in Gamrad et al. (2009) to detect human errors automatically and in Oberheid et al. (2011) to calculate situation specific action spaces. A partial state space could be used in two ways. On the one hand, the method could be used as presented but the detail state space is just partial. As a consequence the information would be missing in all different abstraction levels. On the other hand a small part of the state space The proposed method can be applied to represent an action space with multiple levels of detail. The amount of Information is reduced in the more abstract representations. The different abstraction levels can be combined, similar to the metal model of the human operator. This method could be used to develop assistant systems which propose a plan with multiple levels of detail and thus would have change the proposed plan less often. Due to the concentration on the important information and the reduced amount of changes of the plan, this representation would be better suited to support the operator. As the amount of intermediate states needed to execute a plan depends on the experience of the operator, a system using this method could be adapted to the operator s experience. A drawback of this method is that it requires a complete state space, which cannot be derived in many cases, especially if random effects like disturbances can occur in the task environment. Thus in the next step the method could be modified to be based on partial state spaces to allow for the use in environments with disturbances. REFERENCES Amalberti, R.D.F. (1992). Cognitive modelling of fighter aircraft process control: a step towards an intelligent on-board assistance system. International Journal of Man-Machine Studies, 36(5), Dörner, D. (1989). Die Logik des Mißlingens: Strategisches Denken in komplexen Situationen. Rowohlt, Reinbek bei Hamburg. Gamrad, D., Oberheid, H., and Söffker, D. (2009). Automated detection of human errors based on multiple partial state spaces. In Proc. 6th Vienna Conference on Mathematical Modeling on Dynamical Systems MATH- MOD, Hasselberg, A., Oberheid, H., and Söffker, D. (2009). State-space-based analysis of human decision making in air traffic control. In 7th Workshop on Advanced Control and Diagnosis Hayes-Roth, B. and Hayes-Roth, F. (1979). A cognitive model of planning. Cognitive Science, 3(4),

6 Jensen, K. and Kristensen, L.M. (2009). Coloured Petri Nets: Modelling and Validation of Concurrent Systems. Springer, Berlin, Heidelberg. Möhlenbrink, C., Oberheid, H., and Werther, B. (2008). A model based approach to cognitive work analysis and work process design in air traffic control. In D. de Waard, F. Flemisch, B. Lorenz, H. Oberheid, and K. Brookhuis (eds.), Human factors for assistance and automation. Shaker Publishing, Maastricht, the Netherlands. Mumford, M.D., Schultz, R.A., and van Doorn, J.R. (2001). Performance in planning: Processes, requirements, and errors. Review of General Psychology, 5(3), Oberheid, H., Hasselberg, A., and Söffker, D. (2011). Know your options - analysing human decision making in dynamic task environments with state space methods. In D. de Waard, N. Gérard, L. Onnasch, R. Wiczorek, and D. Manzey (eds.), Human Centred Automation, Shaker Publishing, Maastricht, the Netherlands. Russell, S.J. and Norvig, P. (2010). Artificial intelligence: A modern approach. Prentice-Hall series in artificial intelligence. Pearson, Boston, 3. ed., international edition. Söffker, D. (2003). Systemtheoretische Modellbildung der wissengeleiteten Mensch- Maschine-Interaktion. Logos Wissenschaftsverlag, Berlin. Westergaard, M. and Kristensen, L.M. (2009). The access/cpn framework: A tool for interacting with the cpn tools simulator. In G. Franceschinis and K. Wolf (eds.), Lecture Notes in Computer Science, volume 5606, Springer, Berlin, Heidelberg.