CONCLUSION ACKNOWLEDGMENTS REFERENCES

Transcription

1 tion method produces commands that are suggested by at least one behavior. The disadvantage of the chosen arbitration approach is that it is computationally expensive. We are currently investigating methods to implement it effectively. In the near future, we will finish the experiments of unblocking. We expect these experiments to demonstrate how the arbitration method enables a smooth transition from one behavior to another. After that we will integrate the components based on the optical flow in the Pure system. We will continue to develop the Samba architecture. More complex systems are needed to test the arbitration method. The proposed learning scheme has to be specified in more detail and implemented. Although the learning scheme contains challenging problems, we believe that it also has a great potential as an approach to building complex control systems. CONCLUSION In this paper we discussed behavior cooperation. We described how markers and ed signals can be used to coordinate cooperating behaviors. We discussed the cooperating behaviors for both single and multiple agents. We proposed a method for learning cooperative behaviors. We tested the architecture with real robots. ACKNOWLEDGMENTS The authors would like to thank colleagues at the Electrotechnical Laboratory for their assistance in this research project. The discussions with Paul Bakker, Polly Pook, and Alex Zelinsky were valuable. Special thanks to Kenji Konaka, who helped in the implementation. Further, we are grateful to Nobuyuki Kita and Sebastien Rougeaux, who contributed to the original image processing and robot control programs. REFERENCES. Brooks, R.A. (986) A robust layered control system for a mobile robot. IEEE Journal of Robotics and Automation, RA-2(): Brooks, R.A. (99) Elephants don t play chess. Robotics and Autonomous Systems 6(-2): Tsotsos, J.K. (995) Behaviorist intelligence and the scaling problem. Artificial Intelligence 75: Chapman, D. (99) Vision, instruction, and action, MIT Press. 5. Brill, F.Z., Martin, W.N. & Olson, T.J. (995) Markers elucidated and applied in local 3-space. International Symposium on Computer Vision, Coral Gables, Florida, November 2-23, 995. pages Payton, D.W., Rosenblatt, J.K. & Keirsey, D.M. (99) Plan guided reaction. IEEE Transactions on Systems, Man, and Cybernetics, 2(6): Rosenblatt, J.K. & Thorpe, C.E. (995) Combining goals in a behavior-based architecture. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 95), Pittsburgh, USA, August 995, pp Kuniyoshi, Y., Kita, N., Rougeaux, S., Sakane, S., Ishii, M. & Kakikura, M. (994) Cooperation by observation - The framework and basic task patterns. IEEE International Conference on Robotics and Automation, pp Kuniyoshi, Y., Inaba, M. & Inoue, H. (992) Seeing, understanding and doing human task. 992 IEEE International Conference on Robotics and Automation, May 992, Nice, France.. Kuniyoshi, Y., Riekki, J., Ishii, M., Rougeaux, S., Kita, N., Sakane, S. & Kakikura, M. (994) Visionbased behaviors for multi-robot cooperation. IEEE/RSJ/GI International Conference on Intelligent Robots and Systems (IROS 94), Munchen, Germany, September 994, pp Riekki, J. & Kuniyoshi, Y. (995) Architecture for vision-based purposive behaviors. IEEE/RSJ International Conf. on Intelligent Robots and Systems (IROS 95), Pittsburgh, USA, August 995, pp

2 a) The goal positions sent by Pose. b) The goal positions sent by Unblock x y - x y c) Combined goal positions x y Figure 7. An example of combining goal positions. DISCUSSION The lack of powerful behavior coordination methods prevents applying the behaviorbased architecture to complex tasks. To solve this problem, we proposed behavior coordination methods based on markers and ed signals. The s enable a continuous shift from one behavior to another. Further, s enable the anticipation of tasks, as behaviors can send commands with small s when the task conditions have not yet been fully satisfied. We improved the behavior-based architecture bottom-up, without introducing a symbolic model or reasoning based on the model. As all coordination methods are local, the system is robust and scalable. These coordination methods can also be used to cooperate behaviors of multiple agents. We proposed a method for learning a common task by imitating other agents. As there is no explicit communication between agents, the multi-agent system is robust and extensible. Further, there is no need for transformation between the representations of the agents. The arbitration approach was chosen as it is an open mechanism, that can easily be modified by other modules such as learning modules. Cooperation can be adapted based on experiments. Weighting each command enables dynamic arbitration based on the agent and environment state. The use of gains allows further tuning based on additional information. When we specified the arbiter, we considered first ed averaging. In this approach the resulting command is constrained by the commands sent by the behaviors. In many cases this is too loose a constraint. For example, when triangulation is used to calculate object positions, the cameras must always be turned towards an object. Also, a marker must be bound to one feature, not between several features. The chosen arbitra-

3 POSE FIXATED! camera angles fix egomotion planning MOVETO MARKER ROBOT GAZE PLATFORM active sensing motion control MOVETO robot command gaze command ZDF ZDFCENT MARKER PURSUIT Figure 5. Posing task time [s] Figure 6. Zdfcent. allel. Both behaviors sent commands to the Moveto marker. This is an example of cooperating behaviors of a single agent. The behaviors cooperate to unblock other agents. Pose keeps the agent near the other agent, so the area in front of the other agent can be checked periodically and Unblock can remove the obstacles. In the first version Unblock simply subsumed Pose when an obstacle was found. The behaviors did not cooperate very well. The reason was that during posing the agent tends to drive behind the other agent, which makes unblocking difficult. We are currently implementing a new version in which Pose and Unblock send ed signals to the Moveto marker. The signals are combined by an arbiter. The maximum s of the combined signal specify the goal and target positions of the Moveto marker. This approach enables Unblock to control the agent towards a good unblocking position while Pose keeps the agent near the other agent. When the area in front of the other agent is known to be free, Unblock sends small s and thus does not have big effect on the Moveto marker. But when the known free area shortens, Unblock increases the s and the goal position shifts toward the good unblocking position. Figure 7 illustrates the combining of the goal positions. In the signal sent by Pose, s are large at a constant distance from the target. Unblock specifies large s at a constant distance from the future trajectory of the target. The maximum of the combined signal specifies the goal position of the Moveto marker.

4 cutes the task described by the marker. When the agent observes another agent, it first represents the other agent s actions by commands to its own actuators. Then it replaces sequences of commands by a command to a marker, until there are no more sequences of commands in the observed set that match a set of child nodes in the action tree. The remaining set of actions is produced by the new behavior when the conditions for the task are satisfied. The learned behaviors can also be generalized. When several behaviors produce the same sequence of actions, a new behavior executing the common sequence is created. Also a new marker is created. The marker activates the new behavior. The common sequence of actions is replaced in each behavior by activation of the new marker. The complexity of the learning problem is managed by learning gradually. At each step, the agent learns a new way to combine the existing actions. After learning a new action, the action is added to the action tree. When possible, behaviors are generalized. The agent builds incrementally new layers of behaviors and markers in top of existing ones. EXPERIMENTS We have tested the Samba architecture in the application of Posing, Unblocking, and Receiving. The Pure control system controls an agent to help other agents in their transfer tasks. The agent is equipped with a stereo gaze platform. The gaze platform has 2 DOF vergence control. Zero Disparity Filter (Zdf) extracts from the image data the features that are at the center of both cameras, that is, the features that the cameras are fixated on. Refer to papers [,] for more details on the Pure system. Lately we have improved the Pure system by adding s for signals, activation levels for behaviors and arbiters for resources. So far we have tested the posing and unblocking tasks. Posing and Unblocking Separately In the posing experiment, the agent followed another agent successfully for several minutes. This experiment demonstrates the basic characteristics of our architecture. The active modules are shown in Figure 5. Pose behavior initializes the Moveto marker, which contains a point bound to an object and a goal point for the agent. The Moveto behavior moves the agent to the goal point and turns it towards the object point. The Moveto marker updates the internal representation of the points continuously based on ego-motion. Pursuit behavior controls the cameras towards the posed object based on the output of Zdf module, which is described by the Zdfcent marker. The Moveto marker updates its points based on the fixation point of the cameras. In the unblocking experiment, the agent helped another agent by pushing away an obstacle blocking that agent. This is an example of multi-agent cooperation. When there is an obstacle in the trajectory of another agent, the Unblock behavior controls the Moveto marker in such a way that the agent heads towards the obstacle at a predefined distance and pushes the obstacle away. As an example of calculation, see the of the Zdfcent marker in Figure 6. The is calculated based on the size of the Zdf region (in pixels) and the speeds of the cameras. The other agent was tracked until 245 seconds (approximately). The small valleys in the before that moment are caused by variations in the Zdf region size. When an obstacle is searched the goes to zero. When the obstacle is found, the increases quickly. At the end the varies considerably, as the obstacle fills the images. Cooperation of Posing and Unblocking We tested behavior cooperation also by executing posing and unblocking tasks in par-

5 SENSORS sensor data marker commands MARKER marker data feedback PURPOSIVE BEHAVIOR MOTOR BEHAVIOR actuator commands Arbiters An arbiter combines the commands that behaviors send to a resource. Each input command has a gain that can be modified dynamically. The arbiter multiplies the s of the commands by the gains, sums the s, and selects the command parameter values having the maximum s. Then, the arbiter sets the s of the combined command based on the maximum and its neighborhood. For an actuator arbiter we specify a second element, that sums the commands from stabilizing behaviors to the arbitrated command. The stabilizing behaviors take other actuators into account. For example, the cameras can be stabilized by subtracting the amount of agent rotation from the camera commands. COOPERATION Figure 4. Task execution. Single Agent As behaviors are capable of executing only simple tasks, many behaviors must cooperate to perform a complex task. Cooperation is implemented using markers and ed signals. The signal s specify how much each behavior affects the resulting command. The behaviors produce separate s for each marker position. The arbiter combines the positions separately. Multiple Agents Cooperation among multiple agents is based on markers initialized by visual observations. A marker triggers the behaviors performing the cooperative task. Thus, cooperating multiple agents corresponds to the process of controlling an agent s own behaviors. This cooperation by observation is discussed in more detail by Kuniyoshi et al. [8]. Cooperation among multiple agents can also be learned. An agent can learn tasks by imitating other agents. Imitation consists of seeing, understanding, and doing [9]. The agent observes another agent when it performs the task and represents the observed actions by its own actions. The agent represents also preconditions for the actions by its own sensor and marker values. After this, it can start to execute the task together with the other agents. Commands for markers and actuators form a natural representation for observed actions. Once a set of actions has been observed, the creation of a new behavior is straightforward. The behavior outputs the observed set of actions. Further, sensor and marker values are a natural choice for representing the conditions, as these values are produced also when the agent executes the task itself. The new behavior receives the sensor and marker signals that are used in the conditions. The commands for markers and actuators form the action tree of the agent. A node in the structure describes one action known by the agent, a command to either a marker or to an actuator. Nodes describing actuator commands are leafs in the tree. A node describing a marker command has child nodes for commands produced by the behavior that exe-

6 ..5 WEIGHT MIN MAX Figure 2. A function. PARAMETER Markers A marker connects a task-related environment feature to motor actions. A marker is bound to a feature either by a behavior or automatically based on sensor data. It can be interpreted as binding task parameters to environment features. Binding activates the behavior executing the task. A feature is indexed by its position in the environment. The feature position can be described either in the egocentric coordinate system (mobile space markers) or in the image coordinate system (image space markers). The feature position is updated automatically based on observations on the feature. Mobile space markers are updated also based on ego-motion. Markers are also used to describe goals for the agent as positions related to the feature position. Further, feedback data describes the state of the task. Feedback is sent by the behavior executing the task. A marker specifies s for the different possible feature positions. The s are initialized when the marker is bound. After that, the marker updates the s based on the observations on the feature. The s decay over time. The maximum is the activation level of the marker. When it decreases below a threshold, the marker deactivates itself. A marker specifies s also for the goal positions. Behaviors A behavior transforms input signals received from sensors and markers into commands to actuators and markers (that is, to resources). A behavior calculates first its activation level based on the input s and the previous activation level. If the activation level exceeds a threshold, the behavior transforms input signals into output signals. The activation level describes the importance of the behavior. The s of the output reflect the activation level. A behavior also reports the progress of the task by sending a state signal to the corresponding marker. The system contains two types of behaviors: motor behaviors and purposive behaviors. Motor behaviors control actuators based on signals received from sensors and markers. Purposive behaviors execute tasks by controlling markers. When task conditions are fulfilled, a purposive behavior binds the markers needed in task execution with environment features. It also fills in task-specific data related to the features. For each marker there is either a motor behavior that controls an actuator based on the marker data or a lower-level purposive behavior that decomposes the task described by the marker further and binds the corresponding markers. A purposive behavior monitors the execution of the task by analyzing sensor data and the marker feedback data. Figure 4 illustrates task execution. - x Figure 3. A surface. y

7 The arbitration method was inspired by the distributed command arbitration method reported by Payton et al. [6]. We have generalized the method for more complex commands, for arbitrating markers, and for the image space. In recent work also Rosenblatt & Thorpe [7] extend the distributed command arbitration method for more complex commands. In the following chapters we describe our architecture and discuss cooperation. We also describe the experiments done so far. THE SAMBA ARCHITECTURE The Samba architecture contains Sensor, Actuator, Marker, Behavior, and Arbiter modules. The architecture is presented in Figure. The control system is connected to the external world through sensor and actuator modules. Markers connect task-related environment features to behaviors. Actuators and markers are the resources of the control system. Behaviors execute tasks by sending commands to the markers and actuators. Arbiters solve the conflicts among the behaviors commanding the same resource. SENSORS MARKERS PURPOSIVE BEHAVIORS MOTOR BEHAVIORS A R B IT E R S A R B IT ACTUATORS E R S Figure. The Samba architecture. Markers coordinate behaviors at several stages of task execution. First, markers activate the behaviors needed to perform the task. Secondly, markers share data among the behaviors and focus their attention on the important environment features. Thirdly, markers arbitrate behaviors. Each marker has an arbiter, that combines the commands sent by the behaviors. Actuators have similar arbiters. Finally, markers control cooperating behaviors. When several tasks can be executed in parallel, the commands sent to a marker are combined. Signals Modules communicate by sending signals to each other. A signal specifies s for the different possible values of some data fields. The number and type of the data fields is not restricted. A is a real number in a range [-.,.]. The data fields can be grouped and s can be specified for these groups separately. In the simplest case, a signal contains one set of values. The interpretation is that these values have the maximum and the rest of the possible values have zero s. Or, the one set of values can have a in a range [.,.]. This can be given by a filter producing the values, or as a function of a parameter that has some relation to the values. Such a function is illustrated in Figure 2. The increases linearly from. to. when the parameter value increases from the minimum value to the maximum value. The parameter can be, for example, camera speed for a signal describing an image processing result. In the general case, the s are specified for all possible sets of data field values. For a position in two-dimensional space, the s form a three-dimensional surface. Figure 3 shows a simple surface.

8 BEHAVIOR COOPERATION BASED ON MARKERS AND WEIGHTED SIGNALS JUKKA RIEKKI * University of Oulu 957 Oulu Finland jpr@ee.oulu.fi YASUO KUNIYOSHI Autonomous Systems Section Electrotechnical Laboratory Tsukuba, Japan kuniyosh@etl.go.jp ABSTRACT In this paper we propose markers for coordinating behavior cooperation. Markers ground task-related data on sensor data flow. Behaviors command markers by specifying s for the different possible command parameter values. Cooperation is achieved by combining the commands sent to the same marker. We discuss also multi-agent cooperation and propose a scheme for learning cooperative behaviors. Markers have an important role in multi-agent cooperation and learning. We present experiments on a control system that enables a real robot to help other robots in transferring objects. KEYWORDS: behavior-based, markers, behavior coordination, multi-agent cooperation INTRODUCTION The behavior-based architecture has been applied successfully to mobile robot control [,2]. However, the tasks performed by these robots have been rather simple. The behavior-based architecture does not scale well to complex tasks because they require more powerful behavior coordination methods than the behavior-based architecture offers. The scaling problem has been discussed in more detail by Tsotsos [3]. We propose in this paper Samba, a behavior-based architecture with powerful coordination methods. The key idea is to use markers, which describe tasks and ground taskrelated data on sensor data flow. Behaviors execute tasks by commanding markers. Commands contain s for the different possible values of command parameters. These s are used in arbitrating behaviors. Markers are applied in coordinating cooperating behaviors - also in multi-agent cooperation - and in learning cooperative behaviors. The architecture was inspired by the work of Brooks [] and Chapman [4]. We have extended Chapman s work by attentive behaviors for the 3D domain and by integrating image space data with mobile space data. Also Brill et al. [5] have reported a markerbased architecture. The major difference between these architectures and ours is that in our system, markers have arbitration functionality. * This research was done at the Electrotechnical Laboratory. The research was supported by the Science and Technology Agency program of the Japanese government.