Northwestern University CS 395 Behavior-Based Robotics Behavior Design

Transcription

1 Behavior design In the last chapter, we discussed the design of simple controllers - specifically regulators and tracking systems. In standard feedback control, a single control policy is used all the time. In some cases, such as gain scheduling, you might have a number of similar controllers that you switch between, but more or less you think of there as being a single control system that's trying to do the same thing all the time - follow the set-point. Now we want to think about building systems that, as it were, have a life. Instead of a having a single goal of tracking a set-point, our robots will likely have lots of different goals, some of which can be thought of as tracking a set-point, but others will be more qualitative, like "stay fed," or "avoid predators." To handle multiple tasks, we ll need multiple control policies. The primary topic of this course is how to combine multiple control policies to accomplish multiple tasks or more complicated tasks. The topic of this chapter is how to combine control policies using parallelism, that is, running different pieces of code at the same time, as in a multi-threaded system. We will be using a specific form of parallelism called guarded parallelism, where policies run in parallel but only take control of the motors when specific conditions are met. In the theory of parallel sequential programming, these conditions are called guards, hence the name guarded parallelism. In this course, we ll be calling those conditions triggers or, when we want to consider the possibility of a policy that s half on, activation-levels. We ll call the combination of a policy and an activation level a behavior. This course is on behavior-based systems. Behavior-based systems, roughly, consist of a large number of specialized controllers, each designed to contribute to a specific goal in a specific set of circumstances. In its simplest form, this leads to an architecture consisting of a group of behaviors connected in parallel to the sensors and motors: behavior behavior Sensor data drive-base behavior behavior I ll call this general design the naïve behavior-based architecture because, while it s a useful model, real behavior-based systems tend to have considerably more structure than this. In the coming chapters, we ll discuss the more complicated variants that are used in practice. Most of these will correspond to different ways of combining control policies.

2 In general, the creatures don't act on all their goals at once. It may be that no action is called for; if a creature's just fed, there's no point in feeding again immediately. Or two goals may conflict; if you're being chased by a predator, it's probably a bad time to feed, even if you're hungry. Also, a given goal may be implemented by a number of behaviors. Part of the job of behaviors is to decide when to take action and when to remain quiescent. In general, behaviors don't activate themselves unless they can do something useful. Nonetheless, several behaviors may be in conflict, so researchers have proposed a number of techniques for conflict resolution. We will discuss these in the next chapter. Common robot locomotion behaviors To give a sense of the kinds of things we mean when we talk here about robot behaviors, let s discuss some of the most common kinds of behaviors that people build. Followers There are a number of locomotion behaviors that are referred to as followers of one kind or another. The most common is the freespace follower also called a wanderer or sometimes just collision avoidance system, which steers toward open space and attempts to drive forward. The simplest class of control policies for freespace following is to choose a direction of motion or a rotational velocity that forms a compromise between the different possible directions or velocities based on the amount of free space in different directions. For example, the direction of motion might be a weighted average of the different possible directions of motion with the amount of free space in each direction used as the weights. A variant on freespace following is boundary following, in which the robot tries to trace along the edge of some object or other contour. Line followers sense the presence of lines drawn or otherwise marked on the floor and follow along them. Wall followers attempt to drive along a wall by keeping a constant distance from it and remaining parallel to it. Wall- and boundary followers are sometimes built to have a preference for following the object on one side or the other of the robot, so some robots have separate left- and rightwall followers, which are activated by different triggers. The simplest class of control policies for boundary followers is to steer using a feedback loop on the robot s distance from the boundary. If the robot is too far, it turns toward it; if it s too near, it turns away. Finally, there are object followers, which try to stay close to an object as the object moves. Usually the object is a brightly colored ball or some other kind of object that is easy to sense. Person followers are also common, but are harder to build because it s difficult to build a good person detector. Ballistic actions Most behaviors turn themselves off when their sensors detect that their triggering conditions have ended. However, some behaviors need to keep running until the are done. These kinds of behaviors are called ballistic behaviors or ballistic actions. In biology, they re called fixed action patterns. For example, a ballistic turn behavior is a behavior that always turns a specific angle. Since motors are controlled by torque or velocity and not angle, the ballistic turn must be implemented as a control loop. When activated, the behavior must compute its target orientation and remember it (e.g. with a latch in GRL) so that it can compute the feedback error in its control loop. It must then stay active until that feedback error reaches zero. Another example of a ballistic action is backing up from an obstacle after a collision is sensed. This has to be a ballistic action because collision sensors are usually contact sensors, so as soon as contact is broken, the trigger goes away. Other common locomotion behaviors Control loops often get into stuck states where they haven t accomplished their tasks, but they still generate zero motor output. An example is a freespace follower that has driven into a cul de sac. Many freespace followers will generate a zero turn signal because there is equal freespace on both sides, yet they

3 cannot move forward because there is insufficient freespace ahead. A common solution to the problem is an unwedger which will trigger in a stuck state and forcibly reorient the robot (e.g. using a ballistic turn) to move it out of the stuck state. Another common behavior is to drive to a specified set of Cartesian coordinates based on the robot s estimate of its position. Position estimates are often based on integrating the sensed velocities of each motor. In nautical navigation, this task of computing the integral of velocity to get position is called deduced reckoning, which is usually shortened to DED reckoning, or more commonly, dead reckoning. The behaviors that use this to move to a specified position are often therefore also called dead reckoning behaviors. Homing is a behavior of (somehow) returning to a specified location. Simple homing systems use dead reckoning. More complicated ones can be very complicated indeed. Docking is essentially like object following, except that the object isn t moving and the robot actually needs to touch it. Usually, the idea is to touch it softly and at a specific position and orientation. For example, a robot that recharges itself needs to be able to plug itself in somehow. This can be very hard with today s robots because while they can move forward and backward easily, they can only move side-toside by alternately rotating and translating; so docking can be like trying to parallel park a car. Designing a frog As an example of behavior-based system design, we ll sketch the design of a feeding behavior for a simulated frog. Before we can write code for the behaviors, we will need to formalize what we mean by a behavior. Although behavior means many things to many people, we will take it to mean a combination of some kind of triggering mechanism (usually based on sensory data) and some kind of control loop. We will represent the output of the triggering mechanism as an activation-level signal, which might either be a Boolean value or a real number, depending on whether we want to be able to represent varying degrees of activation. We will represent the output of the control loop with a motor-vector signal (usually a group), suitable for feeding to the low-level drivers for the motors. We can describe these as a data abstraction in GRL: (define-group-type behavior (behavior activation-level motor-vector) (activation-level activation-level) (motor-vector motor-vector)) This says that behaviors have two components, an activation-level, and a motor-vector (the output of the policy). You create behaviors with the behavior procedure, which takes the activationlevel and motor-vector as arguments, and you extract their components with the activation-level and motor-vector procedures. Let's consider a couple of simple behaviors, such as a frog might have. Let's say our robot frog can do three things: turn in place, drive forward, and eat whatever's in front it. We can represent the turning and driving with our rt-vectors, so all we need to do is to add an eat? component to them. Then our motor vectors might look like: (define-group-type frog-motor-vector (frog-motor-vector motion eat?) (motion motion-of) (eat? eat?)) Now suppose our frog has a fly-motion sensor that tells the bearing of any flies that it sees. Then we could describe the frog's feeding behavior as a conjunction of two simpler behaviors:

4 Face-prey When the frog sees a fly, it needs to turn to face it so that it can stick out its tongue and grab it (frog tongues have limited steerability). Eat When the frog sees its prey within tongue range, it shoots out its tongue to eat it We can code each of these as a behavior. The eat behavior is straightforward: (define-signal facing-prey? (< (abs prey-heading) facing-prey-tolerance)) (define-signal sit-still (rt-vector 0 0)) (define-signal eat (behavor (and see-prey? facing-prey?) (frog-motor-vector sit-still #t))) Where we assume that prey-heading is the output of the prey sensor and it tells the frog what direction the prey is in relative to its own body, and where a heading of zero means the frog is perfectly facing the fly. The face-prey behavior is a little more complicated. At first blush, we might expect to use something like: (define-signal face-prey (behavior (and see-prey? (not facing-prey?)) (frog-motor-vector (orientation-p-controller prey-heading 0 face-prey-gain) #f))) which would servo the frog's body until it faced the prey. Unfortunately, most motion sensors go blind as soon as the creature starts moving itself, so the frog has to remember where the prey is and then turn toward it using dead reckoning. We can write this as (changes shown in italics): (define-signal face-prey (behavior (and see-prey? (not facing-prey?)) (frog-motor-vector (orientation-p-controller frog-orientation prey-orientation face-prey-gain) #f))) with the ancillary definitions: (define-signal frog-orientation (modulo (integrate rotational-velocity) 360) (define-signal still? (= rotational-velocity 0)) (define-signal prey-orientation (latch (modulo (+ frog-orientation prey-heading)

5 360) still?)) Note that latch is a transducer that outputs the value of its first input, as long as its second input is true. When the second input goes false, it holds its output until the second input becomes true again. We can now build our frog controller by saying: (define-signal frog-controller (drive-frog eat face-prey)) where we assume that drive-frog is a transducer, like drive-base, that sends a motor vector to the motors, or rather the muscles. However, unlike the previous base controllers we ve used, we will assume here that it takes behaviors as arguments and uses the motor vector of whichever behavior is active. This of course brings up to issues: of how we find the motor vector of the active behavior, and what we do if there are two active behaviors, both of which we ll put off until the next chapter. For now, simply take it for granted that drive-base has a way of handling the situation. Naï ve behavior-based system design To build a naï ve behavior-based system, we need to come up with a set of behaviors that jointly perform the robot s task. In practice, roboticists almost always end up doing this iteratively because there are almost always contingencies that come up in real environments that they didn t think while writing the code. Different people design systems in different ways, but the general outline of how you build a naï ve behavior-based system is basically this: 1. Break the task into a set of motions that, if properly executed at the right times, would allow the robot to solve the task 2. For each motion: Determine the conditions under which it should be performed Design an estimator that can determine whether those conditions occur Design a controller that should implement that motion when used under those conditions Package the controller and estimator together as a behaviors 3. Try the behaviors out as a system 4. Find a failure mode If none of the motions you have implemented could have prevented the failure, implement a new behavior. If the failure was due to firing the wrong behavior, then either Make the activation conditions for the behavior that didn t fire more general Make the activation conditions for the behavior that did fire more specific 5. Repeat until the system works This is an incomplete methodology at best since it s focused only on choosing behaviors and their triggering conditions. It ignores issues like tweaking gains and thresholds, hunting down bugs in device drivers, fixing bugs in sensor code, and so on. These latter tasks often consume much more of a roboticist s time than choosing a set of behaviors and their triggering conditions. Often times, there may be subtle trade-offs between solving these sorts of issues and designing the behaviors. For example, if the orient-to-prey behavior is performing poorly because the fly-motion detector sometimes hallucinates flies, you might try to fix the situation by building a better fly detector. On the other hand, you might instead try to solve the problem by adding another behavior that advances toward an apparent fly to get a better view, then modifying the triggering conditions for the eating behavior to only try to eat flies that are near enough for the fly detector to be trusted.

6 Motor schemas and potential fields Motor schemas are a very powerful method for designing low level behaviors. The term schema is originally due to Kant. In biology it is taken to mean any kind of built-in pattern to an organism s activity. In psychology, it means any kind of structuring or framework that a person uses to understand their world. In robotics, a motor schema is a mapping from positions in the environment to velocity vectors. In other words, it s a vector field over space. Conceptually, the agent just looks up the appropriate velocity vector for its location and sends it to the motors. By arranging for all of the vectors to converge at the agent s goal, you can easily make the agent go to any desired position. In reality, motor schemas are more complicated than this because of two factors. First of all, the agent doesn t necessarily know what its coordinates are in the environment. So in practice, motor schemas are often implemented as mappings from sensor data to velocity, rather than from position to velocity. This also means that the velocity vector will be specified in body-centered or egocentric coordinates, rather than in environment-centered or global coordinates. For example, common obstacle avoidance strategy is to have the velocity vectors point away from the obstacle with magnitude inversely proportional to distance squared. For example: (define-signal (obstacle-force obstacle-vector) (- (/ obstacle-vector (cube (magnitude obstacle-vector))))) (Note that we haven t bothered to write this as a behavior in our formal sense there s no activation level; Don t worry about this for the moment). Of course it s rare that there s a specific obstacle that we know the vector of that we want to avoid. A more common case is when we have an array of depth readings from a ring of sonars pointed in different directions. Then we can generate a vector for each sonar reading and then try to avoid all of them by summing their vectors: ;;; Total number of sonars in the sonar ring (define-signal sonar-count 16) ;;; The direction (in radians) in which the specified sonar faces. (define-signal (sonar-direction sonar-number) (/ (* 2 pi sonar-number) sonar-count)) ;;; The unit vector in a given direction (expressed in radians) (define-signal (unit-vector direction) (xy-vector (cos direction) (sin direction))) ;;; A vector of unit vectors pointing in the directions of ;;; each sonar. (define-signal sonar-unit-vectors (unit-vector ;; The directions of each sonar (sonar-direction ;; This just gives us a vector with the elements ;; sonar-count-1. (index-generator sonar-count))))

7 ;;; A vector of force-vectors for the objects sensed by ;;; each sonar. We assume that the distance readings for the ;;; sonars are in the signal sonar-readings, which is a vector with ;;; one element per sonar (define-signal sonar-forces (/ sonar-unit-vectors (square sonar-readings))) ;;; The final motor vector for avoiding everything (define-signal total-force (vector-reduce + 0 sonar-forces)) This code uses a feature of GRL feature we haven t seen before: vectors. We ve seen xy-vectors and rt-vectors before, but they were implemented as groups with components named x and y, and rotation and translation, respectively. GRL also allows you to use real array-type vectors and to access them using the Scheme primitives vector-ref and vector-set!. Like, groups, primitive procedures automatically map over elements of a vector, so when you add or multiply two vectors, you get a vector back. The vector-reduce procedure is just like the normal reduce procedure it takes a procedure, an initial value, and a vector, and applies the procedure, pairwise, to all elements of the vector, starting from the initial value. In other words, (vector-reduce + 0 v) returns the sum of the elements in v. Between automatic mapping and higher-order functions like vector-reduce, it s very rare to actually have to use the low level procedures vector-ref and vector-set!. Having to compute vectors from sensor data was the first complication in implementing motor schemas. The second complication is that most robot bases can t directly execute x,y velocity vectors, since they can only go forward and backward, and turn in place. Since we re assuming the xy-vectors are in an egocentric (robot-body-centered) coordinate system, that means the robot can only directly execute the xyvector if it happens to be pointing directly forward (or backward). So we need some kind of a wrapper to adapt the xy-vector policy into an rt-vector policy. The simplest, and most common, approach for this is to rotate until the vector is pointing forward, then drive: (define-signal (follow-xy-vector xy) (let ((rotation-error (angle xy))) (rt-vector (* rotate-gain rotation-error) (if (< (abs rotation-error) rotation-tolerance) translate-speed 0)))) where the free variables rotate-gain, rotation-tolerance, and translate-speed are turned for good performance. The function angle returns the angle (in radians) in which the xyvector is pointing relative to the X-axis (which we are taking to be the direction in which the robot is pointing). It can be computed using the familiar 2-argument form of arc tangent: (define-signal (angle xy) (atan (y-of xy) (x-of xy))) Many variants on this policy are possible. For example, the translational and/or rotational velocities could be made to vary with the magnitude of xy (the policy above ignores the magnitude completely). Or the translational velocity could scale smoothly as the orientation shifts toward 0. Another policy that people have obtained good results with is to drive the rotate velocity with the sine of the angle of xy and to drive the translate velocity with its cosine. Since the sine and cosine are the components of the unit vector in xy s direction, we can simply define this as:

8 (define-signal (follow-xy-vector xy) (let ((unit (/ xy (magnitude xy)))) (rt-vector (y-of xy) (x-of xy)))) Obviously, either version of follow-xy-vector would need to be tuned with gains, saturations, dead zones, etc., to suit the needs of the particular robot base being used. Now we can put these together to produce a system that moves into the center of an open space: (define-signal freespace-taxis (base-controller (follow-xy-vector avoid-obstacles)))) or we could write it as a behavior: (define-signal freespace-taxis (behavior want-to-avoid? (follow-xy-vector avoid-obstacles)))) where want-to-avoid? is a placeholder for whatever activation condition you want to use. Potential fields Motor schemas are vector fields that map from positions (or sensor readings) to vectors that point toward better positions. An alternative is to define a field that maps from position to desirability itself. This field is called a potential field after potentials in physics. Low values of potential are typically taken to mean high desirability. The advantage of constructing a potential field is that we can then construct a schema simply by taking the gradient of the field. (Actually, the attentive reader will realize that the motor schema should be minus the gradient. Since gradient vector point toward bigger potentials). All gradient fields are usable as motor schemas (at least in principle), however, not all motor schemas are gradient fields. That is, not all motor schemas are the gradient of some potential. In fact, it is rare for mobile robots to use real potential fields, since most of the motor schemas that are easy to compute from sensor data don t happen to correspond to the gradient of a real potential. In spite of this, motor schemas (and reactive control systems in general) are often referred to informally as potential field methods because they have most of the same advantages, disadvantages, and structure. They are also occasionally referred to as gradient-descent methods because true potential field systems follow the gradient toward lower potentials. Local minima The equilibrium points of a gradient descent system (the points in space where the robot stops, i.e. where the motor vector is zero) are often called local minima because, in a true gradient field, zeros correspond either to local maxima or local minima in the potential function. One of the big advantages of motor schemas and potential fields is that you can combine them just by adding them. If you have one motor schema for approaching a goal, and another for avoiding obstacles, then on a good day, the sum of the two will approach the goal while avoiding the obstacles. Unfortunately, there tend to be unfortunate cases where the two fields cancel each other out. These lead to unwanted local minima, i.e. equilibrium points. This is enough of a problem that when people talk about local minima, they almost always mean unwanted local minima.

9 It s usually impossible to get rid of unwanted local minima entirely. However, there are a number of standard approaches that have been used to help cope with the problem. Most of these involve running another behavior in parallel with the behavior that gets caught in the local minima: Noise Some equilibrium points are stable, meaning that if the robot moves a little bit away from them, the control system moves it right back to the equilibrium point. Other minima, however, are marginally stable or metastable meaning that if the robot moves away from them a little bit, then the control system will naturally move it away from the equilibrium point. If you throw a rock into a valley, it will roll to the bottom, at stay in a stable equilibrium. If you balance it on top of a stick, then it s only in a metastable equilibrium, and probably not for very long. Metastable equilibria can be avoided by adding noise (small random numbers) to the output of the system. If the system ever gets caught in a meta-stable equilibrium, the noise will eventually push it out. Path planning Many researchers have used local minima as an argument for building control systems that compute paths for the robot rather than velocities. Computing the path is called path planning. Once the path is computed, it is executed by a control loop of some kind. Planning has many advantages, but it is expensive. It may also require more information than the robot s sensors can realistically provide. It is also more complicated in dynamic worlds where the obstacles you re trying to avoid are moving. Unwedging A common, simple technique is just to build a separate behavior that takes control whenever the robot is motionless for too long (when it is wedged ). Such an unwedger then moves the robot to a different position or orientation, in hopes that the original behavior will be able to able to achieve its goal. Avoid-recent behavior A very nice technique, due to Tucker Balch, is to keep a simple memory of the last few locations the been and to generate a repulsive field from them. When this repulsive field is added to another field, it will tend to push the robot out of local minima when it gets stuck in them. Exercises 1. Suppose your robot is equipped with a front-mounted gripper that can open and close to grasp objects that are directly in front of the robot. Propose a set of behaviors for grasping and collecting objects. 2. Modify the frog feeding behaviors to handle the case where the frog s tongue has limited range.