Summary A follow-the-leader robot system s mplemented usng Dscrete-Event Supervsory Control methods. The system conssts of three robots, a leader and two followers. The dea s to get the two followers to poston themselves so that they are followng the leader n sngle fle. If robot1 s followng the leader then robot2 wll follow robot1, and vce-versa. A Supervsory Controller s used to ensure the system behaves as desred and to prevent the system from gettng nto a bad state.
Table of Contents SUMMARY... I TABLE OF CONTENTS...II INTRODUCTION... 1 PROJECT DESCRIPTION... 1 BACKGROUND INFORMATION... 2 AGENT CONTROL...2 AUTOMATIC TARGET RECOGNITION... 3 Image-Based versus Feature-Based Target Recognton...4 Image Sensng usng a Webcam... 4 Object Segmentaton usng a Lnear Vector Quantzaton Neural Network...4 Object Classfcaton usng Support Vector Machnes... 7 Optmal Separatng Hyperplane for Lnearly Separable Problems... 7 Extendng the Optmal Separatng Hyperplane to Nonlnear usng Kernels... 10 MOTION CONTROL... 11 AGENT MODELING AS FINITE STATE MACHINES... 11 SUPERVISORY CONTROL... 12 USING THE FINITE STATE MACHINES MODEL TO GENERATE THE PLANT...12 CREATING THE SPEC BASED ON UNDESIRED PLANT STATES... 13 GENERATING THE SUPERVISORY CONTROLLER FROM THE PLANT AND SPEC... 14 RESULTS...14 CONCLUSIONS... 15
Introducton The goal of ths paper s to explan the development of a follow-the-leader robot scenaro. What ths means, s that gven 3 robots, a leader and two followers, the two followers wll follow the leader n sngle fle. That s, robot 1 wll follow robot 2 (or vce-versa) whch wll follow the leader robot. Project Descrpton Automatc target recognton (ATR), artfcal ntellgence (AI), dscrete-event control and supervsory control technques are used n ths mplementaton. The archtecture s a three-ter herarchy, wth ATR and moton control at the lowest layer, ndvdual robot fnte state machnes n the mddle, then supervsory controller at the top. Note that the robot FSM, ATR and moton control are part of the ndvdual robots, whle the supervsory controller s separate and above both. See Fgure 1. Fgure 1. System herarchy. At the lowest layer, ATR and moton control, a webcam s used as the only sensor. It acts as the eyes of the robots. The webcam grabs an mage of ts envronment. A Lnear Vector Quantzaton Neural Network (LVQ Net) s used to segment the targets away from background n an mage. A mnmum boundng box s used to scale the targets to classfer nput sze. A relatvely new technque, Support Vector Machnes (SVMs) are used for object classfcaton. The results of ths dentfy what the robot can see. Once a decson s made as where to move, the moton control executes t. The next layer, the robot fnte state machnes, determnes, based on nformaton from the lower layer what the robot should do. Once each robot can dentfy each other and the leader, a supervsory controller s syntheszed to ensure that the follow-the-leader gudelnes are followed. The supervsory 1
controller nhbts sgnals on each of the robots to ensure the correct operaton of the system. Background Informaton AI and Robotcs are two research felds that are becomng more and more manstream. Computers are becomng faster and cheaper, so t s possble to run the CPU ntensve AI algorthms on them quck enough to make thngs run real-tme, and t s affordable to do so. AI algorthms are able to solve problems that tradtonal approaches are not. Intutvely, as robots become cheaper and better they get used more often. A follow-the-leader scenaro has many applcatons. The mnng ndustry s always lookng for cheaper and more effcent methods of producton. As part of ths ntatve, mne automaton s central. Remotely operated vehcles not only elmnate the need for the drver to get to the vehcle (whch may take up to an hour), but also makes the drver s job safer, snce he can do t from an offce char. To further ths ntatve, less people drvng vehcles means less money the company has to pay out, thus a hgher proft. Often the mnng vehcles are moved n convoys from one locaton to another, each havng a dfferent drver. Usng the follow-the-leader mplementaton there s concevably only one drver needed, the drver of the lead vehcle, and the other vehcles could follow that one n sngle fle. The same dea can be used on Automated Hghways. A group of lorres that are movng cargo from one locaton to another can be controlled by one drver. The drver would drve the lead vehcle and the other vehcles would follow behnd t n sngle fle. Ths would not only reduce the costs of drvers, but also make the hghways safer and less congested. The supervsory controller knows what each vehcle s dong, so the vehcles can be packed closer together on the hghway. Agent Control Each robot, or agent, has ts own ntellgence system. Ths ncludes an Automatc Target Recognton system, a moton control system and a robot Fnte State Machne model of ts lfecycle. The robot FSM layer gets nput from the ATR layer and outputs to the moton control layer. Fgure 2 shows the agent herarchy for robot1. Robot2 s smlar. 2
Fgure 2. Agent Herarchy. Automatc Target Recognton In general, an ATR system deals wth algorthms that can take, as nput, a dgtal mage, and then classfy t accordng to come crteron. Ths s often a three-step process, consstng of pre-processng, feature extracton, and classfcaton. The pre-processng s used to remove redundant nformaton. The feature extracton step s employed to reduce the dmensonalty of the problem. Examples of features nclude peaks n the fourer spectrum, statstcal measures of edge denstes, and gabor flterng. The fnal step s the classfcaton step, whch takes as nput the lower dmenson feature vector and outputs the classfcaton. Often the classfcaton algorthm s also a learnng algorthm. Desgnng an Automatc Target Recognton (ATR) algorthm wth hgh detecton rate and real tme operaton s easy: smply assgn the label target to everythng. Desgnng an ATR algorthm wth hgh detecton rate and low false alarm rate s hard. Such an algorthm must segment potental target objects from all the observed data representng a scene, then each object must be transformed nto a representaton approprate for classfcaton, where t s compared aganst stored representatons of known targets and a classfcaton decson s made f a robust match s found. Each of the three processes segmentaton (extract part of the mage that we are nterested n), transform to classfcaton space (feature extracton), and classfcaton contrbute to false alarms. Segmentaton s always based towards hgh detecton rates because a target mssed n segmentaton wll not get a second chance n later processng. Consequently, a typcal mage generates many more non-target objects than target objects. Classfcaton occurs n a multdmensonal feature space where the true partton between target and non-target regons s very complex and generally not well understood. Constructng a decson surface s therefore error-prone, leadng to msclassfcaton of objects near the true parttons. Wth many more non-target objects than targets n typcal 3
magery, random errors n decson surface layout tend to ncrease false alarms more than mssed detectons. The typcal learnng algorthm s a neural network. Feature vector selecton s very mportant for neural networks snce the complexty of the network scales wth the sze of feature vectors. In fact, the number of free parameters that must be determned s proportonal to the sze of the feature vector and s often many tmes larger. Ths generally necesstates a large number of tranng samples n order to constran the number of errors suffcently. The Support Vector machne (SVM) algorthm avods many of these problems. Here, only the dot product between feature vectors enters the problem. Therefore, the length of the feature vector has lttle effect on the computatonal complexty of the algorthm. By desgn, SVM s a large margn classfer, and can gve reasonable results even for sparse tranng sets, where the number of samples may be less than the sze of the feature vector. SVM can also be made resstant to outlers of a gven sze, by adjustng a parameter. Image-Based versus Feature-Based Target Recognton Many ATRs take dfferent features of a suspected target n an mage to use when classfyng. Some examples of features nclude: sze, shape, and length. Ths usually transforms the suspected target nto a form that can be more easly separated by a classfer. The downsde to usng ths feature-based technque s that t takes tme to compute each feature, so f there are many features the process can slow thngs down too much. Image-based target recognton nvolves usng the mage drectly as nput to the classfer. Ths s much faster snce features aren t computed, but the downsde s that the classfer may have to be more complex n order to separate the targets from the nontargets. Image Sensng usng a Webcam There are many dfferent types of mage sensors such as CCD TV, low-lght level TV (LLTV), forward lookng nfrared (FLIR), synthetc aperture radar (SAR), movng target ndcator radar (MTI Radar) and laser radar (LADAR). The mage sensor used here s a smple webcam. Ths doesn t gve the performance, mage qualty or resoluton of an expensve camera-framegrabber system, but t s very nexpensve and easy to use. Object Segmentaton usng a Lnear Vector Quantzaton Neural Network Object segmentaton attempts to remove target objects from the mage usng varous technques such as foreground-background separaton and slhouettng. Here an LVQ Net s used. 4
In a color mage, there exst three values for each pxel. These correspond to the Red, Green, and Blue color channels (RGB). Each channel can have values from 0 to 255. From these three color channels, other colors can be syntheszed. These three values serve as nput to the neural network. The dea behnd the neural network s that each 3-channel pxel from an mage s fed nto the network and s labeled as belongng to a group. For example, all the red pxels belong to the red group, and all the blue pxels belong to the blue group. The LVQ neural network structure s shown n Fgure 3. Fgure 3. LVQ Net structure. LVQ s a supervsed-learnng neural network. By ths, I mean that for each nput passed to the network, an dentfer was also passed tellng the network what ts class should be. The combnaton of these were used to tran the network. Tranng was completed offlne. Ths means that the network had to be traned wth data for whch the result was known, and then tranng ended (e the weghts for the network were fxed) and use of the network n dentfyng objects began. To gather tranng data, I placed the robot n several locatons along the possble routes that t would follow and took some pctures from ts attached vdeo camera. These saved btmap fles were then brought up n Mcrosoft Pant. Mcrosoft Pant has a faclty where one can clck on a pxel and get ts RGB nformaton. See Fgure 4(b). The RGB nformaton was then coped to a text fle for use n tranng the LVQ network. 5
(a) (b) Fgure 4. (a) Sample mage of robot envronment. (b) RGB nformaton extracton from an Image n Mcrosoft Pant. Snce the number of nputs to the LVQ neural network s three, the weghts can be observed n a 3D envronment. MATLAB s used to vew the convergence of the weghts onto the desred cluster of ponts for each object. See Fgure 5. Fgure 5. LVQ Weghts ( ) after tranng. Stars (*) denote tranng nputs. The dfferent colors denote dfferent objects. A classc example of the results s shown n Fgure 6. The LVQ neural network correctly labels the targets, but also ncorrectly labels other pxels as targets. Ths s due to the fact that some pxels do not fall nto any of the set groups. These pxels are then labeled as the closest match. 6
(a) (b) Fgure 6. (a) Sample mage of robot envronment. (b) Sample mage after segmentaton va LVQ Net Object Classfcaton usng Support Vector Machnes Object Classfcaton nvolves takng, as nput, a dgtal mage, and classfy t accordng to some crteron. There are many dfferent approaches to object classfcaton, ncludng mage analyss, pattern recognton, model-based vson, artfcal neural net classfcaton, and knowledge-based reasonng. Here a new technque, SVMs, s used. SVMs s a learnng technque that can be vewed as a new method for tranng polynomal, neural network, or Radal Bass Functons Classfers. The decson surfaces are found by solvng a lnearly constraned quadratc programmng problem. The two man parts of SVMs nclude usng an Optmal Separatng Hyperplane for lnearly separable problems, and then extend ths concept to non-lnear problems usng kernels. Optmal Separatng Hyperplane for Lnearly Separable Problems To llustrate the concept of the Optmal Separatng Hyperplane (OSH), we wll use an example. Gven examples wth weght, heght and sex of a person, we develop a hypothess whch enables us to determne a persons sex from ther weght and heght. Ths can be done by plottng the weghts and heghts n a 2-D coordnate system and drawng a dvdng lne or separatng hyperplane to separate the weght/heght ponts nto male and female regons. A typcal plot s shown n Fgure 7. 7
Fgure 7. (a) Weght vs Heght of People. The sold dots are female, crcles male. (b) OSH separatng males and females. There are a number of possble lnes that can separate the data, but there s only one that maxmzes the margn (maxmzes the dstance between t and the nearest data pont of each class). Ths lne gves the best results, where best means that t gves the hghest classfcaton rate when new data s used. Intutvely, we would expect ths lne to generalze well as opposed to other possble ones. Ths lne s called the Optmal Separatng Hyperplane. Fgure 7 (b) shows the data redrawn wth the OSH ncluded. The OSH algorthm s based on fndng a par of parallel hyperplanes, whch separate the data and whch has the largest perpendcular dstance between them. It s conjectured that ths provdes a good approxmaton to the best separatng hyperplane. Thus f we can fnd an OSH from the tranng data, as a mathematcal functon, then only basc geometry s used to determne whch sde of the lne any gven pont les and make a classfcaton of an unseen pont. SVMs use geometrc propertes to exactly calculate the OSH drectly from the tranng data. Gven the followng tranng data: (x 1,y 1 ),.,(x m,y m ), x = real, y = {+1,-1} (1) where each data pont s descrbed by a feature vector x and a truth value y, the latter of whch can take the values of +1 and 1, dependng on the class. The two hyperplanes are requred to pass through at least one pont of each class and there can be no ponts between them. The boundary between the classes s then defned to be a thrd parallel hyperplane that s halfway between the other two. The data ponts that the outer hyperplanes pass through, are called Support Vectors. The two outer hyperplanes are descrbed by the followng expressons, (w x) + b = +1, (2) (w x) + b = -1,. wth the frst gong through a pont of class y = +1 and the second gong through a pont of class y = -1. The constants w and b defne the hyperplanes, wth w beng normal to the hyperplanes and b/ w beng the perpendcular dstance from the orgn to the mddle hyperplane. The RHS of Equaton (2) wll be greater than or equal to +1 for all ponts of 8
class y = +1 and wll be less than or equal to 1 for all ponts of class y = -1. These can be combned nto the followng constrant on all the data ponts, y [(w x ) + b] - 1 > 0, = 1, m (3) The perpendcular dstance between the two outer hyperplanes (margn) s equal to 2/ w. Therefore, fndng the hyperplanes wth the largest margn reduces to fndng values for w and b that maxmze 2/ w or equvalently mnmze ½ w 2 = ½ (w w), subject to the constrant n Equaton (3). In other words, Mnmze: f(w) = ½ w 2 Subject to: y [(w x ) + b] > 1, = 1, m A standard method for handlng optmzaton problems wth constrants s through the mnmzaton of the Lagrangan. The constrants are taken nto account by addng terms nvolvng Lagrange multplers to the objectve functon. In ths case, ths results n the followng prmal Lagrangan, m m 1 2 LP = w α y ( w x + b) + α (4) 2 = 1 = 1 where α are the Lagrange multplers assocated wth each of the constrants n Equaton (3). The Lagrangan has to be mnmzed wth respect to the prmal varables w and b, and maxmzed wth respect to the dual varables α (e a saddle pont exsts). At the saddle pont, the dervatves of L p wth respect to the prmal varables must be equal to zero. Dong ths results n the followng expressons, m w = α y x (5) = 1 m α y = 0 (6) = 1 whle from the defnton of the Lagrange multplers, we get, α ( y ( w x + b) 1 ) = 0 (7) Insertng Equatons (5) and (6) nto (4), removes the prmal varables and results n the Wolfe dual Lagrangan where we just have to fnd the α whch maxmze: subject to m 1 LD = α α α j y y j ( x x j ) (8) = 1 2 α > 0, = 1 m, and Equaton (6) Ths s attractve because the problem s reduced to fndng the Lagrange multplers (the dual varables) that maxmze Equaton (8) and satsfy both the non-negatve constrants and the constrants of Equaton (6). Equaton (7) means that only those data ponts whch le on the outer hyperplanes (and hence are actve constrants) wll have non-zero Lagrange multplers. These data ponts are called the support vectors and they are the ponts that determne the poston of the hyperplanes. One can move the other ponts around the feature space or remove them entrely and the soluton wll not change, provded one does not move a pont across one of the outer hyperplanes. 9
Equaton (8) can be solved usng any quadratc programmng solver. Once the Lagrange multplers are known, the soluton for w s gven by Equaton (5), where the sum s over the support vectors, snce they are the only ones wth non-zero α. One can fnd b from Equaton (7), usng any of the support vectors, although one generally averages over all the support vectors for better accuracy. Once these constants are known, the classfcaton of an unknown vector, v, s gven by the sgn of, b + α y x v (9) where the sum s over the support vectors. Ths determnes on whch sde of the boundary (or mddle) hyperplane that the data pont falls. Extendng the Optmal Separatng Hyperplane to Nonlnear usng Kernels SVMs form an extenson to the OSH method. They map the nput space (e the 32x32 pxel mage) nto a hgh-dmensonal feature space through some non-lnear mappng functon and then construct the OSH n the feature space. Ths makes t possble to construct lnear decson surfaces n feature space whch correspond to non-lnear surfaces n nput space. Ths s for the case where a lnear boundary s unable to separate the data n nput space. For ths case, the SVM can map the nput vector, x, nto a hgh dmensonal feature space, z, through a nonlnear transformaton Φ. By choosng the non-lnear mappng before tranng, the SVM constructs an OSH n ths hgher dmensonal space. The most common mappngs are polynomals, radal bass functons and varous sgmod functons. Addng another pont to the set of ponts n the prevous example, we see that an OSH can t be used to separate the data (Fgure 2 (a)), but f we use a non-lnear separatng hyperplane, whch s equvalent to mappng nto a hgh dmensonal space, a separaton s possble. See Fgure 8 (b). Fgure 8. (a) A lnear separatng hyperplane can t be used to separate the data. (b) A non-lnear separatng hyperplane separates the data. Ths results n the Lagrangan n Equaton (8) beng transformed to, L α 1 α α y y Φ( x ) ( x ) (10) j j Φ D = j and the classfcaton relaton n Equaton (9) becomes, b + α y Φ( x ) Φ( v) (11) 2 10
Snce Equatons (10) and (11) depend only on the dot product between the two transformed feature vectors, one can employ a kernel functon, K( x, y) = Φ( x) Φ(y) (12) and never need to compute the transformaton Φ explctly. Equaton (11) then becomes, b + α yk( x, v) (13) wth the test feature vector now nsde the summaton over the support vectors. In general, the mappng Φ wll be to a hgher dmensonal space. Snce one s stll solvng the lnear problem, just n a dfferent space, the computatonal overhead s essentally the same. The soluton and parameters for the hyperplane are n the hgher dmensonal space and when one transforms back to the orgnal space the boundary become nonlnear. However, n general, there s no way to analytcally nvert the solutons for w and b. Hence, one must use Equaton (13) to classfy test feature vectors. Moton Control Although not ntegrated wth the rest of the system yet, the dea behnd the moton control s that a forward, reverse, turn-left or turn-rght command followed by a value would determne the drecton and velocty of the vehcle, by settng the speed of rotaton of the wheels. Agent Modelng As Fnte State Machnes At a level of abstracton hgher than the ATR and moton control exsts a fnte state machne (FSM) model of each ndvdual robot or agent. For smplcty and llustratve purposes the model has three states: dle (dle_r1), follow-the-leader (fol_leader1) and follow-the-other-robot (fol_r2). The robot starts n the dle state where t doesn t move. To get from dle to follow-the-leader, the robot has to see-the-leader (sl1 event). Whle n the follow-the-leader state the robot follows the leader robot. If at any tme whle n state follow-the-leader, the robot doesn t-see-the-leader (dsl1 event), then t goes back nto the dle state. But, f at any tme whle n state follow-the-leader, the robot sees-theother-robot (sr2 event), then t goes nto follow-the-other-robot state. In ths state, the robot stops followng the leader robot and follows the other robot. Fnally, whle n follow-the-other-robot state, f the robot doesn t-see-the-other-robot (dsr2 event), then t goes back nto the dle state. Robot2 s desgned smlarly. See Fgure 9. 11
(a) (b) Fgure 9. The FSM model of (a) robot1 and (b) robot2. Supervsory Control Supervsory control s used to coordnate the FSMs of the 2 robots to ensure the whole system acts as t should. It s generated from the FSMs of the 2 robots and a desgn spec. Usng the Fnte State Machnes Model to Generate the Plant The plant of a system s just the synchronous product of each of the FSMs. In ths case, t s the synchronous product of the FSMs of both of the robots. See Fgure 10 12
Fgure 10. The plant s the synchronous product of both the robots FSMs. Dong ths synchronous product n OTCT yelds the FSM n Fgure 11. Fgure 11. The FSM of the plant. Creatng the Spec based on undesred Plant States Fgure 11 shows the possble states that the plant (e the synchronous product of robot1 and robot2) can be n. Here there are two states that are undesrable. These correspond to both of the robots followng the leader ( [fol_leader1, fol_leader2] ) at the same tme and to both of the robots tryng to follow each other ( [fol_r2, fol_r1] ). These are undesrable snce they don t follow the gudelnes of follow the leader n sngle fle and also because they could potentally cause a crash. The purpose of the spec s to prevent these undesrable states from happenng. To remove these two states restrctons are placed on ther events. Smply put, to prevent state [fol_leader1, fol_leader2] from happenng the followng restrcton s mposed: delta2 can only happen after a marker state. So, after a delta2 happens the system must pass through a marker state before delta2 s allowed to happen agan. Ths prevents both 13
robots from beng n state fol_leader at the same tme. The same s done to prevent state [fol_r2, fol_r1] from happenng. The followng restrcton s mposed: delta can only happen after a marker state. So, after a delta2 happens the system must pass through a marker state before delta2 s allowed to happen agan. Ths prevents both robots from beng n state fol_r. Fgure 12 shows ths spec. Fgure 12. The Spec. Generatng the Supervsory Controller from the Plant and Spec From the plant and spec, a supervsory controller s bult to let the two robots functon n ther respectve FSMs whle preventng them from enterng a bad state. The supervsory controller s very smlar to the plant, except the bad states are removed accordng to the spec. Fgure 13 shows the supervsory controller. Fgure 13. The Supervsory Controller. Results To demonstrate the effectveness of the system, two software packages were wrtten. The frst software package lets the system run wthout supervsory control, whereas the second software package adds supervsory control to the system. When the frst software package was run, the system got nto bad states qute often. However, when the second software package was run, the supervsory controller prevented the system from enterng any of the bad states. 14
Conclusons Supervsory control works well n ths stuaton. The FSM model of the robots was very smplstc, further research wll nclude expandng the FSM models of both robots and regeneratng the supervsory controller to see the results. Also, t would be nterestng to see the effect of addng more robots to the system. 15