DogBOT An Interactive Robot Dog for Entertainment

Transcription

1 DogBOT An Interctive Robot Dog for Entertinment Kun-Ting Yu, Ping-Che Hsio, Wei-Ho Mou, Yi-Shu Li Deprtment of Computer Science & Informtion Engineering, Ntionl Tiwn University Abstrct This pper describes robot dog. First, by using the Kinect sensor the user cn use more nturl 3D dynmic hnd gestures to interct. Unlike previous work, either the user must wer some device to sense the humn motion or the gestures re limited to 2D or sttic poses. Second, the DogBOT cn follow the user with the lser rnge finder by utilizing n existing humn trcking lgorithm. Third, with color er on, the DogBOT cn chse bll smoothly by pplying hue histogrm bck projection. The bll detection system cn tolerte moderte illumintion chnge. Lstly, we hve designed the fcil expression to let the DogBOT rect in more nturl wy. W I. INTRODUCTION ith the rpid popultion growth in modern cities, people re becoming physiclly busy but psychologiclly lonely. A loyl compnion, such s pet, cn be good plymte for ll ges. However, rel pet need extr cre to feed it, to wlk it, or to cope with its excrement, which re problems tht people doesn t think of t the first moment. In contrst, robot dog only needs chrging, nd its behviors re controlled by engineers design (i.e. won t spontneously brk in the midnight). There hve been severl successful robot pet products or prototypes. The bility to recognize fces nd receive voice commnds re the most common communiction chnnel. However, few of them hve utilized the gesture recognition, one of the most nturl communiction techniques between humn nd pets. The hnd gesture recognition depends on highly relible hnd trcking method, on which Kinect does pretty good job. In this project, we built robot dog for entertinment which cn recognize humn s hnd gesture. It cn ct like pet dog which hve fcil expressions nd voices nd ply with the mster. Besides, the robot cn ctively detect nd void obstcles, follow the mster, nd chse bll. II. RELATED WORK The most common extrioceptive sensor cn be ctegorized into three kinds: tctile, udition nd er. For tctile sensors, they could embed in the hed, chin nd bck, so the robot cn recognize stroking nd ptting. Like Pro[7], n dvnced interctive robot developed by AIST, he feels being stroked nd beten by tctile sensor, or being held by the posture sensor. Sony's Aibo dog [8] hs Hed touch sensor nd Chin Touch Sensor, it will immeditely rect Fig. 1. View of the robotic wlker we designed. when you stroke this touch sensor. Aibo is constntly lerning from intercting with you. For udition sensors, it is usully in the hed, so it cn detect sound nd recognize the source of the sound. When Aibo hers something it will nlyze the sound nd recognize words. The direction of the sound e from is lso perceived nd Aibo will turn his hed towrd the source. Pro cn lso recognize the direction of voice nd words such s its nme, greetings, nd prise with its udio sensor. For er, Aibo uses the Color vision er to interct with you nd its environment in number of wys. Aibo cn record movie clips, remember your fce, tke nd send color pictures by emil, or ply progrmmed CD trcks by recognizing the CD cover. Within its scope of vision, Necoro[9], very lifelike Robotic ct, cn perceive the direction of moving objects. With the light sensor, Pro cn recognize light nd drk. For hnd recognition, Zhi Li et l proposed to recognize hnd position by nlyzing the histogrm of depth imge [11]. In contrst to trditionl methods bsed on color informtion, hnd segmenttion is extrct by depth imge. It ssumes tht the hnd position is the nerest object from rnge er, nd put the depth dt into n bins of histogrm. The distnce of hnd is then the nerest bin which contins enough number of points.

2 Fig. 3. DogBOT stte trnsition III. SYSTEM OVERVIEW A. Physicl System Overview IPC server. The other components will receive the messge then do the correspondent tsks. E.g., fter detecting wving gesture, we send messge INTRO to IPC server. Another process strts the procedure of self-introduction fter receiving INTRO messge. IV. FUNCTIONALITY Fig. 2. Side view of DogBOT B. Softwre System Overview We bsiclly construct the entire control system in FSM design, which minly consist of five ctions. In IDLE stte, it does nothing but detect the hnd gesture by user; it greets nd sys hi to user in HELLO stte; it introduce itself in INTRO stte; it go chsing the bll in CHASE stte; finlly in COME stte, it follows humn until Stedy gesture is detected. Fig. X shows the four gestures we hve defined: circle, wving, stedy nd beckoning. We summrize the whole trnsition finite stte mchine in fig. 3. About the trnsition between sttes, we communicte with ech component by CMU Inter Process Communiction (IPC) [1]. IPC provides efficient messge pssing between processes. When the process responsible for gesture recognition detects new gesture by user, it sends messge to A. Gesture Recognition We define how to determine whether gesture is finished by mens of constructing finite stte mchines (FSM). First, we construct the FSM for ech gesture. By these FSMs, we could be wre of which of them is occurred. In the FSM, few steps re defined to finish. If user doesn t finish the hnd gesture in time limit or the behvior violtes the rule of gesture, it jumps to strt stte gin. Tht is to sy, user hs to do the hnd gesture from beginning. For exmple, wving gesture consists of 4 sttes, including repet twice of wving the hnd from left to right nd reverse. If the 4 tsk hs been completed, then we determine tht wving gesture is triggered by user. Fig. 4. Implementtion exmple of wving gesture. Begin with strt stte; it is finished fter going through ll steps sequentilly. It needs to do gesture gin when it comes to time exceeded or rule violtion.

3 B. Humn Following The reserches on humn s position detection nd trcking using only LRF sensor re incresing [2][3][4]. Horiuchi et l. [5] hve proposed pedestrin trcking system bsed on sensing from LRF mounted on mobile robot to identify potentil moving pedestrin trgets in the environment. Since our system is mde to follow the nerest humn, we filter out humns tht showing too close or too fr wy from the dog robot. Our finl project build hybrid pproch to Lser Rnge Finder (LRF) bsed humn leg detection system tht returns not only "true" or "flse" type of nswer but lso probbility. We first obtin the geometric informtion from mesurements mde by the lser rnge finder, nd this set of mesurement dt is further decomposed into severl sectors using segmenttion. And then we pply probbilistic model to compre these sectors with leg ptterns to check if they belong to the set of humn leg ptterns or not. Moreover, we lso use motion detector to check if these objects move or not s n enhncement of the detection. coordinte, we cn driving the pioneer to follow humn, since we only see the humn cndidte front of pioneer by 8 cm to 18 cm s the object of we wnt to follow, we cn void pioneer to hit the humn. Also during the wlk, our obstcle voiding system cn work on the first priority, our obstcle voiding is lser bsed, front of pioneer from to, the distnce is 6cm. Segment E Fig. 6. Y Segment D Y Segment C Segment B Segment A Segments re composed of severl lser points. X B d AB Figure. 5. is the system flowchrt of humn following system in our Robotics finl project. First, we mounted the lser rnge finder 4cm bove ground level to get the distnce of front of pioneer, the lser we use is SICK-LMS 1. The rnge of LMS 1 is 27 degrees nd 2 meters, resolution is.5 degree, the smpling rte of LMS1 is 5Hz. After sequence of scnning points is collected by the lser rnge finder, we segment the lser points s shown in figure 6. Then, we do the geometric ptterns mtching to find out tht whether the segment possible to be the humn or not. In figure 7, if 2 2 d OA OB 2OA OB cos b Fig. 5. Humn following system flowchrt. is in the threshold we defined, where ngle is the constnt vlue,, we cn define the segment s humn cndidte. Then we mtch the humn cndidtes to geometric ptterns to see if the segment like legs or not, the leg ptterns is shown in figure 8, where, b nd c re following stndrd of norml humn s leg size. And the forth step, we chnge the humn position coordinte to pioneer s coordinte. The fifth step, we use motion detector to exmine if the detected object moves or not, we use this formul to do motion exmintion, (1) Where is n rry of humn position of X-xis t time t, is n rry of humn position of Y-xis t time t, is n rry of humn position of X-xis t time t-1, is n rry of humn position of Y-xis t time t-1, if formul (1) is less thn threshold we defined, we will tke it out of humn cndidte. After we chnge the position to the pioneer C. Bll Chsing Our bll chsing system includes the following four steps : 1) Clculte hue histogrm of the bll 2) Bck projection of the histogrm on the current imge 3) Thresholding nd finding contour 4) Coordinte trnsform from imge to bse To chieve better nvigtion bility (i.e. Nerness Digrm), precise trget locliztion is required. bse bse p T T p, where O bse p nd n n 1 Fig. 7. Representtion of the point-distnce-bsed-method segmenttion. LA Ptterns FS Ptterns SL Ptterns b Fig. 8. bse nd imge respectively, b Schemtic representtion of the leg ptterns A p re the coordinte of the bll w.r.t. bse T is the c X

4 trnsformtion from er s coordinte to pioneer p3d x s, nd is the trnsformtion from imge to er. T D. Fcil Expression () norml Fig. 8. Representtion of the bll coordintes trnsformtion from imge to bse. We cn clculte bse T by using er clibrtion tool with the eqution: bse bse T T T, where T cn be mesured s extrinsic prmeter nd bse T cn be mesured by hnd. Here mens the checker used for er clibrtion. T, known s the inverse perspective trnsform, cn be solved by combing the constrint of the er mtrix (2) nd generl plne eqution of the ground (3). (b) wve (c) introduction px fc1 p y 1 fc2 cc1 cc2 1 ( p o ) n p p x y 1 pz pz (3) (2) (d) serch (e) hppy Fig. 9. Solve the trnsformtion mtrix with er clibrtion tool nd checker. (f) cry Fig. 1. Our dogbot s fce

5 A. Stereo Vision V. RESULTS Stereo vision is sensor which is used to produce depth imge of environment. The principle behind stereo vision is to compre the disprities of two imges, nd then depth imge is produced. However, we did some experiments on it, nd the performnce is not enough for us to extrct hnds in our ppliction. Becuse of the principle of stereo vision, the depth imge is rther incomplete when it comes to objects with less texture. Figure 11 shows rnging imge comprison between stereo vision (left) nd Kinect sensor. D. Bll Chsing Fig.12 Humn Following Fig.13 Hue histogrm Clculte the bse T bse T : Fig. 11. Rnging imge comprison between stereo vision (up) nd Kinect sensor (bottom). The imge produced by stereo vision is obviously rther broken. B. Kinect Hnd Trcking Cpbility The distnce cn be detected by Kinect sensor rnges widely. It rnges from.5 meter to 4 meter bove. And it costs 3 seconds to loclize the hnd of user in the first time use. And it just costs second to re-ctch the position of hnd if the hnd is going bck to the rnge of Kinect gin. C. Humn Following Our humn following system cn demonstrted in the open spce, with lots humn crossing by, s it will only chse the nerest humn, it won t follow the humn who fr wy from it. Although it presented well in Demo the lst course of Robotics, it my flsely detect some chirs s humns, so we hve to put some s to void this sitution. The figure 12 is the demonstrtion of humn following. To perform T, the inverse perspective trnsform, we combine the (1) nd (2) with the following mesured prmeters to obtin p. fc = [ ], cc = [ ] o [ ] n VI. DIVISION OF LABOR Gesture recognition: Ping-Che Hsio Humn following: Wei-Ho Mou Bll chsing: Kun-Ting Yu Fcil expression nd voice: Yi-Shu Li VII. CONCLUSION We hve implemented gesture recognition, the humn following system on our Robotics finl project, nd successfully demonstrte it on the lst course, lthough it my flsely detect some chirs s humn, but fter using some s to msk the chirs out, the humn following result is quite stisfied. The future work my be fusing web or Kinect to enhnce the ccurcy of humn correctness.

6 REFERENCES [1] C.-T. Chou et l, Multi-robot Coopertion Bsed Humn Trcking, in Proceedings of IEEE Interntionl Conference on Robotics nd Automtion (ICRA), 211. [2] E.A. Topp, H.I. Christensen, Trcking for Following nd Pssing Persons, IEEE Int. Conf. on Intelligent Robots nd Systems, pp , 25. [3] A. Fod, A. Howrd, M. J. Mtric, Lser-Bsed People Trcking, Proceedings of IEEE Int. Conf. on Robotics nd Automtion, vol. 3, pp , 22. [4] Z. Huijing, S. Ryosuke, I. Nobuki, A Novel System for Trcking Pedestrins Using Multiple Single-Row Lser-Rnge Scnners, IEEE Int. Trns. Systems, Mn nd Cybernetics, vol. 35, pp , 25. [5] T. Horiuchi, S. Thompson, S. Kgmi, Y. Ehr, Pedestrin Trcking From Mobile Robot Using Lser Rnge Finder, IEEE Int. Conf. on Systems, Mn nd Cybernetics, pp , 27. [6] [7] [8] [9] [1] Inter Process Communiction, [11] Zhi Li, Ry Jrvis, Rel time Hnd Gesture Recognition using Rnge Cmer, Austrlsin Conference on Robotics nd Automtion (ACRA), Dec 29.