6-dof Eye-Vergence Visual Servoing by 1-step GA Pose Tracking

Transcription

1 International Journal of Applied Electromagnetics and Mechanics 41 (214) 1 1 IOS Press 6-dof Ee-Vergence Visual Servoing b 1-step GA Pose Tracking Yu Cui a, Kenta Nishimura a, Yusuke Sunami a, Mamoru Minami a and Akira Yanou a, a Graduate School of Nature Science and Technolog, Okaama Universit, Tsushimanaka, 7-853, Okaama, Japan Abstract. Visual servoing to moving target with hand-ee cameras fied at hand is inevitabl affected b hand dnamical oscillations, then it is hard to keep target at the centre of camera s image, since nonlinear dnamical effects of whole manipulator stand against tracking abilities of robots. In order to solve this problem, we come up with a sstem where the visual servoing controller of the hand and ee-vergence is separated independentl, so that the cameras can observe the target object at the center of the camera images through ee-vergence functions. The ee with light masses make the cameras ee-sight direction rotate quickl than the hand s, so the track abilit of the ee-vergence motion is superior to the one of hand s. In this report, merits of ee-vengence visual servoing for pose tracking have been confirmed through frequenc response eperiments on condition of full 6-dof pose being estimated in real time. Kewords: Visual Servoing, Ee-vergence, GA 1. Introduction Nowadas, in a field of robot vision, a control method called a visual servoing attracts attention [1]-[4]. Visual servoing, a method for controlling a robot using visual information in a feedback loop, is epected to be able to allow the robot to adapt to changing or unknown environments. An ee-vergence visual servoing sstem, where the visual servoing controller of hand and eevergence is separated independentl, is said to be superior to fied ones [5], which have been reported in [1]-[4]. When humans want to keep tracking a fast-moving object, it is not eas to catch up with it b head rotation motion, because it is hard for human to rotate the face to squarel position the object that moves quickl since the mass of head affects dnamicall to the rotating motions. However human s ees can keep staring at the object because of their small masses and inertial moments. On top of this, generall the final objective of visual servoing is deemed to lie in approaching the end-effector to a target object and then work on it, such as grasping. To achieve the above environment-adapting of visual servoing in which the desired distance between cameras and the target is time-varing, ee-vergence camera sstem is indispensable to keep suitable viewpoint all the time during the approaching visual servoing with camera rotation so as the cameras to observe the target at the center of camera images during the approaching motions. To verif the superiorit of ee-vergence sstem, frequenc response eperiments where target object moves with sinusoidal time profile have been conducted. In this report, the results of the eperiments are * Corresponding author: Mamoru Minami, Graduate School of Nature Science and Technolog, Okaama Universit, Tsushimanaka, 7-853, Okaama, Japan minami-m@cc.okaama-u.ac.jp /14/$8. c 214 IOS Press. All rights reserved

2 2 Yu Cui / presented. Based on them, it can be confirmed that ee-vergence have a better stabilit and traceabilit than fied-ee visual servoing sstem. 2. 3D Pose Tracking Method 2.1. Kinematics of Stereo-Vision We utilie perspective projection as projection transformation. Fig.1 shows the coordinate sstem of the dual-ees vision sstem. The target object s coordinate sstem is represented b Σ M, the i-th solid model coordinate sstem is represented b Σ Mi and the image coordinate sstems of the left and right cameras are represented b Σ IL and Σ IR. Σ W means world coordinate sstem and Σ H dose hand coordinate sstem of the robot. Camera R Camera L Robot Hand Σ Image R Σ Image L Searching Area j-th point of i-th solid model Σ real target object Fig. 1. Coordinate sstem of dual ees whose two cameras are set at the hand (represented b Σ H) of robot are defined as Σ CL and Σ CR. The coordinates of camera images are Σ IL and Σ IR, in which real target object defined b Σ M and i-th solid model defined b Σ Mi are projected from the 3D space depicted at right side) The position vectors projected in the Σ IR and Σ IL of arbitrar j-th point on target object can be described as IR r i j and IL r i j. The can be calculated as below: IR r i j = f R ( W φ i, Mi r j, q), IL r i j = f L ( W φ i, Mi r j, q). (1) These relations connect the defined points, Mi r j, on the 3D object and projected points on the left and right images, since the joint angle vector of robot, q, can be detected b robot. IR r i j and IL r i j can be calculated when the value of W φ i, which means pose(position and orientation) of the i-th model, is given b an assumption in Genetic Algorithm(GA) process Recognition Method Using Genetic Algorithm The 3D marker as a target with color and shape information used in this paper is shown in Fig.2. The pose of the target is represented b the coordinate sstems, Σ M, which is fied at the target. In order to measure the pose of the target object, man solid models having the same shape and color with the real target object but having different poses are prepared. The state that the i-th solid model and the real target object eist in 3D space is shown in Fig.3(the right side). The pose of the i-th solid model is represented as W φ i = [t i, t i, t i, ɛ i 1, ɛi 2, ɛi 3 ] (ɛ = [ɛi 1, ɛi 2, ɛi 3 ] is quaternion posture parameter) in Σ W.

3 Yu Cui / 3 The si elements of W φ i are epressed in binar with 12 bits. The shape and color of the solid model and the real target object projected to right camera image are shown in Fig.3 (the left side). Then a correlation function representing a matching degree of projected model against the real target object in the image is used as a fitness function in GA process. When the pose of the model and that of the real target object are identical, the fitness function value is to be maimied. We have confirmed that a correlation function having the highest peak with a condition of the pose of real target completel being identical to the model could be designed [6], [7]. So the problem of recognition for target object can be changed into an optimiation problem that eplores the maimum of fitness function and finds the W φ i to give the maimum. There are man methods for the eploration of maimum of functions. The easiest method is full eploration that calculates function value for all and find the maimum, which takes too much time for real-time recognition. Therefore, in this research GA, which can find the maimum more efficientl in shorter time, is used to solve it [8]. Furthermore, a 1-step GA is proposed for real-time visual control [9]. Targetobjectarea\S in" Backgroundarea\S out" 4mm [ IR r i j ]Sout [ IR r i j ]Sin i-th solid model 3mm 4mm 3mm 3mm 3mm 4mm Right Camera Image real target object in right image real target object Fig. 2. 3D marker Fig. 3. i-th object model is projected to right image plane whose target object area denoted b S in" and background area denoted b S out." Projected points IR r i j on S in" are represented as [ IR r i j] sin and on S out as [ IR r i j] sout. 3. Hand & Ee Visual Servoing Controller The block diagram of the proposed hand & ee-vergence visual servoing controller is shown in Fig.4. Based on the above analsis of the desired-trajector generation, the desired hand velocit W ṙ d is calculated as, W ṙ d = K Pp W r E,Ed + K Vp W ṙ E,Ed, (2) where W r E,Ed, W ṙ E,Ed can be calculated from homogeneous Matri connecting End-effector coordinate Σ E and desired pose of end-effector Σ Ed, E T Ed and E Ṫ Ed. K Pp and K Vp are positive definite matri to determine PD gain. The desired hand angular velocit W ω d is calculated as, W ω d = K Po W R E E ɛ + K Vo W ω E,Ed, (3)

4 4 Yu Cui / where E ɛ is a quaternion error [1] calculated from the pose tracking result, and W ω E,Ed can be computed b transforming into variables on base coordinates of Σ W using E T Ed and E Ṫ Ed. Also, K Po and K Vo are suitable feedback matri gains. The desired hand pose is represented as W ψ T d = [ W r T d,w ɛ T d ]T. The desired joint variable q d = [q 1d,..., q 7d ] T and q d is obtained b q d = f 1 ( W ψ d ) (4) [ W ] q d = K p (q Ed q E ) + K v J + E (q) ṙ d W ω d (5) where K p, K v is positive definite matrices and f 1 ( W ψ T d ) is the inverse kinematic function and J + E (q) is the pseudo-inverse matri of J E (q), and J + E (q) = J T E (J EJ T E ) 1. The Mitsubishi PA-1 robot arm is a 7 links manipulator, and the end-effector has 6-DoF, so it has a redundance. The hardware control sstem of the PD servo sstem of the robot PA1 is epressed as τ = K SP (q d q) + K SD ( q d q) (6) where K SP and K SD are smmetric positive definite matrices to determine PD gain. Furthermore, two rotatable cameras whose controller is separated from the hand s are installed at the end-effector, which constitutes ee-vergence sstem. Objective of visual servoing Ed TM, Ed_ TM Inverse kinematics q d Desired- trajector generation E M E M E M E TEd E_TEd E!Ed Coordinate transform ÜE ÜM W re;ed W _re;ed E Åè W!E;Ed Hand visual servoing block Velocit controller block Camera q Camera cd inverse controller _q cd kinematics block ú ú Ee visual servoing block Motion-FeedForward compensation Manipulator & Camera dnamics J(q) q+c(q;_q)_q +G(q)=ú r E q E q C P CR CL W E T ^M E_ T ^M Recognition b model-based matching and 1-step GA Raw image Camera Visual feedback block f(q) Fig. 5. Eperimental visual servoing sstem Fig. 4. Block diagram of the hand visual servoing sstem 4. Eperiment of Hand & Ee-vergence Visual Servoing 4.1. Eperimental Conditions The eperimental sstem is shown in Fig. 5, and the target 3D marker moves in the landscape direction. The initial hand pose is defined as Σ E and the initial object pose is defined as Σ M. Both are stationar in space as shown in Fig. 5. The relation between the object and the desired end-effector is set as:

5 Yu Cui / 5 Ed ψ M = [, 9[mm], 545[mm],,, ]. (7) Target object moves in landscape direction according to the following time function as: M M (t) = 15 cos(ωt)[mm], (8) 4.2. Definition of Trackabilit Here, to compare the trackabilit of the ee-vergence sstem and fied camera sstem, we define a concept of gaing point. As it is shown in Fig. 6, the intersection of both cameras gaing directions is defined as the gaing point of the cameras. Since the tilt angle of both left and right cameras is common b kinematical design, there alwas eists the intersection of gaing directions, i.e. gaing point in space. Gaing point Ü ^M M M Ü M ec Object M Ed M Ed L ee E R R ÜL L E Ü E Ü R Fig. 6. Cameras gaing point 4.3. Eperiment Results Fig.7 shows the results of the visual servoing eperiments in landscape direction, where the position and orientation of the object are recognied b the cameras respectivel(6-dof). We evaluated eevergence sstem comparing to fied camera sstem using gaing point performance with different angular velocit, ω(ω=.314, ω=.628, ω=1.256) as shown in Fig.7 (a)(b)(c). We did each eperiment for 6s for each value of ω, and got the average dela time and the amplitude to draw the frequence response curve. When the target object moves slowl, both hand and cameras can keep tracking with it, as shown in Fig.7 (a). While, as the raise of the velocit of the target object, the hand can not keep tracking with it, but cameras can gae at the object ever time, as shown in Fig.7 (b) and (c). The amplitude-frequence curve and the dela frequenc curve are shown in Fig.7 (d) and (e). Here, A means the amplitude of target object shown in (a), (b) and (c), and B means the maimum amplitude of end effector or gaing point. In these two figures the abscissa aes are ω. From (d), the curve of the fied camera sstem is alwas below the curves of the cameras, we can see that the amplitude of the ee-vergence sstem is more closed to the target object than the fied camera sstem. And from (e), the curve of the fied camera sstem is also below the curves of the cameras, which means that the dela time in the fied camera sstem is longer than in the ee-vergence sstem.

6 6 Yu Cui / position 2 Target object 15 End effector Gaing point position 2 Target object End effector Gaing point Time[s] (a) ω=.314 (b) ω=.628 Time[s] position 2 Target object End effector Gaing point (c) ω=1.256 Time[s] Gain 2log(B/A)(dB) 1 Gaing point End effector (d) amplitude-frequenc Frequenc[rad/s] Phase( ) 1 Gaing point End effector (e) angle-frequenc Frequenc[rad/s] Fig. 7. The object moves in landscape direction and it s position and orientation are estimated through 1-step GA," then the are informed to the visual servoing controller as shown Visual Feedback Block in Fig.6. Then the frequenc response results in this figure include robot s dnamics and pose-tracking dnamics. 5. Conclusion In this paper, we have carried out visual servoing eperiments on frequenc responses to evaluate the observabilit and trackabilit of hand & ee-vergence visual servoing with 1-step GA. Based on the eperimental results, we have analed the trackabilit, amplitude-frequenc and phase-frequenc responses of the cameras of the ee-vergence sstem under moving object with different angular velocit. The eperiments have shown that the trackabilit and observabilit of the ee-vergence sstem has been confirmed to function and the ee-vergence sstem s abilit to track a target in camera-depth direction has been clarified through frequenc response eperiments. References [1] S.Hutchinson, G.Hager, and P.Corke, A Tutorial on Visual Servo Control", IEEE Trans. on Robotics and Automation, vol. 12, no. 5, pp , [2] P.Y.Oh, and P.K.Allen, Visual Servoing b Partitioning Degrees of Freedom", IEEE Trans. on Robotics and Automation, vol. 17, no. 1, pp. 1-17, 21. [3] E.Malis, F.Chaumentte and S.Boudet, 2-1/2-D Visual Servoing", IEEE Trans. on Robotics and Automation, vol. 15, no. 2, pp , [4] P.K.Allen, A.Timchenko, B.Yoshimi, and P.Michelman, Automated Tracking and Grasping of a Moving object with a Robotic Hand-Ee Sstem", IEEE Trans. on Robotics and Automation, vol. 9, no. 2, pp , [5] Wei. Song, M. Minami, Fujia Yu, Yanan Zhang and Akira Yanou 3-D Hand & Ee-Vergence Approaching Visual Servoing with Lapunouv-Stable Pose Tracking ", IEEE Int. Conf. on Robotics and Automation (ICRA), pp.11, 211. [6] Koichi Maeda, Mamoru Minami, Akira Yanou, Hiroaki Matsumoto, Fujia Yu, Sen Hou Frequenc Response Eperiments of 3-D Full Tracking Visual Servoing with Ee-Vergence Hand-Ee Robot Sstem SICE Annual Conference pp.11-17, 212. [7] W. Song, Y. Fujia, M. Minami, 3D Visual Servoing b Feedforward Evolutionar Recognition, Journal of Advanced Mechanical Design, Sstems, and Manufacturing (JSME), Vol.4, No.4, pp , 21. [8] T.Nagata, K.Konishi and H.Zha: "Cooperative manipulations based on Genetic Algorithms using contact information", Proceedings of the International Conference on Intelligent Robots and Sstems, pp.4-5, 1995 [9] M.Minami, W.Song, Hand-ee-motion Invariant Pose Estimation with On-line 1-step GA -3D Pose Tracking Accurac Evaluation in Dnamic Hand-ee Oscillation", Journal of Robotics and Mechatronics, Vol.21, No.6, pp (29.12) [1] W. Song, M. Minami, S. Aoagi, On-line Stable Evolutionar Recognition Based on Unit Quaternion Representation b Motion-Feedforward Compensation", International Journal of Intelligent Computing in Medical Sciences and Image Processing (IC-MED) Vol. 2, No. 2, Page (27).