VISUAL POSITIONING OF A DELTA ROBOT

Transcription

1 VISUAL POSITIONING OF A DLTA ROBOT Warin Poomarin, Kamol Chuengsatiansup, and Ratchatin Chancharoen Mechanical ngineering Department, Chulalongkorn University, Bangkok, 10330, Thailand mail: Ratchatin.C@chula.ac.th ABSTRACT An uncalibrated dynamic look and move visual servoing is successfully implemented on a light weight Delta robot to position it relative to an object which its position is sensed by a USB camera and is driven by Dynamixel motors. The online estimated Jacobian is proposed to map the deference in joint space to the difference of parameters in image space. The paper demonstrates that the estimated Jacobian is accurate and can be used for a dynamic look and move visual servoing in this case. The experiment demonstrates that the robot is three dimensional point to point position controlled in image space with its accuracy is defined in this space as well. KY WORDS Visual Servoing, Delta Robot. 1. Introduction An accuracy of a robot manipulator does not exceed its repeatability if the pose of its end effector is determined based on a robot s kinematic model. The kinematic model is a map between the joint positions and the pose of the manipulator in Catesian space, containing a lot of robot parameters including link lengths, link twists, joint offsets, and joint angles [1]. The accuracy of the kinematic model, thus, depends on the accuracy of these parameters. Normally, these parameters have nonlinear uncertainties that are affected by the environment factors (especially temperature), load conditions, and inherent dynamic properties from backlash and friction. One technique to design a high accuracy robot manipulator is to use highly rigid links and joints such that all the parameters in the kinematic model are constant and static calibration is used to determine all the parameters. This technique results in a heavy robot manipulator and still requires calibration which is normally time consuming and expensive. In this project, the digital CCD camera is used as a sensory feedback at the end effector and an uncalibrated position based visual servoing technique [2, 3, 4] is used to control a Delta robot [5] such that the accuracy can be defined in image space. The low cost and light weight Delta robot is designed and built with the web camera installed at its end plate. The robot is driven by three USB Dynamixel motors which is point to point (PTP) positioned control at 1024 resolution step by Dynamixel e electronics. The aim is to position the Delta robot relative to an object which its relative position is sensed by the camera. The challenge is the control technique that is flexible and robust for this task. An inverse Jacobian control [1] where the Jacobian is online estimated is successfully implemented to control the robot such that the robot is positioned controlled relative to an object. The experiment demonstrates that the robot can be PTP position controlled in image space which the accuracy is also defined in this space. 2. Image Processing Technique 2.1 Feature xtraction In this paper, the objects are assumed to be in circular shape and in different color with the background. Color based tracking technique [6] is used to filter the background. The digital camera gives an RGB image that contains the objects as in Fig. 1a. The RGB image is a three-dimensional image including red, green and blue images. To simplify the work, the objects are in red, green, and blue which are primary colors. The RGB image is processed to construct the gray image. To extract the red object, the red image is subtracted with the gray image. The resulting image is a one dimensional image that the red object is enhanced as shown in Fig. 1b. The thresholding [7] is then used to convert the resulting image into black and white image as shown in Fig. 1c. Then, Blob analysis [8] is used to determine the object s parameters including the centroid and the bounding box. These parameters are further processed to determine the Catesian position of an object. a. An original RGB image c. BW image by Thresholding technique b. RD enhancement by color analysis d. Centroid of a white object by Blob analysis Fig 1. Centroid of a RD circle in RGB image.

2 2.2 Camera Calibration In the paper, the technique, to positioning the robot to the target, is based on the Catesian position. Thus, camera calibration is required to convert the information in 2D image into the 3D positions of the monitored object. Basically, there are three categories of calibration techniques including Photogrammetric calibration [9], Self-calibration [10], and the hybrid technique [9, 11]. In Photogrammetric calibration, the calibration object is a specially designed three dimensional object which its geometry is precisely known. Thus, the actual geometry and its shape in image view are then used to determine the camera model. This requires an expensive hardware and also effort to setup. In Self-calibration, calibration object is not required. The camera takes a number of images at its various positions with a static object and background. The correspondences between the images are used to determine the camera model. Although the technique is flexible, the resulting camera model is not reliable due to many parameters involved. In this work, the technique proposed by Zhang [11] is used. In this technique, the object is a planar chess like pattern (see Fig. 3 for an example) and multiple views of the planar pattern are used to determine the camera model. Thus, it combines the advantage of the Photogrammetric calibration and Self-calibration such that it is flexible and the resulting camera model is reliable. In the project, Logitech HD Pro Webcam C920 is used. The camera s intrinsic parameters are determined as Resulting Intrinsic parameters: Focal Length: fc = [ ] Principal point: cc = [ ] Skew: alpha_c = [ ] Distortion: kc = [ ] Fig 4. Calibration images Fig 2. Calibration procedure In this paper, the calibration object is a chess like pattern as in Fig. 3 and six views from various positions (Fig. 2) of the camera are used to determine the camera model. The resulting images are shown in Fig. 4. The calibration toolbox [12, 13] is used to determine the camera intrinsic parameters including the focal length, the principal point, the skew coefficient, and distortion coefficient. Fig 3. Calibration pattern Fig 5. stimated camera positions from captured images Once the camera s model is determined, the model is capable of mapping the image coordinate (u,v) to normalized projection, (u n,v n ), by taken focal length, principal point, skew, and distortion into account. 2.3 Catesian Position of an object The Catesian position of an object relative to the camera is determined using camera projection model. The perspective projection [14] is used to map the normalize projection, (u n,v n ), into the Catesian position, C X, relative to the camera. The projection model in Fig. 6 shows the relation between the actual position in three dimensional space and its projection in image plane, (u n,v n ). The equation that maps the image coordinate to the Catesian

3 position is given in equation (1). The z-position of an object can be estimated from the relation between the actual size of a circular object and its size in the image plane. However, the actual size of an object is unknown in this project. Thus, the z-position is estimated from the relative positions of the camera poses which is determined from forward kinematic of the robot. proposed such that those homogeneous transforms are not required. The technique is an enhance version of the previously proposed online estimation of the image Jacobian as in [2] such that the estimated Jacobian is stable during the small step motion of the robot towards the goal. U B X T B X Fig 6. Projection model n u x ( xyz,, ) n v z y (1) Fig 7. Pose stimation Actually, the camera s intrinsic parameters can be used to estimate the pose of a point in the calibration pattern as shown in Fig. 7. The resulting pose (shown in green in Fig. 7) in this case is determined as T X C C x T Fig 8. Relevant coordinate frames between world, end-effector, camera and target frames 3. Delta Robot The Delta robot consists of three parallel mechanisms to constrain the end effector such that it parallel to the base at all time. Three motors sit on the base where their joints, called Hip joints, connect to the rigid link called Thigh. The Thigh link connects to the parallelogram mechanism called Shin using two spherical joints where the other side of Shin connect to the end plate via the other two spherical joints. In this way, the rotation of the three motors relates to the three translational position of the end plate. The torques from the motors can be transmitted to the end effector via low inertia rigid link. Thus, the robot is very rigid with very low inertia. In total, there are eleven links, twelve spherical joints, and three active rotational joints that are driven by three motors. Resulting xtrinsic parameters: Translation vector: Transl=[ ] Rotation vector: [ ] Rotation matrix: R= [ ] A technique to map the position of an object in an image into the three dimensional position of the object relative to the camera is already mentioned. Once C x, is T determined, this position can be mapped further to the object position relative to the end effector, to the base, and to the universal frame via uniform homogeneous transforms [1], T, B T C, and U BT respectively. Although these transforms can be determined by calibration technique, they are not a priori known in this project. An advance online estimation of the image Jacobian is Fig 9. The model of the CHULA parallel delta robot.

4 3.1 Kinematics of the Delta robot The kinematics of a Delta robot is the mathematics that maps the three actuator s positions to the three dimensional position of the end effector. The map is nonlinear three-dimensional space. However, there are twelve passive spherical joints involved which causing a complicated solutions to the inverse kinematics. Passing joint s variables must be eliminated to map the actuator s positions to the end effector s position in the closed form kinematics solution. Several techniques to solve kinematics of a Delta robot are proposed including [15, 16, 17, 18, 19, 20]. In this paper, a technique described in [5] is used to determine the inverse kinematic solution. 3. Hardware Setup Fig 10. Hardware Setup. The Delta robot in this project is driven by three USB dynamixel motors which is 1024 step position controlled by the dynamixel s electronics. The position command is updated at 0.2 second update rate. The actual motor s positions are read at the same rate. The Logitech HD Pro Webcam C920 is installed at the end effector to give a video feedback at about 30 Hz refresh rate. The Bluetooth gamepad is also used as human machine interface to control the flow of the program. In this way, USB is used as a bus in real time control. The objects are the printed color circles on the planar paper including red, green, and blue objects with different sizes. The overall system is simple and low cost and the position of the web camera that fixed to the end plate is not precisely known. The challenge is the control technique that can complete the task which is robust enough for this system. 5. Control Law Dynamic position based look and move (DPLM) [14] with online estimation of Jacobian is used to control the robot. The camera is in eye in hand configuration [14] and can sense the position of the object relative to the camera. The different between the proposed control law and the DPLM mentioned in [14] is that 1) the pose estimation is relative to the camera and the desire position is also defined relative to the camera. Thus, the error signal is relative to the camera. 2) The inverse kinematics [1] is used to convert the command Catesian position of the robot into the command joint s positions. In this way, the Jacobian maps the change in the error to the change in command Catesian position. This will uniform the Jacobian over the entire space and ease an online estimation of Jacobian. Although the Jacobian is uniform, it is updated at every step the robot move. This causes the technique to be more flexible and less sensitive to all of the system parameters. The performance of the technique is then relied on the accuracy of the Jacobian. C x ref T C x T 1 J Fig 11. Dynamic position-based look-and-move. Since the Jacobian can map the change of the commanded end effector s position, B X, to the change of the target position, C x, its inverse can map the error signal back to the change of the commanded end effector s position such that the robot move to a goal in the next step. The is introduced in the control law to control the maximum distant of the walk. The control law is thus designed as B X J ˆ[ C x ref C x ] (2) 6. Online estimation of jacobian The performance of the proposed visual positioning technique relies on the accuracy of the Jacobian that maps the change of the commanded end effector s position, B X, to the change of the target position, C x, relative to the camera frame. The Position based servoing technique is used to give the Jacobian uniform over the entire space. Thus, the camera calibration technique mentioned in section 2 is used only to gives the Jacobian uniform and aids the online estimation of the Jacobian. Although the Jacobian is mathematically uniform, it is updated at every step the robot move to increase the accuracy of the Jacobian using new information from the movement. Consider three step motions, at j-2, j-1, and j steps, of the Delta robot that the changes in end effector s positions, B X, [X x X y X z ] T, at each step are stacked in matrix as B B j2 B j1 B j D X X X X (3) As the robot moves with the change of end effector s positions, B X, the position of the target, C x, changes correspondingly. The changes of the C x, [ C x x C x y C x z ] T, in the past three steps are stacked and written in matrix form as C j2 j1 j D xx x x as (4) These two matrices are used to estimate the Jacobian ˆ C B 1 D x D X (5) J This technique is very simple but efficient [2]. A technique to estimate the Jacobian in realtime is a key for

5 uncalibrated dynamic vusual servoing as seen in [3, 4]. In this work, both end effector s positions, B X, and the target positions, C x, are monitored and recorded at every time step. Thus, the Jacobian is updated at every time step as well. However, the robot moves in the same direction causes matrix D B X to lose its rank as the column vector of D B X are parallel. One of the main contributions of the paper is a technique to estimate the Jacobian in realtime without losing its rank. When the robot is commanded to incremental move by B X, the perpendicular direction to X B is determined and noted it as X 1. Then, the perpendicular direction to both B X and X 1 is determined by cross product of those two vectors and noted it as X 2. X 1 and X 2 are then used to estimate C x if the robot moves in these two directions by C 1 ˆ 1 x J X C 2 ˆ 2 x J X (6) When the robot moves by X B, C x is sensed corresponding to the move. However, the Jacobian estimates C x as C* ˆ B x J X The x C that is actually sensed is used to update the C* x by C* x (new) = (0.9) C* x (old) + (0.1) C x. Then set DX B and Dx C as D B X = [ B X X 1 X 2 ] D C x = [ C* x C x 1 C x 2 ] In this way, the Jacobian is smoothly updated without losing its rank. 7. Task Performance valuation The task to evaluate the performance of the proposed technique is designed as point to point control in image space. There are three objects in different colors, i.e., red, green, and blue and in different sizes, i.e., 30, 25, and 35 mm respectively. The Delta robot is controlled such that the object is seen in the image by USB camera and its position is at the center of the image and its size is as marked by blue circle in Fig. 13 which is equivalent to z=-250 for red object. This is a three dimensional task. Starting at any position, the robot is to move to the red object and position itself such that the error is within the specified accuracy. Since the error is defined in image space, the task is to move the robot such that the object fits in the blue circle as seen in Fig 13. Once the positioning on the red object is complete, the robot moves to the green and then the blue object. Since the sizes of the objects are different, the robot will have to move in three-dimensional space to complete the task. (7) (8) Fig 12. Blue, green, and red targets. 8. Result The result shown in Fig. 13 demonstrates that the proposed technique can complete the task. In this result, the robot took 22 step moves to go from the start point as seen in Fig 13a to the position at the red object. Noted that in the last 7 steps to the red object, the robot moved around the object (see Table 1) until the error is within the specified limit (5 mm in this case). The robot took 11 and 7 steps to move to the green and blue objects respectively. Images, taken when the robot is positioning on the objects, are shown in Fig 13 to demonstrate that the robot can complete the task. The PTP trajectory is defined in image space. a. Start position b. Positioning at red target c. Positioning at green target d. Positioning at blue target Fig 13. PTP in image space positioning results.

6 The task can be designed as connect the dots game to control the robot to follow the trajectory that is defined in image plane. The accuracy is also defined based on end point feedback directly. The control performance is less sensitive to the robot kinematic. Table 1 Trajectory relative to camera during positioning. Step Target position in image space Note x y z Start Fig 13a Red Pos Fig 13b Green Pos Fig 13c Blue Pos Fig 13d 9. Conclusion The Position based dynamic look and move is successfully implemented on the low cost Delta robot system that is driven by USB dynamixel motors with a visual feedback by a USB web cam camera. The motors are positioned control by dynamixel electronics with 1024 step per revolution. The Logitech Web Camera is operated in VGA mode at about 30 Hz refresh rate. The camera is rigidly fixed on the end effector but its position is not precisely known. The proposed Position based look and move that estimate the Jacobian in realtime is able to control the robot to position itself to the objects. The robot is successfully controlled PTP in image space. References [1] John J. Craig (1989), Introduction to ROBOTICS: mechanics and control, Addison-Wesley, USA. [2] V. Sangveraphunsiri, R. Chancharoen, A Hybrid Force/Visual Servo Control for Industrial Robots, JSA Spring Convention, [3] J. A. Piepmeier, G.V. McMurray, H. Lipkin, Uncalibrated Dynamic Visual Servoing, I Trans on ROBOTICS AND AUTOMATION, [4] A. Shademan, A. Farahmand, M. Jagersand, Robust Jacobian stimation for Uncalibrated Visual Servoing, I International Conference on Robotics and Automation, Alaska, USA, [5] Olsson, A. (2009). Modeling and Control of a Delta-3 robot, Lund University, Sweden. [6] J. Weijer, C. Schmid, Coloring Local Feature xtraction, The 9 th uropean conference on Computer Vision, [7] M. Sezgi, B. Sankur, Survey over image thresholding techniques and quantitative performance evaluation, Journal of lectronic Imaging 13(1), pp , [8] A. Ming, H. Ma, A blob detector in color images, The 6 th ACM international conference on Image and video retrieval, New York, USA, [9] F. Remondino, C. Fraser, DIGITAL CAMRA CALIBRATION MTHODS: CONSIDRATIONS AND COMPARISONS, Image ngineering and Vision Metrology, [10] C. S. Fraser, Invited Review Paper Digital camera selfcalibration, ISPRS Journal of Photogrammetry & Remote Sensing 52 (1997) [11] Z Zhang, Flexible camera calibration by viewing a plane from unknown orientations, The Proceedings of the 7 th I International Conference on Computer Vision, [12] J. Heikkilä O. Silvén, A Four-step Camera Calibration Procedure with Implicit Image Correction, Computer Vision and Pattern Recognition, [13] P. F. Sturm S. J. Maybank On Plane-Based Camera Calibration: A General Algorithm, Singularities, Applications, Computer Vision and Pattern Recognition, [14] S. Hutchinson, G. D. Hager, P. I. Corke, A tutorial on Visual Servo Control, I Trans on ROBOTICS AND AUTOMATION, [15] M.A. Laribi, L. Romdhane, S. Zeghloul, Analysis and dimensional synthesis of the DLTA robot for a prescribed workspace, Mechanism and Machine Theory 42 (2007) [16] P.J. Zsombor-Murray, Descriptive Geometric Kinematic Analysis of Clavel s Delta Robot, Centre of Intelligent Machines, McGill University [17] M Lopez, Castillo, G Garcıa, and A Bashir, Delta robot: inverse, direct, and intermediate Jacobians, Proceedings of the I MCH Part C Journal of Mechanical ngineering Science, Vol. 220, No. 1. (2006), pp [18] Miller, K., xperimental verification of modeling of DLTA robot dynamics by direct application of Hamilton's principle, I conference on Robotics and Automation, [19] M.A. Laribia, L. Romdhanea, and S. Zeghloulb, Analysis and dimensional synthesis of the DLTA robot for a prescribed workspace, Mechanism and Machine Theory 42 (2007) [20] Staicu, St., Dynamic analysis of Clavel's Delta parallel robot, I conference on Robotics and Automation, 2003.