FABRICATION AND CONTROL OF VISION BASED WALL MOUNTED ROBOTIC ARM FOR POSE IDENTIFICATION AND MANIPULATION

Transcription

1 FABRICATION AND CONTROL OF VISION BASED WALL MOUNTED ROBOTIC ARM FOR POSE IDENTIFICATION AND MANIPULATION Induraj R 1*, Sudheer A P 2 1* M Tech Scholar, NIT Calicut, , indurajrk@gmail.com: 2 Assistant Professor, NIT Calicut, , apsudheer@nitc.ac.in Abstract Locating and handling different work part configuration in a material handling system is more effective with the assistance of a vision system. Usually object identification done through RGB image which is illumination sensitive and other interferences from the surrounding. These limitations can be avoided if object recognition is done by depth segmentation. This work is a 3D vision sensor based work piece re-orientation. The work piece is changed to the desired orientation by a wall mounted robotic arm. In industries delta robots are used for fast response but it is not capable to handle all work part configurations. This is solved using a serial manipulator which is having four degrees of freedom. This work also presents modeling, kinematic and dynamic analysis of the robotic arm. This project work focus on real-time object pose recognition based on point cloud approach. It can be applied for the automation of any work cell with the vision assistance. Main focus of the work is on control of the manipulator by processing the point cloud data. Keywords: Work piece handling, Vision Assisted Robot, 3D Vision, Re-orientation 1 Introduction As a comprehensive technology, machine vision has been used widely in various fields including industrial fields making a significant contribution to ensure competitiveness in modern industrial manufacturing. On automatic sorting lines, machine vision is used to detect and track the moving target and guide sorting robot complete the sorting task. Application of computer vision is extensively used in the food industries for automated sorting of vegetables, inspection, labeling, packing, etc. In an automated production line such as Flexible Manufacturing System (FMS), material handling is completely automated by Automated Guided Vehicle (AGV), conveyor, Automated Storage and Retrieval System (ASRS), robot, etc. With the invention of NC machines the entire production process were automated without any human intervention. The technological advances lead to the development of CNC and DNC by which the operator is able to control the production line with few mouse clicks. As far as the computer vision in industry is concerned, it is still growing at a fast pace. It may be due to the need for more processing power or lack of innovative sensing technology. The present scenario of the vision system used in industry and its drawbacks are described in the following section. Object recognition systems have been an active research topic over the last decade. Wenchang Zhang et al. [1] proposed a vision based control strategy to perform high speed pick-and-place tasks on automation product line. CCD camera used for image capturing by which position and shape of objects obtained. The sorting is done by a delta robot. This model used a target tracking method based on servo motor with synchronous conveyer and used LabVIEW for control. Recent technology advances have enabled the development of devices, as for example the Microsoft Kinect that captures 3D data from the real world. Many works have been done as for obstacle avoidance in indoor environments. Jose-Juan Hernandez-Lopez et al.[2] used both color and depth data for object detection from a scene. RGB image is converted to the CIE-Lab color space to avoid light sensitivity. A mask is generated using depth information to extract object of interest. Since RGB and depth cameras of Kinect is having an offset, homography relationship is used. Antonio Sgorbissa et al.[3] used a structure based object representation with Scale Invariant Feature Transform (SIFT) algorithm for object identification in indoor environment. In this work he used Euclidean clustering and RANSAC plane estimation to identify different furniture in the scene. Most of the processing techniques for 3D point clouds are computationally intensive. The processing power requirement for depth image is reduced with fovea approach [4, 5]. It proposed a new method in which more processing power is dedicated to a small area in the in the image where the object of interest is present. Data obtained from a 3D spin image has been used for 3D difference detection between a CAD model 355-1

2 FABRICATION AND CONTROL OF VISION BASED WALL MOUNTED ROBOTIC ARM FOR POSE IDENTIFICATION AND MANIPULATION and a real object by Svenja Kahn et al.[7]. Rostislav Hulik et al.[8] used Hough transform techniques in 3D image processing and it is applied for the continuous plane detection from a point cloud. The proposed system is then compared with RANSAC-based plane detector.. 2 Computer vision in industry Out of the vision systems used in industries, most of them are based on 2D vision sensors. Image will be taken by CCD camera and processing will be done by 2D processing techniques. Such systems rely on color segmentation or boundary detection for the extraction of the required data from the RGB image. This is having fast response since low processing power is required. On the downside it has drawbacks such as color of background affects the performance and proper illumination methods are necessary. 3D vision sensors such as Time of flight (TOF) camera and stereo imaging are very rarely used in industry because of the lack of fast response. A product from Microsoft called Kinect can be used as a solution for this problem. Kinect has its own infrared light source for depth imaging and its sensor is capable of giving data at a rate of 30 frames per second. In machining process, where re-orientation of the work piece is required frequently, robotic arm can be used with the assistance of kinect vision. This avoids the need for continuously recording the orientation of work piece in a transfer line. Since a vision system based on depth imaging is used there will be no need of illumination facility and is more reliable. Depth segmentation is used rather than color segmentation for object identification. The robotic arm is mounted on wall which results in less utilization of the floor space. 3 Model of robotic manipulator Most of the industrial robots or serial manipulators are having five or six degrees of freedom (DOF). In this work a robotic arm with four degrees of freedom is fabricated for the work piece re-orientation. By reducing the degrees of freedom of a serial manipulator the control becomes easy. A 4 DOF robotic arm is designed and fabricated for a particular workspace configuration. Since reduction in DOF leads to reduced flexibility, it cannot adapt to the changes in work space configuration. 3.1 Four - DOF robotic arm The design with axes representation is as shown in figure 1. Joint 1 provides yaw. Pitch is provided by joint 2 and 3. Roll is done through joint 4. Figure 1 Schematic diagram of robotic arm Joint θ d a α (degree) () () (degree) 1 θ 1 0 l θ 2 0 l θ θ l 3 +l Table 1 DH parameters of robotic arm By conducting dynamic analysis (Lagrange-Euler formulation) servomotors were selected for actuation of each joint. The maximum payload considered in the design is 0.5 kg. The fabricated robotic arm is shown in figure Workspace Figure 2 Fabricated robotic arm With the help of Denavit-Hartenberg (DH) parameters of the robotic arm given in Table 1, forward kinematic relations are formulated and workspace plotted in MATLAB using scatter3 function as shown in figure 3. This plot helps to study effect of reduced DOF on reachable workspace of the robotic arm

3 Figure 3 Workspace of robotic arm Rotation of 180º is provided for joints 1, 3 and 4. For joint 2 rotation is limited to 90º downward since the robotic arm is vertically mounted at a height from the floor and its work space of interest lies below to it. For re-orienting the work piece to the desired pose rotations of joint 2, 3 and 4 are used. 4 Vision and control The vision sensor used is Microsoft Kinect is capable of both RGB imaging and depth imaging of object. In this work only the depth imaging capability is utilized. Kinect has its own infrared light source for structured lighting and depth imaging. It can work even in the absolute darkness. Data from Kinect is used for database recording of different poses and then pose identification by the software called Processing. Processing also controls the robotic arm, for reorientating objects, with the help of Arduino uno microcontroller. An external dc power supply of 4.9V is used for actuating robotic arm. Schematic diagram is shown in figure 4. Kinect 10 Z Database collection Y X Processing Figure 4 Schematic diagram 0 Arduino Pose identification Robotic Arm 4.1 Three dimensional vision Point cloud approach Point cloud is a collection of co-ordinate data of the points on the surface of the object on which infrared rays from the Kinect can reach. The co-ordinate system (XYZ) of Kinect and Processing are different which is considered while using inverse kinematics. To obtain the point cloud data of the object, segmentation algorithm (Refer Algorithm 1) is used in which boundary is limited in x, y and z axes. To compare the different point clouds the first step is to align a common reference point. To attain this, the co-ordinates of each point cloud is updated with its centroid as origin. Algorithm 1 Processing algorithm for Point cloud segmentation Input point cloud {P} in from kinect Input boundary limits in XYZ directions Output point cloud {P} out for every point in {P} in if point in {P} in inside boundary limits store the point in {P} out CG = average of {P} out //finding centroid of {P} out for every point in {P} out substract CG //updating {P} out with CG as origin Algorithm 2 Processing algorithm for Point cloud refining Input point cloud {P} out //output of algorithm 1 Input number of scans parameter K 1 Input point elimination parameter d 1max Output point cloud {P} 1 //each time of using {P} out it get updated {P} 1 = {P} out //storing first point cloud for K 1 iterations {P} 2 = {P} out //storing updated point cloud for every point in {P} 1 find closest point in {P} 2 if distance to closest point < d 1max find average and update {P} 1 else delete point from {P} 1 While scanning some junk co-ordinates also get included due to improper reflection of infrared rays. To obtain more accurate results average point cloud of K 1 number of consecutive scans is taken. At each scan some points are deleted deping on the parameter 355-3

4 FABRICATION AND CONTROL OF VISION BASED WALL MOUNTED ROBOTIC ARM FOR POSE IDENTIFICATION AND MANIPULATION d 1max which is the maximum allowable distance of each point to the closest point in consecutive scans. If the distance is more than d 1max it corresponds to a junk data. The variables K 1 and d 1max are changed to get an optimum result. With increase in these values number of points in output point cloud {P} 1 also reduces which may also give inaccurate results. Algorithm 2 is used to attain these results. Same algorithm is used for collecting the database point clouds of different poses and for scanning point cloud for pose identification. Algorithm 3 Processing algorithm for Point cloud matching Input point clouds {PC} 1 {PC} 2 {PC} 3 //point clouds from database Input point cloud {P} 1 //point cloud for matching Input d 2max //maximum allowable distance while comparing two points for similarity Output score1, score2, score3 //matching score with each point cloud in database score1=0, score2=0, score3=0 for every point in {P} 1 if distance with closest point in {PC} 1 < d 2max score1++ if distance with closest point in {PC} 2 < d 2max score2++ if distance with closest point in {PC} 3 < d 2max score3++ Matching of a scanned point cloud with that in the database is according to Algorithm 3. It is controlled by a factor d 2max which is the allowable distance with the corresponding point in the database. For experimental purpose database of three poses are collected as {PC} 1 {PC} 2 and {PC} Controlling robotic arm Based on the score value the pose is identified and Processing selects the sequence of joint rotations for reorienting the object into required pose. Picking operation is made through the inverse kinematic relations from centroid of the point cloud. Position of the Kinect in the base frame of the robot is essential for proper functioning of inverse kinematics. Communication with micro controller is done through serial communication. Micro controller contains coding in its memory for the control of six servo motors connected to it. 5 Results and discussion Design and fabrication of robotic arm are completed. Robot is integrated with vision sensor. Threshold values: d 1max = 3 and d 2max = 1.5 are selected after testing the results with different values of the distance threshold parameters for image processing and analysis. The number of iterations K 1 is taken as 10 for getting average point cloud. The algorithm has identified the pose of object accurately in every test runs. However, the centroid of the object calculated is different from the actual value. This is mainly because three sides of object is only scanned by the Kinect sensor during the processing. Due to this, small errors in inverse kinematics are occurred, which has affected the picking operation. Based on the identified pose the algorithm selects the sequence of joint movements from memory to reorient the object. One of the limitations of the Kinect based vision system is the minimum range of depth camera (about 20 inches) within which no data will be available. Another serious issue is occlusion which has reduced by carefully selecting the position of Kinect. Proposed vision based robotic manipulator has the limitation of adaptation to some points in workspace, where it cannot perform picking operation. This is due to the low resolution of sensors/actuators and problems with vision sensor mentioned above. 6 Conclusion and outlook The proposed robotic arm is fabricated and automated with microcontroller. Algorithm is presented for pose identification with point cloud approach. Based on the identified pose re-orientation is carried out by selecting programmed cycles. Experiments conducted on serial robotic arm to validate the proposed control strategy. Further improvements can be done by incorporating color data and by full scanning of the object. The system can be integrated with work cells and by changing the programming, the system can be applied also in the sorting operation of work parts. References [1] Wenchang Zhang., Jiangping Mei. and Yabin Ding. (2012), Design and development of a high speed sorting system based on machine vision guiding, Physics Procedia, Vol. 25, pp [2] Jose-Juan Hernandez-Lopez., Ana-Linnet Quintanilla-Olvera., Jose-Luis Lopez-Ramırez., Francisco-Javier Rangel-Butanda., Mario-Alberto Ibarra-Manzano. and Dora-Luz Almanza-Ojeda. (2012), Detecting objects using color and depth segmentation with kinect sensor, Procedia Technology, Vol. 3, pp [3] Antonio Sgorbissa. and Damiano Verda. (2013), Structure-based object representation and classification in mobile robotics through a Microsoft Kinect, Robotics and Autonomous Systems, Vol. 61, pp [4] Isil Bozma, H. and Hulya Yalcin. (2002), Visual processing and classification of items on a moving conveyor: a selective perception approach, Robotics and 355-4

5 Computer Integrated manufacturing, Vol. 18, pp [5] Rafael Beserra Gomes., Bruno Marques Ferreira da Silva., Lourena Karin de Medeiros Rocha., Rafael Vidal Aroca., Luiz Carlos Pacheco Rodrigues Velho. and Luiz Marcos GarciaGonçalves. (2013), Efficient 3D object recognition using foveated point cloud, Computers & Graphics, Vol. 37, pp [6] Mattone, R., Campagiorni, G. and Galati, F. (2000), Sorting of items on a moving conveyor belt. Part 1: a technique for detecting and classifying objects, Robotics and Computer Integrated Manufacturing, Vol. 16, pp [7] Svenja Kahn., Ulrich Bockholt., Arjan Kuijper. and Dieter, W. F. (2013), Towards precise real-time 3D difference detection for industrial application, Computers in Industry, Vol. 64, pp [8] Rostislav Hulik., Michal Spanel., Pavel Smrz. and Zdenek Materna. (2013), Continuous plane detection in point-cloud data based on 3D Hough Transform, Journal of visual communication and image representation, Vol. 25, pp [9] Han-Pang Chiu., Leslie Pack Kaelbling. and Tomas Lozano-Perez. (2009), Learning to generate novel views of objects for class recognition, Computer Vision and Image Understanding, Vol. 113, pp