Low Cost Robot Arm with Visual Guided Positioning

Size: px

Start display at page:

Download "Low Cost Robot Arm with Visual Guided Positioning"

April Reynolds
6 years ago
Views:

Low ost Robot Arm wth Vsual Guded Postonng Petra Đurovć, Ratko Grbć, Robert upec and Damr Flko Josp Juraj Strossmayer Unversty of Osjek, Faculty of Electrcal Engneerng, omputer Scence and Informaton

1 Low ost Robot Arm wth Vsual Guded Postonng Petra Đurovć, Ratko Grbć, Robert upec and Damr Flko Josp Juraj Strossmayer Unversty of Osjek, Faculty of Electrcal Engneerng, omputer Scence and Informaton Technology Osjek, Osjek, roata Abstract Low cost robotc solutons are of great mportance for mprovement and development of robotcs. In ths paper, two vsually guded low cost robot arms are proposed. The proposed system performs automatc handeye calbraton and, after the calbraton, postons ts end effector above the object of nterest usng vsual servong based on off the shelf marker tracker. The presented experments demonstrate postonng accuracy of the proposed setup. I. INTRODUTION Robot arms are today wdely used n ndustral automaton. Ther accuracy, relablty and flexblty make them unavodable components of many producton processes. However, hgh cost of ndustral robot arms hnders ther wde applcaton n households and educaton. The prces of the smallest ndustral robot arms wth sub-mllmeter repeatablty are ncomparably hgher than prces of average household machnes. Therefore, low cost robotc solutons are of the great mportance for the further development of robotcs, snce they broaden the populaton of robot users and also expand the base of researchers n ths feld. II. RELATED RESEARH There exst many researches where vsual servong s appled for control of a robotc arm. In some of them, ndustral robots [1] are beng used and eye-n-hand [2], or hybrd eye-to-hand/eye-n-hand [3] confguraton s appled. Also, varety of hand-eye calbraton methods are appled, such as convex lnear matrx nequalty [4]. Our setup conssts of a low-cost robotc arm and an off-theshelf camera n eye-to-hand confguraton. Ths setup, whch conssts of sgnfcantly less expensve components n comparson to the hardware used n the aforementoned related research, together wth an evaluaton of the absolute postonng accuracy acheved by the proposed system represents the contrbuton of ths paper. III. A. omponents VISION BASED ROBOT ANIPULATION SYSTE Snce the man goal of ths paper s to provde an nsght n the postonng accuracy whch can be acheved by low cost vson guded robot manpulators, all In ths paper, development of low cost robot arms s consdered. The proposed soluton conssts of only basc components, whch allow mplementaton of a vsonbased robot manpulaton system. The presented research s bologcally nspred n the sense that hand-eye coordnaton, typcal for lvng bengs, s used to acheve postonng accuracy nstead of relyng on proproceptve sensors n the robot jonts, common n a vast majorty of ndustral robots. The goal of ths nvestgaton s to fnd out what accuracy can be acheved wth such a smple confguraton. The paper s structured as follows. Secton 2 provdes a short overvew of related research. Secton 3 explans components of the proposed vson-based robot manpulaton system and mplementaton of the calbraton and vsual servong algorthms, whle Secton 4 presents the performed experments and ther results. Fnally, oncluson comments on the results and offers deas for future work. Fgure 1. Dobot Arm mounted on the stand

software. Two robots are consdered: Dobot Arm V1.0 and a custom made robot arm n SARA confguraton, named VIRA (Vson ontrolled Robot Arm).

2 components are acqured from the lower prce specter. The robot manpulaton system shown n Fgure 1 conssts of a robot arm, a camera mounted on a stand, a marker used for vsual servong nstalled on the robot s end effector and a vson-based robot control software. Two robots are consdered: Dobot Arm V1.0 and a custom made robot arm n SARA confguraton, named VIRA (Vson ontrolled Robot Arm). The prce of the Dobot Arm was 900$ and the manufacturng of the mechancal constructon whch connects the robot arm wth a camera on a pan-tlt head costed 320 EUR. The development and manufacturng of VIRA prototype costed approxmately 3000 EUR. The Dobot package ncludes Dobot Arm, Dobot controller, 5 peces of effectors, power adaptor, USB cable, toolkt, extenson cable and base. Dobot Arm has 4 axes and 0.5 kg payload wth maxmum reach of 320 mm and +/- 0.2 mm poston repeatablty. Dobot Arm has three jonts wth stepper motors: base, rear arm and forearm, and the fourth jont wth rotaton servo. It weghs 3 kg and t s made of alumnum alloy. Dobot Arm s controlled by Dobot ontroller whch s based on Arduno ega 2560 and FPGA board placed nsde a control box. It can communcate wth a P va USB- UART. A Dobot Arm s shown n Fgure 2. The Dobot arm and the RGB-D camera are mounted on a stand thereby formng a compact vson-based robot manpulaton unt. The stand was desgned n SketchUp [13] and bult n a local workshop. The stand shown n Fgure 1 was desgned to provde secure camera holdng n adequate poston w.r.t. the robotc arm. The camera s postoned n such a way that the robot arm workspace s completely contaned nsde the camera's feld of vew. It s located suffcently close to the robot's workspace n order to acheve precse measurement of objects whch the robot should manpulate wth. In the same tme, the dstance to the closest object must be greater than the lower bound of the camera range. Furthermore, the marker used for vsual servong must always be clearly vsble. Fgure 3. Robot manpulaton system wth VIRA VIRA, shown n Fgure 3, s a custom made robot arm n SARA confguraton. It has one translatonal and three rotatonal jonts. The frst three jonts, whch poston the tool, are drven by stepper motors, whle the fourth jont, whch defnes the tool orentaton, s drven by a D servo motor. The currently used tool s a grpper drven by a servo motor, but t can easly be replaced by alternatve tools. The frst translatonal jont enables vertcal reach of approxmately 0.6 m and the two rotatonal jonts enable horzontal reach of approxmately 0.6 m. The weght of the robot s approxmately 8 kg, whch makes t sutable for mountng on a moble platform. The robot s controlled by Arduno ega 2560 wth RAPS 1.4 sheld whch communcates wth a P va USB. The robot s desgned for a pan-tlt head wth a camera to be mounted on the top of the frst axs, n order to support computer vson-based control. Also, nexpensve mcro-swtches are used as end-stop for all three axes. Each of the two consdered robot arms has ts advantages and drawbacks. VIRA s more expensve than Dobot, but t has greater payload and wder operatve range. Also, because of ts confguraton, n specfc postons, Dobot s jonts nterfere wth camera s vew of the marker used for vsual feedback. On the other hand, VIRA provdes undsturbed vew of the marker n each poston. Fgure 2. Dobot Arm V1.0 Both consdered robot confguratons are four axes robot arms, whose absolute postonng s acheved by a 3D percepton sensor beng the only sensor appled. The frst three axes of the robot are drven by stepper motors wthout encoders and the fourth axs s drven by a D servo motor. The robots are controlled by an Ardunobased controller and the appled 3D percepton sensor s

3 an affordable off-the-shelf RGB-D camera, Orbbec Astra S, costng 150 US$. Although a common RGB-camera can be used for mplementaton of vsual feedback, a RGB-D camera Orbbec Astra S s used n the consdered system because we wanted to buld a system wth 3D vson capablty, snce results of recent computer vson research clearly demonstrate advantages of 3D vson systems [6] [10]. Orbbec Astra S s a camera optmzed for short-range use cases, from 0.35 m to 2.5 m. It s compatble wth exstng OpenNI [14] applcatons and provded wth Astra SDK software. Astra SDK s used for acqurng mage from the camera whch s then processed by algorthms presented n Sectons III.B and III.. Even though presented manpulaton module currently doesn t use depth obtaned from the RGB-D camera, t s planned to use ths nformaton n future versons. Both robot manpulators are based on stepper motors and no absolute encoders or any other proproceptve sensor are used n ths research for measurng absolute poston. Stepper motors are controlled by seres of mpulses defnng relatve changes n jont angles. Hence, n order to acheve absolute postonng, the ntal poston must be known. Furthermore, stepper motors could lose mpulses, whch could lead to wrong postonng. In order to acheve precse absolute postonng, vsual servong usng RGB-D camera s appled. Vsual servong s mplemented by detecton and localzaton of a marker mounted on the robot arm close to the end-effector. arker detecton and pose estmaton s mplemented usng ArUco lbrary for augmented realty [11] based on Open V [12]. Snce an object of nterest s not always captured n the current camera feld of vew, the camera s mounted on a pan-tlt head. Ths way, the camera s able to rotate up and down, left and rght and fnd the rght angle to successfully capture objects of nterest. The pan-tlt head also allows navgaton of moble robot manpulator, whch can be created by mountng of the presented vson guded robot arm on a moble platform. In the experments reported n ths paper, the capabltes of the pan-tlt head are not used,.e. the camera orentaton remans fxed durng the experment. The control software conssts of functons for controllng robot s jonts [15], bult n Python, and functons for vsual servong wrtten n ++. B. Hand-Eye albraton Systems where a robotc hand s guded by a camera, whch represents eyes, are called Hand-eye systems. There are two man types of these systems: () the eye-n-hand confguraton, where the camera s mounted on the arm, close to ts end effector, and moves along wth the end-effector, and () the eye-to-hand confguraton, where the camera s statonary and overlooks the robot s movements. In ths paper, the second confguraton s used, prmarly because the camera was too bg to be mounted on the appled robot arms. In order to be able to coordnate the robot movement wth the nformaton obtaned from the camera mages, the pose of the robot reference frame S R w.r.t. the camera reference frame S must be determned. A calbraton procedure used to compute ths pose s known as hand-eye calbraton. To perform a hand-eye calbraton, a set of mages of a calbraton object s acqured. The calbraton object used n the research presented n ths paper s a marker mounted close to the end effector of the robot arm, whch s also used for vsual servong. Frst, let s ntroduce the notaton used n the rest of ths paper. In ths paper, B T A denotes the homogeneous transformaton matrx representng the pose of a reference frame S A w.r.t. a reference frame S B. Analogously, B R A and B t A denote rotaton matrx and translaton vector defnng the orentaton and poston of S A w.r.t. S B respectvely. A rotaton matrx and a translaton vector are ncluded n a homogeneous transformaton matrx as follows B B RA t B A TA Furthermore, the followng notaton s used for reference frames: R represents robot, camera, G tool, and marker reference frame. The pose of S R w.r.t. S can be computed from three marker postons, referred to n ths secton as poston 0, 1 and 2. Assumng that the marker s clearly vsble, the appled vson software provdes us wth 3D poston of the marker denoted by t (k), for each robot poston k = 0, 1, 2. If the robot moves from poston 0 to poston 1 along the x-axs of ts reference frame, then the marker postons t (0) and t (1) defne the x-axs of the robot reference frame vewed from the camera reference frame,.e. unt vector t 1 t 0 xr t 1 t 0 represents the x-axs of S R w.r.t. S., (1) Furthermore, f the robot moves from poston 1 to poston 2, where the both postons have the same z- coordnate n the robot reference frame, then the marker postons t (1) and t (2) defne a vector perpendcular to the z-axs of the robot reference frame. onsequently, unt vector z R xr b, (2) x b R

4 where b t 2 t 1, (3) represents the z-axs of S R w.r.t. S. Vectors x R and z R completely defne the orentaton of S R w.r.t. S, whch can be represented by rotaton matrx where R R R R R x y z (4) y z x. (5) R R R The poston of the orgn of S R w.r.t. S can be computed from t (0), the prevously determned rotaton matrx R R and the poston of the marker w.r.t. the robot end effector G t, whch s expected to be known n advance. The relaton between t (0) and the correspondng poston of the end effector w.r.t. robot reference frame can be descrbed by the followng equaton 0 G t t R TR T G. (6) 1 1 The both consdered robots are constructed n such a way that the z-axs of the tool reference frame S G s always parallel to the z-axs of S R. Hence, f poston 0 s selected so that the x-axs of S G s parallel to the x-axs of S R, then R R G s the dentty matrx and (6) can be wrtten as t 0 R t t 0 t. G R R G R onsequently, vector t R can be computed by t t 0 R t t 0. (7) G R R R G The eye-to-hand calbraton s performed n the followng steps. Step 1. Poston the robot tool at an arbtrary poston R t G(0) on the x-axs of the robot RF. Step 2. apture an mage and measure t (0). Step 3. ove the robot tool n the x-drecton for dstance a. The resultng poston s R t G(1). Step 4. apture an mage and measure t (1) usng the trackng tool. Step 5. ove the robot tool n the y-drecton for dstance b. The resultng poston s R t G(2). Step 6. apture an mage and measure t (2) usng the trackng tool. Step 7. ompute R R usng equatons (1) (5) and t R by (7).. Vsual Servong The vsual servong algorthm presented n ths secton computes changes of jonts varables requred to move the marker n a wanted poston based on the mage acqured by the camera, a desred marker poston t r w.r.t. the camera reference frame and the pose of the robot reference frame w.r.t. the camera reference frame T R, obtaned by calbraton. In a practcal applcaton, the desred marker poston t,r s computed from a pose of an object of nterest obtaned by an approprate computer vson software, whch s not a subject of ths paper. The task of the presented vsual servong algorthm s to poston the marker at t,r wthn a gven tolerance. The vsual servong algorthm conssts of the followng steps. Step 1. Based on a gven mage, the current marker poston t w.r.t. the camera s computed usng a trackng tool (n the current mplementaton, ths s ArUco software). Step 2. ompute the current marker poston t the robot reference frame by R t = R T t. Step 3. ompute jont angles q from R t by nverse knematcs. Step 4. ompute the reference marker poston n the robot reference frame by R t r = R T t r. Step 5. ompute new jont angles q r from R t r by nverse knematcs. Step 6. ompute the requred changes n jont angles byq = q r q. Step 7. Send q to the robot controller. These steps are repeated teratvely untl the marker reaches the desred poston wthn a gven tolerance. IV. EXPERIENTAL EVALUATION In order to test the developed system, two test case scenaros were performed. In the frst experment, the accuracy of the camera calbraton was evaluated by comparng the estmaton of the tool poston measured by vson, the tool poston gven to the robot and the actual tool poston measured manually. In the second experment, the goal was to test the accuracy of the mplemented vsual servong algorthm. The experments were performed usng Dobot Arm. Snce the postonng accuracy depends on vsual feedback and not on the mechancal constructon of the robot, smlar results are expected to be obtaned by VIRA. Nevertheless, experments wth VIRA are not ncluded n ths paper, snce ts development s stll n progress. In order to facltate manual measurements, n the frst experment, a laser ponter s mounted nstead of a grppng tool. The manually measured postons are consdered n ths analyss as the ground truth postons. The tool postons gven to the robot are referred to n ths secton as robot postons and the postons measured by

5 TABLE I. x max y max d max x y d STATISTIAL DATA ANALYSIS OF TOOL OORDINATES EASURED BY VISION. BEFORE ORRETION AFTER ORRETION TABLE II. x max y max d max x y d STATISTIAL DATA ANALYSIS OF TOOL POSITIONS GIVEN TO THE ROBOT. BEFORE ORRETION AFTER ORRETION vson are referred to as vson measurements. After camera calbraton, a square matrx of ponts s gven to the robot as reference postons. The robot's task was to move to these ponts, one by one, wat for the vson system to calculate the tool pose and a human operator to measure the tool poston. In order to allow comparson of the measurement obtaned by vson to the open-loop robot postonng and the manual measurements, the pose estmated by the vson system s transformed n the robot reference frame. Snce the robot has a laser as the end effector, the pont projected by the laser on a mllmeter paper allowed an easy measurement n the robot reference frame. After all 100 ponts were processed, a statstcal analyss s performed. The results of ths analyss are shown n the mddle column of Table I and Table II. In Table I, the statstcs of the dfference between the ground truth postons and the vson measurements s shown, where x and max ymax represent the maxmal absolute dfference n the x- and y- coordnates, d represents the maxmal dstance, whle x, y, and max d represent the standard devatons of the x-coordnates, y-coordnates and dstances. The analogous statstcal analyss of the dfference between the ground truth postons and the tool postons gven to the robot s shown n Table II. The presented analyss shows that the calbraton procedure proposed n Secton III.B, although fast and smple, provdes the transformaton T R wth sgnfcant naccuracy. We notced that both the measurements obtaned by vson and the robot open-loop postonng have sgnfcant bas, whch could possbly be reduced by an addtonal calbraton step. In order to estmate to whch extent the consdered postonng errors represent random nose and to whch extent these errors contan a bas, whch could be reduced by an addtonal correcton step or a more thorough calbraton procedure, we performed the followng analyss. We computed values a x, b x, a y and b y whch mnmze the followng error functons a, b x x 2, x x x r, a, b y 2, y, y y y r where x r, and y r, are reference values obtaned by manual measurement, whle and are values obtaned by lnear mappng of the measured values x and y defned by x axx b, x y ay y b. (8) y x The statstcs of the vson error and robot error for the values obtaned by mappng (8) s shown n the last column of Table I and Table II. Ths analyss ndcates that the postonng error can be sgnfcantly reduced by an addtonal correcton step, such as the lnear mappng proposed n ths secton. If the robot s requred to have a good postonng accuracy wthn an approxmately planar workspace, then the presented lnear mappng can be used as the fnal calbraton step, resultng n a more accurate postonng. If the robot s requred to have accurate postonng n a wder workspace ncludng sgnfcant changes n z-drecton, then a more thorough calbraton procedure should be appled. In the second experment, vsual servong accuracy was evaluated. A marker whch represented an object on the scene, was put n 30 dfferent postons, one by one, n the robot s workng regon. Ths marker s referred to n ths paper as target marker. After capturng an mage, the center of the target marker representng the desred poston was determned. The robot ntally moved to ths poston by settng ts jonts to the values computed by nverse knematcs. If the dfference between the robot s poston and the target marker after ths movement was greater than a gven threshold, whch was 3 mm n ths case, the vsual servong algorthm descrbed n Secton III. was appled. Ths process s repeated untl the postonng error falls below the gven threshold. After the robot stopped movng, the dstance between the laser pont on the mllmeter paper, whch represents the poston of the end effector, and the center of the target marker was measured by the human operator. The results of the y

6 TABLE III. STATISTIAL DATA ANALYSIS OF DISTANES BETWEEN TARGET AND OBTAINED POSITIONS BASED ON VISUAL SERVOING ALGORITH d max d descrbed experment are gven n Table III, where d max represents the maxmum dstance between the center of the target marker and the laser poston and represents the standard devaton of ths dstance. d V. ONLUSION AND FUTURE WORK In ths paper, a smple vson guded robot manpulaton system consstng of a low cost robot arm and an off-the-shelf RGB-D camera s presented. As the man contrbuton, we emphasze the development of a low-cost soluton whch can be used by a wde populaton of developers. Furthermore, an analyss of the postonng accuracy of the developed system acheved by vsual servong wthout proproceptve sensors s presented. From the results of ths analyss t can be concluded that the postonng error s wthn 6 mm wth standard devaton of 1.42 mm. The man error source n the proposed postonng system comes from naccurate determnaton of the marker dstance w.r.t. the camera reference frame. We expect that a sgnfcant mprovement to the gven system can be acheved by fusng the measurement obtaned by ArUco software wth depth nformaton provded by a RGB-D camera. Ths wll be the topc of our future research. AKNOWLEDGENT Ths work has been fully supported by the roatan Scence Foundaton under the project number IP REFERENES [1] S. van Delden and F. Hardy, Robotc Eye-n-hand albraton n an Uncalbrated Envronment. [2] K. H. Strobl and G. Hrznger, Optmal hand-eye calbraton, n Intellgent Robots and Systems, 2006 IEEE/RSJ Internatonal onference on, 2006, pp [3] V. Lppello, B. Sclano, and L. Vllan, Eye-n-hand/eye-tohand mult-camera vsual servong, n Decson and ontrol, 2005 and 2005 European ontrol onference. D-E th IEEE onference on, 2005, pp [4] J. Heller, D. Henron, and T. Pajdla, Hand-eye and robot-world calbraton by global polynomal optmzaton, n 2014 IEEE Internatonal onference on Robotcs and Automaton (IRA), 2014, pp [5] A. Aldoma, F. Tombar F, L. D Stefano and. Vncze, "A global hypothess verfcaton method for 3d object recognton," European onference on omputer Vson (EV), [6]. Papazov and D. Burschka, "An Effcent RANSA for 3D Object Recognton n Nosy and Occluded Scenes," Asan onference on omputer Vson (AV), Part I, 2010, pp [7] S. Hnterstosser,. agnart, S. Ilc, P. Sturm, N. Navab, P. Fua and V. Lepett, "Gradent Response aps for Real-Tme Detecton of Textureless Objects," IEEE Transactons on Pattern Analyss and achne Intellgence, vol. 34, no. 5, 2012, pp [8]. A. ueller, K. Pathak and A. Brk, "Object Shape ategorzaton n RGBD Images usng Herarchcal Graph onstellaton odels based on Unsupervsedly Learned Shape Parts descrbed by a Set of Shape Specfcty Levels," IEEE/RSJ Internatonal onference on Intellgent Robots and Systems (IROS), hcago, IL, USA, 2014, pp [9] R. Detry,. H. Ek,. adry and D. Kragć, "Learnng a Dctonary of Prototypcal Grasp-predctng Parts from Graspng Experence," n 2014 IEEE Internatonal onference on Robotcs and Automaton (IRA), 2014, pp [10] L. Twardon and H. Rtter, Interacton sklls for a coat-check robot: dentfyng and handlng the boundary components of clothes, IEEE Internatonal onference on Robotcs and Automaton (IRA), 2015, pp [11] ArUco: a mnmal lbrary for Augmented Realty applcatons based on OpenV, node/26, February [12] A. Kaehler and G. Bradsk, Learnng OpenV 3, O'Relly eda, December [13] SketchUp, arch [14] OpenNI, The standard framework for 3D sensorng, arch [15] open-dobot, arch 2017

SLAM Summer School 2006 Practical 2: SLAM using Monocular Vision

SLAM Summer School 2006 Practical 2: SLAM using Monocular Vision SLAM Summer School 2006 Practcal 2: SLAM usng Monocular Vson Javer Cvera, Unversty of Zaragoza Andrew J. Davson, Imperal College London J.M.M Montel, Unversty of Zaragoza. josemar@unzar.es, jcvera@unzar.es,