METHODS FOR MOBILE MANIPULATORS

Transcription

1 COMPARISON OF REGISTRATION METHODS FOR MOBILE MANIPULATORS ROGER BOSTELMAN 1,2, ROGER EASTMAN 3, TSAI HONG 1, OMAR ABOUL ENEIN 1, STEVEN LEGOWIK 4, SEBTI FOUFOU 5 1 National Institute of Standards and Technology, Engineering Laboratory, Intelligent Systems Division, Gaithersburg, MD, USA 2 Le2i Lab, Université de Bourgogne, BP 47870, Dijon, France 3 Loyola University Maryland, Baltimore, MD, USA 4 Robotic Research, LLC, Gaithersburg, MD, USA 5 CSE Dept, College of Engineering, Qatar University, Qatar.

2 Introduction Past registration research Registration Methods Visual Fiducials Search (Fine and Bisect) Experiments and Results Conclusions Outline

3 Introduction Mobile Manipulators Current mobile manipulators application examples: 1. unloading trucks 2. bagged-goods (e.g., dog food bags) handling 3. conveyer loading/unloading 4. picking canned and boxed goods from shelves in supermarkets 5. delivering, placing, and manipulating semiconductor wafer pods within wafer fabrication facilities 1-4 applications require loose tolerance on registration (e.g., vacuum grip, etc.) 5 and other assembly-type operations, (e.g., peg-in-hole), require much tighter tolerances on positioning from the mobile manipulator

4 Introduction NIST Program, Definitions NIST Robotic Systems for Smart Manufacturing Program currently researching automatic guided vehicle (AGV) (soon, mobile robot)/mobile manipulator performance and vision performance standards develops and deploys advances in measurement science by improving performances of robotic systems to achieve dynamic production for assembly-centric manufacturing. Registration = process of measuring and mapping the feedback from one system (e.g., mobile manipulator) to the model of another (e.g., artifact), correcting for differences in resolution, scale, direction, and timing Calibration = instrument (e.g., camera) adjustment or output correlation of the instrument readings with its known accuracy. These two terms are sometimes interchanged in the literature.

5 Introduction Ex. Research Registration Methods Quick Reference (QR) codes [1] combined with calibrated vision [2] - tracking error: under 20 mm, maximum errors: 45 mm at the largest camera-target distance. 1 - R. Andersen, O. Madsen, T. Moeslund, Hand-Eye of Depth Cameras based on Planar Surfaces, Abstract from Int l Workshop on Intell. Robot Assistants, Padova, Italy, B. Hamner, S. Koterba, J. Shi, R. Simmons, S. Singh, An autonomous mobile manipulator for assembly tasks Auton. Robot 28: (2010). QR codes for mobile robot registration and end effector error [3] - maximum positional repeatability: 1.1 mm (one point) to 4.0 mm (multiple points). 3 - R. Andersen, J. Damgaard, O. Madsen, T. Moeslund, "Fast calibration of industrial mobile robots to workstations using QR codes." 44th Int l Symp. Robotics (ISR), pp IEEE, High-precision calibration - average errors based on the Tsai hand-eye calibration combined with a high-speed calibration - average errors: ± 0.1 mm and ± based on a combination of laser triangulation and image processing [4]. 4 - M. Hvilshøj, et al., "Calibration techniques for industrial mobile manipulators: Theoretical configurations and best practices." Robotics (ISR), st International Symposium on and th German Conference on Robotics (ROBOTIK). VDE, Constrained manipulator endpoint to a single contact point while executing manipulator motion where manipulator joint angles are read to develop a calibration model [5]. 5 - M. Meggiolaro, G. Scriffignano, S. Dubowsky, Manipulator Calibration Using A Single Endpoint Contact Constraint, Proc. of DETC2000: 2000 ASME Design Engineering Technical Conference, DETC2000/MECH-14129, Baltimore, MD, September Touch probing using peg-in-hole and particle filter solutions [6, 7]. 6 - S. Chhatpar, M. Branicky, Localization for Robotic Assemblies Using Probing and Particle Filtering, Proc. of the 2005 IEEE/ASME Int l Conf. on Advanced Intelligent Mechatronics, Monterey, CA, USA, July Y. Taguchi, T. Marks, J. Hershey, Entropy-Based Motion Selection for Touch-Based Registration Using Rao-Blackwellized Particle Filtering, TR , September 2011.

6 NIST-Tested Registration Methods Fine Search and Bisect with Fine Search use a laser retroreflector to determine fiducial location with respect to the mobile manipulator. Visual Fiducials builds upon [3] from Aalborg University - QR codes were used in combination with vision processing. QR codes for mobile robot registration and end effector error [3] - maximum positional repeatability: 1.1 mm (one point) to 4.0 mm (multiple points). 3 - R. Andersen, J. Damgaard, O. Madsen, T. Moeslund, "Fast calibration of industrial mobile robots to workstations using QR codes." 44th Int l Symp. Robotics (ISR), pp IEEE, 2013.

7 Fine Search and Bisect with Fine Search Uses a laser retroreflector detector carried by the manipulator to detect reflective fiducials on the Reconfigurable Mobile Manipulator Apparatus (RMMA) RMMA: metal plate with fiducial mount points at precise locations Fiducial: collimated reflector on a base that attaches to the RMMA CAD model of paths and docking points used by AGV control program to move the vehicle from one docking pose to another near the RMMA. Manipulator program then corrected for vehicle pose allowing it to register with pre-taught targets using the two methods: fine and bisect search Two pairs of 1 mm dia fiducials were positioned at: two corners of a 457 mm square pattern of 4 fiducials at opposing points along a 305 mm diameter circle pattern of 6 fiducials.

8 Performance Measurement Concept Reconfigurable Mobile Manipulator Artifacts RMMA-1 can be reconfigured to be at any reachable height and angle to the mobile manipulator, including overhead RMMA-2 can include flat, concave and convex patterns RMMAs include square, circle, triangle, straight and curved lines, and sinusoidal geometric patterns of tapped holes drilled into machined plates with tolerance of +/ mm. 8

9 Performance Measurement using Artifacts Performance evaluation criteria includes: Time and pass/fail to register the mobile manipulator to the artifact Time and pass/fail to move from the registration points to the assembly points Repeatability after registration Number of search steps equating to the initial distance from registration/assembly points Detection of reflectors with varying diameters Target application = assembly (e.g., peg-in-hole) However, initially perform the measurement without contacting the artifact Align to the fiducials using a laser

10 Performance Measurement using Artifacts Bisect Method with Fine Search 10

11 Results: Fine Search and Bisect with Fine Search Only the first 6 vehicle poses were completed for the fine search method due to the long registration time Square Search spiral pattern with 0.5 mm steps to detect 1 mm and 3 mm fiducials The 3 mm fiducial had the highest average number of search steps at 869 with 1921 maximum steps and 1740 s causing an average search time of 360 s with a maximum of 893 s. The root-mean-square deviation (RMSD) from the mean was 776 steps (403 s)

12 Results: Bisect with Fine Search Table 1: Mobile manipulator registering to the RMMA using the bisect search method. AGV Position Number Pose Angle Pattern Average num. of search steps to register Total bisect + fine search time to register (s) 1 90ᵒ circle ᵒ square ᵒ circle ᵒ square ᵒ circle ᵒ square ᵒ circle ᵒ square ᵒ circle ᵒ square Mean Search Steps/Time (s) 1.8 / 0.8 RMSD Search Steps/Time (s) 3.8 / 1.8 nearly 90% less time

13 Registration with Visual Fiducials allows for six degrees-of-freedom (6DOF) positional tracking of fiducial targets, systems are well-developed, and can be implemented with open source software, inexpensive cameras, and virtually free printed targets, number of advantages for use in robotics research and testing procedures for industrial robot evaluation and validation. We reviewed augmented reality systems: ARTag, April tags, ARToolKit, and ALVAR (A software Library for creating Virtual and Augmented Reality) Used ALVAR because it integrates with Robot Operating System (ROS). allowed the use of ROS preprocessing, visualization, and message-passing facilities. ALVAR uses similar rectangular black and white targets with a black outer square for location, and an internal matrix of squares that codes the identity of the target. Other fiducial targets used include standard camera calibration targets (i.e., checkerboards), QR codes, and application-specific targets.

14 Registration with Visual Fiducials ALVAR et al. advantages - flexibility and cost over other 6DOF tracking systems that may require more expensive and extensive installations. For ALVAR use in testing procedures and standards, we wished to understand system: robustness, working range and orientations, response time accuracy, These don t have universal solutions since the system performance depends on implementation details. robustness depends on occlusion and camera details; working range depends on target size, camera resolution, and camera focal length; response time depends on camera frame rate, resolution, and computer processing unit speed; ultimate accuracy depends on all these factors, including target motion speed relative to frame rate.

15 Experiment 1: Visual Fiducials Two sets of experiments using the ALVAR implementation 17 mm machine vision camera resolution of 1296 pixels x 964 pixels fixed 4.5 mm focal length lens Experiment 1: static repeatability of the ALVAR system when the 200 mm x 200 mm target was moved to static positions by a pan-tilt mechanism. allowed to settle before static measurements camera to - target separation distance 800 mm to 1000 mm moved to 26 positions of differing tilt and pan For each position, 306 measurements were taken over 30 s

16 Experiment 1 Results: Visual Fiducials Repeatable measurements indicate systemic biases can be corrected by calibration Maximum difference from the mean in any one position in any dimension was 0.8 mm (along the Z-axis) Maximum angular error (in angle axis representation) was 0.18ᵒ From initial results, we judge that the basic capabilities of ALVAR are adequate as a subsystem in workpiece registration. Expect that target detection (i.e., registration) can occur near camera polling speeds (e.g., 120 Hz = 1/120 s = 8 ms x # samples)

17 Experiment and Results: Visual Fiducials Experiment 2: Integrated ALVAR with the systems on an AGV docking with the RMMA Spacing between the RMMA square and circle patterns was 508 mm. Camera onboard the AGV Repeatedly measured the AGV positioning at the square and circle patterns and consistently communicated the position to the robot controller) Since writing the paper Camera onboard robot arm

18 Recent ALVAR Experiments and Results 1. Intrinsic Lens Barrel Distortion, or Fish-eye effect 2. Camera to End Effector Offset Camera? Laser? P P 3. Camera Measurement Error Calib_Camera Calibration Program Included with ARToolkit SDK.

19 Recent ALVAR Experiments and Results ARToolkit Camera Offset Calibration Ground Truth: Manipulator X and Y position, as measured by the robot controller. Laser centered on the marker origin. Parameters: Measurement Range: ±180⁰ Increments: 15⁰ Sample Size: 100 X 2 R 1 Y 1 15⁰ Y 2 R 2 X 1 Offset Result: mm

20 Recent ALVAR Experiments and Results ARToolkit Camera Error Calibration Ground Truth: Change in manipulator position. (± 0.2 mm) Camera centered on marker origin. Marker rotationally aligned with laser. Laser Camera Parameters: Measurement Range: ±100 mm Increments: 10 mm Samples: 100

21 ARToolkit Absolute Error (mm) Recent ALToolkit Results ARToolkit: Average Absolute Error vs. Camera Y Position Camera Y Position Relative to Marker (mm) Standard Deviation for each measurement less than 0.03 mm

22 Conclusions Visual Fiducials Experimental results for visual fiducials are consistent with the various registration methods from the literature Under optimal conditions, we estimated repeatability of a visual fiducial at under 1 mm and 0.2ᵒ from a single image. Expect that basic capabilities of ALVAR are adequate as a subsystem in workpiece registration The second ALVAR experiment provided successful integration with the mobile manipulator. Given other elements in the system, including calibration of camera-to-base, and base-to-arm, and the propagation of error, we would expect total error for the system to be higher. Most recent experiments: linear error progression of ~ 0.3 mm / 10 mm away from target center Fine search and Bisect with Fine Search Fine search = high number of search steps and time (average steps: 776, average/maximum time: 360 s/893 s) to register Bisect method prior to the fine search, = maximum of 184 steps/86 s or nearly 90% less time than using only the fine search method. Larger bisect search steps, among many other improvements, could be used although would increase the number of registration search steps on the 1 mm or 2 mm fiducials to a potentially unknown amount. Future registration tests will combine the visual fiducial with the search methods to minimize the search time.

23 Misc.

24 ARToolkit Pose Server Software Development Custom program used to implement and assess ARToolkit marker position and orientation (pose) tracking measurements.