Multi Sensor Fusion in Robot Assembly Using Particle Filters Ulrike Thomas, Sven Molkenstruck, René Iser, and Friedrich M. Wahl Technical University of Braunschweig Germany - 1/15 -
Contents Introduction / Motivation Force Torque Maps: Definition Automatic computation from CAD data Use during assembly Computer Vision Sensor Fusion using Particle Filter Experiments Conclusion / Outlook - /15 -
Introduction / Motivation Robot Assembly should be Automatically planned and executed CAD data as the only input Tolerant to variations during execution Sizes of parts Poses of parts Both together is difficult. We suggest an approach towards these goals, based on Force Torque Maps - 3/15 -
Workcell Setup basis T hand known sensor T hand hand sensor camera robot basis dof uncertain Δ hand Grasp T task worldt camera calibrated 6 dof uncertain taskt object camera T object uncertain world T object uncertain object 3d pose uncertain z y x world - /15 -
Force Torque Maps: Introduction Given: Robot arm with Force Torque Sensor Assembly Task (e.g. Shaft-Fits-Hole) Formal definition of FT-Maps: r r f : ω s r ω Ω Tolerance space r s S Sensor space Force Torque Maps map each vector of uncertain parameters (e.g. relative object pose R ) to a vector of expected sensor values (e.g. torques (t x,t y ) R ) - 5/15 -
Force Torque Maps: Example Resolution Res. 1 Res. Res. 3 Res. [mm, ] [1,1] [.5,.5] [.,.] [.1,.] No. of configurations 35 35 71 31 71 1 3 31 Computation time [s] 9 136 13 931-6/15 -
Force Torque Maps: Computation I Distance computation on GPU camera direction A camera direction B object A z-buffer A separating plane object B contact points z-buffer B sum of z-buffer values - 7/15 -
Force Torque Maps: Computation II Physical model f r imposed (z-axis of TF) Computation of moment arm as perpendicular vector between z-axis and convex hull of contact points r r r r instable contact torques occur Usually static friction occurs. compute torques as r τ = r r ( fimposed ) - 8/15 -
Force Torque Maps: Quality Gemessene Karte für Rot (,, ) Gemessene Karte für Rot (,, ) Measured map for Rot(,, ) Measured map for Rot(,, ) torque Drehmoment x-axis X-Achse [Nm] torque Drehmoment x-axis X-Achse [Nm] 1.5 1.5 -.5-1 -1.5 position y-axis Position Y-Achse [mm].5 -.5-1 -1.5 - - Simulierte Karte für Rot (,, ) Simulierte Karte für Rot (,, ) Simulated map for Rot(,, ) Simulated map for Rot(,, ) position y-axis Position Y-Achse [mm] - - position x-axis Position X-Achse [mm] position x-axis Position X-Achse [mm] CAD-Model of assembly task torque y-axis [Nm] Drehmoment Y-Achse [Nm] 1.5 1.5 -.5-1 -1.5 Deviation:.59 Nm Drehmoment Y-Achse [Nm] torque y-axis [Nm] 1.5 1.5 -.5-1 -1.5 - position y-axis Position Y-Achse [mm] position y-axis Position Y-Achse [mm] - - - - position x-axis Position X-Achse [mm] position x-axis Position X-Achse [mm] Measured maps Simulated maps - 9/15 -
Particle Filter Each particle represents a relative pose hypothesis initialization of particles with gaussian distribution move robot with respect to estimated position drift and distribute particles according to weights move robot down into contact s 1 1 σ ² p i = e πσ ( ( h i m ) s, s s ) hole found? no yes estimate position as best fitted particle evaluate all particles (based on forces, torques, and vision) measure forces and torques in contact - 1/15 -
Computer Vision Here: Edge detection of left and right edges, one camera only Estimation of relative image parameters (angle, distance) comparison to particle evaluation of particle Uses only 3d-to-d calculation no d-to-3d ambiguity Future: Automatic feature selection α im age particle - 11/15 -
Experiments I 1. Initialisation. Measurement and Evaluation 3. Particle drift and Robot move (,) = ideal matching pose True relative pose Estimated relative pose. Measurement and Evaluation 5. Robot move finished - 1/15 -
Experiments II Number of contact measurements without vs. with vision sensor (total 1 experiments each) Sensor fusion is reasonable Occurrence (% of experiments) 5 3 1 1 3 5 6 7 8 9 1 Number of necessary contact measurements Force/torque sensor only Occurrence (% of experiments) 5 3 1 1 3 5 6 7 8 9 1 Number of necessary contact measurements Force Torque Sensor and Vision Sensor - 13/15 -
Conclusion / Outlook Towards automated and tolerant execution of robot tasks Automatic computation of FT Maps from CAD data Possible for any object shape Vision (still) manually adopted to task Automatic planning in the future Particle filter allows easy fusion of two or more various sensors Robot stiffness could be considered in FT Map simulation Usefulness to be proved with more complex assembly parts - 1/15 -
Questions, Comments, Discussion - 15/15 -