Real-time Scalable 6DOF Pose Estimation for Textureless Objects

Transcription

1 Real-time Scalable 6DOF Pose Estimation for Textureless Objects Zhe Cao 1, Yaser Sheikh 1, Natasha Kholgade Banerjee 2 1 Robotics Institute, Carnegie Mellon University, PA, USA 2 Department of Computer Science, Clarkson University, NY, USA 1

2 Object Pose Estimation for Robotic Manipulation Object detection is not enough 3D object pose estimation Manipulation task: robot image from Toyota America Research Center 2

3 Real-time GPU-based Pose Estimation for Textureless Objects Moving Camera RGB Stream Object Pose Estimation Result 3

4 4 Related Work Feature-based Pose Estimation Template-based Pose Estimation RGBD-based Pose Estimation Collet et al., 2011 Hinterstoisser et al., 2010 Song et al., 2014 Xie et al., 2013 Choi et al., 2012 Hinterstoisser et al., 2013

5 Model-based Object Pose Estimation for Textureless Object Camera Frame 3D Model Pose Estimation Result 5

6 Challenges in Model-based Method Viewpoint Variance Camera Frame Scale Variance Illumination 3D Model Variance 6

7 GPU-based Exhaustive Search for Viewpoint and Scale Scale Viewpoint 3D Model Camera Frame Rendered Image (Template) 7

8 Transformation Function for Illumination Robustness Transformation function: Scale f( ) =(f mvnorm f LoG )( ) Viewpoint Where, f mvnorm ( ) is the mean-variance normalization, f LoG ( ) is the Laplacian of Transformed Image 3D Model Guassian (LoG) transformation 8 Transformed Templates

9 Normalized Cross-correlation (NCC) CPU-based NCC [1] Sequential matching GPU-based NCC [2] Parallel over pixel Our Vectorized-NCC Parallel over templates Easy but slow Does not fully utilize the modern GPU Fastest Jp [1] Lewis J P. Fast normalized cross-correlation[c] Visionbinterface, [2] Babenko P, Shah M. MinGPU: a minimum GPU library for computer vision[j]. 2008

10 Template Matrix Construction Rendered Templates Normalized LoG Feature Vectorized Template Matrix T T 0 = t 0 1 t 0 2 t 0 n Viewpoint ti T 3D Model T 10

11 Image Patch Matrix Construction Image Pyramid Normalized LoG Feature Vectorized Image Patch Matrix Scale 1 P 0 = p 0 1 p 0 2 p 0 m P pi 11

12 Image Patch Matrix Construction Image Pyramid Normalized LoG Feature Vectorized Image Patch Matrix Scale 1 P 0 = p 0 1 p 0 2 p 0 m Scale 2 P Scale n pi 12

13 Vectorized Normalized Cross-correlation (VNCC) Score matrix S = Template matrix T x Image matrix P i i j j By reshaping the template set and the image, we reformulate large-scale template matching as one matrix product Our VNCC is 20 times faster than previous GPU-based NCC

14 Vectorized Normalized Cross-correlation (VNCC) Score matrix S = Template matrix T x Image matrix P i i j j Cross-correlation value between ti and pj Template ti Image patch pj

15 SVD-based Dimensionality Reduction Score matrix S = Template matrix T x Image matrix P i i j j To further speed up the computation, we perform SVD decomposition on template matrix: T = U * D * V T = A * Z Weights Bases 15

16 SVD-based Dimensionality Reduction Score matrix S = Template matrix T x Image matrix P i i j j To further speed up the computation, we perform SVD decomposition on template matrix: T = U * D * V T = A * Z Decrease the runtime by 25%! Weights Bases 16

17 RGB-based Pose Estimation Results computationally expensive 17

18 Real-time RGB-based Object Pose Estimation Response map of matched template over the image Matched template Detected object contours (multiple hypotheses) select pose hypotheses number

19 RGB-D Object Scale Prior Imposed object scale prior Camera projection based on depth value 19

20 Multiple Object Pose Estimation S = Template X Image Matrix Matrix

21 Multiple Object Pose Estimation Image Matrix 21

22 Multiple Object Pose Estimation Template Matrix 22

23 Multiple Object Pose Estimation S = Template X Image Matrix Matrix 23

24 Multiple Object Pose Estimation S = A X Z X Image Matrix Principal Component Analysis 24

25 Multiple Object Pose Estimation S = A X Z X Image Matrix Principal Component Analysis 25

26 RGB-D Object Pose Estimation Results Eggbox Duck toy 26

27 RGB-D Object Pose Estimation Color stream for the teapot and sugar bag Real-time 3D model alignment result

28 Application on Real Robot 28 Robot in Toyota America Research Center

29 Runtime for Different Number of Objects Runtime (ms) # objects VNCC-PCA VNCC DDT-3D [1] Linemod [2] one VNCC-PCA VNCC DDT-3D two five ten Runtime (ms) fifteen [1] Rios-Cabrera et al. Discriminatively trained templates for 3d object detection: A real time scalable approach ICCV [2] Hinterstoisser et al. Model based training, detection and pose estimation of texture-less 3d objects in heavily cluttered scenes ACCV Number of objects Sub-linear increase 29

30 3D Mesh Model Dataset for Evaluation 10 textureless models 6 textured objects 13 models are created from Autodesk 123D Catch 3 models are from online repository 30

31 Accuracy comparison Average error in our dataset Pitch Roll Yaw X Y Runtime VNCC ms Line2D-5 [5] ms VNCC-PCA ms Accuracy in ACCV12 dataset Method DDT-3d [1] Hintersoisser [2] VNCC- PCA-10 VNCC-10 VNCC-1 Linemod[3] Drost [4] Accuracy 97.2% 96.6% 96.0% 96.2% 84.2% 83.0% 79.3% 31

32 Future Work Patch-based Matching Non-rigid pose estimation: deformable objects and articulated objects Object tracking based on particle filtering 32