Scenario/Motivation. Introduction System Overview Surface Estimation Geometric Category and Model Applications Conclusions

Transcription

1 Single View Categorization and Modelling Nico Blodow, Zoltan-Csaba Marton, Dejan Pangercic, Michael Beetz Intelligent Autonomous Systems Group Technische Universität München IROS 2010 Workshop on Defining and Solving Realistic Perception Problems in Personal Robotics Taipei, October 2010

2 Scenario/Motivation! # #! % # # % #! &

3 Motivation Scenario: our robot is operating in a household environment, we have tables with objects on them, we want to reason about their arrangements, and perform manipulation. Problem: classify the clusters to find out their semantic meaning for reasoning, reconstruct the clusters to be able to grasp them. Solution: RSD, GRSD, SVM, Model Reconstruction

4 System Overview 1 1, ( ) + #! % & % 222 / 0..

5 System Overview 1 1, ( ) + #! % & % 222 / 0..

6 System Overview!

7 Segmentation and Smoothing of Data Detect tables and segment cluster in Euclidean sense Smooth data and compute accurate surface normals using Moving Least Squares fit of polynomial surfaces

8 Radius-based Surface Descriptor (RSD) Local variation of normal angles by distance: Synthetic Data plane sphere sphere side corner cylinder cylinder top cylinder side edge handle Real Data small cylinder medium cylinder big cylinder handle1 handle2 handle3 Estimate minimum and maximum curvature radius from minimum and maximum angle/distance pairs: d (α) = 2r 1 cos(α) d (α) = rα + rα O(α5 ) d = rα

9 Radius-based Surface Descriptor (RSD) Voxelization of space and simple feature estimation to reduce computational complexity Since the minimum and maximum radius have physical meanings, simple intuitive rule-based local surface classification is possible [IROS2010:] Marton et al., General 3D Modelling of Novel Objects from a Single View

10 Global Radius-based Surface Descriptor A global feature is computed from each cluster by counting transitions between surface types Use geometric class to decide on what model to fit Each view is considered a training example, but only object category is checked Total time is around 0.2 seconds, from raw clusters to geometric category [Humanoids2010:] Marton, Pangercic, Hierarchical Object Geometric Categorization and Appearance Classification for Mobile Manipulation

11 Geometric Categorization using GRSD Categories are hand-picked for now, automatic grouping would be preferred The object categories of previously unseen views was successfully found in 89% of the time Each category has a best fitting geometric model whose parameters will be estimated

12 Model Reconstruction (Box, Cylinder) Box fitting: we fit a rectangular model directly to the points having normals perpendicular to the up axis (as we assume boxes to be standing on one of their sides). Cylinder fitting: we use a SaC approach which is based on the observation that on a cylinder surface, all normals are orthogonal to the cylinder axis, and intersect it. We consider the two lines defined by two sample points and their corresponding normals as two skew lines, and the shortest connecting line segment as the axis. Determining the radius is then a matter of computing the distance of one of the sample points to the axis.

13 Model Reconstruction (Box, Cylinder)

14 Model Reconstruction (Arbitrary Rot.-Sym.) Non-linear minimization using Levenberg-Marquardt for axis, and least squares fit for contour m d l,l ( a, a, p i, n i ) 2 i=0 Needs checking of the generated surface for plausibility (parts only in occupied and occluded space) as in Blodow et. al. [Humanoids2009]

15 Short Demonstration roslaunch cloud_algos rotational_estimation_triangulation.launch

16 Grasping of Unmodelled Objects Estimating surface type and reconstruction using symmetries *

17 Interpretations of Scenes Missing Objects )! * +! # #! % # # % #! & ' ( perceivedobjectsonplane(plane, Perceived) :- onplane(plane), setof(obj-hyp, ( on(obj, Plane), category(obj,cat), uniqueid(id), objectinstace(obj,knownobj), Obj-Hyp = [Id,Obj,Cat,KnownObj]), Perceived). [IROS2010:] Pangercic et al., Combining Perception and Knowledge Processing for Everyday Manipulation

18 Pros and Cons Advantages automatic model reconstruction for previously unseen objects more accurate and robust grasping through completed geometric model potential speedup of visual classification based on geometrical info connection to high-level reasoning Constraints segmentable horizontal supporting plane physically separated objects apriori trained geometric categories objects with planar and rotational symmetry (plus handles)

19 Discussion Thanks! Intelligent Autonomus Systems Group: TUM ROS Package Repository: tum-ros-pkg/mapping/ Contact: