Polygonal representation of 3D urban terrain point-cloud data part I Borislav Karaivanov The research is supported by ARO MURI 23 February 2011 USC 1
Description of problem Assumptions: unstructured 3D point cloud with unknown noise Segment: extract curves (~1D subsets) extract surfaces (~2D subsets) extract 3D regions residual is noise Represent: approximate the subsets by polygonal curves and surfaces Compress: encode the boundaries Reconstruct: for use in other applications: simulation, line of site, path planning, autonomous navigation. for visualization. 23 February 2011 USC 2
segment storage compress visualize represent 23 February 2011 USC 3
Motivation Local picture is uncertain in presence of noise. Curves, surfaces, and 3D textures each requires different tools to efficiently represent in 3D broken lines approximate curves well, polyhedra with triangular faces are more convenient for surfaces. Extracted as separate point clouds, each lower dimensional manifold can be processed with the most appropriate technique independently from the rest of the cloud. Instead of modeling singularities, get rid of them. Pay price for putting it back together. Offset by the savings from working in a simpler space. 23 February 2011 USC 4
Mahalanobis distance 23 February 2011 USC 5
Geometric interpretation Mahalanobis distance is closely connected to Principal Component Analysis (PCA). Measures how point is situated relative to a specific collection of points, not to a fixed coordinate system. Consider the ellipsoid with axes along the eigenvectors (principal components) and radii equal to the square roots of the eigenvalues of the covariance matrix. Mahalanobis distance from a point is the Euclidean distance to the center of the ellipsoid divided by the radius of the ellipsoid in direction of the point. 23 February 2011 USC 6
Geometric interpretation 23 February 2011 USC 7
Self-Adjusting Region Growing (SARG) 23 February 2011 USC 8
Selecting an initial set Initial radius is varied for each seed to find initial set with just enough variance. Inappropriate seed should be dismissed. Indicated by: large initial radius in comparison with the point cloud s range; large portion of initial set being included in already extracted regions; small cardinality of the initial set; Improperly small initial radius would result in growth process that is unable to start or is picking up a single scan line. Improperly large initial radius would result in grabbing points from more than one region. 23 February 2011 USC 9
Selecting growth radius Growth radius can be chosen independently of the scale of data since Mahalanobis distance is unit-free. Constant growth radius is suitable for growing 1D regions. It translates into regions growing exponentially in Euclidean terms. Properly decreasing sequence of growth radii is more reliable for planar regions as it can make regions grow linearly in Euclidean terms. Mahalanobius radius needs to be greater than 2.7 for balls to grow. 23 February 2011 USC 10
23 February 2011 USC 11
23 February 2011 USC 12
23 February 2011 USC 13
23 February 2011 USC 14
23 February 2011 USC 15
23 February 2011 USC 16
23 February 2011 USC 17
23 February 2011 USC 18
23 February 2011 USC 19
23 February 2011 USC 20
23 February 2011 USC 21
23 February 2011 USC 22
23 February 2011 USC 23
23 February 2011 USC 24
23 February 2011 USC 25
23 February 2011 USC 26
23 February 2011 USC 27
23 February 2011 USC 28
23 February 2011 USC 29
23 February 2011 USC 30
a street light and a vehicle 23 February 2011 USC 31
Examples of trees Canadian DRDC-Valcartier data collections 23 February 2011 USC 32
Examples of flat regions Canadian DRDC-Valcartier data collections 23 February 2011 USC 33
Examples of power poles and wires Canadian DRDC-Valcartier data collections 23 February 2011 USC 34
Comparison to other methods Local PCA methods: only locals can vote on matters happening on their block. $$ very difficult to piece together the big picture. Tensor voting: everybody votes on everything (gets very expensive). $$$$ digs out perfect shapes buried in dominating noise. SARG algorithm: everybody votes once for local representatives which then vote on the specific issues. $$ gets the big picture, leaves out the noise. 23 February 2011 USC 35
Polygonal representation Build compressed representation of the regions generated by the SARG algorithm using only a small number of simple objects: line segments polygons closed 3D polyhedra of standard shape. 23 February 2011 USC 36
Linear regions power lines Found in communication and electric power transmission infrastructure (antennas, power lines, poles, pipes); characteristic parts of equipment and fortifications in military settings (artillery, boom barriers, barbed wire fences). Recognizable through PCA as having exactly two eigenvalues close to zero. Represented as either line segment, by its endpoints, or cylinder with radius derived from relative sizes of its two smaller eigenvalues. cannon 23 February 2011 USC 37
Planar regions Prevailing ingredient in point clouds obtained from lidar scans; points from solid objects are near their surfaces. Recognizable through PCA as having exactly one eigenvalue close to zero. Represented as a polygon, need not be convex or simply connected; described by boundary curves or triangulated. 23 February 2011 USC 38
3D regions Typical for vegetation (trees, bushes) due to the ragged texture and slight movement of the foliage; in human made structures - hurricane fences. Recognizable as having 3 large PCA eigenvalues. Visualized as an ellipsoid (or parallelepiped) with semiaxes in the same ratios as square roots of PCA eigenvalues and the same orientations as PCA eigenvectors. Appearance could be improved with textures from imaging. 23 February 2011 USC 39
Polygonizing in a nutshell polygonization Transform points to 2D via projection, 1D translation, and 3D rotation. Pixelize 2D point cloud. Clean using morphological operations. Extract boundary curves. Thin boundary curves w.r.t. Hausdorff metric. Inversely transform boundary curves to 3D. 23 February 2011 USC 40
Transformation transformations Orthogonal projection onto the best fitting plane. 1D translation parallel to the best fitting plane s normal. 3D rotation aligning region s principal components with coordinate axes, the best fitting plane's normal becoming vertical. Polygonization is effectively reduced to 2D. 23 February 2011 USC 41
Pixelization custom-made pixel grid Advantageous to employ the existing image processing tools. Rectangular grid of certain cell size and orientation is overlaid on 2D data points. Occupied pixels are made white, the rest are left black. Size and orientation of pixels is crucial for faithfully capturing the shape of region. Pixel sides are always oriented along the principal components of the 2D point cloud. 23 February 2011 USC 42
Pixelization parameters Arbitrary, heterogeneously distributed point cloud can be handled. Regularities are sought which, if found, are exploited to improve results considerably. Formation of approximately parallel scan lines with approximately constant densities along and across. Formation of diverging scan lines with common center and approximately the same angular spacing. Results deteriorate gracefully as point cloud departs from the stated conditions. 23 February 2011 USC 43
Pixelization parameters Delaunay triangulation Build Delaunay triangulation of the reduced 2D point cloud. Distance between scan lines as a dominant value among the longest altitudes of all triangles. Distance between points along a scan line as a dominant value among the shortest sides of all triangles. Slope of scan lines as dominant value among slopes of individual scan lines grown with SARG. 23 February 2011 USC 44
Morphological operations Pixelized image typically has many imperfections. Morphological closing fills in the small holes due to distortion or lack of data, or underestimated pixel size. pixelized Morphological opening removes the outliers and whiskers due to noise or grabbing points from other regions. closing Larger holes and protrusions are left mostly unchanged. opening 23 February 2011 USC 45
Boundary curves extraction xor = cleaned eroded boundary pixels Boundary pixels are found by logical exclusive-or on the cleaned and eroded images. Pixelized curves are tracked one at a time until boundary pixels are used up. Tracking moves from a boundary pixel to a not visited boundary neighbor in a prescribed fashion. Boundary curves are formed by connecting centers of pixels. boundary curves 23 February 2011 USC 46
Adaptive thinning adaptive thinning raw boundary thinned boundary Iteratively removes the least significant node. Significance is defined as Hausdorff distance between the current curve and the one obtained by removing the node and placing an edge between its two neighbors. Fast implementation using priority queue. Controlled by allowed error or complexity of output. 23 February 2011 USC 47
Encoding 1D region is encoded by two endpoints of a line segment (6 d.o.f.). optionally, adding a radius of a cylinder (1 d.o.f.). 2D region is encoded by vertices of its planar curves (2 d.o.f. per vetex). 3D rotation (3 d.o.f.). 1D translation (1 d.o.f.). 3D region is encoded by center of mass (3 d.o.f.). orientations of its three principal components (3 d.o.f.). lengths of its three principal components (3 d.o.f.). 23 February 2011 USC 48
Example of compression Data type number d.o.f. total 3D points 12,188 3 36,564 Total d.o.f. 36,564 Model type number d.o.f. total Raw compression ratio ~ 230:1 linear regions 0 6 0 planar regions 7 3 + 1 28 vertices 64 2 128 Total d.o.f. 156 23 February 2011 USC 49
Example of compression Data type number d.o.f. total 3D points 25,698 3 77,094 Total d.o.f. 77,094 Model type number d.o.f. total Raw compression ratio ~ 803:1 linear regions 0 6 0 planar regions 6 3 + 1 24 vertices 36 2 72 Total d.o.f. 96 23 February 2011 USC 50
Acknowledgments Presenter is very thankful to Peter Binev, Karl-Heinz Brakhage, Matthew Hielsberg, Scott Johnson, Luke Owens, and Robert Sharpley for the useful discussions, supply of challenging data sets, and opportunity to work on this exciting problem. All simulated point-cloud data sets used in this presentation are generated by Matthew Hielsberg using his lidar simulator developed at USC. 23 February 2011 USC 51
Thank you 23 February 2011 USC 52