IMPLEMENTATION OF 3D POINT CLOUDS SEGMENTATION BASED ON PLANE GROWING

Transcription

1 IMPLEMENTATION OF 3D POINT CLOUDS SEGMENTATION BASED ON PLANE GROWING METHOD Małgorzata Jarząbek-Rychard 1 Abstract In the recent years we can observe growng demand for the extracton of physcal objects from 3D spatal nformaton. Segmentaton s the frst step of pont cloud nterpretaton whch further allows object modelng. Ths paper presents an mplementaton of an automatc tool for 3D data segmentaton. The segmentaton algorthm s based on the plane growng approach, whch uses least squares plane estmaton and pont connectvty provded by the k-nearest neghbours search. Two refnement parameters are requred, angle and dstance thresholds. In order to avod a loss of nformaton caused by data resamplng, presented software works on raw 3D data. Keywords Segmentaton, laser scannng, 3D modelng 1 INTRODUCTION In the recent years, we can observe growng demand for the extracton of physcal objects from three-dmensonal spatal nformaton. Laser scannng s consdered a leadng technology for the acquston of such data. As well, magematchng algorthms that work on mages acqured by dgtal cameras, experence a renassance. Both technques allow extractng a dense data set that contans sub-randomly dstrbuted ponts, referred to as pont cloud. Dense pont clouds are hghly desrable; however, they may present a challenge wth regard to the management of huge volumes of data and processng lmtatons on PCs [1]. Pont cloud data sets allow for the analyses of complex constructons by establshng accurate and detaled 3D models. The feld of applcaton of object modelng s very wde (e.g. ndustry, urban plannng or water management) and stll grows as new methods appear and old ones mprove. Usually the modelng s used for the purpose of manmade objects extracton, such as buldngs, brdges or ndustral objects. Due to ncreasng demands for such products, numerous dscplnes deal wth the problem of retrevng explct spatal nformaton from the abundant pont cloud. The capablty of carryng out useful analyses s smplfed for data wth some organzatonal structure. It becomes consderably lmted for unstructured 3D ponts. Therefore, before fnal products can be derved, raw pont clouds requre a great deal of processng. Acqured data feature the lack of structure and mplctness of ncluded nformaton. Therefore, the problem of automatc extracton of the shape of the recorded objects becomes hghly complex. Accountng to both effectveness of extractng nformaton and the constantly ncreasng number of data, there s the necessty to ntroduce some level of organzaton nto a data set []. Such organzaton nvolves data segmentaton that provdes planar faces (for nstance n case of buldng modelng) or specfc geometrc shapes by aggregatng ponts wth smlar attrbutes. Ths process enables afterwards to reconstruct geometrc 3D models. For ths reason, the segmentaton process emerges as one of the most crucal part of automatc pont clouds processng. The mportance of ths task s clearly vsble n the number of segmentaton algorthms developed n the last few years. Snce many segmentaton algorthms have ther orgnaton n range mage analyses, they requre.5d nput data. Because raw data acqured by laser scannng or multple mage matchng feature 3D character, they have to be transformed to a raster. Such a transformaton usually causes loss of nformaton. Therefore, the current research leans to mprove algorthms that are workng on raw 3D data. One of these approaches plane growng based segmentaton s used for the software descrbed n ths paper. SEGMENTATION Segmentaton s the frst step of pont cloud nterpretaton. It allows parttonng of a data n the object space n order to create meanngful, coherent and connected subsets, referred to as segments. Each segment s supposed to nclude ponts or pxels wth smlar attrbutes and represents a surface, an object or a part thereof. The outcome of the segmentaton s the set of extracted segments. 1 Małgorzata Jarząbek-Rychard, M.Sc., Wroclaw Unversty of Envronmental and Lfe Scences, The Faculty of Envronmental Engneerng and Geodesy, Insttute of Geodesy and Geonformatcs, Grunwaldzka 53, Wroclaw, malgorzata.jarzabek-rychard@up.wroc.pl 1

2 Denotng whole data set as R and each segment as R the segmentaton process may be formally descrbed as follows [3] : n 1) =1 R = R, ) R s a connected segment, = 1,,, n, 3) R R j = {} for all and j, j, 4) P ( R ) = TRUE for all = 1,,, n, 5) P ( R R j ) = FALSE for j, where P denotes probablty. The whole data set s decomposed nto segments (1) that are dsjoned (3). Each of them s a connected component n the object space (), whch means that groupng has to take nto consderaton a neghbourhood between the ponts. The predcate (4) defnes the smlarty among the ponts provdng that smlar ponts wll le on the same object plane. Requrement (5) ensures that the ponts belongng to dfferent segments le on separate faces. The features that are consdered whle parttonng a pont cloud may be ether geometrcal or spectral. However, the ntensty data that provded by laser scannng has nosy character. Due to ths fact, ponts are usually grouped wth respect to spatal propertes only..1 Plane growng method A segmentaton method used for the presented mplementaton s based on extracton of planar faces from unstructured data sets, usng the scheme of the plane growng approach. The algorthm s drven by the best-fttng plane estmaton. The am of segmentaton process,.e. smooth surfaces extracton s acheved by groupng neghbourng ponts that share some propertes, such as the drecton of a locally estmated surface normal. In ths work, the processed data s a raw unstructured 3D pont cloud. The assumpton about ths knd of nput makes presented method more general. The presented algorthm conssts of the followng core steps: structurng data usng nearest neghbours search, computng the best-fttng plane, the plane growng process. Ultmately, segment mergng s performed n order to avod over-segmentaton..1.1 Nearest neghbours search wth usage of a k-d trees structure The nearest neghbour (NN) search problem requres preprocessng a data set nto a structure that provdes a quck fndng of the closest pont to a query pont n the metrc space [4]. Ths dea mght be easly extended to searchng for k-nearest neghbours. The prncple of ths approach s based on geometrcal relatons between ponts. In order to optmze search tme performance many optmzaton strateges have been developed n the recent years. One of them s space parttonng [5]. k-d tree s one of space parttonng data structure for storng a fnte set of ponts from a k-dmensonal space. It s a specal knd of a bnary tree - whch s the result of recursve subdvson of the three dmensonal space [4]. Space partton and ts assocated node structure are outlned n Fg. 1. Fg. 1 Two dmensonal k-d tree - space partton and assocated node structure [6]. The dea of the nearest neghbours search algorthm operatng on k-d trees structures conssts on vstng cells successvely, accordng to the ncreasng dstance between them and the consdered pont. The frst met pont s consdered as the nearest neghbour. The dstance between ths pont and the query one determnes the radus of the hyper-sphere centered n the query pont. Consequently, any other pont replacng ntally approxmated nearest neghbour, needs to be closer,.e. wthn the volume enclosed be the hypesphere. Accountng for tme performance, the sequence of pont examnaton s based on parent-chld dependency. The k-nearest neghbour search s the preprocessng step for the normal vectors estmaton.

3 .1. Least squares plane fttng To fully defne a poston of the plane n the 3D space, the coordnates of the ponts and the plane normal have to be known. Therefore, the dervaton of normal vectors s of crucal mportance for the further steps of data segmentaton. Plane normals should be computed and assocated wth each pont of the data set. The calculaton s performed by extracton of the best-fttng plane through the consdered pont and ts nearest neghbours. Ths may be done by a least-squares ft. In terms of fttng the plane n 3D space, multple lnear regressons s appled,.e. estmatng the coeffcents that provde the mnmum sum of squares resduals [7]. For a gven set of pont coordnates, the set of parameters that mnmzes the square sum of the orthogonal dstances of the ponts from the plane has to be found [8]. A plane n 3D space may be formulated subject to Ax + By + Cz + D = 0, (1) Wth the parameters A, B, C defnng the plane normal vector: n = ( A, B, C). () Accordng to the condton of mnmzng the sum of squared dstances between the -th pont P ( x, y, z ) and the plane N! d = 1 = mn, N N d = = 1 = 1 ( Ax + By + Cz + D), (4) a nonlnear least squares problem has to be solved. The problem can be reduced to an egenvalue problem (M λ I)x = 0, where M represents the coeffcents matrx yelded from mnmzaton and I s an dentty matrx. As the egenvalue s the sum of squared resduals, the best-fttng plane s assocated wth the smallest obtaned egenvalue. The ganed set of parameters A, B, C s defned by the correspondng egenvector x..1.3 Plane growng Plane growng ncorporates the prevously computed local surface propertes n order to merge together ponts that are spatally close and have smlar geometrc features. The frst step of ths task s to fnd a seed pont of the surface that has to be grown. For each pont of the data set, the resduals of the fttng plane determned by ths pont and ts nearest neghbors are computed. The pont wth the lowest square sum of resduals s consdered a seed pont and ts parameters are assumed to determne a seed surface. The second step provdes segments based on the former calculaton of normal vectors and dstances from the best-fttng plane. The algorthm mplemented n ths work s affected by two parameters: α and d that respectvely stem from followng geometrcal restrctons: smoothness crteron, realzed by settng a maxmum value for the angle between the normal vectors of the seed plane and the canddate pont (α) proxmty crteron, enforced by settng a threshold on the dstance between the seed plane and the canddate pont (d) (3) (5) α d Fg. Segmentaton parameters α and d; black vector: normal to the seed plane, blue vectors: accepted, red vectors: rejected 3

4 These two requrements ensure the smoothness of the plane [9]. As a result, ponts are grouped nto unbroken segments so that the normal vectors of ponts belongng to the same segment do not vary too much from each other. Fgure presents the dea of the algorthm. The surface growng process starts wth the seed pont and contnuously examnes canddate ponts whether they fulfll or not the crterons mentoned above. Ths s an teratve procedure, whch requres recomputng the surface equaton after every loop of checkng the pont set and addng ponts. Ths updatng enables to extract more precse plane parameters, as more ponts are used for ther calculaton. The procedure s repeated and consequently the surface s growng untl no canddate meetng the crterons s found. Thus, no more pont can be accepted and assgned to the segment. In ths case, a new seed pont has to be found and the surface growng process s carred out agan..1.4 Segments mergng Ths last stage enables to avod over-segmentaton n the fnal result. Startng wth the algorthm executon frst, the near neghbours search s performed, whch returns k neghbours for each query pont. In ths paper k value s set to ten. Afterwards, the best-fttng plane for these ponts s computed and the plane normal s derved. Unfortunately, all of the neghbours do not necessarly belong to the same plane. In such a stuaton, the calculated normal vector devates from the real one. The devaton ncreases when the more ponts do not le on the plane. Also n case of surface roughness, the surface normals do not pont n exactly the same drecton. These two facts affect the plane growng resultng n groupng ponts n some numbers of very close planes nstead of one. After performng each loop of examnng canddate ponts, ther plane parameters (thus the plane normal) are updated. Therefore, splt planes that n realty make up one segment should have almost the same normal vectors (possble varatons are depended on a pre-defned threshold). For ths reason, all segments are checked agan n order to merge smlar segments. The dea of segments mergng s very smlar to the dea of groupng raw ponts durng the stage of plane growng. The dfference s only the nput - nstead of 3D sngular ponts, the already created clusters are examned. Ths s agan done by checkng the smlarty of normal vectors and the orthogonal dstance between two planes. Mergng smlar neghbourng segments yelds denser and larger planes. 3 IMPLEMENTATION 3.1 Implementaton envronment The basc part of the program s coded wth C++. In order to extend the computatonal functonaltes of ths language, addtonal external lbrares were ncluded. The extenson starts wth ncorporatng two classes belongng to LASTools. They enable to read and wrte the processed data that are stored n the LAS (Log ASCII Standard) format. The GNU Scentfc Lbrary and the Approxmate Nearest Neghbour Lbrary smplfy the mplementaton of segmentaton algorthm. For the purpose of vsualzaton the program uses the Open Graphcs Lbrary. To enable external nteracton a Graphc User Interface was developed, whch was done wth the usage of the Qt platform. 3. Algorthm The segmentaton algorthm mplemented n ths work, conssts of the four man steps: 1) Data preparaton : a) Fndng for each pont the n-nearest neghbours n the 3D space usng k-d trees, n = 10. b) Computng normal vectors - ths step s assocated wth dervng for each pont the equaton of the bestfttng plane, determned through that pont and ts nearest neghbours. c) Computng mean resdual and standard devaton of the dstances from ponts to ther best-fttng plane. The respectve values are assocated wth each pont. ) Settng the crterons the surface growng procedure s depended on and specfyng thresholds assocated wth them. 3) Fndng segments : a) Pckng the seed pont by analyzng the best fttng-plane s resduals. b) Performng an teratve plane growng process. c) When the extendng process s fnshed, detecton of a new seed pont and repeatng the step (a). 4) Mergng homogenous segments dvded nto dfferent sets (performed n order to avod over-segmentaton). The prncple of ths step s the same lke for the thrd one. Instead of sngular ponts, segments are used as an nput. a) Pckng the seed segment by analyzng the best fttng-plane s resduals. b) Performng an teratve plane growng process. c) When the extendng process s fnshed, detecton of a new seed pont and repeatng the step (a). 4

5 4 APPLICATIONS AND RESULTS Tests are performed n order to evaluate the applcablty of the mplemented surface growng approach. Therefore, the algorthm s examned aganst pont clouds derved from three sources. For the purpose of results comparson, the segmentaton of each consdered area s performed wth the same parameter settngs. Afterwards the segments that can be clearly dstngushed n each outcome are chosen and vsually compared. To facltate readablty, small clusters are dscarded and the number of presented segments s lmted to 30. Addtonally, statstcs computed for chosen objects enable to assess the segmentaton performance. The provded values gve nformaton about the number of ponts wthn the chosen segment, mean resdual from the best-fttng plane and standard devaton. 4.1 Study area and data The used data consst of three pont clouds. Each of them refers to the same object - relatvely smple house wth a gable roof. The pont clouds are acqured from dfferent technques: arborne laser scannng and multple mage matchng. LDAR data were collected by an ALS50 system durng ten flghts wth 45º FOV and the densty of 5 ponts/m. The frst matched pont cloud s generated by matchng mages acqured by a UCX Camera. The second one s derved by processng photos delvered by DMC. Image blocks of both cameras were acqured wth 8 cm GSD and 60% overlap along and across the strps. The vsualzaton of data s presented n Fg.3. (a) LDAR (b) UCX wth 8 cm GSD Fg. 3 (c) DMC wth 8 cm GSD Consdered buldng. Data provded by dfferent technques 5

6 4. Investgaton For each segmentaton process, the angle threshold α was set to 5º and the maxmum dstance equals 0.5 m. The results of buldng segmentaton are hghlghted n Fg. 4. Snce the roof segment s clearly dstngushed n each data set, t was chosen to conduct the comparson of segmentaton statstcs. Table 1 presents the comparson of results obtaned from three dfferent sources. From the fgure and the table t s seen that the most plausble results are obtaned from processng arborne laser scannng data (Fg. 4(a)). Both the groupng of smooth planes and the avodng of overand under-segmentaton have been successfully acheved n ths case. Fgure 4(b) outlnes the outcome of processng the data acqured by the UCX Camera. Besdes a roof surface, the segmentaton results n extracton of short edges nstead of planes. The raw ponts provded by mage matchng are not equal dstrbuted and concentrated at dscontnutes. Moreover, parts of the depcted objects are not smoothly connected. Therefore, due to the character of the data, applcaton of the plane growng algorthm gves a poor performance. Another pont cloud derved from mage matchng arses from photos acqured by DMC. Outcomes of ts segmentaton are presented n 4(c). The results are better than n the UCX case, as the pont cover s denser and the pont dstrbuton s more homogeneous. However, stll the qualty of segmentaton s not satsfyng. (a) LDAR (b) UCX wth 8 cm GSD Fg. 4 (c) DMC wth 8 cm GSD Orgnal pont clouds derved from dfferent sources (left mages) and ther segmentaton wth hghlghted roof ponts (rght mages) 6

7 Tab. 1 Comparson of roof segments Data source Number of ponts Mean resdual [m] Standard devaton [m] LDAR UCX DMC CONCLUSION Dffcultes wthn the mplementaton arose for the k-nearest neghbours search and normal vector computaton. The latter s provded for each pont, based on ts k-nearest neghbours and ther best-fttng plane. However, the selected ponts do not necessary le n the same plane. In ths case, the least squares plane estmaton s based, resultng n false normal vectors. Therefore, the frst groupng ponts nto a segments (called pre-clusters ) s not suffcent. Next, t s requred to update best-fttng plane s equaton for each pre-cluster. After updatng, plane growng has to be carred out agan usng pre-clusters nstead of sngular ponts. Orgnally, the mplemented algorthm was characterzed by two segmentaton parameters, angle threshold and dstance threshold. Due to the problems arsng for the computaton of accurate normal vectors, the run tme of algorthm ncreased consderably. To meet a tradeoff between the number of segments and the tme performance, an addtonal refnement parameter was ntroduced,.e. number of created pre-clusters. Ths threshold enables to skp the computaton of small clusters durng the segmentaton process. In order to facltate dstngushng the large number of objects n vsualzaton process, small segments are not dsplayed. Focusng on large segments only, sgnfcantly mproves the algorthm performance. It can be nferred from the conducted tests, that the performance of the mplemented algorthm s strongly depended on the character of the nput data. Pont clouds derved by multple mage matchng are denser at the edges than at the contnuous surfaces. Therefore, the segmentaton of matched pont clouds often returns clusters that depct several parts of object nstead of one whole object. However, among matched pont clouds there are some that sut much better than others do. In both tests, segmentaton of data stemmng from DMC systems gve results most comparable wth LDAR counterpart. The segmentaton approach s talored for pont sets that feature some level of order, lke for example even dstrbuton or regular spacng. As the ponts provded by LDAR are collected strp-wse, ths knd of nput gves very plausble results. Lterature [1] BIOSCA J.M., LERMA J.L. Unsupervsed robust planar segmentaton of terrestral laser scanner pont cloud based on fuzzy clusterng methods. ISPRS Journal of Photogrammetry and Remote Sensng, 63, 008, pp [] FILIN S., PFEIFER N. Segmentaton of arborne laser scannng data usng a slope adaptve neghborhood. ISPRS Journal of Photogrammetry and Remote Sensng, 60, 006, pp [3] HOOVER A., JEAN-BAPTISTE G., JIANG X., FLYNN P.J., BUNK, H., GOLDGOF D.B., BOWYER K., EGGERT D.W., FITZGIBBON A., FISHER R.B. An Expermental Comparson of Range Image Segmentaton Algorthms. IEEE Trans. Pattern Analyss and Machne Intellgence, 18, No. 7, 1996, pp [4] GOODMAN J.E., O OURKE J. Handbook of dscrete and computatonal geometry. CRC Press, Inc., Boca Raton, New York, [5] MOORE A.W. An ntroductory tutoral on k-d trees. Extract from PhD Thess: Effcent Memory-based Learnng for Robot Control, Unversty of Cambrdge, UK, [6] MOUNT D.M. (006) ANN Programmng Manual [onlne] [7] CHAPRA S.C., CANALE R.P. Numercal methods for engneers: wth software and programmng applcatons. The McGraw-Hll Companes, Inc., New York, 00. [8] WANG M., TSENG Y-H. LDAR data segmentaton and classfcaton based on OCTREE structure. Geo- Imagery Brdgng Contnents 0th ISPRS Congress, Istanbul, Turkey, 004. [9] TOVARI D. Segmentaton based classfcaton of arborne laser scanner data. PhD Thess, Unversty of Karlsruhe, Germany, 006. Revewer Andrzej Borkowsk, prof. dr hab. nż., Wroclaw Unversty of Envronmental and Lfe Scences, The Faculty of Envronmental Engneerng and Geodesy, Insttute of Geodesy and Geonformatcs, Grunwaldzka 53, Wroclaw, Poland, , andrzej.borkowsk@ up.wroc.pl 7