Accurate Object Localization in 3D Laser Range Scans

Size: px

Start display at page:

Download "Accurate Object Localization in 3D Laser Range Scans"

Daniela Green
5 years ago
Views:

50 40 30 20 10 Point Distibution numbe of points 0.

Reseach Goup Albechtstaße 28 D-49069 Osnabück, Gemany {nuechte lingemann hetzbeg}@infomatik.uni-osnabueck.

de Abstact This pape pesents a novel method fo object detection and classification in 3D lase ange data that is acquied by an autonomous mobile obot.

The pefomance of the classification is impoved by combining both senso modalities, which ae independent fom extenal light.

INTRODUCTION Envionment peception is a basic poblem in the design of autonomous mobile cognitive systems, i.e., of a mobile obot.

The pefomance of such a obot highly depends on the accuacy and eliability of its pecepts and on the computational effot of the involved intepetation pocess.

1 Point Distibution numbe of points distances [cm] 7 Accuate Object Localization in 3D Lase Range Scans Andeas Nüchte, Kai Lingemann, Joachim Hetzbeg Univesity of Osnabück, Institute fo Compute Science Knowledge-Based Systems Reseach Goup Albechtstaße 28 D Osnabück, Gemany {nuechte lingemann hetzbeg}@infomatik.uni-osnabueck.de Hatmut Sumann Faunhofe Institute fo Autonomous Intelligent Systems (AIS) Schloss Bilinghoven D Sankt Augustin, Gemany hatmut.sumann@ais.faunhofe.de Abstact This pape pesents a novel method fo object detection and classification in 3D lase ange data that is acquied by an autonomous mobile obot. Unesticted objects ae leaned using classification and egession tees (CARTs) and using an Ada Boost leaning pocedue. Off-sceen endeed depth and eflectance images seve as an input fo leaning. The pefomance of the classification is impoved by combining both senso modalities, which ae independent fom extenal light. This enables highly accuate, fast and eliable 3D object localization with point matching. Competitive leaning is used fo evaluating the accuacy of the object localization. I. INTRODUCTION Envionment peception is a basic poblem in the design of autonomous mobile cognitive systems, i.e., of a mobile obot. A cucial pat of the peception is to lean, detect, localize and ecognize objects, which has to be done with limited esouces. The pefomance of such a obot highly depends on the accuacy and eliability of its pecepts and on the computational effot of the involved intepetation pocess. Pecise localization of objects is the all-dominant step in any navigation o manipulation task. This pape poposes a new method fo the leaning, fast detection and localization of instances of 3D object classes. The appoach uses 3D lase ange and eflectance data acquied by an autonomous mobile obot to peceive the 3D objects. The 3D ange and eflectance data ae tansfomed into images by off-sceen endeing. Based on the ideas of Viola and Jones [25], we built a cascade of classifies, i.e., a linea decision tee. The classifies ae composed of classification and egession tees (CARTs) and model the objects with thei view dependencies. Each CART makes its decisions based on featue classifies and leaned etun values. The featues ae edge, line, cente suound, o otated featues. Lienhat et. al and Viola and Jones have implemented a method fo computing effectivly these featues using an intemediate epesentation, namely, integal image [12], [25]. Fo leaning object classes, a boosting technique, paticulaly, Ada Boost, is used [6]. Afte detection, the object is localized using a matching technique. Heeby the pose is detemined with six degees of feedom, i.e., with espect to the x, y, and z positions and the oll, yaw and pitch angles. Finally the quality of the object localization is evaluated by fast subsampling of the scanned 3D data. The esulting appoach fo object detection is eliable and eal-time capable and combines ecent esults in compute vision with the emeging technology of 3D lase scannes. Fig. 1 gives an oveview of the implemented system. II. STATE OF THE ART Common appoaches of object detection use infomation of CCD-cameas that povide a view of the obot s envionment. Nevetheless, cameas ae difficult to use in natual envionments with changing light conditions. Robot contol achitectues that include obot vision mainly ely on tacking, e.g., distinctive, local, scale invaiant featues [18], light souces [11] o the ceilings [5]. Othe camea-based appoaches to obot vision, e.g., steeo cameas and stuctue fom motion, have difficulties poviding navigation infomation fo a mobile obot in eal-time. Camea-based systems have poblems localizing objects pecisely, i.e., single cameas estimate the object distance only oughly using the known object size due to the esolution. Estimating depth with steeo is impecise eithe: Fo obots, the width of the steeo base line is limited 3D Scan Off Sceen Classification Ray Tacing Model Matching Evaluation Result endeing Visualization depth efl. depth efl. compaing subsampled 3D models "... ///// database Fig. 1. System Oveview. Afte acquiing 3D scans, depth and eflection images ae geneated. In these images, objects ae detected using a leaned epesentation fom a database. Ray tacing selects the points coesponding to the 2D pojection of the object. A 3D model is matched into these points, followed by an evaluation step.

Cuently a geneal tend exists to use 3D lase ange findes and build 3D maps [2], [19], [22], [23]. Nevetheless, only little wok has been done in intepeting the obtained 3D models.

2 to small values (e.g., < 20 cm), esulting in a typical z-axis eo of about 78 cm fo objects at the scanne s maximum anging distance of about 8 m. Many cuent successful obots ae equipped with distance sensos, mainly 2D lase ange findes [24]. 2D scannes cannot detect 3D obstacles outside thei scan plane. Cuently a geneal tend exists to use 3D lase ange findes and build 3D maps [2], [19], [22], [23]. Nevetheless, only little wok has been done in intepeting the obtained 3D models. In [14] we show how complete scenes, made of seveal automatically egisteed 3D scans, ae labeled using elations given in a semantic net. Object detection in 3D lase scans fom mobile obots was pesented in [13]. This appoach is extended hee: Fist, CARTs ae used fo a moe sophisticated object detection, second, objects ae localized in 3D space using point based matching, and thid, the accuacy of the matching is evaluated. In the aea of object ecognition and classification in 3D ange data, Johnson and Hebet use the well-known ICP algoithm [4] fo egisteing 3D shapes into a common coodinate system [10]. The necessay initial guess of the ICP algoithm is done by detecting the object with spin images [10]. This appoach was extended by Shapio et al. [16]. In contast to ou poposed method, both appoaches use local, memoy consuming suface signatues based on pio ceated mesh epesentations of the objects. Futhemoe, spin images ae not able to model complicated objects, i.e., objects with nonsmooth, o non-poducible mesh epesentation. One of the objects used in this pape, the volksbot [1], is of such a stuctue (Fig. 9). Besides spin images, seveal suface epesentation schemes ae in use fo computing an initial alignment. Stein and Medioni pesented the notion of splash to epesent the nomals along a geodesic cicle of a cente point, which is the local Gauss map fo 3D object ecognition with a database [20]. Ashock et al. poposed a paiwise geometic histogam to find coesponding facets between two sufaces that ae epesented by tiangle meshes [3]. Hamonic maps and thei use in suface matching have been used by Zhang and Hebet [26]. Recently, Sun and colleagues have suggested so-called point fingepints : They compute a set of 2D contous that ae pojections of geodesic cicles onto the tangent plane and compute similaities between them [21]. All these appoaches take the local geomety of the sufaces into account, i.e., meshes. They have poblems coping with unstuctued point clouds. III. THE AUTONOMOUS MOBILE ROBOT KURT3D A. The Kut Robot Platfom Kut3D (Fig. 2) is a mobile obot platfom with a size of 45 cm (length) 33 cm (width) 26 cm (height) and a weight of 15.6 kg, both indoo as well as outdoo models exist. Equipped with the 3D lase ange finde, the height inceases to 47 cm and the weight inceases to 22.6 kg. 1 1 Videos of the exploation with the autonomous mobile obot can be found at: Fig. 2. The autonomous mobile obot Kut3D equipped with the 3D lase ange finde as pesented at RoboCup The scannes technical basis is a SICK 2D lase ange finde (LMS-200). Kut3D s maximum velocity is 5.2 m/s (autonomously contolled: 4.0 m/s). Two 90W motos ae used to powe the 6 wheels. Compaed to the oiginal Kut3D obot platfom, this cuent vesion has lage wheels, whee the middle wheels ae shifted outwads. Font and eal wheels have no tead patten to enhance otating. Kut3D opeates fo about 4 hous with one battey (28 NiMH cells, capacity: 4500 mah) chage. The coe of the obot is an Intel-Centino-1400 MHz with 768 MB RAM and a Linux opeating system. An embedded 16-Bit CMOS micocontolle is used to contol the moto. B. The 3D Lase Scanne The 3D lase ange finde (Fig. 2) is built on the basis of a 2D ange finde by extension with a mount and a standad sevo moto [23]. The 2D lase ange finde is attached to the mount in the cente of otation fo achieving a contolled pitch motion. The sevo is connected on the left side (Fig. 2). The 3D lase scanne opeates up to 5h (Scanne: 17 W, 20 NiMH cells with a capacity of 4500 mah, Sevo: 0.85 W, 4.5 V with batteies of 4500 mah) on one battey pack. IV. DETECTING OBJECTS IN 3D LASER DATA A. Rendeing Images fom Scan Data Afte scanning, the 3D data points ae pojected by an offsceen OpenGL-based endeing module onto an image plane to ceate 2D images. The camea fo this pojection is located in the lase souce, thus all points ae unifomly distibuted and enlaged to emove gaps between them on the image plane. Fig. 9 shows eflectance images and endeed depth images (distances encoded by gey-values) as well as point clouds. B. Featue Detection using Integal Images Thee ae many motivations fo using featues athe than pixels diectly. Fo mobile obots, a citical motivation is that featue-based systems opeate much faste than pixel-based systems [25]. The featues used hee have the same stuctue as the Haa basis functions, i.e., step functions intoduced by Alfed Haa to define wavelets [8]. They ae also used in [12], [15], [25]. Fig. 3 (left) shows the eleven basis featues, i.e.,

3 ============<<<<<<<<<<< edge, line, diagonal and cente suound featues. The base esolution of the object detecto is fo instance pixels, thus, the set of possible featues in this aea is vey lage ( featues, see [12] fo calculation details). In contast to the Haa basis functions, the set of ectangle featues is not minimal. A single featue is effectively computed on input images using integal images [25], also known as summed aea tables [12]. An integal image I is an intemediate epesentation fo the image and contains the sum of gay scale pixel values of image N with height y and width x, i.e., I(x, y) = x x =0 y =0 y N(x, y ). The integal image is computed ecusively, by the fomulas: I(x, y) = I(x, y 1) + I(x 1, y) + N(x, y) I(x 1, y 1) with I( 1, y) = I(x, 1) = I( 1, 1) = 0, theefoe equiing only one scan ove the input data. This intemediate epesentation I(x, y) allows the computation of a ectangle featue value at (x, y) with height and width (h, w) using fou efeences (see Fig. 3 (ight)): F (x, y, h, w) = I(x, y) + I(x + w, y + h) I(x, y + h) I(x + w, y). Fo computing the otated featues, Lienhat et. al. intoduced otated summed aea tables that contain the sum of the pixels of the ectangle otated by 45 with the bottom-most cone at (x, y) and extending till the boundaies of the image (see Fig. 3 (bottom left)) [12]: I (x, y) = x x =0 x x y y =0 N(x, y ). The otated integal image I is computed ecusively, i.e., I (x, y) = I (x 1, y 1) + I (x + 1, y 1) + I (x, y 1) + N(x, y) + N(x, y 1) using the stat values I ( 1, y) = I (x, 1) = I (x, 2) = I ( 1, 1) = I ( 1, 2) = 0. Fou table lookups ae equied to compute the pixel sum of any otated ectangle with the fomula: F (x, y, h, w) = I (x + w h, y + w + h 1) + I (x, y 1) I (x h, y + h 1) I (x + w, y + w 1). Since the featues ae compositions of ectangles, they ae computed with seveal lookups and subtactions weighted with the aea of the black and white ectangles. To detect a featue, a theshold is equied. This theshold is automatically detemined duing a fitting pocess, such that a minimal numbe of examples ae misclassified. Futhemoe, the etun values (α, β) of the featue ae detemined, such that the eo on the examples is minimized. The examples ae given in a set of images that ae classified as positive o negative samples. The set is also used in the leaning phase that is biefly descibed next ::::: ;;;;;; ::::: ;;;;;; ::::: ;;;;;; ::::: ;;;;;; ::::: ;;;;;; > > >???? > > >???? > > >???? > > >???? > > >???? > > AAAAA BBBBB CCCCCC BBBBB CCCCCC I (x,y 2) DDDEEEE DDDEEEE I (x,y) y GMGMGMGMGM JKJKJKJKJKJKJKJKJKJKJKJKJKJKJKJKJKJKJKJK x x+w GLGLGLGLGL GMGMGMGMGM JIJIJIJIJIJIJIJIJIJIJIJIJIJIJIJIJIJIJIJI JKJKJKJKJKJKJKJKJKJKJKJKJKJKJKJKJKJKJKJK GLGLGLGLGL GMGMGMGMGM JIJIJIJIJIJIJIJIJIJIJIJIJIJIJIJIJIJIJIJI JKJKJKJKJKJKJKJKJKJKJKJKJKJKJKJKJKJKJKJK y+h I (x h,y+h 1) w h ONONONONONONO ONONONONONONO OPOPOPOPOPOPO OPOPOPOPOPOPO POPOPOPOPOPOPOP OTOTOTOT RJRJRJRJRJRJRJRJRJRJRJRJR ONONONONO ONONONONO OPOPOPOPO OPOPOPOPO OSOSOSO OTOTOTO JQJQJQJQJQJQJQJQJQJ JRJRJRJRJRJRJRJRJRJ ONONO ONONO OPOPO OPOPO OSO OTO JQJQJQJQJQJ JRJRJRJRJRJ O JRJJ I (x h+w,y+w+h 1) I (x+w,y+w 1) Fig. 3. Top left: Edge, line, diagonal and cente suound featues ae used fo classification. Top ight: Computation of featue values F in the shaded egion is based on the fou uppe ectangles. Bottom left: Calculation of the otated integal image I. Bottom ight: Fou lookups in the otated integal image ae equied to compute the featue value of a otated featue F. (a) th. = (b) th. = th. = th. = th. = Fig. 4. Left: A simple featue classifie with its theshold and etun values α and β. Right: A Classification and Regession Tee with 4 splits. Accoding to the specific filte applied to the image input section x, the output of the tee, h(x) is calculated, depending on the theshold values. C. Leaning Classification Functions 1) Classification and Regession Tees: Fo all possible featues a Classification and Regession Tee (CART) is ceated. CART analysis is a fom of binay ecusive patitioning. Each node is split into two child nodes, the oiginal node is called a paent node. The tem ecusive efes to the fact that the binay patitioning pocess is applied ove and ove to each a given numbe of splits (i.e., 6 splits in the case of the object volksbot). In ode to find the best possible split featues, we compute all possible splits, as well as all possible etun values to be used in a split node. The pogam seeks to maximize the aveage puity of the two child nodes using the misclassification eo measue [17]. Fig. 4 (left) shows a simple featue classifie and a simple CART (ight). 2) Gentle Ada Boost fo CARTs: The Gentle Ada Boost Algoithm is a vaiant of the poweful boosting leaning technique [6]. It is used to select a set of simple CARTs to achieve a given detection and eo ate. In the following, a detection is efeed to as a hit and an eo as a false alam. The vaious Ada Boost algoithms diffe in the update scheme of the weights. Accoding to Lienhat et al., the Gentle Ada Boost Algoithm is cuently the most successful leaning pocedue tested fo face detection applications [12].

4 The leaning is based on N weighted taining examples (x 1, y 1 ),..., (x N, y N ), whee x i ae the images and y i { 1, 1} the classified output i {1,..., N}. At the beginning of the leaning phase, the weights w i ae initialized with w i = 1/N. The following thee steps ae epeated to select simple CARTs until a given detection ate d is eached: 1) Evey simple classifie, i.e., a CART, is fit to the data. Heeby the eo e is calculated with espect to the weights w i. 2) The best CART h t is chosen fo the classification function. The counte t is incemented. 3) The weights ae updated with w i := w i e yiht(xi) and enomalized. The final output of the classifie is sign( T t=1 h t(x)) > 0, with h(x) the weighted etun value of the CART. Next, a cascade based on these classifies is built. D. The Cascade of Classifies The pefomance of a single classifie is not suitable fo object classification, since it poduces a high hit ate, e.g., 0.999, but also a high eo ate, e.g., 0.5. Nevetheless, the hit ate is significantly highe than the eo ate. To constuct an oveall good classifie, seveal classifies ae aanged in a cascade, i.e., a degeneated decision tee. In evey stage of the cascade, a decision is made whethe the image contains the object o not. This computation educes both ates. Since the hit ate is close to one, thei multiplication esults also in a value close to one, while the multiplication of the smalle eo ates appoaches zeo. Futhemoe, this speeds up the whole classification pocess, since lage pats of the image do not contain elevant data. These aeas can be discaded quickly in the fist stages. An oveall effective cascade is leaned by a simple iteative method. Fo evey stage, the classification function h(x) is leaned until the equied hit ate is eached. The pocess continues with the next stage using the coectly classified positive and the cuently misclassified negative examples. These negative examples ae andom image pats geneated fom the given negative examples that pass the pevious stages and thus ae misclassified. This bootstapping pocess is the most time consuming of the taining phase. The numbe of CARTs used in each classifie may incease with additional stages. Fig. 5 shows an example cascade of classifies fo detecting a volksbot in 2D depth images, whose esults ae given in Table I. E. Application of the Cascades Seveal expeiments wee made to evaluate the pefomance of the poposed appoach with two diffeent kinds of images, namely, eflectance and depth images. Both types ae acquied by the 3D lase ange finde and ae pactically light invaiant. About 200 epesentation of the objects wee taken in addition to a wide vaiety of negative examples without any taget object. The detection stats with the smallest classifie size, e.g., pixel fo the human classifie, fo the volksbot classifie. The image is seached fom top left to bottom ight by applications of the cascade. To detect th. = th. = th. = th. = th. = th. = th. = th. = th. = th. = th. = th. = th. = th. = th. = th. = th. = th. = th. = th. = th. = th. = th. = th. = th. = th. = th. = th. = th. = th. = th. = Σ h(x) < 0 Σ h(x) < 0 th. = th. = th. = th. = th. = th. = th. = th. = 6.485e 06 th. = th. = th. = h(x) >= 0 Σ h(x) >= 0 Σ h(x) >= 0 Σ Fig. 5. The fist thee stages of a cascade of classifies to detect the object volksbot. Evey stage contains seveal simple classifie tees that use Haa-like featues with a theshold th. and etun values of P h(x). h(x) is detemined by the path though the tees. objects on lage scales, the detecto is escaled. An advantage of the Haa-like featues is that they ae easily scalable.

Fig. 6. Object points estimation by ay tacing. Top left: All points inside a detection aea ae extacted. Top Right and bottom: 3D view. 3D points inside the detecto aea (viewing cone) ae ed coloed.

9 show examples of the detection. To decease the false detection ate, we combine the cascades of the depth and eflectance images.

To avoid the eduction of the hit ate, seveal diffeent offsceen endeed images ae used, whee the vitual camea is otated and the apex angle is changed [13]. A. Object Points Estimation V.

5 Fig. 6. Object points estimation by ay tacing. Top left: All points inside a detection aea ae extacted. Top Right and bottom: 3D view. 3D points inside the detecto aea (viewing cone) ae ed coloed. Each featue equies only a fixed numbe of look-ups in the integal image, independent of the scale. Time-consuming pictue scales ae not necessay to achieve scale invaiance. Fig. 9 show examples of the detection. To decease the false detection ate, we combine the cascades of the depth and eflectance images. Thee ae two possible ways fo combining: Eithe the two cascades un inteleaved o seial and epesent a logical and [13]. The joint cascade deceases the false detection ate close to zeo. To avoid the eduction of the hit ate, seveal diffeent offsceen endeed images ae used, whee the vitual camea is otated and the apex angle is changed [13]. A. Object Points Estimation V. OBJECT LOCALIZATION Afte object detection in a 2D pojection the algoithm finds the coesponding 3D points using ay tacing. All 3D points that have been pojected into the classified aea ae etieved using a special OpenGL pojection matix. Fig. 6 (ight) shows a endeing of aytaced 3D points. B. Model Matching Afte the 3D data (set D) that contain the object is found, a given 3D model fom the object database is matched into the point cloud. The model M is also saved as 3D point cloud in the database. The well known iteative closest points algoithm (ICP) is used to find a matching [4]. The ICP algoithm calculates iteatively the point coespondences. In each iteation step, the algoithm selects the closest points as coespondences and calculates the tansfomation, i.e., otation and tanslation (R, t) fo minimizing the equation E(R, t) = w i,j d i (Rm j + t) 2, N m N d i=1 j=1 1 N N m i (Rd i + t) 2 (1) i=1 whee N m and N d, ae the numbe of points in the model set M o data set D, espectively, and w ji ae the weights fo a point match. The weights ae assigned as follows: w ji = 1, if m i is the closest point to d j within a close limit, w ji = 0 othewise. It is shown that the iteation teminates in a minimum [4]. In each iteation, the tansfomation is calculated by the quatenion based method of Hon [9]. The assumption is that the point coespondences ae coect in the last iteation step. Finally, the pose of the model coesponds to the one in the data set. C. Evaluating the Match Geneally, one is inteested in the quality of the matching, i.e., the accuacy of the model pose inside the 3D data. Many application specific tasks equie this estimation, e.g., complicated obot navigation tasks. Howeve, the value of the eo function (1) does not give this infomation, since point densities influence this value. Diffeent point densities ae the esult of the scanning pocess, i.e., the spheical and continuous measuement of the lase. The scanne emits the lase beams in a spheical way such that the data points close to the souce ae moe dense. A competitive leaning technique is used to subsample the model and data set. 1) Competitive Object Leaning: In addition to subsampling, goals of competitive object leaning ae the minimization of the expected quantization eo and entopy maximization. A finite set of 3D scan points D is subsambled to the set A = {w 1, w 2,..., w N }. Eo minimization is done with espect to the following function: E(D, A) = 1 D w i A ξ R c ξ w i, with the set A of samples and the Voonoi set R c of unit c, i.e., R c = {ξ D s(ξ) = c} and s(ξ) = ag min c A ξ w c. Entopy maximization guaantees inheent obustness. The failue of efeence vectos, i.e., missing 3D points, fects only a limited faction of the data. Intepeting the geneation of an input signal and the subsequent mapping onto the neaest sample in A as a andom expeiment which assigns a value x A to the andom vaiable X, then maximizing the entopy H(X) = x A P (x) log(p (x)) is equivalent to equipobable samples. The following neual gas algoithm leans and subsamples 3D points clouds [7]: i.) Initialize the set A to contain N vectos, andomly fom the input set. Set t = 0.

6 _ h gh i gh i Point Distibution 50 numbe of points a ac ad ac ad ac ae ad 10 ac ae ad ac ae ad ac ae ad ae m a aq a a a 0 a a distances [cm] Fig. 8. A typical distibution of distances between closest points te egisteing two models with a fixed (hee: 250) numbe of points. Fig. 7. Top: 3D models (point clouds) of the database. Bottom: sumbsampled models with 250 points. ii.) Geneate at andom an input element ξ, i.e., select a point fom D. iii.) Ode all elements of A accoding to thei distance to ξ, i.e., find the sequence of indices (i 0, i 1,..., i N 1 ) such that w i0 is the efeence vecto closest to ξ, w i1 is the efeence vecto second closest to ξ, etc., w ik, k = 0,..., N 1 is the efeence vecto such that k vectos w j exists that ae close to ξ than w ik. k i (ξ, A) denotes the numbe k associated with w i. iv.) Adapt the efeence vectos accoding to w i = ε(t)h λ (k i (ξ, A)) (ξ w i ), with the following time dependencies: λ(t) = λ i (λ f /λ i ) t/tmax, ε(t) = ε i (ε f /ε i ) t/tmax, h λ (k) = exp( k/λ(t)). v.) Incease the time paamete t. The neual gas algoithms is used with the following paametes: λ f = 0.01, λ i = 10.0, ε i = 0.5, ε f = 0.005, t max = Note that t max contols the un time. Fig. 7 shows 3D models of the database (top ow) and subsampled vesions (bottom) with 250 points. 2) Estimating Matching Quality: Given two egisteed point sets that contain an equal numbe of points, e.g., 250 points deived unde the pemise of minimization of the expected quantization eo and entopy maximization, the quality of a matching can be evaluated using the following method: The distibution of shotest distances d ij between the ith and the jth point (closest points) te egisteing two models with a fixed (hee: 250) numbe of points show a typical stuctue (Fig. 8). Many distances ae vey small, i.e., less than 0.3 cm, and thee ae also many lage distances, e.g., geate than 1 cm. To ou expeience it is always easy to find a good theshold to sepaate the two maximas. Afte dividing the set of distances d i, the algoithm computes the mean and the standad deviation of the matching, i.e., µ = 1 N N d i σ = 1 N N (d i µ) 2 i=1 i=1 Based on these values one estimates the matching quality by computing a measue D as a function of µ and σ (we have been using D = µ + 3σ). Small values of D coespond to a high quality matching wheeas inceasing values epesent lowe qualities. VI. RESULTS AND CONCLUSION The pocess of geneating the cascade of classifies is elatively time-consuming, but it poduces quite pomising esults. The fist thee stages of a leaned cascade ae shown in Fig. 5. The time pefomance of the object detection cucially depends on the bootstapping, i.e., on the geneation of false positive examples duing the stage leaning. Nevetheless, leaning has to be executed only once, the application of the cascade if vey fast (300 ms). Thus the majo time fo the accuate object localization is spent duing the model alignment and evaluation step ( 1.4 s). The capabilities of the chosen appoach have been evaluated in vaious expeiments. Fig. 9 shows fou examples of successful detections and Table I summaizes the object localization esults. A. Futue Wok Needless to say, much wok emains to be done. Futue wok will concentate on fou majo aspects: 1) Impove the computational pefomance of the system to impove obot/envionment inteaction. 2) Geneate high level desciptions and semantic maps including the 3D infomation, e.g., in XML fomat.

7 Fig. 9. Examples of object detection and localization. Fom Left to ight: (1) Detection using single cascade of classifies. Geen: detection in eflection image, yellow: detection in depth image. (2) Detection using the combined cascade. (3) Supeimposed to the depth image is the matched 3D model. (4) Detected object in the aw scanne data, i.e., point epesentation. TABLE I OBJECT NAME, NUMBER OF STAGES USED FOR CLASSIFICATION VERSUS HIT RATE AND THE TOTAL NUMBER OF FALSE ALARMS USING THE SINGLE AND COMBINED CASCADES. THE TEST SETS CONSIST OF 89 IMAGES RENDERED FROM 20 3D SCANS. THE AVERAGE PROCESSING TIME IS ALSO GIVEN, INCLUDING THE RENDERING, CLASSIFICATION, RAY TRACING, MATCHING AND EVALUATION TIME. object # stages detection ate (eflect. img. / depth img.) false alams (eflect. img. / depth img.) aveage poc. time chai (0.867 / 0.767) 12 (47 / 33) 1.9 sec kut obot (0.912 / 0.947) 0 ( 5 / 7) 1.7 sec volksbot obot (0.844 / 0.851) 5 (42 / 23) 2.3 sec human (0.963 / 0.961) 1 (13 / 17) 1.6 sec

8 The semantic maps will contain spatial 3D data with desciptions and labels. 3) Integate a camea and enhance the semantic intepetation by fusing colo images with ange data. The apetue angle of the camea will be enlaged using a pan and tilt unit to acquie colo infomation fo all measued ange points. 4) Enlage the database with moe objects of an indoo and outdoo envionment and build an explicit knowledge base, i.e., specifying a semantic net containing geneal object elations as well as links to the object database [14]. The final goal of object detection and localization is to develop unesticted, automatic and highly eliable algoithms that could be used in scenaios like RoboCup Rescue. B. Conclusions This pape has pesented a novel method fo the leaning, fast detection and localization of instances of 3D object classes. The 3D ange and eflectance lase scanne data ae tansfomed into images by off-sceen endeing. Fo fast object detection, a cascade of classifies is built, i.e., a linea decision tee [25]. The classifies ae composed of classification and egession tees (CARTs) and model the objects with thei view dependencies. Each CART makes its decisions based on featue classifies. The featues ae edge, line, cente suound, o otated featues. Afte object detection the object is localized using a point matching stategy. The pose is detemined with six degees of feedom, i.e., with espect to the x, y, and z positions and the oll, yaw and pitch angles. A final computation etuns a quality measue fo the object localization. The pesented combination of algoithms, i.e., the system achitectue enables high accuate, fast and eliable 3D object localization fo autonomous mobile obots. REFERENCES [1] The volksbot obot, volksbot/, [2] P. Allen, I. Stamos, A. Gueoguiev, E. Gold, and P. Blae. AVENUE: Automated Site Modeling in Uban Envionments. In Poceedings of the thid Intenational Confeence on 3D Digital Imaging and Modeling (3DIM 01), Quebec City, Canada, May [3] A. P. Ashock, R. B. Fishe, C. Robetson, and N. Weghi. Finding suface coespondences fo object ecognition and egistation using paiwise histoams. In Poceedings of the Euopean Confeence on Compute Vision (ECCV 98), pages , Feibug, Gemany, June [4] P. Besl and N. McKay. A method fo Registation of 3 D Shapes. IEEE Tansactions on Patten Analysis and Machine Intelligence, 14(2): , Febuay [5] F. Dellaet, W. Bugad, D. Fox, and S. Thun. Using the Condensation Algoithm fo Robust, Vision-based Mobile Robot Localization. In Poceedings of the IEEE Confeence on Compute Vision and Patten Recognition (CVPR 99), Ft. Collins, USA, June [6] Y. Feund and R. E. Schapie. Expeiments with a new boosting algoithm. In Machine Leaning: Poceedings of the 13th Intenational Confeence, pages , [7] B. Fitzke. A gowing neual gas netwok leans topologies. In Advances in Neual Infomation Pocessing Systems 7 - Poceedings of the 7th Advances in Neual Infomation Pocessing Systems (NIPS 95), pages , Cambidge, MA, USA, [8] A. Haa. Zu Theoie de othogonalen Funktionensysteme. Mathematische Annalen, (69): , [9] B. Hon. Closed fom solution of absolute oientation using unit quatenions. Jounal of the Optical Society of Ameica A, 4(4): , Apil [10] A. Johnson and M. Hebet. Using spin images fo efficient object ecognition in clutteed 3D scenes. IEEE Tansactions on Patten Analysis and Machine Intelligence, 21(5): , May [11] F. Launay, A. Ohya, and S. Yuta. Autonomous Indoo Mobile Robot Navigation by detecting Fluoescent Tubes. In Poccedings of the 10th Intenational Confeence on Advanced Robotics (ICAR 01), Budapest, Hungay, August [12] R. Lienhat and J. Maydt. An Extended Set of Haa-like Featues fo Rapid Object Detection. In Poceedings of the IEEE Confeence on Image Pocessing (ICIP 02), pages , New Yok, USA, Septmbe [13] A. Nüchte, H. Sumann,, and J. Hetzbeg. Automatic Classification of Objects in 3D Lase Range Scans. In Poceedings of the 8th Confeence on Intelligent Autonomous Systems (IAS 04), pages , Amstedam, The Nethelands, Mach [14] A. Nüchte, H. Sumann, and J. Hetzbeg. Automatic Model Refinement fo 3D Reconstuction with Mobile Robots. In Poceedings of the 4th IEEE Intenational Confeence on Recent Advances in 3D Digital Imaging and Modeling (3DIM 03), pages , Banff, Canada, Octobe [15] C. Papageogiou, M. Oen, and T. Poggio. A geneal famewok fo object detection. In Poceedings of the 6th Intenational Confeence on Compute Vision (ICCV 98), Bombay, India, Januay [16] S. Ruiz-Coea, L. G. Shapio, and M. Meila. A New Paadigm fo Recognizing 3-D Object Shapes fom Range Data. In Poceedings of the IEEE Confeence on Compute Vision and Patten Recognition (CVPR 03), Madison, USA, June [17] S. Russell and P. Novig. Atificial Intelligence, A Moden Appoach. Pentice Hall, Inc., Uppe Sanddle Rive, NJ, USA, [18] S. Se, D. Lowe, and J. Little. Local and Global Localization fo Mobile Robots using Visual Landmaks. In Poceedings of the IEEE/RSJ Intenational Confeence on Intelligent Robots and Systems (IROS 01), Hawaii, USA, Octobe [19] V. Sequeia, K. Ng, E. Wolfat, J. Goncalves, and D. Hogg. Automated 3D econstuction of inteios with multiple scan views. In Poceedings of SPIE, Electonic Imaging 99, The Society fo Imaging Science and Technology /SPIE s 11th Annual Symposium, San Jose, CA, USA, Januay [20] F. Stein and G. Medioni. Stuctual indexing: Efficient 3d object ecognition. Tansaction on Patten Analysis and machine Vision (PAMI), 14: , Febuay [21] Y. Sun, J. Paik, A. Koschan, D. Page, and M. Abidi. Point Fingepint: An New 3D Object Repesention Scheme. IEEE tansaction on Systems, Man, and Cybenetics Pat B: Cybenetics, 33(4), [22] H. Sumann, K. Lingemann, A. Nüchte, and J. Hetzbeg. A 3D lase ange finde fo autonomous mobile obots. In Poceedings of the of the 32nd Intenational Symposium on Robotics (ISR 01), pages , Seoul, Koea, Apil [23] H. Sumann, A. Nüchte, and J. Hetzbeg. An autonomous mobile obot with a 3D lase ange finde fo 3D exploation and digitalization of indoo en vionments. Robotics and Autonomous Systems, 45(3 4): , Decembe [24] S. Thun, D. Fox, and W. Bugad. A eal-time algoithm fo mobile obot mapping with application to multi obot and 3D mapping. In Poceedings of the IEEE Intenational Confeence on Robotics and Automation (ICRA 00), San Fancisco, CA, USA, Apil [25] Paul Viola and Michael J. Jones. Robust eal-time face detection. Intenational Jounal of Compute Vision, 57(2): , May [26] D. Zhang and M. Hebet. Hamonic maps and thei application in suface matching. In Poceedings of the IEEE Confeence on Compute Vision and Patten Recognition (CVPR 99), pages , Ft. Collins, CO, USA, June ACKNOWLEDGMENTS The wok was done duing the authos time at the Faunhofe Institute fo Autonomous intelligent Systems. We would like to thank Saa Miti, Simone Fintop, Kai Pevölz and Matthias Hennig.

Robust Object Detection at Regions of Interest with an Application in Ball Recognition

Robust Object Detection at Regions of Inteest with an Application in Ball Recognition Saa Miti, Simone Fintop, Kai Pevölz, Hatmut Sumann Faunhofe Institute fo Autonomous Intelligent Systems (AIS) Schloss