3-D Object Modeling and Recognition for Telerobotic Manipulation

Transcription

1 Research CMU Roboics Insiue School of Compuer Science D Objec Modeling and Recogniion for Teleroboic Manipulaion Andrew Johnson Parick Leger Regis Hoffman Marial Heber James Osborn Follow his and addiional works a: hp://reposiory.cmu.edu/roboics Par of he Roboics Commons Published In Proceedings of IEEE Inelligen Robos and Sysems, This Conference Proceeding is brough o you for free and open access by he School of Compuer Science a Research CMU. I has been acceped for inclusion in Roboics Insiue by an auhorized adminisraor of Research CMU. For more informaion, please conac research-showcase@andrew.cmu.edu.

2 3-D Objec Modeling and Recogniion for Teleroboic Manipulaion Andrew Johnson Parick Leger Regis Hoffman Marial Heber James Osborn The Roboics Insiue Pisburgh, PA Absrac This paper describes a sysem ha semi-auomaically builds a virual world for remoe operaions by consrucing 3-D models of a robo s work environmen. Wih a minimum of human ineracion, planar and quadric surface represenaions of objecs ypically found in manmade faciliies are generaed from laser rangefinder daa. The surface represenaions are used o recognize complex models of objecs in he scene. These objec models are incorporaed ino a larger world model ha can be viewed and analyzed by he operaor, accessed by moion planning and robo safeguarding algorihms, and ulimaely used by he operaor o command he robo hrough graphical programming and oher high level consrucs. Limied operaor ineracion, combined wih assumpions abou he robos ask environmen, make he problem of modeling and recognizing objecs racable and yields a soluion ha can be readily incorporaed ino many eleroboic conrol schemes. 1 Inroducion umerous eleoperaion asks require manipulaors o move objecs and posiion end effecors; ypically, he operaor uilizes 2-D daa from a suie of video cameras o plan and execue asks. The reliance on cameras limis he effeciveness of such eleoperaed sysems because video images provide only 2-D informaion. For many complex asks, 3-D informaion is vial. Previous work uses a variey of sensors and represenaions o consruc models of inerior workspaces. Chrisensen [1] describes a supervised eleoperaed sysem wih a world model ha includes a priori knowledge, he robo configuraion, and informaion gahered by he robo s sensors. The world model allows he operaor o preview he operaion, have he compuer auomaically plan an end-effecor rajecory, and view he ask from many differen viewing posiions. Trivedi [2] repors on anoher model-based sysem using range sensors; his sysem allows esing of robo plans in simulaion. Azarbayejani [3] has a sysem o semiauomaically consruc CAD models from uncalibraed video images. Thayer [4] compues an objec s locaion and orienaion using sereo vision; he operaor performs he sereo maching of some poins on he objec and he sysem compues he pose of he objec. These and oher modeling sysems share common characerisics ha jusify heir developmen: 1.They serve as a graphical aid o he operaor by allowing viewing he robo and is workspace from many more angles han are possible wih on-board cameras. 2. They sore knowledge of he world in he form of a mahemaical model; his capabiliy is essenial for auomaic planning agens. 3. They augmen uncerain or unrecorded a priori informaion wih up-o-dae in siu daa. The sysem we describe generaes 3-D models of manmade environmens from laser rangefinder daa o produce a virual 3-D world for he human operaor. This capabiliy is criical for fuure effors in which eleoperaed sysems evolve ino eleroboic and auonomous sysems. 2 Teleroboics and 3-D objec modeling The Deparmen of Energy has declared ha robos and remoe sysems will play crucial roles in fuure deconaminaion and decommissioning (D&D) of nuclear weapons faciliies [5]. Mobile worksysems will be used in he near erm for selecive equipmen removal, in which some par of an apparaus is exricaed while minimally disurbing he surrounding objecs. An example of a mobile Figure 1: The Rosie Mobile Worksysem removing a secion of pipe (lef). The IODOX faciliy, a DOE es sie for selecive equipmen removal (righ).

3 worksysem is Rosie, an advanced prooype for esing, evaluaing and demonsraing roboic selecive equipmen removal[6]. Rosie includes a locomoor, a heavy manipulaor, an operaor conrol cener, and a conrol sysem for robo operaion. An key componen of he operaor conrol cener is he 3-D objec modeling and recogniion sysem. Figure 1 shows he worksysem cuing and removing a secion of pipe using he DOE dualarm work module, Rosie s principal payload. A faciliy represenaive of hose in which Rosie and oher worksysems will operae is shown in Figure 1. The near erm goal of he work described herein is o enable dismanlemen of such faciliies using eleroboic conrol schemes ha provide subsanial produciviy gains relaive o baseline eleoperaion. Accurae 3-D objec modeling is he essenial foundaion for eleroboic planning and moion conrol. Our modeling sysem is a combinaion of sensors, modeling and analysis sofware, and an operaor inerface ha creaes 3-D models of indoor man-made environmens as hey are discovered. The ineracive sysem described performs he following funcions: 1.Acquire 3-D scene daa in he form of range images. 2. Selec a region of ineres in each range image. 3. Generae a surface mesh wihin he region of ineres. 4. Segmen mesh ino planar and quadric surfaces paches. 5. Mach surface paches o objec models for objec recogniion. 6. Incorporae objec models ino robo s world model. A block diagram of he above operaions and daa flow is shown in Figure 2. Auomaic acions are shown on he lef while operaor ineracions are given on he righ. The remainder of his paper deails hese seps and concludes wih a deailed descripion of he propagaion of errors hrough he sysem 3 Daa acquisiion The sensor used in our experimens is a 3-D scanning laser rangefinder manufacured by Percepron, Inc. I acquires 256 x 256 pixel range and inensiy images over a verical and horizonal field of view of 60 degrees a a frame rae of 2 Hz. The scanner s range is 2 o 40 meers and is range precision is 5-7 cm [7]; he sensor ha will be used in he final sysem will have much beer accuracy and precision. To map a faciliy, he scanner is remoely posiioned by a mobile worksysem and commanded by he human operaor o acquire a sequence of images. Linear fiing of image daa a known range values is used o calibrae he sensor for range scale and offse due o emperaure drif of he sensor which improves he range accuracy o he cenimeer range. The range and inensiy images acquired are displayed on he operaor s console. Figure 3 shows a sample image and reflecance image pair aken a our experimenal es sie. To focus he sysem on specific objecs in he scene o be analyzed, he operaor selecs a recangular region of ineres in he image. This limis he amoun of daa o ha Range Image selecion Region of Ineres Viable Objecs ransformaion Caresian Surface Mesh segmenaion Segmened Mesh recogniion Recognized Objecs confirmaion World Model Figure 2: Modeling sysem block diagram. which is imporan o recognizing he objecs of ineres. Figure 3 shows a region of ineres (inerior of he whie recangle in he range image) seleced by an operaor ineresed in deermining he posiion of he -join and diagonal pipe imaged by he sensor. Once he region of ineres is deermined a spaial smoohing filer is applied o eliminae ouliers while preserving range disconinuiies. The resuling range image is hen convered from spherical sensor coordinaes (ρ,θ,φ) o caresian world coordinaes (x,y,z). To group poins ino surface paches, a surface mesh is creaed ha esablishes he local conneciviy of he poins in space based on he conneciviy of pixels in he range image. eares neighboring range poins are conneced if he disance beween hem is less han a specified hreshold. (his preserves range sep disconinuiies when convering from he sensor o world coordinaes). The resuling se of conneced poins consiues a surface mesh in caresian space which is used in all subsequen processing sages. Figure 4 shows he surface mesh generaed from he region of ineres seleced in Figure 3. For presenaion purposes, he wo pipes ha form a T-join are labelled T1 and T2 ; he pipe beneah (and occluded by) he T- join is labelled P and he wo prominen fla regions are labelled F1 and F2. 4 Objec represenaion We have chosen a surface based objec represenaion (as opposed o volume based) because we are building world models which will be used o plan complex asks. A volume based represenaion is no sufficien for planning asks such as grasping an objec, applying a surface coaing, or obaining a sample. The robo workspace conains large numbers of man-made objecs wih fairly simple geomeries bu wih complex arrangemens and numerous inerconnecions (e.g., Figure 1). Curved objecs, such as pipes and vessels, can be concisely modeled wih quadric surfaces - second order surface descripions having en parameers, while fla objecs, such as walls, floors and suppor srucures can be modeled wih planar surfaces. Quadric and planar surfaces are suiable

4 Figure 3: Range image and seleced region of ineres (lef) and reflecance image (righ) of experimenal esbed. descripions for 3-D modeling of man-made ineriors because heir parameric represenaions do no vary when surfaces are parially occluded or he sensing viewpoin changes. Furhermore, he posiion and orienaion of surface paches in he scene can be easily compued from heir corresponding parameers. In he remainder of his paper he following noaion will be used. A 3-D poin x lies on a planar surface wih surface normal n and caresian disance d from he origin of he world coordinae frame if i saisfies: n x + d = 0 (1) A 3-D poin x lying on a quadric surface saisfies: x Ax + B x+ d = 0 (2) where A is a 3x3 marix describing he curvaure of he surface, B is a 3x1 marix describing he posiion of he quadric wih respec o he world coordinae sysem origin, and d is a scalar relaed o he size of he quadric. 5 Segmenaion Our segmenaion mehod is based on surface meshes ha group range poins ino surfaces and is independen of he means by which he range daa is colleced as long as he daa is ransformable ino a mesh represenaion. This is aracive as i allows he use of he range sensing modaliy (srucured ligh, deph from focus, laser rangefinding, sereo vision, ec.) ha is bes suied o he ask environmen. Though he resuls described herein are based on meshes obained from single range images, he echnique can be applied o daa obained from muliple views or differen sensors. Subsanial performance improvemens hrough reducing he effecs of occlusions, isolaed range errors and increasing he sensor field of view are hus possible. A general surface mesh segmenaion echnique is essenial o realizing hese benefis. The similariy measure for segmenaion in our algorihm is he proximiy of he poins o planar or quadric surfaces in he scene. The segmenaion groups poins ino surface paches which are described by a se of poins and he bes fi parameers of he surface fi o he poins. This parameric descripion of surface paches makes he ask of objec recogniion more efficien. Our algorihm, which is based on he algorihm for segmening range images presened by Faugeras and Heber [8], is presened in deail in [9]. I is a generalizaion of heir echnique because i performs he segmenaion using he arbirary conneciviy of a surface mesh. Furhermore, he similariy measure ha defines when o merge wo regions can be easily changed o measure any propery of he surface. The firs sage in he algorihm segmens a mesh ino planar surface paches using global region growing. Iniially, each poin in he mesh is considered a region; he wo regions wih he minimum merge error given by n x i + d 2 (3) where i is over poin in boh regions are merged. This merging coninues unil he oal merge error exceeds some user defined hreshold. The resul is a se of planar surface paches on he surface mesh. The algorihm for creaing quadric surface paches is very similar o ha used o generae planar surface paches. Firs, quadric paches are generaed from each sufficienly large planar pach by calculaing he quadric parameers of each pach. Then he wo quadric paches wih he minimum merge error given by x Axi i B + x i + d 2 (4) are merged. This merging coninues unil he oal merge error exceeds some user defined hreshold. The resul of he segmenaion is a se of quadric and planar surface paches. Figure 5 shows he resul of applying he segmenaion o he mesh given in Figure 4. The poins ha lie on he curved surfaces in he scene have been merged ino single curved quadric surfaces (T1,T2,P) while poins lying on fla surfaces have been merged ino planar surfaces (F1,F2), no quadric surfaces. Poins ha lie on differen pipes of he T-join have no been merged because hey belong o he disinc quadrics. The wo paches ha lie on P have no been merged due o he occlusion by he pipe T-join in he foreground. The increased uiliy of performing he segmenaion on a surface mesh comes a he cos of added represenaional complexiy of he daa in he algorihm. F1 T1 P Figure 4: Resul of consrucing a surface mesh from he region of ineres shown in Figure 3. T2 F2

5 T1 T2 ( Rx + T) A m ( Rx + T) + B m ( Rx + T) + d m = x A s x+ B s x + ds (5) from which can be derived P A m = R A m R = A s B m 2T = A m R + B m R = Bs (6) F2 (7) F1 Figure 5: An oblique view of he final planar and quadric surface paches. Prominen paches are idenified wih labels given in Figure 4. The surface mesh and he region adjacency graphs mus be represened wih undireced parameerized graphs o describe heir full 3-D conneciviy. Segmenaion echniques based on images can use simple funcions o access pixels, while lookup funcions on graphs are complicaed o ensure efficiency. 6 Objec recogniion The purpose of he mesh segmenaion is o group poins in he scene ino higher level consrucs ha can be used more efficienly by he objec recogniion sysem. The basic purpose of he objec recogniion sysem is o locae models of objecs in he scene. To make objec recogniion racable, he sysem relies on he naure of he srucured environmen o limi he number of possible objecs in he scene, and on operaor ineracion o verify he locaion of he objecs found by he compuer. Firs, he operaor selecs one or more objec ypes he recognizes in he region of ineres from a menu of objecs in a library. Then, he objec recogniion module searches for hese models in he scene. When a model is found, i is presened o he operaor for verificaion before being insered ino he world model a he pose compued by he objec recogniion module. The human ineracion is limied o selecing he objecs o look for and verifying heir locaion once hey are found. The objec recogniion mehodology presened, menioned by Besl in [10], relies on aligning principal direcions of he scene paches wih principal direcions of model paches o compue a ransformaion. 6.1 Maches beween quadrics Our objec recogniion sysem deermines he ransformaion beween a model surface pach represened by a se of surface parameers and a scene surface pach represened by a se of surface parameers and a se of poins ha are grouped by he pach. In he firs case, we will consider a mach beween a model quadric pach, denoed wih subscrip m, and a scene quadric pach, denoed wih subscrip s. Suppose ha he scene pach corresponds o a roaed and ranslaed version of he model pach. Then here exiss some roaion R and ranslaion T such ha d m = T A m T+ B m T+ d m = d s (8) The eigenvecors of A m correspond o he principal direcions of he model quadric and he eigenvecors of A s correspond o he principal direcions of he scene quadric. If he wo paches mach each oher hen Equaion 6 saes ha he R mus roae he principal direcions of he scene pach ono he principal direcions of he model pach. I is unknown which principal direcions in he model correspond o which principal direcions in he scene, so here exiss six possible roaions ha will roae he model principal direcions ono he scene principal direcions. These roaions are generaed by maching eigenvecors of he model pach o eigenvecors of he scene pach. The eigenvecors of he scene and he model have arbirary signs (i.e., hey can poin in eiher direcion along he principle axes of he quadric), so for each of he six ransformaions ha align he principle axes of he quadrics here exis four differen roaions. Hence, here exiss weny four possible roaions for he ransformaion of model o scene. Once he weny four maches beween model and scene eigenvecors have been enumeraed, he corresponding weny four roaion marices are generaed using he quaernion represenaion presened in [8]. For each of hese weny four roaions, Equaion 7 is solved for T, resuling in weny four (R,T) pairs. For each of he (R,T) pairs he new quadric parameers A m, B m and d m of he roaed model pach are calculaed from he lef hand sides of Equaion 6, 9 and 10. The bes (R,T) pair for he mach is hen seleced as he one ha minimizes he mach error measure 1 E --- x Am x B x d = i i m i m (9) for all of he poins in he scene pach. In he ideal case, where scene poins lie on a roaed and ranslaed version of model pach, he bes (R,T) pair will se A m, B m and d m equal o A s, B s and d s respecively, and Equaion 9 will equal zero. This mach error measure will reward ransformaions ha ransform he model pach parameers o similar values as he scene pach parameers and will penalize ones ha do no. oe, however, ha he mach error measure does no give he Euclidean disance beween he scene poins and he roaed model pach. Each possible roaion resuls from pairing axes along he principal direcions of he scene pach wih axes along he principal direcions of he model pach. A mach beween a scene pach and a model pach will consis of he pairings of hese axes, he ransformaion ha maps he

6 model pach o he scene pach and he mach error. In he case of a non-singular model quadric, he mach conains hree axis pairs. Special cases occur when he principal direcions of a quadric are ambiguous due o an axis of symmery. These special cases are deal wih horoughly, bu limied paper space precludes presening he deails. 6.2 Maches beween planes The procedure for finding he bes ransformaion beween a model planar pach and a scene planar pach is similar o ha presened for quadric paches. Suppose ha he planar scene pach corresponds o a roaed and ranslaed version of he planar model pach. Then here exiss some roaion R and ranslaion T such ha n m ( Rx + T) + d m = n s x + ds (10) from which can be derived n m = n m R = ns (11) d m = n m T + dm = d s (12) Equaion 11 requires he roaion R o roae he uni scene normal o he uni model normal. This roaion is he roaion abou n m x n s by an angle cos -1 (n m n s ). The appropriae ranslaion T can be deermined by fixing wo coordinaes of T and solving for he hird using Equaion 12. The ambiguiy in he scene surface normal is eliminaed by forcing he inner produc beween he scene surface normal and he ray from he sensor o be posiive. Similar o he case of maching quadrics, he roaed model plane parameers n m and d m are calculaed from he lef hand sides of Equaion 11 and 13 and he mach error for a model and scene planar pach is 1 E (13) --- n xi d = + 2 m m A mach beween a model planar pach and a scene pach will have one axis mach, he normals of he scene and model paches. Any model planar pach can be ransformed exacly o any scene planar pach, so Equaion 13 will always be zero when only wo planar paches are mached. However, complex objecs will have more han one plane making an exac ransformaion less common when more han one model plane is mached o he scene. 6.3 Searching for complex objecs Single quadric or planar surfaces are sufficien for modeling simple objecs like walls and pipes bu are no sufficien for modeling more complex objecs. Forunaely, models of complex objecs which are prevalen in manmade environmens, like pipe joins, I-beams and holding vessels, can be creaed by grouping ogeher muliple planar and quadric surfaces. This represenaion conveys only he surfaces ha make up a model and no does no represen he boundaries beween surfaces in a model, so explici conneciviy beween surfaces is no represened. However, for locaing he pose of objecs in he scene, his represenaion is sufficien. The size of he model is explici in he parameers of he surface ha make up he model, so differen models mus be creaed for similar objecs of varying size. Searching for models made from muliple surfaces in he scene relies on he echniques presened for finding a mach beween single surfaces as presened in he previous wo secions. The model in he scene is found by maching model paches o scene paches and compuing he ransformaion of he model from he mach. Firs, he model pach wih he highes discernabiliy (a measure based on he size of he pach and is abiliy o fix he pose of he pach in he scene) M 1 is mached o each scene pach and he ransformaions beween scene and model paches are compued. Maches wih errors (as given by Equaion 9 and Equaion 13) ha are below a pre-deermined mach error hreshold are insered ono a lis of feasible maches F because hey are good maches beween he parameers of he model pach and he poins grouped by he scene pach. All of he oher maches are eliminaed. ex, he model pach M 2 wih he second highes discernabiliy is mached o every pach in he scene and he ransformaions and he mach errors beween scene paches and M 2 are compued. Of hese maches, he ones wih errors below he mach error hreshold are kep in a emporary lis of feasible maches F 2. Then each mach in F 2 is combined wih each mach in F by concaenaing he liss of axis pairs of each mach. The roaions, ranslaions and mach errors of he combined maches are compued as explained below. The combined maches wih mach errors below he mach error hreshold are insered ino F. This process of creaing maches coninues, in he order of discernabiliy, unil all of he paches in he model have been searched for in he scene. A he end of he search, F will conain all full and parial feasible maches beween he model and he scene. The bes roaion R for a combined mach maps he axes of he model paches o he axes of he mached scene paches. I is found by minimizing he roaion error: i i 2 Re m e s (14) e i m are he model pach axes e i s are he scene pach axes The bes ranslaion of he combined pach is found by manipulaing Equaion 7 and Equaion 12 ino he forms 1 A m T = -- ( RB (15) 2 s B m ) n m T = ds d m (16) respecively. Then for each mach beween quadric

7 he eigenvalues of he ineria marix of he pach) and only maching planar model paches o planar scene paches and quadric model paches o quadric scene paches. The surface area consrain has been rudimenarily implemened by requiring ha scene paches have a minimum surface area before being considered for maching 7 World Model Figure 6: Oblique view (op) and sensor view (boom) of he locaion of recognized objecs superimposed on he surface mesh from Figure 4. paches in he mach, he hree equaions ha saisfy Equaion 15 are creaed, and for each mach beween planar paches in he mach Equaion 16 is creaed. These ses of equaions are sacked o creae one marix equaion ha can be solved for T using singular value decomposiion. SVD will produce he bes soluion in a leas squares sense even when here are less han hree axes maches which are required o fully deermine he ransformaion. The mach error for a combined mach is compued as he sum of he mach errors as found from Equaion 9 and Equaion 13 for each mach beween paches in he combinaion. Once all of he feasible maches have been found for he model in he scene, he bes one is shown o he user in is calculaed pose for verificaion. If he user believes ha he model is in is correc pose, he model is acceped and i is insered ino he robos world model. This user verificaion eliminaes he need for compuer reasoning abou feasible maches ha would be required in a compleely auomaic objec recogniion sysem. The nex bes feasible mach is presened o he user and he verificaion process is repeaed unil he user is saisfied wih he modeling or he lis of feasible maches is exhaused. This procedure is repeaed for each model seleced by he user. Figure 6 shows a locaed model pipe - join (made from wo quadric surfaces) mached o scene paches T1 and T2, and a pipe (made from one quadric surface) mached o scene pach P superimposed on he mesh from Figure 4. The pose of each model comes from he bes mach of model o scene. The combinaorial explosion of he search space is limied because afer each model pach is searched for in he scene only he maches wih a low error when combined wih maches on he lis of feasible maches are kep. Because he mach error is low when paches are proximal in pose space, his is a form of a rigidiy consrain. In addiion o rigidiy consrains, consrains on he surface area and curvaure of paches ha are mached are employed o limi he number of maches beween a single model pach and he scene paches. The curvaure consrain has been implemened by deciding which paches in he scene correspond o planes (by comparing The final sep in he modeling process is o presen useful geomeric informaion o he human operaor. A virual 3-D world provides richer undersanding of he environmen han 2-D camera daa. To effecively convey a view of he robo in is workspace and he recognized objecs a commercial robo simulaion package, TELEGRIP from Deneb Roboics Inc., is used. This package allows viewing of he synheic scene from any angle, manipulaor pah-planning, and off-line simulaion of robo acions. The pre-sored model of he robo vehicle, manipulaors and ooling is updaed wih he sensor-based models as hey are consruced.a represenaive view of he robos world model is given in Figure 7. 8 Error modeling The accuracy of he world model dicaes he possible funcions he operaor can perform wih he virual world and he exen o which he sensed world geomery can be rused as a basis for auomaic moion and ask planning. To perform a ask, he operaor mus undersand accuracy and errors ha are presen in he modeling process. Modeling he sources and propagaion of errors hrough he sysem characerizes he errors presen in he sysem and helps he operaor o make decisions on wha asks are possible, given he curren sensing condiions. Ulimaely hese errors depend on he error of individual poins sensed in he scene and he errors in he conrol of he manipulaor ha performs he ask. In he curren scenario, he locaion of a poin P o r wih respec o a coordinae sysem aached o a manipulaor is specified based on sensor daa. For example, P o r is he locaion o which he manipulaor mus Figure 7: TELEGRIP model of robo workspace.

8 sensor manipulaor Figure 8: Geomery for error modeling. move in order o cu a pipe. In his case P o r would have been compued by fiing a surface o daa poins segmened ou of a range image. Because of noise in he sensors, inaccuracies in he conrol of he manipulaor and calibraion errors, he manipulaor is acually going o move o a differen posiion P r = P o r + ε. The goal of his secion is o characerize he error ε. We characerize ε by is mean value E f and is covariance marix C f ( f sands for final ). E f characerizes he bias of he sysem. C f characerizes he average variaion of P r around P o r + E f. oice ha, if all he errors in he sysem were unbiased, E f would be zero. However, we will see ha calibraion errors do inroduce a bias ino he sysem. 8.1 Sensor errors T s r Assuming ha he range sensor provides range measuremens in spherical coordinaes, ρ(ϕ,θ), he sensor error can be modeled by he sandard deviaion of errors on he measured range, σ ρ, and on he scanning angles σ ϕ and σ θ. This model assumes ha he measuremen error has zero mean. Typically, σ ρ is a funcion of ρ 2, while σ ϕ and σ θ are consan [11]. Wih his model, he uncerainy C s on he locaion of a poin P of coordinaes P s = [x s y s z s ], wih respec o a coordinae frame aached o he sensor, corresponding o a measuremen ρ(ϕ,θ) is given by: C s = J s D s J s (17) where J s is he Jacobian of he ransformaion S(ρ, θ, φ) from spherical coordinaes o Caresian coordinaes. D s is he uncerainy marix of he measuremen in spherical coordinaes: D s diag σ 2 ρ σ2 θ σ 2 =,, (18) ϕ Because he measuremen errors are of zero mean, he error on he Caresian poin P s is also of zero mean. Therefore, he covariance marix C s is sufficien o characerize he error. I is assumed ha he range quanizaion noise is negligible compared o he measuremen noise. This implies ha he range is large compared o he resoluion of he measuremen. an assumpion ha is necessary for he covariance model o work. Oher assumpions are ha he mean range and angular error is zero and he errors are small enough ha his linear approximaion is accepable. P s P r 0 P r P r 8.2 Calibraion errors The coordinaes of P are ransformed o a coordinae sysem aached o he manipulaor, P r = [x r y r z r ]. The ransformaion beween manipulaor and robo coordinae sysem, T r s is a fixed rigid ransformaion compued by calibraing off-line he manipulaor wih respec o he sensor. T r s is composed of a roaion R(θ x,θ y,θ z ) and a ranslaion = [ x y z ]. The error is a fixed error, as opposed o he sensor error which is sochasic in naure. More precisely, we can assume he calibraion errors are represened by fixed errors δθ {x,y,z} and δ {x,y,z} on he hree angles of roaion and he hree componens of he ranslaion, respecively. Denoing by δt r s he ransformaion corresponding o he error parameers, he relaion beween he coordinaes P r in robo coordinaes and he coordinaes P s in sensor coordinaes can be wrien as: r r P r = T Ps s + δt Ps s (19) o r Since P r = T Ps, he error erm in robo coordinaes r s is δt Ps s which has mean E r and covariance C r given by: r r r E r = δt Ps s C r = δr s Cs δr s (20) In his expression, δr r s is he roaion par of δt r s (upper lef 3x3 marix). Assuming ha he calibraion errors are small, he marix δt r s can be easily calculaed as: r δt s = (21) The assumpions inheren in his error modeling are ha he errors are small enough for he linear approximaion of δt s r above o work and ha he calibraion of he sensor remains fixed 8.3 Manipulaor errors 0 δθ z δθ y δ x δθ z 0 δθ x δ y δθ y δθ x 0 δ z Assuming ha a arge poin P r has been seleced from he sensor daa, he manipulaor is commanded o move o P r. Errors are inroduced in he conrol of he manipulaor so ha insead of reaching he ideal poin P r, he manipulaor moves o a poin P r = P r + ε. Assuming posiion conrol, he error ε is due o he uncerainy on sensing he posiion of each join q i. More precisely, if each join posiion θ i (angle or linear displacemen) is measured wih a sandard deviaion σ i, and if he kinemaics of he manipulaor is described by a relaion of he ype: P r = F(q 1,.,q n ), where Q is he vecor of join posiions, he error ε on P r is of zero mean and of covariance marix C : C = J q C q J q C q = diag( σ 2 1,..., σ2 n) (22) In his model, J q is he Jacobian of he kinemaic ransformaion F.

9 The assumpions inheren in his error modeling of he manipulaor are ha he errors on join posiions are uncorrelaed, he errors on join posiions are independen of he configuraion, and he errors are unbiased. 8.4 Compounding of errors on a single poin In he model of he error aribuable o he manipulaor, he arge poin P r is iself uncerain because i comes from a sensor measuremen. Therefore, we need o compound (assuming ha he errors inroduced are uncorrelaed) he errors due o sensor noise and o calibraion. By combining he models developed above, we end up wih P r = P o r + ε sc + ε, where Po r is he rue locaion of he poin in space, ε sc is he error due o he combinaion of sensor noise and calibraion error and ε is he error due o he manipulaor. Since ε is of zero mean, he mean error E f on P r is he mean of ε sc which was compued in he firs secion: (23) where P s is he poin compued from he measuremen (ρ,ϕ,θ).the covariance C f of he error ε sc + ε is he sum of he covariances on ε sc and ε: r C f δr s J s D s J r = s δr s + J q C q J q (24) A complee error model for locaing he posiion of a poin in space has been presened. The error inroduced on he posiion of he poin afer a surface has been fi o he sensed daa has ye o be invesigaed. 9 Conclusion r r E f = δt Ps s = δt s ( S ( ρθϕ,, )) Many inernal deails of he sysem operaion have been described and illusraed, from he collecion of he range image, hrough segmenaion, o final model display. The sysem as used by a human operaor hides mos of his deail. The human operaor iniiaes range image acquisiion, chooses a region of ineres, and selecs he objec models ha exis in he scene. The planar and quadric segmenaion and he subsequen locaion of objec models in he scene are performed auomaically. Locaed models ha ruly correspond o objecs in he scene are hen verified by he operaor and insered ino he world model. The world model conains a model of he robo, all recognized objecs and any available a priori informaion abou he worksie, so i is an excellen ool for planning remoe deconaminaion and decommissioning asks. In he near fuure, he complee sysem will be inegraed o DOE s Dual Arm Work Module and hen mouned on he Rosie worksysem and an overhead ransporer. The mobiliy hus afforded will enable collecion of muliple range images of he same ask space. Fuure work will focus on means o effecively combine ha daa ino more deailed, more accurae and more complee ask space models. Acknowledgmens This work is funded under conrac DE-AC21-92MC29104 from he Unied Saes Deparmen of Energy Morganown Energy Technology Cener. The auhors wish o acknowledge he echnical suppor and guidance of he DOE Roboics Technology Developmen Program, paricularly Dr. Linon Yarbourgh and Dr. William Hamel, as well as he echnical saff of he Teleroboics Sysem Secion of Oak Ridge aional Laboraories and Sandia aional Laboraories Inelligen sysems and Roboics Cener. We also acknowledge he programmaic suppor of Mr. Vijay Kohari of he Morganown Energy Technology Cener. Wenfan Shi, Herman Herman, Cecelia Shepherd and Marie Elm conribued echnical suppor a Carnegie Mellon Universiy. References [1] B.K. Chrisensen, e al. Graphic Model Based Conrol of Roboics Sysems for Wase Remediaion. Proceedings of he AS Fifh Topical Meeing on Roboics and Remoe Sysems, pp , [2] M. Trivedi and C. X. Chen. Developing Sensor-based Roboics Sysems Using Virual Realiy Conceps. Proceedings of he AS Fifh Topical Meeing on Roboics and Remoe Sysems, pp , [3] A.J. Azarbayejani e al. Recursive Esimaion for CAD Model Recovery. Proceedings of he Second CAD-Based Vision Workshop, IEEE Compuer Sociey, pp , [4] S. Thayer e al. On-line Sereo Vision and Graphical Inerface for Deconaminaion and Decommissioning Applicaions Using he Advanced Servo Manipulaor. Proceedings of he AS Fifh Topical Meeing on Roboics and Remoe Sysems, pp , [5] Environmenal Resoraion and Wase Managemen Five- Year Plan Fiscal Years Repor # DOE/ P, US Deparmen of Energy, [6] L. Conley, W. Hamel, and B. Thompson. Rosie: A Mobile Worksysem for Deconaminaion and Dismanlemen Operaions. Proceedings of he AS Sixh Topical Meeing on Roboics and Remoe Sysems, pp , [7] M. Heber and E. Krokov. 3-D Measuremens from Imaging Laser Radars: How Good Are They? Inernaional Journal of Image and Vision Compuing, 10(3), pp , [8] O.D. Faugeras and M. Heber. The Represenaion, Recogniion and Locaing of 3-D Objecs. Inernaional Journal of Roboics Research, 5 (3), pp , [9] M. Heber, R. Hoffman, A. Johnson, and J. Osborn. Sensor- Based Inerior Modeling. Proceedings of he AS Sixh Topical Meeing on Roboics and Remoe Sysems, pp , [10]P. J. Besl and. D. McKay A Mehod for Regisraion of 3- D Shapes. IEEE Transacions on Paern Analysis and Machine Inelligence, 14 (2), pp , [11]I. Kweon, R. Hoffman, and M. Heber. Experimenal Characerizaion of he Percepron Laser Rangefinder. Technical Repor# CMU-RI-TR-91-1, The Roboics Insiue, Carnegie Mellon Universiy, 1991.