A High-Speed Adaptive Multi-Module Structured Light Scanner

Transcription

1 A High-Speed Adapive Muli-Module Srucured Ligh Scanner Andreas Griesser 1 Luc Van Gool 1,2 1 Swiss Fed.Ins.of Techn.(ETH) 2 Kaholieke Univ. Leuven D-ITET/Compuer Vision Lab ESAT/VISICS Zürich, Swizerland Leuven, Belgium {griesser,vangool}@vision.ee.ehz.ch luc.vangool@esa.kuleuven.ac.be Absrac Moion capuring sysems ha are based on 3D models require high-speed scanning mehods. One-sho srucured ligh echniques aim a a good balance beween speed and accuracy. Due o paern inerference, currenly available seups capure 3D only from one single viewpoin. We propose work o enable opposiely posiioned scanning modules o acquire 3D daa simulaneously and hereby o speed up he acquisiion even furher. Key is he applicaion of dynamic projecion masks, ha limi he srucured ligh projecion o he relevan par of he scene, i.e. he person. This requires racking of he person s ouline. Keywords: 3D acquisiion, reconsrucion, muliple srucured ligh sysems 1 Inroducion Currenly 3D scanning sysems based on srucured ligh echniques work as sand-alone modules and acquire 3D from only one viewpoin. In order o build complee models, he cameraprojecor pair and he scanned objec have o be roaed wih respec o each oher. Highspeed applicaions, like moion capure, canno be deal wih in his way. They require ha muliple scanning devices work in parallel. Wihou he usage of specialized hardware, like ferroelecric shuer-glasses in fron of he projecor lenses, acivaing one module afer he oher would slow down he process o a poin where he dynamics are no longer capured well. Ye, we wan o build sysems ha are low-cos and capure human 3D moion well, boh in erms of spaial and emporal resoluion. Here, we propose a sysem ha is based upon muliple sae-of-he-ar srucured ligh modules and ha quickly capures 360 o 3D. By having more han one such module working simulaneously we have o mee he following challenges: Synchronizaion Paern inerference Robusness Anoher imporan issue is ha scanning sysems - acive or passive alike - also capure daa from cluered backgrounds, which unnecessarily consumes compuaion ime. In his paper we propose a soluion, ha already resuls in much denser 3D daa han hose obained wih markerbased sysems, and his a 13 Hz. The paper is organized as follows: In secion 2 imporan relaed work is described. Secion 3 discusses our sysem archiecure and how he synchronizaion scheme looks like. The inerference of differen paerns is addressed in Secion 4. Secion 5 describes a echnique o increase robusness of he scanning process and secion 6

2 Maser Machine Module 2 Module 3 Module 4 Module 1 Figure 1: Our muli-module srucured ligh seup consising of four scanning modules and one maser compuer. focuses on he inegraion of racking mehods, which allow for scanning objecs under moion. Finally, he resuls are shown in secion 7. Secion 8 concludes he paper. 2 Relaed Work Hall-Hol and Rusinkiewicz [1] and Rusinkiewicz e al. [2] already demonsraed he feasabiliy of real-ime srucured ligh wih a sequence of differen, projeced sripe paerns, bu heir mehod was resriced o relaively slow moion. We build upon anoher srand of fas srucured ligh sysems, ha ge 3D daa from he projecion of a single paern (so-called one-sho 3D acquisiion). These sysems have he obvious advanage ha hey ake only a very shor ime o acquire he necessary daa. As a consequence, hey are ideal o capure dynamic shapes [3, 4, 5]. Mos of hese sysems do no process he daa on-line, however. A one-sho sysem ha allows o also exrac he 3D daa while scanning has recenly been proposed by Koninckx e al. [6, 7, 8]. One such module only requires off-he-shelf hardware (a sandard color video camera, an LCD projecor, and a PC wih framegrabber card). The srucured ligh paern consiss of horizonal whie sripes, which are idenified via heir inersecion wih diagonal colored sripes. 3 Archiecure and Synchronisaion Our sysem combines four of he aforemenioned 3D acquisiion modules [7, 8]. One dedicaed PC, furher referred o as maser, is responsible for he synchronizaion of he enire susem componens. Our goal is o le wo opposiely posiioned modules work in parallel. The oher module pair is acived direcly afer he firs pair has grabbed he camera images. To minimize moion blur and loss of informaion, he ime needed for his swiching procedure mus be kep shor. All cameras in our seup are exernally riggered by he maser compuer. This iniiaes an acquisiion cycle by grabbing he camera s image, processing i, sending he deph map o he maser for furher model inegraion, and applying masking o he nex projeced paern. Figure 2 demonsraes he iming sequence for wo opposiely posiioned scanning modules. A firs rigger causes boh cameras of a module pair o ake an image and hen send he image over he camera-pc inerface o a framegrabber card. The processing sep aferwards involves deph reconsrucion as well as paern adapaion. Since boh modules graphic cards work independenly, heir verical frame pulses (Vsync), which idenify he sar of a new projecion frame, are unaligned wih respec o each oher. In figure 2 he firs and hird row show

3 Module 3 Module 1 firs rigger VSync Cam. exp. & ransfer Processing nex possible rigger diff_exp Addiional Proj. Frame Nex Projecion Frame Figure 2: Timing sequence for wo opposiely posiioned scanning modules wihou projecor synchronizaion. he emporal behaviour of he cameras and PCs, whereas he second and fourh row visualize he oupu of he projecion devices. Assuming ha boh modules finish daa processing a he same ime, he adaped paern is displayed firs on module 1 because he nex possible display frame is earlier han on module 3. Therefore, he nex possible simulaneous rigger for boh cameras can only occur afer module 3 has also begun he new projecion. Our goal now is o swich o he second module pair as soon as possible so ha moion blur in he final inegraed 3D model is minimized. The earlies possible momen for his swich is afer he previous image exposure and he urn-off of module 3 s projecor. In case of unsynchronized graphic cards, he ime inerval beween he beginning of he firs pair s cameras exposure and he earlies projecor being urned off is shown as diff exp. If his inerval is less han he camera s exposure ime, he capured daa of module 3 are incomplee. Avoiding such siuaions requires an addiional projecion frame. Indeed, his also increases he swiching ime beween module pairs. However, synchronizing all graphic cards wihin he seup simplifies he deecion of he nex possible rigger poins and also eliminaes he addiional projecion frames. Our aemp for such synchronizaion is based on SofGenLock [9]. This video genlock soluion uilizes he Vsync signal of he maser compuer o drive he Vsyncs of all modules. This is done by disribuing he maser s Vsync signal over a simple one-wire nework o each module. There he pixel clock of he graphics card is regulaed unil frame synchronizaion is reached. Our measuremens show a maximum jier of approximaely 50µs. 4 Masking By projecing sripe paerns ha fill complee frames, he reconsrucion will also include pars of he background. Such cluer needlessly consumes addiional compuaion ime. Masking ou he paern ouside he silhouee of he scanned objec prevens he background from being illuminaed and herefore from being processed any furher. Moreover, by projecing also ouside of he objec, here is a risk of projecing direcly ino he opposie camera, which would basically render i blind. This masking process is splied ino he following seps: sripe deecion and sripe indexing, ransfer of each labelled sripe ino projecor coordinaes via epipolar geomery, compuaion of he objec silhouee, and finally limiing he paern o his silhouee for he nex frame. Figure 3 visualizes his process. In order o idenify he objec silhouee, he exraced sripes lying oo far from he 3D modules are elimaed. The scanning area s dimensions are known from he calibraion of he seup. By discarding sripes a oo large disances, only sripes on he foreground objec are reained. This clipping is based on he endpoins of he sripes. Figure 4: Difference beween full scene illuminaion (l.) and masked paern (r.). In figure 4 he difference beween 3D reconsrucions wih full paern and masked paern illuminaion is shown. The speedup in processing is remarkable as i is abou a facor of 5!

4 a b c d e Figure 3: Masking of he sripe paern: a:camera image, b:deeced sripes, c:sripes in proj. coord., d:compued silhouee, e: new projecion paern for he nex frame. 5 Mask Offse When applying a projecion mask o he paern, i is also essenial o deal wih objecs under moion. When our scanned objec is moving ou of he illuminaed region, we are no longer able o recover he new objec s conour since we don have any sripe informaion ouside of his region. An approximae polygon offse mehod, which les he paern mask grow uniformly around he objec s silhouee, is used o remedy his recovery problem. Our implemenaion is based on a discree Voronoi diagram, as described in [10], and replaces each verex of he silhouee polygon by a disc and each edge by a recangle. The resul is overlayed wih a essellaion of he original polygon in order o rerieve he new projecion mask. This process is shown in figure 5. Since all compuaions are done on he GPU, here is jus a minimal processing overhead needed on he CPU for ransferring he drawing commands o he graphics card. The amoun of mask offse needs o srike a balance beween higher compuaion load for clipping and occlusion handling (big offse) and shorer recovery imes (small offse). 6 Mask Tracking Especially for moion capuring applicaions, he speed of model acquisiion is of umos imporance. Therefore we decided o use smaller mask offses and o compensae for his by adding silhouee racking. The goal is o predic he nex posiion of our objec of ineres, so ha we ge an esimaion for he nex paern Figure 5: Mask-Offse. Top-lef: inpu silhouee-polygon (dashed) compued from horizonal sripes in projecor coordinaes. Top-righ: essellaion. Boom-lef: replacing verices by discs and edges by recangles. Boom-righ: final overlayed mask. mask. Raher han aking full 2D or 3D informaion ino accoun - his would lead o unaccepable compuaion imes - he sysem focuses on deeced sripes in projecor coordinaes. Our algorihm calculaes a per-sripe predicion of i s endpoins by aking only heir horizonal shifs over ime ino accoun. A simple linear esimaor predics he nex posiion bes on he previous and he curren one. This mehod proved sufficien and is exremly fas.

5 Figure 6: Snapsho of a 3D-scan aken from wo opposie sides a he same ime. 7 Resuls The overall processing pipeline, as depiced in figure 7, is he same for all scanning modules and is iniiaed by an exernal camera rigger from he maser compuer. Firs he curren image is grabbed and processed, i.e. he sripes are deeced and indexed. Those sripes, which pass he deph clipping es, are used for he complee mesh reconsrucion and ransfer o he maser compuer for model inegraion. In parallel, hey are also used for silhouee racking. The nex frame predicion is he basis for furher paern masking, mask-offseing and paern projecion. Figure 7: Processing pipeline for each scanning module. The following able summarizes he measured compuaion imes for clipping, silhouee creaion, racking, mask essellaion and mask offseing. The underlying CPU is a Penium-4 ype wih 2.67GHz and he number of deeced sripes is 500. Task avg. ime [ms] Compue Mask 2.1 Mask Tracking 0.2 Tessellae Mask 1.2 Offse Mask 1.0 descripion Clipping, Conour- Compuaion CPU-GPU- Transfer CPU-GPU- Transfer Figure 6 shows a snapsho of a 3D scan aken from wo opposie modules a he same ime. Two consecuive illuminaions are used: he masked sripe paern for 3D mesh reconsrucion and a homogeneous whie paern o acquire he objec exure. The problem of selfocclusion, which possibly leads o mislabeled sripes, is addresses in [8]. A combinaion of geomeric coding, color coding and racking over ime aims o o solve he correspondence problem. The posiion and orienaion of each module relaive o a common world coordinae sysem (WCS) is known from sysem calibraion. Therefore each module compues a 3D mesh relaive o he WCS, which simplifies he model inegraion on he maser compuer. Even wihou

6 a closed mesh in figure 6, he person can be idenified well and enough feaures can be exraced for moion capure applicaions. The scanning speed is highly dependen on he scene complexiy, bu frame raes of approximaely 13Hz can currenly be reached. 8 Conclusions and Fuure Work This work focuses on he simulaneous use of muliple one-sho srucured ligh sysems for high-speed scanning of human bodies under moion. We demonsraed ha wo nearly opposiely posiioned modules can work in parallel and ha paern inerference is avoided by adding a projecion mask. This sraegy also reduces he processing ime by eliminaing unnecessary pars of he background. The robusness, i.e. he recovery of he paern on he scanned objec, is increased by applying rapid polygonoffseing combined wih a simple racking. The sysem consiss of four Modules bu currenly only wo of hem are used. We sared working on flexible swiching mechanisms beween pairs of scanning modules o increase he compleeness of he inegraed model. Acknowledgmens The auhors graefully acknowledge he suppor of Canon AG, Swizerland. The auhors also graefully acknowledge suppor by KULeuven GOA projec MARVEL and SNF NCCR projec IM2. References [1] O. Hall-Hol and S. Rusinkiewicz. Sripe boundary codes for real-ime rucuredligh range scanning of moving objecs. In In. Conf. on Compuer Vision, pages , [4] M. Maruyama and S. Abe. Range sensing by projecing muliple slis wih random cus. IEEE Trans. Paern Anal. Mach. Inell., 15(6): , [5] M. Proesmans and L. Van Gool. One-sho acive 3D image capure. In Proc. SPIE, volume 3023, pages 50 61, [6] T.P. Koninckx and L. Van Gool. Efficien, acive 3D acquisiion, based on paernspecific snake. In Proceedings 24h DAGM symposium on paern recogniion, pages , Sepember [7] T. Koninckx, A. Griesser, and L. Van Gool. Real-ime range scanning of deformable surfaces by adapively coded srucured ligh. In S. Kawada, edior, 4. inern. conf. on 3-d dig. imaging and modeling, pages , Ocober [8] T. Koninckx, I. Geys, T. Jaeggli, and L. Van Gool. A graph cu based adapive srucured ligh approach for realime range acquisiion. In Proceedings of he 2nd Inernaional Symposium 3D Daa Processing, Visualizaion, and Transmission, Thessaloniki, Greece, [9] J. Allard, V. Gouranon, G. Lamarque, E. Melin, and B. Raffin. Sofgenlock: Acive sereo and genlock for pc cluser. In Proceedings of he Join IPT/EGVE 03 Workshop, Zurich, Swizerland, May [10] Kenneh E. Hoff, III, John Keyser, Ming Lin, Dinesh Manocha, and Tim Culver. Fas compuaion of generalized voronoi diagrams using graphics hardware. In Proc. of he 26h conf. on comp. graph. and ineracive echniques, pages , [2] S. Rusinkiewicz, O. Hall-Hol, and M. Levoy. Real-Time 3D Model Acquisiion. In Proc. of Siggraph, [3] M. Proesmans, L. Van Gool, and A. Ooserlinck. Acive acquisiion of 3d shape for moving objecs. In In. Conf. on Image Processing, pages , 1996.