Searching the Optimal Threshold for Voxel Coloring in 3D Reconstruction

Transcription

1 Searchg the Optmal Threshold or oxel Colorg D Recostructo Youg-Youl Y, Hyo-Sug Km, Soo-Youg Ye, K-Go Nam Departmet o Electroc Egeerg, Pusa Natoal Uversty Emal: mustapha@pusa.ac.r Abstract oxel colorg s oe o the well-ow methods or recostructg a D shape rom D mages. The covetoal methods cause a trade-o problem betwee precso ad stablty, whe they recostruct D shapes. I ths paper, we preset a ovel approach to solve the trade-o problems. Ths method searches the real ace voxel o comparg the photo-cosstecy o a sde voxel o the optc ray wth the ace voxel o a ceter camera. As teratg proposed voxel colorg, the method ca search the optmal threshold by tsel. The graph cut method s also used or reducg the ace ose. Keywords: oxel Colorg, optmal threshold, photo-cosstecy, optcal ray, D recostructo Itroducto The desre o huma beg craves ot the meda o D plae ace but the meda D space, because o the mproved computer perormace ad the wde spread o hgh speed Iteret. The vrtual realty that s emboded D space s stll the begg level, but t s already used some elds, or example mult meda cotets, game, move, ad educato/trag smulato. I ear uture, vrtual realty wll be used all ds o elds. The most mportat thg vrtual realty techque s costructg the D model. The mage-based D shape recostructo has bee studed or a log tme. The techques o recostructg D model ca be classed to two large groups. Oe s actve sesg, ad the other s passve oe. The actve sesg aalyzes structured lght that s relected o real obect. The passve sesg aalyzes mages that are acqured uder geeral llumato or atural lght. The passve sesg has lower precso tha actve sesg, but t s hady method usg oly geeral CCD camera, so t ca be wdely used may elds. I ths thess, we propose a ovel method to recostruct D shape rom mult-vew slhouettebased mages. The prevous voxel colorg method meaes photo-cosstecy o sgle ace voxel ad compares t wth pre-establshed sgle threshold, the decdes to elmate the voxel or ot. It s a very streuous wor to d the best sgle threshold. Eve the best threshold has bee oud, applyg t to all ace voxel maes a tradeo problem betwee precso ad stablty. I the proposed method, we compare the photo-cosstecy o ace voxel wth ts eghbor sde voxel, ad elmate ace voxel ts photo-cosstecy s lower tha ts eghbor. A graph cut method s also used to reduce the rregular ose o ace. Prevous wor As metoed prevously, there have bee may trals o recostructg D shape rom mult-vew slhouette-based mages. Baer[] used the slhouette o a obect rotatg o a turtable to costruct a wre-rame model o the obect. Mart ad Aggarwal[] used volumetrc descrptos to represet the recostructed shape. Potmesl[] suggested a octree model usg arbtrary vews to speed up shapg rom slhouette. For each o the vews, he costructed the octree represetg rom coc volume ad tersected octrees. Szels[] rst created a low resoluto octree model qucly ad the reed ths model teratvely, by tersectg each ew slhouette wth the already exstg model. Geerally, the voxel carvg method usg slhouette mages ca qucly recostruct the D shape o a obect voxel level, but the method also has some problems Setz ad Dyer[] proposed the voxel colorg method usg photo-cosstecy wthout volume carvg. It ca reduce model errors. The voxel colorg method has a dsadvatage that the posto o the camera havg to satsy the ordal vsblty costrat. Culbertso ad Malzbeder[] proposed a geeralzed voxel colorg method (GC) that ca be used wth a radomly postoed camera. Our approach The recostructed D shapes usg volume tersecto have errors mage acqusto ad the cocave porto o the obect modelg. I gure (a), we assume that ace_a s recostructed ace that s derved by the volume carvg method

2 C - p C C + a b c d e g h l m o p q (a) a b c d e g h l m o p q (b) a b c d e g h l m o p q (c) ace_a ace_b ad ace_b s the real ace o obect at the camera C -, C, C +. A voxel,, s deed as the xy voxel located o the optcal axs o ceter camera C to be located by coordate (x, z) o x-z axs. Gve ay voxel, we ca obta two voxels o the real ace through the voxel, xy, rom two camers, C -,C +. Usg the photo-cosstecy o the two voxels, we ca obta the dssmlarty o the voxel. I the Fgure (b), usg the voxel, ad two cameras, C -,C +, we ca obta two voxels, h + ad o the real ace. From the photo-cosstecy o h + ad, we ca get the dssmlarty o. h + ad are close to o the real ace ad the dssmlarty o s more decreasg. At a voxel,, Fgure (c), we ca obta the lowest dssmlarty, because the voxel s correct voxel o real ace. At a voxel, Fg. (d), the dssmlarty was calculated by hgher value tha. The smaller value o the dssmlarty, the closer the voxel s located at the real ace. Thereore, the dssmlarty o ay voxel xy has to compared wth the dssmlarty o aother voxel o the optcal axs. Ad the dssmlarty o the xy s larger tha the dssmlarty o ext voxel o the optcal axs, the voxel should be elmated. Ths process should teratvely be perormed utl dg the mmum dssmlarty to estmate the voxel. I the Fgure, the characterstc chage o dssmlarty was show. Whe s, the dssmlarty o a voxel has the lowest value. So we decded the voxel as real ace voxel. From all ceter camera postos, dssmlarty was calculated, ad the the optmal threshold value was decded. Ths method ca be decrease modelg error comparg wth covetoal method usg the sgle-xed threshold because o mult-varable threshold o the all voxels. dssmlarty o the optcal ray at ceter camera optmal thresh a b c d e g h l m o p q (d) /terato Fgure : Dssmlarty calculato at the ceter camera o the optcal ray. Fgure : Balace o orces

3 Proposed oxel colorg Steps The proposed voxel colorg s bascally a orm o GC algorthm[] ad searchg optmal threshold method s added. The specc method ollows ext steps.. Calculatg camera posto T T T Let P, P, P are the row vector o the gve camera proecto matrx P. P T X = 0 ad P T X = 0 mea axs plae. P T X = 0 meas prcpal plae le Fgure. The camera posto C s calculated by Eq. ().. Searchg vsble ace voxels The searched ace voxel s proected o a mage plae to search vsble ace voxel. I two voxels are overlaped, voxel dex be saved the vsble dex buer wth mmum depth rom camera ceter le Fgure. Ater proectg all ace voxel, there s the oly oe dex o the voxel whch s see rom a camera the vsble dex buer. Ater ths process s perormed, the ormato o all voxel s acqured at each camera. sble Idex buer P C P u v Y O X Z 0 0,, 0,, 0,, Select voxel dex wth mmum depth P Icrease depth Fgure : Three plaes deed by the row vectors o the camera matrx PC=0 (). Searchg ace voxels We ted to search the ace voxel whch are o D voxel matrx that s recostructed by carvg method. The D voxel matrx has the ormato that whe the voxel s carved, the voxel value s, otherwse s 0. To search ace voxel, the value o tsel would be ad the oe o the -coected eghbor voxels would be at least 0, we wll allocate a voxel to ace voxel, the Eq. (). p { = = 0} () = p p p N Where, s the voxel o D voxel matrx at p arbtrary posto, s the eghbor voxel o p. p Fgure s -coected eghbor voxels. p p Fgure : -coected eghbor voxels. Fgure : Estmato o a vsble ace voxel Camera ceter. Calculato o the ceter camera To decde the ceter camera C, we search vsble camera at a voxel ace. I voxel s see at the,, ad camera, ceter camera wll be, le Fgure. C C C C 0 0 C C C C sble ceter camera Fgure : Calculato o ceter camera. sble camera Ivsble camera. Calculato o optcal ray We calculate optcal ray rom vsble ceter camera. At rst, ut vector s calculated wth Eq. (). C = () C Where, C s vsble ceter camera ad s vsble ace voxel. The Fgure shows that s ut vector o the optcal ray at a vsble ceter camera, s ace voxel ad s the sde voxel o.

4 sble ceter camera Optcal ray : ace voxel : sde : voxel o ace voxel Fgure : oxels o the optcal ray.. Calculato dssmlarty o the optcal Ray To decde the threshold value, we calculate dssmlarty o the optcal ray. I the covetoal voxel colorg method, dssmlarty was calculated rom vsble ace voxel. But ths paper, order to solve the sgle-xed threshold problem, dssmlarty s calculated rom ot oly ace voxel but also sde voxel o optcal ray the Fgure. sble mages C - C C + cosst( ) model error ad the voxel s elmated. The other case, the s cosdered as the ace voxel o real obect ad the voxel s remded. Ater ths process s perormed teratvely, the oly real ace voxel s remaed at the optmal threshold.. Decso o voxel elmato usg graph cut We used the graph cut method to ally decde ace voxels. We classed ace voxel to two categores, Opaque ad Carvg odes as Fgure (a). The result s show Fgure (b). We used the E() to mmze the eergy o ace voxels. The Eq. () represets eergy ucto. E( ) = D ( ) + {, } (, ) q q {, q } N () I the above eergy ucto, D ( ) s the expese o data ad ( {, }, ) s the expese o q q smoothg. D ( ) ca be dvded to two cases. Oe s that was assged to Opaque, the other s Carvg. Opaque Opaque cosst( ) Fgure : Calculato o dssmlarty or voxels o optcal ray. Dssmlarty s calculated betwee ace voxel ad sde voxel by usg the photocosstecy. The relato o photo-cosstecy ad dssmlarty s as ollowg Eq. (). a cosst( ) = () dssmlarty( ) + Where, cosst( ) s photo-cosstecy value, a s arbtrary costat, ad dssmlar ty( ) s dssmlarty value. Ad dssmlarty( ) ca be represeted as ollowg equato, Eq. (). red red gree gree blue blue { µ µ + µ µ + µ } dssmlar ty( ) = µ r, () Where,, represets the dex o ace ad sde voxels. red gree blue µ, µ, µ are the average value o RGB o ace voxel. The dssmlarty o ad o the optcal ray s calculated. I the dssmlarty o s larger tha, the s cosdered as the Carvg Carvg Fgure : Costructo o graph cut or voxel colorg. (a) Costructo o graph. (b) oxel labelg resultg graph cuts. I Eq. (), the expese o data term has the ollowg orm whe s assged to Opaque. D ( ) = () 0 cosst( ) > avg _ cosst cosst( ) avg _ cosst & cosst( ) cosst( ) cosst( ) avg _ cosst & cosst( ) < cosst( ) Where, the avg_cosst s photo-cosstecy value o all ace voxels as Eq. () avg _ cosst = N cosst ( ) () I Eq. (), the codto s cosst( ) > avg _ cosst, the expese o ace voxel s assged a low value, 0, ot to cut the voxel at the graph because the

5 photo-cosstecy o s hgh. I the codto s cosst( ) avg _ cosst, the expese s decded by cosderg the photo-cosstecy o sde voxel o the optcal ray. That s, the codto s cosst( ) cosst( ), the codto s assged lower value,, more tha cosst ( ) < cosst( ) because the photo cosstecy o s larger tha. I the codto s cosst ( ) < cosst( ), the hghest expese value,, s assged. I s assged to Carvg level, the expese value relatoshps are oppostely rom the Opaque. Expese o smoothg term, {, }(, ) q, s as q the ollowg, Eq. (). ( {, q }, q 0 ) = = q q () Where, the q meas -coupled eghbor voxel o. To d the mmum eergy, we used the graph cut algorthm that s proposed by Kolmogrov[]. Expermet I ths expermet, the color CCD camera, JAI C- S00 was used. The acqusto mage s bt colors, ad ts sze s 0*0. We used the sual C++ or compler ad the OpeGL to dsplay D mage. The Petum computer was used or smulato. We acqured the slhouette mages rom a real D obect. The mages are 0 mage sldes wth the agle o about. Followg mages are some o the acqured mages, the Fgure 0. We used the mages or the put mages. (a) I (b) GC_Th0 (c) GC_Th Table : Expermetal codtos. (d) GC_GC_Th0 (e) GC_GC_Th () OTC Algorthm olume Itersecto Geeralzed oxel Colorg Geeralzed oxel Colorg Geeralzed oxel Colorg Geeralzed oxel Colorg Optmal oxel Colorg Threshold Graph Cuts -- The expermetal codtos (b), (c) used the geeral voxel colorg method o Culbertso ad Malzbeder[], ad the threshold value o dssmlarty was set to 0 ad. Threshold meas dssmlarty o the ace voxels. I the threshold value s small, the photo-cosstecy s hgh, ad the other case the photo-cosstecy s low. The relato o betwee photo- cosstecy ad dssmlarty s verse proporto. We also appled the expermetal codto (d), (e) to graph cut method at the same codto (b), (c). The expermetal codto () s optmal threshold method usg voxel colorg. (a) (b) (c) Fgure 0: Iput mages. Expermetal codtos whch are used or evaluatg the proposed voxel colorg method are show the Table. I the Table, I meas the volume carvg method o Szels[] ad s used or crtero to evaluate the eect o optmal threshold method. (d) (e) () Fgure : Depth map o the recostructed o expermetal codtos. (a) C, (b) GC_TH0, (c) GC_TH (d) GC_GC_TH0, (e) GC_GC_TH, () OTC. We show the recostructed results by usg depth map o each expermetal codto, Fgure 0. Fgure0 (a) s the recostructed shape usg I. It shows model errors because o cocave ace. I expermetal codtos (b), (c), the threshold value s large, the modelg result ca be smlar to the real obect, but cocave model error s large. I the threshold s small, cocave model error ca be small, but the precso o the recostructo s low. Expermetal codtos (d) ad (e) are smlar to (b) ad (c), addtoally graph cut method was appled. The result o codtos (d) ad (e) shows that the ace ose s elmated comparg wth codtos

6 (b) ad (c). But model error was large. Fgure 0() was show the result o usg optmal threshold at the codto (). We decreased the model error by usg the optmal thresholdg method, ad creased the stablty o recostructo by usg the graph cut method. Fgure shows the average dssmlarty o recostructed D shape by usg the expermetal codtos show Table. We ow that the smaller the dssmlarty value s the closer the voxel s. Ad also we oud optmal threshold at the mmum dssmlarty. Avg. Dssmlarty. 0 comparso o dssmlarty or each voxel colorg algorthm 0 terato G C_Th0 G C_Th G C_GC_Th0 G C_GC_Th MT C Fgure : Comparso graph o dssmlarty or expermetal codtos. I ths paper, proposed algorthm s better result tha coveto method. Coclusos We proposed the mproved searchg optmal threshold method usg the voxel colorg algorthm or the mage-based D shape recostructo. The proposed voxel colorg algorthm preseted good result comparg wth covetoal voxel colorg algorthm usg the sgle-xed threshold value. The threshold s approached to the optmal value as the dssmlarty o voxel s small. The process s terated to d out the optmal threshold. Ad to elmate the ose o ace voxel, we appled the graph cut method. Graph cut algorthm was used to mmze eergy, ad rregulartes o ace were elmated by eergy o smooth term. Expermets were perormed wth covetoal ad proposed method uder varous codtos. I covetoal voxel colorg algorthm, the trade-o problem o accuracy ad stablty was caused by the sglevalued threshold o dssmlarty. We resolved the problem by usg optmal threshold ad graph cut method. The recostructo ececy o proposed algorthm s much better tha covetoal oe. REFERENCES [] H. Baer, Three-dmesoal modelg, It. Jot Co. o Artcal Itellgece, pp. -,. [] W. N. Mart ad J. K. Aggarwal, olumetrc descrpto o obects rom multple vews, IEEE Tras. o Patter Aalyss ad Mache Itellgece, vol., o., pp. 0-,. [] M. Potmesl, Geeratg octree models o D obects rom ther slhouettes a sequece o mage, Computer so, Graphcs, ad Image Processg, vol. 0, pp.-,. [] R. Szels, Rapd Octree Costructo rom Image Sequeces, Computer so, Graphcs, ad Image Processg, vol., o., pp. -, Jul... [] S. M. Setz ad C. R. Dyer, Photorealstc Scee Recostructo by oxel Colorg, Proc. Compurter vso ad Patter Recogto Co., pp. 0-0,. [] W. B. Culbertso ad T. Malzbeder, Geeralzed voxel colorg, Proc. o the ICC, pp. 00-,. []. Kolmogorov ad R. Zabh, What Eergy Fuctos ca be Mmzed va Graph Cuts?, IEEE Tras. o Patter Aalyss ad Mache Itellgece, 00 Acowledgmets Ths wor was supported by "Research Ceter or Logstcs Iormato Techology (LIT)" hosted by the Mstry o Educato & Huma Resources Developmet Korea.