Normal Map Acquisition of Nearly Flat Objects Using a Flatbed. Scanner

Transcription

1 Normal Map Acquisitio of Nearl Flat Objects Usig a Flatbed Scaer Rogjiag Pa 1, Vaclav Skala 1 School of Computer Sciece ad Techolog, Shadog Uiversit, Jia, Chia, parj@sdu.edu.c Facult of Applied Scieces, Uiversit of West Bohemia, Ple, Cech Republic skala@kiv.cu.c Abstract We propose a ew efficiet techique for acquirig ormal maps of earl flat objects usig a flatbed scaer. The techique requires o hardware modificatio or dedicated calibratio. Our setup cosists of a covetioal flatbed scaer ad a clamp o which a plaar checkerboard is glued. We take four scas of the object mauall rotated b roughl 90 degrees. The scas are the registered b subpiel checkerboard corers. Detailed ormal map of the scaed surface is obtaied b solvig a liear least squares problem. Kewords ormal map recostructio, subpiel corers, flatbed scaer, checkerboard 1 Itroductio Ma small atiques such as seals ad cois have a earl flat surface as show i Figure 1. The surface details, icludig scratches ad impressios, are importat iformatio for archaeological stud. However, eistig rage scaers have difficulties i capturig such subtle details o the surface due to their low resolutio. Although microgeometr capture sstems usig elastomeric sesor are proposed recetl [1][], the are still i prototpe stage ad require special sesor materials. Images with ormal maps, which cotai a color ad a surface ormal chael for each piel, are a tpe of data lig betwee simple D images ad full 3D models. I cotrast to photographs ad 3D models, ormal maps make sharp chages i ormal directio clearl visible [3]. Images with ormal vectors ca also be used to produce detailed ad uderstadable illustratios of comple real-world objects [4]. Figure 1: Two small objects which have a earl flat surface I this paper we itroduce a method to acquire ormal maps of earl flat objects usig a flatbed scaer. Nowadas flatbed scaers are low cost ad mature products to digitie documets ad photographs at high resolutio. The hardware setup cosists of a covetioal flatbed scaer ad a clamp o which a plaar checkerboard patter is glued. Our cotributios 1

2 iclude: A ormal maps acquisitio sstem that requires o hardware modificatio or dedicated calibratio. A robust algorithm for registerig multiple scas b virtue of subpiel checkerboard corers. Easil cotrolled ad accurate lightig directios for four scas rotated b 90 degrees with a clamp. Related Work Normal maps are widel used i real-time rederig [5] ad geometric modelig [6]. It is a techique used for fakig the lightig of bumps ad dets, addig details without usig more polgos. Normal maps ca be etracted from a high resolutio modeled mesh ad ehace the appearace ad details of a low polgo model. For real world objects, ormal maps are ofte recostructed from multiple images uder varig illumiatio based o the priciple of photometric stereo [7]. Digital cameras ca be used to take multiple images from the same view poit illumiated with differet light sources of kow positio. Uder the assumptios of perfect diffuse shadig, per-piel surface ormals of a object ca also be obtaied from multiple scas at differet orietatios with a flatbed scaer. Brow et al. [3] obtaied a ormal map of a fresco fragmet from four scas rotated b approimatel 90 degrees with a flatbed scaer. The scaed images are registered via a affie trasform usig a ormalied cross-correlatio. The also required a dedicated calibratio phase usig a diffuse plae iclied at kow agles with respect to the scaer platform. Clarkso et al applied photometric stereo to four scas of the paper usig a commodit flatbed scaer [9]. The used a prited bo o the test pages to alig differet scas ad recovered each surface ormal s projectio ito -plae up to a scale, (, ) T. Pitus et al etracted ormals ad albedo of objects usig a two-bulb flatbed scaer [8]. The activated each of the bulbs separatel b modifig scaer hardware or usig private software uavailable to others. The developed a registratio method b simultaeous registratio ad photometric stereo. However, chromatic differece betwee bulbs ad imperfect mechaical repeatabilit of the scaer are problematic for photometric stereo. The modeled the scaer s light as poit light sources. Istead, flatbed scaers cotai a liear light source, which is placed at a fied agle with respect to the movig CCD [3] [8]. Our work is a improvemet over Brow s [3] ad Clarkso s [9] methods. Compared with Brow s method, our method does ot require calibratio procedure ad is more adaptable. I cotrast to Clarkso s method, we recover the ormalied 3D surface ormals, (,, ) T. As will be eplaied i et sectio, it has a importat effect o the etracted ormals to rotate each sca b approimatel 90 degrees. To obtai four scas rotated b 90 degrees as accurate as possible, we make use of a clamp which ca be easil placed alog the side of the scaer plate. 3 Acquisitio Workflow Photometric stereo usuall requires at least three images of the surface to estimate surface ormals [10]. Tpicall, more tha the miimum three images are used, providig oise reductio. To make it easier ad robust, we take scas at orietatios 0,90,180 ad 70 although

3 other orietatios could be used as well. I order to make the positio of the object as accurate as possible withi the checkerboard, we utilie a clamp from umerical cotrol machies (Figure ) to fi the scaed object. A plaar checkerboard patter is glued o the clamp ad utilied i the multiple scas registratio. However, other fiig sstems that keep the object colliear with the scaer scaig surface ca be used as well. Figure : A clamp to fi the scaed object 3.1 Multiple Scas Registratio We make use of checkerboard corers to register multiple scas. A custom-desiged user iterface was provided for a semi-automated subpiel corer etractio. At least three corers are selected o the rectagular checkerboard patter. Afterwards, subpiel detectio techiques are applied to corers for refiig their locatios to subpiel accurac [11]. Assume that i two scas there eist correspodig corers { i } ad { i }, i 1,,, such that the are related b: R t (1) i i where R is a stadard rotatio matri ad t is a D traslatio vector. Solvig for the optimal trasformatio tpicall requires miimiig S i Ri t () i1 The two sets of corers { i } ad { i } should have the same cetroid. Defiig i, i i ad i1 S i, i i, the Eq. () ca be reduced to i1 i Ri X RY (3) F i1 where X ad Y are matrices whose colums are formed from the corer positio vectors, the Frobeius orm. Miimiig T X RY is equivalet to maimiig Trace F YX R is F ( ). Accordig to the Lemma 1 i [1], the optimal rotatio matri is 1 0 R T 0 det( ) UV ad the traslatio vector is t R where USV T is the Sigular Value Decompositio 3

4 (SVD) of XY T, U ad V are orthogoal matrices, S is a diagoal matri. After aligig the other three scas to the first sca, we cut out the image regios that are out of iterest. 3. Normal Recostructio The light source of a flatbed scaer is liear ad placed at a fied agle α with respect to the scaer plate. Each poit o the scaed surface has a ormal ad albedo ρ. For the scaig take at orietatio 0, we defie a left-haded coordiate sstem as show i Figure 3(a) whose origi is at the surface poit cosidered, -ais is parallel to the scaer s liear light source, -plae is the scaer plate ad -ais poits dow. The light source is approimated b a lie segmet etedig from -l to +l i the -ais directio ad is offset b a i the -directio, b i the -directio. We defie =(,, ) T as a ormalied surface ormal at the poit, (, a, b) T l is the ormalied lightig directio vector alog the liear light source. The a b observed itesit of such a surface poit is the: l l a b l 1 I, ( ) ( ) 0 l d d l l a b d s l a b a b a b l where 1 l l a b s l d l a b a b. α b a α b -a (a) 0 (b) 180 α b -a α b a (c) 90 (d) 70 Figure 3: Four scaig take at differet orietatios 4

5 Aalogousl, scaig the same surface poit with the object rotated b 180 whose ormalied lightig directio vector alog the liear light source is l (, a, b) T a b as show i Figure 3(b) leads to: l l a b l 1 I, ( ) ( ) 180 l d d l l a b d s l a b a b a b Scaig the same surface poit with the object rotated b 90 whose ormalied lightig directio vector alog the liear light source is l ( a,, b) T a b as show i Figure 3(c) leads to: l l a b l 1 I, ( ) ( ) 90 l d d l l a b d s l a b a b a b Scaig the same surface poit with the object rotated b 70 whose ormalied lightig directio vector alog the liear light source is l ( a,, b) T a b as show i Figure 3(d) leads to: l l a b l 1 I, ( ) ( ) 70 l d d l l a b d s l a b a b a b We ma arrage the four equatios ito a matri equatio: 0 ta 1 I 0 sb 0 ta 1 I 180 sb ta 0 1 I 90 sb ta 0 1 I70 (4) where ta sb = a b I. Equatio 4 ca be solved usig liear least-squares. Its solutio is I ta I I, I + I + I + I, sb = ta 5 sb= Accordig to [8], the lightig geometr is approimated with at this surface poit is obtaied ad ormalied. 3.3 Mea Curvature Estimatio Fiall, the ormal Oce we have obtaied ormal vectors at ever surface poit, we estimate the curvatures of the surface similarl to [4]. Assumig the surface has the eplicit form (, ), we first trasform the ormal =(,, ) T ito =( /, /,1) T. The mea curvature is half the trace of

6 1 I II where I is the first fudametal tesor ad II is the secod fudametal tesor uu I= uv u u II= v u vu vv u v v v, ad ( uv, ) are the aes of a arbitrar coordiate sstem i the taget frame. We take u (1,0, ) T ad v (0,1, ) T such that u =0 ad v =0. The derivates of the ormal are approimated usig cetered fiite differeces which is secod-order accurate: u i1, j i1, j i, j, v i, j 1 i, j 1 i, j 4 Results I our eperimets, a HP ScaJet G4050 was used at a 400 dpi ad images were saved i RGB mode. Although HP ScaJet G4050 offers si-color scaig b two light sources which is emploed i [8], we perform the scas i three-color mode b oe light source. The itesit image for recostructio was computed as I=0.99R+0.587G+0.114B. We tested the algorithm o a coi ad a seal show i Figure 1. Figure 4 ad 5 show the acquired ormal maps ad mea curvature maps, respectivel. The ormal maps are displaed as RGB images where the RGB compoets correspod to the, ad values of the surface ormal, respectivel. I our implemetatio, maps from (-1.0,1.0) to red (0,55), maps from (-1.0,1.0) to gree (0,55) ad maps from (0.0,1.0) to blue (0,55). The curvature maps are displaed as RGB images where positive curvatures correspod to red ad egative to blue. The absolute values of curvatures are mapped to saturatio i HSV color sstem. As see from the figures, ormal maps reveal fie surface detail that is difficult to see i color images. 6

7 Figure 4: From left to right: color image, ormal map ad mea curvature map of a coi Figure 5: From left to right: color image, ormal map ad mea curvature map of a sculpture. To evaluate the accurac of our method, we make a check o the itegrabilit costrait which boils dow to check the mied secod partials are equal,. So we epect that ( ) ( ) should be small at each poit. The partial derivatives are estimated umericall ad the differeces betwee them are show i Figure 6. Sice most parts of the sculpture are flat, the differece betwee partial derivatives is small ecept at the egraved parts. 7

8 (a) Differece betwee partial derivatives for a coi (b) Differece betwee partial derivatives for a sculpture Figure 6: Validatios of the itegrabilit costrait We also compute the residual error i the least-squares solutio of Eq. (4), which provides a check o our measuremet as well. Figure 7 shows the RMS residual errors for the coi ad the sculpture. The average RMS error is 10.1 for the coi ad 1. for the sculpture. (a) a coi Figure 7: RMS residual error (b) a sculpture Our algorithm is implemeted i C++ laguage ad the eperimets were made o a 8

9 otebook with 1.6 GH Itel Core Q70 processor ad 3GB RAM memor. It takes about 10 secods to etract the ormal map ad mea curvature map for images of 9 M piels. 5 Coclusio I this paper we preseted a method of acquirig ormal maps of earl flat objects with a flatbed scaer. The method computes surface ormals from four scas with differet object rotatios based o the priciple of photometric stereo. B eploitig the flatbed scaer s high resolutio ad the simple setup, the approach is applicable to small objects havig a earl flat surface such as i archaeological domai. As stated i [8], the limitatio of our techique is due to the low depth-of-field of the scaer ad the depth of geometr we ca hadle is approimatel a few millimeters. I the future the proposed techique will be cosidered for depth recostructio of historical artifacts b itegratig the ormals. 6 Ackowledgmets Authors are grateful to colleagues for costructive commets ad suggestios, to aomous reviewers for their costructive commets ad hits that helped to improve the mauscript sigificatl. This work was supported b Chia-Cech Scietific ad Techological Collaboratio Project (40-8), Shadog Natural Sciece Foudatio of Chia (Grat No. ZR010FM046) ad Project of Miistr of Educatio of the Cech Republic (project LH1181). Refereces [1] Micah K. Johso, Forrester Cole, Alvi Raj, ad Edward H. Adelso. Microgeometr capture usig a elastomeric sesor. ACM Tras. Graph. 30, 4, Article 46 (Jul 011), 8 pages. [] Micah K. Johso, ad Edward H. Adelso. Retrographic sesig for the measuremet of surface teture ad shape. I Proceedigs of the IEEE Computer Societ Coferece o Computer Visio ad Patter Recogitio, 009, [3] B. Brow, C. Toler-Frakli, D. Nehab, M. Burs, D. Dobki, A. Vlachopoulos, C. Doumas, S. Rusikiewic, ad T. Weric. A sstem for high-volume acquisitio ad matchig of fresco fragmets: Reassemblig Thera wallpaitigs. ACM Tras. Graph., 7(3):84:1-9, 008. [4] Core Toler-Frakli, Adam Fikelstei, ad Smo Rusikiewic. Illustratio of comple real-world objects usig images with ormals. I Proceedigs of the 5th iteratioal smposium o No-photorealistic aimatio ad rederig (NPAR '07). ACM, New York, NY, USA, , 007. [5] J. F. Bli. Simulatio of wrikled surfaces. SIGGRAPH Comput. Graph., vol. 1, o. 3, pp. 86 9, [6] Tai-Pag Wu, Chi-Keug Tag, Michael S. Brow, ad Heug-Yeug Shum ShapePalettes: iteractive ormal trasfer via sketchig. ACM Tras. Graph. 6, 3, Article 44 (Jul 007). [7] Robert J. Woodham. Photometric stereo: A reflectace map techique for determiig surface orietatio from image itesit. I Proc. d SPIE Aual Techical Smposium, vol. 155, 1978, pp [8] Ruggero Pitus, Thomas Malbeder, Oliver Wag, Ruth Bergma, Hila Nachlieli, Gitit Ruckestei. Photo Repair ad 3D Structure from Flatbed Scaers Usig 4- ad -Source Photometric Stereo. Computer Visio, 9

10 Imagig ad Computer Graphics. Theor ad Applicatios, Spriger Berli Heidelberg, Volume 68, page 36-34, 010. [9] William Clarkso, Tim Werich, Adam Fikelstei, Nadia Heiger, J. Ale Halderma, ad Edward W. Felte. Figer pritig Blak Paper Usig Commodit Scaers. I Proceedigs of the th IEEE Smposium o Securit ad Privac (SP '09). IEEE Computer Societ, Washigto, DC, USA, [10] Robert J. Woodham. Photometric Method for Determiig Surface Orietatio from Multiple Images. Optical Egieerig, 19(1): , [11] Gar Bradski, Adria Kaehler. Learig OpeCV: Computer Visio with the OpeCV Librar. O'Reill Media; 1st editio (October 1, 008) [1] Adri Mroeko, Xubo B. Sog. O the closed-form solutio of the rotatio matri arisig i computer visio problems. CoRR abs/ ,