The Great Buddha Project: Modeling Cultural Heritage for VR Systems through Observation

Transcription

1 The Great Buddha Project: Modelng Cultural Hertage for VR Systems through Observaton Katsush Ieuch, Atsush Naazawa, Kazuhde Hasegawa, Taesh Ohsh Insttute of Industral Scence, The Unversty of Toyo {, naazawa, -hase, Abstract Ths paper overvews our research on dgtal preservaton of cultural assets and dgtal restoraton of ther orgnal appearance. Geometrc models are dgtally acheved through a ppelne consstng of scannng, regsterng and mergng multple range mages. We have developed a robust smultaneous regstraton method and an effcent and robust voxel-based ntegraton method. On the geometrc models created, we have to algn texture mages acqured from a color camera. We have developed two texture mappng methods. In an attempt to restore the orgnal appearance of hstorcal hertage objects, we have syntheszed several buldngs and statues usng scanned data and lterature survey wth advce from experts. 1. Introducton Currently, a large number of cultural hertage objects around the world are deteroratng or beng destroyed because of natural weatherng, dsasters and cvl wars. Among them, Japanese cultural hertage objects are qute vulnerable to fres and other natural dsasters because most of them were constructed of wood and paper. One of the best ways to prevent them from loss and deteroraton s to dgtally preserve them. Dgtal data of hertage objects can be obtaned by usng computer vson technques. Once these data have been acqured, they can be preserved permanently, and then safely passed down to future generatons. In addton, such dgtal data s sutable for many applcatons, ncludng smulaton, restoraton, and creatng mult-meda contents. Such dgtal contents can be vewed through the nternet from anywhere n the world, wthout movng the objects nor vstng the stes. We have been worng to develop such dgtal archval methods by usng computer vson and computer graphcs technologes [1]. Smlar projects nclude: Stanford s Mchelangelo Project [2], IBM s Peta Project [3], and Columba s project [4], to name a few. Our project has a number of unque features; among them s ts ablty to dgtze relatvely large objects outdoor such as the Kamaura great Buddha, and Camboda s Bayon. Ths presents several challenges. Also, our project conssts not only of geometrc modelng but also of photometrc and envronmental modelng, as shown n Fgure 1. Geometry Photometry Partal observaton Envronment Color mages Illumnaton Fgure 1. Three components of our project The remander of ths paper s organzed as follows. Secton 2 descrbes the outlne of the geometrc ppelne developed, a parallel smultaneous algnment algorthm and a parallel voxel-based mergng algorthm. Secton 3 descrbes methods to algn observed textures from a dgtal camera wth a range data for texture mappng. Secton 4 reports our efforts to restore the orgnal appearance of these objects usng acqured dgtal data and the lterature survey. Secton 5 summarzes ths paper. 2. Geometrc Modelng Several computer vson technques, such as tradtonal shape-from-x and bnocular stereo, or modern range sensors, provde cloud of pont nformaton. The cloud of pont nformaton certanly carres three-dmensonal nformaton pertanng to observed objects. However, there s no structural nformaton among these ponts. Namely, there s no nformaton to represent adjacency among the ponts. The frst step of geometrc modelng s to convert the cloud of ponts nto a surface representaton such as a mesh model.

2 Snce each observaton provdes only partal nformaton, we have to combne these partal mesh representatons nto a whole geometrc mesh representaton. Thus, the second step n geometrc modelng s to algn these meshes so that ther correspondng parts overlap one another (algnment). To accomplsh ths step, we have developed a smultaneous algnment method to avod accumulaton of errors [3]. The thrd geometrc modelng step s to ntegrate the pre-regstered multple range mages and to compose one complete geometrc model, a step usually called 'mergng'. The procedure can be consdered as extractng one surface from multple overlapped surfaces. In the mergng procedure, t s mportant to mae the ntegraton framewor robust aganst any nose whch may be n the scanned range mages and can also be nherted from the regstraton procedure. to algn a very large data set [12]. The fundamental algorthm, due to the utlzaton of a graphcs hardware, s slghtly dfferent from the one descrbed above [11]. The correspondng pars are searched along the lne of sght to use the graphcs hardware. Assume that a base range mage s the model mage and that a correspondng range mage s the scene mage. In Fgure 4, a model mage s depcted as ponts and a scene mage s shown as meshes. When a lne extended from a vertex of model mage along the lne of sght crosses a mesh of a scene mage, the ntersectng pont s the correspondng pont. In order to elmnate false correspondences, f the dstance between correspondng ponts s larger than a certan threshold value or f the angle of lnes of sght s beyond a threshold angle, the correspondence s removed. Ths correspondence search s computed for every combnaton of range mages, usng the Z-buffer capablty of the graphcs hardware. Scannng Algnment Mergng Fgure 2. Three steps of Geometrc Modelng. 2.1 Smultaneous Regstraton Method To avod accumulaton of errors, we have developed a smultaneous algnment method [11]. Tradtonal sequental methods [5,8,9] such as Iteratve Closest Pont Algorthm (ICP) algn these meshes one by one, and progressvely algn a new partal mesh wth prevously algned meshes. If a few partal meshes can cover an object, the accumulaton of algnment errors s relatvely small and can be gnored. A sequental algnment wors well. However, some cultural hertage objects are very large; we need more than one hundred vews. In such cases, the error accumulaton caused by sequental algnment methods s very large. Thus, by consderng algnment of all the pars smultaneously, we algn all partal meshes so as to reduce the errors among all the pars smultaneously [7,10,11]. Our smultaneous algorthm [11] employs ponts and planes to evaluate relatve dstance as does the Chen and Medon and Gagnon et.al. methods [8,10]. Further, to avod the effects from the outlers, the algorthm uses M-estmator, n partcular, the Lorentan functon as the error measure [13]. Currently, we are extendng the smultaneous algnment algorthm nto a a parallel mplementaton so as to be able Fgure 3. Search correspondng ponts The error measure between correspondng ponts s the dstance between the pont and the plane. Let the vertex of model mage be x r and let the pont correspondng to the vertex be y r, the error measure between the pars s wrtten as n ( y x) (1) where n s the vertex normal of x r. The transformaton matrces of the model and scene mages are computed so that ths error measure s mnmzed. The error evaluaton functon s rewrtten as r r r r r RM n {( RS y + ts ) ( RM x + tm )} (2) Here, the rotaton matrx and the translaton vector of the r t M,t model and scene mage are R, M, Rs s, respectvely. Then, the dstance between the model mage and the scene mage s expressed as 2 r r r r r 2 ε = mn (3) ( RM n {( RS y + ts ) ( RM x + tm )}) R, t If t s assumed that the angles of rotaton are small, the rotaton matrx R can be approxmated as

3 1 c3 c2 4 R = c3 1 c1 c2 c1 1 The translaton vector s expressed as ( t t t ) t = 5 x y z After some algebrac manpulatons [16], (3) s rewrtten as 2 ε = mn s δ j A δ j s j 2 6 = n x y ) 7 ( j Fgure 4 Removng redundant dependences We can defne and remove redundant dependences n the range mages. A par of range mages that does not satsfes all these three condtons wll be removed as redundant relatons. A j { { { { { = 0...0C + j C The boundng-boxes of two range mages overlap each j 6 1 6( l 1) 1 6 j other. 6( l j 1) The angle between ray drectons of two range mages s less than a threshold value. n yj C = 9 3. Two range mages are adjecent to each other. j n = ( m0 Lmn 1 δ ) 10 ( c c c t t t ) m = 11 x where the number of range mages s n. By (6) δ s wrtten as 1 y T T δ = A j Aj A j s 12 j For the parallel mplementaton of the algnment algorthm, both tme and memory performance have to be consdered. The smultaneous algnment algorthm, orgnally desgned, requres all range mages to be read nto memory; even when the computaton s dstrbuted over a PC cluster, the amount of memory used on each PC s not reduced. We wll remove redundant or wea data dependency relatons. Fgure 4 shows overlappng data-dependency relatons. Each node n the graph represents one range data and each arc represents an overlappng dependency relatons among range data. In the left graph, all the data are overlappng each other. In order to compute algnment of one data, we have to read, nto a PC memory, all the remanng range data. By removng some of redundant overlappng dependences, we can transform the orgnal graph nto a smpler one as shown n the rght fgure. By usng ths smpler relatonal graph, we only need adjacent data wth respect to a vertex for algnment of a vertex, and can ncrease the effcency n memory usage. z Under the assumpton that ntal postons of two range mages are accurately estmated, Requrement 1 s satsfed when a suffcent overlapped regon exsts between two range mages. Requrement 2 s satsfed when two vews are relatvely near. As the sde effect, ths requrement also reduces the possblty of false correspondences. Snce our method utlzes the ray drecton to establsh correspondences, by settng the threshold, as θ = 90, we can avod correspondence pars between one and the other sde. Requrement 3 s set up for removng non-adjacent relaton sequentally. For example, as shown n Fgure 5, f the length from I to n s larger than the length from to 0 I I m I n (l 0m < l0n), the arc between I 0 and I n s removed. Here, the dstance s evaluated from the center of a range mage s surface. Fgure 5 Defnton of the dstance measure For our parallel algorthm to read data only related, we can set up a relatonal table as shown n Fgure 6. The row of the table represents scene mages, and the column represents model mages. Each element d represents dependency between a scene, j and model mage,. 1 j and have a dependency relaton d j, = 0 otherwse Snce each par of a scene and model mage can be

4 evaluated ndependently n equaton (6), roughly speang, we wll assgn those pars to nodes of a PC cluster so as for each node to have an average number of pars. Here, we assgn pars to nodes based on scene mages. t = ts v j + tm ( v d ) + tc n0, j ( n0, + n ) n0, j ( n0, + n ) (16) where n scene mages are assgned to node from the number n through n. Here, v s the number 0, 0, j + n of vertces; t s s the computaton tme for the number of vertces ncluded n scene mages, and t m s computaton tme for the number of vertces ncluded n model mages. t s ndependent of the number of vertces, and c because t s very small, c t can be consdered to be 0. c Assumng that the computaton tme for the number of scene mages and model mages t and t s equal to, s m tv (16) s rewrtten as Fgure 6 Par Assgnment table t = tv v j + ( v d ) (17) n0, j ( n0, + n ) n0, j ( n0, + n ) The amount of memory used depends on the number of vertces ncluded n range mages requred for each node. By assumng the number of vertces of each range mage s roughly equal, the amount of memory requred, M for each node s proportonal to the number of range mages assgned to the node r, the number of 1 s n the assgnment. More precsely, M = M r r + M 13 c s the average of the amount of data contaned n each M r range mages, and M s the amount of memory used for c computaton and ndependent to the number of range mages (constant). The number of range mages read nto each node s expressed as unon of sets of model mage and scene mage. r mod el, r scene, r rmodel, rscene, t = t = t (18) s m v That s, the computaton tme s proportonal to the number of vertces computed. Then, n consderaton of a scene mage beng assgned to multple nodes, assumng that the number of vertces of the dvded scene mage s the average of the number of vertces of all range mages v, the number of vertces that are computed on average v node s expressed as follows. v = v j ( v j d j ) vaverage ( nnode 1) +,, + / nnode 0 j n 0 j n 0 n (19) n expresses the number of nodes. By usng ths, node v expressed n (19) as an ndex, we wll assgn nodes to the scene mages. = (14) We mplemented our method as a server/server system. The procedures of the computaton s are as follows For the sae of computatonal effcency descrbed below, we wll re-arrange the table. The range mages, read nto each node, are unons of model and scene mages to be read n Eq (14). The assgnments to the nodes are based on the scene mages, as shown n Fgure 8; t s preferable to read the same model mages wth respect to the scene mages We defne the dstance l between two scene mages l (, j) as follows:, j ( d ), d j, d xor d, j,, j = n ) 0 n The column of the table, the scene mages, are sorted n the descendng order of ths dstance under the assumpton that the adjacent scene mages are more lely to share the same set of model mages. Algorthm Procedure of Parallel Algnment /* Chec correspondence of all par of */ /* the range mages */ CreateParTable; /* Create the lsts of the fles for each processor */ CreateFleLsts: whle(error > threshold){ ( (15) /* Clent Process*/ for( = 0; < nimage; ++) for(j = 0; j < nimage; ++j) f(lst[][j]){ CorrespondenceSearch(, j); CalculatonEachMatrx(, j); } /* Server Process */ Averagng computatonal tme CalculatonMatrx(all); Next, we wll mae even dstrbuton of the computatonal /* Server & clent process */ load over a PC cluster. The computatonal cost of the I UpdatePoston; node, s wrtten as: } t

5 The server program holds boundng-boxes and transformaton matrces from ntal poston to current poston of all range mages, checs all combnatons, and creates the lst of computatons for each node. The lst of computatons for each clent s computed at the begnnng of the entre teratons based on the relatonal table usng the algorthm descrbed above The clent program receves the lst and reads the requred range mages nto memory. T Then, each clent computes the matrces A and j Aj T A js n (12) ndependently, and sends the matrces to j the server program. The server program computes the rotaton matrces and translaton vectors of all range T mages from the matrces and T A receved j Aj A jsj from the clent program. The results are appled to all server/clent data. Then, each teraton process s contnued untl the error falls below a certan threshold value. Computaton tme s defned as the tme taen for one teraton, and an average of tme of all teratons s used for the evaluaton. We used the 20 range mages created from hand model (model A) and the 15 range mages measured the great Buddha of Kamaura (model-b) for tme evaluaton. Fgure 4 shows the result of ntal and smultaneous algnment of Kamaura Buddha (after 20 teratons). Fgure 7. Smultaneous regstraton result. We have observed the computaton effcency lnearly proportonal to the number of processors. In Table 1, the computaton tmes wth 1 processor and 16 processors are descrbed. The computaton tme wth 16 processors s approxmately 8.4 tmes faster than that wth 1 processor for the hand model, and 8.9 tmes for the Great Buddha of Kamaura. As for the memory usage, we can reduce the amount of memory used decreases as the number of processor ncreases. Table 1 Computatonal tme As shown n Table 2, our method can reduce the amount of memory used n approxmately 60% for the hand model and n approxmately 43% for the model of the Great Buddha of Kamaura. See [12] n more detals. Table 2 Amount of memory rato 2.2 Voxel-based Mergng Algorthm After all range mages have been algned, a volumetrc vew-mergng algorthm generates a consensus surface of the objects from them. Our method merges a set of range mages nto a volumetrc mplct-surface representaton, whch s converted to a surface mesh by usng a varant of the marchng-cubes algorthm [14]. Unle prevous technques based on mplct-surface representatons, our method estmates the sgned dstance to the object surface by determnng a consensus of locally coherent observatons of the surface [15,16,17,18]. We utlze octrees to represent volumetrc mplct surfaces, thereby effectvely reducng the computaton and memory requrements of the volumetrc representaton wthout sacrfcng accuracy of the resultng surface. We orgnally desgn software that merges the algned 20 range data. However, recent nput data s unpredctably huge, we decde to bult up a PC cluster to run ths mergng software; the cluster parallel-processes the mergng algorthm for savng the computaton tme and utlzng a large memory space of many PCs. We produced one ntegrated dgtal Great Buddha wth ths software. The whole data set conssts of 3.3 M ponts, and 5 M polygons that can be merged n approxmately 20 mnutes on the PC cluster. We have dgtally archved Japanese Buddhas, ncludng Asua, Kamaura, Nara, and foregn ones, ncludng Thaland s Wat S Chum and Camboda s Byon. We are contnung ths effort toward completng the world Buddha lbrary, as shown n Fgure 8, whch dgtally dsplay transtons of Buddha shapes n tme and regon. 3. Texture Mappng and Renderng The geometrc model s vtal nformaton regardng the cultural hertage objects because t enables us to analyze object n detals. In addton to the geometrc model, surface color dstrbuton (texture) s also very mportant for some nds of cultural propertes. Fgure 9 shows one of the Japanese natonal treasures, the 'Koumou-Ten' statue. Ths clay statue stll retans the surface colors that were panted at the tme the fgure was orgnally made. For such types of cultural propertes, we have to archve geometry nformaton, texture nformaton and ther

6 Afghanstan Chna Asua (9AD) Inda Kamaura (12AD) Thaland Camboda Nara (16AD) Japan Fgure 8. The World Buddha Lbrary (Current Status) relatonshp at the same tme. For ths purpose, we have developed two nds of texture mappng methods: calbraton-based and reflectance-edge based methods. ponts n mage coordnate x c ={x c,y c } can be easly calculated as: 3.1 Calbraton-based Method The problem of the texture mappng method s how to determne the relatonshp between mage sensors and geometrcal sensors. When a short-dstance range sensors can be used, as shown n Fgure 9, the most promsng method s to calbrate the geometrcal relatonshp between the mage sensor and the range sensor before scannng. Assume that the coordnate system of the mage sensor s (x c,y c ) and the correspondng pont n range mage s (X,Y,Z); the relatonshp between them can be descrbed as: The matrx C 34 represents the relatonshp between mage coordnate and world coordnate, and t can be calculated by scannng the calbraton box. Inversely, when we map the texture mages onto the geometrcal trangular mesh Xn={(Xn,Yn,Zn) 1<= n <=3}, the correspondng Fgure 9. The scannng scene of the Koumou-Ten x clay fgure usng the VIVID 910 c X + c Y+ c Z + c 11 n 12 n 13 n 14 c=, c31x c32y c33z c34 c21x c22y c23z c yc = c X + c Y+ c Z + c 31 n 32n 33 n For the modelng of the Koumou-Ten clay fgure, we used 60 range mages and color mages that taen at the same tme. Fgure 10 shows the geometrcal model (upper), the texture mapped result (lower) and synthess results under the dfferent lghtng condtons generated usng the texture mapped result (See Fgure 11)

7 Fgure 10. Geometrcal model of Koumou-Ten and texture-mapped model. returned strength provdes a reflectance measurement. A reflectance mage s a collecton of the strength of returned laser energy at each pxel. Ths reflectance mage s algned wth the range mage because both mages are obtaned through the same optcal recevng devce. Commonly avalable range sensors, ncludng ERIM, Preceptron, and our man sensor, CYRAX, provde ths reflectance mage. We employ ths reflectance mage as a vehcle for the algnment of range mages wth color mages [3,16]. Reflectance mages share characterstcs smlar to color mages due to the fact that both mages are somehow related wth surface roughness as shown n Fgure 9. Snce our CYRAX range scanner uses a green laser dode, reflectance edges can be observed along the boundary between two colors or materal boundares along dfference reflectance ratos for ths wavelength. Snce dfferent materals are of dfferent colors, a dscontnuty also appears n the color mages. Jump edges along small ranges n a range mage also appear as jump edges n a reflectance mage as well as n a color mage. Occludng boundares are observed both n reflectance mages and n color mages. Reflectance Materal dfference Jump edge Color Occluson Fgure 11 Syntheszed results under varous lghtng condtons (Sunrse->Daytme->Sunset). 3.2 Reflectance-Edge based Method One soluton for determnng the relatonshp between range and color mage s, as shown n the prevous secton, through calbraton usng a calbraton fxture. However, ths method requres that the range and color sensors be fxed on the fxture once the relatonshp s calbrated. Further, the calbraton-based method s accurate only around the poston occuped by the calbraton fxture. When a target object s very large, ths method becomes unrelable due to the lens dstorton. Thus, we also need a method that does not rely on calbraton. Generally speang, range sensors often provde reflectance mages as sde products of range mages. The returned tmng provdes a depth measurement, whle the Fgure 12. Reflectance and color edges. They share the smlar characterstcs. Pror to the algnments, we paste the necessary reflectance edges onto the 3D geometrc model. As mentoned above, snce occludng boundares vary dependng on the vewng drecton, edges along the occludng boundares are frst removed from the reflectance mages. On the other hand, edges along the current occludng boundares wll be estmated from the 3D geometrc model and the current vewng drecton. Our algorthm extracts them automatcally, and uses them for the algnment. We algn edges extracted from reflectance mages wth those n color mages so that the 3D poston error of those edges s mnmzed by teratve calculaton as shown n Fgure 13. Extracted edges are represented as a collecton of ponts along them. The algnment s done between 3D reflectance ponts on 3D geometrc model projected on the mage plane and 2D color edge ponts n the 2D mage. To establsh correspondence, the system fnds the color mage ponts that are nearest to the projected reflectance

8 Projecton to the mage plane Nearest pont 3D error z 3D edge pont Geometrc model method, we can obtan the confguraton P that mnmzes the error term and gves the relatve relatonshp between the camera and the range sensor. Fgure 14 shows the texture mapped Kamaura Buddha. Snce ths method mnmzes a non-lnear equaton, we need an ntal algnment. The ntal algnment s gven manually usng our GUI. For the current mplementaton, relatvely accurate algnment s necessary for rotaton, but t s not the case for translaton. Image plane Fgure 13. Correspondence between 3D reflectance and 2D color edges ponts. Ths operaton s smlar to the ICP operaton. To determne the relatve pose that concdes wth the poston of 2D color edges and projected 3D reflectance edges, we use the M-estmator. Frst the dstance between correspondng 2D color edge ponts and 3D reflectance edge ponts s evaluated as shown n Fgure 13 : where z s a 3D error vector whch s on a perpendcular lne from a 3D reflectance edge pont to the stretched lne between the optcal center and a 2D color edge pont on the mage plane. ε = Z snθ where Z s the dstance between the optcal center and a 3D reflectance edge pont, and θ s the angle between the color edge pont and the reflectance edge pont. The system fnds the confguraton, P, whch mnmzes the total error, E, where ρ s an error functon. The mnmum of E( p) can be obtaned by: E = P ( ε ) ρ ε ε = 0 P We can consder ( ε ) error terms. ω ( ε ) ρ = 1 ε ε ω as a weght functon to evaluate Thus, the mnmzaton can be derved as the followng least squared equaton: E = ρ ω ε P ( ε ) ε = 0 We choose the Lorentzan functon for ths functon ( ) ε ω ε = 1+ 2 σ By solvng ths equaton usng the conjugate gradent Fgure 14. Texture Mapped Kamaura Buddha. 4. Restorng Hypotheszed Orgnal State After we obtan the precse geometry and photometry nformaton of the cultural assets n the current state, we can restore them to ther hypotheszed orgnal state. In ths secton, we descrbe one of the examples: the restoraton of the Nara Great Buddha and ts man hall 4.1 Restorng Nara Great Buddha Nara Great Buddha s the man statue of Toudaj Temple. Unfortunately, the current statue s a rebult and repared one because the orgnal statue was burned and melted down due to a couple of cvl wars. Accordngly, the shape of the current Great Buddha s dfferent from that of the orgnal one. By usng the geometrcal modelng shown n Secton 2, we have acqured the complete 3D geometrcal model of Buddha n ts current state. From ths model, we have attempted to synthesze the orgnal state by morphng the 3D geometry of the model. From some lterature nherted at the temple, we now the szes of varous face parts such as the nose and mouth. Usng these data, we desgn a two-step morphng algorthm.

9 Frst, we globally change the scale of the whole portons (e.g., Sttng Heght, Face Length, Nose Length); these are gradually modfed. In the 2nd stage, vertces are moved one by one teratvely, smlar to the constrant propagaton algorthm, usng smoothness and unform constrants. The 2-stage morphng enables us to obtan the complete model of the orgnal Great Buddha. Fgure 15 shows the 3D models of the current (a) and the orgnal Great Buddha (b). We can easly recognze that the orgnal Buddha s larger and rather snny. Fgure 16 Mnature and ts 3D scanned models. Fgure 17 Partal models acqured at Tousyoudaj. (a) Current (b) Hypotheszed Fgure 15. Restored result of Nara Great Buddha. 4.2 Restoratng Toudaj Man Halle The man hall of the Toudaj Temple was bult durng the same decades as those of the Great Buddha (8th century). It was also rebult twce: n the 12th and 18th centures. In the 12th century, Tenju archtecture was mported from Chna and the man hall was rebult n a totally dfferent archtecture style. The rebuldng n the 18th century followed the same new style. As a result, the style of the current man hall s entrely dfferent from that of the orgnal buldng. Fortunately, the Toudaj temple has been dsplayng a mnature model of the orgnal hall, constructed for the Pars Expo n 1900, as shown n Fgure 16(a). We dgtzed t usng the Pulstec TDS-1500 and scaled t up to the orgnal sze as shown n Fgure 16(b). Due to the lmtaton of resoluton, the detal parts cannot be obtaned precsely. Accordng to Prof. Kesue Fu an archtecture professor at the Unversty of Toyo, one of the experts on buldng style n the era, the Toudaj and Toushou-daj Temples share a smlar format. Here, the man hall of Toshoudaj Temple were also bult durng the same perod (8th century). After scannng varous ey parts of the man hall at Toushouda as shown n Fgure 14, we morphed these partal range data by expandng and shrnng the Touhoudaj parts (Fgure 17) to the scaled-up range data of the Toudaj (Fgure 16). The process was conducted by an extended algnment algorthm that allows scale change as well as confguraton dfferences. Fgure 18 shows the current Nara Great Buddha man hall and the (a) Current Toudaj man hall bult n 18AD. (b) Hypothesze 8AD Toudaj man hall. (c) Hypotheszed 8AD Nara Buddha n the hypotheszed 8AD man hall. Fgure 18 Dgtally restored Nara Great Buddha.

10 orgnal one dgtally restored by our method. 5. Concluson In ths paper, we ntroduced our project to dgtally archve and restore cultural hertage objects. Our project s man goal s to develop a method of 'modelng from realty', n whch the dgtal model of cultural propertes s created by usng varous computer vson methods. For the observaton of geometrcal nformaton, we used laser range fnders and post process algorthms, ncludng regstratng and mergng the range mages. For the texture nformaton, we have developed several texture mappng methods. For the short dstance range sensors, we calbrated the relatonshp between the range sensor and the mage sensor for complete texture algnment. For the long dstance range sensors, we developed a non-calbrated texture algnment method by usng laser reflectance features. Dgtal restoraton of lost cultural hertage objects has a bg advantage compared wth other restoraton methods such as physcal constructon of actual temples, because we can examne varous hypotheses wthout any physcal changes nor long buldng perods. We demonstrated the effectveness of ths method through the restoraton of the Nara Great Buddha and ts man hall. We are also conductng a project to create a dgtal lbrary of the world great Buddhas, ncludng three Japanese Buddhas, Sr Chum Buddha n Thaland, and Byon s n Camboda. The models and restoraton results constructed so far can be vewed at [21] Acnowledgment Ths research s sponsored, n part, by JST under Ieuch Crest program. The Bayon n Camboda was dgtzed wth the cooperaton of Japanese Government Team for Safeguardng Anor (JSA). References [1] K. Ieuch and Y. Sato, Modelng from Realty, Kluwer Academc Press, [2] M. Levoy et. al., The dgtal Mchelangelo project, SIGGRAPH 2000, New Orleans. [3] J. Wasserman, Mchelangelo s Florence Peta, Prnceton Unversty Press [4] I. Stamos and P. Allen, Automatc regstraton of 2-D wth 3-D magery n urban envronments, ICCV2001, Vancouver. [5] P.J. Besl and N.D. McKay, "A method for regstraton of 3-d shapes," IEEE Trans. Patt. Anal. Machne Intell., 14(2): , [6] R. Benjemma and F. Schmtt, Fast global regstraton of 3D shample surfaces usng a multple-z-buffer technque, Int. Conf on Recent Advances n 3-D Dgtal Imagng and Modelng, pp , May [7] P. Neugebauer, Geometrcal clonng of 3D objects va smultaneous regstraton of multple range mages, Int. Conf on Shape Modelng and Applcaton, pp , March [8] Y. Chen and G. Medon, Object modelng by regsteraton of multple range mages, Image and Vson Computng, 10(3): , Aprl [9] S. Rusnewcz and M. Levoy, Effcent varants of the IPC algorthm, Int. Conf 3-D Dgtal Imagng and Modelng, pp , May [10] H. Gagnon, M. Soucy, R. Bergevn, and D. Laurendeau, Regsteraton of multple range vews for automatc 3-D model buldng, CVPR94, pp [11] K. Nshno and K. Ieuch, "Robust Smultaneous Regstraton of Multple Range Images", Ffth Asan Conference on Computer Vson ACCV '02, pp , [12] T. Osh, R. Sagawa, A. Naazawa, R. Kurazume, and K. Ieuch, Parallel Algnment of a Large Number of Range Images on PC Cluster, Int. Conf 3-D Dgtal Imagng and Modelng, Oct [13] M. D. Wheeler and K. Ieuch, "Sensor Modelng, Probablstc Hypothess Generaton, and Robust Localzaton for ObjectRecognton", IEEE PAMI, 17(3): , [14] B. Curless and M. Levoy, A volumetrc method for buldng complex models from range mages, SIGGRAPH 96, New Orleans, LA. [15] M. Wheeler, Y. Sato, and K. Ieuch, Consensus surfaces for modelng 3D object from multple range mages, ICCV98. [16] R. Sagawa, K. Nshno, M.D. Wheeler and K. Ieuch, "Parallel Processng of Range Data Mergng", IEEE/RSJ Internatonal Conference on Intellgent Robots and Systems, Vol. 1, pp , 2001 [17] R. Sagawa, T. Masuda, and K. Ieuch, Effectve Nearest Neghbor Search for Algnng and Mergng Range Images, Int. Conf 3-D Dgtal Imagng and Modelng, Oct 2003 [18] R. Sagawa and K. Ieuch, Tang Consensus of Sgned Dstance Feld for Complementng Unobservable Surface, Int. Conf 3-D Dgtal Imagng and Modelng, Oct 2003 [19] R. Kurazume, M. D. Wheeler, and K. Ieuch, "Mappng textures on 3D geometrc model usng reflectance mage," Data Fuson Worshop n IEEE Int. Conf. on Robotcs and Automaton, [20] R. Kurazume, K. Nshno, Z. Zhang, and K. Ieuch, "Smultaneous 2D mages and 3D geometrc model regstraton for texture mappng utlzng reflectance attrbute," Ffth Asan Conference on Computer Vson, [21]