INTEGRATED MODELING SYSTEMS FOR 3D VISION

INTEGRATED MODELING SYSTEMS FOR 3D VISION A. Guarnieri a, R. M. Levy b, A. Vettore a a CIRGEO Interdept. Research Center of Geomatics, University of Padova, Italy - cirgeo@unipd.it b Faculty of Environmental Design, University of Calgary, Canada rmlevy@ucalgary.ca KEYWORDS: Laser Scanning, 3D Modeling, Virtual Reality ABSTRACT: A joint project between the Interdept. Research Center of the University of Padova (CIRGEO) and Dr. Richard. M. Levy, teaching Professor at the Faculty of Environmental Design of the University of Calgary (Canada), has been undertaken aimed to the generation of a full 3D model of an historical building and the surrounding environment, based on a terrestrial laser scanning survey. The main goal of this project is to provide a 3D representation where two different contents are merged together: the object s geometry on one hand and a set of related historical and cultural information on the other hand. Indeed, through suitable VR authoring tools, like the Virtools Dev. software, it is now possible to build VR environments around laser scanning-based 3D models, which allow the user with a certain level of interaction with the model itself. This solution opens interesting perspectives towards the use of 3D models as a mean to promote the national Cultural Heritage content among remote people: portable cave systems, comprising of double projector devices, stereo 3D converter and a wide screen display, could be profitable employed to this end. In this paper we report the first results of our VR project, i.e. creating a TLS-based 3D model of the church of Pozzoveggiani, an ancient historical building located 15 km south of Padua (Italy). The second stage of the work, i..e. the generation of an interactive VR environment based on such 3D model is still in progress at current date. Therefore we will focus here on different issues related to the generation of a fully closed model of a complex structure, a situation frequently found in the Cultural Heritage field. 1. INTRODUCTION A major advantage of 3D models relies on the fact that they allow to represent real objects more adequately than through a single picture or collection of pictures, by providing a higher level of detail together with a good metric accuracy. These models are currently used for cultural heritage, industrial, land management or also medical applications. In the cultural heritage field, 3D models represent an interesting tool for asbuilt documentation and interactive visualization purposes, e.g. to create virtual reality environments. In some cases [2] 3D models obtained by laser scanning were used to fill a virtual environment with real objects, in order to get a faithful copy of a real environment, such as the interior of a museum or historical building. Furthermore, in these VR applications the user point of view can be easily changed, providing in this way a useful tool by which the human direct inspection can be well simulated. Regardless the measuring system empoyed (close-range or long-range laser scanner), the target shape complexity, in the field of 3D modeling there is an increasing demand for a final product allowing the user to retrieve not only the object s geometry but also the information content related to the surveyed structure. In this sense, dedicated VR authoring softwares, like Virtools Dev. [9], could be profitably employed to satisfy this demand, given their capability to combine in a unique solution 3D models with tools for interactive information content retrivial. This topic has been already explored in past years (Fangi, Corfu) even if most of proposed works were built around web-oriented applications, i.e. the resulting VR environment could be viewed on a PC monitor, preventing therefore to exploit all the advantages of a 3D representation. A possible solution to this intrinsic limit of current PC technology is represented by socalled cave systems, a set of devices comprising of workstations, wall screens, projectors and touch sensors, which allow the user to live full immersive experiences in fascinating and stunning 3D VR environments. With a little effort of imagination, it should be easy to figure out the advantage of combining together TLS-based 3D models and their historical and cultural related information in an interactive VR environment to be explored through a portable cave system. Despite the very high cost of VR sensors needed for this goal, such integration of different technologies would represent indoubtely a powerful mean to promote the national Cultural Heritage content in remote sites. In terms of practical application, portable caves could be used as moving virtual museums, where user would experience a new and, of course, more immersive way of getting in touch with the History, Art and Culture of remote sites respect with it is allowed today through the Internet. According with this new concept for the employment of TLS based 3D models, a joint project between the Interdept. Research Center of the University of Padova (CIRGEO) and. Dr. Richard M. Levy, teaching Professor at the Dept. of Urban Planning, Faculty of Environmental Design of the University of Calgary (Canada), has been undertaken. The main aim of this project is to create a full 3D model of an historical building and the surrounding environment in order to provide a 3D representation where two different contents are merged together: the object s geometry and a set of related historical and cultural information. As target object, the ancient little church of Pozzoveggiani, 15 km south of Padua (Italy), was chosen given the limited degree of geometry complexity and its importance in the panorama of the historical heritage of Padua and its province. The work has been arranged as follows: the research group of CIRGEO has been charged to perform the laser scanning survey and to generate the full 3D model of the church, while Prof. Levy should build a full VR environment comprising of the 3D model of the church, its surrounding and of related information contents. An example of how the expected final model should look like is depicted in figure 1, showing the 3D

reconstruction of the temple site at Phimai in Thailand, performed by Prof. Levy. Then the resulting final product will be employed in two different ways: as full VR environment from which the user can interactively retrieve a set of information about the geometry and history of surveyed structure and as data source for immersive 3D exploration in a portable cave. The latter will be based on a double stereo projector system, provided by the Geamedia Group italian company (Figure 2.). In this paper we report the first results of our VR project, e. g. the TLS-based 3D modeling of the church of Pozzoveggiani. At the present the second stage of the work, e.g. the generation of an interactive VR environment based on such 3D model, is still in progress. Therefore we will focus here on different issues related to the generation of a fully closed model of a complex structure, a situation frequently found in the Cultural Heritage field. instrument is provided with a rotating head and two scanning windows (figure 3) which allow to acquire a scene with a large field of view (FOV = 360 H x 270 V), reducing therefore the need of several measuring stations. Though the acquisition speed is not very high (1800 pts/sec), depth accuracy is claimed to be 6 mm (up to at 50 m distance) with a beam diameter divergence of 3 mm at 50 m. The laser comes with a built-in color CCD camera capable to provide 3 different user selectable resolutions (low, middle and high), up to 1 Mpixel. A short summary of the main features of the HDS 3000 is reported in figure 4. In order to perform a model texturing and to build the 3D view of the outdoor environment surrounding the church, a set of digital images were taken with a prosumer digital camera Nikon COOLPIX 5700. This is a 5 Mega pixel camera with a 2/3" CCD sensor size and a maximum image resolution of 2560 x 1920 pixels, corresponding to 3.4 mm pixel size. As in this project digital images had to be used for different purposes, image capture was set as follows: about 60 out of 100 pictures were acquired with a 1600 x 1200 resolution for the texture mapping while for the remaining 40, to be used for the modeling of the outdoor, a 1024 x 768 resolution was chosen. In both cases, the digital images were saved in JPG format. Figure 1: The 3D virtual reconstruction of the temple site at Phimai, Thailand (courtesy from Dr. M. R. Levy) Figure 3: The Leica HDS 3000 laser scanner 3. THE TLS-BASED 3D MODEL Figure 2: The stereoscopic projector system 3D Vyz 2. INSTRUMENTS In this work, two different 3D environments can be distinguished: the 3D model of the church, based on a terrestrial laser scanning survey, and a VR representation of the surrounding based on color digital images and a cadastral map. The former has been accomplished employing the Leica HDS 3000 terrestrial laser scanner [5], the more recent product of Leica s high definition TLS systems (Figure 3). This Given the relative small size of the targeted structure and the wide field of view of the laser scanner, only 9 measuring stations were needed to completely survey the church. Such stations were located around the church in order to be able to capture its whole geometry while limiting the number of the stations. However due to unavoidable obstacles to the laser line of sight, a few parts of the building could not be surveyed, what led to quite large holes in the resulting 3D models. For example the roof was not surveyed at all, as it would have required to raise up the laser scanner on higher position like a terrace of surrounding houses. Basically, 20 scans were acquired with an average resolution of 7 mm, however all the range images were subsequently downsampled at 1 cm in order to reduce the size of the dataset and make it more manageable with the available hardware. Data processing has been fully accomplished using Polyworks/Modeler, a software package provid ing a very powerful environment for the interactive 3D modeling of real objects, especially when high point density is required. It is composed of several modules, which allow the user no only to

regarded as a confirmation of the goodness of the registration procedure implemented in Polyworks: the residual error is only due to the inherent accuracy of the employed laser scanner. Figure 4: Technical specifications for the HDS 3000 Figure 5: Example of overexposure image acquired by the built-in CCD camera carry out all the processing steps required in a classical modeling framework, but also to keep in the same time the control over the entire process and to verify the accuracy of the results through a number of dedicated tools. In this phase of the work the following modules has been employed: IMAlign for the scan alignment, IMMerge for the mesh generation IMCompress for the model decimation IMEdit for the hole filling 3.1 Range data alignment The interactive manual N-points alignment procedure was adopted in this case to register the scans of the outer walls of the church each other. Matching points have been pretty easily recognized on adjacent range images thanks to the color per point information acquired by the laser built-in CCD camera, whose resolution was set to 1 Mpixel (high level). However a few scans did not present a good point coloring due to exposure issues of the CCD camera. Indeed the survey of the church was carried out in one single day, what did not allow to choose the best position for the laser scanner according to the part to be surveyed and the sun elevation. As a result some scans presented disturbing phenomena like overexposure, backlighting and low illumination (Figures 5-6), which made a bit more difficult the manual selection of the matching points. In this case intensity data could not be employed to solve for this issue as Polyworks does not allow to interactively switch between color or gray coded point clouds. After the N-points pre-alignment step, an approximate transformation matrix for each scan pair was obtained and then used in the second stage as starting point for the refined registration based on the well-known ICP algorithm. Here, a scan group was locked in order to define the reference frame of the model. Then, the ICP-based global alignment was run twice in order to refine the results of the first stage. Such approach [8] yielded a very good registration, with an average RMS alignment error of 0.004 m (figure 7). This result can be Figure 6: Example of image acquired in dark light Figure 7: Results of the global registration Another way to assess the quality of the global registration is to look at the range image histograms computed by Polyworks. In this case the well gaussian shaped curves of the residual

alignment error (Figure 8) indicate a good level of overlap between adjacent views and a sufficient number of points needed to compute accurate transformation matrices. the 3D distance between the reduced triangulation and the original one. IMCompress thus aims at removing the maximum number of vertices while maintaining the decimated model as close as possible to the original 3D model. There are three methods for specifying reduction levels with IMCompress. The user may choose to specify one or more reduction levels based on the number of triangles of resulting model, a tolerance value, which specifies absolute tolerance reduction level, or a relative tolerance value (tolerance %), given in percentage of the largest side of the model bounding box. In this case the decimation procedure was run twice by choosing two different values for the first option: 3 millions of traingles and then 2 millions, respectively, while keeping a sampling step of 1 cm. After a simple viewing inspection the model comprising of 3 millions of triangles was chosen as it featured a good reduction level (half size) while keeping enough good shape quality. The resulting model, rendered with the color per pixel information, is shown in figure 9, where some major holes can be easily noticed. Figure 8: Residual alignment error histograms 3.2 The scan merging As a result of the scan registration a unique point cloud was obtained presenting a large overlap between adjacent component views. Though this overlap is needed for the pairwise alignment, it implies in the same time a high redundancy level not necessary for the merging step. Therefore prior to build a poligonal model form the global point cloud, a overlap reduction procedure was applied. Basically, the best non overlapping points among all the scans were automatically detected and selected by Polyworks, then inverting this selection the overlapping areas were highlighted and shrinked in order to get only a tiny band (5 points) of overlap, just the minimum needed for the merging algorithm. This strategy allowed reduce the number of points involved in the triangulation process, while preserving enough information about the object s shape. For the merging the same subsampling step adopted the scan alignment was set (1 cm) in order to keep as much as possible the same level of detail. As a result a 3D model consisting of 6 millions of points was obtained, corresponding to a.pol file of about 185 MBytes in the Innovmetric binary format. 3.3 Mesh decimation and model editing Since the main aim of this project was to build a VR environment comprising of the 3D model of the church so as a 3D representation of the surrounding, in order to avoid possible difficulties for managing a too high data set, a further model decimation was applied prior to perform any kind of editing operation. To this end IMCompress was employed, a Polyworks module consisting of a multi-resolution triangulation algorithm that optimally reduces the number of triangulation vertices. Basically, a sequential optimization process iteratively removes triangulation vertices, minimizing Figure 9: The 3 million points 3D model of Pozzoveggiani During in field data acquisition it always advisable to place the measurement stations in such a way to be able to capture as much as possible the object details, avoiding possible large holes in the resulting model. However, this was not feasible in this case given the limited room for a clear view of the church, as depicted in figure 10. Therefore some parts of the outer walls, so as the whole roof could not be surveyed, which resulted in very large holes in the final model. Furthermore all the eave lines had to be removed given the high amount of little holes to be filled. The editing of these elements, so as most of the windows, required the application of surface patches (Figure 11) by manual anchoring control points of Bezier or Hermite surface representations to existing hole bounding triangles, and performing a subsequent fitting and triangulation operations. All these steps had to be reapeated for most of the holes, increasing heavily the global processing

time. From this point of view the budget of the working time can be summarized as follows: 1 full day for the laser scanning survey, one day for the range data alignment, one day for the mesh decimation and 4 days for the model editing. Figure 10: View of the North side of the church Figure 12: Adjacent scans acquired by the laser upper window showing longitudinal holes. Figure 11: Hermite surface patch applied for hole filling As regards the hole filling step, it should be noted that even the HDS 3000 laser scanner contributed to produce areas of missing data. Surprisingly, indeed, during scan registration areas of the bell tower surveyed with the upper window of the laser scanner revealed a lack of overlap between adjacent point clouds, as clearly shown in figure 12. Actually this strange behavior was noticed during the surveying of another historical building, an ancient venetian villa in the surrounding of Padua. Even in this case, a few parts of the structure were acquired by the laser upper window as non overlapping stripes. At the present it is not clear if this was due to laser failure or to other unpredictable factors, since such event appeared randomly. Of course, further checks and tests of the laser scanner are needed to find a solution for this issue. The final model resulting after extensive editing is shown in figure 13, where it is rendered according the color per point information. It can be clearly seen that major hole are completely disappeared. Figure 13: The filled final 3D model (compare with figure 9) CONCLUSIONS AND FUTURE WORKS In this paper the results of the first stage of a joint project between CIRGEO, the Interdept. Research Center for Geomatics of the Univerisyt of Padua (Italy), and Prof. Richard M. Levy of the Faculty of Environmental Design of the University of Calgary (Canada) have been presented. The main goal of the work is to build a VR environment comprising of the TLS-based 3D model of the church of Pozzoveggiani and its surrounding. The end product should allow the user not only to explore the site in a virtual way but also to retrieve interactively from it historical and architectural contents related

to the church. Moreover, it is planned to use such product as data sample to be displayed on a portable cave through a stereoscopic projector system, in order to allow even remote users to experience a full immersive 3D visit of an historical site. This application could be regarded as a first attempt to realize a kind of moving virtual museum. The project is quite complex and it is still in progress. Here the first stage has been discussed, i.e. the steps needed to generate a full 3D model of the church starting from a terrestrial laser scanning survey. Procedures adopted for the 3D modeling and some issues related to the registration and edititing steps were discussed. The work done so far has demonstrated that the availability of laser built-in CCD cameras with resolutions higher than 1 Mpixel can theoretically provide well coloured point clouds, so that to avoid the subsequent texture mapping phase. However, issues related to the rigth camera exposure settings show that the acquisition of digital images with enough good quality could require longer surveying times depending upon the outdoor light conditions, site geometry, approriate choice for the measuring station locations and on the sun elevation. Furthermore, though modern TLS systems allow for high acquisition speeds, the generation of a full closed mesh of the surveyed object can be often become a very time consuming process according with to the level of detail of the 3D model and the object s shape complexity. [8] Soucy, M., and Laurendeau D., 1995. A general surface approach to the integration of a set of range views. IEEE Trans on PAMI, Vol. 17, No 4, pp 344-358. [9] Virtools, The Behavior Company, 2005. http://www.virtools.com (accessed April 30) ACKNOWLEDGEMENTS This work has been accomplished in the context of the National Research Project 2004, titled Integrated Methods of Laser Scanner and Photogrammetric Surveys for 3D Model Generation in Culture Heritage. National Coordinator: Prof. Carlo Monti, Local Coordinator: Prof. Antonio Vettore. REFERENCES [1] Bologna R., Guarnieri A., Minchilli M., Vettore A., 2002. Automatic registration of 3D views. Proceedings of the ISPRS Comm. V Symposium Close Range Imaging- Long-Range Vision, 2-6 September, Corfù, Greece. [2] El-Hakim S. F., 2001. 3D Modeling of Complex Environments. Videometrics and Optical Methods for 3D Shape measurement, Proceedings of SPIE, vol 4309. [3] Guarnieri A., Remondino F., Vettore A., 2004. Photogrammetry and Ground-based Laser Scanning: Assessment of Metric Accuracy of the 3D Model of Pozzoveggiani Church. Proc. of FIG Working week 2004, The Olympic Spirit in Surveying, Athens, Greece. [4] Innovmetric Software Inc., 2004. http://www.innovmetric.com (accessed April 30). [5] Leica Geosystems, 2005. http://www.leicageosystems.com (accessed April 30). [6] Levy R., 2001. Computer Reconstruction: Temple site at Phimai, Computer generated images, Canadian Architect. [7] Malinverni E. S., Gagliardini G., Fangi G., 2002. Virtualisation of an archaeological site. Proceedings of the ISPRS Comm. V Symposium Close Range Imaging- Long-Range Vis ion, 2-6 September, Corfù, Greece.