RECENT TECHNIQUES IN DOCUMENTING HISTORIC PLACES

Transcription

1 RECENT TECHNIQUES IN DOCUMENTING HISTORIC PLACES Dr. Eng. Ahmed Abdelhafiz Civil Engineering Department, Faculty of Engineering, Assuit University, Egypt ABSTRACT: Three dimensional models serve in many fields such as architectural conservation and cultural heritage documentation. Historical sites can be then accessed virtually for all people all over the world, not only as tourists but also as researchers. Virtual models offer a good solution to save time, effort, and of course money. Laser scanning and photogrammetry are the recent techniques for the 3D documentation. Data processing is a long and effective step in the work flow using such techniques. This paper focuses mainly on simplifying the work flow of documentation which will give the chance for more historical sites to be documented. We are working on some European monuments with very fine surfaces made of frescoes and/or mosaics like the Domitilla catacomb at Rome/Italy and the terrace houses at Ephesos/Turkey. Some of the results on such projects will be presented using the recent developed techniques. KEY WORDS: Architectural conservation, laser scanning, coloured point cloud, 3D model, documentation, Software for automatic texture mapping

2 1. INTRODUCTION Three dimensional models are the key now for cultural heritage documentation and architectural conservation. Laser scanning technique is the most recent and accurate technology for that type of documentation. Digital photogrammetric technique is always the important partner technique for laser scanning. Laser scanners capture huge amount of points for objects in a very short time in order to deliver what is called point cloud. Multiple scans from different positions have to be taken in order to achieve all object faces and details in a 3D environment. Unfortunately the laser ray can t detect objects colour which is of highest interest for historical places i.e. the acquired 3D point cloud has no colour. Therefore digital images are taken at the same time with each scan by either a free hand camera or by a mounted camera on the top of the scanner. In the free hand camera approach, a registration step is required to put the captured images in the same coordinate system of the point cloud. This can be achieved through an interactive selection of points between the images and the point cloud. Afterwards the coloring process is applied. This step of registration takes much manual work and increase significantly the processing time. For the approach of mounted camera on the top of the laser scanner, a colored point cloud for each scan is delivered in one step. Afterwards all colored point clouds of the object are registered in one coordinate system and merged. The results are not with enough quality suitable for heritage documentation. Colours of a part of an object come from more than one image. On the other hand, images captured from different positions always have illumination variation and so noisy colours are resulted. In this paper, a good solution will be presented for colouring the merged point clouds from the same simultaneously captured images taking into account the previous colour issues using developed software. The main advantage of the developed algorithm in this software is that each part of the object will be coloured from one image with a good level of automation. So, homogenous colours are achieved. In some applications, the colored point cloud is not enough, as it is still present points. One expects to get real model with surfaces. Therefore the point cloud can be converted to a mesh model. Having high resolution digital images for the scene, the texture of the model can be added. Manual processes are required for texture mapping process.

3 The solution is found in an automatic algorithm for texture mapping like the ML3DImage algorithm with its corresponding software (3DImage). An overview on the laser scanners and the resulted point clouds is given in section 2. The registration step of multiple point clouds is described in section 3. The coloured point cloud using the free hand camera approach and the mounted camera on the laser scanner approach are presented in section 4 and section 5, followed by the reprocessing technique of data captured by mounted camera on laser scanner approach in section 6 applied on a real data for a historic place in Turkey. Finally, the generation of photorealistic 3D models technique is presented and applied on a famous site in Rom/Italy in section LASER SCANNER POINT CLOUDS Terrestrial laser scanning has already found its place between the standard technologies for objects acquisition. The laser scanner can be described as a motorized total station, which measures automatically all points in its horizontal and vertical field. For each measured point, its distance to the laser scanner together with the horizontal and the vertical angles are recorded. So, the space coordinates relative to the scanner position can be easily computed, Abdelhafiz, (2009). This means that at one position of the laser scanner, a dense point cloud is immediately delivered. Modern scanners, like Z+F laser scanner and Leica HDS, can capture the entire hemisphere from one position with a field of view. Old scanner types, like Cyra2500, have a limited field of view (40 60 ), see Fig. 1 for different types of modern laser scanners.

4 Leica Riegl Z+F Faro Fig. 1: Different types of modern laser scanners A powerful advantage of the scanning technology is that, the laser scanning is an active technology therefore no problems with daylight or illumination conditions are encountered. Along with the points space coordinates, the laser scanner measures also an intensity value for each point. These measurements are commonly used to support the visual analysis of the point cloud. Terrestrial laser scanners employed in architectural applications use mainly two methods of scanning. These methods are the time of flight (e.g. Leica and Trimble) and the phase difference (e.g. Faro and Z+F). While the scanning speed of phase difference scanners is faster than the time of flight scanners, the point clouds resulted from scanners use the phase difference method is more noisy than those resulted from scanners use the time of flight method Mechaleke et al., (2007). The measuring range of scanners employ the time-of-flight method ( m) is longer than the measuring range of scanners employ phase difference method (70-80m). Terrestrial laser scanners deliver points with millimetres accuracy. Different researches investigate scanners accuracy can be found in the following literature Boehler et al., (2003); Kersten et al., (2005); Clark and Robson, (2004). See Fig. 2 for two captured scans by a time of flight laser scanner for a status in the city centre, Braunschweig, Germany.

5 Fig. 2: Two captured scans of a status in Braunschweig, Germany 3. REGISTRATING MULTIPLE LASER SCANNER POINT CLOUDS Recovering the geometry of a complete 3D object requires more than one stand position for the laser scanner. The resulted point clouds captured at each stand point of the scanner have different coordinate systems. These point clouds have to be registered together in one coordinate system in order to achieve a complete object representation. A layout of scanner positions and targets distribution around the Lion status which is the city symbol of Braunschweig, Germany is shown in Fig. 3 together with the final registered point cloud. A significant amount of research in this direction has been done in the last few years, in which manual, semi automatic, and automatic methods of point clouds registration are developed and/or evaluated Al-Manasir and Fraser, (2006); Akca and Gruen, (2008); and Wendt, (2008). Fig. 3: Left: A layout of scanner positions and the targets (balls). Right: the registered point clouds of the lion status, Braunschweig, Germany.

6 The registration between two point clouds can be achieved through assigning the space coordinates of three corresponding points or more at each point cloud. In case of having more than three points, the least squares technique is applied. These points can be either artificial or natural targets. Whereas natural targets are assigned manually, artificial targets can be assigned automatically by employing certain algorithms. The automatic assigning process is based on detecting certain shapes of targets like spheres or black and white targets. Fig. 4 shows different types of used artificial targets. Fig. 4 : Different types of artificial targets used with laser scanners 4. COLORED POINT CLOUDS The laser scanner almost immediately delivers a 3D data set with no further complicated compilation processes. But, the laser ray can t detect objects color. On the other hand, High resolution digital images can offer very good colors. These images are taken to the site in parallel. The images are captured either by a camera mounted on the laser scanner itself or by a free hand camera. In order to achieve the real color for each point of the laser scanner point cloud, a registration step has to be executed first. The registration aims to put the concerning data sets in a common coordinate system so that they can be integrated into a single 3D colored point cloud. The registration refers here for the registration of the digital photos in the same coordinate system of the registered point clouds, see Fig. 5.

7 Fig. 5: The captured photos registered in the point cloud coordinate system In the coloring procedure from an image, the effect of the occlusion has to be considered. In case of blind coloring without considering the occlusions, false colors might be achieved, see Fig. 6 for a real example for a traffic sign and a building in the background falsely colored. So, it is required for the data fusion to know the correct corresponding image pixel for each point from the point cloud considering the occlusion existence Abdelhafiz and Niemeier, (2006). + Original point cloud A photo Coloured point cloud (false colour) Fig. 6: Occlusion effect, a real example for a traffic sign and a building in the background to see the effect of points like B on the coloring process

8 Application 1: Vehicles manufactures are interested in the 3D reconstruction of traffic crossings to investigate the traffic safety on those cross-sections not only from the geometric design point of view but also from the vehicles safety point of view. Here a professional digital camera, Canon EOS 350D, eight MP was employed to capture a set of photos for a crossing in Germany. Three scans using the Z+F Imager 5003 laser scanner are also captured. This scanner uses laser safety class 3R (DIN EN ) and capable of capturing up to 500,000 points per second. The maximum captured rang for this scanner is 53.5 meters with linear error less than 5 mm. At Fig. 7, the three registered scans displayed in grey colors are shown. Fig. 7: The three registered scans for a traffic crossing displayed in grey colors In order to color such a point cloud, the following two steps are executed: Registering: the registration is achieved here through interactive manual selection for natural points between the images and point cloud. A photogrammetric model is then reconstructed for the captured images using the extracted points from the point cloud to compute the exterior orientations for all images using photogrammetric software. Colouring: the 3DImage software is employed here to automatically colour the point cloud using all the available images (eighteen photo). Fig. 8 shows some snapshots for the final 3D coloured point cloud.

9 Fig. 8 : Some snapshots for a colored point cloud of the crossing in Germany 5. MOUNTED CAMERA APPROACH Nowadays the optional true colour channel, integrated in some laser scanner types (e.g. Riegl LMS-Z420), provides the colour of the target surface as additional information to each space measurement. This channel, which is co-aligned with the range measuring channel, is a passive colour channel which means that valuable results can only be achieved for outdoor scanning under bright sunlight conditions. Other laser scanner types (e.g. Riegl LMS-Z210ii) are equipped with mechanically mounted digital cameras to capture the object colour. The attached camera has to be firmly fixed to the scanner body then the relative orientations between the camera and the scanner can be calculated through a certain calibration process. Having the relative orientations between both sensors, the corresponding image pixel colour for each space

10 point can be extracted. After that, the point cloud can be displayed using the true colour channel instead of intensity values. Application 2: Ephesus site is considered one of the great outdoor museums of Turkey. It is located on the south of Izmir's Selcuk county. The links of Ephesus with the Amazons and the myths had survived throughout history. For documentation purpose, a Riegl terrestrial laser scanner mounted with high resolution digital camera is employed. At one step, coloured scans are achieved. Fig. 9 shows two coloured scans for Ephesus. Fig. 9: Two scans from different positions for Ephesus, Turkey 6. RE-PROCESSING DATA CAPTURED BY MOUNTED CAMERA ON LASER SCANNER The capturing procedure of scanners equipped with cameras is executed in two steps. The scanner starts first to scan the scene geometry. Then it begins to rotate again to capture the required photos. At each scan position of our example in Ephesus, six images are obtained. The colour of individual scans look good. But after registering the scans together, the illumination variation between the different images at each scan shows bad colours. Registered scans are shown in Fig. 10- left with their colours. One can see the defects along the whole point cloud. A good solution is offered by the 3DImage software ( The software reprocesses the data again automatically to colour each part from one image. The

11 result of reprocessing the same data for this site is shown at Fig. 10-right in comparison with the resulted scans before the reprocessing. One can see nice and homogenous colour after the reprocessing step using the 3DImage software. Snapshots for multiple registered scans captured by the modern Riegl laser scanner Snapshots for the same point clouds reprocessed by the (3DImage software) using the same images resulted from the mounted camera Fig. 10: Registered multiple scans captured by Riegl laser scanner for Ephesus, Turkey in comparison with reprocessed data using 3DImage software

12 7. PHOTO REALISTIC 3D MODELS Photo realistic 3D models are the preferable product for almost all customers in the civil market especially for those working in fields of architectural documentations and cultural heritage. All the time, customers seek to achieve the best accuracy and the minimum capturing and processing time. Coloured point clouds achieved in the previous section provide the customer with a reliable representation for the concerned object. It offers a good geometry and an acceptable visualization with real colours. But in many cases, these coloured point clouds can not fulfil customers requirements as it is still points and not a real model, see Fig. 11 for a status in points form, mesh and textured model. It is clear that the 3D model is superior in comparison with the point cloud. Fig. 11: Lion status model, Braunschweig, Germany, Left: points, Middle: Mesh, Right: Textured model In order to get a 3D model, the object points have to be connected together to be surfaces through what is called the meshing process. The photo texture is then warped to the geometry through the texture mapping process in order to finally achieve the desired photo-realistic models. Plenty of algorithms and software are already available for the meshing process. After some extra manual editing processes, smooth mesh for the concerning object can be achieved.

13 Review papers about the mesh generation technology written by Owen, (1998) and Edelsbrunner, (2001) give more details of the available techniques and software. The procedure of texture mapping needs also several manual processes which require too much time. Several days and sometimes several weeks for more complex objects are needed to achieve good results. This is with no doubt time consuming and consequently cost consuming. Therefore the automation presents a challenge in executing this process. An automatic texture mapping algorithm and the 3DImage software are used in this work to generate photorealistic 3D model for our third application shown in the next section. Application3: The Domitilla catacomb is the largest catacomb in Rom. It is considered to be one of the most complex sites from the architectural structure point of view. It consists of a disordered grid of about 15 km long of underground galleries in up to four levels. Moreover, it saves inside about eighty painted areas of highest interest for archaeologists. These paintings are partially known and not yet accessible to all scientists Abdelhafiz et al (2009). The main goal of the documentation process here is to produce, for further archaeological studies, a high quality documentation of the architecture and the paintings of the catacomb, based on laser scanning technique and digital photography Abdelhafiz and Zimmermann, (2011), see Fig. 12. Fig. 12: Domitilla catacomb point cloud

14 The geometry of such complex structure is recovered by a terrestrial laser scanner, while its texture is captured by a high resolution digital camera. For the geo-referencing purpose, 3D points measured by a total station are employed. These points are necessary not only to connect the output with the local coordinate system but also to reduce the number of scan positions and to control the registration of all scans. The scans are achieved by a terrestrial laser scanner Riegl LMS Z420i mounted by a digital camera. It permits a measurement range between 1 and 800 metres, measurement accuracy of 8 mm standard deviation, and measurement rate up to points per second. The scanner field of view is 80 x 360, more information about scanner specifications can be found in Riegl homepage ( The camera mounted on the scanner delivers poor textures that can t be used for high quality texture for the important mural paintings. That is why a Canon EOS 1Ds digital camera with a 14 mm objective is used to capture free hand photos from the best available positions. A professional lighting system with a flexible light panels made by KINO FLO is also used to overcome the problem of the underground light conditions. Photo realistic 3D-models for the most important models with wall paintings are textured full automatically using our own 3DImage software, see Fig. 13 for a photorealistic 3D model from the Domittila catacomb project.

15 Fig. 13: A photo realistic 3D model of one painting from the Domitilla catacomb Rom, Italy automatically textured with the 3DImage software

16 8. CONCLUSIONS In this paper, the recent techniques used for architectural documentation are presented. These techniques are laser scanning technique, photogrammetry, and their combination. The results are either coloured point clouds and/or photorealistic 3D models. In case of a coloured point cloud is required, a laser scanner with mounted camera is recommended together with the data reprocessing software 3DImage ( On the other hand, if a photorealistic 3D model is required, the 3DImage software gives also reliable documentation solution employing its developed algorithm for automatic texture mapping. The work is also extended to show successful documented international historical sites using the recent techniques and software. These sites are the Lion status in Braunschweig, Germany, the Domittila catacomb in Rom, Italy, and the Ephesus in Turkey. 9. REFERENCES Abdelhafiz, A., Integrating Digital Photogrammetry and Terrestrial Laser Scanning. PhD thesis, Institute of Geodesy and Photogrammetry, TU-Braunschweig, GERMANY. Abdelhafiz, A., and Zimmermann, N., Toward higher level of automation in texture mapping applied to Domitilla Catacombs in Rome. Cultural heritage data acquisition and processing, ISPRS working group V/2 conference, York, United Kingdom, 17th 19th August 2011 Abdelhafiz, A., Zimmermann, N., Eßer, G., and Mayer, I., Generating a Photo Realistic Virtual Model for the large Domitilla-Catacomb in Rome. 9th Conf. Optical 3-D Measurement Techniques, Vienna, Austria, 1-3 July Abdelhafiz A., and Niemeier W., Sept Developed Technique for Automatic Point Cloud Texturing from Multi Images Applied to a Complex Site. ISPRS commission V Symposium, Dresden, Germany. Al-Manasir and Fraser, Registration of terrestrial laser scanner data using imagery, The Photogrammetric Record, 09/2006, Volume 21, Issue 115, p.14, (2006)

17 Akca, D. and Gruen, A., Co-registration of pointclouds by 3D Least Squares matching. The International LIDAR Mapping Forum, Denver, Colorado, US, February (CD- ROM). Boehler, W., Bordas Vicent, M., Marbs, A., Investigating Laser Scanner Accuracy, Proceedings of XIXth CIPA Symposium, Antalya, Turkey, Sept Oct. 4. Clark, J. and Robson, S, Accuracy Of Measurements Made With A Cyrax 2500 Laser Scanner Against Surfaces Of Known Colors, XXth ISPRS Congress, Commission 4, July Istanbul, Turkey. Edelsbrunner, H., Geometry and Topology for Mesh Generation. Cambridge University Press, Cambridge. Cambridge Monographs on Applied and Computational Mathematics, No. 7, 190 pages. Kersten, Th., Sternberg, H., Mechelke, K., Investigations into the Accuracy Behaviour of the Terrestrial Laser Scanning System Trimble GS100, Optical 3D Measurement Techniques VII, Gruen & Kahmen (Eds.), Volume 1, pp Mechelke, k., Kersten, T., and Lindstaedt, M., Comparative investigations into the accuracy behaviour of the new generation of terrestrial laser scanning systems. International Conference on Optical 3-D Measurement Techniques VIII, Zurich, Switzerland, 9-12 July Owen, S. J., A survey of unstructured mesh generation technology. Proceedings 7th International Meshing Roundtable, Dearborn, Michigan, October, pp Wendt, A., Objektraumbasierte simultane Multisensorale orientierung. PhD Thesis, Institute of photogrammetry and Geoinformation, Hannover, Germany.