3D modeling, texturing and applications in Cultural Heritage Devrim Akca Chair of Photogrammetry and Remote Sensing ETH Zurich http://www.photogrammetry.ethz.ch 1
Why 3D modeling of Cultural Heritage objects? Case of documentation and taxonomy 2
Why 3D modeling of Cultural Heritage objects? Case of lost or damage Buddha of Bamiyan, Afghanistan: Before March 2001: - 53 m high - tallest representation of a standing Buddha - niche full of frescos 3D modeling from old images After March 2001: - empty niche - no more frescos - risk of collapse 3
Why 3D modeling of Cultural Heritage objects? Case of physical replica 4
Why 3D modeling of Cultural Heritage objects?. Education resources Interaction without risk of damage Virtual tourism and virtual museums Maintenance 5
3D modeling of Cultural Heritage objects Technologies Image-based techniques: (Photogrammetry) (3D) measurements from images - It can be time consuming - Possible smooth effects on small details + 3D and texture from the images + Results independent from the scene/object + Low costs Triangulation principle (at least 2 images) Range-based techniques: (Active Sensors) Sensors which provide directly 3D info - High costs - Usually no texture - Results depend on the type/quality of surface - Long processing / editing + Very fast acquisition + Able to acquire all the small details Triangulation or time delay principle Surveying & CAD tools 6
3D modeling of Cultural Heritage objects Technologies We usually need some surveying steps (scale, georeferencing) The choice is less obvious between image- and range-based Budget? Requirements? Project size? Time Which camera shall I use? Which scanner shall I use? 7
3D modeling of Cultural Heritage objects Technologies F. Remondino?? IMAGE-BASED OR RANGE SENSORS?? 8
3D modeling of Cultural Heritage objects Technologies F. Remondino?? IMAGE-BASED OR RANGE SENSORS?? 9
3D modeling of Cultural Heritage objects Technologies Image-based techniques: (Photogrammetry) Range-based techniques: (Active Sensors) Surveying & CAD tools 10
3D modeling of Cultural Heritage objects Active Sensors Technology They provide directly 3D information -> no mathematical model like imagebased approach Active sensor Triangulation: Time delay: - Structured light projection (projector + camera) - Laser light (beamer + camera) - Time-of-flight - Amplitude modulation ShapeGrabber Breuckmann Leica 11
3D modeling of Cultural Heritage objects Active Sensors - Coded Structural Light System Projecting a set of known patterns onto the object Grabbing the images with the a camera Correspondence problem solved by system calibration parameters & known geometry of the patterns (decodification) Time-multiplexing Binary codes n-ary codes Gray code + phase shifting Hybrid methods Spatial codification Non-formal codification De Bruijn sequences M-arrays Direct Codification Grey levels Color 12
3D modeling of Cultural Heritage objects Active Sensors Triangulation based systems Triangulation based systems Weight and price Speed Sensitivity to ambient light Speckle noise Penetration into object surface Imaging for texture mapping Depth of view Eye safety Laser Identical Less Larger Coded light Identical Faster Less No Yes Better Laser light Coded structured light 13
3D modeling of the Weary Herakles statue with a coded structured light system 14
The Weary Herakles Statue - Story Antalya Boston Marble statue of the Greek demi-god Herakles (2 nd c.ad). Copy of the original bronze statue of famous sculptor Lyssipos of Sicyon (4 th c.bc) Broken in two parts. The upper half, seen in the USA in the early 1980s (Boston Museum of Fine Arts). The lower half, excavated in Perge (Antalya, TR) in 1980 by Prof. J. Inan, (now in the Antalya Museum). 15
The Weary Herakles Statue - Story According to Turkish law, Turkish antiquities state property since Ottoman times 1906. The Turkish government asked the upper half. The Boston MFA refused the petition, saying that: the statue may have broken in ancient times and the upper torso may have been taken from Turkey before the year 1906. 16
Aim of the Project The Aim To record and model both the lower and the upper part and bring these partial models together in the computer, so that at least there the complete statue could be seen, appreciated and analyzed. The lower part in the Antalya Museum was scanned in September 2005. Access to the upper part in the Boston MFA was denied. The Project In cooperation with 17
Herakles in Mythology In Greek Mythology: Herakles (or Heracles) In Roman Mythology: Hercules Naples National Archaeological Museum Herakles Farnese type, due to similarity to a complete copy in Naples National Archaeological Museum (Italy). Mythical hero, demigod, best known for his superhuman strength. After to perform twelve great tasks (The Twelve Labours of Herakles), became a god. The first task: strangling the Nemean Lion. Killed, and used the its skin as armor. Mostly portrayed nude & leaning (hence Weary Herakles), with lion s skin. 18
Data Acquisition Digitization in the Antalya Museum in September 2005 Breuckmann optotop-he coded structural light system 19
Coded Structural Light System Coded Structural Light Technique A kind of active stereo triangulation technique for surface measurement Replacing one of the cameras with a pattern projector Figure from Dr. B. Breuckmann Projecting a set of known patterns onto object Grabbing the images with the other camera Correspondence problem solved by system calibration parameters & known geometry of the patterns (decodification) 20
Coded Structural Light System - Pattern encoding techniques Time-multiplexing Binary codes n-ary codes Gray code + phase shifting (Breuckmann) Hybrid methods Spatial codification Non-formal codification De Bruijn sequences M-arrays Direct Codification Grey levels Color Salvi et al., 2003 http://eia.udg.es/~jpages/reportcodedlight03.pdf 21
Coded Structural Light System - Gray code + phase shifting Gray code: (Frank Gray, 1953) A sequence of (Gray encoded) binary fringe patterns are projected, dividing into sections. A codeword is associated for each pixel, establishing the correspondence: image pixel -> projector stripe no 011 3D coordinates by triangulation Resolution limit, half the size of the finest pattern Phase shifting: A periodical pattern (sinusoidal) is projected several times by shifting it in one direction Phase unwrapping Each camera pixel -> projector stripe number (sub-stripe accuracy) http://eia.udg.es/~jpages/examples/examples.html Figure: Line shifting, Gühring, 2001 22
The Scanner: Breuckmann optotop-he Gray code + phase shifting Field of view (FOV): 480x360 mm measuring depth: 320 mm acquisition time: <1sec. weight: 2-3 kg digitization: 1280x1024 = 1.25M points base length: 600 mm triangulation angle: 30 0 Lateral resolution: ~360 microns feature accuracy: 1/15000 of img. diagonal feature accuracy: ~50 microns http://www.breuckmann.com 23
Scanning in the Antalya Museum Breuckmann optotop-he system 1 ½ days on site work with 67 scans (56+11) Each scan 1.25M points Totally 83.75M points preparation scanning 24
Scanning in the Antalya Museum optotop-he, very flexible system 25
Postprocessing Workflow Registration + Pairwise registration + Global registration (LS3D) Point cloud editing (Geomagic Studio 6) + Cropping the Area Of Interest + Noise reduction + Down-sampling Surface triangulation and editing (Geomagic Studio 6) Texture Mapping (VCLab s 3D Scanning Tool, CNR, Pisa) Visualization (PolyWorks 9.0.2) 26
Registration Pairwise registration 234 consecutive pairwise LS3D matching. The average sigma naught is 81 microns. Example: Registration of 1 st and 2 nd scans Note: 3x3 down-sampling for better visualization 27
Registration Global registration Global registration with the block adjustment by independent models solution Sigma naught 47 microns, in agreement with the system specifications Example: Registration of first 10 scans Note: 3x3 down-sampling for better visualization 28
Point Cloud Editing Noise reduction Registration (ALL= 83.75M = 87.8 million points) Merging all as one XYZ file, discarding the NODATA points (36.2 million points) Cropping the AOI (33.9 million points) Noise reduction with Reduce Noise function (medium level) 29
Point Cloud Editing Down-sampling Down-sampling with Curvature Sampling function, eliminates points at flat regions and preserves at hi-curvature regions (9.0 million points) 30
Surface Triangulation and Editing Finally 9.0 million points => 5.2 million triangles Memory problems with Geomagic if greater number of target triangles, e.g. 10 million Data holes due to complexity & inner concave parts Filling the holes is the most tedious step of the project 31
Texture Mapping Leica Digilux1, 4Mpixel CCD camera The Veawer module of VCLab s 3D Scanning Tool (ISTI-CNR, Pisa, Italy) 32
Visualization (gray shaded) Better lighting & shading with PolyWorks IMView. 33
Visualization Back projection of the 3D model into image space http://www.photogrammetry.ethz.ch/research/herakles/ 34
Conclusions The coded structural light system is a mature technology and allows high resolution documentation of cultural heritage objects. The hardware component, optotop-he worked well. Editing the surface is the most tedious step of the whole modeling pipeline. Need for sophisticated algorithms & software. Texture mapping is not fully available in either software. http://www.photogrammetry.ethz.ch/research/herakles/ 35
Conclusions http://www.photogrammetry.ethz.ch/research/herakles/ 36
3D Modeling of a Khmer Head 37
The Khmer head project Bodhisattva Head (the Lord of compassion who looks down on the suffering of the world) Cambodia - Khmer period Bayon style, 12 th -13 th a.c. 28 cm in height Sandstone Collection of Rietberg Museum, Zurich
The Khmer head project Data acquisition Data acquisition: 3-4 hours on site work Breuckmann OptoTOP-SE coded structural light system 18 scans, each scan 1.3 million points (totally 23.6 million points)
The Khmer head project Data processing Point cloud registration 52 Pairwise registration with the Least Squares 3D Surface Matching (LS3D) method + global registration (final 28 microns sigma0 value accuracy) Modeling Data reduction Surface generation Editing Texturing Geomagic Studio 3.9 million triangles PolyWorks 0.6 million triangles
The Khmer head project Texture mapping & Viz Special illumination conditions to avoid shadows
3D Modeling of the pre-inca site Pinchango Alto, Peru 42
Experimental Results (#7): Pinchango Alto 300 m 150 m Aerial Image area: Pinchango Alto, a pre-inkaic site in Peru scanner: Riegl LMS Z-420i average point spacing 1~30 cm, changing with range Data set is courtesy of Riegl GmbH. 43
Experimental Results (#7): Pinchango Alto The data set: large data set (57 scans with 144 million points) large occlusions due to topography 44
Experimental Results (#7): Pinchango Alto Tot. no. points Aver. no. iters. Average sigma Global registration 144M ~7-8 ~1 cm 0.5 cm 45
Experimental Results (#7): Pinchango Alto Difficulties in modelling: Geomagic Studio 6, memory limitations in surface wrapping data holes due to occlusions of walls and hollows non-static objects on the site during 5 days field work 46
3D Modeling of Alfred Escher statue in Zurich 47
Scanner: FARO LS880 HE80 Range: 0.6-76 meters FOV: 320 0 x360 0 Accuracy: ±3 mm @ 10 meters 48
49
Scanning at 2 nights: 30 March 2007, Friday, 01:00am 05:15am 1 April 2007, Sunday, 01:00am 05:00 am 36 scans with 4.4 million points in the AOI 50
Intensity image of a scan 51
Modeling with Geomagic Studio 52
53
Technology selection (a recall) Budget? Requirements? Project size? Time right selection for each object 54
Technology selection (terrestrial case) Passive sensors Active sensors Laser + Triangulation Structured light + Triangulation Laser + TOF 0.3 5K E costs Ca 50K E Ca 50K E Ca 100K E fast acquisition time minutes seconds minutes small large field of view small (very) small large Fast with the right algorithms/sw Time consuming if manual processing time time consuming 55
Technology selection Passive sensors Active sensors ca 4 hours (some mm) ca 4 days (ca 50-100 μm) 56
Thank you for your interest! http://www.photogrammetry.ethz.ch 57