Image-based 3D Modeling Close-Range Photogrammetry: Extraction of 3D Geometry & Quality Information

Image-based 3D Modeling Close-Range Photogrammetry: Extraction of 3D Geometry & Quality Information Athanasios D. STYLIADIS - EPOCHE 2013

Image-based 3D Modeling (A) Introduction (B) CLOSE-RANGE PHOTOGRAMMETRIC SURVEY (C) SURFACE RECONSTRUCTION (D) Texture-Mapping & VISUALIZATION (E) Applications Architecture (Landscape) Archaeology (F) Conclusions & Web Resources The case-study: single three-point perspective historic photography-based CAAD modeling (amateur camera calibration, pose and 3D reconstruction) of manmade environments, buildings and monuments, rich in geometrical regularity.

(A) Introduction 3D ACQUISITION SYSTEMS 3D Modeling (Range-Based Techniques) GPS Lidar Total Station Laser Scanner (Computer Graphics Techniques) Satellite Imagery Arial Photogrammetry Close-Range Photogrammetry Historical Photography POSSIBLE OUTPUTS Textured 3D models of objects/buildings at high or low level of details (3D: Architecture, Landscape Architecture, Archaeology) 3D models with semantic information (3D: GIS, Archaeology) Extraction of Plans and Sections (2D: Architecture, Archaeology) Orthophotos & Rectified Photos for Elevations (2D: Architecture)

IBMR* (Image-based Modeling and Rendering) Image-based Modeling Techniques all rely on some type of visual cues, the following is a list of example visual cues that some of these techniques rely on. Visual Cues Shading Texture Motion Focus Highlights Shadows *This is quite a wide field that covers various topics such as: Silhouette Modeling geometric objects from images, inverse lighting, Inter-reflections light fields, layered depth images, 3D reconstruction, etc Symmetry Light polarization Features such as corners and edges

Image-based 3D Modeling Outputs 3D 2D

Image-Based 3D Modeling DEFINITION Photogrammetry is the science, and art, of determining the size and shape of objects as a consequence of analysing images recorded on film or electronic media. Albrecht Durer (Close Range Photogrammetry and Machine Vision K.B. Atkinson, 2003)

Image-based 3D Modeling APPLICATIONS in: Architecture Landscape Architecture Archeology Cultural Heritage Engineering 3D City Modeling Planning Geology Film & Animation Accident Reconstruction Forensic Medical & Biological

Image-based 3D Modeling The Equipment Hardware

Image-based 3D Modeling The Software

So, what are the most important Steps to carry out a Photogrammetric Survey? The traditional pipe-line is as follows: Project Requirement Network Design Camera Calibration Image Acquisition Camera Orientation (Recording) Close-range form 3D Measurements

(B) Close-Range Photogrammetric SURVEY. B.1 Project Requirements B.2 Network Design B.3 Camera Calibration B.4 Image Acquisition B.5 Camera Orientation B.6 Close-range form 3D Measurements

B.1 Project Requirements Close co-operation with client or contractor Defining object or area to be measured Type of output results required Resolution of fine details (smallest object feature to be measured) Scale Coordinate System Accuracy to be achieved (it has to fit the purpose) Time schedule for on-site work, data processing and output delivery Staff and cost management

B.2 Network Design Image Scale Resolution / Image quality Object environment / surroundings Depth of field Imaging angle Number and distribution of image points Intersection angles / Ray intersection Field of view Visibility / Occlusion / Self- Occlusion

B.3 Camera Calibration (Internal Orientation) Camera's Focal Length Lens Distortion (Radial & Decentering) Sensor Width and Height (W, H) Principal Point (xp, yp)

B.3 Camera Calibration (Internal Orientation) Test-field Calibration Self-Calibration

B.4 Image Acquisition Single Image Stereo Image Multi Image

B.5 Camera Orientation (External Orientation) The Exterior Orientation consist of Six (6) Parameters, which describe the Spatial Position (x, y, z) and the Orientation (ω, φ, κ) of the Camera s Co-ordinate system with respect to the global object Co-ordinate System (X, Y, Z). The Viewing Direction The Object Co-ordinate System The Camera Co-ordinate System

(C) Surface RECONSTRUCTION Free-Form Shapes: Point Clouds The MeshLab Software Pre-processing operations Data sampling Noise reduction Hole filling Data registration Mesh generation Post-processing operations Data smoothing Data decimation Hole filling

(D) Texture Mapping & VISUALIZATION Regular Geometric Shapes

(D) Texture Mapping & VISUALIZATION Free-form Shapes

(D) Texture Mapping & VISUALIZATION Export to a number of different File Formats.dxf.ply.obj.dae (Collada).skp (SketchUp).kml.kmz (Google Earth).pdf (3D).3ds etc. (other video formats)

(E) Applications The Façade Software

(F) Conclusions & Web Resources Benefits: Mobility of the system Multiple distance range Low cost No restriction for type of surface (as far as they are textured) Accuracy and resolution of a Laser Scanner Problems: Not suitable for very large and complex sites (best used to integrate with other techniques) Lighting conditions Web Resources www.photomodeler.com/ meshlab.sourceforge.net/ cipa.icomos.org/ www.isprs.org/publications/archives.aspx www.isprs.org/technical_commissions/wgtc_5.aspx 3dom.fbk.eu/en/home www.toposmagazine.com/ (Int l Review of LA & Urban Design) www0.cs.ucl.ac.uk/staff/d.nahmias/log.html (Overview of Image-based Modeling Techniques) graphics.stanford.edu/ www.pauldebevec.com/; www.debevec.org www.pauldebevec.com/research/ The Façade Application (Modeling & Rendering Architecture from Photographs) www.robots.ox.ac.uk/~vgg/publications/papers/crimi nisi00a.pdf (Single View Metrology, A. Criminisi, I. Reid, A. Zisserman)

A Case-Study Photo-based 3D Modeling Historical photography-based Computer-Aided Architectural Design: Demolished buildings Information Modeling with reverse engineering functionality Athanasios D. STYLIADIS - EPOCHE 2013 (Thassos Island)

Geometry (Shape) from Single View There are image-based modeling techniques that rely solely on a single image for its input. These techniques require such little input data they usually necessitate user interaction. In this domain: A close-form technique is presented by A. Criminisi, I. Reid and A. Zisserman (ICCV 99) which makes use of the following assumptions: Three (3) Orthogonal Sets of Parallel Lines Four (4) known Points on ground plane One (1) Height in the scene An alternative technique for single view that also Scales to Multiple Views is presented by Paul Debevec and used in his Façade application. Another interesting approach is presented in Image-Based Modeling and Photo Editing. Also, in Potential and limitation for the 3D documentation of cultural heritage from a single image certain commercially available software capable of 3D reconstruction from a single view is examined.

Geometry (Shape) from Single View Refs: Potential and limitation for the 3D documentation of cultural heritage from a single image André Streilein & Frank A. van den Heuvel Single View Metrology A. Criminisi, I. Reid & A. Zisserman Modeling and Rendering Architecture from Photographs Paul Ernest Debevec Modeling and Rendering Architecture fromphotographs: A hybrid geometry- and image-based approach Paul E. Debevec, Camillo J. Taylor & Jitendra Malik Recovering Arches in Façade using Ray - Plane intersections in 3-D G. D. Borshukov & P. Debevec Image-Based Modeling and Photo Editing Byong Mok Oh, Max Chen, Julie Dorsey & Frédo Durand (Computer Science Lab, MIT)

The Aghios Nikolaos Tranos Cathedral Thessaloniki, Greece T.S. Mantopoulou- Panagiotopoulou, The monastery of Aghios Menas in Thessaloniki, Dumbarton Oaks (Trustees for Harvard University) papers, 50 (1996) 239-262. G. Moutsopoulos, Thessaloniki 1900-1917, M. Molho Editions [in Greek]. E. Marki-Aggelkou, Aghios Nikolaos Tranos excavation, Macedonika 19 (1979) 271-298. T.S. Mantopoulou- Panagiotopoulou, New data for the Aghios Nikolaos Tranos church in Thessaloniki (1863) - A metabyzantine typology approach, Makedonika 29 (1993-1994) 132-174. A non-existing today meta-byzantine church (1863) -The only one available historic photography! The image of this photography was digitized with resolution 484 x 319 pixels and then has been published to the Internet by the Municipality of Thessaloniki (www. thessalonikicity.gr/eikones). Also, from a recent excavation, the metric data (dimensions) of this church were found to be 35.50m x 15.50m.

The great Thessaloniki fire (August, 1917) The church was demolished in the great Thessaloniki fire on August 18th (NS) or August 5th (OS), 1917.

The proposed method, takes profit from the presence in the image (historic photography) of: Three (3) Vanishing Directions, and Two (2) Orthogonal Object-Edges with known length ration and then focuses on the graphical estimation of the skew intrinsic parameter of the uncalibrated camera (i.e. the angle of dot's x and y optical axes, in photography plane), dealing in this way even with the skew presence case (non-rectangle dot). The presence of Skew is not a negligible factor in historic photography of early 20th century years, due to dot optical axes failure (carelessness manufacturing) or collapse, as well as the twist effect (distortion) from the undocumented film development process in these years.

The graphical recovery of the skew factor is the main contribution of the presented method to the Pose and CAAD literature. It is shown that one single three-point perspective amateur photography, even with the presence of skew, is adequate for calibration, pose and planar structure (building façades) recovery, if the usual in building's architecture geometric clues are present (i.e. planarity, orthogonality and parallelism) and some metric data (e.g. length and width of demolished building's dimensions) are available. The presented method is of interest for: Architecture, Landscape Architecture, Archaeology, Reverse Engineering, and Virtual Reality

The proposed method was validated on a simulated cuboid and demonstrated on a number of demolished historical buildings for which only one uncalibrated (and skewed) photography was available. The accuracy evaluation shows that the method is suitable for CAAD modeling applications regarding demolished buildings and monuments of the early 20th century years (2% relative accuracy, i.e. 40cm for a 20m façade, included the metric data inaccuracy). According to the proposed method results (mean values of 100 trial-and-error processes), the historic photography was acquired by a camera with: Focal Length f=33.33mm, Aspect Ratio r=1.03, Field of View FoV=43.65, Principal Point co-ordinates u x =20.25mm, u y =13.35mm, Skew Angle α = 89.07 and Film Format 40.5mm (width) x 26.7mm (height)

Also, the recovered camera pose (historic photography vantage point) has the following values for its six orientation parameters: X 0 =14.79m, Y 0 =-27.73m, Z 0 =-1.78m, ω=15.96, φ=35.28 and κ=1.52. The error (measured as in the previous Section in respect with the known building's dimensions) in the metric data for camera pose and façades 3D control points, is approximately: 0.32cm and 0.40 for historic photography vantage point pose recovery and 0.65cm for façades 3D control points recovery (i.e. a relative error in the worst case of 2% for this 35.50m x 15.50m church).

The EU Tax sufferers, The European Union, Thanks! The LLP Erasmus Programme & All the EPOCHE 2013 Partners (Teachers and Students)