Multiple-View Reconstruction from Scene Knowledge 3D Reconstruction from Scene Knowledge SYMMETRY & MULTIPLE-VIEW GEOMETRY Fundamental types of symmetry Equivalent views Symmetry based reconstruction MUTIPLE-VIEW MULTIPLE-OBJECT ALIGNMENT: Scale alignment: adjacent cells in a single view Scale alignment: same cell in multiple views APPLICATIONS & Experiments: Building 3D geometric models Symmetry extraction, detection, and matching Camera calibration 1 2 Scene knowledge and symmetry Scene knowledge and symmetry Parallelism (vanishing point) Orthogonality Translational invariance Rotation and reflection Symmetry is ubiquitous in man-made or natural environments 3 4
Wrong assumptions SYMMETRY & MUTIPLE-VIEW GEOMETRY Ames room illusion Necker s cube illusion Why does an image of a symmetric object give away its structure? Why does an image of a symmetric object give away its pose? What else can we get from an image of a symmetric object? 5 6 Equivalent views from rotational symmetry Equivalent views from reflectional symmetry 90 O 7 8
Equivalent views from translational symmetry Recovery using rectangular structure Recovery of the camera displacement from a planar rectangular structure Rectangle reflectional symmetry Square rotational and reflectional symmetric Special cases of symmetric objects 9 10 Camera pose recovery Homography Estimation Assume partially calibrated camera Decouple known and unknown structure Explicitely parametrize the homography Estimate the unknown homography Explicitely parametrize the unknown structure 11 12
z Homography Factorization Example Exploiting orthogonality constraints 100 Directly estimate the focal length 200 300 400 0 20 40 60 80 100 500 50 120 y 0 50 600 50 x 0 50 100 150 200 250 300 350 400 450 100 50 remaining parameters and final pose With shared segment the pose can be reconciled and we obtain single consistent pose recovery up to scale and error ~ 3 degrees 13 14 Example for multiview matching Example Recovery of the camera displacement from a planar structure 15 Recovery of the camera displacement from a planar structure 16
Symmetric multiview geometry (chap 10.) Definition. A set of 3-D features S is called a symmetric structure if there exists a non-trivial subgroup G of E(3) that acts on it such that for every g in G, the map Symmetry-based reconstruction (experiments) Rotational symmetry is an (isometric) automorphism of S. We say the structure S has a group symmetry G. Detailed treatment in chapter 10 (some examples next) 17 18 Symmetry-based reconstruction (experiments) Symmetry-based reconstruction (experiments) Reflectional symmetry Translational symmetry Algorithm 1. Recover R,T from homography constraint Algorithm 1. Recover T from homography constraint (no rotation in this case) known, recovered Projection of the origin of the Symmetric structure 19 20
Alignment of multiple symmetric structures Alignment of multiple symmetric structures Pick the image of a point on the intersection line? 21 22 Alignment of multiple symmetric structures Alignment of multiple symmetric images 23 24
Alignment of multiple symmetric images Alignment of multiple symmetric images 25 26 Alignment of multiple symmetric images APPLICATION: Symmetry detection and matching Extract, detect, match symmetric objects across images (over an arbitrary motion), and recover the camera poses. 27 28
Segmentation & polygon fitting Symmetry verification & pose recovery Symmetry verification (rectangles) Color-based segmentation (mean shift) Polygon fitting. Single-view pose recovery 29 30 Symmetry-based matching Pose and 3D recovery Only one set of camera poses is consistent with all correctly detected symmetry objects 87.6 o Accuracy (aspect ratios) Whiteboard Table Reconstruction 1.54 1.01 Ground Truth 1.51 1.00 31 32
Multiple-view matching and recovery APPLICATION: Building 3D geometric models 33 34 Building 3D geometric models (camera poses) Building 3D geometric models (rendering) 35 36
APPLICATION: Calibration from symmetry Calibrated homography Uncalibrated homography APPLICATION: Calibration from symmetry Calibration with a rig is also simplified: we only need to know that there is sufficient symmetries, not necessarily the 3D coordinates of points on the rig. (vanishing point) 37 38 SUMMARY Multiple-view 3-D reconstruction in presence of symmetry Symmetry based algorithms are accurate, robust, and simple. Methods are baseline independent and object centered. Alignment and matching can and should take place in 3D space. Camera self-calibration and calibration are simplified and linear. Related applications Using symmetry to overcome occlusion. Reconstruction and rendering with non-symmetric area. Large scale 3D map building of man-made environments. 39