Camera Registration in a 3D City Model Min Ding CS294-6 Final Presentation Dec 13, 2006
Goal: Reconstruct 3D city model usable for virtual walk- and fly-throughs Virtual reality Urban planning Simulation Special effects Car navigation Objectives: Automated Photorealistic Fast Scalable
Aerial Image Registration for airborne modeling
Shortcomings of the existing approach 3D city model reconstruction from aerial LIDAR and oblique aerial photos alone High scalability, fast acquisition Manual correspondence or extensive computation for aerial photo texture mapping Automated texture mapping system is necessary
Camera registration algorithm overview Need to recover the intrinsic (focal length) and extrinsic (rotation, translation) parameters of a camera Assume zero skew, unit aspect ratio and principal point at the image center Stamp GPS and electronic compass readings to each aerial image obtain estimate of translation parameters and yaw angle(φ) Find focal length, pitch(θ) and roll(ψ) angles from vanishing points Refine estimates by projecting 3D points to an image and solving point correspondences on this image use 3D corners as feature points
Vanishing points detection literature review Existing techniques look for intersections among groups of lines Expectation Maximization Algorithm [Kosecka et al. 2002] RANSAC [Aguilera 2005] Gaussian sphere / Hough transform [Barnard 1983, Shufelt 1999] GPCA [Vidal et al. 2004] Perform well on indoor image or outdoor image with only a few buildings of simple geometry Difficult to apply to aerial image of complex urban scenes where multiple vanishing points exist
Vanishing points detection literature review
Vanishing points detection detection algorithm Iteratively find vanishing points Does not require a priori knowledge of number of vanishing points Remove line segments in each iteration Guaranteed convergence Initialize vanishing point to be intersection among nearly parallel lines Not pick up real intersection in 3D Refine vanishing point position with Levenberg-Marquardt algorithm at the end
Vanishing points detection selection algorithm 1. Fix the vanishing point with most number of segments 2. Choose two other points which make the orthocenter of the formed triangle closest to the image center Assume principal point at the image center v i v i v j v k v k v j
Vanishing point detection entire process
Camera calibration intrinsic parameter Standard uncalibrated camera model Three orthogonal vanishing points correspond to in homogenous coordinate
Camera calibration extrinsic parameters Obtained R does not belong to SO(3) R is the closest unitary matrix to R in Frobenius norm Decompose R into yaw, pitch and roll angles R = Rroll*Rpitch*Ryaw Update yaw angle from GPS reading R = Rroll*Rpitch*R yaw
3D corners detection depth map From 299 Harris corners to 189 3D corners 1. Apply Harris corner detection on digital surface model (DSM) 2. Label a Harris corner as a 3D corner when two sufficiently long lines intersect at a right angle
3D corners detection aerial image From 1964 end points to 283 3D corners (99 are real 3D corners) 1. Start from the end points of all the segments corresponding to the identified three orthogonal vanishing points 2. Label an end point as a 3D corner if there are two sufficiently long lines converging to the other two vanishing points in a region near this end point
Point correspondences on an image (?) Originally intended to run RANSAC to identify correspondence pairs based on the same fundamental matrix Vanishing point based automatic algorithm: f = 2566.2 Pitch = 50.1502 Roll = -5.0192 Manual correspondence Lowe s algorithm: f = 2555.7 Pitch = 60.5141 Roll = -0.9834
Precision analysis in a controlled experiment Fix camera pose and rotate a calibration rig Apply vanishing points based automatic calibration algorithm to find pitch and roll which should be constant pitch: 82.5 (2.2 ) roll: -2.43 (0.17 ) pitch: 66.3 (1.3 ) roll: -14.5 (0.5 )
Conclusions and future directions Developed a fast and robust vanishing point detection for complex urban scenes Examined precision of vanishing point based camera calibration Difficult to obtain accurate parameters just from vanishing points in a complex urban setting Possible improvements include additional hardware (eg. 3-axis compass) apply stereo-vision (eg. video sequence)