A PROBABILISTIC NOTION OF CAMERA GEOMETRY: CALIBRATED VS. UNCALIBRATED

Size: px

Start display at page:

Download "A PROBABILISTIC NOTION OF CAMERA GEOMETRY: CALIBRATED VS. UNCALIBRATED"

Reynold Parrish
6 years ago
Views:

1 A PROBABILISTIC NOTION OF CAMERA GEOMETRY: CALIBRATED VS. UNCALIBRATED Jutin Domke and Yianni Aloimono Computational Viion Laboratory, Center for Automation Reearch Univerity of Maryland College Park, MD, 274, USA domke/, yianni/ KEY WORDS: Correpondence, Matching, Fundamental Matrix, Eential Matrix, Egomotion, Multiple View Geometry, Probabilitic, Stochatic ABSTRACT: We ugget altering the fundamental trategy in Fundamental or Eential Matrix etimation. The traditional approach firt etimate correpondence, and then etimate the camera geometry on the bai of thoe correpondence. Though the econd half of thi approach i very well developed, uch algorithm often fail in practice at the correpondence tep. Here, we ugget altering the trategy. Firt, etimate probability ditribution of correpondence, and then etimate camera geometry directly from thee ditribution. Thi trategy ha the effect of making the correpondence tep far eaier, and the camera geometry tep omewhat harder. The ucce of our approach hinge on if thi trade-off i wie. We will preent an algorithm baed on thi trategy. Fairly extenive experiment ugget that thi trade-off might be profitable. 1 INTRODUCTION The problem of etimating camera geometry from image lie at the heart of both Photogrammetry and Computer Viion. In our view, the enduring difficulty of creating fully automatic method for thi problem i due to the neceity to integrate image proceing with multiple view geometry. One i given image a input, but geometry i baed on the language of point, line, etc. Bridging thi gap- uing image proceing technique to create object ueful to multiple view geometry- remain difficult. In both the Photogrammetric and Computer Viion literature, the object at interface between image proceing and geometry i generally correpondence, or matched point. Thi i natural in Photogrammetry, becaue correpondence are readily etablihed by hand. However, algorithmically etimating correpondence directly from image remain a tubbornly difficult problem. One may think of mot of the previou work on Eential or Fundamental matrix etimation a falling into one of two categorie. Firt, there i a rather mature literature on Multiple View Geometry. Thi i well ummarized in Hartley and Zierman recent book (Hartley and Zierman, 24), emphaizing the uncalibrated technique leading to Fundamental Matrix etimation. Specifically, there are technique for etimating the Fundamental Matrix from the minimum ofeven correpondence (Bartoli and Sturm, 24). In the calibrated cae, the Eential Matrix can be efficiently etimated from five correpondence (Nitér, 24). Given perfect matche, it i fair to ay that the problem i nearly olved. The econd category of work concern the etimation of the correpondence themelve. Here commonly a feature detector (e.g. the Harri corner detector (Harri and Stephen, 1988)) i firt ued to try to find point whoe correpondence i mot eaily etablihed. Next, matching technique are ued to find probable matche between the feature point in both image (e.g. normalized cro correlation, or SIFT feature (Lowe, 24)). Thee are active reearch area, and progre continue up to the preent. Neverthele, no fully atifactory algorithm exit. Current algorithm often uffer from problem uch a change in cale or urface orientation (Schmid et al., 2). Furthermore, there are many ituation in which it i eentially impoible to etimate correpondence with out uing a higher-level undertanding of the cene. Thee include repeated tructure in the image, the aperture effect, lack of texture, etc. When human etimate correpondence, they ue thi high-level information. Neverthele, it i unavailable to algorithm. Reearch in multiple view geometry, of coure, ha conidered the difficultie in the underlying algorithm for correpondence etimation. A uch, robut technique uch a RANSAC (Fichler and Bolle, 1981) are traditionally ued to etimate a camera geometry from a et of correpondence known to include many incorrect matche. Thee technique are fairly ucceful, but becaue even inlying correct matche include noie there i a difficulty in dicriminating between inlying matche with noie, and outlying, wrong, matche. When imultaneouly adjuting the camera geometry, and 3-D point in a final optimization, bundle adjutment method frequently ue more ophiticated noie model which moothly account for due to both noie, and outlying matche (Trigg et al., 1999). In thi paper, we ugget that it i worth tepping back and reconidering if correpondence are the correct tructure to ue at the interface between image proceing and multiple view geometry. Point correpondence are natural in Photogrammetry becaue they are eaily etimated by human. Neverthele they are very difficult to etimate algorithmically. Here, we ugget intead uing. We can ee immediately that thi make the image proceing ide of the problem much eaier. If repetitive tructure or the aperture effect preent itelf, it i imply incorporated into the probability ditribution. We will preent a imple, contrat invariant, technique for etimating thee correpondence from the phae of tuned Gabor filter. The more difficult ide of thi trategy concern multiple view geometry. One mut etimate the camera geometry from only di-

tribution of correpondence. A we will ee, one can quite eaily define a probability for any given camera motion, from only thee ditribution of correpondence.

In practice, thi pace ha a imilar tructure to the leat-quare epipolar pace, (Olieni, 25) in that it contain relatively few local minima. 1.1 Previou Work Other work ha aked imilar quetion.

Thee technique then find motion which are compatible with thee feature, in the ene that each feature tend to have a compatible feature along the epipolar line in the econd image.

2 tribution of correpondence. A we will ee, one can quite eaily define a probability for any given camera motion, from only thee ditribution of correpondence. We then preent a heuritic nonlinear optimization cheme to find the mot probable geometry. In practice, thi pace ha a imilar tructure to the leat-quare epipolar pace, (Olieni, 25) in that it contain relatively few local minima. 1.1 Previou Work Other work ha aked imilar quetion. Firt, there are technique which generate from image feature point, and local image profile, with out etimating an explicit correpondence (Makadia et al., 25). Thee technique then find motion which are compatible with thee feature, in the ene that each feature tend to have a compatible feature along the epipolar line in the econd image. Here, α repreent the probability that the information given by the Gabor filter i mileading. Thi would be the cae, for example, were the point to become occluded in the econd image. Notice that adding the contant of α i equivalent to combining the ditribution with the flat ditribution in which all point q are equally likely. In all experiment decribed in thi paper, we have ued α = 1. Correpondence ditribution for everal image are hown in Figure 2. Other work ha created weaker notion of correpondence, uch a the normal flow. If a point i along a texturele edge in one image, local meaurement can only contrain it to lie along the ame edge in the econd image. Thi contraint i eentially the normal flow, and algorithm exit to etimate 3-D motion directly from it (Brodky et al., 2). Though thee technique will not uffer from the aperture effect, they cannot cope with ituation uch a repeated tructure in the image. It i alo important to notice that the normal flow will give up information unnecearily at point which do not happen to uffer from the aperture effect. 2 CORRESPONDENCE PROBABILITY DISTRIBUTIONS Given a point q in the firt image, we would like the probability that thi correpondence mot cloely to each pixel in the econd image. It i important to note that there i no obviou way to ue traditional matching technique here. Wherea traditional technique try to find the mot probably point correponding to q, we require the relative probabilitie of all point. Our approach i baed on the phae of tuned Gabor filter. Let φ l,γ () denote the phae of the filter with cale l and orientation γ at point. Now, given a ingle filter, (l, γ), we take the probability that correpond to a given point to be proportional to exp((φ l,γ () φ l,γ ()) 2 ) + 1. (1) Figure 1: Correpondence Probability Ditribution. Left: Firt image, with point in conideration marked. Center: Second image: Right: Probability ditribution over the point in the econd image, with probability encoded a color. Combining the probability ditribution given by all filter then yield the probability that correpond to, which we denote by ρ (). ρ () exp((φ l,γ () φ l,γ ()) 2 ) + 1. (2) l,ω Note here that correpond to a particular pixel in the econd image. Since we are computing probabilitie over a dicrete grid, we approximate the probability that correpond to an arbitrary point, having non-integer coordinate, though the ue of a Gauian function. ρ (q) max ρ () exp( q 2 ) + α (3) 3 ESSENTIAL AND FUNDAMENTAL MATRIX ESTIMATION Given the correpondence ditribution, we will define natural ditribution over the pace of the Fundamental and Eential Matrice. Becaue the pace of thee matrice are of high dimenion (7 and 5 repectively), it i impractical to attempt to calculate a full ditribution, by ampling. It i poible that future work will directly ue thee ditribution. Neverthele, we ue a imple heuritic optimization to maximize the probability in the Eential or Fundamental Matrix pace. Thi make it poible to examine the behavior of thee ditribution more eaily. 3.1 Fundamental and Eential Matrix Probability Given the correpondence ditribution for a ingle point, ρ ( ), we define a ditribution over the pace of fundamental matrice.

3 ρ(f) max ρ (q) (4) q:q T F = Thu, the probability of a given Fundamental Matrix F i proportional to the maximum probability correpondence compatible with the epipolar contraint. Now, to ue all correpondence ditribution, imply take the product of the ditribution given by each point. ρ(f) max q:q T F = ρ (q) (5) Subtituting our expreion for ρ (q) from Equation (3), we obtain ρ(f) [ max q:q T F = Rearranging term, thi i ρ(f) [max ρ () max ρ () exp( q 2 ) + α]. (6) max q:q T F = exp( q 2 ) + α]. (7) Notice here, that we do not need to explicitly find the point q. Only required i max q:q T F q. Notice that thi i exactly the minimum ditance of the point from the line F. Therefore, we can write the probability of F in it final form. ρ(f) [max ρ () exp( ( T l (F,) ) 2 ) + α] (8) Here, l (F,) i the line F normalized uch that r T l (F,) give the minimum ditance between r and the line F on the plane z = 1. If F i i the ith row of F, then l (F,) = F (F1) 2 + (F 2) 2. (9) When earching for the mot probable F, a parameterization of the fundamental matrice i required. We found it convenient to ue three parameter f, p x, and p y repreenting the focal length, and x and y coordinate of the principal point. Next, keeping the magnitude of the tranlation vector t fixed to one, we took two parameter to parameterize it axi and angle. Finally, we ued 3 parameter to repreent the rotation vector ω. Thi correpond to a rotation of an angle ω about the axi ω/ ω. K = [ f px f p y 1 ] (1) E = [t] R(ω) (11) F = K T EK 1 (12) Notice there are a total of 8 free parameter, depite the fact that the Fundamental Matrix ha only 7 degree of freedom. Though thi preent no problem to the etimation of F, it doe mean that an ambiguity i preent in the underlying parameter. To extend thi to the calibrated cae, we take K to be known. Thu, there are now 5 free parameter: 2 for the tranlation t, and 3 for the rotation ω. It would be trivial to extend thi to the cae that only certain calibration parameter were known, or to include a contant for camera kew. 3.2 Optimization To explore the behavior of the probability ditribution over the Fundamental and Eential Matrice, we will ue a heuritic optimization to try to find arg max F ρ(f) and arg max E ρ(e), repectively. The optimization proceed a follow: Firt, elect N random point in the Fundamental or Eential matrix pace. Evaluate ρ(e) or ρ(f) at each of thee point. Next, take the M highet coring point, and run a nonlinear optimization, initialized to each of thee point. We have ued both Simplex and Newton type optimization, with little change in performance. The final, highet coring point i taken a the max. For the calibrated cae, we have found that uing N = 25 and M = 25 wa ufficient to obtain a value very near the global maximum in almot all cae. A in the cae for the tandard leat-quare urface (Olieni, 25) (Tian et al., 1996), there are generally everal, but only everal local minima. Uually, a ignificant number of the nonlinear earche lead to the ame (global) point. In the uncalibrated cae, we ued N = M = 1. (Thu earche are taken from 1 random point.) We found that it wa neceary to increae M to 1 to obtain reaonable certainty of obtaining the global maximum. At the ame time, we found that increaing N did not improve reult, and may even be counterproductive. Still, the pace of ρ(f) appear to have more local minima, and even thi increaed method doe not alway appear to achieve the global maximum. 4 EXPERIMENTS To analyze the performance of the framework, we prepared three different 3-D Model with the POV-Ray oftware. Each model wa choen for it difficulty, including repetitive tructure, lack of texture, or little image motion. The ue of ynthetic model make the exact motion and calibration parameter available. For each model, we generated two different image equence- one with a forward motion, and one with a motion parallel to the image plane. For each image pair, 1, were created. Next, the calibrated and uncalibrated algorithm were both run acro a range of input ize. For each input ize, 1 random ubet of the correpondence were generated, and the algorithm wa run on each input. In the calibrated cae, the meaurement of i imple. Let the true tranlation vector be t, normalized o that t = 1. Let the vector parameterizing the true rotation matrix be ω. The metric we ue are imply the Euclidean ditance between the etimated and true motion vector, t t, and ω ω repectively. For each input ize, mean are taken over the for all reulting motion etimate.

In the uncalibrated cae, we mut meaure the of a given fundamental matrix F. Commonly ued metric uch a the Frobeniu norm are difficult to interpret, and allow no comparion to the calibrated cae.

4 In the uncalibrated cae, we mut meaure the of a given fundamental matrix F. Commonly ued metric uch a the Frobeniu norm are difficult to interpret, and allow no comparion to the calibrated cae. Intead, we ue the known ground truth calibration matrix K to obtain E. (Hartley and Zierman, 24) E = K T FK (13) Next, Singular Value Decompoition i ued to decompoe E into the tranlation and rotational component, E = [t] R(ω). From thi, it i imple to recover the underlying motion parameter, t and ω. The i then meaured in the ame way a the calibrated cae. Reult for the Cloud, Aby, and Bicuit model are hown in figure 2, 3, and 4 repectively. Several obervation are clear from the data. Firt, motion etimation i alway more accurate when the epipole i in the middle of the image than when it i parallel to it. Surpriingly, perhap, neither the calibrated nor uncalibrated approach clearly outperform the other. The performance of the uncalibrated approach relative to the calibrated approach i better when the epipole i further from the image. Two frame from the Catle equence, along with the epipolar line are hown in Figure 5. Two frame from the popular Oxford Corridor equence are hown in Figure 6. In both cae, approximately 2 correpondence ditribution were ued. Though no ground truth calibration or motion i available, the reader can oberve the cloe correpondence among epipolar line. The running time of the algorithm i dominated by the time to generate correpondence ditribution. In practice, the motion etimation tep run on the order of a minute on a modern laptop. "Cloud" model, t = [ 1 ] T, ω [.35 ] T 5 CONCLUSIONS AND FUTURE WORK With real camera, neither the fully calibrated, nor fully uncalibrated approach i fully realitic. In practice, one ha ome idea of the calibration parameter, even if only from knowledge of typical camera. At the ame time, even when a camera i calibrated, the true calibration i not found exactly. It would be quite natural to extend thi paper work to create a unifying approach between the two cae. Write the prior ditribution over the focal length by ρ(f). Similarly, we can write the prior ditribution of the principal point by ρ(p x, p y). Now, we can make the Bayeian nature of thi approach more explicit by writing 14 a "Cloud" model, t = [ 1] T, ω [.35 ] T ρ(e f, p x, p y) max ρ (q) (14) q:q T K T EK 1 = Now, in the optimization tep, intead of eeking arg max ρ(f), (15) F the optimization would be over arg max E,f,px,p y ρ(e f, p x, p y)ρ(f)ρ(p x, p y). (16) In thi way, in one tep, the mot likely calibration parameter would be found a well a the mot likely motion. Thi could be particularly ueful in the common cae that the camera calibration i approximately known, but the focal length change, perhap due to change of focu. Figure 2: Cloud model, and mean for two different motion.

5 "Aby" model, t = [1 ] T, ω [.175] T "Bicuit" model, t = [ 1 ] T, ω [.47.17] T "Aby" model, t = [ 1] T, ω [.175] T "Bicuit" model, t = [ 1] T, ω [.17.47] T t t t t t t t t Figure 3: Aby model, and mean for two different motion. Figure 4: Bicuit model, and mean for two different motion.

Figure 5: Two frame from the Catle equence, with epipolar line overlaid ACKNOWLEDGEMENTS The author would like to thank Gille Tran for providing the 3- D model, The Oxford Viual Geometry Group for

and Sturm, P., 24. Non-linear etimation of the fundamental matrix with minimal parameter. IEEE Tranaction on Pattern Analyi and Machine Intelligence 26(4), pp. 426 432. Brodky, T., Fermuller, C.

Random ample conenu: a paradigm for model fitting with application to image analyi and automated cartography. Commun. ACM 24(6), pp. 381 395. Harri, C. G. and Stephen, M., 1988.

G., 24. Ditinctive image feature from cale-invariant keypoint. Int. J. Comput. Viion 6(2), pp. 91 11. Makadia, A., Geyer, C. and Daniilidi, K., 25.

6 Figure 5: Two frame from the Catle equence, with epipolar line overlaid ACKNOWLEDGEMENTS The author would like to thank Gille Tran for providing the 3- D model, The Oxford Viual Geometry Group for providing the Oxford Corridor equence, and Marc Pollefey for providing the Catle equence. Figure 6: Two frame from the Oxford Corridor equence, with epipolar line overlaid REFERENCES Bartoli, A. and Sturm, P., 24. Non-linear etimation of the fundamental matrix with minimal parameter. IEEE Tranaction on Pattern Analyi and Machine Intelligence 26(4), pp Brodky, T., Fermuller, C. and Aloimono, Y., 2. Structure from motion: Beyond the epipolar contraint. International Journal of Computer Viion 37(3), pp Fichler, M. A. and Bolle, R. C., Random ample conenu: a paradigm for model fitting with application to image analyi and automated cartography. Commun. ACM 24(6), pp Harri, C. G. and Stephen, M., A combined corner and edge detector. In: AVC88, pp Hartley, R. I. and Zierman, A., 24. Multiple View Geometry in Computer Viion. Second edn, Cambridge Univerity Pre, ISBN: Lowe, D. G., 24. Ditinctive image feature from cale-invariant keypoint. Int. J. Comput. Viion 6(2), pp Makadia, A., Geyer, C. and Daniilidi, K., 25. Radon-baed tructure from motion without correpondence. In: CVPR. Nitér, D., 24. An efficient olution to the five-point relative poe problem. IEEE Tran. Pattern Anal. Mach. Intell. 26(6), pp Olieni, J., 25. The leat-quare for tructure from infiniteimal motion. Int. J. Comput. Viion 61(3), pp Pollefey, M., Gool, L. V., Vergauwen, M., Verbiet, F., Corneli, K., Top, J. and Koch, R., 24. Viual modeling with a hand-held camera. Int. J. Comput. Viion 59(3), pp Schmid, C., Mohr, R. and Bauckhage, C., 2. Evaluation of interet point detector. International Journal of Computer Viion 37(2), pp Tian, T., Tomai, C. and Heeger, D., Comparion of approache to egomotion computation. Trigg, B., McLauchlan, P. F., Hartley, R. I. and Fitzgibbon, A. W., Bundle adjutment - a modern ynthei. In: Workhop on Viion Algorithm, pp

3D MODELLING WITH LINEAR APPROACHES USING GEOMETRIC PRIMITIVES

MAKARA, TEKNOLOGI, VOL. 9, NO., APRIL 5: 3-35 3D MODELLING WITH LINEAR APPROACHES USING GEOMETRIC PRIMITIVES Mochammad Zulianyah Informatic Engineering, Faculty of Engineering, ARS International Univerity,