Unknown Radial Distortion Centers in Multiple View Geometry Problems

Transcription

1 Unknown Raial Distortion Centers in Multiple View Geometry Problems José Henrique Brito 1,2, Rolan Angst 3, Kevin Köser 3, Christopher Zach 4, Pero Branco 2, Manuel João Ferreira 2, Marc Pollefeys 3 1 Intituto Politécnico o Cávao e o Ave, Barcelos, Portugal 2 Universiae o Minho, Guimarães, Portugal 3 Computer Vision an Geometry Group, ETH Zürich, Switzerlan 4 Microsoft Research, Cambrige, UK Abstract. The raial unistortion moel propose by Fitzgibbon an the raial funamental matrix were early steps to exten classical epipolar geometry to istorte cameras. Later minimal solvers have been propose to fin relative pose an raial istortion, given point corresponences between images. However, a big rawback of all these approaches is that they require the istortion center to be exactly known. In this paper we show how the istortion center can be absorbe into a new raial funamental matrix. This new formulation is much more practical in reality as it allows also igital zoom, croppe images an camera-lens systems where the istortion center oes not exactly coincie with the image center. In particular we start from the setting where only one of the two images contains raial istortion, analyze the structure of the particular raial funamental matrix an show that the technique also generalizes to other linear multi-view relationships like trifocal tensor an homography. For the new raial funamental matrix we propose ifferent estimation algorithms from 9,10 an 11 points. We show how to extract the epipoles an prove the practical applicability on several epipolar geometry image pairs with strong istortion that - to the best of our knowlege - no other existing algorithm can hanle properly. 1 Introuction When trying to relate images, the robust estimation of the funamental matrix base on local feature corresponences is a very powerful approach. Stochastic estimation algorithms such as RANSAC can fin the correct two-view relation with high probability an at the same time istinguish inliers an outliers to the moel i.e. mismatches). However, this approach relies on the appropriateness of the moel, i.e. it assumes that the images strictly obey to the pinhole camera moel. In practice however, images can contain significant istortion inuce by the lens system) of a real camera. Consequently, in the literature several camera moels an techniques have been propose to moel such istortion [3, 6, 16, 1, 17, 4, 9]. However, for automatic registration of images obtaine from internet sources or archives, an offline camera calibration phase is not feasible.

2 2 Brito, Angst, Köser, Zach, Branco, Ferreira, Pollefeys In such cases lens istortion has to be consiere irectly in the multi-view geometry estimation stage. This was the iea of the unistortion moel propose by Fitzgibbon [6] that has been extene to the raial funamental matrix by Barreto an Daniiliis [1]. The assumption is that unistortion can be moele in a raial fashion with respect to a istortion center. The main rawback in both formulations is that the istortion center must be known in avance, which we argue is not practical when images stem from sources like archives or internet photo collections. Using a wrong istortion center reners the whole concept of raial istortion meaningless, although assuming the istortion center to be at the center of the image can sometimes still be a vali approximation. However, in the case of croppe images or images taken with igital zoom no heuristics exist where to place the istortion center. Consequently, in this contribution we generalize the raial funamental matrix an all other multilinear multiple view relations) to unknown istortion centers. This is very analogue to the ieal pinhole case where the essential matrix was generalize to the funamental matrix [5] that coul then account for any principal point. Also in the case of the raial funamental the imensions of the matrix o not change once the istortion center is consiere an linear algorithms require the same number of points for estimating it. For clarity of presentation we start from the setting where only one of the two images contains raial istortion an analyze the structure of the particular raial funamental matrix. It will turn out that a change of istortion center acts linearly on the lifte point representation, allowing to o the same generalization for other multi-view geometry relations like homograpy, trifocal tensor an so forth. We then continue to erive ifferent estimation algorithms for our raial funamental matrix from 9,10 an 11 points that exploit the specific algebraic structure an show how to extract the epipoles. Finally, we prove the practical applicability of the new theory on several epipolar geometry image pairs with strong istortion that - to the best of our knowlege - no other existing algorithm can hanle properly. 2 Previous Work For ieal pinhole cameras the essential matrix has been introuce by Longuet- Higgins [14] an it allowe efficient computation of the relative pose between two views. However, pre-calibration of these views was manatory an prevente using this technique for images with unknown calibration parameters since one ha to know e.g. focal length an principal point of both cameras. Much later, the introuction of the funamental matrix [5, 15] remove this restriction an allowe to work with unknown images, zoom cameras an le to a whole theory of auto-calibration from images an projective reconstruction cf. to [10]). Practically, alreay the original 8-point algorithm from [14] coul have been applie to the uncalibrate setting, but ue to notation an for historic reasons this was not clear before the proposal of the funamental matrix. Nowaays, the core of

3 Unknown Raial Distortion Centers in Multiple View Geometry Problems 3 Longuet-Higgins algorithm is known as the 8- point algorithm for funamental matrix estimation [10]. The funamental matrix applies to ieal pinhole cameras, but real cameras have lenses that sometimes result in istortion of the image an, ue to the shape of the lens, this istortion is typically raially-symmetric with respect to a istortion center. Many formulations exist to cope with this problem e.g. [6, 16, 1, 17, 4, 9]. Accoring to one of the classical istortion moels [3] the eformation of an unistorte point into a istorte point as cause by the lens) is represente by a polynomial equation, but ue to the nature of the istortion function it was not easily possible to estimate the inverse of the istortion irectly from point corresponences. Fitzgibbon[6] has suggeste to irectly moel the unistortion of a point rather than the istortion an argue that earlier istortion moels were as goo or ba empiric approximations to the true lens behaviour as an unistortion moel might be. Having an unistortion moel has the avantage that one can irectly work with istorte coorinates, which is what is measure in an image. However, similarly as in the erivation of the essential matrix of Longuet- Higgins, now Fitzgibbon assume that the istortion center is known beforehan. Later, his moel was reformulate into the raial funamental matrix by Barreto an Daniiliis[1]. They propose a linear 15 point metho to estimate the matrix an recently it has been shown by Kukelova et al. [12] that this view relation can be estimate actually from only 9 corresponences in a minimal solver. All of the above mentione papers kept the strong requirement that the istortion center nees to be known in avance, which practically prevente the use of these techniques for unknown, croppe images or in the case of igital) zoom. Li et al. [13] aresse the unknown istortion center problem, but they nee a calibration gri or a very high number of noise-free point corresponences, among other restrictions. In this paper we will show that the position of the istortion center can be absorbe into the raial funamental matrix in very much the same way as the principal point is absorbe into the funamental matrix. 3 The Lifting-Trick for Raial Distortion 3.1 Secon-Orer Raial Distortion Moels The traitionally use secon-orer istortion moel in computer vision with unknown center of istortion x, y ) T R 2 escribes the raial istortion as x y ) = xu y u ) ) + λ r 2 xu y u x y )), 1) where x, y ) T R 2 an x u, y u ) T R 2 are the istorte an the unistorte point, respectively, whereas λ R is the istortion coefficient an r 2 = x u, y u ) T x, y ) T 2 is the square Eucliean istance between the center

4 4 Brito, Angst, Köser, Zach, Branco, Ferreira, Pollefeys of istortion an the unistorte point. Eq. 1 is a istortion moel since it actually escribes the istorte point in explicit form: given the unistorte point x u, y u ) T an the istortion parameters λ an x, y ) T, the istorte point can be compute easily by evaluating the right-han sie of Eq. 1. Fitzgibbon [6] has propose a slightly ifferent moel, which he showe to be equivalently powerful as the moel above, i.e. it provies the same approximation accuracy to the unerlying true istortion. However, his moel enjoys an interesting property. Specifically, this raial istortion moel can conveniently be expresse with homogeneous coorinates p u = x u y u 1 = x y 1 + λr 2, 2) with r 2 = x 2 + y2 an where = enotes equality up to a scalar multiple. In this paper, we exten his formulation to the case where not only the istortion coefficient λ is unknown, but the center of raial istortion x, y ) T as well. In this case, his moel can be extene by starting with x ) = y xu ) ) + λr 2 xu y u y u x y )), 3) where r 2 = x, y ) T x, y ) T 2. The only istinction to the moel in Eq. 1 is that the istance is now measure between the istorte point an the center of raial istortion. In contrast to the istortion moel in Eq. 1 however, Fitzgibbon s moel actually is an unistortion moel: the right-han sie of Eq. 3 is linear in the unistorte point x u, y u ) T an hence one can compute an explicit form for this unistorte point given the istorte point x, y ) an the istortion parameters λ an x, y ) T. In the following section, we are going to show how this more complex formulation can be conveniently hanle with a lifting trick. 3.2 Lifting to 4D Space Lifting is a process in polynomial algebra which embes a problem with nonlinear polynomial terms in a higher imensional linear space. In our case, raially istorte points in the projective 2-plane P 2 will be mappe to points in projective 3-space P 3. A istorte point with homogeneous coorinates p = x, y, z ) T P 2 will be mappe to the point x z, y z, z 2, x2 + y2 )T P 3. Hence, the projective 2-plane P 2 is mappe to a quaric surface in P 3 efine through { x, y, z, w) P 3 zw x 2 y 2 = 0 }1. Interestingly an most importantly, the lifte istorte points can be mappe to the unistorte points by a fixe linear transformation, as we will erive shortly. Note that the same lifting scheme has been propose by Barreto an Daniiliis [1] see Eq. 7 in their 1 Points of the form x z, y z, z, x 2 + y 2 ) T fulfill this equation zw x 2 y 2 = 0 as can easily be verifie by setting x = x z, y = y z, z = z 2, w = x 2 + y 2.

5 Unknown Raial Distortion Centers in Multiple View Geometry Problems 5 paper). Their erivation is however closely linke to the funamental matrix, but we woul like to highlight that this lifting trick can be applie inepenently of the type of multiple view constraint, i.e. it applies to homographies, trifocal tensors, etc as well. Furthermore, Barreto an Daniiliis assume a known center of raial istortion. In the following, we show in etail how the same lifting scheme can be generalize to the case of unknown istortion center, resulting in a ifferent linear transformation matrix than the one erive in [1], though. Let us now present this lifting trick in etail, starting from the istortion moel in Eq. 3. Simple algebraic manipulation of Eq. 3 leas to ) ) x + λr 2 x = 1 + λr 2) ) x u, 4) y y y u which shows that the unistorte point x u, y u ) T is a scalar multiple of x, y ) T + λr 2 x, y ) T. The scalar factor can be absorbe with a homogeneous representation x u x + λr 2 x 1 x x x p u = y u = y + λr 2 y = 1 y y y. 5) λr λr 2 For aitional generality an in orer to stay closer to [1], let us represent the istorte point p = x, y, z ) P 2 as an element of projective 2-space. The previous equation Eq. 5 then becomes 1 x x z 1 x p u = 1 y y z 1 y, 6) λr 2 with r 2 = x z 1 ) 2 x + y z 1 ) 2. y Some further algebraic manipulations allow us to expose all the components ue to the istorte point on the right han sie 1 x x z 1 x p u = 1 y y z 1 y x λ z 1 ) 2 x + y z 1 ) 2 ) 7) y = 1 x z 1 x x λ x 1 y λ y y z 1 y 1 8) 1 λ x z 1 ) 2 x + y z 1 ) 2 y x x x z x = 1 y y 1 1 y y z 1 1 z 1 1 2, 9) λ 2 x 2 y 2 x + 2 y 1 x 2 }{{} + y2 =L R 3 4

6 6 Brito, Angst, Köser, Zach, Branco, Ferreira, Pollefeys where in Eq. 8 the lifting trick has been use an Eq. 9 is equal to Eq. 8 up to a scale factor of z 2 which oes not matter since p u P 2 is an element of projective 2-space 2. This erivation provies an important insight an leas to one of the main contributions of this paper. Eq. 9 shows that the unistorte homogeneous coorinates p u can be expresse by a 3 4 linear transformation L applie from the left to the lifte ata vector x z, y z, z 2, x2 + y2 ) T P 4. This linear algebraic representation has far reaching consequences. All multiple-view geometry entities, such as homographies or funamental matrices, act on homogeneous coorinates of unistorte points. Unfortunately, if the input images are raially istorte, these entities are no longer applicable. However, these entities can be lifte to a higher imensional space by multiplying them either from the left an/or the right) with the 3-by-4 matrix L thereby acting on raially istorte lifte coorinates. The matrix L is a function of the raial istortion parameters an therefore also unknown. However, given sufficiently many istorte image observations, the lifte multiple view entities can be estimate nonetheless. This will be emonstrate in the following sections with the funamental matrix. 4 Single-Sie Raial Funamental Matrix The funamental matrix captures the projective relation between two camera views [10]. Given a homogeneous point corresponence p u an q u between two images of the same 3D point, the funamental matrix relates these points by the constraint q T u F p u = 0. The funamental matrix actually maps a point in one image to an epipolar line in the other image. Since neither q u nor p u can be the zero vector, F has a non-trivial left an right nullspace. These nullspaces actually correspon to the two epipoles. The next section shows how the funamental matrix can be extene to hanle a raially istorte point measurement p instea of an unistorte measurement p u. 4.1 Derivation of the Single-Sie Raial Funamental Matrix Let us now assume that one of the two images is raially istorte, say the one where feature point p u has been observe. This means that only the raially istorte point x, y ) T is known. Thanks to the erivation in Sec. 3.2, we know how to hanle this situation. A simple right-multiplication by L lifts the funamental matrix on one sie) to a 4D projective space which allows to use the raially istorte measurements x z x z 0 = qu T F p u = qu T F L y z z 2 = qt ˆF y z u z 2, 10) x 2 + y2 x 2 + y2 2 Of course in practice, the measurement will be normalize such that z = 1 an the formulas simplify slightly. Nevertheless, the more general representation is easier to interpret in terms of mappings between projective spaces.

7 Unknown Raial Distortion Centers in Multiple View Geometry Problems 7 where the single-sie raial funamental matrix ˆF = F L R 3 4 has been introuce. The ecomposition L = [I 0]+λ x, y, 1) T 2 x, 2 y, 2 x + 2 y, 1 ) leas to another interesting representation of the single-sie raial funamental matrix ˆF = F L = [F 0] + F λ x y 2 x 2 y 2 x + 2 y 1 ). 11) Properties of the Single-Sie Raial Funamental Matrix Since the single-sie raial funamental matrix ˆF = F L is given as the prouct between the orinary rank-2 funamental matrix F an the matrix L, its rank equals 2. As a 3 4 matrix of rank 2, the single-sie raial funamental matrix has = 9 egrees of freeom minus one ue to the scale ambiguity) 3. Unfortunately, the number of parameters we are looking for equals 7 for the stanar funamental matrix plus 3 for the raial istortion parameters. Hence, there are 10 parameters but only 9 egrees of freeom in the single-sie raial funamental matrix. This implies that there is a one-parametric family of perfectly vali solutions. Hence, given a single-sie raial funamental matrix, it is not possible to uniquely extract the unerlying funamental matrix an the 3 raial istortion parameters. This is in contrast to previous work [1, 12] which assume a known raial istortion center which ecrease the number of parameters by 2. This allowe the unique extraction of all the 8 parameters. Nevertheless, in the remainer of this section, we will show that the epipoles are unique an how they can be extracte from the single-sie raial funamental matrix even if the raial istortion center is unknown. The extraction of the left epipole e from the rank-2 matrix ˆF is easy: Since ˆF = F L, both ˆF an F share the same left-nullspace an hence e equals the left nullspace of ˆF. This nullspace can be easily compute e.g. with the singular-value ecomposition of ˆF. The right epipole is more tricky since there is a two-imensional right nullspace N = [n 1, n 2 ] R 4 2 of ˆF R 3 4, i.e. ˆF N = 0 R 3 2. This nullspace can again be compute with the singularvalue ecomposition of ˆF. The lifte coorinates of the istorte right epipole e P 3 must lie in this nullspace since the unistorte epipole lies in the right nullspace of the stanar funamental matrix F which is a factor of ˆF = F L. Hence, ue to this fact an since the istorte coorinates are only efine up to scale, the lifte coorinates of the istorte epipole eα) = αn α)n 2 can be parametrize with one parameter α R. As escribe at the beginning of Sec. 3.2, vali points x, y, z, w) P 3 in the lifte space are restricte to a quaric surface efine through the equation zw x 2 y 2 = 0. Plugging the 3 A matrix A R m n of rank r can be factorize A = BC with B R m r an C R r n. The matrix factors are unique up to a multiplication with a regular matrix Q R r r, i.e. A = BQQ 1 C an as such A has mr + rn r 2 egrees of freeom.

8 8 Brito, Angst, Köser, Zach, Branco, Ferreira, Pollefeys one-parametric representation eα) into this quaric equation yiels a quaratic equation in α which can be solve easily in close form. This results in two equally vali solutions for the istorte coorinates of the right epipole. Note that this is an inherent characteristics of the Fitzgibbon istortion moel which always provies two possible istorte points, given the unistorte point an the raial istortion parameters Further Examples - Two-Sie Raial Funamental an Homographies As alreay previously mentione, the same lifting trick can be applie to other entities in multiple view geometry in the presence of raial istortion with unknown center of raial istortion. For example, the two-sie raial funamental matrix where both images are raially istorte is given by left- an rightmultiplying the stanar funamental matrix with the transformations mapping lifte points to unistorte points, i.e. x, y, 1, x 2 + y 2 ) L T F L x, y, 1, x 2 + y 2 ) T = 0 12) If both images have the same raial istortion, then L = L. This results in a 4 4 two-sie raial funamental matrix which is again of rank 2 an has therefore = 11 egrees of freeom. There are = 13 parameters 7 ue to the stanar funamental matrix an twice times 3 parameters for the two istortion moels), an again, there is no unique solution for the parameters. However, the two epipoles can be extracte analogously to the single-sie raial funamental matrix. Another example is given by a one-sie raial homography. Again multiplying the lifte coorinates x of the istorte image from the left by L yiels x = HL x an hence the one-sie raial homography HL is a full rank 3 4 matrix. Both the two-sie raial funamental matrix an the one-sie raial homography can be estimate with linear methos analogously to the algorithms presente next for the single-sie raial funamental matrix. 5 Single-Sie Raial Funamental Matrix Estimation The algebraic epipolar constraint q T ˆF x y 1 x 2 + y 2 ) T = 0 13) can be rewritten using kronecker proucts [7] as x y 1 x 2 + y 2 ) q T vec ˆF ) = 0 14) }{{}}{{} A f 4 Solving Eq. 3 for the istorte coorinates x, y ) T given x u, y u) T, λ, an x, y) T asks for intersecting two conics which in this specific instance can have up to two solutions.

9 Unknown Raial Distortion Centers in Multiple View Geometry Problems 9 From each corresponence, we obtain a ifferent row vector A i. Stacking 11 of these equations on top of each other we obtain an matrix an f must lie in the null space of that matrix, like in the 8-point algorithm for estimating the funamental matrix. Similarly, rank two of the resulting matrix can be enforce via a singular value ecomposition afterwars. The ten point algorithm In an analogous way to the 7-point-algorithm for classical funamental matrix estimation, we use one corresponence less than is require for the linear solution above an obtain a two-imensional null-space spanne by f 1 an f 2. The true f must thus be a linear combination of both, where we can fix one of the coefficients, since f is only efine up to scale. f = αf 1 + f 2 15) We now perform the inverse operation to vectorization an reassemble the matrix ˆF from the vector f, an for convenience of notation, explicitely write own the columns: ) ˆF = ˆF1 ˆF2 ˆF3 ˆF4 16) We now choose alpha such that ) et ˆF1 ˆF2 ˆF3 = 0 17) which is the same step as in the stanar seven-point algorithm. Thus, from ten corresponences an one cubic eterminant constraint we estimate the matrix ˆF. However, in the presence of noise, it is however not guarantee that ˆF will have rank two, since the last column of ˆF can vary freely. Again, rank two of the resulting matrix can be enforce via SVD afterwars. The nine point algorithm As mentione above, in the ten-point-algorithm only the first three columns of F are force to be in a 2D subspace, however, the last column coul still vary freely in the presence of noise. Consequently, we might enforce also the last three columns of F to be linearly epenent. To start, we can use only nine corresponences an obtain a 3D nullspace f = αf 1 + βf 2 + f 3 18) We now choose α an β such that ) ) et ˆF1 ˆF2 ˆF3 = 0 et ˆF2 ˆF3 ˆF4 = 0 19) These are two cubic equations in α an β an accoring to Bezout s theorem there cannot be more than nine iscrete solutions. The erivation of the exact solution is out of the scope of this paper, however the intereste reaer is refere to Groebner basis methos [11]. As argue before, there are nine egrees of freeom in ˆF an so there can be no solution base on less than nine points.

10 10 Brito, Angst, Köser, Zach, Branco, Ferreira, Pollefeys 6 Experiments In this section we emonstrate the usefulness of the presente formulation an prove empirically that the new moel can cope with arbitrary istortion centers while earlier methos cannot. We first analyse this using synthetic ata an then with real images. In the experiments, we use image pairs in which one image has known instrinsics an the other image has unknown focal length, raial istortion center an raial istortion coefficients. In the experiments with synthetic ata we use ranom camera configurations, ifferent istortion centers an ifferent istortion parameters. Due to the lack of earlier methos for our setting, the results are compare to those obtaine with the state of the art raial istortion solver from Kukelova et al.[12], although this latter metho assumes the istortion center to be at the center of the image an also estimates istortion for both cameras. In contrast, in our setting, one of the images in each image pair has known intrinsics an the istortion center is not at the center of the image. We then test the algorithms with real worl images which were taken with cameras that exhibit a significant level of istortion. We generate croppe versions of these images, so that the center of istortion woul not lie at the center of the image. 6.1 Evaluation with synthetic ata The first set of tests for the semicalibrate case was performe using synthetic ata. All tests with synthetic ata were performe with a set of 100 ranom 3D points an 1000 generate ranom camera poses. The first camera was place at the origin, with fixe parameters, pointe towars the set of 3D points. The 1000 ranom poses were generate for the secon camera, by generating ranom translations, ranom rotations an ranom focal lengths, varying between 1/2 an 2x the focal length of the first camera. For each camera pose we projecte the 3D points on both cameras, istorte the points on the image of the secon camera accoring to the istortion moel in Eq. 3, setting the isplacement of the istortion center to vary between 0 an the with/height of the image an using ifferent values for the istortion coefficient. For each setting we compute the number of inliers with each algorithm. Results are presente in Fig. 1. We can see that, as the center of istortion is place further away from the image center, the number of ientifie corresponences is constant for both implementations of our metho, whereas metho [12] increasingly fails to correctly ientify the corresponences, as it oes not correctly moel the position of the istortion center. 6.2 Test with real images To test the theory on real images we first matche a set of uncalibrate, istorte, croppe images to an image with known calibration parameters using ifferent atasets. The unistorte images were croppe in such a way that the center of istortion woul be locate away from the center of the resulting image.

11 Unknown Raial Distortion Centers in Multiple View Geometry Problems 11 Semicalibrate, varying istortion center lamba=0.01) 100 Semicalibrate, varying istortion center lamba=0.1) 100 Number of inliers New 11 point New 10 point Kukelova et al Distortion center isplacement/image size a) Distortion parameter of λ = 0.1. Number of inliers New 11 point New 10 point Kukelova et al Distortion center isplacement/image size b) Distortion parameter of λ = Fig. 1: Boxplots of the number of inliers for 1000 ranomly generate camera poses with varying istortion center. Note that our 11- an 10-point algorithms nearly always fin all the 100 inliers. To extract features in the images we use SURF[2], an then we compute a number of putative matches in each image pair by stanar feature space matching. This prouce a number of matches for each image pair, not all of which were correct corresponences. We then ran both the 11 point an 10 point implementations of our algorithm an [12], in a RANSAC framework with same parameters an constructing hypotheses on the same sample sets. In the en we compute the number of inliers with a threshol of 3 pixels. To obtain the epipolar error use for classifying outliers) we compute the istance in pixels between a point an the epipolar line in the unistorte image. Before applying the point corresponences to the ifferent algorithms, we normalise the image measurements similarly to the 8-point algorithm [8]. For the calibrate image we use the inverse of the camera intrinsics for the normalization, an for the uncalibrate/istorte/croppe one we use an initial estimate of the focal length, W/2 f guess = tanfov, where W is the image with an fov guess/2) guess = 50 is an a- priori estimate of the fiel of view. Furthermore, the image points are normalise with respect to the center of the image. As the normalisation is only performe to enhance the conitioning of the system, any similarity transformation is a vali normalisation in our formulation. Results for one of the teste atasets are shown in Fig. 2 where we can see the inliers ientifie by metho of [12] an our new metho in an image pair where the image on the left has been previously unistorte with an offline camera calibration phase an the image on the right has unknown istortion parameters an was also croppe so that the istortion center is now in the upper part of the image. One can visualy see that our metho is able to ientify a higher number of inliers, especially in areas away from the istortion center. Fig. 3a shows a irect comparison of which inliers are ientifie by our new metho an [12]. Again we

12 12 Brito, Angst, Köser, Zach, Branco, Ferreira, Pollefeys a) 146 inliers ientifie by the metho from Kukelova et al. [12] b) 294 inliers ientifie by our new metho. Fig. 2: Results for ataset Shopping. can see that both methos ientify inliers close to the center of istortion but our metho ientifies extra inliers away from the istortion center. Similar results can be obtaine for ifferent atasets in Fig. 3b an Fig. 3c. For the image pair in Fig. 3b the istorte image was croppe so that the istortion center was place in the bottom right region of the image. For the image pair in Fig. 3c the istorte image was croppe so that the istortion center was place in the top left region of the image. Also for these image pairs, the metho from [12] foun only a spatially confine set of corresponences near the center of istortion, whereas our metho woul be able to use more corresponences also far away from the center of istortion, where raial istortion is more severe. The inliers foun by the metho of [12] must be explaine as an algebraic fit to the ata, because the algorithm was not geometrically esigne to cope with an unknown istortion center. To the best of our knowlege, the approach presente in this paper is the only one esigne to hanle epipolar geometry problems with fully unknown raial istortion. 7 Conclusion We have shown that the lifting of image points into 4-space can consier the istortion center in a linear way. This allows for instance to generalize the raial funamental matrix to the case of unknown istortion centers, facilitating now

13 Unknown Raial Distortion Centers in Multiple View Geometry Problems 13 a) Dataset Shopping b) ataset Church c) ataset Corner Fig. 3: Comparison between our metho an [12]: re are the inliers foun by both methos; green are the extra inliers foun by our metho; blue are inliers foun by the metho from Kukelova et al. not foun by our metho. practical use of the raial funamental matrix even with croppe or zoome images or more generally with images where the center of istortion is unknown. We have proven this by evising ifferent algorithms to estimate the matrix from point corresponences an have shown results on real images that we believe cannot be obtaine with any other existing framework. Furthermore, since a change of istortion center can be expresse linearly in 4-space, now the raial istortion moel with unknown center can be applie to all multilinear multiple view relations, such as the trifocal tensor an homographies. Besies this, the insight about the istortion center might pave the way for a series of new minimal solvers with unknown istortion center. On top of this we believe that the new raial funamental matrix can open the oor to a theory of raial istortion self calibration, i.e. on top of focal length an principal point one coul now look for the istortion coefficient an the istortion center when given multiple image pairs or image sequences, enforce some constraints e.g. constant istortion center throughout a sequence) an so forth.

14 14 Brito, Angst, Köser, Zach, Branco, Ferreira, Pollefeys References 1. Barreto, João P. an Kostas Daniiliis: Funamental Matrix for Cameras with Raial Distortion International Conference on Computer Vision ICCV) 2005) Bay, Herbert an Ess, Anreas an Tuytelaars, Tinne an Van Gool, Luc: SURF: Speee Up Robust Features Computer Vision an Image Unerstaning CVIU) 3 110) 2008) Brown, D. C.: Close-range camera calibration Photogrammetric Engineering 8 37) 1971) Claus, D. an Fitzgibbon, A. W.: A Rational Function Lens Distortion Moel for General Cameras Computer Vision an Pattern Recognition CVPR) 2005) Faugeras, O. an Luong, Q. an Maybank, S.: Camera self-calibration: Theory an experiments European Conference on Computer Vision ECCV) Fitzgibbon, A.W.: Simultaneous linear estimation of multiple view geometry an lens istortion Computer Vision an Pattern Recognition CVPR) ) Anrea Fusiello: A matter of notation: Several uses of the Kronecker prouct in 3D computer vision Pattern Recognition Letters 15 28) 2007) Hartley, R.: In efense of the eight-point algorithm Pattern Analysis an Machine Intelligence PAMI) 6 19) 1997) Hartley, R. an Kang, Sing Bing: Parameter-Free Raial Distortion Correction with Center of Distortion Estimation Pattern Analysis an Machine Intelligence PAMI) 8 29) 2007) Hartley, R. an Zisserman, A.: Multiple View Geometry in Computer Vision Cambrige University Press 2004) 11. Kukelova, Zuzana an Bujnak, Martin an Pajla, Tomas: Automatic Generator of Minimal Problem Solvers European Conference on Computer Vision ECCV) 2008) Kukelova, Zuzana an Byrö, Martin an Josephson, Klas an Pajla, Tomas an Aström, Kalle: Fast an robust numerical solutions to minimal problems for cameras with raial istortion Computer Vision an Image Unerstaning CVIU) 2 114) 2010) Hongong Li an Richar Hartley: A Non-iterative Metho for Correcting Lens Distortion from Nine-Point Corresponences In Proc. OmniVision05, ICCVworkshop 2005) 14. Longuet: A computer algorithm for reconstructing a scene from two projections Nature ) Quan-Tuan Luong an Olivier D. Faugeras: The funamental matrix: Theory, algorithms, an stability analysis International Journal of Computer Vision IJCV) 1 17) 1996) Micusik, B. an Pajla, T.: Estimation of omniirectional camera moel from epipolar geometry Computer Vision an Pattern Recognition CVPR) ) Thirthala, S. an Pollefeys, M.: Multi-view geometry of 1D raial cameras an its application to omniirectional camera calibration International Conference on Computer Vision ICCV) )