3D MODELLING WITH LINEAR APPROACHES USING GEOMETRIC PRIMITIVES

Transcription

1 MAKARA, TEKNOLOGI, VOL. 9, NO., APRIL 5: D MODELLING WITH LINEAR APPROACHES USING GEOMETRIC PRIMITIVES Mochammad Zulianyah Informatic Engineering, Faculty of Engineering, ARS International Univerity, Bandung 48, Indoneia Abtract In thi paper, we tudy linear approache for 3D model acquiition from non-calibrated image. Firt, the intrinic and extrinic camera calibration i taken into conideration. In particular, we tudy the ue of a pecific calibration primitive: the parallelepiped. Parallelepiped are frequently preent in man-made environment and naturally encode the affine tructure of the cene. Any information about their euclidean tructure (angle or ratio of edge length), poibly combined with information about camera parameter i ueful to obtain the euclidean recontruction. We propoe an elegant formalim to incorporate uch information, in which camera parameter are dual to parallelepiped parameter, i.e. any knowledge about one entity provide contraint on the parameter of the other. Conequently, an image a parallelepiped with known Euclidean tructure allow to compute the intrinic camera parameter, and reciprocally, a calibrated image of a parallelepiped allow to recover it euclidean hape (up to ize). On the conceptual level, thi duality can be een a an alternative way to undertand camera calibration: uually, calibration i conidered to be equivalent to localizing the abolute conic or quadric in an image, wherea here we how that other primitive, uch a canonic parallelepiped, can be ued a well. While the main contribution of thi work concern the etimation of camera and parallelepiped parameter. The complete ytem allow both calibration and 3D model acquiition from a mall number of arbitrary image with a reaonable amount of uer interaction. Keyword: parallelepiped, the affine tructure, euclidean tructure, 3D model. Introduction The camera and parallelepiped parameter are recovered in two tep. Firt, their intrinic and orientation parameter are computed. The original approach for thi tep wa introduced in Wilczkowiak et al. [] and conit on parameterization of the intrinic matrice of all the object (camera and parallelepiped) in term of the intrinic matrix of a reference object. The algorithm propoed later Wilczkowiak et al. [] exploit the fact that the parallelepiped projection matrice can be factorized into two part, repreenting repectively camera and parallelepiped parameter and allow to treat the available data imultaneouly without privileging any primitive. Our calibration approach i conceptually cloe to elfcalibration method, epecially thoe that upgrade affine to euclidean tructure [3,4] or thoe that conider pecial camera motion [5-7]. The way metric information on a parallelepiped i ued i alo imilar to vanihing point baed method [8-]. Some propertie of our algorithm are alo common with plane-baed approache [-7]. While more flexible than tandard calibration technique, plane-baed approache till require either euclidean information or, for elf-calibration, many image in general poition [8]. In thi ene, our approach i a generalization of plane-baed method with metric information to three-dimenional parallelepipedic pattern. Thi allow to handle miing data and unknown cale factor and implifie the formulation of calibration contraint. Finally, our approach can be compared to method uing complex primitive for the cene repreentation. However, unlike mot method of thi type, we ue the parallelepiped parameter directly to olve the calibration problem without requiring non-linear optimization method. We preent our parameterization of parallelepiped and tudy the propertie of their perpective image. We how that, in analogy to point, projection of a parallelepiped can be repreented by a projection matrix. A hown in Figure, it can be computed auming that the vertice of a parallelepiped projected in the image belong to a canonical cube, and thu will be called in the following the canonical parallelepiped projection matrix. 3

2 3 MAKARA, TEKNOLOGI, VOL. 9, NO., APRIL 5: 3-35 A parallelepiped encode naturally the affine propertie of the cene and facilitate modeling the remaining metric part. The repreentation of a parallelepiped given below i baed on the formalim propoed in Wilczkowiak et al. [,9]. A parallelepiped i defined by twelve parameter: ix extrinic parameter decribing it orientation and poition, and ix intrinic parameter decribing it metric hape: three dimenion parameter (edge length l; l and l3) and three angle between edge (µ; µ3; µ3). The parallelepiped may be repreented compactly in matrix form by a 4 x 4 matrix N: S v ~ N = O L T () where S i a rotation matrix and v a vector, repreenting the parallelepiped' poe (extrinic parameter). The 4 x 4 matrix ~L repreent the parallelepiped' hape: l ~ L = cij With: l c l l3 l3c3 c3 c c l c ( c c c ) 3 ] π [, l = ij ij ij ij i co θ, = inθ, θ > () The matrix ~L repreent the affine tranformation between a canonical cube and a parallelepiped with the given hape. Formally, a vertex (±; ±; ±; )T of the canonical cube i mapped, by ~L, to a vertex of our parallelepiped' intrinic hape. Then, the poe part of N map the vertice into the world coordinate ytem. Other parameterization for ~L may be choen, but the above one i attractive due to it upper triangular form. Thi underline the fact that ~L play the ame role for the parallelepiped a the calibration matrix K for a camera. The analogou entity to a camera' Image of the Abolute Conic(IAC) ω, i the matrix µ, defined by: l ll coθ ll3 coθ3 µ ~ L T L ~ ll coθ l ll3 coθ 3 ll3 coθ3 ll3 coθ 3 l3 where L i the upper left 3 x 3 matrix of ~L. (3) Hence, there i a eemingly perfect ymmetry between intrinic parameter of camera and parallelepiped. The only difference i that in ome cae, the ize of a parallelepiped matter, a will be explained below. A for camera, the fact that K33 = allow u to fix the cale factor in the relation ω K-TK-, and thu to extract K uniquely from the IAC ω, e.g. uing Choleky decompoition. A for parallelepiped, however, we have no uch contraint on it calibration matrix" L, o the relation µ LTL give u a parallelepiped' Euclidean hape, but not it (abolute) ize. Thi doe Figure. The projection of the canonic parallelepiped (cube) into the image. Matrice K, L correpond to intrinic parameter of camera and parallelepiped and (R; t),(s; v) correpond to extrinic parameter of camera and parallelepiped, repectively. not matter in general, ince we are uually only intereted in recontructing a cene up to ome cale. However, when recontructing everal parallelepiped, one need to recover at leat their relative ize. There are many poibilitie of defining the ize of parallelepiped. We chooe the following definition, due to it appropriatene in the equation underlying our calibration and recontruction algorithm below: the ize of a parallelepiped i defined a = (det L)/3 (4) Thi definition i actually directly linked to the parallelepiped' volume: 3 = det L = Vol=8 (the factor 8 arie ince our canonic cube ha an edge length of ).. Method Uing parallelepiped a natural calibration object offer everal advantage over tandard elf-calibration approache. Firtly, fewer correpondence are needed; five and a half point extracted per image are ufficient, and even fewer inter-image correpondence are needed. Secondly, the reduced canonical parallelepiped projection matrice encode the affine propertie of the cene. In conequence, the calibration problem i reduced to a elf-calibration problem where the plane at infinity i already localized [3,4] or where the camera are tationary [5-7]. The calibration algorithm decribed in thi ection i baed on the parameterization of the intrinic matrice of all the object in term of the intrinic matrix of a reference primitive. Thi i done uing parallelepiped projection matrice. Due to the neceity of choice of reference object the propertie of thi algorithm are le intereting than propertie of the factorization algorithm given in the following ection. In the current implementation, a cene i repreented by a bi-partitioned graph, whoe node are the camera a

3 MAKARA, TEKNOLOGI, VOL. 9, NO., APRIL 5: well a parallelepiped and the edge are the projection. We aume that the graph i connected and, conequently, for each object i the tranformation Gi uch that GiT µgi can be computed. When thi i not a cae, all the connected part of the graph have to be calibrated eparately. Our calibration approach conit of two tage. Firt, all the available linear equation are ued to determine µ (the ytem i olved uing SVD). If there i a unique olution, then we are done (from µ, all the camera and parallelepiped intrinic can be computed uing the Gik). The deciion if the ytem i under-contrained may be taken on the bai of a ingular value analyi. Thi alo give the degree of the ambiguity (dimenion of the olution pace). In practice, thi i uually two or lower. Hence, two quadratic equation are in general uffcient to obtain a finite number of olution. Once the matrice ωi and µk are etimated, the matrice Ki and Li can be computed via Choleky decompoition. Finally, we propoe the following algorithm:. Contruct the graph with camera and parallelepiped a node and projection a edge,. Etimate the canonical projection matrice ~ Xik, 3. Chooe a reference parallelepiped repreented by µ 4. Compute path (hortet for example) connecting all the camera i and parallelepiped k to µ and ue them to compute tranformation G i, G k, 5. Etablih linear equation ytem on µ baed on prior knowledge of intrinic parameter of camera and parallelepiped, 6. Solve the ytem to leat quare, 7. If neceary, ue the non-linear equation to reolve the remaining ambiguitie on µ, 8. Compute the matrice ω i, µ k uing µ and tranformation G i, G k 9. Extract the K i, L k from the ω i and the µ k uing e.g. QR-decompoition. Note that at thi tage the L k can only be recovered up to cale, i.e. the parallelepiped (relative) ize remain undetermined,. Compute rotation matrice by factorization of a meaurement matrix compoed of matrice X ik. The meaurement matrix X contain all information that can be recovered from the parallelepiped' image point alone. Since the meaurement matrix i the product of a motion matrix" ~M of 4 column, with a hape matrix" ~S of 4 row, it rank can be 4 at mot (in the abence of noie). 3. Reult and Dicuion The calibration approach preented before i well adapted to interactive 3D modeling from a few image. It ha a major advantage over other method: implicity. Indeed, only a mall amount of uer interaction i needed for both calibration and recontruction: a few point mut be picked in the image to define the primitive image poition. To recontruct cene element not belonging to the parallelepiped, contrained by bilinear relation uch a collinearity, coplanarity or parallelim, we have implemented a multi-linear recontruction method, introduced originally in Wilczkowiak et al. []. The recontruction tep i actually independent from the calibration method, although it ue the ame input in the firt tep. Interetingly, it allow 3D model to be computed from non-overlapping photograph. The global cheme of our ytem i preented in Figure. Figure. The calibration and recontruction algorithm. Finally, we propoe the following recontruction algorithm: : while!top_condition do : for object=point,line+plane: do n 3: N= i = nb _ of _ coordinate( objectr[ i]) 4: initialize an empty linear equation ytem A xnx nx=b x 5: compute the indexing function (bijection) F:idx (I,j);idx є [..N], where idx i the index in X Nx of the j-th coordinate of the i-th object. 6: for all contraint c[k]: do 7: compute k k ( Amk N, Bmk ): = equation ( c[ k]. type, c[ k]. object 8: add equation to the ytem: A k B A : = B : = k k A B 9: end for : olve AX = B : for idx=..n: do : if variable_computed(idx) then 3: et (i,j):=f(idx) 4: et object[i].coord[j]:=x(idx) 5: end if 6: end for 7: end for 8: end while

4 34 MAKARA, TEKNOLOGI, VOL. 9, NO., APRIL 5: 3-35 Let u tudy in more detail the reult of the intrinic and extrinic calibration. Let u firt conider the behavior of our calibration method in a proximity of ingular configuration (left and right extremitie of Figure 3-5). It can be een that at both initial and final poition P and Pn, calibration i very untable, a expected. However, when the minimal angle between the camera axe i larger than 5 the method can be conidered table. All calibration are ucceful (Figure 3a), the relative error on the obtained median value i not larger than 7% (Figure 4).A expected, calibration reult obtained with the reference primitive method are le table than thoe obtained with the factorization method. Indeed, a hown in Figure 4 the relative error on median value obtained with the reference primitive method i up to 9% (v 7% for the factorization method). Alo, reprojection error i more important (~,5 pixel for primitive and ~8 pixel for primitive baed calibration v ~,5 pixel and ~6 pixel for the factorization method). Naturally, calibration reult are more table uing two parallelepiped than with a ingle one. Firt, uing larger number of primitive decreae the poibility for ingulararity. Figure 5b how reult for the econd camera for one and two parallelepiped baed calibration uing information on right parallelepiped angle. While a relative poition between the firt parallelepiped and econd camera i ingular for the calibration, adding a econd parallelepiped tabilize the configuration. In non-ingular configuration the larger number of primitive and related increaed number of equation reult in more accurate calibration (Figure 3-5). However, introducing additional primitive increae the reprojection error (ee Figure 3b). Comparing Figure 4 and 5 it can be een that uing the information about the right parallelepiped angle decreae importantly the number of ingularitie. Figure 6 preent the reult for the etimation of the camera' rotation and tranlation. Extrinic calibration wa performed only when the previou internal Figure 4. Figure 5. Figure 6. Median value obtained for the focal length of (a) firt camera; (b) econd camera. Notation are imilar to Figure 3. Calibration reult uing the factorization method for the etimation of local length of (a) the t camera and (b) the econd camera, uing the known right parallelepiped angle. Extrinic calibration reult. (a) Median value of the rotation error; (b) Median value of the angle between the true and the etimated direction between the camera-parallelepiped relative tranlation vector. Notation are imilar to Figure 3. Figure 3. (a) Number of ucceful calibration; (b) Median value of reprojection error. Plot decribed by (F-*) and (P-*) correpond, repectively, to factorization and reference primitive method; plot decribed by (*-) and (*-) correpond to calibration baed on and parallelepiped. calibration wa ucceful. Diplayed error value on recovered rotation (Figure 6a) are the angle of the relative rotation between the true R and the etimated R (the preented value are computed a the mean rotation error for both camera). Tranlation error (Figure 6b) i repreented by the mean of the two angle between the true and etimated relative camera-parallelepiped poition vector. Thi allow to evaluate the geometry of the recontructed camera independently from error caued by global caling and error on the rotation. For a tandard deviation of δ = and a minimal angle of between the camera axe, the rotation error i never larger than, with a 5 median value, and the tranlation error i imilar. Thoe error quite ignificant, but note, that were obtained uing

5 MAKARA, TEKNOLOGI, VOL. 9, NO., APRIL 5: projection of at mot point per image (6 per parallelepiped projection). However, increaing the number of cene contraint (i.e. number of calibration primitive) increae alo the reprojection error. The increae of the reprojection error for the method baed on two calibration parallelepiped can be een in chart 3b. 4. Concluion An image of parallelepiped with known Euclidean tructure allow the intrinic camera parameter computing, and reciprocally, a calibrated image of a parallelepiped allow it euclidean hape (up to ize) recovering. On the conceptual level, thi duality can be een a an alternative way to undertand camera calibration which i uually conidered to be equivalent to localizing the abolute conic or quadric of an image, other primitive, uch a canonic parallelepiped, can be ued a well. The complete ytem allow both calibration and 3D model acquiition from a mall number of arbitrary image with a reaonable amount of uer interaction. Reference [] M. Wilczkowiak, E. Boyer, P. Sturm, In: A. Heyden, G. Sparr, M. Nielen, P. Johanen (Ed.), Proceeding of the 7th European Conference on Computer Viion, Copenhagen, Denmark,, p.. [] M. Wilczkowiak, P. Sturm, E. Boyer. Uing Geometric Contraint Through Parallelepiped for Calibration and 3D Modelling, Reearch Report 555, inria, Grenoble, France, 3. [3] R.I. Hartley, Proceeding of the Darpaepritworkhop on Application of Invariant in Computer Viion, Azore, Portugal, 993, p.87. [4] M. Pollefey, L. van Gool, Proceeding of the Conference on Computer Viion and Pattern Recognition, Puerto Rico, USA, 997, p. 47. [5] R.I. Hartley. International Journal of Computer Viion (997) 5. [6] L. de Agapito, R.I. Hartley, E. Hayman, Proceeding of the Conference on Computer Viion and Pattern Recognition, Fort Collin, Colorado, USA, 999. [7] M. Armtrong, A. Zierman, P. Beardley, In: E. Hancock (Ed.), Proceeding of the fifth Britih Machine Viion Conference, York, England, 994, p. 59. [8] B. Caprile, V. Torre, International Journal of Computer Viion 4 (99) 7. [9] R. Cipolla, E. Boyer, Proceeding of IAPR Workhop on Computer Viion, Chiba, Japan, 998, p [] C.S. Chen, C.G. Yu, Y.P. Hung, Proceeding of the 7th International Conference on Computer Viion, Kerkyra, Greece, 999, p. 3. [] J. Koecka, W. Zhang, Proceeding of the 7th European Conference on Computer Viion, Copenhagen, Denmark,, p [] P. Sturm, S. Maybank, Proceeding of the Conference on Computer Viion and Pattern Recognition, Fort Collin, Colorado, USA, 999, p. 43. [3] Z. Zhang. Proceeding of the 7th International Conference on Computer Viion, Kerkyra, Greece, 999. [4] B. Trigg, Proceeding of the 5th European Conference on Computer Viion, Freiburg, Germany, 998. [5] E. Mali, R. Cipolla, IEEE Tranaction on Pattern Analyi and Machine Intelligence 4(). [6] B. Trigg, Proceeding of the 6th European Conference on Computer Viion, Dublin, Ireland,, p. 5. [7] P. Sturm, Proceeding of the Conference on Computer Viion and Pattern Recognition, Hilton Head Iland, South Carolina, USA,, p.. [8] B. Trigg, Proceeding of the Conference on Computer Viion and Pattern Recognition, Puerto Rico, USA, 997, p. 69. [9] M. Wilczkowiak, E. Boyer, P. Sturm, Proceeding of the 8th International Conference on Computer Viion, Vancouver, Canada,, p. 4.