Mirror shape recovery from image curves and intrinsic parameters: Rotationally symmetric and conic mirrors. Abstract. 2. Mirror shape recovery

Transcription

1 Mirror hape recovery from image curve and intrinic parameter: Rotationally ymmetric and conic mirror Nuno Gonçalve and Helder Araújo Λ Intitute of Sytem and Robotic Univerity of Coimbra Pinhal de Marroco - POLO II - Coimbra PORTUGAL Abtract Thi paper analyze the problem of the etimation of the local mirror hape in a catadioptric imaging ytem. We propoe a method to recover the D coordinate of mirror urface point a long a there are image of thoe point, i.e., a long a there i an image of a D geometric element that i reflected by thoe point. For that purpoe the information required i the image, the intrinic parameter of the camera, and the D coordinate of point in the cene. The etimation of the local hape can be ued to calibrate the ytem even though that problem i not addreed in thi paper. We addre the problem of the hape recovery for conic haped mirror and rotationally ymmetric mirror. Experimental reult for ynthetic image are preented.. Introduction Panoramic and omnidirectional image are being increaingly ued in many application. New and intereting application are being developed. Omnidirectional image are obtained by combining camera with mirror. Many of thee ytem ue configuration that aure that the projection i central. In what concern the type of mirror employed, rotationally ymmetric conic mirror are uually ued. However, thoe mirror provide central projection only for pecial poition of the camera [,, 9]. Suppoe that the type of the mirror urface i not known. Can the mirror urface be recovered? Uing which information? Previou work on thi topic include ome work on reflectance model, polarization, color, photometric characteritic and tructured light [, 8, 4,, ]. Other approache to the problem include uing tereo or multiple view [, ] and alo a moving oberver or moving urface [, ]. In thi paper we are intereted in non-central projection ytem with a perpective camera and a rotationally ymmetric mirror. The pecial cae of conic mirror i alo ad- Λ (nunogon,helder)@ir.uc.pt dreed. In thi paper we propoe a method to recover the mirror urface locally uing the following a priori information: the image, the intrinic parameter of the camera and D point in the cene. In the next ection the problem of the mirror urface recovery i addreed and in ection we addre the initial value problem. Thi problem reult from the fact that the urface recontruction i obtained from the integration of an ordinary differential equation. In ection 4 the validity of the model i demontrated with experimental reult and then we draw the concluion.. Mirror hape recovery In thi ection the problem of etimating the D mirror point correponding to the image of a moving point in the cene i addreed. The point can decribe any curve in the real cene. The correponding curve on the image (after the reflection) i tracked. The only a priori information required are the image of that very point, the intrinic parameter of the camera and the D coordinate of two or three point in the cene (depending on the model ued). Let x () be the point on the mirror urface that we wih to recover. A curve in D pace will be projected in the image plane and let the curve be parameterized by the variable which hould not be confued with time (ee figure ). It i poible to expre x () a the um of it two perpendicular component (ee figure ). x () = L ()+ < x (); Vr() > Vr() () where Vr() i the unitary reflected ray and <:;:>i the inner product. Vector L () repreent the ditance vector from the origin of coordinate to the reflected ray (notice that < L (); Vr() >= ). Let u now differentiate equation with repect to the

2 optical center P(Xcam,Xcam,Xcam) image plane [x_img(),x_img(),f image plane x (x_img,x_img,f) x L() Vr() x P(X,X,X) x mirror urface X() O x Vr N x Vi Figure : Perpendicular component of the reflected ray. mirror urface Figure : Reflection through a pecular mirror parameter. It yield: x () = L ()+ < x (); Vr() > Vr()+ _ + < x _ (); Vr() > Vr()+ < _ x (); Vr() > Vr() () Let u now calculate the inner product of equation with _ Vr() yielding: < x (); Vr() >=< L (); Vr() > + + < x (); Vr() >< Vr(); Vr() > () Since Vri a unit vector, < _ Vr; Vr >= tand. Rearranging the term with repect to < x (); Vr() > and ubtituting in equation one obtain (the notation v () i ubtituted throughout the paper by the horter form v ): X < x; Vr> Vr x = + L < L; Vr> Vr Vrk (4) Vrk k _ k _ which i a linear ytem of differential equation on x. Let u now analyze each term of thi expreion. The reflected ray pae through two known point: the optical center of the camera (P (x cam ;x cam ;x cam )) and the image point (P i (x cam x img ;x cam x img ;x cam f )), where f i the focal length and (x img ;x img ) are the image coordinate. Thee expreion however are valid only if there i no rotation between the world coordinate ytem and the camera coordinate ytem. Thi aumption can be conidered without lo of generality. The expreion of the reflected ray i thu known and it unit vector i given by: (x img;x img ;f) T Vr= () qx img + x img + f and it derivative with repect to i traightforward. Both Vrand Vrdepend _ on the image point and it motion in the image, which can be etimated by tracking a point moving along a curve in the image. Equation 4 i alo a function of the vector L and it derivative with repect to parameter _ - L. Since L i the ditance vector from the origin of coordinate ( O ) to the reflected ray it i traightforward to compute it. The following expreion can be ued: L = P i ( P i O ) ( P P i ) k P ( P P i ) () P i k and it derivative i alo traightforward. Since for each image point it i poible to know or etimate all coefficient of equation 4 it can be rewritten in matrix form a: A() _ x () = x () +'() () where the ma matrix A() and the vector '() can be eaily calculated. Notice however that thoe entitie vary with the parameter and therefore thee differential equation have variable coefficient.

3 The nature/type of thi ytem of differential equation depend on the matrix A(). A a matter of fact it will depend on whether A() i ingular or not. An analyi of the determinant of A() how that it i actually ingular and therefore one could be led to think that the ytem of equation i a DAE (Differential ytem of Algebraic Equation). However, matrix A() i made up of only two linearly independent vector and it i not poible to tranform it into an ODE (Ordinary Differential ytem of Equation) or a DAE. That mean that new retriction mut be conidered o that matrix A() ha rank. Two new retriction (although one would be enough) can be added to the ytem by conidering the nature of the image projection (perpective projection or orthographic projection if that i the cae). The retriction are x img = f (x x cam )=(x x cam ) and x img = f (x x cam )=(x x cam ), where x, x and x are the coordinate of the D point to be recovered and x cam, x cam and x cam are the camera coordinate. Then the ytem become: 8 a x_ + a x_ + a _ >< a x_ + a x_ + a _ x = x + k x = x + k a x_ + a x_ + a x_ = x + k x cam x img fx cam = fx + x img x >: x cam x img fx cam = fx + x img x Thi equation i a DAE of index. The index of a DAE i the minimum number of derivative that have to be taken on ome of the equation o that an explicit ODE can be obtained. So, taking the derivative of the new retriction with repect to the parameter the reulting ODE ytem i obtained: 8 a x_ + a x_ + a _ >< a x_ + a x_ + a _ f _ >: f _ x = x + k x = x + k a x_ + a x_ + a x_ = x + k x x img x_ x x img x_ or in matrix form: a a a a a a a a a 4 f x img f x img = v x x v x x cam = v x x v x x cam _ x = 4 v x v x k (8) (9) k x + k 4k 4 k () A() _ x = B() x + ' () () where k 4 = v x x cam and k = v x x cam. Since the new retriction introduce a third linearly independent condition, the rank of matrix A() i. Furthermore, matrix A() i now over-determined and o it peudo-invere matrix can be ued to etimate x _ (). The final ytem become: _ x () =(A T A) A T B x () +(A T A) A T ' () which an ODE ytem with variable coefficient matrice.. Initial Value Problem The initial value problem i till unolved. Since the main interet of thi method i the recovery of the D hape of the reflecting mirror with minimal a priori data, it i important that the initial value problem be olved with minimal initial knowledge of the ytem. Let u now preent a poible olution for thi problem... Knowing the Starting Point Solving equation require that the initial value x = x () i known. Conider that one line (or any arbitrary curve) i tracked in the image. Equation can be ued to etimate the D coordinate of the mirror point correponding to the point being tracked in the image, a long a the D coordinate of the initial mirror point are known... Cloed Contour Aume that the cene contain ome cloed contour uch a rectangle, triangle, circle or any arbitrary cloed contour (example of thi are door or che board-like pavement). The interet of thoe contour lie on the fact that equation can be ued to etimate the D coordinate of the mirror point that reflect the contour without the knowledge of an initial point. A a matter of fact all that i required i that the contour i travered by iterating the method until the tarting point coincide with the final point (once the figure i cloed). If one ha a good initial gue, the method hould converge rapidly uing wellknown method for olving non-linear equation... Conic reflector The previou olution i intereting but a we hall ee, experimental reult how that the convergence of uch method i uually poor unle good initial etimate are known. Since the many of the mirror ued in omnidirectional viion correpond to conic, thi can be ued to obtain an initial gue. Conider then that the reflector mirror correpond to a conic (including elliptic, parabolic, hyperbolic and pherical). The equation for uch mirror i: x + x + Ax + Bx = C ()

4 or rewriting: x + x + A x + B A = B 4A + C () where thi equation aume that the conic i centered in the origin of the coordinate ytem and aligned with the coordinate axe. The general conic equation with an arbitrary orientation can be ued although it increae the number of unknown. Thi problem i being currently addreed. The generic mirror point i then given by: x = 4 x x q C+ B 4A x x A B A (4) Taking the partial derivative with repect to the patial coordinate x and x it yield: 8 >< = 4 4 r C+ B r C+ B x 4A x x A x 4A x x A B A B A and the normal vector to the mirror urface = x k () () Furthermore, the incident ray Vican be recovered uing the normal vector and the reflected ray Vr: Vi = V r < N; V r > N () where Vr can be calculated uing equation and Vi = ff( x P ), being P the vector with the D world coordinate of the cene point (ee figure ). If the D coordinate of a point are conidered to be known a well a the reflected ray Vr, equation provide equation for the following even unknown: x, x, x, A, B, C and ff. However, by uing the perpective projection equation the number of unknown can be decreaed by two ince x = x img (x x cam )=f + x cam and x = x img (x x cam )=f + x cam. The ytem of equation then include equation for five unknown: x, A, B, C and ff. We till have more unknown than equation. Each additional D point add equation and two new unknown:x j and ff j. Therefore the minimal number of Surface point coordinate....9 Etimation of the mirror urface Figure : Etimated value (dahed line) and true value (olid line) of the mirror urface. There are three pair of line in the graphic: the upper i the x coordinate and the two in the bottom of the plot are the x and x coordinate. Elliptic mirror hape. D point i three ince in that cae there will be 9 equation with 9 unknown (the hape parameter are the ame). The knowledge of the D coordinate of three point in the cene and of their correponding image coordinate allow the computation of an etimate of the initial value required by equation. Additionally etimate for the conic hape parameter A, B and C are alo obtained. Thee value are obtained a olution of the nonlinear ytem of equation. 4. Experimental reult In thi ection experimental reult obtained with ynthetic image are preented. For thi et of experimental reult an elliptic mirror and a perpective camera with a focal length of mm were conidered. We alo performed tet with an hyperbolic mirror. Uing equation the image of everal geometrical element (line, rectangle and curve) were obtained. The ground truth value computed were the image coordinate (x img () and x img ()), the image flow (v x () and v x ()), the coordinate of the correponding point on the mirror urface ( x (), to be recovered) and the D coordinate of the point in the cene. The firt experimental reult correpond to the etimation of the coordinate of mirror urface point obtained by uing a D quare. The reult correpond to the olution of equation with known initial value, i.e., when x i known. Figure and 4 how the etimated value and their relative error. The error are mall for all the coordinate. Figure diplay in D the ground truth point and the point recovered on the mirror urface. A it can be een the recovered curve i not cloed. To find out a good tarting point the approach decribed in ub- 4

5 . Etimation of the mirror urface Etimation reult of the quare in the mirror urface in D Relative error (%) Figure 4: Relative error in the etimation of the point on the mirror urface. x.. x Figure : D pace repreentation of the curve recovered (thicker line) and the ground truth curve, after the iterative matching of the initial and final point..9 x. Etimation reult of the quare in the mirror urface in D.4... Etimation of the mirror urface x x.. x.9 Surface point coordinate..9 Figure : D repreentation of the curve recovered (thicker line) and the ground truth curve. ection. wa ued. A a reult of the nonlinear iterative matching proce a cloed curve wa recovered. The reult are plotted in figure. Finally, the retriction of the conic mirror wa ued to etimate the initial value. In thi tet the curve ued wa a D line egment line intead of a cloed curve. Figure and 8 how repectively the comparion of the etimated and ground truth value for each patial coordinate and the relative error in the etimation the curve recovered. We alo performed tet with an hyperbolic mirror. Figure 9 how that alo in hyperbolic the etimation reult are good.. Summary and Concluion In thi paper the problem of the etimation the mirror hape of an omnidirectional viion ytem i addreed. A method to locally recover the D coordinate of the mirror point that reflect a D curve into the image plane i propoed Figure : Etimated value (dahed line) and true value (olid line) of the mirror urface. There are three pair of line in the graphic: the upper i the x coordinate and the two in the bottom of the plot are the x and x coordinate. Conic mirror hape wa aumed. The etimation of the local hape can be ued to calibrate the omnidirectional ytem. If the curve in the image plane i travered the problem of etimating the mirror point i olved by a ytem of differential equation, once the initial value i available. However a good initial value i required and that i the main difficulty. If the mirror i conidered to be conic (including elliptic, parabolic, hyperbolic and pherical) then the D coordinate of three point in the cene can be ued to get a good initial etimate. The experimental reult how that the relative error in the etimation of the D coordinate of the mirror point i mall. In the future everal extenion are being conidered namely to other type of mirror and to tudy the method numerical tability.

6 Etimation of the mirror urface Etimation of the mirror urface.8. Relative error (%)..4. Surface point coordinate Figure 8: Relative error in the etimation of the line egment point on the mirror urface. Conic mirror hape wa aumed. Acknowledgment The author gratefully acknowledge the upport of project OMNISYS-POSI/SRI/4/, funded by the Portuguee Foundation for Science and Technology. Reference [] L. Wei A. Sanderon and S. Nayar. Structured highlight inpection of pecular urface. PAMI, ():44, January 988. [] S. Baker and S. Nayar. A theory of catadioptric image formation. In IEEE ICCV, page 4, Bombay, 998. [] Max Born and Emil Wolf. Principle of Optic. Pergamon Pre, 9. [4] Donald Burkhard and David Shealy. Flux denity for ray propagation in geometrical optic. J. Optical Soc. of America, ():99 4, March 9. [] C. Geyer and K. Daniilidi. A unifying theory for central panoramic and practical implication. In ECCV, page 44 4, Dublin,. [] G. Healey and T. Binford. Local hape from pecularity. CVGIP, 4(): 8, April 988. [] K. Ikeuchi. Determinig urface orientation of pecular urface by uing the photometric tereo method. PAMI, (): 9, November 98. [8] J. Koenderink and A. Doorn. Photometric invariant related to olid hape. Optica Acta, ():98 99, 98. Figure 9: Etimated value (dahed line) and true value (olid line) of the mirror urface. There are three pair of line in the graphic. From the top to bottom: the upper i the x coordinate, the middle pair i x and in the bottom the pair of line correpond to x coordinate. Hyperbolic mirror hape. [9] S. Lin and R. Bajcy. True ingle view point cone mirror omni-directional catadioptric ytem. In IEEE ICCV, Vancouver, July. [] M. Longuet-Higgin. Reflection and refraction at a random moving urface: i, ii and iii. J. Optical Soc. of America, (9):88 8, September 9. [] Shree Nayar and Simon Baker. Catadioptric image formation. In DARPA Image Undertanding Workhop, New Orlean, May 99. [] Michael Oren and Shree Nayar. A theory of pecular urface geometry. IJCV, 99. [] Michael Groberg Rahul Swaminathan and Shree Nayar. Cautic of catadioptric camera. In ICCV, Vancouver, Canada, July. IEEE. [4] K. Ikeuchi S. Nayar and T. Kanade. Determinig hape and reflectance of hybrid urface by photometric ampling. IEEE Tranaction on Robotic and Automation, (4):48 4, Augut 99. [] X. Fang S. Nayar and T. Boult. Separation of reflection component uing color and polarization. IJCV, (): 8, February 99. [] H. Schultz. Retrieving hape information from multiple image of a pecular urface. PAMI, ():9, February 994. [] K. Torrence and E. Sparrow. Theory for off-pecular reflection from roughened urface. J. Optical Soc. of America, (9): 4, September 9.