3D Shape of Specular Surface Measurement Using Five Degrees of Freedom Camera System

Transcription

1 Khar usuf, Prasetyo Ed, Amr Radz Abdul Ghan 3D Shape of Specular Measurement Usng Fve Degrees of Freedom Camera System KHAIRI USUF, PRASETO EDI and AMIR RADI ABDUL GHANI Department of Engneerng Desgn and Manufacture Faculty of Engneerng, Unversty of Malaya 63 Kuala Lumpur MALASIA Abstract : In ths report, we descrbe a new method n obtanng a 3D shape of specular surface by usng fve degrees of freedom (-DOF) camera. We use the prncple that the normal vectors of the surface are extracted by algnng the camera axs and the surface normal vector. From the normal vector data, the shape of the surface s reconstructed. After the flat and the ball surface, the obects of the research are contnued to convex and concave specular surface. The convex obect s an automoble glass wndow, where n ths obect we show that the back reflecton of a parallel surface of a thn glass can be elmnated. In the concave obect we have a lmtaton that the radus of obect s not smaller than twce of the nearest dstance from the camera to the surface. The result shows that the methodology mproves the 3-D shape of obect measurement wth good accuracy. Key-Words: - shape reconstructon, pattern recognton, computer vson, mage processng, fve degrees of freedom camera, specular surface, normal vector, robotcs. 1 Introducton Specular surface has become nterestng obect snce t s used wdely especally n glass and metal ndustres. Besdes that, many ndustral materals are made of metal and have strong specularty and lttle dffuse reflecton. Moreover, many practcal tasks n robot vson and nspecton requre nterpretaton of mages of specular or shny surfaces.[1][]. Several methods of obtanng the shape of a surface have been proposed such as multple vew, slt ray proecton, moré method, shape from shadng and some of them are wdely used n practcal applcatons. An approach utlzes graded llumnaton felds to llumnate the surface has been proposed by Ikeuch[1]. It uses the assumpton that from a dstant source and known obect poston, the angular relaton between an observed reflected brghtness and the poston n the source feld wth that brghtness could be used to derve surface orentaton. Ths approach requres measurement of reflected brghtness wth suffcent accuracy to solate a partcular poston n the source feld. It uses the reflectance map and photometrcs stereo. Sanderson et. al. [] proposed an approach of structured hghlght to obtan the 3-D shape of specular surface. Ths approach uses multple pont lght sources and mages the resultng hghlght pattern reflected from the surface. It uses a sphercal array of fxed pont sources to scan all possble poston and drecton of ncdent lght rays relatve to a fxed camera. It s assumed that the pont source are dstant from the surface, that the surface s at a fxed reference heght, and the extent of the surface from the orgn s much smaller than the dstance to the source. Such that the angle of ncdence of llumnaton s determned only by the poston of the source and does not depend on the relatve poston of llumnaton of the surface. Babu et al. [3] uses a specular surface model to estmate orentaton of an extended planar surface based on estmaton of parameters n the shadng model usng contours of constant brghtness n the mage. Ths approach utlzes a parallel source and perspectve camera model so that observed brghtness of reflected llumnaton depends on surface poston relatve to the camera axs. Schulz [4] showed some smulaton of retrevng shape nformaton from multple mages of a specular surface under natural lghtng condton. The technque whch s known as specular surface stereo consst of two parts, that are: a surface orentaton procedure that computes the gradent at a pont on the surface by mnmzng the dfference between observed and syntheszed rradance value, and a surface propagaton procedure that retreves ISSN: Issue, Volume 4, Aprl 9

2 Khar usuf, Prasetyo Ed, Amr Radz Abdul Ghan the shape of whole surface. The method s stll n consderaton snce addtonal factors must be ncluded n the mage synthess and llumnaton models f the method s appled to the real word. Ths research conductng a new method of obtanng the shape of specular surface by extractng the surface normal vectors based on the reflecton theorem and mage processng. A -DOF camera s used to obtan the normal vector by scannng the algnment between the ncdent and reflected rays. A LED(Lght Emttng Dode) s placed on the camera axs to gve an ncdent ray to the surface. The camera axs s assumed ntersects the center of mage plane, hence, the algnment s reached when the reflected mages concde wth the center of mage plane. The locaton of the lens center and the drecton of the camera axs on the algnng condton s recorded as the normal of the surface. From the normal vector data, the shape of the surface s reconstructed. Ths reconstructon gves a lmtaton n applcaton that s used only to a smooth curve of obect wth no hole. In the prevous work we had verfed the flat surface obect and the convex surface obect. The research s contnued wth examnng a convex and a concave surface. At ths tme, the convex surface has a back surface reflecton (lke an automoble glass wndow) and has the lmtaton that the gradents at each correspondng ponts on both surfaces should be parallel. For a concave obect, t also has a lmtaton that the radus of the surface s not smaller then twce of the nearest dstance from the camera to the surface, where the camera stll be able to receve the mage n focus. Prncple z φ Lens center A (u,v) f x θ v F (U o,v o ) A(,,) Fg.1. A smple camera model. A, whch has the coordnates (,,). The coordnates of A n the mage are A (x A, z A ), where x A = f ; z = f ; (1). A s called the depth of A, whch measured from the lens to the obect. The dstance from the lens to the mage plane s denoted by f, the focal length. In the mage coordnate system, A (x A, z A ) becomes A (u,v), after convertng x and z by scale factor α to u and β to v as follows u = U o + x A α (pxels); v = V o - z A β ; (). where α, β n pxel/mm. Lens center u Image plane Lght source y Specular.1 Extractng the Normal vector. A specular surface always reflects the ncdent ray n the same angle. The normal vector at that pont on a surface can be found f the ncdent and reflected angles are o or the ncdent ray algn wth the reflected ray. A smple camera model (see Fg. 1) has a camera axs wth the center of mage and the lens center on the camera axs. Snce the gradent of a specular surface at a pont s not drectly perpendcular to the ncdent ray, the camera must be panned, tlted and moved n three coordnate axes to satsfy the algnment. Fgure shows the formaton of an mage of a sngle pont, denoted by Fg.. Lght source and Vewer To get the normal vector, the mage of pont A must be moved to the center of mage plane (U o, V o ). As shown n Fg. 3, the camera must be panned and tlted as much as : θ = tan -1 ( (U o - u)/f) ; φ = tan -1 ((V o - v)/f) ; (3) where F = fα (pxels), θ s the pan angle, and φ s the tlt angle. We can express the normal lne of a pont on the specular surface as follows. ISSN: Issue, Volume 4, Aprl 9

3 Khar usuf, Prasetyo Ed, Amr Radz Abdul Ghan z φ x z θ Normal vector θ x Fg. 3. Pan and tlt Angle. y y c = m x x c = l z z c n φ Normal lne η (l, m, n) y y (, cos φ, sn φ) b(,1,) y (4) x c, y c, z c, are the coordnates of a pont on the normal lne and l, m, n are the drecton number of the normal lne. We can derve that l = m tan θ () n = m tan θ/cos φ y = p ; y x z = q where p l n = ; q = (7) m m Dfferentaton of eq. (6) wth respect to x and z are = α + α x + 3α x + α z + α xz + α z (8) = α + α x + α x + α z + α xz + 3α z (9) z Eq. (7), (8), and (9) gve l = α + α x + 3α x + α z + α xz + α z (1) m n = α + α x + α x + α z + α xz + 3α z (11) m By usng eq. (1) and (11), a procedure of reconstructon can be made. η(l, m, n) = C. Shape Reconstructon. Reconstructon of the shape begns wth the assumpton that the shape s smooth otherwse we can not determne the shape of obect from the surface normal vectors. Snce there are nfnte number of surfaces have the same normal vectors, the shape of obect can not drectly be reconstructed from the extracted normal vectors. Ths problem s solved by knowng at least one pont coordnates on the surface as a startng pont to lead the reconstructon. Cubc polynomal functon of a surface shape s used here and expressed as follows : y = α + α 1 x + α x + α 3 x 3 + α 4 z + α xz + α 6 x z + α 7 z + α 8 xz + α 9 z 3 (6) The partal dervatves of ths surface functon s related to the components of surface normals, that are : = f(x, z) startng pont Fg. 4. Reconstructon of the shape. As shown n Fg. 4 (two-dmensonal vew), a plane whch s perpendcular to the -axs and passng through the startng pont, s made. The equaton of the plane can be wrtten as y = f ( xo, yo, zo ) + ( x xo ) + ( z zo ) (1) z (x o, y o, z o ) s the startng pont, y s the gradent of the functon (6) n the x drecton, y s the gradent of the functon (6) n the z z drecton. Snce ths plane s perpendcular to the -axs, the ISSN: Issue, Volume 4, Aprl 9

4 Khar usuf, Prasetyo Ed, Amr Radz Abdul Ghan plane equaton becomes smple as, = C (13) where C s the y coordnate of the startng pont. The ntersecton ponts between the normal lne and the plane and the gradent at each pont can easly be obtaned, whle the coeffcent drecton of the normal lne are avalable. These are the requrements of reconstructng the shape. The dfferences between the measured normal drecton (l,m,n) and the normal drecton of the plane data s l ( ) + m ( ) + ( ) n ε = ( ) z m ε= p u q v (14) s β c s β s = f(x, z) β c l, m, n Target surface Fg.. Reconstructon of the Shape. Startng pont = C As shown more detal n Fg., the reconstructon s started by a plane data(coordnates and drectons), where the plane data contans the ntersecton ponts between the measured normal vector and the plane surface. The dfference angle β ( n - plane) and γ (n - plane) between the measured normals and the plane normals s mnmzed. It s done by nputtng the value of x and z of ntersecton ponts to the eq.(1) and (11). We wll get the value of α. Ths value then s used n eq.(6) and the startng pont to get the frst curve of, where wth coordnates x and z n the curve s s. In other word, we get the curve through s. These curve has the ntersecton ponts wth the measured normals at c. These ponts coordnates, the normals of the curve and the angle β at these ponts can be calculated. Then the next dfferences mnmzaton usng the curve and the measured normals can be done wth the same procedure. Ths teraton s done untl the target curve s found and the stop condton s acheved (Σ and Σ γ < small number e). β where p = ; q = ; z l n u = ; v = ; (1) m m Mnmzaton : = ( = --- 9) (16), [( )] = p u = ; =, 1 ( p u ) x = ; [ q v ] = p u 3x = ; 3 =, ( ) = ; 4 =, ( p u ) z ( q v ) x + = ; ε, ( ) =, 6 ( ) p u x z + ( q v ) x = ; 7 =, ( ) q u z = ; ISSN: Issue, Volume 4, Aprl 9

5 Khar usuf, Prasetyo Ed, Amr Radz Abdul Ghan 8 =, [( p u ) z + ( q v ) x y ] = 9 [( )] =, q u 3 z = ; Soluton of = k ( k =,...,9) can be wrtten as the followng smultaneous equaton. ;.3 Reconstructon sequence. The ntersecton ponts are belong to several shapes of the surface, however, snce a known pont s ncluded n the nput data, the teraton wll lead to an approprate curve. However, the reconstructon s not done drectly usng all the normal vector snce that way can not acheved an approxmate shape of the surface. It s done n a pecewse reconstructon patch by patch. At every patch, to solve eq. (6), ponts are need- A T A α = A T v (17) where A= 1 x1 3x1 z1 x1z1 z 1 1 x1 x1 z1 x1z1 3z xn 3xn zn xnzn zn 1 xn xn zn xnzn 3zn 1 st patch of tracng nd patches of tracng 3 rd patches of tracng 4 th patches of tracng u1 v 1.. v =., α =.. un v n α1 α α 9 The equaton means that approxmate value of the coeffcents are obtaned n a sense of least mean square or regresson [11]. Fg. 6. Reconstructon Sequences. ed, otherwse the column vector of A are not lnearly ndependent. In our method, we use 9 ponts ncluded n 3 x 3 square matrx. All the normal vectors are traced patch by patch on the ISSN: Issue, Volume 4, Aprl 9

6 Khar usuf, Prasetyo Ed, Amr Radz Abdul Ghan surface carefully n a sequence as shown n Fg Reconstructon error. In ths reconstructon technque, some errors may occur because of two thngs that are, the dfference n surface functon, and the error of measured normal vectors. The frst knd of error may occur because the surface functon can not be approached precsely by a polynomal functon. For example, a cylndrcal surface can not be acheved precsely by a polynomal functon although t can be changed nto a polynomal functon by Taylor seres. Snce we used the bcubc polynomal functon, the hgher order of the seres wll cause the error. However, ths knd of error s not sgnfcant and always n the range of tolerance. Below a smulaton of the frst error s shown. Say that there are two knds of surface wth the functon as and y = ax, for every z, y + x = r, or y = r x for every z. Consder that, both of surface have a set of deal normal vector (l, m, n) whch s calculated as l =, n =. m where l + m + n = 1, such that 1 m =, and l = m. + 1 The frst surface can be reconstructed from the normal vector precsely wth no error as shown n Fg. 7.a, whle the second reconstructed surface has some error as shown n Fg. 7.b. (mm) (mm) (a) (mm) (b) (mm) Error(mm) Error(mm) Fg. 7 Frst Reconstructon Error. The second knd of error occurs because of the error of normal vector. However, the normal vector error does not cause the propagaton error. Superposton of normal vector error makes the error of reconstructon stable around some value. It can be shown by the followng smulaton. The smple surface functon of y = ax for every z s agan used to get easy understandng. As presented before, reconstructon of ths surface from ts normal vector has no error. Here three knds of normal vector sets are presented. The frst set as shown n Fg. 8(a) contans error of +.1 o for the ISSN: Issue, Volume 4, Aprl 9

7 Khar usuf, Prasetyo Ed, Amr Radz Abdul Ghan Msalgned normal vector Ideal normal vector Msalgned normal vector Ideal normal vector.1 o.1 o Startng pont Startng pont (a) (mm) Error(mm) (a) (mm) Error(mm) (mm) (mm) (b) (a) Fg. 8 (a).1 o (exagerated) error to the left and rght of the normal vector at the left and rght of the startngpont, and (b) ts reconstructon error. Fg. 9 (a).1 o (exagerated) error to the rght and left of the normal vector at the left and rght of the startng pont, and (b) ts reconstructon error. normal vectors at the left of startng pont and -.1 o for the normal vector at the rght of startng pont. Ths s the test of error f the orentaton s comng from the left and rght from startng pont when the path of camera comng from the left and rght of startng pont. The second set as shown n Fg. 9(a) contans error of.1 o for the normal vectors at the left of the startng pont and +.1 o for the normal vectors at the rght of the startng pont. Ths s the opposte test of error of the above orentaton. The thrd set as shown n Fg. 1 (a) contans error of +.1 o and.1 o respectvely for the left and the rght normal vector from the startng pont. Ths test s the combnaton of both orentatons of normal vectors. From these three sets of normal vector, ther surface s reconstructed and ther error s calculated by dfferentatng them to the deal surface. The error s obtaned as shown n Fg. 8(b), 9(b), and 1(b). Ths fgures show that f every normal vector has the same orentaton of error, the reconstructon error wll be large. However, as shown n Fg. 1(b), f the orentaton s opposte at two neghborhood normal vectors, the error s stable around some value. Ths s one example of normal vector data where the summaton of error s zero. As the matter of fact, f the reconstructon s performed to a set of statstcally random error normal vector, wth the summaton s zero, the error of reconstructon s stable around some value. It concludes that the propagaton error does not occur n ths method of reconstructon ISSN: Issue, Volume 4, Aprl 9

8 Khar usuf, Prasetyo Ed, Amr Radz Abdul Ghan Msalgned normal vector Ideal normal vector reflected ray s analyzed by an mage processor and a computer..1 o Startng pont Parallelzaton of Axs and the Camera Axs. When the system s started, we don t know whether the camera axs s parallel or not to the axs snce the zero value of the rotary table s not set to a certan coordnate. A procedure usng a plane wth a (mm) Error(mm) (mm) (b) Fg o (exagerated) and -.1 o error respectvely for normal vector at the left and rght of the startng pont, and ts reconstructon error. 3 Experment 3.1 Preparaton. In order to obtan surface normal vectors, we use fve degrees of freedom moblng camera system. The camera s equpped wth a LED whch s attached on the camera axs and acts as the lght source. Fve degrees of freedom camera system s shown n Fg. 11. Ths system conssts of a moble camera, a movement control equpment, an mage processor and a personal computer. The camera s attached on two rotary tables whch are mounted on one of three lnear stages n such a way that the camera can be moved along,, and axes and also can be panned and tlted by computer control. The ray from the LED s reflected by the specular surface and caught by the camera. The mage of the Movement control Specular Ca mera loca t on & angle nformaton Command Lgh t source Computer θ φ Camera Image nforma ton Com mand Ima ge Fg. 11. Fve Degrees of Freedom Camera System. Camera Plane A mark Camera axs Fg. 1. Parallelzaton of the and the Camera Axs. spot mark s performed to make the and camera axs parallel by movng the camera along the camera axs. (Fg. 1) Camera Scannng Path As shown n Fg. 13, camera s moved n x and z axs from the startng pont, however t must be panned and tlted f the obect has a curved surface. The dstance between camera and obect ISSN: Issue, Volume 4, Aprl 9

9 Khar usuf, Prasetyo Ed, Amr Radz Abdul Ghan θ y Startng pont x Fg. 13. Camera scannng Path n - Plane. Specular surface Camera Lens center Fg. 14. Camera scannng Path n - Plane. wll change and the camera wll not catch the reflected mage f the path n - plane s not arranged. Therefore, after frst movement, the dstance of the camera to the obect s calculated and the calculated dstance s used for the next movement (see Fg. 14). Fg. 1. Measurement of fxed pont by trangulaton. wll elmnate the effect of back reflecton. As the matter of fact the back surface reflected ray also algn wth the ncdent ray. However, the gradents at the correspondng ponts on both surface should be parallel, otherwse we wll have two knds of the same ntensty mage. We can not dstngush and elmnate between the back surface mage and the front surface mage. Therefore, parallelsm of the gradents at the correspondng ponts of both surface s the prerequste (See Fg. 16). The concave obect of the next step s a concave mrror. The mage of the lght source should be a vrtual mage n the surface. If the radus s smaller than twce of the nearest dstance ( that can be acheved by the camera wth focus mage) from the camera to the surface, the mage wll occur as real mage and t needs more consderaton n calculaton and technques. Therefore, at ths stage we have the lmtaton that the radus should not be smaller than twce of the nearest dstance from the camera to the surface (Fg 17) Obtanng the Startng Pont As shown n Fg. 1, the startng pont s obtaned by puttng a spot on the obect and measured the dstance between the lens center and the spot mark by trangulaton. Image plane Image 1 LED Image Measurng the Convex Obect wth back surface reflecton, and the Concave Obect. The convex obect of the next step has the characterstcs of back surface reflecton. In ths case, the algnng of the reflected and ncdent ray Fg. 16. The lmtaton of the thn glass ISSN: Issue, Volume 4, Aprl 9

10 Khar usuf, Prasetyo Ed, Amr Radz Abdul Ghan Real mage C Bad poston of LED C s the center pont F s the focus pont f Good poston of LED Fg. 17 The lmtaton of the concave obect. Vrtual mage 3. Expermental Result We present here some of the expermental result that are a ball shape surface as the result of the prevous work, an automoble glass wndow, and a radal concave mrror. The ball shape surface s a steel ball bearng surface wth known radus. The radus of the ball s 7. mm. From ths surface, 1 (1 x 1) data were scanned. Measurement were executed 1 tmes and the result are averaged. The average radus of the ball whch was attaned s mm. The average error of the radus was.66 mm. Reconstructed shape of the ball s shown n Fg. 18. The automoble glass wndow shows the ablty of the system to measure a transparent thn(3. mm) and large sze obect. The dstrbuton of error was obtaned by a comparson wth the contact measurement data usng a matchng program. We got a bg maxmum error, that was 167 μm. Ths error may be dstrbuted by the dfferent set up between the contact measurement and ths method. Fg. 19. Reconstructon of the Shape of the Automoble Glass Wndow and ts Error Dstrbuton. The surface has a cylndrcal form and has an elastc deformaton. It s weght may deform the shape n the contact measurement snce t was measured horzontally whle ths deformaton dd not occur when measured by our method vertcally. Reconstructon of the shape of ths obect and ts dstrbuton of error s shown n Fg 19. A concave mrror was used to check the ablty of the method to measure the concave obect. The obect radus known as 1 mm was used to check the result by comparng t to the reconstructed radus. We found that the maxmum error was μm. The reconstructed shape of ths obect s shown n Fg.. (a) n the contour lne and the dstrbuton of error s shown n Fg.. (b). Fg. 18. Reconstructed Ball Shape. ISSN: Issue, Volume 4, Aprl 9

11 Khar usuf, Prasetyo Ed, Amr Radz Abdul Ghan (a) (b) Fg.. Reconstructon of the shape of the concave mrror and error dstrbuton. 4 Concluson The new method of obtanng a 3D shape of specular surface by usng fve degrees of freedom (-DOF) camera has been developed. Ths method uses normal vector to reconstruct the shape of the surface leadng by a startng pont of the surface. The normal vector s obtaned by scannng the locaton and drecton of the pont on the surface by makng the concdent condton of the ncdent ray on and reflected ray from the surface. Present result have a good accuracy whch are shown by a ball shape surface (66 μm), automoble glass wndow (167 μm), and concave mrror ( μm). More research and development s needed to acheve the better result especally for a large surface as automoble glass wndow. Both hardware and software equpment have ther own source of error, however, the hardware development need other precson technology to make precse machne. References: [1] K. Ikeuch, Determnng surface orentaton of specular surfaces by usng the photometrc ste reo, IEEE Trans. Pattern Anal. Machne Intell., vol PAMI-3, Nov. 1981, pp [] A. C. Sanderson, L. E., Wess, and S. K. Nayar, Structured hghlght nspecton of specular surfa-ce, IEEE Trans. Pattern Anal. Machne Intell., vol PAMI-1, January 1988, pp 44-. [3] M. D. R. Babu, C-H Lee, and A. Rosenfeld, Determnng plane orentaton from specular reflectance, Pattern Recognton, vol. 18, no. 1, pp. 3-6, 198. [4] H. Schultz, Retrevng Shape Informaton from Multple Image of specular, IEEE Trans. Pattern Anal. Machne Intell., vol PAMI - 16, pp. 19-1, February, [] M. Baba, T. Konsh, H. Handa, Shape Measurement of Columnar Obects wth Specular s by Slt Ray Proecton method, IEICE Trans. On Informaton And Systems, Vol. J83-D-II, no. 8, pp , August. [6] P. Ed. A Flow Control for a Hgh Subsonc Regonal Arcraft Explotng a Varable Camber Wng wth Hybrd Lamnar Flow Control. IASME TRANSACTIONS Journal on Flud Mechancs and Aerodynamcs, Issue 6, Volume, August, ISSN , page [7] P. Ed. The Development of N- Mltary Verson. WSEAS TRANSACTIONS on Flud Mechancs, Issue 8, Volume 1, August 6, ISSN , page [8] P. Ed, Nukman. and Aznar A.. The Applcaton of Computatonal Flud Dynamc (CFD) on the Desgn of Hgh Subsonc Wng. WSEAS TRANSACTIONS on APPLIED and THEORETICAL MECHANICS, Issue 9, Volume 3, September 8, ISSN : [9] K. Jan, Fundamental of Dgtal Image Processng, Prentce Hall, Inc, Englewood Clffs, New Jersey, [1] B. K. P. Horn, Determnng shape from shadng, n the Psychology of Computer Vson, P. H. Wnston Ed., New ork, Mc Graw Hll, 197. [11] Hald, A. Statstcs Theory wth Engneerng Aplcaton, New ork: John Wley & Sons, Inc ISSN: Issue, Volume 4, Aprl 9