The Comparison of Calibration Method of Binocular Stereo Vision System Ke Zhang a *, Zhao Gao b

Transcription

1 3rd Internatonal Conference on Materal, Mechancal and Manufacturng Engneerng (IC3ME 2015) The Comparson of Calbraton Method of Bnocular Stereo Vson System Ke Zhang a *, Zhao Gao b College of Engneerng, Zhejang Normal Unversty, Jnhua, , P.R.Chna a jhkeanu@zjnu.cn, b gz@zjnu.cn Keywords: Stereo vson, Calbraton method, Genetc algorthm, BP neural network Abstract. Stereo vson s a non-contact method to realze 3D reconstructon of free-form surface, and has a wde range of applcatons n the feld of reverse engneerng and vrtual realty. The technology of camera calbraton has always been the research hotspot and dffculty. In ths paper, frstly the expermental system of bnocular stereo vson s constructed, and then the stereo vson system s calbrated respectvely by usng genetc algorthm and BP neural network. In the method of genetc algorthm, the encodng rule of adaptve adjustment of parameter search nterval s proposed, whch effectvely realze the camera calbraton of the hgh dmenson and non-lnear system of bnocular stereo vson. In the calbraton method of BP neural network, wth the help of CNC precson moble workstaton to obtan the densty calbraton sample data, by means of BP neural network whch smulatng the mappng relaton between the 3D space and 2D mage plane of stereo vson system, the mplct calbraton model of bnocular stereo vson system s constructed, whch avods the system error because of mperfect mathematcal models. The two methods have respectve advantages, and are sutable for dfferent applcaton stuaton. Introducton The poneerng work of stereo vson started n the md 60 s of the last century. Roberts of Amerca MIT extended the past smple 2D mage analyss to the complex 3D scene by extractng 3D structure of smple and regular polyhedron, such as cube, wedges and prsms n dgtal mages, and by descrbng the shapes of objects and spatal relatonshps, whch marks the brth of stereo vson technology. Wth the development of research, the scope of study was enlargng from the feature extracton of edges and corners, and the analyss of geometrcal elements such as lnes, planes, surfaces, drectly to the analyss of mage shadng, mage texture, and movement and magng geometry, and varous data structure and rules of nference were establshed. Especally n the early 80 of the last century, Marr [1] generalzed the research results n dscplnes, such as mage processng, mental physcs, neuro-physology, and clncal psychatry, from the angle of nformaton processng for the frst tme, and then created the theoretcal framework of vsual computng. The fundamental theory greatly promoted the development of stereo vson technology, so the complete system from mage acquston to vsual surface reconstructon of 3D scene was formed, whch made the stereo vson to be an mportant branch of the computer vson. After decades of development, the applcaton of stereo vson n the felds of robot vson, aeral mappng, reverse engneerng, mltary, medcal magng and ndustral nspecton become more wdely [2,3]. Stereo vson smulates the mechansm of human vson usng bnocular cue to perceve the around scene, and realzes understandng and recognton of 3D nformaton by sensng equpment. In the practcal applcaton, stereo vson always take mages of the same object from dfferent vewponts usng two cameras, and then reconstructs the 3D shape of space objects from the two mages accordng to trangulaton method. In the system of bnocular stereo vson, the geometry layouts between two cameras wll drectly affect the common vson of the two cameras and the search scope of mage matchng. Therefore, t s needed to perform analyss and selecton of the system structure, and then the hardware system can be constructed. The research of the problem of camera calbraton of stereo vson has always been the focus and dffculty n the feld of stereo vson. In the practcal use, because of the exstence of varous dstorton of camera lens n the system of stereo vson and varous error of devce nstallaton, the The authors - Publshed by Atlants Press 896

2 deal lnear model can not accurately descrbe the relatonshp of camera s magng geometry. The exstence of varous non-lnear factors makes the relatonshp between objects and mages complex, and can t even use a precse mathematcal model to express the relatonshp, the only way to do that s to adopt a smplfed approxmaton of the actual model on the premse of satsfyng the precson requrement. There are two methods to solve the problem of camera calbraton of stereo vson system, the one s based on non-lnear mathematcal model, whch uses mathematcal analyss equaton to approxmately express the non-lnear geometry mappng relatonshp between the 3D geometry poston of space object surface ponts and correspondng mage ponts on the left and rght mage planes, and the camera calbraton of stereo vson system s to solve all external and nternal parameters whch make up the mathematcal model. The other one s based on artfcal neural network, whch has no need to establsh the camera mathematcal model of stereo vson system, and t constructs the mappng relatonshp between 2D mage coordnates and 3D world coordnates by usng non-lnear fttng capablty of artfcal neural network. Constructon of Bnocular Stereo Vson System There are three optonal structures of bnocular stereo vson system,.e. parallel optcal axs structure, common optcal axs structure and ntersectng axs structure. In the stereo vson system of parallel optcal axs structure [4], the focal length and other nternal parameters of two cameras are all equal wth the optcal axs perpendcular to the camera s magng plane, the horzontal axes of two cameras to be overlap, and the vertcal axes to be parallel to each other, so the left camera can be concdent wth the rght camera by movng the left camera along the horzontal axs for a certan dstance. The stereo vson system of ths structure form has very convenent condtons when the matchng relatonshp of mages s beng establshed, however t s very dffcult to make the two axes keep parallel, because the axes can t be observed n the process of nstallng the cameras. In addton, n order to make two cameras observe of parallel optcal axs at the same tme the vsble surface of the object to be measured, the two cameras must be nstalled very closely, but dong so wll take the dsparty between the projectons on the left and rght magng planes to be smaller, whch wll affect the precson of measurement. In the stereo vson system of common optcal axs structure [5], the optcal axes of two cameras of front and back layout are concded. The eppolar plane composed of the space object pont and two optcal center of the camera ntersect wth the magng planes of front and back layout, and the ntersecton lnes are conjugate lne, whch go through ther respectve center of the magng plane and have the same slope. Obvously, the conjugate lnes through mage center and wth same slope provded convenent condtons for matchng between the two mages. However, as s the case wth stereo vson system of parallel optcal axs structure, t s also very hard to make the optcal axs of the two cameras to be collnear. Meanwhle, n order to observe smultaneously the same part of the scene, the dstance between the front and back cameras must be large enough, but, f the baselne dstance s too large, then the occluson on two mages wll appear, t s to say that the some surface area of the measured object wll appear on the one mage, and not on the other mage, or wll not appear on the two mages. Even more so, when the curvature radus of the measured 3D surface s larger and the surface change s dramatc. The ntersectng optcal axs structure s shown as Fg.1. In ths structure form, the optcal axes z l, z r of left and rght cameras was arranged at an angle. The cameras of ths structure form were nstalled easly, and the dstance and slope drecton of two cameras could be adjusted flexbly accordng to the characterstcs of the measured object and system requrements, whch could avod from the structure the stuatons of small feld of vew and mage occluson exstng n the above two forms. But, t can be observed from the fgure that the dstrbuton of conjugate eppolar lnes p l, p r s not conducve to the matchng of left and rght mages. 897

3 A z l z r p l a l C l C r a r p r O l y l x l b y r O r x r Fg.1 Intersectng optcal axs structure Consderng comprehensvely the advantages and dsadvantages of three knds of structures, the bnocular stereo vson system set up n the ntersectng optcal axs structure both can be easly nstalled, adjusted, and can realze smoothly mage matchng under the exstng condtons, whch s a common structure form n the practcal applcaton. System Calbraton Based on Genetc Algorthm Constructng the model of bnocular stereo vson system, f consderng the nfluence of radal dstorton factor, the calbraton of double camera s a nonlnear complex functon optmzaton problem. The tradtonal optmzaton methods to solve such problems nclude Newton method, gradent descent method and conjugate gradent method, etc. But, these methods need to set the approprate ntal values, and f the ntal value s not set properly, these methods can lead to poor convergence of optmzaton process or easy to fall nto local extremum. Especally for the large scale nonlnear optmzaton problems, the dffculty of fndng the global best soluton wll ncrease wth the ncrease of local extremum. Genetc algorthm [6] s an adaptve searchng probablstc method for global optmzaton, whch s developed based on natural selecton and genetc evoluton mechansm. It uses group search technology to make the communty evolve gradually to the state of ncludng or nearng the optmal soluton by mposng a serous of operatons on the current group, such as optons, overlappng and mutaton, and generatng new group. The applcaton of genetc algorthm on the camera calbraton has made great achevements, but t manly focused on sngle camera parameter calbraton. Qang [7] adopted genetc algorthm to calbrate camera parameters, and got good performance n terms of convergence, accuracy and robustness. But, snce the model of camera ddn t nclude dstorton factor of lens, the precson was dffcult to ensure for 3D measurement. In addton, Roberts [8] ponted out that the standard genetc algorthm mght not meet the precson requrements for the hgh-dmensonal optmzaton space wth number of parameters greater than 10. Meanwhle, hs research results ndcated that the performance of genetc algorthm would be degraded sharply wth the ncreasng of the dmensons of optmzaton space. The genetc algorthm appled n the camera calbraton of stereo vson system should be mproved to meet the varous demands. The adaptve varable nterval adjustment functon was added to the standard genetc algorthm by mprovng the encodng scheme. The mproved genetc algorthm can satsfy smultaneously the requrements of search space and encodng precson wth the encoded length of the chromosome keepng unchanged, and eventually realze the parameters calbraton of the hgh-dmenson and nonlnear system of bnocular vson wth the mproved genetc algorthm. The adaptve adjustment method of varable search nterval s acheved by movng through the 898

4 double nterval. There are two ntervals correspondng to the varable, the large nterval mn max [ A, A ] represents the range of the possble varable value, and ts boundary s fxed, the + boundary value s determned by ntal estmaton. The small nterval [ a, a ] can move wthn the large ntervals, and the nterval sze remans unchanged n the process of movng. When encodng varable, small nterval s used so as to ensure the accuracy of codng. When searchng the optmal soluton, the search wll cover the entre varable value range by makng the small ntervals movng wthn the large ntervals. The calbraton plate wth black and whte pattern was taken mages by left and rght CCD camera. The total 126 test ponts were obtaned. After mage processng and actual measurement, correspondng 126 groups data were ready, and every group data ncluded left and rght mage coordnates (u l, v l ) and (u r, v r ), and 3D world coordnates (X w, Y w, Z w ). The all test ponts data were dvded nto two groups of calbraton and testng ponts, whch 24 ponts were used as calbraton calculaton and the other 102 ponts used to test the 3D measurement precson after calbraton of stereo vson system. Takng the mage coordnates (u l, v l ) and (u r, v r ) of testng ponts to be the nput of the calbrated measurement system of bnocular stereo vson, the 3D space coordnates (X w, Y w, Z w ) can be obtaned from 3D reconstructon. Then after comparson of the 3D space coordnates (X w, Y w, Z w ) and the true 3D coordnates (X w, Y w, Z w ) of testng ponts, the average error of measurement of the stereo vson system was shown as n Table 1, whch was compared wth the measurement results of 24 calbraton ponts. Table 1 comparson of average measurement error X w drecton(mm) Yw drecton(m Z w drecton(mm) m) Calbraton pont Testng pont The expermental results ndcates that although the soluton space of the nonlnear complex and hgh dmenson optmzaton problem of the bnocular stereo vson system s very large, the mproved genetc algorthm have good convergence n the near-optmal soluton. Calbraton Based on BP Neural Network For stereo vson system, the ultmate goal of camera calbraton s to construct the mappng relatonshp between the 2D coordnates of object ponts and the 3D world coordnates, whch make the 3D world coordnates of object ponts to be drectly derved from the 2D coordnates on the left and rght mage planes. So there s not need to solve the every physcal parameter of camera, and the physcal parameters of the camera can be merged nto some ntermedate parameter to be calculated, whch make the process of calculaton smplfed and mprove the precson of camera calbraton to some extent. Ths method s called mplct calbraton. Correspondng to ths method, the way to construct mathematcal model of camera and accurately solve the nternal and external physcal parameters of the camera called explct calbraton of camera [9]. The nternal mechansm of the Artfcal Neural Network (ANN) s strkngly smlar wth the process of mplct calbraton of a camera, and the both methods construct system model by some known data and then calculate unknown data. The method of camera calbraton by means of ANN has no need to construct mathematcal model of stereo vson system, whch can reduce errors caused by mperfect mathematcal model, and can avod the defects of explct calbraton. Dong so can mprove the measurement precson. The ablty of extenson or generalzaton of ANN s an mportant ndex to measure the performance of neural network. A neural network of good generalzaton ablty not only has hgher matchng effects for the tranng sample set, but also wll smulate the approxmate output of a new 899

5 sample nput vector relatve to destnaton vector. The standard BP algorthm s easy to get nto trouble of the local mnmum, slow convergence and excessve tranng, whch not only affects tranng results and extends the convergence tme, but also reduces ablty to promote. The regularzaton method can mprove the generalzaton ablty by amendments to the tranng performance functon of the standard BP neural network. Generally, the performance functon of BP algorthm adopts Mean Square Error functon mse. Assumng that there are N tranng sample [X, Y ] (=1, 2,, N), and the network weghts and ntal value of thresholds s arbtrary, when there s nput Mean Square Error of all sample s as follows. N 1 2 mse= ( Y Y) N = 1 X, the output Y s obtaned, so the (1) In the method of regularzaton, the performance functon s amended as follows. E= γ mse+ (1 γ) msw (2) Where, γ represents scale factor and msw represents average sum of squares of all the network weghts value. 1 M 2 wj M j = 1 msw= (3) The method of regularzaton adaptvely adjusts the value of γ n the process of network tranng to make sure that the network has less weght n the case of network tranng error E as small as possble, whch s equvalent to reducng the sze of the network automatcally to reduce the opportunty of overtranng, and obtan the objectve to mprove network generalzaton ablty. When acqurng mages of calbraton plate, postons of calbraton are total 21 and calbraton ponts of every calbraton poston are 216, so all calbraton ponts are The true 3D coordnates of each calbraton pont are determned by the poston of calbraton plate and the poston on the calbraton plate. The mage coordnates of every calbraton pont on the left and rght mage planes are obtaned by means of corner extracton procedure, as shown n Fg.2. Fg.2 corner extracton The data sample s composed of the left and rght mage coordnates (u l, v l, u r, v r ) and the 3D world coordnates (X w, Y w, Z w ) correspondng to every calbraton pont. The total 4536 samples s dvded nto tranng samples 800 and testng samples The tranng results of BP neural network are shown as n Table 2. Table 2 tranng error of neural network Average error(mm) Max error(mm) X Y Z X Y Z

6 The testng errors are obtaned by smulatng the traned BP neural network usng testng samples, shown as n Table 3. Table 3 testng errors of neural network Average error(mm) Max error(mm) X Y Z X Y Z The expermental results ndcate that the precson of tranng and testng of BP neural network are almost equal, whch means the BP neural network has good generalzaton ablty n the effectve calbraton 3D space. Concluson The genetc algorthm used n the calbraton of double camera can solve respectvely the nternal and external parameters of each camera. Because of the adopton of adaptve encodng method, the ablty to solve the problem of nonlnear and complex functon optmzaton got strengthened. The Artfcal Neural Network can realze hgh approxmaton of the complex nonlnear mappng, whch provdes a effectve method to construct camera model of stereo vson system. The method doesn t need to consder the effects of lens dstorton and envronmental factors, so t can reduce errors caused by mperfect of mathematcal model and mprove the measurement precson. References [1] Marr D.Vson.W.H.Freeman and Company, San Francsco. Chnese verson: computatonal theory of vson. G.Z. Yao,L.Lu and Y.J.Wang,Scence Press. Bejng, Chna, 1988, pp. 6-37(In Chnese). [2] S.Y. You and G.Y. Xu: Journal of Image and Graphcs. Vol. 2, No. 1, (1997), pp. 17~24(In Chnese). [3] Y.D. Ja: Machne Vson. Scence Press. Bejng, Chna, 2000, pp. 1-25(In Chnese). [4] S.D. Ma and Z.Y. Zhang: Computer Vson-Fundamentals of Computatonal Theory and Algorthms. Scence Press. Bejng, Chna, 1998, pp (In Chnese). [5] M. Kong and S.M. Wang: Journal of Metrology. Vol. 25, No. 4, (2004), pp. 294~297(In Chnese). [6] M. Zhou and S.D Sun: Prncple and Applcaton of Genetc Algorthm. Natonal Defense Industry Press. Bejng, Chna, 2002, pp. 4-64(In Chnese). [7] J. Qang and Y.M. Zhang: IEEE Transactons on Systems, Man, and Cybernetcs-Part A: Systems and Humans. Vol. 31, No. 2, (2001), pp. 120~130. [8] Roberts M and Naftel A J: Genetc Algorthms n Image Processng and Vson. IEE Colloquum on, 1994: 12/1~12/5 [9] G.Q. We and S.D. Ma: IEEE Transactons on Pattern Analyss and Machne Intellgence. Vol. 16, No. 5, (1994), pp. 469~