OPTIMAL CALIBRATION OF A PRISM-BASED VIDEOENDOSCOPIC SYSTEM FOR PRECISE 3D MEASUREMENTS A.V. Gorevoy 1, 2, 3, A.S.

Transcription

1 OPIMAL CALIBRAION OF A PRISM-BASED VIDEOENDOSCOPIC SYSEM FOR PRECISE 3D MEASUREMENS A.V. Gorevoy A.S. Machkhn Scentfc and echnologcal Center of Unque Instrumentaton Russan Academy of Scences Mosco Russa 2 Bauman Mosco State echncal Unversty Mosco Russa 3 Mosco Poer Engneerng Unversty Mosco Russa Abstract Modern vdeoendoscopes are capable of performng precse three-dmensonal (3D) measurements of hard-to-reach elements. An attachable prsm-based stereo adapter allos one to regster mages from to dfferent veponts usng a sngle sensor and apply stereoscopc methods. he key condton for achevng hgh measurement accuracy s the optmal choce of a mathematcal model for calbraton and 3D reconstructon procedures. In ths paper the conventonal pnhole camera models th polynomal dstorton approxmaton ere analyzed and compared to the ray tracng model based on the vector form of Snell s la. We frst conducted a seres of experments usng an ndustral vdeoendoscope and utlzed the crtera based on the measurement error of a segment length to evaluate the mathematcal models consdered. he expermental results confrmed a theoretcal concluson that the ray tracng model outperforms the pnhole models n a de range of orkng dstances. he results may be useful for the development of ne stereoscopc measurement tools and algorthms for remote vsual nspecton n ndustral and medcal applcatons. Keyords: remote vsual nspecton 3D spatal measurements camera calbraton dstorton model prsm dstorton. Ctaton: Gorevoy AV Machkhn AS. Optmal calbraton of a prsm-based vdeoendoscopc system for precse 3D measurements. Computer Optcs 2017; 41(4): DOI: / Acknoledgments: he Russan Scence Foundaton (project # ) fnancally supported the ork. he authors thank A. Naumov A. Shurygn and D. Khokhlov for contnuous techncal support. Introducton Measurement technologes are no transformng vdeoendoscopes from pure vsual nspecton tools to precse equpment for characterzaton of hard-to-reach elements. Wth the measurement technologes avalable today remote vsual nspecton s complementng or even replacng other non-destructve testng methods n the rocket and arcraft engneerng ol and gas ndustry and other felds [1 2]. he mostly used approach to vdeoendoscope-based 3D measurements s the utlzaton of a mnature prsm-based optcs and matchng to mages of a surface from to dfferent ponts [3]. hs stereoscopc technque employs a calbraton procedure and processng algorthms to compute a 3D coordnate for every matched pxel pror to the start of the measurement process resultng n a full 3D surface map of the nspected object [4 6]. he data processng ppelne of a 3D measurement endoscopc system th a prsm-based stereo adapter s shon n Fg. 1. Many researchers have proposed ther understandngs and mplementaton methods for prsm-based sngle-lens stereovson systems n the last to decades. Lee and Keon [7] consdered an arbtrary pont n 3D space as to vrtual ponts after the refracton of a prsm and derved 3D reconstructon technque for ths knd of systems. Lm and Xao [8] descrbed that one mage acqured by the system could be dvded nto to halves consdered as captured by to vrtual cameras. hs concept converted the prsm-based stereovson system to the conventonal stereoscopc system usng to cameras hch made possble to use conventonal camera calbraton mage processng and 3D reconstructon methods. Furthermore ths concept as extended from toocular to multocular system [9]. hs approach as also used to mplement eppolar lnes calculaton method [10] to analyze the nfluence of prsm s angle and poston on a common feld of ve for the vrtual cameras [11] and to estmate depth measurement error caused by errors of the prsm and camera parameters estmaton [12]. Hoever they made sgnfcant assumptons about the relatve camera to prsm poston and analyzed refracton n 2D space only. Fg. 1. he data processng ppelne of a 3D measurement endoscopy system Computer Optcs 2017 Vol. 41(4) 535

2 Cu et al. [13] ponted out the lmtatons of the 2D approach and proposed an accurate geometrcal optcs model of the prsm-based stereovson system usng the 3D vector form of Snell s la. It made possble to derve a transformaton matrx hch could express the relatonshp beteen an object pont and ts mage as 3D affne transformaton. hs matrx could be composed th other transformaton matrces descrbng rotaton translaton and projecton n tradtonal projectve geometry to buld eppolar lnes and 3D reconstructon. hey noted that ths matrx s dfferent for each mage pont and the prsm-based stereovson system can not be represented th pure projectve camera models th hgh precson. Wu et al. [14] used smlar approach for the dgtal mage correlaton system. hey mplemented a modfed vrtual pont model based on backard ray tracng from the mage plane to the object space consderng the refracton on prsm faces and adapted t for blateral telecentrc lens later [15]. In all reveed papers t as supposed that the camera parameters could be ether estmated through a standard calbraton procedure [16] or knon from a techncal documentaton and that the poston and the parameters of the prsm ere knon exactly. In contrast Cu et al. [17] proposed a calbraton technque to estmate multocular prsm poston after a separate camera calbraton. hey used transformaton matrces from [13] and made several assumptons about prsm to reduce the number of parameters. In ths paper e ntroduce a classfcaton and analyss of mathematcal models for a prsm-based sngle-lens stereovson system based on a consdered model of an optcal dstorton nduced by a prsm. he frst group [8 12] ncludes pure projectve pnhole models gnorng prsm dstorton. Nevertheless the concept of vrtual camera allos to use any camera model th dstorton to calbrate vrtual cameras as ndependent ones. Follong ths ay Cu et al. [18] approxmated scalng factor for every mage pont n [13] by radal dstorton. Lm and Qan [19] made aylor seres analyss of the equatons for ray refracton and offered the extended polynomal model to undstort acqured mages. hese models can be unfed as the second group contanng pnhole camera models th polynomal dstorton. he thrd group [ ] conssts of the accurate geometrcal optcs models derved from the 3D vector form of Snell s la. Fnally e may refuse global dstorton model and use model-free local dstorton correcton as shon n [20]. Furthermore the same approach can be used n 3D object space to correct the result of 3D measurements drectly or to undstort mages. Genovese et al. [21] used an addtonal complcated equpment and the second camera thout a prsm to measure devatons of 3D coordnates for thousands of control ponts n the orkng volume to obtan correcton tables. Hoever an endoscopc system th a prsm-based stereo adapter has some peculartes. Frst of all the man lens has a very short focal dstance and a der feld of ve the prsm angles are steeper and the prsm s placed close to the lens. hese factors lead to sgnfcantly bgger mage dstortons than shon n the most of reveed papers. he lack of technques allong to assess 3D measurement error and the applcablty of the proposed mathematcal models for dfferent camera and prsm parameters makes t problematc to apply the results for large-scale prsm-based systems to endoscopes. Moreover the stereo adapter contans an addtonal lens. We consder the prsm parameters and poston to be unknon and to be estmated durng a calbraton procedure. Hence e can not apply technques based on knon prsm parameters as ell as ndependent calbraton of camera thout a prsm and estmaton of prsm parameters afterards. he patent [22] descrbng the stereo-measurement borescope ncluded the descrpton of calbraton procedure usng a sngle mage of the calbraton tool manufactured as a flat grdded base th a hgh step. It contans the dstorton correcton equatons generated from the dstorted postons of the grd ponts on the mage but ddn t provde further detals. Another patent [23] offered to apply polynomal dstorton model to the man lens and to the prsm th addtonal lens separately. Hence e can refer t to the second group of our classfcaton. akata et al. [24] developed dstorton correcton method for mages captured by endoscope th to edge prsms usng backard ray tracng smlar to [14]. Unfortunately none of these papers presented any results to estmate the uncertanty of 3D measurements hch can be acheved for the endoscope th stereo adapter. he results of computer smulaton n our prevous ork [25] have shon that the pnhole camera model th polynomal dstorton could not be equal to the ray tracng model based on the 3D vector form of Snell s la for the typcal ndustral endoscope th a prsm-based stereoscopc adapter. he reason s that pnhole model doesn t consder a ray shft and non-homocentrc beams after refracton. Addtonally the best crtera for ndustral endoscope evaluaton s based on the measurement uncertantes of length or area. hese uncertantes vary sgnfcantly dependng on poston and drecton nsde the orkng space [25 26] hch makes a problem to choose an optmal mathematcal model for these systems challengng. hs paper presents theoretcal analyss and expermental evaluaton of the mathematcal models for the ndustral endoscopes th the prsm-based adapter consderng the peculartes of ther optcal systems and the specfcty of the applcaton. We ntroduce the basc descrpton of three models and the calbraton procedure; present the expermental verfcaton of each model effectveness and the comparson of the models. Mathematcal models he general geometrc model of the mage formaton may be rtten n the form p = P E (l ) here l s the vector of 3D lne coordnates n the orld coordnate system (WCS) and p s the vector of 2D coordnates for the correspondng pont on the mage plane of the -th camera = 1...N N number of cameras [4 5 26]. he vector of lne coordnates l ncludes 3D coordnates of the orgn pont c and the drecton vector v : ( ) l = c v. We use E to denote Eucldean mappng 536 Computer Optcs 2017 Vol. 41(4)

3 x = E (x ) = R x +t translatng 3D pont x from the WCS to the coordnate system (CS) of -th camera (R s the rotaton matrx and t s the translaton vector). Hence ( ) ( ) l = E l = c v th c = R c +t v = R v. he mappng P determnes the unque correspondence p = P (l ) beteen the ray l n the -th camera object space and the pont p n the mage plane. We use notaton here to defne a composton of transformatons.e. P E (l ) P (E (l )). he set of transformatons E and P s parametrzed by the vector k of ntrnsc and extrnsc camera parameters hch components are to be determned durng the system calbraton. Accordng to [ ] the prsm-based stereoscopc mager can be consdered as to vrtual cameras and the same notaton can be used. Pnhole camera models. he pnhole camera model s dely used n computer vson and s ell descrbed [ ]. It assumes that all rays I for the -th camera ( 000 ) pass through the central pont.e. = ( ) l v. Hence t s possble to use one pont x for each ray and rte x nstead of I n the equatons for the pnhole models (see Fg. 2). Usng the same approach as [5 27] e consder the mappng P as the combnaton of smple transformatons ( ) ( ) p x x (1) = P E = A D F E here F s the central projecton onto the unt plane x = F x ; D s the 2D dstorton transformaton ( ) ( ) mage coordnates n pxels = A ( ) x = D x and A s the 2D affne transformaton to the p x [4 5]. Fg. 2. A prsm-based stereoscopc mager consdered as to vrtual pnhole cameras One can fnd many possble approxmatons for the lens dstorton based on polynomal [ ] or more complex [5] mathematcal models. he dely used model (ntroduced by Bron [28]) represents dstorton as the combnaton of radal (3rd 5th and 7th orders) and tangental part to consder lens decenterng as follos x x c = ( 1 + k1r + k2r + k3r ) + y y с 2 2 ( 2 ) xy c c 2( r yc) ρ 1 r + xc + ρ 2xy c c x ρ +ρ + 2 y here r = x c + y c x c = x x 0 y c = y y 0 and x y s the dstorton center of the mage hch s ( ) 0 0 usually assumed to concde th the center of projecton; k1 k2 k3 ρ1 ρ 2 are the dstorton parameters. We further refer to ths model as "model 1". hs model requres 10 parameters to descrbe each camera (5 parameters for A and 5 parameters for D ) and 6 parameters for relatve orentaton of to cameras [4 16]. As a result vector k ncludes 26 parameters. We can extend the polynomal dstorton model usng the generc equaton (2) x B 1 B 1 kxmn m n x 0 = x c y c + y m= 0 n= 0 ky mn y (3) 0 here coeffcents kx mn and ky mn are the dstorton parameters (2B 2 totally). We further refer to the extended polynomal model as "model 2". Generally ths model requres 2(5+2B 2 )+6 = 16+4B 2 parameters but the number of the dstorton coeffcents can be reduced sgnfcantly based on theoretcal analyss or computer smulaton of the system typcal parameters [19 25] to smplfy optmzaton procedures on the calbraton stage. In order to allo reconstructon of a ray n 3D object space for each mage pont camera model should be nvertble and should provde the back-projecton transformaton E 1 l = P 1 p. Hence each of the transformatons n Eq. ( ) (1) should have a correspondng nverse one: F 1 D 1 and A 1. It s essental that polynomal models descrbed by Eqs. (2) and (3) do not allo fndng a closed-form soluton for the nverse transformaton D 1 an teratve solver should be used n ths case [5]. hus D 1 notaton actually stands for ths teratve procedure. Ray tracng models. he alternatve model usng a ray tracng from the mage plane to the object space s formulated smlar to [14 24]. We use I k = S k(i k 1) notaton to descrbe the refracton of each ray I k on the k-th surface for the -th mage part. Accordng to the vector form of Snell s la e can derve transformaton S k as 2 2 ( ( ) ) ( ck 1 dk ) sk v = b v + 1 b 1 g b g s b k k k 1 k k k k k c = c v k k 1 k 1 gk n = g = v s k 1 k k k 1 k nk here n k s the refractve ndex beteen the k-th and the (k + 1)-th surfaces s k s the normal to the k-th surface and d k s the 3D coordnates of any pont on the k-th surface (see Fg. 3). In order to fnd the ray coordnates I 2 n the object space for each mage pxel p e should (4) Computer Optcs 2017 Vol. 41(4) 537

4 defne ray coordnates I 0. We consder the pnhole model th dstorton for the man lens and use Eqs. (1) and (2) 1 l = P 1 p. he nverse transformaton to fnd ( ) 0 P depends only on the parameters of the mage sensor and the man lens t s the same for both mage parts. Smlar to Eq. (1) the complex nverse transformaton for the ray tracng model can be represented n the form ( ) ( ) l l p (5) = E 2 = E S2 S1 F D A here Eucldean mappng E s the same for both channels due to our choce of the same CS for the left and the rght parts of the prsm vectors s k and d k are determned n ths CS. Fg. 3. Ray tracng through a prsm We assume that s k s the unt vector and the pont d k s on the z-axs so e can use 3 parameters for each prsm surface. Accordngly e have 9 parameters for 3 surfaces and one more parameter for the refractve ndex. Agan f e let E to be the untary transformaton e need the vector k of 20 parameters to descrbe ths model. In contrast to the pnhole model the ray-tracng model cannot provde closed-form soluton for the forard transformaton p = P E(I ) because t requres ntally unknon drecton vector of the lne I from 3D pont x (shon n Fg. 3). hs problem s usually solved by the teratve technque called ray amng or by look-up-table nterpolaton. We can formally rte the forard transformaton as p = P E(x ) and extend t the same ay as Eq. (5) f e consder that ths notaton stands for the teratve solver. We further refer to the ray tracng model as "model 3". 3D reconstructon problem. he reconstructon of 3D pont coordnates x from N projectons p may be consdered as the ray ntersecton problem. Because of the uncertantes n coordnates p these rays are ske and the trangulaton algorthm s used for the estmaton of ˆx by mnmzng a functonal C.e. ( ) ( ) argmn C( ˆ ) xˆ = pk = x pk (6) xˆ here = ( N ) p p p p. he selecton of the approprate cost functons s not trval n the general case because t depends on a pror data about the poston of target pont x ts projectons p the statstcs of coordnate measurement errors and the propertes of camera transformatons P. If e assume that the devaton of the measured mage coordnates p from ther true values p follos the Gaussan dstrbuton th zero mean and covaraton matrx Σ p then e can use the Maxmum Lkelhood Estmator hch mnmzes the Mahalanobs dstance n the mage plane [4 29] ( xˆ pk) p ( ˆ x) C = P E = N =1 1 ( p p) p ( p p) = ˆ Σ ˆ =1 here p = P E ( x ) N ˆ ˆ s the estmated poston of the 3D pont ˆx projecton on the mage plane of the -th camera and 1 Σ p s the nverse (rank-constraned generalzed nverse) covaraton matrx of the coordnate measurement error for p. We assume that the measurements errors are ndependent for each mage. he cost functon (7) and ts dervatves are calculated at each teraton so t s suffcent to have a closed-form soluton for the forard transformaton. Hence e can use ths cost functon for models 1 and 2. Alternatvely e can formulate the cost functon based on the sum of dstances from the 3D pont ˆx to the rays l for example e can use the md-pont of the common perpendcular for to ske rays [4]. hs alternatve cost functon does not allo to acheve the theoretcal bound for mnmal varance (the Cramer-Rao loer bound) [29] but t s more sutable for the models that can faster calculate the nverse transformaton than the forard one such as model 3 though e can also use t for models 1 and 2. Calbraton problem. he am of the calbraton procedure s to determne the value of parameter vector k usng dfferent types of calbraton targets (such as boards and corners [16] or steps [22]). We consder that e regster mages of the calbraton target th M ponts n R postons; 3D coordnates of the ponts x tj are knon th a certan accuracy n the CS of the calbraton target. Next the mage coordnates p jk are calculated for each pont j = 1...M and each poston k = 1...R. he complete transformaton for these ponts s p ( x ) jk k t j 2 Σ (7) = P E E here E k stands for the Eucldean mappng from the CS of the calbraton target to the WCS. In addton to the parameter vector k for transformatons P and E e also need to fnd the vector k t descrbng transformatons E k for each k. If e ntroduce the composte vector x t of all 3D ponts and the composte vector p of all projectons the calbraton algorthm K may be rtten as kˆ = K x p = argmn C x p kˆ k ˆ. (8) ( ) ( t ) ( t t) kk ˆ ˆ t It s not necessary for each pont at each poston to be projected for each camera some ponts or postons may be skpped. Follong the same reasons as for the trangulaton algorthm above the most approprate choce of the cost functon C s the Mahalanobs dstance n the mage plane. he 538 Computer Optcs 2017 Vol. 41(4)

5 equaton for ths cost functon s smlar to Eq. (7) but e should rte the sum for all values j and k.e. N M R ( xt pkk ˆ ˆ t ) p ˆ ˆ ˆ jk k( xt j) C = P E E = Σ =1 j=1 k=1 (9) = ˆ Σ ˆ. 1 ( p jk p jk ) p jk ( p jk p jk ) jk Agan e can use ths cost functon for models 1 and 2 because they provde the closed-form soluton for the forard transformaton. For the model 3 e should derve the alternatve cost functon based on the sum of dstances from 3D pont x t to rays ˆl t. If e use a flat calbraton target e can replace rays ˆl t jk by ponts x ˆ t jk of ray ntersectons th XOY plane n the CS of the target. In that case the cost functon can be represented as follos here ( ˆ ˆ t t ) ˆ ˆ ˆ x pkk x t j k ( p jk ) C = E E P = 1 x jk jk 1 ( x x t t ) x ( x x j jk jk t j t jk ) = ˆ Σ ˆ. jk 2 Σ 2 (10) Σ s the nverse covaraton matrx of calbraton target coordnate measurement error for x tjk (see [29] for detals about the estmaton of ths matrx). We can also perform calbraton for models 1 and 2 th the cost functon (10) hch leads to slghtly dfferent results than calbraton th (9). Expermental results Equpment. In order to evaluate and compare the consdered mathematcal models e have conducted a seres of experments usng ndustral vdeoendoscope Mentor Vsual Q Vdeoprobe (GE Inspecton echnologes) th forardve stereo adapter. he probe has a dameter 6.1 mm and a 1/6" CCD mage sensor th pxels. A prsm-based stereo adapter provdes to regstraton channels th the felds of ve 55 /55. o acqure mages for calbraton and tests e utlzed to types of plane calbraton targets: (1) th black crcular dots on a hte background and (2) th black and hte chessboard pattern. Due to suffcent range of dstances and magnfcatons three samples of each calbraton target type th 0.5 mm (small-szed) 1 mm (mddle-szed) and 2 mm (large-szed) dstance beteen markers have been manufactured. hs dstance corresponded to the dstance beteen the centers of the dots and to the chessboard square sze for to types of target respectvely. Each target had a grd of markers. he dstal end of the endoscope as fxed on a mechancal stand that allos adjustable movement along z-axs and rotaton around y and z-axs. he axs of the probe as approxmately concdent th the z-axs of the stand. Another mechancal stand as placed n front of the endoscope to mount calbraton targets. he plane of calbraton target as set approxmately perpendcular to the z-axs of the stand. Image acquston and processng. We have conducted a fe experments usng dfferent types of calbraton targets descrbed above. At frst dot and chessboard targets have been used for ndependent experments. Next e have carred out an experment replacng calbraton target n each poston to obtan equal condtons for to types of targets. For each experment e have captured to seres of mages: calbraton sequence and test sequence. he calbraton sequence ncluded 18 mages captured at dfferent postons of three calbraton targets: smallszed medum-szed and large-szed. Each of these targets has been placed approxmately perpendcular to z- axs of the stand at the dstance hch allos to see at least eght markers n a horzontal ro on both mage halves. hen the dstal end as consequently rotated 30 around x and y axes to capture mages at oblque angles. Fnally e have postoned the target perpendcular to z- axs at approxmately 1.5 larger orkng dstance than ntal poston. As a result e have acqured 6 mages for each sze of the calbraton target coverng the total range of dstances from 8 to 40 mm. est sequences ere regstered for medum-szed and large-szed calbraton targets n the follong ay. Agan the calbraton target has been placed approxmately perpendcular to z-axs of the stand at reasonably close dstance and orented so that horzontal ros of markers ere approxmately horzontal on the mage. We used translaton stage to shft the dstal end of the endoscope along z- axs and captured mages th 1 mm step. hus e have obtaned seres of mages for each calbraton target. hese seres contaned 16 mages n the range from 12 to 27 mm for medum-sze targets and 19 mages n the range from 22 to 40 mm for large-szed ones. he range of dstances as lmted by the condton that the number of vsble markers should not be too small and the condton of mnmal marker scale sutable for robust mage processng. In order to calculate mage coordnates pjk for each marker e have mplemented n MALAB the mage processng algorthms for detecton of dot and chessboard pattern from [30]. All automatcally processed mages of calbraton and test sequences have been checked manually afterards to delete ponts n lo-llumnated areas around glare etc. Captured mages of to calbraton targets th estmated marker postons are shon n Fg. 4. hese mages correspond to the calbraton seres. Calbraton. We have mplemented the calbraton algorthms for each earler consdered mathematcal model usng non-lnear teratve solver from MALAB for the constraned mnmzaton problem formulated n Eq. (8). he constrants for the values of calbraton parameters k are determned by the physcal lmtatons of prsm parameters: sze (less than 5 mm) refracton ndex (from 1.4 to 1.8) etc. We used the cost functon C based on dstance n the mage plane for models 1 (usual pnhole model) and 2 (pnhole model th extended polynomal dstorton) accordng to Eq. (9). he covaraton matrx Σ p jk as assumed to be the dentty matrx due to the lack of a prory nformaton about the error dstrbuton of the mage coordnates p jk. As mentoned above model 3 (ray tracng model) does not provde closed-form soluton for the forard transformaton hch makes ths cost functon napproprate. Hence e used another cost functon based on the dstance n the XOY plane of the CS of the calbraton target as shon n Eq. (10). Computer Optcs 2017 Vol. 41(4) 539

6 Fg. 4. Captured mages th estmated marker postons for to types of calbraton target For smplcty the covaraton matrx Σ x jk as also consdered as the dentty matrx but ths s suffcently less reasonable assumpton than the same one for Σ p jk accordng to [4 29]. In order to assess the mpact of the calbraton algorthm on the measurement accuracy e Camera 1 Camera 2 able 1. Calbraton parameters have calbrated model 1 usng the cost functon (10). We further refer to ths hybrd model as model 1 (usual pnhole model calbrated n 3D space). We have processed four calbraton sequences and have calculated four vectors of parameters for each of the consdered mathematcal models. o of these sequences have been captured durng the frst experment. he frst sequence ncluded mages of the dot calbraton target and the second one ncluded mages of the chessboard target. hese sequences ere acqured ndependently and the postons of calbraton target ere dfferent. he next to sequences have been captured replacng the calbraton target n each poston durng the second experment. hus the planes of the targets ere approxmately the same for each poston. We have compared the calbraton results for each model calculated th dfferent targets to assess the mpact of ths factor. Our analyss has shon that the dfference of parameters calculated for to sequences of the second experment s close to the dfference of parameters calculated for to seres of the same calbraton target. he calbraton results for the chessboard target from the frst experment are lsted n able 1. he affne transformaton A s descrbed by the focal length f x f y and the mage center c x c y [4 16] the ske parameter s consdered to be zero. he parameters of dstorton transformaton D for model 1 1 and 3 correspond to Eq. (2). Model 2 uses 17 parameters for the dstorton transformaton: kx 14 kx 04 kx 32 kx 12 kx 02 kx 11 kx 50 kx 30 kx 20 ky 05 ky 23 ky 03 ky 02 ky 41 ky 21 ky 11 and ky 20 accordng to Eq. (3). he prsm parameters estmated for model 3 are as follos: Back face: s 11 x = s 11 y = d 11 z = 0.014; Front face 1: s 21 x = s 21 y = d 21 z = 3.401; Front face 2: s 22 x = s 22 y = d 22 y = (all measured n mm) and the ndex of refracton n 1 = (see Fg. 3 for notaton). Parameters Unts Model 1 Model 1 Model 2 Model 3 focal length fx fy pxel ; ; ; ; mage center cx cy pxel 40.28; ; ; ; dstorton k1 k2 0.55; ; parameters 0.567; dstorton ρ1 ρ ; ; for dstorton focal length fx fy pxel ; ; ; mage center cx equal to cy pxel 748.7; ; ; the parameters of dstorton k1 k2 0.51; ; parameters camera 1 dstorton ρ1 ρ ; ; for dstorton Rotaton ω1 ω2 ω3 radans 0.012; 0.398; ; ; ; 0.331; ranslaton t1 t2 t3 mm 1.264; 0.025; ; 0.058; ; 0.021; Error analyss. he mages of the test sequences have been used to calculate the 3D coordnates for each marker accordng to Eq. (6). he algorthm have been mplemented n MALAB smlar to the calbraton algorthm. he same ay e used the cost functon (7) based on the dstance n the mage plane for models 1 and 2 th the dentty covaraton matrx. he cost functon based on the sum of the dstances from the 3D pont to the back-projected rays as used for models 1 and 3. We have consdered all covaraton matrces equal to the dentty ones so that all coordnates ponts and rays had equal eghts n the cost functons. In most applcatons of 3D measurement endoscopc systems the calculated 3D pont coordnates are used to measure the geometrc parameters such as dstances or areas. Hence e should utlze the crtera based on the devatons of the length or the area to evaluate the mathematcal models for these systems. We have processed 3D ponts calculated for each mage of the test sequences to estmate the dstance beteen neghbourng markers. Due to the chosen orentaton of 540 Computer Optcs 2017 Vol. 41(4)

7 the calbraton targets these segments are approxmately parallel to x- and y-axs of the stand. he orentaton of the CS s the same as shon n Fg. 3. he true value of the dstance s assumed to be 1 mm for the medum-szed target and 2 mm for the large-szed one. Next e have processed 3D ponts for pars of consequently captured mages to estmate the dstance beteen ponts correspondng to the same marker. he obtaned segment s approxmately parallel to z-axs and ts true length s assumed to be 1 mm (equal to the shft step for the dstal end of the endoscope). Addtonally the poston of the calbraton target has been estmated for each mage relyng on the calculated 3D pont coordnates. We have estmated the dstance measurement errors for all captured test sequences and all consdered mathematcal models usng correspondng vectors of parameters from the calbraton stage. Snce the results for dfferent types of calbraton targets are almost concdent e present the results for the chessboard target from the frst experment only. he absolute errors r are shon n Fg. 5 for the medum-szed target. Fg. 5. Absolute error of dstance measurements for medum-szed chessboard calbraton target. he true value of each segment s 1 mm all values are n mm. he value of error s ndcated by dots color accordng to the grayscale bar n the rght sde of the fgure. Each ro of plots represents one of four consdered mathematcal models each column corresponds to segments parallel to x- y and z-axs shon n the bottom part respectvely Computer Optcs 2017 Vol. 41(4) 541

8 he error value s ndcated by dots color accordng to the grayscale bar n the rght sde of the fgure. he dots ndcatng segments th error value larger than 0.2 mm are flled th the black color ntended for 0.2 mm values. he coordnates of the dots correspond to the estmated postons of the calbraton target for a vsual clarty the coordnates of the frst pont of segment are shon. Each ro of the plots represents one of four consdered mathematcal models each column corresponds to the segments parallel to x- y- and z-axs respectvely. he CSs for the plots are slghtly dfferent. he orgn for model 3 s determned by the center of the nternal pnhole camera as shon n Fg. 3. he mddle pont beteen to camera centers s chosen as the orgn for models 1 1 and 2 (see Fg. 2). One can see that the error value vares sgnfcantly for dfferent parts of the observed volume and the dfferent orentaton of the segment. hus the choce of the ntegral crteron for the quanttatve evaluaton representng the values dstrbuton s not clear. Accordng to common applcatons of 3D measurement endoscopc systems e have dvded the obtaned data set nto zones by z-coordnate and calculated mean and standard devaton of the segment length for every zone. he number of zones s equal to the number of mages n the test sequence hence each captured mage belongs to sngle zone. he results are shon n Fg. 6. he top ro of the plots represents test sequence for large-szed calbraton target the true value of x- and y-segments s 2 mm and the true value of z-segments s 1 mm. he bottom ro of plots represents test sequence for medum-szed target the true value of all segments s 1 mm. he results presented n Fgs. 5 and 6 ndcate that the error for x-segments gros sgnfcantly on the edges of the feld of ve for models 1 1 and 2. In contrast all models demonstrate relatvely smlar performance for y- segments. Furthermore the error for x- and y-segments vares for each zone hch leads to a large standard devaton due to a chosen evaluaton method. In contrast the standard devaton for z-segments s comparable for all models but the usage of models 1 1 and 2 leads to unacceptable bas hch ncreases th dstance. he correlaton of spkes on the plots for the mean error of the length for z-segments reveals uncertantes n shftng of the dstal end durng the experment. Model 1 demonstrates slghtly better results than model 1 for x-segments but the overall performance s relatvely smlar. Hence the choce of the cost functon has no great mpact on the accepted evaluaton of models. he man concluson based on the expermental results s that model 3 has a superb performance n comparson to other models. Model 2 may be used for the orkng dstances less than 20 mm but ts usage s lmted due to bas n measurements of length along z-axs at larger dstances. Fg. 6. Mean and standard devaton of dstance: large-szed calbraton target (top ro) and medum-szed target (bottom ro). Each column corresponds to segments parallel to x- y- and z-axs respectvely. he true value of x- and y-segments s 2 mm for the large-szed target and 1 mm for the medum-szed one. he true value of z-segments s 1 mm 542 Computer Optcs 2017 Vol. 41(4)

9 Concluson In ths paper e frst theoretcally and expermentally analyzed and compared three dfferent mathematcal models of prsm-based stereoscopc magers. he expermental data confrmed the results prevously obtaned by computer smulaton: any of consdered pnhole models can not provde the requred accuracy of the measurement over a de range of orkng dstances. he model based on ray tracng through the prsm shos much better results. An ncrease n the number of degrees for dstorton polynomal model does not help to mprove the accuracy apparently because of the assumpton that all rays pass through the sngle pont. he error of the length segment measurement has a complex spatal dstrbuton and depends strongly on the orentaton of the segment. hs fact should be taken nto account for evaluaton of the vdeoendoscopes and formulatng the crtera for ther measurement accuracy testng. he dstrbuton can be used for the nteractve user assstance by ndcatng the estmated measurement errors or gvng recommendatons on the locaton of the probe relatve to the object. We have shon that the smple calbraton method usng a flat target allos to conduct the calbraton of the endoscope th attached prsm-based stereo adapter and reach the desred accuracy. he method does not requre the results of other specfc calbratons or the prsm parameters. ype of the calbraton target pattern selected for the calbraton and test measurements has an nsuffcent effect on the result. he obtaned theoretcal and expermental results may be useful for the mprovement of the exstng prsm-based vdeoendoscopc magers n terms of the measurement range and accuracy as ell as for the development of the ne small-sze tools for remote vsual nspecton especally for the mportant ndustral tasks here hgh precson of defect characterzaton s crucal. References [1] Mx PE. Introducton to nondestructve testng: A tranng gude. 2 nd ed. Ne York: John Wley & Sons; ISBN: [2] Zhu Y-K an G-Y Lu R-S Zhang H. A reve of optcal ND technologes. Sensors (Basel) 2011; 11(8): DOI: /s [3] Olympus Corp. Demonstraton of stereo measurement. Source: [4] Hartley RI Zsserman A. Multple ve geometry n computer vson. 2 nd ed. Cambrdge: Cambrdge Unversty Press; ISBN: [5] Kannala J Brandt SS. A generc camera model and calbraton method for conventonal de-angle and fsh-eye lenses. IEEE rans Pattern Analyss and Machne Intellgence 2006; 28(8): DOI: /PAMI [6] Zhang S ed. Handbook of 3D machne vson: Optcal metrology and magng. Boca Raton Fl: CRC Press; ISBN [7] Lee D Keon I. A novel stereo camera system by a bprsm. IEEE ransactons on Robotcs and Automaton 2000; 16(5): DOI: / [8] Xao Y Lm KB. Vrtual stereovson system: Ne understandng on sngle-lens stereovson usng a bprsm. Journal of Electronc Imagng 2005; 14(4): DOI: / [9] Xao Y Lm KB. A prsm-based sngle-lens stereovson system: From trnocular to mult-ocular. Image and Vson Computng 2007; 25(11): DOI: /j.mavs [10] Wang D Lm KB Kee WL. Geometrcal approach for rectfcaton of sngle-lens stereovson system th a trprsm. Machne Vson and Applcatons 2013; 24(4): DOI: /s [11] Kee WL Lm KB un ZL Yadng B. Ne understandng on the effects of angle and poston of bprsm on snglelens bprsm stereovson system. J Electron Imagng 2014; 23(3): DOI: /1.JEI [12] Kee WL Ba Y Lm KB. Parameter error analyss of snglelens prsm-based stereovson system. J Opt Soc Am A 2015; 32(3): DOI: /JOSAA [13] Cu X Lm KB Guo Q Wang D. Accurate geometrcal optc model for sngle-lens stereovson system usng a prsm. J Opt Soc Am A 2012; 29(9): DOI: /JOSAA [14] Wu L Zhu J Xe H. A modfed vrtual pont model of the 3D dc technque usng a sngle camera and a b-prsm. Measurement Scence and echnology 2014; 25: DOI: / /25/11/ [15] Wu L Zhu J Xe H. Sngle-lens 3D dgtal mage correlato system based on a blateral telecentrc lens and a bprsm: valdaton and applcaton. Appl Opt 2015; 54(26): DOI: /AO [16] Zhang Z. Flexble camera calbraton by veng a plane from unknon orentatons. Proc Internatonal Conference on Computer Vson 1999; [17] Cu X Lm KB Zhao Y Kee WL. Sngle-lens stereovson system usng a prsm: poston estmaton of a mult-ocular prsm. J Opt Soc Am A 2014; 31(5): DOI: /JOSAA [18] Cu X Zhao Y Lm K Wu. Perspectve projecton model for prsm-based stereovson. Opt Express 2015; 23(21): DOI: /OE [19] Lm KB Qan B. Bprsm dstorton modelng and calbraton for a sngle-lens stereovson system. J Opt Soc Am A 2016; 33(11): DOI: /JOSAA [20] Qan B Lm KB. Image dstorton correcton for snglelens stereo vson system employng a bprsm. Journal of Electronc Imagng 2016; 25(4): DOI: /1.JEI [21] Genovese K Casaletto L Rayas J Flores V Martnez A. Stereo-dgtal mage correlaton (DIC) measurements th a sngle camera usng a bprsm. Optcs and Lasers n Engneerng 2013; 51(3): DOI: /j.optlaseng [22] Bendall C La R Salvat J Chlek. Stereo-measurement borescope th 3-D veng. US Patent ; publshed on Jule [23] Nakano S Hor F Ogaa K. Endoscope devce and endoscop mage dstorton correcton method. US Patent ; publshed of March [24] akata Y orgoe Kobayash E Sakuma I. Dstorton correcton of edge prsm 3D endoscopc mages. Proc 7th Asan-Pacfc Conference on Medcal and Bologcal Engneerng (APCMBE 2008) 2008: DOI: / _185. [25] Machkhn AS Gorevoy AV. Calbraton of mnature prsm-based stereoscopc magers for precse spatal meas- Computer Optcs 2017 Vol. 41(4) 543

10 urements. Proc SPIE 2015; 9917: DOI: / [26] Machkhn AS Gorevoy AV Perflov AM. Accuracy evaluaton for the non-contact area defect measurement at the complex shape-surfaces under vdeoendoscopc control [In Russan]. Scentfc and echncal Journal of Informaton echnologes Mechancs and Optcs 2014; 14: [27] Wu C Jaramaz B Narasmhan S. A full geometrc and photometrc calbraton method for oblque-veng endoscope. Computer Aded Surgery: Offcal Journal of the Internatonal Socety For Computer Aded Surgery 2010; 15(1-3): DOI: / [28] Bron DC. Decenterng dstorton of lenses. Photometrc Engneerng 1966; 32: [29] Kanatan K. Statstcal optmzaton for geometrc computaton: heory and practce. Mneola: Dover Publcatons; [30] Mallon J Whelan PF. Whch pattern? Basng aspects of planar calbraton patterns and detecton methods. Pattern Recogn Lett 2007; 28(8): DOI: /j.patrec Authors nformaton Alexey Vladmrovch Gorevoy (b. 1987) graduated from Bauman Mosco echncal Unversty n 2010 Laser and Optc-Electronc Systems subdepartment. Currently he orks as the researcher at the Scentfc and echnologcal Center of Unque Instrumentaton оf RAS. Research nterests nclude mage processng optcal system desgn and computer graphcs. E-mal: gorevoy.a@gmal.com. Alexander Sergeevch Machkhn (b. 1984) graduated from Bauman Mosco echncal Unversty n 2006 Development of Optc-Electronc Systems for Scentfc Research subdepartment. PhD. Currently he orks as the leadng researcher at the Scentfc and echnologcal Center of Unque Instrumentaton оf RAS. Research nterests nclude machne vson spectral magng acousto-optcs and spectroscopy. E-mal: aalexanderr@mal.ru. Code of State Categores Scentfc and echncal Informaton (n Russan GRNI): Receved May he fnal verson July p 544 Computer Optcs 2017 Vol. 41(4)