Calibration based on part set-up measurement for on-machine inspection using vision

Transcription

1 Calibration based on part set-p measrement for on-machine inspection sing ision Lorène Dbreil, Yann Qinsat, Claire Lartige To cite this ersion: Lorène Dbreil, Yann Qinsat, Claire Lartige. Calibration based on part set-p measrement for on-machine inspection sing ision. International Jornal on Interactie Design and Manfactring, Springer Verlag, 205, pp.-7. <0.007/s >. <hal > HAL Id: hal Sbmitted on 7 Sep 205 HAL is a mlti-disciplinary open access archie for the deposit and dissemination of scientific research docments, whether they are pblished or not. The docments may come from teaching and research instittions in France or abroad, or from pblic or priate research centers. L archie oerte plridisciplinaire HAL, est destinée a dépôt et à la diffsion de docments scientifiqes de niea recherche, pbliés o non, émanant des établissements d enseignement et de recherche français o étrangers, des laboratoires pblics o priés.

2 Noname manscript No. (will be inserted by the editor) Calibration based on part set-p measrement for on-machine inspection sing ision Lorène Dbreil Yann Qinsat Claire Lartige Receied: date / Accepted: date Abstract Integrating inspection procedres in the machining process contribtes to process optimization. To be highly efficient, inspection operations mst be performed in the same frame as machining operations. Within the context of on-machine inspection sing ision, a method of calibration based on part set-p measrement is proposed in this paper. The measrement frame ths coincides with the CAD/CAM frame in which the CAD model is bilt. The CAD-based calibration is performed by sing 3D featres of the part set-p, which are easily identifiable in the 2D pictre of the scene. The robstness of the method with regard to 2D featre identification in real enironment is assessed thanks to Monte-Carlo simlations. Keywords CAD-based Calibration On-machine inspection Vision Introdction The integration of inspection procedres in the prodction process contribtes to the improement of the manfactring process. Associated benefits inclde high speed of inspection, measrement flexibility, on-machine inspection, and the possibility of 00% inspection []. On-machine inspection (or in-sit inspection) is performed while the part is still located on the machine tool and while the machining process is stopped [2]. On-machine inspection allows to achiee high prodctiity, to redce lead times, and to make a rapid decision regarding the conformity of the prodced part [3]. Frthermore, de to the great inter-operability between machining and inspection, in-sit inspection, either for dimen- C. Dbreil Y. Qinsat C. Lartige LURPA, ENS Cachan, Uni Paris-Sd, F Cachan, France Tel.: Fax: qinsat@lrpa.ens-cachan.fr Prodct before machining Machining Manfactring Corrections In-sit measrement Fig. On-machine inspection sing ision Measrement Prodct KO Prodct OK Prodct after machining sional metrology or for srface roghness, contribtes to process optimization [4], [5], [6]. On-machine inspection is often performed sing ision. Vision-based systems are sed for the erification of machining set-ps [7], [8], the srey of tool trajectories [9], part inspection of socket prodction, and so on []. These systems present arios adantages: low cost, rapidity, flexibility and simplicity of implementation in the context of prodction. A minor drawback is the system calibration. Calibration is necessary to transform a 2D point of the pictre into a 3D point expressed in the reference frame. Within the context of on-machine inspection, the positioning of the pictre frame relatiely to the reference frame mst be performed for each measrement. On the other hand, measrement and machining operations mst be performed in the same frame to facilitate the comparison of the machined part to its CAD model. Therefore, to enhance the interoperability between machining and inspection, it cold be interesting that the reference frame sed for calibration coincides with the CAM frame associated to machining operations. This remoes an additional step of frame registration which cold be penalizing in terms of comptational time and qality. The idea is ths to bild the reference frame from featres belonging to the machine tool scene. In this direction the se of featres belonging to the part set-p seems releant. This paper deals with a calibration method based on part set-

3 2 Lorène Dbreil et al. p measrement for on-machine inspection. In the proposed approach, featres belonging to the part set-p are sed to map the ision calibration frame to the machining frame in which both measrements and machining operations will be performed. The final aim of or stdy is to deelop a tool for manfactring monitoring with the aim of improing the interoperability between CAM, machining and inspection dring the prodction start of a new part. This tool based on ision (more particlarly image correlation) performs onmachine part inspection at the beginning of the machining in order to locate potential defects (Fig. ). It ths gies the ser a rapid and interactie decision tool to act at the earliest to make corrections. The paper is organized as follows. Section 2 is dedicated to the basics of calibration from setp measrements. Applications of or approach are detailed in section 3. In section 4, a sensitiity stdy is proposed to assess the robstness of the method with regard to enironment parameters. 2 Calibration from set-p measrements Before detailing or approach, let s reiew the basic principle of ision-system calibration along with classical methods proposed in the literatre. 2. Calibration principle The calibration aims at identifying the model parameters that define the relationship between the coordinates of a 2D point, expressed in the pictre R p, and the coordinates of a 3D point expressed in the reference frame R w. c c Rc Rp Fig. 2 Pinhole model f c c c Zc c Yc Rr y x Z Rw Y = A.D.P.T. Y Z Rp R w = M.T. Y Z Based on the pinhole model (Fig. 2), the relationship between R p and R w reslts from the combination of the geometric transformations A, D, P, and T as defined in (Eq. ). The matrix A corresponds to the affine transformation from the center pictre frame R r to the pictre frame R p.the transformation P is a perspectie projection of R c, the frame attached to the camera, into the retinal plane R r. The transformation D corresponds to camera distortions. The parameters associated with the matrices A, D, and P correspond to the internal calibration parameters also called the intrinsic parameters: the focal f c, the pictre center position C c and C c, the pixel size s p, the pixel nmber n p, and the distortion parameters k i (for i = to 5). The transformation T between the reference frame R w and the camera frame R c, corresponds to the combination of a rotation R(3 3) and of a translation t. T can be described thanks to a homogeneos matrix, and its parameters are called extrinsic parameters (Eq. 2). c Y c Z c R c = T. Y Z R w R w ( ) R33 t =. Y 0 Z Seeral techniqes of calibration exist. Most of them allow the determination of extrinsic and intrinsic parameters simltaneosly. Starting from a correspondence between 2D to 3D featres, classical techniqes rely on the minimization of a projection error fnction [0][][2]. Seeral methods are based on point correspondence. Fageras and Toscani [] propose a method leading to the identification of the pinhole model parameters withot distortions. As they only consider the jst necessary nmber of points, the problem is linear and easy to sole. Laest et al. [2] extend the approach by considering a non-limited nmber of points. The correspondence problem is ths non-linear, and is soled by minimizing projection errors. This method has been implemented as a Matlab R204 c toolbox by Boget [3]. Doilly [4] propose a calibration method for a mlti-camera system based on a ball-bar artifact. Inspired from the method deeloped by Tsai [5], this approach reqires a high nmber of pictres to be efficient. A second family of methods se line correspondences. The difference lies in the model sed to described the lines: parametric description in [0], or plane intersection in [6]. A few stdies rely on the CAD definition of the stdied scene to perform the calibration. Beabier et al. [7] proposed a CAD-based calibration sing stereo-correlation. Unfortnately, this approach, which R w () (2)

4 Calibration based on part set-p measrement for on-machine inspection sing ision RP T? zcam OP Table Featres identifiable on pictres M 3D featre RCAM O xcam Cylinder Sphere Intersection of 2 planes Circle ycam OC RC 2D featre Extracted element 2 generatrices Circle Line Ellipse Line Point Line Point zc xc yc Fig. 3 Identification of the transformation T takes place dring machining, relies on the part which has geometrical defects. Frthermore, the stereo-correlation reqires a pattern projection which is not appropriate for onmachine inspection de to low contrast, the difficlty to extract more than a few featres, and it may be time consming. 2.2 Proposed approach As the ision system is dedicated to on-machine inspection, the calibration method proposed in this paper relies on identifiable geometric featres of the machine tool scene. As aforementioned, the idea is to define a niqe frame for calibration, ision-based measrements, and machining operations. In this direction, we propose that the reference frame Rw coincides with the frame in which machining operations are defined. Machining operations are generally defined in a frame associated to the CAD model. It ths seems releant to calibrate the system thanks to featres that belong to the machining set-p as they belong to both the CAD model and the machining enironment, and as they are also not modified dring machining. This will make the integration of the measrement in the machining process easier. Dring on-machine ision, intrinsic parameters are fixed and do not ary. Therefore, the transformation M = A.D.P between R p and Rc is completely defined, and the stdy focses on the identification of the extrinsic parameters i.e. on the identification of the elements of the transformation matrix T (Fig. 3). The method we propose aims at establishing the correspondence between a set of 3D featres of set-p CAD model and the corresponding 2D featres in the pictre of the actal machining set-p. Once the correspondence is established, the transformation G between R p and Rw = RCAM is known, and T is simply gien by: T = M.G 3 (3) It is important to nderline that the set-p defects are neglected: the actal machining set-p is assimilated to its CAD model. The idea is ths to se featres that can be easily identifiable as displayed in Tab.. Table describes the correspondence between a 3D geometrical featre, the associated 2D element belonging to the pictre, and the extracted 2D element. The transformation T is characterized by 6 extrinsic parameters, 3 Eler angles (φ, θ, ψ for the rotation R(3 3)) and 3 distances for the translation t. Let s denote Φ = (φ, θ, ψ,tx,ty,tz ) the ector gathering the extrinsic parameters we want to identify. The set of featres mst lead to a minimm of 6 independent eqations to completely define the transformation. The method consists in 3 main steps. An initial ector Φ0 is chosen defining an initial transformation Tinit. The 3D featres of the CAD model are projected onto the pictre frame thanks to eqation (Fig. 4(a)). In parallel, eqialent 2D featres are extracted from the pictre (Fig. 4(b)). To illstrate or prpose, let s consider the case of the point C belonging to the CAD model. C is projected onto the pictre frame R p in C0 by the combination of the transformation Tinit and the internal transformation M. The same goes for the line dcad belonging to the CAD model which is projected in d pict into the pictre frame. In parallel, corresponding elements are extracted from the pictre: the point D and the line ddetect (Fig. 4(b)). It is ths possible to calclate the projection error for each type of featre. For points, the error is defined as the distance in pixels between the extracted point and the projected point as represented in figre 5. For lines, two errors are calclated corresponding to the distances of the most extreme points of the projected line d pict to the extracted line ddetect (Eq. 4). e point = d(c0, D) e = d(a0, ddetect ) axis 2 eaxis = d(b0, ddetect ) (4) For a set of m lines and n points, this yields to an error ector E(Φ) of dimension 2m+n. The matching problem between 3D featres and 2D featres, i.e. the calibration, leads to the minimization of the fnction F = E T E. A first-order expansion of the error near the initial point Φ0 is performed: E(Φ) = E(Φ0 ) + E Φ (5) Φ0

5 4 Lorène Dbreil et al. z R CAM CAM C O xcam y CAM R 3*3 init,t init A d CAM R C O C y C z C A' ' x C C' d pict (a) 3D featre projection Fig. 4 Main steps of the approach e line d detect A' d pict Fig. 5 Calclation of the projection error C' d detect (b) 2D featre extraction e point D B' e 2 line [ ] E Let s denote J = the jacobi of E. The problem Φ Φ 0 is ths to find Φ so that F is minimized. This gies: [ ] [ ] F E = 2 E T = 0 (6) Φ Φ Considering the Jacobi of E, this yields to: J T E = 0 (7) Then: J T E = J T (E(Φ 0 )+J dφ) = J T E(Φ 0 )+J T J dφ = 0 And finally: D (8) Extracted featres Fig. 6 CAD-based calibration 3D featres projected before optimization 3D featres projected after optimization 2.3 Validation of CAD-based calibration The approach is applied to the set-p presented in figre 3. First, the intrinsic calibration is performed sing the Camera Calibration toolbox of Matlab c [3]. Intrinsic parameters are identified and the matrix M is known. The initial transformation T init is defined considering the manal matching of 6 points defined in the CAD model to the corresponding points on the pictre. 3D featres are projected onto the pictre according to eqation, giing elements in red in figre 6. In parallel, featre extraction from the pictre is performed sing the toolbox Image Processing of Matlab c. After filtering the pictre sing the Canny filter, featres are extracted from the pictre thanks to the Hogh transform. When necessary, the extracted element is constrcted as it is the case for axis cylinders (see figre 7). Extracted featres are reported in yellow in figre 6. The error fnction is ths calclated and the optimization problem is soled thanks to the least-sqare method. Reslts are displayed in figre reffig:alidation, in which the green elements are the optimized calclated featres. As it can be obsered, positions of the optimized featres are close to the nominal ones, assessing the releance of CAD-based calibration. Neertheless, there are still deiations between the projected and the extracted featres de to 2 main cases. The first one is that the set-p, which is assmed to be perfect, presents some defects that shold be taken into accont. Dimensions and positions of the components belonging to the set-p do not perfectly match their configration of the CAD model. On the other hand, the method sed to extract 2D featres from the pictre inoles some errors. The extraction is dependent on the qality of the pictre along with the parameters associated to the extraction method. Both sorces of er- dφ = (J T J) (J T E(Φ 0 )) (9) The problem is soled sing Matlab c psedoinerse. If the transformation T init is close enogh to the optimized soltion, the problem can be soled with only one iteration. If not, the optimization is reiterated considering that the initial transformation is gien by Φ = Φ 0 + dφ. Fig. 7 2D featre extraction

6 Calibration based on part set-p measrement for on-machine inspection sing ision Légende Extracted featre from pictre 3D featre projected on pictre (before calibration) 3D featre projected on pictre (after calibration) Fig. 8 Calibration from set-p measrement in real enironment rors hae the same importance on the deiations. In order to qantify the ncertainty associated to the proposed method, the inflence of 2D featre extraction is inestigated in the next section. 3 Application and sensitiity stdy 3. Description of the case stdy The inflence of 2D featre extraction is inestigated considering a real case for which the set-p is in sitation in the machine tool enironment. The calibration is performed at the beginning of the prodction, problems of coolants or remoed chips/strings obscring ision iews are aoided. The approach is applied by considering a set of 7 lines to perform the calibration (Fig. 8). To inestigate errors associated to the calibration, the projection error is calclated for a set of points belonging to the set-p. These points, which correspond to circle centers, were not sed for calibration (Fig. 9). Table 2 Reprojection errors Mean (pixel) Standard deiation (pixel) e e Fig. 9 Projection errors for a set of points Reslts reported in table 2 enhance the qality of the calibration. Actally, the absolte mean ale of the reprojection error does not exceed 0.24 pixel for both direction, and. Taking into accont the dimensions of the measred space (70 mm x 70 mm), related to the image size (3200 x 4500 pixels), the error is consistent with the order of magnitde of the machining defects we want to ealate. Neertheless, one of the key points of sing this method for on-machine inspection deals with its sensitiity to featre extraction and changes in calibration conditions. A sensitiity stdy is inestigated in the next section. 3.2 Sensitiity of the proposed method The sensitiity stdy is condcted by considering ariability in position and orientation of the extracted 2D featres. A 2D line is defined in the pictre by its explicit model, y = a x + b, where a represents the slope and b the intercept. An ncertainty is ths associated to each parameter a and b, and is modeled by a Gassian distribtion characterized by a standard deiation δ a, and δ b respectiely. δ a, and δ b are obtained by considering the ariations in position and orientation of a set of extracted lines for different calibrations for which lighting conditions are modified along with the size of the area arond the featre to be extracted. Indeed, lighting conditions can ary widely in the machine tool enironment, which can affect the qality of the 2D pictre, and as a reslt, can inflence 2D featre extraction. The size of the pictre area from which the 2D featre is extracted also plays a role in the qality of featre s extraction. On the other hand, it is necessary to accont for the ariability of the intrinsic parameters, which also cold affect the global calibration. These intrinsic parameters obtained sing the the Camera Calibration toolbox of Matlab c [3] and their standard deiations are displayed in table 3. An ncertainty is then affected to each intrinsic parameter, which is modeled by a Gassian distribtion characterized by its standard deiation. Once ncertainties are affected to each parameter to accont for its ariability, the ncertainty associated with CAD-based calibration is obtained thanks to 000 Monte-Carlo simlations (Fig. 0). The sensitiity stdy is condcted by considering the inflence of the ariability of all the parameters on the one hand, and on the other hand, by only taking into accont the inflence of the 2D featre extraction. Reslts, reported in figres and 2, bring ot that the standard deiation of the reprojection errors do not exceed 4.37 pixels for σe and 4.2 pixels for σe in the second case. These ales are twice lower than those obtained when all sorces of ncertainty are taken into accont. This obseration also applies to mean error ales. These reslts enhance the good stability of the CAD-based calibration method with regard to 2D featre extraction. Note that, a part of the error is likely de

7 6 Lorène Dbreil et al. Intrinsic calibration 2D extracted element R p Part set-p calibration R CAO zcao y CAO 3D Featre R CAM Φ = f(d pict, d detect, C',D) {ϕ, θ, ψ, t x, t y, t z } Φ Maximal error (pixels) O x CAO Fig. 0 Monte carlo description Table 3 Calibration parameters Mean Standard deiation f c.δ f c.δ C c C c k k k e e 4 k 4 8 e e 4 k to the set-p defects: dimensions and positions of the featres belonging to the actal set-p do not perfectly match their configration in the CAD model. As a conclsion, the ariation in lighting conditions does not hae more inflence on the reprojection error than the ariability of the intrinsic parameters, which appears to be significant in the ncertainty bdget. This confirms that calibration based on measrements of featres belonging to the part set-p is releant for on-machine inspection sing ision. Neertheless, some key isses still need to be addressed sch as improement of intrinsic calibration, and consideration of the actal set-p. 4 Conclsion In this stdy, we presented a CAD-based calibration method. Based on the CAD model of a machining set-p, this new techniqe is particlarly well-adapted for on-machine part inspection, as it allows measrements in the manfactring frame. It relies on the matching of 3D CAD-featres of the machining set-p with corresponding featres extracted from the 2D image. The first reslts show the releance of the proposed approach, attested by the good stability of the method with regard to image analysis. Ftre works will focs on the following stage of on-machine measrement for part inspection sing ision. This stage deals with the implementation of a correlation-based method to localize geometrical defects by ision. References. Chen, F. and S, C., Vision-based atomated inspection system in compter integrated manfactring, The International Jornal of Adanced Manfactring Technology,, pp , (996) 2. Vacharankl, F., Mekid, S., In-process dimensional inspection sensors, Measrement, 38, pp , (2005) 3. Yandayan, T., Brdekin, M., In-process dimensional measrement and control of workpiece accracy, Internationa Jornal of Machine Tool and Manfactring, 37(0) pp , (997) 4. Qinsat, Y., Tornier, C., In sit non-contact measrements of srface roghness, Precision Engineering, 36, pp 97-03, (202) Density Density Density Density e (pixel) e (pixel) e (pixel) e (pixel) (a) e = σ e = (b) e =.682 σ e = 9.27 (a) e = σ e = 4.37 (b) e = 0.39 σ e = 4.2 Fig. Distribtion of reprojection errors considering all sorces of ncertainty Fig. 2 Distribtion of reprojection errors considering 2D featre extraction ariability only

8 Calibration based on part set-p measrement for on-machine inspection sing ision 7 5. Polahon, F.,Génération en ligne de trajectoires d sinage par ne approche de rédction de modéle, PhD thesis, Ecole Centrale de Nantes, (205) 6. Polhaon, F., Rach, M., Leyge, A., Hascoet, J.Y., Chinesta, F.,Online Prediction of Machining Distortion of Aeronatical Parts Cased by Re-Eqilibration of Residal Stresses, Key Engineering Materials ol. 62, pp (204) 7. Karabagli, B, Vérification atomatiqe des montages d sinage par ision : application la sécrisation de l sinage, PhD thesis, Uniersité de Tolose Le Mirail, 6 noembre Karabagli, B., Simon T., Orte J.,J. Compter-ision-based atomatic erification of machining setps: method for atomatic pose estimation Joint International Conference of ISEM-ACEM-SEM-7th ISEM 2, Taipei (Taiwan) noember Ahmad, R., Tichado, S., Hascoet, J.-Y., Integration of ision based image processing for mlti-axis CNC machine tool safe and efficient trajectory generation and collision aoidance, Jornal of Machine engineering, ol. 0, pp53-65, (200) 0. Dornaika, F., Garcia, C., Robst camera calibration sing 2D to 3D featre correspondences, Proceedings of the International Symposim SPIE Optical Science Engineering and Instrmentation, Videometrics V, ol. 374, pp 23-33, (997). Fageras, O., Toscani, G., Camera calibration for 3D compter ision, International Workshop on Machine Vision and Machine Intelligence, pp , (987) 2. Laest, J.-M., Viala, M., Dhome, M., Do we really need an accrate calibration pattern to achiee a reliable camera calibration?, Compter Vision ECCV98, ol. 406, pp 58-74, (998) 3. Boget, J.Y. Camera Calibration Toolbox for Matlab, Comptational Vision at the California Institte of Technology. (200) 4. Doilly, M., Contribtion lamélioration de la qalit gomtriqe de positionnement dn robot anthropomorphe dans le cas dassemblage de strctres aronatiqes, PhD thesis, Ecole Normale Spérier de Cachan, 24 mars Tsai.R. Y., An Efficient and Accrate Camera Calibration Techniqe for 3D Machine Vision, Proceedings of IEEE Conference on Compter Vision and Pattern Recognition, Miami Beach, FL, pp , (986) 6. Ma, S., Wei,G.,Hang J., Segment based camera calibration. Jornal of Compter Science and Technology, ol. 8(), pp -6, (993) 7. Beabier, B., Dfor, J.-E., Hild, F., Rox, S., Laernhe, S., Laernhe-Taillard, K., CAD-based calibration and shape measrement with stereodic, Experimental Mechanics, ol. 54, pp , (204)