Mchine vision system for surfce inspection on rushed industril prts. Nicols Bonnot, Rlph Seulin, Frederic Merienne Lortoire Le2i, CNRS UMR 5158, University of Burgundy, Le Creusot, Frnce. ABSTRACT This work ims t detecting defects on metllic industril prts with streked surfce. The orienttion of those prllel streks is totlly rndom. The serched defects re scrtch nd lck of mchining. A specific mchine vision system hs een designed to del with the prticulr inspected surfce fetures. One imge is cquired with n nnulr lighting in right field nd six imges re cquired with rotting lighting in drk field. A prticulr imge processing is pplied on the six imges in order to get one imge tht represents ll the reveled imperfections. A thresholding processing is then pplied on this imge in order to segment the imperfections. A trined clssifiction, creted with well known typicl ojects of ech clss, is performed. The clssifiction hs to recognize the different defects nd the smll imperfections tht re not defects. The decision phse is used to know if the defects re cceptle, nd therefore if the inspected prt is cceptle. Some cceptility rules re defined for every defect clss. The developed mchine vision system hs een implemented on n experimentl industril production line nd it gives 2 % of su-detection nd 16 % of over-detection. Keywords: Mchine Vision, rushed metllic surfce inspection, smrt lighting, defect detection 1. INTRODUCTION A mnufcturer tht produces metllic prts wnts to utomte its production line. All the steps of the production line re prtilly utomted. Inspection is the lst step tht is completely performed y humn opertors. Severl experts re visully serching defects. The humn control implies n importnt sujectivity in the inspection. Indeed the sme prt could e cceptle for n expert ut not for nother. The disprity etween experts is incresed y the disprity, for the sme expert, etween the eginning nd the end of the dy. Besides, there is no reference for the relevnce of the defects. No rules llow the experts to define exctly prt s cceptle or not. The mnufcturer wnts n inspection system tht reches the humn control qulity. Mny industril ctivities hve enefited from mchine vision systems 1 5 nd especilly the metllic prts production. As humn control is visully performed, mchine vision system is good wy to utomte the inspection. The experts method used to detect the defects is to revel the defects with prticulr lighting. Therefore, this project will tke importnt enefits of using mchine vision system to inspect the prts. Finlly, the recorded results of the utomted inspection would llow to oserve exctly the evolution of the numer nd the type of detected defects. 1.1. Industril prt The industril prt is metllic. Its surfce is mchined in order to otin flt nd non smooth surfce. The result of this mchining is streked surfce, where the streks re prllel. We cll it streked pttern (figure 1). Those streks re very importnt for the mechnicl chrcteristics of the prt. Becuse of the specificity of the mchining, the orienttion of the streked pttern is totlly rndom, nd some secondry streked ptterns, with less intensity, could pper on the surfce with nother rndom orienttion. Nicols Bonnot, IUT, 12 rue de l fonderie, Le Creusot, FRANCE, - www.le2i.com E-mil: nonnot@iutlecreusot.u-ourgogne.fr ; Phone: +33 (0)3.85.73.10.90 ; Fx: +33 (0)3.85.73.10.97
Figure 1. Streked pttern 1.2. Defects Two different defects cn occur on the surfce of prt. Scrtch is due to the mnipultion of the prt fter the mchining nd is viewed s lengthened mrking in the prt. The position of scrtches is completely rndom on the prt s surfce. Lck of mchining is region of the prt without ny streks due to fltness prolem of the prt which surfce is not completely mchined. A lck of mchining is rough region of the prt s surfce. Lck of mchining cn pper on two different prt s zones. The first one is positioned on the edge of the prt, the second is on its center. Figure 2 presents the typicl defects. c Figure 2. (c) Scrtch Lck of mchining on the prt s edge. Lck of mchining on the prt s center. 2.1. Principle of reveltion 2.1.1. Shpe reveltion 2. ACQUISITION SETUP For ech prt, its position in the imge cn slightly vry. The position of the prt in the imge hs to e well known in order to pply correctly the imge processing on the prt s surfce. A region of interest, representing the inspected surfce, hs to e defined for ech prt. The prt is imged in right field 6 in order to revel the surfce ut not the environment of the prt (figure 3). In right field, the light is directly reflected y the surfce towrds the cmer nd the surfce ppers s white in the imge. The streked surfce of the prt is reveled in white, wheres the environment of the prt is drk. 2.1.2. Defects reveltion The mnufcturer wnts the mchine vision system to rech the qulity of the humn control. Its study hs shown tht the defects re reveled when the streked pttern is not visile. So the principle of the lighting is to revel the defects ut not the streked pttern. The study of the interction of the defects nd the streked pttern with light hs shown tht they re reveled when they re perpendiculr to the direction of the light nd when they re imged in drk field. 6 In tht cse, the surfce is right in the imge. When the direction of the light is prllel to the orienttion of the streked pttern (in drk field), the surfce ppers s drk in the imge (figure 4). A scrtch intercts with the light in the sme wy thn strek. When the light s direction is perpendiculr to the streked pttern orienttion, the prt s surfce is imged s white nd the scrtch is not visile. Therefore the scrtches re reveled when the light s direction is prllel to the streked pttern orienttion (figure 5.). Due to its rough spect, lck of mchining diffuse the light in ll the directions whtever the direction of the incident light. Therefore it is reveled with ny light direction nd is imged with medium intensity (figure 5.).
Figure 3. Reveltion of the prt s position. Figure 4. Reveltion of the streked pttern. Lighting perpendiculr to the streked pttern. Lighting prllel to the streked pttern. Figure 5. Scrtch reveltion. Lck of mchining reveltion.
Due to the rndom orienttion of the streked pttern specific lighting device ws designed nd relized in order to imge the prt with severl orienttions enling n efficient reveltion of the scrtches. 2.2. Lighting nd cquisition fetures The design of the lighting device is the most importnt step of the project. Better the cquired imges of the defects re nd simpler nd stronger the imge processing re. 7,8 A fixed cmer imges the inspected surfce of the prt. The cmer imges two prts. The cmer sensor resolution is 768*580 pixels, coded upon 8 its in lck nd white. Seven imges re cquired with two different lighting devices. One imge is cquired with n nnulr lighting in right field, in order to detect the exct position of the prt. It is clled nnulr imge. Six imges re then cquired with rotting lighting in drk field, in order to revel the scrtches with different light directions nd the lck of mchining (figure 6.(,)). They re clled rotting imges. This rotting lighting is composed of twelve pie segments tht cn e independently ctivted (figure 6.c). For ech rotting imge, two opposite segments of 30 in, re lighting the industril prt. Therefore, when the six rotting imges re cquired, the industril prt receives light from ll the directions. All the defects nd streked pttern tht exist on the surfce re then reveled in the rotting imges. Figure 7 presents the seven imges cquired for scrtch. Figure 7. presents the nnulr imge nd figure 7.(,c,d,e,f,g) presents the six rotting imges. The corresponding lighting configurtion is lso schemticlly represented. 2 1 1 1 30 1 Cmer 3 4 2 4 30 30 3 2 3 2 3 4 Annulr lighting Rotting lighting Prt's support c Figure 6. Lighting device 3D view. Lighting device front view. (c) Lighting device top view (12*30 ). c d e f g Figure 7. (,c,d,e,f,g) Annulr imge Rotting imges
3. IMAGE PROCESSING The imge processing re seprted into three steps. The detection phse hs to list the imperfections oserved on the prt. The clssifiction phse hs to recognize ech imperfection. The decision phse hs to decide if the prt is cceptle or not. 3.1. Detection phse The detection phse is seprted into three steps. In the first step the region of interest is creted. In the second step the rotting imges tht potentilly revel defects re selected. In the lst step, thresholding processing is performed in order to segment the imperfections. 3.1.1. Region of interest definition In the first step of the detection phse, region of interest, representing the inspected streked surfce, is creted. A fixed threshold is performed on the nnulr imge, s the cptured imge is sturted (grey level = 255) in the prt region. In order to exclude the edge s effects of the prt, morphologicl erosion processing, using [7*7] structuring element, is pplied on the msk. Figure 8. (c) c Annulr imge Thresholded nnulr imge Eroded imge s the region of interest 3.1.2. Rotting imge selection In the second step of the detection phse, the rotting imges re selected. A scrtch is oserved when the streked pttern is not reveled, tht is when rotting imge is drk. A lck of mchining is reveled in ech rotting imge. Therefore we work only on the drk rotting imges tht potentilly revel the defects. The drk rotting imges re selected upon their men nd their stndrd devition. A rotting imge with low men nd low stndrd devition mens tht the streked pttern is not reveled (figure 9). The oundries of men nd stndrd devition (figure 10) were determined on representtive smple prts oservtion. Men (µ) Stndrd devition (σ) Figure 9. (c) c 190 31 54 28 19 5 Streks reveled : high µ, high or low σ Streks reveled : low µ, high σ Streks unreveled : low µ, low σ
85 30 Men (µ) 0 8 15 µ = -7.86σ +147.88 Stndrd Devition (σ) Figure 10. Imge Selection s rules Defect reveltion vries etween the different rotting imges. Moreover, defect cn e prtilly reveled through different imges; it is due to its orienttion vrition (figure 7.(d,e)). A specific imge processing is pplied on the selected imges in order to get one imge tht completely represents ll the reveled imperfections. This imge is clled reveled imge. In rotting imge, righter pixel, representing n imperfection, is nd etter the reveltion is. The reveled imge (RI) is computed from the selected rotting imges (Rot i ) s follows: for ech pixel P(x,y) RI x,y = Mx(Rot i x,y ) with i = [1, 6] end for The rotting imges re so merged in unique imge where defects re fully reveled nd well contrsted. 3.1.3. Thresholding processing In the third step of the detection phse, thresholding processing is pplied on the reveled imge to detect the imperfections. A segmented imperfection is clled oject nd do not necessrily refers to defect. Two thresholds re computed on the reveled imge. The first fixed threshold, clled shpe threshold (1), llows us to segment the shpe of the existing ojects. After the shpe thresholding is performed, n oject cn e segmented in different los. Therefore, morphologicl closing processing is computed on the otined imge fter the shpe thresholding in order to connect the los of the oject. An re filter is performed on the ojects in order to remove the smll ojects, which re is less thn fixed vlue determined from oservtions. A new region of interest (ROI), representing the detected ojects, is creted. The second dptive threshold, clled detection threshold (2), is pplied on the new region of interest. It llows the selection of the segmented ojects tht re right enough to e potentilly defect (figure 11). The lock digrm of the thresholding processing performed on the reveled imge is presented in figure 12. For the rotting imges cquired in figure 7, the computed reveled imge is presented in figure 13., nd the oject s detection is presented in figure 13.. The shpe prmeters nd the intensity prmeters of the ojects re finlly computed in order to perform the clssifiction nd the decision phses. ShpeThreshold = Men + CsteST W ith CsteST = F ixed threshold vlue DetectionT hreshold = M en + CsteDT StndrdDevition W ith CsteDT = V lue for the dptive threshold (1) (2) 3.2. Clssifiction phse The clssifiction phse is sed on trined clssifiction opertor. 9,10 Due to the specific mchining, mny ojects, tht re not necessrily defects, re segmented on the surfce. The clssifiction hs to recognize those imperfections in order to distinguish them from rel defects. Moreover, there is not the sme level of cceptility etween the defect clsses. The clssifiction ws creted with well known typicl ojects of ech clss. For the clssifiction s trining, the typicl ojects were ll nmed y n expert. They re given s input to the clssifier, with their nme, their shpe nd their intensity prmeters. The clssifier gives us some clssifiction rules tht re pplied to the detected ojects. We otin the nme of every ojects detected on the prt.
!" #$%$#& ' () * +,-.)/ 1 2 + 3 + 0.)/ ), 00 Figure 11. Principles of the Shpe nd the Detection threshold. Reveled Imge Shpe Threshold Connection of close ojects ( Morphologicl Closing ) Smll ojects removl Oject's Are < Minimum Are Oject s ROI definition Detection Threshold Oject s shpe prmeter Oject s intensity prmeter Figure 12. Block digrm of the thresholding processing Figure 13. Reveled imge. Detection imge. Defects re surrounded in white.
3.3. Decision phse The decision phse is the lst step of the processing. It is used to know if defects re cceptle, nd therefore if the inspected prt is cceptle. Some decision rules re defined for the defect clss Scrtch nd lck of Mchining. As no simple decision rules were found, trined clssifiction sed on discriminnt nlysis is used in order to seprte the cceptle defects from the non cceptle defects. These decision rules re creted using the ojects correctly recognized y the clssifiction phse. The prts, on which these ojects were detected, were ll inspected y n expert. The ojects nd the prts were defined s cceptle or non cceptle. These definitions re compred with the results of the decision s rules nd llow the vlidtion of these rules. The clsses Acceptle (Acc) nd Non cceptle (NAc) re defined for ech defect clss. At the end, prt will not e ccepted if it contins t lest one non cceptle defect. 4. RESULTS The developed mchine vision system ws tested on 4000 prts. Ech prt ws inspected y the mchine vision system nd the humn opertor. The comprison etween the two controls re presented in tle 1. Only the prts on which ojects were detected, nd so on clssifiction nd decision phse re performed, re presented. The reported percentges represent the correct decision (digonl) nd the confusion (out of digonl). System Expert NAc Acc NAc 74.5% 25.5% Acc 23.5% 76.5% Tle 1. Glol percentge of correct decision for the prts Tle 1 does not include the prts for which no ojects were detected. Those prts re included in the computtion of the over-detection nd the su-detection. The comprison etween the glol results of humn nd mchine control gives 2.1 % of su-detection nd 16.6 % of over-detection. 5. CONCLUSION AND FUTURE WORKS The mchine vision system is vlidted y the mnufcturer. It hs een implemented on n experimentl industril production line. Both humn nd mchine control will e done on the prts in order to vlidte the mchine vision system on long time. The shpe nd intensity prmeters of the non cceptle defects re studied in order to oserve the evolution of these defects. The chrcteristics nd the numer of the defects will give informtion to the mnufcturer if there is prolem in the production line. The mchine vision system rings importnt enefits to the mnufcturer. The sujectivity induced y the humn control is removed nd enles n ccurte monitoring of the fctory s production line. The min restriction of this mchine vision system is tht it is not relly dptive. The mjor prmeters vlues of the processing (imge selection, threshold constnts,...) were determined from oservtions. The results presented in 4 re good for the defects nd the surfce s spect presented in 1.1 nd 1.2. An importnt evolution of the defects, or if new defect occurs, it will induce very d results. Moreover, the mnufcturer does not exclude the possiility of modifying the spect prt s surfce. Therefore, future work concerns dptive processing in order to consider ny future evolution. REFERENCES 1. E. N. Mlms, E. G. M. Petrkis, M. Zervhis, L. Petit nd J-D. Legt, A survey on industril vision systems, pplictions nd tools, in Imge nd Vision Computing, Volume 21, pp. 171 1881, 2003. 2. T. Pfeifer nd L. Wiegers, Relile tool wer monitoring y optimized imge nd illumintion control in mchine vision, in Mesurement, Volume 28, Issue 3, pp. 209 218, 2000.
3. A. D. H. Thoms, M. G. Rodd, J. D. Holt nd C. J Neill, Rel-time industril Visul Inspection: A Review, in Rel-Time Imging, Volume 1, Issue 2, pp. 139 158, June 1995. 4. AC. Legrnd, E. Renier, P. Suzeu, F. Truchetet, P. Gorri nd F. Meriudeu, Mchine vision systems in metllurgy industry, in Journl of Electronic Imging SPIE, 10(1), pp. 274 282, Jnury 2001. 5. P. Bourget, F. Meriudeu nd P. Gorri, Defect detection nd clssifiction on metllic prt, in Proc. SPIE Mchine vision industril Inspection X, Sn-Jose, USA, Volume 4664, pp. 182 189, Jnury 2002. 6. F. Pernkopf nd P. O Lery, Imge cquisition techniques for utomtic visul inspection of metllic surfces, in NDT & E Interntionl, Volume 36, Issue 8, pp. 609 617,Decemer 2003. 7. R. Seulin, N. Bonnot, F. Merienne, P. Gorri, Simultion process for the design nd optimiztion of mchine vision system for speculr surfce inspection, in Conference on Mchine Vision nd Three-Dimensionl Imging Systems for Inspection nd Metrology II, SPIE, Boston, USA, 4567, pp. 129 140, Octoer 2001. 8. R. Seulin, F. Merienne, P. Gorri, Simultion of speculr surfce imging sed on computer grphics : ppliction on vision inspection system, in Journl of Applied Signl Processing - Specil issue on Applied Visul Inspection, EURASIP, 2002 (7), pp. 649 658, July 2002. 9. P. Bourget, K. Toin, F. Meriudeu, P. Gorri, Pttern Wfer segmenttion, in Journl of the Society of Mnufcturing Engineers, Volume 242, pp. 2 11, Jnury 2003. 10. J. Mitern, S. Bouillnt nd E. Bourennne SVM pproximtion for rel-time imge segmenttion y using n improved hyperrectngles-sed method, in Rel-Time Imging, Volume 9, Issue 3, pp. 179 188, June 2003.