AN ARTIFICIAL NEURAL NETWORK -BASED ROTATION CORRECTION METHOD FOR 3D-MEASUREMENT USING A SINGLE PERSPECTIVE VIEW

Transcription

1 Ma 8, 9 and 30, 1997 Le Ceusot, Bougogne, FRANCE AN ARIFICIAL NEURAL NEWORK -BASED ROAION CORRECION MEHOD FOR 3D-MEASUREMEN USING A SINGLE PERSPECIVE VIEW Kauko Väinämö, Juha Röning Depatment of Electical Engineeing, Univesit of Oulu, FIN OULU, FINLAND el: , , fa: , venei@ee.oulu.fi, jj@ee.oulu.fi Abstact - High accuac in woking lining wea ate measuements of a metallugical vessel is ve essential. If the molten steel buns though the vessel because of ecessive lining wea, the consequences ma be substantial losses in poduction, damage to the instumentation, and equipment and dange to the staff. Such lining measuements ae done with lase scanning equipment. he positioning of the vessel fo the lase scanning is done with a CCD-camea. Each vessel contains a positioning plate, which is a plana object containing fou ings at the cones. A pocedue ealised with a special image pocessing hadwae locates the ings on a ectangula plane with known dimensions. With that infomation the position of the metallugical vessel is calculated befoe each lase scanning. he ing seach method is based on coss-coelation, which is a ve accuate method in D situations. hee is, howeve, a poblem of the pespective pojection of the ing at diffeent angles, which causes a little inaccuac in otation tansfomation calculation. In this pape we popose an adaptive atificial neual netwok (ANN) based method to estimate and to coect the eo in otation measuement due to pespective view of the ings and fom some othe possible eo souces like diffeences in optical components and thei lens distotion calibation. he adaptabilit is equied since installation and the measuement geomet of each lase measuement sstem installation is diffeent. he sstem has been veified with measuements in a steel plant. 1. INRODUCION In steel plants, metallugical vessels ae used fo tanspoting liquid steel. Because of the eteme heat and flow of the molten steel the lining of the vessel weas and good monitoing is equied so that the ceamic lining will not wea ecessivel. A lase measuement sstem is ve useful fo contolling the wea of these vessels [1]. he 3D-model of the vessel s inne suface is calculated b measuing seveal hundeds of points and using Delaune tiangulation to ceate a suface consisting of tiangula pimitives []. o get eliable measuement esults it is essential to ensue that individual esults ae compaable with each othe. Each time the vessel is measued, it is positioned b a CCD-camea. he pupose of the positioning is to establish the otation and tanslation elationship between the lase measuement fame and the vessel s co-odinate fame. If thee is an eo in the positioning infomation, the measuement is not compaable with pevious measuements. Howeve, the measuement is compaative in natue, so the eos due to some offsets do not affect the final esults consideabl. hee is a lage vaiation in the vessels and thei envionment. he installation geomets of the measuement sstem diffe ve much fom each othe. he mounting of the vessel esticts the movement. hat wa each installation place has its own chaacteistics. Atificial Neual Netwoks (ANN) ae a ve useful technique to solve poblems that need an adaptive appoach [3,4]. he goal of this wok is to ceate an epeimental pocedue to minimise the eo, which is due to uncetainties in plate fame tansfomation calculation. his is established b doing measuements on the actual measuement scene and compaing lase measuements to the CCD-camea measuements. he model of the envionmental uncetainties is taught to the ANN duing the installation peiod. Aftewads the ANN taught is capable of coecting otation eos in the pocess use of the measuement device. he CCD-camea positioning is based on the pinciple of calculating a 3D pose of a known object in a single pespective view. he measued object is a ectangle, which is defined with fou ings. he cental point of each ing defines the cone point of the ectangula object. he algoithm ealised with special hadwae etacts the image and seaches fo the cente points of each ing. he cente points define a ectangula plane that is deliveed to the basic 3D-positioning algoithm. IMAGE EXRACION RING SEARCH 3D POSE CALCULAION VESSEL'S POSIIONING 3D MEASURE- MEN PLAN 3D SURFACE MEASUREMEN WIH LASER Figue 1. he wokflow of the CCD-camea positioning pocedue and the actual 3D measuement with lase. he positioning pocedue with a single CCD-camea is quite accuate, howeve, because of the lage sie of the vessel even a small eo in otation tansfomation leads

2 Ma 8, 9 and 30, 1997 Le Ceusot, Bougogne, FRANCE to an eo of seveal millimetes in the lining thickness calculation. he vessels ae measued on at special measuement station whee the toleance in the placement of a vessel is limited. So it is possible to model the woking envionment of the vessel. In each station the accuate otation angles can be measued with the lase measuement sstem using the cicle positioning method. he pinciple of the cicle positioning method is to measue seveal points fom the edge of the vessel s mouth in a cicula manne. his wa the eos in otation tansfom can be modelled as a function of the otation measued with the CCD-camea. Howeve, the cicle positioning method is not applicable since it equies manual and caeful wok to poceed and it is time consuming as well. Faste and moe obust CCD-camea positioning method is used fo that eason. Eos in the otation mati come fom seveal souces. he envionment in a steel plant is etemel ough. Fo eample, vibation of the CCD-camea is one eo souce as well as electomagnetic noise in the video signal, but the most consideable poblem is the method of finding cicle pimitives fom the positioning plate. he plate consists of fou ings that ae machined ve accuatel. he ings can be distinguished fom the etacted image cleal because the contast between the ings and the backgound is high. he pinciple of the ing seach is based on a coss-coelation method. he model of the ing is fist taught to the algoithm and then maimum coelation is sought fom the image based on the taught model. he ehaustive seach is a ve slow pocedue, howeve the pocedue used is optimised with the hill climbing optimisation method. Fist the optimal stating points ae found fom the fequenc domain and the final coelation matching is done using the hillclimbing algoithm efeed to above. he pinciple of the seach is to go to the diection whee the coelation esult gows. his seach method woks fine, when the pojection of the ing is othogonal to the CCD-element. Howeve, the optimal wa to measue the positioning plate otation is to aange some angle between the - and -aes of the CCD-element s co-odinate sstem and the positioning plate s co-odinate sstem. his aangement maimises the sensitivit of the otation measuement. If the plate is othogonal to the optical ais of the camea, a small otation of the plate poduces a ve small movement of ing cental co-odinates in the Dfame of the CCD-element. When thee is a clea angle between co-odinate fames the sensitivit is consideabl bette. So even a small otation of the plate poduces a significant movement of the ing cental co-odinate points. Figue shows two positioning plates whee the fist is neal othogonal to the camea and the second is otated 8 degees fom the othogonal placement. Figue. he positioning plates of the metallugical vessel. he poblem in the cuent method is that the ing model is oiginall taught in one position and aftewads when the plate is otated some angle, the pespective pojection commences to poduce an eo in the otation measuement. he oiginal model used in coelation matching diffes a little compaed to the model in the otated image. he ing is found, but the coelation and the D-placement accuac have deceased. Figue 3 shows two of the ings found at the angle of 8 degees. he fist ing is the lowe left ing of the plate and the second is located in the lowe ight position. he effect of the pespective pojection is cleal noticeable. Figue 3. Eample ings that define the cone points of the ectangula object plane. In this pape we popose an adaptive method to coect the otation measuement eos due to the pespective distotion of the D-model in the CCD-camea positioning. his calibation of the otation elationship simpl coects the eo in the otation mati. he method is based on the pinciple of adaptive installation. Atificial neual netwoks (ANN) wee used fo modelling the function of seveal vaiables. his wa each constuction of a metallugical vessel measuement sstem can adapt to the eos due to envionmental factos. he teaching mateial needed is obtained with seveal measuements using CCD-camea and lase measuements. he neual netwok is tained with the otation mateial obtained. Afte pope taining the netwok is used to fine-tune the CCD-positioning sstem. he pape is divided into seveal sections. In chapte 1 and the basic pinciple of the metallugical vessel measuement sstem is descibed as the motivation of the pape. Chapte 3 includes the mathematical notation of the calculation of the 3D pose of the ectangula object used fom a single pespective view. Chapte 4 descibes ou method to impove the accuac of the CCD-camea positioning accuac using atificial neual netwoks. Finall, chapte 5 indicates the esults of the eseach. he poposed otation measuement impovement fo the sstem is an oiginal and new method contibuted b the authos of this pape.

3 Ma 8, 9 and 30, 1997 Le Ceusot, Bougogne, FRANCE. NAURE OF MEASURING HE WEAR OF HE MEALLURGICAL VESSEL he pupose of the lase measuement sstem is to monito the weaing of metallugical vessels. Measuements consist of individual measued points in the vessel fame. he inne suface of the vessel is modelled with tiangula epesentation. he wea o the thickness of the lining is calculated fom the infomation fom these models. he cuent measuement is compaed to the fist measuement of the campaign. he campaign is the time between the eplacements of the woking lining of the vessel. he distance of the sufaces is calculated at each actual 3D-measuement point. he won thickness value of the lining is the distance of the sufaces at each point. he diection of the thickness vecto is nomal to the fist measued suface, which is called the efeence suface. he wea of the vessel s woking lining is optimised because eplacement of the lining is ve epensive. On the othe hand the lining is not allowed to wea too much because it is then possible that the vessel will bust which causes significantl moe epense. he basic elationship of the co-odinate fames used is descibed in Figue 4. he oigins of the co-odinate sstems ae defined as follows; the lase measuement head O LR, the CCDcamea O C, the positioning plate O P and the vessel O V. established. Fom the sstem co-odinate calibation the elationship between the lase and the CCD-camea is known befoehand. he cicle positioning method with the lase measuement sstems is based on co-odinate tansfomation with thee measued points. he points ae measued fom the uppe edge of the vessel with the lase measuement device. At least thee points define a co-odinate fame. he fist two points define the diection of the -ais. he thid point defines the - -plane with othe two points. he -ais is calculated with the coss poduct of the vectos' v 1 and v, whee v 1 is the vecto fom p 1 to p and v is a vecto fom p 1 to p 3. Finall the diection of the -ais is calculated using the coss poduct of the unit vectos of the - and -ais. he otation s tansfom is simpl a mati containing unit vectos of co-odinate aes as column vectos. he basic situation is descibed in Figue 5. Points p 1, p, and p 3 ae pesented in the measuement head co-odinate fame. he same points define the vectos p 1, p and p 3, stating fom the oigin of the measuement head. P1 v1 v v P3 C R, v1 LASER SENSOR P O LR O C R, R p, p O P O V R v, v R c, c Figue 4. he basic envionmental pinciple of the woking lining wea measuements of the metallugical vessels. he placement of the vessel in the measuement station is mechanicall esticted. he toleance of the placement is a few hunded millimetes in tanslation and a few degees in otation. he lase measuement sstem is located above the vessel. he positioning CCD-camea is located alongside the vessel in such a wa that the positioning plate of the vessel is seen without obstacles. o define the placement of the positioning plate in the vessel s co-odinate fame, the cicle positioning pocedue is done. With cicle positioning the elationship between the lase s fame and the vessel s fame is Figue 5. he tansfomation between the vessels and lase fame. he otation elationship is established though the pinciple of the nsa co-odinate fame tansfomation [5]. he plane of the vessel's mouth is detemined with vectos v 1 and v : v = p p v = p p he nomal of the plane is calculated though the coss poduct: v3 = v1 v () o ceate the othogonal tansfomation the vecto v is ecalculated: v = v3 v1 (3) Now the final otation mati R is constucted: whee, (1) R = [ n s a ], (4)

4 Ma 8, 9 and 30, 1997 Le Ceusot, Bougogne, FRANCE v1 n = v1 v s = v3 v a = v 3 he tanslation vecto is calculated fom the otation elationship between the co-odinate fames. he positioning points ae pojected on the plane defined b the mouth of the vessel. In the beginning the tanslation vecto is defined with the point P 1 and it is the oigin of the vessel s co-odinate fame. he cental point of the vessel s mouth is calculated with b fitting a second ode cuve to the points and calculating the cental point of the cuve. o get the best esults, moe than thee points should be used. he basic second ode fitting technique is the use of the ellipse equation. (5) A + B + C + D + E = 0, (6) whee, A,B,C,D,E ae polnomial coefficients,, ae co-odinates in D fame. Fom Equation 6 we can ceate a mati definition of the second ode equation. With Q and w the cental point of the ellipse can be detemined with Equation 9. q q A Q = 1 q q = 3 B B C [ ] [ ] (7) w = q4 q5 = D E (8) o = Q 1 w (9) he poblem is to solve the coefficients of the ellipse equation. Howeve, coefficients can be solved using the least-squaes estimation [6]. Fist the H mati is defined and then the vecto q including final coefficients is solved b pseudo-invese calculation. 1 H = f 1 1 [ ] O (10) = (11) q = [ q q q q q ] (1) q = ( H H) 1 H f (13) Finall the tanslation is solved with vecto p 1 and o. o = R o + p 0 1 (14) Afte this positioning calculation, the inne suface of the vessel is scanned with a lase beam. he points ae fist pesented in the lase fame but the ae moved to the vessel s fame. Each measued point in the lase s coodinate fame is tansfomed to the vessel s fame with Equation 15. p = R 1 ( p ), (15) whee, p v v p l l is a point in the vessel s fame, is a point in the lase s fame. When the cicle positioning is done the position and the otation of the positioning plate can be calculated in the vessel s co-odinate fame. he tansfomation between the positioning plate and the vessel is saved to the database and it is used aftewads in CCD-camea positioning. he otation R v (Fig. 4) can be calculated fom Equation 16 and the tanslation v fom Equation R = R R R, (16) v p c = R 1 ( R + ), (17) v p c p In CCD-camea positioning the tansfomation between the lase head and the vessel is calculated fom the esults stoed afte the cicle positioning pocedue. 3. CALCULAION OF HE 3D POSE OF HE POSIIONING PLAE IN A SINGLE PERSPECIVE VIEW he algoithm to detemine the position and the placement of the positioning plate is based on a pioi infomation of the plate dimensions and CCD-camea paametes and on the measuement esults; the co-odinates of the plate s vetices in the CCD-element D-pojection. he camea paametes include the focal length, which is the distance between the CCD-element and the optical cente of the lens, the lens distotion coection paametes, the hoiontal piel width and vetical piel height of the CCD element and the location of the optical cente of the lens on the CCD-elements D -co-odinate fame. Seveal methods ae poposed as a solution to this poblem [7,8,9,10,11]. Iovenitti et al. has poposed a method of thee-dimensional measuement using a single object [7,8]. Howeve, the method is based on a situation whee the distance to the object is about the same as the focal length of the objective. hat is wh the poposed method is not usable in this situation whee the measuement distance is about 5 metes. Hoaud et al. has poposed an analtical solution fo the pespective 4-point poblem [9]. Jäviluoma has poposed some impovements to the Hoaud s method [10]. In this epeiment the

5 Ma 8, 9 and 30, 1997 Le Ceusot, Bougogne, FRANCE measuement of the plate position in a 3D-pose was based on Hoaud s and Jäviluoma s wok. he analtical solution fo the pespective 4-point poblem is based on the situation descibed in Figue 6. With the method the placement elationship to the camea fame of the taget is calculated with fou image pojection points when the distance and the nomal of the taget points ae known. he camea model used is a simple pin hole model. he oigin of the camea s coodinate fame is in the focus point of the camea s lens. he image plane is pependicula to the diection of the - and -ais of the camea fame. he distance to the image plane is the same as the focal length of the lens. Camea fame J3 J J1 l3 l1 l J P3 k' Pictue fame M3 M L3 M1 L1 L M P3 aget fame L3 P3 Figue 6. he co-odinate fame elationship of the model used. One of the fou points is selected as the pinciple point. he pinciple point in Figue 6 is the point M and the coesponding pojection point in the image plane is J. he oigin of the image fame is placed at the point J and the oigin of the taget is placed at the point M. he detailed analtical solution fo computing the eteio camea paametes fom fou coespondence points in geneal positions can be found fom Hoaud s pape [9]. 4. PRINCIPLE OF A ROAION RANSFORMAION UNING WIH NEURAL NEWORKS he basic poblem equies adaptive settlement. Eve installation of the measuement sstem diffes ve much fom each othe. his was the eason wh ANN s wee selected to solve the poblem at hand. A multilae pecepton tained with the back-popagation algoithm ma be viewed as a pactical vehicle fo pefoming a non-linea input-output mapping of a geneal natue. he basic issue to solve was the topolog of the netwok, what would be inputs and outputs. he poblem at hand was the otation tansfom accuac so thee wee thee input paametes to use, thee otational diections to coect and, because of that, the numbe of outputs wee the same. Figue 7 illustates the ANN s topolog at the beginning of the topolog optimisation. Howeve, the otational accuac ove the -co-odinate ais was ve good and the poblem was focused to the otation measuements ove - and -co-odinate aes. his allows a eduction of the paametes used. It was not necessa to ca out the -otation coection. a a a Figue 7. he fist ANN topolog used in the eseach. Eos in otations wee calculated with known elative placement infomation obtained fom the lase measuement sstem. he netwok stuctue was taught with the mateial obtained. Seveal stuctue sies and activation functions wee tested and the sigmoid activation function was found to be the most usable. In the beginning the hbid model was used fo the coection function. he model included all the otation degees of feedom (DOF). his eduction of paametes used led to the simplification of the ANN stuctue used. Late the teaching was done sepaatel fo each otation diection and in that wa it was possible to visualise the teaching pocedue. his wa each DOF to be coected got its own ANN stuctue. he poblem of ovetaining was also avoided with the coss validation [3]. On the othe hand D-pesentation of the taining samples and the esults of the ANN stuctue output wee pesented in the same diagam. In this wa the ovetaining poblem was eliminated b the visual inspection of the esults. Figue 8 descibes the teaching angles aound the vessel s -ais. In the diagam the functional envionment is placed on the -ais and the eo values in the otation is placed on the -ais. he unit is in degees. Figue 8. he taining angles used fo taining the ANN stuctue. In this epeimental case onl one diection of otation was used. In case of multiple diections, the same pocedue will wok equall well. he otation measuements wee conveted to the netwok teaching mateial. Input samples consisted of the actual esults fom the CCD-camea positioning pocedue. aget values consisted of the calculated eos in otation. d d d

6 Ma 8, 9 and 30, 1997 Le Ceusot, Bougogne, FRANCE Samples wee scaled between and the netwok wee taught with the Levenbeg-Maquadt ANN leaning algoithm [3]. 5. RESULS he sstem has been veified with eal measuements in a steel plant. he eal measuements wee done with a metallugical vessel, whee inne diamete was 1.7 metes and height 3.8 metes. he position of the plate was in the uppe pat of the vessel. he angle measuement eos befoe the coection wee 0.1 degees on aveage. Afte the coection the accuac was impoved to a value of 0.05 degees eo on aveage. he test was epeimental in natue, howeve, the method is useful in an installation to impove the oveall accuac of the ceamic lining measuement sstem. aditional statistical methods could also be consideed, but the poblem is non-linea in some installations and theefoe basic least squae line fitting is not alwas useful. Howeve, statistical methods povide a benchmak fo used ANNs. Oiginal idea was ceated duing one installation case in Japan. he time to ceate the calibation was 48 hous. he eason wh the ANN was selected as a solution was the non-lineait of the phenomena and the need of adaptation. he standad deviation of the otation eo befoe the tuning was degees and maimum absolute eo was -0.3 degees. hat means a thickness calculation eo of 0 millimetes at the bottom of the metallugical vessel when the positioning plate is located on uppe pat of the vessel s wall. Figue 9 shows the otation eos afte the otation tuning. he standad deviation of the eo is degees. his means.7 millimetes standad deviation in thickness calculation. In the diagam the functional envionment is placed on the -ais and the eo values in the otation is placed on the -ais. he unit is in degees. Figue 9. Rotation eos afte tuning calculation. 6. DISCUSSION AND CONCLUSIONS An ANN-based otation coection method fo 3Dmeasuement using a single pespective view was ceated as a esult of this epeimental wok. he wok was done almost completel in a steel plant. he sstem has been veified with eal lase scanning measuements. he esults of this epeimental wok ae pomising, but moe eseach is needed to impove the method s capabilit in pactical situations. he ing seach method is eplaceable, but the coss-coelation seach is ve efficient when the ings ae patl coveed with dit. hee ae moe than two was to go in the futue. One possibilit is to eliminate the effect of the pespective eo b impoving the cente point seach algoithm. It will be challenging wok because of the etemel sevee envionment. he othe wa to go is to use statistical methods to coect the known eo in otation measuements. Howeve, non-linea cases will cause poblems. he thid wa is to go ahead with the ANN method and to make it a plug-in tpe poduct fo each plant installation. In that case automation of the calibation pocess is needed, as well as an atificial method to define the ANN stuctue fo each installation. 7. ACKNOWLEDGEMENS he authos would like to epess ou appeciation to ou fiends in the Specta-Phsics Vision ech ltd., and paticulal to M. Hannu Jokinen, fo thei helpful comments and assistance in the eseach. Specta-Phsics Vision ech ltd. holds the patent ights of the CCD-camea positioning method. 8. REFERENCES [1] Määttä, K., Pulsed time-of-flight lase angefinding techniques and devices fo hot suface pofiling and othe industial applications, Doctoal thesis, Univesit of Oulu, [] Mücke, E.P., A Robust Implementation fo 3D Delaune iangulations, Compute Reseach & Applications Goup, Los Alamos National Laboato. [3] Hakin, S., Neual Netwoks; A Compehensive Foundation, MacMillan Collage Publishing Compan, New Yok, [4] Feeman, J., A., Skapua, D., M., Neual Netwoks; Algoithms, Applications and Pogamming echniques, Addison-Wesle Publishing Compan, New Yok, 199. [5] Koivo, A., J., Fundamentals fo contol of obotic manipulatos, John Wile & Sons Inc., New Yok, [6] Kesig, E., Advanced Engineeing Mathematics, John Wile & Sons Inc., New Yok, [7] Iovenitti, P., hompson, W., Singh, M., heedimensional measuement using a single image, Optical Engineeing, Vol. 35 No. 5, Ma [8] Iovenitti, P., Accuac of thee-dimensional measuement using a single image, Optical Engineeing, Vol. 36 No. 1, Janua [9] Hoaud, R., Conio, B., Leboulleu, O., An analtic solution fo the pespective 4-point poblem, Compute Vision Gaphics and Image Pocess. Vol. 47, pp , [10] Jäviluoma, M., Ei-iteoiva laskentamenettel tähslevn paikoitukseen kameaan nähden, V

7 Automation, commecial poject apot fo SPV ltd., 1996, in Finnish. [11] Pehkonen, K., Calculation of 3-D pose of a known object in a single pespective view, V Publications, 199. INERNAIONAL CONFERENCE ON QUALIY CONROL Ma 8, 9 and 30, 1997 Le Ceusot, Bougogne, FRANCE