Tracking and Guiding Multiple Laser Beams for Beam and Target Alignment

Transcription

1 Internatonal Journal of Automaton and Computng 12(6), December 2015, DOI: /s Trackng and Gudng Multple Laser Beams for Beam and Target Algnment Peng-Cheng Zhang De Xu Research Center of Precson Sensng and Control, Insttute of Automaton, Chnese Academy of Scences, Bejng , Chna Abstract: An accurate and robust approach for trackng and gudng multple laser beams s developed, whch can be appled to the task of beam and target algnment. Multple laser spots are frstly detected and recognzed from the mage sequences of the target and laser spots. Then, the contour trackng algorthm based on the chan code s nvestgated, n whch the shape matchng scheme based on the nvarant moments s employed to dstngush dfferent spots. When occluson occurs n the multple spots trackng procedure, the contour trackng combned wth Kalman flter predcton s proposed to obtan the postons of multple spots n real-tme. In order to gude 3 spots to algn the target, an ncremental proportonal ntegral (PI) controller s employed to make the mage features of spots converge to the desred ones. Comparatve experments show that, the proposed trackng method can successfully cope wth the fast moton, partal or complete occluson. The experment results on spots gudng also exhbt the accurate and robust performance of the strategy. The proposed vsual system solves the problem of spots mxng, reduces the algnment tme, mproves the shootng accuracy and has been successfully appled to the expermental platform. Keywords: Laser spot, spot segmentaton, contour trackng, vsual feedback, beam and target algnment. 1 Introducton In order to apply the theores of nertal confnement fuson (ICF) to practce, the challenges assocated wth beam and target algnment system must be overcome [1].Itsnecessary to take measures to realze rapd and precse algnment n a tme as short as possble. In order to acheve hgher shootng accuracy, the laser spots should be guded through contnuous vsual feedback n real-tme [2]. Normally, the process of algnng multple laser beams to the desredregononthetargetscalledabeamandtarget algnment process, and a sensor for algnng multple laser beams to the target s called a target algnment sensor, whch manly conssts of the conjugate reflectvty mrror and the mcroscope lens, as shown n Fg. 1. In ths work, 3 dedcated laser algnment devces are ntegrated to gude 3 laser beams to acheve the beam and target algnment. For each laser beam, the laser from the transmtter s amplfed, processed through a flter wheel, focused by a set of convergent lenses, and then fltered va an aperture of a crcular hole of 6 mm dameter. Fnally, the laser s focused agan by a convex lens and s reflected to the end surface of target by a hgh-reflectvty mrror. Therefore, the detectng, trackng and gudng technologes of laser spots become very mportant so that multple laser beams can shoot the desred regon of the target n the beam and target algnment procedure. Regular Paper Manuscrpt receved December25, 2013; accepted December 4, 2014 Recommended by Assocate Edtor Nazm Mr-Nasr Ths work was supported by Natonal Natural Scence Foundaton of Chna (Nos and ). c Insttute of Automaton, Chnese Academy of Scence and Sprnger-Verlag Berln Hedelberg 2015 The detecton and segmentaton of multple spots n mage sequences s a fundamental step before the trackng or when the trackng s faled. The laser spot s assumed to be the only part of the mage wth a hgh lght ntensty. The common approach s to extract the spots by background subtracton. However, n practce, the ntensty of laser may randomly change, the llumnaton n the scene may not be very stable and even the ntensty of a cylndrcal target may vbrate slghtly. All of these may nduce the changes of background. Stauffer and Grmsor [3] modeled each pxel as a mxture of Gaussans and used an on-lne approxmaton to update the model. However, the method suffers from slow learnng at the begnnng and s not senstve to the small motons. An mproved Gaussan mxture model was presented by Zvkovc [4], n whch not only the parameters but also the number of components of the mxture were constantly selected for each pxel. However, ths method has hgh computatonal cost, especally, n the face of the hgh mage resoluton. The trackng algorthm can provde the poston feedback for the spot n real-tme. In [5], the object trackng was dvded nto pont trackng, kernel trackng and slhouette trackng. Features, such as color, edges, optcal flow and texture are often chosen for trackng. They can be represented by a probablstc model and then detected n consecutve frames. In general, the most desrable features of spot are the ones whch can be used to dstngush one spot from the others. Consderng the gray mage, the bggest dfference between dfferent spots s the contour then the ntensty. Moreover, due to the fact that the spot s non-

2 P. C. Zhang and D. Xu / Trackng and Gudng Multple Laser Beams for Beam and Target Algnment 601 Fg. 1 Beam and target algnment system confguraton, n whch the mcroscope s used n the magng procedure of the target whle t s not used n the magng procedure of laser spots rgd, the trackng algorthm based on ts contour s more practcal. Many contour models have been reported for trackng n the prevous lterature, such as optcal flow, level set, snakes, balloons and actve contours model [6 8]. Some contour trackng methods based on boundary codes were nvestgated n [9, 10]. However, they are lmted to the boundary-based nformaton and are senstve to nose. In order to overcome the drawbacks of the senstveness to nose and poor mage contrast, a partcle flterng algorthm for geometrc actve contours trackng was proposed n [11]. However, these technques requre a number of teratons and are computatonally too expensve for real-tme applcaton on multple spots trackng. It s necessary to develop one rapd and robust contour trackng scheme for spots. Another mportant problem s that the spots may mx together and nterfere wth each other when multple laser beams smultaneously shoot the target. The problem may be solved by gudng a sngle beam at a tme, however, whch s tme-consumng. Therefore, occluson handlng s a dffcult ssue n the face of multple spots trackng. In the prevous lterature, the appearance model was ncorporated, or the target was treated as a blob whch may merge and splt, or an excluson prncple was employed by usng the jont probablstc data assocaton flter and the partcle flter was employed to avod the hgh computatonal load [12 15]. The pror nformaton of shape s often ntegrated nto contour representaton. Ylmaz et al. [16] proposed a non-rgd trackng method, whch was acheved by evolvng the contour from frame to frame wth some energy functons. The contour represented by level sets was used to recover the mssng object regons durng occluson. However, these methods requre precse model, ncrease the computatonal complexty and may fal n the face of the rapd motons. The recognton of multple spots s also a key problem durng multple spots trackng. In order to dstngush dfferent spots, shape representaton and matchng technques should be consdered. A number of successful shape matchng algorthms were proposed. One of the most popular methods s to use Hausdorff dstance, although t s very senstve to outlers. Some methods compare shape by the feature vector whch contans the descrptors such as area, geometrc moments, shape matrx, appearance va gray hstograms, optcal flow vectors, etc. [17], whle others drectly do wth the ad of pxel brghtness [18]. Belongge et al. [19] proposed the shape context for shape matchng and object recognton, whch descrbed the contour ponts by hstogram n the log-polar space. The smlarty between two shapes was computed by a sum of matchng errors between correspondng ponts. However, a large amount of calculatons wll be needed and they are hard to satsfy the realtme requrement. The motvaton of ths paper s to develop the detecton, trackng and gudng schemes for multple laser spots based on a beam and target algnment expermental platform. An accurate and real-tme system for multple laser beams shootng s presented, whch conssts of spots segmentaton, spots contour trackng even under occluson and spots gudng based on vsual feedback control. The rest of ths paper s organzed as follows. Secton 2 descrbes multple spots detecton and trackng strategy, n whch the spot segmentaton, the contour trackng and the shape matchng algorthms are ntroduced. In order to accomplsh the beam and target algnment task, Secton 3 presents multple spots gudng scheme. Secton 4 provdes the expermental confguraton and the analyss of experment results. Fnally, the paper s concluded n Secton 5. 2 Multple spots detecton and trackng strategy The achevement of the beam and target algnment based on vsual feedback needs to obtan the postons of multple

3 602 Internatonal Journal of Automaton and Computng 12(6), December 2015 spots n real-tme. The accurate postons need to be determned by the detecton and trackng stages. As shown n Fg. 2, the proposed strategy s ntalzed wth a status flag L = 1. Each spot s recognzed by ts shape and s numbered after the detecton. Meanwhle, ts locaton, wdth, heght, mage moments and so on are also calculated. If all spots are found, then flag L s set to 0. When a new mage s captured, the trackng algorthm s employed to obtan the new features of spots. When occluson occurs, the trackng scheme combned wth a predcton mechansm wll determne the new postons of spots. If the shape matchng between the current contour and the prevous one s faled, then, flag L changes to 0. Otherwse, the postons of the current spots wll be provded to the gudng stage. Then, the spots n the next frame wll be tracked untl stoppng. Fg. 2 Block dagram of multple spots detecton, trackng and gudng strategy 3 Spots detecton stage 3.1 Spots detecton stage Frst of all, the movng spots are segmented from the beam and target mages. However, nose may be ntroduced and the dfferental mage may be ncomplete by statonary background subtracton. Many solutons n the prevous lterature have been proposed for real-tme foreground detecton from movng background. Here, the adaptve Gaussan mxture model (GMM) s employed to obtan the spots background [20]. A Bayes decson rule for classfyng background and foreground s formulated and the learnng strategy s ntroduced to adapt to slght changes n background. The probablty of each pxel value at tme t can be wrtten by p(x t)= K w η(x t,μ,q ) (1) =1 where K s the number of Gaussan dstrbutons, w s the weght of the -th Gaussan component, μ s the mean value, Q s the covarance matrx and η(x t,μ,q )sthe -th Gaussan probablty densty functon, whch s represented by 1 η(x t,μ,q )= e 1 (2π) D 2 Q 1 2 (xt μ ) T Q 1 (x t μ ). (2) 2 In order to avod the complex matrx computaton, let Q = σ 2 I. Consderng the hgh resoluton of the spot mage, no more than 3 Gaussan models are ntalzed. Each pxel s frstly classfed as ether background or a foreground pxel by the models. The K dstrbutons are ordered based on the ftness value w /σ. The frst B dstrbutons are selected as a model of the background, whch are estmated by ( b ) B = arg mn w >T (3) b where T s the threshold of background weght value. The Gaussan model that matches the current pxel value wll be updated by the followng formulae. ŵ t+1 ˆμ t+1 ˆQ t+1 =ŵ t + α ( ) 1 ŵ t =ˆμ t + α ( ) x t+1 ˆμ t = ˆQ t + α ( (x t+1 ˆμ t+1 =1 )(x t+1 ˆμ t+1 ) T ˆQ t ). (4) If no Gaussan model matches the current pxel value, then the least probable model s replaced by the formulae w t+1 = α, μ t+1 = x t and Q t+1 = σ 0. If the maxmum number of components s reached, the component wth the smallest weght wll be dscarded. However, the foreground regon segmented by the Gaussan mxture model s not suffcent to be clear. Frstly, a bnary mage s obtaned through adaptve threshold segmentaton, n whch the value of each pxel s compared wth the weghted average value around the pxel. { 255, f (Ib (u, v) -I b(u, v)) >I T I b (u, v) = (5) 0, otherwse where I b(u, v) s the convoluton of pxel by one Gaussan kernel operator and I T s the constant value. Then, the regon-based nose cleanng s appled, such as the morphologcal open operaton ncludng the eroson by the structurng element A then the dlaton by the structurng element B, denoted by (X A) B. (6) Ths openng operaton can generally remove very small regons, elmnate thn protrusons and smooth the contour

4 P. C. Zhang and D. Xu / Trackng and Gudng Multple Laser Beams for Beam and Target Algnment 603 of the spot. Then, the shape flter s used to classfy the spots. So far, all spots are numbered and each one has ts own ID number. 3.2 Spots trackng stage In ths subsecton, the contour trackng algorthm based on the chan code s ntroduced. Many applcatons usng chan code representaton have been reported n prevous lteratures. The frst approach for representng the arbtrary geometrc curve usng chan code was proposed by Freeman [21]. In ths approach, an arbtrary contour can be represented by a sequence of small vectors of unt length and a set of possble drectons. There are two standard code defntons used to represent contour, ncludng the crack code based on 4-connectvty and the chan code based on 8-connectvty. In ths work, the real-tme and robustness are very mportant for multple spots trackng. Here, the 8-connectvty chan code s employed for representng the contour of spot, whch s based on the connectvty of neghborng pxels [22]. Fg. 3 llustrates the changes of 8 possble absolute drectons, whch are ndcated by numbers. 0 ndcates the drecton change to the east, 1 ndcates the drecton change to the northeast, 2 ndcates the drecton change to the north, 3 ndcates the drecton change to the northwest, 4 ndcates the drecton change to the west, 5 ndcates the drecton change to the southwest, 6 ndcates the drecton change to the south and 7 ndcates the drecton change to the southeast. A change between two consecutve chan codes means a change n the drecton of the contour. Thus, each spot scontourcanbecodedby the chan code n the mage space. Fg. 3 Spot s contour trackng based on chan code Generally, the basc prncple of the trackng based on chan code of 8 drectons s to separately encode each connect component. It can be dvded nto 3 steps as follows. Frstly, the ntal center of the tracked spot should be specfed so that the frst border pxel can be found. The ntal poston can be obtaned from the detecton procedure. Startng from the frst center, the frst border pxel can be detected along the u-axs drecton. In order to search a new contour n a small regon of nterest (ROI) when the trackng s faled, one sutable search wndow s set. A copy of the current spot should be created n order to determne whether the current spot s smlar to the next one. If no smlar spot s detected, ths copy s set to the current spot. Secondly, the next border pxel s consdered by updatng the pxel coordnates along the 8 drectons. The coordnate transformaton rules are descrbed as u +1 = u +1, v +1 = v, f d =0 u +1 = u +1, v +1 = v 1, f d =1 u +1 = u, v +1 = v 1, f d =2 u +1 = u 1, v +1 = v 1, f d =3 u +1 = u 1, v +1 = v, f d =4 u +1 = u 1, v +1 = v +1, f d =5 u +1 = u, v +1 = v +1, f d =6 u +1 = u +1, v +1 = v +1, f d =7 where u, v are the coordnates of the current pxel and d s the ndcaton of the drecton change. Meanwhle, the spot features, such as the center of mass, surface, boundng box and mage moments, are calculated by ther correspondng ncrements. Ths contnues untl the encoder returns to the startng border pxel. Compared to other contour-based methods, t does not go through all the neghbors of the pxel, so faster. So far, the boundares of the spot scontour are represented by the chan codes. Thrdly, f the trackng s faled, a search area wll be requred. Thanks to the short mage samplng nterval, the movng dstance of the spot durng two consecutve frames s lmted n a small range. So the dynamc wndow technque s appled to searchng. Once the spot s found n the new frame, the center of the dynamc wndow wll be updated. Therefore, f the trackng s faled or the spots go out of the feld of vew, the searchng regon n the next frame can be confned n ths dynamc wndow. If the spot s stll not detected n ths small wndow, the searchng wndow wll be changed to the entre mage regon [23]. In addton, when the trackng s faled, not every pxel wll be treated as a startng pont for contour detecton, but a large searchng step s separately set along the u-axs and v-axs drectons. If the new contour s smlar enough to the prevous one, the chan codes of spot contour wll be updated. Ths trackng procedure contnues untl stoppng. Durng the multple spots trackng procedure, f the relatve chan codes of spot contour are consdered for matchng, t must be ndependent of the choce of the frst boundary pxel. Usually, the normalzed dfferental chan codes nstead of the relatve ones are used to represent the contour boundary and ths can be computed by subtractng each element from the prevous one [24]. However, the chan codes of spot contour are usually senstve to nose, so the shape matchng by chan code s mpractcal. In ths work, consderng the real-tme and smplfcaton, the mage moments of spot have been employed for shape matchng. The Hu nvarant moments that are nvarant to translaton, rotaton and changes n scale are adopted to evaluate the resemblance of two contours [25].

5 604 Internatonal Journal of Automaton and Computng 12(6), December 2015 The mage moments are obtaned by m pq = u p v q I(u, v) (7) u v where the order of the moment s p+q, thecentralmoments μ pq are calculated by μ pq = (u ū) p (v v) q I(u, v), ū = m10, v = m01. m 00 m 00 u v (8) n shapes. So the mass centers of spots calculated by the common contour become naccurate. In the prevous lteratures, ths case s defned as a flterng and data assocaton problem. Then, the normalzed central moment η pq can be determned by η pq = μpq, λ = p + q, p+ q 2. (9) μ λ 00 2 Further, 7 Hu nvarant moments that contan mage momentsuptoorder3areshownas h 1 = η 20 + η 02 h 2 =(η 20 η 02) 2 +4η 2 11 h 3 =(η 30 3η 12) 2 +(3η 21 η 03) 2 h 4 =(η 30 + η 12) 2 +(η 21 + η 03) 2 h 5 =(η 30 3η 12)(η 30 + η 12)((η 30 + η 12) 2 3(η 21 + η 03) 2 )+ (3η 21 η 03)(η 21 + η 03)(3(η 30 + η 12) 2 (η 21 + η 03) 2 ) h 6 =(η 20 η 02)((η 30 + η 12) 2 (η 21 + η 03) 2 )+4η 11(η 30+ η 12)(η 21 + η 03) h 7 =(3η 21 η 30)(η 30 + η 12)((η 30 + η 12) 2 3(η 21 + η 03) 2 )+ (3η 21 η 03)(η 21 + η 03)(3(η 30 + η 12) 2 (η 21 + η 03) 2 ). (10) The Hu moments advantages of nvarance to poston, sze and orentaton make the spot matchng become practcal. Therefore, the 7 Hu moments combned wth the spot s wdth, heght and surface are appled to shape matchng. However, the spot s shape s not very stable, but slghtly changes n a small range. So a certan changng range s set for each moment. If the changes of moments are beyond a certan range, the correspondng shape wll be rejected. 3.3 Occluson handlng Importantly, the trackng should contnue even n the event that the spot s partally or even completely occluded. Especally, the mxng phenomenon between dfferent spots s very common. When multple spots move to near the desred postons n the gudng procedure, occluson may occur. In ths case the spots may even completely merge for a long tme. Here, the boundng box dstance s used for determnng whether the spots merge or splt, as shown n Fg. 4. It s more stable as opposed to the Eucldean dstance between the centers of spots. Therefore, occluson detecton s performed by determnng the dstance between the boundng boxes of dfferent spots. When occluson occurs, multple spots mx each other, whch nduces sudden changes Fg. 4 Boundng box dstances between dfferent spots The most common expresson of the flterng and data assocaton process s the state space approach, whch can model the dscrete dynamc system by lnear dfference equatons [26]. Kalman flter technque can predct moton nformaton and reduce the computatonal complexty, so t s employed to predct and update the spot sfeatures n ths work. The features of the spot are defned by the state sequences x k (k =0, 1, ), whch are specfed by the dynamc equatons x k = f k (x k 1,v k ). Theavalablemeasurements z k (k =1, 2, ) are related to the correspondng states through the measurement equatons z k = h k (x k,n k ). The functons v k and n k are the nose sequences, whch are assumed to be ndependent and are of Gaussan dstrbuton wth zero mean. The functons f k and h k are lnear. The dynamc equatons are defned by x k = Fx k 1 + v k and the measurement equatons are defned by z k = Hx k + n k. F s the transton matrx and H s the measurement matrx. The contour trackng technque combned wth the Kalman flterng s appled when occluson occurs. The weghted sum of the measured poston and the predcted poston s used for determnng the spot s poston n the next frame. Typcally, a Kalman flter can be dvded nto the predcted phase and the corrected phase [26]. In the frst phase, a pror state estmaton of the spot at tme k s evolved from the state at tme k 1 accordng to x k k 1 = F k x k 1 k 1 + B k u k 1 (11) where F k s the state transton matrx, B k s the control matrx whch s not used n ths work. Meanwhle, a posteror error covarance matrx can be estmated by P k k 1 = F k P k 1 k 1 F T k + Q k (12) where Q k s the process nose covarance matrx. In the update phase, the state estmaton s updated by the formula as x k k 1 = x k k 1 + K k (z k H k x k k 1 ) (13) where H k s the measurement matrx, K k s the optmal Kalman gan matrx, whch s a functon of the relatve

6 P. C. Zhang and D. Xu / Trackng and Gudng Multple Laser Beams for Beam and Target Algnment 605 certanty of the measurements and current state estmate. Wth a hgh gan, the flter places more weght on measurement. Wth a low gan, the flter follows the predcton more closely. At the extreme, a gan of zero causes the measurement to be gnored. The gan can be set up to acheve better performance and s expressed by K k = P k k 1 H T k (H k P k k 1 H T k + R k ) 1 (14) where R k s the measurement nose covarance matrx and the posteror estmate covarance matrx s updated by P k k =(I K k H k )P k k 1. (15) The predcton step and correcton step are executed recursvely. From (15), the Kalman gan s nversely proportonal to the measurement covarance matrx R k. Therefore, the smaller R k s, the greater the gan becomes and the hgher the weght of the predcted value becomes. Lkewse, the closer the pror estmate error P k k 1 gets to zero, the hgher the weght of the measurement becomes. In Kalman flter algorthm, the movng model of spot should be constructed. Because of the small changes on spot s surface n consecutve frames, only 4 nput states are taken nto account, whch are the postons and the veloctes of the spot along the u-axs and v-axs drectons n the mage space. The poston and velocty of the spot are evolved from the state at tme k 1accordngto u k u k 1 v k Δu k = v k 1 (16) Δu k 1 Δv k Δv k 1 where Δu k and Δv k are the poston ncrements durng the contnuous samplng tme along the u-axs and v-axs drectons, and they can replace the veloctes, u k and v k are the pxel coordnates of the mass centers of spot. The state transton matrx s a 4 4 one, the measurement matrx s a2 4 one, the measurement nose covarance matrx R k s a 2 2 dagonal one, the process nose covarance matrx Q k s also a 4 4 dagonal one, and the posteror error estmate covarance matrx P k s a 4 4one. When occluson occurs, the entre contour ncludng the merge boundares can be detected by the chan code. At ths moment, the poston of spot s the weghted sum of the mass centers of the common contour and the predcton poston. Importantly, the practcal mplementaton of Kalman flter s often dffcult to obtan a good estmaton of the nose covarance matrces Q k and R k. The autocovarance least squares technque was used to estmate the covarance [27]. More practcally, n order to reduce the computaton cost, an occluson rate s defned to determne the values of nose covarance, shown as Ns N(t), f occluson occurs ζ(t) = (17) 1, otherwse where N s s the number of pxels of spot before occluson, N(t) s the number of pxels n the common area of multple spots under occluson. So, N(t) s greater than N s. Here, t s assumed that the nose covarance s nversely proportonal to occluson rate ζ(t). When occluson occurs, the measurement nose covarance becomes greater, and then the weght of the predcted value becomes hgher. As the common area of multple spots becomes smaller, the occluson rate becomes greater and the nose covarance becomes smaller, so the weght of the measurement becomes hgher. Accordng to the occluson rate, the Kalman flter s adjusted adaptvely. 4 Multple spots gudng based on vsual feedback The objectve of the spots trackng s to gude the laser spots to accomplsh the beam and target algnment. From the prevous secton, the postons of spots are obtaned n real-tme, so the poston errors n pxels between the current and the desred ones are known. Fg. 5 llustrates the desred postons of 3 spots. Note that due to the pose relatonshp between the target algnment sensor and the hgh-reflectvty mrror, the two axes of the sub-coordnate system of each spot are not perpendcular to each other. The control objectve s to mnmze the errors by choosng an approprate control vector at each samplng tme. The control scheme for elmnatng the poston devatons of the spotssthedscretencremental proportonal ntegral (PI) controller. The lnear control law s gven by [ ] ([ ] [ [ ] Δu x(k) Δx(k) Δx(k 1) Δx(k) =K p ])+K Δu y(k) Δy(k) Δy(k 1) Δy(k) (18) where parameters Δu x(k) andδu y(k) are the output of the PI controller at the k-th control cycle, Δx(k) andδy(k) are the postons errors n Cartesan space at the k-th control cycle. K p s the proportonal factor, whch s a dagonal matrx. K s the ntegral factor, whch s also a dagonal matrx. The Zegler-Nchols method s employed to tune the PI parameters. The control structure s shown n Fg. 6. The coordnates of the mass centers n pxels are obtaned from the camera system. Then, the errors Δu(k) andδv(k) n the mage space between the current poston and the desred one are calculated. The movements of the spot are acheved by the rotatonal motons of the two servo motors wth a hghreflectvty mrror along the ptch and yaw drectons. The rotatonal ncrements of two motors Δx(k) andδy(k) can be acqured by [ ] [ ][ ] Δx(k) p u 0 Δu(k) = (19) Δy(k) 0 p v Δv(k) where parameters p u and p v are the proportonal relatonshp between the translatonal motons of the spot n pxels

7 606 Internatonal Journal of Automaton and Computng 12(6), December 2015 Fg. 5 The ntal and desred postons of 3 spots n the beam and target algnment procedure and the axes of the subcoordnate system of each spot are not perpendcular to each other and the rotatonal motons of the motor. The values of p u and p v can be separately calbrated off-lne by multple motons, as shown n (20). N =1 Δx Δm p u = N, pv = (20) N where Δx s the ncrement of the motor s moton only along the ptch drecton and Δm s the correspondng ncrement of the spot s moton n the mage plane. Smlarly, Δy s the ncrement of the motor s moton only along the yaw drecton and Δn s the correspondng ncrement of the spot s moton n the mage plane. Meanwhle, the correspondng unt vector can be obtaned by lne fttng. In the control procedure, the rotatonal ncrements Δx(k) andδy(k) are obtaned by the ncrements n mage space, whch can be treated as the nput of PI controller. Fnally, the outputs Δu x(k) andδu y(k) are used as the nput to the motor s controller. N =1 5 Experment and analyss 5.1 Experment system Δy Δn The whole expermental confguraton conssts of 3 beam and target algnment subsystems, as shown n Fg. 7. Each laser subsystem of laser s composed of a laser transmtter, a set of convergent lenses, a convex lens wth a translatonal moton platform and a hgh-reflectvty mrror wth two rotatonal motons platform. Before passng through the aperture, the laser needs to be adjusted manually. Then, t s focused agan by a convex lens wth a translatonal moton platform. Fnally, the laser s reflected by a hgh-reflectvty mrror whch s drven by one ptch motor and one yaw motor. In addton, the camera system could smultaneously mage the laser spots and the cylndrcal target, whch could provde the poston feedback to ensure the algnment accuracy. Therefore, the magng camera and the gudng motors ncludng the ptch and yaw ones consttuted one vsual servong system. Importantly, after beng reflected by the hgh-reflectvty mrror, the laser s reflected agan by the mrror on the target algnment sensor, and then t s captured by the camera system. So, the charge-coupled devce (CCD) plane and the target s end surface should be conjugate relatve to the plane of target algnment sensor so that the laser could shoot the target s surface after the target algnment sensor s removed. It effectvely prevents multple laser beams from drectly shootng the target. The above manpulators are the Aerotech s motor named ANT95-50-XY n the translatonal drecton, one named ANT R n the yaw drecton and one named ANT- 20G-90 n the ptch drecton. The accuracy of the rotatonal motor s ±50 μrad and the resoluton s 0.25 μrad. In order to guarantee that the reflected spots are not out of vew, the rotatonal angle of each motor s lmted wthn rad. The magng detector s GRAS-50S5M-C from Pont Grey Research. The camera could run at 15 frames per second for 2448(H) 2048(V) n pxels. The software system has two processes ncludng a user nterface process and an applcaton one. The nterprocess communcaton s acheved by shared memory and the socket technology. Meanwhle, mult-threadng technology s used n multple spots trackng procedure. The experment was mplemented on a PC (Intel GHz). 5.2 Expermental result and analyss In the spots detecton experments, the nose varance was set to 6.4, the ntal weght value was 0.04, the varance threshold was 0.64, the threshold of background weght value was 0.9, and the learnng rate was set to The procedure of spots segmentaton based on the Gaussan mxturemodelsshownnfgs.8(e)to8(g). Asacontrast, the procedure of statonary background subtracton s exhbted n Fgs. 8 (b) to 8 (d). Compared to the statonary background subtracton, the segmentaton results based on GMM are more stable and precse, especally n the face Fg. 6 Control archtecture of spots gudng based on vsual feedback

8 P. C. Zhang and D. Xu / Trackng and Gudng Multple Laser Beams for Beam and Target Algnment 607 of the complex beam and target mages or the rapd movement of spot. Then, the adaptve threshold operaton, the morphologcal open operaton ncludng the eroson wth a 3 3 operator and the dlaton wth a 5 5 operator are used to clean nose. Fg. 7 Expermental system spot and ms for the thrd spot. The real-tme contour trackng technque based on the chan code could meet the requrement of the expermental system. The experments on handlng the problem of spots mxng have been conducted. As dscussed n Subsecton 3.3, whether the spots merge wth each other s determned by the boundng box dstances. In the Kalman flter algorthm, the ntal values of the dagonal elements of matrx R k are set to 3, the ntal values of the dagonal elements of matrx Q k are 1e 6 and the ntal values of the dagonal elements of matrx P k are 0.1. When multple spots mx wth each other n the gudng procedure, the center of the searchng wndow calculated by the common contour would have a sudden shft. However, thanks to the predcton mechansm based on Kalman flter, the sudden shft would be decreased. A range of trackng results combned wth Kalman flter predcton s shown n Fgs. 9 (a) 9(f), n whch the trajectory of the trackng center s marked wth the sold lne. Fg. 10 exhbts the trackng results of 3 spots n the face of mergng and splttng. The proposed algorthm s able to track multple spots n the presence of partal or complete occluson, whch ncreases the feedback accuracy. Fg. 8 The spot segmentatons based on statonary background subtracton and Gaussan mxture model. (a) The beam and target mages; (b) (d) The spot after the statonary background subtracton, the spot after the threshold segmentaton and the spot after the morphologcal operaton, respectvely; (e) (g) The spot after subtracton based on GMM, the spot after adaptve threshold segmentaton and the spot after the morphologcal operaton, respectvely; (h) The mage segmentaton and contour extracton of 3 spots. In ths work, 3 laser beams are used for trackng and gudng. The frst laser spot s about 51 pxels n wdth, 49 pxels n heght and pxels n surface. The second spot s about 70 pxels n wdth, 65 pxels n heght and pxels n surface. The thrd spot s about 64 pxels n wdth, 64 pxels n heght and pxels n surface. In the trackng procedure, f the trackng s faled, the sze of searchng wndow would be set to 15 tmes the spot ssze and the correspondng searchng steps along u-axs and v- axs drectons would be respectvely set to 0.75 tmes the spot s wdth and heght. In addton, the trackng tmes have been tested. The average tmes spent on searchng the contour based on chan code are ms for the frst spot, ms for the second spot and ms for the thrd spot. The whole trackng tmes ncludng the spot segmentaton are ms for the frst spot, ms for the second Fg. 9 The trackng mage sequences under two spots mxng, n whch the frst spot remaned n moton, merged wth the second spot and then splt durng ts movement. The second spot remaned motonless. (a) (f) Image sequence frames 1 502, 1 520, 1 531, 1 550, 1 571, and are shown. Fg. 10 The trackng results n the case of 3 spots mxng. These are the mages sequences for the 3 laser spots n the cases of mergng, splttng, partally occludng and completely occludng. (a) (f) Image sequence frames 621, 1 005, 1 032, 1 050, 1 061, and are shown, n whch the entre contour s detected under the occluson.

9 608 Internatonal Journal of Automaton and Computng 12(6), December 2015 In order to show the better performance of the proposed trackng algorthm, the comparatve experments have been carred out as follows. The most popular shape representaton s the shape context proposed n [19]. Yet n the face of the condtons that the strct requrement of real-tme, the hgh resoluton mage and multple targets, many complex contour-based trackng algorthms are dffcult to be used n our expermental system. Shape context has been employed n the contour-based trackng. Frstly, the spot s represented by a set of ponts sampled from the contour. Then, the shape context s calculated by the log-polar hstograms (5 bns for logr and 12 bns for θ, whch are the same as [28]). The pont correspondences are obtaned by the matchng cost functon wth χ 2 dstance, as shown n Fg 11. Fnally, the smlarty based on shape dstance could be obtaned. The translaton and rotaton between two frames are also calculated by the affne transformaton. The contour of each spot s saved as the pror shape before matchng. Each spot could be recognzed by shape smlarty between the current shape and the pror one. Fg. 12 reveals the tmes of shape matchng between each spot s shape n arbtrary frame and the correspondng pror shape. The average matchng tmes are ms and ms. It s tme-consumng and hard to be accepted. Furthermore, when two spots mx wth each other, the shape matchng based on shape context wll fal. Therefore, the trackng method based on shape context matchng s not practcal for our system, although the shape context represent the spot s contour better. Consderng the real-tme requrement, the contour trackng scheme based on the chan code detecton and the recognton based on the nvarant mage moments are more practcal for our expermental system. trackng 2D blobs, such as the spots n ths work. The mean shft method s a nonparametrc statstcal one for trackng wth an sotropc kernel [29, 30]. Here, the kernel functon s smplfed as a rectangular one. In the trackng procedure, the mass center s computed. Meanwhle, a mean shft vector s determned and the searchng wndow center shfted to the current mass center accordngly. Ths procedure s repeated untl the number of teratons reaches the specfed one or untl the shftng dstance was less than the specfed one. Snce the spot s shape changed not too much, here the sze and orentaton of the trackng wndow do not change durng the trackng procedure. In addton, the searchng wndow s set to a rectangle regon of n pxels. The average searchng tme s ms and the whole trackng tme ncludng pre-processng s ms. Compared to the proposed trackng scheme, the tmes are almost smlar. However, t s hard to contnue trackng when multple spots are close to each other, as shown n Fg. 13. Although ths method has the better trackng tme performance, t cannot handle the case of occluson. Therefore, ths dsadvantage lmted ts applcatons on multple spots trackng. Fg. 13 Trackng mage sequences by the mean-shft. When two spots mx wth each other, the trackng s faled. (a) (c) Image sequence frames 912, 937 and 979 are shown. Fg. 11 frames Shape matchng by the shape context between two Fg. 14 Trackng mage sequences by polar coordnate scannng. When two spots mx wth each other, the trackng was faled. (a) (c) Image sequence frames 628, 645 and 682 are shown. Table 1 The comparatve experment results Spots trackng Average trackng tme Chan code ms( ms) Mean shft ms( ms) (ROI: ) Polar coordnate ms( ms) (ROI: ) Shape matchng Average matchng tme Invarant moments Very short Shape context ms Fg. 12 Tme of shape matchng by the shape context n the sequental frames In the second comparatve experment, the mean shft algorthm was employed, whch s an effcent technque for In addton, on the condton that the spot sshapesapproxmately treated as a crcle, the contour can be detected by scannng along the polar coordnate. The least square method combned wth random samplng consensus technque s utlzed to ft crcle wth the contour ponts. Then,

10 P. C. Zhang and D. Xu / Trackng and Gudng Multple Laser Beams for Beam and Target Algnment 609 the trackng wndow s updated by the new center of crcle. The searchng wndow s also set to a rectangle regon of n pxels. The average tme of detectng the spot contour s ms and the whole trackng tme ncludng pre-processng s ms. By contrast, not only s ts realtme performance s not better, but also t cannot handle the case n whch multple spots are very near each other, as shown n Fg. 14. In addton, when the spots shape change suddenly and cannot be treated as a crcle, the trackng wll be faled. The runnng tme of the comparatve experment are concluded n Table 1. Through many tests, the proposed contour trackng scheme proved to be robust under the adverse condtons, ncludng rapd motons, large-scaled changes n velocty, slght vbratons of target and slght shakng of the laser spot. Therefore, the proposed trackng method s optmal and practcal. Fnally, a seres of mages that show 3 spots gudng wth PI controller are dsplayed n Fgs. 15 (a) 15 (f). Three threads for gudng 3 laser beams execute synchronously. Three spots mx wth each other when they are close to the target area. In ths case, the contour trackng method combned wth the Kalman flter s appled to determne the current poston of each spot. The weght of the measurement s adjusted by the occluson rate. Meanwhle, the poston error of each spot between the current one and the desred one s treated as a feedback. Here, the vald shootng objectve s a crcular regon wth 150 pxels n dameter and the desred poston s the center of the target send surface. The detecton, trackng and gudng strategy have realzed the hgh accurate shootng. The average tme of 3 spots gudng s wthn 38 s, whch meets the expermental system s requrement. 6 Conclusons In ths paper, the man contrbuton s that a new detectng, trackng and gudng strategy for multple laser spots based on beam and target algnment expermental system are developed. Frst, the laser spots are segmented and recognzed by the contour-based analyss. Then, the contour trackng technque based on the chan code and Kalman flter predcton s proposed, n whch the shape matchng scheme s employed to dstngush dfferent spots. Fnally, a vsual feedback system s ntroduced n order to gude 3 laser spots to shoot the desred poston of the target. The experment results show the rapd, accurate and robust performance of the laser shootng. In future, the new technque can be further combned wth more data assocaton approaches, such as multple hypotheses trackng, especally, when multple spots mx together or completely nterfere wth each other. Fg. 15 The gudng procedure of 3 spots for beam and target algnment. (a) (f) Image sequence frames of the spots gudng 1 109, 1 183, 1 282, 1 638, and are shown. The lnes are the drectons of spot moton only drven by the yaw or ptch motor. And n the experment, the lnear nterpolaton algorthm was appled to two motors. References [1] S.J.Boege,E.S.Blss,C.J.Chocol,F.R.Holdener,J. L. Mller, J. S. Toeppen, C. S. Vann, R. A. Zacharas. NIF pontng and centerng systems and target algnment usng a 351-nm laser source. In Proceedngs of the SPIE 3047, SPIE, Pars, France, pp , [2] P. D Ncola, D. Kalantar, T. McCarvlle, J. Klngmann, S. Alvarez, R. Lowe-Webb, J. Lawson, P. Datte, P. Danforth, M. Schneder, J. M. D Ncola, J. Jackson, C. Orth, S. Azevedo, R. Tommasn, A. Manuel, R. Wallace. Beam and target algnment at the natonal gnton faclty usng the target algnment sensor. In Proceedngs of the SPIE 8505, SPIE, San Dego, USA, pp. 1 9, [3] C. Stauffer, W. E. L. Grmson. Adaptve background mxture models for real-tme trackng. In Proceedngs of the IEEE Computer Socety Conference on Computer Vson and Pattern Recognton, IEEE, Fort Collns, USA, vol. 2, pp , [4] Z. Zvkovc. Improved adaptve Gaussan mxture model for background subtracton. In Proceedngs of the 17th Internatonal Conference on Pattern Recognton, IEEE, Cambrdge, UK, vol. 2, pp , [5] A.Ylmaz,O.Javed,M.Shah.Objecttrackng: Asurvey. ACM Computng Surveys, vol. 38, no. 4, pp. 1 45, [6] M. Yokoyama, T. Poggo. A contour-based movng object detecton and trackng. In Proceedngs of the 2nd Jont IEEE Internatonal Workshop on Vsual Survellance and Performance Evaluaton of Trackng and Survellance, IEEE, Tokyo, Japan, pp , [7] N. Paragos, R. Derche. Geodesc actve contours and level sets for the detecton and trackng of movng objects. IEEE Transactons on Pattern Analyss and Machne Intellgence, vol. 22, no. 3, pp , [8] M. Fussenegger. Level Set Based Segmentaton and Trackng of Multple Objects, Ph. D. dssertaton, Graz Unversty of Technology, Austra, pp , [9] G. Wagenknecht. A contour tracng and codng algorthm for generatng 2D contour codes from 3D classfed objects. Pattern Recognton, vol. 40, no. 4, pp , [10] M. W. Ren, J. Y. Yang, H. Sun. Tracng boundary contours n a bnary mage. Image and Vson Computng, vol. 20, no. 2, pp , 2000.

11 610 Internatonal Journal of Automaton and Computng 12(6), December 2015 [11] Y. Rath, N. Vaswan, A. Tannenbaum, A. Yezz. Trackng deformng objects usng partcle flterng for geometrc actve contours. IEEE Transactons on Pattern Analyss and Machne Intellgence. vol. 29, no. 8, pp , [12] A. Senor, A. Hampapur, Y. L. Tan, L. Brown, S. Pankant, R. Bolle. Appearance models for occluson handlng. Image and Vson Computng, vol. 24, no. 11, pp , [13] M. Isard, J. MacCormck. Bramble: A Bayesan multpleblob tracker. In Proceedngs of the 8th IEEE Internatonal Conference on Computer Vson, IEEE, Vancouver, Canada, vol. 2, pp , [14] J. MacCormck, A. Blake. A probablstc excluson prncple for trackng multple objects. Internatonal Journal of Computer Vson, vol. 39, no. 1, pp , [15] L. Bazzan, D. Blos, V. Murno. A comparson of mult hypothess Kalman flter and partcle flter for mult-target trackng. In Proceedngs of the CVPR Workshop on Performance Evaluaton of Trackng and Survellance, Mam, USA, pp , [16] A. Ylmaz, X. L, M. Shah. Contour-based object trackng wth occluson handlng n vdeo acqured usng moble cameras. IEEE Transactons on Pattern Analyss and Machne Intellgence, vol. 26, no. 11, pp , [17] N. Sebe, M. Lew. Robust shape matchng. In Proceedngs of Internatonal Conference on Image and Vdeo Retreval, London, UK, pp , [18] D. S. Zhang, G. J. Lu. Revew of shape representaton and descrpton technques. Pattern Recognton, vol. 37, no. 1, pp. 1 19, [19] S. Belonge, J. Malk, J. Puzcha. Shape matchng and object recognton usng shape contexts. IEEE Transactons on Pattern Analyss and Machne Intellgence, vol. 24, no. 4, pp , [20] P. KaewTraKulPong, R. Bowden. An mproved adaptve background mxture model for real-tme trackng wth shadow detecton. In Proceedngs of the 2nd European Workshop on Advanced Vdeo Based Survellance Systems, Sprnger, Berln, Germany, pp , [21] H. Freeman. On the encodng of arbtrary geometrc confguratons. IEEE Transactons on Electronc Computers, vol. 10, no. 2, pp , [22] INRIA, Vsual Servong Platform, [Onlne], Avalable: [23] D. Xu, H. W. Wang, Y. F. L, M. Tan. A new calbraton method for an nertal and vsual sensng system. Internatonal Journal of Automaton and Computng, vol. 9, no. 3, pp , [24] M. Sonka, V. Hlavac, R. Boyle. Image Processng, Analyss and Machne Vson, Thomson Learnng, Stamford, USA, pp , [25] M. K. Hu. Vsual pattern recognton by moment nvarants. IEEE Transactons on Informaton Theory, vol. 8, no. 2, pp , [26] G. Bshop, G. Welch, An ntroducton to the Kalman flter, [Onlne], Avalable: TR95-041, UNC-Chapel Hll, USA, [27] M. R. Rajaman, J. B. Rawlngs. Estmaton of the dsturbance structure from data usng semdefnte programmng and optmal weghtng. Automatca, vol. 45, no. 1, pp , [28] Z. Lu, H. Shen, G. Y. Feng, D. W. Hu. Trackng objects usng shape context matchng. Neurocomputng, vol. 83, pp , [29] D. Comancu, V. Ramesh, P. Meer. Kernel-based object trackng. IEEE Transactons on Pattern Analyss and Machne Intellgence, vol. 25, no. 5, pp , [30] R. T. Collns. Mean-shft blob trackng through scale space. In Proceedngs of the IEEE Computer Socety Conference on Computer Vson and Pattern Recognton, IEEE, Pttsburgh, USA, vol. 2, pp , Peng-Cheng Zhang receved the B. Sc. degree n mechancal and automotve engneerng from Hefe Unversty of Technology, Chna n 2008, and the M. Sc. degree n mechancal and automotve engneerng from South Chna Unversty of Technology, Chna n He s currently a Ph. D. degree canddate n the Research Center of Precson Sensng and Control, Insttute of Automaton, Chnese Academy of Scences, Chna. Hs research nterests nclude target trackng and postonng, vsual servong, and computer vson. E-mal: pengcheng.zhang@a.ac.cn (Correspondng author) ORCID D: De Xu receved the B. Sc. and M. Sc. degrees n control scence and engneerng from Shandong Unversty of Technology, Chna n 1985 and 1990, respectvely, and the Ph. D. degree n control scence and engneerng from Zhejang Unversty, Chna n Snce 2001, he has been wth the Insttute of Automaton, Chnese Academy of Scences, Chna, where he s currently a professor n the Research Center of Precson Sensng and Control. Hs research nterests nclude robotcs and automaton, the control of robots, such as vsual control and ntellgent control. E-mal: de.xu@a.ac.cn