Vol. 2, No. 3, Page 185-195 Copyrght 2008, TSI Press Prnted n the USA. All rghts reserved Optmzed Regon Competton Algorthm Appled to the Segmentaton of Artfcal Muscles n Stereoscopc Images Rafael Verdú-Monedero, Juan Morales-Sánchez, and Jorge Larrey-Ruz Department of Informaton Technologes and Communcatons Techncal Unversty of Cartagena {rafael.verdu, juan.morales, jorge.larrey}@upct.es Receved 29 June 2007; revsed 30 November 2007, accepted 25 December 2007 Abstract Ths paper addresses the mplementaton of the Regon Competton algorthm for segmentng stereoscopc vdeo sequences. The segmentaton performed by ths algorthm s an essental stage for the 3D characterzaton of artfcal muscles. Image sequences are acqured by a two-cam computer vson system. The am of ths work s to come up wth optmal and effcent segmentaton of these mages; the nformaton obtaned from the frst segmented frame of the vdeo sequence s used for segmentng the next frame and so on. Redundancy between stereoscopc pars of mages s also used to optmze the segmentaton. In ths paper, the Regon Competton algorthm s descrbed and a specfc mplementaton s detaled. It also shows how to solve partcular problems of stereoscopc vdeo segmentaton. Fnally, results yelded from experments are presented and dscussed. Keywords mage segmentaton, stereo vson, artfcal muscles 1. INTRODUCTION In ths paper, an algorthm specfcally desgned for the segmentaton of stereoscopc mage sequences s descrbed. The mages to be segmented are sequental pctures whch contan artfcal muscles n moton. These devces called artfcal muscles [1] are bult wth polymer conductors provdng the devce wth the ablty to transform electrcal energy nto mechanc work. When the artfcal muscles are exposed to an electrcal ntensty, they can bend or stretch themselves dependng on the sgn of that ntensty. There's a wde range of applcatons for artfcal muscles, from human prosthess to motor systems for small robots, and new ones are appearng every day. Inspred by natural muscles, polymer-made muscles are excted by electrcal ntenstes and ther movement s due to dmensonal varatons of the polymers caused by electrochemcal reactons. Ths stuaton creates the need of a system whch measures small varatons of the artfcal muscles and whch controls ther behavour under dfferent condtons. Observaton and characterzaton of these devces are essental tasks for ts nvestgaton and mprovement. In [2][3] a new method based on actve contours s developed n order to track the moton of the muscle and obtan ts 3D characterzaton. Two dgtal cameras take orthogonal pctures of the muscle n moton. Usng stereoscopc vson technques and dgtal mage processng we get an automatc way to extract the parameters we're lookng for, [4]. Ths method s based on artfcal vson and mage processng. The dfferent stages are:
Camera control and stereoscopc mages acquston (see Fgure 1). Image Processng, whch ncludes mage segmentaton (.e., detecton of the muscle n the mages and separaton from the background) and trackng usng a 3D actve contour model [5]. Data mnng: parameters of movement, energy of curvature, etc. Ths work focuses on the mage processng stage, whch s a key stage for the whole process snce t s mpossble to characterze the muscle f the obtaned parameters are naccurate. In ths paper we descrbe the proposed algorthm that carres out the optmzed segmentaton. The startng pont for our segmentaton algorthm s Zhu and Yulle's Regon Competton algorthm [6]. Fgure 1. Stereoscopc magng system for characterzaton of artfcal muscles. On the left, a par of acqured stereo-frames. On the rght, the arrangement of the cameras and the muscle s shown. Segmentaton technques When analysng the objects n mages t becomes essental to dstngush between the objects of nterest (target) and the rest ' (also referred to as the background). Before developng our algorthm, a wde study of several segmentaton technques has been carred out: edge detecton technques, thresholdng, regon growng, actve contours, etc. All of these technques have somethng n common: they formulate some hypotheses about the mage, test features, and make decsons by applyng thresholds explctly or mplctly. The man dfference between these dfferent approaches les n the domans on whch the hypotheses, tests, and decsons are based. Edge based technques only make use of local nformaton and cannot guarantee contnuous closed edge contours. Actve contour models make use only of nformaton along the boundary and requre good ntal estmates to yeld correct convergence. An advantage of regon growng s that t tests the statstcs nsde the regon; however t often generates rregular boundares and small holes. In addton, all of these three methods lack of a global crteron for segmentng the entre mage. In contrast, global optmzaton approaches based on energy functons or Bayesan and MDL (Mnmum Descrpton Length) crtera often have problems when tryng to fnd ther mnma. 186
Zhu and Yulle presented n [6] a statstcal framework for mage segmentaton usng a novel algorthm whch they called Regon Competton (RC). It s derved by mnmzng a generalzed Bayes/MDL crteron and combnes the attractve geometrcal features of snake/balloon models and the statstcal technques of regon growng. RC uses a samplng wndow whose sze depends on the sgnal to nose rato. The precson of the boundary locaton depends on the sze of these samplng wndows. Lke many other algorthms, the performance of regon competton depends on the ntal condtons, more precsely on the choce of ntal "seeds. Zhu and Yulle showed a crteron for choosng such seeds and also descrbed an mportant domno effect whereby bad seeds are transformed nto good seeds. RC can be generalzed to multband segmentaton (colour mages) and s able to use several descrptors (as materals texture) to carry out the segmentaton. These features avod the key drawback of many approaches to mage segmentaton whch only look for dscontnutes n ntenstes, and whch can gve msleadng results. The work descrbed n ths paper focuses on gray level mages but the number of descrptors can be ncreased n order to segment more complex mages. The Regon Competton algorthm The goal of mage segmentaton s to partton the mage nto subregons wth homogeneous ntensty (colour or texture) propertes whch wll hopefully correspond to objects or object parts. It s assumed that a regon R s consdered to be homogeneous f ts ntensty values are consstent wth havng been generated by one of a famly of prespecfed probablty dstrbutons P ( I ), where denotes the parameters of the dstrbuton. It s also supposed that the entre mage doman R has been ntally segmented nto M pecewse homogeneous underlyng regons R where =1,2,..., M and R = M R, =1 R R j = 0 f j. Let Γ = R be the R boundary of regon, where we defne the drecton of to be counter-clockwse. Let M = 1 R Γ = Γ be the edges or segmentaton boundares of the entre mage. Now consder an MDL crteron (a global energy functonal). Ths gves: E[ Γ, ] = M μ { ds ({ I( x, y) : ( x, y) R } ) + λ} 2 log P, = 1 R (1) where the frst term wthn the braces s the length of the boundary curve R, for regon R. We smply assume that the code length s proportonal to the curve length, where μ s the code length for unt arc length. Snce all edge segments are shared by two adjacent regons, we dvde the frst term by a factor of 2. The second term s the sum of the cost for codng the ntensty of every pxel ( x, y) nsde regon R, accordng to a dstrbuton P( { I( x, y) : ( x, y) R } ). λ s the output of a cost functon whch s related to the code length needed to descrbe the dstrbuton and code system for regon R, and we smply assume that λ s common to all regons. Because the energy E n (1) depends on two groups of varables (the segmentaton Γ and the parameters ), Zhu and Yulle proposed a greedy algorthm whch conssts of two alternatng stages. The frst stage locally mnmzes the energy wth the number of regons fxed. It proceeds by teratng two steps both of whch cause the energy to decrease. The second stage merges regons provded ths decreases the energy. The frst stage ncludes two steps. In the frst step, we fx Γ and we solve for to mnmze the descrpton cost for each regon. In other words, parameters are estmated by maxmzng the condtonal probabltes. In the second step, we fx the 187
parameters of the dstrbuton and do steepest descent wth respect to Γ. Both steps n ths frst stage of the algorthm cause the energy functon to decrease. In addton, the functon s bounded below and so the algorthm s guaranteed to converge to a local mnmum. Ths two step process, however, does not allow us to alter the number of regons. Thus a second stage s added where adjacent regons are merged f ths causes the energy to decrease. Ths s followed by the two step teraton stage agan, and so on. Overall each operaton reduces the energy and so a local mnmum s reached. The whole process, as descrbed n [6], goes as follows: 1. Intalze the segmentaton; put N seeds randomly across the mage; all background s treated as a sngle regon wth unform probablty dstrbuton. 2. Fx the boundary Γ ; compute the parameters by maxmzng P( I : ). 3. Fx, move the boundary Γ by mnmzng the energy functon. When two seed regons meet, an edge s formed at ther common boundary, and then these two regons compete along ths boundary. 4. Execute step 2, 3 teratvely untl the moton of boundary Γ converges. Then go to step 5. 5. If there s any background regon not occuped by any seed regons, then put a new seed n the background, and go to step 2; else go to step 6. 6. Merge two adjacent regons so that the mergng causes the largest energy decrease; go to step 2. If no merge can decrease the energy, then go to step 7. 7. Stop. 2. RC BASED SEGMENTATION OF STEREOSCOPIC SEQUENCES OF ARTIFICIAL MUSCLES We descrbe n ths Secton our adaptaton of RC algorthm for artfcal muscles segmentaton. As shown n the ntroducton of ths paper, our segmentaton work s a key stage n the method desgned to obtan a characterzaton of artfcal muscles. The nputs for our applcaton are pars of orthogonal pctures of an artfcal muscle n moton (see Fgure 1). Our algorthm s responsble of segmentng these mages along the stereoscopc sequence. The output generated wll be used by actve contours to obtan the parameters that characterze the muscle movements. In the followng, we expose a general overvew of how our algorthm works, and we show how t deals wth mportant ssues such as the ntal segmentaton, the use of redundancy n stereoscopc pars of frames and the trackng of the target along the sequence. General overvew of the algorthm operaton The easest way to understand our algorthm s to descrbe ts operaton from the frst par of frames acqured. Let t) be the frame captured n the XZ plane and let B(t) be the frame captured n the YZ plane at the same nstantt. Fgure 2 shows a sequence of three stereo frames. As shown n the fgure, dfferences between consecutve frames are small. Ths wll help the target trackng stage later. Also, both cameras are calbrated so that the relatve poston of the muscle s the same n both frames t) and B(t). The presented applcaton operates wth pars of frames t), B(t) smultaneously. The frst par 0), B(0) s segmented wthout any a pror nformaton, followng the algorthm descrbed n Secton 1.2. The RC 188
Input Intal segmentaton Output t=0 Trackng Output t=1 Trackng Output t=2 Fgure 2. Sequence segmentaton flow. Whte arrows show nformaton transfer. 0) algorthm s appled to wth N seeds randomly put n the mage. The process follows the algorthm descrbed n the prevous secton untl the segmentaton Γ s complete. Once the ntal segmentaton of 0) s done (.e., the whole mage I( x, y) s parttoned nto N regons R ), the regon of nterest (the artfcal muscle) can be chosen from the descrptors of each regon or t s possble that the user can be prompted to choose the regon of nterest (t s useful n other applcatons wth more complex mages to segment). Now the algorthm knows what to look for and then t uses that nformaton to segment B(0) much faster. Ths ntal segmentaton could be accelerated by usng the knowledge we already have about the mages, but ths way we ensure that the algorthm wll work wth mages of any knd. At ths pont, we have the segmentaton Γ of both ntal frames A (0) and B(0). From now on, the algorthm wll work n a statonary mode untl the end of the whole sequence. In ths stage, we do have nformaton about prevous frames and we can use t to segment consecutve mages much faster. In ths statonary mode, the RC algorthm s not fully used as descrbed n the prevous secton. Instead of that, the number of regons s lmted to target and background and the mergng step almost dsappears, so the computatonal cost s reduced. Target trackng through the stereoscopc vdeo s done by change detecton n consecutve frames. Frame dfference s calculated between frames t) and A ( t 1) and the regon competton algorthm only takes care about the pxels that have changed. 189
Intal segmentaton As explaned n the prevous secton, segmentaton of the frst frame 0) s carred out by the RC algorthm descrbed before. Ths frst segmentaton s mportant because the next frames ( B(0), 1) ) are segmented much faster thanks to the nformaton obtaned. Moreover, f the muscle segmentaton n the frst frame s wrong our applcaton wll probably not be able to track ts movements. In ths secton we explan some mportant ssues related to ths part of the process. Frst step s to put N seeds randomly across the mage (see Fgure 3(b)). The sze of these ntal seeds s 1 pxel but the ntal descrptors,.e. the parameters that characterze the dstrbuton, are calculated wth a 5 5 pxel samplng wndow. Ths s to ensure that ntal descrptors do not correspond to nosy peaks n the mage. If one ntal seed falls nto one of these peaks, the statstcal forces would not let t grow as neghbour pxels do not pass the lkelhood test. On the other hand, the samplng wndow cannot be too bg because f t covers a sgnfcant edge t may be gnored. In fact, the optmal sze of the samplng wndow depends on the mage sgnal-to-nose rato, n [6] a dscusson about seeds and samplng wndows can be found. Now we have an approxmaton of the fnal segmentaton Γ 0). Ths s a m n matrx of labels, whch s the same sze of the frame. Ths matrx has zeros n every poston except n those N pxels wth ntal seeds R. Every seed regon has an assocated descrptor whch s used to control ts growng., Next step s to make those N regons grow. Ths s done by morphologc operators that dlate the ntal regons wth a 3 3 pxels structurng element. New pxels are tested aganst the regon descrptors and only become part of the regon f they ft the dstrbuton P I ). Usng gray level ( (a) (b) (c) (d) (e) (f) (g) (h) Fgure 3. Intal segmentaton 0) wth N = 8 random seeds (a) Frame 0). (b) Segmentaton Γ 0) after the frst growng teraton. (c), (d), (e), (f), (g) show Γ 0) evoluton every 10 teratons. (h) Fnal segmentaton Γ 0) after 160 teratons. 190
mages and one descrptor per regon, ths decson s taken by comparng the pxel value wth the descrptors, whch are, n ths case, the mean of the pxels n the regon. If the dfference s not hgher than the growng threshold T g, the new pxel s fnally added. Descrptors are recalculated n every growng teraton to keep them representatve of ther regons R. When the values of Γ 0) converge (.e., regons cannot grow anymore) the algorthm looks for pxels wthout any regon assgned and, f there s any, t puts a new seed and the prevous steps are repeated. Once every pxel n the mage has been labelled, the next step s to merge regons f ths causes the energy to decrease, that s, regons wth smlar descrptors are merged usng a new threshold called T m (see Fgure 3 (g) and (h)). If the values chosen for Tg and Tm are too tght, regons wll have problems to grow and new seeds wll be added to complete Γ 0). Ths means more regons, more descrptors and much more processng tme to obtan an over-segmented mage as fnal result. On the other hand, f Tg and Tm are too relaxed our algorthm wll be less senstve to detals and we may loose some regons of nterest. However, most artfcal muscle pctures have a good contrast between target and background, and by knowng the descrptors of prevous segmentatons, t makes easy to tune both thresholds properly (common values are T =50 and T =30). g m Redundancy between stereoscopc pars of mages As stated before, our applcaton only uses the nformaton obtaned from Γ t) to segment B(t) when t = 0. In other words, nformaton transfer between stereoscopc pars only occurs for the frst par of mages (see Fgure 2). It s possble to mplement ths transfer for every frame and t may be useful for mages wth dfferent characterstcs. However, experments carred out wth artfcal muscle sequences do not show any mprovement. In ths secton we descrbe how we use the nformaton obtaned from Γ t) to speed up the segmentaton of B(t). Assumng that Γ t) s correct, the nformaton we can use to segment B(t) s: Muscle poston t ) R n frame A (t). t) nterest Descrptor nterest of the muscle regon t) Rnterest n A (t). We use the muscle poston n t) to delmt the range of pxels where the algorthm wll search for the muscle n B(t). Cameras are calbrated so that both vews of the muscle are n the same relatve poston and scale. Therefore we can assume that muscle coordnates n z axs are the same n t) and B(t). Pxels lyng out of ths range are labelled as background (see Fgure 4(c)). t) The value of descrptor nterest s used to fnd the y axs coordnates of the muscle n B(t). The algorthm looks for pxels wth values t ) smlar to nterest and labels the rest as background (see Fgure 4(d)). At ths pont, the segmentaton of frame B(t) starts by applyng the RC algorthm. However, as more than 90% of the mage s already segmented (labeled as background) there s no competton between regons except n some specal cases. Part of the target gets occluded when the muscle's end reaches the water surface. In ths stuaton our applcaton needs the regon competton strength to complete the segmentaton correctly. Once we have delmted the muscle poston n B(t), the segmentaton starts by puttng one seed at the centre of the delmted zone and then we let t grow untl the segmentaton s fnshed. More 191
(a) (b) (c) (d) (e) Fgure 4. Informaton transfer between stereoscopc pars. (a) Frame A (t). (b) Frame B(t). (c) Segmentaton Γ. (d) Search range n B(t). One seed s put n the z coordnate of the t) t) t ) gravty centre of R, y coordnate s found by lookng for smlar pxels to. (e) nterest shows the regons growng. nterest (a) (b) (c) Fgure 5. Estmated target poston usng prevous knowledge. (a) Frame B ( t 1). (b) Target estmaton dlatng prevous segmentaton. (c) Frame B(t). specfcally, z axs (.e., vertcal) coordnate of the ntal seed s found by calculatng t) R gravty centre. The y axs (.e., nterest horzontal) coordnate s chosen by fndng the t ) most smlar pxel to nterest. Intal seed poston s carefully chosen because we do not want regons to compete when t s not necessary. Vdeo Segmentaton: Muscle Trackng In ths secton t s shown how t) and B(t) frames are segmented by usng the segmentaton of prevous frames. The followng explanaton s referred to YZ plane frames ( B(t) and B( t 1) ), but segmentaton of t) frames s analogous. The technque appled to track the muscle through the vdeo sequence s based on change detecton between consecutve frames. These changes are usually detected by calculatng the dfference I(t) I(t 1) between consecutve frames. In ths case, we use the knowledge that we already have about the mage and ts movement to optmze the segmentaton. As the muscle speed s lmted, we assume that ts poston wll be only slghtly dfferent n B(t). Therefore, we can estmate the muscle poston n B (t) f we know Γ B ( t 1). In order to do ths, the muscle poston n B ( t 1) s dlated morphologcally whch gves us a lmted space where the muscle mght be n B(t) (see Fgure 5 (b)). Once more, most pxels n B(t) are already labelled as background before startng RC algorthm. Now we apply the change detecton technque by calculatng the dfference between B ( t 1) and B(t) only n the estmated zone. The processng tme s reduced 192
ths way because most pxels of both frames are gnored. Besdes, we prevent the algorthm from beng dstracted by slght changes s water surface. Next step conssts n fndng the optmal poston to place our ntal seed. We use the B( t 1) nterest descrptor to test the pxels not excluded yet by our estmaton. A new seed s put at the most smlar pxel found n B(t) and t starts growng (see Fgure 6 (b)). In the RC algorthm all regons grow alternatvely to ensure a far ' competton but at ths pont we let the regon of nterest grow frst. Ths s a way to avod problems wth possble changes n llumnaton (due to muscle movement) that may mslead our algorthm. Once the muscle regon s grown, the background regon starts growng untl segmentaton of B (t) s fnshed. Experments show that the trackng process s remarkably accelerated thanks to ths technque because regon competton only occurs n specal cases lke the occluson problem mentoned before. Nevertheless, the performed segmentaton s correct n every case. 3. SEGMENTATION RESULTS Several tests have been run wth dfferent sequences and dfferent types of artfcal muscles and the results yelded are satsfactory. The obtaned segmentatons represent fathfully the muscle contour and are sutable to extract ts parameters. Fgure 7. Fgure 8 and Fgure 9 show the results obtaned n the frst frames of a 60 pars sequences of 150 150 pxels. In Fgure 7 the muscle has been loaded wth a pece of metal n order to test ts strength. XZ plane segmentaton s correct as t does not add the metal rng to the target regon. Notce the changes n lghtng between frames t) and B(t) and the presence of hard nose. Results obtaned are also satsfactory n ths stuaton. 4. CONCLUSIONS Ths paper shows the applcaton developed for segmentng stereoscopc vdeo sequences based on the regon competton algorthm. The orgnal RC algorthm has been tuned and optmzed for artfcal muscle vdeo segmentaton n gray level mages and results yelded are satsfactory. Parameters of muscle behavour can be obtaned from the output of our algorthm even n adverse stuatons of nose and lghtng. The process can be run wthout any human partcpaton after thresholds are adjusted. Redundancy between stereoscopc pars of mages has been used to optmze the segmentaton wth satsfactory results. Muscle trackng has also been optmzed by estmatng target poston n consecutve frames. As a future work t could be nterestng to extend the method by addng some topologcal model to ensure coherent segmentatons n both planes. 5. ACKNOWLEDGEMENTS Ths work s partally supported by Mnstero de Educacón y Cenca, under grant TEC2006-13338/TCM, and by Fundacón Séneca, project 03122/PI/05. (a) (b) (c) Fgure 6. (a) Change detecton between B (t) and B ( t 1). (b) Intal seed s placed usng and t starts growng n frst place. (c) Fnal segmentaton. B( t 1) nterest Γ B(t) 193
(a) A (0) (b) B (0) (c) A (1) (d) B(1) (e) A (2) (f) B (2) (g) A (3) (h) B(3) Fgure 7. Stereoscopc vdeo segmentaton. The muscle s loaded wth a metal rng. (a) A (0) (b) B (0) (c) A (3) (d) B(3) (e) A (6) (f) B (6) (g) A (9) (h) B(9) Fgure 8. Stereoscopc vdeo segmentaton. Intal stereo frames (a) and (b) and stereo frames 6 and 9. 194
(a) A (0) (b) B (0) (c) A (3) (d) B(3) (e) A (6) (f) B (6) (g) A (9) (h) B(9) Fgure 9. Stereoscopc vdeo segmentaton. Intal stereo frames (a) and (b) and stereo frames 6 and 9. REFERENCES [5] Lang, J., McInerney, T., Terzopoulos, D. Unted snakes. Proc. Int. Conf. Computer Vson, pp. 933-940, 1999. [1] Otero, T.F., Sansñena, J.M. Blayer dmensons and movement of artfcal muscles. Boelectrochem. Boenergetcs, 47, 1997. [2] Verdú, R., Morales, J., Fernandez-Romero, A.J., Cortés, M.T., Otero, T.F., Weruaga, L. Mechancal characterzaton of artfcal muscles wth computer vson. Proc. of SPIE Annual Int. Symposum on Smart Structures and Materals, 2002 [3] Verdú, R., Berenguer, R., Morales, J., Vázquez, G., Otero, T. F., Weruaga, L. 3D mechancal characterzaton of artfcal muscles wth stereoscopc computer vson and actve contours. Proc. of IEEE Int. Conf. on Image Processng, 2003. [4] Verdú, R., Berenguer, R., Morales, J., Vázquez, G., Otero, T., Weruaga, L. Mechancal characterzaton of the lfe cycle of artfcal muscles through stereoscopc computer vson and actve contours. IEEE Int. Conf. on Image Processng, 2005 [6] Zhu, S.C., Yulle, A.L. Regon competton: Unfyng snakes, regon growng, and bayes/mdl for multband mage segmentaton. IEEE Trans. Pattern Anal. Machne Intell, 18, 9, pp. 884-900, 1996. AUTHOR INFORMATION Rafael Verdu receved the MS degree n telecommuncatons engneerng n 2000 from the Techncal Unversty of Valenca (UPV), and the PhD n the same feld n 2005 from the Techncal Unversty of Cartagena (UPCT). He s currently a researcher and assstant teacher n sgnal and communcatons theory at UPCT. Hs techncal nterests are modelng and nverse problems n mage analyss. 195