COLOR HISTOGRAM SIMILARITY FOR ROBOT-ARM GUIDING

Transcription

1 COLOR HITOGRAM IMILARITY FOR ROBOT-ARM GUIDING J.L. BUELER, J.P. URBAN, G. HERMANN, H. KIHL MIP, Unversté e Haute Alsace Mulhouse, France ABTRACT Ths paper evaluates the potental of color hstogram technques for the approach phase n the vsual servong of a camera mounte on a robot arm. We propose the Color Hstogram mlarty metho to locate the object of nterest. For each poston n the mage, the hstogram of the moel s compare wth the hstogram of a search wnow of the same sze as the moel. The object s locate when the smlarty measure s maxmze. We evaluate the extenson of ths metho to the estmaton of the sze of the object n the mage. KEYWORD Color hstograms, Color smlarty, Multcolore objects, Object tracng. Introucton We aress the problem of approachng an graspng famlar objects n an noor home envronment wth a moble robot arm. An eye-n-han vson confguraton s retane, where a veo camera s mounte on the robot s en-effector. The tas can then be efne n terms of vsual servong []: characterstc mage features of the object wll be extracte to control the poston of the robot s en-effector relatve to the object of nterest. Intally, the object of nterest s presente n front of the camera n ts fnal esre poston (reay for the graspng phase that wll not be scusse n ths paper) an vsual object features are extracte from the corresponng camera mage to bul a reference moel of the object. tartng generally from a stant ntal poston, the camera that s mounte on the robot-arm wll be vsually servoe such as to see the object of nterest n ts fnal poston, where the object s typcally centere an close. The tas s conclue wth a bln/pre-programme graspng movement to pc-up the object. In ths paper we evaluate the possblty of usng a color hstogram smlarty technque as an mage feature for the approachng phase of the tas, not aressng the fnal precse postonng an graspng phases. The moel that wll be use s the color hstogram of the object n ts esre mage poston. In a prevous paper we showe the potental of a color hstogram smlarty approach to robustly trac an object of nterest through a sequence of mages n a varety of scenes an contons [2]. ecton 2 ntrouces the technque, scusses ts performance wth the smlarty surface an ts real-tme mplementaton. For the approach phase, the object must be etecte an localze n the mage, but ts range must also be evaluate. Can the hstogram smlarty algorthm that allows for etecton an localzaton also be use to rescale the object, thus provng range nformaton to poston the camera? Ths s the subject of ecton 3 an contrbutng ea of ths paper. In ecton 4, the approach s llustrate wth varous mage sequences where the recorng camera s move bac an forth, eepng the object n the fel of vew. The metho s scusse an puts t nto perspectve. 2. Color Hstogram mlarty Color hstograms are very appealng: they are nvarant to translaton an rotaton n the mage plane, they change slowly uner change of angle of vew, change n scale an occluson, an they are also computatonally cheap to mplement [3]. To us, usng untl recently a lght-bulb marer to extract a center of gravty n a treshole mage, they represent an alternatve well worth nvestgatng. CIMTA 2004 eptember 9-24, 2004, Cherbourg, France page /5

2 2. Hstogram Intersecton A color hstogram s obtane by countng the number of tmes each screte color occurs n an mage DI, of sze L x L 2, where L an L 2 are the rows an columns of DI. The object of nterest s efne by reference mage RI, of sze l x l 2, where l an l 2 are the rows an columns of RI. The moel hstogram M of RI s compose of n bns, corresponng to the scretzaton of the color space. The Hstogram Intersecton algorthm, ntrouce by wan & Ballar [4], s a basc but powerful vsual cue for object etecton an locaton. The moel hstogram M s compare to hstogram H of the mage n whch the object s to be entfe. The Hstogram Intersecton c s efne as: c = n = = ( H M ) mn, n M where M an H represent respectvely the number of pxels n the th bn of the moel an the mage. () The secon assumpton s a strong one, nowng that varyng llumnaton wll affect the color bnnng, an constant llumnaton s not concevable when navgatng n natural envronments. Much effort s therefore put nto efnng color constancy spaces an effcent color quantzaton technques [5, 6, 7]. Fgure llustrates the CH metho on two examples. A frst stuy [2] prove the robustness of the approach n terms of color sgnatures. Ths pont wll be scusse further n the ecton 3. Doctor Desel example Reference mage Toy box example Reference mage When the object s color sgnature s clearly an unquely scrmnate n the mage, the algorthm s robust to object rotaton n the mage plane, reszng, partal occlusons, eformaton (changng angle of vew). 2.2 mlarty urface The Hstogram Intersecton n equaton () has prmarly been evse to etect the presence of an object n the mage. To locate the object, a sub-mage or search wnow (W) of the sze of the reference mage RI s forme an systematcally swept across the mage pxel by pxel. For each pxel poston p the hstogram Ip of the W s compare to M, resultng n a smlarty measure c(p), accorng to equaton (). The systematc sweepng of the mage forms a smlarty surface C, of sze L -l + tmes L 2 -l 2 +. As a frst approach, we conser that the object s poston s gven by the maxmum of the smlarty surface. We wll refer to ths metho as the Color Hstogram mlarty (CH). Localze object mlarty urface (for vsual clarty we represent -C) Localze object mlarty urface (-C) The man assumptons for ths estmaton to gve goo results are: - the moel hstogram must be suffcently scrmnatng; - the object s color n the sequence of mages must stay close to the ones of the moel; - the sze of object an ts moel are suppose entcal. Ths last assumpton can be taen practcally n a very loose way, the object wll stll be localze but wth less accuracy. mlarty urface 3-D representaton mlarty urface 3-D representaton Fgure. Two examples (left an rght columns of fgure) of object localzaton wth the Color Hstogram mlarty metho. The maxmum of the smlarty surface s retane as the object s locaton. The Doctor Desel example (left column) results n a well-efne unque maxmum. The toy box example (rght column) shows a car rver character n a rch brght color envronment an the character s angle of vew has conserably change. CIMTA 2004 eptember 9-24, 2004, Cherbourg, France page 2/5

3 2.3 Color Hstogram etup The veo mages are quantze urng acquston usng a same color map. Our experence s that object localzaton wth the CH algorthm s not very senstve to the quantfcaton metho (unform, optmze ), to the hstogram resoluton an to the color space. Ieally, we are n nee of a fast algorthm that preserves color constancy urng the camera movements. o far we favor a basc RGB unform quantfcaton that s computatonally very lght an gves goo results n many cases. Other color spaces are currently nvestgate. 2.4 Real-Tme Implementaton The algorthmc cost of the hstogram smlarty surface for a full sze mage may at frst seem prohbtve. We however were able to optmze t for real-tme veo compatblty. We esgne an mplementaton base on the E2 nstructons set of the Pentum 4 processor. Usng a 2.4 Ghz processor, the computaton of the exhaustve smlarty surface of a 576x768 pxels veo mage DI taes 50 to 00 ms accorng to the sze of the W. where: - s the apparent surface, or the number of pxels belongng to the object n mage ; - ref s the surface of the object n the reference mage; - an ref represent the stances between the object an the camera, respectvely for the th mage an the reference mage. Rather than etermnng the pxels belongng to the target, we efne a search rectangle that wll cover most of the object an has the object s characterstc color sgnature. The ses of ths rectangle are ept proportonal to the ses of the reference mage. We ntrouce rate r such as: = ref = ref. (3) r Is t possble to estmate ths rate r base on color hstograms? An teratve search algorthm base on a smlarty maxmzaton crteron can be use. The hypothess to be verfe s that the smlarty s maxmze when the sze of the search wnow correspons to sze of the moel. 3. Reszng wth Hstogram mlarty Im Im 2 The en-effector mounte camera must be gue towars an object that appears n the mage. The camera must both be orente an move forwar n the recton of the object. The esre poston s etermne by the moel that has been recore prevously. An estmaton of the stance between object an camera s therefore crucal to control the camera movement. Due to the vsual servong scheme, vsual feebac upates the robot s movements contnuously an ensures the approach movement, even f the range nformaton s mprecse. However, precse fnal postonng cannot be consere wth naccurate control sgnals. Theses remars le us to evaluate the CH metho to etermne the apparent sze of the object n the mage. The correlaton can be evaluate wth the relatonshp between stances an surfaces: ref = (2) 2 ref 2 Im 3 Im 4 Im 5 RCX FlunchBot Images for the moels Fgure 2. ample mages extracte from an expermental sequence showng the camera approachng the object of nterest. The same sequence has been use for both the RCX an the FlunchBot objects. The last mages have been use to etermne the moels. CIMTA 2004 eptember 9-24, 2004, Cherbourg, France page 3/5

4 Table. Rate estmaton for the mages of fgure 2 Preetermne rate RCX Estmate rate r FlunchBot Estmate rate r Moel Im Im Im Im Im The sze of the search wnow can be vare accorng to rate r. Ths wnow wll be note W r. nce the reference mage RI s only use to bult the moel hstogram M, the reszng of the moel s obtane by ajustng hstogram M r = r 2 M. As escrbe above, the smlarty surface s obtane by comparng hstograms W r (p) wth M r for every pxel poston. Let C (r) be the maxmum of the smlarty surface for mage an rate r. However, the metho wors well an s smple, an s therefore worth further nvestgaton. In partcular, we explore hstogram comparsons, wth the ea of mang the comparson more meanngful. The reszng technque has a complementary effect: by ajustng the apparent sze of the object, t ncreases the precson of the object locaton through CH. Ths observaton s confrme by the analyss of the smlarty surfaces, as shown n fgure 3 where the smlarty surfaces for mage 2 are represente for two fferent reszng rates, 0.5 an.3, where the latter s close to the actual rate of the object. 5. Concluson The vsual servong of a robot arm wth an eye-n-han camera can be mplemente rectly wth mage-base nformaton. Image processng s often smplfe, for example wth the aton of lght-bulbs. To use robotc vson n a less constranng way, t seems nterestng to use the color of objects. The color hstogram smlarty metho turns out to be a goo canate, especally for ts smplcty an robustness. Let R be the proportonal rate between object n the mage an the reference moel. If the hypothess s verfe, the maxmum CM() = max C (r) s obtane for r = R. 4. Experments To explore the valty of our hypothess, we too a number of natural scene mage sequences. Two examples are presente n Fgure 2. An mage n the sequence s use to efne the reference: a rectangular zone that covers the object s taen as moel. The purpose s then to etermne the sze of that object n every mage of the sequence. The target object s searche for n the mage usng the CH metho for r beng vare n the nterval [0. 2]. Fgure 4 shows the graph of C (r) for two examples: a case where the object s foun closer than the reference, an a case where the object s farther away an appears smaller than the reference. We etermne rate R that maxmzes hstogram smlarty. Table compares estmate rate R to a manual etermnaton obtane by vsual analyss. a. rate r = 0.5 Dscusson Table shows that the crteron of maxmum smlarty s not entrely satsfyng. It unerestmates systematcally the sze of the object. Ths bas can be explane by a shft n the color strbuton n the hstogram. b. rate r =.3 Fgure 3. mlarty surface for the RCX object n mage 5 for two fferent rates r. CIMTA 2004 eptember 9-24, 2004, Cherbourg, France page 4/5

5 C (r) max Our experments show that the algorthm can etect an locate an object n the mage wth goo precson. Vsual servong n the approach phase requres also an estmate of the range of the object. We propose a smple technque to show the nterest of the CH algorthm: the smlarty s maxmze when the search wnow s of the same sze than the object n the mage. C (r) a. mage 5 object s closer than the reference max b. mage 2 object s farther away Rate r Rate r Fgure 4. Graph of C (r), the maxmum CH as a functon of the rate coeffcent r for the RCX object n mage 2 an mage 5. It s however necessary to enhance the hstogram comparson to reuce the estmaton error ntrouce by the shft phenomenon n the color hstogram. References [] E. K. Hashmoto, Vsual ervong (Worl centfc, 993). [2] G. Hermann, D. Greboval, H. Khl, J.P. Urban, Evaluaton of Color-Base Technques for Robotc Postonng Tass, Proc. 8th Worl Mult-Conference on ystemcs, Cybernetcs an Informatcs, Orlano, UA, [3] M. J. wan, Color Inexng, TR 360, November 990, Unversty of Rochester [4] M. J. wan, D. H. Ballar, Color Inexng, Internatonal Journal of Computer Vson, 7(), 99, [5] B. Funt, G. Fnlayson, Color constant color nexng, IEEE Transactons on Pattern Analyss an Machne Intellgence, 7(5), 995, [6] T. Gevers, A. W.M. meulers, Color-base object recognton, Pattern Recognton, 32, 999, [7] N. Papamaros et al., Aaptve Color reucton, IEEE Transactons on ystems, Man, an Cybernetcs part B: Cybernetcs, 32(), February 2002 CIMTA 2004 eptember 9-24, 2004, Cherbourg, France page 5/5