Object Segmentation based on Saliency Extraction and Bounding Box Jian Ma, Lei Huang, Bo Yin, Fengfu Yin

Transcription

1 Iteratoal Coferece o Lostcs Eeer, Maaemet ad Computer Scece (LEMCS 204) Object Semetato based o Salecy Extracto ad Boud Box Ja Ma, Le Hua, Bo Y, Fefu Y Ja Ma Collee of Evrometal Scece ad Eeer yff3@26.com Bo Y Collee of Iformato Scece ad Eeer, ybfrst@26.com Le Hua Collee of Iformato Scece ad Eeer jasohuasc@mal.com Fefu Y Eco-des ad maufactur Isttute, Haer R&D Departmet, yff3@26.com Abstract Object semetato s desrable may practcal applcatos, e.., object classfcato. However, due to varous object appearaces ad shapes, cofus backrouds, object semetato a effectve way s stll a challe ssue. I ths paper, a ovel alorthm of object semetato based o salecy extracto ad boud boxes s proposed. The semetato performace s sfcatly mproved by troduc salecy extracto to the semetato scheme. Frstly, boud boxes are acqured by object detecto alorthms, foreroud ad backroud model s costructed us boud boxes. The, salecy extracto procedure s troduced, ad adaptve wehts for each pxel are computed based o the salecy extracto. Fally, udrected raph whch corporates the adaptve wehts for each pxel s costructed ad raph cuts s mplemeted to obta the semetato results. Comprehesve ad comparatve expermets demostrate that our proposed alorthm has acheved proms performace over a challe publc avalable dataset. Keywords - object semetato; salecy extracto; raph cuts; boud box; adaptve weht I. INTRODUCTION Object semetato statc maes s a mportat ad challe ssue for uderstad maes. May approaches have bee proposed to solve ths problem. Iteractve methods focus o mae semetato wth pror foreroud ad backroud seeds whch are ofte labeled maually. These methods ca acheve better semetato results, but are ot sutable for practcal applcatos. To avod the teractve operato, researchers proposed automatc semetato approaches whch use object detecto techques to et the rouh object reos stead of maual labels. However, due to varous object appearaces ad shapes, cofus backroud, object semetato remas a challe problem. There s plety of prevous work related to object semetato. Geerally, teractve methods[][2][3] ca acheve proms results, but are ot sutable for practcal applcatos that requre automatc operatos. Therefore, automatc scheme for object semetato s proposed. The automatc methods use specfed foreroud ad backroud reos as put. I eeral, the form of specfed foreroud reos s boud box[4][5]. The semetato alorthm based o eery mmzato s used to et the result wth boud boxes. To et more accurate semetato results wth the detected boud box, Lemptsky et al.[6] adopted the raph cuts alorthm wth the costrat that the desred semetato should have parts that are suffcetly close to each of the sdes of the boud box. Ths s reasoable uder the codto that the boud box s accurate. Ya et al.[7] developed the adaptve edelet features as the uary term of Codtoal Radom Feld Model to explot the spatal coherece of labels of ehbor pxels. These methods employ appearace models for the foreroud ad backroud whch are estmated throuh object detecto alorthms ad acheve semetato by solv eery mmzato problems. However, All the above methods whch us boud box as put, suppose that all the pxels the boud box ca ve equal cotrbuto for the foreroud model costructo. Ths wll lead false foreroud pror, especally whe the structure of the object s ot compact. Therefore, ths paper, we propose a ovel object semetato method based o salecy extracto ad boud boxes, wth the salecy of the pxels, we ve each pxel a specfc wehts for costruct the foreroud ad backroud models. Our cotrbuto s that we propose a ovel object semetato method whch troduce salecy extracto to the boud box-based scheme. Wth the salecy extracto results, adaptve wehts for each pxels s 204. The authors - Publshed by Atlats Press 564

2 computed for costruct the udrected raph for raph cuts method. II. OBJECT SEGMENTATION ALGORITHM Frst, foreroud ad backroud model s costructed us boud boxes. Secod, adaptve wehts for each pxel are computed based o the salecy extracto. A. System Overvew The framework of the proposed method s llustrated Fure. Geerally, our method cossts of three staes. Iput mae Boud box Salecy detecto Foreroud ad backroud Seeds Adaptve wehts Foreroud ad backroud model Udrected raph costructo Graph cuts Fure. Framework of the proposed method. Fally, udrected raph s costructed ad raph cuts s mplemeted to obta the semetato results.the detals are preseted below. B. Foreroud ad backroud model costructo I order to obta foreroud ad backroud model, object detecto method s appled. I our scheme, we use the part based methods to et the boud boxes[5][8]. The, the foreroud ad backroud model s costructed by Gaussa mxture model (GMM). The probablty desty fucto for GMM of a observato x ca be wrtte as p T - ( x; )= p ( x; )= exp{ ( x- μ ) ( - )} x μ = = d /2 2 (2 ) 2 where p ( x; ) s the p. d. f. () correspod to the th Gaussa model G, x s a RGB color vector, cossts of the elemets of the mea vectors u ad the covarace matrces, d s the dmeso of vector x, T T T ad d 3 our alorthm, (, ), s the mx parameter, = ad 0, s the umber = of the Gaussa models for costructo of the GMM. C. Adaptve wehts determato I order to use the pror appearace for foreroud ad backroud model costructo a effectve way, we troduce the adaptve wehts for each pxel. Imae salecy s oe of the hot research ssues mae process doma. Che et al.[9] proposed a fast reoal cotrast based salecy extracto alorthm, ad had ot proms results. I ths paper, we troduce t to our semetato scheme. The mae salecy of reoal cotrast based salecy extracto ca be represeted as follows: 2 S( rk ) exp( Ds ( rk, r ) / s ) ( r ) Dr ( rk, r ), (2) where rk r 2 D ( r, r ) f ( c, ) f ( c, j) D( c, c ), r 2 2, 2, j j D( c,, c2, j) s the color dstace metrc betwee pxels c, ad c 2, j, Dr ( r, r 2 ) s the color dstace betwee reos r ad r 2. Ds ( rk, r ) s the spatal dstace betwee reos r k ad r, s cotrols the streth of spatal weht. Larer values of s reduce the streth of spatal weht. f ( ck, ) s the frequecy of the -th color c k, amo all k colors the k-th reo r k wth k {,2}. The detals of the salecy extracto alorthm ca refer to Ref. [9]. Wth the salecy extracto alorthm, the salecy map s acqured. Fure 2 ve some examples, the color of the pxel meas the salecy, the whter, the more salet. Wth the salecy results, we compute the adaptve wehts for each pxel as follows: (, j) ( s(, j) ) / (0* ), f s (, j) > (3) ( s(, j) ) / (0* ), f s (, j) < (, j) ( s(, j) ) / (0* ), f s (, j) > (4) where (, j) s the adaptve wehts for each pxel at locato (, j ). s (, j) s the salecy of pxel (, j ). s the threshold that ves 95% recall rate for the tra maes, ad s chose emprcally. For pxels the boud boxes, we use Eq.(3), ad for pxels outsde the boud boxes, we use Eq.(4). 565

3 E(,, z) U(,, z) V(, z) (6) where =[, 2, N ]. Wth the adaptve wehts ad the foreroud ad backroud models, the udrected raph ca be costructed. The the semetato results s obtaed throuh the raph cuts method. I our scheme, we use the optmzed verso Ref.[2]. Fure 2. Samples of salecy extracto. The frst le s the put maes, ad the secod le s the correspod salecy extracto results. D. Udrected raph costructo ad raph cuts Graph cuts are famous methods whch have bee successfully used for seeded mae semetato. ' Represet the mae as a array z ( z,... z,... z ) wth z correspod to the color or rey value of pxel, the udrected raph G ( V, E) s costructed wth the mae pxels as the odes( V ) ad the ehborhood relatoshp betwee pxels(e.. 4-ehborhood) as ede( E ). There are also two specally desated termal odes F ad B that represet foreroud ad backroud labels. Edes betwee pxels are called ehborhood lks (-lks) ad edes coect pxels ad termal odes are called termal lks (t-lks). The, the mae semetato correspods to a odes partto the raph G. Def a array of opacty values {,...,... N} for all pxels, where {0,} wth 0 for the backroud ad for the foreroud, the mae semetato also ca be expressed as a soluto for ferr the ukow varables from a ve mae z. Fally, the lobal optmal soluto of s solved by mmz a Gbbs eery fucto E(,, z) as follows: E(,, z) U(,, z) V(, z) (5) where U(,, z) lo( p( z; )), { m, } m. U(,, z) ( m, ) C V (, z) B [ ] N s the reo term, whch defes the cost of t-lks. V(, z) s the boudary term, whch defes the cost of -lks. The tradtoal raph cuts method compute the cost of t-lks wth the same weht for all the pxels. I our scheme, we corporate the adaptve wehts for each pxel to the cost computato of t-lks. The, E(,, z) ca be defed as: III. EXPERIMENTAL RESULTS I ths secto, we coduct comprehesve evaluatos of our method. The dataset, basele alorthms ad evaluato metrcs are descrbed frst. A. Dataset, basele alorthms, evaluato metrcs To evaluate the effectveess of our method, we test t o a challe publc datasets: Parse dataset from[0]. The parse dataset cotas 305 maes of full body wth a wde varety of actvtes ra from stad ad walk to dac ad perform exercses. The dataset clude a stadard tra/test splt. Ad we use the 205 test maes to evaluate our method. We compared our method wth the optmzed verso of raph cuts method [2](Grabcut) ad salecy based semetato method[9]. These two methods are deoted as Grabcut- Boudbox ad Grabcut-Salecy. The Grabcut- Boudbox method us the boud box as put, the same wth ours. The Grabcut-Salecy method use the salecy extracto results as the put. F-metrc as follows s used to evaluate the performace of our method, whch s smlar wth Ref. []: ( rs( ) r( )) F-metrc (7) ( r ( ) r ( )) s where rs ad r deote the semeted bary body ad roud truth respectvely, s the pxel ad the operators ad perform pxel-wse AND ad OR, respectvely. B. Comparso wth other methods As descrbed Secto 2.3, the threshold s obtaed from the tra maes. A larer meas there are more pxels assed as mportat oes for foreroud model costructo. I our expermets, 40, whch s determed by aalyz the threshold that ves 95% recall rate for the tra maes. Fure 3 plots the comparso wth basele methods, where the red le represets our method, ad the yellow ad blue les correspod to the Grabcut-Boud box[2] ad Grabcut-salecy[9] methods respectvely. From the results we ca see that our method performs best amo the compared methods. To quattatve llustrate the performace of our method, the mea ad stadard devato of F-metrc whch have bee used the prevous work [] s employed. Table provdes the values of the each compared methods. From the table, we ca see that our method outperforms the other methods by more tha 5%. 566

4 Beyod these quattatve comparsos, we hhlht the qualtatve mprovemet Fure 4. From Fure 4, we ca see our method ca et more accurate semetato results tha other methods. I our expermets, we fd that, the Grabcut- Boudbox method ves the worst performace for that us rectale reo as pror foreroud drectly may cota backroud whch wll lead to a bad semetato. I cotrast, our method s more robust ad proms sce we troduced adaptve wehts for each pxel for udrected raph costructo, whch ca allevate the fluece from the backroud reos the boud boxes. IV. CONCLUSIONS AND FUTURE WORK We have proposed a ovel method of object semetato. Ulke the other methods, we ve each pxel a adaptve wehts by troduc the salecy extracto alorthm. The adaptve wehts s used the Grabcut-BoudBox Grabcut-Salecy Our Method F-Metrc Test maes(-205) Fure 3. Comparso of our method wth Grabcut-BoudBox ad Grabcut-Salecy. TABLE. COMPARISON OF OUR METHOD WITH OTHER METHODS. Method Our Method Grabcut-BoudBox Grabcut-Salecy Mea Std. dev udrected raph costructo. Based o the udrected raph, raph cuts s executed to et the semetato results. Comprehesve expermets show that our method ca et proms results. I the future, we pla to employ more formato, e,, shape formato, for boud box-based object semetato. ACKNOWLEDGEMENT Ths work was supported by the Fudametal Research Fuds for the Cetral Uverstes uder Grat ; by the Scece ad Techoloy Fud Pla Project of Qdao uder Grat hy; by the Natoal Nature Scece Foudato of Cha uder Grat (a) (b) (c) (d) (e) (f) Fure 4. Performace comparsos for dfferet methods. (a) Iput maes, (b) Boud boxes, (c) Salecy extracto results, (d)-(f) are semetato results. (d) Our method, (e) Grabcut- Boudbox, (f) Grabcut-Salecy. REFERENCES [] Y. Y. Boykov ad M.-P. Jolly, Iteractve raph cuts for optmal boudary & reo semetato of objects ND maes, Proc. of IEEE Iteratoal Coferece o Computer Vso, 200, vol., pp [2] C. Rother, V. Kolmoorov, ad A. Blake, Grabcut: Iteractve foreroud extracto us terated raph cuts, ACM Trasactos o Graphcs (TOG), vol. 23, pp , [3] A. Deecke, H. Wers, J. J. Stel, ad E. Körer, Ole fure roud semetato wth adaptve metrcs eeralzed LVQ, Neurocomput, vol. 72, o. 7, pp , [4] N. Dalal ad B. Trs, Hstorams of oreted radets for huma detecto, Proc. of IEEE Coferece o Computer Vso ad Patter Recoto, 2005, vol., pp vol.. [5] P. F. Felzeszwalb, R. B. Grshck, D. McAllester, ad D. Ramaa, Object detecto wth dscrmatvely traed part-based models, 567

5 IEEE Trasactos o Patter Aalyss ad Mache Itellece, vol. 32, o. 9, pp , 200. [6] V. Lemptsky, P. Kohl, C. Rother, ad T. Sharp, Imae semetato wth a boud box pror, Proc. of IEEE Iteratoal Coferece o Computer Vso, 2009, pp [7] B. Ya, C. Hua, ad R. Nevata, Semetato of objects a detecto wdow by Noparametrc Ihomoeeous CRFs, Computer Vso ad Imae Uderstad, vol. 5, o., pp , 20. [8] Y. Ya ad D. Ramaa, Artculated pose estmato wth flexble mxtures-of-parts, Proc. of IEEE Coferece o Computer Vso ad Patter Recoto, 20, pp [9] M.-M. Che, G.-X. Zha, N. J. Mtra, X. Hua, ad S.-M. Hu, Global cotrast based salet reo detecto, Proc. of IEEE Coferece o Computer Vso ad Patter Recoto, 20, pp [0] D. Ramaa, Lear to parse maes of artculated bodes, Advaces Neural Iformato Process Systems, vol. 9, p. 29, [] S. L, H. Lu, ad L. Zha, Arbtrary body semetato statc maes, Patter Recoto, vol. 45, o. 9, pp , Sep