An Improved Stereo Matching Algorithm Based on Guided Image Filter

Transcription

1 nd Internatonal Conference on Modellng, Identfcaton and Control (MIC 015 An Improved Stereo Matchng Algorthm Based on Guded Image Flter Rudong Gao, Yun Chen, Lna Yan School of nstrumentaton Scence and Opto-electroncs Engneerng, Behang Unversty, Bejng , Chna Abstract Stereo matchng s a challengng ssue n computer vson feld. To address the poor accuracy behavor of local algorthms, we propose an mproved stereo matchng algorthm based on guded mage flter. Frstly, we put forward a combned matchng cost by ncorporatng the absolute dfference and mproved color census transform (ICCT. Secondly, we use the guded mage flter to flter the cost volume, whch can aggregate the costs fast and effcently. Then, n the dsparty computng step, we desgn a modfed dynamc programmng algorthm, whch can weaken the scannng lne effect. At last, the fnal dsparty maps are ganed after post-processng. The expermental results are evaluated on the Mddlebury stereo dataset, showng that our approach can acheve good results both n low texture and depth dscontnuty areas wth an average error rate of 5.14%. Keywords-stereo matchng; census transform; guded mage flter; dynamc programmng I. INTRODUCTION Stereo matchng s one of the most actve research areas n computer vson. It refers to the process of estmatng the scene depth by fndng the correspondng ponts n the bnocular or mult-ocular mages [1]. It s wdely used for vsual realty, object recognton, and depth-mage based renderng []-[5]. Stereo matchng algorthms can be classfed nto local and global algorthms. Local algorthms compute each pxel s dsparty value n the lght of the ntensty values wthn a wndow of fnte sze [6], [7]. However, global algorthms regard stereo matchng as an energy mnmzaton problem and obtan global dsparty allocaton va optmzaton methods such as dynamc programmng (DP [8], graph cuts [9], and belef propagaton [10]. The tradtonal matchng cost computaton methods encompass the pxel-based cost, regon-based cost, flterbased cost, and the non-parameter transformaton-based cost. [11] gave a new dea of combnng AD and census transform, achevng good results by explotng the advantages of dfferent matchng costs computaton methods. [1] poneered the use of adaptve weght n the cost aggregaton process, greatly mprovng the accuracy of the local stereo matchng algorthm. Ths method s actually equvalent to the computaton of the weght usng the blateral flter, but ts computatonal complexty s hgh. On ths bass, many new self-adaptve weghtng methods are proposed. [13] determned the weghts by computng the geodesc dstance between the wndow pxel and the central pxel, ncreasng the matchng accuracy through mposng the connectvty constrant. But the computatonal complexty was not reduced. Guded flter has good edge-preservng smoothng propertes lke the blateral flter, and t does not suffer from the gradent reversal artfacts [14]-[16]. To the best of our knowledge, t s one of the best edge-preservng flterng. To address these problems mentoned above, we combne AD and mproved color census transform (ICCT to compute matchng cost. Then, the guded mage flter s used to generate the adaptve weght n the process of cost aggregaton. Moreover, the dsparty value s selected usng the modfed dynamc programmng algorthm to obtan the hgh-accuracy dsparty map quckly and effectvely. II. ALGORITHMS The proposed method s an effcent correlaton and flter-based local method. An overvew of the block dagram of the approach s provded n Fg. 1. Frstly, the algorthm transforms the mages before calculatng the matchng costs under each dsparty level. Then we use AD + ICCT to measure the smlarty between two ponts and adopt guded flter to compute support weghts. Fnally, n the postprocessng process, we employ three steps ncludng regon votng, nterpolaton and weghted medan flter for dsparty refnement. The man components wll be dscussed n detals. Left Image Rght Image Dsparty Map AD+Improved Color Census Transform AD+Improved Color Census Transform Weghted Medan Flter LR Consstence Check Fgure 1. The block dagram of the proposed approach. Guded Image Flter Regon Votng A. Matchng cost computaton Motved by color census transform, we propose an mproved color census transform. Frst, we convert the mage from RGB color space to Gaussan color model (GCM space, because RGB color space s senstve to radometrc. Then the Eucldean dstance s used to measure the dfference between two pxels p and q. We defne Dm ( p as the mean value of all these dstance n the wndow centered at p. DG ( q ndcates the dstance 015. The authors - Publshed by Atlants Press 139

2 between p and other pxels n the wndow centered at p. The relaton between Dm ( p and DG ( q s defned as: { 1, G ( (, ( ( < m ξ D = f D q D ( p m p DG q (1 0, else For each pxel p n the mage I, the census transform encodes the local neghborhood around p to a bt strng, representng the set of surroundng pxels wth lower dstance value than Dm ( p. The transform results can be ndcated as: ( = ( m(, G ( T p ξ D p D q ( q Np Where denotes the act of concatenaton and N p s the neghborhood area of p. For a partcular dsparty d, the matchng cost between a pxel p n the reference mage and ts correspondng pxel pd n the matchng mage, s calculated through the Hammng dstance. The Hammng dstance represents the number of unequal elements n the two bt streams: (, (, C p d = Hammng T T = T T (3 census p pd p pd For mage regons wth smlar local structures, census transform could brng n matchng ambgutes. Thus, we combne the absolute dfference n color and the census transform to form a more accurate metrc. Equaton (4 s used to calculate the AD value. 1 C p d I p I p d (4 AD Where I ( L reference mage and I (, 3 L R R, G, B (, = ( (, p denote the ntensty values of pxels n the pd represent the ntensty values R of pxels n the target mage. They are both calculated n three color channels. The fnal pxel-wse matchng cost s defned as (5: (, Census (, CAD p d C p d C( p, d = 1 exp + 1 exp (5 λad λcensus Where λ AD and λ Census are normalzng parameters. B. Guded flter based on adaptve weghts Accordng to the above defnton, we can compute every cost for assgnng dsparty d at pxel p = ( x, y. Then we use a three dmenson array C, whch s usually called the cost volume or dsparty space mage (DSI, to store all these costs. Each cost can be ndexed by C( p, d. As pxel-topxel comparson s too senstve to nose, t s common to aggregate the matchng costs n a support regon usually defned by a wndow. To mprove the accuracy of matchng results, the method of adaptve weghts was developed and has been wdely used for ts great effectveness. In [1], the aggregaton step was formulated as: ( ( ( ( ω pq, ω p,,, d qd C qd q Np qd Npd C pd, = ω( pq, ω( p, q N, q N d qd p d pd Where N p and N p are the set of pxels n the support d wndow of correspondng pxels p and p d respectvely. ω ( pq, and ω ( pd, qd are the desgnated weghts n the reference and target wndows. The denomnator s used to normalze the weghts. As combnng the support weghts n both wndows only helps to mprove correspondence search slghtly, we can omt the term of ω ( pd, qd. Thus the followng equaton s obtaned. (, pq, (, (6 C p d = q W C q d (7 N In ths way, cost aggregaton s used to compute the weghted average n a support wndow for every pxel. It can be mplemented by smoothng the cost volume wth a flter. So the weghts are defned by the flter kernels. In fact, the method n [1] s equvalent to the blateral flterng. Here we use the newly developed guded flter to compute weghts. Guded flter gves a weght between two pxels p and q, accordng to ther statstcal analogy of the reference ntenstes based on the averages and the varances of several squared wndows. For color gudance mages, the weght s descrbed as follow: W 1 = 1+ p T ( I ( p μk ( I ( q μk pq, ω k ω σ p ωq k ε + U (8 Where, I ( p, I ( q and μ k are 3 1 vectors as they have R, G, B channels. σ k and U are 3 3 matrxes. μ k and σ k are the mean and co-varance matrx of I n a square wndow wth dmensons r r centered at pxel k. The number of pxels n ths wndow s denoted wth ω. ε s a smoothness parameter and U s an dentty matrx. Guded flter has an edge-preservng property whch can be easly understood by nvestgatng the above equaton. Parameter ε controls the strength of smoothng and can be tuned expermentally. To show the property of guded flter ntutvely, we vsualzed the flter weghts for some mage 140

3 regons n Fg.. In order to observe dfferent mage areas clearly, we gave the partal enlarged vews. Here, the data term comes drectly from the aggregated matchng costs, whch s defned n (13. The smoothness term s defned n (14: Edata ( d = C( x, y, d (13 ( xy, smooth ( = (, ( 1, E d λ d x y d x y (14 ( xy, Fgure. The demonstratons of generatng adaptve weght by usng guded flter. The weghts are hgh n regons whch are self-smlar to the central pxel and low otherwse, agreeng wth the local mage structure. In the real mplementaton, nstead of computng W p, q explctly, the guded flter can be performed n the followng equvalent way: ( a ( p p C p, d = I p + b (9 1 ak = I μ ω k ω 1 ( σ k + ε U ( C(, d kc( k, d T (, k (10 b = C k d a μ (11 k I s the reference mage and acts as the role of guded mage. a p and b p are the average of a k and bk n the square wndow ω k whch nvolves p. (, nput cost slce C(, d n ω k. k C k d s the mean of C. Dsparty selecton Dynamc programmng (DP s a technque used to solve a complex problem by breakng t down nto several subproblems. It solves each sub-problem each tme, whch greatly reduces the computatonal complexty. DP-based algorthms formulate stereo correspondence as a least-cost path fndng problem. Gven an mage scannng lne S = p, y, DP fnds an optmal path through a D slce y { ( } C( ; y ; of the 3D cost-volume. The optmal path s equvalent to a dsparty assgnment functon d( p that mnmzes the global energy functon: ( = data ( + smooth ( E d E d E d (1 The smoothness term mplctly assumes that the dsparty changes smoothly and mposes penalty for abrupt change of dsparty. We modfed the tradtonal dynamc programmng (DP algorthm and proposed the WTA gudance-based DP algorthm. The proposed DP algorthm wll be descrbed n detals below: Consder the scan lne y n the reference mage. Each pxel p = ( x, y s vsted from left to rght. Its energy correspondng to all dspartes d s computed. At the same tme, mnmum energy and the poston of the correspondng dsparty are saved. Ths can be gven by: (,, = C( x, y, d + mn ( M ( x 1, y, d' + d d' M x y d d' [ 0, d ] max λ (15 Where p = x 1, y, whch s the neghborng pxel of p. The dynamc programmng algorthm mentoned above has a computaton load of OWD for each scan lne, where W denotes the mage ( d denotes the dsparty of ( wdth and D denotes the dsparty range. So t s not suted for real-tme applcatons. To reduce the computatonal load, we consder the dsparty smoothness assumpton. By assumng that the dsparty of p s neghborng pxel s close to that of p and lmtng the value of d wthn d 1, d, d + 1, we obtan the modfed equaton as follows: { } (,, = C( xyd,, + mn ( M ( x 1, y, d' + d d' M xyd d' [ d 1, d, d= 1] λ (16 The computatonal complexty of the modfed algorthm s OWD (, but the dsparty computed by the smplfed algorthm changes too slowly n the depth dscontnuty areas. Ths may blur the depth edges and cause the scan lne effect. To avod the over-smoothng phenomenon, the orgnal dsparty mage from the WTA method s used to gude the dynamc programmng process. The core dea s to provde an addtonal dsparty canddate for the dynamc programmng process by usng WTA method. The orgnal dsparty mage d 0 s computed by (17. For the pxel 141

4 p = ( x, y, let 0 ( 1, of p : ( = C( x y d d x y be the fourth dsparty canddate M x, y, d,, + mn 1,, ' λ ' (17 d' d 1, d, d+ 1, d0 ( x 1, y ( M ( x y d + d d Because the orgnal dsparty mage d 0 has been computed at the begnnng, the computatonal complexty of the modfed dynamc programmng algorthm (WTA-DP s not ncreased, whch s stll d 0. D. Dsparty post-processng There are some msmatches n the orgnal dsparty mages, so t needs to be post-processed. Frst, left-rght consstency check s adopted to detect occluded ponts. Let dl ( p denotes the left dsparty mage and dr ( p denotes the rght dsparty mage. If the dsparty of the pont s not consstent wth that of ts correspondng pont, d p d p d p. Then we regard the pont p as the ( ( ( L R L occluded pont and label ts dsparty value as zero. Next, we search the scan lne, whch pont p locates on, for the frst non-occluded pont at ts left and rght drectons, Afterwards the occluded pont s flled by selectng the smaller dsparty value as the dsparty of the occluded pont. Fnally, the weghted medan flter s used to smooth the dsparty mage and obtan the fnal dsparty mage. III. RESULTS To valdate the effectveness of the proposed algorthm, C Language was used to mplement the proposed algorthm. The experments were carred out on the standard stereo mage pars from the recognzed Mddlebury Platform [17] for testng stereo matchng algorthms. Ths webste provdes four groups of baselne color mage pars: Tsukuba, Venus, Teddy, and Cones. The quantfed matchng errors are obtaned by comparng the expermental results wth the ground truth, thus enablng the algorthm s accuracy to be evaluated objectvely. Unless specfed descrpton, the parameter settng of all experments s as follows: { λad, λc, λg, β, r, ε } = { 45,30,15,50,9,0.0001} (18 Fg. 3 ntutvely shows the accuracy of the proposed algorthm. The expermental results of Tsukuba, Venus, Teddy, and Cones are sequentally gven n Fg. 3(a-(d. In ths fgure, the frst column s the orgnal mage. The second column dsplays the related ground truth. The dsparty mage from the proposed algorthm s gven n the thrd column. The fourth column presents the msmatched pxel mage of the proposed algorthm. In the fourth column, the large whte areas are the correctly matched ponts, whereas, the grey and black areas represent the msmatched ponts n the occluded areas and the non-occluded areas, respectvely. It can be seen that the dsparty mage of the proposed algorthm s smooth overall, and t s capable of achevng desred matchng results n the less-textured areas and dsparty edges. 14

5 (a (b (c (d Fgure 3. The comparson of the results. (a The four reference mages, from top to bottom, they are Tsukuba, Venus, Teddy, and Cones. (b Ground Truth maps. (c Matchng results usng the proposed method. (d Error maps usng the proposed method. TABLE I. AVERAGE PERCENTAGE OF BAD PIXELS WITH DISPARITY ERROR THRESHOLD OF 1 USING THE MIDDLEBURY BENCHMARK Algorthms Tsukuba Venus Teddy Cones Average nooocc all dsc nooocc all dsc nooocc all dsc nooocc all dsc Error Proposed DTAggr-P [18] HEBF [19] GlobalGCP [0] ASSW [1] The obtaned dsparty mage can be compared wth the ground truth to evaluate the accuracy of the algorthm objectvely by defnng the accuracy as the proporton of the msmatched pxels. And the msmatched pxel s defned as the pont whose absolute dfference wth the ground truth s larger than 1: 1 B = ( d x y d x y > δ (19 N C (, T (, d ( xy, dt Where dc (, (, x y denotes the computed dsparty mage, x y represents the true dsparty map, N denotes the total number of pxels, and δ d denotes the specfed error threshold. The pont whose matchng dfference wth the true dsparty map s over 1 pxel s regarded as the msmatched pont. Table I provdes the percentage data of msmatched pxels for the proposed algorthm when δ d = 1. From the table, t can be seen that the matchng accuracy of the proposed algorthm s hgher than the compared stereo matchng algorthms. 143

6 IV. CONCLUSION A novel local stereo matchng algorthm whch can obtan the dsparty mage accurately s proposed n ths paper. A combned matchng cost based on AD and mproved color census transform s adopted to overcome the dsadvantages of the sngle matchng cost. We perform the adaptve cost aggregaton by flterng the cost volume usng the guded mage flter. Moreover, a modfed dynamc programmng algorthm s used to weaken the scannng lne effect from the obtaned dsparty mage. The expermental results on Mddlebury platform show that our method performs well among the local stereo matchng methods. ACKNOWLEDGMENT Ths research s funded by the Natonal Natural Scence Foundaton of Chna (NSFC under grants No REFERENCES [1] D. Scharsteren and R. Szelsk, A taxonomy and evaluaton of dense two-frame stereo correspondence algorthms, Internatonal Journal of Computer Vson, vol. 47, no. 1, pp: 7-4, 00. [] S. P. Zhu, Z. L and Y. Yu, Vrtual vew synthess usng stereo vson based on the sum of absolute dfference, Computers and Electrcal Engneerng, vol. 40, no. 8, pp , November 014. [3] S. P. Zhu, L. Y. L and Z. K. Wang, A novel fractal monocular and stereo vdeo codec wth object-based functonalty, EURASIP Journal on Advances n Sgnal Processng, vol. 01, Artcle number 7, pp. 1-14, October 01. [4] S. P. Zhu and Y. Gao, Noncontact 3-D coordnates measurement of cross-cuttng feature ponts on the surface of a large-scale workpece based on the machne vson method, IEEE Transactons on Instrumentaton and Measurement, vol. 59, no. 7, pp , July 010. [5] S. P. Zhu, J. C. Fang, R. Zhou, J. H. Zhao and W. B. Yu, A new noncontact flatness measurng system of large -D flat workpece, IEEE Transactons on Instrumentaton and Measurement, vol. 57, no. 1, pp , December 008. [6] S. P. Zhu and Z. L, Local stereo matchng usng combned matchng cost and adaptve cost aggregaton, KSII Transactons on Internet and Informaton Systems, vol. 9, no. 1, pp. 4-41, January 015. [7] S. P. Zhu, Y. S. Hou, Z. K. Wang and K. Belloulata, Fractal vdeo sequences codng wth regon-based functonalty, Appled Mathematcal Modellng, vol. 36, no. 11, pp , November 01. [8] J. K. Km, K. M. Lee, and B. T. Cho, A dense stereo matchng usng two-pass dynamc programmng wth generalzed ground control ponts, IEEE Internatonal Conference on Computer Vson and Pattern Recognton, vol., pp: , June 005. [9] N. Papadaks and V. Caselles, Mult-label depth estmaton for graph cuts stereo problems, Journal of Mathematcal Imagng and Vson, vol. 38, no. 1, pp: 70-8, September 010. [10] F. Besse, C. Rother, and A. Ftzgbbon, PMBP: Patch match belef propagaton for correspondence feld estmaton, Internatonal Journal of Computer Vson, vol. 110, no. 1, pp: -13 October 013. [11] X. Me, X. Sun, and M. Zhou, On buldng an accurate stereo matchng system on graphcs hardware, IEEE Internatonal Conference on Computer Vson Workshops, pp , November 011. [1] K. Yoon, S. Kweon, Locally adaptve support weght approach for vsual correspondence search, IEEE Internatonal Conference on Computer Vson and Pattern Recognton, vol., pp: , June 006. [13] A. Hosn, M. Bleyer, and M. Gelautz, Local stereo matchng usng geodesc support weghts, IEEE Internatonal Conference on Image Processng, pp , August 009. [14] K. He, J. Shun, and X. Tang, Guded mage flter, IEEE Pattern Analyss and Machne Intellgence, vol. 35, no. 6, pp: , June 013. [15] A. Hosn, C. Rhemann, and M. Bleyer, Fast cost-volume flterng for vsual correspondence and beyond, IEEE Pattern Analyss and Machne Intellgence, vol. 35, no., pp: , February 013. [16] Q. Yang, P. J, D. L, S. Yao, and M. Zhang, Fast stereo matchng usng adaptve guded flterng, Image and Vson Computng, vol. 3, no. 3, pp: March 014. [17] D. Scharsten, R. Szelsk, Mddlebury Stereo Vson Page, [18] C. Pham, J. Jeon, Doman transformaton-based effcent cost aggregaton for local stereo matchng, IEEE Transactons on Crcuts and systems for Vdeo Technology, vol. 3, no. 7, pp: , July 01. [19] Q. Yang, Hardware-effcent blateral flterng for stereo matchng, IEEE Transactons on Pattern Analyss and Machne Intellgence, vol. 36, no. 5, pp: , May 014. [0] L. Wang, R. Yang, Global stereo matchng leveraged by sparse ground control ponts, IEEE Internatonal Conference on Computer Vson and Pattern Recognton, pp , June 011. [1] Y. Xu, Y. Zhao, M. J, Local stereo matchng wth adaptve shape support wndow based cost aggregaton, Appled optcs, vol. 53, no.9, pp: ,