DENSE CORRESPONDING PIXEL MATCHING USING A FIXED WINDOW WITH RGB INDEPENDENT INFORMATION

Transcription

1 ISRS Annal of the hotograetry, Reote Sening and Spatial Inforation Science Volue I-4, 202 XXII ISRS Congre 25 Augut 0 Septeber 202, Melbourne, Autralia DENSE CORRESONDING IXEL MATCHING USING A FIXED WINDOW WITH RGB INDEENDENT INFORMATION Teng Hu, Huayi Wu* State Key Laboratory of Inforation Engineering in Surveying, Mapping and Reote Sening, Wuhan Univerity, Wuhan, huteng@whueducn, wuhuayi@whueducn, obleap@gailco KEY WORDS: Iage proce Dene correponding pixel atching, Markov Rando Field, Fixed Window, RGB independent Inforation ABSTRACT: A fixed window baed algorith i propoed for dene correponding pixel atching between epipolar iage The algorith ake full ue of three colour coponent R, G and B, and iplifie deterining the optial window In order to reduce the ipact of obviou colour difference between correponding pixel the propoed algorith ue ore inforation fro neighbouring pixel and adjut the contribution of the central pixel and their neighbouring pixel Two pair of epipolar iage are teted uing the fixed window baed algorith The experiental reult including boundary teting, apled correponding pixel evaluation and 3D point cloud check, deontrate the effectivene of the propoed algorith INTRODUCTION Correponding pixel atching i one of the baic topic in photograetry Acquiring a large quantity of accurate correponding pixel between epipolar iage reain a difficult proble Four ajor interference caue the difficulty: obviou colour difference between correponding pixel enor noie, oved object and occluion Generally, obviou colour difference between correponding pixel are the priary error ource for atching a pair of epipolar iage Thu a reliable algorith ut eriouly reduce the ipact of thi priary error ource Current dene window-baed pixel-to-pixel atching technique contain two ain type: adaptive-window and ultiple-window ethod Adaptive-window ethod find a proper optial upport window for each pixel Kanade and Okutoi (994) elect an optial window by evaluating local change of intenity and diparity, the ethod depend on the initial diparity Meanwhile, the upport window i a rectangle and therefore not uitable for pixel in dicontinuou area Boykov et al (998) choe an arbitrarily haped connected window uing plauibility hypothei teting for each pixel, but the coputation wa very coplex Vekler(2002,2003) found a few ueful window ize and hape which work well in deterining upport ize, however the ethod i not generalizable Yoon (2006) preented an adaptive upportweigh approach for correponding earch; the approach can produce accurate piecewie ooth diparity ap but a long runtie i inevitable Multiple-window ethod chooe an optial upport window fro everal predefined window having the ae hape with different poition Fuiello et al (997) aigned each pixel 9 different window and retained a the optial window the one window with the allet atching cot Kang et al (200) exained each window containing the pixel of interet in ulti-view proceing Although thee ethod perfor well in building relationhip between atching pixel under oe condition they have iitation: the local upport window i not generalizable, require any uer-pecified paraeter or need long runtie Baed on the baic Bayeian rule, a Markov Rando Field (Gean and Gean, 984) baed algorith i forulated for dene correponding pixel atching between epipolar iage The ethod take the independent phyical propertie of RGB inforation into conideration The RGB difference fro neighbouring pixel are treated independently rather than a cobined value In thi ethod, for each pair of atching pixel 2 RGB difference fro four pair of neighbouring pixel deterine 2 prior probabilitie of atching pixel in RGB colour pace, while one cobined value between two atching pixel i deterined a a one likelihood probability in CIELab colour pace The algorith ake three iproveent ) treat RGB colour coponent independently a baic inforation ource; 2) reduce the direct negative ipact of the correponding point colour difference and increae the contribution of neighbouring pixel colour difference; and 3) ue a fixed window working in the firt-order Markov Rando Field Iproveent ) and 2) work together to depre the interference of correponding pixel colour difference and iprove the correct atching rate of pixel in the regional background Iproveent 3) iplifie optial window deterination The ret of thi paper i organized a follow Section 2 preent detail of the propoed algorith Section 3 contain experiental data, reult and evaluation Finally, Section 4 draw concluion with a dicuion of potential future work 2 THE ROOSED ALGORITHM For a given pair of epipolar iage I={I L, I R }, where I L and I R are the left (reference) and the right (target) iage, the joint poterior probability of a caual pixel in the reference iage with a given diparity dx i expreed a: ( ( dx ) I ) ( ( dx ( I ( dx ( I ) () Here, (() and (I () are the ynthetic likelihood probability and ynthetic prior probability repectively In the firt order Markov Rando Field, each atching pixel contain four neighbouring pixel; additionally, each neighbouring pixel ha four neighbouring pixel Thee 3 pixel build a fixed window * Correponding author 89

2 ISRS Annal of the hotograetry, Reote Sening and Spatial Inforation Science Volue I-4, 202 XXII ISRS Congre 25 Augut 0 Septeber 202, Melbourne, Autralia In thi paper, the propoed algorith take the independent phyical propertie of RGB inforation into conideration The RGB difference of neighbouring pixel are treated independently rather than a cobined value 2 RGB difference fro four pair of neighbouring pixel deterine 2 prior probabilitie in RGB colour pace, while one cobined value between two atching pixel i deterined a a likelihood probability in CIELab colour pace Thu () can be rewritten a: ( ( dx ) I ) CIELab ( I ( ( dx S { r, g, b } ( ( dx A et of poterior probabilitie containing all the poible value for reference pixel were deterined Then, the larget poterior i the optial reult 2 Likelihood robability According to (2), the ynthetic likelihood probability (() i: ( ( dx ( I ( ( dx CIELab (2) (3) For atching pixel and (, the colour difference between the in CIELab colour pace can be cobined a: * * 2 * * 2 * * ( ( L L ( ( ( a a ( ( ) ( b b ( ( dx Then, ( i convert to a proper value a: ( ( dx exp( D ( ( dx ) ) 2 (4) (5) (5) indicate that (() i deterined by paraeter D and the cobined value of the atching pixel colour difference in CIELab colour pace For the ae colour difference between atching pixel enlarging D reduce the enitivity of the correponding pixel actual difference 22 rior robability In thi paper, we olve prior probabilitie in RGB colour pace There are three tep: 22 The difference initialization of neighbouring pixel For a caual atching pixel, and (, auing that a caual pixel n exit in their fixed window The difference between n and n( in RGB colour pace are expreed a: ( n ( dx I L ( ) ( n ) I R ( ) ( n ( dx { r, g, b } (6) Here, a iple unitary proceing i ued to converted colour difference to value in interval [0, ] The value of RGB colour difference are now defined a r / g / b ( D2 n( ) exp( ( r / g / b ( n( ) / (7) The colour difference of neighbouring pixel work in RGB colour pace, unlike the per-pixel cale-invariant feature tranfor (per-pixel SIFT, Liu, 20) or other color-invariant pace algorith The per-pixel SIFT aign a 28-dienional vector to each pixel a it baic atching feature, where too any dienion ake it difficult to et paraeter The colour pace ued in other colour-invariant pace ethod i alo copoed by three vector; however, thoe three coponent do not have ae range or weigh, and thu they cannot be converted to a proper value uing a ingle paraeter In contrat, the RGB colour pace i iple and avoid the difficultie when any paraeter ut be et to convert each coponent to it proper value in the other colour-invariant pace ethod 222 Coputing rior In the propoed algorith prior probabilitie are deterined by 2 RGB difference of four connected neighbouring pixel Auing the prior ent fro the neighbouring pixel q to central pixel in colour with a given value dx i (, then, the ynthetic prior probability of joint poterior probability i expreed a: ( I ( ) S { r, g, b} ( I ( ( ) Here, the prior ( i olved a: { r, g, b}, qn exp( D 223 rior noralization 2 { r, g, b}, qn ( q( ( (8) pn ( q) \ ( p, (9) Conidering the dicontinuou boundary, a paraeter λ i et to copel the dicontinuou area { r, g, b}, qn ax( (, ax( Here, dax i the given earching range for The paraeter λ have two choice: J J 2 ( ( dx[0, d ax] (, q ) bund (, q ) bund ) ) (0) () If a colour coponent ditance between central pixel and it neighbouring pixel i larger than the threhold bund, a jup exit The paraeter bund ut be et to a uitable value o that the inner colour juping within a natural continuou object i ignored and boundarie between object are identified J and J 2 are deigned to enure that the updated prior i within a proper range for the ret prior J i larger than J 2 o that the algorith enlarge the updated prior between two juping pixel and therefore enlarge the poterior probability to copel their diiilarity Then, prior are noralized a: { r, g, b}, qn ( d ax) /, qn dx[0, d ax] ( ( (2) 90

3 ISRS Annal of the hotograetry, Reote Sening and Spatial Inforation Science Volue I-4, 202 XXII ISRS Congre 25 Augut 0 Septeber 202, Melbourne, Autralia 23 oterior After 2 prior probabilitie and one likelihood probability are deterined, the joint poterior probability of reference pixel with a given value dx i expreed a: ( dx ) exp( D ( ( dx ) (, q,, dx ) { r, g, b }, q N ( ) (3) Though (3), each pixel in a reference iage acquire a group of poterior probabilitie for all poible value Adjuting paraeter D and D 2 will control the different contribution of the central pixel and their neighbouring pixel colour difference For pixel the optial olution i: MA ax( dx [ 0, d ax] ( dx (4) In uary, thi algorith retain the independence of RGB colour difference Thi algorith doe not cobine the neighbouring pixel RGB difference but treat all RGB difference a independent variable Thi technique ake full ue of RGB colour difference with no need for additional preproceing 3 EXERIMENTS Figure Two pair of epipolar iage 3 The reult calculated by the propoed algorith In thee experient paraeter D, D 2, bund, J and J 2 are all et to 80, 255, 5, 085 and 075 baed on our experiental experience The axiu reearch range for reference pixel in both two epipolar iage pair i 60 pixel After cobining all the diparitie of the pixel in the reference iage two diparity pace iage (DSI) with the ae reolution a the reference iage wa generated a hown in Figure 2 (pixel gray value = reference pixel diparity * 4) In thi paper, the propoed fixed-window baed algorith olved two pair of epipolar iage Figure how both two pair of epipolar iage belonging to the ae flight trip collected by an unanned aerial vehicle at a 3500*200 reolution with any local error ource Many correponding pixel in the rectified iage have colour difference epecially in building roof and on the ground 9

4 ISRS Annal of the hotograetry, Reote Sening and Spatial Inforation Science Volue I-4, 202 XXII ISRS Congre 25 Augut 0 Septeber 202, Melbourne, Autralia Figure2: Two DSI calculated by the propoed algorith (a) i the DSI between Figure (a) and Figure (b), (b) i the DSI between Figure (c) and Figure (d) Figure (a) and Figure (c) are the reference iage (ixel gray value = reference pixel diparity * 4) Obviouly, both of the DSI in Figure 2 have two ain part: ground and building roof For the ground, all diparitie in the background are ditributed evenly For building roof the conitency in the inner roof how local diparity iilarity The roof nonconforitie indicate that the roof have altitude dicrepancie All outward appearance noted are in accordance with the phyical condition in both epipolar iage pair 32 Boundary teting Boundarie are coon phenoena in iage The quality of the boundarie i the key to 3D recontruction For evaluating boundarie two ynoptic ap overlapped by reference iage and their DSI are hown in Figure 3 Four characteritic can be een in Figure3: The boundarie in DSI agree with the actual condition Firt, ot of building boundarie in DSI can atch the actual tructure edge Second, oe independent platfor are iolated for background area 2 Mot of occluion area are correctly calculated The diparitie of occluion pixel are in confority with the actual continuou area although they have no correponding pixel in the target iage 3 Soe local error ource uch a oved object are iolated autoatically Soe local error ource diparitie are different fro the urrounding pixel and are naturally iolated after ot pixel in the background are uccefully atched 4 Soe atching error exit in coplex area When dene local building exit, the boundarie of building are not eay to be identified; thee lead to any incorrect atche Figure 3 Two overlapped ap fro the reference iage and their DSI 33 Sapled correponding pixel evaluation In order to evaluate the accuracy of the propoed algorith,000 reference pixel were elected fro Figure (a) and Figure (b) with a regular grid (20X50), then their actual correponding pixel were anually identified fro the target iage The correponding pixel actual reult are copared with the calculated reult a hown in Table Table give error detail For iage pair Figure (a) and Figure (b), 575% of apled correponding pixel are accurate, while 882% are controlled in pixel, and 924% are controlled in jut 2 pixel In addition, the variance i

5 ISRS Annal of the hotograetry, Reote Sening and Spatial Inforation Science Volue I-4, 202 XXII ISRS Congre 25 Augut 0 Septeber 202, Melbourne, Autralia ince the axiu diparity error i 50 and 3 error are ore than 0 pixel The accuulated percentage of Figure (c) and Figure (d) i 502%, 730% and 89%; with a variance of Error (unit: pixel) Aount Accuulated ercentage 0 575/ %/502% 307/ %/730% 2 42/89 924%/89% [3, 9] 45/95 969%/94% [0, 60] 3/86 00%/00% Max error 50/55 (unit: pixel) Variance 06354/24687 (unit: pixel) Table The apled correponding pixel tatitical error for the two epipolar iage pair fro Figure The left data belong to Figure (a) and Figure (b) The right data belong to Figure (c) and Figure (d) The reult fro Figure (c) and Figure (d) are wore than the reult fro Figure (a) and Figure (b) Becaue of exiting dene local building in the left lower corner of Figure (c) and Figure (d), any boundarie could not be identified Thu any incorrect atche caue a bigger variance and wore tatitical reult 34 3D point cloud check Figure 4 how 3D view of the point cloud containing 5,620,35 patial point reulting fro the two epipolar iage pair hown in Figure and their DSI Mot incorrect atche including oved object and enor noie were reoved after the correponding pixel colour difference were anually liited The patial ditribution of the 3D point cloud followed the phyical condition in two epipolar iage pair hown in Figure Thi 3D point cloud deontrate that the propoed algorith i capable of generating dene correponding pixel of the propoed algorith; however, it ha oe weaknee Firt, it created oe wrong atche in the coplex background epecially in dene tructure-containing area given the difficulty in ditinguihing object boundarie efficiently Second, the pixel-leveled preciion lead to pixel-leveled juping in DSI and caue pace cliff in 3D point cloud In the future, we plan to judge object boundarie by aking full ue of RGB recognition of different phyical propertie and invetigate how to iprove DSI uing ub-pixel etiation REFERENCES T Kanade and M Okutoi, 994 A Stereo Matching Algorith with an Adaptive Window: Theory and Experient IEEE Tran attern Analyi and Machine Intelligence, vol 6, no 9, pp A Fuiello, V Roberto, and E Trucco, 997 Efficient Stereo with Multiple Windowing roc IEEE Conf Coputer Viion and attern Recognition San Juan, uerto Rico, USA, pp Y Boykov, O Vekler, and R Zabih, 998 A Variable Window Approach to Early Viion, IEEE Tran attern Analyi and Machine Intelligence vol 20, no 2, pp SGean and DGean 984 Stochatic Relaxation, Gibb Ditribution, and the Bayeian Retoration of iage", IEEE Tran attern Analyi and Machine Intelligence, Vol6, No6, pp72-74 SB Kang, R Szeliki, and C Jinxjang, 200 Handling Occluion in Dene Multi-View Stereo roc IEEE Conf Coputer Viion and attern Recognition, Kauai, Hawaii, USA, vol, pp 03-0 O Vekler, 2002 Stereo Correpondence with Copact Window via Miniu Ratio Cycle IEEE Tran attern Analyi and Machine Intelligence, vol 24, no 2, pp O Vekler, 2003 Fat Variable Window for Stereo Correpondence uing Integral Iage roc IEEE Conf Coputer Viion and attern Recognition, Madion, Wiconin, USA, vol, pp Figure 4 The 3D point cloud reulting fro two epipolar iage pair in Figure 4 CONCLUSION AND FUTURE WORK The propoed algorith conider independently all three colour coponent and work well in a iple fixed window Experiental reult how that it i capable of uppreing colour difference exiting between correponding pixel and correctly identifie ot of the correponding pixel Thu thi algorith can effectively olve the dene correponding point atching proble between rectified iage C Liu, J Yuen and A Torralba, 20 SIFT flow: dene correpondence acro different cene and it application IEEE Tranaction on attern Analyi and Machine Intelligence, vol 33, no5, pp K-J Yoon and I-S Kweon, 2006 Adaptive Support-Weight Approach for Correpondence Search IEEE Tran attern Analyi and Machine Intelligence, vol 28, pp ACKNOWLEDGEMENT Thi work wa upported by the National Natural Science Foundation of China under Grant , and The reult in Section 3 deontrate the validity and rationality 93