THE RENDERING OF BUILDING TEXTURE FROM LAND-BASED VIDEO

Transcription

1 THE RENDERING OF BUILDING TEXTURE FROM LAND-BASED VIDEO Zuxun Zhang, Zhzhong Kang School o Remote Sensng and Inormaton Engneerng, Wuhan Unv., Wuhan 40079, P.R. Chna - (zxzhang, zzkang)@supresot.com.cn KEY WORDS: Renderng, Moble Mappng; Image Sequences; Texture; Mosac; Rectcaton; Dstorton ABSTRACT: Moble mappng s a hot ssue n the communtes o photogrammetry and computer vson nowadays. Moble mappng systems represent a sgncant advance n ntegrated, mult-sensor dgtal mappng technology, provdng an nnovatve path towards rapd and cost-eectve collecton o hgh-qualty and up-to-date spatal normaton. Thereore how to acqure geometrc and texture normaton rom the mages or reducng human operatons and enhancng ecency s a challenge n moble mappng data processng. In ths paper, the renderng o buldng texture rom land-based vdeo s mplemented. Frstly, some preprocessng o mages s mplemented. The occluson o tree s removed because t goes aganst the extracton o lnes rom buldngs. Close-ranged DC mages are greatly aected by lens dstorton, so lens dstorton s calbrated and the nluence s elmnated accordngly. Then, Land-based Vdeo mage sequences are automatcally rected to elmnate nonlnear dstorton thanks to the large oblquty o the posture o DC. The buldng texture s nally mosac rom rected mage sequence and rened by the process o texture analyss. 1. INTRODUCTION Real texture data,.e. mages, are requred to make buldng objects more genune n D cty modelng(dcm). Textures o terrans and the top o buldngs are acqured rom aeral mages presently. The textures o buldng acades, however, are stll acqured rom mages taken manually wth dgtal camera. Snce lackng ndex o mages, the processes o searchng mages, preprocessng mages and correspondng mages wth buldngs are mplemented manually. As a result, the problems o low ecency and hgh labour ntensty emerge. Thereore, t s becomng a hot ssue n communtes o dgtal photogrammetry, GIS and computer vson that how to solve these problems and relevant research are carred out (Debevec.P et al., 1996 ; Cpolla.R et al., 1999). The acquston o land-based vdeo or buldngs has the characterstcs o convenence and rapdness. The resoluton o textures obtaned rom land-based vdeo s urthermore hgher than those obtaned rom helcopter-based vdeo. Thereore, t wll be aded greatly by the automatc processng o textures rom land-based vdeo that the rapd reconstructon o texture or buldng acades n DCM. Accordngly, t s dscussed n ths artcle that the renderng o buldng textures rom landbased vdeo. Frstly two groups lnes are extracted rom a sngle mage. The occluson o tree s removed employed the normaton o hue and lnes wth regular orentaton because t goes aganst the extracton o lnes rom buldngs. Close-ranged DC mages have nonlnear dstorton due to the large oblquty o the posture o DC and obvous lens dstorton. Radal lens dstorton parameter k1 s estmated because the major part o the lens dstorton s elmnated by applyng t only. Due to lens dstorton, straght lnes n object space appear as slghtly curved lnes n the mage. Here we break up long curved mage lnes nto smaller sectons extracted later. Thereore, those sectons extracted along the same curved mage lne are bound to be collnear. A model o adjustment system s then evolved accordng to ths concluson to estmate the radal lens dstorton parameter k1.to elmnate the nonlnear dstorton cause by large oblquty o the posture o DC, the automatc rectcaton o mages s mplemented. The key o automatc rectcaton o mages s how to acqure camera parameters rom the mage normaton. Snce the shape o buldngs s always regular, there are many straght lnes n buldng acades parallel to X-axs and Y-axs n object space. Namely the orentaton o these lnes s known. Thereore, t s presented to calbrate DC parameters that an algorthm combned wth the constrant o straght lnes bundle and the constrant o known orentaton o parallel lnes n object space. The automatcally mosac process o açade texture s nally mplemented employed the strategy o selectng mosac pont par combned wth correlaton coecent and geometrc constrant o relatve orentaton. Tone adjustment s completed or the color derence between each two mages contrbutng to llumnaton.. ESTIMATION OF LENS DISTORTION As we know, close-ranged DC mages are greatly aected by lens dstorton and the nluence should be elmnated accordngly. In ths artcle, an algorthm that estmates lens dstorton accordng to the curvature o straght lnes thanks to lens dstorton. Because the major part o the lens dstorton s A a. A straght lne extracted b. Lnes extracted ater dmdaton C' d C c. The curvature o straght lnes Fg.1 The determnaton o curvature o straght lnes caused by lens dstorton B

2 elmnated by applyng parameter k 1, only parameter k 1 s estmated n the artcle. As Fg.1a llustrated, a straght lne s extracted n the mage whle the actual lne s slghtly curved due to lens dstorton. Thereore the lne extracted s dmdated and each lne secton s extracted agan. The extracted result s llustrated as Fg.1b. The two lnes extracted cannot obvously satsy the condton o collnearty thanks to lens dstorton. As Fg.1c llustrated, d denotes the dsplacement o pont C to pont Ccaused by lens dstorton. Thereore, d can be employed to calbrate the lens dstorton. A model o least adjustment s then evolved accordng to ths concluson to estmate the radal lens dstorton parameter k1.. OCCLUSION REMOVAL Snce the occluson o trees goes aganst the extracton o lnes rom buldngs, the occluson area should be automatcally dstngushed and removed..1 Occluson Removal Employed Hue Inormaton As Fg. a llustrated, t s taken or granted that color normaton can be employed to remove the occluson o trees because o the color smlarty o trees. Thereore, a 80~00 range o hue value s chosen to determne whether a pxel les n the occluson area. The result o occluson removal s llustrated n Fg.b. As we can see, some pxels n the red ellpse area whch hue values are out o the range we chosen are remaned although they are lyng n the occluson area, meanwhle some pxels n the green ellpse area vce versa. A concluson s drawn, thereupon, that the result o occluson removal only employed hue normaton comes out lackng stablty. Here t s presented to mprove the stablty that an algorthm o Occluson removal employed normaton o hue and lnes.. Occluson Removal Employed Inormaton o Hue and Lnes Lnes are Frstly extracted n the orgnal mage as Fg. a llustrated. As we can see, lnes on the buldng açade are b. Occluson removal employed hue normaton a. Lnes extracted c. Occluson removal employed normaton o hue and lnes Fg. Occluson removal almost correspondng to object lnes parallel to X-axs and Y- axs n the object space, thereore ther orentaton are regular whle lnes n the trees vce versa. The normaton o lnes wth regular orentaton, thereupon, can be used to detect the occluson area combned wth hue normaton. The process s presented below: 1) Dvde the orgnal mage nto rectangular grds wth grd sze o 18 pxels 1 pxels. ) Count the number o pxels whch hue values are n the range o 80~00 n every grd. I the number s 80% o total, ths grd s deemed to le n occluson area and removed. Otherwse, the grd wll be quartered and the same operaton wll mpose on new grds. ) The densty o lnes wth regular orentaton n the whole mage s determned by Eqs.1 n = = 1 L ρ l (1) S Wth, ρ l : The densty o lnes wth regular orentaton n the whole mage; L : The length o lne (pxel); S: The area o the whole mage(pxel ). 4) Calculate the denstes o lnes wth regular orentaton respectvely n grds havng not been removed. I the densty o a grd s smaller than that o whole mage, ths grd s deemed to le n occluson area and removed. The result o occluson removal, as llustrated n Fg.c, s obvously better than that llustrated n Fg.b. 4. THE AUTOMATIC RECTIFICATION OF FACADE TEXTURE p: xy Snce buldng acades are planar, there are two groups o lnes p 0 : x 0 y 0 parallel to X-axs S and Y-axs n object space respectvely on Fg. The process o rectcaton each o them. As a result, only ocus and three angular orentaton parameters ϕ, ω, k can be calculated accordng to theores o y vanshng pont geometry. Thereore Y x açade textures are rected employed P 0 S and ϕ, ω, k wth prncple pont xed X at the center o Z mage. The process o rectcaton s Fg. 4 the relatonshp between p 0 and object space coordnate system

3 llustrated as Fg.. For close-ranged DC mages, the coordnate system o object space s set up as Fg.4 llustrated. The açade textures are rected onto plumb plane accordngly. 4.1 The Determnaton o Vanshng Pont and Intal Values o DC Angular Parameters. Vanshng pont V x, y ) s the ntersecton pont o a set o ( V V straght lnes whch are parallel n object space. It means that each straght lne such as lne j belong to ths set must be passed through the vanshng pont as shown n Fg.5. So Eqs. s evolved. ( xv x ) ( yv y ) d = ( y j y ) ( x j x ) () s s V where, d s the dstance o pont j to lne V. From Eqs. a model o adjustment system o observatons and parameters s deduced to determne vanshng pont. Then ntal values o DC angular parameters are decomposed accordng to vanshng pont geometry (B. Caprle. et al., 1990). 4. The Calbraton o DC Angular Parameters Two algorthms o calbraton o DC angular parameters are presented here The Calbraton Employed the Constrant o Straght Lnes Bundle: Accordng to vanshng pont geometry vanshng pont coordnate ( x V, y V ) n Eqs. s replaced wth nteror and exteror orentaton (angular) parameters. Then a model o adjustment system o observatons and parameters employed the constrant o straght lnes bundle s deduced to calbrate DC angular parameters whle nteror orentaton parameters are xed n the adjustment system. Because the ntal value o orentaton parameters s decomposed rom vanshng pont coordnates, the accuracy s not always hgh owng to the qualty o mages and the error o lne extracton. Thereore, It has a small convergent radus that the model o adjustment system employed the constrant o straght lnes bundle. As a result, the model o adjustment system s not robust. In ths artcle, t s ntroduced to control the adjustment system that the constrant o known orentaton o parallel lnes n object space. 4.. The Model o Adjustment System Controlled by the Constrant o Known Orentaton o Parallel Lnes n Object Space: There are two group lnes parallel to X-axs and Y-axs n object space respectvely. Ther projectons n mage space are nvolved n the adjustment system, consequently the projectons o these two group lnes n rected mage should be parallel to X-axs and Y-axs n object space respectvely n deal nstance (As llustrated n Fg. and Fg.4). Namely Eqs. and 4 are deduced. θ V d Fg. 5 The constran o Straght lnes bundle j In X drecton: b b x b y b 1x + b y b 1 j + j = () c x + c y c c x + c y c 1 In Y drecton: 1 j a x a y a a x a y a 1 m + m 1 n + n = (4) c x + c y c c x + c y c 1 m m 1 n Wth: ( x y ),( x j y j ): the coordnates o two end ponts o lne parallel to X-axs ( xm y m ),( xn y n ): the coordnates o two end ponts o lne parallel to Y-axs : ocal length a1 b1 c1 a b c a b : rotaton matrx c Eqs. and 4 can be control condton or the calbraton o DC angular parameters. Thereore t s deduced that a model o adjustment system combned wth constrants o straght lnes bundle and known orentaton o parallel lnes n object space. As the expermental results showng, the model o adjustment system deduced here has the characterstcs o large convergent radus and hgh stablty. 5. THE AUTOMATIC MOSAIC OF FACADE TEXTURE Usually or bg buldngs, açade texture s covered by several mages nstead o only one mage. Thereore to acqure the whole açade texture, mosac process s requred. Snce açade textures are all rected onto plumb plane, only the dsplacements d x, d y n X-axs and Y-axs respectvely are needed to determne n mosac process. Thereupon, the queston s how to determne the dsplacements d x, d y. Fg.6 Correspondng ponts matched Frstly, correspondng ponts are automatcally matched between each two adjacent mages. As Fg.6 llustrated, there are 95 correspondng pont pars acqured. Because o the smlarty o texture, some alse pont pars are also obtaned. Thereore, an optmal pont par should be pcked out as mosac pont par by whch the dsplacements d x, d y are determned. The strategy o selectng mosac pont par s then presented. 5.1 Selectng Mosac Pont Par by Maxmum Correlaton Coecent j n

4 As we know, correlaton coecent s employed to determne whether the pont par o nterest s correspondng. The larger the correlaton coecent s, the more lkely t s that the pont par o nterest s correspondng. Consequently t s taken or granted that the pont par wth maxmum correlaton coecent should be mosac pont par. The dsplacements d x, d y are calculated by the mosac pont par and açade textures are mosac as Fg.7 llustrated. The result s not satsyng because even the pont par wth maxmum correlaton coecent s not really correspondng thanks to the smlarty o texture. Accordng to the correspondng actor, the mosac pont par s obtaned and açade textures are mosac as Fg.8 llustrated. The result s obvously satsyng. 5. Tone Adjustment Because o the llumnaton, the color derence exsts between each two mages. In ths paper, t s presented that an algorthm o tone adjustment by maxmum ntensty derence detected to elmnate the color derence. Frstly, the ntersectonal area s determned accordng to the correspondng ponts matched. The maxmum ntensty derence s then detected n ths area and the tone o the rght mage s adjusted by lnear mappng accordng to t. 6. EXPERIMENTAL RESULTS Fg.7 The mosac result employed the strategy o maxmum correlaton coecent Hence to acqure a real correspondng pont par, geometrc constrant needs to be ntroduced. 5. Selectng Mosac Pont Par Combned wth Correlaton Coecent and Geometrc Constrant Relatve orentaton process s mplemented employed the correspondng pont pars matched above and the vertcal parallax Q o every correspondng pont par s acqured meanwhle. Thereore the correspondng actor s calculated as below. Q F = 1+ ρ (5) Façade textures are acqured rom mage sequence extracted rom vdeo obtaned by DV. The sze o mage s 70 pxels 480 pxels. 6.1 The Comparson between Two Knds o Least Square Adjustment Models To compare the derence o convergent radus and stablty between the model employed the constrant o straght lnes bundle (Model A) and the model controlled by the constrant o known orentaton o parallel lnes n object space (Model B), (a) Model B (b) Model A Fg.9 Rected Images (Hstogram o Angle) wth: F : the correspondng actor o pont par, the larger the more lkely t s that pont par s correspondng Q : the vertcal parallax o pont par 1 s, F ρ : the correlaton coecent o pont par, the value o ρ s 100 when pont par s ully correlated (a) Model B (b) Model A Fg. 10 Rected Images (Hstogram o Angle wth the Geometrc Constran o Normal Vector to Interpretaton plane) Fg.8 The mosac result employed the strategy combned wth correlaton coecent and geometrc constrant (a) Model B (b) Model A Fg. 11 Rected Images (Hstogram o Angle wth the Geometrc Constran o Normal Vector to Interpretaton plane) the least square adjustments employed these two model are mplemented based on the results o groupng lnes by three methods,.e. groupng lnes by angle hstogram, by angle hstogram wth geometrc constrant o normal vector to nterpretaton plane and by ntal value acqured artcally. The results o rectcaton are llustrated as Fg.9~11.

5 vanshng pont coordnates, angular parameters and ocus s shown n Fg.1 based on the results o least square adjustment above (The hollow denotes model A whle the sold denotes model B). vanshng pont X (Model B) From the dstrbutng above, a concluson s easly drawn that model B has larger convergent radus and hgher stablty than those o model A. Namely, the values acqured by model B are generally optmal whle those acqured by model A are locally optmal. 6. The Elmnaton o Lens Dstorton (a) vanshng pont y vanshng pont Y (Model B) vanshng pont Y (Model A) vanshng pont X (Model A) The algorthm o renderng o buldng açade texture rom land-based mage sequence has been presented. Some experments has been conducted wth land-based mage sequence acqured by DV. The result s satsyng and several Fg. 1 Convergence Dstrbutng x o o ω angular parameters (Model B) ϕ (c) ocus angular parameters (Model A) (b) angular parameters κ There are 408 lnes extracted n the mage as Fg.1a llustrated. And then 68 lnes whch lengths are over 60 pxels are dmdated and extracted agan. The result s shown n Fg.1b. The lens dstorton s estmated used those lnes extracted ater dmdaton and the result o correcton o lens dstorton s llustrated n Fg.1c. As we can see, The lnes curvng thanks to lens dstorton obvously became straght ater lens dstorton correcton. That s the lens dstorton was elmnated. a. maxmum correlaton coecent b. combned wth correlaton coecent and geometrc constrant Fg.14 the comparson o mosac results o açade texture a.lnes extracted b. Lnes extracted ater dmdataon Fg1. Lens dstorton correcton 6. The Mosac o Façade Texture Fg.14 s the comparson o mosac results o açade texture employed two strategy presented above. The mosac result used the strategy o selectng mosac pont par combned wth correlaton coecent and geometrc constrant s obvously much better than that employed the strategy o selectng mosac pont par by maxmum correlaton coecent. Ater the processes o tone adjustment and texture analyss, more rened açade texture s acqured as Fg.15 llustrated. 7. CONCLUSIONS c. Result o lens dstorton correcton conclusons are drawn as below. The curvature o straght lnes caused by lens dstorton can be detected used those lnes extracted ater the dmdaton o every long straght lne. Thereore the lens dstorton can be estmated accordng to the curvature detected. The method to estmate lens dstorton presented n ths artcle s proven to be straghtorward and eectve by the experments carred out. t s taken or granted that color normaton can be employed to remove the occluson o trees because o the Fg. 15 Rened Texture or Buldng Facade color smlarty o trees. However the result o occluson

6 removal only employed hue normaton comes out lackng stablty. As we can see, lnes on the buldng açade are almost correspondng to object lnes parallel to X-axs and Y-axs n the object space, thereore ther orentaton are regular whle those n the trees vce versa. The normaton o lnes wth regular orentaton, thereupon, can be used to detect the occluson area combned wth hue normaton. It s proven that more stable result o occluson removal can be acqured employed ths method. Snce t has a small convergent radus that the model o least square adjustment employed the constrant o straght lnes bundle, the model s senstve to ntal value whch tends to be aected by the qualty o mage and the errors o lne extracton. However, t has a larger convergent radus and hgher stablty that the model o adjustment system controlled by the constrant o known orentaton o parallel lnes n object space presented n ths artcle, thanks to the strctly geometrc constrant to control the calbraton o DC parameters. Two strateges o selectng mosac pont par are compared n ths artcle. It has been proven by experments that the strategy combned wth correlaton coecent and geometrc constrant o relatve orentaton s better than that takng only correlaton coecent nto consder. REFERENCES B. Caprle and V. Torre, Usng Vanshng Ponts or Camera Calbraton, IJCV, 4(), pp Cpolla.R. Robertson. D.P and Boyer.E.G. Photobulder, (June) D models o archtectural scenes rom uncalbrated mages. Proc. IEEE Internatonal Conerence on Multmeda Computng and Systems, Frenze, volume I, pp.5-1. Debevec.P Taylor.C.J Malk.J., Modelng and Renderng Archtecture rom Photographs: A Hybrd Geometry- and Image-Based Approach, Proc. O SIGGRAPH 96. ACKNOWLEDGMENTS We wsh to thank the project o Natural Scentc Foundaton o Chna ttled Three-Dmensonal Cty Model Generaton by Vdeogrammetry wth the seral number or ts support to our research.