Chinese Academy of Surveying and Mapping, Beijing , China - - (zhangjx, b

Transcription

1 SEMI-AUTOMATIC EXTRACTION OF RIBBON ROADS FORM HIGH RESOLUTION REMOTELY SENSED IMAGERY BY COOPERATION BETWEEN ANGULAR TEXTURE SIGNATURE AND TEMPLATE MATCHING X. G. Lin,,, J. X. Zhng, Z. J. Liu, J. Shen Chinese Acdemy of Surveying nd Mpping, Beijing 00039, Chin - linxingguo@gmil.com, - (zhngjx, zjliu)@csm.cn School of Resources nd Environment, Wuhn University, Wuhn , Chin Commission III, WG III/5 KEY WORDS: Rod extrction; Semi-utomtic; Angulr texture signture; Templte mtching ABSTRACT: Rod trcking is promising technique to increse the efficiency of rod mpping. In this pper n improved rod trcker, sed on coopertion etween ngulr texture signture nd templte mtching, is presented. Our trcker uses prol to model the rod trjectory nd to predict the position of next rod centreline point. It employs ngulr texture signture to get the exct moving direction of current rod centreline point, nd moves forwrd one predefined step long the direction to rech new position, nd then uses curvture chnge to verify the new dded rod point whether right enough. We lso uild compctness of ngulr texture signture polygon to check whether the ngulr texture signture is suitle to e used to go on trcking. When ngulr texture signture fils, lest squres templte mtching is then employed insted. Coopertion etween ngulr texture signture nd templte mtching cn relily extrct continuous nd homogenous rion rods on high resolution remotely sensed imgery.. INTRODUCTION Extrction of rod from digitl eril/stellite imgery is not only sceniclly chllenging ut lso of mjor importnce for sptil dt cquisition nd updte of geodtses. Trditionl mnul plotting is time consuming nd expensive, so utomtic cquisition nd updte of rod dt is gretly needed. In (Bjcsy nd Tvkoli, 976; Wng nd Newkirkr, 988; Trinder nd Wng, 997; Long nd Zho, 005; Hverkmp, 00; Hinz nd Bumgrter, 003; Zhng nd Couluigner, 006; Brzohr nd Cooper, 996; Grdner nd Roerts, 00; Btz nd Schpe, 004), vrious fully utomtic pproches re proposed. But the rod chrcteristics vry considerly with ground resolution, rod type, density of surrounding ojects, nd light conditions nd so on, dding tht the limits of stte of the rt on computer vision nd photogrmetry, the desired fully utomtion could not e chieved y now, however, semiutomtic pproch tht retins the humn opertor in the loop where computer re used to ssist humn performing is considered to e good compromise etween the fst computing speed of computer nd the efficient interprettion skills of n opertor. And quite lot of promising pproches for semi-utomtic rod extrction hve een proposed in the lst two decdes. Qum (978) trcked rod y rod surfce model nd profile model; Nevti nd Bu (980) proposed edge-sed technique; Mckeown nd Denlinger (988) comined edge-sed nd profile correltion sed pproch; Vosselmn nd de Knecht (995), Bumgrtner (00) nd Zhou (006) used lest squre profile mtching; Prk nd Kim (00), Hu, Zhng nd Zhng (000) employed templte mtching; Grun nd Li (995), Merlet nd Zerui (996) connected rod seeds y dynmic progrmming; Grun nd Li (997) used snkes to optimize the pth of rod seed points; Vndn nd ChndrKnth nd Rmchndrn (00) employs minimum cost to follow pth; Bltsvis (004) revised rod mp sed on existing geodt nd knowledge. But stndrd cliché of rod extrction is tht every lgorithm hs its limits, so we elieve tht numer of techniques developed for different clsses of rod will led to mny-rnched solution for rod extrction tht will e effective for wide rnge of rod types. Improved ngulr texture signture is proposed nd coopertion etween ngulr texture signture nd templte mtching is employed to semi-utomticlly extrct rod network in this pper. Rod chrcteristics nd the principles of the proposed lgorithm re descried in Sect.. In Sect. 3 we introduce the process of our trcker. Section 4 compres our trcker with clssic lgorithms. Section 5 evlutes the trcker y cse study. Section 6 summrizes the results of our study nd mkes conclusion.. Rod chrcteristics. METHOD Rod chrcteristics cn e clssified in five groups: geometricl, photometric, topologicl, functionl nd contextul chrcteristics (Vosselmn nd de Knecht, 995; Grun nd Li, 997; Zhou, 006) on high-resolution imgery. Detils of these chrcteristics re: ) Geometry ) Rods re elongted rions rther thn liner fetures; ) A rod segment hs mximum curvture; Corresponding uthor 539

2 The Interntionl Archives of the Photogrmmetry, Remote Sensing nd Sptil Informtion Sciences. Vol. XXXVII. Prt B3. Beijing 008 c) A rod segment hs constnt width. ) Rdiometry d) The rod surfce usully is smooth nd homogeneous; e) The rod hs good contrst with its djcent res. 3) Topology f) The rod will continue nd do not stop without reson; g) The rods intersect nd form network. 4) Function h) The rods connect humn settlements. 5) Context i) Overpsses, higher rods, djcent uildings nd tress my cst shdow; j) Rods my e occluded y vehicles nd other ostcles. The opertor use the ove chrcters nd prior knowledge to detect nd identify rod segment, nd the proposed trcker works sed on ), ), c), d), e) properties. Principles of ngulr texture signture A texture mesure is descried in (Hverkmp, 00) nd defined nd extended y us s follows. At ech pixel p of grey imge, ngulr texture signture (ATS) T ( α, w, h, p) is defined s the men, stndrd devition, vrince or entropy for rectngulr set of pixels of width w nd height h round the potentil rod pixel p whose principl xis lies t n ngle of α from the potentil rod direction. This mesure is computed for set of ngles α 0,,α n t pixel p.at the point p, the ATS is defined s the set of vlues {T ( α, w, 0, w, h, p), T ( α, w, h, p),,t ( α n h, p. The grph nd vlues of n ATS for single point p re shown in Fig.. The locl limits on this grph correspond to the most likely directions of the rod t point p (e.g. directions 3, 0, 0, 9). At ech pixel p, the numer k nd loction of the strong locl limits re computed from the ATS. For exmple, the signture shown in Fig. () hs 4 limits tht re significnt. We refer to the numer k of limit s the degree of the pixel. The texture mesures tht re used in rod detection re: the degree of the pixel nd the direction of the limit. In our pproch, on the ssumption tht the rods hve the ove ), ), c), d), e) properties, rectngulr templte is extended from nd rotted 80 deg from the perpendiculr of the potentil rod direction out ech rod pixel. At discrete intervls out the pixel, the ATS is clculted; nd the direction of the limit is regrded s the rod direction. If the ATS tkes the vrince or entropy s mesure, the direction of locl minimum is tken; while if the ATS tkes the men s mesure, the direction of locl minimum is tkes for right rods nd the direction of locl mximum is tken for drker rod s shown in Fig.. Once the rod direction is given, move on one step long the direction nd iterte the ove process until the trcker fils or reches to nother trcked rod trjectory or reches to the oundry of the imge..3 Rod trjectory model Rod trjectory cn e modelled y B-splines ( Trinder nd Li, 997), stright line (Hverkmp, 00), Klmn filter (Vosselmn nd de Knecht, 995), extended Klmn filter (Zhou, 006), prticle filter (Zhou, 006), nd prol (Mckeown nd Denlinger, 988). )} x y plne cn hve ritrry The prol in the orienttion, hving n eqution of the form: Ax + Bxy + Cy + Dx + Ey + F = 0 () where x nd re coordintes of point on the prol, y A, B, C, D, E, F Angulr Stndrd devition Angulr Men re prmeters Fig. Angulr texture signture () The effect of rotting templtes in 36 discrete ngles (the rectngles whose indexes re odd is invisile for convenience) () The stndrd devitions of 36 templtes (c) The mens of 36 templtes Direction Index Direction Index 4 c

3 The Interntionl Archives of the Photogrmmetry, Remote Sensing nd Sptil Informtion Sciences. Vol. XXXVII. Prt B3. Beijing 008 We don t use this eqution since the prmetric form is more convenient for our purpose. We represent the rod pth prmetriclly s two seprte functions x(l) nd y(l) where l is the totl length in steps tht we hve trversed long the rod s pth. We use multiple regression with l nd l s the independent vriles to fit prols to x(l) nd y(l), getting pproximte functions: X ( l) = l Y ( l) = l X (l) Y(l) x coordintes nd y coordintes, + l + c + l + c where nd re the rod centreline points the prmeters., c, c,,, To get the most possile potentil direction of the rod, we resort to compute X ( l +) nd Y( l +), tht is the most possile loction of the next rod centreline point. Given tht the rod segment hs mximum curvture, then the chnge of the curvture of two djcent rod points must e less thn predefined threshold T. The curvture K t some point on the prol cn e computed s follows: K c c = (3) 3/ [4( c + c) l + 4( c + c ) l + + ], c, c,,, where re the sme s in Eq. (). () re ATS for some interesting pixels with the ATS polygons shown in red. If the rod hs good contrst with its surrounding ojects, the polygon looks like n ellipse or -Shpe. The compctness of ATS is defined s the compctness of the ATS polygon using Eq. (4). We must note tht the ATS polygon in our trcker is just hlf of the ove illustrted polygon ecuse we just rotte the rectngulr templte 80 deg, nd our ATS polygon is form y plotting the ATS vlues round the pixel under considertion with corresponding direction nd link the lst point nd the first point to the current pixel. The compctness tells us whether the ATS polygon looks like circle. A circle-like ATS polygon usully mens tht the trcker is no longer fit to e used to follow rod centerline point, nd it needs mnul plotting. ATS compctness π A = P where A nd P re the re nd perimeter of the ATS polygon, respectively. Note tht P doesn t include the distnce etween the first point nd the current pixel nd the distnce etween the lst point nd the current pixel..5 Lest squre templte mtching Our lest squres templte mtching is s the sme s Prk nd Kim one (00). 3. THE PROCESS OF OUR TRACKER Semi-utomtic rod extrction here is undertken s follows. 3. Pre-process the input imge If the originl imge doesn t hve good contrst etween rod nd other fetures, it needs stretching. Then the imge is convolved with Gussin filter to smooth the imge nd reduce the high-frequency noise. (4) where σ = G x + y exp( σ = (5) pixels. ) 3. The opertor detects rod segment Fig. The wind rose chrt of Angulr Texture Signture () The men ATS of drker rod () The men ATS of righter rod.4 Compctness of ATS When we tke closer look t the ATS rotting full 360 deg of ech pixel, we cn find some interesting links etween the shpe of the ATS polygon nd corresponding pixel types. To form the ATS polygon, insted of plotting the ATS vlues for ech direction long horizontl line, we plot the ATS vlues round the pixel under considertion with corresponding direction nd link the lst point to the first point. The resulting polygon is clled the ATS polygon. Fig. shows the clculted A humn opertor hs to identify short prt of rod xis; this rod prt serves s initiliztion for n utomtic trcker. The trcker need strting point on the rod centreline nd second point to define the direction of the rod nd third point to define the width w of the rod. Then the rod trcker move forwrd t lest 5 steps long the initil direction, then the rod trjectory model cn e uilt y Eq. (). Predict the next rod position nd get the most possile potentil rod direction. 3.3 Compute ATS From the lst rod centreline point in the trjectory, rectngulr templte is formed with width w nd height * w, nd rotte 80 deg from the perpendiculr of the predicted direction. At discrete intervls out the pixel, the ngulr 54

4 The Interntionl Archives of the Photogrmmetry, Remote Sensing nd Sptil Informtion Sciences. Vol. XXXVII. Prt B3. Beijing 008 texture signture is clculted. Selecting which texture signture s the mesure of ATS, it should judge y the rod conditions. After lot of experiment, we conclude tht, tking vrince, strnd devition or entropy s the mesure of ATS if the rod hs slient chrcteristic d) while tking men s the mesure of ATS if the rod hs ovious chrcteristic e). 3.4 Compute hed ATS compctness ATS compctness nd move forwrd one step Clculte the y Eq. (4). If the vlue is lrger thn predefined threshold T, it tells us tht the ATS is not fit ny more to trck rod, nd it needs lest squres templte mtching insted. Otherwise the direction of the limit is regrded s the rod direction, nd moves the rod trjectory one step. 3.5 Compute the chnge of the curvture Clculte the curvture of the new dded rod centreline y Eq. (5), compre it with the curvture of lst point, if the difference is lrger thn predefined threshold T, delete the new point from the rod trjectory, nd resort to mnully plot. Otherwise, predict the next rod position y prol eqution nd iterte from 3.3 if the trjectory doesn t rech to nother trjectory or the oundry of the imge. Once the user ccomplishes one rod segment or the trcker reches to one trcked rod or the oundry of imge, initilizes nother rod segment nd restrt from 3. gin until ll rods re trcked. From the opertor point of view, the procedure is s follows: the opertor hs to initilize the trcker y three input points to indicte the strting, the moving direction nd the width of the rod segment, nd then the trcker is triggered. Whenever the internl evlution of the trcking tool indictes tht the trcker might lost the rod xis or e no longer fit, it demnds for interction of the opertor. Then the opertor hs to confirm the trcker or the user must edit the extrcted rod nd put the trcker ck on the rod. finishes immeditely fter initiliztion. The snkes is slower thn profile mtching. The templte mtching is slower thn snkes ut fster thn ATS. We cn get the conclusion tht our proposed lgorithm is more roust thn other trckers. c d 5. EXPERIAMENT AND EVALUATION 4. COMPARISION OF FOUR TRACKERS To verify our lgorithm, we mke comprison etween lest squre templte mtching, lest squre profile mtching, snkes nd our trcker on sme Quickird imge of Hui rou County in Beijing, Chin, whose size is 355 y 066 pixels. On this imge, there is righter centreline on the homogenous drker rod surfce with righter ckground. The results re shown in Fig. 3. All trckers extrct the rod centreline with different precision in red colour. For profile mtching, the front prt of the pth is quite good ut the lst prt of the extrcted rod trjectory hs lrger devition due to the lrger curving of the rod. For templte mtching, the extrcted rod trjectory is good ut with some lrger devite points long the trjectory. For snkes, if there is only two seed points on centreline, the extrcted rod trjectory is quite wrong, s shown in Fig. 3(c), the up line; ut if there is 5 rod seed points, the result re quite good, s shown in Fig. 3(c), the down line. For ATS tking vrince s mesure, there is some cceptle devition in the middle prt of the rod. For ATS tking men s mesure, the extrcted rod trjectory is very good. We lso record the time needed y ech trcker. Profile mtching is so fst tht it e Fig. 3 Comprison of lgorithms ()The result of profile mtching () The result of templte mtching (c) The result of snkes (d) The result of ngulr texture signture tle stndrd devition s mesure (e) The result of ngulr texture signture tke men s mesure The lgorithm proposed here ws tested y one Quickird imge over Hefei City, An hui Province, Chin. The imge with 8 y 998 pixels contins mny different rod types such s stright rods, curves, nd crossings t different orienttions. And for ech segment, the surfce mteril is sme, ut there is sudden chnge in rdiometry, s shown in Fig. 4. The rods hve different disturing ojects such s shdows of trees nd 54

5 The Interntionl Archives of the Photogrmmetry, Remote Sensing nd Sptil Informtion Sciences. Vol. XXXVII. Prt B3. Beijing 008 occlusions y vehicles, ut the shdow nd occlusion is not serious. In the procedure of trcking, there re 8 times the trcker deviting the pth, nd then the thred is cesed s soon s possile y the opertor. There re 6 times prompts notifying the user tht the trcker is no longer suitle nd it needs mnul plotting. And there re only 76 mnul input points, nd the whole process tkes 543 seconds. But if the opertor wnt mnully plot ll pth point with the sme precision s the trcker, it needs 08 inputs nd it tkes 776 seconds. In generl, the qulity of the result of mnul nd semi-utomtic plotting is equivlent, since the opertor supervises the results of the semi-utomtic system nd filures re edited online. On verge, the geometric ccurcy is comprle, too. We lso test our trcker on mny other grey scle imges with different resolution vried from 0. to.5 m, nd the results re similr in mnul input sving out 90% nd time sving out 30%. The result shows tht our trcker is quite roust when the photometric property of sme rod segment chnges suddenly, nd when the trcker reches to the junctions nd it will go on without stop. And the trcker cn detect the rod centreline of the rods in ny orienttion with moderte curvture ccurtely, nd lso works successfully for rods hve some ostcles cused y shdow nd occlusion. compctness coefficient to evlute the ptness of itself to go on trcking, so the lgorithm hs some ility of higher-level resoning. The current limittions re tht the lgorithm my not work on the rod cst y much shdow nd occlusion in complex scenes, it cn t judge the vlidity of input seeds, it cn only trck long rion rods on grey scle imgery, nd it need more computing times. These limittions re currently eing exmined now. The min contriution of this pper is tht it employs ngulr texture signture semi-utomticlly extrct rod with precise results. REFERENCES Bjcsy, R., Tvkoli, M., 976. Computer recognition of rods from stellite pictures. IEEE Trnsctions on Systerms, Mn, nd Cyernetics, 6(9), pp Bltsvis, E. P., 004. Oject extrction nd revision y imge nlysis using existing geodt nd knowledge: current sttus nd steps towrds opertionl systems.isprs Journl of Photogrmmetry & Remote Sensing, 58, pp Brzohr, M., Cooper, D. B., 996. Automtic finding min rods in eril imges y using geometric-stochstic models nd estimtion. IEEE Trnsctions on Pttern Anlysis nd Mchine Intelligence, 8(7), pp Bumgrtner, A., Hinz, S., Wiedemnn, C., 00. Efficient methods nd interfces for rod trcking. Interntionl Archives of Photogrmmetry nd Remote Sensing, 34(3B), pp Benz, U. C., Hofmnn, P., Willhuck, G., et l., 004. Multiresolution, oject-oriented fuzzy nlysis of remote sensing dt for GIS-redy informtion. ISPRS Journl of Photogrmmetry & Remote Sensing, 58, pp Doucette, P., Agouris, P., Stefnidis, A., 004. Automted rod extrction from high resolution multi-spectrl imgery. Photogrmmetric Engineering &Remote Sensing, 70(), pp Grdner, M. E., Roert, D. A., Funk, C., Noronh, V., 00. Rod extrction from AVIRIS using spectrl mixture nd Q-tree filter techniques. In: Proc. AVIRIS Airorne Geosciences Workshop, Psden, Cliforni, URL:ftp://popo.jpl.ns.gov/pu/docs/workshops/0_docs/toc.h tml (lst ccessed : April 0,007). Gemn, D., Jedynk, B., 996. An ctive testing model for trcking rods in stellite imges. IEEE Trnsction on Pttern Anlysis nd Mchine Intelligence, 8(), pp. -4.Gruen, A., 985. Adptive lest squre correltion-a Powerful Imge Mtching Technique. South Africn Journl of Photogrmmetry, Remote Sensing nd Crtogrphy, 4(3), pp Fig. 4 Semi-utomtic extrcted rod network on Quickird imge () Overview () Locl result. 6. CONCLUSIONS Gruen, A., Li, H., 995. Rod extrction from eril nd stellite imges y dynmic progrmming. ISPRS Journl of Photogrmmetry nd Remote Sensing, 50(4), pp. -0. The proposed trcker sed on ATS is very roust, ecuse it mkes est use of the rod chrcteristics on high-resolution imgery. Our lgorithm employ prol eqution to fit the trjectory of the rod nd to predict the rod position nd moving direction nd to judge whether the new dded rod point is right y check the curvture chnge, it lso utilize 543 Gruen, A., Li, H., 997. Semi-utomtic liner feture extrction y dynmic progrmming nd LSB-Snkes. Photogrmmetic Engineering nd Remote Sensing, 997, 63(8), pp

6 The Interntionl Archives of the Photogrmmetry, Remote Sensing nd Sptil Informtion Sciences. Vol. XXXVII. Prt B3. Beijing 008 Hverkmp, D., 00. Extrcting stright rod structure in urn environments using IKONOS stellite imgery. Opticl Engineering, 4(9), pp Hrvey, W., McGlone, J., MKeown, D., Irvine, J., 004. Usercentric evlution of semi-utomtic rod network extrction. Photogrmmetric Engineering nd Remote Sensing, 70(), pp Hinz, S., Bumgrtner, A., 003. Automtic extrction of urn rod network from multi-view eril imgery. ISPRS Journl of Photogrmmetry & Remote Sensing, 58, pp Hu, X., Zhng, Z., Zhng, J., 000. An pproch of semiutomted rod extrction form eril imges sed on templte mtching nd Neurl Network. Interntionl Achieves of Photogrmmetry nd Remote Sensing, Amsterdm, Netherlnds, Vol. XXXIII, Prt B3, pp Long, H., Zho, Z., 005. Urn rod extrction from highresolution opticl stellite imge. Interntionl Journl of Remote Sensing, 6(), pp Mckeown, D., Denlinger, J., 988. Coopertive methods for rod trcing in eril imgery. In: Proceedings of the IEEE Conference in Computer Vision nd Pttern Recognition, pp Ann Aror, MI. Men, J.B, 003. Stte of the rt on utomtic rod extrction for GIS updte : novel clssifiction. Pttern Recognition Letters, 4,pp Merlet, N., Zerui, J., 996. New prospects in line detection y dynmic progrmming. IEEE Trnsction on Pttern Anlysis nd Mchine Intelligence, 8(4), pp Prk, S., Kim, T., 00. Semi-utomtic rod extrction lgorithm from IKONOS imges using templte mtching. In: Proc.nd Asin Conference on Remote Sensing, Singpore, pp Shukl, V., Chndrknth, R., Rmchndrn, R., 00. Semiutomtic rod extrction lgorithm for high resolution imges using pth following pproch.in:icvgip0, Ahmdd, pp Trinder, J. C., Li, H., 997. Semi-utomtic feture extrction y snkes. In: Automtic Extrction of Mn-mde Ojects from Aeril nd Spce Imges (). Bsel, Zürich,, pp Trinder, J. C., Wng, Y. D, Sowm, Y. A., et l., 997. Artificil intelligence in 3D feture extrction. In: Automtic Extrction of Mn-mde Ojects from Aeril nd Spce Imges (), Bsel, Zürich, pp Vosselmn, G. nd de Knecht, J., 995. Rod trcing y profile mtching nd Klmn filtering. In Proceedings of the Workshop on Automtic Extrction of Mn-Mde Ojects from Aeril nd Spce Imges, pp Birkheuser, Germny. Wng, F. G., Newkirkr, R., 988. A knowledge-sed system for highwy network extrction. IEEE Trnsction on Geoscience nd Remote Sensing, 6(5), pp Zhng, Q., Couloigner, I., 006. Benefit of the ngulr texture signture for the seprtion of prking lots nd rods on high resolution multi-spectrl imgery. Pttern Recognition Letters, 7, pp Zhou, J., Bischof, W. F., Celli, T., 006. Rod trcking in eril imges sed on humn-computer interction nd Bysin filtering. ISPRS Journl of Photogrmmetry & Remote Sensing, 6, pp ACKNOWLEDGEMENTS Our reserch is funded y The Ntionl Key Bsic Reserch nd Develop Progrm under Grnt No. 0006CB Qum, L. H., 978. Rod trcking nd nomly detection in eril imgery. In: Imge Understnding Workshop, London, UK, pp