A Probabilistic Approach to Detect Urban Regions from Remotely Sensed Images Based on Combination of Local Features

Transcription

1 A Probablstc Approach to Detect Urban Regons from Remotely Sensed Images Based on Combnaton of Local Features Berl Sırmaçek German Aerospace Center (DLR) Remote Sensng Technology Insttute Weßlng, 82234, Germany E-mal: Cem Ünsalan Computer Vson Research Laboratory Yedtepe Unversty Istanbul, 34755, Turkey E-mal: Abstract Detectng urban regons from very hgh resoluton aeral and satellte mages provdes very useful results for urban plannng, and land use analyss. Snce manual detecton s very tme consumng and prone to errors, automated systems to detecton of urban regons from very hgh resoluton aeral and satellte mages are needed. Unfortunately, dverse characterstcs of urban regons, and uncontrolled appearance of remote sensng mages (llumnaton, vewng angle, etc.) ncrease dffculty to develop automated systems. In order to overcome these dffcultes, heren we propose a novel urban regon detecton method usng local features and a probablstc framework. Frst, we ntroduce four dfferent local feature extracton methods. Extracted local feature vectors serve as observatons of the probablty densty functon to be estmated. Usng a varable kernel densty estmaton method, we estmate the correspondng probablty functon. Usng modes of the estmated densty, as well as other probablstc propertes, we detect urban regon boundares n the mage. We also ntroduce data and decson fuson methods to fuse nformaton comng from dfferent feature extracton methods. Extensve tests on very hgh resoluton grayscale aeral and panchromatc Ikonos satellte mages ndcate practcal usefulness of proposed method to detect urban regons automatcally n a robust and fast manner. I. INTRODUCTION Remotely sensed satellte and aeral mages provde very valuable nformaton. However, ther coverng areas and resoluton make manual analyss very dffcult and prone to errors. Furthermore, especally urban areas are dynamc envronments. Hence, they should be montored and analyzed perodcally. Due to these problems, developng algorthms to detect partcular objects n remotely sensed mages s a very mportant research feld. Especally, automatc detecton of urban regon boundares can provde useful nformaton to muncpaltes, mappng agences, mltary, government agences, or unmanned aeral vehcle developers. Unfortunately, automatc object detecton algorthms cope wth some dffcultes on remotely sensed mages. Frst, buldngs generally have dverse characterstcs wth dfferent texture, color, and shape. In addton to that, n ther appearance on the mage, the llumnaton, vew angle, scalng, occluson effects are uncontrolled. Therefore, classcal object detecton algorthms cannot provde an acceptable detecton performance. Therefore, more advanced methods are requred for robust detecton of the objects n remotely sensed mages. In related lterature, due to the mportance of the problem, many researchers concentrated on developng automated systems to detect urban regons. Karathanass et al. [7] used buldng densty nformaton to classfy resdental regons. They beneft from texture nformaton and segmentaton to extract the resdental areas. Unfortunately, they had several parameters to be adjusted manually. Benedktsson et al. [1] used mathematcal morphologcal operatons to extract structural nformaton to detect the urban regon boundares n satellte mages. Ther method s based on usng a neural network whch s traned usng tranng urban area regons to classfy nput mages. Ünsalan and Boyer [16], [18] used structural features to classfy urban regons n panchromatc satellte mages. Snce they use statstcal classfers, they also need tranng data to detect the urban area n the mage. In a followng study, Ünsalan and Boyer [17] assocated structural features wth graph theoretcal measures n order to grade the satellte mages and extract the resdental regons from them. Fonte et al. [5] consdered corner detectors to obtan the type of the structure n a satellte mage. They concluded that corner detectors mght gve dstnctve nformaton on the type of structure n an mage. Bhagavathy and Manjunath [2] used texture motfs for modelng and detectng regons (such as golf parks and harbors) n satellte mages. They focused on repettve patterns n the mage. Bruzzone and Carln [3] proposed a context-based system to classfy very hgh resoluton satellte mages. They used support vector machnes fed wth a novel feature extractor. Fauvel et al. [4] fused dfferent classfers to extract and classfy urban regons n panchromatc satellte mages. Zhong and Wang [20] extracted urban regons n grayscale satellte mages usng a multple-classfer approach. These last three studes also need tranng data for urban area classfcaton. In a related study, Sırmaçek and Ünsalan [16], used scale-nvarant feature transform (SIFT) and graph theory to detect urban areas and buldngs n grayscale Ikonos mages. They used template buldng mages for ths purpose. Although graph theoretcal methods are sutable for urban

2 area detecton, they need consderable computaton power and operaton tme. In a followng study, we proposed a method to detect urban regon boundares usng Gabor features alone [14]. In ths study, we extend our prevous work by a new approach for robust detecton of urban regon boundares based on four sets of local nvarant features, ther fuson n dfferent levels, and a probablstc framework. To ths end, frst we ntroduce four dfferent local feature extracton methods as: Harrs corners, Gradent Magntude Support Regon (GMSR) based features, Gabor features, and FAST features. The extracted local features serve as observatons of the probablty densty functon (pdf) to be estmated. We formed pdf by usng the nonparametrc varable-kernel densty estmaton method. Usng the estmated densty, we detect urban regon boundares n the gven mage. To ncrease robustness, and to merge nformaton comng from dfferent local features, we present two dfferent fuson methods n data and decson levels. In our experments, we use a dataset obtaned from dfferent ctes by two dfferent sensors to test our method. Specfcally the data set s formed of very hgh resoluton panchromatc Ikonos satellte mages and grayscale aeral mages. These test mages have also dfferent spatal resolutons and dverse characters. Ths data set s the same as n our prevous study. Therefore, we have a chance to compare both our exstng and novel method on the same bass. II. LOCAL FEATURE EXTRACTION We detect the urban area n a test mage usng local feature ponts. As frst step, before extractng local features, s to apply smoothng to the nput test mage by usng medan flter [15]. Ths step elmnates small nose terms n the mage. Then, we apply Gabor flterng n dfferent drectons. The maxma n these flter responses lead to local feature ponts. Next, we wll explore these steps n detal. A. Gabor Features Gabor flters are extensvely used n texture segmentaton and object recognton [8]. They exhbt desrable characterstcs as spatal localty and orentaton selectvty [19]. Mathematcally, the 2D Gabor flter can be defned as the product of a Gaussan and a complex exponental functon as follows, F ϕ (x, y) = 1 2πσ 2 g exp( u2 + v 2 2σg 2 )exp(j2πf u) (1) Here, u = x cos ϕ + y sn ϕ and v = x sn ϕ + y cos ϕ. f s the frequency of the complex exponental sgnal, ϕ s the drecton of the Gabor flter, and σ g s the scale parameter. These parameters should be adjusted wth respect to the mage resoluton at hand. We can detect the edge-orented urban characterstcs (such as buldng edges) n a test mage usng Gabor flterng. Therefore, for a test mage I(x, y) (wth sze N xm), we beneft from the real part of the Gabor flter response as G ϕ (x, y) = RI(x, y) F ϕ (x, y) (2) where stands for the 2D convoluton operaton. Here, G ϕ (x, y) s the maxmum for mage regons havng smlar characterstcs wth the flter. In order to extract Gabor features, we frst search for the local maxma n G ϕ (x, y) for x = 1,..., N and y = 1,..., M. If any pxel (x o, y o ) n G ϕ (x, y) has the largest value among ts neghbors, G ϕ (x o, y o ) > G ϕ (x n, y n ) (x n, y n ) (x o 1, y o 1), (x o, y o 1),..., (x o + 1, y o + 1); we call t as a local maxmum. It s a canddate for beng a local feature pont. Next, we check the ampltude of the flter response G ϕ (x o, y o ). We call our local maxmum (x o, y o ) as a canddate local feature pont f and only f G ϕ (x o, y o ) > α. To handle dfferent mages, we obtan α usng Otsu s method on G ϕ (x, y) n an adaptve manner for each mage separately [9]. Therefore, we elmnate the weak canddate local feature ponts n future calculatons. To represent each canddate local feature pont further, we assgn an orentaton and weght to them. We obtan the weght for each local feature vector smlar to the methods n prevous sectons. Here, we obtan our bnary mage B m (x, y) by thresholdng G ϕ (x, y) wth α for weght calculatons. However, we assgn the orentaton dfferent than the two prevous methods as follows. We check for the orentatons n the eght-neghborhood of (x j, y j ) and pck the orentaton, θ j as the one havng hghes magntude. We apply ths procedure to obtan a robust orentaton nformaton. We apply ths procedure n all ϕ drectons and obtan Gabor flterng based local feature vectors as k f = (x j, y j, θ j, w j ) for j = 1,..., K f, where K f s the total number of extracted Gabor features. B. Harrs Corner Features Fonte et al. [5] consdered Harrs and Susan corner detectors to obtan the type of structure n a satellte mage. They concluded that, corner detectors are not suffcent alone to gve dstnctve nformaton on the type of structure n an mage. Schmd et al. [11] on the other hand evaluated and compared dfferent corner detectors for general mage processng applcatons. They concluded that the best results are provded by the Harrs corner detector [6]. Therefore, n our study we also beneft from nformaton comng from Harrs corner detector. Harrs corners are extracted n three steps: gradent calculaton, matrx formaton, and egenvalue computaton. Frst, we should calculate smoothed gradents n x and y drectons to detect corners n a gven grayscale mage I(x, y). We defne smoothed gradent flters for x and y drectons as; g x (x, y) = x 2πτg 4 g y (x, y) = y 2πτg 4 exp( x2 + y 2 2τg 2 ) (3) exp( x2 + y 2 2τg 2 ) (4) where τ g s the smoothng parameter that we select as unty due to the scale of Ikonos and aeral mages at hand. Although our method s farly robust to ths parameter, t should be adjusted by the resoluton of the mage to be analyzed n future studes.

3 We calculate the smoothed gradents for the mage I(x, y) as I x = g x (x, y) I(x, y) (5) I y = g y (x, y) I(x, y) (6) where stands for the two dmensonal convoluton operaton. Harrs corner detector depends on calculatng a matrx (related to autocorrelaton functon) as ( ) axx a A(x, y) = xy (7) where a xx = a xy = a xy x W y W x W y W a yy = x W y W a yy I 2 x(x, y ) (8) I x (x, y )I y (x, y ) (9) I 2 y(x, y ) (10) Here, a xx, a xy, a yy are gradent magntudes averaged over a wndow W. We pck ths averagng wndow wdth as seven pxels n ths study. Further analyss of choosng ths parameter can be found n our prevous study [13]. Egenvalues of the matrx A provde nformaton about the edge n a gven locaton. If both egenvalues of the matrx at a gven locaton s large, then there s a corner there. Harrs and Stephens suggested that, exact egenvalue computaton can be avoded by calculatng the response functon R(A) = A κtrace 2 (A) (11) where κ s a tunable parameter wth values from 0.04 to 0.15 were reported as approprate n the lterature. We pcked κ = 0.06 n ths study. We also tested the effect of usng ths parameter n remote sensng mage n our prevous study [13]. Corner ponts are detected by checkng the local maxma of R(A). As we obtan corner ponts wth ther spatal coordnates, we defne our local features usng them. Besdes spatal coordnates, we also add orentaton and weght nformaton to features. Frst, we calculate the gradent orentaton O(x, y), and magntude M(x, y), for each mage as, O(x, y) = arctan( I y(x, y) I x (x, y) ) (12) M(x, y) = Ix(x, 2 y) + Iy(x, 2 y) (13) For the corner pont coordnate (x j, y j ), the correspondng orentaton s θ j = O(x j, y j ). To assgn a weght for the local feature vector, we threshold M(x, y) usng Otsu s method n an adaptve manner [9]. As a result, we obtan B m (x, y) as a bnary mage. In ths mage, pxels havng values one correspondng to strong responses. We obtan connected pxels to (x j, y j ), we assgn ther sum as the weght w j. Therefore, f a canddate local feature vector has more connected pxels, t has more weght. Fnally, we have our Harrs corner based local features as k h (j) = (x j, y j, θ j, w j ) for j = 1,..., K h where K h s the total number of detected Harrs features. C. GMSR Features Next we pck our prevous study, to extract gradent magntude based support regons (GMSR) [16]. There, we used these features to extract structural and condtonal statstcal features to classfy land use. Heren, we extract support regons usng smoothed gradent values, namely I x and I y gven n 5 and 6 respectvely. To extract support regons, we threshold M(x, y) wth 10% of the maxmum gradent magntude n the consdered mage I(x, y). The ratonale here s as follows. We take the maxmum gradent magntude as a benchmark. After experments, we observed that even 10% of ths value stll gves nformaton about the structure n the mage. Therefore, we have an adaptve threshold value. Smlar to the Harrs corner detecton method, we obtan B(x, y) as a bnary mage after thresholdng. In ths mage, pxels havng a value of one correspond to support regons. We defne local feature vectors based on the extracted support regons. Therefore, we pck each support regon pxel as a local feature vector coordnate. Assume that we have a local feature vector (x j, y j ). By defnton, B(x j, y j ) = 1. We defne the orentaton and magntude of the local feature vector havng spatal coordnate (x j, y j ) wth the same method that we used n the prevous secton. As a result, we obtan our local features as k g (j) = (x j, y j, θ j, w j ) for j = 1,..., K g where K g s the total number of extracted GMSR features. D. FAST Features Rosten et al. [10] ntroduced the FAST method to detect corners n mages n a fast and relable manner. The method depends on wedge-model-style corner detecton and machne learnng technques. Ths method can brefly be explaned as follows. For each corner canddate pxel, ts 16 neghbors are checked. If there exst nne contguous pxels passng a set of tests, the canddate pxel s labeled as a corner. These tests are done usng machne learnng technques to speed up the operaton. In our study, we fnally pck FAST to extract local features from gven test mage. Assume that, we have a local feature (FAST corner) at (x j, y j ) coordnate. We defne the orentaton and magntude of the local feature, usng the same method that we used n the Harrs-corner-based local feature extracton. As result, we obtan local features as k s (j) = (x j, y j, θ j, w j ) for j = 1,..., K s where K s s the total number of extracted FASTbased features. III. URBAN REGION DETECTION Each extracted local feature ndcates a characterstc part of a buldng to be detected n the mage. However, only one

4 feature s not suffcent enough to detect urbanzed regon. In fact, more local features ncreases detecton probablty of nterest regon. To solve problem, we formulate our urban regon detecton method wth a probablstc framework. To do so, we represent locatons whch are holdng possble buldng characterstcs as dscrete jont random varables. We then estmate ther pdf by takng local features as observatons. Here, we beneft from a varable-kernel densty estmaton method. Next, we ntroduce our probablstc urban regon detecton framework usng t. Fnally, we ntroduce data and decson fuson methods based on our probablstc framework to detect buldngs. A. Kernel based Densty Estmaton Slverman [12]defnes the kernel densty estmator for a dscrete and bvarate pdf as follows. Frst, the bvarate kernel functon, N(x, y) should satsfy the condtons and N(x, y) = 1 (14) x y N(x, y) 0 (x, y) (15) The pdf estmator wth kernel N(x, y) s defned by p(x, y) = 1 nh n ( x x N, y y ) h h =1 (16) where h s the wndow wdth (also called the smoothng parameter) and (x, y ) for = 1,..., n are observatons from the pdf to be estmated. If observatons can not be represented relably by a fxed kernel functon, then a varable kernel functon can be used. Ths s acheved by adaptaton of the amount of smoothng to the local densty of the data (observaton). Hence, the scale parameter s allowed to vary from one observaton pont to another. Besdes, the estmate s constructed smlarly to the classcal kernel estmate. The pdf estmate gven n Eqn. 16 then becomes p v (x, y) = 1 nh n =1 ( 1 x x N, y y ) σ hσ hσ where σ s the varable scale parameter for = 1,..., n. (17) B. Detectng Urban Regons Usng Varable Kernel based Densty Estmaton As we mentoned prevously, we use local features (k h, k g, k f, k s ) as observatons to estmate the pdf. Wthout loss of generalty, we explan pdf estmaton on a generc local feature vector k = (x, y, θ, w ) for = 1,..., K. These features provde nformaton about urban regon havng buldngs characterstcs. In order to estmate pdf, each local feature wll have ts effect on (x, y ) coordnate. Usng N(x, y) n Eqn. 17 as a Gaussan symmetrc pdf, whch s used n most densty estmaton applcatons, we form the estmated pdf as K 1 2πσ exp ( (x ˆx ) 2 + (y ŷ ) 2 ) p(x, y) = 1 R 2σ =1 (18) where σ = w and R s the normalzng constant. We wll use p(x, y) pdf functon to detect urban regon boundares. To do so, we apply automatc thresholdng on obtan pdf??? C. Data Fuson and Decson Fuson for Urban Regon Detecton The four local feature extracton methods extract dfferent nformaton from the same mage. In the prevous secton, we ntroduced how to separately use ths nformaton to detect urban regon boundares. However, ther fuson can also mprove our detecton performance. Fortunately, our probablstc urban regon detecton approach allows fuson of nformaton. Therefore, n ths secton we ntroduce two feature fuson methods usng probablstc framework. Our frst method s based on data fuson. Ths method s straghtforward, such that we use all the local features extracted wth dfferent methods as one unque group. In other sayng, k F = {k h, k g, k f, k s }. We estmate the pdf usng Eqn. 18 wth the local feature set k F. We detect urban regon boundares from the estmated pdf wth the same method n the prevous secton. Our second method s based on decson fuson. Here, we mx the estmated pdfs by dfferent methods and obtan a fnal pdf. Whle mxng the estmated pdfs, we assgn a weght to each of them drectly proportonal to ther maxmum mode value. As we mentoned n the prevous secton, n detectng buldng locatons from the estmated pdf we label the mode wth the maxmum value as a buldng. By normalzng four dfferent pdfs ths way, we can mx them and obtan the fnal pdf estmate as p D (x, y) = 1 R l={h,g,f,s} p l (x, y) max (x,y) p l (x, y) (19) where p h (x, y), p g (x, y), p f (x, y), and p s (x, y) are the estmated pdfs from k h, k g, k f, and k s. R s agan the normalzng constant. We call ths method as decson fuson, snce we apply the fuson operaton close to the urban regon detecton step. Agan, we use the urban regon detecton method n the prevous secton on p D (x, y) to detect urban regon boundares. IV. EXPERIMENTS We test our probablstc urban regon detecton methods on panchromatc Ikonos and grayscale aeral mages of Istanbul and Adana ctes of Turkey. V. CONCLUSION Heren, we ntroduced a novel urban regon detecton method based on a probablstc framework. To do so, we defned urban regon pxels to be detected as jont random varables. We formed a pdf whch shows probabltes of belongng

5 to an urban regon for each mage pxel. In estmatng the pdf, we used local features whch are extracted from mage usng four dfferent methods. However, our probablstc buldng detecton framework s not lmted to these four methods. It can be appled to other local feature extracton methods as well. Then, we detected urban regon boundares usng the estmated pdf. We further mproved our urban regon detecton method by ntroducng two feature fuson methods; data fuson and decson fuson. Obtaned performances of the proposed approach on panchromatc Ikonos satellte and grayscale aeral mages show robustness of algorthm even on mages obtaned from very dfferent knd of sensors. We can conclude that our probablstc urban regon detecton method can be used n real-lfe applcatons n a very fast and relable manner. [20] P. Zhong and R. Wang, A multple condtonal random felds ensemble model for urban area detecton n remote sensng optcal mages, IEEE Transactons on Geoscence and Remote Sensng, vol. 45 (12), pp , Dec REFERENCES [1] J. Benedktsson, M. Pesares, and K. Arnason, Classfcaton and feature extracton for remote sensng mages from urban areas based on morphologcal transformatons, IEEE Transactons on Geoscence and Remote Sensng, vol. 41 (9), pp , Sep [2] S. Bhagavathy and B. Manjunath, Modelng and detecton of geospatal objects usng texture motfs, IEEE Transactons on Geoscence and Remote Sensng, vol. 44 (12), pp , Dec [3] L. Bruzzone and L. Carln, A multlevel context-based system for classfcaton of very hgh spatal resoluton mages, IEEE Transactons on Geoscence and Remote Sensng, vol. 44 (9), pp , Sep [4] M. Fauvel, J. Chanussot, and J. Benedktsson, Decson fuson for the classfcaton of urban remote sensng mages, IEEE Transactons on Geoscence and Remote Sensng, vol. 44 (10), pp , Oct [5] L. Fonte, S. Gautama, W. Phlps, and W. Goeman, Evaluatng corner detectors for the extracton of man made structures n urban areas, n Proceedngs of IEEE Internatonal Geoscence and Remote Sensng Symposum, pp , [6] C. Harrs and M. Stephens. [7] V. Karathanass, C. Iossfds, and D. Rokos, A texture-based classfcaton method for classfyng bult areas accordng to ther densty, Internatonal Journal of Remote Sensng, vol. 21 (9), pp , Jun [8] V. Kyrk, J. Kamaranen, and H. Kalvanen, Smple gabor feature space for nvarant object recognton, Pattern Recognton Letters, vol. 25 (3), pp , Feb [9] N. Otsu, A threshold selecton method from gray-level hstograms, IEEE Transactons on Systems, Man, and Cybernetcs, vol. SMC-9 (1), pp , Jan [10] E. Rosten, R. Porter, and T. Drummond. [11] C. Schmd, R. Mohr, and C. Bauckhage. [12] B. W. Slverman, Densty Estmaton for Statstcs and Data Analyss, 1st ed. Chapman Hall, [13] B. Sırmaçek and C. Ünsalan. [14], Urban area detecton usng local feature ponts and spatal votng, IEEE Geoscence and Remote Sensng Letters, vol. 7 (1), pp , Jan [15] M. Sonka, V. Hlavac, and R. Boyle, Image Processng, Analyss and Machne Vson, 3rd ed. CL Engneerng, [16] C. Ünsalan, Gradent magntude based support regons n structural land use classfcaton, IEEE Geoscence and Remote Sensng Letters, vol. 3 (4), pp , Oct [17] C. Ünsalan and K. Boyer, A theoretcal and expermental nvestgaton of graph theoretcal measures for land development n satellte magery, IEEE Transactons on Pattern Analyss and Machne Intellgence, vol. 27 (4), pp , Apr [18], Classfyng land development n hgh resoluton satellte magery usng hybrd structural multspectral features, IEEE Transactons on Geoscence and Remote Sensng, vol. 42 (12), pp , Dec [19] M. Vetterl and J. Kovacevc, Wavelets and Subband Codng, 1st ed. Prentce Hall, 1995.