EXTRACTION OF MAIN AND SECONDARY ROADS IN VHR IMAGES USING A HIGHER-ORDER PHASE FIELD MODEL

Transcription

1 EXTRACTION OF MAIN AND SECONDARY ROADS IN VHR IMAGES USING A HIGHER-ORDER PHASE FIELD MODEL Ting Peng a,b, Ian H. Jermyn b, Véronique Prinet a, Josiane Zerubia b a LIAMA & PR, Institute of Automation, Chinese Acaemy of Sciences, Beijing 0090, China (tpeng, prinet)@nlpr.ia.ac.cn b Project-Team Ariana, INRIA/I3S, 0690 Sophia Antipolis, France (tpeng, ijermyn, jzerubia)@sophia.inria.fr KEY WORDS: Roa, Urban, Satellite, Segmentation, Moelling, Variational methos ABSTRACT: This paper aresses the issue of extracting main an seconary roa networks in ense urban areas from very high resolution (VHR, ~0.6m) satellite images. The ifficulty with seconary roas lies in the low iscriminative power of the grey-level istributions of roa regions an the backgroun, an the greater effect of occlusions an other noise on narrower roas. To tackle this problem, we use a previously evelope higher-orer active contour (HOAC) phase fiel moel an augment it with an aitional non-linear non-local term. The aitional term allows separate control of roa with an roa curvature; thus more precise prior knowlege can be incorporate, an better roa prolongation can be achieve for the same with. Promising results on QuickBir panchromatic images at reuce resolutions an comparisons with other moels emonstrate the role an the efficiency of our new moel.. INTRODUCTION Roa extraction from remotely sense imagery has been extensively stuie for the last few ecaes ue to the variety an importance of the potential applications of an automatic extraction metho. While a great number of approaches exist in the literature (Fortier et al., 999, Mena, 003), the evelopment of reliable proceures is still a challenge. The relative failure of existing approaches stems from the complexity of the image scene, i.e. the variety of 'objects' (cars, builings, shaows...) that it contains. Existing algorithms can be classifie into three main categories, accoring to strategy: bottom-up, top-own, an combine. A bottom-up strategy assumes that first basic features are etecte, an then constraints are progressively ae, up to higher-level recognition. This category inclues mathematical morphology (Zhang et al., 999), knowlege representation an reasoning (Wang an Newkirk, 988), an roa tracking (Geman an Jeynak, 996, Merlet an Zerubia, 996). Due to their high sensitivity to nuisance factors, bottomup methos show strong limitations, an in general, low robustness. A top-own strategy moels the objects, an then searches for them in the image. Examples inclue active contours (Fortier et al., 00, Mayer et al., 998) an marke point processes (Stoica et al., 004, Lacoste et al., 005). These methos are relatively less sensitive to incomplete an ambiguous information, but the computations neee are usually expensive. In fact, the borerline between the two strategies is not clear. Most methos make use of both bottom-up an top-own processing. In aition, there are also "transversal" techniques, for instance, multi-scale analysis (Mayer et al., 998), neural networks (Bhattacharya an Parui, 997), Kalman filters (Vosselman an e Knecht, 995), an so on. The algorithms mentione above are mostly restricte to lowresolution images, an particularly to rural an semi-urban areas, where the roa network is reaily visible, with little shaow an few occlusion artefacts compare to inner cities. For very high resolution (VHR) imagery in ense urban areas, the complexity of the image scenes, which contain many roalike features in the backgroun, unesire noise on the roa, an occlusions cause by trees an the shaows of high builings (see Figure ), often results in unreliable roa extraction. This is particularly true for seconary roas. Being narrower, they are more affecte by the various types of geometric noise present in the image; they are more easily completely occlue, for example. This suggests that more sophisticate roa moelling approaches must be evelope in orer to extract these types of roas in an urban context. Figure. A QuickBir panchromatic image (size = , 0.6m/pixel) of a ense urban area. In this paper, our goal is to extract the main an seconary roa networks in ense urban areas from VHR QuickBir panchromatic images. To o so, we introuce a novel nonlinear non-local higher-orer active contour (HOAC) phase fiel energy, base on the energy use for the extraction of main roa networks in (Peng et al., 007a). The new, non-linear term allows the long-range interactions between pairs of bounary points on opposite sies of a roa to have a ifferent magnitue an/or range from those between pairs of bounary points on the same sie of a roa. In practical terms, this allows separate control of roa with an roa curvature: the scale on which the roa is moelle as being 'straight' can be ifferent (usually it is larger) from the roa with, unlike the moel in (Peng et al., 007a). The effect of this more precise prior 5

2 knowlege is better roa prolongation an thus better extraction of those roas for which this prior knowlege is crucial, i.e. narrower roas. Promising results on QuickBir panchromatic images at reuce resolutions an comparisons with other moels emonstrate the role an the efficacy of our new moel. The paper is organise as follows. In section, we recall briefly the main principles of the HOAC phase fiel moel of (Peng et al., 007a), eicate to the extraction of main roa networks. Section 3 introuces the new, non-linear term that ifferentiates between interactions along the roa an across the roa, an thus enables more sophisticate prior knowlege to be inclue. We explain the optimization scheme in section 4. In section 5, the benefits of the new moel are illustrate via experiments on real QuickBir panchromatic images. Section 6 conclues.. HOAC MODEL FOR MAIN ROAD EXTRACTION In (Rochery et al., 006), HOACs were propose for roa network extraction from low to meium resolution images. In contrast to conventional active contours, HOACs incorporate long-range interactions between points on the contour (better calle the 'region bounary'), an thus encoe complex prior knowlege of roa network geometry. For this reason, HOACs are more robust, an can be initialize generically an hence automatically. In (Rochery et al., 005), 'phase fiels' were introuce for region moelling in image processing, an HOACs were reformulate as (non-local) phase fiel moels. We use the phase fiel framework in this paper, because it has several avantages over parametric active contours or stanar level set methos: a linear representation space; ease of implementation; a neutral initialization; an greater topological freeom. To aapt the original phase fiel HOAC moel of (Rochery et al., 005) to the extraction of the main roa network from VHR QuickBir panchromatic images, (Peng et al., 007b) first propose a multi-resolution ata energy to eal with the complexity of VHR images. For the purpose of further eliminating false etections in the backgroun, (Peng et al., 007a) introuce specific prior knowlege in the form of an outate GIS map, to complement the generic prior knowlege carrie by the HOAC prior. It has been experimentally emonstrate that for the main roas, at full resolution, this moel is able to keep the unchange roas, to correct the mistakes, an to extract new roas. However, it is still not capable of retrieving the narrower roas very accurately. In this section, we will briefly review the basic moel in (Rochery et al., 005). We wish to fin the region R in the image omain containing the roa network. The phase fiel framework represents a region by a functionφ : R, where R is the image omain. The functionφ efines a region via a threshol ζ : R = { x : φ( x ) > ζ}. To moel regions, we efine a functional of φ, E ( φ; I) = θe ( φ) + E ( I, φ), () M P D where I : R is the image ata, an θ balances the contributions of the prior energy, which moels the E P geometry of the region sought, i.e. in our case, the region containing the roa network, an the ata energy E, which D moels the image to be expecte given the region. Minimization of such a functional with respect to φ gives a minimizing function φ * *, an hence a region R. The functional * must be esigne so that R is the region sought. The prior term E P in (Rochery et al., 005) is the sum of two terms: a local phase fiel term E 0, an a non-local HOAC phase fiel term E : P, { φ W φ x } E ( φ) = x x ( x) )) (a) 0 φ ( ) + ( (, β x x' E, ( φ ) = P xx' φ ( x ) φ ( x ') Ψ( ), (b) where controls the range of the interaction. The potential functio n W is W z = λ z z + α z z ( ) ( ) ( ) (3) where λ an α are cons tants. The potential W effectively constrains φ( x) for x R an φ( x) for x R = \ R. As a result, the quantitiesφ = ( ± φ) / are approxi- mately equal to the characteristic functions of R an R. The local graient prouct φ( x) φ( x) smoothes this result. Hence, it prouces a narrow interface, which is centre aroun the region bounary R. The interaction function Ψ is ( r + sin( π r )) if r <, Ψ ( r) = π (4) 0 else. E P The energy is equivalent to an active contour moel whose,0 energy is a linear combination of region bounary length an region area. E escribes long-range interactions between the graient vectors of φ at pairs of points. Since φ is only nonzero in R, this is the same as long-range interactions between C pairs of region bounary points an their normal vectors. Its effect is to prevent pairs of bounary points with antiparallel normal vectors from coming too close, an to encourage the growth of arm-like shapes. Therefore, E favours regions P ± R C, 6

3 compose of long, low curvature 'arms' of roughly constant with that meet at junctions, i.e. it moels network structures. as from the back-groun. We moel the one point statistics of the image intensi-ties, i.e. their histograms. It consists of two parts: information from the roas an from the backgroun. E can be written as D D { } E ( I, φ) = x ln P ( I( x)) φ ( x) + ln P ( I( x)) φ ( x).(5) + + P ± are two-component Gaussian mixture istributions, moelling the image intensities, where + enotes the roas an - enotes the backgroun. Their parameters are learne on samples of roa an non-roa in the image by supervise learning. 3. NON-LINEAR TERM FOR SECONDARY ROAD EXTRACTION Compare to the main roas, the seconary roas are much more ifficult to eal with, for the following reasons. First, the raiometric properties of narrower roas are similar to those of the backgroun. Secon, narrower roas are more often obscure by shaows an trees, which can cause gaps in the extracte network. For both reasons, ata riven/bottom-up moels fail to retrieve the roas correctly: strong geometric prior information is neee. The network moel of equation (b) contains such prior knowlege, but it suffers from a limitation that is severe in the case of seconary roas: the interaction between points on opposite sies of a roa ( φ( x) φ( x') < 0) is of the same strength an range as the interaction between points on the same sie of a roa ( φ( x) φ( x') > 0). This means that the scale on which the roa is expecte to be straight is the same as the with of the roa, whereas in fact roa with gives only an (approximate) upper boun on the raius of curvature of the roa: most roas are straighter than they are wie. For narrow roas this is particularly problematic, since the roa region is relatively unconstraine ue to the small roa with. In particular, roa prolongation is short-range just when, ue to the effects of geometric noise mentione above, we want it to be long-range. In orer to change this situation, we nee to be able to moel longer-range, stronger interactions along the roa without changing the interactions across the roa, i.e. we have to separate the two interactions. In this section, we will achieve this goal by introucing a new energy term. To motivate this term, we return to the contour formulation. 3. Interaction functions The ata term E D takes into account the raiometric properties of ense urban areas, which iscriminate ro Since roas are elongate structures, the interaction between points on the same sie of a roa must have longer range (or be stronger, which often amounts to nearly the same thing) than the interaction between points on opposite sies of a roa. To achieve this goal, a straightforwar solution is to separate the interaction function along one sie of a roa Ψ from the one across a roa Ψ, as shown in Figure. In other wors, the // interaction function must epen on the tangent/normal vectors at the pairs of points that are its argument. Although the length scale in the interaction function,, coul be mae to epen on the inner prouct between the tangent/normal vectors at the two pixels, it woul lea to complicate functional erivatives. Alternatively, we prefer to perform a linear interpolation between two interaction functions. In the contour formulation, our new HOAC prior energy E takes the form: HO E ( γ) = ss' f ( γ( s) γ( s')) Ψ { PHO, S S f ( γ( s) γ( s')) Ψ, (6) // // w here γ : S, is an arc length parameterization of the region bounary R, instea of the entire image omain; γ () s is the tangent vector to the bounary at s (thus γ () s γ ( s') [,] ); ' 'enotes parallel vectors an ' //' enotes antiparallel vectors. f // ( x), f // ( x):[,] [0,] are two + linear functions: f ( x ) = ( + x ) /, (7a) + // f ( x ) = ( x ) /. (7b) // Ψ an Ψ are the same types of function as in equation (4), // but have ifferent range or magnitue. The two interaction functions compete with each other: when γ () s γ ( s') [0,], i.e. the two interacting tangent vectors are more parallel, Ψ is ominant; while when γ () s γ ( s') [,0], i.e. the two inter acting tangent vectors are more antiparalle l, Ψ is ominant. // To simplify the formulation, we ajust only the magnitue of the interaction (although this effectively changes its range also), an we further assume that the magnitue of the interaction of parallel vectors is stronger than that of antiparallel vectors, i.e. Ψ = aψ, where a >, is a constant. Equation (6) // becomes } Figure. The interactions an Ψ Ψ // 7

4 E ( γ) = ss' {[( a ) ( a ) ( s) ( s')] }.(8) PHO, + + γ γ Ψ // S S 3. Non-linear non-local HOAC phase fiel energy In orer to implement E ( γ ) in the phase fiel framework, it PHO, nees to be reformulate as a function of the phase fiel φ, instea of the arc length parameterization γ use in equation (8). To this en, we replace tangent vectors by normal vectors, an then normal vectors by φ. Subsequently, the range of interactions is extene from the region bounary R to the whole of the image omain. Due to the fact that φ( x) is approximately equal to zero everywhere outsie the narrow interface R C in, the bounary inicator function S( φ) = ( φ( x) φ( x))( φ( x') φ( x')), (9) is inserte into the first term of equation (8). Thus we have E ( φ) = xx' {[( a ) S( φ) PHO, x x' + ( a+ ) φ( x) φ( x')] Ψ( )}. (0) When a =, this reuces to the non-local HOAC phase fiel term E of equation (b) (up to a factor of β /). Therefore, we efine our new aitional energy term by β E xx S { x x' ( φ) = ' ( φ) Ψ( )}. () 4 stron-ger than that between pairs of points on opposite sies of a roa. 4. OPTIMIZATION We now want to fin the function φ that minimizes our new total energy E. Using graient escent, at convergence, the optimal etermines an estimate of the roa network. * φ Following (Rochery et al., 005), we use a neutral initialization, i.e. the initial value ofφ is set equal to a constant everywhere in. During the iterations, no re-initialization or regularization is require. To perform graient escent we nee only the functional erivative of E: δ E P = + + δφ( x) P 3 ln θ { φ ( x ) λ ( φ ( x ) φ ( x )) + ' ( ( )) ' ( ') ( x + α φ x + β x φ x Ψ x ) x x' + β [ φ( x) x'( φ( x') φ( x')) ( )]}. () Ψ The erivatives δ E P / δφ an δ E /, P, δφ are non-local. To avoi performing explicit convolutions, they are calculate in the Fourier omain. The resulting evolution equation is φ( x) P 3 ln + = + θ { φ ( x ) λ ( φ ( x ) φ ( x )) t P α( φ ( x)) + βf { k Ψˆ ( k) ˆ φ( k)} β φ( x) F { Ψˆ ( k) F{ φ( x) φ( x)}} β φ( x) { F { Ψˆ ( k) F{ φ( x) φ( x)}}}}, (3) Since the functional erivative of E will, unlike that of 4 E, contain a term non-linear inφ ue to S( φ) being O( φ ), we refer to it as the non-linear non-local term. We now efine our new moel as E = θ ( E + E + E ) + E. P,0 D Note that whether the two tangent/normal vectors at a pair of in-teracting points are parallel or antiparallel, the effect of E is always to encourage two points insie the range of the interac-tion to attract each other. Thus E reinforces the effect of E in the case that the tangent/normal vectors are parallel, an it partly annuls the effect of E in the case that the tan gent/normal vectors are antiparallel. Accoringly, the interaction between pairs of points on the same sie of a roa is F where F an enote the Fourier an the inverse Fourier transform respectively, an ^ inicates the Fourier transform of a variable. In the iscretize implementation, all erivatives are compute in the Fourier omain, while the time evolution uses the forwar Euler metho. The parameters of the prior energy, i.e. θ, α, λ, β, β, an are constraine by stability conitions that ensure that a long bar of a given with is a stable configuration of the moel. This enables a choice of λ, β, β, an base on the expecte roa with: only α an θ remain. 8

5 (a) (b) (c) () (e) Figure 3. Experiments at /4 resolution. 3(a)-3(b): image ata (size = , roa with = 3~5 pixels) an its partial enlargement. 3(c)-3(e): results obtaine using the energy E (with E ) at iterations,,500 an 7,000. (a) (b) (c) Figure 4. Experiments at /4 resolution. Left to right: results obtaine respectively using the energy E (without E M, likelihoo estimation, an a stanar, non-higher-orer active contour moel (with neither E nor E ). P ), maximum 5. RESULTS AND COMPARISONS In this section, we emonstrate the behaviour of our new moel containing the non-linear, non-local HOAC term E via experiments on real QuickBir panchromatic images in ense urban areas, at reuce resolutions. Figure 3(a) shows one of the input images at /4 resolution. Figure 3(b) shows its partial enlargement, to isplay the complexity remaining at this resolution. The parameters ( θ, α, λ, β, β, ) were (00, 0., 3.8, , , 4). The results obtaine using the energy with the new non-linear nonlocal term E at iterations,,500 an 7,000 are illustrate in Figures 3(c)-3(e). The result obtaine using the energy E P NE E M (without ) is shown in Figure 4(a). We see that aing, W E enables the recovery of the main an seconary roa network, whereas the moel without E misses a seconary roa. In orer to illustrate the effects of other terms in the moel, we compute results using maximum likelihoo estimation (i.e. θ = 0 ) an a stanar, non-higher- orer active contour, i.e. β = β = 0 (see Figures 4(b) an 4(c)). The MLE result shows that local image information alone is not sufficient to istinguish the roas from the backgroun, while the stanar active contour result shows the importance of the geometric knowlege introuce by HOACs. Quantitative evaluations base on stanar criteria (Heipke et al., 997) are shown in Table. Groun truth for the evaluations was segmente by han. On the other han, the computation time for the result in Figure 3 was aroun 80 minutes, which is consierably slower than the next nearest time, that obtaine with the moel E M (Figure 4(a)). Figure 5 presents more results at reuce resolutions. The first column shows the input image ata, which is either at /4 or / resolution. The two columns on the right show the corresponing results obtaine with an without the new non-linear, non-local term E. 9

6 The importance of E is clear: it facilitates greatly the retrieval of seconary roas. Image New moel E (Figure 3(e)) E ( β = 0 ) M (Figure 4(a)) θ E + E 0 D (Figure 4(c)) MLE ( θ = 0 ) (Figure 4(b)) Completeness TP/(TP+FN) Correctness TP/(TP+FP) Quality TP/(TP+FP+F N) Table. Quantitative evaluation criteria of the ifferent methos (T = True, F = False, P = Positive, N = Negative) 6. CONCLUSIONS Narrow seconary roas in VHR images are very ifficult to extract, because of occlusion effects an the similar raiometric properties of the roa region an backgroun. To tackle this problem, the incorporation of strong geometric prior knowlege of roa networks is essential. Builing upon previous work eicate to segmenting only main roa networks, we have presente, in this paper, a novel non-linear, non-local phase fiel term, an applie it to the extraction of main an seconary roa networks in VHR images. This novel term causes pairs of points insie the range of the interaction to attract each other. In conjunction with the original HOAC geometric term, it allows the interaction between points on the same sie of a roa to be stronger than the interaction between points on opposite sies of a roa. Therefore, the incorporation of the term enables the generation of longer arm-like branches an better prolongation. Experiments on roa network extraction from QuickBir panchromatic images in ense urban areas at reuce resolution show that roa networks are completely recovere using the moel with the aitional term. The new moel clearly outperforms the previous moel in terms of quality of results. However, the new moel is computationally expensive, which is why the metho was applie at reuce resolution. To solve this problem, which is ue to the non-linear nature of the new term, we are now working on a novel linear non-local prior term that will have a similar effect. Figure 5. More experiments at reuce resolutions. First column: input images, first row: / resolution, size = , roa with = ~4 pixels; secon row: /4 resolution, size = , roa with = 3~5 pixels; thir row: /4 resolution, size = , roa with = 3~6 pixels; last row: /4 resolution, size = 5 5, roa with = 3~5 pixels. Two rightmost columns: corresponing results obtaine using the energy E (with E ) an the energy E (without E ). M 0

7 ACKNOWLEDGEMENTS This work was partially supporte by European Union Network of Excellence MUSCLE (FP ). The work of the first author is supporte by an MAE/Thales Alenia Space/LIAMA grant. REFERENCES Bhattacharya, U. an Parui, S., 997. An improve backpropagation neural network for etection of roa-like features in satellite imagery. International Journal of Remote Sensing 8(6), pp Fortier, M. F. A., Ziou, D., Armenakis, C. an Wang, S., 999. Survey of work on roa extraction in aerial an satellite images. Technical Report 4, Université e Sherbrooke, Quebec, Canaa. Fortier, M. F. A., Ziou, D., Armenakis, C. an Wang, S., 00. Automate correction an upating of roa atabases from high-resolution imagery. Canaian Journal of Remote Sensing 7(), pp Geman, D. an Jeynak, B., 996. An active testing moel for tracking roas in satellite images. IEEE Trans. Pattern Analysis an Machine Intelligence 8(), pp. 4. Heipke, C., Mayr, H.,Wieemann, C. an Jamet, O., 997. Evaluation of automatic roa extraction. International Archives of Photogrammetry an Remote Sensing XXXII, pp Lacoste, C., Descombes, X. an Zerubia, J., 005. Point processes for unsupervise line network extraction in remote sensing. IEEE Trans. Pattern Analysis an Machine Intelligence 7(0), pp Mayer, H., Laptev, I. an Baumgartner, A., 998. Multi-scale an snakes for automatic roa extraction. In: Proc. European Conference on Computer Vision (ECCV), Vol., Freiburg, Germany. Mena, J., 003. State of the art on automatic roa extraction for GIS upate: A novel classification. Pattern Recognition Letters 4(6), pp Merlet, N. an Zerubia, J., 996. New prospects in line etection by ynamic programming. IEEE Trans. Pattern Analysis an Machine Intelligence 8(4), pp Peng, T., Jermyn, I. H., Prinet, V., Zerubia, J. an Hu, B., 007a. A phase fiel moel incorporating generic an specific prior knowlege applie to roa network extraction from VHR satellite images. In: Proc. British Machine Vision Conference (BMVC), Warwick, Englan. Peng, T., Jermyn, I. H., Prinet, V., Zerubia, J. an Hu, B., 007b. Urban roa extraction from VHR images using a multiscale approach an a phase fiel moel of network geometry. In: Proc. 4th IEEE GRSS/ISPRS Joint Workshop on Remote Sensing an Data Fusion over Urban Areas (URBAN), Paris, France. Rochery, M., Jermyn, I. H. an Zerubia, J., 005. Phase fiel moels an higher-orer active contours. In: Proc. IEEE International Conference on Computer Vision (ICCV), Beijing, China. Rochery, M., Jermyn, I. H. an Zerubia, J., 006. Higher-orer active contours. International Journal of Computer Vision 69(), pp Stoica, R., Descombes, X. an Zerubia, J., 004. A Gibbs point process for roa extraction from remotely sense images. International Journal of Computer Vision 57(), pp. 36. Vosselman, G. an e Knecht, J., 995. Roa tracing by profile matching an Kalman filtering. In: Proc. Workshop on Automatic Extraction of Man-Mae Objects from Aerial an Space Images, Birkhuser Verlag, Basel-Boston-Berlin. Wang, F. an Newkirk, R., 988. A knowlege-base system for highway network extraction. IEEE Trans. Geoscience an Remote Sensing 6, pp Zhang, C., Murai, S. an Baltsavias, E. P., 999. Roa network etection by mathematical morphology. In: Proc. of ISPRS Workshop '3D Geospatial Data Prouction: Meeting Application Requirements', Paris, France.

8