Multivariate Texture-based Segmentation of Remotely Sensed. Imagery for Extraction of Objects and Their Uncertainty

Transcription

1 Multivariate Texture-based Segmentation of Remotely Sensed Imagery for Extration of Objets and Their Unertainty Arko Luieer*, Alfred Stein* & Peter Fisher** * International Institute for Geo-Information Siene and Earth Observation (ITC), Department of Earth Observation Siene, P.O. Box 6, 7500 AA Enshede, the Netherlands Telephone: +31 (0) , Fax: +31 (0) arko@luieer.net stein@it.nl ** University of Leiester, Department of Geography, Leiester, LE1 7RH, United Kingdom Telephone: +44 (0) , Fax: +44 (0) pff1@le.a.uk Keywords: multivariate texture, image segmentation, Loal Binary Pattern (LBP) operator, unertainty Abstrat In this study, a segmentation proedure is proposed based on grey-level and multivariate texture to extrat spatial objets from an image sene. Objet unertainty was quantified to identify transitions zones of objets with indeterminate boundaries. The Loal Binary Pattern 1

2 (LBP) operator, modeling texture, was integrated into a hierarhial splitting segmentation to identify homogeneous texture regions in an image. We proposed a multivariate extension of the standard univariate LBP operator to desribe olor texture. The paper is illustrated with two ase studies. The first onsiders an image with a omposite of texture regions. The two LBP operators provided good segmentation results on both grey-sale and olor textures, depited by auray values of 96% and 98% respetively. The seond ase study involved segmentation of oastal land over objets from a multi-spetral Compat Airborne Spetral Imager (CASI) image, of a oastal area in the UK. Segmentation based on the univariate LBP measure provided unsatisfatory segmentation results from a single CASI band (70% auray). A multivariate LBP based segmentation of three CASI bands improved segmentation results onsiderably (77% auray). Unertainty values for objet building bloks provided valuable information for identifiation of objet transition zones. We onlude that the (multivariate) LBP texture model in ombination with a hierarhial splitting segmentation framework is suitable for identifying objets and for quantifying their unertainty. 1. Introdution Geospatial data quality is a topi frequently overed in reent sientifi literature on GIS and remote sensing (Foody and Atkinson 2002). An important omponent of data quality is data unertainty. Poor lass definition, gradual transition zones or fuzzy boundaries, mixed pixels, and inomplete or imperfet data give rise to unertainty in remotely sensed image lassifiation results. Both fuzzy and probabilisti lassifiation tehniques an help to model and quantify unertainty. In reent years, muh researh has foused on modeling unertainty in remotely sensed image lassifiation (Foody 1996, Hootsmans 1996, Canters 1997, Fisher 1999, van der Wel 2000, Zhang and Foody 2001, Foody and Atkinson 2002). It mainly 2

3 foused on unertainty of spetral lassifiation on a pixel-by-pixel basis. As suh, it partially ignored potentially useful spatial relations between pixels. Objet-oriented approahes to remotely sensed image proessing have beome popular with the growing amount of high-resolution satellite and airborne imagery. Segmentation tehniques extrat spatial objets from an image (Gorte and Stein 1998, Luieer and Stein 2002). It extends lassifiation, as spatial ontiguity is an expliit goal of segmentation whereas it is only impliit in lassifiation. Unertainty in a segmented or lassified image an affet further image proessing. In partiular, in areas where fuzzy objets or objets with indeterminate boundaries dominate, an indiation of segmentation unertainty is important. A straightforward approah to identify fuzzy objets is to apply a fuzzy -means (FCM) lassifiation. This lassifier gives membership values of belonging to eah lass for eah pixel. A themati map an be obtained from this result by labeling the pixels aording to the lass with the maximum membership value. However, pixel-based lassifiers, like the FCM, do not take spatial relations between pixels into aount, also known as pattern or texture. We argue that a texture-based segmentation approah (i.e. inluding the spatial omponent) an help to identify fuzzy objets. Texture reflets the spatial struture of pixel values and it is therefore indispensable in segmenting an area into sensible geographial units. Texture analysis has been addressed and suessfully applied in remote sensing studies in the past. An interesting overview paper onerning texture measures is Randen and Husøy (1999). Reently, Ojala and his o-workers have further pursued an effiient implementation and appliation towards texture-based segmentation (Ojala et al. 1996, 2002a; Ojala and 3

4 Pietikäinen 1999; Pietikäinen et al. 2000). Their Loal Binary Pattern (LBP) measure is superior to most of the traditional texture measures in segmentation of texture images (Ojala et al. 1996). LBP is a rotation invariant grey sale texture measure. The aim of this study is to develop and apply a supervised multivariate texture segmentation tehnique to identify objets from remotely sensed imagery. It is applied to an image with a texture omposition and to an airborne multispetral image of a oastal area in northwest England. It builds on work of Luieer and Stein (2002) and Luieer et al. (2004) and further explores the use of multivariate texture. In addition, we fous on quantifiation of objet unertainty to identify transition zones. 2. Methods 2.1 Texture Image texture an provide valuable information for identifiation of objets. The human visual system not only an distinguish objets based on olor, but texture plays an important role as well. A major harateristi of texture is the repetition of a pattern or patterns over a region. The pattern may be repeated exatly, or as a set of small variations, possibly as a funtion of position. There is also a random aspet to texture, beause size, shape, olor and orientation of pattern elements (sometimes alled textons) an vary over a region. A omparative study of texture measures is given in Randen and Husøy (1999). They onlude that a diretion for future researh is the development of powerful texture measures that an be extrated and lassified with a low omputational omplexity. A relatively new and simple texture model is the Loal Binary Pattern operator (LBP) (Pietikäinen et al. 2000, 4

5 Ojala et al. 2002a). It is a theoretially simple yet effiient approah to grey sale and rotation invariant texture segmentation based on loal binary patterns and non-parametri disrimination of sample and referene texture distributions. 2.2 Texture model the Loal Binary Pattern Operator (LBP) Ojala et al. (2002a) derived the loal binary pattern operator (LBP) by defining texture T in a loal neighborhood of a grey sale image as the joint distribution of grey levels of P image pixels T = t( g, g0,..., gp 1) (1) where g orresponds to the grey-sale value of the enter pixel ( p ) of the loal neighborhood and g i ( i= 0,..., P 1) orresponds to the grey-sale value of a pixel in the neighborhood of p. In this study, we apply a irle of radius R with P equally spaed pixels that form a irularly symmetri neighborhood set (Ojala et al. 2002a, Luieer et al. 2004). A irular neighborhood enables a definition of a rotation invariant texture measure. Invariane with respet to the saling of pixel values or illumination differenes is ahieved by onsidering the signs of the differenes instead of their numerial values * T t sign g0 g sign g1 g sign gp 1 g ( ( ), ( ),..., ( )) (2) Ojala et al. (2002a) found that not all loal binary patterns desribe properties of texture well. LBP aptures the uniformity of the entral pixel towards its neighborhood, but it does not 5

6 apture the uniformity of the neighborhood itself. Therefore, they introdued a uniformity measure U to define uniformity in a neighborhood set. U orresponds to the number of spatial transitions or bitwise 0/1 hanges in the pattern. With gp = g0, U is defined as P U = sign( g g ) sign( g g ) (3) C i i 1 i= 1 Patterns with U j are designated as uniform. Ojala et al. (2002a) found that for j = 2 the best texture model is obtained for texture images. This results in the following operator for grey sale and rotation invariant texture desription LBP, j P 1 sign( gi g) if UC j (4) = i= 0 + P 1 otherwise The LBP, j operator thresholds the pixels in a irular neighborhood of P equally spaed pixels on a irle of radius R, at the value of the enter pixel. It allows for deteting uniform patterns for any quantization of the angular spae and for any spatial resolution. Non-uniform patterns are grouped under one label, P A measure for texture The LBP, j measures the spatial struture of loal image texture, but disards ontrast, being another important property of loal image texture. In most ases, its performane an be enhaned by ombining it with a rotation invariant variane measure that haraterizes the ontrast of loal image texture, defined by 6

7 P 1 VAR = ( g µ ) i i= 0 2, where 1 1 P µ = gi (5) P i= 0 LBP, j and VAR values are alulated and assigned to eah individual image pixel, depiting loal texture information. Therefore, two new images are derived from the original image ontaining LBP, j and VAR values for eah pixel. These images form the basis for the final texture measure. Most approahes to texture analysis quantify texture measures by single values (means, varianes, entropy, et.). However, muh important information ontained in the distributions of feature values might be lost. In this study, the final texture feature is the histogram of the joint LBP, j and VAR ourrene, omputed over an image or a region of an image. The joint distribution of ( LBP, j, VAR ) is approximated by a disrete two-dimensional histogram of size P + 2 by b, where P is the number of neighbors in a irular neighborhood and b is the number of bins forvar. The number of bins used in quantization of the feature spae plays a ruial role. Histograms with too modest a number of bins fail to provide enough disriminative information about the distributions, however, if we go to the other extreme the number of entries per bin is very small and histograms beome sparse and unstable. In this study following Ojala et al. (1996), the number of bins b is omputed by taking the total feature distribution of ( LBP, j, VAR ) for the whole image. This distribution is divided into 32 bins having an equal number of entries. Ojala et al. (2002a) showed that the twodimensional ( LBP, j, VAR ) histogram is a powerful tool for rotation invariant texture segmentation. 7

8 2.4 Texture similarity Similarity between different textures is evaluated as a test of goodness-of-fit using a nonparametri statisti, the log-likelihood ratio statisti, also known as the G-statisti (Sokal and Rohlf 1987). The G-statisti ompares the bins of a texture sample histogram with a texture model histogram. The G-statisti is defined as tb tb tb filog fi fi log fi sm, i= 1 sm, i= 1 i= 1 G = 2 tb tb tb fi log fi + fi log f i i= 1 s, m s, m s, m i= 1 s, m i= 1 (6) where, the sample s is a histogram of the texture measure distribution of an image blok, the model m is a histogram of a referene area in the image of a partiular texture, tb is the total number of bins and f i is the probability for bin i. By using a nonparametri test we avoid making any, possibly erroneous, assumptions about the feature distributions. The value of the G-statisti indiates the probability that two sample distributions ome from the same population: the higher the value, the lower the probability that the two samples are from the same population. The more alike the histograms are the smaller is the value of G. Texture is modeled for ertain image bloks. The blok size should be appropriate for the omputation of the texture features. As we onsider bloks of inreased size, however, the probability that regions ontain a mixture of textures is inreased. This an bias the omparison, sine the referene textures ontain only features of individual patterns. On the other hand, if the blok size is too small it is impossible to alulate a texture measure. 8

9 Within this onstraint, it is impossible to define an optimum size for segmenting the entire image. Therefore, segmenting regions of a fixed blok size is inappropriate (Aguado et al. 1998). Alternatively, a top-down hierarhial segmentation proess, as disussed in the next setion, offers a very suitable framework for segmenting image regions based on texture. 2.5 Texture based image segmentation Split-and-merge segmentation onsists of a region-splitting phase and an agglomerative lustering (merging) phase (Haralik and Shapiro 1985, Horowitz and Pavlidis 1976, Gorte and Stein 1998, Luieer and Stein 2002, Luieer et al. 2004). Supervised segmentation uses expliit knowledge about the study area to train the segmentation algorithm on referene textures. Aguado et al. (1998) introdued a segmentation framework with a top-down hierarhial splitting proess based on minimizing unertainty. In this study, we ombine the ( LBP, j, VAR ) texture measure and the segmentation framework as suggested by Aguado et al. (1998). Similar to split-and-merge segmentation eah square image blok in the image is split into four sub-bloks forming a quadtree struture. The riterion used to determine if an image blok is divided is based on a omparison between the unertainty of the blok and the unertainty of the sub-bloks. The image is segmented suh that unertainty is minimized, where unertainty is defined as the ratio between the similarity values (G-statisti), omputed for an image blok B, of the two most likely referene textures (equation 7). The referene textures are histograms of ( LBP, j, VAR ) of harateristi regions in the image. To test for similarity between an image blok texture and a referene texture, the G-statisti is applied. Unertainty U B is then defined as 9

10 U B G G 1 = (7) 2 where G 1 is the lowest G value of all textures (highest similarity) and G 2 is the seond lowest G value. Unertainty is high if G 1 and G 2 are very similar and U B is lose to one. The subdivision of eah image blok is based on this unertainty riterion. An image blok is split into four sub-bloks if 1 UB > ( USB1+ USB2 + USB3 + USB4 ) (8) 4 where the left side of equation 8 defines unertainty obtained when the sub-bloks are labeled aording to the referene lass obtained by onsidering the whole blok (B). The right side of equation 8 defines unertainty if the sub-bloks (SB1, SB2, SB3 and SB4) are labeled by the referene lass obtained by the subdivision. Thus, the basi idea is to subdivide an image blok only if it is omposed of several textures. Additionally, segmentation is always unertain at the boundaries, beause the image blok ontains a mixture of textures. Aordingly, we subdivide bloks that have at least one neighboring region of a different texture (Aguado et al. 1998). Finally, we obtain a partition of the image. We onsider an image objet as a olletion of ontiguous image bloks sharing the same texture label. The building bloks of eah of the objets give information about objet unertainty. We use U B to depit unertainty with whih an objet blok is assigned a texture label. The spatial 10

11 distribution of blok unertainty values within an objet gives information about unertainty of the spatial extent of objets. We expet high unertainty values for the boundary bloks of objets, beause of mixed textures and transition zones. 2.7 Texture example Figure 1(a) shows a omposite of five different textures. The image is derived from the Outex framework for testing texture models (Ojala et al. 2002b). These grey-sale textures were labeled with the following lass names: lass NW (granite), lass NE (fabri), lass SW (grass), lass SE (stone) and lass Center (reed mat). Eah of these lasses is unique in terms of texture. The image shows that the human visual system not only distinguishes image regions based on grey-sale or olor, but also on texture, as one an learly distinguish five homogeneous regions. A pixel-based lassifier does not take into aount texture or spatial information. Figure 1(b) shows why pixel-based lassifiation tehniques might fail. It shows the `defuzzified result of a pixel-based FCM lassifier. In this ase, a supervised fuzzy - means lassifiation was applied with a Mahalanobis distane measure and an overlap parameter of 2.0 (Bezdek 1981, Zhang 2001). Five regions of 40 by 40 pixels were seleted in the enters of the texture regions to train the lassifier. Although, the patterns are still visible, no lear spatial partition of lasses was obtained. A lassifiation validation provided an overall auray of 30.00% and a Kappa oeffiient of [FIGURE 1 ABOUT HERE] A muh better segmentation was obtained when texture was inorporated by applying the unsupervised texture-based segmentation algorithm based on the joint ( LBP, j, VAR ) 11

12 distribution. A detailed desription of these results an be found in Ojala and Pietikäinen (1999) and Luieer et al. (2004). Additionally, a supervised approah might prove to be useful as one an guide the segmentation algorithm with referene texture information. Espeially in geographial appliations, a supervised approah is often feasible, as knowledge about the area might improve segmentation. Figure 2 shows a supervised texture-based segmentation of figure 1(a), applying the unertainty riteria of Aguado et al. (1998). Five referenes regions of 40 by 40 pixels were seleted, orresponding to the five different textures in figure 1(a). Values for P and R were 8 and 1 respetively. An auray assessment of the segmentation results provided a very high overall auray of 96.20% and a Kappa oeffiient of 0.95, showing that good segmentation results an be obtained with the LBP texture measure. Unertainty values were highest in lass SW. This an be explained by the irregularity of this texture, i.e. its pattern is not repetitive and the referene area does not fully represent the whole texture area. In addition, all small bloks at the boundaries of textures show high (>0.9) unertainty values, beause they ontain mixtures of different textures. [FIGURE 2 ABOUT HERE] 2.8 A multivariate texture model The LBP, j texture measure allows a texture desription of a single band. Most remote sensing images, however, onsist of multiple bands. Inluding multiple bands might improve segmentation onsiderably, as a ombination of bands provides more spetral information for identifiation of different land over types. 12

13 In their psyhophysial study Poirson and Wandell (1996) showed that olor and pattern information are proessed separately by the human visual system. Mojsilovi et al. (2000) extrated olor-based information from the luminane and hrominane olor omponents. The ahromati pattern omponent was utilized as texture pattern information. Another approah was that of Panjwani and Healey (1995) whih aptured spatial interations both within and between olor bands with Markov random fields (MRFs). More reently, Pietikäinen et al. (2002) showed that the powerful LBP texture measure an also be applied to olor images. They proessed olor information and texture information separately and obtained good lassifiation results for olor texture images. Most researh on olor texture foused on images of different materials with a lear texture. In standard olor images, the pattern in different bands is often highly orrelated. This makes it possible to summarize pattern information in a single band and proess it separately from olor information. In remote sensing images, however, information is reorded from different parts of the spetrum. Therefore, textures in these bands are not neessarily similar. In between band relations should be taken into aount when looking at multivariate texture measures for remotely sensed imagery. The LBP, j texture measure is a robust, rotation invariant and flexible texture measure. An extension to the multivariate ase is expeted to provide good segmentation results. In this study, a new multivariate texture measure is introdued and implemented. It is based on the univariate LBP, j measure. The Multivariate Loal Binary Pattern operator, MLBP desribes loal pixel relations in three bands. In addition to the spatial interations of pixels 13

14 within one band, interations between bands are onsidered. Thus, the neighborhood set for a pixel onsists of the loal neighbors in all three bands (figure 3). The loal threshold is taken from these bands, whih makes up a total of nine different ombinations. This results in the following operator for a loal olor texture desription MLBP = sign g g + sign g g + sign g g + (9) b1 b1 b2 b1 b3 b1 sign( gi g ) sign( gi g ) sign( gi g ) P 1 b1 b2 b2 b2 b3 b2 ( i ) ( i ) ( i ) i= 0 b1 b3 b2 b3 b3 b3 sign( gi g ) + sign( gi g ) + sign( gi g ) where b1 is the first band, b2 is the seond band, and b3 is the third band. The first part of the equation alulates LBP values for the enter pixel of the first band based on relations with the neighbors in the first band and the two other bands. The seond part of the equation alulates LBP values for the enter pixel of the seond band and the third part of equation 9 alulates LBP values for the enter pixel of the third band. Eah of the three entral pixels is, therefore, ompared with neighborhood pixels in the other bands. MLBP is not just a summation of LBP, j of individual bands, it also models pixel relations between bands. These ross-relations an be important in the distintion of different olor textures. A total of nine LBP values is obtained and summed to derive MLBP. The olor texture measure is the histogram of MLBP ourrene, omputed over an image or a region of an image. This single distribution ontains 2 P 3 bins (e.g. P = 8 results in 72 bins). [FIGURE 3 ABOUT HERE] MLBP measures the binary olor pattern of a texture. To omplete this measure with ontrast and variane information we inluded the olor histogram, RGB-3D. Eah 8-bit band 14

15 is quantized into 32 levels by dividing the pixel values on eah band by 8, resulting in a three-dimensional histogram with 3 32 entries. The two histograms of MLBP and RGB-3D are used to segment a three-band image into objets. In the top-down hierarhial splitting proess we alulate MLBP and RGB-3D histograms for every image blok. G-statisti values are alulated to test for similarity between image blok and referene texture histograms. For two MLBP and RGB-3D histograms, two G-statisti values are obtained. These values are summed to derive a single similarity measure. Based on this measure, unertainty values are alulated using equation 7 and texture labels are assigned to image bloks to form objets To illustrate the solution for segmenting regions of different olor texture, a three-band image (512 by 512 pixels) with a omposition of six different olor textures (figure 4(a)) was used. This image was omposed of textures from the Outex texture library (Ojala et al., 2002b). The following textures were used: Upper Left (UL) = fur, Upper Right (UR) = arpet, Middle Left (ML) = wood, Middle Right (MR) = pasta, Lower Right (LR) = flour, Lower Left (LL) = seeds. It poses a more diffiult segmentation problem than the grey-sale texture omposition of figure 1(a), beause of the high variane in olor and the different texture sales. Six referenes regions of 40 by 40 pixels were seleted, orresponding to the six different texture lasses. Values for P and R were 8 and 1 respetively. Figure 4(b) shows the segmentation result. All regions were identified orretly. In the lower left objet (LL), however, some dark spots were (inorretly) segmented as fur (UL). This was most likely aused by high similarity in olor distributions. Additionally, in the lower right objet (LR) some dark shadow spots were (inorretly) segmented as flour (LL). Unertainty for these inorretly labeled objets and boundary regions was high (>0.9) (figure 4()). 15

16 An auray assessment of the segmentation result provided an overall auray of 98.32% and a Kappa oeffiient of The onfusion matrix with per-texture auray perentages is given in table 1. These auray values show that good segmentation results an be obtained with the multivariate LBP texture measure. [FIGURE 4 ABOUT HERE] [TABLE 1 ABOUT HERE] 3. Case study 3.1 Study area: the Ainsdale Sands The study area, known as the Ainsdale Sands, is on the oast of Northwest England approximately 25km North of Liverpool. The Ainsdale Sand Dunes National Nature Reserve (NNR) totals 508 ha and forms part of the Sefton Coast. The NNR is within the oastal Speial Protetion Area. It ontains a range of habitats, inluding intertidal sand flats, embryo dunes, high mobile yellow dunes, fixed vegetated dunes, wet dune slaks, areas of deiduous srub and a predominantly pine woodland. Management of this area onsists of extending the area of open dune habitat through the removal of pine plantation from the seaward edge of the NNR, maintaining and extending the area of fixed open dune by grazing and progressively reating a more diverse struture within the remaining pine plantation with assoiated benefits for wildlife (Sefton Coast Partnership, 2004). In 1999, 2000 and 2001 the Environment Ageny, UK, olleted fine spatial resolution 16

17 digital surfae models (DSM) by LiDAR, and simultaneously, aquired multi-spetral Compat Airborne Spetral Imager (CASI) imagery (one flight eah year). The airraft flew at approximately 800 m above ground level, aquiring 2 m spatial resolution LiDAR senes and 1 m spatial resolution CASI imagery. In this study, the CASI image of 2001 was used. These images, geometrially orreted by the Environment Ageny, were spatial omposites of multiple flight strips. The area overed by these images was approximately 6km2. We applied the univariate segmentation algorithm on the LiDAR DSM to derive general landform lasses (Luieer et al., 2004). An auray assessment of the segmentation results provided an overall auray of 86%. The results showed that the univariate LBP measure in ombination with the hierarhial splitting algorithm an provide a meaningful segmentation of basi land form lasses with an indiation of objet unertainty. In this study, we fous on segmentation of land over lasses from multispetral CASI imagery. 3.2 Land over segmentation Land over is obtained from spetral information from the CASI image. Four land over lasses an be distinguished: sand, marram grass, willow shrub and woodland. Detailed mapping of these units is required, beause knowledge about the loation and dynamis of these objet types is important for monitoring the rare habitats in this area, as well as, the oastal defense against flooding. Figure 5(a) shows a subset (512 by 512 pixels) of band 12 of the CASI image of the study area. Band 12 at 780 nm (NIR) was hosen for a univariate segmentation based on the joint distribution of LBP, j and VAR values. It is suitable for disrimination of land over types, 17

18 beause of large differenes in refletane for different vegetation types. Four referene areas of 40 by 40 pixels were seleted to train the algorithm. Values for P and R were 8 and 1 respetively. Figure 6(a) shows a segmentation of band 12 with the ( LBP, j, VAR ) texture measure for four land over lasses. The woodland area in the southeast orner of the image was orretly segmented with unertainty values between 0.1 and 0.5 (figure 6(b)). The northeastern orner of the image and small objets in the northern part of the image were also segmented as woodland. However, fieldwork showed that no woodland ourred in this area. The main part of the dune field was segmented as willow shrub land. Fieldwork showed that marram grass was mainly found on the fore dune and on the highest parts of the dune ridges in the dune field. Only a few small pathes of marram grass an be seen in figure 6(a) in the fore dune area. Willow shrub was found all over the dune field, but mainly in the dune slaks. Image texture for these two lasses, however, is very similar in band 12 of the CASI image. High unertainty values (higher than 0.7 in the dune field and higher than 0.95 in the fore dune and dune ridge areas) in figure 6(b) onfirm the similarity of these two land over lasses. The sand on the beah was orretly segmented, beause of its harateristi texture. Unertainty values were lower than 0.2 in this area. Again, a short transition zone an be seen from the fore dune to the beah with dereasing marram grass overage (figure 6(b)). This zone is depited by unertainty values of 0.95 and higher. Field observations showed that the univariate texture-based segmentation algorithm performed unsatisfatorily, espeially in areas where marram grass was severely under-segmented. Table 2 onfirms this observation, showing auray values for individual land over lasses. The overall segmentation auray was 70.53% and the Kappa oeffiient was It an be onluded from table 2 that major marram grass areas were inorretly segmented as willow shrub. 18

19 [FIGURE 6 ABOUT HERE] [TABLE 2 ABOUT HERE] 3.3 Multivariate texture-based land over segmentation Segmentation using only one CASI band disards valuable information in other bands. A multivariate approah towards texture segmentation might improve segmentation results. The ombined MLBP and RGB-3D texture measure, models texture in three bands. CASI band 1 (440 nm), 8 (650 nm) and 12 (780 nm) explain most of the variane in the image sene and haraterize land over lasses well. Figure 5(b) shows a olor omposite of these three bands. Figure 7 shows a supervised segmentation based on MLBP and RGB-3D. Segmentation of the marram grass lass improved onsiderably. The fore dune area and the dune ridges were segmented as marram grass, as was observed in the field. The ore areas showed low unertainty values, whereas the boundaries showed high unertainty values. This orresponds to observations that marram grass gradually hanged to willow shrub land in the dune slaks and to sandier terrain towards the beah side. The woodland area was segmented orretly. In addition, segmentation of the north-eastern part of the area (marram grass and willow shrub) improved, as the segmentation result of a single band (figure 6) showed woodland in this area. The beah area was orretly segmented with low unertainty values. Some small inorretly segmented bloks (marram, willow and woodland) ourred in the beah area where the sand was wet with low refletane values in the image. High unertainty values (>0.9) ourred in all transition areas. These unertainty values are an indiation for the ourrene of fuzzy objets with indeterminate boundaries. 19

20 An auray assessment of the segmentation results provided an overall auray of % and a Kappa oeffiient of The onfusion matrix with per-lass auray perentages is given in table 3. It an be onluded from this onfusion matrix that segmentation of marram grass and willow shrub improved onsiderably ompared to segmentation based on one CASI band. [FIGURE 7 ABOUT HERE] [TABLE 3 ABOUT HERE] 4. Disussion and onlusions In this study, a texture-based supervised segmentation algorithm derived labeled objets from remotely sensed imagery. Texture was modeled with the joint distribution of the loal binary pattern (LBP) operator and loal variane. The segmentation algorithm was based on a hierarhial splitting tehnique, reduing unertainty at the level of the image bloks that were obtained. By applying this tehnique, one does not only obtain a texture-based image segmentation, yet also an indiation of unertainty for all objet building bloks. The spatial distribution of unertainty values provided information about the loation and width of transition zones. The univariate LBP, j texture measure was extended to a multivariate measure, MLBP, to model within band and between band pixel relations in three bands. The MLBP measure was further extended with olor information using a three-dimensional olor histogram, RGB-3D. The ombination of these texture measures, model olor texture as registered on different 20

21 bands. The univariate LBP measure provided good segmentation results for a test ase study with a omposite image of five different grey-sale textures. An overall auray of 96% was obtained. An artifiial image with a omposition of six olor textures was used to demonstrate the use of MLBP and RGB-3D in segmentation. Good segmentation results were obtained from this omplex texture image, depited by an overall auray of 98%. To illustrate the algorithm for mapping oastal objets, a CASI image of a oastal area on the northwest oast of England was used. Land over objets derived from band 12 of the CASI image showed high unertainty values and many inorretly labeled objets. The overall auray was 71%. Additionally, ompared to field observations segmentation results were unsatisfatorily. The ombination of textural and spetral information from more than one CASI band greatly improved segmentation results. The MLBP and RGB-3D based segmentation was applied to band 12, 8 and 1 of the CASI image of the study area. Segmentation results improved onsiderably, depited by and overall auray of 77%. Unertainty values provided valuable information about transition zones between fuzzy objets. In this study, we applied a texture-based segmentation algorithm on airborne imagery for identifiation of oastal objets. The proposed algorithm, however, an easily be applied to other remote sensing images and other study areas. The univariate and multivariate LBP measures an also be used in a different ontext. Contextual lassifiation using the LBP 21

22 texture measure might provide valuable results from image lassifiation. The omputation of the multivariate LBP measure was limited to three bands. More image bands ould be used in the MLBP and RGB-3D texture model. It would, however, inrease omplexity and omputational demands onsiderably, whereas extra bands would possibly not add muh textural information. For multispetral and hyperspetral images, one ould inlude a preproessing step to detet the three bands that explain most variane for a speifi appliation In this study, a uniformity measure is defined for the univariate LBP measure. It depits uniformity of pixel values in a neighborhood set. Ojala et al. (2002) showed that more than 90% of the patterns in a texture image are uniform. In remote sensing images, however, also non-uniform patterns our. Some of these non-uniform patterns might be harateristi for a ertain land over lass. An extension of the uniformity measure to the multivariate ase might provide more information on pattern uniformity in remotely sensed imagery. A multivariate uniformity measure ould be alulated by summation of uniformity in eah band or by ombining the uniformity measure for eah of the nine omponents in the multivariate LBP measure (equation 9). We will assess the effet of a multivariate uniformity measure on segmentation of multispetral remote sensing imagery in future researh. The resolution of the neighborhood set applied during segmentation affets the texture measure and, therefore, the segmentation result. In this study, a (irular) neighborhood set of the nearest eight neighboring pixels was used. Cirular neighborhood sets with large radii and a large number of neighbors might improve desription of large-sale textures. Therefore, a multi-resolution approah with different ombinations of neighborhood sets might provide a meaningful texture desription. In future researh, the effet of different 22

23 neighborhood sets on the segmentation result will be assessed. Referenes AGUADO, A.S., MONTIEL, E., AND NIXON, M.S., 1998, Fuzzy image segmentation via texture density histograms. EU projet Nr. ENV4-CT Fuzzy Land Information from Environmental Remote Sensing (FLIERS) Final Report, BEZDEK, J., 1981, Pattern Reognition with Fuzzy Objetive Funtion Algorithms (Plenum Press, New York). CANTERS, F., 1997, Evaluating the unertainty of area estimates derived from fuzzy land over lassifiation. Photogrammetri Engineering & Remote Sensing, 63, CHENG, T., FISHER, P.F., AND ROGERS, P., 2002, Fuzziness in multi-sale fuzzy assignment of duneness. Auray International Symposium On Spatial Auray Assessment in Natural Resoures and Environmental Sienes, pp CHENG, T. AND MOLENAAR, M., 2001, Formalizing fuzzy objets from unertain lassifiation results. International Journal of Geographial Information Siene, 15, FISHER, P.F., 1999, Models of Unertainty in Spatial Data, In Geographial Information Systems, seond edition (Wiley & Sons, New York), pp FISHER, P., CHENG, T. AND WOOD, J., 2004, Where is Helvellyn? Multisale morphometry and the mountains of the English lake distrit, Transations of the Institute of British Geographers, 29, FOODY, G. M., 1996, Approahes for the prodution and evaluation of fuzzy land over lassifiations from remotely sensed data. International Journal of Remote Sensing, 17, FOODY, G.M. AND ATKINSON, P.M., 2002, Unertainty in GIS and Remote Sensing (John Wiley & Sons Ltd). 23

24 GORTE, B. H. H. AND STEIN, A., 1998, Bayesian lassifiation and lass area estimation of satellite images using stratifiation. IEEE Transations on Geosienes and Remote Sensing, 36, HARALICK, R.M., SHANMUGAM, K., AND DINSTEIN, I., 1973, Textural features for image lassifiation. IEEE Transations on Systems, Man and Cybernetis, 2, HARALICK, R.M. AND SHAPIRO, L.G., 1985, Image segmentation tehniques. Computer Vision, Graphis and Image Proessing, 29, HOOTSMANS, R.M., 1996, Fuzzy Sets and Series analysis for Visual Deision Support in Spatial Data Exploration. PhD thesis, Utreht University. HOROWITZ, S.L. AND PAVLIDIS, T., 1976, Piture segmentation by a tree traversal algorithm. Journal of the Assoiation for Computing Mahinery, 23, LUCIEER, A. AND STEIN, A., 2002, Existential unertainty of spatial objets segmented from remotely sensed imagery. IEEE Transations on Geosiene and Remote Sensing, 40, LUCIEER, A., FISHER, P. AND STEIN, A., 2004, GeoDynamis (CRC Press LLC), hapter Texture-based Segmentation of Remotely Sensed Imagery to Identify Fuzzy Coastal Objets. MOJSILOVIC, A., KOVACEVIC, J., HU, J., SAFRANEK, R. AND GANAPATHY, S., 2000, Mathing and retrieval based on the voabulary and grammar of olor patterns, IEEE Transations on Image Proessing, 9, NIXON, M.S. AND AGUADO, A.S., 2002, Feature extration & image proessing (Butterworth-Heinemann). PANJWANI, D. AND HEALEY, G., 1995, Markov random field models for unsupervised segmentation of textured olor images, IEEE Transations on Pattern Analysis and Mahine Intelligene, 17, PIETIKÄINEN, M., MÄENPÄÄ, T. AND VIERTOLA, J., 2002, Color texture lassifiation 24

25 with olor histograms and loal binary patterns, Proeedings of the Seond International Workshop on Texture Analysis and Synthesis, Copenhagen, Denmark, POIRSON, B. AND WANDELL, B., 1996, Pattern-olor separable pathways predit sensitivity to simple olored patterns, Vision Researh 36, OJALA, T., PIETIKÄINEN M. AND HARWOOD, D., 1996, A omparative study of texture measures with lassifiation based on feature distributions. Pattern Reognition, 29, OJALA, T. AND PIETIKÄINEN, M., 1999, Unsupervised texture segmentation using feature distributions. Pattern Reognition, 32, OJALA T., PIETIKÄINEN M., AND MÄENPÄÄ, T., 2002a, Multiresolution gray-sale and rotation invariant texture lassifiation with loal binary patterns. IEEE Transations on Pattern Analysis and Mahine Intelligene, 24, OJALA, T., MÄENPÄÄ, T., PIETIKÄINEN, M., VIERTOLA, J., KYLLÖNEN, J. AND HUOVINEN, S., 2002b, Outex - New framework for empirial evaluation of texture analysis algorithms. 16th International Conferene on Pattern Reognition, Quebe, Canada, 1: URL: PIETIKÄINEN M., OJALA T., AND XU, Z., 2000, Rotation-invariant texture lassifiation using feature distributions. Pattern Reognition, 33, RANDEN, T. AND HUSØY, J.H., 1999, Filtering for Texture Classifiation: A Comparative Study. IEEE Transations on Pattern Analysis and Mahine Intelligene, 21, SEFTEN COAST PARTNERSHIP, 2004, Sefton oast partnership nature onservation, URL: SOKAL, R.R. AND ROHLF, F.J., 1987, Introdution to Biostatistis, seond edition (W.H. Freeman and Co, New York). WEL, F. VAN DER, 2000, Assessment and Visualisation of Unertainty in Remote Sensing Land Cover Classifiations. PhD thesis, Utreht University. 25

26 ZHANG J. AND FOODY, G. M., 2001, Fully-fuzzy supervised lassifiation of sub-urban land over from remotely sensed imagery: Statistial and artifiial neural network approahes. International Journal of Remote Sensing, 22,

27 Tables Table 1. Confusion matrix with auray values (%) of a multivariate texture-based segmentation. Overall auray was 98.32% and the Kappa oeffiient was Table 2. Confusion matrix with per-lass auray values (%) of a univariate texture-based segmentation of band 12 of the CASI image. Overall auray was 70.53% and the Kappa oeffiient was Table 3. Confusion matrix with per-lass auray values (%) of a (multivariate) texturebased segmentation of bands 12, 8 and 1 of the CASI image. Overall auray was 77.09% and the Kappa oeffiient was Table 1 Referene Class UL UR ML MR LL LR Total UL UR ML MR LL LR Total Table 2 Referene Class Sand Marram Grass Willow Shrub Woodland Total Sand Marram Grass Willow Shrub Woodland Total Table 3 Referene Class Sand Marram Grass Willow Shrub Woodland Total Sand Marram Grass Willow Shrub Woodland Total

28 Figure Captions Figure 1. Texture image omposition: (a) artifiial omposition of five different natural textures with five referene areas (Ojala et al. 2002b); (b) result of a pixel-based lassifier. Figure 2. Supervised texture-based segmentation: (a) segmentation based on the joint LBP, j and VAR distribution with five referene lasses; (b) related unertainty for all objet building bloks. Figure 3. The neighborhood set for the multivariate (three band) LBP texture measure desribes spatial pixel relations within a band and between bands. Figure 4. Segmentation of olor texture image: (a) artifiial omposition of five different olor textures (Ojala et al. 2002b); (b) supervised texture-based segmentation based on the multivariate MLBP distribution and RGB-3D olor histogram with five referene lasses; () related unertainty for all objet building bloks. Figure 5. CASI image of the Ainsdale sands study area, UK: (a) band 12; (b) olor omposite of band 12, 8 and 1 (RGB). Figure 6. Segmentation of land over from CASI image: (a) supervised texture-based segmentation of band 12 (NIR) of the CASI image with four referene land over lasses, based on the joint univariate LBP, j and VAR distribution; (b) related unertainty for all objet building bloks. Figure 7. Multivariate texture-based segmentation of land over from CASI image: (a) supervised segmentation of band 12, 8 and 1 based on olor histogram; (b) related unertainty for all objet building bloks. MLBP distribution and RGB-3D 28

29 Figures Figure 1 (a) (b) 29

30 Figure 2 (a) (b) 30

31 Figure 3 31

32 Figure 4 (a) (b) () 32

33 Figure 5 (a) (b) 33

34 Figure 6 (a) (b) 34

35 Figure 7 (a) (b) 35