Methods for Contextual Classification of Remotely Sensed Data

Transcription

1 FATTI, L.P., ELPHINSTONE, C.D. and LONERGAN, A. T. Methods fo contextual classification of emotely sensed data. APCOM 87. Poceedings of the Twentieth Intenational Symposium on the Application of Computes and Mathematics in the Mineal Industies. Volume 3: Geostatistics. ohannesbug, SAIMM, 987. pp Methods fo Contextual Classification of Remotely Sensed Data L.P. FATTI*, C.D. ELPHINSTONEt and A.T. LONERGANt *Depatment of Statistics, Univesity of the Witwatesand, ohannesbug tnational Reseach Institute fo Mathematical Sciences, CSIR, Petoia An impotant featue affecting the classification of a emotely sensed scene is that the data ae, in geneal, spatially coelated. Vaious appoaches ae discussed towads incopoating the coelation between neighbouing pixels when they ae classified on the basis of thei spectal eflectance data. Intoduction In the emote-sensing context, classification efes to the pocess of allocating pixels in a scene to one of a numbe of distinct classes o goups on the basis of the spectal (and possibly othe) data associated with each pixel. Geneally speaking, the classes coespond to the di ffeent geogaphic featues o land--use pattens of the teain coveed by the scene. A common assumption made in classification studies is that the data vectos associated wi t.h diffeent pixels ae independent and follow a multivaiate nomal distibution, with mean vecto vaying fom class to class, but with a common covaiance matix ove all classes. Unde these assumptions, it tuns out that the function that disciminates optimally between the diffeent classes by minimizing the pobabili ty of misclassification - is the linea disciminant function. Essentially, this function is a weighted linea combination of the elements of the data vecto, the weights depending on the common covaiance matix and the mean vecto of the class in question, plus a constant function of the class mean and the pio pobability of the class. The (Bayesian) classification ule based on this funtion is to allocate the data vecto to the class fo which this function is a maximum. Specifically, denoting the j-th class by., if z denotes the data vecto associated - with a pixel, then the pixel is all ocated to that class. whee and fo which u. (z) "- max u. (z) [] - j:::cl,...,k u. (z) = Z'L, /::!. %f::! '.2: /::! P ~, /::!. is the mean vecto, 2: is the common covaiance matix, p. is the pio pobability of class. k is the numbe of classes epesented in the scene. pio pobabilities p. ae en If the all equal, then the last tem in u.(z) - falls away. To implement the above allocation ule to CONTEXTUAL CLASSIFICATION OF REMOTELY SENSED DATA 3

2 classify a scene, values have to be assigned to the paametes L:,. and p., j=l,...,k. Given taining samples in the scene, fo which the tue class (o gound tuth) ae known, the mean vecto e.. can be estimated fom class mean vectos, and L: can be estimated fom the pooled within--goups covaiance matix. The pio pobab:ili ties p. ae commonly assumed to be equal fo all classes; othewi.se they can be estimated fom the elative fequencies of the diffeent classes jn the taining samples if these ae epesentative of the scene as a whole, o fom (~xtenal infomation about the scene. In two data gell(~ally impotant aspects emote-sensing do not satisfy the assumptions on which the above classification ule is based. Fistly, the data vectos ae not independent, and secondly, the covaiance mati ces of the di ffeent. classes ae often not equal. On the othe hand, the assump-- tion of multivadate nomality IS usually acceptable fo emotely sensed spectal data. 2 Unde the heteoscedastic (unequal covaiance matices) nomal model, the optimal discimination between classes is achieved by a quadatic function of the data vecto, most conveniently expessed in tems of the Mahalanobis distances - = (z-u.)'l:. (z-u.), 6~(z) ~ ~ ~ ~ ~ fom each of the classes ll.,,jool,...,k. The pixe is then allocated to that class ll. fo which: cs~(z) +I!n IL:.I - 2 en p. ~ [2] is a minimum, whee ILj I is the deteminant of the covaiance matix of the data fom ll.. Dawbacks to this quadatic assignment ule ae that it is slowe to compute than 4 the linea ule (l) and that its pefomance is geneally moe sensitive to depatues fom nomality. 3 An inteesting obsevation in this context is that although the covaiance matices in the diffeent classes may diffe, lhis diffeence is usually due to diffeences in the standad devi ahons of the i ndl v.i dual eflectances athe than in thei coelation matjee-l Th:is featue has obvious advantages as egads paamete estimation, and may also be exploited to speed up the computations. This pape concent.ates on the fist t.ype of depatue fom the assumpt ions, llame ly that of non-j ndependence hetw(~(m Uw data vectos in neighbouing pixels, commonly efeed to as spat] a dependency. Spatial dependency between pixels may occu because of scatteing of the eflected electomagnetic ays on the suface of the eath, pat.:icualy Tl boken teain, causing contaminatioll of the eflectances in H pixel by those of nelghbouj ng pi Xf!] s_ This contamination of the spectal dat.a can be aggavl:lled by some foms pocessing of the image data, such as 'ovesampling' and 'esampling'. A discussion of these foms of pepocessing, as well as of thei effects 0 the specta data, is given in Ephinstone et 8.2 Anothe cause of spatial dependency is th~ spatial continuity of the gound clhssesin most emotely sensed scenes, as a esult of single geogaphic featues o and--use pat tens tending to compise lage aeas than that coveed by a s ingl e ( o even a few) pixels. Thus, even if we ignoe the contamination aspect, the spectal data fom neighbouing pixels will not be independent although, given thei gound classes, they may be. (This is efeed to as cassconditional independence.) Spatial dependency can affect the GEOSTA TISTICS: THEORY

3 pefomance of the allocation ule in two ways. Fistly, the obsevations in the taining samples will not be independent, esulting in a biased estimate of the covaiance matix!. Lawoko and McLachlan 4 show that positive seial coelation in the taining samples inceases the classification eo ate, and Foglein and Kittle S show that if the pixel dependencies can be modelled by an ARMA pocess, then class sepaability, and hence the classification eo ate, can be impoved by decoelating the obsevations. Clealy, one way of getting aound the dependencies in the taining samples is to select the pixels so that they ae suff:iciently fa apat fo the coelation between thei data vectos to be negligible. While this will impove the sampling efficiency, it may lead to difficulties in obtaining the gound tuth infomation fo these scatteed pixels. The second way in which the spatial dependency can affect the pefomance of the allocation ule is in the actual classification pocess itself. Both linea () and quadatic (2) allocation ules opeate on the scene on a pixel-by-pixel basis, each allocation being based puely on the data associated with the pixel. independently of the data o classifications of the neighbouing pixels. Since this pocedue takes no cogn:izance of any local spatial cont inui ty of the tue gound classes, it tends to poduce classification maps that ae 'patchie' than the tue scene. The vaious appoaches towads contextual classification dif';cussed in this pape attempt to incopoate both the spatial dependencies between the data fom neighbouing pixels. and the local spatial continuity of the gound classes into the allocation ules to poduce classification maps that moe closely fit the tue gound classifications in the scene. In the next section. vaious appoaches involving some fom of 'pe-smoothing' of the data ae discussed, including those using geostatistical concepts fo obtaining smoothing weights. all of which attempt to incopoate the spatial dependencies amongst neighbouing pixels by smoothing thei associated (spectal) data. Wheeas these foms of smoothing ae all applied to the data pio to classification, 'postsmoothing', which is discussed next, is applied to the actual classifications, o to the classification pobabilities of each of the pixels. This fom of smoothing theefoe takes cognizance of the spatial continuity of the gound tuth classes. Included unde this heading is a 'two-pass' appoach, in which pixel-specific pio pobabilities fo the vaious classes estimated fom the fist pass ae used to classify the pixels in the second pass. Contextual classification based on explicit models of the spatial dependency is discussed in the following section. Thee types of models will be discussed. The fist type diectly models the coelation stuctue of the eflectances in neighbouing pixels, to which classical disciminant analysis pocedues, such as the linea and quadatic allocation ules descibed ealie, can be applied. The second type models the spatial vaiation of the gound classes themselves and incopoates these models into the Bayesian classification fomulation togethe with class-conditionally independent models fo the spectal data. The thid and most complex type of spatial dependency model incopoates models of both the spatial vaiation of the gound classes and the spatial dependency of the spectal data. Seveal of the contextual classification CONTEXTUAL CLASSIFICATION OF REMOTELY SENSED DATA 5

4 algoithms discussed in this section ae, in thei pesent state, fa too heavy computationally to be of pactical inteest to the emote-sensing community. Simplification of thei models, as well as thei implementation on fast paallel pocessos may, howeve, change this pictue. Fo the pupose of the discussion, the simple five-pixel neighbouhood, shown in Figue I below, will geneally be used to illustate the vaious contextual classifies consideed. This consists of the pixel of inteest, numbeed at the cente of the figue, with its fou closest neighbous, nominally along the fou pincipal diections of the compass. When the pixel of' inteest is allocat.ed using a contextual classifie, only the pixels within the neighbouhood ae taken into account FIGURE. The simple five-pixel neighbouhood Clealy, othe foms of neighbouhood could also be consideed, such as the nine-pixel neighbouhood, which, in addition to the fou neaest neighbous, also includes the fou cone pixels that ae next closest to the pixel of inteest. Most of the ideas discussed below, with appopiate modifications, hold fo othe foms of neighbouhood, the main cost associated with using lage neighbouhoods being computational time. 6 Only supevised classification, in which the taining samples ae based on gound tuth infomation, is consideed in this pape. Futhemoe, as the object is to pesent the mathematical and statistical concepts undelying contextual classification, no consideation is given to thei incopoation into image-pocessing systems, and computational expeience with them is only discussed biefly. Contextual classifies that incopoate extenal o subjectively assessed infomation, such as the ECHO (Extaction and Classification of Homogeneous Objects) pocess of Ket ti g and Landgebe 6 in which homogeneous objects that ae lage compaed to the pixel size ae identified and classified simultaneously, will not be consideed. Pe-smoothing the eflectances One of the simplest methods of jncopoating contextual infomation into the classification pocess is to smooth the eflectances of a pixel using those of its neighbous, that is to eplace t.he eflectance vecto of the pixel by a weighted aveage of those of the pixes in its neighbouhood. Fo example, the weighting scheme which coesponds to the numbeing system in Figue. gives equal weight to the eflectances in the cental pixel and to the aveage of the eflectances of the fou suounding pixels. A geneal, symmetical weighting scheme would be of the fom: {-4p,p,p,p,p}, with p ~ ~. A method fo establishing a value fo p would be to apply kiging weights (Kige") obtained fom the semi-vaiogams fitted to the spectal data fom taining samples fom contiguous aeas of the scene. Poblems GEOSTATISTICS: THEORY

5 associated with using this geostatistical appoach fo smoothing the eflectances ae: (a) Since the semi-vaiogams of the diffeent spectal bands ae unlikely all to be the same, it would imply that diffeent values of p would be applied to the diffeent spectal bands. Wheeas the multivaiate stuctue of the data vectos could be incopoated into the weighting pocess via cokiging, a simple appoach would be to obtain a single set of weights fom the semi-vaiogam of the fist canonical component of the spectal data. (b) Since the kiging estimato is an exact intepolato, the geostatistical appoach, if stictly applied, will assign zeo weights to all neighbouing pixels and thus not smooth the data at all. One way to ovecome this poblem would be fist to obtain the kiging estimato fo the cental pixel using the eflectances fom its neighbous alone, via any of the methods descibed unde (a) above. The estimated and actual eflectances could then be combined via, say, canonical vaiate analysis. Cc) By using the above symmetical weighting scheme, possible diectional effects in the image ae not taken into account in the smoothing pocess. Howeve, since Elphinstone et al.2 have found stong evidence of such anisotopy in spectal data, possibly as a esult of pepocessing of the image, it is ecommended that this be taken into account in the weighting scheme. A simple non-symmetical weighting scheme fo the neighbouhood given in Figue would be: {-2(p+q),p,q,p,q} with p+q ~ t. The weights p and q could then be obtained fom sepaate semi-vaiogams fo the 'east-west' and 'noth-south' diections, espectively. A moe efined weighting scheme could be obtained by using two-dimensional semi-vaiogams, accounting fo all diections at once. Smoothing the eflectances wi tend to poduce classification maps in which changes fom class to class happen moe smoothly, yielding moe contiguous classes and fewe pathological pixels classified diffeently fom thei neighbous. On the othe hand, smoothing will also lowe the esolution of the image, esulting in smoothe class boundaies than those possibly existing on the gound, and ~ven eliminating some boundaies altogethe. Post-smoothing the classification maps In contast to the pe-smoothing techniques discussed above, post-smoothing techniques attempt to smooth the clas-sification maps that have been poduced by ignoing any contextual infomation, on the assumption that smoothness is inheent in the tue gound classes of the scene. The most obvious way of doing this is to eclassify a pixel into the class in which most of its neighbous fall. One poblem associated with this appoach is that of esolving pixel classifications nea class boundaies, whee neighbous on diffeent sides of the pixel fall into diffeent classes. A statistically moe satisfactoy appoach would be to pefom the post-smoothing on the (posteio) classification pobabilities of the pixels. These pobabilities ae a poduct of Geisse's8 pedictive appoach to disciminant analysis and give the pobabilities of a pixel having been deived fom each of the k classes. A pixel is assigned to the class coesponding to the CONTEXTUAL CLASSIFICATION OF REMOTELY SENSED DATA 7

6 highest pobability, while the sizes of the pobabilities coesponding to the othe classes give an indication of how much confidence should be placed in the classification. Fomulae fo computing these pobabilities in the multivaiate nomal case ae given in Geisse, e Fatti, Hawkins and Haath.l as well as in Smoothing the classification pobabilities may be done in exactly the same way as the pe-smoothing of the eflectances, usually by some fom of weighted aveaging, with the weights possibly obtained by kiging. ode to obtain a moe appopiate scale fo applying the kiging, Hawkins 9 In has suggested that the logit tansfomation be applied to the classification pobabilities p.: p. Z. ~ en(~), i=l,...,k. -Pi Kiging would then be applied to smooth the tansfomed vaiables z. in exactly the same way as in pe'-smoothing. As descibed ealie, the k components z. could be kiged individually, co-kiging could be applied to the vecto of z. 's, o a common set of weights could be used on all components. In the end, the smoothed z. would be tansfomed back to a 0- scale and standadized to sum to unity. The pixel would then be assigned to the class coesponding to the highest 'smoothed' classification pobability. An attactive featue of the pedictive appoach is that pixels on the bounday of two classes may be identified by the fact that they would tend to have high classification pobabilities fo both classes. Intepeting these pobabilities as membeship functions in the context of fuzzy set theoy would give an indication of the amount of mixing of classes that occus within a bounday pixel. 8 A novel, but simple way of incopoating contextual infomation by vaying the pio pobabilities accoding to the local context has been poposed by Swi tze, Kowalik and Lyon. lo They popose a two- pass method in which the pixels ae initially classified by a non-contextual allocation ule, such as () o (2), in the fist pass. Fo evey pixel, the popotions of the pixels in its neighbouhood allocated to each of the classes in the fist pass ae computed. These ae then used to estimate the pio pobabilities of the vaious classes fo this pixel, coecting fo any biases esulting fom the allocation ule used in the fist pass by the fomula: whee P =: (PI"'" Pk) I is the vecto of estimated pio pobabilities, F is the (kxk) confusion matix whose (i,j)-th element gives the popotion of the taining sample pixels fom class ll. that have been assigned to ll. (pefeably obtained fom the ~ jackknife appoach), and f=(fl,...,f k )' is the vecto of popotions of the pixels in the neighbouhood classified to each of the k classes. These estimated pio pobabilities ae then used to classify the pixel in the second pass. Note that if eithe the linea o quadatic allocation ules () o (2) ae used, then the only ecalculations equied in the second pass ae the en p.' s, since the emainde of the allocation fomulae ae unchanged between the two passes. The size of the neighbouhood used fo estimating the local pio pobabilities will detemine how apidly these pio pobabili ties change fom pixel to pixel, and hence how smooth the final classification map will be. Switze, GEOSTATISTICS: THEORY

7 Kowalik and Lyon 0 apply this appoach to a small LANDSAT scene with thee classes (geological ock types), using squae neighbouhoods vaying in size fom 3x3 to 9x9 pixels. In thei application the lage neighbouhoods povide the best classification maps, but they make the point that the best neighbouhood size will be a function of the vaiability of the tue gound classes of the scene and the noise chaacteistics of the spectal data. Contextual classification based on spatial models As mentioned in the Intoduction, thee types of spatial models ae discussed, nunely those based on the eflectances alone, those based only on the gound tuth pocess and those incopoating models of both. Modelling the eflectances alone The simplest way to classify a pixel on the basis of a joint model of the eflectances of all the pixels in its neighbouhood is to adjoin the vectos of eflectances fom these pixels and then to classify the pixel on the basis of this enlaged vecto of eflectances. Fo example, fo LANDSAT data, with fou eflectances pe pixel, and using the five-pixel neighbouhood in Figue, thi s appoach would entail classifying the cental pixel on the basis of the 20--dimensional joint vecto of eflectances fom the pixels in this neighbouhood. The immediately obvious poblem with this appoach is the fact that computation time apidly becomes excessive as the numbe of neighbouing pixels and hence the dimension of the data vecto incease. Also, since the numbe of paametes in the multivaiate nomal model goes up with the squae of the dimension, statistical estimation consideations pedicate a coesponding incease in the equied taining sample sizes. Featue selection methods could, howeve, be used to educe the dimension of the poblem, as could canonical vaiate analysis in which only the majo components ae used fo classification. Switze '2 poposes that instead of adjoining all the eflectances fom the neighbouing pixels with those of the cental pixel, only the vecto of aveage eflectances fom the fou neighbouing pixels be adjoined to them. This esults in a moe manageable eight-dimensional vecto on which the classical allocation ules can be applied. Switze also shows that a futhe eduction in size is possible if the within-class covaiance matix can be factoized such that the covaiance between two eflectance bands in neighbouing pixels is a constant «) facto of the covaiance between these two eflectance bands in the same pixel. This constant attenuation facto depends only on the distance between the two pixes. Unde this model it tuns out that the optimal (Bayesian) classification ule uses the linea disciminant function () applied to a linea combination of the cental pixel vecto and the mean of those of the fou neighbouing pixels using the weighting scheme {-4p,p,p,p,p}, whee p is a function of the constant attenuation facto. Madia 3 genealizes Switze's model to any neighbouhood configuation, and deives geneal fomulae fo combining the eflectances fom the pixels in the neighbouhood fo optimal classification of the cental pixel via the linea disciminant function. Specifically, denoting by ~(~) the vecto of eflectances of the pixel located at point ~, and assuming the usual homoscedastic nomal model with diffeent mean vectos.l. and common covaiance matix L: fo the vaious CONTEXTUAL CLASSIFICA non OF REMOTELY SENSED DATA 9

8 classes, he uses the following spatial model fo the covaiance between the eflectances ~(~) and ~(~) of two pixels located at points ~ and ~: whee [4] p(.) is the isotopic coelation function, with p(o) =. (Fo the onedimensional case this is the usual assumption in kiging.) Given a paticula neighbouhood of R pixels, the covaiance matix of the adjoined vecto of thei eflectance vectos is then: I.* = P L [5] whee P is the (RxR) spatial coelation matix, which can be deived fom [4] once the neighbouhood is defined, and denotes the Konecke poduct. Assuming local spatial continuity of the neighbouhood, so that with pobability close to all the pixels in the neighbouhood come fom the same class as the cental pixel, Madia B then shows that the linea disciminant function [], fo contextual classification of the cental pixel, is based on the weighted sum of eflectances neighbouhood: z * R!. "I.z(x ) =l - whee the vecto of weights X = ("I '''I 0 ''''R) = P-ll. '- '-' in the [6] He also shows that a measue of the efficiency of this contextual classifie, elative to the non-contextual classifie based only on the eflectance vecto of the cental pixel. is: whee v 2 pixels 20 in has R L "I =l [7] maximum value R when the R the neighbouhood ae all independent, and a minimum value of when they ae all pefectly coelated, so that no exta infomation is contibuted by the neighbouing pixels. Clealy. the lage the neighbouhood, the geate is the value of the efficiency, but the less likely is the assumption of local spatial continuity to be tue ove the whole neighbouhood. Madia illustates these esults on a numbe of specific neighbouhood configuations, and then discusses thei genealization to the heteoscedastic case when the quadatic allocation ule [2) is appopiate. He also discusses the estimation of the paametes in this model fom taining samples of contiguous pixels fom each of the diffeent classes. Campbell and Kiivei 4 follow a diffeent appoach towads incopoating spatial dependency of eflectance data into classification pocedues by consideing a geneal spatial model fo canonical vaiate analysis. Thei spatial covaiance model fo the eflectances in neighbouing pixels is simila to that of Madia, 3 but wheeas his spatial coelation matix P in [5] deives diectly fom the isotopic coelation function' p(.) in [4], they assume that it esults fom eithe a conditional o simultaneous Gaussian autoegessive spatial model (Ripley S). deive an explicit This allows them to - expession fo P in tems of the coeffi ci.ents of eithe of these models. Using this model they show that the calculation of the goup means, within- and between-goups sums of squaes and cosspoducts matices (equied fo canonical vaiate analysis) involves neighboucoected values, and that the effective numbe of obsevations fom a neighhouhood enteing the calculations is equal Madia's efficiency measue v 2 =,p- l l. to GEOSTATISTICS: THEORY

9 Vu and FU 6 model the joint density function of the eflectances in a neighbouhood as a two-dimensional stationay Gaussian Makov pocess, oiginally poposed by Moan. l? This implies that the eflectances fom all the pixels in the neighbouhood ae jointly nomally distibuted, with a covaiance matix of a paticula fom detemined by the model. To make the pocess stationay, the spectal data in each pixel ae standadized to zeo mean and identity covaiance matix, and fo this, the classes of the neighbouing pixels ae equied. The authos popose a ecusive scheme in which altenate pixels in the lattice ae pe-classified (fo example, by a noncontextual classifie) and the emainde ae then classified using the Bayesian ule based on this spatial model of the standadized data. The latte cl assi ficat ions can then be used to update the allocations of the pe-classified pixels via the spatial model, and the pocess can be iteated to stability. Vu and Fu 6 epot encouaging esults in applying this contextual classifie to LANIlSAT data. in which two to thee iteations of the ecusive scheme wee equied fo convegence. Modelling the spatial vaiation of the gound classes The appoaches consideed unde this heading ae those that incopoate models of the spatial vaiation of the gound classes into the Bayesian classification fomulation, togethe with class-conditionally independent models of the spectal data. Besag poposes the following model fo the joint pobability distibution of the class membeships of the pixels in a scene. Denoting the class label of the -th pixel in the scene by x, whee x = i indicates that this pixel falls in gound class i. his model fo ~ = (x l,x 2 '... x n ), the class labels of all n pixels in the scene. is: k k P(x) ex exp( 2: a.n.-2: 2: f.. n.. ) i=l i<j [8] whee n. denotes the numbe of pixels in the scene which fall in class i, n.. denotes the numbe of neighbouing pais of pixels in the scene, counted once only, which fall in classes i and j. espectively, f.. ae paametes. and a. and The paametes f.. in this model eflect the amount neighbouing pixels; of spatial dependency between if the f.. ae all zeo, then the numbe of pixels in each class follow a multinomial distibution. with pobability exp(a.n.) that any pixel falls in gound class i. iespective of the class of its neighbous. Equal f ij, s implies symmetic dependencies between pais of neighbouing pixels fom any two classes. BesaglO assumes that, conditional on the class membeship vecto ~, the spectal data z, =l,...,n, fo each pixel ae in - dependently nomally distibuted with the mean vecto detemined by the class of the pixe and common covaiance matix. The posteio pobability fo ~, given the spectal data Z = (zl.z.2...,z ) n obtained via Bayes' Theoem: whee n - = exp{~ i: (z -~ )'2: (z -~ )+ =l ~ x - x k 2: a.n.-i: i: f.. n.. }. I. <. = is then [9] ~x is the mean of the class coesponding to the label x, and i: is the common within-class covaiance matix. Thus. the optimal assignments fo the scene CONTEXTUAL CLASSIFICATION OF REMOTELY SENSED DATA 2

10 ae obtained by maximizing PC! IZ), o equivalently, by =l,...,n, that minimize: finding the labels [0] whee U ("e) denotes the numbe of x neighbous of pixel (whose class is x ) that fall into class e. Besag e poposes the following heuistic fo minimizing [0]: fist apply the usual (non-spatial) linea disciminant analysis to the eflectances z, and then pefom a - tem-by-tem minimization of [0] using the neighbouing x-values aleady detemined. The last pocess may be epeated seveal times until a stable solution is obtained. When applied to a small simulated scene with thee classes, this technique shows a damatic impovement in pefomance ove linea disciminant analysis, and it would be inteesting to assess its pefomance on eal data. Owen 9 develops a neighbouhood-based classifie using a model developed by Switze in which the scene is potayed as a plane patitioned by staight lines into a set of convex egions. These lines ae geneated by a andom pocess called a Poisson field. Each egion thus fomed is assigned at andom to a class accoding to its pio pobability. It is thus possible fo adjacent egions to be assigned to the same class. The class of a pixel is taken to be the class assigned to its cente point. Using this model fo the gound classes, and the five-pixel neighbouhood given in Figue, and assuming that classes change on a scale that is lage compaed to the size of a pixel, Owen!! agues that each 22 neighbouhood will fall into one of thee types of pattens, namely: an X-patten in which all five pixels ae of the same class; an L-patten in which thee adjacent pixels (say, 2 and 3 in Figue ) fall into one egion and the emainde (4 and 5) fall into anothe; and a T-patten in which fou adjacent pixels (say,2,3 and 4 in Figue ) fall into one egion and the emaining one (5) falls into anothe. Futhemoe, each neighbouhood has the same pobability f (which has to be estimated) of intesecting a bounday line of a egion, foming an L- o T-pat ten, and a negligib le pobability of intesecting moe than one line. All fou otations (though 90 ) of the L-patten ae equally likely to occu, as ae all fou otations of the T-patten. Owen 9 also shows that the conditional pobability of an L-patten, given that a single bounday intesects a neighbouhood, is a constant a ~ 0.4 (and hence the conditional pobability of at-patten js I-a). Fom the above esults, and using the pio pobabilities of all the classes, the conditional pobability of any configuation of classes in a neighbouhood, given the tue class of the cental pixel, can be calculated. Fo example, using the numbeing scheme of Figue and let ting ~ = (xl"'" x 5 ) denote the vecto of class membeships of the pixels in the neighbouhood, it can be shown that: P[~ P[~ P[~ = (i,i,i,j,j)lxl=i] = p.afi4 (i,i,i,i,j) Ixl=i] = (i,i,i,i,i)lxl=i] = P. (l-a)f/4 I-f+fP i [] whee p. denotes the pio pobability of class j. As usual, the spectal data associated with each pixel ae assumed to be nomally distibuted, and, conditional on the class GEOSTATISTICS: THEORY

11 membeships, the data fo the pixels in a neighbouhood independent. ae consideed to be (Accoding to Owen 9, modelling the conditional coelation stuctue between pixels inceased the computation tenfold but did not impove the accuacy of the classifie.) In contast to Besag,8 the within-class covaiance matices ae allowed to esticts his be covaiance matices. Thus, diffeent, attention to but Owen!! popotional the Bayes classification ule fo this model is to assign the pixel at the cente of the neighbouhood to the class i that maximizes P(xl=iIZ) «Pif(?lxl=i) 5 = p. L [ H fez Ix )]P(Xlxl=i) ~ '" x =l [2] whee the sum is taken ove all assignments X of the pixels in the neighbouhood consistent with Xl=i. Owen 9 discusses the estimation of the paametes in the model fom taining samples, and demonstates the supeioity of this classifie ove the coesponding non--contextual one by applying it to a small LANDSAT scene. It is inteesting to note that the geneal fom of the contextual classification ule [2] is discussed by Swain, Vademan and Tilton,20 who efe to the joint pobability function P(.'S) of the classes in the neighbouhood as the 'context function'. a subsequent pape (Tilton, Vademan and Swain 2 ) these authos popose vaious nonpaametic function. In estimatos of this context Hill, Hinkley, Kostal and Mois 22 apply paametic empiical Bayes theoy, togethe with Makov models fo the distibution of the gound classes, classification and estimation of pio to the the classification pobabilities of the pixels in a emotely sensed scene. Denoting the pio pobability distibution of the gound tuth pocess by P (x) and the density of a'" the spectal data, given the gound tuth, by f(~i~), the maginal density of the spectal data is: ha(~) = f f(~i~)pa(~)~' [3] x '" Hill et al. 22 conside Makov models, including spatial autoegessive models, fo the pio distibution P (x) of the gound a ~ classes, and use empiical Bayes theoy to estimate the paametes a - fom samples. taining (Note that since the maginal density h (z) of the taining sample data a- does not depend on the actual classes ~, ~ may be estimated without gound tuth data.) These estimates ~ may then be used to obtain A the classification pobabilities P(~I~,~) via the usual Bayes fomula. The theoy is complex, and theefoe the authos only apply it to athe simple models. Futhe development would be equied befoe this appoach could yield pactical algoithms. Besag 23 contextual classification genealizes his ealie appoach by modelling the distibution of the gound classes by means of a locally dependent Makov andom field. The geneal fom of this model is, fo evey pixel scene: whee X I in the P(x IX -s I» = p (x - IXa ) V (4] ~s denotes the gound classification of the whole scene, excluding pixel, and denotes the coesponding classifications of the pixels in the CONTEXTUAL CLASSIFICATION OF REMOTELY SENSED DATA 23

12 neighbouhood of pixel. A simple fom of this model is: k P(x IX" )=exp{pu (x )}/ L exp{pu Ce)} (5]... v 'e=l whee u (e) denotes the numbes of neighbous of pixel fom class e and p is a fixed paamete. A slightly moe complex fom is (8] poposed by Besag,8 which simplifies to: when (=p). P(x Ix" ) <X exp{a +pu (x)} (6] '" x the coefficients P ke ae all equal A 'paiwise inteaction' Makov field has the model: n P(X) <X exp{ L G. (x. )+L! G.. (x.,x.)}... i=l i<j l andom (7] of which the above models ae clealy all special cases. discusses two pobabilistic appoaches towads classifying a scene. The fist appoach is that which finds the maximum a posteioi (m.a.p.) classi- A A fication! = (Xl'".,x n ) fo the whole scene that maximizes: P(!IZ) <X e(zi!)p(!) n whee e(zi!) = fcz Ix), =l... by class-conditional independence. (l8] The computational poblem associated with maximizing [8] ove all X clealy is enomous, but Geman and Geman 24 tackle it via an ingenious demanding) annealing. method (but involving computationally simulated The second appoach maximizes the maginal classification pobability of each pixel in the scene: (9] sepaately fo each pixel. Unfotunately, apat fom special models such as [7] in the next section, p(x,x~ )... v is geneally unavailable in closed fom. To get aound the computational (and othe) pob lems associated with maximiz ing [8] o [9], and to avoid the undesiable lage- scale popeties of Makov andom fields (fo example, ealizations often consist of a single classification fo a whole scene), Besag23 poposes the following iteative heuistic, which he calls estimation by (CM) : Iteated Conditional Modes Given an initial classification X of the... tue scene, pixel that maximizes: A find the classification fo PCx IZ,X I ) <X fez Ix )P(x Ix,,) (20]... s '" by class-conditional independence eflectances. A of the This is applied to all pixels in the scene to give classification... XCI) fo the an scene. updated The updated classification is then used in the next update of the scene, and this pocess is epeated until the classifications convege. The initial classification ~(o)may be obtained using the usual non-contextual maximum likelihood classifie, () o (2), and Besag23 epots that the pocedue conveges apidly to a (possibly local) maximum of PC! IZ) with few, if any, changes occuing afte about the sixth iteation. Fo example, applying this appoach to the simple model [5] fo the gound classes, and assuming Gaussian eflectances, with mean ~(x) depending on the class x of the pixel, and common covaiance matix L, the CM equies the successive minimization of %(z -u(x»'l-l(z -u(x»-pu (x) (2]... ~... ~ 24 GEOSTA TISTICS: THEORY

13 with espect to x, fo each pixel in tun, whee u (x ) denotes the cuent numbe of neighbous of pixel falling in class x. Besag 2B descibes how paamete estimation may be incopoated iteatively into the CM scheme, and he epots damatic impovement of the CM ove non-contextual classifies in simulated.scenes using the simple ule [2] fo the one-dimensional case, both with f3 specified befoehand and with it estimated duing the iteations. Incopoating spatial models of both the gound tuth and spectal data As in the pevious section, the classification algoi tluns discussed hee include models of the spatial vaiation of the gound classes, but in addition, models ae specified fo the joint distibution of the spectal data in a neighbouhood, conditionally on the classes of the pixels falling within this neighbouhood. Theefoe, in the geneal Bayesian fomula fo the posteio pobability fo ~ = (xl' x 2,, x R ), the class membeship vecto of the R pixels in a neighbouhood (R=5 in the neighbouhood of Figue ), given the coesponding spectal data Z = (zl,z2'''''z ): "V......n P(~IZ) a f(zi~)p(~), [22] spatial models ae specified fo both the joint pobability function p(~) of the classes in the neighbouhood and fo the joint likelihood f(zi~) data, given the class membeships. of the spectal To classify the cental pixel, labelled I, on the basis of [22], that value of Xl is chosen which maximizes its maginal posteio pobability: P(xlIZ) all f(zlx l,x 2,...,x R ) x x 2 "",x R [23] Kittle and Paiman 24 use the simple Makov model fo the gound classes in a neighbouhood: p(x Ix ),v ~s) = p(x Ix l ) s [24] implying that the knowledge of the class of the cental pixel completely defines the pobability distibution of the class of the j-th pixel in the neighbouhood. Thus the joint pobability of the class membeship vecto can be witten unde this model as: whee p(x l ) class Xl' R p(~) = p(x l ) H p(xlxl) =l [25] is the pio pobability of Fo the conditional joint density function of the spectal data, f{zi~), they use the two-dimensional stationay Gaussian Makov Pocess poposed by Yu and FU. 5 These authos apply the contextual classification algoitlun [23] based on these two models to satellite imagey of clouds, compaing its pefomance with that of the usual non-contextual classifie, as well as with that of a which p(~) fom taining data. contextual classifie in is estimated non-paametically Although vey lage contiguous taining aeas ae equied fo the latte classifie, which computationally heavie to implement, find vey little diffeence in is they the classification maps poduced by the two contextual algoithms. Both, howeve, povided vey much bette maps than those poduced by the non- contextual classifie. Kiivei and Cambell U apply the conditional Gaussian autoegessive spatial models descibed ealie (Campbell and Kiivei ") fo the spectal data in neighbouing pixels, and a Makov model simila to Besag's8 model [8] fo the class membeship vecto of all pixels in the CONTEXTUAL CLASSIFICATION OF REMOTELY SENSED DATA 25

14 scene. Howeve, instead of classifying pixels individually accoding to [23], they use [22] to compute the joint posteio pobability distibution fo the entie class membeship vecto, p(~iz), and classify all the pixels in the scene by maximizing this pobability. The amount of computation equied to find the global maximum of p(~iz) fo the whole scene is enomous, and Kiivei and Campbel 2 & popose two algoithms, namely a cyclic ascent algoithm and the annealing algoithm, to cay out the seach. An inteesting expeimental finding using these models is that little, if any, impovement in classification accuacy is obtained fom using the conditional Gaussian autoegessive spatial model fo the spectal data, athe than assuming them to be class-conditionally independent. Pess 27 poposes a vey geneal, fully Bayesian, appoach to contextual classification, incopoating models of the spectal data and of the class membeship pobabilities of neighbouing pixels. The taining sample data ae incopoated into the Bayesian model to yield the posteio classification pobability fo the a pixel, given the spectal data of the R pixels in its neighbouhood and the taining sample data (which clealy also obey the contextual model) : P(xlIZ,D) [26] whee Z denotes the spectal data of all the pixels in the neighbouhood of pixel, and D denotes the taining sample data. Using Bayes' Theoem, this pobability can be witten as: P(xlIZ,D) «k k p(x ) L... L f(zid,x,x,...,x ) l l 2 R x = x = 2 R 26 x P(x 2,..,x R lx l ) [27] whee, as usual, k is the total numbe of classes in the scene. Pess 2 7 assumes that the data fom the j-th class come fom a Gaussian covaiance stationay pocess. Thus, if ~(~) denotes a data vecto obtained fom the pixel situated at point ~ on the gound, and supposing that this pixel falls in gound class i, then Pess 27 assumes that: z(s) - N(u.,L.). - - t:;;l [28] In this model the covaiance matix between the data fom any two pixels of the same class If i situated at points ~ I and ~2 on th~~ gound depends only on the vecto diffeence (both magnitude and diection) between ~l and ~2: cov(z(sl),z(s2) ISl,s2~If.) = L.(sl-s2) [29) ~ ~ ~ and the covaiance matix between the data fom two pixels, situated at points ~l ~2' and fom diffeent classes If. and If., depends only on the two classes and the vecto diffeence between ~l and ~2: cov(z(sl),z(s2) ISl~If.jS2~If.) Using this model, L.. (sl-s2) - - [30] the joint likelihood function of the entie taining sample D fom all k classes can be expessed in tems of the multivaiate nomal distibution. The joint distibution of D and the paametes u., L. (s) and L.. (s) then follows, assuming ~ - - vague pio distibutions fo the latte. The conditional distibution of the spectal data Z of the pixels in the neighbouhood of pixel, gi ven thei classes ~ = (xl"'" "R) and the paametes l.l., L. (s) and L.. (s), also follows fom this ~ - - model, and finally: f(z ID, X) «f f(z IX, u.,l. (s),l.. (s» - - ~l - I- GEOSTA TISTICS: THEORY

15 x f(d,u.,. (s),.. (s) )llldi:. (S)di:.. (S) ~ ~ ~ ~ ~ ~ assuming that Z and D ae independent. [3] Fo the 'configuation pobability' P(x 2,...,xRlx l ) of the classes of the R pixels in the neighbouhood, given the class of the cental pixel, Pess 27 assumes that the numbe of diffeent classes M pesent in the neighbouhood has a (tuncated) geometic distibution. (This ecognizes the fact that, in any neighbouhood, the most likely situation is fo thee to be a single class in the neighbouhood. the next likely two. etc.) He then adopts a geneal log-linea model fo the conditional configuation pobability: log P(x 2... x R lx l.m=m) R-l = L. L (x..... =o ~2~... ~l (x.,...,x. Ixl,m) whee the inne sum [32] is ove all subsets this model. which is clealy a genealization of Besag's 8 model [8]. the coefficients satisfy the usual analysis of vaiance-type identifying constaints. Substituting [3] and [32] into [27] then yields the posteio classification pobability of the cental pixel. This model is clealy vey geneal, but as it stands it is not pactical fo implementation in a contextual classifie. Howeve. it allows fo a lage numbe of special cases. some of which may tun out to be both ealistic and computationally pactical. Conclusions Gi ven the vast aay of appoaches towads contextual classification available. it is natual to ask which appoach would be most suitable fo application in emote-sensing. The answe inevitably depends on a numbe of factos, including the chaacteistics of the scene in question. the pupose fo which the classification is being pefomed. and the computing powe and time available. Fo example, if the aveage size of single-class fields is simila to the pixel size, o if the pupose is to detect fine featues such as oads, then the appoaches discussed in this pape ae not appopiate. In contast. they should wok well if the pupose is to classify a scene consisting of lage, homogeneous fields. so that the numbe of misclassified pixels. especially in the middle of a field. is minimized. The pe-smoothing techniques descibed fist ae clealy vey easy to apply. especially once the smoothing weights have been detemined. and being linea, the computing costs ae elatively modest. Similaly, the post-smoothing techniques. and in this context the appoach of Switze et al.0 holds paticula pomise, ae easy to apply and only equie the updating of one tem in the disciminant functions in the second iteation. The model-based techniques ae geneally moe difficult to apply and equie moe computation. Among the thee classes descibed, namely those based on models of, espect i vely. the eflectances, the gound classes and both, it is the authos' opinion. based on vey limited expeience. that the appoaches based on models of the gound classes ae possibly the most pomising. Dopping the assumption of class-conditional independence of the spectal data fom these appoaches geneally esults in manyfold inceases in computing time, without necessaily impoving the classifications significantly. CONTEXTUAL CLASSIFICATION OF REMOTELY SENSED DATA 27

16 Howeve, as this aea is cuently the subject of much eseach, the opinions expessed in this pape ae, at best, of povisional validity. Refeences. FATTI, L.P., HAWKINS, D.M. and RAATH, E.L. Disciminant analysis. In: Topics in Applied luitivaiate Analysis. Hawkins. D.M., Cambidge Univesity Pess, 982. pp ELPHINSTONE, C.D., LONERGAN, A. T., FATTI, L.P. and HAWKINS, D.M. An empiical investigation into the application of some statistical techniques to the classification of emotely sensed data. Special Repot, SWISK 40, National Reseach Institute fo Mathematical Sciences, CSIR, Petoia, FATTI, L.P. and HAWK INS, D.M. Vaiable selection in heteoscedastic disciminant analysis.. Ame. Statist. Ass., vol. 8, no. 394, 986. pp I,AWOKO, C.R.O. and McLACHLAN, G.. Some asymptotic esults on the effect of autocoelation on the eo ates of the sample linea disciminant function. Patten Recognition, vol. 6, no., 983. pp FOGLEIN,. and KITTLER,. The effect of pixel coelations on class sepaability. Patten Recognition Lettes, vol. I, 983. pp KETTIG, R.L. and LANDGREBE, D.A. Classification of multispectal image data by extaction and classification of homogeneous objects. IEEE Tans. 28 Geoscience Electonics, vol. GE-4, 976. pp KRIGE, D.M. Lognomal - de Wisian Geosta tisti cs fo Oe Evaluation. ohannesbug, 98. S.A. Inst. Min. Metal., 8. GEISSER, S. Posteio odds fo multivaiate nomal classifications.. ROY. statist. Soc., B, vol. 26, 964. pp HAWKINS, D.M. Discussion of pape by P. Switze. Bull. Int. Statist. Inst. vol. 50 (Bk.3), 983. pp SWITZER, P., KOWALIK, W.S. and LYON, R..P. A pio pobability method fo smoothing disciminant analysis classification maps. lath. Geol. vo. 4, no. 5, 982. pp SWITZER, P. Extensions of linea disciminant analysis fo statistical classification of emotely sensed satellite imagey. lath. Geol. vol. 2, no. 4, 980. pp SWITZER, P. Some spatial (:!tatistics fo the intepetation of satellite data (with discussion). Bull. Int. Statist. Inst. vol. 50 (Bk2), 983. pp MARDIA, K.V. Spatial discimination and classification maps. Commun. Statlst. - Theo. leth. vol. 3, no. 8, 984. pp CAMPBELL, N.A. and KIIVERI, H.T. Discimination with spatiallycoelated data. Technical Repot, Division of Mathematics and Statistics, CSIRO, Wembley 604, Westen Austalia, 986. GEOSTATISTICS: THEORY

17 5. RIPLEY, B.D. Spatial Statistics. New Yok, Wiley, YU, T.-S. and FU, K.-S. Recusive contextual classification using a spatial stochastic model. Patten Recognition, vol. 6, no. I, 983. pp MORAN, P.A.P. A Gaussian Makovian pocess on a lattice.. Appl. Pobab. vol. 0, 973. pp BESAG,. Discussion of pape by P. Switze. Bull. Int. Statist. Inst. vol. 50 (Bk3), 983. pp OWEN, A. A neighbouhood-based classifie fo Landsat data. Canadian. Statist. vol. 2, no. 3, 984. pp SWAIN, P.H., VARDEMAN, S.B. and TILTON,. C. Contextual classification of multispectal image data. Patten Recognition, vol. 3, no. 6, 98. pp TILTON,.C., VARDEMAN, S.B. and SWAIN, P.H. Estimation of context fo statistical classification of multispectal image data. IEEE Tans. Geoscience Remote Sensing, vol. GE-20, no. 4, 982. pp HILL,.R., HINKLEY, D. V., KOSTAL, H. and MORRIS, C.N. Spatial estimation fom emotely sensed data via empiical Bayes models. Technical Repot #0, Cente fo Statistical Sciences, Univesity of Texas at Austin, Austin,, Texas 7872, BESAG,.T. On the statistical analysis of dity pictues.. ROY. Statist. Soc. B, vol. 48, no. 3, 986. pp GEMAN, S. and GEMAN, D. Stochastic elaxation, Gibbs distibutions and the Bayesian estoation of images. IEEE Tans. Patten Anal. Machine Intell. vol. 6, 984. pp KITTLER,.T. and PAIRMAN, D. Contextual patten ecognition applied to cloud detection and identification. IEEE Tans. Geoscience Remote Sensing, vol. GE-23, no. 6, 985. pp KIIVERI, H.T. and CAMPBELL, N.A. Allocation of emotely sensed data using Makov models fo spectal vaiables and pixel labels. Technical Repot, Division of Mathematics and Statistics, CSlRO, Wembley 604, Westen Austalia, PRESS, S.. Spatial coelation in Bayesian classification of emotely sensed cop quality. Technical Repot, Depatment of Statistics, Univesity of Califonia, Riveside, 986. CONTEXTUAL CLASSIFICATION OF REMOTELY SENSED DATA 29