CONTEXT MODELS FOR CRF-BASED CLASSIFICATION OF MULTITEMPORAL REMOTE SENSING DATA

Transcription

1 ISPRS Annals of he Phoogrammery, Remoe Sensing and Spaial Informaion Sciences, Volume I-7, 2012 XXII ISPRS Congress, 25 Augus 01 Sepember 2012, Melbourne, Ausralia CONTEXT MODELS FOR CRF-BASED CLASSIFICATION OF MULTITEMPORAL REMOTE SENSING DATA T. Hoberg*, F. Roenseiner, C. Heipke IPI, Insiue of Phoogrammery and GeoInformaion, Leibniz Universiae Hannover, Germany (hoberg, roenseiner, Commission VII, WG VII/4 KEY WORDS: Conexual, Muliresoluion, Muliemporal, Land Cover, Classificaion, Condiional Random Fields ABSTRACT: The increasing availabiliy of muliemporal saellie remoe sensing daa offers new poenial for land cover analysis. By combining daa acquired a differen epochs i is possible boh o improve he classificaion accuracy and o analyse land cover changes a a high frequency. A simulaneous classificaion of images from differen epochs ha is also capable of deecing changes is achieved by a new classificaion echnique based on Condiional Random Fields (CRF). CRF provide a probabilisic classificaion framework including local spaial and emporal conex. Alhough conex is known o improve image analysis resuls, so far only lile research was carried ou on how o model i. Taking ino accoun conex is he main benefi of CRF in comparison o many oher classificaion mehods. Conex can be already considered by he choice of feaures and in he design of he ineracion poenials ha model he dependencies of ineracing sies in he CRF. In his paper, hese aspecs are more horoughly invesigaed. The impac of he applied feaures on he classificaion resul as well as differen models for he spaial ineracion poenials are evaluaed and compared o he purely label-based Markov Random Field model. 1. INTRODUCTION An increasing number of opical high resoluion (HR) remoe sensing saellie sysems have become available in he las decade. I should hus be possible o improve he classificaion accuracy and o analyse land cover changes more frequenly han his is currenly done based on a muliemporal analysis. However, he purchase of HR muliemporal daa for hese purposes is ofen no economically viable, especially for large areas. Daa having medium resoluion do no offer as much deail, bu cover a larger area and may ofen be preferable from an economical poin of view. Combining he advanages of boh daa ypes requires muliscale and muliemporal analysis. Up o now mos approaches for muliemporal land cover analysis do no make use of emporal dependencies, bu derive resuls by some kind of difference measure beween he monoemporal classificaion resuls of differen epochs (i.e., differen acquisiion imes) (Lu e al., 2004). If daa from all epochs are available, i would seem o be advanageous o use he original observaions, i.e. he image daa, raher han derived daa. This has for insance been done in (Feiosa e al., 2009), where a model of emporal dependencies based on Markov chains is applied. As in mos echniques for muliemporal classificaion, each pixel is classified individually wihou considering spaial conex, which leads o a sal-and-pepperlike appearance of he change deecion resuls. Bruzzone e al. (2004) ry o overcome his problem by using a cascade of hree muliemporal classifiers, one of hem considering he k-neares neighbours of each pixel. A saisical model of spaial conex in image classificaion is given by Markov Random Fields (MRF) (Geman & Geman, 1984), which have also been used for change deecion (Melgani & Serpico, 2003), (Moser e al., 2009). In (Melgani & Serpico, 2003), he MRF framework is exended by a emporal energy erm based on a ransiion probabiliy marix in order o improve he classificaion resuls for wo consecuive images. Moser e al. (2009) applied he MRF framework o deec changes in opical saellie images based on muliscale feaures, bu wihou deermining he changed objec classes. Using MRF, he ineracion beween neighbouring image sies (pixels or segmens) is resriced o he class labels, whereas he feaures exraced from differen sies are assumed o be condiionally independen. This resricion is overcome by Condiional Random Fields (CRF; Kumar & Heber, 2006). CRF provide a discriminaive framework ha can also model dependencies beween feaures from differen image sies and ineracions beween he labels and he feaures. In remoe sensing CRF have been used for monoemporal classificaion, e.g. of selemen areas in HR opical saellie images (Zhong & Wang, 2007) or crop ypes and oher land cover classes in Landsa daa (Roscher e al., 2010). Muliemporal classificaion based on CRF for improving he overall classificaion accuracy as well as deecing changes has firs been applied in (Hoberg e al., 2010). This mehod allows for emporal informaion passing using an exension of he CRF model. Muliscale analysis is moivaed by he fac ha he appearance of objecs in a scene is a funcion of he image resoluion and because i is capable of providing a more global view on image conen and image analysis algorihms (Kao e al., 1993), (Wilsky, 2002). The simples way of considering muliple scales in classificaion is o derive he feaures a muliple scales, e.g. (Kumar & Heber, 2006), which has been applied for change deecion in (Moser e al., 2009). There have also been approaches o combine a muliscale analysis wih CRF. In (Schnizspan e al., 2008), a muliscale CRF is buil on an * Corresponding auhor. 129

2 ISPRS Annals of he Phoogrammery, Remoe Sensing and Spaial Informaion Sciences, Volume I-7, 2012 XXII ISPRS Congress, 25 Augus 01 Sepember 2012, Melbourne, Ausralia image grid ha in addiion o he spaial neighbourhood relaions also considers neighbours in scale based on a regular pyramid srucure. Differen classes are represened a differen scale levels by a par-based objec model: a finer resoluions, he classes o be discerned correspond o objec pars, whereas a coarser resoluions, hey correspond o compound objecs. In (Yang e al., 2010) his mehod is exended o an irregular pyramid based on a muli-scale waershed segmenaion of he original image. A combinaion of muliemporal and muliscale analysis of remoe sensing daa using CRF is presened by Hoberg e al. (2011). A se of mulispecral images of differen resoluion is classified simulaneously in order o increase he accuracy and reliabiliy of he classificaion resuls and o deec land cover changes beween he individual epochs. This approach allows o model dependencies beween image regions a idenical posiions in he differen epochs ha may addiionally be characerized by differen scales and, hence, by differen (hough relaed) class srucures. Unforunaely in publicaions abou CRF here is only lile informaion abou feaure selecion and he influence of differen feaures on he classificaion resul. Moreover in mos cases only one model for he ineracion poenial is applied, wihou jusificaion of he choice of he paricular model. These issues are invesigaed in his paper. We compare differen conex models wih differen subses of feaures ha are exraced a differen scales. Firs, o find he bes subse of feaures depending on he maximum scale we apply a feaure selecion process. Nex he bes feaure subse is seleced for he associaion poenial. Based on he seleced associaion poenial, we invesigae hree differen conex models for he spaial ineracion poenial, again comparing differen feaure subses. Finally he resuls of hese invesigaions are applied in a muliemporal CRF-based classificaion approach. Tess are performed using wo se-ups, one of hem using images having idenical resoluion and one wih images of differen resoluion. The remainder of his paper is srucured as follows. In Secion 2, he principles of CRF and he exensions for he classificaion of muliemporal and muliscale daa are presened. Secion 3 focuses on he descripion of he feaures and on feaure selecion. In Secion 4, he es sie is described. A qualiaive analysis of he differen ways of modelling conex is given in Secion 5, followed by quaniaive resuls in Secion 6. Conclusions and an oulook are given in Secion MULTITEMPORAL AND MULTISCALE CRF In many classificaion algorihms he decision for a class a a cerain image sie is jus based on informaion derived a he regarded sie (i.e., a pixel, a square block of pixels in a regular grid, or a segmen). In fac, he class labels and also he daa of spaially and emporally neighbouring sies are ofen similar or show characerisic paerns, which can be modelled using CRF. In monoemporal classificaion, we wan o deermine he vecor of class labels x whose componens x i correspond o he classes of image sies i S and S being he se of all sies for given image daa y by maximizing he poserior probabiliy P(x y) (Kumar & Heber, 2006): 1 P x y i i ij i j Z exp A ( x, ) I x,x, y y i S i S j (1) Ni In (1), N i is he spaial neighbourhood of image sie i (hus, j is a spaial neighbour of i), and Z is a normalizaion consan called he pariion funcion. The associaion poenial A i links he class label x i of image sie i o he daa y, whereas he erm I ij, called ineracion poenial, models he dependencies beween he labels x i and x j of neighbouring sies i and j and he daa y. The model is very general in erms of he definiion of he funcional model for boh A i and I ij. In he muliemporal case, we have M co-regisered images. In addiion o he ineracions of spaial neighbours, he emporal neighbourhood is aken ino accoun. Each node is only linked o is direc emporal neighbours a is spaial posiion (Figure 1). The componens of he image daa vecor y are sie-wise daa vecors y i, wih i S and S being he se of sies of all images (i.e., i does no refer o a paricular spaial posiion, bu i refers o one spaial posiion in one of he images). The index indicaes he membership of image sie i o he relaed epoch T and T = {1, M}. The componens of x are he class labels of he image sies i, x i, also wih epoch index T. For each image sie we wan o deermine he class x i from a se of pre-defined classes. The class srucure and hus he number of classes are dependen on. In order o model he muual dependency of he class labels a an image sie a differen epochs, he model for P(x y) in (1) has o be exended: 1 Px y exp A( x i, y ) IS x i,x j, y Z is is jni is ke k lli k k k i l y y IT x,x,, As he differen funcional models for he poenial funcions A, IS, and IT k are shif-invarian, he subscrips of he poenial funcions in (1) have been omied in (2). In (2), A is he associaion poenial, IS he spaial ineracion poenial ha corresponds o he ineracion poenial I ij in (1), and IT k he emporal ineracion poenial. In IT k, y and y k are he images observed a epochs and k, respecively. E is he se of epochs in he emporal neighbourhood of he epoch o which image sie i belongs, hus k is he ime index of an epoch in emporal neighbourhood of. The se of image sies a epoch k E ha are emporal neighbours of he image sie i is denoed by L i k, hus l L i k is an image sie ha is a emporal neighbour of i in epoch k. The emporal ineracion poenial models he dependency beween he class labels and he observed daa a consecuive epochs. The image sies are chosen o be individual pixels and hus are arranged in a regular grid for each image. Figure 1 shows he spaial and emporal neighbourhood for images having idenical or differen resoluions. Figure 1. Muliemporal graph srucure. Lef: images having he same resoluion. Righ: images having differen resoluions. Red nodes: processed primiives; orange / green nodes: spaial / emporal neighbours. (2) 130

3 ISPRS Annals of he Phoogrammery, Remoe Sensing and Spaial Informaion Sciences, Volume I-7, 2012 XXII ISPRS Congress, 25 Augus 01 Sepember 2012, Melbourne, Ausralia 2.1 Associaion poenial The associaion poenial A(x i, y ) in (2) is relaed o he probabiliy of label x i aking a value c given he image y a epoch by A(x i, y ) = log{p[x i =c f i (y )]}. The image daa are represened by sie-wise feaure vecors f i (y ) ha may depend on he enire image a epoch, e.g. by using feaures a differen scales (Kumar & Heber, 2006). We use a mulivariae Gaussian model for P[x i =c f i (y )] (Bishop, 2006): P x c 1 n 2 de Σ fc i fi yi e T 1 1 fi y E fc Σ fc fi y E fc 2 In (3), E fc and fc are he mean and co-variance marix of he feaures of class c, respecively. I is imporan o noe ha boh he definiion of he feaures and he dimension of he feaure vecors f i (y ) may vary from image o image, because he definiion of appropriae and expressive feaures depends on he image resoluion and also on he specral informaion conained in he images (see also Secion 3). 2.2 Spaial ineracion poenial The spaial ineracion poenial IS(x i, x j, y ) in (2) is a measure for he influence of he daa y and he neighbouring labels x j on he class x i of image sie i a epoch. In his poenial, he daa are represened by sie-wise vecors of ineracion feaures ij (y ). In his work we compare hree differen models for he spaial ineracion poenial. The firs model only depends on he labels. I is commonly used wih MRF and has a smoohing effec on he labels: if x i = x j IS1 x i,x j, y = (4) 0 if x i x j The second model is based on (Shoon e al., 2007): 2 µ ij y exp IS2 x i,x j, y = R 0 if x i = x j if x i x j The hird model is used by Hoberg e al. (2010): 2 µ ij y exp if x i = x j R IS3 x i,x j, y = 2 µ ij y 1 exp if x i x j R In (5) and (6), µ ij (y ) denoes he Euclidean norm of µ ij (y ) and is a weighing facor for he influence of he spaial ineracion poenial in he classificaion process. We use he componen-wise differences of he feaure vecors h i (y ) for he ineracion feaures ij (y ), i.e. ij (y ) = [µ ij1, µ ijr ] T, where R is he dimension of he vecors h i (y ) ha may vary wih. Noe ha he feaure vecor h i (y ) used for he ineracion poenial migh differ from he feaure vecor f i (y ) used for he associaion poenial (Kumar & Heber, 2006). Denoing he m h (3) (5) (6) componen of h i (y ) by h im (y ), he m h componen of ij (y ) is µ ijm = h im (y ) h jm (y ). Division by he number of feaures R in (5) and (6) guaranees an idenical influence of he spaial ineracion poenials for all images. In IS 2 a poenial of zero is assigned in he case of wo sies have differen labels. Differing labels a neighbouring sies are penalized unless he feaures of he sies are also very differen. IS 3 penalizes boh local changes of he class labels if he daa are similar and also idenical class labels if he feaures are differen. 2.3 Temporal ineracion poenial The emporal ineracion poenial IT k (x i, x l k, y, y k ) models he dependencies beween he daa y and he labels x i and x l k of sie i a epoch and sie l of epoch k. In principle, IT k could be modelled similarly o IS by penalizing emporal change of labels unless i is indicaed by differences in he daa. However, a more sophisicaed funcional model would be required o compensae for amospheric effecs and varying illuminaion condiions, differen resoluions, and seasonal effecs of he vegeaion. We use a simple model for he emporal ineracion poenial ha neglecs he dependency of IT k of he daa: ssk k TM x i,x k k k k k l IT x i,x l, y, y =IT x i,x l = (7) k Q In (7), is a weigh facor. TM s()s(k) is a emporal ransiion marix similar o he ransiion probabiliy marix in (Bruzzone e al., 2004). The elemens of TM s()s(k) (x i, x l k ) can be seen as condiional probabiliies P(x i =c x l k =c k ) of an image sie i belonging o class c a epoch if he image sie l ha occupies he same spaial posiion as i in epoch k belongs o class c k in ha epoch. Q i k is he number of elemens in L i k and acs as a normalizaion facor ensuring an idenical influence of he sum of all emporal ineracion poenials in any epoch, no maer how many emporal neighbours exis. The scales s() and s(k) of he daa a epochs and k may differ; here is one marix TM s()s(k) for each combinaion of scales available in he daa. For furher informaion we refer o (Hoberg e al., 2011). 2.4 Training and Inference Exac raining and inference is compuaionally inracable for CRF (Kumar & Heber, 2006). In our applicaion, we only rain he parameers of he associaion poenials, i.e. he mean E fc and he co-variance marix fc of he feaures of each class c. They are deermined from he feaures f i (y ) in raining sies individually for each epoch and each class c. The oher model parameers, i.e. he weighing facors and of he spaial and emporal ineracion poenials and he elemens of he ransiion marices TM s()s(k), were found empirically. For inference, we use Loopy Belief Propagaion (LBP) (Nocedal & Wrigh, 2006), a sandard echnique for probabiliy propagaion in graphs wih cycles ha has shown o give good resuls in he comparison repored in (Vishwanahan e al., 2006). 3. FEATURES AND FEATURE SELECTION In order o apply he CRF framework, he sie-wise feaure vecors f i (y ) for he associaion and h i (y ) for he spaial ineracion poenials for each epoch mus be defined. Boh mus consis of appropriae feaures ha can help o discriminae he individual classes. In our applicaion, we used several groups of feaures, namely colour-based, exural and i 131

4 ISPRS Annals of he Phoogrammery, Remoe Sensing and Spaial Informaion Sciences, Volume I-7, 2012 XXII ISPRS Congress, 25 Augus 01 Sepember 2012, Melbourne, Ausralia srucural feaures. All feaures are compued a five differen scales d, wih d indicaing he scale. Whereas in 1 only individual pixels are aken ino accoun, in 2 o 5 he feaures are exraced in a square window of size 3, 5, 9, and 13 pixels, respecively, cenred a he cenre of image sie i. Hence we do no only consider informaion derived a sie i for he sie-wise feaure vecors f i (y) and h i (y), bu we also model dependencies beween he image informaion of neighbouring sies. The colour-based feaures are direcly derived from he pixel values of he specral channels, four in our case. We used he mean and variance of he red (E d r, V d r), green (E d g, V d g), blue (E d b, V d b), and near infrared (E d nir, V d nir) channel, he variance of he hue (V d hue), and he mean of he difference of red and green (E d r-g), near infrared and red (E d nir-r), and near infrared and green (E d nir-g). Moreover he mean and variance of he normalized difference vegeaion index (E d ndvi, V d ndvi) and he relaional vegeaion index (E d rvi, V d rvi) were compued. The exural feaures consis of conras (con d ), correlaion (cor d ), energy (ene d ), homogeneiy (hom d ), and enropy (en d ) as defined by Haralick e al. (1973). They are all derived from he gray-level co-occurence marix ha represens he disribuion of co-occurring values a a given offse (1 in our case). The srucural feaures are derived from a weighed hisogram of oriened gradiens (HOG) (Dalal & Triggs, 2005). Each hisogram has 30 bins, so ha each bin corresponds o an orienaion inerval of 6 widh. Each bin conains he sum of he magniudes of all gradiens having an orienaion ha is wihin he inerval corresponding o he bin. Summing over he magniudes and no jus couning he numbers of gradiens falling ino each bin is done o ake ino accoun he impac of srong magniudes. From he hisogram we derive five feaures: The mean of all gradien magniudes (E d grad) he variance of he hisogram enries (V d grad), he number of bins wih magniudes above he mean magniude (num d ), he value of he maximum hisogram enry (mag d ) and he angle beween he firs wo maxima (ang d ). All he feaures are normalised so ha he values are in he inerval [0, 1]. We define he feaure vecors corresponding o a maximum scale λ max o consis no only of he feaures exraced a λ max, bu also of all feaures of lower scales. Hence, for insance he feaure vecor corresponding o max = 5 conains 113 elemens, nine of hem exraced a 1 and 26 feaures exraced a each addiional scale. Using he large number of feaures jus described makes he classificaion quie ime consuming for wo reasons: All he feaures have o be exraced and all have o be considered for deermining he poenials. As many of he feaures are highly correlaed or may only marginally suppor he classificaion, we apply a feaure selecion procedure o find ou which feaures are relevan for our aims and o reduce he number of feaures accordingly. For ha purpose we use he correlaion-based feaure selecion approach by Hall (1999). Firs, he single feaure which bes classifies he daa se is deermined. Afer ha, oher feaures are chosen according o crieria ha ensure he selecion of a subse ha conains feaures ha are highly correlaed wih he classes, ye uncorrelaed wih each oher. 4. TEST SITE AND DATA Our es area is siuaed near Herne, Germany, and covers an area of 8.6 x 5.9 km². We used mulispecral Ikonos daa wih 4 m ground sampling disance (GSD) acquired in 2005 and 2007, and Landsa daa of 30 m GSD acquired in All images were recorded in summer. The area was spli ino 54 secions, which were processed separaely. Seven secions served as raining daa, he res as es sies. Ground ruh was obained by manually labelling he images a pixel level. The classes o be disinguished wih Ikonos imagery are residenial areas (res), indusrial areas (ind), foress (for), and cropland (crp). Because here is no clear disincion of he classes res and ind in he medium resoluion Landsa imagery hey are fused o a new class buil-up areas (bui) in ha resoluion. 5. FEATURE AND MODEL SELECTION In his secion he impac of using feaures a differen scales and of differen conex models on he classificaion resul is invesigaed. We ry o find a suiable subse of feaures for each maximum scale max and hen analyse he resuls o find he bes maximum scale and, hus, he opimal feaure subse for he associaion poenial. Then we compare differen conex models for he spaial ineracion poenial, using he opimal feaure subses for each maximum scale max. To invesigae how many feaures should be used for our CRFclassificaion we applied a sandard maximum likelihood (ML) classificaion in subses wih feaures derived a an increasing number of scales up o a maximum scale λ max, ordering he feaures according o he resuls of he feaure selecion process described above. The ML-classificaion was chosen because is model is also used for he associaion poenial. For all values of λ max we found ha using he six bes feaures was sufficien. Addiional feaures did no furher increase he classificaion accuracy. Hence each of he feaure vecors f i (y ) and h i (y ) was reduced o jus six feaures depending on max : max = 1 : E 1 r, E 1 g, E 1 b, E 1 nir, E 1 ndvi, E 1 rvi max = 2 : E 2 nir, V 2 nir, V 2 hue, E 2 nir-r, V 2 ndvi, E 2 grad max = 3 : E 3 nir, V 3 nir, V 3 hue, E 3 nir-r, E 3 grad, en 3 max = 4 : E 4 g, E 4 nir, V 4 hue, E 4 grad, en 4, V 3 hue max = 5 : E 5 nir, V 5 hue, E 5 grad, hom 5, E 4 g, en 4 I is obvious ha in each subse he feaures exraced in he larges scale are dominan. The impac of using feaures exraced a differen maximum scales max on he associaion poenial was evaluaed by comparing he resuls of ML classificaion obained for he seleced subses for each value of max. Figure 2 shows exemplary resuls for wo of he secions using Ikonos imagery; he highes overall accuracy is achieved wih max =4. Neverheless, by visual inerpreaion mos users would consider he resul of max =3 o be bes, because many finer srucures (for insance he road in he upper example of figure 2) are much beer preserved. Because informaion ha is los a his sage canno be re-inroduced in furher processing seps, we decided o apply he feaure vecor f i (y ) for max =3 for he associaion poenial of our furher compuaions. The hree conex models for he spaial ineracion poenial (Secion 2.2) are evaluaed by a monoemporal classificaion on Ikonos imagery. For he wo daa-dependen models we used he feaure vecors seleced for he associaion poenials in he maximum scales max = 2, 3 and 4 (see above) for h i (y ). In general, he purely label-based model IS 1 resuls in srong smoohing, while he daa-dependen models preserve finer srucures beer, e.g. he road passing hrough cropland in Figure 3. However, his does no necessarily lead o a higher overall accuracy. In all scales IS 2 performs slighly beer han IS 3, which favours addiional class ransiions if he feaures a neighbouring sies are differen. The maximum scale of he 132

5 ISPRS Annals of he Phoogrammery, Remoe Sensing and Spaial Informaion Sciences, Volume I-7, 2012 XXII ISPRS Congress, 25 Augus 01 Sepember 2012, Melbourne, Ausralia reference λ max = 1 (80.4%) λ max = 2 (84.1%) feaures in h i (y ) only has a minor effec on he resuls. Using max = 2, some sal-and-pepper effecs remain, whereas he oher scales lead o sronger smoohing. Overall, using IS 2 wih h i (y ) from max = 3 delivers he bes rade-off of overall accuracy and preservaion of deails assessed by visual impression, which is why his combinaion is applied in our experimens (cf. Secion 6). Hence, in hese experimens, f i (y ) and h i (y ) are idenical. 6. QUANTITATIVE EVALUTION λ max = 3 (85.0%) λ max = 4 (86.9%) λ max = 5 (86.9%) reference λ max = 1 (67.0%) λ max = 2 (72.6%) λ max = 3 (77.4%) λ max = 4 (81.8%) λ max = 5 (81.4%) Figure 2. Overall accuracy of ML-classificaion in dependence on applied maximum scale λ max for feaure exracion. Red: res; blue: ind; green: for; yellow: crp. reference IS 1 (90.7%) IS 2, λ max = 2 (88.7%) IS 3, λ max = 2 (88.2%) IS 2, λ max = 3 (88.6%) IS 3, λ max = 3 (88.0%) IS 2, λ max = 4 (88.4%) IS 3, λ max = 4 (87.3%) Figure 3. Overall accuracy of CRF-classificaion wih differen conex models and varying scale λ max for he spaial ineracion poenial. We esed our muliemporal approach for wo daa se-ups: Seup I has only one scale and consiss of wo Ikonos images. In he muliscale se-up II we combined one Ikonos and one Landsa scene. For he Ikonos scenes we used he feaures as defined in Secion 5, for he Landsa scene hey were exraced only in he original resoluion. The emporal ransiion marix TM beween Ikonos and Landsa used in our experimens is shown in Table 1. A similar marix was defined for he ransiion beween he wo HR images in se-up I. The choice of hese values is dependen on he land cover srucure and he assumed changes. We assume ha i is mos likely o have no changes in any region. Neverheless each class ransiion migh happen, bu wih differen probabiliy. +1 x i = bui +1 x i = for +1 x i = crp x i = res x i = ind x i = for x i = crp Table 1: Temporal ransiion marix; corresponds o he Ikonos image, +1 corresponds o he Landsa image. For boh se-ups, we compared our mehod (scenario CRF muli ) o a Maximum Likelihood classificaion using he Gaussian model in (3) (scenario ML) and o a muliemporal MRFclassificaion (scenario MRF) using he same graph srucure as for our CRF muli approach, bu applying IS 1. For hese hree scenarios, he overall classificaion accuracy and he kappa coefficiens are compared for all epochs in Table 2. In boh seups we achieved an overall accuracy of over 80% for all images wih CRF and MRF, which is an increase of abou 8% compared o he monoemporal ML-classificaion for he Ikonos images and even 15% for he Landsa scene (Figure 4). The impac of he muli-emporal approach is highlighed by he overall accuracy achieved in he scenario CRF muli in comparison wih he resuls of a monoemporal CRF classificaion (CRF mono ) for he Landsa scene. Using CRF mono only leads o an accuracy of 72%, which is 12% lower han wih CRF muli. The higher informaion conen of he HR images clearly propagaes o he medium resoluion scene and yields a significan increase. Neverheless he accuracy of he HR image also increases. There was hardly any difference beween scenarios MRF and CRF muli. Only in a few regions finer srucures are beer preserved by he CRF-approach. S/E ML CRF muli MRF I / % / % / % / 0.73 I / % / % / % / 0.72 II / % / % / % / 0.73 II / % / % / % / 0.74 Table 2: Overall classificaion accuracy / kappa coefficiens; S/ E: Se-up/epoch; se up I: 1 : Ikonos, 2005; 2 : Ikonos, 2007; se up II. 1 : Ikonos, 2005; 2 : Landsa,

6 ISPRS Annals of he Phoogrammery, Remoe Sensing and Spaial Informaion Sciences, Volume I-7, 2012 XXII ISPRS Congress, 25 Augus 01 Sepember 2012, Melbourne, Ausralia Figure 4. lef: Reference of full es scene (Landsa); middle: Resuls of ML; righ: Resuls of CRF muli : yellow: crp; green: for; dark red: bui; whie paches: raining areas. 7. CONCLUSION In his work, we evaluaed wo possibiliies for modelling spaial conex wihin a CRF-framework. Firs he impac of using feaures exraced a differen scales for he associaion poenial was invesigaed. Neighbourhood dependencies are already aken ino accoun in his sep. Large scales resul in a severe smoohing, while iny srucures are los. Furhermore, differen conex models for he spaial ineracion poenial were compared. I could be shown ha daa-dependen models as used for CRF have a beer abiliy o preserve fine srucures. The resuls of hese invesigaions were applied in a CRF-based approach for muliemporal and muliscale image classificaion. Besides incorporaing spaial conex, his mehod uses a model of emporal conex by inroducing a emporal ineracion poenial. The overall classificaion accuracy of all images was improved by a leas 8%. The effec of he muliemporal ineracion was highlighed in a se-up of an Ikonos and a Landsa image. The overall accuracy of CRF muli in comparison o CRF mono for he Landsa scene increased a abou 12%. Furher research will concenrae on an improvemen of he model for he emporal ineracion poenial, which was kep quie simple in his work. Moreover, ess on differen daa ses wih a focus on he abiliy of he mehod for change deecion will be carried ou. ACKNOWLEDGEMENT The research was funded by he German Science Foundaion (DFG) under gran HE 1822/22-1. Our CRF-classificaion is based on he UGM oolbox by Mark Schmid: hp://people.cs.ubc.ca/~schmidm/sofware/ugm.hml. REFERENCES Bishop, C. M., Paern recogniion and machine learning. 1 s ediion, Springer New York. Bruzzone, L., Cossu, R., Vernazza, G., Deecion of land-cover ransiions by combining mulidae classifiers. Paern Recogniion Leers, 25(13): Dalal, N. and Triggs, B., Hisograms of Oriened Gradiens for Human Deecion. Proc. of IEEE Conference Compuer Vision and Paern Recogniion: Feiosa, R. Q., Cosa, G. A. O. P., Moa, G. L. A., Pakzad, K., Cosa, M. C. O., Cascade muliemporal classificaion based on fuzzy Markov chains. ISPRS J. Phoogrammery Remoe Sens. 64(2): Geman, G. and Geman, D., Sochasic relaxaion, Gibbs disribuion and Bayesian resoraion of images. IEEE Trans. on Paern Analysis and Machine Inelligence, 6(6): Hall, M. A. (1999). Correlaion-based Feaure Subse Selecion for Machine Learning, PhD disseraion, Deparmen of Compuer Science, Universiy of Waikao. Haralick, R.M., Shanmugam, K. und Dinsein, I. (1973). Texure feaures for image classificaion. IEEE Transacions on Sysems, Man, and Cyberneics, 3(6): Hoberg, T., Roenseiner, F. and Heipke, C Classificaion of Muliemporal Remoe Sensing Daa Using Condiional Random Fields. 6 h IAPR TC 7 Workshop Paern Recogniion in Remoe Sens.: 4p. Hoberg, T., Roenseiner, F. und Heipke, C. (2011). Classificaion of muliemporal remoe sensing daa of differen resoluion using condiional random fields. 1s IEEE/ISPRS Workshop on Compuer Vision for Remoe Sensing of he Environmen, IEEE ICCV Workshops, Barcelona, Kao, Z., Berhod, M., Zerubia, J., Muliscale Markov random field models for parallel image classificaion. Proc. Fourh In. Conference on Compuer Vision: Kumar, S. and Heber, M., Discriminaive Random Fields. In l. J. Compuer Vision, 68(2): Lu, D., Mausel, P., Brondizio, E., Moran, E., Change deecion echniques. In. J. Remoe Sensing, 25(12): Melgani, F. and Serpico, S. B., A Markov Random Field approach o spaio-emporal conexual image classificaion. IEEE-TGARS, 41(11): Moser, G., Angiai, E., Serpico, S. B., A conexual muliscale unsupervised mehod for change deecion wih muliemporal remoe-sensing images. Proc. 9 h Conf. Inelligen Sysems Design & Applicaions: Nocedal, J. and Wrigh, S. J., Numerical Opimizaion. 2 nd ediion, Springer New York. Roscher, R., Waske, B., Försner, W., Kernel discriminaive random fields for land cover classificaion. 6 h IAPR TC 7 Workshop Paern Recogniion in Remoe Sens.: 5p. Schnizspan, P., Friz, M., Schiele, B., Hierarchical suppor vecor random fields: join raining o combine local and global feaures. Proc. ECCV II: Shoon, J., Winn, J., Roher, C., Criminisi, A., TexonBoos for Image Undersanding: Muli-Class Objec Recogniion and Segmenaion by Joinly Modeling Texure, Layou, and Conex. Inernaional Journal of Compuer Vision, 81(1): Vishwanahan, S., Schraudolph, N. N., Schmid, M. W., Murphy, K. P., Acceleraed raining of condiional random fields wih sochasic gradien mehods. 23 rd In. Conf. on Machine Learning: Wilsky, A. S., Muliresoluion Markov models for signal and image processing. Proc. IEEE, 90(8): Yang, M. Y., Försner, W., Drauschke, M., Hierarchical condiional random field for muli-class image classificaion. Proc. In l. Conf. Compuer Vision Theory and Applicaions (VISAPP): Zhong, P. and Wang, R., A muliple condiional random fields ensemble model for urban area deecion in remoe sensing opical images. IEEE-TGARS, 45(12):