CASCAM: Crisp and Soft Classification Accuracy Measurement Software

Transcription

1 CASCAM: Crs and Soft Classfaton Auray Measurement Software Mohammad A. Shalan, Manoj K. Arora and John Elgy Shool of Engneerng and Aled Senes, Aston Unversty, Brmngham, UK Deartment of Cvl Engneerng, IIT Roorkee, Inda Abstrat No mage lassfaton s omlete untl an assessment of auray has been erformed. Auray serves as the bass for the analyss of errors, whh may ree n durng the lassfaton roess due to omlex nteratons between the satal struture of landsae, sensor resolutons and lassfaton algorthms. Indeed muh researh has been onduted on devsng a number of auray measures for both rs and soft lassfatons, they have not been mlemented n ommeral mage roessng software. The am of ths aer s to rovde the detals and the utlty of an ndgenous software for auray assessment of both rs and soft lassfaton of remote sensng data.. Introduton Classfaton s the fundamental mage roessng task to extrat nformaton from remote sensng data. Both rs and soft lassfatons may be erformed. In a rs lassfaton, eah mage xel s assumed ure and s lassfed to one lass. Often, artularly n oarse satal resoluton mages, the xels may be mxed ontanng two or more lasses. Soft lassfatons that assgn multle lass membershs to a xel may be arorate for mages domnated by mxed xels. Both suervsed and unsuervsed aroahes may be adoted. Generally, s uervsed lassfaton s adoted that nvolves three stages; tranng, alloaton and testng. Whether the goal s to rodue a rs or a soft lassfaton, the assessment of lassfaton auray n the testng stage s a rtal ste as t allow s a degree of onfdene to be attahed to the lassfaton. In essene, the assessment of auray nvolves omarson of lassfed mage wth referene or ground data (also known as ground truth. The referene data may be gathered from feld surveys, exstng mas, aeral hotograhs and any datasets at a hgher resoluton than the mage beng lassfed. The auray of rs lassfaton may be determned usng onventonal error matrx based measures suh as the overall auray, user s and roduer s auray, kaa oeffent of agreement et. (Congalton, 99. A number of other measures, for examle, Tau oeffent (Ma and Redmond, 995, and lassfaton suess ndex (Koukoulas and Blakburn, 00 have also been roosed, though used sarngly. To evaluate the auray of soft lassfaton, the lassfed oututs are often hardened so that the error matrx based measures may be used. Degradng soft lassfaton to rs results n the loss of nformaton ontaned n the soft oututs thereby hamerng ts roer evaluaton. Moreover, the referene data are also not always error- free and may ontan unertantes, and therefore may be treated as soft or fuzzy (Foody, 995. Hene, --

2 alternatve auray measures that may arorately nlude the fuzzness n the lassfaton oututs and/or referene data have been roosed. These nlude root mean square error, orrelaton oeffent (Foody and Cox, 994, entroy (Masell et al. 994 and dstane based measures (Foody, 996, fuzzy set (Goal and Woodok, 994 and fuzzy error matrx based measures (Bnagh et al The growth of so many auray measures both for rs and soft lassfaton learly ndates the urrent researh otental of lassfaton auray assessmen t roedures as no sngle measure may unversally be adoted. Deendng uon the nature of lassfaton oututs, unertantes n referene data and the qualty of nformaton desred by the end user, t may be neessary to adot not one but a ombnaton of auray measures. Deste muh of the researh beng onduted on lassfaton auray assessment and ts mortane, the urrent mage roessng software are lmted n rovdng suffent auray nformaton to the user. For examle, the well known and the most wdely used software namely ERDAS Imagne, ENVI and IDRISI, ontan auray assessment modules that an reort only a few rs auray measures namely overall, roduer s and user s auray and kaa oeffent. No other omettv e auray measures have been nluded, whh may aeal to a user n ases where the assumtons regardng the urrent auray measures are not met by the dataset. Also, there s no rovson for assessng the auray of soft lassfaton. Only IDRISI has a measure alled lassfaton unertanty that sefes the qualty of lassfaton on er xel bass, and thus may not be treated as a measure to ndate the auray of whole lassfaton. Thus, to evaluate the auray of soft lassfaton, the users ether have to deend on other statstal and mathematal software, where mort/exort of the data from one akage to another may be a tedous task, or they may have to ode ther own software. Further, to rtally examne the usefulness of a artular auray measure vs a vs other measures, dedated software for lassfaton auray assessment needs to be develoed. The am of ths aer s to ntrodue an auray assessment software sefally wrtten to evaluate the qualt y of both rs and soft lassfatons of remote sensng data. The mlementaton of the software wll be demonstrated through a soft land over lassfaton, the results of whh have reently been aeted for ublaton n IJRS Letters (Shalan et al., Detals of Software The software has been wrtten n Matlab and has been named as Crs And Soft Classfaton Auray Measurement (CASCAM. MATLAB language s a hgh - erformane language for tehnal omutng and artularly well -su ted to desgnng redtve mathematal models and develong alaton -sef algorthms, for more detals the reader may vst the MathWorks web ste ( To faltate dfferent oeratons, a number of Matlab funtons and grahal user nterfae resoures have been used to develo ths software. The mnmum requrement to run ths software s Wndows 95 oeratng system. 3. Data Fle Management The nut and outut (I/O mages are read n standard ASCII text fles wth extenson ether.as or.txt. Ths format orresonds to the ERDAS Imagne ASCII fle format that --

3 onssts of xels n eah row wth sequental olumns ndatng loaton of xels (X and Y oordnates n any measurement unt and ther ntensty values n varous bands numbered as b, b, et. For lassfaton outut fles n ASCII format, olumns reresentng bands wll ndate lass membershs of eah xel (.e., lass dentty n rs lassfaton and lass roortons n soft lassfaton. Samle data fles for nut mages and lassfaton oututs (rs and soft are gven n Tables a, b and. The mages an also be morted and exorted n three standard grahs format namely JG, TIFF and BM. However, these mages need to be onverted nto ASCII format before these an be used for further roessng n ths software. All the fles an be read from and saved to the arorate data dretores. Table a Inut mage fle format X Y Band Band Band Table b Crs lassfaton outut fle format X Y C l a s s I d e n t t y Table Fuzzy lassfaton outut fle format X Y Class Class Class3 Class4 Class Salent Features of CASCAM The software CASCAM onssts of fve bas modules namely dslay, tranng, lassfaton, testng and auray measurement modules (Table. The toral layout of menu bar, varous ou menus and button bars are shown n Fgure. Eah of these modules s now desrbed n detal. -3-

4 Table Fve bas modules of CASCAM software Module ou Menu Dslay Dslay nut mages Dslay rs lassfed mages Dslay soft lassfed mages Tranng Generate tranng data lot tranng data Dslay hstogram Classfaton Maxmum lkelhood lassfer Fuzzy C- means lassfer Testng Generate testng data from lassfaton Generate rs/soft referene mage Auray Measurement Crs auray measures Fuzzy auray measures Fgure, Man Menu of Software CASCAM 4. Dslay module Ths module has been wrtten to vew nut and outut mages stored as ASCII, JG, TIFF and BM fles. A anhromat nut mage wll be dslayed as a graysale mage reresentng varous shades of gray rangng from blak to whte. The mult- setral nut mage may be dslayed as a sngle band graysale mage or as a False Color Comoste (FCC by reresentng any three bands n three rmary olours blue, green and red. To dslay a rs lassfaton outut, the user has the oton of assgnng a artular olour to eah lass from the n-bult olour allet. Soft lassfaton oututs n the form -4-

5 of fraton or roorton mages are dslayed as graysale mages wth brght areas denotng hgher lass roorton and ve versa. The number of fraton mages s equal to the number of lasses to be maed. 4. Tranng module T r anng s the frst stage of a suervsed lassfaton roess. Ths module wll allow the user to nteratvely defne tranng areas for eah lass on the dslayed mage. The areas may be marked both as olygon boundares and on er xel bass. The satal loaton of these tranng areas may also be reresented grahally n a lot (Fgure. The marked tranng areas are used to extrat mage nformaton and are stored as an ASCII fle on xel by xel bass for eah lass sequentally (Table 3. Table 3 Tranng data fle format X Y B B B 3.. Class ID Fgure, Samle lot for Tranng Areas Often and n artular, for statstal lassfers, t s neessary to examne the qualty of tranng areas of a lass by examnng the hstogram. True mult modal lasses should be broken down nto a number of ure setral sgnature lasses. A un- modal hstogram -5-

6 ndates good qualty tranng data of a lass (.e., the xels defnng the tranng areas of that lass are relatvely ure. Therefore, the rovson for the dslay of hstograms of lasses n eah band has also been made n ths module. Fgure 3 shows a samle lot of the hstogram for tranng areas of a lass n one of the bands. Fgure 3, Hstogram for a Class n Band The user also has the oton of dretly nuttng the tranng data fle n ASCII format reated n any other software. In ths ase, the user wll be romted to rovde the number of lasses to be maed and the total number of tranng xels n eah lass. 4.3 Classfaton module The fous of CASCAM s on auray measurement. Therefore, for demonstraton uroses, the formulatons of two lassfers wth markedly dfferent haratersts, a robablst - the maxmum lkelhood lassfer (MLC (Mather, 999 and a dstrbuton free - t h e f u z z y- means lassfer (FCM (Bezdek, 984, have been nororated n ths module. The former s hghly deendent on the data dstrbutonal assumtons. However, these assumtons are often not met and, therefore, an alternatve non -arametr lassfer suh as the FCM may be advantageous to rodue rs and soft lassfatons. MLC s the most wdely used lassfer n remote sensng ommunty. In majorty of studes, ths lassfer has been used as a rs lassfer. However, the outut of an MLC n the form of a osteror lass robabltes may be related to the atual lass roortons for eah xel on ground thereby rovdng soft lassfaton. To run MLC n ths software, the fle ontanng aror lass robabltes of eah xel may also be suled. If the fle s unavalable, equal aror lass robabltes are onsdered by default. The FCM s based on an teratve lusterng algorthm that may be emloyed to artton xels of mage nto lass roortons. It s essentally an unsuervsed lusterng algorthm. However, here t has also been mlemented n the suervsed mode (Wang, 990. The user has the oton of seletng dfferent arameters ertnent to ths lassfer. In the suervsed mode, luster means are omuted from the tranng areas -6-

7 reated n the tranng module. The soft lassfaton oututs from ths lassfer are reresented n a lass membersh matrx, whh an be hardened, f desred, to rodue a rs lassfaton where a xel s assgned a lass havng the hghest membersh value. The oututs of rs and soft lassfatons from these lassfers are stored n ASCII fles as er the format sefed n Tables b and resetvely, and may be used subsequently n t estng and auray measurement modules. These fles an dretly be aessed n the dslay module to dslay rs and soft lassfaton mages. 4.4 Testng module Testng stage s the last stage of suervsed lassfaton, where auray s assessed. An arorate samle of testng xels wth known lass dentty (rs referene data or known lass roortons (soft referene data s seleted. Among a number of samlng shemes (Congalton, 988, smle random samlng has been used here to gen erate the testng datasets. The testng data samle sze s to be rovded by the user. If we assume that the unertanty of lassfaton s due to a mxture of land over lasses wthn a xel then the reaton of soft referene data s based on dervng lass roortons from an exstng ma at a fner resoluton than the mage beng lassfed. For nstane, n the workng examle here, a land over ma rodued from IRS AN mage at 5m satal resoluton, has been used as referene data to assess th e auray of lassfaton rodued from IRS LISS mage at 5m satal resoluton. Thus, a xel of LISS mage orresonds to an even number of 5m xels (n ths ase 5 xels to faltate n determnng lass roortons that sum to one for eah xel. These lass roortons of xels are alled as soft referene data. The soft referene data are hardened to rodue rs referene data for auray assessment of rs lassfaton. The testng xels, ether obtaned from rs or soft referene data, are stored n an ASCII fle aordng to the format sefed n Tables b and resetvely. Ths fle s later used n the auray measurement module. 4.5 Auray measurement module The auray of rs and soft lassfatons s determned n ths module. A number of rs and soft auray measures have been nororated. For rs lassfaton auray assessment (Fgure 4, frst an error matrx s generated from the testng data set. The user has to suly two fles: rs lassfed mage obtaned n lassfaton module and rs referene data obtaned n testng module. Alternatvely, an error matrx generated from other soures may also be rovded. The elements of the error matrx are used to derve a number of auray measures, w h h have been dvded nto three grous n ths module: erent orret measures Kaa oeffents Tau oeffents The formulatons of all the auray measures onsdered under these groungs are gven n Table 4. Further detals on these formulatons an be found n the resetve referenes ted n ths table. -7-

8 Fgure 4, Menu for Crs Classfaton Auray Assessment In the frst grou, fve auray measures overall, user s and roduer s auray, (Story and Congalton, 986, average and ombned auray (Fung and LeDrew, 988 have been nororated. Whle overall, average and ombned auray sgnfy the qualty of whole lassfaton, user s and roduer s ndate the qualty of ndvdual lass. Although, overall auray may be based towards the lass wth a large number of testng samles, average auray omuted from user s and roduer s ersetve may be based towards the lass havng a small number of samles (Fung and LeDrew, 988. Combned auray may be used to redue the bases of overall and average auray. roduer s and user s auray are used to ndate auray of ndvdual lasses. roduer s auray s so atly alled, sne the roduer of the lassfaton s nterested n know ng how well the samles from the referene data an be maed usng remotely sensed mage. In ontrast, user s auray ndates the robablty that a samle from the lassfaton reresents an atual lass on referene data (Story and Congalton, 986. However, auray measures n the erent orret grou do not take nto aount the agreement between the data sets (.e., lassfed outut and referene data that arses due to hane alone. Thus, these measures tend to overestmate the lassfaton auray (Ma and Redmond, 995. The kaa oeffent of agreement has the ablty to ontrol the hane agreement that s the result of the mslassfatons reresented by the off - dagonal elements of the error matrx. Thus, the seond grou of auray measures n ths module s formed to onsst four measures from kaa famly - kaa oeffent of agreement (Congalton et al. 983, weghted kaa (Rosenfeld and Ftzatrk- Lns, 986 and ondtonal kaa (user s and roduer s ersetve (Rosenfeld and Ftzatrk- Lns, 986. When some lasses have more onfuson than others, weghted kaa may be mlemented sne t does not treat all the mslassfatons (dsagreements equally and tends to gve more weght to the onfusons that are mo re serous than others (Cohen, 968; Hubert, 978. In ths software, a weght matrx has to be rovded by the user. To determne the auray of ndvdual lasses, a ondtonal kaa may be omuted. -8-

9 Table 4: Crs lassfaton auray measures Measure Base Referene Formulaton Defnton of terms Overall auray Story and Congalton N s total number of testng (986 n N xels. = User s auray Story and Congalton n /N n s the number of samles orretly (986 roduer s auray Story and Congalton (986 Average auray (User s Average auray (roduer s Combned auray (User s Combned auray (roduer s Kaa oeffent of agreement n /M Fung and LeDrew n (988 = N Fung and LeDrew n (988 = M Fung and LeDrew OA + AA u (988 [ ] Fung and LeDrew (988 [ O A + A A ] Congalton et al. (983 Weghted Kaa Rosenfeld and Ftzatrk- Lns (986 Condtonal Kaa (User s Condtonal Kaa (roduer s Tau oeffent (equal robablty Tau oeffent (unequal robablty Condtonal Tau (User s Condtonal Tau (roduer s Rosenfeld and Ftzatrk- Lns (986 Rosenfeld and Ftzatrk- Lns (986 Ma and Redmond (995 Ma and Redmond (995 Naesset (996 Naesset (996 o e e v j v o ( + o ( + j o e(+ o r o ( + r o ( + oj j e(+ e( + e( + lassfed. N s the row total for lass. M s the o lumn total for lass. = s the o n N = observed roorton of agreement e = N M s t h e N = exeted hane agreement v j s the weght s the observed ell o j roorton e j s the exeted ell roorton o ( + s the observed agreement aordng to user s aroah e ( + s the agreement exeted by hane for th r o w o (+ s the observed agreement aordng to roduer s aroah e ( + s the agreement exeted by hane for th olumn r = n + x N = x s the unequal ror robablty of lass membersh s the a ror robablty of lass membersh -9-

10 Nevertheless, as argued by Foody (99 and later suorted by Ma and Redmond (995, the kaa famly may overrate the hane agreement that may result nto an underestmaton of auray. Therefore, alternatves to kaa oeffents suh as Tau oeffent have been roosed. These form the thrd grou of auray measures n ths module. The rtal dfferene between the two oeffents s that Tau s based on a ror robabltes of lass membersh whereas kaa uses the a osteror robabltes. The a ror robabltes for Tau oeffents may be equal or unequal and need to be suled by the user n ths software. A ondtonal Tau oeffent s used to ndate the auray of an ndvdual lass (Naesset, 996. The error matrx based measures nherently assume that eah xel s assoated wth only one lass n the rs lassfaton and only one lass n the referene data. Use of these measures to assess the auray of soft lassfaton may therefore under or over estmate ther auray, as the soft lassfaton oututs have to be degraded to adhere to ths assumton. For evaluaton of soft lassfaton (Fgure 5, at the frst nstane, entroy may be used as a measure to ndate the unertanty n the lassfaton (Table 5. Entroy shows how the strength of lass members h (.e., soft oututs s arttoned between the lasses for eah xel (Foody, 995. The entroy for a xel s maxmsed when the xel has equal lass membershs for all the lasses. Conversely, ts value s mnmum, when the xel s entrely alloated to one lass. It, thus, shows the degree to whh a lassfaton outut s soft (.e., unertan or rs. The auray of soft lassfaton oututs s determned by omarng these wth soft referene data, as generated n testng module for a samle of testng samles. A number of auray measures may be used. For smlty, these measures have been dvded nto three grous n ths module, Measures of loseness Measures based on fuzzy error matrx Correlaton oeffent. Fgure 5, Menu for Fuzzy Classfaton Auray Assessment - 0 -

11 The formulatons of all the auray measures onsdered under these groungs are gven n Table 5. Further detals on these formulatons an be found n the resetve referenes ted n ths table. The frst grou nludes ross entroy, L and Euldean dstanes and generalzed measure of nformaton loseness. These measures estmate the searaton of two data sets based on the relatve extent or roorton of eah lass n the xel (Food y a n d Arora, 996. Lower the values of these measures, hgher s the auray of lassfaton. The dstane measures and ross entroy may be alable when there s omatblty between the robablty dstrbutons of the lassfed oututs and refe rene data. On the other hand, the generalzed measure of nformaton loseness may be used even f the robablty dstrbutons of the two datasets are not omatble (Foody, 996. Reently, Bnagh et al. (999 have roosed the onet of fuzzy erro r matrx, whh an be generated on the lnes of onventonal error matrx for rs lassfaton. The lass roortons of a xel n soft lassfaton outut are omared wth the lass roortons of that xel n the soft referene data (reated n the testng module. Fuzzy set mn oerator s used to fnd mnmum of the two values, whh s reorded n the arorate olumn, and summed over all the xels to generate the fuzzy error matrx. The elements of ths matrx are used to omute overall, user s and roduer s auray, whh have the same nterretaton as desrbed earler. From the ont of vew of standardzng the auray assessment roedures for both rs and soft lassfaton, these measures aear more logal to be used n assessng the qualty of remotely sensed derved lassfatons. The orrelaton oeffents may also be used to ndate the auray of ndvdual lasses and have been defned n the thrd grou of auray measures. The hgher the orrelaton oeffent better s the auray of an ndvdual lass. The user has the oton of omutng the rs and soft auray measures ndvdually or all n one go. The error matrx, the fuzzy error matrx and the values of the seleted auray measures an also be saved to a text fle. - -

12 Table 5: Soft lassfaton auray measures Measure Base Referene Formulaton Defnton of terms Entroy (H Masell et al. (994 s the roorton ( log ( o f th lass n a xel = from the fuzzy refe rene data. Euldean dstane (E L (Cty Blok dstane Kent and Marda (988, Foody (996 ( Foody and Arora (996 = = s the roorton of th lass n a xel from the fuzzy lassfaton. Cross-entroy o r Dret dvergene (D Measure of nformaton loseness (I Foody (995 Foody (996 D(, = + = = + I(, = D, ( log ( ( log ( + + D, s the robablty dstrbuton of fuzzy referene data. s the robablty dstrbuton of fuzzy lassfaton outut. Correlaton oeffents (R Foody and Cox (994, Masell et al. (996 C o v (, S t d ( Std( C o v (, s the ovaran e between the two datasets. Std( and Std( are the standard devatons of the resetve datasets - -

13 5. A Workng Examle To demonstrate the software, a ase study on auray assessment of soft lassfaton from IRS C remote sensng data s brefly resented here. More detals an be found n Shalan et al. (003. IRS C LISS mage (Fgure 6 was used as rmary mage to rodue soft lassfaton. Fve domnant land over lasses n the regon namely agrulture, forest, grassland, urban and sandy areas were onsdered. An MLC derved rs lassfaton of the AN mage nto fve land over lasses was used as referene data (Fgure 7. The LISS mage was regstered to AN mage derved land over lassfaton to an auray of /3rd of a xel, usng frst order olynomal transformaton and nearest neghbourhood resamlng. The regstered LISS and AN mages were resamled to 5 m and 5 m resetvely suh that a LISS xel orresonds to an even number of AN xels (n ths ase 5 xels to faltate n generatng soft referene data n the form of lass roortons. Fgure 6 IRS C LISS III FCC (Red: band 4, Blue: band, Green: band Fgure 7 Classfed AN Image used as Referene Data - 3 -

14 The two lassfers mlemented n lassfaton module of ths software, MLC and FCM, were used to erform soft lassfaton. For effetual omarson wth MLC, the suervsed verson of FCM was aled here. In the formulaton of FCM, a weghtng fator m that desrbes the degree of fuzzness has to be rovded. Here, m was set to.0, motvated by the study onduted by Foody (996, where ths value was found to rodue the most aurate fuzzy lassfaton. The tranng data fle was reated usng tranng module of ths software and onssted of randomly seleted 997, 86, 79, 596 and 805 xels for agrulture, urban area, sandy area, forest and grassland resetvely. In the testng module, a total of 650 testng xels from the entre mage were randomly seleted for auray assessment. The auray of soft lassfaton was evaluated usng ross entroy, Euldean dstane, and orrelaton oeffents as defned n the auray measurement module of the software. Entroy was used to examne the degree of unertanty n the soft lassfaton oututs (Table 6. For a fve- lass roblem, the maxmum value of entroy s From table, t an be seen that the average entroy values (omuted over all the xels for the soft lassfatons rodued from both the lassfers, are very lose to the maxmum entroy value. Ths learly llustrates the resene of lass mxtures (or unertantes n the dataset. In Table 6, the lower values of ross entroy and Euldean dstane for MLC demonstrate that ths lassfer has rodued more aurate lassfatons than FCM for the dataset onsdered. From orrelaton oeffents, t an be seen that the lass sandy area has been lassfed as the most aurate lass by both the lassfers, as ths lass was very well setrally searable from the other lasses. Table 6 Auray of fuzzy lassfatons rodued from MLC and FC M Auray Measure MLC F C M (Average E n t r o y Cross-entroy Euldean Dstane Table 7 Correlaton oeffents of lasses from soft lassfatons Class MLC F C M Agrulture Urban area Sandy area Forest Grassland To nset soft lassfatons vsually, fraton mages ortrayng the satal dstrbuton of fve land over lasses were rodued usng the dslay module of the software (Fgure 8. From ths fgure also, t an be observed that for all the lasses, MLC has generally redted loser relatonsh of lass roortons wth the referene data than the FCM

15 FCM MLC Soft Referene Data A g r u l t u r e U r b a n a r e a s Forest G r a s s l a n d S a n d y a r e a s Fgure 8: Fraton Images from MLC and FCM Comared wth Soft Referene Data - 5 -

16 6. Summary Classfaton auray assessment s an mortant ste of mage lassfaton roess. A number of auray measures for both rs and soft lassfatons have been roosed. No measure has been unversally adoted. Often, a ombnaton of auray measures may have to be used to desrbe the qualty of lassfaton omletely. However, the urrent mage roessng software lak the rovson of varous auray measures. In ths aer, detals of auray measurement software named as CASCAM, wrtten exlusvely for the auray assessment of rs and soft lassfaton assessment of remote sensng data have been rovded. A range of auray measures has been nororated. The software, develoed on Matlab latform, s nteratve and userfrendly. The aabltes of the software have been demonstrated through a ase study on soft land over lassfaton from IRS C LISS remote sensng data. 7. Aknowledgments Ths software was develoed when M. A. Slalan was a ost- graduate at IIT Roorkee, Inda, on a sholarsh from Mnstry of Hgh Eduaton (MHE, Syra, under the sheme of Indan Counl for Cultural Relaton (ICCR. Ths aer has been wrtten when M. A. Shalan was at Aston Unversty, UK, under a sholar sh from the MHE. 8. Referenes Bezdek, J.C., Ehrlh, R., and Full W., 984, FCM: The Fuzzy - Means Clusterng Algorthm, Comuters and Geosenes, Vol. 0, Bnagh, E., Brvo,.A., Ghezz,., Ramn, A., 999, A fuzzy set -based a uray assessment of soft lassfaton, attern Reognton Letters, 0, Cohen, J. 968, Weghted kaa: nomnal sale agreement wth rovson for saled dsagreement or artal redt, syhologal Bulletn, (70: 3-0. Congalton, R. G. 9 88, A omarson of samlng shemes used n generatng error matres for assessng the auray of mas generated from remotely sensed data. hotogrammetr Engneerng and Remote Sensng, 54, Congalton, R.G., Oderwald, R. G., and Mead, R. A. 98 3, Assessng Landsat Classfaton Auray usng Dsrete Multvarate Analyss Statstal Tehnques, hotogrammetr Engneerng and Remote Sensng, Vol. 49, Congalton, R.G. 99, A Revew of Assessng the Auray of Classfatons o f Remotely Sensed Data, Remote Sensng of Envronment, Vol. 37, Foody, G. M., 99, On omensaton for hane agreement n mage lassfaton auray assessment, hotogrammetr Engneerng & Remote Sensng, 58, Foody, G. M., 995, Cross-entroy for the evaluaton of the auray of a fuzzy land over lassfaton wth fuzzy ground data. ISRS Journal of hotogrammetry and Remote Sensng, 50, -. Foody, G. M., 996, Aroahes for the roduton and evaluaton of fuzzy land ov er lassfatons from remotely sensed data, Internatonal Journal of Remote Sensng, 7, Foody, G. M., and Arora, M. K., 996, Inororatng mxed xel n the tranng, - 6 -

17 alloaton and testng stages of suervsed lassfaton, attern Reognton Letters, 7, Foody, G.M., and Cox, D.., 994, Sub -xel land over omoston estmaton usng a lnear mxture model and fuzzy membersh funtons, Internatonal Journal of Remote Sensng, 5, Fung, T., and LeDrew, E., 988, The Determnaton of Otmal Threshold Levels for Change Deteton usng Varous Auray Indes, hotogrammetr Engneerng & Remote Sensng, Vo. 54, Goal, S., and Woodok, C., 994, Theory and Methods for Auray Assessment of Themat Mas Usng Fuzzy Sets, hotogrammetr Engneerng and Remote Sensng, Vol. 60, No., Hubert J.L, 978, A general formula for the varane of Cohen s weghted kaa, syhology Bulletn, 85, Jensen, J.R., 986, Introdutory Dgtal Image roessng, (New York, rente Hall Kent, J. T., and Marda, K. V., 988, Satal Classfaton Usng Fuzzy Membersh Models, I.E.E.E. Transatons on attern Analyss and Mahne Intellgene, Vol. 3, Koukoulas, S., and Blakburn, G.A., 00, Introdung New Indes for Auray Evaluaton of Classfed Images Reresentng Sem -Natural Woodland Envronments, hotogrammetr Engneerng & Remote Sensng, 67, Ma, Z., and Redmond, R.L., 995, Tau oeffents for auray assessment of lassfaton of remote sensng data, hotogrammetr Engneerng & Remote Sensng, 6, Masell, F., Rodolf, A., and Conese, C., 996, Fuzzy Classfaton of Satally Degraded Themat Maer Data for the Estmaton of Sub - xel Comonents, Internatonal Journal of Remote Sensng, 7, Masell, F., Conese, C., and etkov, L., 994, Use of robablty entroy for the estmaton and grahal reresentaton of the auray of maxmum lkelhood lassfatons, ISRS Journal of hotogrammetry and Remote Sensng, 49(: 3-0 Mather,. M., 999, Comuter roessng of Remote Sensng Data, Seond Edton, Chhester, John Wley. Naesset, E., 996, Condtonal tau oeffent for assessment of roduer s auray of lassfed remotely sensed data, ISRS Journal of hotogrammetry and Remote sensng, 5, Rosenfeld, G. H., and Ftzatrk- Lns, 986, A oeffent of agreement as a measure of themat lassfaton auray, hotogrammetr Engneerng & Remote S e n s ng, 5, 3-7 Shalan, M. A., Arora, M. K., and Ghosh, S. K. 003, Evaluaton of fuzzy lassfatons from IRS C LISS III data, Internatonal Journal of Remote Sensng Letters (In ress. Story, M., and Congalton, R. G., 986, Auray assessment: A user s ersetve, hotogrammetr Engneerng & Remote Sensng, 5, Wang, F., 990, Fuzzy suervsed lassfaton of remote sensng mages, IEEE Transatons on Geosene and Remote Sensng, 8,