A Comparative Study of Conventional and Neural Network Classification of Multispectral Data B.Solaiman & M.C.Mouchot Ecole Nationale Supérieure des Télécommunications de Bretagne B.P. 832, 29285 BREST Cedex FRANCE Tel: (33) 98 00 13 08, Fax: (33) 98 00 10 98 ABSTRACT In this study, the classification of remotely sensed data using several classifiers and neural networks is considered. The application was conducted using a test scene containing mainly agricultural areas. The main result obtained in this study is that the application of topological map based neural networks to classify the intensity vectors issued from agricultural classes are more suited than other neural network methods, especially the Multi Layer Perceptron (MLP) usually employed. Obtained results are very close to those of the Maximum Likelihood Classifier (MLC). INTRODUCTION The aim of remotely sensed data classification is to produce the best accuracy map of the earth surface assigning each pixel (using the corresponding feature vector) to its appropriate category of the real-world. Among all supervised non contextual classification methods (K-means, K-Nearest Neighbours,...), the Maximum Likelihood Classifier (MLC) based on Gaussian probability distribution functions for the probability estimation in the discriminate function is considered to be the best in the sense of obtaining optimal recognition rates [Schowengerdt,1983]-[Mather,1989]. Recently, the application of neural networks to the classification of multispectral sensed images is increasingly emerging. This is due to the following characteristics : 1) their ability of learning provides an interesting alternative to the MLC, 2) they make no assumptions about the probabilistic model to be made, 3) they are capable of forming highly non-linear decision boundaries in the feature space and therefore, they have the potential of outperforming a parametric Bayes classifier when a feature statistics deviate significantly from the assumed Gaussian statistics. In this study, the application of several conventional classifiers as well as several neural methods to the classification of multispectral remotely sensed data is considered. The main objectives of this work are :1) to compare the results obtained using two learning data bases, 2) to study the performances of several neural classification methods especially those based on the use of clustering approaches : the Learning Vector Quantization 2 (LVQ2) [Kohonen,1989], the Kohonen Self Organisation Feature Maps (SOFM) [Kohonen,1989] and the Hybrid Learning Vector Quantization [Solaiman,1994] networks. Obtained results using these different classifiers are discussed in terms of recognition rate as well as on their generalisation capability on the whole image. This generalisation capability is essentially evaluated by visualising the reconstructed images. LEARNING DATA BASES AND CLASSIFIERS DESCRIPTION In this work, a LANDSAT Thematic Mapper (TM) sub-scene ( 512x512 pixels ) acquired over the Canadian province of Saskatchewan is studied. The satellite data and the associated ground truth are a courtesy of Canada Centre for Remote Sensing. Among the 7 bands of the TM data only the first 5 were kept for processing through this study. Covering most of the visible and the near infra-red spectrum they tend to provide a good discrimination of the different types of land use.
This test area land contains mainly the agricultural category. The main goal is then to discriminate between different sorts of cultures. Two data bases are used in this study. The first one (BASE1), is issued from the ground truth analysis where 8 classes are defined. These classes are : water, humid area, wheat, flax, peas, barley, canary grass and summer fallow. The second data base (BASE2) is obtained by analysing the ground truth information and by observing the spectral intensities associated with different classes in the five spectral bands. This leads to the division of the following classes into several subclasses: class wheat (divided into 5 subclasses), class barley and the class peas (each one is divided into two subclasses). So, BASE2 is considered to represent 14 classes. Each pixel in these data bases is characterised by a vector (spectral vector) containing the five observed spectral intensities. In this study, several classification methods were tested over these two data bases. These methods are : the K-means (K-M), the K-Nearest Neighbours (K-NN), the maximum likelihood (MLC), the MLP network, the LVQ2, the Kohonen SOFM and the HLVQ classifiers. The K-means is a clustering method based on the determination of K centroïds (vector). Each centroïd is considered as a prototype vector of one class. The classification of a spectral vector is based on the Euclidean distance between this vector and the K prototype vectors. The K-Nearest Neighbour classification method of a pixel starts by determining the K nearest spectral vectors from the learning data base. The classification decision is given as the class that is mostly represented in the set of the nearest K spectral vectors. The MLC method is based on a Gaussian probability distribution function for the probability estimation in the discriminate function. Several neural classification methods are used in this study. The Multi Layer Perceptron classifier using the supervised Back-Propagation learning algorithm [Lippman,1987] is the most commonly neural network used into the classification of multispectral data. The LVQ2 network, is a competitive neural network using a supervised learning algorithm. This algorithm is described as follows: when a spectral vector is fed to the network, the two nearest (the winning and the next winning) neurons are determined. The adaptation of the synaptic weights of these two neurons is carried out only if the three following conditions are verified : 1) the class corresponding to the winning neuron is not the same as the class of the input vector, 2) the class corresponding to the next winning neuron is the same as the class of the input vector, 3) the input vector is very close to the discriminating surface between the two classes of the winning and the next winning neurons. The Kohonen SOFMs are generally used as non supervised classification methods permitting the determination of homogeneous categories in the input data space. Since they use a non supervised learning algorithm, their performances in classification problems are generally lower than the MLP classification method. The HLVQ neural network is a Kohonen SOFM using a hybrid non-supervised/supervised learning algorithm based on the combined use of the non-supervised learning algorithm of the SOFMs and the supervised LVQ2 algorithm. The main objective of this network is to obtain high recognition classification rates, while preserving the topology mapping of the SOFM. CLASSIFICATION RESULTS The first simulation was conducted over the data base : BASE2 containing the 14 classes. Each class was represented by 100 pixels. In the K-means (K-M) classifier, K=14 centroïds were used. Only one nearest neighbour (K=1) was used on the K-NN classifier. Concerning the MLP neural network, optimal results in terms of recognition rates using the test data base were obtained with one hidden layer containing 35 neurons. This network was trained using the classical back propagation algorithm. 2
The last three neural networks used in this study (LVQ2, the Kohonen SOFM and the HLVQ networks) were tested using a two dimensional 8x8 map with the same initialisation values for the synaptic weights. For the LVQ2 network, the notion of the topological map makes no sense because all competitive neurones act without topological relations. So, the LVQ2 network is considered as using 64 competitive neurons. The stop-learning criteria used was the achievement of a stable synaptic weights. Results obtained in terms of mean recognition rates for the seven different classifiers using the test data base are given in TABLE 1.: TABLE 1. Mean recognition rate of different classifiers using the data base BASE2 K-M K-NN MLC MLP LVQ2 Kohonen HLVQ 77.8% 56.7% 94.5% 81% 76.1% 89.3% 94.8% These results are very general and do not give a precise description of the capability of each classifier concerning each of the 14 classes. Nevertheless, the following remarks can be given : 1) classification results in terms of mean recognition rate of the MLP network, which is a decisionsurface based classifier, are nearly 10% less than those of the MLC and the topological maps used by the Kohonen SOFM and the HLVQ networks. This means that the multi spectral data representing agricultural areas are organised in some cluster forms in the input space. In fact, using the Principal Component Analysis by making the projection of the spectral vectors into the plane formed by the first two principal components, three classes can be easily discriminated : the water, the humid area, and the "agricultural" classes. This means that if several agricultural classes are grouped together into one agricultural category then, the use of a decision-surface based classifier like the MLP network gives very good results. Other how, if the agricultural category is splitted into several classes, then the use of a clustering based classifier is well adapted. 2) classification results by the K-means, the K-nearest neighbours and and the LVQ2 networks, which are also clustering based classifiers, are nearly 12% less than the Kohonen SOFM and the HLVQ networks. The common clustering idea among the K-M, the K-NN and the LVQ2 classifiers is that each centroïd is determined independently from its "neighbouring centoïds". This confirms the interest of using the topological organisation notion applied in the Kohonen SOFM and the HLVQ networks, where the topology preserving between the centroïds is an important feature of the classifier. When testing the generalization capability of these different classification methods, the best results were obtained with the HLVQ neural network and the MLC classifiers. Obtained images are shown on figure.1 and.2. 3
Figure 1. HLVQ network classification results using 14 classes Figure 2. MLC classification results using 14 classes 4
By analysing these two images, the HLVQ and the MLC methods seem to produce the same important features contained in the original image. The second simulation conducted in this study consists in the use of only the MLC and the HLVQ neural network classifier in the classification of the data base BASE1 containing only the 8 classes. Results obtained by this experimentation are compared to those obtained by the use of the classifiers over the data base BASE2 and by grouping the decisions concerning the different subclasses into their mother classes. Obtained results in terms of mean recognition rates are resumed in TABLE 2.: TABLE 2. MLC HLVQ 8 classes 96.1% 91.5 % 14 classes grouped into 8 97.2 96.5 after classification This confirms that the HLVQ network and the MLC are very comparable in terms of recognition rates. Concerning the generalization capability over the whole studied scene, both the MLC and the HLVQ network preserve wellthe shapes and the edges of the different image features. CONCLUSION In this study, several classification methods were compared and tested over a multispectral scene containing agricultural classes using a limited small size data bases. The best two : the MLC and the HLVQ classifiers proved to be very comparable in terms of recognition rates and generalization capacities. Nevertheless, the computation time of the HLVQ is sensibly less than the MLC. This means that this neural network architecture may be considered as a good alternative to the classical MLC method, especially when processing hyper spectral data where several hundreds of spectral bands have to be considered together. References [Kohonen,1989]T.Kohonen,"Self-Organization and associative Memory", 3d ed, 1989, Berlin :Springer-Verlag. [Lippman,1987]R.P.Lippman,"An Introduction to computing with neural nets", IEEE Acoustic, Speech and Signal Processing Magazine, 4(2), pp4-22, April1987. [Mather,1989]P.M.Mather, "Computer Processing of Remotely Sensed Images, An Introduction", Wiley, New York ( 2nd edition) 1989. [Schowengerdt,1983]R.A.Schowengerdt,"Techniques for Image Processing and Classification in Remote Sensing", Academic Press, 1983. [Solaiman,1994]B.Solaiman, M.C.Mouchot and E.Maillard, "A hybrid algorithm, HLVQ, combining unsupervised and supervised learning approaches", International Conferences on Neural Networks, ICNN94, June26-July2, Orlando, USA, 1994. 5