Application of Imperialist Competitive Algorithm for Automated Classification of Remote Sensing Images

Transcription

1 Application of Imperialist Competitive Algorim for Automated of Remote Sensing Images S. Karami and Sh.B. Shokouhi Abstract Recently, a novel evolutionary global search strategy called Imperialist Competitive Algorim (ICA) has proven its superior capabilities in optimization problems. This paper presents an application of ICA in automated clustering of remote sensing images. The proposed algorim is basically a hierarchical two-phase process. At e first phase e original data set is decomposed into water bodies and land cover classes using near Infrared band s information. At e second phase, ICA has been applied to determine e number and centers of e land cover clusters using RGB band s information during an unsupervised clustering. The optimization is based on Fuzzy C-Means and an additional term for improving e accuracy of clustering. The meod is applied on pan-sharpened IKONOS images of Tehran and 4 artificial data sets wi different properties. Results obtained from applying e proposed meod for bo artificial data sets and RS image, indicate promising ability of is meod in clustering data wi unknown cluster number. Also e results show at e achieved overall accuracy can be available, better an 78% in comparison wi oer applied meods. Index Terms Imperialist Competitive Algorim, Remote Sensing, Hierarchical, Fuzzy C-Means Clustering I. INTRODUCTION The Remote Sensing (RS) technology is currently being offered a wide variety of digital imagery at covers most of ear s surface. Before e RS image data can yield e required information about e obects of interest, ey need to be processed. The analysis and information extraction is e main part of e overall remote sensing process. of RS images is an important process at provides significant information for a variety of applications. Such information is a necessary input to many decision making and planning processes. is e most common meodology in ematic map production from satellite images. Supervised classification requires knowledge of e area at land. But in most of RS applications is knowledge is not sufficient available or e classes of interest are not yet defined. In ese situations an unsupervised classification can be applied. Unsupervised classification of RS images eliminates e need for human operators to perform e time consuming and expensive process of training sample introduction. The art of unsupervised classification has been pursued by many researchers over e years and many different meods have been brought to bear on e subect. Manuscript received February 17, 01; revised March 30, 01. S. Karami is wi e corresponding auor and can be contacted ( Karami_samaneh@yahoo.com). S. B. Shokouhi is wi e international scientific societies such as IEEE, IEICE and also Iranian Machine Vision and Image Processing (MVIP). Clustering is an important unsupervised classification technique used in identifying some inherent structure present in a set of obects [1]-[]. The purpose of cluster analysis is to classify obects into subsets at have some meaning in e context of a particular problem. More specifically, in clustering, a set of patterns, usually vectors in multi-dimensional space, are grouped into clusters in such a way at patterns in e same cluster are similar in some sense and patterns in different clusters are dissimilar in e same sense. In some clustering algorims e number of classes at pixels must be grouped into is known a priori. In such situations clustering is formulated as distribution of pixels in C (number of classes) group, such at e pixels in a group are more similar to each oer an e pixels in oer groups. Most common classification algorims such as K-Means and Fuzzy C-Means algorims are applicable in such situations [3]. However, e maor drawback of ese clustering algorims is at ey often get stuck at local minima and e results are largely dependent on e choice of e initial center of classes. On e oer hand, in most real life situations e number of clusters is not defined priori so e real challenge in is situation is to determine e number of optimal classes automatically. In last several years, automated clustering has attracted a lot of research interests and many approaches have been proposed to determine e number of classes automatically. Genetic algorim, one of e most popular optimization algorims, has been applied for is purpose in many clustering problems [4]-[5]. In is paper we propose a new meod at uses ICA to determine e number of optimal classes automatically and integrate e effectiveness of fuzzy C-Means algorim, wi e capability of ICA for providing e requisite perturbation to bring it out of e local minima. ICA is a new optimization algorim at has recently been introduced for solving different optimization problems. This algorim is a global search strategy at uses e socio-political competition among empires as a source of inspiration [6]. ICA has been applied successfully in different domains namely designing controllers [7]-[8], recommender systems, characterization of elasto-plastic properties of materials [9] and many oer optimization problems [10]-[11]. Comparing e results has shown great performance in bo convergence rate and better global optima achievement. The effectiveness of e ICA based clustering technique is demonstrated on four artificial data sets having different characteristics. A real life application of clustering technique is provided for automatically clustering an IKONOS satellite image of a part of city Tehran into distinct regions. Then e performance of ICA in image clustering is compared wi conventional GA and K-means 137

2 clustering meods. The results obtained by e proposed algorim and its comparison to GA and K-means indicate at e ICA has better performance in terms of bo accuracy and convergence properties. In most of e previous meods RGB properties of e different regions were used as similarity criteria for clustering data sets into different classes. One problem at reduces e clustering accuracy of RS images is at, due to suspended sediment and seaweeds present in e upper layers of e water body, ese layers absorb more of e blue wavelengs and reflect e green, making e water appear greener in color. So in RS images water typically looks blue-green. Since water bodies and vegetation regions appear similar in color, RGB color property is not a comprehensive criterion to discriminate em. Comparing e spectral response of patterns we may be able to distinguish between targets, where we might not be able to, if we only compared em at RGB wavelengs. IR radiation is absorbed more by water an visible wavelengs, us water and vegetation reflect somewhat similarly in e visible wavelengs but are always separable in e IR waveleng [1]. Because of is property of near infrared band, in proposed clustering approach, we use e near infrared Band of information to separate water body. Then RGB properties are used to determine e number of clusters and classification of land covers. In is regard, e proposed algorim in is paper composes of two main steps. The first one separates water body and land cover regions and e second classifies land cover areas. The remaining sections are organized as follows. Section II briefly introduces ICA and in section III, ICA is applied to determine e number of optimal clusters and clustering e RS images in 4-dimensional feature space during a hierarchical process. Section IV provides e experimental results. Eventually conclusion of e paper and remarks to e future are given in section V. II. BRIEF DESCRIPTION OF ICA Imperialism is e policy of extending e power and rule of a government beyond its own boundaries. ICA is a novel global search strategy at uses imperialism and imperialistic competition process as a source of inspiration. This algorim is based on at in real world countries try to extend eir power over oer countries in order to use eir resources and bolster eir own government. In fact, imperialist countries attempt to dominate oer countries and turning em to eir colonies. Also, imperialist countries compete strongly wi each oer for taking possession of oer countries. During is competition stronger empires will get more power and e weakest one will ultimately collapse. This general policy of imperialist competition is used as e basis of e ICA. Fig. 1 shows e flowchart of is algorim [6]. In optimization problems e final goal is to find an optimal solution of e variables of e problem. So in ese problems an array of variables is formed to be optimized. In ICA, e term country is used for is array. In an N-dimensional optimization problem, a country is an 1 N array. The variable values of e country are floating point numbers. Fig. 1. Flowchart of e imperialist competitive algorim. As shown in Fig. 1, algorim starts wi some initial countries at are randomly dispread in search space. Some of e stronger countries (countries wi lower cost) in e population are selected to be e imperialists and all e oer countries are divided among em based on eir power. The initial number of colonies of each imperialist should be directly proportionate to its normalized power. The normalized power of an imperialist is defined by P n C n N imp C i1 i where Cn is e normalized cost of n imperialist and is defined as difference between e cost of imperialist and total cost of all imperialists. Then e initial number of colonies of n empire will be: N n round { Pn. Ncol} where N col is e number of all colonies. To form initial empires, each imperialist randomly picks N n of e colonies. After creating e initial empires, colonies start moving toward eir relevant imperialist state. This process is called assimilation policy. In assimilation, each colony moves on e line at connects e colony and 138

3 its imperialist by x units, which x is a random variable wi uniform distribution. Then for x we have x ~ U(0, d) where is a positive number which is considered to be smaller an and d is e distance between e colony and its imperialist. To create new positions for colonies around e imperialist in different directions, a random deviation angle ( ) is added to e direction of movement, ~ U (, ) where adusts e deviation from original direction. In fact and are arbitrary numbers at modify e area at colonies randomly search around eir corresponding imperialist. If is movement causes to find a colony wi better position (lower cost) an at of imperialist, e colony moves to e position of e imperialist and vice versa. The total power of an empire is defined by e power of imperialist state plus a percent of e mean power of its colonies.in imperialistic competition, all imperialists try to take e possession of oer imperialist s colonies and increase eir own power. In ICA, is imperialistic competition is modeled as simply taking some of e weakest colonies of e weakest empire and giving em to e empire at has e most probability of possessing em. The possession probability of each empire depends on its total power, so stronger imperialists will be more likely to possess selected colonies. The total power of an empire is mainly affected by e power of imperialist. But e power of e colonies has an effect, albeit negligible, on e total power. The total power of an empire is defined by P P meam { power ( colonies )} t im where Pim is e power of e imperialist and is a positive number which is considered to be smaller an 1. As a result of imperialistic competition, e colonies of powerless empires will be divided among oer imperialists and e empire will be collapsed. Collapse mechanism will cause an increase in e power of great empires and eliminating e weaker ones. As a result, all e countries converge to a state in which ere exists only one empire at possesses all oer countries as its colonies. Imperialistic competition and e movement of colonies toward eir relevant imperialist will hopefully cause to make a new world wi one empire at all its colonies have e same position and power as e imperialist. In is situation e imperialist consists of an array of variables which is an optimal solution of e problem. In addition to is stopping condition, in ICA like oer evolutionary algorims reaching a maximum number of iterations (decades) or having no improvement in final results for several consecutive iterations may be chosen as stopping criteria. In ese situations, e solution of problem is determined by e imperialist of e most powerful empire. We applied ICA to find optimal cluster centers of an image during e hierarchical process based on fuzzy C-Means clustering analysis and an additional parameter for improving e accuracy of clustering. III. HIERARCHICAL IMAGE CLUSTERING WITH UNKNOWN CLUSTER NUMBER USING ICA In is section, an attempt has been made to use ICA for automatic clustering a satellite image in 4-dimensional feature space wi unknown cluster numbers. As mentioned in section I, e clustering meod proposed here mainly composes of two steps. The first one uses near IR band of information to separate water bodies and decomposing image into water and land cover regions. In e second one, we use ICA to determine e optimal number of clusters and proportional centers and classification of land cover regions based on RGB properties of image pixels. Each stage of is algorim will be described subsequently. A. Decomposing Image into Water Body and Land Cover Classes Using ICA As described before, at e first step of proposed meod, e grey level values of near Infra Red band is used as a similarity criterion for separating water bodies and land covers. So, two cluster centers should be determined. In is meod we use ICA for determining e appropriate cluster centers and dividing all image pixels among e mentioned cluster centers based on eir near IR band properties. As mentioned in section II, ICA starts wi some initial countries at are randomly dispread in search space. Here e countries are arrays of two real numbers which each of em represents e near infra red gray level of one of e water and land cover cluster centers. After initializing countries, e partition matrices of each country U and e fuzzy cluster centers Z i are calculated by using ui i i 1 1 for1 i Cand1 n Which u i denotes e grade of membership of e pixel to e i cluster, n is e total number of image pixels, i is e gray level value of i cluster center and is e gray level value of image pixel. n ( ui) 1 Z i n ( ui) 1 fori 1, ( Which n, is e total number of image pixels, i is e gray level value of i cluster center and is e gray level value of image pixel. Then we calculate U new using Zold and Z new using new obtained U. We repeat is process until ere is no more change in Z for a fixed tolerance. 139

4 After calculating cluster centers, we determine a crisp cluster structure by replacing e highest membership value of each pixel by 1 and allocating it to e related cluster. Using is meod helps us to provide more information when e clusters present in e data are overlapping in nature. After forming clusters, e power of each country is calculated by using following equation: ( z) z1 z P( z, x) _ ( z, ) n1 n z z i i i1 1 i1 Which n i is e number of pixels belongs to e i cluster. After calculating e power of all countries, some of e best countries are selected as imperialists and all e oer countries are divided among e mentioned imperialist based on eir power. As mentioned in section, after creating initial empires colonies start moving toward eir imperialists (In our implementation is a random number ranging from 0 to and is less an 0.5 (rad)). For all new positions we should use fuzzy C-Means clustering to form new clusters. If e movement creates colonies wi lower cost an eir imperialists, e position of at colonies and imperialists will exchange. Then e total power of all empires is calculated and imperialistic competition start between empires. This process continues until one of e mentioned stopping conditions will be reached. In is situation e selected imperialist consists of an array of two variables which each of em represents gray level of one of e water and land cover cluster centers. B. Clustering Land Covers wi Unknown Cluster Number Using ICA After separating water bodies, land covers should be clustered. The real challenge in is situation is to be able to automatically evolve an optimal value of cluster numbers as well as providing e appropriate clustering of land covers. In is section an attempt has been made to propose an algorim at uses ICA for automatic clustering land covers. In is step countries are directly characterized as a real code string. The string is an array of 3-dimensional vectors, which encodes e centers of clusters in e RGB color space. Each string represents different number of cluster centers in 3 dimensions. Then if a country encodes C i cluster centers, e leng of e string is l 3Ci. The values of elements of each cluster center vector ranging from 0 to 55. After initializing countries, we calculate partition matrix U for each country by using (9) as below C D ( ci, x ) 1 k ( ) ] for1 i Cand1 n (9) D ( c, x ) ui 1 k Which c i is 3D vector of i cluster center, D ( ci, x ) is e Euclidean distance between i cluster center and image pixel and C is e number of cluster centers of each country. Then we use (7) and repeat process until ere is no more change in cluster centers. In such a condition as mentioned before pixels are divided among e cluster centers based on eir membership grade and crisp clusters are formed. After forming initial clusters, e empires are formed based on e power of countries. The power of each country is defined by ( c) min Di P( c, x) ( c, x) ni C D i1 1 ( c ( x i, i, c c ) ) (10) Which p is e power of country, ni is e number of pixels belong to i cluster, c i is 3D vector of i cluster center, D ( ci, c ) is e Euclidean distance between i and cluster centers and D ( x, c ) is e Euclidean distance between i pixel and cluster center. Because of different lengs of countries e assimilation process is different from what mentioned in sec. In is situation each 3D vector (Cluster center) of a colony randomly selects one of e vectors of its imperialist and moves toward it on e line at connects ese two vectors wi a small deviation angle. Then algorim continues as described in last section until e stopping condition (variation in e power of best imperialist during 5 decades be less an a predefined reshold) will be reached. In is situation e selected imperialist consists of an array of 3D vectors which each of em represents a cluster center. IV. IMPLEMENTATION RESULTS In is section, we present e results of our research. In ICA e initial number of countries is set to 300 at 0 of em are chosen as initial imperialists to form initial clusters. Also and are set to 1.5 and 0.5 (rad) empirically. The experimental results are provided for four 3D artificial data sets (Data1, Data, Data3, Data4), wi different properties and a satellite image of a part of city Tehran. Also a genetic based clustering algorim (wi e same cost function used in ICA) and K-means clustering algorim are applied to cluster e satellite image. Comparing e results obtained by ese ree meods shows e capability of our proposed hierarchical ICA based clustering algorim in clustering data wi unknown cluster numbers and overlapped clusters. A. Results on Artificial Data Sets The artificial data sets are generated using a Gaussian distribution in 3dimensional feature space, so for clustering, we utilize e second part of algorim uses RGB properties of obects. Data1 contains 4 clusters which are fully separate from each oer. Data contains 4 clusters where two of em are overlapped. Data3 contains 4 clusters where are fully enclosed two by two. Data4 have 4 clusters where 3 of em i 140

5 are overlapped. Figs. -5 show e four data sets and tables I-IV show e confusion matrices of em, respectively. From e results it can be seen at ICA in conunction wi improved fuzzy clustering meod has successfully found exact number of clusters and clustered em wi high accuracy in all four cases. B. Results of IKONOS Image Hierarchical Clustering In is section e utility of proposed clustering algorim for partitioning pixels into different regions in satellite image, is investigated. IKONOS satellite image of a part of city Tehran in RGB bands is shown in Fig. 6. Fig. 7 shows, e clustered image using ICA based meod and Fig. 8 illustrated a reference map which has been clustered manually. The land cover considered contains various kinds of soil which are different in color. Two clusters are distinguished by e algorim as open space, which are shown in light and dark brown in Fig. 7. In confusion matrix and manually clustered image eses two regions are considered togeer as a single cluster. For evaluating e results all data points are compared wi e reference map. The comparative result is shown in table V. Comparing e classified image wi e reference map and also from e available ground tru, it is evident at ICA has indicated its capability in clustering images wi unknown cluster numbers. Fig.. Data set1; clustered data1 TABLE I: CONFUSION MATRIX FOR RESULTS OF FIG. TABLE II: CONFUSION MATRIX FOR RESULTS OF FIG. 3 Cluster Cluster Cluster Cluster Cluster1: 100% Cluster1: 100% Cluster: 91.30% Cluster: 87.5% Cluster3: 100% Cluster3: 100% Cluster4:88.57% Cluster4: 93.00% Overall accuracy: 95.68% Fig. 4. Data set3; clustered data3 TABLE III: CONFUSION MATRIX FOR RESULTS OF FIG. 4 Cluster Cluster Cluster Cluster Cluster1: 91.95% Cluster1: 95.3% Cluster: 96.36% Cluster: 93.80% Cluster3: 84.35% Cluster3: 89.85% Cluster4: 9.55% Cluster4: 88.3% Overall accuracy: 90.97% Cluster Cluster Cluster Cluster Cluster1: 99.0% Cluster1: 97.14% Cluster: 100% Cluster: 100% Cluster3: 98.9% Cluster3: 99.4% Cluster4: 100% Cluster4: 100% Overall accuracy: 99.51% Fig. 5. Data set4; clustered data4 TABLE IV: CONFUSION MATRIX FOR RESULTS OF FIG. 5 Fig. 3. Data set; clustered data Cluster Cluster Cluster Cluster Cluster1: 95.33% Cluster1: 93.40% Cluster: 100% Cluster: 100% Cluster3: 88.88% Cluster3: 90.56% Cluster4: 94.68% Cluster4: 95.69% Overall accuracy: 95.53% 141

6 C. Comparing e Performance of ICA, GA and K-Means Clustering Algorims In is section e experimental results comparing e ICA-based clustering algorim wi e GA and K-means clustering algorims are provided for IKONOS image. According to e different clusters at can be specified in IKONOS image (Fig. 6), e value of K is chosen to be 5 and 6 in K-means clustering algorim. In GA e initial number of population, mutation and selection rates are set to 300, 30% and 40% respectively and e number of clusters is unknown (as in ICA). The termination criterion for all ree algorims is set to reaching a maximum number of iterations or having no improvement in final results for several consecutive iterations (in ICA, countries converge to a state which ere exists only one empire at possesses all oer countries). Fig. 9 shows e clustered image of implementation GA based clustering algorim. As shown in is figure, 6 clusters are detected by GA-clustering algorim (vegetation, water, road, habitation and clusters for soil). So e variety and number of optimal clusters, determined by GA is e same as ICA. Fig. 9 and Fig. 9(c) depict e results of K-means clustering algorim wi 5 and 6 classes as priori information. As it derived from e results when e number of initial clusters is set to 5, K-means combines habitation, road and a part of soil classes and clusters em as a single class. Likewise, when e number of initial classes is 6, is algorim clusters road and habitation classes as a single class and classify soil into 3 classes. In fact, in is case K-means clustering algorim is unable to provide meaningful clusters. Table VI shows e comparative results of ree clustering algorims considered in is paper. This is clear at, e user's and producer's accuracy of K-means clustering algorim cannot be calculated because of not meaningful classes formed by is meod (combining road, soil and habitation classes). Regarding table VI, it can be concluded at evolutionary algorims can successfully classify data wi unknown clusters and close RGB properties and ey bo perform better an K-means clustering algorim. Also, comparison between ICA and GA indicates at ICA excels GA. Fig. 9. Reference map clustered manually TABLE V: CONFUSION MATRIX FOR RESULTS OF FIG. 7 Reference map soil Vegetation road water habitation soil Vegetation road water habitation Soil: Soil: Vegetation: Vegetation: Road: Road: 4.53 Water: Water: Habitation: Habitation: Overall accuracy: % (c) Fig. 9. clustered image using GA, K-means(5 clusters), (c) K-means clustering (5 clusters) TABLE VI: COMPARISON THE PERFORMANCE OF ICA, K-MEANS(6-CLASS) AND GA INTEMRS OF ACCURACY Clusterin g algorim ICA User's accuracy Roa Soil Veg d Wate r Hab Producer's accuracy Roa Wate Soil Veg d r Hab overall accurac y K-means GA Fig. 6. IKONOS image of Tehran, in RGB bands Fig. 7. Clustered image using e ICA based meod V. CONCLUSION The paper employed e imperialist competitive algorim which is a numerical optimization algorim inspired from socio-political concepts in automatic image clustering. This algorim was used to find e optimal cluster numbers and best cluster centers. The effectiveness of e combination of ICA and two stages fuzzy cluster analysis is demonstrated by e example of pan-sharpened IKONOS imagery and 4 artificial data sets wi different overlapping properties. By using e proposed meod, it was shown at e fuzzy hierarchical ICA based algorim was able to determine e number of optimal clusters in data sets wi various 14

7 properties and clustering em wi a high accuracy in comparison wi two of most common clustering algorims. However, e effect of scene type on automated image clustering needs to be furer investigated. REFERENCES [1] S. Guha, R. Rastogu, and K. shim, An efficient clustering for large data base, Proceeding of e ACM SIGMOD International conference on Management of Data, pp.73-84, [] T. M. Lillesand and R. W. Kiefer, Remote sensing and image interpretation, John Wiley and Sons, New York, 74p, 000. [3] J. Zhang and G. M. Foody, Full fuzzy supervised classification of sub-urban land cover from remotely sensed imagery: statistical neural network approaches, International Journal of Remote Sensing, Vol., pp. 615, 001. [4] F. Rolauf, Representations for genentic and evolutionary algorims, Springer, Neerlands, 314p, 006. [5] Gautam Garai, B.B.Chaudhuri, A novel genetic algorim for automatic clustering, Pattern Recognition Letters. Vol. 5, pp , 004. [6] E. Atashpaz-Gargari and C. Lucas, Imperialist competitive algorim: An algorim for optimization inspired by imperialistic competition, IEEE Congress on Evolutionary Computation, Singapore, 007. [7] R. Raabioun, F. Hashemzadeh, E. atashpaz-gargari, B. Mesgari, and F. R. Salmasi, Decentralized PID controller design for a MIMO evaporator based on colonial competitive algorim, in 17 IFAC World congress, Seoul, Korea, July 6-11, 008. [8] E. Atashpaz-Gargari, F. Hashemzadeh, and C. Lucas, Evolutionary design of PID controller for a MIMO distillation column using colonial competitive algorim, international ournal of intellegent computing and cybernetics, vol. 1, no. 3, pp , 008. [9] Biabangard-Oskouyi, E. Atashpaz-Gargari, N. Soltani, and C. Lucas. Application of imperialist competitive algorim for materials property characterization from sharp indentation test, International Journal of Engineering Simulation, 008. [10] A. Khhabbazi, E. atashpaz, and caro lucas, Imperialist competitive algorim for minimum bit error rate beamforming, internatilonal ournal of Bio-Inspired computation, vol. 1, pp , 009. [11] E. Atashpaz-Gargari and C.Lucas, Colonial aompetitive algorim as a tool for nash equilibrium point achievement, Lecture notes in computer science, Springer Berlin, vol. 5073, pp , 008. [1] E. J. Haic and L. R. Tinney, Analysis of visible and ermal Infrared data, Manual of RS, vol..nd ed., American Society of Photogrammetry, falls church, VA S. Karami was born in Tehran, Iran, March 0, 198. She received her BS degree in Electrical Engineering from Amirkabir University of Technology (Tehran Polytechnic), Tehran, Iran in 006 and e MS degree in Electrical engineering from Iran University of Science and Technology (IUST), Tehran, Iran in 009. Her research interests are image processing, machine vision, pattern recognition and evolutionary computation. M.s Karami is e corresponding auor and can be contacted at:karami_samaneh@yahoo.com S. B. Shokouhi received his B.S. and M.S. degrees in Electronics in 1986 and 1989, bo from e Department of Electrical Engineering of Iran University of Science and Technology (IUST). He received his Ph.D. on Medical Imaging and Image Processing in 1999 from e School of Electronic and Electrical Engineering, University of Ba, England. He is currently an assistant professor of Electrical Engineering Department at IUST. His research interests include Machine Vision Algorims and Hardware Implementations, Pattern Recognition, and Intelligent Systems Design. Dr. Shokouhi is member of some international scientific societies such as IEEE, IEICE and also Iranian Machine Vision and Image Processing (MVIP). 143