Comparative Analysis of Unsupervised and Supervised Image Classification Techniques

Transcription

1 ational Conference on Recent Trends in Engineering & Technology Comparative Analysis of Unsupervised and Supervised Image Classification Techniques Sunayana G. Domadia Dr.Tanish Zaveri Assistant Professor Professor EC Department EC Department Ins. of Comp. & Comm. Tech. for Women irma University ew V.V. agar Ahmedabad Abstract-- Image classification techniques are used to classify different features available in the image. The obective of image classification is to identify the features occurring in an image in terms of the obect. Image classification are applied in many areas such as medical imaging, obect identification in satellite images, traffic control systems, brake light detection, machine vision, etc. Image classification techniques are mainly divided in two categories: supervised image classification techniques and unsupervised image classification techniques. In this paper different supervised and unsupervised image classification techniques are implemented, analyzed and comparison in terms of accuracy & time to classify for each algorithm are also given. II. USUPERVISED IMAGE CLASSIFICATIO TECHIQUES A. K-means Clustering The basic step of K-means clustering is given in figure []. To initialize we determine number of cluster k and we assume center of this cluster. We can take any random obect as initial centroid. This algorithm will do three steps given below until convergence which is shown in Figure. Iterate until stable Determine the centroids coordinate Determine the distance of each obect to the centroids Group the obect based on minimum distance Keywords-- k-means algorithm, EM algorithm, A, Maximum likelihood, Confusion matrix. A I. ITRODUCTIO broad group of digital image processing techniques is directed towards image classification which is done by the automated grouping of pixels into specified categories []. Image classification is mainly divided into two categories () supervised image classification and () unsupervised image classification. In supervised image classification training stage is required, which means first we need to select some pixels form each class called training pixels. Find the characteristics of training pixels and also find other pixels which have same characteristics, this way image classification can be done. In unsupervised image classification, no training stage is required, but different algorithms are used for clustering. In real world, sometimes image does not have much information about data. So, in this case we can use unsupervised image classification as here information is not required before classification, unlike supervised classification. umerous factors affect the classification results, among which important ones being the obective of classification, the spectral and spatial characteristics of the data, the natural variability of terrain conditions in geographic region, and the digital classification technique employed [3]. The success of an image classification depends on many factors likes availability of high-quality remotely sensed imagery and ancillary data, the design of a proper classification procedure, and the analysts skills and experiences are the most important ones. Fig. K means algorithm The criterion for ending the iterative process can be defined in terms of one iteration to the next. Specifically, this is the magnitude change of the mean from iteration i- to iteration i summed over all K clusters. K i i k k () k ( i) B. Expectation-Maximization (EM) Algorithm It is standard method to fit finite mixture models in to observed data. EM is an iterative procedure which converges to a (local) maximum of the marginal a posteriori probability function [4]. P( / x) P( x / ) P( ) () 3-4 May 0 B.V.M. Engineering College, V.V.agar,Guarat,India

2 ational Conference on Recent Trends in Engineering & Technology Where θ is a set of unknown parameters from x.[5] EM is a general method of estimating the features of a given data set, when the data are incomplete or have missing values. Being an iterative procedure, the EM method can present high computational cost. EM algorithm implement by two types of distribution: Univariate normal distribution Multivariate normal distribution For univariate case, EM algorithm uses histogram. In this find image histogram such that the data will become one dimensional. After which x is considered as an index of histogram value of each level. For multivariate case (standard EM), x is 3D space vector of image and randomizes the initial parameter (mean and variance) or parameter obtained by of kmeans algorithm. Steps of EM algorithm are shown in Figure.[5] (t ) k i (t ) k P (t ) P (C / X k ) X k P (C / X k ) k P(C / X k )( X (t ))( X (t ))T P (C / X k ) k (6) (7) P (C k / Xk ) (8) These steps are performed until convergence is reached according the following Equation (t ) (t ) (9) Where. in this implementation, is the Euclidean distance between the vectors µ(t + ) and µ (t), and ε is a threshold chosen by the user. After the calculations, Equation 4 is used to classify the image [4]. Here the final classification is performed. For each of the pixels Xk is associated the class with higher probability that is, find P(C/Xk) > P(Ci/Xk); i and classify Xk as C. III. Fig. EM algorithm Computing EM: The EM algorithm works iteratively by applying two steps:- A. Maximum Likelihood / Bayesian Classifier E-step(expectation) M-step(maximization) (t ) { (t ), (t )},,...M (3) Stands for successive parameter estimates. The method aims to approximate θ(t) to real data distribution when t = 0,,... E-step: This step calculates the conditional posteriori probability function mention in. P (C / X ) (t ) M k k GB (t ) e n P (t ) GB (4) nk e Pk (t ) Where GB = (-/) i ( X i (t ))T i (t )( X i (t )) (5) M-step: This step updates the parameter estimation θ(t) With posterior probabilities, one can now estimate the mean, covariance, and the a priori probability for each cluster, at time t May 0 SUPERVISED IMAGE CLASSIFICATIO TECHIQUES It applies the probability theory to the classification task []. A statistical decision rule that examines the probability function of a pixel for each of classes, and assign the pixel to class with the highest probability [3]. Equation for Maximum likelihood / Bayesian classifier as follows D ln(ac ) [0.5ln( Covc )] [0.5( X Mc )T (Covc )( X Mc )] (0) Where D = likelihood, c = a particular class, X = measurement vector of candidate pixel, Mc = the mean vector of sample of class c, ac = percent probability that any candidate pixel is a member of class c, Covc = the covariance matrix of the pixels in sample of class c, Covc = determinant of Covc, C-ovc = inverse of Covc, Find likelihood for each pixel for each class. Pixel goes to class which has highest likelihood for this pixel, this way classification will be performed B.V.M. Engineering College, V.V.agar,Guarat,India

3 ational Conference on Recent Trends in Engineering & Technology 3 B. Artificial eural etwork Artificial eural etwork (A) is a parallel distributed processor that has a natural tendency for storing experiential knowledge [6]. ) Steps of A for image classification: Take training pixels from each class in Image and its corrected (desired) output d. Let X be input vector of pixel,also set all desired outputs d0,d... typically to zero except for that corresponding to the class the input is from, Training the network using Back propagation algorithm The following assumes the sigmoid function f(x) () e f x x Where W h is weights between hidden nodes and output nodes, S is weighting sum coming into output node (5) Y S e 3. Calculate error term for each output unit Y ( Y )( d Y ) (6) 4. Mean square error(mse) of output node ( ) d i Y e (7) 5. Calculate error term of each of hidden nodes ( W ) Y ( Y ) (8) h h h h h 6. Adust weights to minimize mean square error W W X ( W W ) (9) ih ih h ih ih W W Y ( W W ) (0) h h h h h All the steps excepting are repeated till MSE is within reasonable limits Fig. 3 Block diagram of Artificial eural etwork The back propagation algorithm is implemented using following steps.[7] 7. After training neural network using Training pixels, find Y h and Y for each pixel using weights W h, W ih which is obtain from training of neural network. 8. Pixel goes in Y class if Y have maximum probability for this pixel. According to this all pixels of image are classified.. Initialize weights to small random values W ih,w h.. feed input vector X,X...through etwork and computing weighting sum coming into node and then apply the sigmoid function S ( W ) T X h () ih Where W ih is weights between input nodes and hidden nodes, S h is weighting sum coming into hidden node. (3) Y h S h e Where Y h is probability of each hidden node for each pixel S ( W ) T Y (4) h h IV. SIMULATIO AD RESULTS Two image (.SAR image,.simple digital image) and its ground truth has been taken for simulation purpose. On each image, different unsupervised and supervised algorithms have been applied, then find the confusion matrix and overall accuracy from classification results Confusion Matrix shows the accuracy of a classification result by comparing a classification result with ground truth information. All diagonal elements in confusion matrix gives percentage of corrected classified pixels means accuracy of each class and all other elements gives percentage of misclassification for each class. Figure 4, 6 shows images which are taken for classification Figure 5, 7 shows graphs for accuracy of each class for each algorithm. Correct classification represent by circle or ellipse and misclassification represent by rectangular or square for each classified image. 3-4 May 0 B.V.M. Engineering College, V.V.agar,Guarat,India

4 ational Conference on Recent Trends in Engineering & Technology 4 Table IV COFUSIO MATRIX FOR MAXIMUM LIKELIHOOD CLASSIFIER Farmland Wetland % Woodland Water Table V COFUSIO MATRIX FOR EURAL ETWORK Farmland Wetland % Woodland Water Fig. 4 (a) SAR image, classification result by (b) eural etwork, (c) K-means algorithm, (d) EM algorithm (univariate distribution), (e) EM algorithm (multivariate distribution), (f) Maximum likelihood classifier Using Matlab 7.7 on Intel(R) Core(TM) Duo GHz PC, time to classify of image for K-means algorithm, EM algorithm (univariate distribution), EM algorithm (multivariate distribution), Maximum likelihood classifier, eural etwork, are.8sec, 3.8sec, 990sec, 3.3sec, sec respectively. A. Confusion Matrix Table I COFUSIO MATRIX FOR K-MEAS ALGORITHM Fig. 5. Accuracy of each class and each algorithm Farmland Wetland % Woodland Water Table II COFUSIO MATRIX FOR EM ALGORITHM (UIVARIATE DISTRIBUTIO) Farmland Wetland % Woodland Water Table III COFUSIO MATRIX FOR EM ALGORITHM (MULTIVARIATE DISTRIBUTIO) Farmland Wetland % Woodland Water Fig. 6. (a) Original image, classification result by (b) eural network, (c) K-means algorithm, (d) EM algorithm (univariate distribution), ( e) EM algorithm (multivariate distribution), (f) Maximum likelihood classifier 3-4 May 0 B.V.M. Engineering College, V.V.agar,Guarat,India

5 ational Conference on Recent Trends in Engineering & Technology 5 Time to classify of image for K-means algorithm, EM algorithm (univariate distribution), EM algorithm (multivariate distribution), Maximum likelihood classifier, eural etwork are.8sec, 3.8sec, 300sec, 3.4sec, 80.57sec respectively. B. Confusion Matrix Table VI COFUSIO MATRIX FOR K-MEAS ALGORITHM Water % Bear Table VII COFUSIO MATRIX FOR EM ALGORITHM (UIVARIATE DISTRIBUTIO) Water % Bear Table VIII COFUSIO MATRIX FOR EM ALGORITHM (MULTIVARIATE DISTRIBUTIO) Water % Bear Table XI COFUSIO MATRIX FOR MAXIMUM LIKELIHOOD CLASSIFIER Water % Bear Table X COFUSIO MATRIX FOR EURAL ETWORK Water % Bear Generally probabilistic approach gives better image classification results compare to minimum distance approach. Mixture of Gaussians is a good model for extracting image features instead of the Means so K-means do not give good accuracy compare to EM (multivariate normal distribution) algorithm which has high computational cost. EM (univariate normal distribution) use histogram so its not give good accuracy. Maximum likelihood classifier based on probabilistic approach and also supervised image classification algorithm so it gives better accuracy compare to EM (multivariate normal distribution) within short time period. eural network use more feature of image so it gives better accuracy compare to all algorithms. eural network is also useful for multiple databases, once it is trained for it. V. COCLUSIO The different mage Classification methods like k-means algorithm, EM algorithm, Maximum likelihood, Artificial eural network have been described and implemented. Kmeans algorithm based on a minimum distance whereas other algorithms based on probability distribution. According to simulation results it can be stated that among all unsupervised algorithms and supervised algorithms, EM algorithm (multivariate distribution) and artificial neural network respectively lend good results. But EM algorithm (multivariate distribution) takes time to classify image due to the calculations of inverse covariance matrix and determinant at each iteration for whole set of data. REFERECES [] Tong Hau Lee, Mohammad Faizal, Ahmad Fauzi and Ryoichi Komiya Segmentation of CT Brain Images Using K-means and EM Clustering. 5 th IEEE International Conference on Computer Graphics, Imaging and Visualization, 008. [] B.Bhatta Remote sensing and GIS. Oxford University Press [3] Yousif Ali Hussin Remotely Sensed Image Classification. Department of atural Resources, ITC [4] Tatsuya Yamazaki Introduction of EM Algorithm into Color Image Segmentation. ATR Adaptive Communications Research Laboratories, 998. [5] Thales Sehn Korting and Luciano Vieira Dutra Improvements to Expectation-Maximization approach for unsupervised classification of remote sensing data. ational Institute for Space Research, 007. [6] M.Seetha, I.V. MuraliKrishna and B.L. Deekshatulu Comparison of Advanced Techniques of Image Classification. 00. [7] Prof S K Shah and Fellow V Gandhi Image Classification Based on Textural Features using Artificial eural etwork Fig. 7. Accuracy of each class and each algorithm 3-4 May 0 B.V.M. Engineering College, V.V.agar,Guarat,India