Object Recognition using Particle Swarm Optimization on Fourier Descriptors

Transcription

1 Object Recognition using Particle Swarm Optimization on Fourier Descriptors Muhammad Sarfraz 1 and Ali Taleb Ali Al-Awami 2 1 Department of Information and Computer Science, King Fahd University of Petroleum and Minerals, Dhahran 31261, Saudi Arabia sarfraz@kfupm.edu.sa 2 Department of Electrical Engineering, King Fahd University of Petroleum and Minerals, Dhahran, Saudi Arabia. aliawami@kfupm.edu.sa Abstract. This work presents study and experimentation for object recognition when isolated objects are under discussion. The circumstances of similarity transformations, presence of noise, and occlusion have been included as the part of the study. For simplicity, instead of objects, outlines of the objects have been used for the whole process of the recognition. Fourier Descriptors have been used as features of the objects. From the analysis and results using Fourier Descriptors, the following questions arise: What is the optimum number of descriptors to be used? Are these descriptors of equal importance? To answer these questions, the problem of selecting the best descriptors has been formulated as an optimization problem. Particle Swarm Optimization technique has been mapped and used successfully to have an object recognition system using minimal number of Fourier Descriptors. The proposed method assigns, for each of these descriptors, a weighting factor that reflects the relative importance of that descriptor. 1 Introduction Fourier descriptors [1, 2, 14], like Moment descriptors [9], have been frequently used as features for image processing, remote sensing, shape recognition and classification. Fourier Descriptors can provide characteristics of an object that uniquely represent its shape. Several techniques have been developed that derive invariant features from Fourier Descriptors for object recognition and representation [1-5, 14]. These techniques are distinguished by their definition, such as the type of data exploited and the method for deriving invariant values from the image Fourier Descriptors. Granlund [1] introduced Fourier descriptors using complex representation in This method ensures that a closed curve will correspond to any set of descriptors. The Fourier descriptors have useful properties [3, 4]. They are invariant under similarity transformations like translation, scaling and rotation. The objects having these kind of

2 transformations can be easily recognized using some recognition algorithms with Fourier descriptors as invariant features. For example, the Fourier descriptors, of the boundary [11-13], for recognizing closed contours is proposed in [5]. However, despite its success in some applications, it has certain limitations. Occlusion is the severe shape distortion, when the shape gets distorted, the Fourier descriptors don t work well for recognition [6-8]. This paper has used Fourier descriptors, with different combinations, for the recognition of objects captured by an imaging system which may transform, make noise or can have occlusion in the images. An extensive experimental study, similar to the moment invariants [9], has been made using various similarity measures in the process of recognition. These measures include Euclidean Measure and Percentage error. Comparative study of various cases has provided very interesting observations which may be quite useful for the researchers as well as practitioners working for imaging and computer vision problem solving. Although the whole study has been made for bitmap images, but it can be easily extended to gray level images. From the analysis and results using Fourier Descriptors, the following questions arise: What is the optimum number of descriptors to be used? Are these descriptors of equal importance? To answer these questions, the problem of selecting the best descriptors has been formulated as an optimization problem. Particle Swarm Optimization technique has been mapped and used successfully to have an object recognition system using minimal number of Fourier Descriptors. The goal of the proposed optimization technique is to select the most helpful descriptors that will maximize the recognition rate. The proposed method will assign, for each of these descriptors, a weighting factor that reflects the relative importance of that descriptor. The outline of the remainder of the paper is as follows. Getting of bitmap images and their outline is discussed in Sections 2 and 3 respectively. Section 4 deals with the study of Fourier descriptors. The concepts of similarity measures are explained in Section 5. Algorithm for object recognition problem has been devised in Section 6. Detailed experimental study and analyses are made in Section 7 whereas Section 8 deals with interesting observations during the experimental study. Finally, Section 9 concludes the paper as well as touches some future work. 2 Fourier Theory To characterize objects we use features that remain invariant to translation, rotation and small modification of the object s aspect. The invariant Fourier descriptors of the boundary [11-13] of the object can be used to identify an input shape, independent on the position or size of the shape in the image. Fourier transform theory has played a major role in image processing for many years. It is a commonly used tool in all types of signal processing and is defined both for one and two-dimensional functions. In the scope of this research, the Fourier transform technique is used for shape description in the form of Fourier descriptors. The shape descriptors generated from the Fourier coefficients numerically describe shapes and are normalized to make them independent of translation, scale and rotation.

3 The Fourier transform theory can be applied in different ways for shape description. In this research, the procedure has been implemented in such a way that the boundary of the image is treated as lying in the complex plane. So the row and column co-ordinates of each point on the boundary can be expressed as a complex number. For details, the reader is referred to [5, 14]. Input shape F-16 B-747 M Database of Fourier Descriptors Contour shape Classifier Answer Figure 1. Pictorial Description of the method. 3 Similarity Measures This paper implements two different simple classifiers that calculate different similarity measures of the corresponding Fourier descriptors of the input shape and each of the shapes contained in the database as shown in Figure 1. The similarity measures, attempted for experimental studies, are Euclidean Distance (ED) and Percentage Error (PE). 4 Results and Analysis The recognition system is tested by generating the test objects by translating, rotating, scaling, adding noise, and adding occlusion to the model objects contained in a database of different sizes. The test objects were randomly rotated, translated, and scaled. Some were considered without scale of their model sizes. About 100 test objects were used for each of the experiments for testing similarity transformation. The salt & pepper noise [15-16] of different densities is added to the objects for generating the noisy test objects. Median filter was used in the experiment to filter the noise, so that the noise remains on the boundary of the object. Median filtering is a type of neighborhood processing that is particularly useful for removing 'salt and pepper' noise from an image. The median filter [15-17] considers each pixel in the image and it looks at its nearby neighbors to decide whether or not it is representative of its surroundings. Instead of simply replacing the pixel value with the mean of

4 neighboring pixel values, it replaces it with the median of those values. The median is calculated by first sorting all the pixel values from the surrounding neighborhood into numerical order and then replacing the pixel being considered with the middle pixel value. As would be seen in the experiments, FDs are not promising for the recognition of occluded objects. Around 20% occlusion was added into the objects to make tests. We split the experiments into different categories explained in the following paragraphs. The first series of experiments has been made to view results for different combinations of the Fourier Descriptors. Various experiments can be seen presenting different scenarios of the combination of Fourier Descriptors, similarity measures, and nature of data used. The recognition system is tested by generating the test objects by translating, rotating, and scaling and adding noise to the model objects contained in a database of size 60. The test objects were randomly rotated, scaled and translated. Sixty test objects were used for each of the experiments for testing similarity transformation, 16 test objects were used for noisy objects with similarity transformations, and 60 test objects were used for occluded objects. The salt & pepper noise of density 10% is added to the objects for generating the noisy test objects. Median filter was used in the experiment to filter the noise, so that the noise remains on the boundary of the object. The procedures taken to analyze and test the system are as follows: 1. The base case: That is, the Fourier descriptors FD 1-6 (highlighted in Table 1) are used as features and the Euclidean distance is considered for comparison. The percentage of recognition recorded in case of just similarity transformations is 83.3%. In case of similarity transformations with noise, it is about 93.75%. It is worth noting that in the latter case, only translation is considered as a similarity transformation, i.e., the test images are not rotated or scaled. The recognition rate of occluded objects is only 8.3%, which is very low. The Fourier Descriptors for an object in case of similarity transformations, noise and occlusion are experimented. It was found, by computation, that the Fourier descriptors do not change much in case of similarity transformations and noise. However, the occlusion caused the change in the values of descriptors. Table 1: Recognition rates for different numbers of Fourier descriptors using Euclidean distance. Number FDs Used of 4 6 Base Case % 83.33% 93.33% 90% 93.33% 95% Transformations Noise 75% 93.75% 93.75% 93.75% 93.75% 93.75% Occlusion 5% 8.33% 20% 18.33% 23.33% 23.33% 2. Fourier Descriptors and Euclidean distance: Experiments are made to obtain the recognition rates of transformed, noisy, or occluded images considering differ-

5 ent numbers of FDs using Euclidean distance. The recognition rates for using different numbers of FDs, ranging from 1 to 40, are computed. Some sample results are tabulated in Table 1. For example, using 11 FDs improves the recognition rate of transformed images to 93.33% and improves the recognition rate of occluded images to 20%. From these results, it can be concluded that a good compromise between recognition performance and computational cost is achieved using 11 FDs. That is, increasing the number of FDs beyond 11 does not help much as the maximum recognition rate obtained for transformed images using up to 40 FDs is 95%. Another observation is that the maximum recognition rate is achieved by using 29 FDs. Thus, using more FDs does not improve the recognition performance further. Table 2: Recognition rates for different numbers of Fourier descriptors using percentage of errors. Number of FDs Used 4 6 Base Case Transformations 70% 80% 86.67% 75% 68.33% 68.33% Noise 87.5% 81.25% 81.25% 81.25% 81.25% 81.25% Occlusion 8.33% 11.67% 13.33% 8.33% 6.67% 11.67% 3. Fourier Descriptors and percentage of errors: Similarly, the recognition rates of transformed, noisy, or occluded images using the sum of percentage of error (PE) have been obtained. The recognition rates for using different numbers of FDs, ranging from 1 to 40, are summarized. Some of these numerical results are tabulated in Table 2. It can be seen that, using PE with FDs results in less efficient performance than using ED. Moreover, increasing the number of FDs does not necessarily guarantee a better performance. From Table 2, it can be observed that the best recognition rate of transformed images is achieved using 9 FDs. However, larger number of FDs gives rise to a lower performance. 5 Optimization of the Feature Vector using PSO The problem of selecting the best descriptors can be formulated as an optimization problem. The goal of the optimization is to select the most helpful descriptors that will maximize the recognition rate and assign for each of these descriptors a weighting factor that reflects the relative importance of that descriptor. Since the problem of selecting the best descriptors can be formulated as an optimization problem, one need to define an objective function. The objective function, in this case, is made up of the following two terms: the recognition rate, the number of useful descriptors.

6 In other words, it is required to maximize the recognition rate using the minimum number of descriptors. The function of the optimization algorithm is to assign a weight w i for every descriptor so that the objective function is optimized, where w i belongs to [0,1]. The mathematical formulation of the objective function is defined as follows: J = H + α *min( PE), where H is the number of hits (the number of correct matches), PE is the percentage of errors of all the training images for a given set of weights, α is a factor that is adjusted according to the relative importance of the min(pe) term. In most of the simulations, α = 0.7 is experienced as best case. The first term makes PSO search for the weights that result in the highest recognition rate and the second term makes the PSO reach to the highest recognition with the minimum number of descriptors Particle Swarm Optimization A novel population based optimization approach, called Particle Swarm Optimization (PSO) approach, has been used. PSO was introduced first in 1995 by Eberhart and Kennedy [18]. This new approach features many advantages; it is simple, fast and can be coded in few lines. Also, its storage requirement is minimal. Moreover, this approach is advantageous over evolutionary algorithms in more than one way. First, PSO has memory. That is, every particle remembers its best solution (local best) as well as the group best solution (global best). Another advantage of PSO is that the initial population of the PSO is maintained, and so there is no need for applying operators to the population, a process which is time- and memory-storageconsuming. In addition, PSO is based on constructive cooperation between particles, in contrast with the other artificial algorithms which are based on the survival of the fittest. [19] PSO starts with a population of random solutions. Each particle keeps track of its coordinates in hyperspace which are associated with the fittest solution it has achieved so far. The value of the fitness (pbest) is stored. Another best value is also tracked, which is the global best value. The global version of the PSO keeps track of the overall best value, and its location, obtained thus far by any particle in the population, which is called (gbest). PSO consists of, at each step, changing the velocity of each particle toward its pbest and gbest. Acceleration is weighted by a random term, with separate random numbers being generated for acceleration toward pbest and gbest. Several modifications have been proposed in the literature to improve the PSO algorithm speed and convergence toward the global minimum. One of the most efficient PSO versions uses a time decreasing inertia weight, which leads to a reduction in the number of iterations. The performance of this modified algorithm depends on the method of tuning the inertia weight. The most straightforward way to adjust the inertia weight is to force it to linearly decrease with the number of iterations [20]. As

7 an alternative to inertia weight technique, constriction factors have been proposed [21]. Constriction-factor-based PSO has been proved to be superior upon inertiaweight-based PSO in terms of the number of iterations to reach the best solution. PSO starts with a population of random solutions particles in a D-dimension space. The i th particle is represented by X i = ( x1i, xi2,..., xid ). Each particle keeps track of its coordinates in hyperspace which are associated with the fittest solution it has achieved so far. The value of the fitness for particle i (pbest) is also stored as P i = ( p1i, pi2,..., pid ). The global version of the PSO keeps track of the overall best value (gbest), and its location, obtained thus far by any particle in the population. PSO consists of, at each step, changing the velocity of each particle toward its pbest and gbest according to equation (4). The velocity of particle i is represented as V i = ( v1i, vi2,..., vid ). Acceleration is weighted by a random term w, with separate random numbers being generated for acceleration toward pbest and gbest. The position of the i th particle is then updated according to equation (5). vid = w* vid + c1 * Rand() *( pid xid ) + c1 * Rand() * ( pgd xid ) (4) x id = xid + vid (5) where p id = pbest and p gd = gbest A simplified method of incorporating a constriction factor is represented in equation (6), where K is a function of c 1 and c 2 as illustrated by equation (7). Eberhart and Shi [21] experimented the performance of PSO using an inertia weight as compared with PSO performance using a constriction factor. They concluded that the best approach is to use a constriction factor while limiting the maximum velocity V max to the dynamic range of the variable X max on each dimension. They showed that this approach provides performance superior to any other published results. ( ) ( ) ( ) ( ) vid = K * v id + c1* Rand * pid xid + c1* Rand * pgd x id, (6) kk K = ; kk = 2, φ = c 4 2 φ ϕ c2, φ >. (7) 4φ 5.2. Test Results using PSO The PSO algorithm can be found in the current literature at many places. Therefore, it is not necessary to describe the steps of the PSO algorithm here. However, the reader is referred to [18-19] if interested to see the PSO algorithm details. The proposed PSO based approach was implemented using a MATLAB library built by the authors. In our implementation, the inertia weight w is linearly decreasing from 0.9 to 0.4, c 1 and c 2 are selected as 2, a population of 20 particles is used, and the maximum number of iterations is 400. The search process stops whenever the maximum number of iterations is exceeded. Several experiments have been attempted to use PSO to search for the optimum descriptor weights. These experiments are summarized in Tables 3 and 4. In these

8 tables, No. of FDs considered means the number of Fourier descriptors considered in the optimization process. For example, if this number is F, the PSO is supposed to search for F weights, a single weight for a single FD, that maximize the recognition rate with the minimum number of descriptors. Hence, if only f FDs can do the job, PSO will assign nonzero weights for f FDs and it will assign zero weights for F-f FDs. As a result, the number of descriptors used in the recognition process is only f out of F. Table 3. Optimized weights for different numbers of Fourier descriptors. Experiment No. Training set * X X X O X, O, N No. of FDs Considered Optimized Weights obtained Table 4. Total number of optimized weights used for different numbers of Fourier descriptors and the recognition rate. Experiment No Training set * X X X O X, O, N No. of FDs Considered No. of FDs Used Recognition Rate X 93.33% 95% 95% 90% 98.33% N 93.75% 93.75% % 87.5% O 25% % 20% 25% * X = transformed objects, O = occluded objects, N = noisy objects Table 3 demonstrates the computed optimized weights for different numbers of Fourier descriptors. Table 4 displays about the total number of optimized weights used for different numbers of Fourier descriptors and the recognition rate achieved. In

9 the first experiment when a database of 60 transformed objects, in Table 3 and 4, was considered, one can see a much better recognition results than using even 7 unweighted FDs in Table 2. In the second experiment, the results are better than using 11 un-weighted FDs. One can see, in Table 2, a similar recognition rate is obtained using 29 un-weighted FDs. Experiment 3, when considered for 6FDs, shows generally, much better results than using 6 un-weighted FDs. However, in experiment 4, using occluded images only in the training, does not help improve the recognition rate of occluded images. Experiment 5 for a mixed data set of transformed, noisy and occluded objects has produced wonderful results of 98.33% recognition rate. Such an achievement has not been obtained by any number of un-weighted FDs. 6 Conclusion and Future Work This work has been reported to make a practical study of the Fourier descriptors to the application of Object Recognition. The implementation was done on a P-IV PC using MATLAB. The ultimate results have variations depending upon the selection of number of FDs, similarity transformations, noise, occlusion, and data size. The variety of similarity measures and different combinations of FD features, used in the process, make a difference to the recognition rate. The results have been tested using up to 40 FDs, and different size of databases. Different combinations of these parameters implied different results. Two similarity measures, including ED, and PE, provided different recognition results. The images used are all bitmapped images, further investigations are being done with some more complex images. It can be seen that, using PE with FDs results in less efficient performance than using ED. Moreover, increasing the number of FDs does not necessarily guarantee a better performance. The images that have to be recognized but failed to be recognized by most of the FD combinations are to be analyzed further. This leads to the theory of optimization to find out appropriate features or attributes in the image that made it difficult to be recognized. The methodology of PSO has been utilized successfully for this purpose. Using PSO, to find the most suitable descriptors and to assign weights for these descriptors, improved dramatically the recognition rate using the least number of descriptors. In future, the authors would like to treat the problem as multi-objective optimization method. They would like to maximize the recognition rate while minimizing the number of descriptors. Acknowledgments The authors are thankful to the anonymous referees for the valuable suggestions towards the improvement of this manuscript. This work has been supported by the King Fahd University of Petroleum and Minerals.

10 References 1. G. H. Granlund, Fourier Preprocessing for hand print character recognition, IEEE Trans. Computers, Vol C-21, Febr. 1972, pp A Project led by Julien Boeuf and Pascal Belin, and supervised by Henri Maître: 3. O. Betrand, R. Queval, H. Maître, Shape Interpolation by Fourier Descriptors with Application to Animation Graphics, Signal Processing, June 1981, 4: H. Maître, Le traitement des images, ENST, December 2000, pp C.T. Zahn, R.Z. Rhoskies, Fourier descriptors for plane closed curves, IEEE trans. Compu. 21 (1972) Thomas Bernier, Jacques-Andre landry, A new method for representing and matching shapes of natural objects, Pattern Recognition 36 (2003), N. Ansari, E.J. Delp, Partial Shape Recognition: a landmark based approach, IEEE Trans. PAMI 12 (1990), J. Zhang, X. Zhang, H. Krim, G.G. Walter, Object representation and recognition in shape spaces, Pattern Recognition 36(5), 2003, pp M. Sarfraz, Object Recognition using Moments: Object Recognition using Moments: Some Experiments and Observations: Geometric Modeling and Imaging New Advances, Sarfraz, M. and Banissi, E. (Eds.), ISBN-10: , IEEE Computer Society, USA, 2006, pp John W. Gorman, O. Robert Mitchell, Frank P. Kuhl, Partial shape recognition using dynamic programming, IEEE Transactions on pattern analysis and machine intelligence, Vol.10(2), March G. Avrahami and V. Pratt. Sub-pixel edge detection in character digitization. Raster Imaging and Digital Typography II, pp , Hou Z. J., Wei G. W., A new approach to edge detection, Pattern Recognition Vol. 35, pp , N. Richard, T. Gilbert, Extraction of Dominant Points by estimation of the contour fluctuations, Pattern Recognition Vol. 35, pp , M. Sarfraz, Object Recognition using Fourier Descriptors: Some Experiments and Observations: Computer Graphics, Imaging and Visualization Techniques and Applications, Banissi, E., Sarfraz, M., Huang, M. L., and Wu, Q. (Eds.), ISBN: , IEEE Computer Society, USA, 2006, pp Rafael Gonzalez, Richard Woods and Steven Eddins, Digital Image Processing Using Matlab, Prentice Hall, R. Jain, R. Kasturi, B. Schunk, Machine Vision, McGraw Hill, J. Kennedy, R. Eberhart, Particle swarm optimization, Proc. IEEE Intl. Conf. Neural Networks, 4, Nov/Dec 1995, pp R. Eberhart and J. Kennedy, A new optimizer using particle swarm theory, Proc. the Sixth Intl. Symposium on Micro Machine and Human Science, MHS '95, 4-6 Oct 1995, pp Y. Shi, R. Eberhart, A modified particle swarm optimizer, The 1998 IEEE Intl. Conf. on Evolutionary Computation Proc., IEEE World Congress on Computational Intelligence, 4-9 May 1998, pp R.C. Eberhart, Y. Shi, Comparing inertia weights and constriction factors in particle swarm optimization, Proceedings of the 2000 Congress on Evolutionary Computation, Vol. 1, July 2000, pp