Descriptor Combination using Firefly Algorithm

Transcription

1 Descriptor Combination using Firefly Algorithm J. Matsuoka, A. Mansano, L. Afonso, J. Papa São Paulo State University - UNESP Department of Computing {jekemi, alex.fernandesm, luis.claudio, papa}@fc.unesp.br Abstract Accuracy rate in image classification can be enhanced by combining different types of descriptors as a way to improve the training set. Many works have proposed combining different feature vectors. However, this paper proposes a technique that models the descriptor combination as an optimization problem using evolutionary-based techniques which will compute parameters that increase the separability in the feature space. Among the evolutionary-based techniques in the literature, this work introduces and employs the Firefly algorithm for such purpose which has its performance compared against other evolutionary-based techniques and are assessed by the OPF classifier. Experiments have shown that descriptor combination does improve the accuracy rate for image classification and the proposed technique outperformed single descriptors in wellknown public datasets. I. INTRODUCTION Image classification can provide important information for several applications such as automatic diagnosis in medical systems and target recognition in remote sensing images. In order to perform the classification task, different types of features can be extracted such as shape, texture and color in order to describe images for further recognition. However, each database may require a different type of feature that best fits the sorts of images in that dataset in order to achieve reasonable results, or a set of descriptors can be used and a combination can be performed in that set. Many projects have proposed to combine features by assigning weights to them [1], [2], [3], while other works solve this problem using Linear Discriminant Analysis [4] and Principal Component Analysis [5]. Faria et. al [6] proposed a descriptor combination approach defining it as a pair (D I, I), where DI is an extractor algorithm, and D I the distance function associated to it. This approach combines distance values of single descriptors, resulting in a composite descriptor. It is worth mentioning that many works have evidenced the importance of this kind of modeling, giving the metric space is as much importance as feature space. This paper proposes an extension to Faria et. al s work [6] by introducing the Firefly algorithm for descriptor combination, which has never been employed before. This paper is organized as follows: Section II introduces the concept of descriptor combination. Section III presents the optimization algorithms employed in this work and Section IV describes the OPF classifier. The proposed methodology is presented in Section V and experimental results in Section VI. Finally, Section VII states the conclusions. II. DESCRIPTOR COMBINATION Definition 1. An image Î is a pair (D I, I), where: D I Z 2 is a finite set of pixels, and I : D I D is a function that assigns to each pixel p in D I a vector I(p) of values in some arbitrary space D (for example, D = IR 3 when a color in the RGB system is assigned to a pixel). Definition 2. A simple descriptor (briefly, descriptor) D is defined as a pair (ǫ D,δ D ), where: ǫ D : Î R n is a function, which extracts a feature vector vî from an image Î. δ D : R n R n R is a similarity function that computes the similarity between two images as the inverse of the distance between their corresponding feature vectors. Definition 3. A feature vector vî of an image Î is a point in R n space: vî = (v 1,v 2,...,v n ), where n is the dimension of the vector. Figure 1 illustrates the use of a simple descriptor D to compute the similarity between two images ÎA and ÎB. First, the extraction algorithm ǫ D is used to compute the feature vectors vîa and vîb associated with the images. Next, the similarity function δ D is used to determine the similarity value d between the images. D: Fig. 1. v I^ A ε D I^A d δ D v I^ B ε D I^B Simple descriptor. Definition 4. A composite descriptor ˆD is a pair (D,δ D ), where: D = {D 1,D 2,...,D k } is a set of k pre-defined simple descriptors.

2 δ D is a similarity function which combines the similarity values obtained from each descriptor D i D, i = 1,2,...,k. Figure 2 illustrates the use a composite descriptor ˆD to compute the distance between images ÎA and ÎB. D: δ D1 d 1 d 2 d k δ D2 δ D δ Dk ε D ε 1 D ε 1 D ε 2 D... ε 2 D ε k D k Fig. 2. I ^ A d ^I B Composite descriptor. III. OPTIMIZATION ALGORITHMS In this section we briefly describe the evolutionary-based techniques for descriptor combination addressed in this paper: Firefly Algorithm, Harmony Search and Particle Swarm Optimization. A. Firefly Algorithm As most of the optimization algorithms, the Firefly algorithm (FFA) is another nature-inspired algorithm designed by Yang [7]. The Firefly algorithm is a metaheuristic algorithm based on Lévy flights and firefly s flashing light which is used to attract mating partners, preys and as a protective warning. Also, fireflies can imitate other fireflies flashing pattern in order to attract and eat them. Lévy flights represent the flight patterns of many animals and insects and is used in this algorithm to model the flight of fireflies on their search for an optimum solution. Thus, FFA explores how light intensity is used to attract other fireflies and Lévy flights for optimization problems as particles attract each other and move toward an optimum solution in the Particle Swarm Optimization algorithm. FFA is designed according to three rules: (i) fireflies are unisex so that they attract each other, (ii) attractiveness is proportional to their brightness, in which the less bright moves toward the brighter one, and (iii) the landscape of the objective function affects the brightness. Light intensity and attractiveness are formulated in which attractiveness can be associated to the firefly s brightness and the brightness can be associated to the solution of the problem. However, light intensity is relative to the distance between the firefly source and the firefly that is receiving the light. Yang formulates the light intensity I(r) varying according to the inverse square law: I(r) = I s r2, (1) in which I s is the source s intensity and r the distance. Attractiveness is proportional to the light intensity and defined as: β = β 0 e γr2, (2) where β 0 is the attractiveness at r = 0. So, when the firefly is near the optimum solution its brightness increases and attracts other fireflies which start moving toward the brighter firefly. Their movements in the search space are defined by the Lévy-Flight which is random walks in the space (Algorithm 1). Algorithm 1. FIREFLY ALGORITHM INPUT: OUTPUT: Set of fireflies. Best solution. 1. Generate the initial population of z fireflies. 2. Set light intensity I for each firefly a according to x. 3. For each firefly a = 1:z 4. For each firefly b = 1:z 5. If I a > I b,then 6. Firefly b moves toward a using the Levy-flight. 7. Evaluate the solution and update light intensity. 8. Rank fireflies according to their brightness 9. Return the best solution. In line 2, x stands for the location of the firefly in the space, and fireflies are ranked (Line 8) in order to find the best solution. The distance between two fireflies is their respective Cartesian distance and the movement of a fireflyato a brighter firefly b is: x a = x a +β 0 e γr2 ab (xb x a )+αsign[rand 1 ] Levy, (3) 2 where sign[rand 1 2 ] with rand [0,1] provides a random direction and Levy provides the lenght of the step. Also, the movement is based on the attraction (second term) and a randomization via Lévy-Flight where α is the randomization parameter. B. Harmony Search The Harmony Search algorithm is an evolutionary algorithm based on music composition, considering a process in which musicians improvise to create music [8]. Its main characteristics are the simple concept, a few parameters and speed to find a solution. The idea is to use the process of creating new songs in order to find solutions for optimization problems, in which solutions are modeled as harmonies and each parameter to be optimized is represented as a note. The final solution will be the best harmony computed by the algorithm. The main steps are: 1) Initialize algorithm parameters; 2) Initialize Harmony Memory (HM) with random values;

3 3) Improvise a new harmony to HM; 4) Updates the HM if the new harmony is better than the worst harmony in HM, inserts the new harmony in HM and removes the worst from it; 5) If the stop criteria has not been reached, return to step 3. In Step 3, a new harmony vector x =(x 1,x 2,..., x N ) is generated from the HM based on memory considerations, pitch adjustments, and randomization (music improvisation). As follows: (i) position ( x i ) and (ii) velocity ( v i ). First, we define the number of particles that will be initialized with random values for both velocity and position. Each particle is evaluated to update the local maximum through its fitness function. Finally, the best position is update with respect to the best of the local maximum particles. This process is repeated until some convergence criterion is reached, i.e. number of iterations. The given particle p i has the velocity and position equations at time step t, respectively, given by x j { x j x j φ j { } x j 1,xj 2,...,xj HMS with probability HMCR, with probability (1-HMCR), in which φ j denotes the range of values for variable j, for φ = (φ i,φ 2,...,φ N ). The HMCR is the probability of choosing one value from the historic values stored in the HM, and (1- HMCR) is the probability of randomly choosing one feasible value not limited to those stored in the HM. In order to make it clear, an HMCR= means that 70% of the notes (decision variables) to compose the new harmony h = ( x ) will be picked from HM, and the remaining ones will be randomly generated within the interval φ j. The pitch adjustment for each harmony is often used to improve solutions and to escape from local optimum. This mechanism concerns shifting the neighboring values of some decision variable in the harmony: x j x j +rb, (5) where x j is the note j that composes the new harmony vector, b is an arbitrary distance bandwidth for the continuous design variable, and r is a uniform distribution between 0 and 1. In this paper, we set b = 1. In Step 4, if the new harmony h is better than the worst harmony in the HM, the latter is replaced by this new harmony. Finally, in Step 5, the HS algorithm finishes when it satisfies the stopping criterion. Otherwise, Steps 3 and 4 are repeated in order to improvise a new harmony again. C. Particle Swarm Optimization The Particle Swarm Optimization (PSO) algorithm was proposed by [9] in order to find solutions for optimization problems. The algorithm is based on social behavior dynamics in which each solution for a given problem is represented by particles which fly in the search space. Particle Swarm Optimization (PSO) is an optimization algorithm based on social behavior dynamics [9], that finds a solution in a search space. Each solution is modeled as a particle in the swarm, which has a memory that stores its best local solution (local maxima) and the best global solution (global maxima). Each particle compares its value with the neighborhood based on a objective function, and decides if copies it in order to get the best local and global maxima. The swarm is modeled in a multidimensional space R N, in which each particle p i = ( x i, v i ) R N has two main features: (4) v j i (t+1) = wvj i (t)+c 1r 1 (ˆx i (t) x j i (t))+c 2r 2 ( g x j i (t)) (6) and x j i (t+1) = xj i (t)+vj i (t+1), (7) where w is the inertia weight that controls the interaction between particles, and r 1,r 2 [0,1] are random variables that give the stochastic idea to Particle Swarm Optimization. The variables c 1 and c 2 are used to guide the particles onto good directions. IV. OPTIMUM-PATH FOREST CLASSIFICATION The OPF classifier works by modeling the problem of pattern recognition as a graph partition in a given feature space. The nodes are represented by the feature vectors and the edges connect all pairs of them, defining a full connectedness graph. This kind of representation is straightforward, given that the graph does not need to be explicitly represented, allowing us to save memory. The partition of the graph is carried out by a competition process between some key samples (prototypes), which offer optimum paths to the remaining nodes of the graph. Each prototype sample defines its optimum-path tree (OPT), and the collection of all OPTs defines an optimumpath forest, which gives the name to the classifier [10], [11]. The OPF can be seen as a generalization of the well-known Dijkstra s algorithm to compute optimum paths from a source node to the remaining ones [12]. The main difference relies on the fact that OPF uses a set of source nodes (prototypes) with any smooth path-cost function [13]. In case of Dijkstra s algorithm, a function that summed the arc-weights along a path was applied. In regard to the supervised OPF version addressed here, we have used a function that gives the maximum arcweight along a path, as explained below. Let Z = Z 1 Z 2 Z 3 be a dataset labeled with a function λ, in which Z 1, Z 2 and Z 3 are, respectively, a training, validation and test sets. LetS Z 1 a set of prototype samples. Essentially, the OPF classifier creates a discrete optimum partition of the feature space such that any sample s Z 2 Z 3 can be classified according to this partition. This partition is an optimum path forest (OPF) computed in R n by the Image Foresting Transform (IFT) algorithm [13]. The OPF algorithm may be used with any smooth path-cost function which can group samples with similar properties [13]. Particularly, we used the path-cost function f max, which is

4 computed as follows: { 0 if s S, f max ( s ) = + otherwise f max (π s,t ) = max{f max (π),d(s,t)}, (8) in which d(s,t) means the distance between samples s and t, and a path π is defined as a sequence of adjacent samples. In such a way, we have that f max (π) computes the maximum distance between adjacent samples in π, when π is not a trivial path. The OPF algorithm works with training and testing phases. In the former step, the competition process begins with the prototypes computation. We are interested into finding the elements that fall on the boundary of the classes with different labels. For that purpose, we can compute a Minimum Spanning Tree (MST) over the original graph and then mark as prototypes the connected elements with different labels. Figure 3b displays the MST with the prototypes at the boundary. After that, we can begin the competition process between prototypes in order to build the optimum-path forest, as displayed in Figure 3c. The classification phase is conducted by taking a sample from the test set (blue triangle in Figure 3d) and connecting it to all training samples. The distance to all training nodes are computed and used to weight the edges. Finally, each training node offers to the test sample a cost given by a path-cost function (maximum arc-weight along a path - Equation 8), and the training node that has offered the minimum path-cost will conquer the test sample. This procedure is shown in Figure 3e (a) (b) 0.9 V. DESCRIPTOR COMBINATION USING FIREFLY ALGORITHM Mansano et. al [14] have proposed an extension to Faria s et al. work [6] by introducing a set of parameters β = (β 1,β 2,...,β M ) in order to allow a greater variability of arithmetic computations, which was limited by the linear formulation proposed by Faria et al. [6] (Eq. 9). Equation 10 presents the proposed formulation. δ D = δ D = N α i δ Di, (9) i=1 N i=1 α i δ βi D i, (10) in which 2 α i,β i 2, β i R. The optimization algorithms are responsible for computing the best values for α and β that maximize the accuracy rate over the validation set which is our objective function. As it is known, each agent (particle, harmony or firefly) represents a different solution which changes at the end of each iteration. Each pair (α, β) provides a different training set which is assessed using the validation set. The process of computing new solutions and validation each of them repeats until the number of iterations is reached and the best training set of all computed is saved in order to run the tests. The purpose of validation the training set is to obtain the set that may provide the highest accuracy rates for a given problem. Thus, the methodology uses three sets: (i) training set which will become the final training set, (ii) validation set used to evaluate the descriptor combination parameters at each iteration of the optimization algorithms, and (iii) testing set which is used to assess the final training set. Then, the whole process can be split in two phases: (i) the design phase where the final training set is modeled by means of optimization algorithms which compute the best parameters for the set of descriptors, and (ii) classification phase in which each sample is classified using the training model computed in the previous phase. Figure 4 shows the methodology proposed by Mansano et. al [14] (c) (d) 0.6 (e) Fig. 3. OPF pipeline: (a) complete graph, (b) MST and prototypes bounded, (c) optimum-path forest generated at the final of training step, (d) classification process and (e) the triangle sample is associated to the white circle class. The values above the nodes are their costs after training, and the values above the edges stand for the distance between their corresponding nodes. Fig. 4. Proposed methodology for descriptor combination. Extracted from Mansano et. al [14]

5 VI. EXPERIMENTAL RESULTS Experiments used two well-known public datasets: Corel 1 : It contains 3,906 images labeled in 85 classes, and the number of images per class ranges from 7 to 98 images. Free Photo 2 : A subset of this dataset was employed containing 3,426 images labeled in 9 classes, and the number of images per class ranges from 70 to 854 images. Different types of descriptors were computed for each dataset [15]: Color Autocorrelogram (ACC): descriptor of color representing the space color in 64 bins and 4 values of distance. Border/Interior pixel Classification (BIC): it is a descriptor of color in which color space is represented by 64 bins. Color Coherent Vector (CCV): descriptor of color composed by a combination of two histograms of 64 bins total. Global Color Histogram (GCH): descriptor of color in which the color space was split in 64 bins. Homogeneous Texture Descriptor (HTD): descriptor of texture Local Activity Spectrum (LAS): descriptor of texture in which components are quantized in 4 bins resulting in a histogram of 256 bins. Quantized Compound Change Histogram (QCCH): descriptor of texture composed by 40 bins. There were computed the ACC, BIC, GCH, LAS and QCCH descriptors for Corel dataset; and BIC, CCV, LAS, GCH and HTD for Free Photo dataset [16] The set of descriptors is split in: training, validation and testing sets. Experiments using a single descriptor used just the training and testing sets, and experiments involving combination is performed using the 3 sets where the validation set is employed to evaluate the solution given by the optimization algorithms. The accuracy rate is computed according to Eq. 11. n i=1 Acc = ta i, (11) n in which ta i stands for the accuracy rate of class i and n represents the number of classes. Table I shows the average accuracy rate of 5 rounds of tests for each descriptor and each database using the OPF classifier using a single descriptor. Single descriptor experiments used a training set of 30% of the entire dataset and 50% of the dataset for testing which were randomly generated in each round of test. The purpose of this first experiment is to compare the gain when descriptor combination with optimization algorithms is performed. Results show different performances for each descriptor in each Descriptor/Dataset Corel Free Photo ACC(HTD) 74.27%± %±1.31 LAS 59.75%± %±0.65 BIC 72.67%± %±0.96 GCH 65.82%± %±1.25 QCCH(CCV) 57.38%± %±0.61 TABLE I SINGLE DESCRIPTOR EFFICIENCY. database where most of them had a better performance in the Free Photo database. Experiments combining different descriptors included optimization algorithms which were employed separately in order to assess each algorithm s performance. In this phase, the validation set is included in order to evaluate and improve the training set prior the test step. The parameters of each algorithm were set as shown in Table II where there were used 250 agents (fireflies, particles and harmonies), 100 iterations and parameters with similar functions were set with the same value (e.g. HMCR from HS and w from PSO) in order to make a fair comparison. Technique Parameters FFA α = 0.2, β = 1.0, γ = 1.0 HS HMCR:, PAR: PSO w =, c 1 = 1.6, c 2 = TABLE II PARAMETERS USED IN THE EXPERIMENTS. Since the purpose is to improve accuracy rate in image classification, the OPF classifier was employed to assess each combination computed by each technique and accuracy rates for combinations of 2 and 5 descriptors, in order words, the accuracy rate is the objective function to be maximized. Table III presents the mean accuracy rates for combination of 2 (LAS and GCH, empirically chosen) and 5 descriptors for each technique and dataset. Descriptor/Dataset Corel Free Photo FFA-LAS+GCH 65.82%± %±0.83 HS-LAS+GCH 67.94%± %±3 PSO-LAS+GCH 68.04%± %±1 FFA-ALL 76.02%± %±1.06 HS-ALL 75.34%± %±0.91 PSO-ALL 76.77%± %±2 TABLE III COMPOSITE DESCRIPTOR EFFICIENCY ACCORDING TO THE PROPOSED APPROACH. Comparing the combination of 2 and 5 descriptors, it is visible an improvement of over 8% in Corel database and almost 6% in the Free Photo, considering the best accuracy rates in each scenario. Among the optimization algorithms, PSO has achieved the best accuracy rates in 3 out of 4 situations, although all techniques had similar results. However, comparing performances of single descriptor and descriptor combination using 5 descriptors has shown a slight

6 difference of over 2% for Corel database and almost 1% for Free Photo database. The low gain might be caused due to the usage of a descriptor that has low performance in such configuration of datasets. VII. CONCLUSIONS Many approaches have proposed feature combination in image classification. This paper has presented techniques based on social dynamics to perform descriptor combination using optimization algorithms and Firefly Algorithm is introduced in this context, which is defined by a pair of feature extraction algorithm and a distance function associated with it. The approach consists in a mathematical formulation of a composite descriptor, which is obtained by combining descriptors using PSO, HS and FFA, and the OPF to perform the tests. In this case, the accuracy rate of OPF in a validation set is used as an objective function to be maximized by these techniques. Experiments with color and texture features have shown that combining different descriptors can outperform the sensibility in image classification. [13] A.X. Falcão, J. Stolfi, and R.A. Lotufo, The image foresting transform theory, algorithms, and applications, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 1, pp , [14] A. Mansano, J. A. Matsuoka, L. C S Afonso, J. P. Papa, F. Faria, and R. da S Torres, Improving image classification through descriptor combination, in Graphics, Patterns and Images (SIBGRAPI), th SIBGRAPI Conference on, 2012, pp [15] F. A. Faria, J. A. dos Santos, A. Rocha, and R. S. Torres, Automatic classifier fusion for produce recognition., in SIBGRAPI. 2012, pp , IEEE Computer Society. [16] O.A.B. Penatti, E. Valle, and R.S. Torres, Comparative study of global color and texture descriptors for web image retrieval, J. of Visual Communication and Image Representation, VIII. ACKNOWLEDGMENTS The authors thank FAPESP (Procs. #2011/ , #2012/ and #2009/ ) and CNPq grant #303182/ REFERENCES [1] P. Gehler and S. Nowozin, On feature combination for multiclass object classification, in Proceedings of the 12th International Conference on Computer Vision, 2009, pp [2] J. Hou, B.-P. Zhang, N.-M. Qi, and Y. Yang, Evaluating feature combination in object classification, in Proceedings of the 7th International Conference on Visual Computing, Las Vegas, NV, USA, 2011, pp , Springer-Verlag. [3] D. Okanohara and J. Tsujii, Learning combination features with L1 regularization, in Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the ACL, Stroudsburg, PA, USA, 2009, pp [4] X. Liu, L. Zhang, M. Li, H. Zhang, and D.Wang, Boosting image classification with LDA-based feature combination for digital photograph management, Pattern Recognition, vol. 38, no. 6, pp , June [5] F. Yan and X. Yanming, Image classification based on multi-feature combination and PCA-RBaggSVM, in Proceedings of the IEEE International Conference on Progress in Informatics and Computing, 2010, vol. 2, pp [6] F.A. Faria, J.P. Papa, R.S. Torres, and A.X. Falcão, Multimodal pattern recognition through particle swarm optimization, in Proceedings of the 17th International Conference on Systems, Signals and Image Processing, Rio de Janeiro, Brazil, 2010, pp [7] Xin-She Yang, Firefly algorithm, lévy flights and global optimization, in SGAI Conf., 2009, pp [8] Z. Geem, Novel derivative of harmony search algorithm for discrete design variables, Applied Mathematics and Computation, vol. 199, no. 1, pp , May [9] J. Kennedy and R.C. Eberhart, Swarm Intelligence, M. Kaufman, [10] J. P. Papa, A. X. Falcão, and C. T. N. Suzuki, Supervised pattern classification based on optimum-path forest, International Journal of Imaging Systems and Technology, vol. 19, no. 2, pp , [11] J. P. Papa, A. X. Falcão, V. H. C. Albuquerque, and J. M. R. S. Tavares, Efficient supervised optimum-path forest classification for large datasets, Pattern Recognition, vol. 45, no. 1, pp , [12] E. W. Dijkstra, A note on two problems in connexion with graphs, Numerische Mathematik, vol. 1, pp , 1959.