Constraints in Particle Swarm Optimization of Hidden Markov Models

Transcription

1 Constraints in Particle Swarm Optimization of Hidden Markov Models Martin Macaš, Daniel Novák, and Lenka Lhotská Czech Technical University, Faculty of Electrical Engineering, Dep. of Cybernetics, Prague, Czech Republic Abstract. This paper presents new application of Particle Swarm Optimization (PSO) algorithm for training Hidden Markov Models (HMMs). The problem of finding an optimal set of model parameters is numerical optimization problem constrained by stochastic character of HMM parameters. Constraint handling is carried out using three different ways and the results are compared to Baum-Welch algorithm (BW), commonly used for HMM training. The global searching PSO method is much less sensitive to local extremes and finds better solutions than the local BW algorithm, which often converges to local optima. The advantage of PSO approach was markedly evident, when longer training sequence was used. 1 Introduction Hidden Markov Model (HMM) is statistical modelling technique which enables to characterize real-world stochastic signals in terms of signal models. There are many real world problems which have been solved by HMMs. The examples are speech recognition [1], biomedical signal processing, bioinformatics, robotics, handwriting recognition or economy. There are two main problems connected with HMMs - the choice of model topology (e.g. number of hidden states and type of connections) and search for the model parameters. This problem is mostly solved using Baum-Welch algorithm (BW). The main drawback of the method is frequent convergence to local optima resulting from local character of searching. One solution of this problem is to use some global searching methods [2]. In this paper, the model parameters are searched while the topology is given in advance for all experiments. The model parameters are searched in terms of likelihood maximization. The problem of reaching the global maxima is solved using nature inspired Particle Swarm Optimization (PSO) method, which is described below. However, the problem of searching some parameters of HMM is made difficult by constraints arising from the probabilistic character of this type of parameters. Furthermore, some other parameters representing covariance are searched and therefore a positivity constraint arises from this fact. This implies a need for some constraint handling mechanism to be used in PSO. E. Corchado et al. (Eds.): IDEAL 2006, LNCS 4224, pp , c Springer-Verlag Berlin Heidelberg 2006

2 1400 M. Macaš, D. Novák, and L. Lhotská 2 Methods 2.1 Hidden Markov Models An HMM is a stochastic finite state automata defined by the following parameters: λ =(A, p, B), where A = {a ij } is a matrix of state transition probabilities a ij representing probability of transition from state i to state j, p is the initial state probability and B is the emission probability density function of each state. This paper considers continuous HMMs with B represented by a finite mixture of Gaussian distributions. For simplicity, only one Gaussian distribution was used for all experiments and therefore the B is set of N Gaussian distributions G i (m i,u i ), where the m i and U i are mean and variance of the distribution for ith state respectively. Consider an observation sequence O = O 1,...,O T. There are three problems connected with HMM. The first one is to compute P (O λ), the probability of observing the sequence, given the model (model evaluation). The second problem is to find a corresponding state sequence Q = Q 1,...,Q T, given the observation sequence O and model parameters λ. The third problem is crucial and by far the most difficult one. How to find the model parameters λ maximizing P (O λ) (fitting the training sequence). This problem of model training consists of adjusting the model parameters λ to maximize likelihood - the probability of the observation sequence given the model. There is no known way to analytically solve this problem. Usually the alternative solutions are iterative procedures like Baum-Welch algorithm [3]. Baum-Welch algorithm [1] is a standard solution for solving the training problem based on maximization of data likelihood. However, this hill-climbing algorithm leads to local maxima only, and in most problems of interest, the optimization surface is very complex and has many local optima. The BW algorithm uses a set of re-estimation formulas and guarantees that the likelihood of re-estimated model will be equal to or greater than the likelihood of model in previous iteration. 2.2 Particle Swarm Optimization The PSO method is one of optimization method developed for finding a global optima of some nonlinear function [4]. It has been inspired by social behavior of birds and fish. The method applies the approach of problem solving in groups. Each solution consists of set of parameters and represents a point in multidimensional space. The solution is called particle and the group of particles (population) is called swarm. Each particle i is represented as a D-dimensional position vector x i (t) and has a corresponding instantaneous velocity vector v i (t). Furthermore, it remembers its individual best value of fitness function and position p i which has resulted in that value. During each iteration t, the velocity update rule (1) is applied on each particle in the swarm. The p g is the best position of the entire swarm and represents the social knowledge. v i (t) =w v i (t 1) + ϕ 1 R 1 ( p i x i (t 1)) + ϕ 2 R 2 ( p g x i (t 1)) (1)

3 Constraints in Particle Swarm Optimization of HMMs 1401 The parameter w is called inertia weight and during all iterations decreases linearly from w start to w end.thesymbolsr 1 and R 2 represent the diagonal matrices with random diagonal elements drawn from a uniform distribution between 0 and 1. The parameters ϕ 1 and ϕ 2 are scalar constants that weight influence of particles own experience and the social knowledge. Next, the position update rule (2) is applied: x i (t) = x i (t 1) + v i (t). (2) If any component of v i is less than V max or greater than +V max,thecorresponding value is replaced by V max or +V max, respectively. The V max is maximum velocity parameter. The update formulas (1) and (2) are applied during each iteration and the pi and p g values are updated simultaneously. The algorithm stops if maximum number of iterations is achieved. 2.3 Particle Structure and Evaluation The quality of particle (candidate solution) is evaluated using a fitness function. Each particle is built of A, p,m,u parameters reconfigured to one dimensional vector. Two sorts of structure of particles (encoding) were used here (see the Table 1) - the Type A encoding is used when the particle corresponds to all parametersλ, the type B encoding does not include last component of probabilistic parameters, because it can be computed from stochastic constraints described in the next section. Particles of both types are equivalent to parameters λ of HMM. The Type A representation was used for repair based method and the Type B encoding was used for penalty based method. Table 1. Two types of representation of a particle Type A A{N N} p{n} m{n} U{N} a a 1N... a N1... a NN p 1... p N m 1... m N U 1... U N Type B A{N (N 1)} p{n 1} m{n} U{N} a a 1N 1... a N1... a NN 1 p 1... p N 1 m 1... m N U 1... U N In this paper, the maximization of likelihood is performed as minimization of the value of function described by Equation Constraint Handling eval F (λ) = ln P (O λ) (3) An important aspect of the Baum-Welch algorithm is that the stochastic constraints of the HMM parameters are automatically satisfied at each iteration.

4 1402 M. Macaš, D. Novák, and L. Lhotská Generally, this is not true for all training algorithms. The problem of training must be therefore understood as constraint optimization task and could be described as following: Find λ = {A, p, B}, which maximizes P (O λ), subject to a ik 0 (4) N a ij =1 (5) j=1 p j 0 (6) N p j =1 (7) j=1 U i > 0, (8) where i =1...N, k =1...N and N is number of hidden states. However, the PSO was originally developed for unconstrained optimization and hence the algorithm described in previous section must be modified to be able to handle with constraints [5]. In this paper, the following constraint handling methods are compared: Methods based on repair algorithm (PSOr, PSOrf). The unfeasible solutions are repaired by moving them into feasible space. The unfeasible solutions can be replaced by its repaired version or the repair procedure can be used for fitness evaluation only. The two following repair methods are used in this paper. First approach, reffered to as PSOr, replaces unfeasible particles by their repaired version. The second one, referred to as PSOrf, uses modification of fitness function, which evaluates the repaired version of unfeasible particle, however, the unfeasible particles are not replaced. The repair procedure consists of normalization operations. First, all negative elements of state transition matrix A are assigned to be zero. Next, each element a ij of the matrix is transformed to a ij / N j=1 a ij which ensures that the constraint (5) is not violated. The same procedure is applied on initial probabilities p. Finally, a violation of positivity constraint (8) (for multivariate observations, it is constraint of positive-definity) is repaired by setting all C i which are smaller than a constant c (e.g. c =10 5 )toc. Method based on penalty function (PSOp). The unfeasible solutions are penalized using eval U (λ) =eval F (λ)+penalty(λ). However, the likelihood can not be computed for model parameters violating the stochastic constraints. Therefore, a constant K couldbeusedinplaceofeval F (λ). For instance, the suitable value for K could be the fitness function of the worst particle in the initial swarm. The following form of fitness evaluation was used: eval U (λ) =K + ξ[( (c U i ) 2. (9) a ij<0 a 2 ij + p i<0 p 2 i )] + U i<c

5 Constraints in Particle Swarm Optimization of HMMs 1403 The constant c in Equation 9 has similar meaning as in the previous section and was set to c =10 5. The penalty term computes the degree of constraint violation. The constant ξ is weight which scales an influences of stochastic constraints (4, 6) and the positivity constraint (8). The suitable value of ξ is determined by the observation sequence and was set to ξ = 10 for all experiments. The implementation of penalty function uses different conception of feasibility. The parameters of HMM are encoded into particle using the Type B encoding. Thus, the last column of transition matrix A and last component of vector of initial state probabilities are removed from particle. In the beginning of each evaluation, these parameters must be computed using Equations 5 and 7. For instance, the p N parameter is p N =1 N 1 i=1 p i. Thus, the feasibility is equivalent to satisfaction of the Equations 4, 6 and 8. In the case of violating these three inequality constraints, the particle is evaluated using Equation 9. In the case, that the particle represents feasible solution, it is evaluated using log-likelihood function (3). In experimental results, the penalty method is referred to as PSOp. 3 Results The approaches described above were tested using three observation sequences generated by three predefined artificial HMMs (Model 1-3 ). The number of hidden states was N = 4 because of two reasons. First, authors of this work will utilize the results in application of HMM to classification of 4 substages of non- REM sleep from sleep EEG record. Secondly, the PSO method fits to optimize smaller HMMs as it will be discussed at the end of this section. The initial state probabilities were set uniform (to 0.25) for all three models (p 1 = p 2 = p 3 = (0.25, 0.25, 0.25, 0.25)). For the models 1 and 2, the transition probabilities were set also uniformly to On the other hand, the diagonal elements of matrix A 3 (probabilities of staying in the previous state) were 0.85 and the others (probabilities of transition to different state) were set to The Gaussian distributions representing the emission probabilities are described in Equation 10. G( 2, 0.5) G(0, 0.2) G(0, 0.2) B 1 = G(0, 0.5) G(2, 0.5),B 2 = G(0, 2.0) G( 4, 0.5),B 3 = G(0, 2.0) G( 4, 0.5) (10) G(4, 0.5) G(4, 0.5) G(4, 0.5) The BW algorithm and the three constraint handling PSO methods were compared. The comparison criterion was the final log-likelihood function described by Equation 3 averaged over 30 runs of experiment. First, a training observation sequence of length T was generated by the model (T was 100 and 200). Further, the optimization algorithm was launched 30 times searching for the model which best fits the training sequence. For all PSO experiments, the algorithm parameters were set as following: the weight parameters were ϕ 1 = ϕ 2 = 2, number of iterations was 500, swarm size was 100. The inertia weight w has decreased linearly from w =1tow =0.2

6 1404 M. Macaš, D. Novák, and L. Lhotská during 350 iterations and then it remained constant for the rest of optimization process. The maximum velocity parameter was set variously for different dimensions. Concretely, the change of probabilistic parameters representing the transition matrix and initial state probability was limited by V max =0.2, the rest of velocity dimensions was limited by V max =2.ThevalueofV max is problem dependent and was selected experimentally. Table 2. Experimental results: means, standard deviations of ln P (O λ) obtained from 30 runs of experiments and the results of t-test (α =0.05) of difference between BW results and results for particular PSO modifications. The 1 behind the slash means that the PSO method found significantly better solutions than the BW algorithm. T = 100 Method Model 1 Model 2 Model 3 BW (4.71)/ (5.66)/ (27.13)/- PSOr (2.67)/ (5.46)/ (1.54)/1 PSOrf (3.50)/ (4.69)/ (2.03)/1 PSOp (2.04)/ (4.87)/ (2.46)/1 T=200 Model 1 Model 2 Model 3 BW (3.46)/ (22.52)/ (30.54)/- PSOr (3.44)/ (3.02)/ (2.64)/1 PSOrf (5.22)/ (6.40)/ (1.66)/1 PSOp (1.26)/ (0.72)/ (1.65)/1 The final values of ln P (O λ) obtained for the three models fitting the sequences of two different lengths (T = 100 and T = 200) are depicted in Table 2. The Figure 1 shows the training curves averaged over 30runs. The three PSO variants can be compared from this figure. The difference between BW and PSO algorithms can be seen from the Table 2. The standard deviation of final fitness value represents the ability of method to reach the minimum independently on initialization. In addition, the test of significance was performed using paired t-test (α =0.05) of the hypothesis that two matched samples X, Y come from distributions with equal means. However, the test makes an assumption that the distribution of X Y comes from a normal distribution. This assumption was not satisfied for some samples (e.g. comparison for Model 3) and hence the testing was just additional comparison tool. For all experiments, the means of final fitness obtained for PSO methods are smaller. For the short training sequence (T = 100), only the Model 3 was identified significantly better by all PSO approach. On the other hand, if the longer training sequence is used for training, the PSO algorithms reached much smaller values of the function eval F than the BW algorithm.

7 Constraints in Particle Swarm Optimization of HMMs 1405 (a) (b) Model 1 Model Model 2 Model Model 3 PSOp PSOr PSOrf Model 3 PSOp PSOr PSOrf 420 Fig. 1. The training curves averaged over 30 runs of experiment. An observation sequence with length (a) T = 100 and (b) T = 200 samples was used for training the HMM. Further, the different PSO constraint handling methods may be compared using the Table 2 and the Figure 1. The figure depicts the optimization process using the dependence of the best fitness in the swarm on time. The curves are averaged over 30 runs of experiment. The figures show strong similarity between the two repair methods (PSOr, PSOrf). At the beginning of optimization, the curve decreases faster for the PSOr and PSOrf methods, however, for PSOp, the rapidity of fall increases during first iterations. Furthermore, it even outruns the repair methods and reaches better value. Apparently, the concave-like shape of first part of training is caused by increasing number of feasible particles in the swarm. Finally, the penalty method PSOp seems least sensitive to initialization, because the minimum is reached with the smallest standard deviation and the difference of mean of results from the BW results was statistically significant. Finally, some temporal aspects of the methods must be mentioned. The main disadvantage of all population based techniques are time requirements. During each iteration of PSO using a swarm of 100 particles, the calculation of fitness function must be repeated 100 time. During the whole optimization process with 500 iterations, there are fitness evaluations. The fitness evaluation depends on the order of HMM and on length of the observation sequence. Therefore, it can be concluded, that PSO should be used for HMM training in the case of reasonable model order and not very long training sequence. The detailed experiments with time requirements were not performed for this paper.

8 1406 M. Macaš, D. Novák, and L. Lhotská 4 Conclusions The Particle Swarm Optimization seems useful and suitable method for Hidden Markov Models training. In comparison with conventional method - Baum-Welch algorithm, the PSO is much more capable to overcome the local optima of likelihood, whereas the local BW algorithm sometimes gets stuck in local extreme. In the future, the other methods will be tested. Namely, the preserving feasibility methods using Linear PSO [6] or pareto-ranking methods seem sufficient and promising. Acknowledgments The research was supported by the research program No. MSM Transdisciplinary Research in the Area of Biomedical Engineering II of the CTU in Prague, sponsored by the Ministry of Education, Youth and Sports of the Czech Republic and CTU grant No References 1. Rabiner, R.: A tutorial on hidden markov models and selected applications in speech recognition. Proceedings of the IEEE 77(2) (1989) Kwong, S., Chau, C.: Analysis of paralell genetic algortihms on hmm based speech recognition system. In: ICASSP. (1997) Dempster, A., Laird, N., Rubin, D.: Maximum-likelihood from incomplete data via the em algorithm. Journal of Royal Statistics 39 (1977) Kennedy, J., Eberhart, R.: Particle swarm optimization. In: Proceedings International Conference on Neural Networks IEEE. Volume 4. (1995) X.Hu, R.C.Eberhart: Solving constrained nonlinear optimization problems with particle swarm optimization. In: Proceedings of the Sixth World Multiconference on Systems, Cybernetics and Informatics. (2002) 6. Paquet, U., p Engelbrecht, A.: A New Particle Swarm Optimiser for Linearly Constrained Optimization. In: Proceedings of the Congress on Evolutionary Computation 2003 (CEC 2003). Volume 1., Piscataway, New Jersey, Canberra, Australia, IEEE Service Center (2003)