The Pennsylvania State University. The Graduate School. Department of Electrical Engineering

Transcription

1 The Pennsylvania State University The Graduate School Department of Electrical Engineering COMPARISON ON THE PERFORMANCE BETWEEN THE THREE STRUCTURES OF IIR ADAPTIVE FILTER FOR SYSTEM IDENTIFICATION BASED ON GENETIC ALGORITHMS (GA) A Thesis in Electrical Engineering by Xin Shao 2015 Xin Shao Submitted in Partial Fulfillment of the Requirements for the Degree of Master of Science December 2015

2 The thesis of Xin Shao was reviewed and approved* by the following: Kenneth W. Jenkins Professor of Electrical Engineering Thesis Advisor John F. Doherty Professor of Electrical Engineering Kultegin Aydin Professor of Electrical Engineering Head of the Department of Electrical Engineering *Signatures are on file in the Graduate School

3 iii ABSTRACT Genetic Algorithms (GA) are based on the principles of natural selection and natural genetics that originate in biology. The Genetic Algorithm (GA) has been used for IIR adaptive system identification to deal with its multimodal error surface. The Genetic Algorithm (GA) can be very useful in the all three structures of IIR filters while the Gradient Algorithm experiences many difficulties due to the recursive feedback. This thesis will focus on the different performances of three structures of IIR adaptive filters based on the Genetic Algorithm (GA) and Multi-Parents Genetic Algorithm (MPGA). Experimental results demonstrate that, in general, the standard Genetic Algorithm (GA) direct form will have lower Mean Square Error (MSE), while the cascade and parallel forms will have higher convergence rates. The relative performance of three structures for Multi-Parents Genetic Algorithm (MPGA) is similar to the 2-parent Genetic Algorithm, but the rate of convergence is higher than the standard GA, which means the MPGA converges faster than the standard GA. Furthermore the performances of the three structures for the IIR filter based on modified Multi-Parents Genetic Algorithm (MPGA) are very similar. Simulation results demonstrate that when compared with the GA, the MPGA operates similarly on the three different structures, increases the rate of convergence rate and reduces the computational complexity. Finally, the Genetic Algorithm and the Gradient Algorithm were combined on the direct form to take advantage of each algorithm. When the rate of convergence decreases into a steady level the Gradient Algorithm is then applied so that the MSE will decrease again to a lower value, demonstrating that the combined algorithm obtains a more precise result and improves the performance.

4 iv TABLE OF CONTENTS List of Figures... v List of Tables... vi Acknowledgements... vii Chapter 1 Introduction Development of Adaptive Filtering Gradient Algorithm Introduction of the LMS Algorithm Limitation of the LMS Algorithm on IIR Adaptive Filtering Genetic Algorithm Multi-Parents Genetic Algorithm Overall Thesis Structure... 5 Chapter 2 FIR vs IIR Adaptive System Identifications FIR Adaptive System Identification IIR Adaptive System Identification... 7 Chapter 3 Three Structures of IIR Filters Based on the Genetic Algorithm Direct Form II Structure Cascade Form Structure Parallel Form Structure Comparison of three structure Effect of Population Size Effect of Mutation Rate Effect of Input Signal Effect of the Number of Parents Chapter 4 Combined Genetic Algorithm and Gradient Algorithm Combine Genetic Algorithm and Gradient Algorithm for the FIR filter Combine Genetic Algorithm and Gradient Algorithm on IIR filter Chapter 5 Conclusion and Future Work... 23

5 v LIST OF FIGURES Figure 1-1 GA circle... 4 Figure 2-1. Adaptive system identification configuration... 6 Figure 2-2. Simplified IIR Adaptive Filter Block Diagram[14]... 9 Figure 3-1. Direct form II IIR filter structure Figure 3-2. Cascade form IIR filter structure Figure 3-3. Parallel form IIR filter structure Figure 3-4. Effect of Population Size Figure 3-5. Effect of Mutation Rate Figure 3-6. Effect of Mutation Rate Figure 3-7. Effect of Input Signal Characteristics Figure 3-8. Effect of the Number of Parents Figure 3-9. Effect of Modified Parents Amount in the MPGA Figure 4-1. Gradient Algorithm Applied to the FIR filter Figure 4-2. Genetic Algorithm Applied to the FIR filter Figure 4-3. Combination of Genetic Algorithm and Gradient Algorithm Applied to the FIR filter Figure 4-4. Gradient Algorithm Applied to the IIR filter Figure 4-5. Genetic Algorithm Applied to the IIR filter Figure 4-6. Combination of Genetic Algorithm and Gradient Algorithm Applied to the IIR filter... 22

6 vi ACKNOWLEDGEMENTS I would like to thank my advisor, Professor W. K. Jenkins, for his careful and patient guidance throughout all aspects of my thesis and graduate studies. He is more a mentor than a teacher for me. What he taught me is beyond the knowledge itself. I would also like to thank Professor John F. Doherty and my classmate Guoxin Sun for their advice, support and assistance for my thesis. I am also grateful to my parents for their support and love.

7 1 Chapter 1 Introduction 1.1 Development of Adaptive Filtering Ever since Widrow and Hoff established the theory of adaptive filtering [1], adaptive filters have been widely used in various fields such as inverse modeling, linear prediction and noise and echo cancellation [2]. Adaptive filters are also applied in the system identification area successfully. Finite impulse response (FIR) and infinite impulse response (IIR) adaptive systems are two general classes of models used for unknown system identification. An adaptive filter is a system which has a linear filter, of which the transfer function is manipulated by a set of parameters, also known as tap weights. In each iteration, the system calculates the cost based on a cost function, a criterion for optimization, and the cost serves as a feedback to adjust those parameters (or transfer function) to minimize the cost on the next iteration until an optimization is considered to have been reached. The mean square error (MSE) of the signal is the most commonly used cost function, the most widely used optimization algorithm that minimizes the MSE is known as the least mean square (LMS) algorithm. 1.2 Gradient Algorithm Introduction of the LMS Algorithm The LMS algorithm is an application of the stochastic gradient descent method [2]. Because of its simplicity, ease of computation, requiring no off-line data estimation, the LMS is used widely as an optimization algorithm for adaptive filtering. The LMS algorithm is also

8 2 very suitable for FIR adaptive filters when the error surface is unimodal and quadratic. The LMS algorithm for FIR adaptive filters can be summarized as follows [2]: y(n) = w H(n)u(n), (1-1) e(n) = d(n) y(n), (1-2) w (n + 1) = w (n) + μu(n)e (n), (1-3) where n is the iteration number, u(n) is the input vector, y(n) and d(n) are the corresponding output and desired responses, respectively, e(n) is the error signal, w (n) is an estimate of the unknown tap-weight vector, w(n), of the adaptive system, and μ is the step size parameter. The superscript H denotes Hermitian transposition and the asterisk denotes complex conjugation Limitation of the LMS Algorithm on IIR Adaptive Filtering Generally, the IIR filter is better than the FIR filter for system identification because an FIR filter may not be able to accurately approximate the unknown system with a reasonable length filter. Alternatively, an IIR filter has recursive characteristics and results in less amount of memory requirements to design complex systems [3]. In many situations an IIR system can represent a better model for real-world systems than an FIR system. Although the LMS algorithm is a good choice for FIR adaptive filters, the application on IIR adaptive filters exhibits some drawbacks. Different from FIR adaptive filters, the error surface of an IIR adaptive filter is often multimodal with respect to the filter coefficients, causing the LMS algorithm to become stuck at local minima rather than converge to the global minimum [4]. Due to this reason, the application of IIR system identification is limited until some other optimization algorithms are developed.

9 3 1.3 Genetic Algorithm During the past few decades, many other optimization algorithms in the computational intelligence area, such as the particle swarm optimization (PSO) algorithm, the cat swarm optimization (CATSO) [6] and the genetic algorithm (GA) [7], etc., have been used for overcoming the deficiencies encountered by the IIR LMS algorithm. GAs are global optimization search techniques that are based on the principles of natural selection originated in the late 1980s [7]. Thereafter, the GA has been widely used in a broad variety of engineering applications such as pharmaceutical drug designs, antenna designs, integrated circuit testing, and adaptive infinite impulse response (IIR) filtering [8]. The GA can be viewed as an evolutionary process, which is mainly a cycle composed of three operators, namely, Selection, Crossover and Mutation. The cycle is summarized in Figure 1-1. This process starts with a set of randomly generated N individuals (N is called population size) with M parameters to be optimized. At the beginning of each generation, the fitness function will evaluate the fitness of each individual. Then a portion, generally half, of individuals (parents) with higher fitness are selected and the rest individuals will be removed from the population. After that each two of the selected individuals form a pair to do crossover and generate two new individuals (offspring). During this stage, two parents generate two children and the size of the population remains the same (assume half of the population are selected). Then mutation occurs to all parameters of some of the individuals with a relatively low mutation rate m for each individual. The parameter updated in this stage occurs with the probability m and can be expressed as p ik = p ik + q, where q is some small constant. After the mutation stage, the remaining individuals form the new generation, whose fitness have been evaluated, and then the process continues until the "best"

10 4 Old Generation New Generation Selection Mutation Crossover Figure 1-1 GA flow chart individual is generated. The information exchange in the Crossover stage and the change of information in the Mutation stage are able to get the search of the GA out of local minima when the error surface is multimodal. In spite of that, the slow convergence rate of the GA limits the use of its implementations somehow [3]. In order to improve the convergence rate, a modified GA - the MPGA was proposed and will be introduced in the next paragraph. 1.4 Multi-Parents Genetic Algorithm In recent years, many attempts have been proposed to improve GA because of its low rate of convergence and high computational complexity, including the innovative multiparents genetic algorithm (MPGA), which adopts three or more parents, instead of two, in the crossover stage of the GA. Since it generates more offspring in each generation, the MPGA is believed to have positive effects on the rate of convergence. This will lead the way for broader implementations of the GA in IIR adaptive system identifications in the future [3]. Since the increasing number of parents can increase the rate of convergence, in order

11 5 to achieve lower MSE while maintaining faster rate of convergence, Guoxin Sun proposed a new modified multi-parent GA (MPGA)[3], which makes the number of parents decrease automatically when the difference between the current MSE and the previous one is smaller than a given threshold and the Permutation Crossover MPGA (PC MPGA) gradually becomes the 2-parents GA. This modification (referred to as the variable parent (VP) MPGA algorithm) reduces the computational complexity of the PCMPGA significantly since only the first several generations use the PC MPGA. 1.5 Overall Thesis Structure Firstly, the development of the adaptive filtering, gradient and genetic algorithms are introduces in Chapter 1. Then in terms of the IIR filter and FIR filter based on the GA it will be demonstrated that if the unknown system is an FIR model the adaptive will achieve very good results which the IIR adaptive algorithm cannot. Hence, in the rest of thesis the IIR and FIR Adaptive System Identifications will be briefly compared in chapter 2 and then turn to the IIR filter. Three structures (direct form, cascade form, parallel form) of IIR filter based on the GA and MPGA will be compared in chapter 3. In the last chapter (chapter 4) a new ideal that combining Genetic Algorithm and Gradient Algorithm, which will result in a better performance of Adaptive System Identifications, will be introduced.

12 6 Chapter 2 FIR vs IIR Adaptive System Identifications Over the past several decades, adaptive filters have been applied in many areas, including inverse modeling, linear prediction, noise and echo cancellation, and system identification [2]. In applications dealing with system identifications, an adaptive filter is used iteratively to provide an optimal model for an unknown system or plant, based on some error function, i.e., mean squared error (MSE) of the signal between the output of the adaptive filter and that of the plant. The optimal model is obtained when the error function is minimized. The general adaptive filtering configuration for system identification is shown in Figure 2-1 below. The plant and the adaptive filter are driven by the same input signal. The error function, attained by subtracting the adaptive filter output from the plant output and calculated by Eq. 1-2, is used to update the adaptive filter parameters according to the optimization algorithm adopted. As mentioned above, when the unknown system is nonlinear or contains feedback, an FIR adaptive filter with a reasonable length may not provide an accurate model of the system but an IIR adaptive filter can achieve a better result [3]. u Adaptive filter - y e System input Plant + d System output Figure 2-1. Adaptive system identification configuration

13 7 2.1 FIR Adaptive System Identification Finite Impulse Response (FIR) filters are digital filters with finite impulse responses. They are also known as non-recursive digital filters as they do not have the feedback (a recursive part of a filter), even though recursive algorithms can be used for FIR filter realization [10]. The equation for FIR filters can be expressed as: M 1 y(n) = k=1 ω k (n)u(n k), n = 0,1,2,,, (2-1) where parameters ω k (n), which are called tap weights, are allowed to change at every time step and u(t) is the input signal. As mentioned above, various algorithms can be used to update the tap weights and the LMS algorithm is the most widely used one. If the LMS algorithm is used, then the equation, which updates tap weights, is the same as Eq Since there is no feedback for FIR filters, it is simpler than the IIR. It does a good job if the unknown system is FIR filter as well. 2.2 IIR Adaptive System Identification Different from FIR filters, IIR filters are recursive filters that uses the output of the filter as points in the numerator of the filter coefficients. An Infinite Impulse Response (IIR) adaptive filter is a linear filter and its output depends not only on the weighted inputs but also on the previous outputs, which can be expressed as: y(n) = N k=0 a kj (n)x(n k) M + b j (n)y(n j), (2-2) j=1 where n is the iteration number; a k (n) and b j (n) are the tap weights (coefficients of numerator and denominator, respectively); and N and M are the orders of the numerator and denominator of the IIR filter, respectively [3]. The vector w(n) has the numerator and denominator filter coefficients concatenated and will be used as the previous filter values. w(n) = [a 0 (n)a 1 (n) a N (n)b 1 (n) b M (n)], (2-3)

14 8 The following equation is the error equation where e(n) is the error, d(n) is the desired signal and y (n) is the output of the filter. e(n) = d(n) y(n), (2-4) The LMS algorithm uses stochastic gradients to update the filter coefficients and multiplies them with the error to make new coefficients. The alpha and beta terms are the outcome of the derivation of the gradient operator of the algorithm. The vector η(n), like w (n) from earlier, is the concatenation of the alpha and beta gradient terms that leads to updated filter coefficients. α i (n) = x(n i) + β i (n) = y(n i) + M l=1 N l=1 b l (n)α i (n l), (2-5) b l (n)β i (n l), (2-6) η(n) = [α 0 (n)α 1 (n) α N (n)β 1 (n) β M (n)], (2-7) The final recursion is shown in equation (2-8). It can be seen that η(n), which is the vector that contains the gradient terms, gets multiplied by the error and the μ value; then added to the original filter weight. The constant μ controls the step size of the filter coefficient changes. Equation (2-8) shows how the filter is ultimately updated. w(n + 1) = w(n) + 2μe(n)η(n), (2-8) The following figure shows a simplified IIR adaptive filter block diagram from Farhang[13] and illustrates how the IIR adaptive filter works using the LMS algorithm.

15 9 Figure 2-2. Simplified IIR Adaptive Filter Block Diagram [14] As a recursive filter, the IIR filter can accomplish the filtering of a signal with a lower order than FIR filter and it can also obtain the same performance. However, as the error surface of an IIR adaptive filter tends to be multimodal, the LMS algorithm tends to become stuck at local minima [4]. Furthermore, IIR adaptive filters are susceptible to quantization effects and can more easily become unstable because of the recursive nature of the filter. There are many solutions, which have been developed, to overcome the drawbacks of the IIR adaptive systems. The issue of quantization effects can be reduced by applying a different realization of the IIR structure, such as an adaptive IIR lattice. The drawback of the instability can be solved by applying a stability check, such as converting the filter to an equivalent cascade second order terms and bounding the denominator coefficients to the stability triangle, or by using an IIR lattice structure [9]. To deal with the multimodal error surface in this thesis, the Genetic Algorithm was implemented to update the tap weights rather than the LMS algorithm.

16 10 Chapter 3 Three Structures of IIR Filters Based on the Genetic Algorithm An IIR system can be expressed as equation (2-2), where the output of this kind of system depends on the inputs as well as past outputs (with feedback). The unit impulse response of an IIR system is infinite in length and it contains a feedback loop, which means its structure is recursive. Let N = M in (2-2), then the transfer function H(z) is H(z) = Y(z) X(z) = a 0 + a 1 z 1 + a 2 z a N z N 1 + b 1 z 1 + b 2 z b N z N (3-1) There is not only one structure for IIR systems, but rather the same system function may have different structures. The paragraph below will introduce and compare the three kinds of structures called the direct form, cascade form and parallel form. 3.1 Direct Form II Structure The direct form II structure is one of the most commonly used structures that is shown in Figure 3-1. The direct form II structure can be represented by (3-2) and (3-3). The delay variable u(n) at the middle top of Figure 3-1 satisfies (3-2), and the output y(n) in Figure 3-1 satisfies (3-3). [11] u(n) = x(n) b 1 u(n 1) b 2 u(n 2) b N u(n N), (3-2) y(n) = a 0 u(n) + a 1 u(n 1) + a 2 u(n 2) + + a N u(n N), (3-3) Equation (3-2) and (3-3) are used to design an IIR filter. At the beginning u(n-1), u(n-2), are set to zero. At time n, a new input x(n) is acquired, and (3-2) is used to calculate u(n). Then the filter output becomes y(n) = a 0 u(n) + 0, At time n+1, a newer input x(n+1) is acquired and delay variables are updated by (3-2) u(n + 1) = x(n + 1) b 1 u(n) 0,

17 11 The output is y(n + 1) = a 0 u(n + 1) + a 1 u(n) + 0, and so on, for time n+2, n+3, For each specific time, a new input is acquired, then the delay variables and the output are updated by (3-2) and (3-3). Figure 3-1. Direct form II IIR filter structure 3.2 Cascade Form Structure The transfer function in (3-1) can be factored as H(z) = CH 1 (z)h 2 (z) H r (z), in terms of first- or second-order transfer functions. The cascade (or series) structure is shown in Figure 2. An overall transfer function can be represented with cascaded transfer functions. For each section, the direct form II structure or its transpose version can be used. The transfer function H(z), in terms of cascaded second-order transfer functions, can be written as [11] N/2 H(z) = a 0i + a 1i z 1 + a 2i z b 1i z 1 + b 2i z 2 i=1 Figure 3-2. Cascade form IIR filter structure

18 Parallel Form Structure The transfer function in (3-1) can also be represented as H(z) = C + H 1 (z) + H 2 (z) + + H r (z), that can be obtained via a partial fraction expansion (PFE) of H(z). The resulting parallel form structure is shown in Figure 3. Each of the transfer functions H 1 (z), H 2 (z),... can be either first-order or second-order functions. Figure 3-3. Parallel form IIR filter structure As with the cascade structure, the parallel form can be efficiently represented in terms of second-order direct form II structure sections. H(z) can be expressed as [11] N/2 H(z) = C + a 0i + a 1i z 1 + a 2i z b 1i z 1 + b 2i z 2 i=1 3.4 Comparison of three structure Now the performances of these three structures will be compared to demonstrate the differences between them.

19 Effect of Population Size Figure 3-4. Effect of the Population Size In this section the population size will be the variable and two experiments are compared in the following paragraph. These experiments are based on 2parent GA-5 th order IIR adaptive filter, the magnitude of mutation is 0.08 and the probability of mutation is 0.2. The population sizes are 60 individuals and 150 individuals. From Figure 3-4 it can be seen that the more individuals it has the lower the MSE will become, which means better results are obtained by increasing the number of individuals. Under the same number of individuals the direct form has a faster rate of converge and the lowest mean square error. Hence, the performance of the direct form is the best and parallel form is the worst.

20 Effect of Mutation Rate The mutation is set to modify the information (genes) of the individuals. It maintains the genetic diversity from one generation to the next and is considered to be the key factor to prevent the GA from becoming stuck in local minima. Mutation alters one or more gene values of the individuals and occurs according to a user-defined probability [3]. Figure 3-5. Effect of the Mutation Rate Figure 3-5 demonstrates that the mutation rate will affect the MSE directly. Mutation plays an important role in Genetic Algorithm. When increasing the mutation it will increase the possibility to change the weights during the algorithm and that will decrease the rate of convergence. From the above we can clearly see the dotted lines, whose mutation is 0.4, have a lower rate of convergence and a higher final MSE than the corresponding full line whose mutation is 0.2. In term of filter s structure, it can be seen that the direct form always has a better performance than others. When the mutation is 0.4, the parallel form and cascade form filters have the similar performance and when mutation is 0.2 the cascade form has a better performance than parallel form.

21 15 Figure 3-6. Effect of the Mutation Rate It can be seen from Figure 3-6 that when the mutation becomes large enough, the MSE will increase after a little convergence. Hence, the mutation probability should not be set too large; otherwise, the GA will become a random search and the system will become unstable. It also cannot be too small, otherwise it would take a long time for the search to get out of local minima. It is also obviously to be seen that the cascade form and parallel form filters are easier to be effected by the mutation than the direct form filter. Hence, the cascade form and parallel form filter are more sensitive than the direct form on the mutation.

22 Effect of Input Signal Figure 3-7. Effect of the Input Signal Characteristics From figure 3-7 we can see the different performances for the different inputs. It is easy to see out that when a color noise signal is used as the input the filter has a worse rate of convergence and higher MSE. The input signal that with more noise will take more time to converge. In terms of structure of the filter, the type of input doesn t change the relative performance of the three filters, direct form has the best performance and parallel form has the worst. It can be seen that the GA is not strongly affected by the structure of filters.

23 Effect of the Number of Parents Figure 3-8. Effect of the Number of Parents In figure 3-8 where the variable is the number of parents, it can be seen that when increasing the number of the parents the rate of convergence increased, although, the MSE becomes higher. In terms of their structure, the parallel form and cascade have the similar performance but the direct form appears to have the best performance. Now the performance of the 2-parent and the modified multi-parents will be compared. For multi-parents, the modified MPGA changes the amount of parents automatically. From figure 3-8 and figure 3-9 it can be clearly seen that the rate of convergence of modified multi-parents is faster than the 2-parent one and multi-parents one. But it has a higher MSE than the others. In term of their structure, parallel form and cascade have the similar performance for 2-parent and direct form has a better performance. For the modified multi-parents the direct form and the cascade have similar performance and the parallel form has a worse performance.

24 18 Figure 3-9. Effect of the Modified Number of Parents in the MPGA Chapter 4 Combined Genetic Algorithm and Gradient Algorithm From early research we can see the advantages and disadvantages of each algorithm, then an idea to combine them together came up. Following we will see if we can get better performance when we first apply the Genetic Algorithm and then apply the Gradient Algorithm. The first case will be to apply the combined algorithm approach to the FIR filter. 4.1 Combine Genetic Algorithm and Gradient Algorithm for the FIR filter From figure 4-1 and figure 4-2 we can see Genetic Algorithm (GA) did a bad job on FIR filter when the error surface converged the mean square error is still high. However, the gradient algorithm did a good job on the FIR filter, its error converged to a lower mean square

25 error than the GA did. Now it will be shown how the FIR filter performs if we combine these two algorithms together. 19 Figure 4-1. Gradient Algorithm Applied to the FIR filter Figure 4-2. Genetic Algorithm Applied to the FIR filter

26 20 Figure 4-3. Combination of Genetic Algorithm and Gradient Algorithm Applied to the FIR filter From the figure 4-3 we can easily see the combination performed better than either of those two algorithms separately. It is obvious that the combination improves the performance of FIR filter. However, the LMS part in the combination did not work as well as it did separately. Figure 4-1 indicated that the Mean Square Error (MSE) converged approximately from 0 DB to -50 DB, but in the combination the LMS part only converge the MSE from -10 DB to -55 DB so the difference is 45 DB. And it is also to be seen that if we apply the LMS algorithm at the moment that GA just converged, the total rate of convergence is higher than both of them separately. However, the combination and the separate LMS approximately converge to the similar MSE. Hence, it is not very obvious that the combination is able to achieve a lower error but it clearly improves the rate of convergence of the FIR adaptive filter. Now the same combination strategy will be applied to the IIR adaptive filter to determine if this strategy will achieve a better result.

27 Combine Genetic Algorithm and Gradient Algorithm on IIR filter Figure 4-4. Gradient Algorithm Applied to the IIR filter Figure 4-5. Genetic Algorithm Applied to the IIR filter

28 22 The result is opposite to the FIR one, Genetic Algorithm performed better than Gradient Algorithm. The MSE of Gradient Algorithm made the MSE converge to -1.5 DB and the Genetic Algorithm made the MSE converge to -18 DB. The Genetic Algorithm (GA) got a better result, which indicated the GA performed better on IIR filter than LMS algorithm. Now the combination of these two algorithms will be demonstrated: Figure 4-6. Combination of Genetic Algorithm and Gradient Algorithm Applied to the IIR filter From figure 4-6 it can be easily seen that the MSE converged to -34 DB and it seems to be still converging, which is much better than both of these algorithm separately. From these results can conclude that the combination strategy significantly improved the performance of the IIR adaptive filter. It also can be seen that the LMS part of combination converged lower than the LMS did separately. That means the convergence, which is contributed by GA, allows the LMS algorithm to more easily converge to the global minimum. Comparing the combination on FIR and IIR filters we can conclude that the improvement for performance of combination on IIR is more obvious than it on FIR.

29 23 Chapter 5 Conclusion and Future Work Based on the experimental results in the previous chapter, it can be concluded that, for the performance of three different structures of IIR adaptive filters, the direct form has the best performance, the cascade form is a bit worse than the direct form, and the parallel form has the worst performance. It also can be seen that more individuals will result in a lower mean square error, high mutation may make the result unstable, more parents tend to improve the rate of convergence, and colored noise is more difficult to be handled than white noise. Another conclusion is that no matter what conditions are changed the relative performance of these three structures will not significantly change. This suggests that the Genetic Algorithm can be applied to arbitrary filter structures and the performance will not be strongly influenced by the type of filter structure. Future research will further investigate the use of a final gradient stage to minimize the final converged mean square error. In terms of FIR adaptive filters the Gradient Algorithm LMS generally works better than the Genetic Algorithm. However, in terms of the IIR adaptive filter the Genetic Algorithm appears to do a better job than the Gradient Algorithms, which means that the Genetic Algorithm is suitable to be applied on the IIR adaptive filter and it will improve the performance of the IIR adaptive filter. In this thesis the research came up with a new strategy that combines the Genetic Algorithm and Gradient Algorithm together. From the experimental results we can find the combination really improves the performance of the adaptive filter and it improves the IIR filter a lot. Since the IIR adaptive is very sensitive on Gradient Algorithm, we can use the combination of Genetic Algorithm and Gradient Algorithm to improve the performance of the adaptive filter. Future research will investigate the performance of three different structures based on this combination strategy.

30 24 Bibliography [1] B. Widrow and M. E. Hoff, "Adaptive Switching Circuits," 1960 Wescon Conv. Rec. pt. 4, pp , August [2] S. Haykin, "Adaptive Filter Theory," Englewood Cliffs, NJ: Prentice Hall, [3] Guoxin Sun, "Performance of multi-parents genetic algorithms (MPGA) for IIR adaptive system identification," M.S. Thesis, Department of Electrical Engineering, Penn State University, State College, PA, [4] B. Widrow and S. D. Stearns, "Adaptive Signal Processing," Englewood Cliffs, NJ: Prentice Hall, [5] D. J. Krusienski and W. K. Jenkins, "Design and Performance of Adaptive Systems Based on Structured Stochastic Optimization Strategies," Circuits and Systems Magazine, Vol. 5, No. 1, pp 8 20, February [6] J. So and W. K. Jenkins, "Comparison of CAT Swarm Optimization with Particle Swarm Optimization for IIR System Identification," Proceedings of the Forty-Seventh Annual Asilomar Conference on Signals, Systems, and Computers, Pacific Grove, CA, PP , November 4-7, [7] David E. Goldberg, "Genetic Algorithms in Search, Optimization, and Machine Learning," Addison-Wesley Longman Publishing Co., Inc. Boston, MA, USA, [8] R. Nambiar, C.K.K. Tang and P. Mars, "Genetic and Learning Automata Algorithms for Adaptive Digital Filters", Proc. IEEE Int. Conf. on ASSP, 1992, vol. IV, pp [9] Shynk, J. J., "Adaptive IIR Filtering", IEEE ASSP Magazine, pp. 4-21, April [10] Zoran Milivojević, "Digital Filter Design", MikroElektronika, 1st edition, ( [11] Rulph Chassaing, "Digital Signal Processing and Applications with the C6713 and C6416 DSK", Worcester Polytechnic Institute, Wiley Interscience, A JohnWiley & Sons, Inc.

31 25 [12] Yao L., Sethares W. A., "Nonlinear Parameter Estimation via the Genetic Algorithm," IEEE Transactions on Signal Processing, vol.42, April [13] Z. J. Cavaliero and W. K. Jenkins, " Comparative Analysis of the Use of IIR Adaptive Filters vs. FIR Adaptive Filters to Implement Sequential Processing for Fetal Electrocardiograms," [D] Penn State University, State College, PA [14] Farhang-Boroujeny, B. Adaptive Filters: Theory and Applications, Sussex: John Wiley & Sons, Print.