A Hybrid Maximum Error Algorithm with Neighborhood Training for CMAC

Transcription

1 A Hybrid Maximum Error Algorithm with Neighborhood Training for CMAC Selahattin Sayil Department of Electrical Engineering Pamukkale University Kinikli-Denizli, TURKEY Kwang Y. Lee, Fellow IEEE Department of Electrical Engineering The Pennsylvania State University University Park, PA 16803, U.S.A Abstract In this paper, several possible algorithms and training methods for the CMAC network are analyzed thoroughly. Improvements are then examined and a Hybrid approach has been developed for the Maximum Error Algorithm by using the Neighborhood Training for the initial training period. The employment of the technique yielded faster initial convergence which is very important for many control applications. The proposed hybrid approach is demonstrated in an inverse kinematics problem of a two-link robot arm. Intex Terms: CMAC algorithms, Maximum error algorithm, Neighborhood training. I. INTRODUCTION The CMAC (Cerebellar Modular Articulation Controller) is a mathematical formalism developed by Albus to model the information processing characteristics of the cerebellum [1],[2]. The CMAC, as a controller, computes control values by referring a memory look-up table where those control values are stored. Memory table basically stores the relationship between input and output or the control function. In comparison to other neural networks, CMAC has the advantage of very fast learning and it has the unique property of quickly training certain areas of memory without affecting the whole memory structure. The convergence of a CMAC depends on how input training points are chosen. It has been shown that CMAC learning always converges to some arbitrary accuracy on any set of training data [3],[4]. The CMAC has an advantage in training speed and this property is very important in real-time applications such as in load forecasting [5]. The same benefits may be gained in adaptive control systems [6]. The performance of a CMAC neural network depends on the training algorithms and the selection of input points. Papers have been published that explain CMAC algorithms, however, little work has been done to improve existing algorithms [16]. In this paper, using computational results and the algorithm properties, the existing algorithms for CMAC are first compared. Improvements are then examined and a "Hybrid Algorithm" approach has been developed. In this method, a CMAC network is first trained by using Neighborhood Training Algorithm and then trained by the Maximum Error Algorithm for fine-tuning of the CMAC network. With the hybrid approach, faster initial convergence is achieved. The proposed "Hybrid Algorithm" approach is also applied to solve the inverse kinematics problem of a twolink robot arm and successful results are obtained. The new algorithm may reduce the training time and increase the initial learning rate, which is very important for control applications. The paper is organized as follows: A brief description of CMAC is given in Section II. Various CMAC algorithms are reviewed and their performances are compared in Section III, and a hybrid of the Maximum Error Algorithm and the Neighborhood Training is developed in Section IV. The proposed hybrid algorithm is demonstrated in Section V to solve an inverse kinematics problem of two-link robot arm, and conclusions are drawn in Section VI. II. CMAC In CMAC, learning is iterative and 'teacher' supplies important information about the desired function to be learned and CMAC adapts itself using this feedback information (Figure 1). This is called supervised learning. The input selects the active of CMAC depending on its value. Then active are summed and averaged to calculate an output. This output is compared with the desired output, and CMAC uses the difference to adjust its to get the desired response. Input value s(k) select Adjustable... sum active Adjust active Average Weights Figure 1. Supervised learning in the CMAC structure. Computed output r(k) + Desired output d(k) -

2 Albus based CMAC on the perceptron by Rosenblatt and the mappings [7]: S A P (1) where S is sensory input vector, A is the binary association matrix, and P is the output response. The first mapping is a non-linear transformation that maps the input variable s, into a binary vector that indeed represents the location of chosen by the corresponding input. The generalization factor, ρ, is a design parameter and determines the number of active cells with "ones" in the association vector. In the second mapping of the CMAC, the output values are formed first by summing of all the of the cells in the association vector which have been excited by a particular input and then taking their average. In the training phase, are adjusted in such a way that the output of the CMAC approaches the desired output. Training of takes place in a supervised learning environment. Given the k-th training sample the error between the desired output d(k) and the calculated output r(k), is e(k) = d(k) - r(k). This error is then used as a correction to each of the memory cells excited by the k-th input. For each excited memory element, the correction of weight is W = γ e( k), where, γ is the learning factor. Miller identifies the CMAC training algorithm as the well-known Least Mean Squares (LMS) algorithm [8]: 1 T Wk () = Wk ( 1) + γ{() dk a() kwk ( 1)}() ak (2) ρ where a(k) is the binary association vector, and ρ is the generalization parameter. III. COMPARISON OF CMAC TRAINING ALGORITHMS A. Selection of Training Input Points Parks and Militzer (1992) discusses Cyclic and Random Training methods for the CMAC algorithms. In Cyclic (Sequential) Training, a cycle is defined for the input training points and during the training this cycle is repeated until a desired performance is reached. If the training points are in the same neighborhood of the previous input points, good output convergence may not be obtained. Random Training method could be used to prevent the cross-talk (interference) between the input training points. A pseudo-random number generator could be used to generate uniformly distributed random numbers at each training session. In this manner, learning interference is greatly minimized because of the reduction of repetitive learning at the same neighborhood. B. Comparison of Training Algorithms: The following algorithms were investigated: 1. Albus Instantenous Learning (Nonbatch) Algorithm. 2. Batch Learning Algorithm. 3. Maximum Error Algorithm. 4. Gram-Schmidt Procedure. 5. Partially Optimized Step Length Algorithm. 6. Moving Average Algorithm. In CMAC, weight updates can be made after each presentation (Non-batch Algorithm) or after a batch of presentations (Batch Algorithm). Instantaneous Error Correction Algorithm developed by Albus updates the network weight vector after each training pair is presented to the network [2]. The rate of convergence of the algorithm depends directly on the orthogonality (no interference between patterns) of the training patterns [9]. A new learning algorithm, Gram-Schmidt Procedure are derived which produce weight changes perpendicular to the previous L weight changes (L = user defined parameter) and so the learning is orthogonal for the last L training examples, [9],[10]. The Moving Average Algorithm uses a counter vector c, to indicate how often a weight has been updated and the update is forwarded to less frequently changed weight elements for a particular input [10]. The Maximum Error Training Algorithm, developed by Parks and Militzer [10], is based on the algorithm proposed by Albus [2]. The algorithm takes L training input points consisting of the current training input point and L-1 past input points and selects the one with the largest output error. Partially Optimized Step Length Algorithm tries to minimize the sum of errors for the last L training patterns rather than minimizing the square of error for one input. The details of these algorithms can be found in [1]-[2], and [9]- [11]. In the comparison, the computational results and algorithm properties have been utilized. The training is performed either in a cyclic or in a random manner. The following two functions are used as target functions for generating the training data: Case A) An arbitrary mathematical function f (x) which is defined for 180 x 180 : 1 1 f( x) = 2 sin( x) sin(2 x) + sin(3 x) cos( x) cos(2 x) + cos(3 x) 2 3 Case B) A composite function which has a discontinuous first derivative at x = (N-1)/3 where N=361, and g(x) that is defined for 0 x 360 3x 1 3x 1 g x e N 3 (3) ( ) = (4) e N 1 Desirable properties of learning algorithms are initial fast learning and long-term convergence. The computational time of each algorithm is also taken into consideration. These

3 criteria formed basis for comparison. The following abbreviations are made: ALC: ALR: AVC: AVR: GRC: GRR: MAC: MAR: OSC: OSR: BATC: BATR: Albus (nonbatch) Cyclic Training Albus (nonbatch) Random Training Moving Average Cyclic Training Moving Average Random Training Gram-Schmidt Proc. Cyclic Training Gram-Schmidt Proc. Random Training Maximum Error Cyclic Training Maximum Error Random Training Optimized Step Cyclic Training Optimized Step Random Training Batch Alg. with Cyclic Training Batch Alg. with Random Training In both cases, the target functions are the given functions in (3) and (4), and the parameters in the training are: ρ=10 and B=361 (Resolution of 1 degree). Comparing the two cases, the second function (Case B) is actually harder than the first one in terms of CMAC learning because of its discontinuous first derivative. From the results it can easily be seen that the Albus Nonbatch Algorithm is the least time requiring algorithm for training data and is convenient for on-line learning because of its easy algorithm implementation. But there is a risk if modelling error or measurement noise exists in the network. In that case of mismatch, the weight vector never converges to its optimal value, instead it converges in a domain which surrounds this optimal value, called minimal capture zone [9]. As it can be seen in Tables 1 and 2, when Cyclic Training is applied, slow initial convergence occurs. This is due to the interference which is inherent in the Cyclic Training nature. The employment of Random Training may improve initial convergence. The drawback in the Random Training is that it may sometimes get stuck since the Random Training process can choose points at which it is already reasonably well trained. At some point CMAC may not learn anything. The work done by Ellison also explains this fact [12]. Initial convergence in the Moving Average Algorithm is better than Albus Algorithm when Cyclic Training is used, but later on convergence rate gets slower and slower. The computational effort is the second best after Albus Nonbatch Algorithm. The Optimized Step Algorithm with Cyclic Training shows a slow convergence because the transformed input patterns are parallel to each other. Since the weight vectors are optimized with respect to the last L pieces of data, inputs are still correlated. With Random Training the initial convergence improves but long term convergence still remains the same. The best result is recorded with small L value. Large values may slow down the algorithm. Gram-Schmidt procedure gives the fastest convergence, but it takes almost a quarter of a second just to store one single point. Larger L values, gives more accuracy and faster convergence but also requires tremendous amount of computation time. Algorit. L TABLE I. COMPARISON OF TRAINING ALGORITHMS FOR CASE A. 361 pts. =1epoch 3610 pts. =10 epoch pts. =50 epoch pts. 100 epoch Exec.time / single point t(msec) ALC % 10.20% 0.139% 0.076% 3.30 ALR % 1.130% 0.198% 0.073% 3.18 AVC % 0.910% 0.533% 0.460% 5.50 AVR % 0.920% 0.543% 0.492% 3.18 GRC % % % % GRC % % % % GRR % % % % GRR % % % % MAC % % % % MAC % % % % MAR % % % % MAR % % % % OSC % % % % OSC % % % % OSR % % % % OSR % % % % epch 3610 epoch epoch BATC % 0.176% 0.060% BATR % 0.220% 0.067%

4 Algorit. L TABLE II. COMPARISON OF TRAINING ALGORITHMS FOR CASE B. 361 pts. =1epoch 3610 pts. =10 ep pts. =50 ep pt. =100 ep. Exec.time / single point t(msec) ALC % 13.61% 0.323% 0.165% 3.30 ALR % 1.23% 0.207% 0.110% 3.18 AVC % 1.10% 0.669% 0.566% 7.90 AVR % 1.22% 0.685% 0.592% 7.91 GRC % 0.12 % % % GRC % 0.08 % % % GRR % 1.19 % % % GRR % 0.41 % % % MAC % 0.48 % % % MAC % 0.46 % % % MAR % 0.98 % % % MAR % 0.62 % % % OSC % 2.21 % % % OSC % 0.97 % % % OSR % 1.29 % % % OSR % 1.23 % % % epch 3610 epoch epoch BATC % 0.303% 0.069% BATR % 0.490% 0.077% The Maximum Error Algorithm has a good convergence property. Initial learning rate is especially high and could be made higher using Neighborhood Training technique. The long-term convergence is the best after Gram-Schmidt procedure. It has a moderate computation time compared to others. Choosing L larger increases the accuracy but requires much computation time and effort, and a trade-off may be made between accuracy and the time requirement. Batch Training doesn't look appropriate for our purposes. The requirement that all the information about the training data points in one pass should be known limits the usage of this technique. The recommended algorithm in this paper is the Maximum Error Algorithm because of its moderate computation time, fast initial and long term convergences. The works of Ellison and also Parks and Militzer agrees with this result [10],[12]. IV. HYBRID APPROACH: THE MAXIMUM ERROR ALGORITHM WITH NEIGHBORHOOD TRAINING The Maximum Error Algorithm has been the best algorithm in terms of its moderate computation time and good convergence properties. But can there be any improvement on Maximum Error Algorithm? From Tables I and II and discussions before, we conclude that the initial convergence may be improved by "Neighborhood Training". Thompson and Kwon suggested the Neighborhood Training method to avoid the interference resulting from the CMAC's generalization property [13]. In this method, input training points that lie outside the neighborhood of previous input training points are selected so that no interference will occur between the training sessions. The total number of input points to be trained depends on the resolution of the bins and ρ. For an arbitrary function (eqn. 3) with 180 x 180, a resolution of 1 degree and ρ=10, the total number of Neighborhood Training points, N t is calculated as N t = 360/10+1=37. The input values are computed by: s = s + nρr, n = 1,2,,N t (5) n 0 where, s 0 is the smallest input value, and r is the input resolution. After selecting the input values for training, the Albus Nonbatch Algorithm can be applied for training at the neighborhoods. W = W + µ a( k) k k 1 k T µ = ρ. d( k) a( k) W k k 1 n = 1,2,,N t (6) where, anis ( ) the normalized form of the n-th association vector, and d(n) is the desired output. Data for each memory element are adjusted only one time (with gain=1) during training. After applying the Albus Training Algorithm, the result is appreciable because of the speed advantage of this method. Since Neighborhood Training minimizes the learning interference, Lin and Fisher later used this form of training in a arc-welding process control system [14]. In the proposed hybrid algorithm approach, the CMAC is pre-trained with "Neighborhood Training" before it is fine tuned with the recommended Maximum Error Algorithm. In pre-training, instead of using only one pass of neighborhood at 37 points, a second pass of Neighborhood Training are also utilized in the midpoints. These mid-points corresponded to the half of the previous neighborhoods, but among them there were no interference as in the first session.

5 After the initial neighborhood training, the rms error for Case A was 1.45% and the rms error for Case B was only % (Figures 2 and 3). For both functions, L value is chosen to be 10. From Table III, it can be concluded that the initial convergence improves appreciably. For Case A, after the first epoch of Maximum Error Algorithm in the hybrid approach, the rms error becomes 1.18%. One compares this with 21.24% for Cyclic Trained Maximum Error Algorithm and 5.7% for Randomly trained Maximum Error Algorithm after 1 epoch (cf. Table I). The hybrid approach requires 73 extra training points. But, since we are using Albus Nonbatch Algorithm in pre-training, the computational time is quite low (cf. Tables I and II). After 10 epoch, the rms error further drops down to 0.23% instead of 0.375% and 1.159% corresponding to the Cyclic and Random Trainings, respectively. The initial convergence rate was better for the second function. After the first epoch of Maximum Error Algorithm 0.065% rms error is obtained compared to % for Cyclic Trained Maximum Error Algorithm and 6.71 % for Randomly trained Maximum Error Algorithm (cf. Tables II and III). Long term convergence also well improves from 0.035% to 0.017% for Case B (cf. Tables II and III). In conclusion, the combination of these two algorithms improves CMAC neural network convergence considerably. Figure 2. f(x) after the 2-pass Neighborhood Training. Figure 3. g(x) after the 2-pass Neighborhood Training. TABLE 3. RECOMMENDED ALGORITHM (L=10). Neighborhood Training Algorithm Maximum Error Algorithm with Neighborhood Training (L=10) Case 1st pass 2nd pass 361 pts. =1 epoch 3610 pts. =10 epoch pts. =50 epoch pts 100 epoch Mean Exec. time t (msec) A 3.20% 1.45% 1.180% 0.230% 0.040% 0.016% 30 B 0.14% 0.11% 0.065% 0.034% 0.024% 0.017% 30 V. AN INVERSE KINEMATICS PROBLEM The Hybrid Maximum Error Algorithm with Neighborhood Training can be applied to solve an inverse kinematics problem of a two-link robot arm to demonstrate the effectiveness of the hybrid method. The two-link robot has rotary joints and the tip position is commanded to follow a straight line trajectory. Inverse kinematics deals with determining what joint angles of the robot are needed to force the robot tip to follow the line joining specified end points. The joint angles are given by Yao and Zhang [15]: x + y L1 L2 θ2 = arccos( ) 2LL 1 2 L2sin( θ2) θ1=arctan(y,x) - arctan( ) L + L cos( θ ) where, θ 1 and θ 2 is the first and the second joint angle, respectively, with 0 θ 1, θ 2 π, (x,y) are the coordinates of the arm, L 1 and L 2 are the link lengths. Initial position and the final position of the straight line trajectory were chosen as (X 1,Y 1 ) = (2,3) and (X 2, Y 2 )=(-2,3), respectively, with link lengths given as L 1 = L 2 = 2. The CMAC network can learn the inverse kinematics of a two-link robot arm and then predict the joint angles. Two (7)

6 different CMAC networks have been utilized for learning θ 1 and θ 2 in this problem. The generalization factor ρ chosen was 4 for the two networks. The initial training with Neighborhood method showed a low rms error value of % for θ 1 and an rms error value of % for θ 2. The Maximum Error Algorithm has been employed to reduce these rms error values further down. After the 100-th epoch of the Maximum Error Algorithm, rms values were only 0.025% and %, for θ 1 and θ 2, respectively. These results can also be visualized in Figures 4 and 5. The figures also compare the "Hybrid Algorithm" approach to the original Maximum Error Algorithm if it were to applied alone without Neighborhood Training to predict the joint angles. From these figures, it can be concluded that the initial convergence rates for both joint angles were made considerably higher than the original Maximum Error Algorithm by employing the proposed "Hybrid Algorithm" approach. VI. CONCLUSION Several possible algorithms and training methods for the CMAC network are analyzed thoroughly. The selection of the input training points was found to be important, and for that reason beside the Cyclic Training, Random Training is also taken into consideration in comparing different algorithms. Among the algorithms, Maximum Error Algorithm has been recommended because of its moderate computation time, and fast initial and long-term convergence. An improvement has been made for the recommended Maximum Error Algorithm by using the Neighborhood Training technique and this resulted in faster initial convergence. The hybrid Neighborhood Training and Maximum Error Algorithm was able to reach an appreciable accuracy in a small period of time. The acceleration in the convergence may depend on shape and complexity of a function as well as other network parameters. The hybrid Neighborhood Training and Maximum Error Algorithm is also applied to an inverse kinematics problem of a two-link robot arm and successful results were obtained with given a trajectory. Figure 5. The training results for θ 2. REFERENCES [1] J.S. Albus, "A New Approach To Manipulator Contol: The Cerebellar Model Articulation Controller (CMAC)", Transactions of the ASME, Vol. 97, , [2] J.S. Albus, "Data Storage In The Cerbeller Model Articulation Controller (CMAC)", Transactions of the ASME, Vol. 97, , [3] Y. F. Wong and A. Sideridis, Learning Convergence In The Cerebellar Model Articulation Controller, IEEE Transactions on Neural Networks, Vol.3, , [4] C.-S.Lin and C.-T.Chiang, "Learning Convergence of CMAC Technique", IEEE Transactions on Neural Networks, vol.8, , [5] T. H. Lee and K.Y. Lee, "Application of CMAC for Short-Term Load Forecasting", Proc. International Conference on Intelligent System Applications to Power Systems, pp ,1997. [6] G. Cembrano, G. Wells, J. Sarda, and A Ruggeri, "Dynamic Control Of A Robot Arm Using CMAC Neural Networks", Control Engineering Practice, Vol.5, pp ,1999. [7] F. Rosenblatt, Principles of Neurodynamics: Perceptrons and The Theory of Brain Mechanisms. Washington, DC: Spartan Books, [8] W. T. Miller, H. G. Filson, and L. G. Craft, "Application of A General Learning Algorithm To The Control of Robotic Manipulators", The International Journal of Robotics Research, Vol. 6, , [9] M. Brown and C.J. Harris, Neurofuzzy Adaptive Modelling and Control, Prentice Hall, [10] P. C. Parks and J.Militzer, "A Comparison Of Five Algorithms For The Training of CMAC Memories For Learning Control Systems", Automatica, Vol.28, , [11] M. Brown and C.J. Harris, "Least Mean Square Learning In Associative Memory Networks", Proc. IEEE Int. Symposium on Intelligent Control, Glasgow,UK, [12] D. Ellison, On The Convergence Of The Albus Perceptron, IMA J. Math. Cont. and Information, Vol.5, ,1988. [13] D. E. Thompson and S. Kwon, Neighborhood And Random Training Techniques For CMAC IEEE Transactions on Neural Networks, Vol.6, , [14] R. -H.Lin and G. W. Fischer, "An Online Arc Welding and Quality Monitor and Process Control System", Proc. Int. IEEE/IAS Conf. on Industrial Automation and Control, pp ,1995. [15] S. Yao and B. Zhang, The Learning Convergence Of CMAC In Cyclic Learning", Proc. International Joint Conference on Neural Networks, , 1993 [16] T.Szabo and G.Horvath, "Improving The Generalization Capability Of The Binary CMAC", Proc. International Joint Conference on Neural Networks, Vol.3, 85-90, Figure 4. The training results for θ 1.