COMPARISON BETWEEN ARTIFICIAL NEURAL NETWORKS AND NEURO- FUZZY SYSTEMS IN MODELING AND CONTROL: A CASE STUDY

Transcription

1 COMPARISON BETWEEN ARTIFICIAL NEURAL NETWORKS AND NEURO- FUZZY SYSTEMS IN MODELING AND CONTROL: A CASE STUDY José António Barros Vieira, Fernando Morgado Dias, Alexandre Manuel Mota 3 Escola Superior de Tecnologia de Castelo Branco, Departamento de Engenharia Electrotécnica, Av. Empresário, 6 Castelo Branco, Portugal, Tel: , zevieira@est.ipcb.pt Escola Superior de Tecnologia de Setúbal, Departamento de Engenharia Electrotécnica, Campus do IPS, Estefanilha, 94-8 Setúbal, Portugal Tel: , fmdias@est.ips.pt 3 Departamento de Electrónica e Telecomunicações, Universidade de Aveiro, 38 Aveiro, Portugal, Tel: , alex@det.ua.pt Abstract: This article presents a comparison of Artificial Neural Networks and Neuro- Fuzz Sstems applied for modeling and controlling a real sstem. The main objective is to control the temperature inside of a ceramics kiln. The details of all sstem components are described. The steps taken to arrive at the direct and inverse models using the two architectures: Adaptive Neuro Fuzz Inference Sstem and Feedfoward Neural Networks are described and compared. Finall, real time control results using Internal Control strateg are presented. Using available MATLAB software for both algorithms, the objective is to find which solution performs better comparing the performances of the solutions through different parameters for a specific case. Copright IFAC 3 Kewords: temperature control, fuzz hbrid sstems, artificial neural networks, applied neuro-fuzz control, model based control and real time control.. INTRODUCTION The purpose of the present paper is to compare, using a case stud, two solutions for modeling that became ver popular in the last decades: Artificial Neural Networks (ANN) and Neuro Fuzz Sstems (NFS). This comparison is to be done using available MATLAB software. The field of ANN has crossed different stages of development. One of the most important steps was achieved when Cbenko (Cbenko, et al., 989) proved that the could be used as universal approximators. A negative stage had been brought two decades earlier b the book of Minsk and Papert called Perceptrons (Minsk, et al., 969), where among other examples it was shown that a single laer of perceptrons could not represent a simple function like the Exclusive OR. This negative phase was overcome when algorithms for training of multilaer ANN where proposed in the decade of the 8s. Since then much work has been done regarding ANN and their application to man different fields. A reasonable slice of this work has been in the modelling and control field where ANN hold the promise of being capable of producing nonlinear models and controllers, being able to work under noise conditions and being fault tolerant to the loss of neurons or connections. The field of NFS starts in the end of the 8s and presents a big growth in the decade of the 9s with a large variet of different approaches. These approaches mix the ANN with fuzz inference sstems (FIS) in three was: cooperative, concurrent and fused. The most common architecture is the fused NFS that uses neural networks just to learn some internal parameters of a fixed structure (Nauck, et al., 997). The Adaptive Neuro Fuzz Inference Sstem () belongs to the fused NFS, was introduced b Jang (99) and it is able to approach an linear or non-linear function (universal approximator) (Jang, et al., 99). The present case stud is a reduced scale prototpe kiln, which is non linear and will be working under measurement noise. The work described in this article is still under development and is part of an interdepartmental project at the Universit of Aveiro, which will lead to the control of the atmosphere inside the kiln using two loops for temperature and for air/oxgen ratio control. To test the models in real time control action the Internal Control (IMC) strateg was used.. THE KILN Non-linearit and noise have alwas been a major problem in control sstems. This tpe of kilns are non-linear sstems because their temperature does not depend onl on the heating control variable but also on the exchange of heat with the exterior world and the present sstem also has measurement noise because of the tpe B thermocouple used.

2 The sstem is composed of a kiln, electronics for signal conditioning, power electronics module and a Data Logger from Hewlett Packard HP3497A to interface with a Personal Computer (PC) connected as can be seen in figure. Fig.. Power Module Heating Element Kiln Chamber Thermocouple Data Logger PC Modules that compose the sstem. Through the Data Logger bi-directional real-time information is passed: control signal supplied b the controller and temperature data for the controller. The temperature data is obtained using a thermocouple. The power module receives a signal from the controller implemented in the Personal Computer, which ranges from to 4.9V and converts this signal in a power signal of V applied during a period of time proportional to the input signal. The Data Logger is used as the interface between PC and the rest of the sstem. Since the Data Logger can be programmed using a protocol called Standard Commands for Programmable Instruments (SCPI), a set of functions have been developed to provide MATLAB with the capabilit to communicate through the RS- 3C port to the Data Logger. measured temperature to values superior to 3ºC (due to the characteristics of the thermocouple) and the thermocouple introduces measurement noise. After the observation of the step response of the kiln in several reference temperatures and using the lipschit function (Nørgaard, 996b) to determine the lag space some conclusions were achieved: the kiln can either be considered a first or a second order sstem. The kiln shows a non linear behavior, due to the different behavior in the heating and in the cooling phases. This is because of the energ losses of the kiln structure to the exterior that are dependent of the temperature in the kiln. The identification data has been chosen to respect two important requirements: frequenc and amplitude spectrum wide enough (Jang, et al., 99) (Sørensen, et al., 994). With this concern and with a sampling period (h) of 3 seconds, the operation of collecting data was made. Training structures used for direct and inverse models are described in figures 3 and 4. These structures are the most common solutions for training models and are described in several articles (Pradeep, et al., 988) and (Hunt, et al., 99). Considering the analsis done the kiln direct and inverse models were obtained with two different groups of regressors. First, considering the kiln a first order sstem and second, considering the kiln a second order sstem. Considering that k=n*h, where k is the time instant, n is the iteration and h the sampling time, to obtain the direct models, the prediction of the temperature at time k, pred(k) is given b equation and, for first and second order approaches: pred(k)= f( (k-h),u(k-h) ) () pred(k)= f( (k - h),(k - h),u(k - h),u(k - h))() To obtain the inverse models, the prediction of the control signal at time k, upred(k) is given b equation 3 and 4 for first and second order approaches: upred(k) = f( (k h), (k)) (3) upred(k) = f( (k h),(k),(k h),u(k h)) (4) Fig.. Picture of the kiln and electronics. Using the HP349A (6 analog inputs) and HP3497A (digital inputs and outputs and two Digital to Analog Converters) modules together with the developed functions it is possible to read and write values, analog or digital, from MATLAB. A picture of the sstem can be seen in figure. The kiln can be seen in the centre and at the lower half the prototpes of the electronic modules. 3. IDENTIFICATION / MODULATION In the identification phase two problems arise as the first data is collected: the data logger limits the Considering the kiln s behavior, ANN like and NF sstems like were considered good approaches for building the direct and inverse non linear models of the kiln. (k) (k-h) (k-h) u(k-h) u(k-h) Direct error(k) pred(k) Fig. 3. Structure for direct model training (first and second order sstems ). -

3 u(k) Fig. 4. Structure for inverse model training (first and second order sstems ). 3.. Artificial Neural Netwoks Artificial Neural Networks are artificial and simplified models of the neurons that exist in the human brain. The can be used as a black box approach to create models of sstems profiting of the facilit to model non linear (as well as linear) sstems. Their abilit relies on the qualit of the signals used for training and the performance of the training algorithms and their parameters do not contain information that can be directl understood b the human operator or that can be related to the phsical properties of the sstem to be modelled. are a sub tpe of ANN in which the onl connections allowed between neurons are feedforward, i.e. there are no lateral or feedback connections Feedforward Neural Networks Architecture A Feedforward Neural Network () is a laered structure, which can include non-linearit. The basic element of a is the neuron that is shown in figure. Fig.. I I I3 error(k) - upred(k) w w w 3 F Neuron structure. Inverse The neuron implements the general equation: n = F( Ii.wi) () i= where usual functions for F are sigmoidal, linear and hard limit. A is composed of an input laer, one or more hidden laers with one or more neurons and an output laer where frequentl the neurons are linear. The Multi Input Single Output in figure 6 implements the following general equation: nh ni = F ( w' jf j( wljil )) (6) j= l= A tpical structure of can be seen in figure 4. w 4 (kh) (k) (k-h) u(k-h) I I I3 Fig. 6 w w w w w 3 w3 f (.) w' w 4 w f (.) 4 w' w' 3 F (.) Feedforward Neural Network structure Learning Algorithms of Man algorithms have been developed to use with like the well known Backpropagation or the most effective Levenberg-Marquardt. The algorithms developed or adapted for the use with are based on minimizing a criterion (which is most frequentl based in the error between the desired and the obtained output). Most of them are based on derivative calculations of the error as a mean to minimize it. The Levenberg-Marquardt algorithm was chosen because of the robustness and fastest convergence. 3.. Neuro-Fuzz Sstems A FIS can use human expertise b storing its essentials components in a rule base, and perform fuzz reasoning to infer the overall output value. The derivation of if-then rules and corresponding membership functions depends, a lot, on the a priori knowledge about the sstem. However there is no sstematic wa to transform experiences and knowledge of human experts to the knowledge base of a FIS. There is also a need for adaptabilit or some learning algorithms to produce outputs within the required error rate. On the other hand, ANN learning mechanism does not rel on human expertise. Due to the homogenous structure of ANN, it is difficult to extract structured knowledge from either the weights or the configuration of the ANN. Table summarise the characteristics of FIS and ANN. ANN FIS Black box Interpretable Learning for Making use of linguistic scratch knowledge and heuristics Table Characteristics of FIS and ANN FIS and ANN are complementar which induce the appearance of the NFS that take advantage of the capacit that FIS have to store human expertise knowledge and the capacit of learning of the ANN. A common wa to appl a learning algorithm to a FIS is to represent it in a special ANN like architecture, which is what we have in. In this work was the NFS solution chosen because of the robustness and fastest convergence.

4 3... Anfis Architecture The architecture (Jang, 997) is ilustreted in figure 7. x A X X x Laer A A B B B A B B Laer Laer 3 TT TT Y Y w w w f=pxqr Fig. 7. a) A two input first-order Sugeno fuzz model with two rules; b) equivalent architecture. Assume that the fuzz inference sstem under consideration has two inputs x and and one output z, for example. For the first order Sugeno fuzz model a common rule set with two fuzz if-then rules is the following: Rule: If x is A and is B, then f=pxqr; Rule: If x is A and is B, then f=pxqr. Figure 7 a) illustrates the reasoning mechanism for this Sugeno and the corresponding equivalent architecture is shown in figure 7 b), where nodes of the same laer have similar functions. The output f in Figure 7 b), can be written as: w w f = f f w w w w = (wx)p (w)q (w)r (wx)p (w)q (w)r (7) This wa an adaptive network that is functionall equivalent to a first order Sugeno fuzz model is constructed. From the architecture shown in figure 7 b), it can be seen that when the values of the premise parameters (laer ) are fixed, the overall output can be expressed as a linear combination of the consequent parameters (laer 4) Learning Algorithms of The learning algorithms are composed of two phases: - In the forward pass of the hbrid learning algorithm, node outputs values go forward until laer 4 and the consequent parameters are identified b the least squares method. - In the backward pass, the output errors are propagated backward and the premise parameters are updated b gradient descent method. w a) b) f=pxqr = w(px q r) w(px q r) N N w w Laer 4 x x wf wf f = w w = w f w f wf Laer wf SUM f 3.3. and structures to identification For the structure, as there is no rule to determine the ideal number of neurons in the hidden laer, a wide range of values were tested to search for the best solution for first and second order sstem models. For first order approach models, the best solutions were obtained using three neurons for the direct model and four for the inverse model (hidden laer). For second order approach models, the best solutions were obtained using three neurons for the direct model and six for the inverse model (hidden laer). The models have one output neuron with linear activation function. The standard structure was used to obtain the direct and the inverse models. For first order approach models, the structure contains four rules, two inputs with two membership functions each (bell shaped with three non linear parameters each). For second order approach models, the structure contains sixteen rules with two membership functions each (bell shaped with tree non linear parameters each). All the models have one output that is a linear function of the consequent parameters. Using the data collected and divided into training and test sets, direct and inverse models were identified using and. The identification procedures were performed using MATLAB Fuzz Logic Toolbox (MATHWORKS, 996) tools for models and the NNSYSID (Nørgaard, et al., 996b) and NNCTRL (Nørgaard, et al., 996c) toolboxes for MATLAB for FFN models. Training was performed off-line for both solutions Comparison of the and models Figure 8 shows the training (first 3/4 of the points) and testing sets (last /4 of the points) used to create the models ((k)-temperature and u(k)-control signal). Temp.(ºC) Control Signal(V) Fig. 8. Training and test data sets. The training was made on a PC with a Intel MHz MMX processor with 64Mbtes of RAM memor. The number of training epochs

5 corresponds roughl to stopping the training when the minimum of the test data error was achieved. To compare the precision of the models, the mean square error criterion was used in train and test sets. The results are shown in table and 3 (direct and inverse models respectivel). Mean Square Error Train Test Direct with Direct with.3e-.4e- Direct with.9.94 Direct with.3e-3.e-3 Table. Mean square errors of direct models. Mean Square Error Train Test Inverse with.8e-3.e-3 Inverse with.e-3.e-3 Inverse with.e-3.e-3 Inverse with 3.3e-.6e- Table 3. Mean square errors of inverse models. ( models for first order approach, models for second order approach, models for first order approach, models for second order approach). As can be seen from the train and test errors, in the second order approach the errors are smaller than the ones in the first order approach. In general the structure achieved smaller mean square errors when compared with the same order approach using. The obtained models can also be compared in terms of complexit b measuring the number of parameters, number of training epochs and the training time. The results are summarised in tables 4 and for direct and inverse models respectivel. Complexit Comparison Nº of parameters Nº of training 8 3 epochs Training time (sec) Table 4. Complexit comparison of the direct models. Complexit Comparison Nº of parameters Nº of training epochs Training time (sec) Table. Complexit comparison of the inverse models. From table 4 and, for first order approach, the number of parameters is smaller and the training time is higher in than in. For second order approach, the number of parameters and the training time are smaller in than in. 4. CONTROL STRUCTURE To test the obtained models with both architectures in real time control, the Internal Control (IMC) structure was implemented. Internal Control consists of connecting in series with the plant the inverse model and in parallel with the plant the direct model. The difference between the output of the model and the output of the plant will generate an error that will be feedback (Pradeep, 988), (Dias, ). This solution can be seen in figure 9. r(kh) - Inverse u(k) Fig. 9. Generic structure for IMC. Plant Direct (kh). THE REAL TIME CONTROL ACTION - e(k) pred(kh) The controllers were directl tested in the kiln, when the simulation results were considered satisfactor. Figures and show the results of and internal model controllers for the first and second order approach (reference signal, control and error signals from first order approach and finall control and error signals from second order approach). The values of the mean square errors of the four IMC controllers for first and second order approaches are presented in table 6 after stabilization (from to 3 samples). Internal Controller Mean Square Error Using models.3 Using models.78 Using models.86 Using models.6 Table 6. Comparison between the four controllers. As can be seen from the figures, after initial stabilization, the second order approach achieve the best results.

6 (ºC) u(st)(v) e(st)(ºc) u(nd)(v) e(st)(ºc) Fig. IMC using models for second order approach. (ºC) u(st)(v) e(st)(ºc) u(nd)(v) e(st)(ºc) Fig. IMC using models for second order approach. As can be seen from table 6 the results of first order and second order are similar but the ones achieved b for second order are a better than the ones achieved b. 6. CONCLUSIONS In the modelling of this kiln, the train/test errors achieved are similar in and in structure. Just in the first order approach the errors achieved are smaller in than in structure. The number of parameters and time training of the models are bigger in general in than in structure. This could be caused because it the standard structure was used, which is not ver flexible. It could be interesting test other structures like zero order Sugeno tpe and cluster data before training to get simple structures. The IMC results show that the second order approach gets better results. The first order approach controllers take longer to achieved a zero stationar error, because the models are not so good. The second order approach controllers give better results, because the can hold more information, learning all the characteristics of the kiln, so for these models the direct models and the kiln are almost equal. From the results of table 6 the controllers with models gives smaller errors than the equivalent ones with models. Their complexit is similar for first order models, but becomes more complex for second order models. For the solution the use of second order models is justified b the results obtained while for the solution a first order approach would be enough. Both solutions are valid options for the modelling of real sstems under measurement noise. REFERENCES Cbenko G., (989) Approximation b Superposition of a Sigmoidal Function, Mathematics of Control, Signals and Sstems, pp Jang J. S. R (99) Neuro-Fuzz ing: Architecture, Analses and Applications. PhD Thesis, Department of Electrical Engineering and Computer Science, Universit of California, Berkele, USA. Jang J S. R.,C.-T. Sun, E. Mizutani, (997) Neuro- Fuzz and Soft Computing A Computation Approach to Learning and Machine Intelligence, Matlab Curriculum Series, Prentice Hall. Fuzz Logic Toolbox for MATLAB, MATHWORKS (996), Technical Report. Minsk M. and Papert S. (969), Perceptrons, Cambridge, MA: MIT Press. Nørgaard, M. (996)a. Sstem Identification and Control with Neural Networks. PhD Thesis, Department of Automation, Technical Universit of Denmark. Nørgaard, M. (996)b. Neural Network Sstem Identification Toolbox for MATLAB, Technical Report. Nørgaard, M. (996)c. Neural Network Control Toolbox for MATLAB, Technical Report. Pradeep B. Deshpande, Ramond H. Ash (988) Computer Process Control w/ advanced control applications, ISA. Hunt, K.J., D. Sbarbaro (99) Neural Networks for linear Internal Control, IEEE Proceedings-D, Control theor and applications, 38(): Sørensen, O. (994) Neural Networks in Control Applications. PhD Thesis, Department of Control Engineering, Institute of Electronic Sstems, Aalborg Universit, Denmark. Nauck D., Klawonn F., Kruse R. (997) Foundations of Neuro-Fuzz Sstems. Wile. Dias Fernando M., Mota Alexandre M. () Comparison between Different Control Strategies Using Neural Networks, 9 th Mediterranean Conference on Control and Automation, Dubrovnik, Croatia.