9. Lecture Neural Networks

Transcription

1 Soft Control (AT 3, RMA) 9. Lecture Neural Networks Application in Automation Engineering

2 Outline of the lecture 1. Introduction to Soft Control: definition and limitations, basics of "smart" systems 2. Knowledge representation and knowledge processing (Symbolic AI) Application: expert systems 3. Fuzzy systems: Dealing with fuzzy knowledge Application: Fuzzy Control 4. Connective systems: Neural Networks Application: Identification and neural control 1. Basics 2. Learning method 3. Application in Automation Engineering 5. Genetic algorithms: Stochastic Optimization Application: Optimization 6. Summary & Literature 234

3 Contents of 9 th Lecture Modelling of Systems by NN Preliminaries Direct Model Inverse Model Application Control Virtual Sensors Assessment of NN Comparison of NN und Fuzzy Possible combinations Application examples: Load forecasting 235

4 Preliminaries Neural networks can model any non-linear relations among multiple input and output variables of a system Pure feed-forward networks can only model static relationships Solution 1: Recurrent Networks - Training is difficult Solution 2: External feedback i.e., processing of past values + Simple learning algorithm like backpropagation can be used - The number of past values must be fixed Identification with past values: discrete model 236

5 Generating the model of a process Objective: Modeling of a process Networks models the function y k = f(u k-1, y k-1 ) For systems of higher order: y k = f(u k-1,u k-2,...,y k-1,y k-2,...) Input: Current and past values of the process input u past values of the process output y Output: Current process output y k Example 237

6 Generating inverse process model Objective: Modeling of inverse process model Network models the function u k-1 = f(y k, y k-1 ) or u k-1 = f( y k,y k-1,y k-2,... u k-2,u k-3,... ) Inputs: Current and past values of the process output y Previous process inputs u output: Current process input u k-1 Example 238

7 Application of the direct model Estimation of state variables which are not measurable online to use in closed-loop controllers (virtual sensor, observers) w u y m Controller Route - NN Model y y NN logical interconnection 239

8 Application of the inverse model (ideal) If the model is ideal it is possible to achieve open-loop control using inverse model w u y inverse NN Modell Route But: Model is not ideal There are noises 240

9 Application of the inverse model (real) Use of a controller to remove the noises and to compensate for the errors in the model w - Lin. Controller inverse NN Model u Route y 241

10 Summary of applications of NN in AT (Automation) Besides the "classical" tasks such as pattern recognition, classification, etc. NN can also be used for performing core functionalities of AT (Automation) Observer or virtual Sensor Closed-loop control (in combination with conventional control) Combinations of the above are also possible In addition to the basic structures discussed, there could be many other structures 242

11 Evaluation of NN Neural networks can be trained on the basis of data no modelling of the processes necessary Successful applications show the potential of the method Knowledge is encoded in the structure of the NN A verification, interpretation of the calculated values is virtually not possible (raises acceptance problems!) NN training is extensive Acquisition of "good" data can be problematic To fix the structural parameters, e.g., Number of hidden layers Number of neurons in the hidden layers Type of network Type of activation functions Learning parameters and criteria for stopping training use of heuristics is preferable in most cases. 243

12 Comparison NN vs. Fuzzy Classic Method Fuzzy Control Neuronal Networks Neuro-Fuzzy + In-depth process understanding based on process analysis + Generally the outcome is very good and optimal solutions can be achieved + Stability proves are possible + Simple and comprehensive form of algorithms, + Easily extensible rulebase + Integration of knowledge from more than one source is possible + Adaptive and adaptable to very complex dynamic processes, + Possible to retrain when the process undergoes changes + Like NN but + Better interpretation of knowledge, + Knowledge through learning can gradually be complemented - Application to complex processes very cumbersome and expensive - Control specialists are needed to write and amend the algorithms - There are scarce standard tools for implementing the algorithms on standard hardware (e.g., PLC) - Difficult knowledge acquisition phase - Optimization phase often slow - Unusual way of thinking - Long computing times during training, - Many competing network structures to choose from - Extrapolation not possible, i.e., good results are achieved only in the range of training data - Knowledge in the network hardly interpretable - Often very long computing times - Convergence is not ensured 244

13 Approaches for the combination of NN and fuzzy (A) Cooperative neuro-fuzzy systems: Fuzzy systems which can be trained by neural networks. A neural network connected serially with the fuzzy system can, for example, be used to learn the suitability of a rule in certain situations. (B) Rule-based training of a simple neural network (C) Hybrid Neuro-Fuzzy-systems: simple neural networks that uses "fuzzy neurons" (e.g., min-/max-neurons) and "fuzzy weights". The structure of the fuzzy system can be recognized from the network topology. (D) Neural networks that can be trained by fuzzy-learning method. The changes of the weights between the neurons is calculated by a fuzzy system at each step. (E) topological configuration of a neural network, with more or less complex fuzzy systems as neurons. (F) A mix of classic expert systems and one of the above approaches. Important approaches are A, B and C. Other approaches are not as widespread the previous ones. 245

14 Cooperative neuro-fuzzy systems (2 approaches) Fine-tuning a fuzzy controller by NN A fuzzy controller will be followed by a neural network The output of the fuzzy system will be immediately processed by the NN Thus based upon a basic knowledge (of the fuzzy system) a non-linear system can be built, which additionally renders adaptability to certain special situations which are not defined by the basic knowledge. Thus NN performs the "fine tuning" of output of the fuzzy system. The NN can learn which tuning is necessary for which input. The fuzzy system must not deliver defuzzyfied output this task can also be performed by the NN. Preprocessing the input values of a fuzzy controller by NN fuzzy controller is preceded by a NN The output of the NN is fed to the fuzzy controller for processing. Thus, changes in the input data, which cannot be processed by the fuzzy system can be compensated by NN. 246

15 Rule based training of NN NN can only be trained by numerical data Often a rough knowledge of the process is available in the form of fuzzy rules Solution: mapping of linguistic rules (qualitative) to the training data (quantitative) The linguistic terms are mapped to values (according to the membership functions) The rules are then defined by the corresponding values During training NN interpolates among the values Example: Three variables X1, X2 and Y with values of Small, Medium and Large within the range of [0, 1] have to mapped to numeric values. It is given that small = 0; resources = 0.5; Large = 1. The rule IF X1=small AND X2 = large THEN Y = large Results in the data set X = (0, 1); Y = 1 247

16 Hybrid neuro-fuzzy systems Mapping of a fuzzy controller to a neural network Example: 1 st Layer: input Fuzzy Sets 2 nd Layer: evaluate the degree of fulfilment of the rules 3 rd Layer: output fuzzy sets 4 th Layer: De-fuzzyfication Other variants define the fuzzy sets in the weights Training with data Interpretation of the rules learned as weights (weights between Layer 1 and 2 or 2 to 3) 248

17 Hybrid neuro-fuzzy systems (example) x 1 x 2 249

18 Example: Load forecast in electrical energy supply networks Motivation Last curve analysis Forecast with Artificial Neural Networks (ANN) Wavelet transformation Assessment of results Summary 250

19 Motivation Structure of an electric power supply network Logic on/off deterministic, known not deterministic, only past behaviour known Power Plant P I Network (Low storing capacity) P O Consumer 251

20 Motivation Last curves forecast plays a major role in the operation of power networks Power is cost-effective Electrical energy is difficult to save It should be possible to only produce as much electrical energy as needed P I =P O Therefore one needs to recorded consumption profiles based Forecast Under forecast leads to inadequate provision of spare capacity Over forecast caused unnecessary spare capacity 252

21 Last curve-analysis Network load from to (individual than three weeks) in the control zone RWE's electricity transportation network MW 1. From Monday to Sunday, from 0 clock to 24 clock 2. Given are 15-minute averages 3. 4 * 24 = 96 test points per day, 96 * 7 = 672 measuring points per week 253

22 Forecast with artificial neural networks (ANN) Forecast runoff Last curve normalization Forecast basic idea KNN-Definition Structure, vector input, output vector, activation function KNN training (with a whole week (this week 1)) Back propagation-algorithms Learning rate KNN-Application (with Week 2 oder 3) Results Denormalization Lk ( 1, k 2,. k 8) Lk ( 4) KNN Modell Fig 2 : Einschicht-Neuron Fig 3 : Drei-Schichten-Feed-Forward-Struktur 254

23 Forecast with artificial neural networks (ANN) Last curve-normalization 255

24 Forecast with artificial neural networks (ANN) Three layers feed forward structure L k 1 L k 2... L k 8 Last in an hour alk4 p Last course (distribution) of the last two hours Lk ( 1, k 2,. k 8) Lk ( 4) KNN Modell Forecast basic idea Monolayer neuron 256

25 Forecast with artificial neural networks (ANN) Four-step forecast results Training of KNN with Week 1 Target vector (SimT): Last curve Week 3 Output vector(y): Forecast of Week 3 257

26 Forecast with artificial neural networks (KNN) Relativer Fehler In many places, the relative error is greater than 10% The accuracy must be improved Idea: Installation of Wavelet transformation 258

27 Wavelet transformation Development of Wavelet transformation Fourier transformation Transformation from Time- to Frequency Domain f t Ff Short-Time-Fourier transformation Additional Information which Frequency in occurs which time frame 1 Continuous Wavelet transformation Transformation of time in frequency and time domain f t Discrete Wavelet transformation (DWT) Realization in Computer tt 1 2 ft Ff, Ff,., Ff t0 tt tt A Trous algorithm of Wavelet transformation Shift invariant Same in data length in different frequency domains suitable for real-time systems 259

28 Discrete Wavelet transformation (DWT) (implementation) Analysis of a signal HP TP 2 High pass filter Low pass filter Down sampling x d1 a1 d2 a2 d3 a3 Sampling points N N/2 N/2 N/4 N/4 N/8 N/8 Frequency response f<f s /2 f s /4<f<f s /2 f<f s /4 f s /8<f<f s /4 f<f s /8 f s /16<f<f s /8 f<f s /16 260

29 Discrete Wavelet transformation (DWT) Example 261

30 Discrete Wavelet transformation (DWT) Synthesis of a signal Upsampling 262

31 Discrete wavelet transformation (DWT) Requirement of DWT in the analysis of real-time system Localization time points in different scales Shift invariance of the system Move original curve Wavelettransformation Wavelettransformation Move Coefficient Wavelet- Coefficient Wavelet- Coefficient 263

32 Á-Trous algorithm of Wavelet transformation Properties of the A-Trous algorithm Shift invariance Same data length of all the different scales Wavelet coefficient a3 d3 g[n] : Tiefpassfilter d2 h[n] : Hochpassfilter d1 264

33 Wavelet transformation Example A-Trous algorithm Week 1 load curve is split into 4 layers a4: Approximations signal; d4, d3, d2, d1: detail signals a4 has the largest amplitude and the lowest frequency d1 is the smallest and the largest amplitude frequency 265

34 Forecast: ANN + A Trous Forecast runoff with KNN and A-Trous For each split signal, a ANN model The more layers, the higher the accuracy of the load curve synthesis d1 is the prognosis regarded as noise and neglected. a4 neta4 d4 netd4 Recorded load curves Ã-Trous Wavelet d3 netd3 Wavelet Retransformation Predicted Last curve d2 netd2 d1 netd1 266

35 Forecast: ANN + A Trous Four-step forecast results Training with Week1 Target vector(simt): Last curve Week3 Output vector(y): Forecast of Week3 267

36 Forecast: ANN + A Trous Relativer Fehler At the most points the relative error less than 2% The error is never greater than 6% In comparison to ANN without A-Trous, the accuracy improved significantly 268

37 Summary and learning from the 9th Lecture Know basic applications of NN in AT Model shapes in the identification and their target directly Inverse Neural networks with other approaches to (especially fuzzy) compare Deduce reasons for neuro-fuzzy Know possible ways of combining NN with fuzzy and can explain the basic idea Use of neural networks has been shown to predict Neural networks applied to isolated not bring satisfactory results in the load curve forecasting In combination with wavelet transform results could be significantly improved 269