http://usnet.us.edu.pl/uslugi-sieciowe/oprogramowanie-w-usk-usnet/oprogramowaniestatystyczne/ Neural Networks in Statistica Agnieszka Nowak - Brzezińska
The basic element of each neural network is neuron. x1 x2 w1 w2 Dendrites Terminal Branches of Axon x3 w3 S Axon xn wn
Types of neurons y S- aggregated input value Activation function
Activation function For linear neurons: Linear, sigmoidal, hiperbolic, exponential, sinusoidal, For radial: gauss. Linear is the aggregation. Output value can be taken from nonlinear activation function.
Neuron s learning y
Prediction Input: X 1 X 2 X 3 Output: Y Model: Y = f(x 1 X 2 X 3 ) X 1 =1 X 2 =-1 X 3 =2 0.2 = 0.5 * 1 0.1*(-1) 0.2 * 2 0.5 0.6-0.1 0.1-0.2 0.7 f(x) = e x / (1 + e x ) f(0.2) = e 0.2 / (1 + e 0.2 ) = 0.55 0.2 f (0.2) = 0.55 0.55 0.9 f (0.9) = 0.71 0.71 Prediction Y = 0.478 0.1-0.2-0.087 f (-0.087) = 0.478 0.478 If true id Y = 2 Then prediction error is = (2-0.478) =1.522
Learning process 2. Calculate the value of Y 1. Randomly choose one observation 3. Compare Y with the actual value 4. Modify the weights by calculation the error
Backpropagation It is one of the most popular techniques of learning process for NN.
How to calculate the prediction error? where: Error i is the error ofr i-th node, Output i is the predicted by the network, Actual i is the real value which should be predicted
Weights modification L- is the learning factor from the range [0,1] The less the l values is the slowest the learning process is. Very often l is the highest in the begining and then reducted with the changing of the weights.
Example
Zmiana wag L- is the learning factor from the range [0,1] The less the l values is the slowest the learning process is. Very often l is the highest in the begining and then reducted with the changing of the weights.
How many neurons? The number of neurons in the input layer depends on the number of input variables The number of neurons in output layer depends on the type of the problem to solve by the network The number of neurons in hidden layer depends on the users qualifications
Neural network tasks: clasification NN is to decide about the class of a given object (classes in nominal scale) regresion NN is to predict a value (numerical) of the attribute which is the output value.
Clasification 1. Dataset leukemia.sta 2. choose the type of NN
3. Choose the variables: 4. Automatic generation of NN
You may change the proportions of division of dataset in learning and testing probes
Automatic generation of NN Linears neurons(mlp) Minimal (3) maksimal (10) neurons in hidden layer 20 NN, 5 the best is displayed Error function: SSE The window presents the creation of model 3-6-2 where 3 are neurons in input layer, 6 in hidden and 2 in output layer.
3 best nets are saved
Predictions Graphs Details Liftcharts Custom predictions SUMMARY
Predictions
Details Summary Weights Confusion matrix
Details Ciekawe są opcje: Summary Weights Confusion matrix
Details Ciekawe są opcje: Summary Weights Confusion matrix
Results
Zakładka liftcharts Further read: http://www.statsoft.pl/czytelnia/artykuly/krzywe_roc_czyli_ocena_jakosci.pdf
1 dataset tomatoes.sta Regression 2. type of the network:
3. Choose from the variables: 4. Automatic generation of NN
2 best NN saved
Predictions Graphs Details Liftcharts Custom predictions SUMMARY
Predictions
Graphs
Details Ciekawe są opcje: Summary Weights Correlation coefficients Confusion matrix -
Results
read: http://zsi.tech.us.edu.pl/~nowak/si/si_w4.pdf