Package FCNN4R. March 9, 2016

Transcription

1 Type Package Title Fast Compressed Neural Networks for R Version Date Package FCNN4R March 9, 2016 Author Grzegorz Klima <gklima@users.sourceforge.> Maintainer Grzegorz Klima <gklima@users.sourceforge.> Provides an interface to kernel routines from the FCNN C++ library. FCNN is based on a completely new Artificial Neural Network representation that offers unmatched efficiency, modularity, and extensibility. FCNN4R provides standard teaching (backpropagation, Rprop, simulated annealing, stochastic gradient) and pruning algorithms (minimum magnitude, Optimal Brain Surgeon), but it is first and foremost an efficient computational engine. Users can easily implement their algorithms by taking advantage of fast gradient computing routines, as well as work reconstruction functionality (removing weights and redundant neurons, reordering inputs, merging works). Networks can be exported to C functions in order to integrate them into virtually any software solution. Depends R (>= 3.0), stats, graphics, methods, Rcpp License GPL (>= 2) NeedsCompilation yes Repository CRAN Date/Publication :57:57 R topics documented: FCNN4R-package is.mlp_ mlp_eval mlp_export_c mlp_ mlp_-absolute-weight-indices mlp_-accessing-individual-weights

2 2 FCNN4R-package mlp_-class mlp_-combining-two-works mlp_-display mlp_-export-import mlp_-general-information mlp_-manipulating-work-inputs mlp_-mse-gradients mlp_-names mlp_-weights-access mlp_plot mlp_prune_mag mlp_prune_obs mlp_rm_neurons mlp_rnd_weights mlp_set_activation mlp_teach_bp mlp_teach_grprop mlp_teach_rprop mlp_teach_sa mlp_teach_sgd read-write-fcnndataset Index 28 FCNN4R-package Fast Compressed Neural Networks for R Provides an interface to kernel routines from the FCNN C++ library. FCNN is based on a completely new Artificial Neural Network representation that offers unmatched efficiency, modularity, and extensibility. FCNN4R provides standard teaching (backpropagation, Rprop, simulated annealing, stochastic gradient) and pruning algorithms (minimum magnitude, Optimal Brain Surgeon), but it is first and foremost an efficient computational engine. Users can easily implement their algorithms by taking advantage of fast gradient computing routines, as well as work reconstruction functionality (removing weights and redundant neurons, reordering inputs, merging works). Networks can be exported to C functions in order to integrate them into virtually any software solution. Author(s) Grzegorz Klima <gklima@users.sourceforge.> References G. Klima. A new approach towards implementing artificial neural works. Technical Report,

3 is.mlp_ 3 Examples # set up the XOR problem inputs and outputs inp <- c(0, 0, 1, 1, 0, 1, 0, 1) dim(inp) <- c(4, 2) outp <- c(0, 1, 1, 0) dim(outp) <- c(4, 1) # create a work <- mlp_(c(2, 6, 1)) # set activation function in all layers <- mlp_set_activation(, layer = "a", "sigmoid") # randomise weights <- mlp_rnd_weights() # tolerance level tol <- 0.5e-4 # teach using Rprop, assign trained work and plot learning history mse <- mlp_teach_rprop(, inp, outp, tol_level = tol, max_epochs = 500, report_freq = 10) <- mse$ plot(mse$mse, type = l ) # plot work with weights mlp_plot(, TRUE) # if the algorithm had converged, prune using Optimal Brain Surgeon and plot if (mlp_mse(, inp, outp) <= tol) { <- mlp_prune_obs(, inp, outp, tol_level = tol, max_reteach_epochs = 500, report = TRUE)[[1]] mlp_plot(, TRUE) } # check work output round(mlp_eval(, inp), digits = 3) is.mlp_ Is it? This function checks whether argument is. is.mlp_(x) x an object to be checked Logical value.

4 4 mlp_export_c mlp_eval Evaluation Evaluate work output. mlp_eval(, input) input numeric matrix, each row corresponds to one input vector, the number of columns must be equal to the number of neurons in the work input layer Numeric matrix with rows representing work outputs corresponding to input rows. mlp_export_c Export multilayer perceptron work to a C function This function exports multilayer perceptron work to a C function with optional affine input and output transformations: Ax+b for inputs and Cx+d for outputs. mlp_export_c(fname,, with_bp = FALSE, A = NULL, b = NULL, C = NULL, d = NULL) fname with_bp A b C d character string with the filename logical, should backpropagation code for online learning be exported? numeric matrix (optional), input linear transformation numeric vector (optional), input translation numeric matrix (optional), output linear transformation numeric vector (optional), output translation

5 mlp_ 5 Logical value, TRUE if export was successful, FALSE otherwise. Examples # create a work <- mlp_(c(2, 3, 1)) # randomise weights <- mlp_rnd_weights() # show the work show() # export work to a C function mlp_export_c("test.c", ) # show the output file file.show("test.c") mlp_ Create objects of mlp_ class Function used for creating multilayer perceptron works. mlp_(layers, name = NULL) layers name vector providing numbers of neurons in each layer character string, work name (optional) Returns. See Also mlp_ for details.

6 6 mlp_-absolute-weight-indices Examples # create a work <- mlp_(c(2, 3, 1)) # randomise weights <- mlp_rnd_weights() # show basic information about the work show() mlp_-absolute-weight-indices Retrieving absolute weight index In some situations absolute weight index (i.e. index within all weights including inactive ones) needs to be computed based on information about connected neurons indices or weight index within actives ones. The latter functionality is especially useful in implementation of pruning algorithms. mlp_get_w_idx(, layer, nidx, nplidx) mlp_get_w_abs_idx(, idx) layer nidx nplidx idx integer value (vector), layer index integer value (vector), neuron index integer value (vector), index of the neuron in the previous layer determining connection from neuron nidx in layer, 0 denotes bias of neuron nidx in layer integer value (vector), weight index (indices) within active ones Absolute weight index.

7 mlp_-accessing-individual-weights 7 mlp_-accessing-individual-weights Setting and retrieving status (on/off) and value of individual weight(s) The following functions can be used to access individual weight(s), i.e. set or retrieve status(es) (on/off) and value(s). mlp_set_w_st(, on, idx = NULL, layer = NULL, nidx = NULL, nplidx = NULL) mlp_set_w(, val, idx = NULL, layer = NULL, nidx = NULL, nplidx = NULL) mlp_get_w_st(, idx = NULL, layer = NULL, nidx = NULL, nplidx = NULL) mlp_get_w(, idx = NULL, layer = NULL, nidx = NULL, nplidx = NULL) on idx layer nidx nplidx val logical value (vector), should the weight be set on or off? integer value (vector), weight absolute index integer value (vector), layer index integer value (vector), neuron index integer value (vector), index of the neuron in the previous layer determining connection from neuron nidx in layer, 0 denotes bias of neuron nidx in layer numeric value (vector), connection (or bias) value to be set mlp_set_w_st returns work () with state(s) (on/off) of selected weight(s) set. mlp_set_w returns work () with value(s) of selected weight(s) set. mlp_get_w_st returns logical value (vector), TRUE if connection/bias is active, FALSE otherwise. mlp_get_w returns numeric value (vector), selected weight value(s).

8 8 mlp_-class mlp_-class An S4 class representing Multilayer Perception Network. The mlp_ class represents the Multilayer Perception Network employing the so-called compressed representation, which was inspired by the Compressed Column Storage familiar from sparse matrix algebra. Although the representation and algorithms working with it are somewhat complicated, the user is provided with a simple and intuitive interface that completely hides the internal workings of the package, which in its large part is written in C++. Slots m_name character string, work name m_layers integer vector, stores the numbers of neurons in layers m_n_pointers integer vector, stores the so-called pointers to neurons m_n_prev integer vector, stores the number of connected neurons in the previous layer m_n_next integer vector, stores the number of connected neurons in the next layer m_w_pointers integer vector, stores the so-called pointers to weights m_w_values numeric vector, values of connection weights and biases m_w_flags logical vector, states (active/inactive) of weights and biases m_w_on integer value, the number of active weights m_af integer vector, activation functions indices m_af_p numeric vector, activation functions slope parameters References G. Klima. A new approach towards implementing artificial neural works. Technical Report, See Also mlp_ for creating objects of this class.

9 mlp_-combining-two-works 9 mlp_-combining-two-works Combining two works into one These functions construct new work by merging two works (they must have the same number of layers) or by connecting one work outputs to another work inputs (the numbers of output and input neurons must agree). These functions may be used in constructing deep learning works or constructing works with some special topologies. mlp_merge(1, 2, same_inputs = FALSE) mlp_stack(1, 2) 1 2 same_inputs logical, if TRUE both merged works are assumed to take the same inputs (they share the input layer), default is FALSE Both functions return. Examples # create two works with random weights and plot them 1 <- mlp_(c(2, 2, 2)) 1 <- mlp_rnd_weights(1) mlp_plot(1, TRUE) 2 <- mlp_(c(2, 2, 2)) 2 <- mlp_rnd_weights(2) mlp_plot(2, TRUE) # create a work with random weights and plot it 3 <- mlp_(c(4, 3, 2)) 3 <- mlp_rnd_weights(3) mlp_plot(3, TRUE) # construct new work using 1, 2, and 3 and plot it 4 <- mlp_stack(mlp_merge(1, 2), 3) mlp_plot(4, TRUE)

10 10 mlp_-export-import mlp_-display Displaying works (objects of mlp_ class) These methods can be used to display objects of mlp_ class. show and print provide short information about work structure and activation functions, summary gives detailed information about all work connections. ## S4 method for signature mlp_ show(object) ## S4 method for signature mlp_ print(x) ## S4 method for signature mlp_ summary(object) object x mlp_-export-import Export and import multilayer perceptron work to/from a text file in FCNN format These functions can be used to export and import multilayer perceptron work to/from a text file in FCNN format. mlp_export_fcnn(fname, ) mlp_import_fcnn(fname) fname character string with the filename

11 mlp_-general-information 11 Details See Also Files are organised as follows: the first comment (beginning with #) is treated as work information (name) string, all other comments are ignored, work structure is represented by five block of numbers: the first line determines numbers of neurons in consecutive layers, the second block of 0 s and 1 s determines which weights are turned off/on, the third block contains active weights values, the last block determines hidden and output layers activation functions and their slope parameters - each line contains 2 numbers: the function index and its slope parameter. mlp_export_fcnn returns logical value, TRUE if export was successful, FALSE otherwise. mlp_import_fcnn returns or NULL, if import failed. mlp_ for work representation details. Examples # create a work <- mlp_(c(2, 3, 1)) # randomise weights <- mlp_rnd_weights() # show the work show() # export work mlp_export_fcnn("test.", ) # show the output file file.show("test.") # import work 2 <- mlp_import_fcnn("test.") # show the imported work show(2) mlp_-general-information General information about work The following functions return basic information about the work.

12 12 mlp_-manipulating-work-inputs mlp_get_layers() mlp_get_no_active_w() mlp_get_no_w() mlp_get_layers returns an integer vector with numbers of neurons in consecutive layers. mlp_get_no_active_w returns the number of active weights (connections and biases). mlp_get_no_w returns the total number (including inactive) of weights (connections and biases). See Also mlp_-class for details on internal work representation. mlp_-manipulating-work-inputs Manipulating work inputs These functions construct new work by removing redundant (i.e. not connected to the next layer) inputs or reordering / expanding work inputs. mlp_rm_input_neurons(, report = FALSE) mlp_expand_reorder_inputs(, newnoinputs, inputsmap) report newnoinputs inputsmap logical value, if TRUE, information about removed neurons will be printed on the console (FALSE by default) integer value, determines the number of inputs in the new work integer vector, determines the mapping of old inputs into new ones - the ith value of this vector will be the new index of ith input

13 mlp_-mse-gradients 13 mlp_rm_input_neurons returns a two-element list. The first element () is the work (an object of mlp_ class) with all redundant input neurons removed, the second (ind) - the indices of input neurons that were not removed. mlp_expand_reorder_inputs returns. Examples # construct a work, plot result nn <- mlp_(c(2, 4, 3)) nn <- mlp_rnd_weights(nn) mlp_plot(nn, TRUE) # expand inputs, the new no. of inputs will be 5, with the first input # becoming the 3rd and the second retaining its position, plot result nn <- mlp_expand_reorder_inputs(nn, 5, c(3, 2)) mlp_plot(nn, TRUE) # remove redundant neurons (i.e. 1, 4, 5) and plot result nn <- mlp_rm_input_neurons(nn, TRUE)$ mlp_plot(nn, TRUE) mlp_-mse-gradients Computing mean squared error, its gradient, and output derivatives The functions use fast FCNN kernel routines and are intended for implementing teaching and pruning algorithms. mlp_mse(, input, output) mlp_grad(, input, output) mlp_gradi(, input, output, i) mlp_gradij(, input, i) mlp_jacob(, input, i) input numeric matrix, each row corresponds to one input vector, the number of columns must be equal to the number of neurons in the work input layer

14 14 mlp_-names output i numeric matrix with rows corresponding to expected outputs, the number of columns must be equal to the number of neurons in the work output layer, the number of rows must be equal to the number of input rows data row index Details mlp_mse returns the mean squared error (MSE). MSE is understood as half of the squared error averaged over all outputs and data records. mlp_grad computes the gradient of MSE w.r.t. work weights. This function is useful when implementing batch teaching algorithms. mlp_gradi computes the gradient of MSE w.r.t. work weights at the ith data record. This is normalised by the number of outputs only, the average over all rows (all i) returns the same as grad(input, output). This function is useful for implementing on-line teaching algorithms. mlp_gradij computes gradients of work outputs, i.e the derivatives of outputs w.r.t. active weights, at given data row. The derivatives of outputs are placed in subsequent columns of the returned matrix. Scaled by the output errors and averaged they give the same as gradi(input, output, i). This function is useful in implementing teaching algorithms using second order corrections and Optimal Brain Surgeon pruning algorithm. mlp_jacob computes the Jacobian of work outputs, i.e the derivatives of outputs w.r.t. inputs, at given data row. The derivatives of outputs are placed in subsequent columns of the returned matrix. mlp_mse returns mean squared error (numeric value). mlp_grad returns two-element lists with the first field (grad) containing numeric vector with gradient and the second (mse) - the mean squared error. mlp_gradi returns numeric vector with gradient. mlp_gradij returns numeric matrix with gradients of outputs in consecutive columns. mlp_jacob returns numeric matrix with derivatives of outputs in consecutive columns. mlp_-names Get and set work names The following functions can be used for retrieving and setting work names. mlp_get_name() mlp_set_name(, name)

15 mlp_-weights-access 15 name character string with work name mlp_get_name returns character string with work name. mlp_set_name returns work () with name set to new value. mlp_-weights-access Set and retrieve (active) weights values One of FCNN s design objectives (and main advantages) is the complete separation of teaching (and pruning) algorithms from internal work structure workings. This goal is achieved through fast access to (active) weights vector facilitated by FCNN s compressed work representation. The following two functions allow users to efficiently retrieve and set work (active) weights vector. mlp_set_weights(, weights) mlp_get_weights() weights numeric vector of new active weights values mlp_set_weights returns work () with active weights set to given values. mlp_set_weights returns numeric vector of active weights values.

16 16 mlp_prune_mag mlp_plot Plotting multilayer perceptron work This function plots a multilayer perceptron work s structure. Optionally, weights values are displayed on graph. mlp_plot(, show_weights = FALSE, show_neuron_idx = TRUE) show_weights logical, should weights values be displayed? (FALSE by default) show_neuron_idx logical, should neurons indices be displayed? (TRUE by default) This function does not return value. mlp_prune_mag Minimum magnitude pruning Minimum magnitude pruning is a brute force, easy-to-implement pruning algorithm in which in each step the weight with the smallest absolute value is turned off. This algorithm requires reteaching work in almost every step and yields suboptimal results. mlp_prune_mag(, input, output, tol_level, max_reteach_epochs, report, plots = FALSE) input output numeric matrix, each row corresponds to one input vector, the number of columns must be equal to the number of neurons in the work input layer numeric matrix with rows corresponding to expected outputs, the number of columns must be equal to the number of neurons in the work output layer, the number of rows must be equal to the number of input rows

17 mlp_prune_obs 17 tol_level numeric value, error (MSE) tolerance level max_reteach_epochs integer value, maximal number of epochs (iterations) allowed when reteaching work report plots logical value, if TRUE, information about the pruning process will be printed on the console (FALSE by default) logical value, if TRUE, the initial work is plotted and then replotted every time neuron is removed and at the end of pruning (FALSE by default) Three-element list, the first field () contains the pruned work, the second (wcount) - the number of connections removed (inactivated), the third (ncount) - the number of neurons removed. mlp_prune_obs Optimal Brain Surgeon pruning The Optimal Brain Surgeon algorithm is a robust (yet computationally demanding) pruning algorithm in which candidate weight to be turned off is determined based on information about the inverse of (approximate) Hessian matrix of the MSE. mlp_prune_obs(, input, output, tol_level, max_reteach_epochs, report, plots = FALSE, alpha = 1e-05) input output numeric matrix, each row corresponds to one input vector, the number of columns must be equal to the number of neurons in the work input layer numeric matrix with rows corresponding to expected outputs, the number of columns must be equal to the number of neurons in the work output layer, the number of rows must be equal to the number of input rows tol_level numeric value, error (MSE) tolerance level max_reteach_epochs integer value, maximal number of epochs (iterations) allowed when reteaching work report plots alpha logical value, if TRUE, information about the pruning process will be printed on the console (FALSE by default) logical value, if TRUE, the initial work is plotted and then replotted every time neuron is removed and at the end of pruning (FALSE by default) numeric value, scaling factor used for initial Hessian approximation

18 18 mlp_rm_neurons Three-element list, the first field () contains the pruned work, the second (wcount) - the number of connections removed (inactivated), the third (ncount) - the number of neurons removed. References B. Hassibi, D. G. Stork, and G. J. Wolff. Optimal Brain Surgeon and General Network Pruning. Technical Report CRC-TR-9235, RICOH California Research Centre, mlp_rm_neurons Remove redundant neurons in a multilayer perceptron work This function removes redundant neurons from the work, i.e. hidden layers neurons that are not connected to neurons in the previous layer or the next layer. If a neuron is not connected to neurons in the previous layer but is connected to neurons in the next layer (effectively acts as an additional bias), biases of neurons in the next layer are properly adjusted, therefore, the resulting work behaves just like the initial one. mlp_rm_neurons(, report = FALSE) report logical value, if TRUE, information about removed neurons will be printed on the console (FALSE by default) Three-element list. The first element () is the work () with all redundant neurons removed, the second (ncount) - the number of neurons removed, the third (wcount) - the number of weights removed.

19 mlp_rnd_weights 19 mlp_rnd_weights This function sets work weights to random values drawn from uniform distribution. This function sets work weights to random values drawn from uniform distribution. mlp_rnd_weights(, a = 0.2) a numeric value, values will be drawn from uniform distribution on [-a, a] (by default a = 0.2) Network () with randomised weights. mlp_set_activation Set work activation functions This function sets activation function (and its slope parameter) for neurons in the hidden layers and in the output layer. mlp_set_activation(, layer, activation = c("threshold", "sym_threshold", "linear", "sigmoid", "sym_sigmoid", "tanh", "sigmoid_approx", "sym_sigmoid_approx"), slope = 0) layer activation slope integer vector or character value, index (indices) of layer(s) whose activation function will be changed or character: "a" denotes all layers, "h" - hidden layer(s), "o" - the output layer character string, activation function name, admissible options are: "threshold", "sym_threshold", "linear", "sigmoid", "sym_sigmoid" (and "tanh"), "sigmoid_approx", and "sym_sigmoid_approx" numeric value, activation function slope parameter, if 0 the default parameter value is chosen for each activation function

20 20 mlp_teach_bp This function returns work () with activation function set. mlp_teach_bp Backpropagation (batch) teaching Backpropagation (a teaching algorithm) is a simple steepest descent algorithm for MSE minimisation, in which weights are updated according to (scaled) gradient of MSE. mlp_teach_bp(, input, output, tol_level, max_epochs, learn_rate = 0.7, l2reg = 0, report_freq = 0) input output tol_level max_epochs numeric matrix, each row corresponds to one input vector, the number of columns must be equal to the number of neurons in the work input layer numeric matrix with rows corresponding to expected outputs, the number of columns must be equal to the number of neurons in the work output layer, the number of rows must be equal to the number of input rows numeric value, error (MSE) tolerance level integer value, maximal number of epochs (iterations) learn_rate numeric value, learning rate in the backpropagation algorithm (default 0.7) l2reg numeric value, L2 regularization parameter (default 0) report_freq integer value, progress report frequency, if set to 0 no information is printed on the console (this is the default) Two-element list, the first field () contains the trained work, the second (mse) - the learning history (MSE in consecutive epochs). Note The name backpropagation is commonly used in two contexts, which sometimes causes confusion. Firstly, backpropagation can be understood as an efficient algorithm for MSE gradient computation that was first described by Bryson and Ho in the 60s of 20th century and reinvented in the 80s. Secondly, the name backpropagation is (more often) used to refer to the steepest descent method that uses gradient of MSE computed efficiently by means of the aforementioned algorithm. This ambiguity is probably caused by the fact that in practically all neural work implementations, the derivatives of MSE and weight updates are computed simultaneously in one backward pass (from output layer to input layer).

21 mlp_teach_grprop 21 References A.E. Bryson and Y.C. Ho. Applied optimal control: optimization, estimation, and control. Blaisdell book in the pure and applied sciences. Blaisdell Pub. Co., David E. Rumelhart, Geoffrey E. Hinton, and Ronald J. Williams. Learning representations by back-propagating errors. Nature, 323(6088): , October mlp_teach_grprop Rprop teaching - minimising arbitrary objective function This implementation ( generalisation ) of the Rprop algorithm allows users to teach work to minimise arbitrary objective function provided that functions evaluating objective and computing gradient are provided. mlp_teach_grprop(, obj_func, gradient, epochs, stop = NULL, report_freq = 0, report_action = NULL, u = 1.2, d = 0.5, gmax = 50, gmin = 1e-06) obj_func gradient epochs stop report_freq report_action function taking an object of mlp_class class as a single argument returning objective to be minimised function taking an object of mlp_class class as a single argument returning gradient of the objective integer value, number of epochs (iterations) function (or NULL), a function taking objective history to date and returning Boolean value (if TRUE is returned, algorithm stops) (the default is not to stop until all iterations are performed) integer value, progress report frequency, if set to 0 no information is printed on the console (this is the default) function (or NULL), additional action to be taken while printing progress reports, this should be a function taking work as a single argument (default NULL) u numeric value, Rprop algorithm parameter (default 1.2) d numeric value, Rprop algorithm parameter (default 0.5) gmax numeric value, Rprop algorithm parameter (default 50) gmin numeric value, Rprop algorithm parameter (default 1e-6)

22 22 mlp_teach_grprop Two-element list, the first field () contains the trained work, the second (obj) - the learning history (value of the objective function in consecutive epochs). References M. Riedmiller. Rprop - and Implementation Details: Technical Report. Inst. f. Logik, Komplexitat u. Deduktionssysteme, Examples ## Not run: # set up XOR problem inp <- c(0, 0, 1, 1, 0, 1, 0, 1) dim(inp) <- c(4, 2) outp <- c(0, 1, 1, 0) dim(outp) <- c(4, 1) # objective obj <- function() { return(mlp_mse(, inp, outp)) } # gradient grad <- function() { return(mlp_grad(, inp, outp)$grad) } # stopping citerion tol <- function(oh) { if (oh[length(oh)] <= 5e-5) { return(true); } return(false) } # create a work <- mlp_(c(2, 6, 1)) # set activation function in all layers <- mlp_set_activation(, layer = "a", "sigmoid") # randomise weights <- mlp_rnd_weights() # teach obj <- mlp_teach_grprop(, obj, grad, epochs = 500, stop = tol, report_freq = 1) # plot learning history plot(obj$obj, type = l ) ## End(Not run)

23 mlp_teach_rprop 23 mlp_teach_rprop Rprop teaching Rprop is a fast and robust adaptive step method based on backpropagation. For details, please refer to the original paper given in References section. mlp_teach_rprop(, input, output, tol_level, max_epochs, l2reg = 0, u = 1.2, d = 0.5, gmax = 50, gmin = 1e-06, report_freq = 0) input output tol_level max_epochs numeric matrix, each row corresponds to one input vector, the number of columns must be equal to the number of neurons in the work input layer numeric matrix with rows corresponding to expected outputs, the number of columns must be equal to the number of neurons in the work output layer, the number of rows must be equal to the number of input rows numeric value, error (MSE) tolerance level integer value, maximal number of epochs (iterations) l2reg numeric value, L2 regularization parameter (default 0) u numeric value, Rprop algorithm parameter (default 1.2) d numeric value, Rprop algorithm parameter (default 0.5) gmax numeric value, Rprop algorithm parameter (default 50) gmin report_freq numeric value, Rprop algorithm parameter (default 1e-6) integer value, progress report frequency, if set to 0 no information is printed on the console (this is the default) Two-element list, the first field () contains the trained work, the second (mse) - the learning history (MSE in consecutive epochs). References M. Riedmiller. Rprop - and Implementation Details: Technical Report. Inst. f. Logik, Komplexitat u. Deduktionssysteme, 1994.

24 24 mlp_teach_sa mlp_teach_sa Teaching works using Simulated Annealing This function can be used to teach an ANN to minimise arbitrary objective function. mlp_teach_sa(, obj_func, Tinit = 1, epochs = 1000, report_freq = 0, report_action = NULL) obj_func function taking an object of mlp_class class as a single argument returning objective to be minimised Tinit numeric value, initial temperature (default is 1) epochs integer value, number of epochs (iterations) (default is 1000) report_freq report_action integer value, progress report frequency, if set to 0 no information is printed on the console (this is the default) function (or NULL), additional action to be taken while printing progress reports, this should be a function taking work as a single argument (default NULL) Two-element list, the first field () contains the trained work, the second (obj) - the learning history (value of the objective function in consecutive epochs). Examples ## Not run: # set up XOR problem inp <- c(0, 0, 1, 1, 0, 1, 0, 1) dim(inp) <- c(4, 2) outp <- c(0, 1, 1, 0) dim(outp) <- c(4, 1) # objective obj <- function() { return(mlp_mse(, inp, outp)) } # create a work <- mlp_(c(2, 6, 1)) # set activation function in all layers <- mlp_set_activation(, layer = "a", "sigmoid")

25 mlp_teach_sgd 25 # teach obj <- mlp_teach_sa(, obj, Tinit = 1, epochs = 1000, report_freq = 1) # plot learning history plot(obj$obj, type = l ) ## End(Not run) mlp_teach_sgd Stochastic gradient descent with (optional) RMS weights scaling, weight decay, and momentum This function implements the stochastic gradient descent method with optional modifications: L2 regularization, root mean square gradient scaling, weight decay, and momentum. mlp_teach_sgd(, input, output, tol_level, max_epochs, learn_rate, l2reg = 0, minibatchsz = 100, lambda = 0, gamma = 0, momentum = 0, report_freq = 0) input output tol_level max_epochs learn_rate numeric matrix, each row corresponds to one input vector number of columns must be equal to the number of neurons in the work input layer numeric matrix with rows corresponding to expected outputs, number of columns must be equal to the number of neurons in the work output layer, number of rows must be equal to the number of input rows numeric value, error (MSE) tolerance level integer value, maximal number of epochs (iterations) numeric value, (initial) learning rate, depending on the problem at hand, learning rates of or 0.01 should give satisfactory convergence l2reg numeric value, L2 regularization parameter (default 0) minibatchsz integer value, the size of the mini batch (default 100) lambda numeric value, rmsprop parameter controlling the update of mean squared gradient, reasonable value is 0.1 (default 0) gamma numeric value, weight decay parameter (default 0) momentum report_freq numeric value, momentum parameter, reasonable values are between 0.5 and 0.9 (default 0) integer value, progress report frequency, if set to 0 no information is printed on the console (this is the default)

26 26 read-write-fcnndataset Two-element list, the first field () contains the trained work, the second (mse) - the learning history (MSE in consecutive epochs). read-write-fcnndataset Reading and writing datasets in the FCNN format These functions can be used to read and write datasets from/to a text file in the FCNN format. Datasets in the similar FANN format (comments are not supported by FANN) can also be read by read.fcnndataset. read.fcnndataset(fname) write.fcnndataset(fname, input, output) fname input output character string with the filename numeric matrix, each row corresponds to one input vector numeric matrix with rows corresponding to expected outputs, the number of rows must be equal to the number of input rows Details Files are organised as follows: The first comment (beginning with #) is the dataset information (ignored on read), three numbers determine: number of records, no. of inputs and no. of outputs, each data record has two or three lines: (optional) record information in a comment (beginning with #), line with input values, line with output values. read.fcnndataset returns a dataframe. write.fcnndataset does not return.

27 read-write-fcnndataset 27 Examples # set up the XOR problem inputs and outputs inp <- c(0, 0, 1, 1, 0, 1, 0, 1) dim(inp) <- c(4, 2) outp <- c(0, 1, 1, 0) dim(outp) <- c(4, 1) # write dataset write.fcnndataset("xor.dat", inp, outp) # show the output file file.show("xor.dat") # read dataset xordf <- read.fcnndataset("xor.dat") # show the imported dataset show(xordf)

28 Index Topic classes is.mlp_, 3 mlp_, 5 mlp_-class, 8 Topic package FCNN4R-package, 2 Topic pruning mlp_prune_mag, 16 mlp_prune_obs, 17 Topic teaching mlp_teach_bp, 20 mlp_teach_grprop, 21 mlp_teach_rprop, 23 mlp_teach_sa, 24 mlp_teach_sgd, 25 FCNN4R-package, 2 is.mlp_, 3 mlp_eval, 4 mlp_expand_reorder_inputs mlp_, 5, 5, 8, 11 (mlp_-manipulating-work-inputs), 12 mlp_export_c, 4 mlp_export_fcnn (mlp_-export-import), 10 mlp_get_layers (mlp_-general-information), 11 mlp_get_name (mlp_-names), 14 mlp_get_no_active_w (mlp_-general-information), 11 mlp_get_no_w (mlp_-general-information), 11 mlp_get_w mlp_plot, 16 (mlp_-accessing-individual-weights), mlp_prune_mag, 16 7 mlp_prune_obs, mlp_get_w_abs_idx (mlp_-absolute-weight-indices), 6 mlp_get_w_idx (mlp_-absolute-weight-indices), 6 mlp_get_w_st (mlp_-accessing-individual-weights), 7 mlp_get_weights (mlp_-weights-access), 15 mlp_grad (mlp_-mse-gradients), 13 mlp_gradi (mlp_-mse-gradients), 13 mlp_gradij (mlp_-mse-gradients), 13 mlp_import_fcnn (mlp_-export-import), 10 mlp_jacob (mlp_-mse-gradients), 13 mlp_merge (mlp_-combining-two-works), 9 mlp_mse (mlp_-mse-gradients), 13 mlp_-absolute-weight-indices, 6 mlp_-accessing-individual-weights, 7 mlp_-class, 8 mlp_-combining-two-works, 9 mlp_-display, 10 mlp_-export-import, 10 mlp_-general-information, 11 mlp_-manipulating-work-inputs, 12 mlp_-method (mlp_-class), 8 mlp_-mse-gradients, 13 mlp_-names, 14 mlp_-weights-access, 15

29 INDEX 29 mlp_rm_input_neurons (mlp_-manipulating-work-inputs), 12 mlp_rm_neurons, 18 mlp_rnd_weights, 19 mlp_set_activation, 19 mlp_set_name (mlp_-names), 14 mlp_set_w (mlp_-accessing-individual-weights), 7 mlp_set_w_st (mlp_-accessing-individual-weights), 7 mlp_set_weights (mlp_-weights-access), 15 mlp_stack (mlp_-combining-two-works), 9 mlp_teach_bp, 20 mlp_teach_grprop, 21 mlp_teach_rprop, 23 mlp_teach_sa, 24 mlp_teach_sgd, 25 print,mlp_-method (mlp_-display), 10 read-write-fcnndataset, 26 read.fcnndataset (read-write-fcnndataset), 26 show,mlp_-method (mlp_-display), 10 summary,mlp_-method (mlp_-display), 10 write.fcnndataset (read-write-fcnndataset), 26