Application of neural networks to model catamaran type powerboats

Transcription

1 Application of eural etors to model Catamaran Type Poerboats Application of neural netors to model catamaran type poerboats Garron Fish Mie Dempsey Claytex Services Ltd Edmund House, Rugby Road, Leamington Spa, UK Abstract Poerboats in operation represent a system consisting of a number of complex components such as: surface propellers, aerodynamics and hydrodynamics; hich interact ith each other and ith the ind and ater surface conditions. By measuring the behaviour of the poerboat it is possible to create a mathematical model using system identification methods. A neural netor model has been generated hich can be used to predict ho the poerboat ill perform under different driver inputs for the purpose of optimizing performance. Keyords: neural netors; system identification; poerboats surface propellers and environmental conditions such as the ater surface and ind speed and direction. The aim of generating a mathematical model of the system as to be able to investigate the effect on the boat performance of variations in driver input and boat setup. A neural netor system identification method as selected as the most appropriate ay to model the system. eural netor techniques can be very effective at identifying complex nonlinear systems hen complete model information cannot be obtained [2]. A Modelica library called A_SID has been developed to facilitate system identification using neural netors. The library contains different types of neural netor and several training methods and has been applied to study the poerboat system. Introduction There are many different approaches to mathematical modelling and the decision about the most appropriate method to use is based on hat a priori noledge is non about the system. Modelica is typically used for hite box modelling, hich is based on the application of the universal las and principles. This paper discusses the use of blac box modelling techniques that are entirely based on the use of measurement data to generate the mathematical model []. In blac box modelling, the inputs and outputs of an unnon system are used to create a model that produces an output close to that of the actual system, hen supplied ith the same inputs. eural netor system identification is one method that can be used to create blac box models. In the case of a poerboat, it is convenient to model the system as a blac box, as it is not feasible to model the behaviour of the system as a hite box model. Figure shos a class poerboat under race conditions. To create a hite box model e ould need to create models of the aerodynamic and hydrodynamic effects, and their interaction ith the Figure. Vie of a poerboat during operation. Image courtesy of Victory Team 2 eural netors 2. An artificial neural netor An artificial neural netor is a netor of functions called neurons, hich are connected by eighted signals see Figure 2. This architecture is loosely based on a biological neural netor. eural netors can be used for a variety of tass such as system identification and classification. The A_SID library provides neural netors appropriate for system identification tass. The Modelica Association 247 Modelica 2008, March 3 rd 4 th, 2008

2 G. Fish, M. Dempsey Input layer Hidden layer Output Layer bias eight neurons bias Figure 2. Feedforard neural netor Figure 2 shos a simple diagram of a typical neural netor, commonly called a feedforard neural netor, hich consists of an input layer, a hidden layer and an output layer. In both the hidden and output layers, the eighted sum of the inputs to the layer and the bias, are applied to neuron functions. The formulae that describe the feedforard neural netor in Figure 2 are shon belo. Equation calculates the outputs of the hidden layer and equation 2 calculates the output of the neural netor. 2.2 Types of neural netors implemented in the A_SID library The A_SID library provides pre-defined neural netor models for feedforard neural netors and a form of dynamic recursive neural netors called eural etor Output Error. Within each type of neural netor there can be any number of inputs, hidden neurons, neuron layers and output neurons. In the dynamic recursive neural netor, the output of the neural netor can be used as an input to the neural netor, as shon in Figure 3. This type of recursive netor is used for modelling dynamic systems here the next output is affected by the previous output values and previous input values. The process of minimising the cost function of the neural netor is called training. In the A_SID library both bacpropagation and the Levenbergdelay delay ui vi + vn+ i o + m+ o O y Y 2 here: o is the output of the hidden layer O is the hidden neuron function u i is the input v n+ is the bias eight for the neuron there are n inputs y is the output of the neural netor Y is the output neuron function m+ is the bias eight for the neuron there are m hidden neurons t is the current sample Within the A_SID library the most common neuron functions such as linear, sigmoid and tanh are available. The user can also easily add their on neuron function by extending from the neuron function base class and implementing the required function. Figure 3. Dynamic recursive neural netor 2.3 Training of the neural netor The eights and biases in a neural netor have to be trained so that the output of the neural netor approximates the actual system ell. The mean square error beteen the actual output and predicted output is the cost function determining the measure of the closeness of the approximation of the neural netor to the actual system, as in equation 3. MSE 2 z y 2 3 here: MSE is the mean square error z is the target value output from the actual system y is the output of the neural netor is the total number of target values The Modelica Association 248 Modelica 2008, March 3 rd 4 th, 2008

3 Application of eural etors to model Catamaran Type Poerboats Marquardt training methods are available. These methods have been implemented in Modelica in both continuous and discrete forms. The choice of method to train a neural netor is influenced by the size of the neural netor and the amount of data being used to train the netor. The continuous training methods have the advantage that the gradient, hich is the rate of change of the eights, is accurate everyhere, not only at the linearization points as ith discrete methods. This can result in the search method travelling along the bottom of valleys of the cost function and not oscillating along valley alls. The continuous method interacts ith the variable step solvers to determine the step-size. If the gradient changes suddenly then the solver ill reduce the step size to deal ith this efficiently. The disadvantage of the continuous method is that it generates huge numbers of equations due to the ay that Dymola expands the for loops used in the model. By using Modelica functions and external C functions these problems can be minimized through the reuse of code sections. Data storage and the manipulation of large matrices in Dymola can also generate problems ith large neural netors if the continuous training methods are used. The discrete methods have been implemented to overcome these issues Bacpropagation It is possible to train a neural netor by calculating the gradient of the cost function ith respect to the eights, and to then adust the eights in the appropriate direction to reduce the cost function. This method is called bacpropagation and can be slo to converge to a solution. Appendix A has further information about ho the gradient is calculated Levenberg-Marquardt The Levenberg-Marquardt training method generally requires feer iterations than the bacpropagation method to train a neural netor. Hoever the LM method is more complex and requires more computation and memory to perform each iteration. The rules used to calculate the eights are described in Appendix B Recursive method In this method the partial derivative of the neural netor ith respect to the eights is required. From this partial derivative the gradient and Hessian matrices can be calculated. Once e have determined these matrices either the bacpropagation or Levenberg-Marquardt training methods can be used to minimise the cost function. Modelica provides semantics to define partial derivatives and Dymola is able to utilise these semantics to generate the symbolic derivative of functions. Example shos ho the partial derivatives are defined in Modelica. This method as used to help define a function to calculate the partial derivatives of the neural netor ith respect to the eights. [Example: The specific enthalphy can be computed from a Gibbs-function as follos: function Gibbs input Real p,t; output Real g; algorithm... end Gibbs; function Gibbs_TderGibbs, T; function specificenthalpy input Real p,t; output Real h; algorithm h:gibbsp,t-t*gibbs_tp,t; end specificenthalpy; ] Example. An example of Modelica code for the generation of the partial derivative of a function. Quoted from Modelica 3.0 Specification [4] A_SID Implementation An example of training a neural netor using the A_SID library is shon in Figure 4. The training methods are implemented in the replaceable training component and the user simply selects the required method. Training component Figure 4. A_SID training performed in a model The Modelica Association 249 Modelica 2008, March 3 rd 4 th, 2008

4 G. Fish, M. Dempsey 2.4 Improving the neural netor training When training a neural netor it is important to have confidence that the neural netor ill approximate ell ith inputs that are not part of the training data. This ability is non as generalisation [5]. One ay to investigate this is to divide the data into to sets, one that is used to train the neural netor training data, and one that is used to test the generalisation of the neural netor test data. Generalisation is liely to be improved by reducing the number of eights used in the neural netor [3]. The A_SID library supports both eight decay and pruning methods to reduce the number of eights and improve generalisation. To pruning methods are available in the library and these are non as Optimal Brain Surgery and Optimal Brain Damage. These algorithms determine hich eights to remove from the neural netor. The remaining eights are then updated to reduce the errors introduced by removing the eights for further details refer to [3]. Weight decay is another approach to removing eights from a neural netor. In this method a penalty proportional to the magnitude of the eights is added to the cost function see [3] for further details. All cost functions should contain a measure of the closeness of the neural netor outputs to the desired output. Adding a eight penalty to the cost function generates a trade off beteen reducing the magnitude of the eights and reducing the closeness measure. As a result of this the eights that have little effect on improving the closeness measure ill no be reduced in magnitude. 3 Poerboat operation The type of poerboat that has been modelled using the A_SID library is a Victory Team class offshore poerboat as shon in Figures and 5. These boats have a catamaran hull ith to engines and a central rudder. Each engine drives a height adustable, steerable propeller. The boats are operated by to cremembers: a throttle man and a driver. Beteen them they have 5 controls, hich are: A steering heel that directly controls the rudder angle. The steered angle for the propellers is also controlled by the steering heel angle. The propeller heights are set using to rocer sitches. These control the trim pistons that move the propellers vertically. Throttle position is set using the throttle levers for the left and right engine. trim piston rudder Figure 5. Rear vie of a poerboat. Image courtesy of Victory Team As the boat accelerates it begins to plane and travels higher above the ater, i.e. less of the hull is belo the ater line. As the boat lifts out of the ater, the propellers are loered trimmed don to control their depth in the ater. The trim height or propeller depth also affects the pitch angle at hich the boat travels. In general, the loer the depth of the propellers, the loer the pitch. If the boat is travelling at a pitch angle that is too high for the speed it is doing, it ill flip over a bloover. If the pitch angle of the boat is too lo the result ill be a larger surface area of the boat in the ater and thus an increase in drag. When cornering, the catamaran poerboat rolls to the inside of the corner due to the asymmetrical hull design. If the cornering is too severe for the current speed, the boat ill begin to roll to the outside of the corner, and ill roll over if the drivers do not tae correcting action. A typical cornering manoeuvre requires the throttle man to reduce the throttle to slo the boat to a controllable speed before the corner, and the driver to steer the boat along the course, ensuring that the steering angle is not too steep for the current speed. The Modelica Association 250 Modelica 2008, March 3 rd 4 th, 2008

5 Application of eural etors to model Catamaran Type Poerboats Figure 6. A Racecourse. The poerboats must travel along a course defined by buoys. There are three different types of laps. The start lap in red, the short lap in blac and the long lap in green. Supplied courtesy of IOTA. A race involves the boats travelling around a course defined by buoys laid out in the ater see Figure 6. The drivers try to select good trim height to maximize acceleration hile maintaining stability. The drivers also try to find good throttle, rudder and trim positions for cornering that result in fast and stable cornering. The neural netor can investigate different possible driver inputs and predict their effects on boat performance over a lap. 4 Poerboat model 4. Defining the neural netor To model the poerboat using neural netor techniques, a significant amount of data is used to characterise the system. During races and testing sessions the boats are fitted ith a data logger that records the data required to train the neural netor. The input data required is: The engine throttle positions The rudder angle The trim height of both propellers The target output data required is: The engine speed of both engines The boat speed The ya rate of the boat Using this input and output data e can train a neural netor to represent the poerboat system and then use the neural netor to investigate the system performance ith different inputs. As the boat is an example of a dynamic system, the dynamic recursive neural netor as chosen. The model has been generated from data recorded by Victory Team from their racing boat number 77 during the 2007 Arendal race. During this race the boat completed 2 laps of the course. The measured data as filtered using Basel and Chebyshev filters to reduce the amount of noise and high frequency components in the data. The filtered data as then re-sampled from 00Hz to.7hz to reduce the number of duplicate data points and to decrease the amount of time required to train the neural netor. Finally the data as divided into training and test data sets. 4.2 Training the neural netor Training a neural netor for such a complex system is done in a number of steps. When first training a dynamic recursive neural netor it is not non ho many past outputs and inputs ill result in the model giving a good representation of the poerboat system. It is also not non ho many neurons ill be required, or hich neuron functions should be used. These can only be determined by trying different configurations to find the best setup. The first step in training this type of neural netor is to train it to only predict the next output value from the previous data value. The eights from this training are then used as the initial eights for the recursive training. The recursive training algorithm described in as used ith the Levenberg- Marquardt method to train the neural netor. To improve the generalization, eight decay as used. 4.3 Correlation results After training, the MSE for the neural netor using the training data set as and the MSE for the test data set as This means that the neural netor has been trained successfully and is able to accurately predict the performance of the poerboat, as shon in Figure 7. In Figure 7, the recorded driver inputs have been fed in to the trained neural netor and the outputs for boat speed, engine speed and the ya rate of the boat are compared to the measured data. Overall the results sho that there is very good correlation beteen the neural netor and the real poerboat. There are some small deviations hich could be due to a number of different factors, such as sell and The Modelica Association 25 Modelica 2008, March 3 rd 4 th, 2008

6 G. Fish, M. Dempsey ind conditions along the course, that are not accounted for in the neural netor. neural netor. Using the optimised trimming strategy the poerboat ould tae s less to travel along a 2m straight than using the example trim strategy. Figure 7. Comparison of simulated neural netor ith recorded data for a single lap of the course. 4.4 Optimisation of the trim strategy Section 3 describes ho the propeller height trim heigh affects the performance of the poerboat. By using the neural netor it is possible to determine hat the optimum trimming strategy is for the poerboat. The model shon in Figure 8 uses the trained neural netor to simulate the poerboat accelerating from an initial speed up to its maximum speed. Figure 9. Comparison of a simulated trim strategy ith a real trim strategy. The simulated optimal results Simulated Speed and Simulated Trim assume the boat is travelling perfectly straight. The Example Trim strategy as taen from the race data and applied to the simulator. The neural netor used in the model can only be expected to accurately model an operating region if this region as sufficiently excited during the data recording stage. In Figure 0 the histogram data identifies hat trim position data is available for the operating region of the simulated result. The optimum trim strategy is limited by the availability of data see Figure 0. The upper bound on the trim data is probably due to driver caution because of the ris of a bloover in this operating region. Figure 8. Acceleration model test. The throttle position is set to 00% and the boat is travelling in a straight line. Figure 9 compares an example trimming strategy extracted from the race data and the optimized trim strategy that has been determined ith the use of the Figure 0. Histogram plot of recorded trim position at the operating state. The optimised trim position is plotted over the histogram as hite circles The Modelica Association 252 Modelica 2008, March 3 rd 4 th, 2008

7 Application of eural etors to model Catamaran Type Poerboats 5 Conclusions A library called A_SID has been developed for the development and training of neural netors for system identification. This library as used to generate a blac box model of a poerboat, and this model as then used to determine an improved trimming strategy that should deliver improvements in boat performance. Acnoledgments Victory Team have indly provided the photographs used in this paper and the poerboat data used to carry out this study. IOTA have indly provided the course map used in Figure 9. References [] Estrada-Flores S., et-al. Development and validation of grey-box models for refrigeration applications: a revie of ey concepts. International Journal of Refrigeration, pages , July [2] Wen Yu, onlinear system identification using discrete-time recurrent neural netors ith stable learning algorithms. Information Sciences, pages 3-47, [3] M. ørgaard, O. Ravn,.K.Poulsen and L.K.Hansen, eural etors for Modelling and Control of Dynamic Systems [4] Modelica 3.0 Specifications modelica_3_0/vie [5] eural etor FAQ, part 3 of 7, ftp://ftp.sas.com/pub/neural/faq3.ht ml Bacpropagation al- APPEDIX A: gorithm By calculating the gradient of the cost function see Section 2.3 it is possible to update the eights in a ay that ill reduce the cost function. The example belo is ho bacpropagation ould be used to update a feedforard neural netor using the MSE as the cost function. The folloing equations describe a neural netor. uivi + vn+ i o + m+ o O y Y 2 The cost function is the mean square error i.e. MSE: V 2 z y 2 3 The calculation of the partial derivative of MSE ith respect to output eight follos: 2 Substituting in 2: z y 2 y ε let ε z-y ε Y y ε a o o here a + o m+ + m+ The calculation of the partial derivative of the cost function ith respect to hidden eight follos: i i i 2 y ε v i z y 2 note that ε and y are vectors i i y ε a o y o ε i a bi here: b i O u v + v The discrete eight update method is: i i i n + i ui m+ The Modelica Association 253 Modelica 2008, March 3 rd 4 th, 2008

8 G. Fish, M. Dempsey -η v i By choosing η sufficiently small, the cost function can be decreased at each iterate. The continuous method uses the gradient calculated above to update the existing eights continuously. APPEDIX B: Levenberg-Marquardt algorithm In the bacpropagation algorithm the search direction is calculated from the first order Talyor approximation of the cost function. The Levenberg- Marquardt algorithm maes use of the second order Talyor approximation of the cost function to update the eights. The second order approximation of the cost function follos: ^ V θ Vθ * + θ -θ * V θ * + ^ ½θ -θ * Vθ * θ -θ * V θ Vθ * + θ -θ * G + ½θ -θ * Hθ -θ * here: θ represents all the eights in the neural netor θ* are the eights at hich the Taylor approximation is made. V is dv/dθ and equal to the gradient G V is d 2 V/dθ 2 and equal to the Hessian H minimum of L is far from the current iterate, θ i, a poor search direction may be obtained. [3]. θ i+ here: arg min L θ i θ subect to θ i+ - θ i δ i 5 λ i has a monotonic relationship ith δ i [3]. Where increasing λ i decreases δ i and visa versa. The eights are updated using the folloing rule: [Rθ i + λ i I] θ -Gθ here: θ θ i+ - θ i The update rule for the λ value follos:. If the L i value approximates MSE ell, then λ i+ λ i /2 and thus increasing the search region. 2. If the L i value does not approximates MSE ell, then λ i+ λ i *2 and thus decreasing the search region. 3. Leave λ * if neither the or 2 thresholds are true. To get a more detailed explanation on the update rule for λ * refer to [3]. In the Levenberg-Marquardt method a further approximation is made; the Hessian is approximated by the folloing equation: R θ dy dy dθ dθ This is valid hen the MSE is the cost function. Let the approximation of the cost function be: Lθ Vθ * + θ -θ * G + ½θ -θ * Rθ -θ * 4 This cost function is minimised using an iterative process; here the next eights are limited to a region around the current eights see 5. Limiting the range of the search is often effective as If the The Modelica Association 254 Modelica 2008, March 3 rd 4 th, 2008