The CREDOS Project. Pilot model for wake vortex encounter simulations for take-off and departure D3-6

Transcription

1 The CREDOS Project Pilot model for wake vortex encounter simulations for take-off and departure D3-6 Abstract: A pilot model () is developed for fast-time Monte Carlo Simulations with varying wake vortex encounter conditions for safety analyses. The model consists of different submodules to perform the take-off run, rotate, follow a standard instrument departure route and recover and stabilize the aircraft during and after a wake vortex encounter. The generates sidestick pitch, sidestick roll, pedal, throttle lever, flap lever, and landing gear lever commands and manipulates those control elements as a pilot would do. For design and validation of the, data was generated in a certified A33 Full Flight Simulator and on A3 development simulator with licensed commercial airline pilots. For final validation, the was implemented into the flight simulation to compare it with recorded pilot reactions. Contract Number: AST-CT Proposal Number: 3837 Project Acronym: CREDOS Project Co-ordinator: Document Title: EUROCONTROL Delivery Date: NOV 9 Pilot model for wake vortex encounter simulations for take-off and departure Deliverable Nr: D3-6 Responsible: TU Berlin Nature of Deliverable: R (Report) Dissemination level: PU (Public) File Id N : CREDOS_37_TUB_DLV_D-6_Pilot model for wake vortex encounter simulations for takeoff and departure_v. Status: approved Version:. Date: 3 NOV 9 Approval Status Document Manager Verification Authority Project Approval TU Berlin Airbus Project Management Committee Swantje Amelsberg Sebastian Kauertz C Members Task Manager WP3 Leader Status can be: draft, for approval, approved. Only approved deliverables may be sent to the EC. The first version delivered to the EC should always be version (any previous versions should be a, b etc). Any official update (either requested by the EC or proposed by the C will have subsequent numbering as required). Page

2 Author Swantje Amelsberg, TU Berlin Edition history Edition Nº Date Authors Section Comment va 8 Oct 9 S. Amelsberg All Initial version v Nov 9 S. Amelsberg All Final version v. 3 Nov 9 S. Amelsberg All Dissemination level changed from confidential to public Page

3 GLOSSARY Symbols b Span p Roll rate q Pitch rate Φ Bank angle Θ Pitch angle ψ Azimuth γ Path angle δ Sidestick deflection τ Pilot lag time Γ Vortex Circulation H Altitude above ground x Pos. coordinate with ref. to the longitudinal axis y Pos. coordinate with ref. to the lateral axis z Pos. coordinate with ref. to the yaw axis Indices cmd command disc disconnect gr Geodetic system Pilot Model targ Target WV Wake Vortex Abbreviations AAM Average Angle Measure A/C Aircraft AP Auto Pilot A/THR Auto Throttle FAR False Alarm Rate FCOM Flight Crew Operating Manual FD Flight Director HTR Hit Rate ICAO International Civil Aviation Organization MCS Monte Carlo Simulation MSE Mean Square Error NN Neural Networks Pilot Model POP Probability of prediction SID Standard Instrument Departure Route TUB Technische Universität Berlin (TU Berlin) WVE Wake Vortex Encounter Page 3

4 EXECUTIVE SUMMARY This technical note describes the development of a pilot model () that is able to perform the take-off run, rotate, follow a standard instrument departure route and recover and stabilize the aircraft during and after a wake vortex encounter. It can be used in fast-time Monte-Carlo simulations with varying wake vortex encounter conditions for safety analyses. The generates sidestick pitch, sidestick roll, pedal, throttle lever, flap lever, and landing gear lever commands and manipulates those control elements as a pilot would do. For each control element a sub model is developed. The sub models for sidestick pitch and roll commands are modelled by artificial neural networks (NN). Those for pedal position, flap, landing gear, and throttle lever use classical control schemes. A further part of the is a disconnect logic that disables the autopilot (AP) during wake vortex encounters in automatic flight if the pilot decides to counteract and recover from the vortex-induced disturbances manually. For design and validation of the NN sub models, data was generated in a certified A33 Full Flight Simulator and an A3 development simulator with licensed commercial airline pilots. For final validation, the was implemented into the flight simulation and it flew identical encounters as the airline pilots before. This allowed comparing the outputs with the human pilot control commands. The outputs show a very good agreement with the human pilot behaviour in the roll axis for the A3 tests in all fixed encounter cases. The for sidestick roll commands fulfills in most of the cases the defined validation requirement during wake vortex encounters. Also in longitudinal motion a qualitatively good agreement was achieved between human pilots and pilot model. Page 4

5 Table of Contents EXECUTIVE SUMMARY...4. INTRODUCTION...6. FLIGHT SIMULATOR TESTS PILOT MODEL GROUND TRACKING ROTATION FLAP, LANDING GEAR AND THRUST LEVER NEURAL NET MODELLING Neural net topology Neural network parameter optimisation Pilot model inputs Neural network for lateral control Neural network for longitudinal control AUTO PILOT DISCONNECT MODEL SOFTWARE IMPLEMENTATION PROBABILISTIC ADD-ON MODEL VERIFICATION AND VALIDATION PILOT MODEL OFFLINE VERIFICATION STATISTICAL ANALYSIS OF VARIATION IN HUMAN PILOT BEHAVIOUR Experimental data IDENTIFICATION OF STATISTICAL SCATTER IN TIME HISTORIES Requirements of pilot model validation Pilot model validation...3. CONCLUSION REFERENCES...33 A. APPENDIX...34 A.. TO A3 SIMULATION INTERFACE DOCUMENT...34 A.. PILOT MODEL (SIMULINK MODEL)...36 A.3. TIME HISTORIES OF PILOT RUNS...3 A.4. BOXPLOTS OF STATISTICAL SCATTER...9 A.. VALIDATION PLOTS (PILOTS VS. )...63 Page

6 . INTRODUCTION The CREDOS (Crosswind-Reduced Separations for Departure Operations) project, funded by the European Commission, studies the concept of reduced wake turbulence separation between consecutive departing aircraft for single runway operations []. For the validation of this concept, a risk assessment is required. The chosen method computes wake vortex encounter probabilities and encounter severities by means of fast-time Monte Carlo simulations (MCS). This approach is commonly used for risk assessment as real-time pilotin-the-loop simulations are too time-consuming and therefore not suitable for MCS. The aim of the MCS is to assess the potential impact of the proposed reductions in separation on flight safety and the efficiency of airport operations in crosswind conditions. As simulation results shall be used to revise separation standards without negative impact on safety, they have to be accepted by the authorities. Therefore all models need to have a high fidelity and strict validation requirements apply for all parts of the simulation including the aircraft model, the aircraft vortex interaction model and the model for the pilot control behaviour. As a part of CREDOS, Technische Universität Berlin (TU Berlin) is developing pilot models () for risk assessment that shall mimic the pilot s control behaviour when tracking a Standard Instrument Departure (SID) route and when compensating the disturbances of a wake vortex encounter. Data for development was recorded in flight simulator tests, in which pilots experienced vortex encounters during climb out. The tests were performed on an Airbus A33 Level D Full-Flight Simulator and on an A3 development simulator. Commercial pilots from several airlines with varying flight experience participated in the tests [7]. The objective is to develop a pilot model that performs take-off, departure and wake vortex recovery, that can be used in Monte Carlo simulations, and that is suited for the CREDOS risk assessment, see FIG -. Pilot model design is based on the data acquired during the piloted WVE simulations. Wake vortex simulation model Lamb-Oseen Burnham & Hallock Proctor Winckelmans Pilot model deterministic and probabilistic 4 3 Vortex induced accelerations Desired flight path Δ - Pilot inputs Aileron Elevator Rudder Thrust Aircraft simulation model Aircraft reaction WVE Risk analysis / Hazard rating FIG -: Monte Carlo Simulation (MSC) elements Page 6

7 . FLIGHT SIMULATOR TESTS A detailed description of the piloted simulator tests can be found in [7]. Hence, this chapter gives only a brief overview of how the flight simulator data was gathered. Two flight simulators were used to investigate piloting techniques and pilot control behaviour: The Airbus A33 Full-Flight Simulator, that is located at TU Berlin, Institute of Aeronautics and Astronautics and is operated by the "Zentrum für Flugsimulation Berlin" (ZFB). The A3 THOR Simulator, an Airbus development simulator, is located at Airbus Deutschland GmbH in Hamburg. Both simulators are equipped with research facilities that allow efficient modifications of the simulator software for scientific experiments. The simulator software has been supplemented with the wake vortex encounter software package that was developed in the S-WAKE (Assessment of Wake Vortex Safety) project []. This allows highly realistic simulations of wake vortex encounters. A major difference is that the A33 simulator has a motion system whereas the THOR simulator is fixed-based. The importance and limits of motion simulation during wake vortex encounters was investigated in [] reporting indications that pilot behaviour is affected by the cues from motion simulation. However, a final conclusion was not drawn as the statistical data base was too small. Therefore the hypothesis here is that the impact of motion on pilot response lies within the statistical variations. Using the simulation tool WakeScene [] a number of different test cases for the piloted simulator campaigns were developed that cover the most probable encounter conditions during take-off. Test cases are defined by the encounter geometry, encounter altitudes and vortex characteristics. The test scenarios were the same for both simulators and the conditions are given in TAB -. For each test run, a constant crosswind was defined that is typical for a CREDOS scenario. The aircraft parameters are summarized in TAB -. Scenery Frankfurt/Main airport (EDDF) Runway R 49 SID Crew Visibility Wind QNH Ceiling TOBAKF Pilot flying from the left or right hand seat; supported by an engineer CAVOK 9 / kt 3.hPa /8 TAB -: Simulator test scenarios A3 THOR simulator (Airbus) A33 FFS (TUB) A/C type A3- A33-3 A/C gross weight 66 kg 6 kg CG % MAC 8% MAC Thrust rating Flex 4 Flex 6 V Rot 37 kts (Flaps ) 6 kts (Flaps ) V 4 kts (Flaps ) 33 kts (Flaps ) TAB -: Aircraft data for simulator tests Page 7

8 During a simulator campaign, the pilots were flying departures, in which they experienced wake vortex encounters that varied in magnitude and character. As pilots actions on the controls should be recorded, they were asked to fly most departures manually. However, some departures were flown automatically, i.e. with Autopilot (AP) engaged, to investigate when and why a pilot disconnects the AP during a vortex encounter. In summary, simulator sessions were carried out in the A33 simulator. different airline pilots (6 captains, first officers) flew 69 wake vortex encounters in total. On the A3 simulator, a total of 76 wake vortex encounters in 4 simulator sessions with 4 different pilots ( captains, 9 first officers) were flown. Page 8

9 3. PILOT MODEL For standard departure route tracking and wake vortex recovery, a pilot model is developed for offline flight simulation. The variability of control tasks (see TAB 3-) requires a pilot model consisting of several sub modules; see FIG 3- to manipulate the sidestick pitch, sidestick roll, pedal, throttle lever, flap lever, and landing gear lever and control those elements as a pilot would do. The following table TAB 3- compares the control task the model should be able to perform, with its responsible sub-module and the corresponding output: Control task Sub-Module Output (Control element) Take-off run on the runway centre line Ground tracking Pedal position Rotation Rotation Sidestick pitch Climb along the desired flight path Correct lateral deviations Match the stationary aircraft climb gradient to the actual thrust level SID tracking Sidestick pitch Sidestick roll Maintain required aircraft speed by thrust control Stabilize the a/c after encountering a wake vortex Flap, landing gear, thrust lever control Wake vortex recovery Throttle lever (TL) Flap lever (FL) Landing gear lever (LGL) Sidestick pitch Sidestick roll Disconnect AP in case of high deviation from the nominal flight path AP disconnect model AP disconnect button TAB 3-: Overview of pilot model performance FIG 3- gives an overview of all sub models of the pilot model. Part of it is a mode management logic that selects sub models according to flight phase piloting procedures and switches to a recovery mode if a vortex encounter occurs. The first switch is between manual and automatic flight modes. In the automatic mode the upper loop is active, where the block AP controls the aircraft (block A/C) until the AP disconnect sub model switches to the manual mode. In manual departures either the blocks Take-off run, Rotation, SID tracking or wake vortex recovery can be active at one time, whereas the Flap/LG and Thrust lever models are active in parallel during the whole departure phase. Vector c contains the sidestick pitch and roll control as well as pedal position, where c AP stands for the auto pilot commands and c M for the manual pilot model commands. Vector d includes throttle, flap, and landing gear lever. Page 9

10 AP y c y c y e Take-off run Rotation c M c AP SID tracking WV recover AP disc.model d A/C Flap/LG/ Thrust lever FIG 3-: Sub models of pilot model Sidestick pitch and roll commands that are outputs of the Rotation, SID tracking, and WV recovery model, are modelled with a static feed-forward neural network. The other outputs provided by the sub models for Take-off run, Flap/LG and Thrust lever are based on simple algorithms and logics as functions of speed, altitude, pilot time delay, and Flight Director command. The following sections describe all the sub models in more detail. 3.. GROUND TRACKING In cross-wind conditions the pilot uses the pedals to control the aircraft on the runway centre line until rotation. The pedal command δ Pedal is computed as a function of Flight Director yaw command FD yaw and a pilot lag time τ P. () δ Pedal = f ( FD yaw ) 3.. ROTATION rely on visual attitude information and acceleration cues. Based on these considerations the pilot behaviour during rotation is modelled by using a low-gain pitch rate law. Pitch rate command increases from deg/s to a target pitch rate depending on the aircraft s attitude; and then decreases back to deg/s when reaching θ target = deg. This concept has been augmented by a second order time delay, modelling the smooth character of pilot inputs. The rotation control mode is active from the time the aircraft reaches the rotation speed until achieving the target pitch rate. The pitch rate command q cmd is computed with respect to rotation speed VR and target pitch rate q targ as follows () q cmd = S VR q targ T R s V < V + T Switch S VR = for R R s + S VR = for V VR From the available validation data, a range of.4 s T R.8 s, was identified; for subsequent analysis the mean value T R =.6 s will be used. Sidestick pitch deflection δ col is computed according to the law δ = ( q q) Kq, for Θ < Θ (3) col cmd targ Reasonable results were obtained for gains K q within a range of. K q 3.. Here the mean value of K q = is used. At a given offset from the target pitch attitude, rotation control Page

11 smoothly transitions to tracking mode. The SID tracking model is a neural network and is described in detail in section 3.4. The pilot model needs default values for rotation speed V R, target pitch rate q targ, and the target pitch angle θ targ. The values depend on aircraft type and configuration FLAP, LANDING GEAR AND THRUST LEVER The thrust control function is based on the piloted simulator test and the Flight Crew Operating Manual (FCOM). In a performance calculation, the correct thrust parameters for each aircraft, the defined takeoff configuration and environmental conditions were computed. Airlines train their pilots to use flexible thrust settings for take-off to increase engine life time and reduce maintenance cycles and costs. Based on the general operational procedures, the throttle lever deflection δ TL and the landing gear lever position δ LGL are determined as a function of altitude H above ground and climb rate γ. Whereas the thrust lever is limited to two discrete positions, the take-off trust and after reaching the thrust reduction altitude to continuous climb thrust. The landing gear lever has also two states, it is extended on ground an directly after takeoff, after passing ft and a positive climb angle it adopts the retracted state. The flap lever deflections δ FL depend on altitude H and calibrated airspeed V. There are three discrete states defined for the flap lever position. The first state is flaps plus slats until flap transition speed, afterwards the slats were retracted. When the a/c reaches the acceleration altitude the a/c has to be in a clear configuration. (4) δ TL = f (H) () δ LGL = f ( H, γ ) (6) δ FL = f ( H, V ) 3.4. NEURAL NET MODELLING Reference [8] explains Neural Networks (NN) as follows: Artificial NNs are mathematical models that emulate biological nervous systems and are composed of a large number of interconnected processing elements, similar to neurons. The elements are connected via weights that are analogous to synapses, which serve as the junction through which neurons communicate with each other. The synaptic connections are adjusted by learning processes in biological systems. A priori input and output data is needed for the development and training of the NN. In a training process the data sets are used to adjust the net s parameters. A major advantage of a Neural Network (NN) is that it can be trained to behave like a given non-linear dynamic system. Thus, NN can be used to model complex systems, like nonlinear human behaviour. In this work package an artificial NN is applied to the mathematical modelling of human pilot sidestick control inputs. The experimental data for NN design and training was gathered in the piloted simulator tests as described in section. The following subsections describe the theoretical background for NN design and the development of the NN for sidestick roll and pitch command in detail Neural net topology The structure of an artificial static feed-forward neural network with one hidden layer is shown in FIG 3-. The first layer of a NN is termed the input layer (index i). The activation function f can be found in the hidden layer (index j). The activation function which is generating the final output signal is denoted the output layer marked with index k. Page

12 INPUT LAYER HIDDEN LAYER OUTPUT LAYER f j f j f k f j FIG 3-: Neural network topology A network is created by the combination of a few simple artificial neurons where an artificial neuron can be mathematically described as a processing element, as it is given in FIG 3-3. INPUT LAYER x i, i =... l b j HIDDEN LAYER x w j x w j z j = b j l + w x i = ji i f ( j ) j z y j x l w jl FIG 3-3: Structure and function of a processing element Where x i is the input, w ji is the weight between input and hidden layer, b j is the bias in the hidden layer, f j (z j ) is the activation function in the hidden layer, and y j is the output of the hidden layer. Mathematically, the operation of a network of processing elements can be presented as follows: l (7) z = b + ( x w ), y = f ( z ) j j i= i The output layer usually uses an identity function as activation function (f k =), therefore the output of the network y k is generated as follows: m (8) y k = zk = bk + ( y j wkj ), j= where b k is the bias in the output layer and w kj is the weight between hidden and output layer. ji j j j Page

13 3.4.. Neural network parameter optimisation The NN is usually trained by backpropagation in MATLAB []. This is a supervised learning method, which requires a number of representative input data with corresponding output data. The parameter optimisation was performed in an open loop, this means with previous recorded data and no interaction between and A/C dynamics; see FIG 3-4. Pilot Model Optimization SSRO, SSPI Cost Function SSRO, SSPI Pilot inputs A/C Reactions Piloted WVE Simulation Data FIG 3-4: Neural network parameter optimisation The network training results in an adaptation of the weight and bias matrices to minimize the cost function between the given target output data y* and the calculated network output y k. At time t the network weights are adapted as follows: (9) w( t) = w( t ) + Δw( t) By training, the cost function J, that is the mean square error (MSE) between network output data and training data, is minimized. () N = * ( ( ) N y n n= J y k ( n )), where N is the number of data samples. The input weight adaptation can be described as () J Δw ji = η w, ji where η is the learning rate. The layer weights change Δw kj and the biases change Δb j in the hidden layer and the bias changes Δb k in the output layer are determined in the same way. Eq. (), Eq. (), Eq. (8), and Eq. (7) show what the algorithm s name implies, the errors of the output data is propagated backwards from the output nodes to the inner nodes. The Levenberg-Marquardt algorithm is chosen to minimize the network error. It combines the steepest descent and the Gauss-Newton method [9] One of the major challenges in NN design is to guarantee the validity and generalisation of Page 3

14 the network. Generalization means that the NN produces reasonable outputs for input signals for which it was not trained. A careful choice of network structures is necessary to avoid redundant hidden nodes that can reduce the generalisation capabilities of a network. To avoid over-fitting and to achieve a better generalization, cross validation is used in the training process. In this technique the available data is divided into three subsets. The first subset is the training set, which is used for computing the gradient and updating the network weights and biases. The second subset is the cross validation set and the third data set is used for offline verification (testing), as described in section 4. In training with cross validation the mean square error of the actual NN output is calculated for the training data set as well as for the cross validation data set. The MSE of the validation set is monitored during the training process. Normally, its MSE decreases during the initial phase of training, as does the training set error. However, when the network begins to overfit the data, the error of the validation set typically begins to rise. When the validation error increases for a specified number of iterations the training is stopped (in this case for epochs in a row), and the weights and biases for the minimal validation error are returned as the optimal solution. In addition, a post optimisation in MOPS (Multi Objective Parameter Synthesis) [4] was conducted using the average angle measure (AAM) as a second optimisation criterion; see EQ. () and FIG 3-. As it was not possible to use self defined cost functions in MATLAB [] for the neural net learning process, the AAM was maximized in post processing. () N ( ) ( ) 4 D n α n AAM n= = ; π N D( n) n= α( n) = arccos * y ( n) + y ( n) k ; D( n) D( n) = * y ( n) + y In Eq. () N is the number of recorded samples. D is the resulting vector length of the pilot y* and pilot model y k output and α is the resulting vector angle. In case of a perfect match between pilot and pilot model output, α would be deg for each time step which leads to an AAM of. FIG 3-: Definition of average angle measure Page 4

15 The correlation coefficient is also given here, because it is used in the next section as a parameter to evaluate how the neural network performs on the test data, but is was not an optimisation criterion. (3) R = N n= N * * ( y ( n) y ) ( yk ( n) yk ) = N * * ( y ( n) y ) ( y ( n) y ) n n= y* recorded human pilot outputs y k neural network modelled outputs * y, y k respective mean values of recorded and modelled pilot outputs N number of data samples Compared to the correlation coefficient R, which assesses only the phase characteristics between two signals, the AAM assesses magnitude and phase characteristics between the human pilot s and the modelled sidestick command. A value of corresponds to a perfect magnitude and phase match, - implies perfect magnitude correlation but 8 out of phase, and indicates that neither magnitude nor phase correlation exists. The average angle measure is the cost function that is minimized using the genetic algorithm (GA). The weight and bias values for the minimum MSE from the backpropagation process are taken as initial values for GA optimisation. GA is a global optimization algorithm, based on evolutional strategies that guarantee the survival of the fittest individual. The selection schemes used are tournament or rank selection with a shuffling technique for choosing random pairs for mating [4]. Limited computing power leads to a formulation that input and layer weight matrices of the neural network can be adapted by the optimization algorithm and not each individual weight and bias. This is realised by a factor δ which is handled as a tuning parameter in front of the input weight matrix w = δ w and the layer weight matrix w kj = δ w. The bias in the output layer should be adapted as well in order to avoid an wkj kj undesired offset. Finally, a pattern search (local gradient-free method) is used to make sure that the minimum is found, because GA is only able to converge in the region of the global minimum Pilot model inputs The Flight Director (FD) indicates size and direction of control inputs that are required to guide the airplane along the programmed route. The Flight Director bars are displayed on the Attitude Indicator, see FIG 3-6. ji k w ji ji k Page

16 FIG 3-6: Primary Flight Display Especially in wake vortex operation, the pilot reaction depends on visual cues as well as on motion cues. Piloted simulator data were analysed, using correlation coefficients between selected input parameters and pilot sidestick reaction. The neural network was tested with diverse combinations of selected pilot visual and motion cues, to find the best fit for the human pilot outputs. The following inputs were investigated for lateral control: Flight Director roll command, bank angle, roll rate, roll acceleration, and calibrated airspeed. The best results were achieved with parameter combinations of Flight Director roll command, roll rate and roll acceleration (bank angle is implicitly covered by Flight Director). For longitudinal control: Flight Director pitch command, pitch rate, pitch acceleration, pitch angle, sink rate, and change in sink rate were investigated. The combinations of flight director pitch command and pitch acceleration provide the highest correlation to the sidestick pitch signal. All input and training parameters discussed in the paragraphs above for lateral and longitudinal neural net design are summarized in TAB 3-. Parameter Lateral motion Longitudinal motion Inputs FDroll, p, p (WVE) FDpitch, q (WVE & TT) FDroll, p(tt) Output SS roll (y k ) SS pitch (y k ) Vector of data samples 38 Percentage division of data (training, validation, test) Division Initialize Activation function f j, f j Activation function f k Optimization algorithm Cost function 6%, %, % blockwise randomized Tangens hyperbolicus Identity function Levenberg- Marquardt Epoch number Maximum 6 Quality criterion MSE, AAM (Postprocessing) J train and a negative J val - gradient TAB 3-: Overview of parameters for network analysis Page 6

17 Neural network for lateral control Different net topologies were tested to achieve an optimal trade-off between a good generalisation and fitting properties for the NN pilot model. In close combination with the choice of input parameters, the following net topologies were compared: NN with one hidden layer versus a NN with two hidden layers, and between two neurons and twenty neurons in the first layer and fourteen neurons in the second layer. The second hidden layer gives no significant improvements in sidestick roll and pitch model output. The correlation between increasing the number of neurons and quality is non-linear. There exist some worst case combinations for the number of neurons and inputs. In general above a certain number of neurons a further increase of neurons does not significantly decrease the MSE. Lateral control is structured into two tasks. The first is to track the standard instrument departure route as precise as the average pilot in manual flight operations does; the second is to recover roll attitude when a wake vortex encounter occurs. Vortex recovery is a high gain task, whereas SID tracking is a low gain task, for each of which a neural network was designed. ) For wake vortex recovery a high gain is needed. The NN structure was investigated as described in the paragraph above. The best trade-off between good generalization capabilities and a good fit of pilot sidestick roll inputs yield a feed-forward network for lateral control with one hidden layer and 6 neurons with tangens hyperbolicus activation functions as shown in FIG. The thickness of the lines is proportional to the magnitude of the connection weight, and the shade of the line indicates the sign of the interaction between neurons: black connections are positive and grey connections are negative. The NN for the lateral wake vortex recovery applied on test data achieved a correlation coefficient R=.9 [Eq. (3) and the Average Angle Measure AAM=.67. The test data is measured simulator data which the NN never saw before, neither in training nor for cross validation. f j f j FD roll p p f j f j f j f k SS roll f j FIG 3-7: Neural network for lateral wake vortex recovery ) For the tracking task where accelerations are small a simpler network, see FIG 3-8, with only two neurons with Flight Director roll command and roll rate as inputs was sufficient. For the test data, it achieved a correlation coefficient R=.7, and an AAM=.4. Page 7

18 FD roll f j f k SS roll p f j FIG 3-8: Neural network for lateral control (tracking) Neural network for longitudinal control Pilot model tracking tasks and wake vortex recovery in longitudinal control is addressed similarly. For both tasks, a simple feed-forward NN with one hidden layer and 4 neurons with tangens hyperbolicus activation functions, as given in FIG 3-9, delivered the best results. Applied on test data the described network achieved a correlation coefficient of R=.78, and an AAM=.47. f j FD pitch f j f k SS pitch q f j f j FIG 3-9: Neural Network for longitudinal control 3.. AUTO PILOT DISCONNECT MODEL The pilot model () shall perform departures manually as well as automatically. In automatic flight the pilot monitors the flight situation. If excessive atmospheric disturbances cause deviations from the nominal flight path or from nominal attitudes, the pilot disables the Autopilot (AP) and recovers the aircraft manually. As wake vortex encounters can cause such deviations, a decision model for AP disconnect was developed. All automatic departures, where the pilot disconnected the Autopilot manually during the wake vortex encounter, were used to design the AP disconnect model. A model for Auto Throttle (A/THR) disconnect was not developed, because the data base of cases where the auto throttle was disconnected is too small. Correlations between aircraft parameters and AP disconnect conditions have been investigated. Finally the following decision logic delivered the best results: (4) AP _ disc = f (( Φ ( Φ > 3 p > 4 ) ( p > 3 / s)) > / s) Page 8

19 The boundaries are shown in FIG 3-, together with the used test data generated from simulator tests where pilots performed automatic departures. The black dots outside the boundary mark the departures where the Autopilot was manually disconnected. 4 AP disconnected AP enabled 3 Roll rate [deg/sec] Bank angle FIG 3-: AP disconnect requirements TAB 3-3 gives an overview on prediction capabilities. Correctly predicted were: a= cases for AP engaged and d=8 for AP disconnected. The prediction AP stays enabled was wrong in b= case, and AP was disconnected was falsely predicted in c= cases. AP enabled predicted AP disconnected predicted AP enabled AP disconnected occurred occurred a = b = c=3 d =8 TAB 3-3: AP disconnect statistic data Prediction performance can be quantified by the following indicators that are defined in [3]. The Hit Rate (HTR) represents the percentage of correct prediction. a + d a + b + c + d () HTR = % The probability of prediction (POP) is the ratio ( AP enabled or AP disconnected ) between correct predictions to overall cases. The best value for POP is one, and the worst is zero. a a + c (6) POP enabled = % Page 9

20 POP disconnected d = b + d % The False Alarm Rate (FAR) is the percentage of falsely predicted AP disconnects. The best FAR value is zero and the worst is one. The achieved values are: HTR =89.%, POPenabled=83.3% POPdisconnected= 94.7% and FAR=4.3% c ( c + d) (7) FAR = % These values were considered to be sufficiently accurate for the limited amount of data SOFTWARE IMPLEMENTATION The pilot model is developed under Matlab/ Simulink [] and converted via the Real-Time Workshop Embedded Coder to C-Code. This C-Code is implemented in the A3 simulation used in the VESA wake vortex encounter assessment tool. The Simulink with all submodels can be found in Appendix A in FIG A- to FIG A-44. The pilot model code is running as a submodel within the simulation loop of the A3 simulation. In each timestep the required inputs are fed to the pilot model and its outputs are passed on to the flight controls of the A3. For the simulation loop of the A3 there is no difference if these control inputs come from a human pilot or the pilot model. The same Fixed Encounter scenarios that were conducted with the pilots in the real-time simulator are then simulated again in the offline version, using the same simulation version that was used in the piloted tests as well as the same recorded wake vortex forces and moments. The aircraft dynamical parameters are recorded in the same way than during the piloted tests, so they can be compared easily. The data generated with the pilot model in this way is used in section The Interface definition between pilot model and A3 simulation is listed in TAB A PROBABILISTIC ADD-ON The paragraphs above described the development of a deterministic pilot model. This means that a given input of time histories always generates identical output commands. The provides an average behaviour of different human pilots. However, as a single real pilot also differs in his reactions to the same disturbance, a variability of the pilot model would add another layer of realism. The probabilistic pilot model can be seen as an extension of the deterministic model. When presented with identical input parameters it generates control inputs varying around the mean values generated by the deterministic model. For a reasonable number of simulation runs, the results from the probabilistic pilot model have to exhibit mean values and standard variations comparable to the values of the validation data sets. The statistical scatter of pilot lag times for flap-, throttle-, and landing gear lever tracking were evaluated as well as for sidestick roll and pitch, and pedal deflection. The lag times were computed based on the Page

21 NORMAL PROCEDURES by LTU. The order for flap-, landing gear- and throttle lever transition is a function of speeds and altitude, as it is described in section 3.3. TAB 3- an overview on the basis on which the pilot lag times were computed. LTU A33 NORMAL PROCEDURE TAKEOFF REV 3 SEQ Pilot action Requirement Description Retract landing gear V/S positive+ ALTI (Altimeter) Positive climb rate (vertical speed). Altimeter begins to show an altitude Transition to thrust lever climb At thrust reduction altitude Thrust reduction altitude is airport specific (according to noise abatement). In general ft AGL Transition from CONF + F to CONF Transition from CONF to CONF At acceleration altitude + F speed 3.4 Thrust reduction altitude is airport specific (according to noise abatement). In general ft AGL Minimum speed at which the flaps may be retraced at takeoff. S speed Minimum speed at which the slats may be retraced at takeoff. TAB 3-4: Transition speeds and altitudes The values for pilot lag time τ for landing gear retraction, flap lever transition, and thrust reduction can be found in TAB 3-, derived from the data gathered during the piloted simulator tests. In addition to the pilot lag timeτ a distribution for the amplitudes (stick gain Κ) of sidestick roll and pitch, and pedal deflection was calculated from the recorded data. Handbook for pilots which is the reference for normal procedures of daily operations Page

22 Pilot lag time/ gain Symbol s(x) Description LG lever delay [s] τ LGL 4.7 Pilot lag time in respect to the first point where landing gear lever retraction is allowed, see TAB 3-4, row. Thrust lever delay [s] τ TL 3.6 Pilot lag time in respect to the first point where thrust reduction is allowed, see TAB 3-4, row. Flap lever delay [s] τ FL. Pilot lag time in respect to the first point where flap lever retraction is allowed, see TAB 3-4, row 3. Flap lever delay [s] τ FL. Pilot lag time in respect to the first point where flap lever retraction is allowed, see TAB 3-4, row 4. Sidestick roll gain Κ roll. Sidestick gain is evaluated from TAB 4-. Mean value of the pilots gain for the first extreme deflection for all six fixed encounter cases Sidestick pitch gain Κ pitch. Is assumed to be a bit higher than for lateral reinforcement, because the pilot data for longitudinal control differs a lot. Sidestick roll delay [s] τ roll.4 Value of lag times is evaluated from TAB 4-. Mean value of the pilot lag time for the first extreme deflection for all six fixed encounter cases Sidestick pitch delay [s] τ pitch.6 It is assumed that the pilots have a higher lag time in longitudinal control. This is based on the general rule for the priority in unusual flight situations First aviate, than navigate, and than communicate. TAB 3-: Pilot lag time distribution The implementation of the probabilistic add-on can be described as follows: δ = δ (8) LGL LGL LGL Whereas τ is the individual lag time for each control element derived from the statistical scatter of deflection and K is the gain. A normal distribution is assumed for all probabilistic parameters. The distribution is cut off at ±σ (9% of cases) to prevent unrealistic values, see FIG 3-. An overall sidestick limitation of ± is defined, out of cockpit hardware installation reasons. = δ τ τ (9) TL TL TL () FL FL FL δ δ = δ τ δ = δ K τ () pitch pitch pitch pitch δ = δ K τ () roll roll roll roll Page

23 Density Normal Distribution K P K P Density 3... Normal Distribution τ e τ e [sec] FIG 3-: Assumed normal distribution for pilot lag time and pilot gain In FIG 3- an output example of the is shown for 3 runs were pilot lag time and pilot gain in sidestick deflection were varied normal in the boundaries according to TAB SSRO [-] FIG 3-: output of about 3 runs for one encounter scenario with varying pilot gain and lag time in sidestick roll * Deterministic Probabilistic Extreme Page 3

24 4. MODEL VERIFICATION AND VALIDATION The CREDOS project purpose of subtask 3..7 is the development of a pilot model that can be used in the VESA simulations for further risk assessment. The VESA flight simulation includes the A3 aircraft model; therefore all figures, tables, as well as the validation plots in the Appendix are based on the A3. The verification and validation of the pilot model is accomplished as described below. 4.. PILOT MODEL OFFLINE VERIFICATION verification was performed in an open loop, meaning there was no interaction between and A/C dynamics. The receives the pre-recorded inputs from piloted simulator test but its output does not affect the aircraft response. Verification is achieved by comparing the time histories of the outputs with the recorded pilot responses from the simulator tests. The verification is done after the parameter optimisation described in section FIG 4- shows an example of a verification case with pilot model output superimposed over data obtained from manned simulator trials during a standard departure with a wake vortex encounter. In the beginning of the takeoff run, the aircraft stays on the runway while the pilot pushes the sidestick during aircraft acceleration ( s < t < 8 s). The pilot model shows the same behaviour on the ground. After reaching the rotation speed, the pilot model starts to rotate ( s -3 s) and switches over to the neural net tracking task model, which is coupled to the flight director and the roll rate (3 s - s). If the roll rate exceeds /s the switches into the Wake Vortex Encounter (WVE) mode, to allow high dynamic reactions based on FD roll command, roll rate and roll acceleration for lateral control and FD pitch command and pitch acceleration for longitudinal control. Between s and 6 s the aircraft is encountering a strong wake vortex (Circulation Γ=m/s ). The reacts, recovers the aircraft and stabilises the steady state condition in the longitudinal and lateral axes. At the end of the run the returns into the tracking task mode after the roll rate has dropped below /s for at least 3 s. FIG 8 shows that the reacts comparable to the real pilot across all subtasks during take-off and departure. Side stick pitch. -. SSPI SSPI Pilot Time Side stick roll. -. SSRO SSRO Pilot Time FIG 4-: A3 pilot model (blue) versus commercial pilot (green) during take-off and departure (verification case) Page 4

25 4.. STATISTICAL ANALYSIS OF VARIATION IN HUMAN PILOT BEHAVIOUR For a deterministic pilot model validation, a statistical scatter of human pilot reactions is needed to demonstrate that the output lies within the analysed boundaries of the generated experimental data. This section describes the identification of natural statistical scatter in pilot behaviour and the resulting aircraft motion during simulated vortex encounter. Experimental data from the Fixed Encounter simulator sessions, described in section 4.., was evaluated. Possible requirements for the validation of pilot models are suggested. Generally, pilot response dynamics depend on the specific control task. However, other factors such as pilot training, flight experience, mental constitution, stress, and the workload, also affect pilot behaviour. When presented with identical conditions, the general characteristics of the pilot control inputs are essentially comparable, but the signals also exhibit some variability between individual pilots and also in-between tests repeated by the same individual. FIG 4- presents results from six different pilots who flew through the same fixed encounter, see [7]. During the actual encounter the pilot inputs show the same general characteristics, as each pilot makes an input with a sudden change in sign. However the input signals also show significant differences. Two forms of variation can be identified, a) a scatter in-between the six pilots and, b) an individual variation when comparing the encounters of one pilot. A valid pilot model for WVE offline simulations has to reproduce these variations. Bank angle [deg] Bank angle Sidestick roll Sidestick roll [ ] FIG 4-: A3 pilot variation of one fixed encounter In order to assess pilot model quality and ensure realistic results, especially for the final probabilistic models, objective performance criteria and respective requirements have to be defined for the validation process: Page

26 ) A/C motion is mainly influenced by large pilot control inputs while small changes in the control deflections have only minor effects. So the focus should be on realistically modelling the maxima in the recorded pilot input signals. ) Also, since the A/C motion is very important for the pilot s hazard assessment and also for an offline severity assessment of the encounter in mass simulations, the comparison of A/C reactions from piloted experiment and from offline simulations with the pilot model should be included in the validation process. 3) To account for the pilot variation, the performance criteria have to consider the natural statistical scatter in pilot inputs and aircraft reactions observed in piloted WVE simulations. The following section describes the identification of natural scatter of pilot inputs and flight parameters that can be observed during simulated WVEs and use the results to derive requirements for pilot model validation Experimental data For validation, requirements are needed that quantify the allowable differences between the reactions of the pilot model and human pilots. As human pilots have a typical scatter in their control actions, it was necessary to identify how wide the variations are. Results from simulated WVEs with geodetically fixed vortex systems are not suited, since small changes in the pilot s initial inputs can result in significant differences in the outcome of an encounter, even if the initial scenarios was identical. For the identification of variation parameters, so called fixed encounters are better suited. In a Fixed Encounter, fixed (pre-recorded) vortex-induced forces and moments act on the aircraft, no matter what the actual flight path is. Hence, fixed encounters guarantee identical vortex-induced disturbances so that only variations in the pilot (or pilot model) action can cause the scatter in aircraft reaction. Vortex-induced forces and moments were recorded in the simulator tests, in which encounter geometries and encounter characteristics were defined such that they result in roll encounters of varying strength and duration, see TAB 4-. The Fixed Encounter technique is described in detail in [7]. In the test sessions the Fixed Encounter cases were repeated randomly without notifying the pilot. Δγ WV Δψ WV Γ b WV WV RL H WV Name Δγ WV Δψ WV Γ b WV WV RL H WV FXE [ ] [ ] [m²/s] [m] [ ] [ft] a b TAB 4-: Fixed encounter cases Vertical encounter angle Lateral encounter angle Vortex circulation Vortex span WV reference line definition: +=Left vortex, -=Right vortex, =Center between vortices Encounter altitude above ground level Due to the limited number of identical encounters per pilot only the overall scatter for all pilots is evaluated in this study. Page 6

27 The parameters defining the encounter geometry are visualised in FIG 4-3. FIG 4-3: Wake vortex encounter geometry The six fixed encounter scenarios listed in TAB 4- were repeated times per pilot. The results allowed assessing the typical scatter per pilot when repeating a test and the scatter in between pilots when flying the same scenario IDENTIFICATION OF STATISTICAL SCATTER IN TIME HISTORIES To quantify the statistical scatter the highest amplitudes of the first noticeable sidestick roll input, bank angle, roll rate, and roll acceleration within the timeframe of encounter are chosen for statistical evaluation. For these parameters the mean values x and its standard deviations s(x) are calculated for each scenario: n (3) x = x i n i= n ( n i= (4) s x) = ( x i x) where x is the mean value and x i is the extreme value. Furthermore the standard deviation of time is computed, see FIG 4-4. For analysis, only the time frame of the actual encounter has been used. It starts with the rolling moment exceeding an absolute value of x Nm (value defined by visual inspection, approximately 6 8 % of the largest recorded wake-induced rolling moment for each FXE case) and stops at the time when the roll rate has dropped below an absolute value of 3 /s for at least seconds. / Page 7

28 Mean Mittelwert values der Extrema of extremes x Distance Abstand of der extremes Extrema Δt(x) Mittelwert Mean value der Zeitpunkte of time t (x) Standardabweichung der Zeitpunkte Standard deviation s(t(x)) of time s(t(x)) Standard Standardabweichung deviation of der extremes Extrema s(x) FIG 4-4: Visualisation of parameters used for statistical scatter analysis The time histories of statistical scatter for all other fixed encounter cases can be found in Appendix A.3. Figures FIG A-4 to FIG A- show the experimental data for the time of the encounter. The identified extreme values for each flight parameter are marked by red (for maxima) and green (for minima) stars inside the vertical boundaries. TAB 4- summarizes pilot s statistical scatter in lateral axis for all flown frozen encounter scenarios. In the last column the mean value of the highest amplitudes of the first noticeable sidestick roll input, bank angle, roll rate, and roll acceleration within the timeframe of encounter, generated by the probabilistic pilot model, are given. The probabilistic pilot model scatter (last column) agrees well with the human pilots scatter (first column) in all encounter cases. The bold marked mean values of the probabilistic lie within the standard deviation of the human pilots scatter. Boxplots 3 of the extreme values for the listed flight parameter are given in the Appendix A.4, FIG A- to FIG A-4. For simplicity the results of the different encounter cases (FXEs) are sorted according to the value of the median. For the sidestick roll order the variation decreases with increasing mean stick deflections. This results from the fact that full sidestick is limited to ±. The A/C parameters are as well limited by the normal law, but these boundaries were usually not reached as opposed to the sidestick limits. In the case of a strong disturbance all pilots may potentially perform a full stick deflection resulting in no scatter of the extremal value. The largest spread can be seen over all fixed encounter cases in the sidestick roll command, as expected. Distribution of bank angle and roll rate are tighter, because of the a/c damping. 3 Boxplots are a special way of visualizing data samples. The box has lines at the at the lower quartile, median (red line), and upper quartile values. The whiskers are lines extending from each end of the box to show the extent of the rest of the data. Their length equals. x interquartile range of the sample. Additionally outliers are marked by + symbols. Page 8