STATISTICALAND ADAPTIVE APPROACH FOR VERIFICATION OF A NEURAL-BASED FLIGHT CONTROL SYSTEM

Ronald L. Broderick, GraduafeSchool o f Computer and Information Science, Nova Southeastern University, Ff.Lauderdale, FL

Abstract
This paper presents a combined statistical and adaptive approach for the verification of an adaptive, online leaming, sigma-pi neural network that is used for aircraft damage adaptive flight control. Adaptive flight control systems must have the ability to sense its environment, process flight dynamics, and execute control actions. This project was completed for a class in Complex Adaptive Systems at Nova Southeastem University. Verification of neural-based damage adaptive flight control system is currently an urgent and significant research and engineering topic since these systems are being looked upon as a new approach for aircraft survivability, for both commercial and military applications. The most significant shortcoming of the prior and current approaches to verifying adaptive neural networks is the application of linear approaches to a non-linear problem. Advances in computational power and neural network techniques for estimating aerodynamic stability and control derivatives provide opportunity for real-time adaptive control. New verification techniques are needed that substantially increases confidence in the use of these neural network systems in life, safety, and mission critical systems.

certification is mandatory. This certification requirement imposes stringent verification requisites. This standard requires that avionic software be certified from the process and product viewpoint. Any attempt at the verification of a neural network system must address substantially different issues compared to traditional software. The most significant issue is trustworthiness. Neural network systems are able to address very complex problems, but they raise very difficult verification issues. New verification techniques are needed that substantially increases confidence in the use of these neural network systems in life, safety, and mission critical systems. This paper will focus on the application of statistical confidence interval analysis coupled with the use of complex adaptive system theory to determine the rules of the adaptive sigma-pi neural network.

Background
For the past several years, one of the most interesting applications of neural network systems is damage adaptive flight control. For the damage adaptive flight control application, artificial neural networks have been extensively researched at NASA Ames Research Center because of their powerful ability in approximating non-linear dynamical functions. Called Damage Adaptive Flight Control [l], the objective of this project was to demonstrate a flight control concept that can identify aircraft stability and control characteristics using neural networks, and then to utilize this information to optimize aircraft performance in nominal, off-nominal and failure conditions [2]. Current research being performed by Georgia Institute of Technology [3], Boeing Phantom Works [4], and NASA Dryden Flight Research Center is based on a more adaptive approach that consists of two pre-trained neural networks and a single online leaming neural network. As shown in Figure 1, taken from [4], the Pre-trained Neural Network (PNN) is a table-lookup scheme based on

Introduction
The purpose of this paper is to propose a combined statistical and adaptive approach for the verification of an adaptive, online learning, sigmapi neural network that is being used by NASA for aircraft damage adaptive flight control applications. For avionic systems certification,the Radio Technical Commission for Aeronautics (RTCA) industry standard DO-178B titled Software Considerations in Airbome Systems and Equipment Certificationis recognized and adhered to by the Federal Aviation Administration as the means of demonstrating compliance with the Federal Aviation Regulations. For flight control systems,

0-7803-8539-W04/$20.00 0 2004 IEEE

6.E.1-1

stable aircraft conditions. The Real Time Parameter Identification(PID) neural network uses an equation error technique employing a Fourier Transform Regression algorithm. Both are pretrained, non-adaptive, neural networks. Pre-trained, non-adaptive means that once trained, these neural networks are static during flight operations. The Online Leaming Neural Network (OLNN) is fully adaptive sigma-pi neural network, which is based on Kohonen-like adaptation which changes the values of the network weights during flight operations to adapt to changing flight conditions and to meet desired system performance goals. During off-nominal flight conditions, differences between the outputs of the PNN and PID are accounted for by the OLNN. These differences, called the Stability and Control Derivative (SCD) error, stored by the OLNN. The role of the OLNN is to provide a better estimate of the stored values for new flight conditions and compute control gains needed to make the aircraft stable. The neural-based damage adaptive flight controller has been flight demonstrated on the F-15 ACTIVE aircraft at NASA Dryden Flight Research Center. A recent flight demonstration included a flight envelope of 15000 to 35000 f t .altitude, speeds of subsonic and supersonic, Ig and 4g turns and a 360-degree roll [4]. One of the major problems that this system faces is certification. Current FAA certification relies heavily on testing. Certification based on testing is problematic for adaptive neural networks since they leam after they are deployed and the system tested in the lab is not the system that is being flown in the field.

Prior Research
The major theme of prior research has been that neural networks are created using a very different paradigm fromconventional software, therefore a new framework must be developed [5, 61. Experimental tools that evaluate neural networks can be classified into two groups: 1) tools that assist in the decisions for the development of the neural network system and 2) tools that evaluate the performance of the neural network. The goal is to decrease both system output error and system output error variability. The usefulness of neural networks has motivated the implementation of many systems. Characteristicssuch as fault and noise tolerance, and learning capability have attracted many engineers. Unfortunately,the uncertainty of neural network output has not been addressed in many implementations. In safetycritical systems, output error variability must be evaluated.

A decade ago, in the paper titled A foundation for neural network verification and validation G. E. Peterson, [7] proposed to enhance the current verification and validation approach. This paper also introduced the idea of statistically evaluating true and apparent error. In addition, Peterson addressed the subject of a neural networks boundary of acceptablebehavior and used a confidence statistic to express the performance within the boundary.
The Taguchi method is another analysis procedure, like the design of experiments that has been used successfully in improving manufacturing products and processes. It has been used successfullyto generally evaluate neural networks [SI. As a conclusion Peterson indicates that since relatively little is known about the capabilities and characteristics of neural networks, controlled experiments under varying circumstances must be run since the outcome of a single experiment can be misleading. Fault tolerance refers to the fact that no matter how well a system is designed and tested, faults will remain in the delivered system. Neural networks must be designed to be fault tolerant if their application requires high reliability. Aircraft are subjected to extreme reliability requirements [9]. Such high reliability requirements force the development of redundant systems since the

I
Figure 1. Intelligent Flight Control Architecture

6.E.1-2

hardware cannot live up to such high standards. Triple redundancy with voting is proposed in the paper titled Complete and Partial Fault Tolerance of Feedforward Neural Nets [lo]. In the paper titled Towards Developing Verifiable Neural Network Controller, [ 5 ] , rises the question of completeness of training data and flight envelope. The generation of training data cannot rely solely on recorded flight data. This data would not contain the environment the aircraft would experience in a damaged condition. Also, what is the performance of the neural network in the vicinity of the boarders of the flight envelope? In a more recent paper, Towards V&V of Neural Network Based Controllers, [121, authors propose modification of current verification methods and attention to numerical aspect of neural network verification. In the recent paper titled Certifying Adaptive Flight Control Software [13], the authors take a fault tolerance approach including both sensor failure and actuator failure. With the goal of establishingthe foundations for software certification of adaptive flight control systems, the authors approach is to evaluate the limitations of various combined verification techniques, including formalization of requirements specifications,their formal verification, and techniques for proving the convergence and stability properties of neural networks.

altitude (h) and aircraft pitch angle (a), 20 processing elements in a single hidden layer, and 34 outputs corresponding to each aerodynamic stability derivative. This became the Pre-trained Neural Network (PNN), which is non-adaptive and is shown in Figure 1. To increase the robustness and the ability to handle off-nominal conditions, two additional networks were developed for the Intelligent Flight Control System (IFCS), The Parameter Identification (PID) and the Online Leaming Neural Network. The PID uses a Fourier Transfer Regression Algorithm to estimate stability derivatives.The difference the PNN derivative and the PID derivative is stored and the job of the sigma-pi OLNN is to used these stored values to provide a better estimate of stability derivatives for the current flight condition. The new stability derivatives are use to transform the pilot input to the computed aircraft control surface commands in the aircraft closedloop control system. As shown in Figure 2, the process of using state information to govem the control inputs is known as closing the loop, and the resulting system as a closed-loop control or feedback control.
Confrol

Vehicle

System Actual State (yJ

The Approach
The approach of this paper is to use a combined statistical confidence interval analysis and an adaptive analysis to verify the online, leaming sigma-pi neural network of the Intelligent Flight Control System (IFCS). The following statistical confidence interval analysis will provide a framework to determine, for a given set of flight condition inputs, whether the IFCS will produce a suitable range of command errors to develop a high level of confidence ( 95% or greater). The adaptive analysis will attempt to determine the rules that govem the weights of the sigma-pi neural network using complex adaptive systems analysis. The original neural network concept developed by NASA was to implement a pre-trained multilayer perceptron neural network to model aircraft aerodynamic stability derivatives. The resulting network consisted of three inputs, Mach (M),

Figure 2. A Simple Closed-Loop Control System The aircraft controller compares the desired control commands (yd)to the computed control commands (yJ and determines the aircraft controller error (e) to he used in the next cycle of aircraft control.

Statistical Analysis
The time line of aircraft activity as a function of command error (e) is shown in Figure 3 below.

6.E.1-3

Neuralnetwork

flight envelope, as shown in Figure 4, a specific M (t), h (t) and a(t) would be input to the neural network.

Recovery
Error (er)

I
Time (t) Figure 3. The Point of Damage It is important to note that the point of neural network engagement is some time later than the point of damage (the point in which damage occurs to the aircraft). This is to account for the reaction time of the pilot to command the engagement of the neural network. The point x(t) is the point in which the neural network is engaged with a command error substantially greater than the normal command error of the aircraft. Certification Flight Envelope For certification purposes, the certification flight envelope would be a limited set of the neural networks inputs: Mach (M), altitude (h) and aircraft pitch angle (a). Thus

Figure 4. A Point in the Flight Envelope.

A computational study is required to determine the increment of each parameter to determine the instance of each computer run and the scaling of each input value. For example, the increment of altitude could be 1000 ft, Mach could he 0.25 Mach and angle of attack could be 6 degrees. The increments do not need to be uniform and larger increments could be used to limit the time required for simulations.
Statistical Mean Error/Confidence Interval The simulation would also model control surface damage, i.e., an aileron not working. A new set of 34 aerodynamic stability derivatives (C) would be computed and used to determine the computed control surface deflections. An error (e) would be computed as the difference between the computed command (y.) using the new stability derivatives and the aircraft response vs. the desired command (yd) and aircraft response modeled using the original stability derivates. Letting C be the set of thirty four (34) coefficients that are the output of the neural network - c = I (yd, y e ) I = r (Yd) 1 where, -as stated above, yd is the desired command and yc is the correct computed command for the given desired command input. Let r be the parent function to be modeled by the neural network. L is chosen to be a training set, where L c C and is expressed as follows:

,0 5 t 5 T x(t) = M t ) , h(t), where T is the observation time interval. For a realistic flight envelope, environmental conditions must also be modeled to account for pressure, density, temperature, and speed-of-sound variations with altitude. The test space of all x(t) is infinite and is far to rich to be covered by a reasonable number of test cases. To cover a reasonable set of certification tests at each point in the flight envelope, the certification flight envelope set would be further reduced to a finite interval and to eliminate impossible combinations of inputs. The neural network certification database should range f r o m 0.1 < M < 1.3, at a minimum recovery altitude (h=5000 ft.) to 0.3 < M < 2.0 at 50,000 ft altitude (h). The aircraft pitch angle would range from -1 80 to +I80 degrees. For each point in the database or

6.E.1-4

from 20.6 to 22.6 that states that the neural network would produce a command error between 20.6 and 22.6,95% ofthe time.

Adaptive Anabsis
It is believed that the greatest deficit to verifying neural networks has been the attempt to apply linear methods to non-linear systems. In the previous section, a method to develop statistical information about the behavior of the sigma-pi neural network was proposed. In the following section, adaptive analysis is proposed to understand the behavior of the neural network weighs is convergent for verification purposes and to determine neural network rules. Using adaptive analysis, the neural network weights can be evaluated using a complex adaptive approach. The OLNN is a fully adaptive sigma-pi network with one hidden layer. Sigma-pi is a product unit neural network that was introduced by Durbin and Rumelhart [14]. A normal neural network used a unit summation function: Y=

cwqxi
i=l

"

Sigma-pi neural networks use a unit product function:

where X is the activation value of the unit and W is the weight. Figure 5, taken from [14], shows a sigma-pi network Confidence

Level
lOO(1 -a) 90% 95% 99%

a 0.10 0.05 0.01

d2

w t
1.645 1.960 2.575

0.05 0.025 0.005

Using the formula above a claim with a probability of 95% that an interval contains p if a random sample of size n = 100 is taken from a population having a standard deviation ( 0 ) of 5.1. If the sample mean is 21.6 then the 95% confidence interval for p is computed by 21.6 1.96 * 5.1/ f i . The confidence interval for p would be

Figure 5. Sigma-Pi Net with One Hidden Layer where V are the first to second layer interconnection weights, W are the second to thiid interconnection

6.E.1-5

weights, and a is the hidden layer activation function. An example of a hidden layer activation function is the sigmoidal function:
a(z) = 1 / ( I+

e-")

be recorded. Using chaos theory it could be determined whether the weights naturally converge. Using this approach, rules that govem the neural network weights could be determined. Neural Network Weights In the study of complex adaptive systems [15] a concept of interest is emergence. Emergence is the study of complex adaptive systems that are govemed by elements that follow low-level rules but perform sophisticated behavior. Schools of fish, flocks of birds, bees, ant and termite colonies, and herds of land animals are subjects of emergent study since they all exhibit complex, adaptive behavior [16].

The damage adaptive controller uses a backpropagation controller to change the interconnectionweights to optimize the control response to compensate for damage and failure conditions of the aircraft [3]. For specific cases of M (t), h (t), a(t) and damage condition, how do the weights change as a function of time? Using a network fitness distribution approach, it is proposed that a small number of the weights change significantly,thus the control of the outcome of the neural network would be dominated by a few weights as shown in Figure 6.
j
t

Vsrance of Weights

As shown in Figure 7, the neuron is the processing unit. Each neuron is characterized by an activity level, representing the state of polarization of the neuron, an output value -representing the firing rate of the neuron, a set of input connections -representing synapse on the cell and its dendrite, a bias value -representing an intemal resting level of the neuron, and a set of output connections representing a neuron's axonal projections. Each aspect is represented mathematically by real numbers. Each connection has an associated weight (synaptic strength) that determines the effect of the incoming input on the activation level of the unit. The weights may be positive (excitatoxy) or negative (inhibitory) Input lines are product units that yield an activation value for neuron j at time t.

XI

X*

Figure 6. Neural Network Weight Variance Since, it is proposed that networks weights are characterizedby the power law, there are certain weights in the neural network that dominate. A command error timeline would be simulated which would characterize the control error before the damage condition, at the time of damage, at the time of the neural network engagement, and after the pilot has regained control of the aircraft. As the system changes, the point x(t) will tmce a trajectory. At each point during the simulation, the value of each weight of the neural network would
Transfer Function
Y,=l/(l+C'~

.x,

Figure 7. Backpropagation Learning Neural network learning can be defined as simply the adjustments necessary to the set of weights that allow the network to calculate the desired output. The most popular form of neural network learning is backpropagation, as shown in Figure 7. In sigma-pi architecture, the weights of

6 . E . 1 6

the network are adjusted as a function of error between the actual output and the desired output values. In the error correction learning procedure, if the actual output equals the desired output, the weights of the neural network are not changed. However, if the output differs, change must be made to some of the weights. The problem is to determine which weight contributed to the error. The total performance of the system, given a set of training pairs, is defmed as:

population with the control parameter and a suitable delta time. The logistics equation is suitable in depicting order (convergence to a constant value), complexity (constant osculation between two values) and chaotic activity (persistent instability), as shown in Figure 8.

E=

c
P. I

( d,, - YhP )*

where i indexes the output units, p indexes the is input/output pairs, d,,pis the desired output Y , . ~ the actual output and E is the total error for the system. The goal is to change the weights to minimize this function. That is, change the weights of the system in proportion to the derivative of the error with respect to the weights. The change in w,, is thus proportional to:

aEav,

Figure 8. Chaos Theory Phases

The logistics equation than plots a parabola + the ~ ordinate and with x, on the abscissa and x ~ on the value of x between 0 and 1. The constant k govems the steepness and height of the parabola above the abscissa. The peak chosen is k = 4,with the range of k from 0 to 4.0. The characteristics of the logistic equation with these limits are very interesting. For values of k from 0 to <3.0 the time series trajectory always converges to a constant value or steady state. This condition is true for any initial condition. In chaos theory terms, the forward trajectory is a single attractor, which means that for any value of q the resulting set of points will move towards a constant. An attractor is a dynamical systems set of stable conditions. It is the equilibrium state for the system, such that if the system is started from another state it will evolve until it arrives at the attractor. Each weight will be analyzed to determine if it has a k value from 0 to < 3.0 and is convergent. Chaos theory contents that there are particular universal phases that dynamical systems transition from regular motion to irregular motion. For k from 3.0 to 3.7, the dynamics of the time series system is unstable, transitioning from stable to chaotic. In this phase, systems will continually oscillate which

ay, awl,
According to [17], complex dynamic behavior
has the following characteristics: a) independence: a

large number of relatively independent components, b) dynamic: each component responds to its fellow component, c) adaptiveness: the system conforms to new situations to bring about some realignment, d) self-organization: order forms, e) local rules: govem each component, and f ) hierarchical nature of structure. Developing a non-linear, dynamical algorithm for a system determination could be based on chaos theory. The initial effort in the determination of a new system algorithm based on chaos theory is to identify at least one control parameter [18]. A mathematical concept that characterizes the non-linear, dynamical nature of a control parameter can be expressed by the logistic equation:
Xt+l

= ht(1-xt)

where xt+l is the updated value of the control parameter during a time period. This equation, originally developed to model long-term population dynamics, provides a one-dimensional feedback system with discrete time intervals. To use the logistics equation, would require the substitution of

6.E.1-7

represents complexity. Fork < 3.7 to 4.0 the time series is chaotic. In the chaotic phase, the system will continually oscillate chaotically which represent an unstable system.
Thus, if a weight is analyzed using the logistic equation to determine k as:

k=xt+i /(xt(l-xi)) First consider 0 < k < 3. Whatever value of k is used the time series using that value will converge to constant value. As shown in Figure 9, if k = 2.43 and the control parameter xl is 0.40, then X,+I = kx,(l-x,) yields 2.43(0.4)(1-0.4) = 0.5832. U s i n g the calculated value for X, ,then XW = 0.59076. As shown in Table 2 below, this series converges to a steady state value of 0.58847, which in chaos theory represents stability. Trials show that all iteration at constant k fork < 3.0 decay to a steady state, regardless of xg. Therefore, if the value of k calculated the control parameter is less than 3.0, the weight can determine as convergent or stable.
!
Ordered Welght Behavior: k*A3

Complex Weight Behavbr: W . 3

E, 0.4
i 0 . 3

I!

Figure 9. Steady State Convergent Behavior Similarly, Table 2 demonstrate the case when k = 3.30 and Figure IO shows that persistent oscillation occurs. Figure 10. Persistent Oscillation Behavior Finally, Table 2 demonstrates the case where
k = 3.95 and Figure 1 1 shows a profile of chaotic

oscillation occurring.

6.E.1-8

References
[l] Jorgensen, Charles, 1997. Direct adaptive aircraft control using neural networks. Technical Report TM-47136, NASA.

[2] Totah, Joseph J, 1996. Simulation Evaluation of a Neural-Based Flight Controller, AIAA 963503.

[3] Calise, A. J., S. Lee, M. Sharma, 2000, Development of a Reconfigumble Flight Control Law for the X-36 Tailless Fighter Aircraft, Proceedings of the AIAA Guidance, Navigation, and Control Conference, August 14-17,2000.

Figure 11. Chaotic Behavior

Conclusion
Sigma-pi neural networks are complex adaptive systems. The theories of complexity model the non-linear, dynamical nature of complex adaptive systems. The approach of this paper is to use a combined statistical confidence interval analysis and an adaptive analysis to verify the online, learning sigma-pi neural network of the Intelligent Flight Control System (IFCS). The statistical confidence interval analysis provides a framework to determine, for a given set of flight condition inputs, whether the IFCS will produce a suitable range of command errors to develop a high level of confidence (95% or greater). The adaptive analysis provides a method to determine the rules that govem the weights of the sigma-pi neural network using complex adaptive systems analysis. The logistic equation is an example of a mathematical characterization of complexity and can be used to characterize the change of the sigmapi neural network weights. The logistic equation or similar equations can be used to define the rules in which the weights of the neural network change and can be use to determine their convergent nature. This will lead to a strong verification of the adaptive, online learning sigma-pi neural network that is currently being used for aircraft damage adaptive flight control.

[4] Umes, James. Sr., R. Davidson, S. Jacobson, 2001, A Damage Adaptive Flight Control System Using Neural Network Technology, Proceeding of the American Control Conference, June 25-27, 2001. [5] Wen, W., J. Callahan, & M. Napolitano,l996, Towards Developing Verifiable Neural Network Controller, ICTAI 96 Workshop on Artificial Intelligence for Aeronautics and Space, Toulouse, France. [6] Broderick, R. L., 1997, Knowledge-Based Aircraft Automation, NASA-CR-205078. [7] Peterson, G. E., 1993, A Foundation for Neural Network Verification and Validation, in Science of Artificial Neural Networks 11, Dennis W.Ruck, Editor, Proc. SPIE 1966, pp. 196-207.

[SI Peterson, G. E., D. C. St. Clair, S. R. Aylward, & W. E. Bond, 1995, Using Taguchis Method of Experimental Design to Control Errors in Layered Perceptrons, IEEE Transactions on Neural Networks, 6(4), pp. 949-961.

[9]Rysdyk, R. T., &A. J. C a l k , 1998, Fault Tolerant Flight Control via Adaptive Neural Network Augmentation, AIAA 98-4483, August 1998.
[lo] Phatak, D. S., & I. Koren, 1995, Complete and Partial Fault Tolerance of Feedfonvard Neural Nets, IEEE Transactions on Neural Networks, 6(2), pp. 446-456. [ 1I] Schuman, J. & S. Nelson, 2002, Towards V&V of Neural Network Based Controllers, Proceedings of the First ACM Workshop on Self-healing Systems.

6 . E . 1 9

[U] Cortellessa, V., B. Cukic, D. Del Gobbo, A. Mili, M. Napolitano, M. Shereshevsky, & H. Sandhu, 2003, Certifying Adaptive Flight Control Sohare, Department of Mechanical and Aerospace Engineering, West Virginia University.
[13] Durbin, R. & Rumelhart, D., 1989. A Computationally Powerful and Biologically Plausible Extension to Backpropagation Networks, Neural Computation, Vol. 1: 133-142.
[14] Calise, A. J. & R. T. Rysdyk 1998. Nonlinear Adaptive Flight Control using Neural Networks, IEEE Control Systems Magazine, 21(6): 14-26.

[ 151 Cannady, James, 2002, DCIS 790 Complex Adaptive Systems. (Class Lecture Viewgraphs), Nova Southeastem University, Graduate School of Information Sciences Web site, URL: httu://scis.nova. edd-cannady

[16] Holland, J. H, 1998, Emergence: From Chaos to Order, Perseus Books, Cambridge, MA. [17] Williams, G. P, 1997, Chaos Theory Tamed, Washington DC, Joseph Henry Press. [18] O t t ,E, 2002, Chaos in Dynamical Systems, New York, Cambridge University Press.

6.E.1-10