International Journal of Advanced Computer Science, Vol. 2, No. 1, Pp. 33-36, Jan. 2012.

Manuscript
Received:
21,Sep., 2011
Revised:
5, Nov., 2011
Accepted:
22,Dec., 2011
Published:
15,Feb., 2012
Keywords
Fault
diagnosis,
Kernel
Fisher
discriminant
analysis
(KFDA),
Wavelet
transform,
TE process,
Abstract Sampling data of chemical
industry process are generally large size,
intense nonlinear and noisy. To solve these
problems of process state monitoring, an
improved fisher discriminant analysis (FDA)
algorithm for fault diagnose is proposed. This
new algorithm firstly deals with the process
data with wavelet transform for de-noising.
And then maps the de-noised data onto a
high-dimension feature space by a
nonlinearity mapping which is built by the
kernel function. Finally applies linearity FDA
in the feature space. Tennessee Eastman
process(TE process) simulation results show
that the proposed algorithm is effective, and
improves the accuracy rate of fault diagnosis.


1. Introduction
Chemical industry is very important to each countrys
national economy. One important task of fault diagnosis
of chemical industry process is to ensure for productivity
effect and product quality. Multivariable statistical
analysis (MSA) methods [1]

which use the statistic
projection technique to make high-dimensional process
data to a reduced dimension data place has gained great
attentions in last decades [2]. MSA methods, such as
principal component analysis (PCA), independent
component analysis (ICA) and partial least square (PLS)
have wide applications [3].
Although MSA methods have been applied to fault
diagnosis of chemical industry [4], they mainly focus on
fault detection and feature extraction, but not on fault
isolation and diagnosis. Since data of chemical industry
process have characteristics of large size, intense
nonlinearity and noisy. These characteristics make MSA
methods may not perform well for fault diagnosis. Noise
signals in chemical industry process commonly have very
bad effect to diagnosing result. The conventional MSA
methods such as FDA and PCA which are both linear
methods are linear discriminant analysis cant effectively
solve nonlinear signal problem. In order to handle the


This work was supported by Master Tutor Project of Education
Department of Gansu Province (No. 1003ZTC085).
Xiaoqiang Zhaox.q.zhao@lut.cn, Shiyuan Li: College of Electrical
and Information Engineering, Lanzhou University of Technology; Key
Laboratory of Gansu Advanced Control for Industrial Processes
nonlinearity of process data, kernel function FDA (KFDA)
is proposed by Mika [5]. Although the kernel function
makes KFDA could deal with data nonlinearity, but that still
couldnt make the fault recognition rate satisfy. Because
KFDA algorithm assumes that data meet Gaussian
distribution, but actual data of chemical process dont fully
satisfy Gaussian distribution. This causes misjudgment and
misclassification for partial data. At the same time, kernel
mapping makes discriminant vector solve difficult, although
it can map original data to high dimensional space and solve
nonlinear problem. So a modified KFDA algorithm is
proposed in this paper.
2. Wavelet Transformation
Supposed chemical industry process signal s(t) as
( ) ( ) ( ) s t f t e k = + (Equ. 1)
where f(t) is an useful signal, e(k) is a zero-mean Gaussian
white noise signal.
Wavelet transform theory has been well studied by
many researchers and also been used in many fields [6-8].
Briefly, wavelet transform decomposes a time domain
signal into a series of multi-scale signals by a mother
function (t) [9], the basis function. By performing scaling
and shifting operations of the mother wavelet (t), a family
of continuous wavelet transform functions can be created
and denoted:
) ( ) (
2 /
,
a
b t
a t
n
b a

+ - = +

, 0 , , = e a R b a

(Equ. 2)
Where n is the dimension and (t) is the mother wavelet
function, and coefficient a
-n/2
ensures that energy of the
scaled wavelet is constant. Scale parameter a stretches or
shrinks the wavelet so that it covers a larger or smaller area
and shift parameter b shifts the wavelet to different
positions. These features allow the wavelet transform to
extract local information (by varying b) at multiple scales
(by varying a). This space-localization feature of the
wavelet transform is its major difference from the Fourier
transform. Although the Fourier transform can still identify
scales of a random field, it cannot identify different scales at
different locations. The wavelet transform is capable of
providing a better representation of a random field.
The valid support length of wavelet transform functions
,
( )
a b
t +
and scale function is 2N-1. When N is 2, it
becomes Haar wavelet. Fault data can be handled by
wavelet transformation. Its decomposed schematic diagram
is as Fig. 1.
A Modified Kernel Fisher Discriminant Analysis
Algorithm for Fault Diagnosis
Xiaoqiang Zhao & Shiyuan Li
International Journal of Advanced Computer Science, Vol. 2, No. 1, Pp. 33-36, Jan. 2012.
International Journal Publishers Group (IJPG)

34

Fig. 1 decomposed schematic diagram

Where S is original fault data, Ai(i=1,2, ) is
decomposed low-frequency data and Di(i=1,2, ) is
decomposed high-frequency data. According to the actual
project, the useful signal is always stable and in low
frequency section and the noisy signal is always in high
frequency section. The de-noising process is generally as
following: firstly, initial signals are decomposed; then
coefficients are set to zero which are less than the thresholds,
and the greater ones reserved or shrinked; the approximate
coefficients are given; after that, signals can be de-noised by
wavelet reverse transformation.
The wavelet de-noising methods have hard threshold
method and soft threshold method. The hard threshold
method can handle data by below way: absolute values of
wavelet parameters are constant when they are greater than
the threshold values and they are set to zero when they are
less than the threshold values.
T
T
k j
k j k j
k j
<
>
=
,
, ,
,
0
{
e
e e
e (Equ. 3)
The soft threshold method can handle data by below
way: absolute values of wavelet parameters are set to zero
when they are less than the threshold values and they are
constant when they are greater than the threshold values.
Then they are shrinked to trend to zero. Signals by the soft
threshold method are better than the hard threshold method
in chemical process.
T
T T sign
k j
k j k j k j
k j
<
>
=
,
, , ,
,
0
) )( (
{
e
e e e
e (Equ. 4)
3. Kernel Fisher Discriminant
Analysis
The key idea of kernel fisher discriminant analysis
(KFDA) algorithm enables homogeneous data close each
other and heterogeneous data remote each other by
projecting data to high dimensional space by kernel
mapping [10].
Let dimensionality of original signal feature space be n
and number of signal classes be C, total original signal
X={X
1
,X
2
,,X
C
}, No. j (j=1,2,,C) class X
j
contains N
j

signal, namely X
j
={x
j
1
,x
j
2
,,x
j
Nj
}. Here, x
j
1
,x
j
2
,,x
j
Nj
R
n

denote No. N
j
signal(column vectors) of class j. N is total
number of original signal and then
1
C
j
j
N N
=
=

.
By nonlinear mapping , inputs are extended into high
dimensional feature space as follows:
: ( )
m h
x R x F e e

Mapping of signal X
i
is simply noted (X
i
)=
i
as total
mapping signal set and No. j mapping class are given by
1 2
( ) { ( ), ( ), , ( )}
C
X X X X =
1 2
( ) { ( ), ( ), , ( )}
j j j
j C
X X X X =

Mean of mapping signal class ( )
j
X is given by
1
(1/ ) ( )
j
N
j j i
i
m N X
=
=

(Equ. 5)
And global mean of total mapping samples is given by
1 1
(1/ ) ( )
J
N C
j
i
j i
m N X
= =
=

(Equ. 6)
By constructing matrixes within-class scatter matrix S
W
in F and between-class scatter matrix S
B
in F, projection
vector W can be solved and definite class of fault can be
judged. Given kernel matrix K , original data P is mapped
to feature space F :
K
P X , so new data sample can be
obtained in feature space. Matrix S
W
and S
B
are defined as:
1 1
1
( ( ) )( ( ) )
j
N
C
j j T
W i j i j
j i
S X m X m
N
= =
=

(Equ. 7)
1
1
( )( )
C
T
B j j j
j
S N m m m m
N
=
=

(Equ. 8)
Performing FDA in feature space F means maximizing
between-class scatter matrix S
B
and minimizing the
within-class scatter matrix S
W
. The following function is
defined as fisher discriminate criterion:
( )
T
b
T
w
w S w
J w
w S w
= (Equ.

9)

where w is a non-zero column vector.
The aim of kernel Fisher discriminant criterions is to
find optimal projection discriminant vector w. Although
kernel function can solve nonlinear problem of data, but by
projection, data space dimensionality becomes very huge.
Data of chemical process dont satisfy Gaussian distribution,
so partial heterogeneous data are close by projecting.
Separating effects are poor and this causes bad efficiency
and accuracy for fault diagnosis of chemical process.
4. Modified fisher discriminant
analysis algorithm
First, wavelet transformation is used for de-noising.
Then the de-noised nonlinear signal is mapped by the
nonlinear mapping and is extended into high dimensional
feature space. According to maximizing the function, a
group of optimal discrimination vector of projection matrix
Xiaoqiang Zhao et al.: A Modified Kernel Fisher Discriminant Analysis Algorithm for Fault Diagnosis.
International Journal Publishers Group (IJPG)

35
w is found and projection data of matrix w are used to
discriminate. Since dimension of feature space is very high
even infinite, it is impossible to calculate directly.
According to reproducing kernel theory, any solution vector
w of optimal discriminate criterions must be located in the
high-dimension feature space F. Linear, Gaussian and
polynomial kernel functions are often used. RBF kernel
function is used here:
2
( , ) exp( / )
i i
K x x x x =

(Equ. 10)
Discriminant vector w is expressed as:
1
( )
C
i
i
i
w X
=
= =

(Equ. 11)
where
1 2
( , , , )
C C
R = e
and then
T T
b b
w S w K =

(Equ. 12)
T T
w w
w S w K =
(Equ.

13)

where
1
( )( )
c
T i
b i o i o
i
N
K
N
=
=

(Equ. 14)
1 1
1
( )( )
C N
j j T
w i j i j
j i
K X X
N
= =
=

(Equ. 15)
1
( ( , ), , ( , ))
j
i
j j T
i N i
X
k X X k X X = (Equ. 16)
1 1
1 1
1 1
[ ( ) ( )], , ( ) ( )]
j j
N N
T j T j T
j k k
k k j j
X X X X
N N
= =
=

(

Equ. 17)

0 1
1 1
1 1
[ ( ) ( ), , ( ) ( )]
N N
T T T
k N k
k k
X X X X
N N
= =
=

(Equ. 18)

is defined as kernel discrimination vector and is a
N-dimension non-zero vector.
Then the fisher discriminate criterion can be converted
to
( )
T
b
T
w
K
J
K
= (Equ. 19)

Optimal kernel discrimination vector is maximized
and is defined as a maximal feature value corresponding a
feature vector following equation
b w
K K = (Equ. 20)
Then the projection matrix is composed of optimal
discrimination vector
1
( , , )
p d
W W W = , where
1
( )
N
k
i i k i
k
W X
=
= =

, 1, 2, , i d = .
1 2
( , , , ) ( ) ( )
T T
d p
Z W W W X W X = =

1 2 1
( , , , , , , )
l l d
z z z z z
+
= (Equ. 21)
where ( 1, , , 1, , )
T
i i x
z i l l d = = + .
x
is the kernel
signal vector of original signal.

The basic steps of the modified kernel Fisher
discriminant analysis (KFDA) are as following:
Step 1: select DB5 wavelet function, fault data are
decomposed by 5 layers
Step 2: threshold value is obtained by Stein unbiased
likelihood estimate method, data are judged by threshold
value
Step 3: data with threshold value de-noising is converted
by wavelet, reconstruct fault data
Step 4: select RBF kernel function, make de-noising
fault data to kernel mapping and linearity
Step 5: construct matrix S
W
and S
B
for data with
de-noising and linearity by Fisher discriminant
Step 6: optimal discrimination vector is solved, faults
are classified by similar formula
( )( )
T
opt i
opt i
w w
S
w w
=

5. TE Simulation
TE process is a practical chemical process of Eastman
Chemical Corporation [11-12]. The whole process includes
5 main operation units: reactor, condenser, vapour liquid
separator, recycle compressor and product desorber. TE
process is a large-sample-size complex nonlinear system,
and the TE process simulated data could be downloaded
from http:// brahms. scs. uiuc. edu. TE process includes 12
control variables, 41 measurement variables (including 22
continuous measurement variables and 19 components
measure values), all of the process containing Gauss noise.
The flow chart of TE process is as Fig.2. TE process
includes 21 preset faults which indicate fault types of step,
random variation, slow drift, sticking and constant position.
16 faults are known and 5 faults are unknown.

Fig. 2 Tennessee Eastman process flow chart

Input materials of TE process have 4 types: A, B, C and
D. Reaction equations are:
( ) ( ) ( ) (liquid) product1
( ) ( ) ( ) (liquid) product2
A g C g D g G
A g C g E g H
+ +
+ +


( ) ( ) (liquid) by-product1 A g E g F +


3 ( ) 2 (liquid) by-product2 D g F

International Journal of Advanced Computer Science, Vol. 2, No. 1, Pp. 33-36, Jan. 2012.
International Journal Publishers Group (IJPG)

36
Above 4 reactions are irreversible exothermal reactions.
Faults 4,9,11 are typical faults of TE process. Firstly,
initial signals are de-noised by wavelet transform with soft
threshold value method. Fig. 2 shows that noises of initial
signals are wiped off obviously and the denoising effect is
fine. Then RBF kernel function is chose and KFDA is used,
and the kernel coefficient is 450 = .


Fig. 2 Wavelet de-noise comparison chart

TABLE 1
FAULT RECOGNITION ACCURATE RATE OF DIFFERENT METHODS
Methods
Fault 4
(accurate
rate%)
Fault 9
(accurate
rate%)
Fault 11
(accurate
rate%)
KPCA 85.3 82.6 83.4
KFDA 88.7 85.2 85.5
Modified
KFDA
90.5 87.1 88.3

To evaluate the effectiveness of the modified method, it
is compared with KPCA and KFDA. TABLE 1 gives fault
recognition results. Faults 4 and 11 are selected because
they are associated with reactor cooling water inlet
temperature, but they are different types of faults (fault 4 is
step change and fault 11 is random variation). Fault 9 is
associated with random variation of material D feed
temperature. These three faults are good representations of
the TE process. The results show the modified method is
better than KPCA and KFDA.


6. Conclusions
To solve large size, intense nonlinear and noisy of
chemical industry process data, a modified fisher
discriminant analysis(FDA) algorithm based on wavelet
transformation for fault diagnose is proposed in this paper.
TE process simulation results demonstrate that the proposed
algorithm can improve fault recognition accurate rate of
chemical process data and it is more effective than KPCA
and KFDA.
References
[1] A. AlGhazzawi & B. Lennox, "Monitoring a complex
refining process using multivariate statistics," (2008) Control
Engineering Practice, vol. 16, no. 3, pp. 294307.
[2] F. Akbaryan & P.R. Bishnoi, "Fault diagnosis of multivariate
systems using pattern recognition and multisensor data
analysis technique," (2001) Computers and Chemical
Engineering, vol. 25, no. 9-10, pp. 13131339.
[3] H.W. Cho, "Identification of contributing variables using
kernel based discriminant modeling and reconstruction,"
(2007) Expert Systems with Applications, vol. 33, no. 2, pp.
274285.
[4] D. Lieftucht, U. Kruger & G.W. Irwin, "Improved reliability
in diagnosing faults using multivariate statistics." (2006)
Computers and Chemical Engineering, vol. 30, no. 5, pp.
901912.
[5] S. Mika, G. Rtsch, J. Weston, B. Schlkopf & K. Mller,
"Fishers discriminant analysis with kernels," (1999)
Proceedings of IEEE Neural Networks for Singal Processing.
[6] Zhu Shu-xian, Zhang Ren-jie & Zhang Gang, "Selection and
improvement of kernel function used in face recognition,"
(2009) Computer Engineering and Applications, vol.45, no.
8, pp. 243-245.
[7] Jiang Liying, Xie Lei & Wang Shuqing, "Fault Diagnosis for
Batch Processes by Improved Multi-model Fisher
Discriminant Analysis," (2006) Chinese Journal of Chemical
Engineering, vol.14, no.3, pp.343-348.
[8] Xiao Binhe, Yang Yupu & Yang Yahong, "Fault diagnosis
based on variable-weighted kernel Fisher discriminant
analysis," (2008) Chemometrics and Intelligent Laboratory
Systems, 93, pp.27-33
[9] H. Monsef & S. Lotfifard, "Internal fault current
identification based on wavelet transform in power
transformers, " (2008) Electric Power Systems Research, 77,
pp.16371645
[10] J. Rafiee, M.A. Rafiee & P.W. Tse, "Application of mother
wavelet functions for automatic gear and bearing fault
diagnosis," (2010) Expert Systems with Applications, vol. 37,
pp.45684579
[11] J.J. Downs & E.F. Vogel, "A plant-wide industrial process
control problem", (1993) Computers and Chemical
Engineering, vol.17, no. 3, pp.245-255.
[12] G. Lee, C.H. Han & E.S. Yoon, "Multiple-fault diagnosis of
the Tennessee Eastman process based on system
decomposition and dynamic PLS," (2004) Industrial &
Engineering Chemistry Research, vol. 43, no. 25, pp.
80378048.