European Journal of Operational Research 124 (2000) 224±234

www.elsevier.com/locate/dsw

A neural network representation of linear programming

a,b,* c
Hong-Xing Li , Xu Li Da
a
Department of Mathematics, Beijing Normal University, Beijing 100875, China
b
Department of Computer Science, Tsinghua University, Beijing 100084, China
c
National Key Laboratory of Manufacturing Systems Engineering, Xian Jiaotong University, Xian 710049, China
Received 1 January 1999; accepted 1 July 1999

Abstract

This paper demonstrates the ¯exibility of neural networks for modeling and solving diverse mathematical problems.
Advantages of using neural networks to solve problems include clear visualization, powerful computation and easy to
be made into hardware. In this paper, the well-known exclusive OR (XOR) problem is ®rst introduced. Then, two
examples are discussed in order to show how to use neural networks to represent di€erent problems. One problem is
Taylor series expansion and the other is Weierstrass's ®rst approximation theorem. The neural representation of linear
programming and the neural representation of fuzzy linear programming are also discussed. Ó 2000 Elsevier Science
B.V. All rights reserved.

Keywords: Linear programming; Neural networks; Mathematical neural networks; Functional-link networks; Learning
algorithms; Fuzzy linear programming

1. Introduction Caudill and Butler (1990) have studied perceptron

Neural network representation problems have a
great relevance in many ®elds, such as those relate
to linear programming and fuzzy linear program-
ming (Li and Li, 1999). A feedforward neural
network with several layers is called a perceptron,
proposed by Rosenblatt (1990,1991,1992), Hertz et
al. (1991), Kosko (1992), and Caudill and Butler
(1990) in 1960s. Minsky and Papert (1990,1991,
1992), Hertz et al. (1991), Kosko (1992), and
*
Corresponding author. Address: Department of Mathemat-
ics, Beijing Normal University, Beijing 100875, China.
in detail and ®nd out that a perceptron with one
layer is of low ability of representation and
learning. A typical example is that such simple
perceptron can not solve the exclusive OR (XOR)
problem because XOR function is a logic function
that accepts two binary inputs and generates a
single binary output. A couple of ideas are sug-
gested here to solve the ability of (one layer) per-
ceptron problem:
idea 1: increase number of the hidden layers,
idea 2: add some higher order terms (neurons).
Clearly both ideas 1 and 2 tend to raise non-
linear grade of the neural network, although idea 1
is more e€ective. Li et al. (1999) suggest that it can

0377-2217/00/$ - see front matter Ó 2000 Elsevier Science B.V. All rights reserved.
PII: S 0 3 7 7 - 2 2 1 7 ( 9 9 ) 0 0 3 7 6 - 8
 H.-X. Li, X.L. Da / European Journal of Operational Research 124 (2000) 224±234
make a network more complicated so that the
learning speed of a network moves strikingly.
However, Pao (1989) prefers idea 2. It is interest-
ing to learn that both ideas represent the same
concept.
Example 1. So-called `functional-link' means that
signal transmission between neurons is designed as
function transformation in a neural network so
that the grade of non-linearity is heightened. We
know that the network shown in Fig. 1 can not
solve XOR problem. But if a higher order term
x1
x2 is added in the network, then network
shown in Fig. 2 can deal with XOR problem.
Pao (1989) suggests, by using functional-link,
many problems can be solved by one-layer net-
work. For instance, in Fig. 2, to add the higher
order term x1
x2 means to add a `new' input. But
the input is actually not new. It is only one forming
by x1 and x2 , which can be regarded as interaction
of x1 and x2 . Then the output of the network is as
follows:
y ˆ u…x1 w1 ‡ x2 w2 ‡ x1 x2 w3 ÿ h†; …
Fig. 1. A simple one layer perceptron.
Fig. 2. Add a higher order term x1
x2 .
225
which can simulate XOR function. In fact, taking
w1 ˆ w2 ˆ 1; w3 ˆ 2, and h ˆ 1, Eq. (1) becomes
the following:
y ˆ u…x1 ‡ x2 ‡ 2x1 x2 ÿ 1†: …2†
If we do 45° rotation of axes and the new coor-
dinates are denoted by x01 and x02 , then the curve
…x1 ‡ x2 ‡ 2x1 x2 ˆ 1† can be expressed as follows:
ÿ 0 p
2 ÿ 0
2
x1 ÿ … 2=2† x
ÿ ÿp
2 ‡ ÿp2
2 ˆ 1; …3†
2=2 2=2
which is a hyperbola. It can classify the two pat-
terns (as inputs) into two classes, i.e., 0- and 1-class
(as outputs) (see Fig. 3).
Example 2. Above example solves XOR problem
by just using idea 2. Now we consider the idea 1 to
solve XOR problem.
We design a 2-layer perceptron shown in Fig. 4,
1† where the activation function of neuron fi is
ui …i ˆ 1; 2; 3†, and the activation function of
neuron h is u. We de®ne u1 and u2 to be identity
functions (i.e., u1 …u† ˆ u, u2 …u† ˆ u†, u3 to be a
hyperbolic function (i.e., u3 …u; v† ˆ uv), and u to
be step-up function (i.e., u…u† ˆ 1, u P 0;
u…u† ˆ 0, u < 0).
This comes back to Eq. (1). In other words, the
network shown in Fig. 2 is equivalent to the one
shown in Fig. 4.
Fig. 3. The hyperbola for solving XOR problem.
226 H.-X. Li, X.L. Da / European Journal of Operational Research 124 (2000) 224±234
Fig. 4. A 2-layer perceptron for solving XOR problem.
Fig. 5. The simpli®ed version of Fig. 4.
Remark 1. Clearly the neurons f1 and f2 in Ex-
ample 2 only play the role of `passageways', since
u1 and u2 are identity functions. So we can remove
the two `stations' f1 and f2 (see Fig. 5).
The activation functions of the neurons f and h
are denoted by w and u, and taken for hyperbolic
function and step-up function. So we also have
y ˆ u…w1 x1 ‡ w2 x2 ‡ w3 x1 x2 ÿ h†:
Remark 2. In Example 1, we can add some higher
order terms (see Fig. 6). Then the output of net-
work is the following:
y ˆ u…w1 x1 ‡ w2 x2 ‡ w3 x1 x2 ‡ w4 x21 ‡ w5 x22 ÿ h†:
…4†
Chosen adequately w1 ; . . . ; w5 and h, it can also
simulate XOR function. Of course, the one-layer
network in Fig. 6 can be accomplished by a 2-layer
network shown in Fig. 7, where the activation
function of the neurons f1 , f2 , f3 and h is respec-
tively u1 ; u2 ; u3 and u; u2 is taken for a hyperbolic
function, u1 and u3 are all taken for quadratic
function (i.e., u1 …u† ˆ u2 ; u2 …u† ˆ u2 , and u as a
step-up function. Then the output of the network
is as follows:
Fig. 6. Add more higher terms.
Fig. 7. A network for accomplishing the network Fig. 6.
y ˆ u…w1 u1 …x1 † ‡ w2 u2 …x1 ; x2 † ‡ w3 u3 …x2 †
‡ w4 x1 ‡ w5 x2 ÿ h†
ˆ u…w1 x21 ‡ w2 x1 x2 ‡ w3 x22 ‡ w4 x1 ‡ w5 x2 ÿ h†;
which is the same as Eq. (4). This means that a
one-layer network with higher order terms can be
always accomplished by a multi-layer network,
vice versa.
We would like to say that the idea of a one-
layer network problem is easier than a multi-layer
network problem seems not to be true. On the
contrary, multi-layer networks can resolve the
complexity of one-layer networks so that they are
clearer and can be easily treated. As for the
learning algorithms of multi-layer networks, they
may not be more complicated than the ones of
single layer. Generally speaking, layer number of a
network and complexity of the network are not of
direct ratio.
2. Mathematical essence of functional-link neural
networks
We begin with some examples. Pao (1989) gave
an interesting example of a functional-link net-
 H.-X. Li, X.L. Da / European Journal of Operational Research 124 (2000) 224±234
Fig. 8. A function expansion network.
work called a function expansion network. As
shown in Fig. 8, the output of the network is as
follows:
y ˆ u…w1 x ‡ w2 sin px ‡ w3 cos px ‡ w4 sin 2px
‡ w5 cos 2px ‡ w6 sin 4px ÿ h†: …
In fact, the network is formed by adding other
®ve inputs into a one-input network, where x is
regarded as a generating input and the other ®ve
inputs as some function transformations (for in-
stance, x ! sin px). Thus, the problem is not so
much interesting. If we let the activation function
of the neuron h be an identity function,
w1 ˆ 0; w00 :ˆ ÿh; w0i :ˆ wi‡1 …i ˆ 1; . . . ; 5†, then
Eq. (5) becomes the following:
y ˆ w00 ‡ w01 sin px ‡ w02 cos px ‡ w03 sin px
‡ w04 cos 2px ‡ w05 sin 4px: …
Clearly, it just is a triangular interpolation func-
tion.
Example 3. Fig. 9 gives a functional-link (also
function expansion) network and its output is as
follows:
y ˆ u…w1 x ‡ w2 x2 ‡ w3 x3 ‡


‡ wn xn ÿ h†: …
Fig. 9. Another function expansion network.
227
Fig. 10. A network for representation of a general functional-
link networks.
Now we let wi ˆ 1=i! …i ˆ 1; 2; n†; h ˆ ÿ1 and
5† u be an identity function, then according to Ma-
claurin's expansion (Malik, 1984) of ex , we have
y ˆ u…w1 x ‡ w2 x2 ‡ w3 x3 ‡


‡ wn xn ÿ h†
x x2 x3 xn
ˆ1‡ ‡ ‡ ‡


‡  ex :
1! 2! 3! n!
This is to say, a network can express a function,
vice versa, a function can be expressed by a net-
work.
Example 4. We consider a general form of func-
tional-link networks (see Fig. 10, an example of
6†
one-output is given; as for multi-output, the pre-
sentation is similar). The output of the network is
the following:
y ˆ u…w1 x1 ‡


‡ wn xn ‡ wn‡1 g1 …x1 ; . . . ; xn †
‡


‡ wn‡m gm …x1 ; . . . ; xn † ÿ h†: …8†
Clearly, the functional-link networks discussed
7† above are all special cases of this network.
Although the output function has some non-
linear terms gj …x1 ; . . . ; xn †, we can make them lin-
ear by changing some variables. Let
zi ˆ xi …i ˆ 1; . . . ; n†; and
zj ˆ gj …x1 ; . . . ; xn †; j ˆ n ‡ 1; . . . ; n ‡ m;
The above output function becomes
y ˆ u…w1 z1 ‡ w2 z2 ‡


‡ wn‡m zn‡m ÿ h†: …9†
228 H.-X. Li, X.L. Da / European Journal of Operational Research 124 (2000) 224±234

This becomes a linear expression again.

For a given group of training samples
 k
……x1 ; . . . ; xkn †; yk †jk ˆ 1; 2; . . . ; p ; …10†

assuming u to be an identity function and taking

…k†
notice of zi …i ˆ 1; . . . ; n; k ˆ 1; . . . ; p† and
…k† …k†
zj ˆ gj …x1 ; . . . ; x…k†
n †; j ˆ n ‡ 1; . . . ; n ‡ m;
k ˆ 1; . . . ; p;

we get a system of linear equations regarding

wi …i ˆ 1; 2; . . . ; n ‡ m† as unknowns Fig. 12. A network without `through trains'.

ak1 w1 ‡ ak2 w2 ‡


‡ akq wq ˆ yk ; expressed by the activation functions of the neu-

k ˆ 1; 2; . . . ; p; …
…k†
where akj ˆ zj …j ˆ 1; 2; . . . ; n ‡ m† and q ˆ n ‡ m.
The problems similar to Eq. (11) are discussed in
detail by Li and Yen (1995b).
Remark 3. The functional-link network shown in
Fig. 10 can also be expressed by a 2-layer network
as in Fig. 11.
The activation functions uj of the neurons fj are
taken as the following:
uj …x1 ; . . . ; xn † :ˆ gj …x1 ; . . . ; xn †; j ˆ 1; 2; . . . ; m:
11† rons in the network. Therefore, functional-link
networks have hardly any new things in essence.
Remark 4. In Fig. 11, the input signals x1 ; . . . ; xn
¯ow into the neuron h by the form of `through
trains'. Of course we can set up `relay stations' to
avoid these `through trains' (see Fig. 12).
Let the activation functions of the neurous
f1 ; . . . ; fm be uj …x1 ; . . . ; xn † …j ˆ 1; . . . ; n ‡ m†,
where
ui …x1 ; . . . ; xn † :ˆ xi ; i ˆ 1; . . . ; n:

Then the output of the network is as follows Then the output of the network is the following:
!
Xn X
m
!
yˆu wi xi ‡ wn‡j gj …x1 ; . . . ; xn † ÿ h ; X
n‡m
iˆ1 jˆ1 yˆu wj uj …x1 ; . . . ; xn † ÿ h
iˆ1
which is Eq. (8). This is to say the function that !
X
n X
n‡m
forms in a functional-link network can be perfectly ˆu wi xi ‡ wj uj …x1 ; . . . ; xn † ÿ h ;
iˆ1 jˆn‡1

which is the same as Eq. (8).

3. As visualization means of some mathematical

methods

The above sections lead us to an idea that ar-

ti®cial neural networks can be used for represent-
ing some mathematical methods, mathematical
forms, or mathematical structures. In other words,
Fig. 11. A 2-layer network expression of Fig. 10. arti®cial neural networks can be regarded as a kind
 H.-X. Li, X.L. Da / European Journal of Operational Research 124 (2000) 224±234 229

of visualization means of mathematics. We are

going to use some examples to demonstrate the
above idea.

Example 5. The neural network representation of

Taylor expansion (Malik, 1984).

Given a function f(x) satisfying the condition:

f …x†; f 0 …x†; f 00 …x†; . . . ; f …n† …x† are continuous in
closed interval [a, b] and f …n‡1† …x† is existential in
open interval (a, b). We design a 2-layer forward
functional-link neural network with one-input Fig. 14. A Taylor expansion network designed by using acti-
one-output (see Fig. 13). vation function.
The activation functions of the neurons in the
network are all taken for identity functions, and
the threshold values of the neurons at the ®rst Especially, when a ˆ 0, it is just a Maclaurin's
layer are all taken for zero and the threshold value expansion neglected the remainder term.
of the neuron at the second layer is taken for
ÿf …a†. If we put w1 ˆ f 0 …a†; w2 ˆ f 00 …a†=2!; . . . ; Remark 5. Li and Yen (1995b) suggested a general
wn ˆ f …n† …a†=n!, then the output of the network is activation function (absorbed threshold values).
the following: According to their suggestion, a Taylor expansion
can be expressed as Fig. 14.
X
n The activation functions of the neurons in the
i
yˆ wi …x ÿ a† ÿ …ÿf …a†† network are de®ned as u0 …x†  f …a†; u1 …x† ˆ x ÿ a;
iˆ1 u2 …x† ˆ …x ÿ a†2 ; . . . ; un …x† ˆ …x ÿ a†n ; un‡1 …u† ˆ u;
f 00 …a† 2 the weight values are taken as w0 ˆ 1; w1 ˆ
ˆ f …a† ‡ f 0 …a†…x ÿ a† ‡ …x ÿ a†
2! f 0 …a†; w2 ˆ f 00 …a†=2!; . . . ; wn ˆ f …n† …a†=n!. Then the
f …n† …a† n
output of the network is as follows:
‡


‡ …x ÿ a†  f …x†; …12†
n! X
n
yˆ wi ui …x† ˆ f …a† ‡ f 0 …a†…x ÿ a†
iˆ0
which is clearly a Taylor expansion neglected the
f 00 …a† f …n† …a†
remainder term, and equal approximately to f(x). ‡ …x ÿ a†2 ‡


‡ …x ÿ a†n :
2! n

Example 6. The neural network representation of

Weierstrass's ®rst approximation theorem (Malik,
1984) (see Fig. 15).

The activation functions of the neurons in the

Fig. 13. A network representating Taylor expansion.
network are de®ned as u1 …x† ˆ cos x; u1 …x† ˆ
cos x2x; . . . ; un …x† ˆ cos nx; w1 …x† ˆ sin x; w2 …x† ˆ
sin 2x; . . . ; wn …x† ˆ sin nx; h1 ; h2 ; g are taken for
identity functions. The threshold values of the
neurons at the second layer are zero, and the
threshold value at the third layer is ÿa0 =2. And
ak ; bk …k ˆ 1; 2; . . . ; n† are all weight values. Then
the output of the network is the following:
230 H.-X. Li, X.L. Da / European Journal of Operational Research 124 (2000) 224±234

Fig. 15. A network representing WeierstrassÕs approximation

theorem.
Fig. 16. A network representing linear programming.
Xn X n  a 
Tn …x† ˆ ak uk …x† ‡ bk wk …x† ÿ ÿ
0 !
2 X
n
kˆ1 kˆ1
y0 ˆ u0 c j x j ÿ b0 ;
a0 X n
jˆ1
‡ …ak cos kx ‡ bk sin kx†; …13† ! …15†
2 kˆ1 X
n
yi ˆ ui aij xj ÿ bi ; i ˆ 1; . . . ; m
jˆ1
which is just the trigonometric polynomial in
Weierstrass's ®rst approximation theorem. That is
Problem 1: Let all weight values to be known
to say, for any continuous function f …x† with pe-
and the threshold values b1 ; b2 ; . . . ; bm to be given.
riod 2p, and for any a positive real number e > 0,
We want to ®nd the threshold value b0 such that
there exists a network shown as Fig. 15 such that
the neuron f0 is inhibitory when the neurons
jf …x† ÿ Tn …x†j < e holds uniformly on the whole
f1 ; f2 ; . . . ; fm are all excited.
number axis.
From Eqs. (14) and (15), it is easy to realize that
the problem is actually a linear programming
problem as the following:
4. Neural network representation of linear program- X
n
ming max cj xj ;
jˆ1
A one-layer forward neural network is given in
Fig. 16, where the activation functions ui of the
neurons fi are all taken for step-up functions, s:t:
X n
…16†
8 aij xj P bi ; i ˆ 1; 2; . . . ; m;
< 1; u P 0 jˆ1
ui …u† ˆ i ˆ 0; 1; . . . ; m; …14†
:
0; u < 0 xj P 0; j ˆ 1; 2; . . . ; n:

and their threshold values are b0 ; b1 ; . . . ; bm ; the If the set of feasible solutions
Pn is not empty, we
weight values in the network are cj and aij should take b0 > max jˆ1 cj xj that is just the
…i ˆ 1; 2; . . . ; m; j ˆ 1; 2; . . . ; n†. For given input threshold value that we need.
…x1 ; x2 ; . . . ; xn †, the components of the output Problem 2: Under the same condition as in
…y0 ; y1 ; . . . ; ym † are as follows: Problem 1, we want to ®nd the threshold value b0
 H.-X. Li, X.L. Da / European Journal of Operational Research 124 (2000) 224±234 231

such that the neuron f0 is inhibitory when the xj P 0; j ˆ 1; 2; . . . ; n:

neurons f1 ; f2 ; . . . ; fm are inhibitory. Clearly the
problem is equivalent to the following linear pro- Similar to Problem 2, the activation functions
gramming problem: ui …i ˆ 1; 2; . . . ; m† should be taken for Eq. (18).
Problem 5 (Inverse problem of linear pro-
X
n
gramming): As an example, we only consider the
max cj xj ;
jˆ1
inverse problem of Example 3 (other cases are
similar to it). Given the threshold values
s:t: bi …i ˆ 0; 1; . . . ; m† and a group of training samples
X n
…17† n o
…k† …k† T
aij xj 6 bi ; i ˆ 1; 2; . . . ; m; Xk ˆ …x1 ; x2 ; . . . ; x…k†
n † jk ˆ 1; 2; . . . ; p …21†
jˆ1

satisfying, for every k,

xj P 0; j ˆ 1; 2; . . . ; n:
X
n X
n
Nevertheless the activation functions ui (see Eq. min cj xj ˆ
…k†
c j x j ˆ b0 ;
(14)) have a little di€erence except u0 , jˆ1 jˆ1
8
< 1; u > 0 s:t:
ui …u† ˆ i ˆ 1; 2; . . . ; m; …18†
: X n
…22†
0; u 6 0; …k†
aij xj P bi ; i ˆ 1; 2; . . . ; m;
jˆ1
Problem 3: Under the same condition, we want
to ®nd b0 such that f0 is excited when f1 ; f2 ; . . . ; fm …k†
are excited. The problem can be expressed as the xj P 0; j ˆ 1; 2; . . . ; n;
following linear programming problem: we want to ®nd the weight values cj and aij …i ˆ
Xn
1; 2; . . . ; m; j ˆ 1; 2; . . . ; n†.
min cj xj In fact, if we use `surplus weight values'
jˆ1
ain‡1 P 0 …i ˆ 1; 2; . . . ; m† to force that
s:t:
X
n
…k†
X n
…19† aij xj ÿ ain‡1 ˆ bi ; i ˆ 1; 2; . . . ; m; …23†
aij xj P bi ; i ˆ 1; 2; . . . ; m; jˆ1
jˆ1
then Eq. (22) becomes a `normal form':
xj P 0; j ˆ 1; 2; . . . ; n:
X
n X
n
…k†
If the set of feasible solutions min cj xj ˆ c j x j ˆ b0 ;
Pn is not empty, then we
should take b0 6 min jˆ1 cj xj that is just the jˆ1 jˆ1

threshold value that we need.

Problem 4: Under the same condition, we want s:t:
to ®nd b0 such that f0 is excited when f1 ; f2 ; . . . ; fm X n
…k† …24†
are inhibitory. The problem is corresponding to aij xj ÿ ain‡1 ˆ bi ; i ˆ 1; 2; . . . ; m;
jˆ1
the following linear programming problem:
X
n …k†
xj P 0; j ˆ 1; 2; . . . ; n;
min cj xj ;
jˆ1
which is the inverse problem of linear program-
s:t: ming and can be solved by means of learning al-
X n gorithms of neural networks.
…20†
aij xj < bi ; i ˆ 1; 2; . . . ; m; Problem 6 (Neural network representation of
jˆ1 linear programming with multiple objects): In
232 H.-X. Li, X.L. Da / European Journal of Operational Research 124 (2000) 224±234

Problem 7: Find threshold values ds …s ˆ 1;

2; . . . ; t† such that g1 ; . . . ; gt are inhibitory when
f1 ; . . . ; fm are inhibitory.
Problem 8: Find threshold ds …s ˆ 1; 2; . . . ; t†
such that g1 ; . . . ; gt are excited when f1 ; . . . ; fm are
excited.
Problem 9: Find threshold ds …s ˆ 1; 2; . . . ; t†
such that g1 ; . . . ; gt are excited when f1 ; . . . ; fm are
inhibitory. Moreover, there are some `mixed'
cases. The following example is just an example of
them, and the other cases are similar to it.
Problem 10: Find ds …s ˆ 1; 2; . . . ; t† such that
g1 ; . . . ; gq are excited but fr‡1 ; . . . ; fm inhibitory
…1 6 r < m†.
Problem 11 (Inverse problem of linear pro-
gramming with multiple objects): As an example,
we only consider the case in Problem 6 (other cases
are similar to it). Given threshold values ds
…s ˆ 1; 2; . . . ; t† and bi …i ˆ 1; 2; . . . ; m† and a
Fig. 17. A network representing linear programming with
several objectives. group of training samples like expression 21 sat-
isfying, for every k …k ˆ 1; 2; . . . ; p†,

Fig. 17, activation functions of all neurons are X

n X
n
…k†
max csj xj ˆ csj xj ˆ ds ; s ˆ 1; 2; . . . ; t;
taken as step-up functions. Given weight values csj jˆ1 jˆ1
and aij …s ˆ 1; 2; . . . ; t; i ˆ 1; 2; . . . ; m; j ˆ 1; 2; . . . ; n†,
and threshold values bi …i ˆ 1; 2; . . . ; m†, we want to s:t:
®nd threshold values ds …s ˆ 1; 2; . . . ; t† such that X n
…k† …26†
the neurons g1 ; g2 ; . . . ; gt are inhibitory when the aij xj P bi ; i ˆ 1; 2; . . . ; m;
jˆ1
neurons f1 ; f2 ; . . . ; fm are excited.
The problem can be clearly expressed by the …k†
xj P 0; j ˆ 1; 2; . . . ; n;
following linear programming with several objects
we want to ®nd weight values csj and aij …s ˆ 1;
X
n
max csj xj ; s ˆ 1; 2; . . . ; t 2; . . . ; t; i ˆ 1; 2; . . . ; m; j ˆ 1; 2; . . . ; n†.
jˆ1

5. Neural network representation of fuzzy linear

s:t: programming
X n
…25†
aij xj P bi ; i ˆ 1; 2; . . . ; m; Based on the network shown as Fig. 16, taking
jˆ1 Problem 1 for an example (other cases are similar
to it), we consider fuzzy linear programming in-
troduced by Li (1986, 1995a, 1997). As a matter of
xj P 0; j ˆ 1; 2; . . . ; n: fact, if the activation functions u1 ; u2 ; . . . ; um of
the neurons f1 ; f2 ; . . . ; fm are taken as membership
If the set of feasible solutions
Pn is not empty, we functions, i.e., ui : R ! ‰0; 1Š; i ˆ 1; 2; . . . ; m (for
should take ds > max jˆ1 csj xj …s ˆ 1; 2; . . . ; t† convenience, ui themselves can be regarded as
that we need. Under the same condition as Prob- fuzzy sets, i.e., ui 2 F …R†; i ˆ 1; 2; . . . ; m†, the
lem 6, similar to Problems 7±10. network shown as Fig. 16 is a fuzzy neural
 H.-X. Li, X.L. Da / European Journal of Operational Research 124 (2000) 224±234 233

network (Li, 1990), where R is the real number

®eld. Seeing from the linear programming corre-
sponding to the network, the m fuzzy activation
functions ui …i ˆ 1; 2; . . . ; m† de®ne m `elastic
constraints' (i.e., m fuzzy constraints). Actually,
we can consider the problem by such way that,
when u1 ; . . . ; um are `crisp' functions (i.e., step-up
functions), the m constraints are just m inequalities
expressed as the following:
!
X n
ui aij xj ÿ bi Fig. 18. The polygonal form of u …u†.
jˆ1
8 n
> P
>
> 1; aij xj P bi Example 8. If we think of Eq. (29) to be compli-
>
>
< jˆ1 cated, then ui …u† can be taken as the following
ˆ i ˆ 1; 2; . . . ; m; simple forms (see Fig. 18):
>
>
>
> P
n
>
: 0; aij xj < bi …27†
jˆ1
ui …u†
8
which are characteristic functions toP re¯ect law of < 1;
> uP0
excluded middle with respect to n ˆ u‡e i
; ÿei 6 u < 0; …ei > 0; i ˆ 1; 2; . . . ; m†:
jˆ1 aij xj P bi > e
: i
…i ˆ 1; 2; . . . ; m†. 0; u < ÿei
When u1 ; . . . ; um are fuzzy sets, the law of ex-
cluded middle re¯ected by Eq. (27) no longer
holds, thus ` P ' becomes `fuzzy' denoted by `>'. 6. Conclusions
So Eq. (16) should be written as follows:
In this paper, the ¯exibility of neural network
X
n
for modeling and solving diverse mathematical
max cj xj ;
jˆ1
problems has been demonstrated. The advantages
of neural network such as clear visualization,
s:t: powerful computation and easy to be made into
X n hardware have been shown. The well-known XOR
…k† …28†
aij xj > bi ; i ˆ 1; 2; . . . ; m; problem is introduced ®rst, followed by Taylor
jˆ1 series expansion and Weierstrass's ®rst approxi-
mation theorem. The neural representation of
xj P 0; j ˆ 1; 2; . . . ; n; linear programming and fuzzy linear programming
is also discussed.
which is a kind of fuzzy linear programming with
one object.
References
Example 7. In Eq. (28), the activation functions
f1 ; . . . ; fm can be taken as the following: Caudill, M., Butler, C., 1990. Naurally Intelligent Systems. The
MIT Press, Cambridge, MA.
ui …u† ˆ …1 ‡ exp … ÿ ai u ‡ bi ††ÿ1 ; Hertz, J., Krogh, A., Palmer, R.G., 1991. Introduction to the
Theory of Neural Computation. Addison-Wesley, New
i ˆ 1; 2; . . . ; m; …29†
York.
Kosko, B., 1992. Neural Networks and Fuzzy Systems.
where the parameters ai and bi can control the Prentice-Hall, Englewood Cli€s, NJ.
slope and parallel translation of the curve of Li, H.X., 1986. Multifactorial fuzzy sets and multifactorial
ui …u†; i ˆ 1; 2; . . . ; m. degree of nearness. Fuzzy Sets and Systems 19, 291±297.
234 H.-X. Li, X.L. Da / European Journal of Operational Research 124 (2000) 224±234
Li, H.X., 1990. Multifactorial functions in fuzzy sets theory.
Fuzzy Sets and Systems 35, 69±84.
Li, H.X., 1995a. To see the success of fuzzy logic from
mathematical essence of fuzzy control: On The paradoxical
success of fuzzy logic. Fuzzy Systems and Mathematics 9
(4), 1±14.
Li, H.X., 1997. Fuzzy Information Processing and Fuzzy
Computers. Science Press, New York, Beijing.
Li, H.X., Li, L., 1999. Representing diverse mathematical
problems using neural networks in hybrid intelligent
systems. Expert Systems 16, 262±272.
Li, H.X., Yen, V.C., 1995b. Fuzzy Sets and Fuzzy Decision-
Making. CRC Press, Boca Raton, FL.
Li, H.X., Chen, C.L.P., Huang, H.P., 1999. Fuzzy Neural
Intelligent Systems: Mathematical Foundation and the
Applications in Engineering. CRC press, Boca Raton,
FL.
Malik, S.C., 1984. Mathematical Analysis. Wiley, New York.
Minsky, M., Papert, S., 1990. In: Naturally Intelligent Systems.
Caudill, M., Butler, C. (Eds.), The MIT Press, Cambridge,
MA.
Minsky, M., Papert, S., 1991. In: Introduction to the Theory of
Neural Computation. Hertz, J., Krogh, A., Palmer, R.G.
(Eds.), Addison-Wesley, New York.
Minsky, M., Papert, S., 1992. In: Neural Networks and Fuzzy
Systems. Kosko, B. (Ed.), Prentice-Hall Inc., Englewood
Cli€s, NJ.
Pao, Y.H., 1989. Adaptive Pattern Recognition and Neural
Networks. Addison-Wesley, New York.
Rosenblatt, 1990. In: Naturally Intelligent Systems. Caudill,
M., Butler, C. (Eds.), The MIT Press, Cambridge, MA.
Rosenblatt, 1991. In: Introduction to the Theory of Neural
Computation. Hertz, J., Krogh, A., Palmer, R.G. (Eds.),
Addison-Wesley, New York.
Rosenblatt, 1992. In: Neural Networks and Fuzzy Systems.
Kosko, B. (Ed.), Prentice-Hall Inc., Englewood Cli€s, NJ.