Escolar Documentos
Profissional Documentos
Cultura Documentos
In this paper we reconsider the existence of worstcase Nash equilibria in a noncooperative multi-player
setting, where the dynamics are described by a linear
differential equation and players preferences are modeled by quadratic cost functions. We consider an openloop information structure. We show that these equilibria can be obtained by determining the open-loop Nash
equilibria of an associated linear quadratic differential
game with an additional initial state constraint. We derive both necessary and sufficient conditions for solvability of the finite planning horizon problem. In particular we demonstrate that unlike in the standard linear
quadratic differential game setting uniqueness of equilibria may fail to hold. A both necessary and sufficient
condition under which there is a unique equilibrium is
provided.
Keywords robust control, noncooperative differential
games, linear optimal control, Riccati equations.
AMS 91A10, 49N70, 93D09.
1. Introduction
In the last decades, there is an increased interest in
studying diverse problems in economics, optimal control theory and engineering using dynamic games. Particularly the framework of linear quadratic differential
games is often used to analyse problems due to its analytic tractability. In environmental economics, marketing and macroeconomic policy coordination, policy coordination problems are frequently modeled as dynamic
games (see, e.g., the books and references in [8], [21],
[12], [29], [26] and [19]). In optimal control theory the
derivation of robust control strategies (in particular the
H control problem) can be approached using the theory of (linear quadratic zero-sum) dynamic games (see
the seminal work of [5]). In the area of military operations, pursuit-evasion problems and, more recently,
Corresponding Author: J.C. Engwerda, Tilburg University, Dept.
of Econometrics and O.R. P.O. Box: 90153, 5000 LE Tilburg, The
Netherlands; e-mail: engwerda@uvt.nl
ISBN: 978-90-367-6321-9
200
MTNS 2014
Groningen, The Netherlands
particular we will show that for this problem also situations occur where there exist an infinite number of
Nash/worst-case equilibria. Necessary and sufficient
conditions are given under which a unique equilibrium
occurs.
The outline of the paper is as follows. In section 2
we show that the problem can be reformulated as a linear quadratic differential game with an additional initial
state constraint. This result is then used in section 3 to
present existence conditions. Furthermore we present
an example where an infinite number of equilibria exist. In the concluding remarks we discuss a number of
issues left for future research. Finally, in the appendix
we present the proof of the main theorem from section
3.
2. An equivalence result
In this paper we consider the problem to find
Nash equilibria for a linear quadratic differential game
that is subject to deterministic noise. With u(t) :=
[uT1 (t) uTN ]T this game is defined by the cost functions
Ji (t f , x0 , u, w) :=
Z tf
0
Qi
VT
i11
where Mi =
T
Vi1N
Vi11
Ri1
Vi22
..
.
T
Vi2N
Vi1N
Vi2N
RiN
(1)
Mi is assumed to be symmetric, Rii and Rwi positive definite (> 0), i N 1 , and x(t) IRn is the solution of the
linear differential equation
N
x(t)
= Ax(t) + Bi ui (t) + Dw(t), x(0) = x0 .
(2)
i=1
The function w(.) W := Lk2 [0,t f ] is an unknown disturbance. The controls ui (t) IRmi considered by player
i, i N, are assumed to be such that u belong to U :=
Lm
2 [0,t f ]. Notice we make no definiteness assumptions
w.r.t. matrix Qi .
We assume that in this uncertain environment every
player wants to minimize his individual cost function Ji
by choosing ui appropriately. Since all players interact,
it is obvious that without making any further specifications the outcome of the game cannot be predicted
as every player will preferably base his action on the
1N
:= {1, ,N}
201
MTNS 2014
Groningen, The Netherlands
i=1
i=1
i=1
x(t)
= Ax(t) + Bi ui (t) + Dw(t), x(0) = x0 .
i=1
Proof:
Assume u is a Nash/worst-case equilibrium. Then,
by definition, (ui , w i (u )) constitutes a saddle-point solution for player i if the other players, j, play uj . Consequently (see e.g. [12, Theorem 3.26]),
Ji (u , w i (u )) =
Ji ((ui , ui ), w(u
i , ui ))
From (4) it follows that this lastmentioned minimum exists and is attained at ui . That is, (u , wi ) satisfies also
condition 2.(b) of Definition 2.1.
)), i N, folFinally, that Ji (u , w(u
)) = J i (u , w(u
lows by a direct comparison of both functions.
(3)
))
Next, consider the minimization of J i (ui , ui , w(u
w.r.t. ui . Some elementary rewriting shows that this
is equivalent with the minimization of Ji :=
Z T
0
)).
min Ji ((ui , ui ), w(u
ui
ui
Ji ((ui , ui ), w(u
))
{[xTi (t), (uTi (t), ui (t))]Mi [xTi (t), (uTi (t), ui (t))]T
w Ti (u )(t)Rwi w i (u )(t)}dt
j6=i
x(t)
= Ax(t) + B1 u1 (t) + B2u2 (t) + Dw(t),
202
(6)
MTNS 2014
Groningen, The Netherlands
Z T
0
1
+ xT (T )QiT x(T ), i = 1, 2,
2
(7)
Here
A
0
S1 S2 SD1
0
A
S1 S2
0
T
Q1
0
A
0
0
T
0
Q
0
A
0
2
Q1
0
0
0
AT
0
Q2
0
0
0
0
SD2
0
0
0
AT
I00
0
P = 0 0 0 and Q = QT
000
QT
T
1 T
ui (t) = R1
ii Bi vi (t) and wi (t) = Rwi D vi+2 (t),
i = 1, 2, respectively.
. (8)
Ki (T ) = QiT , i = 1, 2,
Li (t) = AT Li (t) Li (t)A + Li (t)SDi Li (t) + Qi,
(9)
k 1 (t)
k 2 (t)
H(T )H (T )
I
I
I
I
l1 (t)
l2 (t)
.
(11)
wi (t) =
1
= 2k1 (t) + k12 (t) 1, k1 (1) = 0.9398;
4
= 2k2 (t) + 0.1764k22(t) 1, k2 (1) = 1.497;
1
= 2l1 (t) + l12 (t) + 1, l1 (1) = 0.9398;
4
= 2l2 (t) + 0.1764l22(t) + 1, l2 (1) = 1.497.
Some elementary analysis shows that all four differential equations have a solution on the time interval [0, 1].
Matrix
H(T ), introduced in (11),
equals
0.3771 0.3817
.
Consequently, H + =
0.3771 0.3817
0.6549 0.6629
.
It is easily verified that
0.6549 0.6629
1
1
H(T )H + (T )
=
. So, by Theorem 3.1,
1
1
for every x0 this game has an infinite number of
Li (T ) = QiT , i = 1, 2, (10)
0 0
I 0 .
0 I
Denoting [yT0 (t), vT1 (t), vT2 (t), vT3 (t), vT4 (t)]T := y(t),
with y0 IR2n , and vi IRn , i = 1, , 4, the equilibrium
actions are
(12)
T T
R1
ii Bi E6,i+2 v(t), i = 1, 2,
T T
R1
wi D E6,i+4 v(t), i = 1, 2.
203
MTNS 2014
Groningen, The Netherlands
Nash/worst-case equilibria.
x0
+ (I H + H)q
With
z1 := H +
x
0
1.3179
0.503
x0 +
q, q IR, and
1.3179
0.497
1.91e1.1(t1)
8
0.17e 9 (t1)
0.36e 98 (t1)
4.26e1.1(t1)
0
0
0.45
0
0
0.54
0
0
4.5
5
0
5.4
0
6.3
4.5
5 4.46
5.4 7.1 6.3
0.09e1.1(t1)
8
1.08e 9 (t1)
8 (t1)
x0
1.86e 9
,
q
0.117e1.1(t1)
0
0
u1 (t)
0.254 0.27
0.45 0.45
u (t) 0.30
0.324 0.54 0.54
2
=
w1 (t) 0.254
0.27
0.45 0.45
w2 (t)
0.30
0.324 0.54
0.54
1.1(t1)
1.1(t1)
1.91e
0.09e
8
98 (t1)
1.08e 9 (t1) x0
0.17e
.
8
8
0.36e 9 (t1)
1.86e 9 (t1) q
4.26e1.1(t1)
0.117e1.1(t1)
Appendix
INn
H(T ) = [INn 0Nn2Nn ]eMT diag(QiT ) .
diag(QiT )
4. Concluding remarks
In this paper we reconsidered the finite planning horizon open-loop linear quadratic differential game that is
disrupted by deterministic noise. We derived both necessary and sufficient conditions under which this game
has an open-loop Nash/worst-case equilibrium. We
showed that Nash/worst-case equilibria can be calculated from a virtual linear quadratic differential game
where with every player a nature player is introduced
that tries to maximize the performance criterium w.r.t.
204
MTNS 2014
Groningen, The Netherlands
Z T
0
1
wTi (t)Rwi wi (t)}dt + xT (T )QiT x(T ),
2
and Ji+2 (u1 , u2 , wi ) = Ji (u1 , u2 , wi ), i = 1, 2.
Unfortunately, the initial state of this extended system
can not be arbitrarily chosen. Therefore, we cannot use
directly existing results on open-loop LQ games to derive both necessary and sufficient existence conditions
for a Nash equilibrium. However, we can follow the
lines of the proof for the standard case (see, e.g., proof
of [12, Theorem 7.1]) to obtain these conditions.
Suppose that (ui (.), wi (.)) is a Nash equilibrium. Then,
by the maximum principle, the Hamiltonian Hi =
y(t)
= M e y(t), with Py(0) + Qy(T) = [xT0 0 0 0 0]T .
(14)
e
A
Se SD
Se
0
Qe AT
0
0
0
e
e
Here M = Q
0
0 ;
0
A
T
0
0
0
A
0
T
0
0
0
0
A
I0000
0
0 0 0 0
00000
QT I 0 0 0
0 I 0 0
P = 0 0 0 0 0 and Q =
QT
.
00000
0
0 0 I 0
00000
0
0 0 0 I
1 T
(x Qi x + uT1 Ri1 u1 + uT2 Ri2 u2 wTi (t)Rwi wi (t)) +
2
iT (Ax + B1 u1 + B2 u2 + D1 w1 + D2 w2 ),
is minimized by player i w.r.t. ui , i = 1, 2, and Hi+2 =
1
(xT Qi x uT1 Ri1 u1 uT2 Ri2 u2 + wTi (t)Rwi wi (t)) +
2
T
i+2
(Ax + B1 u1 + B2 u2 + D1 w1 + D2 w2 ),
is minimized by player i + 2 w.r.t. wi , i = 1, 2. This
yields the necessary conditions
T
Some elementary rewriting shows that the above twopoint boundary value problem (14) has a solution for
every initial state x0 if and only if
ui (t) = R1
ii Bi i (t) and wi (t) = Rwi Di i+2 (t),
or, equivalently,
(PeM
i = 1, 2, and
x (t) =
e
e
e
e
Ax(t) S1 1 (t) S2 2 (t) SD1 3 (t) SD2 4 (t);
Si Si
e
with
, SD1 :=
:=
=
Here
S
S
i
i
0 0
SD1 0
e
. In other words,
and SD2 :=
0 SD2
0 0
if the problem has an open-loop Nash equilibrium then,
with y(t)
:= [xT (t), 1T (t), , 4T (t)]T , the differential
equation
e
e
e
e
A
S1 S2 SD1 SD2
Q AT
0
0
0
1
T
= Q2
y(t)
0
A
0
0 y(t)
Q1
0
0
A
0
T
Q2
0
0
0
A
(13)
xT (0)
[xT0
xT0 ].
e
Si
eT
(15)
W1 W2 W3 W4 W5
x0
QT
0
I
0
0
0
QT
z = 0 (16)
0
I
0
0
0
0
0
0
I
0
0
0
0
0
I
0
has a solution for every x0 . An elementary analysis
shows that a solution exists for every x0 if and only if
(W1 + W2 QT W3 QT )z1 = x0
(17)
205
MTNS 2014
Groningen, The Netherlands
or [I 0 0 0 0]eM T [I QT QT 0 0]T z1 = x0 has a solution for every x0 . Using the structure of matrix M e
it is now easily verified that this equation has a solution if and only if with matrix H(T ) := [I 0 0]eMT Q T
the equation H(T ) = x0 has a solution for every x0 . This
last problem is equivalent with the problem
under which
I
conditions the matrix equation H(T )X =
has a
I
solution X. From this one directly obtains condition
(11) (see e.g. [1, Exercise 10.49]).
If (11) holds, all solutions of equation (17) are
T
wi (t) = R1
wi D (Li (t)xi (t) + mi+2 (t)) i = 1, 2.
z =
[zT1
Z T
0
M(tT )
e
Q T z1
e
e
.
y(t) = eM t y0 = eM (tT ) z =
0
0
T
e
K 1 (t) = A K 1 (t) K 1 (t)A + K 1 (t)S1 K 1 (t) Q1 ,
(t)
i1
1 T
T
librium controls: ui (t) = Rii [Bi Bi ]
=
i2 (t)
T
R1 BT E T eM(tT ) Q T z1 and wi (t) = R1
wi Di
ii i 6,i+2
i+21 (t)
T T
M(tT ) Q
T z1 , i = 1, 2.
= R1
wi D E6,i+4 e
i+22 (t)
part By assumption the Riccati differential equations (9,10) have a solution on [0, T ]. Since
H(T ) satisfies (11), it is clear from the part of
the proof that the two-point boundary value problem
(12) has for every x0 a consistent initial value y0 yielding a unique solution for the boundary value problem. Assume y0 is a with x0 consistent initial value
of the boundary value problem. Denote the solution
y(t) of this two-point boundary value problem (12) by
[xT1 (t), xT2 (t), 1 (t), , 4 (t)]T , where the dimension
of xi and i is n. Now consider
(18)
i=1
K1 (t) 0
It is easily verified that K 1 (t) :=
solves
0
0
the Riccati differential equation, where
K1 (t) solves
(9).
m 1 (t)
, where
Using this in (18) shows that m1 (t) =
0
m 1 (t) solves the differential equation
m 1 (t) = (K1 (t)S1 AT )m 1 (t)
(19)
T
Consequently, u1 (t) = R1
1 (t)),
11 B1 (K1 (t)x1 (t) + m
where x1 (t) is the through this optimal control implied
solution of the differential equation, with x1 (0) = x0 ,
x 1 (t) = (A S1K1 )x1 (t) S1m 1 (t) + B2u2 (t) + Dw1 (t).
(20)
Substitution of u2 (t) and w1 (t) into (19,20) shows, that
the variables m 1 and x satisfy
206
MTNS 2014
Groningen, The Netherlands
It is easily verified that a solution of this set of differential equations is given by m 1 (t) = m1 (t) and x1 (t) =
x1 (t). Since its solution is unique this implies that
u1 (t) = u1 (t). Or, stated differently,
References
[1] Abadir, K., Magnus, J. (2005). Matrix Algebra. Cambridge University Press, Cambridge.
[2] Abou-Kandil, H., Freiling, G., Jank, G., Necessary and sufficient conditions for constant solutions
of coupled Riccati equations in Nash games, Systems&Control Letters 21 (1993) 295-306.
[3] Altman, E., Basar, T., Multiuser rate-based flow control, IEEE Trans. Commun. 46 (1998) 940-949.
[4] Azevedo Perdicoulis, T.P., Jank, G. (2011). Existence
and uniqueness of disturbed open-loop Nash equilibria for affine-quadratic differential games. Advances
in Dynamic Games, Eds. Breton, M., and Szajowski,
K., 25-41.
[5] Basar, T., Bernhard, P., H -Optimal Control and Related Minimax Design Problems, Birkhauser, Boston,
1995.
[6] Basar, T., Olsder, G.J., Dynamic Noncooperative
Game Theory, SIAM, Philadelphia, 1999.
[7] Basar, T., Zhu, Q., Prices of anarchy, information, and
cooperation in differential games, Dynamic Games
and Applications 1 (2011) 50-73.
[8] Dockner, E., Jrgensen, S., Long, N.V., Sorger, G.,
Differential Games in Economics and Management
Science, Cambridge University Press, Cambridge,
2000.
[9] Eisele, T., Nonexistence and nonuniqueness of openloop equilibria in linear-quadratic differential games,
Journal of Optimization Theory and Applications 37
(1982) 443-468.
[10] Engwerda, J.C., On the open-loop Nash equilibrium
in LQ-games, Journal of Economic Dynamics and
Control 22 (1998) 729-762.
[11] Engwerda, J.C., Computational aspects of the openloop Nash equilibrium in LQ-games, Journal of Economic Dynamics and Control 22 (1998) 1487-1506.
[12] Engwerda, J.C., LQ Dynamic Optimization and Differential Games, John Wiley & Sons, Chichester,
2005.
[13] Engwerda, J.C., Uniqueness conditions for the affine
open-loop linear quadratic differential game, Automatica 44 (2008) 504-511.
207