Proceedings of the
1
47th IEEE Conference on Decision and Control
Cancun, Mexico, Dec. 9-11, 2008
Networked Control Systems with Packet Delays and Losses
C. L. Robinson
Dept. of Industrial and Enterprise Systems Engineering,
and the Coordinated Science Lab,
University of Illinois at Urbana-Champaign
clrobnsn@uiuc.edu
Abstract We investigate the effect of packet delays
and packet drops on networked control systems. First we
consider the problem of where to locate a controller or state
estimator in a network, and show that under a Long Packets
Assumption (LPA) it is optimal to collocate it with the
actuator. We then show that under the LPA, stabilizability
is only determined by the packet drop probability and
not the packet delay probabilities. We also consider a sub-
optimal state estimator without the LPA, based on inverting
submatrices of the observability Krylov sequence.
I. I
Contention for the medium, channel fading, and
interference in networks, lead to packet delays and
losses. Even though observations may be taken at
regular instants, their arrivals after passage through the
network may be random since collision detection and
avoidance algorithms use random backoffs and delays,
or they may even be dropped due to the losses in the
wireless medium or collisions. Hence, we address the
issue of random packet delays as well as packet drops
in networked control systems.
We study an LQG system by employing a Long
Packets Assumption (LPA) [15], which allows packets
to be arbitrarily long, and in particular to contain a
history of all past observations. The LPA can be realized
even without long packets by having an encoder at the
sensor and a decoder at the actuator, as shown in [5].
We first address the question of where the control logic
should be placed within the network subject to random
packet delays and losses, and show that it is optimal to
collocate the controller with the actuator. This extends
an earlier result for the case of packet drops only [14],
[15].
Then we address the question of when such a system
is stabilizable. We show that the condition
1
pDrop , (1)
max (A)2
where max (A) is magnitude of the eigenvalue of A with
the largest magnitude is necessary, and also sufficient
when the inequality is strict. This result shows that
stabilizability under the LPA depends only on the loss
probability and not the delay probabilities. Thus the
condition for stabilizability is essentially the same as
in the packet drop only case examined in [15], [18], [6],
978-1-4244-3124-3/08/$25.00 2008 IEEE
Authorized licensed use limited to: The Ohio State University. Downloaded on July 19,2010 at 21:28:36 UTC from IEEE Xplore. Restrictions apply.
ThB06.
P. R. Kumar
Dept. of Electrical and Computer Engineering,
and the Coordinated Science Lab,
University of Illinois at Urbana-Champaign
prkumar@uiuc.edu
[19]. It is interesting that under LPA the delay distribu-
tion only affects system performance and not stabiliz-
ability. This is illustrated via simulations in Section IV.
The result has implications not only for a system with
long packets, but also, as mentioned above, for the
encoder-decoder scheme in [5] which realizes the LPA
but without long packets. The result on stabilizability
not depending on delays but only on drop probability
is analogous to that of [16], [17], where the estimation
problem without the LPA is considered, and a similar
independence of estimator stability on delays is shown.
The stabilizability condition is different in that case
since the LPA does not hold.
We next analyze window based schemes without an
LPA, as has been considered in [16], [17]. We consider a
family of suboptimal schemes and obtain upper bounds
on packet drop probability, similar to (1), that are
sufficient for their stability.
Useful references for networked control include [1],
[3]. Packet delays and drops result in generally in-
tractable non-classical information patterns [21], [12].
The effect of random sampling times on optimal con-
troller design [2], state estimation [20], [13] and overall
system performance [8] have been considered. Packet
delays are considered in [11], with delay assumed to be
less than one sampling period. Another approach is to
focus on eliminating the effect of random delay. In [10],
a buffer is maintained at the receiving end of the channel
for randomly delayed packets, which releases them at
regular intervals. A similar actuator buffer has actually
been deployed in [4]. The condition (1) is studied vis-
a-vis packet drops in [7], [18], [19].
II. C P
We begin by determining where to locate the con-
troller; see Fig. 1. We show that placing it on a path
with best delay characteristics is optimal.
Let q() be the probability mass function for delay
on anyP link j. Packets are dropped with probability
1 t=0 q(t). Delays of packets on links are assumed
iid. Let hg be the path of a packet from a node h to
node g. The path delay is the sum of the link delays
on the path, with probability distribution denoted by
Fhg . We say that distribution F1 dominates F2 if F1 (D)
4602
47th IEEE CDC, Cancun, Mexico, Dec. 9-11, 2008 ThB06.1

Sensor Plant Actuator transmitted observation, and under the LPA is comprised
of all state observations {yk : 0 k j}. The sensor takes
C and transmits observations at each sampling instant k.
Control actions are implemented every s sample
S i
instants, and are held constant during the intermediate
interval. These are called actuation instants. Transmitted
observations are subject to delay. Since control actions
A
are only implemented at actuation instants, a delayed
Fig. 1. The controller is at node C, and SA is a minimal path. measurement can arrive at any time, but can only be
used at the next actuation instant.
We now describe the probabilistic model for the end-
F2 (D)D. A path between nodes h and g with the best to-end packet delay; see Figure 3. A packet transmitted
delay characteristics, i.e., one whose delay distribution at time j will contain z j under the LPA and is subject
dominates those of all other paths between h and g, to delay. There is no guarantee that packets will arrive
if one exists, is referred to as a shortest path and is in order. Delays of individual packets are i.i.d.
denoted by hg ; see Figure 1. pi :=Prob(Packet transmitted at k arrives P at k + i).
Lemma 2.1: Suppose the same packet is sent along pDrop
:=Prob(Packet
 is dropped)
 = 1 P i=0 pi .
two chains of nodes C1 and C2 , as in Fig. 2. Under pi := P Packet delay i = pDrop + j=i p j .
the LPA, the information at each node in C1 with more I(k) :=Set of observations known to controller at k
nodes is dominated by that at a corresponding node in is the information set.
C2 . (k) := age of the information set = ki, if the latest
packet to have arrived before actuation instant
C1
S A C k is zi . Note that I(k) = {y j : j k (k)}.
C2
(k) P((k) = ). Note that limk (k)
S B exists . Denote {0 , 1 , . . .}.
Fig. 2. Two chains of nodes with source node denoted by node S. With (a) below representing packet k arriving by
k, and (b) below packets {k + 1, k + 2, . . . , k} that
Proof: Node A on C1 at the same hop distance did not arrive by time k, see Figure 3, we can write:
from the source S as B (see Fig. 2, has FSA = FSB ). The
(a) (b)
information arrival processes at these two nodes can

z }| { z }| {

be stochastically coupled. Since packets will be further (k) =
(1 p+1 ) p1 p2 p3 . . . p for k ,

delayed between nodes A and C in C1 , the information
0 for > k.
at node B stochastically dominates that at C.
Corollary 2.2: Under the LPA there is an optimal con-
troller placement that is on path SA .
Theorem 2.3: Placing the controller at the actuator is
optimal. Actuation instant
Proof: Consider Case 1 with controller located at Sample instant
node i SA , and Case 2 where it is located at A. We
stochastically couple the cases so that events of packets
from i reaching A are identical. Hence, using LPA, node
Sensor
A can receive the same observation information in both p1
cases. However, the controller located at A additionally p2
pDrop p3
has the history of all implemented controls.
p4
p5
III. S P D M Estimator p0 p1 p2 p3 p4 p5
Consider the observable and controllable system:
k (k) k
xk+1 = Axk + Buk + wk , s
yj = C jx j + v j,
zj = {y j , y j1 , . . . y0 }. (2) Fig. 3. State observations occur at sample instants. Control actions are
computed and implemented at actuation instants. Actuation instants
are separated by s sample instants. Sensor observations are sent over
Noises wk and v j are zero mean iid Gaussian processes the network and incur a delayP i with probability pi , or are dropped
with covariances w and v . Packet delays, x0 , {wk } with probability pDrop = 1 i=0 pi .
and {vk } are mutually independent. y j is the state
observation made by the sensor at time j. z j is the

4603

Authorized licensed use limited to: The Ohio State University. Downloaded on July 19,2010 at 21:28:36 UTC from IEEE Xplore. Restrictions apply.
47th IEEE CDC, Cancun, Mexico, Dec. 9-11, 2008 ThB06.1

IV. N C B E If v is the eigenvector of A corresponding to the eigen-

E C value of A with the largest magnitude, max (A), then:

We address boundedness of the quadratic cost: h i Y
X i X i


N1 v E x v v x v
p j

p j
max (A)2i .

1 X

i=0 j=1 j=0
J = lim sup E xk Qxk + uk Ruk .
N N k=0 We now use the ratio test, noting that limi pi = pDrop :
Q 
Denoting by xk
the state estimation error covariance i+1
2i+2
j=1 p j 1 pi+2 max (A)
at k, which is random because of the randomness in lim sup Qi 
2i
obtaining measurements, we can write i j=1 p j 1 pi+1 max (A)


N1 pi+1 1 pi+2
1 X  x
 = lim sup max (A)2
J = lim sup E xk Qxk + uk Ruk + Tr(k Q) , i (1 pi+1 )
N N
k=0
h i = pDrop max (A)2 ,
So we need only study boundedness of E xk . establishing the necessity of (1).
A. State Estimation: Kalman Filter Theorem
  4.2: A sufficient condition for boundedness
of E x is
The Time update is, with the usual notation: 1
pDrop < .
xk+1|k = Axk|k + Buk , (3) max (A)2

Proof: Because 1 I x 2 I, and the same is true
xk+1|k = Axk|k A + , w
(4) for Ai W A i , we only need to consider boundedness of:
The Measurement update is: h i
X
 1 E x = i xk+i|k
Kk+1 = xk+1|k C Cxk+1|k C + v , (5) i=0


X
xk+1|k+1 = xk+1|k + Kk+1 yk cxk+1|k , (6)
3 i Ai A i
xk+1|k+1 = (I Kk+1 C) xk+1|k , (7) i=0

Y i
At each actuation sample instant, (3) and (4) are used X


p j max (A)2i ,

3
to update the system state estimate. Iterating D steps,
i=0 j=1
D1
X where we have upper bounded several terms by 1. By
xk+D|k = AD xk|k A D + Ai w A i . (8)
the ratio test, a sufficient condition for stability is:
i=0
Q 
i+1 2i+2
j=1 p j max (A)
1
If there is no packet loss or delay, then since [A, w 2 ] is lim sup Qi  < 1
controllable and [A, C] detectable, the error covariance i j=1 p j max (A)
2i
converges to a positive definite limit, which we denote
by x . For simplicity we assume that the system is pDrop max (A)2 < 1.
started with x0|0 = x . Under the LPA, whenever a
packet z j arrives, the estimation error covariance xj|j Hence stabilizability under LPA only depends on packet
reverts to x . Hence it is the durations of the excursions drop probability and system dynamics. Performance is
from x that determine boundedness. adversely affected by larger delay, but not stabilizabil-
ity. This has potential design implications. One can set
B. Bounded Estimation Error Covariance the critical number of delivery attempts at the transport
 
PThe expected estimation error covariance is E x = layer, similar to the MAC layer retry limit in IEEE
x 802.11, so as to meet a desired pDrop .
i=0 i k+i|k .
Theorem 4.1: (1) is necessary
 Q for bounded cost. The performance of several specific delay distribu-
P
Proof: = p
j=0 j j=1 p j . So, using (8),
tions in Figure 4 is illustrated in Figures 5, 6 and 7.

h i
X pi pi pi
E x = i xk+i|k pDrop pDrop pDrop
i=0


X X i1
i i j w j (a) Uniform (b) Exponential (c) Positve
= i A x A + A A (9)

i=0 j=0
Fig. 4. The drop probability, pDrop , is chosen to be the same. The
Y
i i P
X X density function, is chosen so to satisfy i=0 pi + pDrop = 1.
p j Ai x A i .


p j

i=0 j=1 j=0

4604

Authorized licensed use limited to: The Ohio State University. Downloaded on July 19,2010 at 21:28:36 UTC from IEEE Xplore. Restrictions apply.
47th IEEE CDC, Cancun, Mexico, Dec. 9-11, 2008 ThB06.1

Packets Dropped After 10 Sample Instants

600
400
max(A) = 1.2
300 500 max(A) = 1.1

200
400

Cost
100

300
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7

200
Packets Dropped After 20 Sample Instants
3000

2500 100
2000
Cost

1500 0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8
1000 Packet Drop Probability

500

0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 Fig. 7. Each curve represents a different system. Packets are delayed
with uniform probability (Figure 4(a)), and dropped when delay
Packets Dropped After 30 Sample Instants exceeds 30. The stability upper bound on pDrop is indicated by the
12000
vertical asymptotes. The upper curve represents a system with a
10000
Single Observation largest eigenvalue of 1.2 and hence a stability bound of pDrop = 0.69.
Long Packets The lower curve is a system with largest eigenvalue as 1.1 and
8000
stability bound of pDrop = 0.83.
6000

4000

2000

0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
V. S S C LPA
We consider a suboptimal scheme without LPA.

A. A Sub-Optimal Estimation Scheme

Fig. 5. Packets are delivered with uniform probability of delay as
shown in Fig. 4(a). In the top figure, packets delayed by more than
10 sample instants are dropped. In the second and third figures the
delay threshold is 20 and 30, respectively. The x-axis is the packet
drop probability. The lower curve in each is the cost under LPA,
and the upper curve that for system which only transmits a single
observation. Notice that the cost diverges at the same packet drop
probability in each figure.
200
Uniform
Exponential
For an n-dimensional observable system, n consecu-
tive observations yield a state estimate with bounded
error covariance [9], denoted x . We consider a sub-
optimal filter that uses only the most recent set of n
consecutive measurements that have arrived. Open loop
prediction is done in between such batches of n or more
consecutive observation arrivals.
Denote by nk the elapsed time since the most recent
time at which n consecutive packets were delivered and
the current time k. Denote this set of n observations as:
n o
Y(k) = y j : k nk n + 1 j k nk .

150
Positive
Bound
The resulting estimation error covariance at k, xk is:

k 1
nX
Cost

100 xk|nk = Ank x Ank + A j w A j .
j=0

50
nk being random, we compute the expectation
E[E[xk|n |nk ]].
k
Theorem 5.1: The expected estimation error covari-
0 ance is bounded if
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
Packet Drop Probability
pk 1
lim inf >1 . (10)
k pk+1 max (A)2
Fig. 6. Packets delayed 30 time steps are dropped. The lowest curve Proof:
is the cost for exponential packet delay distribution (Figure 4(b)), the
middle curve for uniform distribution (Figure 4(a)), and upper curve If packets (k nk n + 1, k nk 1 n + 2, . . . , k nk ) are
for linear distribution (Figure 4(c)). the latest n consecutive packets to arrive before k, then
packet knk +1 should have not arrived by k; see Fig. 8.

4605

Authorized licensed use limited to: The Ohio State University. Downloaded on July 19,2010 at 21:28:36 UTC from IEEE Xplore. Restrictions apply.
47th IEEE CDC, Cancun, Mexico, Dec. 9-11, 2008 ThB06.1

n pknk
Interestingly, the condition is independent of n. For a
geometric delay pk = (1 p)k p and pk = (1 p)k the
sufficient condition is p > 1 max1(A)2 . This is illustrated
p0 in Figure 9 as the = 0 curve.
pknk
B. An improved suboptimal scheme
An improved estimator can use any n observations
from n + consecutive observations. Denote by the
largest integer such that any r observations drawn from
z }| { ... k nk k {ykr+1 , ykr+2 , . . . , yk } yield an estimate of xk with
k nk n + 1 bounded error covariance, and call k an estimation epoch,
the number of allowable misses, and the estimator an
Fig. 8. The Figure illustrates an event where n consecutive packets
arrive before time k. The packets which arrive may do so at anytime
allowable misses estimator.
between their transmission time and time k. The potential arrival Theorem 5.2: With allowable misses, a state esti-
times are represented by the shaded area. The sequence of consec- mate with bounded error covariance can be formed if
utive packets is broken by a packet being delayed with probability 1
pnk .
! +1
pk 1
lim inf >1 . (13)
k pk max (A)2

This non-arrival occurs with probability pnk . Hence, Proof: The set of observations used at k is
" #
Packets (k nk n + 1, k nk n + 2, . . . , k nk ) Y(k) {y j : k r + 1 j k and y j delivered}.
P
are the last n consecutive ones to arrive before k (14)
n +n1 n +n2 n
kX kX X k
Define the oldest possible observation in Y(k) as k
pi
pi ...
pi pnk . (11) k r + 1. Say that Y(k) is full if it is missing exactly
i=0 i=0 i=0 observations in the interval k n + 1 j k.
Using (11) and (8) we can compute an upper bound For k to be the most recent estimation epoch, clearly
+n1 n +n2 n packet yk+1 should have been dropped, and in [k r
X nkX kX X k
+ 1, k] there need to be drops. Hence,
E[xk|knk ]




pi

pi
...
pi
p
nk
" #
nk =0 i=0 i=0 i=0 k is the most
P
nXk 1
recent epoch before k

. Ank x A nk + A j w A j
!
. r+ +1


pkk pkk . . . pkk . pkk Kpkk ,
j=0 |{z}
It is sufficient to consider the boundedness of | {z }
yk+1
+n1 n +n2 n missing from r +
X nkX kX X k
missing
p p ... p p Ank A nk



i
i
i
k n


nk =0 i=0 i=0 i=0 for a sufficiently large constant K. Substituting j = k k:
n +n1 n +n1 n +n1
X kX kX kX

p Ank A nk



p i

pi
...
p i
k n

X
j1
X

+1
j i

nk =0
+n1 n
i=0 i=0 i=0 E[xk|k ] Kp j j
A

x
k|Y(k) A +
i w
A A ,
X nkX j=0 i=0


pi pnk Ank A nk

=
nk =0 i=0
It is enough to consider conditions for boundedness of
| {z }
X
X
+1
  +1
1 p j A j A j p j max (A)2j . (15)

X
X

j=0 j=0
pnk Ank A nk pnk max (A) 2nk
.
nk =0 nk =0 By the ratio test, this is bounded if
For boundedness, by the ratio test it is sufficient if +1
p j+1 max (A)2(j+1)
pnk +1 max (A) 2nk +2 lim sup +1
< 1, i.e., if
lim sup < 1. j p j max (A)2j
nk pnk max (A)2nk
! 1
! +1
pj 1
This is assured if lim sup 1 < .
pn
!
1 j p j max (A)2
lim sup 1 k < . (12)
nk pnk max (A)2

4606

Authorized licensed use limited to: The Ohio State University. Downloaded on July 19,2010 at 21:28:36 UTC from IEEE Xplore. Restrictions apply.
47th IEEE CDC, Cancun, Mexico, Dec. 9-11, 2008 ThB06.1

For a geometric delay distribution, the sufficient con- R

 1
 +1
dition is p > 1 max1(A)2 . This condition is illustrated [1] Special Issue: Technology of Networked Control Systems. Proceed-
ings of the IEEE, Jan 2007.
in Figure 9 for different values of the term ( + 1). [2] M. Ades, P. E. Caines, and R. P. Malhame. Stochastic optimal
control under poisson-distributed observations. IEEE Transac-
1 tions on Automatic Control, 45(1), January 2000.
[3] M. Chow and Y. Tipsuwan. Network-based control systems:
0.9
1 A tutorial. In The 27th Annual Conference of the IEEE Industrial
Packet Delivery Probability

0.8 2 Electronics Society., pages 15931602, 2001.

4
0.7 [4] S. Graham. Issues in the convergence of control with communication
8
0.6 16 and computation. PhD thesis, University of Illinois at Urbana-
Champaign, 2004.
0.5 [5] V. Gupta, B. Hassibi, and R. M. Murray. Optimal LQG con-
0.4 trol across packet-dropping links. Systems & Control Letters,
0.3
56(6):439446, June 2007.
[6] O. C. Imer, S. Yuksel, and T. Basar. Optimal control of dynamical
0.2 systems over unreliable communication links. In NOLCOS,
0.1 2004.
0 [7] O. C. Imer, S. Yuksel, and T. Basar. Optimal control of dynamical
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 systems over unreliable communication links. Automatica, June
Largest Eigenvalue of A
2006.
[8] A. Khan, N. Agarwal, D. Tilbury, and J. Moyne. The impact of
random device processing delays on networked control system
Fig. 9. Illustration of Thm. 5.2 for a geometric delay distribution performance. In 42nd Annual Allerton Conference on Communica-
for different values of the term ( + 1). The upper curve ( = 0) tion, Control, and Computing., 2004.
corresponds to Thm. 5.1. [9] P. R. Kumar and P. Varaiya. Stochastic Systems: Estimation,
Identification and Adaptive Control, volume 1. Englewood Cliffs,
1986.
[10] R. Luck and A. Ray. An observer-based compensator for
C. Existence of distributed delays. In Automatica, volume 26, pages 903908,
1990.
For random A, usually 1. For example, = 6 for [11] J. Nilsson, B. Bernhardsson, and B. Wittenmark. Stochastic
analysis and control of real-time systems with random time
delays. Automatica, 34(1):5764, 1998.
0.65510 0.58530 0.89090 0.84070 [12] C. H. Papadimitriou and J. Tsitsiklis. Intractable problems
0.16260 0.22380 0.95930 0.25430
in control theory. SIAM Journal on Control and Optimization,
A = , (16) 24(4):639654, July 1986.
0.11900 0.75130 0.54720 0.81430
[13] I. B. Rhodes and D. L. Snyder. Estimation and control per-
0.49840 0.25510 0.13860 0.24350 formance for space-time point-process observations. In IEEE
Transactions on Automatic Control, volume AC-22, pages 338346,
(C = [1 0 0 . . . ] throughout). Systems can have large . June 1977.
For example, A1 has > 70, but A2 yields = 0: [14] C. L. Robinson and P. R. Kumar. Control over networks of un-
reliable links: Location of controllers, control laws and bounds
on performance. In Proceedings of Control over Communication
2 1 0 2 1 0 Channels (ConCom), Cyprus, April 2007.
2 1 , and A2 = 0 2 0 .

A1 = 0 [15] C. L. Robinson and P. R. Kumar. Optimizing controller location
in networked control systems with packet drops. IEEE Journal
0 0 3 0 0 3
on Selected Areas in Communications, 26(4):661671, May 2008.
[16] L. Schenato. Optimal sensor fusion for distributed sensors
VI. C subject to random delay and packet loss. In Proc. of the IEEE
Conf. on Decision and Control., New Orleans, December 2007.
We have considered the effect of packet losses and [17] L. Schenato. Optimal estimation in networked control systems
subject to random delay and packet drop. IEEE Trans. on
delay on networked control system stabilizability un- Automatic Control, 53(5):13311317, 2008.
der a Long Packets Assumption. We have established [18] P. Seiler and R. Sengupta. Analysis of communication losses in
an optimal controller location and obtained a neces- vehicle control problems. In Proc. of the American Controls Conf.,
June 2001.
sary condition for stability which is sufficient if the [19] B. Sinopoli, L. Schenato, M. Franceschetti, K. Poolla, M. I.
inequality is strict. The latter depends only on the drop Jordan, and S. S. Sastry. Kalman filtering with intermittent ob-
probability and not the delay probability. We have also servations. In IEEE Transactions on Automatic Control, volume 49,
2004.
considered a sub-optimal scheme that may possibly be [20] D. L. Snyder and P. M. Fishman. How to track a swarm of
of interest. fireflies by observing their flashes. In IEEE Transactions on
Information Theory, 1975.
[21] H. S. Witsenhausen. A counter example in stochastic optimal
A control. Siam J. Control, 6:131147, 1968.

This material is based upon work partially sup-

ported by USARO under Contract Nos. W911NF-08-1-
0238 and W-911-NF-0710287, NSF under Contract Nos.
ECCS-0701604, CNS-07-21992, CNS-0626584, CNS-05-
19535 and CCR-0325716, and AFOSR under Contract
No. F49620-02-1-0217.

4607

Authorized licensed use limited to: The Ohio State University. Downloaded on July 19,2010 at 21:28:36 UTC from IEEE Xplore. Restrictions apply.