ROBUSTNESS ASPECTS OF EXPERIMENTAL DESIGNS

R. Srivastava, Rajender Parsad and V. K Gupta

I.A.S.R.I., Library Avenue, New Delhi -110012
1.
Introduction
Design of Experiments forms a fascinating branch of Statistics, which owes its origin to
agricultural experiments. During 1930s to 1950s there have been major contributions
from Fisher and his co-workers in the development of classical theory of design of
experiments. Last five decades have witnessed a quantum jump in the development of
this branch of Statistics. A careful glance in classical theory reveals that the designs
developed have inherent simplicity, which helps in analyzing the data easily. The
principal reason for this was lack of computational facilities available in the earlier days.
The emphasis on simplicity, with the advent of high-speed computers, shifted to designs,
which have some statistical properties or are optimal in some statistical sense.
The optimal design theory has been developed under various assumptions like
homoscedasticity, independence and normality of errors, etc. The optimal design theory
assumes absence from disturbances like missing observations, outlying observations or
inadequacy of assumed model, etc. These assumptions may, however, be violated in real
life; thus rendering even an optimal design poor. In order to overcome such a situation
we have to think of designs, which are insensitive or robust against such disturbances. It
may be emphasized that in the literature robustness has been defined with reference to a
particular disturbance and broadly speaking the following disturbances have been
identified which tend to spoil the ideal structure of the design.
(i) Presence of one or more outliers, (ii) Missing data, (iii) Systematic trend in blocks in
a block design, (iv) Model inadequacy, (v) Disturbance due to mechanical handling like
interchange and exchange of treatments.
A design d is said to be robust against one or more of the above disturbances if it remains
insensitive to presence of one or more of the above disturbances in terms of design
properties. For a review on robust designs one may refer to Dey, Srivastava and Gupta
(1991).
Consider a design d having some property A. Let d* be the resulting design after some
disturbance(s) has taken place. Then the design d is said to be robust against
disturbance if d* possesses property A .For example suppose a design d is connected
i.e., all the elementary treatment contrasts are estimable, then the design d1 obtained after
losing some observation(s) remains connected then we say that the design d is robust
against loss of observation(s) as per the connectedness property of the design. The
criterion of robustness is disturbance specific. In subsequent sections, therefore, we
describe relevant criterion as per disturbance and the important work done in that area of
robustness.

Robustness of Designs

2.
Robustness Against Missing Data
In the field experimentation once the experiment has been laid in the field the observation
can be destroyed or lost during the course of experimentation. Missing data may cause
serious problems and may render even a well-planned experiment useless. This is one of
the reasons for the popularity of this area of research over last two decades. There are
various criteria for studying robustness of designs, which are discussed in the sequel.
Ghosh (1978) introduced this criterion of robustness in connection with balanced
incomplete block (BIB) designs.
Criterion 2.1: Consider a connected design d. Let d* be the residual design obtained
after some disturbance, say , has occurred. Then the design d is robust against the
disturbance if design d* is connected.
In order to make ideas clear we give below some examples.
Example 2.1 Consider a design d with parameters v = 8, n =26, b =13, k =2 with
columns representing blocks
BLOCKS
1

Suppose that the observation pertaining to treatment 5 in block 13 is lost. The resulting
design is disconnected and hence the original design is not robustness per connectedness
criterion. It may be mentioned here that connectedness criterion is an essential property
of a design as it enables an experimenter to estimate all the paired differences among
treatment effects through the design.
Criterion 2.2: Although the design may be robust in the sense of connectedness against
the type of disturbances described above, the residual design obtained may not be
efficient as compared to the original design. This criterion is, therefore based on the
efficiency of the residual design. According to this criterion a design is robust against
loss of observation(s) if the efficiency of the residual design is close to the efficiency of
original design. Consider a design d with information matrix Cd . Let d* is the residual
design with information matrix Cd * . Then the efficiency E of the residual design is given
by
E=

Harmonic mean of positive eigen values of C d *

.
Harmonic mean of positive eigen values of C d

It may be observed that this criterion is equivalent to A- efficiency of the design.

Using criteria 2.1 and 2.2, Srivastava, Gupta and Dey (1990,1991) established the
robustness of the following designs:

430

Robustness of Designs

(i)
(ii)
(iii)

BIB designs against the loss of all observations pertaining to a block.

Youden Square designs against the loss of all observations in a column.
Semi regular Group Divisible designs against the loss of all observations in
a block.

Gupta and Srivastava (1992) investigated the robustness of binary variance balanced
block designs when m (1) disjoint blocks are lost. As a special case, robustness of
resolvable BIB designs was investigated when all the blocks of a complete replicate are
lost. It was observed that these designs are fairly robust against loss of m disjoint blocks.
The robustness of reinforced BIB designs (obtained by augmenting all the blocks of a
BIB design with a new treatment) was also investigated using criterion 2.2 and it was
observed that these designs were fairly robust when one block is lost.
Srivastava, Parsad and Gupta (1996) obtained a sufficient condition for the robustness of
block designs used for test treatments- control comparisons against the loss of a single
observation pertaining to a test treatment. The designs considered are binary in test
treatments. In particular, a lower bound to the efficiency E0 of Balanced Treatment
Incomplete Block (BTIB) designs against loss of a single observation of a test treatment
was obtained. E0 was computed for standard-reinforced designs (which is a particular
case of BTIB designs) obtained by adding a single replication of control in every block of
BIB design of Hall Jr. (1986) satisfying r > k, where r is the replication number and k is
the block-size. It was observed that these designs are fairly robust against loss of single
observation pertaining to a test treatment
Chakraborty (1996) has studied the robustness of Balanced Nested Row Column (BNRC)
designs against the loss of all observations in block(s), row(s), column(s), utilising
criterion 2.2. It is observed that the loss in efficiency in the residual design after the loss
of a block in these designs is marginal for v > 8 . For a missing row in BNRC designs
where each block is a Latin Square design, the loss in efficiency is marginal for v > 4 .

Criterion 2.3: Consider the usual Gauss-Markoff linear model

E ( y) = X , D( y) = 2 In , Rank ( X) = p ,

(2.1)

Where y is an nx1 observation vector, X is an nxp design matrix, is a px1 vector of

parameters. Let d1 be the residual design when an observation (collected through d) is
lost. Obviously there are n possible d1 designs. Let x represent the row in X
X
corresponding to the missing observation, i.e. X = 1 . According to Ghosh (1978),
x
measure of information contained in an observation is I (x ) = x (X X )1 x It can easily
be seen that

I (x ) = tr [X(X X)1 X ] = p .

For a robust design 0 < I (x ) < 1. Clearly,

for a robust design I (x ) should be small.

431

Robustness of Designs

Srivastava, Gupta and Dey (1991), using criterion 2.3, established that under the
restricted linear model when the first column of matrix X pertaining to mean is
dropped, the orthogonal main effect plans are robust against one missing observation and
the amount of information contained in any observation is (n-1)/n and is thus constant for
given n. For a second order rotatable design, Srivastava, Gupta and Dey (1991) observed
that the information measure I ( x ) is proportional to the variance of the estimated
response at point b.
Criterion 2.4: Hedayat and John (1974) introduced the concept of resistant designs in the
context of balanced incomplete block (BIB) designs. Suppose d is a BIB design with
parameters v, b, r, k,. Let = {1,2,...,v } denote the set of treatments. Suppose T is a

subset of containing m ( v-2) treatments. Let d0 be the design obtained by deleting

from d, all the experimental units allocated to the treatments in T. Then, Hedayat and
John (1974) made the following definitions.
Definition 2.1: A BIB design d is said to be globally resistant of degree m if d0 is
variance-balanced with respect to the loss of any subset T of treatments, where the
cardinality of T is m.
Definition 2.2: A BIB design d is said to be locally resistant of degree m if d0 is
variance-balanced with respect to the loss of some subsets (but not all) T of cardinality m.
In order to make ideas clear we give below the following example:
Example 2.2 Consider a BIB design with parameters v=8, b=
columns as blocks
BLOCKS
1
2
3
4
5
6
7
3
4
5
6
2
3
4
5
6
7
1
5
6
7
1
4
5
6
7
1
2
3
6
7
1
2
8
8
8
8
8
8
8
7
1
2
3

14, r=7, k=4, =3 with

7
2
3
4

1
3
4
5

2
4
5
6

Suppose that all the observations pertaining to treatment 8 are lost, then it can be
observed that the residual design obtained after deleting this treatment remains variance
balanced and hence original design is resistant to the loss of treatment 8. In fact the
design is resistant to loss of any single treatment. Therefore, this design is globally
resistant of degree one.
Hedayat and John (1974) proved that a necessary and sufficient condition for a BIB
design to be locally resistant of degree one is that the two sub-designs obtained after
removal of one treatment are BIB designs themselves. They have also obtained
characrterisation of globally resistant BIB designs.
Singh and Gupta (1991) obtained sufficient conditions for a non binary, non-proper
variance balanced block design to be resistant as described under criterion 2.4.

432

Robustness of Designs

Srivastava, Parsad and Gupta (1997) have introduced structure resistant block designs
with factorial structure. These designs are resistant with respect to loss of all observations
pertaining to treatment combinations of one particular level of a given factor. They have
also provided some important results involving resistant block designs having orthogonal
factorial structure with balance.
3.
Robustness Against Outliers
1
Criterion 3.1: Consider the usual linear model (2.1). Let H = X( XX) X = hij .

(( ) )

Suppose the uth observation has added to itself an aberration c making it an outlier.
Assuming that c could occur with any of the n observations with equal probability, Box
2
and Draper (1975) showed that if h = huu is minimised then the design is robust in the
u

sense that at no design point can the addition of c cause an unduly large discrepancy in
the predicted y ' s, y in full rank models.
The criterion of Box and Draper has been extended by Gopalan and Dey (1976) to the
situations when X is not of full rank. Using criterion 3.1, Gopalan and Dey (1976) have
established the robustness of following block designs against the presence of a single
outlier
Randomised (complete) block designs;
BIB designs;
All non group divisible, two associate class, connected PBIB designs with 2 = 0 ;
All semi - regular group divisible designs;
All triangular PBIB designs satisfying r + ( n 4 ) 1 ( n 3) 2 = 0 ;
All L2 type PBIB designs satisfying r + ( s 1) 1 ( s 1) 2 = 0 .
Bora, Singh and Singh (1986) studied the robustness of change over designs, which
permit the estimation of direct and first order residual effects using the criterion 3.1
4.
Robustness Against Systematic Trend in Blocks
Criterion 4.1: In many experimental situations, the response to a treatment is affected by
the spatial or temporal position of experimental unit or a plot within a group of units (say,
block) and an assumption of common polynomial trend of a specified degree is
appropriate. In such situations, it is desirable to estimate the treatment differences as
accurately as possible under the model that the measured quantity equals (treatment
constants)+(smooth trend)+(random quantity). One way of getting accurate estimates of
the treatment differences is to randomize completely the order of application of the
treatments and to use analysis of covariance, using trend components as covariates. Cox
(1951) however pointed out that in the presence of trends information is lost by using
such a method, the loss being serious if small number of treatments and/or small number
of replicates are involved.

Another way of getting around the same problem was suggested by Bradley and Yeh
(1980), who concentrate on block designs and suggest that a design be so constructed that
treatment effects are orthogonal to trend effects. If the trend effects within the blocks can
433

Robustness of Designs

be represented by an orthogonal polynomial of suitable degree, Bradley and Yeh defined

a block design as trend free if the adjusted treatment sum of squares under a model
with a trend component is the same as the adjusted treatment sum of squares under a
model without any trend component. Bradley and Yeh have showed that whenever a
trend free block design exists, it has optimality properties relative to a block design with
full randomization over units. We give below an example of a trend free BIB design
when systematic trend is assumed to be of degree one.
Example4.1 Consider a BIB design with parameters v=6,
columns representing blocks.
BLOCKS
0
1
2
3
4
5
1
2
1
2
3
4
0
0
5
5
2
3
4
0
1
2
3
4

b=10, r=5, k= 3, =2; with

3
0
5

4
5
1

This is a trend free design for systematic trend of degree one.

Dhall (1986) has given a general algorithm for the construction of trend-free BIB designs
for block-size 2. Also trend-free BIB designs have been constructed in the presence of
linear trends for all k in the range 3 k 6 . Dhall has also prepared a catalogue of trend
free incomplete block designs which consists of BIB and PBIB designs that are trend free
against the presence of a linear trend.
5.
Robustness of Designs Against Interchange and Exchange of Treatments
Suppose in example 2.1 during the laying out of the experiment treatment 1 in block 13
gets exchanged with treatment 5 .The resulting design d* is disconnected and hence
design d is not robust for such a kind of disturbance. This is also true if treatment 1gets
exchanged with treatment 6, 7, or 8.
Example 5.1 Consider a design d with parameters v = 8, n =30, b=10, k =3 with columns
representing blocks.
BLOCKS

Suppose that during the experimentation treatments 5 and 4 in blocks 3 and 4 get
interchanged. The resulting design is disconnected and hence design d is not robust for
this disturbance against interchange of a pair of treatments.
Batra, Sreenath and Parsad (1997) have shown that binary, variance balanced block
designs viz. RBD, BIB designs, non proper equireplicated variance balanced block
designs of Gupta and Jones (1986) are robust against interchange of a pair of treatments

434

Robustness of Designs

according to criteria 2.1 and 2.2. Batra (1993) and Batra, Sreenath and Parsad (1997)
have obtained similar results for the exchange of a single treatment.
6.
Robustness Against Model Inadequacy
Under model inadequacy we consider situations where the model is either incompletely
or incorrectly specified. One of the earliest works in this direction is due to Box and
Draper (1959,1963) in the context of response surface designs. Suppose a design is used
for fitting a polynomial of degree d1. While the experimenter believes that the
polynomial degree d1 adequately represents the true relationship, he also wants to guard
against the possibility of presence of higher order terms in the model. The problem then
is to choose a design, which allows the fitting of polynomial of degree d1 and also
minimizes the bias due to the true model being a polynomial of degree d2 (d2>d1). Let
y ( x ) be the estimated response at a point x, obtained by fitting a polynomial of degree
d1 and (x) be the true response at x. Box and Draper (1959) consider the integrated

mean square error, E{y ( x ) - ( x )}2 as a measure of robustness of a design.

The concept of robustness against model inadequacy can be extended to areas other than
response surface set up. In single factor experiments, some designs can be used for one
way elimination of heterogeneity setting and two-way elimination of heterogeneity
setting. For example, the symmetric BIB designs are optimal in the competing class of
block designs and if we consider it as a Youden designs it is optimal in the class of row
column design settings. Similarly, row-regular generalized Youden square design are
robust for a two way elimination of heterogeneity setting as well as one way elimination
of heterogeneity settings.
The connected block designs under a fixed effects homoscedastic model remains
connected under a fixed effects heteroscedastic model when the intrablock variances are
assumed to be non-negative real power of block sizes. The designs for making all
possible paired comparisons (test treatments -control (s)) comparisons, which are,
efficient under a fixed effect model remain efficient under a mixed effect models as well.
References
Batra,P.K.(1993). Studies on robustness of block designs against exchange or interchange
of treatments. Ph.D. thesis submitted to IARI, New Delhi.
Batra, P.K., Sreenath, P.R. and Parsad, R. (1997). Robustness of block designs against
interchange of treatments. Jour. Ind. Soc. Agril. Statist ,50(2), 156-167
Batra ,P.K., Sreenath P.R. and Parsad , R. (1997) Robustness of block designs against
exchange of a treatment in one observation. Jour.Ind, Soc. Agril. Statist.
(Communicated)
Bora, A.C., Singh, M. and Singh, G. (1896). Robust changeover designs. Statistician, 35,
449-454.
Box, G.E.P. and Draper, N.R. (1975). Robust Designs. Biometrika, 62, 347 - 352.
Bradley, R.A. and Yeh, C.M. (1980). Trend Free block designs, I: Theory. Ann. Statist.,
8, 883 - 893.

435

Robustness of Designs

Chakraborty, A.K. (1996). Studies on block designs with nested rows and columns. Ph.D.
thesis submitted to IARI, New Delhi.
Cox, D.R. (1951). Some systematic experimental designs. Biometrika, 38, 312 - 323
Dey, A., Srivastava, R. and Gupta, V.K. (1991). Robust Designs and a bibliography.
Cahiers du CERO, 33(1-2), 51-62.
Dhall (1986). Some studies on robustness of designs. Ph.D. thesis submitted to I.A.R.I.,
New Delhi.
Ghosh, S. (1978). On robustness of designs against incomplete data. Sankhya, B40, 204208.
Gopalan, R. and Dey, A. (!976). On robust experimental designs. Sankhya,B38, 297 -299.
Gupta, V.K. and Srivastava, R. (1992). Investigations on robustness of block designs
against missing observations. Sankhya B54, Pt.1, 100-106.
Hedayat, A. and John, P.W.M. (1974). Resistant and susceptible BIB designs. Ann.
Statist. 2, 148 - 158.
Singh, R. and Gupta, V.K. (1991). Resistance of block designs. Jour. Satist.Plann.Inf. ,
27,263-269.
Srivastava, R., Gupta, V. K. and Dey, A.(1990). Robustness of some designs against
missing observations. Commn.Statist. Theor. Meth. ,19,1,121-126.
Srivastava, R., Gupta, V.K. and Dey, A. (1991). Robustness of some designs against
missing data. Jour.Appl. Statist. , 18, 313-318.
Srivastava, R., Parsad, R. and Gupta, V. K. (1996). Robustness of block designs for
making test treatments-control comparisons against a missing observation.
Sankhya B 58,Pt.3,407-413.
Srivastava, R., Parsad, R. and Gupta V.K. (2000). Structure resistant factorial designs.
Sankhya B 62(2), 257-265.

436