PRINCIPLES OF EXPERIMENTAL DESIGN

Learning Objective

At the end of this lesson, the participants should be able to

• define experimental error, bias, precision, treatment, experimental unit,

treatment effect and interaction effect;
• choose appropriate treatments to answer the objectives of the experiment and
formulate contrasts for each objective;
• choose dependent variables or responses to be measured to provide
information about the problem of the experiment; and
• describe the three essential components of an experimental design and explain
the importance of each component.

Introduction

The term experimental design refers to a plan for assigning experimental conditions to
subjects and the statistical analysis associated with the plan (Kirk, 1982). Its purpose is to
ensure that measurements taken from experimental units (say, plot, plant, leaf, cow, etc.)
are free from bias and give results as precise as practicable.

Experimental Error

Experimental error results from the natural variation that exists among the experimental
units and random variations in procedure. For example, if one harvests two equal areas
of rice from a field, grain yield from the two areas will seldom be equal; the weight of
fruit from adjacent trees in an orchard is seldom the same; rates of weight gain of any two
animals of the same species and breed nearly always differ. Such differences among crop
or animal units result from genetic and environmental differences beyond the control of
the experimenter and from variation in application of treatments. Although they are not
errors in the sense of being wrong, they result in variability among measurements of
experimental units which is referred to as experimental error.

Principles of Experimental Design 89

Bias and Precision

Bias is defined as the difference of the expected value of an estimate from the true value.
Bias can result from many factors such as badly calibrated measuring instruments,
subjective allocation of treatments to units and subjective measurements as scoring.
Some sources of bias can be avoided through randomization which is the process of
randomly allocating the treatments to the experimental units.

Precision on the other hand, refers to the closeness with which the measurement
approaches the average. Precision is achieved by the following:
1. Wise choice of experimental units (before randomization) which should be as
similar as possible while still representing the range of material to which the
results should apply.
2. Careful conduct of all operations on units before and during the experiment.
This includes high standard of care, accuracy of treatment, protection from
damage and irrelevant sources of variability and trustworthy measurement and
recording.
3. Replication, that is, repeating one treatment more than once.

Treatment

Treatment is a dosage of material or a method that is to be tested in the experiment. A

crop variety, a fertilizer level or a management practice is a kind of treatment.

When the treatments of an experiment consist of several levels of one factor, say different
varieties of a rice cultivar, then the experiment is said to be a single factor experiment.
On the other hand, when the treatments are a combination of two or more factors, the
experiment is said to be a factorial experiment. For example, testing five varieties under
four nitrogen rates.

Experimental Unit

Experimental unit refers to the unit of experimental material to which a treatment is

applied. It can be a single leaf, a whole plant, an area of land containing many plants, a
pot or a bench in the greenhouse, a single animal, several animals, or an entire herd.

Treatment and Interactions Effects

A treatment or main effect is defined as the expected increase or decrease in average

response with the application of a particular treatment. On the other hand, an interaction
effect is the combined effect of two or more factors. That is, factors A and B interact if
the effect of A is different at different levels of B or equivalently if the effect of B is
different at different levels of A.

90 Treatment and Interactions Effects

Preliminary Considerations when Planning an Experiment

The most important step in planning an experiment is to decide what the objectives of the
experiment are. It is often necessary not only to state what is to be tested, but also to
specify the population to which results are to apply. For example, it may be desired to
estimate the increase in the yield of rice resulting from a fertilizer application. The type
of fertilizer and time and method of application need first to be decided in specifying the
treatment. It is then necessary to decide to what population the results are to apply. The
widest possible population consists of all rice varieties and the whole range of conditions
under which they might be cultivated, i.e., all possible soil types, fertilizers and other
cultivation, weather conditions and so on. Practically we must be content with less
general, but still meaningful populations.

Choice of Treatment

The choice of treatments is basically dependent on the questions the experimenter wants
to be answered. For a simple question like, “Which is higher yielding, Variety A or
Variety B?” may have Varieties A and B as the treatment in an experiment. A simple
experiment such as this gives a simple conclusion like, “Variety A is higher yielding than
Variety B.” However, often times, conclusions are not as simple as this. More
realistically, one could say that Variety A is higher yielding if there is no tungro
infestation, otherwise Variety B is higher yielding because it is more resistant to tungro.
Or one could also say that Variety A is higher yielding if this amount of nitrogen is used.
There is no information as to which will give higher yield at different levels of nitrogen.
It is under this situation that an experimenter may include another factor, say level of
nitrogen, to widen the scope of his conclusions. If one is assessing the effect of weeds on
different rice varieties, he may have different varieties treated with different types of
herbicide. He may also wish to include a control treatment, such as no herbicide
application, against which to assess each herbicide. That is, a herbicide is said to be
effective if its use results in yield significantly higher than the yield of the control. One
may also wish to test different levels of a factor, say nitrogen, to assess the optimum level
that would optimize the response. One may choose the levels to be tested at random from
all possible levels of application or purposely selecting the levels to be included. One
may include more levels at the point where curvature is expected and less levels near
zero. For this type of experiment, one may or may not include a zero level. If it is known
that yield is very much lower with no nitrogen, then it may not be necessary to include a
zero level in the experiment. However one level is usually a reference point, such as the
level recommended for the current variety, this serves as the control treatment.

Principles of Experimental Design 91

Contrasts

As mentioned above, the choice of treatments depends mostly on the questions the
experimenter wants to answer. Often these questions can be formulated statistically by
contrasts. Contrasts are combinations of the treatment means designed to have specified
expected values if there are no treatment effects and different values if there are effects.
Consider an experiment with four treatments, O, A, B and C where O is the control, say
no herbicide control and A, B and C are different herbicides. The experimenter may wish
to test the effectiveness of each herbicide by comparing each with the control. In such
case the contrasts are the following:

O A B C
1 -1 0 0
1 0 -1 0
1 0 0 -1

The numbers in the rows of table are the coefficients to be multiplied with the
corresponding means, the products are then added to form the contrast value.

Unfortunately the contrasts set up in this table will not lead to independent test so that the
researcher would have to be careful in interpreting the results. This is because the
coefficient sets are not orthogonal. One can easily test whether two contrasts are
orthogonal if their sum of products of corresponding coefficients will be zero. In order to
obtain independent tests this control-treatment comparison is usually formulated as
follows:

O A B C
3 -1 -1 -1
0 2 -1 -1
0 0 1 -1

The first contrast tests the control against the average of the three treatments A, B and C,
the second compares A to the average of B and C and the third compares B and C. Note
these contrasts are orthogonal.

Given k treatments it is only possible to construct sets with k-1 orthogonal contrasts
which are related to the df for the treatments SS.

Note that the sum of the coefficients of each contrast should be equal to zero. The first
contrast compares treatment A with the control O, ignoring treatments B and C. If A has
no effect compared to O, the value of this contrast should be near zero. If one wishes to
compare the average effect of A, B and C with O then the contrast coefficients are 3, -1, -
1, -1 or equivalently 1 -1/3 -1/3 -1/3.

92 Choice of Measurements
If the levels of the treatment to be tested are quantitative, say 0, 30, 60 and 90 units of
some application then comparisons between individual pairs of means are not usually of
interest. What is of interest is to know the relationship between the response variable and
the treatment. Two specific questions may be whether this relationship is linear or
quadratic over the range of values tested. In such case, the contrasts are the following:

Contrast 0 30 60 90
Linear -3 -1 1 3
Quadratic 1 -1 -1 1

Coefficients for orthogonal polynomials such as those given above are usually provided
in most experimental design books (e.g., Gomez & Gomez p. 641). On the other hand,
coefficients for treatment comparisons are constructed using the following rules:
1. If two groups of equal size are to be compared, simply assign coefficients of +1 to the
members of one group and -1 to members of the other group. It is immaterial which
group is assigned the positive coefficients.
2. Treatments not involved in the comparison are assigned a coefficient of zero.
3. In comparing groups containing different numbers of treatments, assign to the first
group coefficients equal to the number of treatments in the second group, and to the
second group coefficients of the opposite sign equal to the number of treatments in
the first group. Thus, if among five treatments, the first two were to be compared to
the last three, the coefficients would be +3, +3, -2, -2, -2.
4. Reduce coefficients to the smallest possible integers. For example, in comparing a
group of two treatments with a group of four, by rule 3, the coefficients are +4, +4, -2,
-2, -2, -2, but these can be reduced to +2, +2, -1, -1, -1, -1.
5. Interaction coefficients can always be found by multiplying the corresponding
coefficients of the main effects.

It is important for an experimenter to formulate the contrasts relating to the objectives of

his experiment as early as possible so as to make adjustments in the experimental design
to give higher precision to more important contrasts. For example, in the case of the
herbicide trial given above, if C is a newly developed herbicide which is less expensive
than A and B, the recommendation for its usage will depend on its comparative
performance with A, B and O. Hence the experimenter may wish to give a higher
precision to the estimate of the mean of C. This could be done by having more plots
treated with herbicide C that is, C will have more replications than treatments A, B
and O.

Principles of Experimental Design 93

Choice of Measurements to be Made

In choosing a response or dependent variable, the experimenter must be sure that the
response to be measured really provides information about the problem under study.

Measurements collected during an experiment can be classified into the following types:

1. Primary observations. These measure characters for which the treatment

comparisons are of utmost importance or are needed in the calculation of such
quantities. For example, in rice research the most common primary character is grain
yield. A treatment is said to be better that another treatment if its yield is higher.

2. Substitute primary observations. If a particular primary observation is difficult to

obtain, it may be profitable to use a substitute observation that is easier to get. For
example, photosynthesis in rice is almost impossible to measure because of its highly
variable nature due to its dependence to environmental conditions. Besides, the
equipment needed to measure it is very expensive. A substitute character that can be
used to characterize photosynthesis or the amount of light intercepted by the plant is
the amount of nonstructural carbohydrates present in the plant.

3. Explanatory observations. These are taken to attempt an explanation of any treatment

effects found on the primary observations. For example, to explain the variation in
grain yield, number of tillers per hill may be measured to show that an increase in
tiller number coincides with increases in grain yield.

4. Covariate or supplementary observations for increasing precision. For example, if

during the course of the experiment, the plants were infested by tungro, data on
percent infestation may be recorded. This variable may then be used as a covariate in
an analysis of covariance later.

5. Observations for checking the application of the treatments. For example if the
treatments correspond to different temperatures, it will be natural to check
independently that the appropriate temperature has in fact been achieved.

94 Choice of Measurements
Choice of Experimental Design

The choice of an experimental design is dependent upon the nature and variability in the
experimental area or material and on the number and nature of the treatments to be
applied. The chosen design must provide estimates of the most important contrasts, in
terms of the experimental objectives, with the greatest precision possible given the
resources available. It must also give reasonably good estimates of the variances of these
contrasts (Dyke, 1997).

By this we mean that in addition to requiring high precision (small variance), we would
like a reliable estimate of variance to provide high power for testing the contrasts.

This depends on the number of error degrees of freedom (DF) as can be seen from critical
t-values which decrease with the number of error DF, rapidly at first and then slowly to
the critical normal variate for large DF.

Requirements of a Valid Experimental Design

The statistical design of experiments refers to the process of planning the experiment so
that appropriate data will be collected and can be analyzed by statistical methods resulting
in valid and objective conclusions. A statistically valid experimental design is necessary
if meaningful conclusions are to be drawn from the data. There are three basic
requirements for a valid experimental design. It should be unbiased, precise enough to
detect differences of practical importance and it should provide a valid estimate of
experimental error so that the variability of estimates can be assessed and hypotheses can
be tested. Three principles which can ensure the validity of experimental designs are:
replication, randomization and error control.

Replication

Replication means that a treatment is repeated two or more times. Its function is to
provide an estimate of experimental error by directly measuring the variability between
similarly treated units and to provide a more precise measure of treatment effects by
averaging the individual observations. The number of replication that will be required in
a particular experiment depends on the magnitude of the differences the experimenter
wishes to detect and the variability of the data to be collected. Considering these two
things at the beginning of an experiment will save much frustration.

Randomization

Randomization is the assignment of treatments to experimental units so that all units

considered have an equal chance of receiving a treatment. It guarantees unbiased
estimates of treatment means and of the experimental error. In most situations it is

Principles of Experimental Design 95

required to assure the validity of the statistical theory underlying the tests of significance
in the analysis.

Error Control

The principle of experimental design allows for certain restriction on randomization to

reduce experimental error. For example, in the Randomized Complete Block Design,
treatments are grouped into blocks that are expected to perform differently, allowing a
block effect to be removed from the total variation in the trial.

References
Box, G.E.P., Hunter, W.G. & Hunter, J.S. (1978). Statistics for Experiments, John Wiley
& Sons, Inc.
Cochran, W.G. & Cox, G. M. (1950). Experimental Design, John Wiley & Sons, Inc.
Dyke, E. (1997). How to Avoid Bad Statistics. Field Crop Research, Vol. 51, 165-187.
Federer, W.T. (1955). Experimental Design, McMillan.
Gomez, K.A. & Gomez, A.A. (1984). Statistical Procedures for Agricultural Research,
John Wiley & Sons, Inc.
John, P.W.M. (1971). Statistical Design and Analysis of Experiments, The McMillan
Company.
Montgomery, D.C. (1984). Design and Analysis of Experiments, 2nd Ed., John Wiley &
Sons, Inc.
Ostle, B. & Mensing, R.W. (1975). Statistics in Research, 3rd. ed., The Iowa State
University Press.

96