This article was downloaded by: [141.214.17.

222]
On: 22 October 2014, At: 20:16
Publisher: Taylor & Francis
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House,
37-41 Mortimer Street, London W1T 3JH, UK

The American Statistician

Publication details, including instructions for authors and subscription information:
http://www.tandfonline.com/loi/utas20

A Consonance Criterion for Choosing Sample Size

a b
W. Alan Nicewander & James M. Price
a
Personnel Testing Division , Defense Manpower Data Center , Seaside , CA , 93955 , USA
b
Department of Psychology , Oklahoma State University , Stillwater , OK , 74078 , USA
Published online: 17 Feb 2012.

To cite this article: W. Alan Nicewander & James M. Price (1997) A Consonance Criterion for Choosing Sample Size, The
American Statistician, 51:4, 311-317

To link to this article: http://dx.doi.org/10.1080/00031305.1997.10474404

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the Content) contained
in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no
representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the
Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and
are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and
should be independently verified with primary sources of information. Taylor and Francis shall not be liable for
any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever
or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of
the Content.

This article may be used for research, teaching, and private study purposes. Any substantial or systematic
reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any
form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://
www.tandfonline.com/page/terms-and-conditions
 A Consonance Criterion for Choosing Sample Size
and James M. PRICE
W. Alan NICEWANDER
Sample sizes determined by the proposed criterion ensure
that subjectively important parameter estimates will be
statistically significant; alternatively, subjectively trivial
values will be excluded from confidence intervals for those
parameters. Formulas are given for simple mean differ-
ences, multiple correlations, and mean comparison methods
in multiple group settings, with suggestions for extension to
larger, factorial designs. Although no assumptions are nec-
essary about the true values of unknown parameters or a
desired level of power, adequate power is provided for im-
portant results. Only central distributions are required for
either the exact iterative solutions given or their closed form
approximations.
KEY WORDS: Effect size; Mean difference; Power;
Downloaded by [141.214.17.222] at 20:16 22 October 2014
ScheffLs method; Tukeys method.
1. INTRODUCTION
For both the consultant and the client one of the most
vexing problems in the design of an experimental study is
the determination of the number of observations to include.
Even more vexing is the situation in which a critical or
important difference is apparent in the sample data, but
statistical statements do not preclude trivial interpreta-
tions (e.g., nonsignificance or zero as a possible value for
a parameter). Previously proposed methods for determin-
ing sample size n have generally been based on optimizing
certain aspects of statistical inference. For example, sam-
ple size is often chosen so as to yield a preferred value of
power (at some specified value of a ) for an effect size that
has been designated as important by the investigator (e.g.,
Scheffk 1959; Cohen 1988). Alternatively, sample size may
be determined by a preference for a given absolute width
of a confidence interval (e.g., Stein 1945).
The present inquiry proposes a consonance criterion
as an alternative to both the power based and the absolute
interval width calculations for sample size. This criterion
allows quick and easy sample size determination without
the necessity for making assumptions about factors that in-
fluence the parameters of the noncentral distribution of a
statistic (e.g., the configuration of population parameters).
All calculations are based on using the central distribu-
tion of the statistic involved. For single degrees of freedom
W. Alan Nicewander is Chief, Personnel Testing Division, Defense Man-
power Data Center, Seaside, CA 93955. James M. Price is Associate Pro-
fessor, Department of Psychology, Oklahoma State University, Stillwa-
ter, OK 74078 (E-mail: psyjmp@vms.ucc.okstate.edu). The authors thank
Richard Harris, Frank Schmidt, the associate editor, and the referees for
their helpful comments on an earlier version of this paper.
@ 1997 American Statistical Association
methods the consonance criterion produces formulas that
are similar, or identical, to some power-based methods, es-
pecially the recent work of Harris and Quade (1992) and
Willan (1994). Following an explication of the consonance
criterion in the case of two-sample differences, we present
generalizations to more complex situations, approximation
formulas for all cases presented, power computations, and
a discussion of the other proposals just mentioned.
2. THE CONSONANCE CRITERION
As used in this inquiry, consonance means bringing into
agreement the values of sample statistics and the statistical
statements based upon them. As such, our use of the term
is quite similar to Gabriels (1969) usage in the context of
multiple comparison techniques. In what follows we assume
that observations are normal and independent, with a com-
mon population standard deviation 0 , and that samples are
to have an equal number of observations n.
Let S represent the difference between the population
means of two groups in standard deviation units, that is,
= 1p1 - P2l/O. (1)
(In the context of a designed experiment 6 is often desig-
nated as the effect size.) Let 6, be such a standardized
difference that is deemed critical or important by the inves-
tigator. The proposed criterion for determining the value
for n is:
Choose TI so that if the sample estimate d of 6 is equal to or greater than
6,, then a (1- a ) l O O % confidence interval for pi - p2 will just exclude
zero.
This very simple criterion brings into accord (or conso-
nance) the parameter estimation aspect of a study (i.e., es-
timating p1 - pa) and the inferential aspect (i.e., hypothesis
testing). Whenever d equals or exceeds 6, in magnitude,
the null hypothesis Ho: pl - p2 = 0 will always be re-
jected at the a level of significance; likewise, a (1- a )100%
confidence interval for p.1 - p2 will never include zero as
a possible value. We consider both aspects in the follow-
ing two sections. In these sections, and the supporting ta-
bles, it should be understood that S is used as a convenient,
unit-free metric for mean differences (especially in situa-
tions in which o may not be known to the experimenter),
and that all probabilistic statements are oriented towards
the difference p1 - p2 in raw score form.
2.1 Two-Group Hypothesis Testing
In order to compute the minimum sample size that will
always lead to the rejection of HOwhen the estimated effect
size equals or exceeds S,, it is necessary to show the rela-
tionship between the estimated effect size and the sample
value of the t statistic. Let
1x1- X 2 l
d= (2)
SP
The American Statistician, November 1997, Vol. 51, No. 4 311
 be the sample estimate of the effect size, where
s2P =
s;
~
+ s; (3)
2 sizes such that if the (standardized) mean difference d is
is a pooled unbiased estimate of the (assumed common)
variance, and s: and s: are the unbiased variance estimates
for the two samples. It is easy to show that
- a somewhat similar method for sample size determination
It1 = d,/E2 (4)
is the value of the sample t for testing the hypothesis that
the group means are equal. In order to find, a priori, the
value of n for which one will always reject Ho whenever d
equals or exceeds a, one must adjust n iteratively until
be,/; = L 4 2 ; 2 ( n - l ) (5)
where tl--cl/2;2(n-1) is the two-tailed critical value of t for
2 ( n - 1) df at the a level of significance. Equivalently, the
consonant sample size can be found by solving, iteratively,
the implicit equation
Downloaded by [141.214.17.222] at 20:16 22 October 2014
For example, suppose that an effect size of 6, = .5 is
deemed to be critical. What should n be in order that HO
is always rejected (at Q = .05, two tailed) when the sample
estimate of 6 is equal to or greater than 6, = .5? Trying
successive values for n until the equation holds, it will be
found that when n is set to 32, t = 2.00, just exceeding the
.05 critical value o f t for 62 df. A sample of size n = 32 will
always lead to the rejection of Ho in experiments that yield
an estimated effect size of .5 or greater; samples smaller
than n = 32 may well yield estimated effect sizes of .5 or
greater, but without the certainty of rejecting Ho.
2.2 Interval Estimation of a Difference
In the context of effect sizes a confidence interval can be
thought of as the set of all numbers that, based on the data,
are not excluded as possible values of the true effect size, on
the basis of chance. Suppose that a sample size is wanted so
that if some important difference (in terms of 6) is found in
the sample means, then the (1- a)100% confidence interval
located by this obtained difference will exclude zero as a
possible population value. In order to find this value of n,
which distinguishes important differences from trivial ones,
one must solve (again iteratively) the following equation:
(7)
Dividing both sides by s p gives
or equivalently
where p is the grand mean (see, e.g., Cohen 1988, p. 277;
(9) Scheffk 1959, pp. 63-64). This configuration of population
312 General
If, as before, 6, is substituted for d in Equation (S), then
an equation identical to (5) is obtained; further simplifica-
tion leads to Equation (6), and one can solve for sample
6, or greater, then the value of zero will fall outside the
(1 - a)100% confidence interval for (PI- p 2 ) .
Stein (1945; Steel and Torrie 1960, pp. 86-87) proposed
that is based on setting the absolute width of a confidence
interval to some a priori value. Such an interval will have
a desired absolute precision (say, f 5 mm), but may also
include 0 as a possible value; the present method employs a
specified relative level of precision to guarantee exclusion
of 0.
For the sake of simplicity and brevity only the hypoth-
esis testing approach is detailed in the following sections.
However, a derivation based on the estimation aspect will
yield identical formulas.
3. GENERALIZATIONS TO MULTIPLE-GROUP
EXPERIMENTS
In what follows the same principles will be used for other
techniques: 1) specify a sample difference that is important,
2) restate the appropriate test statistic in terms of that dif-
ference, and 3) solve for the value of n that sets the statistic
just equal to the a-level critical value. The resulting formu-
las are simple enough for presentation to students at any
level, and may well be easier for even occasional users to
remember. The methods to be presented are quite straight-
forward to derive; their derivations are included here for
completeness and to point out similarities to, and differ-
ences from, the standard power-based techniques.
For multiple-group studies matters are a bit more compli-
cated, and there are at least two ways in which to generalize
the consonance criterion discussed above. In the simplest of
these generalizations one might want to determine n such
that if the two most extreme sample means differ by 6, or
more, one will always reject the hypothesis of equal group
means using: 1) Scheffes method, or 2) Tukeys method.
These two situations are considered separately below.
In this section and those that follow we make use of what
we call the Tang Condition. In an unpublished paper in
1938 (Pearson and Hartley 1953), P. C. Tang proposed a
worst case situation for power calculations for omnibus
tests in the analysis of variance (ANOVA). In this scenario
one assumes that (with the population means ordered from
largest to smallest)
Pl = P + (&c/2)
P2 = P
P3 = P
 means represents the minimum departure from HOsuch that
at least one pair of population means differs by &CT. As
such, this arrangement yields the minimum important value
for the noncentrality parameter of the appropriate sampling
distribution. If one calculates the sample size under the as-
sumptions of the Tang Condition, then the value of n pro-
duced will be an upper bound to the sample size actually
needed to produce a specified level of power.
The consonance criterion makes no assumptions about
population means or desired power. Aside from the later
comparisons of n and power produced by different methods,
we will assume that the configuration of sample means is
as described by (10) when population means are replaced
by corresponding sample means and the common variance
o2 is replaced by MS,,,,,.
3.1 Scheffes Method of Simultaneous Confidence
Intervals
Given an experiment with J 2 2 treatment conditions in
a single factor layout, let
Downloaded by [141.214.17.222] at 20:16 22 October 2014
be the sample estimate of 6 for the pair of sample means
with the largest difference, where X L and XS are, respec-
tively, the largest and smallest sample means. The t test
statistic for this comparison is given by
In order to satisfy Scheffks criterion, with a common
consonant sample size, n is adjusted iteratively until
n6,2 -
2 ( J - 1) - F l - a ; ( J - l ) ; J ( n - l ) .
Equivalently, this consonance criterion-based sample size
can be found using an iterative solution to the implicit equa-
tion
A bonus of this approach results from the isomorphism
between the omnibus ANOVA F test and the maximum nor-
malized comparison among means. If the value of n given
by the previous equations is used in a J-sample layout, both
the ANOVA F and the t for the maximum pairwise com-
parison of means will be significant whenever d, 2 6,.
Put another way, when the ANOVA F test is significant and
d, L 6,,at least one painvise mean comparison will also
be significant by Scheffks method.
3.2 -keys Method for the Maximum Pairwise
Difference
If, in the multiple-group situation just described, one
prefers using Tukeys method to investigate differences in
group means, then the following approach can be used. As
before, let d, be the largest standardized pairwise differ-
ence in the sample means. Then the studentized range test
statistic q is given by
XL - xs
= dmax fi (15)
q= lpzJi
where MS,,,,, is the mean-square error from a correspond-
ing one-way ANOVA. Given this relationship between max-
imum sample effect size d, and the studentized range
statistic q, it is obvious from previously developed logic that
the value of n that will guarantee rejection of the hypoth-
esis of equality of the population means when d, 2 6,
must satisfy
6 c f i = ql--a;J;J(n-l) (16)
where ql--a;J,J(n-l) is the 1 - a percentile point of the
sampling distribution of q. An alternative implicit formula
for n is
2
ql--a;J;J(n-l)
n=
6
:
4. COMPARISONS OF SAMPLE SIZES
Table 1 lists the multiple-sample ( J 2 2 ) consonance
based sample sizes for both the Scheffk and Tukey meth-
ods, along with the standard sample size estimates based
on the Tang Condition, as a function of values of 6, and
the number of groups J . The standard Tang-based sample
sizes require a (user-fixed) value of power for their com-
putation; for this comparison, power was arbitrarily set to
.8. All F-based sample sizes were determined by means of
an iterative routine written in the DATA step of SAS that
was programmed to mimic the aGtions of a user employing
the appendixes of a statistics text, the same method that
was actually used to find the q-based values of n. Step-
ping from an initial value of n = 2, the consonant sample
sizes were found by computing the critical value of F (us-
ing FCNV) and integrating the central F distribution (using
PROBF) until the convergence dictated by Equation (14)
was attained. The standard sample size values were deter-
mined by the same method, except that for each value of n
the value of the noncentrality parameter was computed for
the Tang Condition and the appropriate noncentral F dis-
tribution integrated (again using SASs PROBF function).
(Note: Under the Tang Condition, the noncentrality param-
eter used by PROBF is X = (n6,)/2. The parameter for
entering the Pearson and Hartley charts is 4 = m.)
Table 1 shows that the consonance criterion generally
leads to smaller sample sizes than does the Tang Upper
Bound when power is set at .8 for the standard method,
with the greatest differences in values of n occurring for
smaller numbers of samples. The Tang Upper Bound Val-
ues of n change monotonically with power, whereas the
consonant values of n are constant for a given design and
value of 6,. Therefore, neither method will necessarily al-
ways produce lower values of n; however, it should be ex-
pected that, in situations involving higher values of power
and smaller numbers of samples, the consonance criterion
will require smaller sample sizes. The actual power asso-
The American Statistician, November 1997, Vol. 51, No. 4 313
 Table 1. A Comparison of Sample Sizes Yieldedby the Two Consonance are lower than 6,are by definition too small to be of interest
Criterion Methods and the Standard Power-Based Method to the investigator. Examination of Table 2 indicates that, in
Method for Determining Sample Size such cases, the power of the consonance criterion samples
is never less than .5, and is often 30 and above.
No. of Consonance Consonance Tang Upper Bound
6, groups F-based q-based (power = .SO) The minimum value of .5 arises in the case of J = 2
samples when true S equals S,,, in which case power is the
.1 2 770 770 1,571
3 1,200 1,096 1,928
probability that the sample estimate d exceeds S,, and the
5 1,899 1,490 2,389 mean of the appropriate noncentral t distribution exactly
7 2319 1,739 2,726 equals the critical value of the test. As true S increases, an
9 3,102 1,928 3,006 increasing proportion of the noncentral distribution exceeds
.5 2 32 32 64 the critical value, and power increases. As J increases, this
3 49 44 79 minimum value of power is augmented by the proportion
5 77 60 97
7 102 70 110 of rejections that occur when several mean differences are
9 125 78 122 combined and yield a significant F statistic. Together, these
1.o 2 9 9 17 two factors yield the observed pattern of results in Table 2.
3 14 12 21
5 20 16 25 6. APPROXIMATIONS TO CONSONANT
7 26 18 29 SAMPLE SIZES
9 32 20 31
If one assumes that the population variance is known,
NOTE The consonance criterion F-based and q-based sample sizes are those values of n then simple equations with closed form solutions for con-
that ensure that the HOBfor F and 9 (the studentized range statistic) will be rejected if the most
sonant sample sizes can be developed. The resulting values
Downloaded by [141.214.17.222] at 20:16 22 October 2014

different means differ by &u or more.

ciated with consonance-based sample sizes is discussed in
the next section.
It should also be noted that the q-based consonant values
of n are uniformly smaller than those for the F-based con-
sonant values (except for the trivial case of J = 2). As a
consequence and a second bonus, if one chooses n on the
basis of an omnibus F test (Scheffk's method), then when-
ever the ANOVA F test is significant and d, 2 S,, at
least one pairwise mean difference will also be significant
by Tukey's method.
5. POWER ASSOCIATED WITH CONSONANT
SAMPLE SIZES
An advantage of the consonance criterion for determin-
yield approximate values for the true consonant sample size,
and may be used as a starting point in the iterative solutions.
The implicit equations for consonant sample size-
requiring iterative solutions-are given in Equation (6) for
the two-group case and in (14) and (17) for the two gen-
eralizations to the multiple-group case. Under the assump-
tion of known variance the consonance sample sizes are
simple functions of the standard normal distribution, a chi-
square distribution, and the distribution of the range statis-
tic, respectively. For convenience the approximations are
presented below following the implicit equations giving the
exact solution.
Exact Solution Approximate Solution
J Groups (Scheffe) n = 2 ( J - 1 ) F l - a , ( J - l ) , J ( n - l ) ~ "f-a,(J-l)

ing sample size is that it requires no specification of the 62 e

J Groups (Tukey) 11.= q?-a,J,J(n-i) q?-a,J,-
power desired to reject a null hypothesis for a given (but un- ~

known) pattern of differences among the population means.
One might reasonably be concerned about the actual power
associated with using a consonant sample size instead of the
standard value. Table 2 shows the power levels associated
with the Scheffk and Tukey methods (under the Tang Con-
dition) as a function of values of s,, the number of groups
J, the (unknown) true value of 6,and the sample sizes given
in Table 1. Again, SAS's PROBF function was used to in-
tegrate the appropriate noncentral F distribution for each
of the Scheffk cases. Because of the problems associated
with a noncentral studentized range distribution (Hochberg
and Tamhane 1987) a noncentral F distribution was used to
compute lower bounds to power for the studentized range-
determined sample sizes; these values should be considered
quite conservative.
Notice that a number of power lower bounds are enclosed
in boxes. These boxed values are the lower bounds to power
for true values of S that are equal to or greater than 6,.
Ordinarily, one is most interested in the performance of a
statistical test only for such values of S because values that
314 General
*: 6:
Unlike the exact solutions the approximations employ n
only on the left-hand side, yielding simple, closed form
solutions. Note that in the case of consonant samples for
J samples using Tukey's method, the approximate sample
sizes are based on the distribution of the range-which is
given as the last row (infinite n) of the tabled distribution
of the studentized range statistic q.
It has been our experience, and that of Harris and Quade
(1991), that the approximate solutions for the consonance
criterion sample sizes rarely differ by 1 or 2 (and occa-
sionally 3) from the exact solution. Kupper and Hafner
(1989) noted similar empirical discrepancies in power-based
approximation formulas, and offered tables for correcting
the value of n so derived. Guenther (1981), also in a pa-
per discussing power-based sample size determination, sug-
gested a correction to a similar approximate solution for
n in the two-group case that can be used in conjunction
with the consonance-based sample sizes for two-group ex-
periments, namely, increase the computed approximate n
 Table 2. Conditional Lower Bounds to Power for Consonance Criterion Sample Sizes for Selected Values of J and 6

True 6
10 .25 .50 .75 1.00 1.25 1.50

6, = .I0
J n
2 770 50 .99 .99 .99 .99 .99 .99
3 1,200(1,096) 58 (54) .99 (.99) .99(.99) .99(.99) .99 (.99) .99 (.99) .99 (.99)
5 1,899(1,490) .69(57) .99(.99) .99(.99) .99(.99) .99(.99) .99 (.99) .99 (.99)
7 2,519(1,739) .76 ( 5 7 ) .99(.99) .99 (.99) .99 (.99) .99(.99) .99(.99) .99 (.99)
9 3,102(1,928) .82(57) .99(.99) .99 (.99) .99 (.99) .99(.99) .99(.99) .99 (.99)

6, = .50
J n
2 32 .07 .I 7 50 .84 .98 .99 .99
3 49 (47) .07(.07) .I8(.16) 58 (53) .92(.88) .99(.99) .99(.99) .99 (.99)
5 77 (60) .07(.06) .20(.16) .69 (56) .97(.92) .99(.99) .99(.99) .99 (.99)
7 102 (70) .07(.06) .22 (.16) .76 (57) .99 (.93) .99(.99) .99 (.99) .99(.99)
9 125 (78) .07(.06) .23(.16) .82 (57) .99 (.94) .99(.99) .99 (.99) .99(.99)

6, = 1.00
J n
2 9 .05 .08 .I 7 .32
3 14 (12) .05 (.05) .08(.08) .I9 (.16) .38(.33) 6 2 (54) .82(.75) .94(.89)
Downloaded by [141.214.17.222] at 20:16 22 October 2014

5 20 (16) .05 (.05) .08(.07) .21 (.15) .47(.32) .76(56) .94(.79) .99(.93)
7 26 (18) .05 (.05) .08(.07) .21 (.15) .47(.32) .76 (56) .94(.79) .99(.93)
9 32 (20) .05 (.05) .08(.07) .23(.15) 52 (.32)

NOTE: All values of n are based on the consonance criterion, and are shown in Table 1. For J > 2, sample sizes are based on both the omnibus F test and the studentized range statistic (values
of n in parentheses). All conditional lower bounds to power are based on rejectingthe hypothesis tested by the omnibus F test, that is, power values in parentheses are for sample sizes determined
from the studentized range statistic, but power was evaluated in terms of rejecting the omnibus F test (see text).

by 0.252,2-,/, observations. This modification gives val-
ues that correspond roughly to the empirical discrepancies
mentioned above, and it has the advantage of automatically
adjusting for the level of a and the nature of the test (one-
or two-tailed). Whether this modification can be improved
and/or extended to multiple-group techniques is an open
question.
7. COMPARISON WITH OTHER PROPOSALS
As mentioned at the outset, in the simplest case, involv-
ing the simple difference between two sample means, the
formulas produced by satisfying the consonance criterion
are similar or identical to those produced by others. This is
due, in part, to similarities or identities among those other
approaches. For instance, both Guenther (1981) and Kupper
and Hafner (1989) make use of the formula (modified for
consistency with the present notation)
n 2 2[(21-,/2 + z1-a)/6l2
which determines sample size for a two-tailed test of signif-
icance for two independent samples at the a level of signifi-
cance with power equal to 1- ,O (exact for known common
variance and approximate otherwise). Harris and Quades
(1992, p. 46) formula for this situation is the special case of
(18) that occurs when their minimally important difference
significant (or MIDS) criterion (that the optimal power for
a test of significance is S O ) is adopted and 21-p = 0. Willan
(1994), working in the context of controlled management
trials, derives a similar formula based on setting power at
S O for the value of (our) 6 that represents the researchers
point of indifference between two competing treatment reg-
imens.
The expression in (18) is not, in itself, hard to derive from
a power-based standpoint, and seemingly quite a few others
have done so because each paper just mentioned credits
a different set of predecessors for its origin. The present
paper arrives at (18) by dropping all reference to power in
the derivation, relying instead on a requirement of assured
significance or certain exclusion of zero from a confidence
interval when Id1 2 6,. This represents one major difference
between the present paper and those of Harris and Quade
(1992) and Willan (1994).
A second major difference with the work of Harris and
Quade (1992) is our approach to multiple sample settings.
Although they concentrate on single degree of freedom
methods, Harris and Quade point out, quite correctly, that
their MIDS criterion may produce values of n in an ANOVA
setting that are larger than necessary for specific (i.e., a pri-
ori) contrasts among means, and they recommend methods
for reducing n until 50% power is achieved for those con-
(18) trasts and the chosen method of familywise control of a
(p. 40).
The present paper gives simple formulas for tests of mean
contrasts by both Scheffks and Tukeys methods if an a
priori approach is taken; these formulas may readily be ex-
tended to other criteria for significance. If, however, the
more traditional a posteriori approach is appropriate, then
we recommend setting n according to (14) or its approxi-
mation. Under the sample Tang Condition, if the omnibus
F test is significant, then at least one test of pairwise mean
differences will also be significant by both Scheffks and
Tukey s methods.
Willans (1994) alternative approach for choosing n in
clinical management trials represents yet another departure
The American Statistician, November 1997, Vol. 51, No. 4 315
 from the single degrees of freedom work previously cited.
As the author states (p. 212), the objective of manage-
ment trials is aimed at deciding which treatment should be
used[,] as opposed to explanatory trials, which are con-
ducted to determine whether a difference in treatments ex-
ists at all. The management trial approach is in the tradi-
tion of Wald (1947), in which one desires strong evidence
for one or the other of two distinct hypotheses. As such,
Willans S (for his symmetric case, p. 215) is half the
value used in the present paper. Substituting this value into
the present formulas (and recognizing that his n is the total
number of observations, not n per sample) leads to Willans
equation (6). Blind use of the present formulas in the man-
agement trial setting would lead to dramatic underestimates
(by a factor of 8) relative to Willans criterion. The ap-
proaches taken by all the other works cited in the present
paper, as well as our approach, lead to values that would
be appropriate for explanatory trials only, without such ad-
justments.
8. SUMMARY AND DISCUSSION
Downloaded by [141.214.17.222] at 20:16 22 October 2014
differ by 6, or more, under any pattern of differences in
the population means. The consulting statistician can tell
clients, I have determined a sample size such that if some-
thing important happens in your experiment (as measured
by d, relative to S,), it will be statistically significant.
Finally, the approximate solutions, based on the central nor-
mal, chi-square, and range distributions, have closed form
solutions that are trivial to compute; even the iterative ex-
act solutions are easier to find than those involving noncen-
tral distributions. This simplicity makes the present method
easier to present to students or clients, and may prove ap-
pealing to those who need such formulas only occasionally.
With a little reflection it is easy to see that the consonance
criterion may be used for determining n in other situations.
For instance, main effect and cell means tests in multifac-
tor designs may be handled by substituting the appropriate
degrees of freedom in Equations (14) and (17) or their ap-
proximations. Although derived from a different standpoint,
Harris and Quades (1991) formulas for Pearsons T and x2
are also such extensions, which may be further generalized.
As an illustration, suppose one wanted to test the multiple

The investigations that led to the consonance criterion correlation coefficient for relating several independent vari-
were inspired by a recent paper in which Schmidt (1992) ables to some dependent variable. The consonance criterion
discussed situations in which the values of n were such that sample size in this situation is the value of n that guaran-
the sample estimate of S had to exceed 6, by 20% or more tees the rejection of Ho: p 2 = 0 if the (squared) sample
in order for HO to be rejected. As previously stated, such multiple correlation exceeds some critical size, say pz. It is
results can be vexing (embarrassing?)to both the researcher easy to show that the exact implicit equation for the conso-
and the statistical consultant. The consonance criterion was nance sample size and the (known-variance) approximation
proposed because it determines a value of n such that re- are, respectively,
jection of HO is assured whenever the largest sample effect 2
size d, equals or exceeds some critical effect size 6,. For- n = P F l - - a ; p ; n - p - l ( l - P,) + p + l
mulas for determining this value of n were proposed and P:
used to construct tables of sample sizes and minimal tradi- and
tional power for simple experiments. A striking feature of
the sample size table is how large these sample sizes are
for effect sizes as small as .lo, and how small they are for
effect sizes as large as 1.0. This range of values emphasizes where p is the number of independent (predictor) variables.
an observation made not long ago by Tukey (1986): For example, given five independent variables, and given
that a (squared) multiple correlation of .4is considered im-
With a reasonable amount of data, things of size 50 are nearly trivial to portant, the exact formula yieldsa sample size of n = 58,
find-anyone should be able to find them-whereas things of 0 . 0 5 ~ can with an approximate sample size of 56.
be nearly impossible to find, once we face the presence of systematic error Cohen (1992) recounts his seemingly futile efforts over
as well as those very nice errors whose effects come down like cr/J;E
(P. 76).
three decades to make sample size selection easier for the
everyday research worker, through simpler (and fewer) for-
A double bonus of the generalizations to multiple sample mulas and improved tables. Like him, we hope that our
situations is that choosing n for the ANOVAs omnibus F proposed methods will help to reverse the negative answer
test assures rejection of at least one a posteriori pairwise to Sedlmeier and Gigerenzers (1 989) title question.
mean comparison, by either Scheffks or Tukeys method.
One question that will surely arise in connection with [Received September 1993. Revised September 1996.1
the consonance criterion is, What advantage does this cri-
terion for determining n have over older methods in which REFERENCES
one specifies values of a, S,, and power, and then calculates Cohen, J. (1988), Statistical Power Analysis for the Behavioral Sciences
sample size based on some assumed pattern of differences (2nd ed.), New York: Academic Press.
(19921, A Power Primer, Psychological Bulletin, 112, 155-159.
among the population treatment means? For one thing a Gabriel, K. R. (1969), Simultaneous Test Procedures-Some Theory of
value of power for a particular configuration of unknown Multiple Comparisons, Annals of Mathematical Statistics, 40,224-250.
population means does not have to be specified in order to Guenther, W. G. (1981), Sample Size Formulas for Normal Theory T
do the calculations. Second, the sample sizes given by the Tests, The American Statistician, 35, 243-244.
Harris, R. J., and Quade, D. (1992), The Minimally Important Difference
consonance criterion deliver their promise of certain rejec- SignificantCriterion for Sample Size, Journal of Educational Statistics,
tion of HO whenever one or more pairs of sample means 17, 27-49.

316 General
 Hochberg, Y., and Tamhane, A. C. (19871, Multiple Comparison Proce-
dures, New York: Wiley.
Kupper, L. L., and Hafner, K. B. (1989), How Appropriate are Popular
Sample Size Formulas?, The American Statistician, 43, 101-105.
Pearson, E. S., and Hartley, H. 0. (19511, Charts of the Power Function
of the Analysis of Variance Tests, Derived from the Non-Central F -
Distribution, Biometrika, 38, 112-1 30.
Scheffk, H. A. (1959), The Analysis of Variance, New York: Wiley.
Schmidt, F. L. (1992), What do Data Really Mean?, Anrerican Psychol-
ogist, 47, 1173-1 180.
Sedlmeier, P., and Gigerenzer, G. (1989), Do Studies of Statistical Power
have an Effect on the Power of Studies?, Psychological Bulletin, 105,
Downloaded by [141.214.17.222] at 20:16 22 October 2014
309-3 16.
Steel, R. G. D., and Torrie, J. H. (1960), Principles and Procedures of
Statistics, New York: McGraw-Hill.
Stein, C. (1945), A Two-Sample Test for a Linear Hypothesis Whose
Power is Independent of Variance, Annals of Mathematical Statistics,
16,243-258.
Tang, P. C. (1938), unpublished manuscript.
Tukey, J. W. (1986), Sunset Salvo, The American Statistician, 40, 72-76.
Wald, A. (1947), Sequential Analysis (1973 reprint by Dover Publications,
New York).
Willan, A. R. (1994), Alternative Approach for Analyzing Management
Trials, Controlled Clinical Trials, 15, 21 1-219.
The American Statistician, November 1997, Vol. 51, No. 4 317