BAHADAR SHAH CHAIRMAN,DEPARTMENT OF MANAGEMENT SCIENCES

SAMPLING: A Scientific Method of Data Collection

SAMPLE
It is a Unit that selected from population Representers of the population

Purpose to draw the inference

Very

difficult to study each and every unit of the population when population unit are heterogeneous

Time Constraints Finance

The population having significant variations (Heterogeneous), observation of multiple individual needed to find all possible characteristics that may exist

Population The entire group of people of interest from whom the researcher needs to obtain information Element (sampling unit) One unit from a population Sampling The selection of a subset of the population through various sampling techniques Sampling Frame Listing of population from which a sample is chosen. The sampling frame for any probability sample is a complete list of all the cases in the population from which your sample will be drown

Parameter
The variable of interest

Statistic
The information obtained from the sample about the parameter

Population Vs. Sample

Population of Interest

Population Sample Parameter

Sample

Statistic

We measure the sample using statistics in order to draw inferences about the population and its parameters.

Universe
Census Sample Population Sample Frame

Elements

Representative

Accessible
Low cost

SAMPLING

What you want to talk about

Population

What you actually observe in the data

Sampling Process Sampling Frame

Sample

Inference

Define the population

Identify the sampling frame

Select a

sampling design or procedure

Determine the sample size Draw the sample

Sampling Design Process

Define Population
Determine Sampling Frame

Determine Sampling Procedure Probability Sampling Simple Random Sampling Stratified Sampling Cluster Sampling Systematic Sampling Multistage Sampling Determine Appropriate Sample Size Execute Sampling Design
Non-Probability Sampling

Convenient Judgmental Quota Snow ball Sampling

Classification of Sampling Methods

Sampling Methods

Probability Samples
Systematic Stratified Multistage Cluster Simple Random

Nonprobability
Convenience Snowball

Judgment

Quota

Each and every unit of the population has the equal chance for selection as a sampling unit Also called formal sampling or random sampling

Probability samples are more accurate

Probability samples allow us to estimate the accuracy of the sample

Simple Random Sampling Stratified Sampling Cluster Sampling

Systematic Sampling
Multistage Sampling

Simple Random Sampling

The

purest form of probability sampling

Assures

each element in the population has an equal chance of being included in the sample number generators

Random

Simple random sampling

Types of Simple Random Sample

With replacement

Without replacement

With replacement
The unit once selected has the chance for again selection

Without replacement
The unit once selected can not be selected again

 Tippet method

Lottery Method Random Table

684257954125632140 582032154785962024 362333254789120325 985263017424503686

 Minimal knowledge of population needed External

validity high; internal validity high; statistical estimation of error

Easy to analyze data

Disadvantage
High cost; low frequency of use
Requires sampling frame Does not use researchers expertise Larger risk of random error than stratified

Population is divided into two or more groups called strata, according to some criterion, such as geographic location, grade level, age, or income, and subsamples are randomly selected from each strata.

Elements within each strata are homogeneous, but are heterogeneous across strata

Stratified Random Sampling

Types of Stratified Random Sampling

Proportionate Stratified Random Sampling

Equal proportion of sample unit are selected from each strata

Disproportionate Stratified Random Sampling

Also called as equal allocation technique and sample unit decided according to analytical consideration

Advantage
Assures

representation of all groups in sample population needed of each stratum can be estimated and comparisons made

Characteristics

Reduces variability from systematic

Disadvantage
Requires accurate information on proportions

of each stratum

Stratified lists costly to prepare

Cluster Sampling

Section 1

Section 2

Section 3

Section 5 Section 4

Advantage

Low cost/high frequency of use

Requires list of all clusters, but only of individuals within chosen clusters Can estimate characteristics of both cluster and population For multistage, has strengths of used methods Researchers lack a good sampling frame for a dispersed population

Disadvantage
The cost to reach an element to sample is very high Usually less expensive than SRS but not as accurate

Each stage in cluster sampling introduces sampling error the more stages there are, the more error there tends to be

Systematic Random Sampling

Order all units in the sampling frame based on some variable and then every nth number on the list is selected Gaps between elements are equal and Constant There is periodicity. N= Sampling Interval

Systematic Random Sampling

Advantage
Moderate cost; moderate usage

External

validity high; internal validity high; statistical estimation of error

Simple to draw sample; easy to verify

 Periodic ordering Requires sampling frame

Multistage Random Sampling

Prim a ry Cl u s te r s 1 2 3 4 5 6 7 8 9 10

S e co n d a ry Cl u s te r s 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 S i m p l e R a n d o m S a m p l i n g w i th i n S e c o n d a r y

Select all schools; then sample within schools Sample schools; then measure all students Sample schools; then sample students

Non Probability Sampling

 Involves non random methods in selection of sample All have not equal chance of being selected Selection depend upon situation Considerably less expensive Convenient Sample chosen in many ways

 Purposive Sampling Quota sampling (larger populations) Snowball sampling

Self-selection sampling
Convenience sampling

Purposive Sampling
Also called judgment Sampling
The sampling procedure in which an experienced research selects the sample based on some appropriate characteristic of sample members to serve a purpose When taking sample reject, people who do not fit for a particular profile Start with a purpose in mind

Advantage

Demerit
Bias selection of sample may occur Time consuming process

Quota Sampling
The population is divided into cells on the basis of relevant control characteristics.

A quota of sample units is established for each cell

A convenience sample is drawn for each cell until the quota is met It is entirely non random and it is normally used for interview surveys

Used when research budget limited Very extensively used/understood No need for list of population elements Introduces some elements of stratification

Demerit
Variability and bias cannot be measured or controlled Time Consuming Projecting data beyond sample not justified

The research starts with a key person and introduce the next one to become a chain

Make contact with one or two cases in the population

Ask these cases to identify further cases. Stop when either no new cases are given or the sample is as large as manageable

low cost Useful in specific circumstances Useful for locating rare populations

Demerit

Bias because sampling units not independent

Projecting data beyond sample not justified

It occurs when you allow each case usually individuals, to identify their desire to take part in the research you therefore Publicize your need for cases, either by advertising through appropriate media or by asking them to take part Collect data from those who respond

More accurate Useful in specific circumstances to serve the purpose

Demerit

More costly due to Advertizing

Mass are left

Convenience Sampling
Called as Accidental / Incidental Sampling Selecting haphazardly those cases that are easiest to obtain Sample most available are chosen It is done at the convenience of the researcher

Merit
Very low cost Extensively used/understood No need for list of population elements

Demerit

Variability and bias cannot be measured or controlled Projecting data beyond sample not justified

Restriction of Generalization

The ever increasing demand for research has created a need for an efficient method of determining the sample size needed to be representative of a given population. In the article Small Sample Techniques, the research division of the National Education Association has published a formula for determining sample size. Regrettably a table has not bee available for ready, easy reference which could have been constructed using the following formula.

s = XNP(1 P) d (N 1) + XP(1 P). s = required sample size. X = the table value of chi-square for 1 degree of freedom at the desired confidence level (3.841). N = the population size. P = the population proportion (assumed to be .50 since this would provide the maximum sample size). d = the degree of accuracy expressed as a proportion (.05).

No calculations are needed to use Table 1. For example, one may wish to know the sample size required to be representative of the opinions of 9000 high school teachers relative to merit pay increases. To obtain the required sample size enter Table 1 at N = 9000. The sample size representative of the teachers in this example is 368. Table 1 is applicable to any defined population. The relationship between sample size and total population is illustrated in Figure 1. It should be noted that as the population increases the sample size increases at a diminishing rate and remains relatively constant at slightly more than 380 cases.

N
10 15 20 25 30

S
10 14 19 24 28

N
220 230 240 250 260

S
140 144 148 152 155

N
1200 1300 1400 1500 1600

S
291 297 302 306 310

35
40 45 50 55 60 65 70 75 80 85 90 95 100 110 120 130 140 150 160 170 180 190

32
36 40 44 48 52 56 59 63 66 70 73 76 80 86 92 97 103 108 113 118 123 127

270
280 290 300 320 340 360 380 400 420 440 60 480 500 550 600 650 700 750 800 850 900 950

159
162 165 169 175 181 186 191 196 201 205 210 214 217 226 234 242 248 254 260 265 269 274

1700
1800 1900 2000 2200 2400 2600 2800 3000 3500 4000 4500 5000 6000 7000 8000 9000 10000 15000 20000 30000 40000 50000

313
317 320 322 327 331 335 338 341 346 351 354 357 361 364 367 368 370 375 377 379 380 381

200
210

132
136

1000
1100

278
285

75000
1000000

382
384

In addition to the purpose of the study and population size, three criteria usually will need to be specified to determine the appropriate sample size: the level of precision, the level of confidence or risk, and the degree of variability in the attributes being measured (Miaoulis and Michener, 1976). Each of these is reviewed below.

The level of precision, sometimes called sampling error, is the range in which the true value of the population is estimated to be. This range is often expressed in percentage points (e.g., 5 percent) in the same way that results for political campaign polls are reported by the media. Thus, if a researcher finds that 60% of farmers in the sample have adopted a recommended practice with a precision rate of 5%, then he or she can conclude that between 55% and 65% of farmers in the population have adopted the practice.

The confidence or risk level is based on ideas encompassed under the Central Limit Theorem. The key idea encompassed in the Central Limit Theorem is that when a population is repeatedly sampled, the average value of the attribute obtained by those samples is equal to the true population value. Furthermore, the values obtained by these samples are distributed normally about the true value, with some samples having a higher value and some obtaining a lower score than the true population value. In a normal distribution, approximately 95% of the sample values are within two standard deviations of the true population value (e.g., mean). In other words, this means that if a 95% confidence level is selected, 95 out of 100 samples will have the true population value within the range of precision specified earlier (Figure 1). There is always a chance that the sample you obtain does not represent the true population value. Such samples with extreme values are represented by the shaded areas in Figure 1. This risk is reduced for 99% confidence levels and increased for 90% (or lower) confidence levels.

The third criterion, the degree of variability in the attributes being measured, refers to the distribution of attributes in the population. The more heterogeneous a population, the larger the sample size required to obtain a given level of precision. The less variable (more homogeneous) a population, the smaller the sample size. Note that a proportion of 50% indicates a greater level of variability than either 20% or 80%. This is because 20% and 80% indicate that a large majority do not or do, respectively, have the attribute of interest. Because a proportion of .5 indicates the maximum variability in a population, it is often used in determining a more conservative sample size, that is, the sample size may be larger than if the true variability of the population attribute were used.

There are several approaches to determining the sample size. These include using a census for small populations, imitating a sample size of similar studies, using published tables, and applying formulas to calculate a sample size. Each strategy is discussed below.

1. Using a Census for Small Populations One approach is to use the entire population as the sample. Although cost considerations make this impossible for large populations, a census is attractive for small populations (e.g., 200 or less). A census eliminates sampling error and provides data on all the individuals in the population. In addition, some costs such as questionnaire design and developing the sampling frame are fixed, that is, they will be the same for samples of 50 or 200. Finally, virtually the entire population would have to be sampled in small populations to achieve a desirable level of precision. 2. Using a Sample Size of a Similar Study Another approach is to use the same sample size as those of studies similar to the one you plan. Without reviewing the procedures employed in these studies you may run the risk of repeating errors that were made in determining the sample size for another study. However, a review of the literature in your discipline can provide guidance about typical sample sizes that are used.

3. Using Published Tables A third way to determine sample size is to rely on published tables, which provide the sample size for a given set of criteria. Table 1 and Table 2 present sample sizes that would be necessary for given combinations of precision, confidence levels, and variability. Please note two things. First, these sample sizes reflect the number of obtained responses and not necessarily the number of surveys mailed or interviews planned (this number is often increased to compensate for nonresponse). Second, the sample sizes in Table 2 presume that the attributes being measured are distributed normally or nearly so. If this assumption cannot be met, then the entire population may need to be surveyed.

Size of Population
500 600 700 800 900 1,000 2,000 3,000 4,000 5,000 a a a a a a 714 811 870 909

Sample Size (n) for Precision (e) of:

3% 222 240 255 267 277 286 333 353 364 370 5% 145 152 158 163 166 169 185 191 194 196 7% 83 86 88 89 90 91 95 97 98 98 10%

6,000
7,000 8,000 9,000 10,000 15,000 20,000 25,000 50,000 100,000 >100,000

938
959 976 989 1,000 1,034 1,053 1,064 1,087

375
378 381 383 385 390 392 394 397 1,099 1,111

197
198 199 200 200 201 204 204 204 398 400

98
99 99 99 99 99 100 100 100 204 204 100 100

a = Assumption of normal population is poor (Yamane, 1967). The entire population should be sampled.

Size of Population 100 125 150 175 200 225 250 275 300 325 350 375 400 425 450

5%

Sample Size (n) for Precision (e) of: 7% 10% 81 67 96 78 110 86 122 94 134 101 144 107 154 112 163 117 172 121 180 125 187 129 194 132 201 135 207 138 212 140

51 56 61 64 67 70 72 74 76 77 78 80 81 82 82

4. Using Formulas to Calculate a Sample Size Although tables can provide a useful guide for determining the sample size, you may need to calculate the necessary sample size for a different combination of levels of precision, confidence, and variability. The fourth approach to determining sample size is the application of one of several formulas (Equation 5 was used to calculate the sample sizes in Table 1 and Table 2 ).

A. Formula For Calcul ating A Sample For Proportions For populations that are large, Cochran (1963:75) developed the Equation 1 to yield a epresentative sample for proportions. Zpq n = ---------e Which is valid where n0 is the sample size, Z2 is the abscissa of the normal curve that cuts off an area at the tails (1 equals the desired confidence level, e.g., 95%)1, e is the desired level of precision, p is the estimated proportion of an attribute that is present in the population, and q is 1-p. The value for Z is found in statistical tables which contain the area under the normal curve. To illustrate, suppose we wish to evaluate a state-wide Extension program in which farmers were encouraged to adopt a new practice. Assume there is a large population but that we do not know the variability in the proportion that will adopt the practice; therefore, assume p=.5 (maximum variability). Furthermore, suppose we desire a 95% confidence level and 5% precision. The resulting sample size is demonstrated in Equation 2. Zpq (1.96) (.5) (.5) n = ---------- = ------------------------- = 385 farmers e (.5)

B. Finite Population Correct ion For Proportions If the population is small then the sample size can be reduced slightly. This is because a given sample size provides proportionately more information for a small population than for a large population. The sample size (n0) can be adjusted using Equation 3. n n = -------------------n-1 1+ ------------N Where n is the sample size and N is the population size. Suppose our evaluation of farmers adoption of the new practice only affected 2,000 farmers. The sample size that would now be necessary is shown in Equation 4. n 385 n = ---------------= -----------------= 323 farmers n-1 385-1 1+ ------------1 + ------------N 2000 As you can see, this adjustment (called the finite population correction) can substantially reduce the necessary sample size for small populations.

A Simplified Formula For Proportions Yamane (1967:886) provides a simplified formula to calculate sample sizes. This formula was used to calculate the sample sizes in Tables 2 and 3 and is shown below. A 95% confidence level and P = .5 are assumed for Equation 5. N n= -------------1+ (e) Where n is the sample size, N is the population size, and e is the level of precision. When this formula is applied to the above sample, we get Equation 6. N 2000 n= ----------- = -------------- = 333 1+ (e) 1+2000 (.5)

Formula For Sample Size For The Mean The use of tables and formulas to determine sample size in the above discussion employed proportions that assume a dichotomous response for the attributes being measured. There are two methods to determine sample size for variables that are polytomous or continuous. One method is to combine responses into two categories and then use a sample size based on proportion (Smith, 1983). The second method is to use the formula for the sample size for the mean. The formula of the sample size for the mean is similar to that of the proportion, except for the measure of variability. The formula for the mean employs 2 instead of (p x q), as shown in Equation 7. Z n= --------e Where n0 is the sample size, z is the abscissa of the normal curve that cuts off an area at the tails, e is the desired level of precision (in the same unit of measure as the variance), and 2 is the variance of an attribute in the population. The disadvantage of the sample size based on the mean is that a good estimate of the population variance is necessary. Often, an estimate is not available. Furthermore, the sample size can vary widely from one attribute to another because each is likely to have a different variance. Because of these problems, the sample size for the proportion is frequently preferred

In completing this discussion of determining sample size, there are three additional issues. First, the above approaches to determining sample size have assumed that a simple random sample is the sampling design. More complex designs, e.g., stratified random samples, must take into account the variances of subpopulations, strata, or clusters before an estimate of the variability in the population as a whole can be made. Another consideration with sample size is the number needed for the data analysis. If descriptive statistics are to be used, e.g., mean, frequencies, then nearly any sample size will suffice. On the other hand, a good size sample, e.g., 200-500, is needed for multiple regression, analysis of covariance, or log-linear analysis, which might be performed for more rigorous state impact evaluations. The sample size should be appropriate for the analysis that is planned.

In addition, an adjustment in the sample size may be needed to accommodate a comparative analysis of subgroups (e.g., such as an evaluation of program participants with nonparticipants). Sudman (1976) suggests that a minimum of 100 elements is needed for each major group or subgroup in the sample and for each minor subgroup, a sample of 20 to 50 elements is necessary. Similarly, Kish (1965) says that 30 to 200 elements are sufficient when the attribute is present 20 to 80 percent of the time (i.e., the distribution approaches normality). On the other hand, skewed distributions can result in serious departures from normality even for moderate size samples (Kish, 1965:17). Then a larger sample or a census is required. Finally, the sample size formulas provide the number of responses that need to be obtained. Many researchers commonly add 10% to the sample size to compensate for persons that the researcher is unable to contact. The sample size also is often increased by 30% to compensate for nonresponse. Thus, the number of mailed surveys or planned interviews can be substantially larger than the number required for a desired level of confidence and precision.

A hypothesis is a kind of truth claim about some aspect of the world: for instance, the attitudes of patients or the prevalence of

a disease in a population. Research sets out to try to prove this truth claim (or, more properly, to reject the null hypothesis - a truth claim phrased as a negative). For example, let us think about the following hypothesis: Levels of Efficiency are affected by Satisfaction and the related null hypothesis: Levels of Efficiency are not affected by Satisfaction Let us imagine that we have this as our research hypothesis, and we are planning research to test it. We will undertake a trial, comparing groups of employees who are working in different organization, to assess the extent of efficiency in these different groupings. Obviously the findings of a study -- while interesting in themselves -- only have value if they can be generalised, to discover something about the topic which can be applied in other organizations. If we find an association, then we will want to do something to increase efficiency (by increasing satisfaction). So our study has to have external validity, that is, the capacity to be generalised beyond the subjects actually in the study

Sample Errors Non Sample Errors

Error caused by the act of taking a sample

They cause sample results to be different from the results of census Differences between the sample and the population that exist only because of the observations that happened to be selected for the sample Statistical Errors are sample error We have no control over

Not Control by Sample Size

Non Response Error Response Error

A non-response error occurs when units selected as part of the sampling procedure do not respond in whole or in part

A response or data error is any systematic bias that occurs during data collection, analysis or interpretation
Respondent error (e.g., lying, forgetting, etc.) Interviewer bias Recording errors Poorly designed questionnaires Measurement error

 respondent gives an incorrect answer, e.g. due to prestige or competence implications, or due to sensitivity or social undesirability of question respondent misunderstands the requirements lack of motivation to give an accurate answer lazy respondent gives an average answer question requires memory/recall proxy respondents are used, i.e. taking answers from someone other than the respondent

 Different interviewers administer a survey in different ways Differences occur in reactions of respondents to different interviewers, e.g. to interviewers of their own sex or own ethnic group Inadequate training of interviewers Inadequate attention to the selection of interviewers There is too high a workload for the interviewer

 The question is unclear, ambiguous or difficult to answer

The list of possible answers suggested in the recording instrument is incomplete Requested information assumes a framework unfamiliar to the respondent The definitions used by the survey are different from those used by the respondent (e.g. how many part-time employees do you have? See next slide for an example)

Non-sampling errors are inevitable in production of national statistics. Important that: At planning stage, all potential non-sampling errors are listed and steps taken to minimise them are considered. If data are collected from other sources, question procedures adopted for data collection, and data verification at each step of the data chain. Critically view the data collected and attempt to resolve queries immediately they arise. Document sources of non-sampling errors so that results presented can be interpreted meaningfully.

What any researcher wants is to be right! They want to discover that there is an association between two variables: say, asthma and traffic pollution, but only if such an association really exists.

If there is no such association, then they want their study to

support the null hypothesis that the two are not related. (While the former may be more exciting, both are important findings). What no researcher wants is to be wrong! No-one wants to find an association which does not really exist, or - just as importantly - not find an association which does exist. Both such situations can arise in any piece of research. The first (finding an association which is not really there) is called a Type I

error. It is the error of falsely rejecting a true null hypothesis.

(Think through this carefully. What we are talking about here could also be called a false positive. An example would be a

study which rejects the null hypothesis that there is no

association between ill-health and deprivation. The findings suggest such an association, but in reality, no such relationship exists.)

The measurement of such generalisability of a study is done by statistical tests of inference. You may be familiar with some such tests: tests such as the chi-squared test, the t-test, and tests of correlation. We will not look at these tests in any detail, but we need to understand that the purpose of these and other tests of statistical inference is to assess the extent to which the findings of a study can be accepted as valid for the population from which the study sample has been drawn. If the statistics we use suggest that the findings are 'true', then we can be happy to conclude (within certain limits of probability), that the study's findings can be generalised, and we can act on them (to improve nutrition among children under five years, for instance). From common sense, we see that the larger the sample is, the easier it is to be satisfied that it is representative of the population from which it is drawn: but how large does it need to be? This is the question that we need to answer, and to do so, we need to think a little more about the possibilities that our findings may not reflect reality: that we have committed an error in our conclusions.

The second kind of error, called a Type 2 error (usually written as Type II), occurs when a study fails to find an association which really does exist. It is then a matter of

wrongly accepting a false null hypothesis. (This is a false negative: using the ill-health and deprivation example again,

we conduct a study and find no association, missing one which really does exist.) Both types of error are serious. Both have consequences: imagine the one which might be spent on reducing traffic pollution, and all the time it does not really affect asthma (a Type I error). Or imagine allowing traffic pollution to continue, while it really is affecting children's health (a Type II error). Good research will minimise the chances of committing both Type I and Type II errors as far as possible, although they can never be ruled out absolutely.

For any piece of research that tries to make inferences from a sample to a population there are four possible outcomes: two are desirable, two render the research worthless. Figure 1 shows these four possible outcomes diagrammatically.
Null Hypothesis is False POPULATION False Cell 1 Current Result Cell 3 Type II error (beta) True Cell 2 Type 1 Error (Alpha) Cell 4 Current Result

True

Cell 2. In this cell, as in cell 1, the study results reject the null hypothesis, indicating some kind of association between the variables of deprivation and health. However, these study results are false, because in the population from which we drew our sample the null hypothesis is actually true and there is no such association. This is the Type I error: the error of rejecting a true null hypothesis. The likelihood of committing a Type I error (finding an association which does not really exist) is known as the alpha (a) value or the statistical significance of a statistical test. Some of you may be familiar with a as p, the quoted level of significance of a test. The p value marks the probability of committing a Type I error; thus a p value of 0.05 (a widely used conventional level of significance) indicates a five per cent or one in 20 -- chance of committing a Type I error. Cell 2 thus reflects an incorrect finding from a study, and the a value represents the probability of this occurring.

Cell 3. This cell similarly reflects an undesirable outcome of a study. Here, as in Cell 4, a study supports the null hypothesis, implying that there is no association between ill health and deprivation in the population under investigation. But in reality, the null hypothesis is false and there is an association in the real world which the study does not find. This mistake is the Type II error of accepting a false null hypothesis. and is the result of having a sample size which is too small to allow detection of the association by statistical tests at an acceptable level of significance (say p = 0.05). The likelihood of committing a Type II error is the beta () value of a statistical test, and the value (1 - ) is the statistical power of the test. Thus, the statistical power of a test is

the likelihood of avoiding a Type II error i.e. the probability that the test will reject the null hypothesis when the null hypothesis is false.

Conventionally, a value of 0.80 or 80% is the target value for statistical power, representing a likelihood that four times out of five a study will reject a false null hypothesis, although values greater than 80% e.g. 90% are also sometimes used. Outcomes of studies which fall into cell 3 are incorrect; or its complement (1-) are the measures of the likelihood of such an outcome of a study.

All research should seek to avoid both Type I and Type II errors, which lead to incorrect inferences about the world beyond the study. In practice, there is a trade-off. Reducing the likelihood of committing a Type I error by increasing the level of significance at which one is willing to accept a positive finding reduces the statistical power of the test, thus increasing the possibility of a Type II error (missing an association which exists). Conversely, if a researcher makes it a priority to avoid committing a Type II error, it becomes more likely that a Type I error will occur (finding an association which does not exist).

Not all quantitative studies involve hypothesis-testing, some studies merely seek to describe the phenomena under examination. Whereas hypothesis testing will involve comparing the characteristics of two or more groups, a descriptive survey may be concerned solely with describing the characteristics of a single group. The aim of this type of survey is often to obtain an accurate estimate of a particular figure, such as a mean or a proportion. For example, we may want to know how many times, in an average week, that a general practitioner sees patients newly presenting with asthma. In addition we may also want to know what proportion of these patients admit to smoking five or more cigarettes a day. In these circumstances, the aim is not to compare this figure with another group, but rather, to accurately reflect the real figure in the wider population.

To calculate the required sample size in this situation, there are certain things that we need to establish. We need to know: 1. The level of confidence we require concerning the true value of a mean or proportion. This is closely connected with the level of significance for statistical tests, such as a t-test. For example, we can be 95% confident that the true mean value lies somewhere within a valid 95% confidence interval, and this corresponds to significance testing at the 5% level (P < 0.05) of significance. Likewise, we can be 99% confident that the true mean value lies somewhere within a valid 99% confidence interval (which is a bit wider), and this corresponds to significance testing at the 1% level (P < 0.01) of significance.

2. The degree of precision which we can accept. This is often

presented in the form of a confidence interval. For example, a survey of a sample of patients indicates that 35 per cent smoke.

Are we willing to accept that the figure for the wider population lies between 25 and 45 per cent, (allowing a margin for random error (MRE) of 10% either way), or do we want to be more precise, such that the confidence interval is three per cent each way, and the true figure falls between 32 and 38 per cent? As we can see from the following table, the smaller the allowed margin for random error, the larger the sample must be.

Margin for random error + or 10% + or 5% + or 3% + or 2% + or 1%

Sample size 88 350 971 2188 8750

How large must a sample be to estimate the mean value of the population? Suppose we wish to measure the number of times that the average patient with asthma consults her/his general practitioner for treatment? a) First, the SE (standard error) is calculated by deciding upon the accuracy level which you require. If, for instance, you wish your survey to produce a very accurate answer with only a small confidence interval, then you might decide that you want to be 95% confident that the mean average figure produced by your survey is no more than plus or minus two visits to the GP. For example, if you thought that your survey might produce a mean estimate of 12.5 visits per year, then your confidence interval in this case would be 12.5 two visits. Your confidence interval would then tell you that you could reasonably (more detail on what reasonably means below!) expect the true average rate of visits in the population to be somewhere between 10.5 and 14.5 visits per year. Now decide on your required significance level. If you decide on 95%, (meaning that 19 times out of 20 the true population mean falls within the confidence limit of 10.5 and 14.5 visits), the standard error is calculated by dividing the MRE by 1.96. So, in this case, the standard error is 2 divided by 1.96 = 1.02. If you want a 95% confidence interval, then divide the maximum acceptable MRE (margin for random error) by 1.96 to calculate the SE. If instead you want a 99% confidence interval, then divide the maximum

b) The formula to calculate the sample size for a mean (or point) estimate is: N= SE 2 SD where N = the required sample size, SD = the stand ard deviation, and SE = the standard error of the mean The standard deviation could be estimated either by looking at some previous study or by carrying out a pilot study. Suppose that previous data showed that the standard deviation of the number of visits made to a GP in a year was 10, then we would input this into the formula as follows: N= SE = 10 = ( 9.8) = 96.12 = 97 (rounded to nearest patient) SD 1.02 If we are to be 95% confident that the answer achieved is correct two visits, then the required sample is 97 - before making allowance for a proportion of the people leaving the study early and failing to provide outcome data.

Suppose that we were interested in finding out what percentage of the local patient population were satisfied with the service they had received from their GP over the previous 12 months. We want to carry out a survey, but of how many people? Once again we need to know: The confidence level (usually 95% or 99%) The Confidence Interval we are willing to accept, for example that our survey finding lies within plus or minus five per cent of the population figure. Assume that we decide that the precision with which we decide the proportion of respondents who say that they are satisfied with the service must be plus or minus 5%. This then is our confidence interval. To calculate the standard error, we divide the confidence interval by 1.96. In this case the standard error is 5/1.96 = 2.55. We also need to estimate the proportion which we expect to find who are satisfied. In order to estimate P (the estimated percentage) we should consult previous surveys or conduct a pilot. Assume, for the time being, that a similar survey carried out three years ago indicated that 70% of the respondents said they were satisfied. We then use the following formula:

N =

P(100% -P (SE) With P = 70% and SE=2.55, we have: N= 70% (100% 70%) = 2100 = 323.28 = 324 (rounded upwards) 2.55% 6.50 So, in order to be 95% confident that the true proportion of people saying they are satisfied lies within 5% of the answer, we will require a sample size of 324. This assumes that the likely answer is around 70% with a range between 65% and 75%. Of course, in real life, we often have absolutely no idea what the likely proportion is going to be. There may be no previous data and no time to carry out a pilot. In these circumstances, it is safer to assume the worst case scenario and assume that

the proportion is likely to be 50%. Other things being equal, this will allow for the largest possible sample size - and in most circumstances it is preferable to have a

slight overestimate of the number of people needed, rather than an underestimate. (If we wished to use a 99% level of significance, so we might be 99% confident that our confidence parameters include the true figure, then we need to divide the confidence interval by 2.56. In this case, the standard error would be 5/2.56 = 1.94. Using the formula above, we find that this would require a sample size of 558.)

As we saw earlier in this pack, studies which test hypotheses (seeking to generalise from a study to a population), need sufficient power to minimise the likelihood of Type I and Type II errors. Both statistical significance and statistical power are affected by sample size. The chances of gaining a statistically significant result will be increased by enlarging a study's sample. Put another way, the statistical power of a study is enhanced as sample size increases. Let us look at each of these aspects of inferential research in turn.

When a researcher uses a statistical test, what they are doing is testing their results against a gold standard. If the test gives a positive result (this is usually known as 'achieving statistical significance'), then they can be relatively satisfied that their results are 'true', and that the real world situation is that discovered in the study (Cell 1 in Fig 1). If the test does not give significant results (nonsignificant or NS), then they can be reasonably satisfied that the results reflect Cell 4, where they have found no association and no such association exists. However, we can never be absolutely certain that we have a result which falls in Cells 1 or 4. Statistical significance represents the likelihood of committing a Type I error (Cell 2).Let us imagine that we have results suggesting an association between ill-health and deprivation, and a t-test (a test to compare the results of two different groups) gives a value which indicates that at the 5% or 0.05 level of statistical significance, there is more ill-health among a group of high scorers on the Jarman Index of deprivation than among a group of low scorers. What this means is that 95 per cent of the time, we can be certain that this result reflects a true effect (Cell 1). Five per cent of the time, it is a chance result, resulting from random associations in the sample we chose. If the t-test value is higher, we might reach 1% or 0.01 significance. Now, the result will only be a chance association one per cent of the time . Tests of statistical significance are designed to account for sample size, thus the larger a sample; the 'easier' it is for results to reach significance. A study which compares two groups of 10 patients will have to demonstrate a much greater difference between the groups than a study with 1000 patients in each group. This is fair: the larger study is much more likely to be 'representative' of a population than the smaller one. To summarize: statistical significance is a measure of the likelihood that positive results reflect a real effect, and that the findings can be used to make conclusions about differences which really exist.

Because of the way statistical tests are designed, as we have just seen, they build in a safety margin to avoid generalising false positive results which could have disastrous or expensive consequences. But researchers who use small samples also run the risk of not being able to demonstrate differences or associations which really do exist. Thus they are in danger of committing a Type II error (Cell 3 in Fig 1), of accepting a false null hypothesis. Such studies are under-powered, not possessing sufficient statistical power to detect the effects they set out to detect. Conventionally, the target is a power of 80% or 0.8, meaning that a study has an 80 per cent likelihood of detecting a difference or association which really exists. Examination of research undertaken in various fields of study suggests that many studies do not meet this 0.8 conventional target for power (Fox and Mathers 1997). What this means is that many studies have a much reduced likelihood of being able to discern the effects which they set out to seek: a study, with a power of 0.66 for some specified treatment effect, will only detect that effect (if true) two times out of three. A non-significant finding of a study may thus simply reflect the

inadequate power of the study to detect differences or associations at levels which are conventionally accepted as statistically significant.

When a study has only small (say less than 50%) power to detect a useful result, one must ask the simple question of such research: Why did you bother, when your study had little chance of finding what you set out to find? Sample size calculations need to be undertaken prior to a study to avoid both the wasteful consequences of under-powering, (or of overpowering in which sample sizes are excessively large, with higher than necessary study costs and, perhaps, the needless involvement of too many patients, which has ethical implications.). Statistical power calculations are also sometimes undertaken after a study has been completed, to assess the likelihood of a study having discovered effects. Statistical power is a function of three variables: sample size, the chosen level of statistical significance (a) and effect size. While calculation of power entails recourse to tables of values for these variables, the calculation is relatively straightforward in most cases.

As was mentioned earlier, there is a trade-off between significance and power, because as one tries to reduce the chances of generating false negative results, the likelihood of a false positive result increases. Researchers need to decide which is more crucial, and set the significance level accordingly. In Exercise 3 you were asked to decide, in various situations, whether a Type I or Type II error was more serious - based on clinical and other criteria. Fortunately both statistical significance and power are increased by increasing sample size, so increasing sample size will reduce likelihoods of both Type I and Type II errors. However, that does not mean that researchers necessarily need to vastly increase the size of their samples, at great expense of time and resources. The other factor affecting the power of a study is the effect size (ES) which is under investigation in the study. This is a measure of how wrong the null hypothesis is. For example, we might compare the efficacy of two bronchodilators for treating an asthma attack. The ES is the difference in efficacy between the two drugs. An effect size may be a difference between groups or the strength of an association between variables such as ill-health and deprivation. If an ES is small, then many studies with small sample sizes are likely to be underpowered. But if an ES is large, then a relatively small scale study could have sufficient power to identify the effect under investigation. It is sometimes possible to increase the effect size (for example, by making more extreme comparisons, or undertaking a longer or more powerful intervention), but usually this is the intractable element in the equation, and accurate estimation of the effect size is essential for calculating power before a study begins, and hence the necessary sample size.

However, when critiquing business education research, Wunsch (1986) stated that two of the most consistent flaws included (1) is regard for sampling error when determining sample size, and (2) disregard for response and nonresponse bias (p. 31).

The question then is, how large of a sample is required to infer research findings back to a population? Standard textbook authors and researchers offer tested methods that allow studies to take full advantage of statistical measurements, which in turn give researchers the upper hand in determining the correct sample size. Sample size is one of the four interrelated features of a study design that can influence the detection of significant differences, relationships or interactions (Peers, 1996). Generally, these survey designs try to minimize both alpha error (finding a difference that does not actually exist in the population) and beta error (failing to find a difference that actually exists in the population) (Peers, 1996).

However, improvement is needed. Researchers are learning experimental statistics from highly competent statisticians and then doing their best to apply the formulas and approaches they learn to their research design. A simple survey of published manuscripts reveals numerous errors and questionable approaches to sample size selection, and serves as proof that improvement is needed. Many researchers could benefit from a real-life primer on the tools needed to properly conduct research, including, but not limited to, sample size selection.

Primary Variables of Measurement

The researcher must make decisions as to which variables will be incorporated into formula calculations. For example, if the researcher plans to use a seven-point scale to measure a continuous variable, e.g., job satisfaction, and also plans to determine if the respondents differ by certain categorical variables, e.g., gender, tenured, educational level, etc., which ariable(s) should be used as the basis for sample size? This is important because the use of gender as the primary variable will result in a substantially larger sample size than if one used the seven-point scale as the primary variable of measure. Cochran (1977) addressed this issue by stating that One method of determining sample size is to specify margins of error for the items that are regarded as most vital to the survey. An estimation of the sample size needed is first made separately for each of these important items (p. 81). When these calculations are completed, researchers will have a range of ns, usually ranging from smaller ns for scaled, continuous variables, to larger ns for dichotomous or categorical variables. The researcher should make sampling decisions based on these data. If the ns for the variables of interest are relatively close, the researcher can simply use the largest n as the sample size and be confident that the sample size will provide the desired results.

More commonly, there is a sufficient variation among the ns so that we are reluctant to choose the largest, either from budgetary considerations or because this will give an over-all standard of precision substantially higher than originally contemplated. In this event, the desired standard of precision may be relaxed for certain of the items, in order to permit the use of a smaller value of n (Cochran, 1977, p. 81). The researcher may also decide to use this information in deciding whether to keep all of the variables identified in the study. In some cases, the ns are so discordant that certain of them must be dropped from the inquiry; . . . (Cochran, 1977, p. 81)

Cochrans (1977) formula uses two key factors: (1) the risk the researcher is willing to accept in the study, commonly called the margin of error, or the error the researcher is willing to accept, and (2) the alpha level, the level of acceptable risk the researcher is willing to accept that the true margin of error exceeds the acceptable margin of error; i.e., the probability that differences revealed by statistical analyses really do not exist; also known as Type I error. Another type of error will not be addressed further here, namely, Type II error, also known as beta error. Type II error occurs when statistical procedures result in a judgment of no significant differences when these differences do indeed exist

Alpha Level. The alpha level used in determining sample size in most educational research studies is either .05 or .01 (Ary, Jacobs, & Razavieh, 1996). In Cochrans formula, the alpha level is incorporated into the formula by utilizing the t-value for the alpha level selected (e.g., t-value for alpha level of .05 is 1.96 for sample sizes above 120). Researchers should ensure they use the correct t- value when their research involves smaller populations, e.g., t-value for alpha of .05 and a population of 60 is 2.00. In general, an alpha level of .05 is acceptable for most research. An alpha level of .10 or lower may be used if the researcher is more interested in identifying marginal relationships, differences or other statistical phenomena as a precursor to further studies. An alpha level of .01 may be used in those cases where decisions based on the research are critical and errors may cause substantial financial or personal harm, e.g., major programmatic changes.

Acceptable Margin of Error. The general rule relative to acceptable margins of error in educational and social research is as follows: For categorical data, 5% margin of error is acceptable, and, for continuous data, 3% margin of error is acceptable (Krejcie & Morgan, 1970). For example, a 3% margin of error would result in the researcher being confident that the true mean of a seven point scale is within .21 (.03 times seven points on the scale) of the mean calculated from the research sample. For a dichotomous variable, a 5% margin of error would result in the researcher being confident that the proportion of respondents who were male was within 5% of the proportion calculated from the research sample. Researchers may increase these values when a higher margin of error is acceptable or may decrease these values when a higher degree of precision is needed.

A critical component of sample size formulas is the estimation of variance in the primary variables of interest in the study. The researcher does not have direct control over variance and must incorporate variance estimates into research design. Cochran (1977) listed four ways of estimating population variances for sample size determinations: (1) take the sample in two steps, and use the results of the first step to determine how many additional responses are needed to attain an appropriate sample size based on the variance observed in the first step data; (2) use pilot study results; (3) use data from previous studies of the same or a similar population; or (4) estimate or guess the structure of the population assisted by some logical mathematical results. The first three ways are logical and produce valid estimates of variance; therefore, they do not need to be discussed further. However, in many educational and social research studies, it is not feasible to use any of the first three ways and the researcher must estimate variance using the fourth method.

A researcher typically needs to estimate the variance of scaled and categorical variables. To estimate the variance of a scaled variable, one must determine the inclusive range of the scale, and then divide by the number of standard deviations that would include all possible values in the range, and then square this number. For example, if a researcher used a seven-point scale and given that six standard deviations (three to each side of the mean) would capture 98% of all responses, the calculations would be as follows: 7 (number of points on the scale) S = --------------------------------------------6 (number of standard deviations) When estimating the variance of a dichotomous (proportional) variable such as gender, Krejcie and Morgan (1970) recommended that researchers should use .50 as an estimate of the population proportion. This proportion will result in the maximization of variance, which will also produce the maximum sample size. This proportion can be used to estimate variance in the population. For example, squaring .50 will result in a population variance estimate of .25 for a dichotomous variable.

Before proceeding with sample size calculations, assuming continuous data, the researcher should determine if a categorical variable will play a primary role in data analysis. If so, the categorical sample size formulas should be used. If this is not the case, the sample size formulas for continuous data described in this section are appropriate. Assume that a researcher has set the alpha level a priori at .05, plans to use a seven point scale, has set the level of acceptable error at 3%, and has estimated the standard deviation of the scale as 1.167. Cochrans sample size formula for continuous data and an example of its use is presented here along with the explanations as to how these decisions were made. (t)2 * (s)2 (1.96)2(1.167)2 no = ------------- = -------------= 118 (d)2 (7*.03)2 Where t = value for selected alpha level of .025 in each tail = 1.96 (the alpha level of .05 indicates the level of risk the researcher is willing to take that true margin of error may exceed the acceptable margin of error.)

Where s = estimate of standard deviation in the population = 1.167. (estimate of variance deviation for 7 point scale calculated by using 7 [inclusive range of scale] divided by 6 [number of standard deviations that include almost all (approximately 98%) of the possible values in the range]). Where d = acceptable margin of error for mean being estimated = .21. (number of points on primary scale * acceptable margin of error; points on primary scale = 7; acceptable margin of error = .03 [error researcher is willing to except]). Therefore, for a population of 1,679, the required sample size is 118. However, since this sample size exceeds 5% of the population (1,679*.05=84), Cochrans (1977) correction formula should be used to calculate the final sample size. These calculations are as follows: no (118) n= -------------------- = ---------------= 111 (1 + no / Population) (1 + 118/1679) Where population size = 1, 679. Where n0 = required return sample size according to Cochrans formula= 118. Where n1 = required return sample size because sample > 5% of population.

These procedures result in the minimum returned sample size. If a researcher has a captive audience, this sample size may be attained easily. However, since many educational and social research studies often use data collection methods such as surveys and other voluntary participation methods, the response rates are typically well below 100%. Salkind (1997) recommended oversampling when he stated that If you are mailing out surveys or questionnaires, . . . . Count on increasing your sample size by 40%-50% toaccount for lost mail and uncooperative subjects (p. 107). Fink (1995) stated that Oversampling can add costs to the survey but is often necessary (p. 36). Cochran (1977) stated that A second consequence is, of course, that the variances of estimates are increased because the sample actually obtained is smaller than the target sample. This factor can be allowed for, at least approximately, in selecting the size of the sample (p. 396). However, many researchers criticize the use of over-sampling to ensure that this minimum sample size is achieved and suggestions on how to secure the minimal sample size are scarce.

If the researcher decides to use oversampling, four methods may be used to determine the anticipated response rate: (1) take the sample in two steps, and use the results of the first step to estimate how many additional responses may be expected from the second step; (2) use pilot study results; (3) use responses rates from previous studies of the same or a similar population; or (4) estimate the response rate. The first three ways are logical and will produce valid estimates of response rates; therefore, they do not need to be discussed further. Estimating response rates is not an exact science. A researcher may be able to consult other researchers or review the research literature in similar fields to determine the response rates that have been achieved with similar and, if necessary, dissimilar populations Therefore, in this example, it was anticipated that a response rate of 65% would be achieved based on prior research experience. Given a required minimum sample size (corrected) of 111, the following calculations were used to determine the drawn sample size required to produce the minimum sample size: Where anticipated return rate = 65%. Where n2 = sample size adjusted for response rate. Where minimum sample size (corrected) = 111. Therefore, n2 = 111/.65 = 171.

The sample size formulas and procedures used for categorical data are very similar, but some variations do exist. Assume a researcher has set the alpha level a priori at .05, plans to use a proportional variable, has set the level of acceptable error at 5%, and has estimated the standard deviation of the scale as .5. Cochrans sample size formula for categorical data and an example of its use is presented here along with explanations as to how these decisions were made. (t)2 * (p)(q) no= --------------------(d)2 (1.96)2(.5)(.5) no= ---------------------- = 384 (.05)2

Where t = value for selected alpha level of .025 in each tail = 1.96. (the alpha level of .05 indicates the level of risk the researcher is willing to take that true margin of error may exceed the acceptable margin of error). Where (p)(q) = estimate of variance = .25. (maximum possible proportion (.5) * 1maximum possible proportion (.5) produces maximum possible sample size). Where d = acceptable margin of error for proportion being estimated = .05 (error researcher is willing to except). Therefore, for a population of 1,679, the required sample size is 384. However, since this sample size exceeds 5% of the population (1,679*.05=84), Cochrans (1977) correction formula should be used to calculate the final sample size.

These calculations are as follows: no n1= --------------------- (1 + no / Population) (384) n1= --------------------- = 313 (1 + 384/1679) Where population size = 1,679 Where n0 = required return sample size according to Cochrans formula= 384 Where n1 = required return sample size because sample > 5% of population These procedures result in a minimum returned sample size of 313. Using the same oversampling procedures as cited in the continuous data example, and again assuming a response rate of 65%, a minimum drawn sample size of 482 should be used. These calculations were based on the following: Where anticipated return rate = 65%. Where n2 = sample size adjusted for response rate. Where minimum sample size (corrected) = 313. Therefore, n2 = 313/.65 = 482

Table 1 presents sample size values that will be appropriate for many common sampling problems. The table includes sample sizes for both continuous and categorical data assuming alpha levels of .10, .05, or .01. The margins of error used in the table were .03 for continuous data and .05 for 48 Bartlett, Kotrlik, & Higgins categorical data. Researchers may use this table if the margin of error shown is appropriate for their study; however, the appropriate sample size must be calculated if these error rates are not appropriate.

Situations exist where the procedures described in the previous paragraphs will not satisfy the needs of a study and two examples will be addressed here. One situation is when the researcher wishes to use multiple regression analysis in a study. To use multiple regression analysis, the ratio of observations to independent variables should not fall below five. If this minimum is not followed, there is a risk for overfitting, . . . making the results too specific to the sample, thus lacking generalizability (Hair, Anderson, Tatham, & Black, 1995, p. 105). A more conservative ratio, of ten observations for each independent variable was reported optimal by Miller and Kunce (1973) and Halinski and Feldt (1970). These ratios are especially critical in using regression analyses with continuous data because sample sizes for continuous data are typically much smaller than sample sizes for categorical data. Therefore, there is a possibility that the random sample will not be sufficient if multiple variables are used in the regression analysis. For example, in the continuous data illustration, a population of 1,679 was utilized and it was determined that a minimum returned sample size of 111 was required. The sample size for a population of 1,679 in the categorical data example was 313. Table 2, developed based on the recommendations cited in the previous paragraph, uses both the five to one and ten to one ratios.

Sample size
Population size 100 200 300 400 500 600 700 Continuous data (margin of error=.03)
alpha=.10 t=1.65 alpha=.05 t=1.96 alpha=.01 t=2.58

Categorical data (margin of error=.05)

p=.50 t=1.65 p=.50 t=1.96 p=.50 t=2.58

46 59 65 69 72 73 75

55 75 85 92 96 100 102

68 102 123 137 147 155 161

74 116 143 162 176 187 196

80 132 169 196 218 235 249

87 154 207 250 286 316 341

800
900 1000 1500 2000 4000 6000 8000 1000

76
76 77 79 83 83 83 83 83

104
105 106 110 112 119 119 119 119

166
170 173 183 189 198 209 209 209

203
209 213 230 239 254 259 262 264

260
270 278 306 323 351 362 367 370

363
382 399 461 499 570 598 613 623

NOTE: The margins of error used in the table were .03 for continuous data and .05 for categorical data. Researchers may use this table if the margin of error shown is appropriate for their study; however, the appropriate sample size must be calculated if these error rates are not appropriate. Table developed by Bartlett, Kotrlik, & Higgins.

Sample size for:

Maximum number of regressors if ratio is:

5 to 1 10 to 1

Continuous data: n = 111 22

Categorical data: n = 313

11
31

62

As shown in Table 2, if the researcher uses the optimal ratio of ten to one with continuous data, the number of regressors (independent variables) in the multiple regression model would be limited to 11. Larger numbers of regressors could be used with the other situations shown. It should be noted that if a variable such as ethnicity is incorporated into the categorical example, this variable must be dummy coded, which will result in multiple variables utilized in the model rather than a single variable. One variable for each ethnic group, e.g., White, Black, Hispanic, Asian, American Indian would each be coded as 1=yes and 2=no in the regression model, which would result in five variables rather than one in the regression model. In the continuous data example, if a researcher planned to use 14 variables in a multiple regression analysis and wished to use the optimal ratio of ten to one, the returned sample size must be increased from 111 to 140. This sample size of 140 would be calculated from taking the number of independent variables to be entered in the regression (fourteen) and multiplying them by the number of the ratio (ten). Caution should be used when making this decision because raising the sample size above the level indicated by the sample size formula will increase the probability of Type I error.

If the researcher plans to use factor analysis in a study, the same ratio considerations discussed under multiple regression should be used, with one additional criteria, namely, that factor analysis should not be done with less than 100 observations. It should be noted that an increase in sample size will decrease the level at which an item loading on a factor is significant. For example, assuming an alpha level of .05, a factor would have to load at a level of .75 or higher to be significant in a sample size of 50, while a factor would only have to load at a level of .30 to be significant in a sample size of 350 (Hair et al., 1995). Sampling nonrespondents. Donald (1967), Hagbert (1968), Johnson (1959), and Miller and Smith (1983) recommend that the researcher take a random sample of 10-20% of non-respondents to use in non-respondent follow-up analyses. If nonrespondents are treated as a potentially different population, it does not appear that this recommendation is valid or adequate. Rather, the researcher could consider using Cochrans formula to determine an adequate sample of nonrespondents for the non-respondent follow-up response analyses. Budget, time and other constraints. Often, the researcher is faced with various constraints that may force them to use inadequate sample sizes because of practical versus statistical reasons. These constraints may include budget, time, personnel, and other resource limitations. In these cases, researchers should report both the appropriate sample sizes along with the sample sizes actually used in the study, the reasons for using inadequate sample sizes, and a discussion of the effect the inadequate sample sizes may have on the results of the study. The researcher should exercise caution when making programmatic recommendations based on research conducted with inadequate sample sizes

Although it is not unusual for researchers to have different opinions as to how sample size should be calculated, the procedures used in this process should always be reported, allowing the reader to make his or her own judgments as to whether they accept the researchers assumptions and procedures. In general, a researcher could use the standard factors identified in this paper in the sample size determination process. Another issue is that many studies conducted with entire population census data could and probably should have used samples instead. Many of the studies based on population census data achieve low response rates. Using an adequate sample along with high quality data collection efforts will result in more reliable, valid, and generalizable results; it could also result in other resource savings. The bottom line is simple: research studies take substantial time and effort on the part of researchers. This paper was designed as a tool that a researcher could use in planning and conducting quality research. When selecting an appropriate sample size for a study is relatively easy, why wouldnt a researcher want to do it right