t-test for independent samples or Paired T-Test?

Sometimes the choice of which type of t test to run is obvious, and sometimes the choice requires some careful reflection. In all of the cases
below the numbers of data points are low enough that the statistical analysis should be based upon a t distribution rather than a normal
distribution. Assume that the variances are equal for all data sets. Independent samples have no connection between any subject in groups 1
and 2; paired samples are of the same group, tested twice. Determine which test will you use and provide a brief explanation. Answers will be
posted tomorrow at http://sircholo.weebly.com/
Problem #1
As a biochemist working for a pharmaceutical company, your job is to test new drugs for possible side effects, both detrimental and
beneficial. The chemistry of agent TFK-05W suggests that it may have the side effect of reducing the tendency toward obesity. The Zucker
rat is an established genetic model for both obesity and hypertension. As rats of the obese strain age they gain weight much more rapidly than
do the so-called "lean" Zucker strain. You plan to treat a group of animals over a period of one year and compare their average weight with
that of a group of untreated animals.
The odds of the drug actually showing this side effect are small and maintenance of rats for a year is expensive, so you limited the scope of
the study to twelve animals in each group. The null hypothesis is that treated animals will show an average weight gain over one year that is
no different from the average weight gain of untreated animals. What statistical test will you use to compare the two means?
Problem #2
This study follows from problem #1. Not only did your study suggest that the agent TFK-05W indeed does affect weight gain, but it also
proved effective and safe (so far) and it is in clinical trials. Because the drug was designed to treat symptoms that have nothing to do with
obesity, the clinical trials do not focus on that problem and won't answer the question of whether or not the agent is a potential weight loss
drug. The company, however, has permitted you to test the agent on a group of 12 people with morbid obesity, who have signed the
appropriate consent forms.
This time the plan is to treat the 12 obese individuals for a year, having measured their weights on the day treatment was started. The paid
participants will be monitoring their weight regularly, taking the drug, and are required to keep a daily log of activity and eating habits so that
the experiment can be properly controlled. Nevertheless, the simplest initial test of the hypothesis that obese individuals treated with TFK-
05W for a year will show an average weight loss is to compare average weight at the beginning and end of the experiment. Thus, as with
problem #1, you will have two sets of 12 data points each to compare. What statistical test will you use to compare the two means?
Problem #3
Embryonic cells (stem cells) from a single human blastula are genetically equivalent. Any of them has the potential to form any kind of tissue
that is normally found in an adult human body. Exploitation of stem cells for therapeutic purposes has the potential to revolutionize medicine
and expand the average human lifespan considerably. Stem cells have very complex cultural requirements. So far your stem cell lines require
the addition of fetal bovine serum to the culture medium, and the exact composition (and efficacy) of animal sera varies from lot to lot, To
exercise the greatest control over your experiments, it would be valuable to be able to culture your cells in a synthetic medium that includes
only those components that are essential to support survival and growth of your cultures.
You have developed a synthetic medium that keeps your cells going for several days, but not indefinitely. You think that you can extend the
life of your cultures by adding an expensive hormone to the medium. To test the hypothesis that stem cells cultured with medium 2 will
survive longer than stem cells cultured with medium 1, you will set up twenty cultures from ten original embryos, growing them in complex
medium to the point at which each culture contains about 100 cells. You will then remove the original medium and feed the cultures from
now on with synthetic medium. One culture from each original embryo will receive medium 1 and the other medium 2. For your data you
will record the time at which each culture declines to the point of having only 50% of its original viable cells remaining. The null hypothesis
is that this average "survival time" will be the same for cultures treated with either synthetic medium. Alternative hypotheses, of course, are
that feeding with one or the other medium will enhance survival time by comparison.
What statistical test will you run on the two sets of 10 data points each?
Problem #4
You suspect that a cause of decline of your stem cell cultures is a failure to produce sufficient superoxide dismutase to rid the cells of oxygen
free radicals. You have an assay for the enzyme, but to conduct the assay you must destroy the culture. From a single source of stem cells you
can prepare about thirty cultures that remain healthy about 10 days in your synthetic medium after growing to a sufficient number of cells to
permit you to run your assay. From then on, they decline rapidly.
You prepared thirty cultures from the same source and sampled half of them when alll thirty cultures had reached the point at which the assay
was feasible. You then sampled the other half 10 days later. Your null hypothesis is that superoxide dismutase activity will not be
significantly different between the two sets of cultures. What statistical test will you run to determine whether or not the difference is
significant?
Normally, this would be a rather poor experimental design, because all of the cultures are identical. Why conduct replicate sampling on the
exact same culture? The issue is that enzyme assays are notoriously inaccurate. The chances of mulitple comparisons yielding dubious results
are much smaller than for a single comparison. The p value that we obtain will give us a fairly accurate estimate of the level of confidence
with which we can interpret the result.


ANSWERS
Problem #1
You'll run a t test for independent samples. It doesn't matter that the number of animals in each data set is the
same, nor that they are all the same type of animal. You sampled 12 treated individuals and
12 different untreated individuals. There is no special relationship between a data point from one group and
any particular data point from a second. The sampling method was independent.
Problem #2
This study calls for running a paired t test. The same individuals were sampled (weights measured) at the
beginning and at the end of the study. Thus each data point in the first set can be paired with a data point
from the same individual in the second set.
Variability among distinct individuals contributes considerable experimental error to many experiments. Such
error can mask effects, especially small effects, even if the null hypothesis is indeed false. For example, if the
average individual lost 10 pounds but the standard deviation at the beginning of the experiment was 55
pounds, the loss might not show up as a significant difference. By controlling for individual variability the paired t
test can focus on the average change in weight.
Problems 1 and 2 were both easy to call. If you had trouble with either case, then you really should review the
criteria for selecting an appropriate t test and try to clear up any misconceptions.
Problem #3
This case can be thought of as a set of replicate experiments. In each experiment one culture from a single
source was fed one medium and a second culture from the same source was fed the other medium. The
experiment was replicated 10 times, using 10 different sources. Since each replicate experiment consists of a
pair of data points linked by the common origin of the respective cultures, you have a set of 10 pairs of data
(two sets of paired data).
A paired t test is appropriate for the same reasons it was appropriate for problem #2. The paired method
controls for experimental error that might be contributed by the 10 different sources.
Why not conduct all of the replicate experiments on cultures from a single source, eliminating all experimental
error that is contributed by individual variability? Then we run the risk that the result won't hold for cultures from
other embryos. We want to know if the medium we are testing will work for most or all cultures, not only for
cultures from one particular embryo.
Problem #4
This time your samples are all coming from the same population of cultures, presumably all identical except
that half of them were sampled at one time and half at the other time. All of the data points are linked by the
fact that they were obtained from cultures from a common source. However, there is no special one to one
correspondence between any one data point in one set and a unique data point in the other. There is no basis
for a paired t test, so we must run a test for independent samples.
The assay itself is the variable in this example. If the assay was 100% accurate and reliable, we would only have
needed one sample at each time. On the other hand, any significant difference should be considered
preliminary until the experiment can be repeated on at least one or two more sets of cultures.