THERAPY STUDY: Are the results of the trial valid?

(Internal Validity)

 What question did the study ask?

Patients – Patients referred to low-vision clinics on the Manchester Royal Eye Hospital,
Manchester, England, from July 15, 2001, through March 31, 2003, with bilateral AMD and a
visual acuity (VA) of at least 1/60 but no better than 6/18 in the better eye.

Intervention – Standard prism spectacles

Comparison – Custom prism spectacles

Outcome – Distance logMAR (logarithm of the minimum angle of resolution) visual acuity

Was the Yes. This statement can be seen in the title, abstract, and methods
assignment of (under study design).
patients to
treatments Stated in method on first pharagraph: This study was a double-mask,
randomized? randomized, placebo-controlled trial, in which participants received 1 of
the following 3 types of test spectacles...

And stated in method on third pharagraph: Participants were allocated

to groups using computer generated randomization codes prepared in
advance by one of us (B.C.R.). Randomization and the ordering of
spectacles were performed by a principal investigator (R.A.H.) who had
no contact with participants during the study.
Were the groups No, as shown at table 6 that presenting demographic and clinical
similar at the characteristics of participants at baseline, there were significant
start of the trial? differences between each groups in registration status, especially blind.
A different value could lead into a bias result.
Aside from the Yes, the participants were treated equally. There were no other
allocated treatment between each group.
treatment, were
groups treated Stated in method (under prescription of prism spectacles): All
equally? participants received a standard low-vision assessment, including the
prescription of conventional low-vision aids (LVAs) and updated
conventional spectacles 6 weeks before the study intervention.
Were all patients No, there were several participants who lost to follow up. The number
who entered the and reason for losses to follow-up can be seen at Figure 1.
trial accounted
for? – and were And stated in comment: Participant were recruited from a wide
they analysed in catchment population source; randomization was concealed, thus ruling
the groups to out selection bias; there was a high follow-up rate (93%), with no
which they were obvious differential attrition bias; there was no evidence that
randomised? participants or researchers became unmasked during assesment; and
the findings were consistent across a range of outcomes.

And stated in method (under sample size and statistical analysis): All
analyses were by intention-to-treat, and differences were considered
 statistically significant if P<0,5.
Were measures Yes the patients and clinicians kept blind to which it was randomised.
objective or were This is stated in the methods (under study design) that this study is
the patients and double-masked
clinicians kept
“blind” to which Stated in method on first pharagraph: This study was a double-mask,
they were randomized, placebo-controlled trial, in which participants received 1 of
randomised? the following 3 types of test spectacles...

And stated in comment: Participant were recruited from a wide

catchment population source; randomization was concealed, thus ruling
out selection bias; there was a high follow-up rate (93%), with no
obvious differential attrition bias; there was no evidence that
participants or researchers became unmasked during assesment; and
the findings were consistent across a range of outcomes.

 What were the results?

1. How large was the treatment effect

In this study, by far the most frequently reported problems experienced by

participants in the custom and standard groups were dizziness / loss of balance and
distortion. There are 35 people who reported dizziness / loss of balance and
distortion from 74 people in standard groups (46%) and 10 people from 80 people in
placebo groups (12,5%) and 34 people from 73 people in custom groups (46,5%)

What is the measure? What does it mean?

Relative Risk (RR) = risk of the outcome
in the treatment group / risk of the
outcome in the control group.
Standard vs placebo, the RR = 0,125/0,46
= 0,27
Custom vs placebo, the RR = 0,125/0,465
= 0,27
The relative risk of standard and custom prism
spectacles with placebo as the control agent
both 0,27. An RR < 1 means that the treatment
decreases the risk of the outcome. Since the
RR < 1, the treatment decreases the risk of
problems such as dizziness / loss of balance
and distortion.

Absolute Risk Reduction (ARR) = risk of
the outcome in the control group – risk of
the outcome in the treatment group. This
is also known as the absolute risk
difference.
Standard vs placebo, the ARR = 0,46-
0,125 = 0,335 or 33,5 %
The absolute benefit of standard prism
spectacles is a 33,5% reduction of dizziness /
loss of balance and distortion, and
The absolute benefit of custom prism
spectacles is a 34% reduction of dizziness /
loss of balance and distortion

Custom vs placebo, the ARR = 0,465-

0,125 = 0,34 or 34 %

Relative Risk Reduction (RRR) =
absolute risk reduction / risk of the
outcome in the control group. An
alternative way to calculate the RRR is to
subtract the RR from 1 (RRR=1-RR)
The standard and custom prism spectacles
reduced the risk of dizziness / loss of balance
and distortion by 73% relative to that occuring in
the control group.

Standard vs placebo, the RRR = 1-0,27 =

0,73

Custom vs placebo, the RRR = 1-0,27 =

0,73

Number Needed to Treat (NNT) = inverse It means that we need to treat 3 people in order
of the ARR and is calculated as 1 / ARR to prevent disturbance problem.

Standard vs placebo, the NNT = 2,98 = 3

Custom vs placebo, the NNT = 2,94 = 3

2. How precise was the estimate of the treatment effect?

The confidence interval provides us with information about the statistical significance
of the result. In this case, the confidence interval includes the value corresponding to
no effect, so the results are not statistically significant.

Table 5. Overall Effects of Group and Estimates

of Differences Between Group Means

Outcome by Group β Coefficient* (95% Overall P Value

CI)

VA .63
Custom vs placebo −0.02 (−0.07 to 0.02)
Standard vs placebo −0.02 (−0.06 to 0.03)

WPM .58
Custom vs placebo −2.70 (−10.35 to 4.96)
Standard vs placebo 1.39 (−6.09 to 8.87)

CPS .27
Custom vs placebo −0.04 (−0.10 to 0.03)
Standard vs placebo −0.05 (−0.11 to 0.01)

NEI-VFQ-25 .73
Custom vs placebo 1.25 (−1.98 to 4.47)
Standard vs placebo 0.29 (−2.90 to 3.49)

SVF .31
Custom vs placebo 1.87 (−1.18 to 4.91)
Standard vs placebo 2.17 (−0.84 to 5.18)
MLVAI part 1 .36
Custom vs placebo −0.72 (−2.30 to 0.87)
Standard vs placebo 0.45 (−1.11 to 2.01)

OPTV .11
Custom vs placebo −1.44 (−4.47 to 1.59)
Standard vs placebo 1.84 (−1.14 to 4.81)

MLVAI part 2 .87

Custom vs placebo −0.14 (−0.67 to 0.39)
Standard vs placebo −0.07 (−0.59 to 0.45)

ADL .91
Custom vs placebo −0.56 (−3.08 to 1.97)
Standard vs placebo −0.10 (−2.59 to 2.39)

Abbreviations: See Table 4; CI, confidence interval.

*The placebo group is the base category; therefore the β values (regression coefficients) represent the mean
differences at the 3-month follow-up between the custom and standard groups and the placebo groups, after
adjusting for baseline measurements (eg, the mean difference between custom and placebo groups for logMAR
VA is −0.02).