Critical appraisal of

clinical research evidence

Chris Lewis May 2008

How to read a paper

Objectives:
To enable VTS members to have a
good working knowledge of:
Processes of EBM
Conduct of a RCT
Sources of bias in a RCT
Risk assessment terminology (RR
ARR NNT)

Evidence Based Medicine

Incorporating the best available
research evidence into clinical
decision making

Processes of Evidence
Based Medicine
Asking answerable questions (PICO)
Accessing the best information
Appraising the information for
validity and relevance
Applying the information to patient
care

Asking an answerable
question

Population
Intervention
Comparator
Outcome(s)

Types of paper research

evidence
Primary studies

Case studies
Experiments
Surveys
Clinical Trials

Secondary studies
Non-systematic reviews
Systematic reviews
Meta-analyses

Guidelines

Decision analyses
Economic analyses

Types of evidence
Advantages
Summarises all relevant
research about all possible
interventions for a clinical
problem. Explores benefits
and harms.

Disadvantages
May become out-of-date
quickly.
Expert opinion often fills
gaps in evidence.

Systematic Review

Summarises all research

about an intervention.

Usually only one of several

possible interventions is
considered. May not
explore benfits vs harms.

Primary Study

Very specific information

Not comprehensive

Evidence-based
Guideline

Topics of Primary Study and Types of Study Design

Phenomena
Observation / qualitative studies
Aetiology
Cohort studies (or Case-control studies)
Diagnosis and screening
Cross-sectional analytical studies
Prognosis
Cohort studies
Intervention
Randomised Controlled Trials

How do you read a clinical

research paper?

How to read a (clinical

research) paper
Scan abstract for a few seconds

Are the authors conclusions of interest?

Briefly assess study design
Briefly assess statistical precision of results
Formulate a brief summary

Critically appraise methods & results

sections for validity
Critically appraise results section
(especially the tables and figures) for
relevance
Draw your own conclusions about clinical
applicability

What conclusions have you

drawn concerning clinical
application of the Heart
Protection Study?

How to read a (clinical

research) paper
Scan abstract for a few seconds

Are the authors conclusions of interest?

Briefly assess study design
Briefly assess statistical precision of results
Formulate a brief summary

Critically appraise methods & results

sections for validity
Critically appraise results section
(especially the tables and figures) for
relevance
Draw your own conclusions about clinical
applicability

What is the essential information you want to know

about any clinical research evidence?

WHAT INFORMATION WOULD

YOU INCLUDE IN A BRIEF
SUMMARY OF A CLINICAL
RESEARCH PAPER?

Information to include in
summary

Type of study
Size
Study Population
Intervention
Comparator
Duration
Outcome(s)
Main findings (with relevant
statistics)
Conclusion(s)

Produce a brief summary for the

Heart Protection Study
(6 to 8 short sentences)
Note which parts of the paper you have to read
to produce your summary.

Heart Protection Study

(Lancet.v360.pp7-22.6/7/2002)

Randomised placebo-controlled trial

20536 UK adults, aged 40 to 80, with CHD,
other occlusive arterial disease or diabetes.
Effect of Simvastatin 40mg vs placebo on
mortality, fatal and non-fatal vascular events.
5 years follow-up

 All cause mortality reduced in the

simvastatin group
1328/10269 (12.9%) vs 1507/10267 (14.7%);
p=0.0003; RR 0.87 (0.81-0.94); NNT=55
First vascular event rate reduced in the
simvastatin group
2033/10269 (19.8%) vs 2585/10267 (25.2%);
p<0.0001; RR 0.76 (0.72-0.81); NNT=18.
No significant harms identified
All people similar to the study population
should be treated with Simvastatin 40mg

How much of the paper have we

actually read to get to this
summary?

How to read a (clinical

research) paper
Scan abstract for a few seconds

Are the authors conclusions of interest?

Briefly assess study design
Briefly assess statistical precision of results
Formulate a brief summary

Critically appraise methods & results

sections for validity
Critically appraise results section
(especially the tables and figures) for
relevance
Draw your own conclusions about clinical
applicability

Flow Diagram for a RCT / cohort

study
1 Selection and sampling
4 Outcomes
2 Allocation (with or without Randomisation)
3 Follow-up

5 Analysis

Validity - External
Validity

To whom do the results of this trial apply?

Can the results be reasonably applied to a
definable group of patients in a particular
clinical setting in routine practice?
Are the results generalisable beyond the
trial setting?

Appraisal of External Validity

Where were the participants
recruited from (primary care / referral
centre)?
Do the inclusion and exclusion
criteria make sense?
What proportion of the screened
population was recruited?

Where were the participants

recruited from?

Where were the participants

recruited from?
Methods: Recruitment p8
69 UK hospitals

Inclusion Criteria:

Inclusion Criteria:

Methods: Eligibility p8

Men and women aged 40 to 80 +

Blood total cholesterol >= 3.5mmol/L +
Past medical history of any one or more
of
CHD, CVA, TIA, PVD, DM
OR
Men, 65 to 80, treated for hypertension

Exclusion criteria:

Exclusion criteria:
1. Anyone already on a
statin or Dr considered
statin to be clearly
indicated.

2. Contraindications
Chronic liver disease
ALT >67 IU/L (1.5 x ULN)
Child-bearing potential

3. Conditions requiring
a dose reduction
Severe renal disease
Creatinine >200 mmol/L

4. Interactions
Treatment with
ciclosporin, fibrates, niacin

5. Conditions similar to
known unwanted effects
Inflammatory muscle
disease
CK >750 IU/L (3 x ULN)

6. Patient unlikely to
survive 5 years followup
Severe heart failure
Another life threatening
condition

7. Conditions limiting
compliance
Severely disabling stroke
Dementia

What proportion of the

screened population was
recruited?

What proportion of the screened

population
was&recruited?
Results: patient enrolment
Fig.1 p10

49% (31458/63603) of screened population excluded or

refused
We are not given a break-down of the reasons

36% (11609/32145) of population accepted for run-in were

not subsequently randomised.
26% chose not to enter or did not seem likely to be compliant
for 5 years
5% considered to have clear indication for statin
3% raised ALT, CK or Creatinine at pre-treatment screen
2% attributed various problems to run-in treatment
1% cholesterol <3.5mmol/L

Only 32% (20536/63603) of screened population were

randomised.

Validity

- Internal Validity

The extent to which the

observed difference in outcomes
between the two comparison
groups can be attributed to the
intervention rather than other
factors.

What are the possible

causes of an effect in a
RCT?

What are the possible

causes of an effect in a
RCT?

Bias
Placebo
Chance
Real effect

Bias
Allocation (Selection) Bias Failure of
randomisation
Systematic differences in comparison groups

Performance Bias
Systematic differences in interventions received by
the two groups

Attrition Bias
Systematic differences in withdrawals from the trial

Detection (Measurement) Bias Failure of blinding

Systematic differences in outcome assessment

Internal Validity - Sources of Bias in

a RCT
2 Allocation Bias (Failure of Randomisation)
3 Follow-up Performance Bias and Attrition Bias
4 Outcomes Detection Bias (Failure of Blinding)

CONSORT definition:
Selection biasa systematic error
in creating intervention groups,
causing them to differ with respect to
prognosis. The groups differ in
measured or unmeasured baseline
characteristics because of the way in
which participants were selected for
the study or assigned to their study
groups.

Confoundinga situation in which the

estimated intervention effect is biased
because of some difference between the
comparison groups apart from the planned
interventions - such as baseline
characteristics, prognostic factors, or
concomitant interventions.
For a factor to be a confounder, it must
differ between the comparison groups and
predict the outcome of interest.

Comparison of Cohort and

RCT
Cohort

RCT

Population diverse
Allocation by clinical
decision
Outcomes can be defined
retrospectively
Outcomes may be rare
Follow-up may be
retrospective and may be
long-term
Analysis complex
multivariate

Population highly
selected
Allocation by chance
Outcomes defined
prospectively
Outcomes must be
common
Follow-up pre-determined
and usually short-term
Analysis relatively simple

Possible types of comparisons in

cohort study
General population
Intervention v alternative intervention
Intervention v no intervention

Restricted population
Intervention v alternative intervention
Intervention v no intervention

Do patients who receive atypical

antipsychotic drugs have an increased risk
of hip fracture?
All older people
Older people with dementia
Atypical
No
Atypical
No
antipsychotic Intervention antipsychotic intervention
(n=34 960) (n=1 251 435) (n=21 427)
(n=58 754)
Mean (SD) age 80.46 (7.63)
No (%) with
21 427 (61.3)
dementia

74.50 (6.58)
58 754 (4.7)

81.69 (7.11)
21 427 (100)

80.95 (7.64)
58 754 (100)

Effect on age distribution and sample size of restricting comparison of

atypical antipsychotic with no intervention to individuals with dementia

Consider the difference between the three sets of figures

here:

Atypical antipsychotic
(n=21 427)

No intervention
(n=58 754)

Mean (SD) age

81.69 (7.11)

80.95 (7.64)

Mean (SD) age

81.69 (1.11)

80.95 (7.64)

Mean (SD) age

81.69 (7.11)

80.95 (1.64)

Selection / Allocation Bias

Assessed by looking at the
Table of Baseline Characteristics

Womens Health Initiative

JAMA v288, pp321-333, 17th July 2002

A randomised placebo-controlled trial.

16608 American women, aged 50-79,
with intact uterus.
Effect of conjugated equine oestrogens
0.625mg od + medroxyprogesterone
acetate 2.5mg od on incidence of CHD
and Breast Cancer
8.5 years follow-up planned, but stopped
after 5.2 years.

 CHD rate increased in oest+prog group

164/8506 (1.93%) vs 122/8102 (1.51%); RR 1.29 (1.021.63);
ARI 0.42%; NNH 238
Breast Ca rate increased in oest+prog group
166/8506 (1.95%) vs 124/8102 (1.53%); RR 1.26 (1.00
1.59);
ARI 0.42%; NNH 238
Treatment group also had increased rate of venous
thrombo-embolism and reduced rates of fractures and
colo-rectal carcinoma.
Overall long-term harms exceeded benefits
751/8506 (8.83%) vs 623/8102 (7.69%); RR 1.15 (1.031.28);
ARI 1.14%; NNH 88

When prescribing combined HRT in the over-50s shortterm benefits should be balanced by consideration of
long-term harms.

Table of Baseline
Characteristics (WHI)
Are all important characteristics listed?
Are any of the differences between the treatment
and placebo groups statistically significant?
Are there any differences in the two groups that
may bias the results?
What age range includes 95% of the Placebo
group?
Assuming HRT has no effect which group would
you expect to have more heart attacks?
Does this introduce a bias?
If so, in which direction does it operate?

Performance Bias
Contamination:
Provision of the intervention to the
control group
Compliance:
Poor compliance with the allocated
intervention
Co-interventions
Provision of unintended additional
interventions to either group

Attrition Bias
Count (drop-out)
Loss to follow-up rate
should not exceed outcome event
rate and should be equal in all
groups.

Detection (Measurement)
Bias
Best: Double-blind
Both patient and investigator unaware
of treatment allocation
Less important if outcome is objective
(e.g. death)
Critical if outcome is subjective

Impossible for some comparisons eg

medical vs surgical intervention

What are the possible

causes of an effect in a
RCT?

Bias
Placebo
Chance
Real effect

Placebo Effect
You can only know the size of a
placebo effect if a placebo has been
used!

Appraisal of Internal Validity

Was assignment of patients to treatments
randomised?
Were groups similar at start of trial?
Were groups treated similarly, apart from the
experimental treatment?
Were all participants accounted for in the
conclusions?
Were all participants analysed in the groups to
which they were randomised (Intention To Treat
analysis)?
Were participants and clinicians kept blind to
treatment received?

Randomisation?
(with allocation concealment)

Best: Centralised computer randomisation

Should be independent of investigators
Where: Methods
HPS
Methods: Recruitment p8
The central telephone randomisation system

Similar Groups?
Table of baseline characteristics
(Note p-values)

Where: Results
HPS:
Methods: Recruitment p8
The central telephone randomisation system used a minimisation algorithm to
balance the treatment groups with respect to eligibility criteria and other major
prognostic factors

Results: Patient Enrollment p10

good balance between the groups for the main pre-randomisation prognostic
features

But there is no table of baseline characteristics!

We are left to make our assessment from the numbers allocated to each sub-group in
Fig.8 p16

Treated equally?
Where: Methods - for intended
schedule
Results - for actual treatment
HPS:
Methods: Recruitment p8
..randomly allocated to receive 40mg
simvastatin daily or matching placebo tablets
in specially prepared calendar packs..

HPS:
Results: Compliance p11
Placebo-allocated group more likely
to be prescribed a non-study statin
(32% vs 5% at end of study).
Averaged over the 5 years of study
85% of simvastatin group and 17% of
placebo group received a statin.

Non-equal treatment Classic

example
1948 Trial of Vitamin E in pre-term infants
Vitamin E prevented retrolental fibroplasia

(by removal from 100% Oxygen to give the

frequent doses of Vit E)

Loss to follow-up
Rough guide: 5% - OK
>20% - validity doubtful
AND
Must not exceed outcome event
rate
Where: Results

HPS Loss to
Results:
Fig.1 p10
Follow-up
Mortality
Morbidity
Total
Outcome ER

Simvastatin
0.03%
0.33%
0.36%
19.8%

Placebo
0.04%
0.25%
0.29%
25.2%

Intention-to-treat
analysis
Maintains the randomisation
Where: Results
HPS:
Summary: Methods p7
Analysescompare all simvastatinallocated versus all placebo-allocated
participants. These intention-to-treat
comparisons
Results: Compliance p11

Blinding?
Best: Double-blind
Where: Methods
HPS:
Does not mention blinding
But we are given some suggestion the randomisation process kept
the allocation concealed from both patients and investigators.
Methods: Recruitment p9
matching placebo tablets in specially prepared calendar
packs.

Methods: Follow-up p9
.coordinating centre clinical staffwere kept unaware of the
study treatment allocation.

How to read a (clinical

research) paper
Scan abstract for a few seconds

Are the authors conclusions of interest?

Briefly assess study design
Briefly assess statistical precision of results
Formulate a brief summary

Critically appraise methods & results

sections for validity
Critically appraise results section
(especially the tables and figures) for
relevance
Draw your own conclusions about clinical
applicability

Appraisal of Relevance /
Impact
Were all the outcomes studied
important?
Were all the important outcomes
studied?
Was sub-group analysis pre-planned?
Could the treatment effect have
arisen by chance?
How large was the treatment effect?

HPS: Outcomes
Primary Outcomes
Mortality
Non-fatal vascular events
MI, CVA, Revascularisation

Secondary Outcomes
Cancer
Other major morbidity

Where: Summary

Was sub-group analysis

pre-planned?
Where: Methods, Statistical
Analysis
The data analysis plan was
prespecifiedbefore any analysis of
the effects of treatment were
available

Could the treatment effect have arisen

by chance?
p-values
Statistical test of the (null) hypothesis that
the intervention had no effect
If p<0.05 result is statistically significant
i.e. the effect would occur by chance less than 5%
of the time

The smaller the p-value the less likely is the

effect to occur by chance

Confidence Intervals (95%)

Range of values that has a 95% chance of
including the true value

How large was

effect?
RR
Death
0.87
Event
0.76

the treatment
RRR
0.13
0.24

ARR
1.8%
5.4%

NNT
55
18

Expressions of Risk
In the study population, treatment with Simvastatin
40mg
for 5 years, compared with placebo, resulted in:

A 13% reduction in risk of death

A 24% reduction in risk of a major vascular event

A 1.8% reduction in deaths

A 5.4% reduction in major vascular events
We need to treat 55 people to defer one death
We need to treat 18 people to prevent / defer a
major vascular event.

Expressions of Risk
Relative Risk (RR) = EER / CER
Relative Risk Reduction (RRR) = 1 RR
Absolute Risk Reduction (ARR) = CER
EER
Numbers Needed to Treat (NNT) = 1/ARR
EER = Experimental Event Rate
CER = Control Event Rate

Relative Risk
RR & RRR may remain constant despite
huge differences in absolute event rates

They are useful to determine whether a

biological effect exists .
BUT
They do not discriminate between huge
treatment effects and trivial ones.

CER

EER

RRR

0.16

0.10

37.5%

0.016

0.010

37.5%

0.0016 0.0010 37.5%

Absolute Risk Reduction

ARR reflects baseline risk and does
discriminate between huge and
trivial treatment effects.

CER

EER

RRR

ARR

0.16

0.10

37.5% 6%

0.016

0.010

37.5% 0.6%

0.0016 0.0010 37.5% 0.06%

Numbers Needed to
Treat
The number of people you need to treat
for one of them to have the desired
outcome
over a specified period of time.
A good measure of clinical relevance.
Allows calculation of cost per desired
outcome.

CER

EER

RRR

ARR

NNT

0.16

0.10

37.5%

6%

16.7

0.016

0.010

37.5%

0.6%

167

0.0016

0.0010

37.5%

0.06%

1667

NNT for various CERs and RRRs

CER
0.9
0.3
0.1
0.01
0.001

50%
2
7
20
200
2000

40%
3
8
25
250
2500

RRR
30%
4
11
33
333
3333

20%
6
17
50
500
5000

10%
11
33
100
1000
10000

Note that a small RRR for a condition with a high CER is more
clinically important than a large RRR for a condition with a low CER

Cost: HPS
Can be calculated from NNT
It costs (25 x 12 x 5 x 55) 82500
to defer one death.
It costs (25 x 12 x 5 x 18) 27000
to prevent / defer one major vascular
event.

Womens Health Initiative

(WHI)
Estrogen +
Progestin

Placebo

Women (n)

8506

8102

Fractures ~5 yrs

650

788

Rate (annualised)

0.0147

0.0191

Can you calculate RR, RRR, ARR

and NNT?

Womens Health Initiative

(WHI)
Estrogen +
Progestin

Placebo

Women (n)

8506

8102

Fractures ~5 yrs

650

788

Rate (annualised)

0.0147

0.0191

RR = EER/CER = 0.0147/0.0191 = 0.77

RRR = 1-RR = 1 0.77 = 0.23
ARR = CEREER = 0.0191 0.0147 = 0.0044
NNT = 1/ARR = 1/0.0044 = 227

Womens Health Initiative

(WHI)
HRT for one year:
Reduces the risk of fracture by 23%
Reduces the number of fractures by 0.44%
We need to treat 227 post-menopausal
women for one year to prevent one
fracture
At ~20/month (20x12x227) =
54480 per fracture prevented

Number Needed to Harm

Women (n)
Breast Cancer ~5 yrs
Rate (annualised)

Estrogen +
Progestin

Placebo

8506
166
0.0038

8102
124
0.0030

Can you calculate RR, RRI, ARI,

NNH?

Number Needed to Harm

Women (n)
Breast Cancer ~5 yrs
Rate (annualised)

Estrogen +
Progestin

Placebo

8506
166
0.0038

8102
124
0.0030

RR = EER/CER = 0.0038/0.0030 = 1.27

RRI = RR-1 = 1.27-1 = 0.27
ARI = EER-CER = 0.0038-0.0030 = 0.0008
NNH = 1/ARI = 1250

Womens Health Initiative

(WHI)
HRT for one year:
Increases the risk of breast cancer by
27%
Increases the number of breast
cancers by 0.08%
We need to treat 1250 postmenopausal women for one year to
give one of them breast cancer.

How to read a (clinical

research) paper
Scan abstract for a few seconds

Are the authors conclusions of interest?

Briefly assess study design
Briefly assess statistical precision of results
Formulate a brief summary

Critically appraise methods & results

sections for validity
Critically appraise results section
(especially the tables and figures) for
relevance
Draw your own conclusions about clinical
applicability

Appraising Applicability
Is my patient similar to the study
population?
Is the treatment feasible in my
clinical setting?
Will potential benefits of
treatment outweigh potential
harms of treatment for my
patient?

www.nntonline.net

Outcomes with CER 25% and NNT=20

Worse
with Rx

CURE. NEJM 2001 v345 p394

Discuss the evidence for and against the

following conclusions regarding clinical
applicability in relation to the Heart
Protection Study:

Group 1

There is no need to check LFTs prior to

commencing statin therapy.
There is no need to check LFTs during statin
therapy.
There is no need to check CK prior to commencing
statin therapy.
There is no need to check CK during statin therapy.

Group 2

All people with diabetes should be

treated with a statin.

All men >65 who are treated for

hypertension should be treated with
a statin.

Group 3

Amongst people similar to the study

population there is no need to test blood
cholesterol levels prior to commencing a
statin.

Once a decision has been taken to start

statin treatment there is no need to
monitor blood cholesterol levels.

Group 4
Statins should be stopped at age 80.
Women benefit from statin treatment
to the same extent as men.
Once started statin treatment should
be continued indefinitely.

Group 1

There is no need to check LFTs prior to

commencing statin therapy.
There is no need to check LFTs during statin
therapy.
There is no need to check CK prior to commencing
statin therapy.
There is no need to check CK during statin therapy.

 People with raised ALT and CK were

excluded from the study.
We are not told how many screened
people were excluded for this reason,
but 3% of pre-randomisation run-in
group were excluded for raised ALT,
CK or Creatinine (Results para1 p10)

ALT 2-4x
ALT >4x
CK 4-10x
CK >10x
Myo
Rhabdo
Persistant
ALT >4x
Persistant
CK >4x

Simva
1.35%
0.42%
0.19%
0.11%
0.05%
0.05%

Placebo
1.28%
0.31%
0.13%
0.06%
0.01%
0.03%

ARI
0.07%
0.11%
0.06%
0.05%
0.04%
0.02%

NNH
1428
909
1667
2000
2500
5000

0.09%

0.04%

0.05%

2000

0.3

0.07%

0.01%

0.06%

1667

0.07

0.2

 There is good evidence that it is not necessary

to monitor LFTs or CK in follow-up of
Simvastatin 40mg treatment in the absence of
relevant symptoms.

The low (not statistically significant) attributable

risk of persistently raised ALT or CK due to
Simvastatin 40mg makes it improbable that pretreatment testing will be of value to the patient.
This study does not provide evidence about
generalisability to other statins (or other doses
of simvastatin).

Group 2

All people with diabetes should be

treated with a statin.

All men >65 who are treated for

hypertension should also be treated
with a statin.

Event
Simva
rate
Diabetes 20.2%

Placebo ARR

NNT

25.1%

20

4.9%

There is good evidence that people with

diabetes aged 40 to 80, with total
cholesterol >=3.5 mmol/L should be
offered Simvastatin 40mg.

 Only 1% of the trial population was male

>65 on treatment for hypertension and had
no history of vascular disease or diabetes.
This subgroup was not separately analysed
(or if it was analysed it was not reported).
This study provides no evidence concerning
the benefits treatment for this subgroup

Group 3

Amongst people similar to the study

population there is no need to test blood
cholesterol levels prior to commencing a
statin.

Once a decision has been taken to start

statin treatment there is no need to
monitor blood cholesterol levels.

 Patients with Total Cholesterol <3.5 mmol/L

were excluded.

Outcome event rate benefit from simvastatin

40mg did not vary with starting cholesterol
level or pre-randomisation LDL response.

But the study does not tell us whether much

larger reductions in LDL cholesterol would
give rise to larger reductions in outcome
event rates.

 This study provides no evidence of need to

check cholesterol levels during treatment
with simvastatin 40mg.
This information needs to be taken in the
context that other studies have reported
greater benefit for greater reduction in
cholesterol levels, and QOF provides a
financial incentive to treat to a target level.

Group 4
Statins should be stopped at age 80.
Women benefit from statin treatment
to the same extent as men.
Once started statin treatment should
be continued indefinitely.

Trial population were aged 40 to 80 at

outset so the oldest were 85 at end of
trial.
Event rate
<65
65 69
>70

Simva
16.9%
20.9%
23.6%

Placebo
22.1%
27.2%
28.7%

ARR
5.2%
6.3%
5.1%

NNT
19
16
20

Considerable overlap of confidence intervals of

event rate ratios for different age subgroups
suggests no statistically significant difference in
treatment effect with age within the age groups
studied.

Trend chi2 0.73 (nb 3.84 ~ p<0.05)

Note: 67% of trial participants were

male

Event rate
Male
Female

Simva
21.6%
14.4%

Placebo
27.6%
17.7%

ARR
6.0%
3.3%

NNT
17
30

Note the pattern of overlap of confidence intervals

Heterogeneity chi2 0.76

The actual results tell us that there is a difference in

benefit between male and female, but the statistical
tests tell us that this difference is not significant.

 Study duration was 5 years, so strictly

it only informs us of benefits and harms
over the first 5 years of treatment.
But figures 5 & 6 give some information
which allows us to predict that benefits
would continue for longer than 5 years
how much longer is a matter of
opinion / judgment.

Event
rate
Year
1
2
3
4
5+

simva
4.7%
3.9%
3.9%
3.8%
5.8%

placebo
5.1%
5.6%
5.6%
5.2%
7.3%

ARR
0.4%
1.7%
1.7%
1.4%
1.5%

NNT
250
59
59
71
67

Event
rate
Year

ARR

1
2
3
4
5+

0.4%
1.7%
1.7%
1.4%
1.5%

Diff in

Adjusted

Statin
use
85%
76%
67%
59%
50%

ARR

NNT

0.5%
2.2%
2.5%
2.4%
3.0%

200
45
40
42
33

 Before we can apply the results of any

individual primary study we have to place
it in the context of all other relevant
research.

> Systematic Reviews, Guidelines

What is already known
What this study adds
Has anything been added since this study?

Levels of Evidence:
I Systematic Review of all relevant
RCTs
II At least one good quality RCT
III Good non-randomised trials;
cohort or case-control studies.
IV Expert opinion

Further Reading
Cochrane handbook for systematic
reviews of interventions
http://www.cochrane.org/resources/handbook/Handbook4.2.6Sep2
006.pdf

Consort Statements
http://www.consort-statement.org/?o=1001

Bandolier
BMJ

Odds Ratios

Odds of an event = no. of events / no. of nonevents

e.g. 51 boys/100 births

Odds of boy = 51/49 = 1.04

Odds >1 means event more likely to happen than

not
Odds of an impossibility are zero
Odds of a certainty are infinity
Odds ratio = odds in intervention group /
odds in control group

Odds Ratios (cont.)

When events are rare Odds and Risk are
similar
OR and RR are similar
As prevalence (control event rate) and
OR increase the error in using OR as an
approximation for RR becomes
unacceptable.

In HPS:

Simvastatin
Placebo

Event
2033
2585

No Event
8236
7684

10269
10267

In the simvastatin group:

the odds of an event are 2033/8236 =
0.247 or 24.7%
the risk of an event is 2033/10269 =
0.198 or 19.8%

In the placebo group:

The odds of an event are 2585/7684 =
0.336 or 33.6%
The
is 0.247/0.336
= 0.735 or
Theodds
risk ratio
of an(=relative
event isodds)
2585/10267
=
73.5%
0.252 or 25.2%

The risk ratio (=relative risk) is 0.198/0.252 = 0.786 or 78.6%

An absolute difference of 51:1000 and a relative error of ~7%

Odds ratios are used because

of their superior mathematical
properties:

They can always take values between 0 and

infinity; values RR can take are dependent on
CER.
With ORs the relationship between the two
possible outcomes (event or not event) is
reciprocal not so for RR.
ORs always used in case-control studies where
disease prevalence is not known.
When adjusting for confounding factors which
affect event rates, logistic regression models
(the correct approach) use odds and report
effects as odds ratios.