Some Principles of

Evidence-based Medicine

William J. Powers, M.D.

Financial Disclosures for

William J. Powers, M.D.

Salary and Research Support:

National Institutes of Health
Washington University School of
Medicine
Barnes-Jewish Hospital
No personal financial relationships with any
pharmaceutical or medical device companies
including consulting, lecture fees, honoraria,
gifts, licensing revenues, equity interests or

Users' Guides to the Medical Literature: XXV.

Evidence-Based Medicine: Principles for Applying
the Users' Guides to Patient Care.

Hierarchy of Clinical Evidence

Guyatt, et al: JAMA 2000 284:1290-1296

N of 1 RCT
In the N of 1 RCT,
Patients undertake pairs of treatment periods in which they
receive a target
treatment in 1 period and a placebo or
alternative in the other.
Patients and clinicians are blind to allocation,
The order of the target and control are randomized
Patients make quantitative ratings of their symptoms
during each period
N of 1 RCTs can provide definitive evidence of treatment
effectiveness in individual patients, and may lead to longterm differences in treatment administration.
N of 1 RCTs are unsuitable for
short-term problems;
therapies that cure
therapies that act over long periods of time or prevent rare
or unique events
(such
as stroke,
myocardial
infarction, or
Guyatt,
et al:
JAMA 2000
284:1290-1296
death)

Users' Guides to the Medical Literature: XXV.

Evidence-Based Medicine: Principles for Applying
the Users' Guides to Patient Care.

When considering any source of evidence about

treatment other than N of 1 RCTs, clinicians are
generalizing from results in other people to their
patients, inevitably weakening inferences about
treatment impact and introducing complex issues of
how trial results apply to individuals.
Inferences may nevertheless be strong if results
come from a systematic review of methodologically
strong RCTs with consistent results and are generally
somewhat weaker if we are dealing with only a single
RCT unless it is large and has enrolled a diverse
patient population.
Guyatt, et al: JAMA 2000 284:1290-1296

Meta-analysis
Identification of Trials
Definition of Outcomes
2 x 2 outcome tables summed over
all trials
Calculation of treatment effects
Statistical significance

Secondary Prevention of Stroke with

High Dose Aspirin and Dipyridamole
Annualized Stroke Rate

Meta-analysis of 1574 patients from Three Trials*

p=NS

900-1300
150-300

mg/day ASA
mg/day DPM

*Rev Neurol 1082; 138:367-385

Stroke 1983; 14:5-14
Stroke 1985; 16:406-415

European Stroke Prevention Study 2

Annualized Stroke Rate

Low Dose Aspirin plus Extended Release Dipyridamole

(J Neurol Sci 1996; 143:1-13)

**

* Better than placeb

** Better than othe

3 groups

ASA
DPM

50 mg/day
400ER mg/day

Secondary Prevention of Stroke with

Aspirin and Dipyridamole
Meta-analysis of 4873 patients from Four Trials with Any Dose of Aspirin

Diener HC: Neurology1998; 51(Suppl 3):S17-S19

Percent of Vascular Events

Secondary Prevention of Vascular Events in High

Risk Patients
Aspirin vs. Dipyridamole and Aspirin

p=NS

Meta-analysis of 25
Trials

Users' Guides to the Medical Literature:

II. How to Use an Article About Therapy or Prevention:
A. Are the Results of the Study Valid?

Guyatt, et al: JAMA 1993;

Why is the intent-to-treat

principle so important?
It mimics the real life decision
At diagnosis, you have to decide what treatment to try

It accounts for sicker patients going off

protocol
Benefits can be inflated by non-responders dropping out

It accounts for side effects

Treatment A for cancer yields a 90% 5-year survival but
only 10% survive treatment
Treatment B yields a 75% survival and all survive treatment

Measuring the Effects of Therapy

Guyatt, et al: JAMA.1994; 271:59-63

Annualized Stroke Rate (%)

Comparison of Aspirin with other

Anti-platelet Agents for the Secondary Prevention of Stroke:
Absolute Risks & Risk Reduction
2
5

2
5

2
0

2
0

1
5

1
5

1
0

1
0

0
ESPS2
Aspirin

TASS
Agent

ESPS2

TASS

Relative Risk Reduction

Absolute Risk Reduction

% Without Stroke
At One Year

Comparison of Aspirin with other

Anti-platelet Agents for the Secondary Prevention of
Stroke: Prognosis

Confidence Intervals for Risk Reduction

The solid line represents the confidence interval around the first example in
which there were 100 patients per group and the number of events in the
active and control groups were two and four, respectively. The broken line
represents the confidence interval around the second example in which there
were 1000 patients per group and the number of events in the active and
control groups were 20 and 40, respectively.
From: Guyatt: JAMA 1994: 271:59-63

Warfarin-Aspirin Recurrent Stroke Study

(WARSS)
(Mohr et al: NEJM 2001; 345:1444-1451)

PRIMARY RESULTS
Although this trial demonstrated no
significant difference between warfarin
and aspirin, it did not establish
equivalence
Hazard ratio 1.13

95% CI 0.92-1.38

This means that there is less than a 5% chance

that warfarin is more than 8% better than aspirin

HAEST

Lancet 2000; 355:1205-1210

Recurrent ischemic stroke within 14 days in 449

patients with acute (< 30 hrs) ischemic stroke
who had atrial fibrillation within 2 years (89%
present on admission)

Dalteparin
Aspirin

19/224
17/225

(8.5%)
(7.5%)
p=.73

OR(95% CI) = 1.13 (.57 to 2.24)

Number Needed to Treat

The number needed to treat

is 1 divided by the absolute
risk reduction

Number Needed to Treat to Prevent 1 Stroke at 2

Years After Carotid Endarterectomy
Gorelick PB: Stroke. 1999; 30:1745-50.
%
Stenosi NN
s
T
Study
Symptomatic
NASCET
ECST
NASCET
NASCET
Asymptomatic
Veterans Affairs
ACAS

7099
7099
5069
<50

8
8
20
67

48
83

Economic Analysis
Cost-effectiveness analysis - health outcomes are
not valued, but reported in physical units such as life
years gained or cases successfully treated.
Cost-utility analysis - outcomes of different types
are weighted to produce a composite index, such as
the quality-adjusted life year (QALY) or healthy years
equivalent.
Cost-benefit analyses - the health consequences
are valued by asking health care consumers what
they would be willing to pay for health services that
achieve combinations of outcomes of particular types.

Drummond, et al:. JAMA 1997; 277:1552-1557

Cost Utility of PET Selection for EC/IC Bypass

Surgery for Symptomatic Carotid Occlusion
Derdeyn et al, J Nucl Med 2000; 41:800-807

If the Carotid Occlusion Surgery Study

hypothesis is proven true, EC/IC bypass
surgery will improve outcome and reduce
cost:
49 QALY/10 yrs/ 100 patients

cost savings of $11,000 per patient

The Important Issue of the

Primary Endpoint

The Primary Endpoint should be an

event that is clinically important to
patients such as death, stroke or
myocardial infarction

The Important Issue of the

Primary Endpoint
The choice of the Primary Endpoint will affect
the size and duration of the study
Sample size is determined by the number of
events in the control group and the projected
treatment effect

The choice of the Primary Endpoint will affect

the outcome of the study
All components of the primary endpoint are
weighted equally
Serious Adverse Events NOT included in Primary
Endpoint are not included in the primary analysis

Comparison of Stenting with Minimally Invasive Bypass

Surgery for Stenosis of the Left Anterior Descending
Coronary Artery
NEJM 2002; 347:561-566

Stenting
Group
(N=108)
Cardiac Death

Surgical Group
(N=106)

99
Acute MI

3
68

Revascularization

31

003
Primary Endpoint
(All)

34

16
02

2 year Stroke Rate (%)

Secondary Prevention of Stroke with Aspirin

and Dipyridamole ESPS2

Bleeding

Tissue Plasminogen Activator for Acute Ischemic

Stroke (NINDS)

NEJM 1995; 333:1581-1587; Stroke 1997; 28:2109-2118

Outcome at 3 Months in 624 Patients

Good
Rankin 0-1

p<.
001
Poor
Rankin 2- 5

Dead

Symptomatic
Hemorrhage

Observational Studies
Observational studies may provide compelling
evidence if untreated outcomes are consistent and
treatment effects are sufficiently large and
consistent. For instance , observational studies
have allowed extremely strong inferences about
the efficacy of insulin in diabetic ketoacidosis or
penicillin for pneumococcal meningitis, both of
which are almost uniformly fatal if untreated.
However, when outcome is variable and,
especially, when the poor outcome occurs in only a
small proportion of patients, evidence derived only
from non-randomized observational studies should
be regarded with extreme caution.

Divergent Data on Post-Menopausal

Hormone Therapy
NEJM 2003; 348:645-650

Users' Guides to the Medical Literature: XXV.

Evidence-Based Medicine: Principles for Applying
the Users' Guides to Patient Care.

Physiologic studies and unsystematic

clinical observations provide the
weakest inferences about treatment
effects.

Guyatt, et al: JAMA 2000; 284:1290-1296

Problems in the "Evidence" of

"Evidence-Based Medicine"
Feinstein AR & Horwitz RI: Am J Med 1997;103:529-535

The data do not include many types of treatments or

patients seen in clinical practice.
The results show comparative efficacy of treatment for an
"average" randomized patient, not for pertinent subgroups
formed by such cogent clinical features as severity of
symptoms, illness, co-morbidity, and other clinical nuances.

The laudable goal of making clinical decisions based on

evidence can be impaired by the restricted quality and
scope of the "best available evidence." The authoritative
aura, however, may lead to major abuses that produce
inappropriate guidelines or doctrinaire dogmas for clinical
practice.

Extrapolating from Primary Preventio

to Secondary Prevention
% Primary Endpoint
per year

Oral Anticoagulation vs. Aspirin in Atrial Fibrillation

ASAK-1 Lancet 1989; 1:175-178; European Atrial Fibrillation Trial Lancet 1993; 342: 1255-1

Extrapolating from Primary Prevention

to Secondary Prevention
Aspirin for Primary Stroke Prevention
Non-fatal Stroke per 10,000 subject-years

Aspirin

Control

35.6
20.1

31.7
16.8

British1
US2

1
2

BMJ 1988; 296:313-316

NEJM 1989; 321:129-135

Odds
Ratio
1.12
1.20

Extrapolating from Primary Prevention

to Secondary Prevention
Antiplatelet Therapy for Secondary Stroke Prevention1
Non-fatal Stroke

Antiplatelet

Control

8.3%

10.8%

BMJ 2002; 324:71-86

Odds
Ratio
.77*
* p< .
0001

The Problem of Subgroup Analysis

TOAST Primary Analysis
JAMA 1998;279:1265

1281 patients within 24 hrs of ischemic stroke

Dose-adjusted danaparoid IV vs. placebo x 7d
Primary endpoint: favorable functional outcome
(GOS I or II and BI 12-20) at 3 months

Favorable outcome
Danaparoid
Placebo

75.2%
73.7%

p = .49
OR 1.09 (0.85 1.41)

TOAST
Stroke Subtype Analysis
5 subtypes analyzed for two different endpoints in addition
to 4 analyses for total group = 14 analyses
p < .05/14 = p< .0036
Large artery atherosclerosis
Favorable outcome p=.04
Very favorable outcome p=.02
NO STATISTICALLY VALID SIGNIFICANT DIFFERENCE

Lumping vs. Splitting

WARSS

(Mohr et al: NEJM 2001; 345:1444-1451)

Design:
Prospective, randomized, double-blind, multicenter clinical trial
Eligibility:
Non-cardioembolic stroke < 30 d, no CEA planned
Treatment:
Warfarin (INR 1.4-2.8, mean 2.1) vs aspirin 325
mg/day
Primary endpoint:
Recurrent ischemic stroke or death within two
years

WARSS

(Mohr et al: NEJM 2001; 345:1444-1451)

PRIMARY RESULTS
2206 patients followed for 2 years
Ischemic Stroke/ Death

Warfarin
Aspirin

17.8%
16.0%

Major Hemorrhage
/100 pt-yrs

2.22
1.49

p=.25

Hazard ratio 1.13 95% CI 0.921.38

Lumping vs. Splitting

WASID

(Chimowitz MI, et al: Neurology 1995; 45: 1488-1493)

Design:
Retrospective, non-randomized, unblinded,
multi-center concurrent cohort study based on
angiogram review
Eligibility:
Symptomatic 50-99% stenosis of major
intracranial artery by arteriography
Treatment:
MD choice: Warfarin (PT 1.2 1.6 x control) vs
aspirin (usually 325 mg/d)
Primary endpoint:
Stroke, MI or sudden death

WASID

(Chimowitz MI, et al: Neurology 1995; 45: 1488-1493)

RESULTS
OAC
Number of Patients
88
Median follow-up (mo)
14.7
19.3
Primary Endpoint
14
p < 0.01

ASA
63

26

Hazard ratio 0.46 95% CI 0.230.86

Translating Evidence into

Recommendations
(Guyatt, et al: Chest 2001; 119:3S-7S)

In addition to the strength of the evidence, the

following considerations will bear on the strength of
the treatment recommendation:
-the magnitude and precision of the treatment
effect
-patients risk of the target event being
prevented
-the nature of the benefit
-the magnitude of the risk associated with
treatment
-variability in patient preferences
-variability in regional resource availability

Translating Evidence into Recommendations

Factors That May Weaken a Recommendation to Treat
(Guyatt, et al: Chest 2001; 119:3S-7S)
Issues

Examples

Less serious outcome

Preventing postphlebitic syndrome with thrombolytic therapy in DVT rather than

death from PE.

Smaller treatment
effect

Clopidogrel vs aspirin leads to a smaller stroke reduction in TIA (8.7% RRR) than
anticoagulation vs placebo in AF (68% RRR).

Imprecise estimate of
treatment effect

Aspirin vs placebo in AF has a wider CI than aspirin for stroke prevention in patients
with TIA.

Lower risk of target

event

Some surgical patients are at very low risk of postoperative DVT and PE, while
others surgical patients have considerably higher rates of DVT and PE.

Higher risk of therapy

Warfarin has a much higher risk of serious hemorrhage than aspirin.

Higher costs

tPA has much higher cost than streptokinase in acute MI.

Varying values

Most young, healthy people will put a high value on prolonging their lives (and thus
incur suffering to do so); the elderly and infirm are likely to vary in the value they
place on prolonging their lives (and may vary in the suffering they are ready to
experience to do so).

Further Reading on Evidence Based Medicine

Users' Guides to the Medical Literature: Essentials of EvidenceBased Clinical Practice
edited by Gordon Guyatt and Drummond Rennie, includes CDROM, 442 pp, soft cover, $34.95, ISBN 1-57947-191-9,
Chicago, Ill, AMA Press, 2002.
Interpreting the Medical Literature
by Stephen H. Gehlbach, 4th ed, 296 pp, paper, $34.95, ISBN
0-07-138762-5, New York, NY, McGraw-Hill, 2002.
Evidence-Based Medicine: How to Practice and Teach EBM
by David L. Sackett, Sharon E. Straus, W. Scott Richardson,
William Rosenberg, and R. Brian Haynes, 2nd ed, with CDROM, 261 pp, softcover, $43.95, ISBN 0-443-06240-4,
Philadelphia, Pa, Churchill Livingstone, 2000.