Research Methodology Workshop Hanoi 7-12/12/2010

Professor Tuan V. Nguyen

Garvan Institute of Medical Research
University of New South Wales
Sydney Australia
Analysis of Association:
Odds Ratio and Relative Risk
Research Methodology Workshop Hanoi 7-12/12/2010
Contents
Types of study design
Risk, odds ratio, and relative risk
Estimates of OR, RR
Effects of information framing
Interpretation of RR or RR
Research Methodology Workshop Hanoi 7-12/12/2010
Mortality in the Titanic incident
Class Dead Survived Total
I 123 200 (62%) 323
II 158 119 (43%) 277
III 528 181 (26%) 709
Total 809 500 (38%) 1309
http://lib.stat.cmu.edu/S/Harrell/data/descriptions/titanic3info.txt
Is there an association between passenger class and and death?
Research Methodology Workshop Hanoi 7-12/12/2010
Zoledronate and fracture
Randomized controlled clinical trial
Placebo n = 1062, Zoledronate n = 1065
Length of follow-up: 3 years
Lyles KW, et al. Zoledronic acid and clinical fractures and mortality after hip fracture.
N Engl J Med 2007;357. DOI: 10.1056/NEJMoa074941
Research Methodology Workshop Hanoi 7-12/12/2010
Smoking and lung cancer
Sir Richard Doll (1912 2005)
http://en.wikipedia.org/wiki/Richard_Doll
Lung
Cancer
Controls
Smokers 647 622
Non-smokers 2 27
R Doll and B Hill. BMJ 1950; ii:739-748
Is there an association between smoking and lung cancer?
Research Methodology Workshop Hanoi 7-12/12/2010
Study design
PAST
PRESENT FUTURE
Cohort study,
RCT
(longitudinal,
prospective)
Case-control
study
Cross-
sectional study
Research Methodology Workshop Hanoi 7-12/12/2010
Distinction of study designs:
prospective / longitudinal study
N subjects
Risk presence
Risk absence
Event: a
Event: c
No event: b
No event: d
Event No event
Risk presence a b
Risk absence c d
Follow-up
Research Methodology Workshop Hanoi 7-12/12/2010
Distinction of study designs:
case-control study
Event
No event
Risk presence: a
Risk presence: c
Risk absence: b
Risk absence: d
Event No event
Risk presence a b
Risk absence c d
Research Methodology Workshop Hanoi 7-12/12/2010
Distinction of study designs:
cross-sectional study
Event
No event
Risk presence: a
Risk presence: c
Risk absence: b
Risk absence: d
Event No event
Risk presence a b
Risk absence c d
Research Methodology Workshop Hanoi 7-12/12/2010
Risk
The disease risk is defined as the probability that
an individual who is disease-free at time T will
develop a disease over a period of T + k.
In 2000 (T) there are 100 people who were 90
years old with no hip fracture. Two years later
(k = 3), 5 people sustained a hip fracture.
The risk of hip fracture (over 3 years) is threfore:
3%.
Research Methodology Workshop Hanoi 7-12/12/2010
Measures of effect size
Given two proportions p
1
and p
2
, there are several
measures of effect size:
Absolute difference
D = p
1
p
2
Relative risk
RR = p
1
/ p
2

Odds ratio
OR = [p
1
(1 - p
2
)] / [p
2
(1 - p
1
)]
Number needed to treat (NNT):
NNT = 1 / D
Research Methodology Workshop Hanoi 7-12/12/2010
Appropriateness of effect size
Prospective
study
Cross-sectional
study
Case-control
study
Relative risk
Odds ratio
NNT
D
Odds ratio
D
Odds ratio
Research Methodology Workshop Hanoi 7-12/12/2010
Relationship between RR and OR
Odds ratio is an estimate of
relative risk
Research Methodology Workshop Hanoi 7-12/12/2010
Relative risk
Risk of developing disease
Treatment: p
1
= a / N
1
Placebo: p
2
= b / N
2
Relative risk
RR = p
1
/ p
2
Implications:
RR = 1, there is no effect
RR < 1, the treatment is beneficial.
RR > 1, the treatment is harmful.

Treatment Control
Disease
a b
No disease
c d
Sample size
N
1
N
2

Research Methodology Workshop Hanoi 7-12/12/2010
95% confidence interval for RR
Treatment Control
Disease a b
No disease c d
Sample size N
1
N
2

Zoledronat
e
Placebo
Fracture 92 139
No fracture 973 923
Sample size 1065

1062
= 0.514 to 0.847
Research Methodology Workshop Hanoi 7-12/12/2010
Odds of developing disease over no disease
Risk presence: O
1
= a / b

Risk absence: O
0
= c / d

Odds ratio
OR = O
1
/ O
0
Implications:
OR = 1, there is no association
OR < 1, the risk factor is associated with reduced disease risk
RR > 1, the risk factor is associated with increased disease risk
Disease No disease
Risk presence a b
Risk absence c d
Sample size N
1
N
2

Odds ratio
Research Methodology Workshop Hanoi 7-12/12/2010
Disease No disease
Risk
presence
a b
Risk
absence
c d
Lung K Control
Smoking 647 622
No smoking 2 27
= 3.32 to 59.03
95% Confidence interval for OR
Research Methodology Workshop Hanoi 7-12/12/2010
Zoledronate and fracture: RR vs OR
Zoledronat
e
Placebo
Fracture 92 139
No fracture 973 923
Sample size 1065

1062
Not much difference!
Research Methodology Workshop Hanoi 7-12/12/2010
Recommendation for catheterization?
Figure 1.
Patients as Portrayed by Actors in the Video Component of the Survey. Panel A
shows a 55-year-old black woman, Panel B a 55-year-old black man, Panel C a 70-
year-old black woman, Panel D a 70-yearold black man, Panel E a 55-year-old white
woman, Panel F a 55-year-old white man, Panel G a 70-year-old white woman, and
Panel H a 70-year-old white man.
Research Methodology Workshop Hanoi 7-12/12/2010
Race, sex, and catheterization
Catheterizatio
n
No
Catheterizatio
n
Total
White
652 68 720
Black
610 110 720
Schulman et al. New England Journal of Medicine (25/2/1999 )
Logistic regression analysis indicated that women (odds ratio,
0.60; 95 percent confidence interval, 0.4 to 0.9; P=0.02) and
blacks (odds ratio, 0.60; 95 percent confidence interval, 0.4
to 0.9; P=0.02) were less likely to be referred for cardiac
catheterization than men and whites, respectively. Analysis of
racesex interactions showed that black women were
significantly less likely to be referred for catheterization than
white men (odds ratio, 0.4; 95 percent confidence interval, 0.2 to
0.7; P=0.004).
Research Methodology Workshop Hanoi 7-12/12/2010
Media reaction
Research Methodology Workshop Hanoi 7-12/12/2010
Where is the truth?
Catheterization No
catheterizatio
n
Black
610 110
White
652 68
Very much difference!
Research Methodology Workshop Hanoi 7-12/12/2010
Odds ratio overestimates relative risk
Study Risk of disease Odds of disease Comparison
between RR and
OR
Group 1
(p
1
)
Group 2
(p
2
)
Nhm 1
(odd
1
)
Nhm 2
(odd
2
)
RR OR
1 0.001 0.003 0.002 0.003 3 3.01
2 0.01 0.03 0.01 0.03 3 3.06
3 0.02 0.06 0.02 0.06 3 3.13
4 0.05 0.15 0.05 0.18 3 3.35
5 0.10 0.30 0.11 0.43 3 3.86
6 0.15 0.45 0.18 0.82 3 4.64
7 0.20 0.60 0.25 1.50 3 6.00
8 0.25 0.75 0.33 3.00 3 9.00
9 0.30 0.90 0.43 9.00 3 21.0
10 0.33 0.99 0.49 99.0 3 2101.0
Research Methodology Workshop Hanoi 7-12/12/2010
For a constant RR, OR increases with
background risk
Research Methodology Workshop Hanoi 7-12/12/2010
The effect of risk framing (communication)
Which (hypothetical) study is more likely to be funded?
T Fahey , et al. Evidence based purchasing: understanding
results of clinical trials and systematic reviews. BMJ
1995;311:1056-1059
Mammography Cardiac
rehabilitatio
n
RR 0.66 0.80
Absolute risk reduction 0.06% 3%
Percent of event-free 99.82 vs 99.80% 84% vs 87%
Number needed to treat 1592 31
Research Methodology Workshop Hanoi 7-12/12/2010
The effect of risk framing (communication)
Which (hypothetical) study is more likely to be funded?
T Fahey , et al. Evidence based purchasing: understanding
results of clinical trials and systematic reviews. BMJ
1995;311:1056-1059
Research Methodology Workshop Hanoi 7-12/12/2010
Prescription of drug
Questionnaire A on relative risk reduction
1. A cholesterol lowering drug treatment reduces the relative risk of a fatal and non-fatal
myocardial infarction by 34%. This result is significant.
2. 2. A cholesterol lowering drug treatment reduces the relative risk of a fatal myocardial
infarction by 26%. This result is not significant.
3. 3. A cholesterol lowering drug treatment increases the total mortality by 5.7%. This result is
not significant.
4. 4. During a cholesterol lowering drug treatment, 71 patients have to be treated for five
years to prevent one fatal or non-fatal myocardial infarction. This result is significant.
Questionnaire B on absolute risk reduction
1. A cholesterol lowering drug treatment reduces the incidence of fatal and non-fatal
myocardial infarction by 14 per 1000 patients and five years of treatment. This result is
significant.
2. 2. A cholesterol lowering drug treatment decreases the incidence of fatal myocardial
infarction by one per 1000 patients and five years of treatment. This result is not significant.
3. 3. A cholesterol lowering drug treatment increases the total mortality by 1.2 deaths per 1000
patients and five years of treatment. This result is not significant.
4. 4. During a cholesterol lowering drug treatment 71 patients have to be treated for five years
to prevent one fatal or non-fatal myocardial infarction. This result is significant.
Research Methodology Workshop Hanoi 7-12/12/2010
RR result and prescription of drug
Research Methodology Workshop Hanoi 7-12/12/2010
Interpretation of RR: vitamin C and serious
outcomes
NEJM April 27, 2006, 354: 1796-1806:
Authors conclusion: Supplementation with vitamins C
and E during pregnancy does not reduce the risk of
preeclampsia in nulliparous women, the risk of
intrauterine growth restriction, or the risk of death or
other serious outcomes in their infants.
FACTS:
Preeclampsia: RR 1.20 (0.82 1.75)
Growth restriction: RR 0.87 (0.66 1.16)
Serious outcomes: RR 0.79 (0.61 1.02)
Research Methodology Workshop Hanoi 7-12/12/2010
Interpretation of RR: fat and breast cancer
Womens Health Initiative study
on fat consumption and breast
cancer (JAMA): Even diets
with only 29% of calories
coming from fat didnt reduce
the risk of disease.
FACTS:
Invasive Breast Cancer: HR
0.91 (0.83-1.01), p=0.07
Breast Cancer Mortality: HR
0.77 (0.48-1.22)
Research Methodology Workshop Hanoi 7-12/12/2010
What does 95% confidence interval mean?
Technical notes:
logor = log(2.47)
sd = log(5.2/1.17)/3.92
x = rannor(1000, logor, sd)
or = exp(x)
hist(or, breaks=20, xlim=c(0,10),
ylim=c(0,250),
xlab="Odds ratio")
95%CI
OR 2.47, 95%CI: 1.17, 5.20
Research Methodology Workshop Hanoi 7-12/12/2010
Clinically harmful Clinically beneficial
0.7 1.3 1.0 2.0 3.0 4.0 0.5 0.3 0.2
Strong evidence against any substantial harm or benefit because we have strong
evidence against any values outside the CI, both these cases argue strongly that any
effect is clinically unimportant. Note that this is true even though one is statistically
significant.
Relative risk
Clinically harmful Clinically beneficial
0.7 1.3 1.0 2.0 3.0 4.0 0.5 0.3 0.2
May be harmful. The estimate is in the harmful range, with the CI including some
values that would be clinically important harm. Strong evidence against any
substantial benefit The CI does not include any values that would be an important
benefit.
Relative risk
Research Methodology Workshop Hanoi 7-12/12/2010
Clinically harmful Clinically beneficial
0.7 1.3 1.0 2.0 3.0 4.0 0.5 0.3 0.2
Relative risk
Clinically harmful Clinically beneficial
0.7 1.3 1.0 2.0 3.0 4.0 0.5 0.3 0.2
Relative risk
Likely to be harmful. The estimate and most of the CI are in the
substantial harm range, and the CI does not include no effect.
Results suggest substantial harm, but CI is too wide to permit a strong conclusion.
Strong evidence against substantial benefit - the CI does not include any values that would
be an important benefit
Research Methodology Workshop Hanoi 7-12/12/2010
Clinically harmful Clinically beneficial
0.7 1.3 1.0 2.0 3.0 4.0 0.5 0.3 0.2
Relative risk
Suggestion of benefit - Estimate and most of CI are in the benefit range
Clinically meaningful benefit possible but not likely Some of the CI is in this
range, but not the estimate
Strong evidence against substantial harm No values in the CI would
constitute important harm
May be no effect (not statistically significant) The CI includes no effect
Research Methodology Workshop Hanoi 7-12/12/2010
Clinically harmful Clinically beneficial
0.7 1.3 1.0 2.0 3.0 4.0 0.5 0.3 0.2
Relative risk
Suggestion of substantial benefit. The estimate would be an important benefit if
true.
May be no effect (not statistically significant) The CI includes no effect
Which of the two results would be more exciting? I think the lower one is, even
though it has wider uncertainty, because the estimate is better.
Research Methodology Workshop Hanoi 7-12/12/2010
Clinically harmful Clinically beneficial
0.7 1.3 1.0 2.0 3.0 4.0 0.5 0.3 0.2
Relative risk
Strong evidence of benefit (statistically significant)
Substantial benefit appears likely, but CI too wide to rule out
clinically insignificant benefit The CI includes some benefits that would
be too small to be clinically important. Studies almost never include this
observation if they reach statistical significance.
Research Methodology Workshop Hanoi 7-12/12/2010
Clinically harmful Clinically beneficial
0.7 1.3 1.0 2.0 3.0 4.0 0.5 0.3 0.2
Relative risk
Strong evidence of substantial clinical benefit This is the most satisfying
type of result. Even the upper confidence bound is in the
substantial benefit range.
Research Methodology Workshop Hanoi 7-12/12/2010
Clinically harmful Clinically beneficial
0.7 1.3 1.0 2.0 3.0 4.0 0.5 0.3 0.2
No conclusions possible due to very wide CI This is the least satisfying
type of result. There is very little information in the study data.
Relative risk
Research Methodology Workshop Hanoi 7-12/12/2010
Summary
Relative risk is a common effect size measure
Odds ratio is an estimate of relative risk
In the absence of prospective study, case-control
study can estimate RR by OR
For diseases with high risk, OR tends to
overestimate effect size.
Confidence interval is more meaningful and
more informative than p-value.