ORIGINAL CONTRIBUTION
Using Effectiveness and Cost-effectiveness
to Make Drug Coverage Decisions
A Comparison of Britain, Australia, and Canada
Fiona M. Clement, PhD
Anthony Harris, MA, MSc
Jing Jing Li, BPharm, BCom
Karen Yong
Karen M. Lee
Braden J. Manns, MD, MSc
E
XPENDITURES ON PHARMACEUTI-
cals are the fastest growing sec-
tor within health care in devel-
oped countries, including
Canada,1,2 the United Kingdom,3 Aus-
tralia,4 and the United States,5,6 where fed-
eral expenditures for Part D of Medi-
care and Medicaid are projected to reach
$4299 billion cumulatively from 2010 to
2014.7 In an attempt to control expen-
ditures and to assess the value of new
drugs, many countries, including Brit-
ain (National Institute for Health and
Clinical Excellence [NICE]),8-11 Austra-
lia (Pharmaceutical Benefits Advisory
Committee [PBAC]),12-14 and, most re-
cently, Canada (Common Drug Review
[CDR]15,16), have established agencies to
determine whether new pharmaceuti-
cal treatments should be listed in pub-
lic formularies. The US Department of
Veterans Affairs has also established a na-
tional drug review process and formu-
lary to reduce geographic variability of
access to pharmaceuticals and reduce
drug acquisition costs.6,17
Concerns over rising health care costs
have sparked debate within the United
States about different ways health care
costs might be controlled while maintain-
ing high-quality health care.18-21 In 2008,
US legislation was introduced (Senate bill
©2009 American Medical Association. All rights reserved.
Downloaded From: http://jama.jamanetwork.com/ by a California State University - Fresno User on 07/23/2015
Context National public insurance for drugs is often based on evidence of compara-
tive effectiveness and cost-effectiveness. This study describes how that evidence has
been used across 3 jurisdictions (Australia, Canada, and Britain) that have been at the
forefront of evidence-based coverage internationally.
Objectives To describe how clinical and cost-effectiveness evidence is used in cov-
erage decisions both within and across jurisdictions and to identify common issues in
the process of evidence-based coverage.
Design, Setting, and Participants Descriptive analysis of retrospective data from
the Common Drug Review (CDR) of Canada, National Institute for Health and Clini-
cal Excellence (NICE) in Britain, and Pharmaceutical Benefits Advisory Committee (PBAC)
of Australia. All publicly available information as of December 31, 2008, was gath-
ered from each committee’s Web site (data set begins in January 2004 [CDR], Feb-
ruary 2001 [NICE], and July 2005 [PBAC]).
Main Outcome Measure Listing recommendations for each drug by disease indication.
Results NICE recommended 87.4% (174/199) of submissions for listing compared
with a listing rate of 49.6% (60/121) and 54.3% (153/282) for the CDR and PBAC,
respectively. Significant uncertainty around clinical effectiveness, typically resulting from
inadequate study design or the use of inappropriate comparators and unvalidated sur-
rogate end points, was identified as a key issue in coverage decisions. Recommenda-
tions varied considerably across countries, possibly because of differences in the medi-
cations reviewed; different agency processes, including the willingness to negotiate
on price; and the approach to “me too” drugs. The data suggest that the 3 agencies
make recommendations that are consistent with evidence on effectiveness and cost-
effectiveness but that other factors are often important.
Conclusions NICE, PBAC, and CDR face common issues with respect to the quality
and strength of the experimental evidence in support of a clinically meaningful effect.
However, comparative effectiveness and cost-effectiveness, along with other relevant fac-
tors, can be used by national agencies to support drug decision making. The results of
the evaluation process in different countries are influenced by the context, agency pro-
cesses, ability to engage in price negotiation, and perhaps differences in social values.
JAMA. 2009;302(13):1437-1443 www.jama.com
S3408)tocreateapublic-privatecompara- Author Affiliations: Departments of Medicine (Drs
tiveeffectivenessinstitute(theHealthCare Clement and Manns) and Community Health Sci-
ences (Dr Manns), Libin Cardiovascular Institute (Dr
Comparative Effectiveness Research Manns), University of Calgary, Calgary, Alberta,
Institute).22,23 Creation of this institute, Canada; Centre for Health Economics, Monash Uni-
as well as whether such an agency would versity, Melbourne, Australia (Mr Harris, Mss Li and
Yong); and Canadian Agency for Drugs and Tech-
includecost-effectivenessasanadditional nologies in Health, Ottawa, Ontario, Canada (Ms Lee).
criterion20,21 to comparative effectiveness Corresponding Author: Braden J. Manns, MD,
MSc, Foothills Medical Centre, 1403 29th St NW,
whenconsideringreimbursementofphar- Calgary, AB T2N 2T9, Canada (braden.manns
maceuticals, remains a subject of debate @albertahealthservices.ca).
(Reprinted) JAMA, October 7, 2009—Vol 302, No. 13 1437
 USING EFFECTIVENESS AND COST-EFFECTIVENESS IN DRUG COVERAGE DECISIONS
in the United States. NICE, PBAC, and
CDR have included cost-effectiveness as
part of drug coverage decisions,12,16,24
whereas drug reimbursement decisions
withinpubliclyfundedhealthcare(Medi-
care and Medicaid) in the United States
largely exclude consideration of cost and
cost-effectiveness at present.
We describe the key issues in evidence-
based coverage facing 3 jurisdictions that
use effectiveness and cost-effectiveness
data in evidence-based coverage of phar-
maceuticals. Using a retrospective analy-
sis of past decisions, we describe how
these committees use evidence on effec-
tiveness and cost-effectiveness (includ-
ing any barriers to such use), and what
additional factors have influenced deci-
sions, and explore how these issues may
be associated with listing decisions. We
also consider 3 drugs that highlight these
key issues across agencies and report in
detail these deliberations. To frame dis-
cussion within the US context, we also
compare qualitatively these 3 agencies
with the US Veterans Affairs and Medi-
care drug insurance plans.
METHODS
Drugfundingdecisionswerereviewedfor
the CDR, NICE, and PBAC. These agen-
cies were selected because they consider
both effectiveness and cost-effectiveness,
publish information in English on reim-
bursement decisions, and are similar in
their underlying populations and public
pharmaceuticalinsurancecoverage.There
aresomeimportant differences in process
and remit between the agencies, which
are described here and in eTable 1 (avail-
able at http://www.jama.com). To frame
discussion within the US context, we
also describe the US Veterans Affairs and
Medicare drug insurance plans, 2 na-
tional programs that provide insurance
coverage for drugs (eText).
Common Drug Review
Canada has universal, publicly funded
health care,15,25,26 although the avail-
ability of publicly funded drug insur-
ance varies across provinces and terri-
tories (eTable 1). In 2002, federal,
provincial, and territorial ministers of
health, concerned with differences in
1438 JAMA, October 7, 2009—Vol 302, No. 13 (Reprinted)
Downloaded From: http://jama.jamanetwork.com/ by a California State University - Fresno User on 07/23/2015
coverage of prescription medications
across public formularies and duplica-
tion of effort in reviewing new medi-
cines, established the CDR.15 The CDR
provides recommendations for listing
of new drugs to the 18 participating fed-
eral, provincial, and territorial pub-
licly funded drug plans.15 Manufactur-
ers submit their medication to the CDR,
with submissions being considered by
the Canadian Expert Drug Advisory
Committee, an independent commit-
tee composed of 11 professional mem-
bers (physicians, pharmacologists, and
members with expertise in health ser-
vices research and health economics)
and, since 2006, 2 members of the pub-
lic. The committee considers the safety
and clinical effectiveness of the drug in
appropriate populations as well as cost-
effectiveness, both in comparison with
current accepted therapy.2 Participat-
ing drug plans are not required to fol-
low the committee’s recommenda-
tions because they also must consider
their own health care priorities, avail-
able resources, and the precedence of
previous formulary decisions. None-
theless, drug plan decisions are in agree-
ment with the committee’s recommen-
dations about 90% of the time.2
National Institute for Health
and Clinical Excellence
Britain has publicly funded health care
(the National Health Service [NHS]) and
a small parallel private system. Drugs are
provided to all British citizens through
the NHS, although for drugs that have
not been reviewed by NICE, coverage
decisions are made at local levels and may
vary across regions. NICE issues guid-
ance on selected clinical topics as cho-
sen by the British government10 and often
a class of drugs is reviewed in 1 review
(multiple technology assessment). In
addition to an independently con-
ducted systematic review and eco-
nomic evaluation, the manufacturer sub-
mits clinical and economic evidence, and
other interested parties (patients, car-
ers, and advocacy groups) may submit
evidence and experience profiles. In 2006
a process that critiques manufacturer-
submitted evidence was introduced as a
©2009 American Medical Association. All rights reserved.
more streamlined alternative for single
drugs or technologies (single technol-
ogy assessment). The evidence is then
discussed by a committee of 33 mem-
bers selected from the NHS, patients, aca-
demia, and industry. Recommenda-
tions are legally binding in England and
Wales and local resources must be reor-
ganized to implement NICE guidance.
Pharmaceutical Benefit Advisory
Committee
Australia also has a national publicly
funded health care system and a parallel
privatesystem.Druginsuranceisprovided
toallAustraliansthroughthePharmaceu-
tical Benefits Scheme. Drug companies
submit their drug for funding consider-
ation by the PBAC, an independent statu-
tory body established in 1953 to make
recommendations and give advice to the
minister about which drugs should be
madeavailableaspharmaceuticalbenefits.
ThePBACconsiderseffectivenessandcost
in comparison with alternative therapies;
formalconsiderationofcost-effectiveness
beganinJanuary1993.10,27 ThePBACcon-
sistsof18membersappointedbythegov-
ernment, including physicians, health
professionals, 1 health economist, and 1
consumer representative. Recommenda-
tions are acted on by the minister for
health,whocannotlistadrugonthePhar-
maceutical Benefits Scheme unless the
PBAC gives a positive recommendation.
Inthecaseofanegativerecommendation,
resubmissionswithnewevidenceorlower
prices are permitted, as are submissions
to broaden the specified indication.
Data Sources
All publicly available documents as of
December 31, 2008, were included.
Given that each committee adopted
transparency measures at different time
points, the available data sets for the
PBAC, NICE, and CDR begin in July
2005, February 2001, and January 2004,
respectively. During this time frame,
the publicly available documents
reviewed for NICE and PBAC consis-
tently provided an overview of the clini-
cal and economic reports considered by
the committee, as well as a summary
of the committee’s perspective on the
 USING EFFECTIVENESS AND COST-EFFECTIVENESS IN DRUG COVERAGE DECISIONS
drug. Since 2007, this level of detail has
also been publicly available for the CDR.
However, prior to 2007, the publicly
available information for the CDR was
often limited to the funding recom-
mendation and a review of the major
clinical and economic reasons for this
recommendation. To ensure that a simi-
lar level of detail was available for drugs
reviewed by each agency, confidential
summaries of committee discussions
were also reviewed for CDR submis-
sions, along with the clinical and eco-
nomic reports. For NICE, all technol-
ogy appraisals including drugs were
reviewed. Each drug and disease indi-
cation combination was considered a
unique observation.
Variable Definitions
The primary outcome considered was the
final decisions or recommendations of
the committees available within the study
time frame. The 3 categories were list, list
with criteria, and do not list. The list rate
was calculated as the number of list or
list-with-criteria recommendations di-
vided by the number of submissions. Ba-
sic drug information was collected: ge-
neric drug name, indication sought, date
of submission, and whether the drug had
been considered for the same indica-
tion previously. To define the clinical
context of the decision, and to be con-
sistent with previous research,12 vari-
ables were developed to indicate if the
indication for the drug was life threat-
ening (less than 50% 5-year mean sur-
vival rate), if the goal of treatment was
life extension or quality-of-life improve-
ment, and if other treatment options were
available for the condition.
We collected information on the pri-
mary end point used in the supportive
clinical studies and categorized end
points as clinical end points (eg, mor-
tality or occurrence of a myocardial in-
farction), clinical scales (eg, American
College of Rheumatology 20% improve-
ment criteria for rheumatoid arthri-
tis 28 ), or surrogate end points (eg,
changes in blood pressure or parathy-
roid hormone level29).30,31 For surro-
gate end points, we also determined
whether the committee felt the surro-
©2009 American Medical Association. All rights reserved.
Downloaded From: http://jama.jamanetwork.com/ by a California State University - Fresno User on 07/23/2015
gate was a valid predictor of changes in
the relevant clinical end point.29 The ad-
vantages and disadvantages of accept-
ing clinical efficacy on the basis of sur-
rogate end points have been reviewed in
detail,29-33 but surrogate end points, even
ones that have not been formally vali-
dated, continue to be accepted as proof
of efficacy by regulatory agencies.31,32
Based on prior work, we expected that
certain issues would create problems for
review committees and focused our data
collection on these variables. We de-
fined these key issues as clinical and eco-
nomic uncertainty (both categorized as
none, some, and considerable uncer-
tainty). Considerable clinical uncer-
tainty was present if efficacy data were
based on nonrandomized clinical trials
or if the supportive randomized trials
used what the committees felt to be an
inappropriate comparator or an unvali-
dated surrogate end point. Consider-
able economic uncertainty was present
if there were major flaws in structure of
the economic model, if assumptions used
with respect to clinical efficacy or effec-
tiveness were substantively different from
the committee’s view, or if there was in-
appropriate mapping of the clinical evi-
dence to a final end-point quality of life.
In both cases, the definition of uncer-
tainty was also judged on whether the
summary documents (or the summary
of committee discussions in the case of
the CDR) noted considerable clinical or
economic certainty as an issue during
committee deliberation.
With respect to the supportive clini-
cal trials, we also considered the rel-
evance of the clinical trial evidence, in-
cluding the magnitude of the clinical
effect (ie, effect size). We also consid-
ered socially relevant characteristics of
the patient group (ie, unmet need in dis-
advantaged population, severity of con-
dition [eg, life threatening]).
We also considered when estimating
the cost per quality-adjusted life-year
(QALY) was seen as necessary by the de-
cision maker. For example, when the
committee judged that clinical efficacy
was “equivalent” based on head-to-
head studies, only a comparison of cost
was required and the calculation of an
(Reprinted) JAMA, October 7, 2009—Vol 302, No. 13
incremental cost per QALY was not re-
quired for the listing decision made. We
recorded the base-case cost per QALY
submitted by the manufacturer for the
indication in which coverage was being
requested as well as committee’s esti-
mate of the cost per QALY (ie, the cost-
per-QALY estimate that was felt to be
more accurate by the expert commit-
tees based on review of sensitivity analy-
ses or reanalysis of the model con-
ducted by the agencies). All subjective
variables were extracted in duplicate and
disagreements resolved by consensus.
Data Analysis
To identify similarities and differences in
the characteristics of submissions consid-
ered by each country, submission char-
acteristics were compared using ␹2 tests
(2-sided significance of .05). Based on
previousworkandpublications,12 weiden-
tifiedkeyissuesthatmightimpactthelike-
lihood of funding a treatment and de-
scribed whether these key issues were
associated with the likelihood of listing.
Acknowledgingthatthereweredifferences
in the drugs assessed within the 3 coun-
tries, we also determined the listing rates
for the subgroup of drugs that were con-
sidered in common across agencies. For
those drugs for which there was no ma-
jor uncertainty on cost-effectiveness, we
examined whether there was a threshold
rangeofcost-effectiveness(costperQALY)
above which the committee was unlikely
to fund a drug. The existence of a thresh-
oldvaluefordrugswheretherewasaclini-
cal benefit was estimated graphically by
ranking the committee’s estimate of the
cost per QALY from low to high. SAS ver-
sion 9.0 (SAS Institute Inc, Cary, North
Carolina) was used for all analyses.
Case Studies
To illustrate the similarities and differ-
ences between the different agencies, we
selected 3 drugs that had been re-
viewed by all of the agencies. Each serves
to highlight different key issues that are
commonly encountered in the evi-
dence review process, as well as how re-
view processes may be associated with
drug reimbursement decisions. We se-
lected insulin glargine for diabetes melli-
1439
 USING EFFECTIVENESS AND COST-EFFECTIVENESS IN DRUG COVERAGE DECISIONS

tus, ranibizumab for age-related macu- submissions to each agency. Of note, regulatory approval for longer. For the
lar degeneration, and teriparatide for resubmissions were common for the CDR and PBAC, 26 of 121 submissions
osteoporosis. PBAC, with 75 of 282 submissions (21.7%) and 81 of 282 submissions
(26.6%) being resubmissions of previ- (28.8%) reported use of a nonrandom-
RESULTS ously rejected drugs, often resubmit- ized study design or an inappropriate
During the study period, the CDR con- ted for a narrower clinical indication comparator in a randomized controlled
sidered 121 submissions (114 submis- and sometimes at a lower price. trial. Surrogate end points were often the
sions, 7 resubmissions) while PBAC main end points in the clinical studies
considered 282 submissions (207 sub- Problems With Clinical Evidence that formed the basis of the submis-
missions, 75 resubmissions). NICE More than 40% of all submissions re- sions to each of the agencies (Table).
completed 144 technology appraisals: viewed by the CDR and PBAC were as-
of those, 47 were excluded because they sociated with considerable clinical un- Problems With Economic Evidence
did not consider drugs, leaving 97 sub- certainty (Table), which was more A cost-per-QALY estimate was more
missions, which resulted in 199 drug common than for submissions to NICE commonly required in the evaluation of
appraisals (184 submissions, 15 resub- (54/199; 27.3%; P=.009), perhaps re- drugs considered by NICE (Table).
missions) (eFigure 1). The TABLE pre- flecting the fact that NICE typically When a cost-per-QALY estimate was re-
sents the baseline characteristics of all evaluates classes of drugs that have had quired for the decision, considerable eco-
nomic uncertainty existed for 46.1% (90/
Table. Baseline Characteristics of All Submissions to the CDR, NICE, and PBAC
192), 58.2% (118/203), and 55.7% (41/
73) of submissions considered by NICE,
No. (%)
PBAC, and CDR, respectively. Of note,
CDR NICE PBAC P considerable economic uncertainty was
Characteristics (n = 121) (n = 199) (n = 282) Value
based exclusively on the presence of con-
Resubmission 7 (5.7) 15 (7.5) 75 (26.6) ⬍.001
Life-threatening disease (ie, mean 5-y survival ⬍50%) 22 (18.3) 38 (19.1) 70 (24.8) .20
siderable clinical uncertainty in 57 of 245
Goal of drug treatment cases (23.4%) where economic uncer-
Quality-of-life improvement 56 (46.7) 90 (45.2) 116 (41.1) tainty existed, underscoring the central
Life extension 14 (11.7) 60 (30.2) 63 (22.3) .005 role of good-quality clinical evidence in
Both 51 (41.6) 49 (24.6) 103 (36.6) decision making.
Clinical uncertainty
None 14 (11.7) 39 (19.7) 38 (13.3)
Listing Rates for Drugs Reviewed
Some 57 (47.5) 105 (53.0) 121 (43.0) .009
Considerable a 50 (41.8) 54 (27.3) 123 (43.7) NICE recommended 87.4% (174/199) of
Weight of clinical evidence submissions for listing compared with
RCT with appropriate comparator 95 (78.3) 169 (84.9) 201 (71.2) 49.6% (60/121) for the CDR and 54.3%
RCT with inappropriate comparator 23 (19.2) 20 (10.1) 55 (19.4) .002 (153/282) for the PBAC. The list rates for
No RCT evidence 3 (2.5) 10 (5.0) 26 (9.4) CDR and PBAC were lower when there
Study end point for clinical studies was considerable clinical or economic
Clinical end point 29 (24.2) 105 (52.7) 108 (38.4)
Clinical scale 31 (25.8) 40 (20.2) 62 (22.1) ⬍.001
uncertainty (eTable 2). In addition, the
Surrogate 61 (50.0) 54 (27.1) 111 (39.5) use of a relevant clinical end point was
Committee felt surrogate was valid 44 (71.6) 46 (86.0) 91 (82.4) .11 associated with a higher probability of
Medications where calculation of cost per QALY 73 (60.0) 192 (96.5) 203 (72.0) ⬍.001 recommending coverage for the CDR
was necessary to the decision made and PBAC. Alternatively, listing deci-
Type of cost-effectiveness evidence sions by NICE did not appear to be as-
Cost minimization 43 (35.3) 13 (6.6) 88 (31.3)
Cost-effectiveness 17 (14.1) 15 (7.7) 55 (19.5)
sociated with the existence of signifi-
⬍.001 cant clinical or economic uncertainty,
Cost utility 55 (45.8) 171 (85.7) 138 (48.9)
Cost consequence 6 (4.8) 0 1 (0.3) possibly reflecting its approach of iden-
Economic uncertainty b tifying subgroups for which the uncer-
None 4 (5.3) 16 (8.1) 20 (10.0) tainty might be lower and the cost per
Some 28 (73.0) 86 (44.8) 65 (31.8) .14 QALY more acceptable. There did not ap-
Considerable c 41 (55.7) 90 (46.1) 118 (58.2) pear to be an association between the list-
Abbreviations: CDR, Common Drug Review (in Canada); NICE, National Institute for Health and Clinical Excellence (in
Britain); PBAC, Pharmaceutical Benefits Advisory Committee (in Australia); QALY, quality-adjusted life-year; RCT, ing decision and whether the underly-
randomized controlled trial.
a Including, for example, the use of non–randomized clinical evidence or a randomized controlled trial using an inap- ing condition was life threatening.
propriate comparator or an unvalidated surrogate end point.
b Economic uncertainty was calculated for those medications where a cost per QALY was necessary to the decision made.
c Including, for example, major flaws in the structure of the economic model, assumptions around clinical efficacy that were Incorporating Economic Evidence
substantively different from the committee’s view, and models where there was inappropriate mapping of the surrogate Where the calculation of a cost per QALY
to the final end point.
was necessary for the decision made and
1440 JAMA, October 7, 2009—Vol 302, No. 13 (Reprinted) ©2009 American Medical Association. All rights reserved.

Downloaded From: http://jama.jamanetwork.com/ by a California State University - Fresno User on 07/23/2015

 USING EFFECTIVENESS AND COST-EFFECTIVENESS IN DRUG COVERAGE DECISIONS
the committee was confident in the evi-
dence supporting the incremental cost
per QALY, the committees’ decisions ap-
peared consistent with a cost-effective-
ness framework (eFigure 2). There was
some evidence of a threshold range for
each agency. However, it is clear that
even when there was confidence in the
cost per QALY, other factors appeared to
result in decisions to not recommend
coverage for a drug with a cost per QALY
below these ranges and, conversely, to
recommend funding for some drugs
above these ranges.
For 13 submissions (4, 8, and 1 for the
CDR, NICE, and PBAC, respectively), the
drug was rejected for the patient popu-
lation submitted within the economic
evaluation but was recommended for list-
ing in a more restricted patient sub-
group in whom a cost per QALY was not
available—presumably in an attempt to
improve the efficiency of drug use and
reduce expenditures (eFigure 2). For the
66 submissions in which committees
noted significant economic uncertainty
(7, 6, and 53 at the CDR, NICE, and
PBAC, respectively), the listing rates were
28.6% (2/7), 66.6% (4/6), and 3.8% (2/
53), respectively (eFigure 2).
Analysis of Common Submissions
In 91 submissions, the same drug was re-
viewed for the same indication by more
than 1 of the agencies (eFigure 3). There
was poor agreement between funding
recommendations made by the CDR and
PBAC (␬=0.27) and NICE and PBAC
(␬=0.13) and moderate agreement be-
tween the CDR and NICE (␬ = 0.55).
Consistent with the trend observed over-
all, for this subset of common drugs
NICE was more likely to recommend
funding (eFigure 3). For the drugs con-
sidered by all 3 agencies (n=19), the list
rates were 52.6% (10/19), 84.2% (16/
19), and 73.6% (14/19) for the CDR,
NICE, and PBAC, respectively.
We qualitatively analyzed the sets of
common drugs to determine the most
common reasons for the discrepant list-
ing recommendations. The most com-
mon reasons included an apparent inten-
tion by NICE to find limited niches for
drugs rather than recommending not to
©2009 American Medical Association. All rights reserved.
Downloaded From: http://jama.jamanetwork.com/ by a California State University - Fresno User on 07/23/2015
list (n=7), the use of price negotiation
by the PBAC to ensure cost-effective-
ness (n=3), and a different attitude to the
proliferation of drugs within an estab-
lished therapeutic class (n=7). The CDR
appeared reluctant to list subsequent
similar drugs while the PBAC had a “cost-
minimization policy,” which appeared to
encourage proliferation by listing thera-
peutically similar drugs at the same (or
lower) price as the originator, thus using
competition to ultimately lower prices.
In 4 cases, the committees appeared to
base their decisions at least in part on a
different estimate or valuation of cost-
effectiveness data across the agencies. Not
unexpectedly, in the absence of good-
quality clinical trial data with respect to
the best approach to drug sequencing in
conditions where several different agents
are available, the 3 agencies made dis-
crepant recommendations regarding
whether to provide an open listing, to list
as a second-line agent after failure of less
expensive therapies, or not to list (n=4).
Insulin Glargine
Insulin glargine (eTable 3) was recom-
mended for listing in October 2002 by
NICE in patients with type 1 diabetes and
in a subset of type 2 patients on the ba-
sis of 13 randomized trials. In 2006, the
CDR reviewed the results of 20 un-
blinded randomized trials (many of
which were unpublished), noting vari-
able results for overall and nocturnal hy-
poglycemia. Although the CDR felt that
the use of insulin glargine may reduce
the frequency of nocturnal hypoglyce-
mia, it did not feel that these benefits jus-
tified the 3-fold cost and did not recom-
mend listing. The PBAC rejected listing
for insulin glargine on 5 separate occa-
sions on the basis of clinical uncer-
tainty resulting from reporting bias in the
presented meta-analysis as well as un-
acceptable cost-effectiveness. On the fifth
resubmission, and after extraordinary
discussions with the manufacturer, the
PBAC agreed to list insulin glargine as
an unrestricted benefit based on accept-
able cost-effectiveness at a new pro-
posed confidential price. Thus, al-
though each of the committees agreed
that insulin glargine offered small incre-
(Reprinted) JAMA, October 7, 2009—Vol 302, No. 13
mental benefits over insulin NPH, all felt
that unrestricted use at the price sub-
mitted was not cost-effective. In re-
sponse to this, NICE listed insulin
glargine for patients with type I diabe-
tes and identified a small niche of type
II patients who might be more likely to
benefit, while the PBAC was able to ne-
gotiate a price that offered reasonable
cost-effectiveness, an approach outside
of the scope of the CDR’s mandate.
Ranibizumab
Each of the agencies recommended list-
ing for ranibizumab for age-related macu-
lar degeneration (eTable 4). The clini-
cal evidence was from well-performed
randomized trials using the appropri-
ate comparator and demonstrated that ra-
nibizumab reduced the incidence of
blindness in patients with wet macular
degeneration. Despite the high cost, given
the significant negative implications of
blindness, the use of this agent was as-
sociated with a cost per QALY gained in
the range of other funded interven-
tions. Given the very high cost of this
agent, each agency recommended prod-
uct-listing agreements to ensure cost-
effective use of this agent and to ensure
a maximal expenditure per patient, thus
shifting some of the financial risk asso-
ciated with prolonged use of this medi-
cation to the manufacturer.
Teriparatide
Each of the committees agreed that
teriparatide (eTable 5) had been shown
to reduce the incidence of vertebral and
nonvertebral fractures in comparison
with placebo but felt that bisphospho-
nates would have been a more appro-
priate comparator within randomized
trials. The CDR and PBAC were also con-
cerned that there were no clinical trials
in patients who did not tolerate or who
continued to experience fractures de-
spite bisphosphonates, a subgroup of pa-
tients who might benefit from an addi-
tional treatment option. Given significant
clinical uncertainty, the high cost, and
resultant unacceptable cost-effective-
ness, the CDR and PBAC did not rec-
ommend listing. NICE felt that the use
of this agent might be cost-effective in a
1441
 USING EFFECTIVENESS AND COST-EFFECTIVENESS IN DRUG COVERAGE DECISIONS
small subgroup of patients with severe
osteoporosis for whom bisphospho-
nates had failed and listed it for this small
subset of patients.
US Medicare and Veterans Affairs
Given Medicare’s regulations, it is ex-
pected that these 3 medications would be
covered for Medicare beneficiaries. Of
note, the Medicare Evidence Develop-
ment and Coverage Advisory Commit-
tee is planning to review the data in sup-
port of ranibizumab to ensure optimal
use.34 VeteransAffairsreimbursesthecost
of insulin glargine, ranibizumab, and
teriparatide with specific restrictions for
each medication by clinical indication or
specialty.Detailedreasonsforrecommen-
dations were not available publicly.
COMMENT
NICE, CDR, and PBAC all have expert
committees that consider comparative
effectiveness and cost-effectiveness,
among other factors, in their listing de-
cisions. We noted differences in listing
decisions by these committees, even for
the small subgroup of drugs that were
assessed by all agencies. This is not sur-
prising given that the mandates and pro-
cesses of each of the committees differed.
Moreover, the differences in listing de-
cisions often appeared less about the in-
terpretation of the clinical or economic
evidence and more about differences in
agency processes that may reflect differ-
ences in risk attitudes, including the will-
ingness to fund drugs with a given quality
of evidence on effectiveness and cost-
effectiveness and the role of competition.
The Australian system allows spon-
sors to resubmit an application an un-
limited number of times with variation
in requested price, indication, and asso-
ciated evidence. If we consider only the
last submission for drugs that were re-
submitted after rejection for a particu-
lar indication, the listing rate for the
PBAC increases to 62%, suggesting that
resubmissions may impact listing deci-
sions. As occurred for insulin glargine
and teriparatide, however, the final ac-
ceptance often involved tighter restric-
tions in indication and a lower price. Re-
vised submissions are unusual in Canada,
1442 JAMA, October 7, 2009—Vol 302, No. 13 (Reprinted)
Downloaded From: http://jama.jamanetwork.com/ by a California State University - Fresno User on 07/23/2015
given more stringent criteria around re-
submissions and possibly the lack of a
process to enable price negotiation. There
is an increasing use of risk-sharing ar-
rangements, particularly in Australia, to
reduce uncertainty around both finan-
cial costs and cost-effectiveness.
Consistent with previous studies re-
porting problems with the quality of the
evidence presented to reimbursement
agencies,14,35,36 we noted ongoing is-
sues with the quality and strength of the
experimental evidence in support of the
claim of a clinically meaningful effect.
While all of the agencies noted prob-
lems with the quality and validity of
economic evidence, each has adopted
different approaches to handling this.
NICE has sought independent eco-
nomic analysis while the CDR has con-
ducted its own sensitivity analysis of
manufacturer models. The PBAC has
adopted a structured approach to the
presentation of clinical and economic
evidence, emphasizing rigor in the steps
needed to translate clinical trial data
into evidence of cost-effectiveness.37
Our study has limitations, including
that the data set is based on publicly
available data for NICE and PBAC. Al-
though the public summary docu-
ments are thorough, there may be subtle
issues that were not captured, particu-
larly in the deliberation process. An-
other limitation is that there are surpris-
ingly few common drugs across the 3
systems, making comparisons across
committees less conclusive. In part, this
reflects that NICE, unlike the other agen-
cies, selects which drugs or drug classes
to consider and when and that the CDR
has not reviewed chemotherapy drugs
for cancer since 2007. Finally, given the
heterogeneity in both the drugs consid-
ered and the drug funding systems, a for-
mal statistical analysis of the reasons for
decisions was not possible. The results
suggest that there are some differences
in the way these jurisdictions use effec-
tiveness and cost-effectiveness informa-
tion in coverage decisions, but further
research is needed to establish the cause
of these differences.
Although our study does not pro-
vide direct evidence to inform the ques-
©2009 American Medical Association. All rights reserved.
tion of whether these agencies im-
prove efficiency of care for populations,
early work suggests that the system in
Australia has reduced prices below
those in comparable countries38 with-
out compromising health outcomes.
Moreover, research suggests that es-
tablishment of the Veterans Affairs na-
tional formulary has achieved signifi-
cant cost savings while ensuring access
to a wide range of prescription drugs.6,17
What can be learned from this study
by the United States or other health care
systems regarding pharmaceutical reim-
bursement? First, the existence of these
3 agencies confirms that it is feasible to
establish an agency that considers com-
parative effectiveness in pharmaceuti-
cal reimbursement decisions. The suc-
cessful establishment of the CDR, which
operates in an environment of multiple
payers, all with varying budgets, is par-
ticularly relevant to the United States.
While cost-effectiveness is not required
for all drugs, economic evidence is criti-
cal in some cases to provide informa-
tion on comparative value for money.
Second, the differences that exist in
the processes of these agencies con-
firm that they can be adapted to local
health care circumstances. In fact, a key
component of sustainability of these
agencies appears to have been the abil-
ity of each committee to adapt to na-
tional decision-making needs.39,40
Third, a primary concern in the
United States appears to be that the use
of comparative effectiveness and cost-
effectiveness would reduce choice in
therapeutic options. 20,21,23 As illus-
trated by ranibizumab, the use of cost-
effectiveness in coverage decisions need
not be an undue barrier to drug fund-
ing,12,40 even for expensive medica-
tions, when there is robust evidence of
effectiveness, at least in some patient
subgroup, or where there are factors
that appeal to the values of decision
makers beyond the simple metric of cost
and health gain.11 Moreover, a system
need not necessarily consider a simple
dichotomous listing decision. Medica-
tions can be reimbursed in specific sub-
groups where they are felt to be cost-
effective or can be listed with a higher
 USING EFFECTIVENESS AND COST-EFFECTIVENESS IN DRUG COVERAGE DECISIONS
co-payment if choice and access to
therapy are valued highly. Informa-
tion on cost-effectiveness could be used
to inform the payment level (or co-
payment) for a particular drug. In the
end, however, in any health care sys-
tem, coverage and pricing choices need
to be made.23 As Wilensky puts it:
Cost-effectiveness information should be an
important consideration in setting reimburse-
ment rates by public and private payers. If an
intervention doesn’t do more, why should a
payer pay more for it? If it does do more, ask-
ing how much more and for what addi-
tional price becomes relevant. Payers will have
to make difficult decisions, and different pay-
ers may make different decisions.21
In summary, our study demonstrates
that comparative effectiveness and cost-
effectiveness can be used by national re-
imbursement agencies, although several
key issues exist, the most important of
which is clinical uncertainty. Perhaps the
main lesson from the experience of the
3 countries is that systematic, durable,
and widely accepted decisions can be
made using comparative effectiveness
and cost-effectiveness, although it is evi-
dent that other information beyond these
2 criteria can be incorporated into de-
cision making. Given that the number
of expensive, targeted pharmaceuticals
for cancer and other chronic conditions
is increasing, pharmaceutical reimburse-
ment will continue to be a key challenge
to formularies in all countries.
Author Contributions: Dr Clement had full access to
all of the data in the study and takes responsibility for
the integrity of the data and the accuracy of the data
analysis.
Study concept and design: Clement, Harris, Li, Lee,
Manns.
Acquisition of data: Clement, Harris, Li, Manns.
Analysis and interpretation of data: Clement, Harris,
Li, Yong, Lee, Manns.
Drafting of the manuscript: Clement, Harris, Li, Manns.
Critical revision of the manuscript for important in-
tellectual content: Clement, Harris, Li, Yong, Lee,
Manns.
Statistical analysis: Clement, Yong.
Obtained funding: Clement, Lee, Manns.
Administrative, technical, or material support:
Clement, Lee.
Study supervision: Manns.
Financial Disclosures: None reported.
Funding/Support: Dr Clement is supported by a post-
doctoral fellowship from the Canadian Health Ser-
vices Research Foundation and the Alberta Heritage
Foundation for Medical Research. Dr Manns is sup-
ported by a Canadian Institutes for Health Research
New Investigator Award. This project was supported
by an unrestricted grant from the Canadian Agency
for Drugs and Technologies in Health.
Role of the Sponsor: The funding agencies had no role
©2009 American Medical Association. All rights reserved.
Downloaded From: http://jama.jamanetwork.com/ by a California State University - Fresno User on 07/23/2015
in the design and conduct of the study; in the collec-
tion, management, analysis, and interpretation of the
data; or in the preparation, review, or approval of the
manuscript.
Additional Information: The eTables, eText, and eFig-
ures are available at http://www.jama.com.
REFERENCES
1. Canadian Institute for Health Information. Drug Ex-
penditure in Canada, 1985 to 2006. Ottawa, ON: Ca-
nadian Institute for Health Information; 2007.
2. Canadian Agency for Drugs and Technologies in
Health. CADTH Presentation on the Common Drug
Review to the House of Commons Standing Com-
mittee on Health. Ottawa, ON: Canadian Agency for
Drugs and Technologies in Health; 2007.
3. Duerden M, Gogna N, Godman B, Eden K, Mallinson
M, Sullivan N. Current national initiatives and poli-
cies to control drug costs in Europe: UK perspective.
J Ambul Care Manage. 2004;27(2):132-138.
4. Health expenditure Australia 2006-07 [Health and
welfare expenditure series No. 35]. Australian Insti-
tute of Health and Welfare. http://www.aihw.gov.au
/publications/index.cfm/title/10659. Accessed Sep-
tember 10, 2009.
5. Poisal JA. Medicaid drugs. Health Care Financ Rev.
2004;25(3):1-4.
6. Sales MM, Cunningham FE, Glassman PA, Valentino
MA, Good CB. Pharmacy benefits management in the
Veterans Health Administration: 1995 to 2003. Am J
Manag Care. 2005;11(2):104-112.
7. The budget and economic outlook: fiscal years 2009
to 2019. Congressional Budget Office. http://www
.cbo.gov/ftpdocs/99xx/doc9957/toc.htm. Accessed
September 3, 2009.
8. Collier J. Parliamentary review asks NICE to do bet-
ter still. BMJ. 2008;336(7635):56-57.
9. Miners AH, Garau M, Fidan D, Fischer AJ. Com-
paring estimates of cost effectiveness submitted to the
National Institute for Clinical Excellence (NICE) by dif-
ferent organisations: retrospective study. BMJ. 2005;
330(7482):65.
10. Morgan SG, McMahon M, Mitton C, et al. Cen-
tralized drug review processes in Australia, Canada,
New Zealand, and the United Kingdom. Health Aff
(Millwood). 2006;25(2):337-347.
11. Steinbrook R. Saying no isn’t NICE: the travails
of Britain’s National Institute for Health and Clinical
Excellence. N Engl J Med. 2008;359(19):1977-
1981.
12. Harris AH, Hill SR, Chin G, Li JJ, Walkom E. The
role of value for money in public insurance coverage
decisions for drugs in Australia: a retrospective analy-
sis 1994-2004. Med Decis Making. 2008;28(5):
713-722.
13. Henry DA, Hill SR, Harris A. Drug prices and value
for money: the Australian Pharmaceutical Benefits
Scheme. JAMA. 2005;294(20):2630-2632.
14. Hill SR, Mitchell AS, Henry DA. Problems with the
interpretation of pharmacoeconomic analyses: a re-
view of submissions to the Australian Pharmaceutical
Benefits Scheme. JAMA. 2000;283(16):2116-2121.
15. Tierney M, Manns B; Members of the Canadian
Expert Drug Advisory Committee. Optimizing the use
of prescription drugs in Canada through the Com-
mon Drug Review. CMAJ. 2008;178(4):432-435.
16. McMahon M, Morgan S, Mitton C. The Com-
mon Drug Review: a NICE start for Canada? Health
Policy. 2006;77(3):339-351.
17. Aspinal SL, Good CB, Glassman PA, Valentino MA.
The evolving use of cost-effectiveness analysis in formu-
lary management within the Department of Veterans
Affairs. Med Care. 2005;43(7 suppl):20-26.
18. Aaron HJ. Waste, we know you are out there. N Engl
J Med. 2008;359(18):1865-1867.
19. Fuchs VR. Three “inconvenient truths” about health
care. N Engl J Med. 2008;359(17):1749-1751.
(Reprinted) JAMA, October 7, 2009—Vol 302, No. 13
20. American College of Physicians. Information on
cost-effectiveness: an essential product of a national
comparative effectiveness program. Ann Intern Med.
2008;148(12):956-961.
21. Wilensky GR. Cost-effectiveness information: yes,
it’s important, but keep it separate, please! Ann In-
tern Med. 2008;148(12):967-968.
22. Comparative Effectiveness Research Act of 2008,
S 3408, 110th Cong, 2nd Session 2008.
23. Alexander GC, Stafford RS. Does comparative ef-
fectiveness have a comparative edge? JAMA. 2009;
301(23):2488-2490.
24. Devlin N, Parkin D. Does NICE have a cost-
effectiveness threshold and what other factors influ-
ence its decisions? a binary choice analysis. Health Econ.
2004;13(5):437-452.
25. Health care system. Health Canada. http://www
.hc-sc.gc.ca/hcs-sss/index-eng.php. Accessed No-
vember 5, 2008.
26. Manns BJ, Mendelssohn DC, Taub KJ. The eco-
nomics of end-stage renal disease care in Canada: in-
centives and impact on delivery of care. Int J Health
Care Finance Econ. 2007;7(2-3):149-169.
27. National Health Act 1953. Australian Govern-
ment, Office of Legislative Drafting and Publishing.
http://www.comlaw.gov.au/comlaw/management
.nsf/lookupindexpagesbyid/IP200401580
?OpenDocument. Accessed September 3, 2009.
28. American College of Rheumatology Committee
to Reevaluate Improvement Criteria. A proposed re-
vision to the ACR20: the hybrid measure of Ameri-
can College of Rheumatology response. Arthritis
Rheum. 2007;57(2):193-202.
29. Psaty BM, Lumley T. Surrogate end points and
FDA approval: a tale of 2 lipid-altering drugs. JAMA.
2008;299(12):1474-1476.
30. Fleming TR, DeMets DL. Surrogate end points in
clinical trials: are we being misled? Ann Intern Med.
1996;125(7):605-613.
31. Manns BJ, Owen WF Jr, Winkelmayer WC,
Devereaux PJ, Tonelli M. Surrogate markers in clini-
cal studies: problems solved or created? Am J Kidney
Dis. 2006;48(1):159-166.
32. Psaty BM, Weiss NS, Furberg CD, et al. Surrogate
end points, health outcomes, and the drug-approval pro-
cess for the treatment of risk factors for cardiovascular
disease. JAMA. 1999;282(8):786-790.
33. Temple R. Are surrogate markers adequate to as-
sess cardiovascular disease drugs? JAMA. 1999;
282(8):790-795.
34. Potential NCD (national coverage determin-
ation) topics. Centers for Medicare & Medicaid
Services. http://www.cms.hhs.gov/mcd/ncpc
_view_document.asp?id=19.
35. Hill S, Henry D, Mitchell A. Problems in pharma-
coeconomic analyses. JAMA. 2000;284(15):1922-
1924.
36. Chalkidou K, Tunis S, Lopert R, Rochaix L, Sawicki
PT, Nasser M, Xerri B. Comparative effectiveness research
and evidence-based health policy: experience from four
countries. Milbank Q. 2009;87(2):339-367.
37. Guidelines for preparing submissions to the Phar-
maceutical Benefits Advisory Committee (version 4.3).
PBS Publications. http://www.health.gov.au/internet
/main/publishing.nsf/Content/pbacguidelines
-index. Accessed September 14, 2009.
38. International pharmaceutical price differences: re-
search report [2001]. Australian Government. http:
//www.pc.gov.au/projects/study/pbsprices/docs
/finalreport. Accessed September 3, 2009.
39. National Institute for Health and Clinical Excellence.
First Report of the Health Committee 2007-2008.
London, United Kingdom: Stationary Office; 2008.
40. Williams I, Bryan S, McIver S. How should cost-
effectiveness analysis be used in health technology cov-
erage decisions? evidence from the National Insti-
tute for Health and Clinical Excellence approach.
J Health Serv Res Policy. 2007;12(2):73-79.
1443